2024-09-15T21:48:49.455 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T21:48:49.489 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:48:49.588 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906224' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGWnVGrkA/JjmxvjmjBRgthXLz7VOvD4mfmEMxMSQAr01SACfqNWvfPx7LAUWe7iayvYRaGbGDbW4jveoCPpQ8= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJvj8jyIONxV3JASNE1uXrdqzjrqZq3OCDJvNQdQDkXnunTsT6zNWLZwEcXkd185KF28yNJhn1qubBItF8LdCBw= smithi129.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB/+HhoZ+TADQjGgrmFhZJXhKCwWzcBEI1owlxiRJ47TvMml/REl29vbNMTKZkkGrNkiHl2/u0Xh7g4UVY8C+f4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - quiescer: null - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 32 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T21:48:49.588 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-15T21:48:49.589 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-15T21:48:49.589 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T21:48:49.591 INFO:teuthology.task.internal:Checking packages... 2024-09-15T21:48:49.613 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-15T21:48:49.613 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T21:48:49.613 INFO:teuthology.packaging:ref: None 2024-09-15T21:48:49.613 INFO:teuthology.packaging:tag: None 2024-09-15T21:48:49.613 INFO:teuthology.packaging:branch: squid 2024-09-15T21:48:49.613 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:48:49.613 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-15T21:48:49.830 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-326.g8921c8ef 2024-09-15T21:48:49.831 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T21:48:49.838 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T21:48:49.839 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T21:48:49.846 INFO:teuthology.task.internal:Saving configuration 2024-09-15T21:48:49.870 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T21:48:49.879 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T21:48:49.904 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:43:44.730256', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGGWnVGrkA/JjmxvjmjBRgthXLz7VOvD4mfmEMxMSQAr01SACfqNWvfPx7LAUWe7iayvYRaGbGDbW4jveoCPpQ8='} 2024-09-15T21:48:49.927 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:43:44.728877', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJvj8jyIONxV3JASNE1uXrdqzjrqZq3OCDJvNQdQDkXnunTsT6zNWLZwEcXkd185KF28yNJhn1qubBItF8LdCBw='} 2024-09-15T21:48:49.954 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi129.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:43:44.731463', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB/+HhoZ+TADQjGgrmFhZJXhKCwWzcBEI1owlxiRJ47TvMml/REl29vbNMTKZkkGrNkiHl2/u0Xh7g4UVY8C+f4='} 2024-09-15T21:48:49.954 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T21:48:49.960 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-15T21:48:49.960 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-15T21:48:49.960 INFO:teuthology.task.internal:roles: ubuntu@smithi129.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-15T21:48:49.960 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T21:48:50.043 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2215de7be0>, signals=[15]) 2024-09-15T21:48:50.043 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T21:48:50.050 INFO:teuthology.task.internal:Opening connections... 2024-09-15T21:48:50.050 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-09-15T21:48:50.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:48:50.137 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-09-15T21:48:50.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:48:50.219 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi129.front.sepia.ceph.com 2024-09-15T21:48:50.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:48:50.299 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T21:48:50.306 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-09-15T21:48:50.324 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-09-15T21:48:50.324 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-09-15T21:48:50.382 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-09-15T21:48:50.382 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-09-15T21:48:50.382 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-09-15T21:48:50.383 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:48:50.383 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-09-15T21:48:50.383 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:48:50.383 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:48:50.383 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-09-15T21:48:50.383 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:48:50.383 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:48:50.383 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:48:50.383 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:48:50.383 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:48:50.383 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:48:50.384 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-09-15T21:48:50.409 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-09-15T21:48:50.428 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-09-15T21:48:50.428 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-09-15T21:48:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2024-09-15T21:48:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2024-09-15T21:48:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2024-09-15T21:48:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:48:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2024-09-15T21:48:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:48:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:48:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-09-15T21:48:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:48:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:48:50.486 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:48:50.487 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:48:50.487 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:48:50.487 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:48:50.487 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-09-15T21:48:50.509 DEBUG:teuthology.orchestra.run.smithi129:> uname -m 2024-09-15T21:48:50.528 INFO:teuthology.orchestra.run.smithi129.stdout:x86_64 2024-09-15T21:48:50.529 DEBUG:teuthology.orchestra.run.smithi129:> cat /etc/os-release 2024-09-15T21:48:50.586 INFO:teuthology.orchestra.run.smithi129.stdout:NAME="CentOS Stream" 2024-09-15T21:48:50.586 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION="9" 2024-09-15T21:48:50.586 INFO:teuthology.orchestra.run.smithi129.stdout:ID="centos" 2024-09-15T21:48:50.586 INFO:teuthology.orchestra.run.smithi129.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:48:50.586 INFO:teuthology.orchestra.run.smithi129.stdout:VERSION_ID="9" 2024-09-15T21:48:50.586 INFO:teuthology.orchestra.run.smithi129.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:48:50.587 INFO:teuthology.orchestra.run.smithi129.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:48:50.587 INFO:teuthology.orchestra.run.smithi129.stdout:ANSI_COLOR="0;31" 2024-09-15T21:48:50.587 INFO:teuthology.orchestra.run.smithi129.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:48:50.587 INFO:teuthology.orchestra.run.smithi129.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:48:50.587 INFO:teuthology.orchestra.run.smithi129.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:48:50.587 INFO:teuthology.orchestra.run.smithi129.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:48:50.587 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:48:50.587 INFO:teuthology.orchestra.run.smithi129.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:48:50.587 INFO:teuthology.lock.ops:Updating smithi129.front.sepia.ceph.com on lock server 2024-09-15T21:48:50.610 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T21:48:50.622 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T21:48:50.628 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T21:48:50.629 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:48:50.631 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:48:50.633 DEBUG:teuthology.orchestra.run.smithi129:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:48:50.650 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T21:48:50.659 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T21:48:50.659 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:48:50.692 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:48:50.694 DEBUG:teuthology.orchestra.run.smithi129:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:48:50.720 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T21:48:50.791 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T21:48:50.811 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-15T21:48:50.811 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-15T21:48:50.811 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-15T21:48:50.811 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:48:50.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:48:50.829 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-15T21:48:50.888 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-503.el9.x86_64 2024-09-15T21:48:50.889 INFO:teuthology.task.kernel:Running kernel on smithi005: 5.14.0-503.el9.x86_64 2024-09-15T21:48:50.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum install -y kernel 2024-09-15T21:48:53.211 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-15T21:48:54.963 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-09-15T21:49:00.655 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-15T21:49:02.205 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 19 kB 00:00 2024-09-15T21:49:03.284 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-15T21:49:08.948 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-15T21:49:11.055 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T21:49:11.055 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T21:49:11.157 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-09-15T21:49:11.168 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-15T21:49:11.169 INFO:teuthology.orchestra.run.smithi005.stdout:Install 4 Packages 2024-09-15T21:49:11.170 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:49:11.170 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 79 M 2024-09-15T21:49:11.170 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 126 M 2024-09-15T21:49:11.170 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-09-15T21:49:11.283 INFO:teuthology.orchestra.run.smithi005.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 252 kB/s | 23 kB 00:00 2024-09-15T21:49:14.650 INFO:teuthology.orchestra.run.smithi005.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.5 MB/s | 16 MB 00:03 2024-09-15T21:49:16.200 INFO:teuthology.orchestra.run.smithi005.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.9 MB/s | 35 MB 00:05 2024-09-15T21:49:20.008 INFO:teuthology.orchestra.run.smithi005.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 3.3 MB/s | 29 MB 00:08 2024-09-15T21:49:20.009 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:49:20.009 INFO:teuthology.orchestra.run.smithi005.stdout:Total 8.9 MB/s | 79 MB 00:08 2024-09-15T21:49:20.716 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-09-15T21:49:21.144 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-09-15T21:49:21.144 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-09-15T21:49:22.131 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-09-15T21:49:22.131 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-09-15T21:49:24.819 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-09-15T21:49:25.486 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:49:25.575 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:49:28.397 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:49:28.521 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:49:33.643 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:49:33.687 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:49:38.831 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:50:18.159 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:50:18.208 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:50:18.577 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:50:18.577 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:50:18.577 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:50:18.577 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:50:19.456 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:50:19.457 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:50:19.457 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-09-15T21:50:19.457 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T21:50:19.457 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T21:50:19.457 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T21:50:19.457 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T21:50:19.457 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:50:19.457 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-09-15T21:50:19.807 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:50:19.891 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:50:19.891 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:50:19.891 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T21:50:19.892 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T21:50:19.892 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T21:50:19.892 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-15T21:50:19.892 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T21:50:19.892 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:50:19.951 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:50:19.951 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:50:19.951 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2024-09-15T21:50:19.984 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2024-09-15T21:50:19.984 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2024-09-15T21:50:19.984 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2024-09-15T21:50:19.984 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 82.el9 2024-09-15T21:50:19.984 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2024-09-15T21:50:19.984 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T21:50:19.984 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2024-09-15T21:50:19.984 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8274670 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T21:50:19.985 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T21:50:19.986 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T21:50:19.986 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2024-09-15T21:50:19.986 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:50:19.986 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T21:50:19.987 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T21:50:19.987 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 5.14.0-505.el9.x86_64 2024-09-15T21:50:19.987 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T21:50:20.649 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-09-15T21:50:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T21:50:21.940 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-09-15T21:50:21.942 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-09-15T21:50:21.957 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.5XRBPX6F7b 2024-09-15T21:50:21.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5XRBPX6F7b 2024-09-15T21:50:22.024 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.5XRBPX6F7b 2024-09-15T21:50:22.163 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.5XRBPX6F7b is 6KB 2024-09-15T21:50:22.212 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.5XRBPX6F7b 2024-09-15T21:50:22.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T21:50:22.297 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T21:50:22.297 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T21:50:22.297 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T21:50:22.297 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T21:50:22.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T21:50:22.463 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-09-15T21:50:52.466 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-15T21:50:52.466 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-15T21:50:52.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:51:52.469 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T21:52:01.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-15T21:52:01.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:52:01.867 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-15T21:52:02.239 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-15T21:52:02.239 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-15T21:52:02.240 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-15T21:52:02.255 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-505.el9.x86_64 2024-09-15T21:52:02.255 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T21:52:02.255 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T21:52:02.255 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T21:52:03.257 DEBUG:teuthology.parallel:result is None 2024-09-15T21:52:03.257 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T21:52:03.265 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T21:52:03.266 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:52:03.268 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:52:03.271 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:52:03.290 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T21:52:03.336 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T21:52:03.353 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T21:52:03.353 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:52:03.356 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:52:03.358 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:52:03.422 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T21:52:03.430 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T21:52:03.430 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:52:03.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:52:03.448 DEBUG:teuthology.orchestra.run.smithi129:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:52:03.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:52:03.466 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:52:03.468 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:52:03.491 DEBUG:teuthology.orchestra.run.smithi129:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:52:03.533 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:52:03.547 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:52:03.567 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:52:03.575 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:52:03.597 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:52:03.606 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:52:03.608 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T21:52:03.615 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T21:52:03.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:52:03.617 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:52:03.620 DEBUG:teuthology.orchestra.run.smithi129:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:52:03.648 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T21:52:03.657 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T21:52:03.657 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:52:03.688 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:52:03.690 DEBUG:teuthology.orchestra.run.smithi129:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:52:03.707 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:52:03.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:52:03.851 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:52:03.939 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:52:03.985 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T21:52:03.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:52:04.055 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:52:04.100 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:52:04.163 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:52:04.205 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:52:04.248 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-15T21:52:04.248 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:52:04.314 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:52:04.358 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:52:04.415 DEBUG:teuthology.orchestra.run.smithi129:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:52:04.502 DEBUG:teuthology.orchestra.run.smithi129:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:52:04.546 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-15T21:52:04.546 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:52:04.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-09-15T21:52:04.616 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-09-15T21:52:04.619 DEBUG:teuthology.orchestra.run.smithi129:> sudo service rsyslog restart 2024-09-15T21:52:04.657 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:52:04.668 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:52:04.689 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:52:05.223 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T21:52:05.232 INFO:teuthology.task.internal:Starting timer... 2024-09-15T21:52:05.232 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T21:52:05.331 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T21:52:05.342 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-15T21:52:05.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-09-15T21:52:05.427 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-09-15T21:52:05.428 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-09-15T21:52:05.481 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-09-15T21:52:05.483 DEBUG:teuthology.orchestra.run.smithi129:> sudo service auditd rotate 2024-09-15T21:52:05.534 INFO:teuthology.orchestra.run.smithi129.stdout:Rotating logs: 2024-09-15T21:52:05.536 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T21:52:05.536 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-09-15T21:52:05.564 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-09-15T21:52:05.564 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-09-15T21:52:05.584 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-09-15T21:52:05.585 DEBUG:teuthology.orchestra.run.smithi129:> /usr/sbin/getenforce 2024-09-15T21:52:05.608 INFO:teuthology.orchestra.run.smithi129.stdout:Permissive 2024-09-15T21:52:05.609 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi099.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi129.front.sepia.ceph.com': 'permissive'} 2024-09-15T21:52:05.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:52:05.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:52:05.636 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:52:05.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:52:05.664 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:52:05.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:52:05.693 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T21:52:05.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:52:05.722 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:52:05.754 DEBUG:teuthology.orchestra.run.smithi129:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:52:05.787 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T21:52:05.796 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T21:52:05.815 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T21:52:05.816 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi099.front.sepia.ceph.com,smithi129.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T21:56:08.010 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com'), Remote(name='ubuntu@smithi129.front.sepia.ceph.com')] 2024-09-15T21:56:08.011 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-15T21:56:08.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:56:08.091 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-15T21:56:08.176 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-15T21:56:08.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-15T21:56:08.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:56:08.251 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-15T21:56:08.324 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-15T21:56:08.325 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-15T21:56:08.325 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi129.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:56:08.401 DEBUG:teuthology.orchestra.run.smithi129:> true 2024-09-15T21:56:08.481 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi129.front.sepia.ceph.com' 2024-09-15T21:56:08.484 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T21:56:08.494 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T21:56:08.494 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:56:08.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:56:08.498 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:56:08.498 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:56:08.501 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:56:08.501 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:56:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:56:08.530 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:56:08.545 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:56:08.551 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:56:08.566 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:56:08.570 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-09-15T21:56:08.571 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-09-15T21:56:08.582 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-09-15T21:56:08.583 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-09-15T21:56:08.591 INFO:teuthology.orchestra.run.smithi129.stderr:sudo: ntpd: command not found 2024-09-15T21:56:08.597 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:56:08.598 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:56:08.602 INFO:teuthology.orchestra.run.smithi129.stdout:506 Cannot talk to daemon 2024-09-15T21:56:08.611 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:56:08.613 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:56:08.616 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:56:08.630 INFO:teuthology.orchestra.run.smithi129.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:56:08.667 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:56:08.670 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:56:08.670 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-15T21:56:08.670 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.670 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.670 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.671 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.671 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:56:08.675 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:56:08.675 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-09-15T21:56:08.675 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.675 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.675 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.675 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.689 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:56:08.692 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:56:08.692 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-15T21:56:08.692 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.692 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.693 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.693 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:56:08.693 INFO:teuthology.run_tasks:Running task install... 2024-09-15T21:56:08.702 DEBUG:teuthology.task.install:project ceph 2024-09-15T21:56:08.702 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-15T21:56:08.703 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-15T21:56:08.703 INFO:teuthology.task.install:Using flavor: default 2024-09-15T21:56:08.708 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T21:56:08.708 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-15T21:56:08.709 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T21:56:08.709 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:56:08.711 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T21:56:08.711 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:56:08.712 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T21:56:08.712 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:56:08.856 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T21:56:08.856 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T21:56:08.868 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T21:56:08.868 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T21:56:08.871 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T21:56:08.871 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T21:56:08.967 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:56:08.967 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T21:56:08.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:56:08.979 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:56:08.979 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-15T21:56:08.979 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:56:08.985 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:56:08.985 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-15T21:56:08.985 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:56:08.995 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T21:56:08.995 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:56:09.005 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T21:56:09.005 DEBUG:teuthology.orchestra.run.smithi129:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:56:09.011 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T21:56:09.011 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:56:09.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:56:09.071 DEBUG:teuthology.orchestra.run.smithi129:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:56:09.077 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:56:09.141 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:56:09.146 DEBUG:teuthology.orchestra.run.smithi129:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:56:09.152 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:56:09.206 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-09-15T21:56:09.208 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-09-15T21:56:09.210 INFO:teuthology.orchestra.run.smithi129.stdout:check_obsoletes = 1 2024-09-15T21:56:09.212 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum clean all 2024-09-15T21:56:09.216 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-09-15T21:56:09.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-09-15T21:56:09.435 INFO:teuthology.orchestra.run.smithi129.stdout:45 files removed 2024-09-15T21:56:09.443 INFO:teuthology.orchestra.run.smithi099.stdout:45 files removed 2024-09-15T21:56:09.445 INFO:teuthology.orchestra.run.smithi005.stdout:45 files removed 2024-09-15T21:56:09.462 DEBUG:teuthology.orchestra.run.smithi129:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T21:56:09.470 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T21:56:09.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T21:56:10.022 INFO:teuthology.orchestra.run.smithi129.stdout:ceph packages for x86_64 292 kB/s | 83 kB 00:00 2024-09-15T21:56:10.023 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 302 kB/s | 83 kB 00:00 2024-09-15T21:56:10.037 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 318 kB/s | 83 kB 00:00 2024-09-15T21:56:10.289 INFO:teuthology.orchestra.run.smithi129.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-15T21:56:10.291 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-15T21:56:10.329 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-15T21:56:10.541 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 8.8 kB/s | 1.9 kB 00:00 2024-09-15T21:56:10.548 INFO:teuthology.orchestra.run.smithi129.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-15T21:56:10.605 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-15T21:56:11.082 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-15T21:56:11.689 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.2 MB 00:01 2024-09-15T21:56:12.008 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.2 MB 00:01 2024-09-15T21:56:13.523 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-15T21:56:13.742 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-15T21:56:15.270 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 20 MB 00:03 2024-09-15T21:56:18.085 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-15T21:56:18.226 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-15T21:56:19.529 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-15T21:56:19.719 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 19 kB 00:00 2024-09-15T21:56:19.810 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-15T21:56:21.098 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-15T21:56:21.130 INFO:teuthology.orchestra.run.smithi129.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 19 kB 00:00 2024-09-15T21:56:21.172 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-15T21:56:22.392 INFO:teuthology.orchestra.run.smithi129.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-15T21:56:26.529 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-15T21:56:26.820 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-09-15T21:56:27.849 INFO:teuthology.orchestra.run.smithi129.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-15T21:56:28.824 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T21:56:28.825 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T21:56:28.839 INFO:teuthology.orchestra.run.smithi099.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T21:56:28.840 INFO:teuthology.orchestra.run.smithi099.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T21:56:28.841 INFO:teuthology.orchestra.run.smithi099.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T21:56:28.842 INFO:teuthology.orchestra.run.smithi099.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T21:56:28.844 INFO:teuthology.orchestra.run.smithi099.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T21:56:28.847 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T21:56:28.848 INFO:teuthology.orchestra.run.smithi099.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T21:56:28.911 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-15T21:56:28.933 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-15T21:56:28.933 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-15T21:56:28.933 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-15T21:56:28.933 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-15T21:56:28.933 INFO:teuthology.orchestra.run.smithi099.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T21:56:28.933 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T21:56:28.933 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T21:56:28.934 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T21:56:28.934 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T21:56:28.934 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T21:56:28.934 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T21:56:28.934 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T21:56:28.934 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T21:56:28.934 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T21:56:28.934 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T21:56:28.934 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T21:56:28.934 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T21:56:28.935 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T21:56:28.936 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T21:56:28.936 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T21:56:28.936 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T21:56:28.936 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T21:56:28.936 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T21:56:28.936 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T21:56:28.936 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T21:56:28.936 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T21:56:28.936 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T21:56:28.936 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T21:56:28.937 INFO:teuthology.orchestra.run.smithi099.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T21:56:28.938 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T21:56:28.938 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T21:56:28.938 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T21:56:28.938 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T21:56:28.938 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T21:56:28.938 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T21:56:28.938 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T21:56:28.938 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T21:56:28.938 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T21:56:28.938 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T21:56:28.939 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T21:56:28.940 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T21:56:28.941 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T21:56:28.942 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T21:56:28.943 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T21:56:28.944 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T21:56:28.945 INFO:teuthology.orchestra.run.smithi099.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T21:56:28.946 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T21:56:28.946 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T21:56:28.946 INFO:teuthology.orchestra.run.smithi099.stdout:Installing weak dependencies: 2024-09-15T21:56:28.946 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T21:56:28.946 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:56:28.946 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-15T21:56:28.946 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-15T21:56:28.946 INFO:teuthology.orchestra.run.smithi099.stdout:Install 148 Packages 2024-09-15T21:56:28.946 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 8 Packages 2024-09-15T21:56:28.946 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:56:28.947 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 226 M 2024-09-15T21:56:28.947 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-15T21:56:29.229 INFO:teuthology.orchestra.run.smithi099.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-15T21:56:29.282 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T21:56:29.283 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T21:56:29.295 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T21:56:29.295 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T21:56:29.296 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T21:56:29.297 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T21:56:29.299 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T21:56:29.301 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T21:56:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T21:56:29.368 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-09-15T21:56:29.391 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-15T21:56:29.391 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-09-15T21:56:29.391 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-15T21:56:29.391 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-09-15T21:56:29.391 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T21:56:29.391 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T21:56:29.391 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T21:56:29.392 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T21:56:29.393 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T21:56:29.393 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T21:56:29.393 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T21:56:29.393 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T21:56:29.393 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T21:56:29.393 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T21:56:29.393 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T21:56:29.393 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T21:56:29.393 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T21:56:29.393 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T21:56:29.393 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T21:56:29.394 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T21:56:29.395 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T21:56:29.396 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T21:56:29.397 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T21:56:29.398 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T21:56:29.399 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T21:56:29.400 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T21:56:29.401 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T21:56:29.402 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T21:56:29.403 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T21:56:29.404 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T21:56:29.404 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-09-15T21:56:29.404 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T21:56:29.404 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:56:29.404 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-09-15T21:56:29.404 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-15T21:56:29.404 INFO:teuthology.orchestra.run.smithi005.stdout:Install 148 Packages 2024-09-15T21:56:29.404 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 8 Packages 2024-09-15T21:56:29.404 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:56:29.405 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 226 M 2024-09-15T21:56:29.405 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-09-15T21:56:29.463 INFO:teuthology.orchestra.run.smithi099.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 3.8 MB/s | 896 kB 00:00 2024-09-15T21:56:29.555 INFO:teuthology.orchestra.run.smithi099.stdout:(3/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 12 MB/s | 5.4 MB 00:00 2024-09-15T21:56:29.580 INFO:teuthology.orchestra.run.smithi099.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 1.2 MB/s | 145 kB 00:00 2024-09-15T21:56:29.681 INFO:teuthology.orchestra.run.smithi099.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 19 MB/s | 2.4 MB 00:00 2024-09-15T21:56:29.823 INFO:teuthology.orchestra.run.smithi099.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 6.3 MB/s | 1.5 MB 00:00 2024-09-15T21:56:29.931 INFO:teuthology.orchestra.run.smithi099.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 19 MB/s | 4.7 MB 00:00 2024-09-15T21:56:29.963 INFO:teuthology.orchestra.run.smithi005.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 24 kB/s | 6.5 kB 00:00 2024-09-15T21:56:30.154 INFO:teuthology.orchestra.run.smithi129.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T21:56:30.155 INFO:teuthology.orchestra.run.smithi129.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T21:56:30.168 INFO:teuthology.orchestra.run.smithi129.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T21:56:30.169 INFO:teuthology.orchestra.run.smithi129.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T21:56:30.170 INFO:teuthology.orchestra.run.smithi129.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T21:56:30.171 INFO:teuthology.orchestra.run.smithi129.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T21:56:30.173 INFO:teuthology.orchestra.run.smithi129.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T21:56:30.176 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T21:56:30.177 INFO:teuthology.orchestra.run.smithi129.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T21:56:30.240 INFO:teuthology.orchestra.run.smithi129.stdout:Dependencies resolved. 2024-09-15T21:56:30.256 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-15T21:56:30.256 INFO:teuthology.orchestra.run.smithi129.stdout: Package Arch Version Repository Size 2024-09-15T21:56:30.256 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-15T21:56:30.256 INFO:teuthology.orchestra.run.smithi129.stdout:Installing: 2024-09-15T21:56:30.256 INFO:teuthology.orchestra.run.smithi129.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T21:56:30.256 INFO:teuthology.orchestra.run.smithi129.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T21:56:30.256 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T21:56:30.256 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T21:56:30.257 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T21:56:30.258 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T21:56:30.258 INFO:teuthology.orchestra.run.smithi129.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T21:56:30.258 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T21:56:30.258 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T21:56:30.258 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T21:56:30.258 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T21:56:30.258 INFO:teuthology.orchestra.run.smithi129.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T21:56:30.258 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T21:56:30.258 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T21:56:30.258 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrading: 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T21:56:30.259 INFO:teuthology.orchestra.run.smithi129.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout:Installing dependencies: 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T21:56:30.260 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T21:56:30.261 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T21:56:30.262 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T21:56:30.263 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T21:56:30.264 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T21:56:30.265 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T21:56:30.266 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T21:56:30.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T21:56:30.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T21:56:30.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T21:56:30.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T21:56:30.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T21:56:30.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T21:56:30.267 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout:Installing weak dependencies: 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T21:56:30.268 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:56:30.269 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction Summary 2024-09-15T21:56:30.269 INFO:teuthology.orchestra.run.smithi129.stdout:====================================================================================== 2024-09-15T21:56:30.269 INFO:teuthology.orchestra.run.smithi129.stdout:Install 148 Packages 2024-09-15T21:56:30.269 INFO:teuthology.orchestra.run.smithi129.stdout:Upgrade 8 Packages 2024-09-15T21:56:30.269 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:56:30.269 INFO:teuthology.orchestra.run.smithi129.stdout:Total download size: 226 M 2024-09-15T21:56:30.270 INFO:teuthology.orchestra.run.smithi129.stdout:Downloading Packages: 2024-09-15T21:56:30.338 INFO:teuthology.orchestra.run.smithi005.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 2.3 MB/s | 896 kB 00:00 2024-09-15T21:56:30.365 INFO:teuthology.orchestra.run.smithi099.stdout:(8/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9. 24 MB/s | 11 MB 00:00 2024-09-15T21:56:30.381 INFO:teuthology.orchestra.run.smithi005.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 3.4 MB/s | 145 kB 00:00 2024-09-15T21:56:30.406 INFO:teuthology.orchestra.run.smithi099.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 609 kB/s | 25 kB 00:00 2024-09-15T21:56:30.807 INFO:teuthology.orchestra.run.smithi099.stdout:(10/156): ceph-common-19.1.1-326.g8921c8ef.el9. 12 MB/s | 21 MB 00:01 2024-09-15T21:56:30.857 INFO:teuthology.orchestra.run.smithi099.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 4.5 MB/s | 228 kB 00:00 2024-09-15T21:56:30.873 INFO:teuthology.orchestra.run.smithi005.stdout:(4/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 4.9 MB/s | 2.4 MB 00:00 2024-09-15T21:56:31.040 INFO:teuthology.orchestra.run.smithi005.stdout:(5/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 4.0 MB/s | 5.4 MB 00:01 2024-09-15T21:56:31.066 INFO:teuthology.orchestra.run.smithi099.stdout:(12/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 14 MB/s | 17 MB 00:01 2024-09-15T21:56:31.081 INFO:teuthology.orchestra.run.smithi005.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 7.3 MB/s | 1.5 MB 00:00 2024-09-15T21:56:31.091 INFO:teuthology.orchestra.run.smithi099.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 116 kB/s | 27 kB 00:00 2024-09-15T21:56:31.124 INFO:teuthology.orchestra.run.smithi099.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 548 kB/s | 32 kB 00:00 2024-09-15T21:56:31.158 INFO:teuthology.orchestra.run.smithi099.stdout:(15/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 751 kB 00:00 2024-09-15T21:56:31.183 INFO:teuthology.orchestra.run.smithi099.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.7 MB/s | 163 kB 00:00 2024-09-15T21:56:31.217 INFO:teuthology.orchestra.run.smithi099.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 2.1 MB/s | 127 kB 00:00 2024-09-15T21:56:31.258 INFO:teuthology.orchestra.run.smithi099.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 6.6 MB/s | 506 kB 00:00 2024-09-15T21:56:31.359 INFO:teuthology.orchestra.run.smithi099.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 450 kB/s | 45 kB 00:00 2024-09-15T21:56:31.425 INFO:teuthology.orchestra.run.smithi129.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 25 kB/s | 6.5 kB 00:00 2024-09-15T21:56:31.459 INFO:teuthology.orchestra.run.smithi099.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.3 MB/s | 136 kB 00:00 2024-09-15T21:56:31.543 INFO:teuthology.orchestra.run.smithi099.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.9 MB/s | 161 kB 00:00 2024-09-15T21:56:31.618 INFO:teuthology.orchestra.run.smithi099.stdout:(22/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-15T21:56:31.632 INFO:teuthology.orchestra.run.smithi005.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 8.0 MB/s | 4.7 MB 00:00 2024-09-15T21:56:31.652 INFO:teuthology.orchestra.run.smithi099.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 2.9 MB/s | 320 kB 00:00 2024-09-15T21:56:31.760 INFO:teuthology.orchestra.run.smithi099.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 2.1 MB/s | 300 kB 00:00 2024-09-15T21:56:31.786 INFO:teuthology.orchestra.run.smithi099.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 744 kB/s | 99 kB 00:00 2024-09-15T21:56:31.861 INFO:teuthology.orchestra.run.smithi099.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 850 kB/s | 85 kB 00:00 2024-09-15T21:56:31.884 INFO:teuthology.orchestra.run.smithi129.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.9 MB/s | 896 kB 00:00 2024-09-15T21:56:31.961 INFO:teuthology.orchestra.run.smithi099.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 1.7 MB/s | 170 kB 00:00 2024-09-15T21:56:31.992 INFO:teuthology.orchestra.run.smithi129.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.3 MB/s | 145 kB 00:00 2024-09-15T21:56:32.037 INFO:teuthology.orchestra.run.smithi099.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 424 kB/s | 32 kB 00:00 2024-09-15T21:56:32.076 INFO:teuthology.orchestra.run.smithi129.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 5.9 MB/s | 5.4 MB 00:00 2024-09-15T21:56:32.112 INFO:teuthology.orchestra.run.smithi099.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.9 MB/s | 146 kB 00:00 2024-09-15T21:56:32.335 INFO:teuthology.orchestra.run.smithi129.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 5.9 MB/s | 1.5 MB 00:00 2024-09-15T21:56:32.421 INFO:teuthology.orchestra.run.smithi099.stdout:(30/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 4.9 MB/s | 3.1 MB 00:00 2024-09-15T21:56:32.829 INFO:teuthology.orchestra.run.smithi099.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 5.0 MB/s | 3.6 MB 00:00 2024-09-15T21:56:32.927 INFO:teuthology.orchestra.run.smithi129.stdout:(6/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 8.0 MB/s | 4.7 MB 00:00 2024-09-15T21:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 2.4 MB/s | 249 kB 00:00 2024-09-15T21:56:32.969 INFO:teuthology.orchestra.run.smithi129.stdout:(7/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 2.4 MB/s | 2.4 MB 00:00 2024-09-15T21:56:33.021 INFO:teuthology.orchestra.run.smithi099.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 537 kB/s | 49 kB 00:00 2024-09-15T21:56:33.105 INFO:teuthology.orchestra.run.smithi099.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 198 kB/s | 16 kB 00:00 2024-09-15T21:56:33.197 INFO:teuthology.orchestra.run.smithi099.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 3.0 MB/s | 285 kB 00:00 2024-09-15T21:56:33.314 INFO:teuthology.orchestra.run.smithi099.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 8.3 MB/s | 7.4 MB 00:00 2024-09-15T21:56:33.347 INFO:teuthology.orchestra.run.smithi099.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 5.0 MB/s | 765 kB 00:00 2024-09-15T21:56:33.372 INFO:teuthology.orchestra.run.smithi099.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 312 kB/s | 18 kB 00:00 2024-09-15T21:56:33.469 INFO:teuthology.orchestra.run.smithi129.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 9.2 MB/s | 21 MB 00:02 2024-09-15T21:56:33.511 INFO:teuthology.orchestra.run.smithi129.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 609 kB/s | 25 kB 00:00 2024-09-15T21:56:33.523 INFO:teuthology.orchestra.run.smithi005.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 5.5 MB/s | 21 MB 00:03 2024-09-15T21:56:33.582 INFO:teuthology.orchestra.run.smithi005.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 431 kB/s | 25 kB 00:00 2024-09-15T21:56:33.908 INFO:teuthology.orchestra.run.smithi005.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 6.0 MB/s | 17 MB 00:02 2024-09-15T21:56:34.006 INFO:teuthology.orchestra.run.smithi099.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 121 kB/s | 80 kB 00:00 2024-09-15T21:56:34.033 INFO:teuthology.orchestra.run.smithi005.stdout:(11/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 4.4 MB/s | 11 MB 00:02 2024-09-15T21:56:34.058 INFO:teuthology.orchestra.run.smithi005.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.5 MB/s | 228 kB 00:00 2024-09-15T21:56:34.129 INFO:teuthology.orchestra.run.smithi005.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 284 kB/s | 27 kB 00:00 2024-09-15T21:56:34.150 INFO:teuthology.orchestra.run.smithi005.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 345 kB/s | 32 kB 00:00 2024-09-15T21:56:34.217 INFO:teuthology.orchestra.run.smithi005.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.4 MB/s | 163 kB 00:00 2024-09-15T21:56:34.257 INFO:teuthology.orchestra.run.smithi099.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 907 kB/s | 801 kB 00:00 2024-09-15T21:56:34.284 INFO:teuthology.orchestra.run.smithi005.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.9 MB/s | 127 kB 00:00 2024-09-15T21:56:34.310 INFO:teuthology.orchestra.run.smithi005.stdout:(17/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 4.1 MB/s | 751 kB 00:00 2024-09-15T21:56:34.328 INFO:teuthology.orchestra.run.smithi129.stdout:(10/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 12 MB/s | 17 MB 00:01 2024-09-15T21:56:34.368 INFO:teuthology.orchestra.run.smithi005.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 5.9 MB/s | 506 kB 00:00 2024-09-15T21:56:34.394 INFO:teuthology.orchestra.run.smithi129.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 3.4 MB/s | 228 kB 00:00 2024-09-15T21:56:34.435 INFO:teuthology.orchestra.run.smithi005.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 678 kB/s | 45 kB 00:00 2024-09-15T21:56:34.444 INFO:teuthology.orchestra.run.smithi129.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 542 kB/s | 27 kB 00:00 2024-09-15T21:56:34.485 INFO:teuthology.orchestra.run.smithi005.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.7 MB/s | 136 kB 00:00 2024-09-15T21:56:34.495 INFO:teuthology.orchestra.run.smithi129.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 634 kB/s | 32 kB 00:00 2024-09-15T21:56:34.536 INFO:teuthology.orchestra.run.smithi005.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 3.2 MB/s | 161 kB 00:00 2024-09-15T21:56:34.595 INFO:teuthology.orchestra.run.smithi129.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 7.3 MB/s | 751 kB 00:00 2024-09-15T21:56:34.636 INFO:teuthology.orchestra.run.smithi005.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.1 MB/s | 320 kB 00:00 2024-09-15T21:56:34.654 INFO:teuthology.orchestra.run.smithi129.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.7 MB/s | 163 kB 00:00 2024-09-15T21:56:34.694 INFO:teuthology.orchestra.run.smithi005.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 5.0 MB/s | 300 kB 00:00 2024-09-15T21:56:34.721 INFO:teuthology.orchestra.run.smithi129.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.9 MB/s | 127 kB 00:00 2024-09-15T21:56:34.753 INFO:teuthology.orchestra.run.smithi005.stdout:(24/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.7 MB/s | 99 kB 00:00 2024-09-15T21:56:34.788 INFO:teuthology.orchestra.run.smithi129.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 7.4 MB/s | 506 kB 00:00 2024-09-15T21:56:34.820 INFO:teuthology.orchestra.run.smithi005.stdout:(25/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-15T21:56:34.865 INFO:teuthology.orchestra.run.smithi099.stdout:(41/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 11 MB/s | 49 MB 00:04 2024-09-15T21:56:35.238 INFO:teuthology.orchestra.run.smithi129.stdout:(18/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 12 MB/s | 5.3 MB 00:00 2024-09-15T21:56:35.240 INFO:teuthology.orchestra.run.smithi099.stdout:(42/156): libquadmath-11.5.0-2.el9.x86_64.rpm 155 kB/s | 191 kB 00:01 2024-09-15T21:56:35.279 INFO:teuthology.orchestra.run.smithi005.stdout:(26/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 5.5 MB/s | 5.3 MB 00:00 2024-09-15T21:56:35.297 INFO:teuthology.orchestra.run.smithi129.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 775 kB/s | 45 kB 00:00 2024-09-15T21:56:35.338 INFO:teuthology.orchestra.run.smithi005.stdout:(27/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 6.0 MB/s | 3.1 MB 00:00 2024-09-15T21:56:35.355 INFO:teuthology.orchestra.run.smithi129.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.3 MB/s | 136 kB 00:00 2024-09-15T21:56:35.363 INFO:teuthology.orchestra.run.smithi005.stdout:(28/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.0 MB/s | 170 kB 00:00 2024-09-15T21:56:35.388 INFO:teuthology.orchestra.run.smithi005.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 635 kB/s | 32 kB 00:00 2024-09-15T21:56:35.414 INFO:teuthology.orchestra.run.smithi129.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.7 MB/s | 161 kB 00:00 2024-09-15T21:56:35.421 INFO:teuthology.orchestra.run.smithi005.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.4 MB/s | 146 kB 00:00 2024-09-15T21:56:35.457 INFO:teuthology.orchestra.run.smithi099.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 211 kB/s | 253 kB 00:01 2024-09-15T21:56:35.481 INFO:teuthology.orchestra.run.smithi129.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 4.7 MB/s | 320 kB 00:00 2024-09-15T21:56:35.548 INFO:teuthology.orchestra.run.smithi129.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 4.4 MB/s | 300 kB 00:00 2024-09-15T21:56:35.764 INFO:teuthology.orchestra.run.smithi005.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 9.5 MB/s | 3.6 MB 00:00 2024-09-15T21:56:35.814 INFO:teuthology.orchestra.run.smithi005.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 4.9 MB/s | 249 kB 00:00 2024-09-15T21:56:35.856 INFO:teuthology.orchestra.run.smithi005.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 1.2 MB/s | 49 kB 00:00 2024-09-15T21:56:35.857 INFO:teuthology.orchestra.run.smithi099.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:00 2024-09-15T21:56:35.890 INFO:teuthology.orchestra.run.smithi005.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 492 kB/s | 16 kB 00:00 2024-09-15T21:56:35.908 INFO:teuthology.orchestra.run.smithi099.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 160 kB/s | 106 kB 00:00 2024-09-15T21:56:35.940 INFO:teuthology.orchestra.run.smithi005.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 5.6 MB/s | 285 kB 00:00 2024-09-15T21:56:35.990 INFO:teuthology.orchestra.run.smithi129.stdout:(24/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 20 MB/s | 49 MB 00:02 2024-09-15T21:56:36.016 INFO:teuthology.orchestra.run.smithi129.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 213 kB/s | 99 kB 00:00 2024-09-15T21:56:36.032 INFO:teuthology.orchestra.run.smithi005.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 8.1 MB/s | 765 kB 00:00 2024-09-15T21:56:36.044 INFO:teuthology.orchestra.run.smithi129.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.6 MB/s | 85 kB 00:00 2024-09-15T21:56:36.074 INFO:teuthology.orchestra.run.smithi005.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 439 kB/s | 18 kB 00:00 2024-09-15T21:56:36.166 INFO:teuthology.orchestra.run.smithi129.stdout:(27/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 3.3 MB/s | 11 MB 00:03 2024-09-15T21:56:36.183 INFO:teuthology.orchestra.run.smithi099.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 188 kB/s | 135 kB 00:00 2024-09-15T21:56:36.200 INFO:teuthology.orchestra.run.smithi129.stdout:(28/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 1.1 MB/s | 170 kB 00:00 2024-09-15T21:56:36.225 INFO:teuthology.orchestra.run.smithi129.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 546 kB/s | 32 kB 00:00 2024-09-15T21:56:36.250 INFO:teuthology.orchestra.run.smithi129.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.8 MB/s | 146 kB 00:00 2024-09-15T21:56:36.283 INFO:teuthology.orchestra.run.smithi005.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 8.6 MB/s | 7.4 MB 00:00 2024-09-15T21:56:36.317 INFO:teuthology.orchestra.run.smithi129.stdout:(31/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 10 MB/s | 3.1 MB 00:00 2024-09-15T21:56:36.341 INFO:teuthology.orchestra.run.smithi099.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 4.3 MB/s | 689 kB 00:00 2024-09-15T21:56:36.367 INFO:teuthology.orchestra.run.smithi129.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 4.9 MB/s | 249 kB 00:00 2024-09-15T21:56:36.418 INFO:teuthology.orchestra.run.smithi129.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 985 kB/s | 49 kB 00:00 2024-09-15T21:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 7.4 MB/s | 943 kB 00:00 2024-09-15T21:56:36.468 INFO:teuthology.orchestra.run.smithi129.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 329 kB/s | 16 kB 00:00 2024-09-15T21:56:36.527 INFO:teuthology.orchestra.run.smithi129.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 4.8 MB/s | 285 kB 00:00 2024-09-15T21:56:36.533 INFO:teuthology.orchestra.run.smithi099.stdout:(49/156): python3-requests-2.25.1-8.el9.noarch. 185 kB/s | 125 kB 00:00 2024-09-15T21:56:36.559 INFO:teuthology.orchestra.run.smithi099.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.1 MB/s | 107 kB 00:00 2024-09-15T21:56:36.601 INFO:teuthology.orchestra.run.smithi099.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 326 kB/s | 22 kB 00:00 2024-09-15T21:56:36.610 INFO:teuthology.orchestra.run.smithi129.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 21 MB/s | 7.4 MB 00:00 2024-09-15T21:56:36.643 INFO:teuthology.orchestra.run.smithi099.stdout:(52/156): python3-urllib3-1.26.5-6.el9.noarch.r 292 kB/s | 215 kB 00:00 2024-09-15T21:56:36.644 INFO:teuthology.orchestra.run.smithi129.stdout:(37/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-15T21:56:36.661 INFO:teuthology.orchestra.run.smithi129.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 365 kB/s | 18 kB 00:00 2024-09-15T21:56:36.668 INFO:teuthology.orchestra.run.smithi099.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-09-15T21:56:36.693 INFO:teuthology.orchestra.run.smithi099.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-15T21:56:36.735 INFO:teuthology.orchestra.run.smithi099.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-09-15T21:56:36.760 INFO:teuthology.orchestra.run.smithi099.stdout:(56/156): flexiblas-openblas-openmp-3.0.4-8.el9 198 kB/s | 18 kB 00:00 2024-09-15T21:56:36.775 INFO:teuthology.orchestra.run.smithi005.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 114 kB/s | 80 kB 00:00 2024-09-15T21:56:36.795 INFO:teuthology.orchestra.run.smithi129.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 532 kB/s | 80 kB 00:00 2024-09-15T21:56:36.845 INFO:teuthology.orchestra.run.smithi129.stdout:(40/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 5.7 MB/s | 3.6 MB 00:00 2024-09-15T21:56:36.853 INFO:teuthology.orchestra.run.smithi099.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-09-15T21:56:36.878 INFO:teuthology.orchestra.run.smithi099.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-15T21:56:36.892 INFO:teuthology.orchestra.run.smithi129.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-15T21:56:36.917 INFO:teuthology.orchestra.run.smithi129.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.1 MB/s | 801 kB 00:00 2024-09-15T21:56:36.959 INFO:teuthology.orchestra.run.smithi129.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-15T21:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.7 MB/s | 3.0 MB 00:00 2024-09-15T21:56:37.018 INFO:teuthology.orchestra.run.smithi129.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-09-15T21:56:37.020 INFO:teuthology.orchestra.run.smithi099.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-15T21:56:37.043 INFO:teuthology.orchestra.run.smithi129.stdout:(45/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-15T21:56:37.054 INFO:teuthology.orchestra.run.smithi099.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-15T21:56:37.068 INFO:teuthology.orchestra.run.smithi129.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-15T21:56:37.071 INFO:teuthology.orchestra.run.smithi099.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 292 kB/s | 24 kB 00:00 2024-09-15T21:56:37.093 INFO:teuthology.orchestra.run.smithi129.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-15T21:56:37.104 INFO:teuthology.orchestra.run.smithi099.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-09-15T21:56:37.119 INFO:teuthology.orchestra.run.smithi129.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-15T21:56:37.155 INFO:teuthology.orchestra.run.smithi099.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-15T21:56:37.180 INFO:teuthology.orchestra.run.smithi099.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-15T21:56:37.205 INFO:teuthology.orchestra.run.smithi099.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-15T21:56:37.233 INFO:teuthology.orchestra.run.smithi005.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 843 kB/s | 801 kB 00:00 2024-09-15T21:56:37.286 INFO:teuthology.orchestra.run.smithi129.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-09-15T21:56:37.336 INFO:teuthology.orchestra.run.smithi129.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-15T21:56:37.355 INFO:teuthology.orchestra.run.smithi099.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-15T21:56:37.389 INFO:teuthology.orchestra.run.smithi129.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-15T21:56:37.412 INFO:teuthology.orchestra.run.smithi129.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-09-15T21:56:37.422 INFO:teuthology.orchestra.run.smithi099.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-15T21:56:37.437 INFO:teuthology.orchestra.run.smithi129.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-15T21:56:37.487 INFO:teuthology.orchestra.run.smithi129.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-15T21:56:37.506 INFO:teuthology.orchestra.run.smithi099.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-09-15T21:56:37.521 INFO:teuthology.orchestra.run.smithi129.stdout:(55/156): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-09-15T21:56:37.546 INFO:teuthology.orchestra.run.smithi129.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 190 kB/s | 11 kB 00:00 2024-09-15T21:56:37.548 INFO:teuthology.orchestra.run.smithi099.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-15T21:56:37.567 INFO:teuthology.orchestra.run.smithi005.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 241 kB/s | 191 kB 00:00 2024-09-15T21:56:37.572 INFO:teuthology.orchestra.run.smithi129.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-15T21:56:37.597 INFO:teuthology.orchestra.run.smithi129.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-09-15T21:56:37.640 INFO:teuthology.orchestra.run.smithi099.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 10 MB/s | 4.9 MB 00:00 2024-09-15T21:56:37.689 INFO:teuthology.orchestra.run.smithi129.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-15T21:56:37.714 INFO:teuthology.orchestra.run.smithi129.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.9 MB/s | 581 kB 00:00 2024-09-15T21:56:37.732 INFO:teuthology.orchestra.run.smithi099.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2024-09-15T21:56:37.739 INFO:teuthology.orchestra.run.smithi129.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 488 kB/s | 24 kB 00:00 2024-09-15T21:56:37.765 INFO:teuthology.orchestra.run.smithi129.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-15T21:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 699 kB/s | 87 kB 00:00 2024-09-15T21:56:37.823 INFO:teuthology.orchestra.run.smithi129.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-09-15T21:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 819 kB/s | 61 kB 00:00 2024-09-15T21:56:37.849 INFO:teuthology.orchestra.run.smithi129.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-15T21:56:37.874 INFO:teuthology.orchestra.run.smithi129.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 762 kB/s | 38 kB 00:00 2024-09-15T21:56:37.974 INFO:teuthology.orchestra.run.smithi099.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-15T21:56:38.034 INFO:teuthology.orchestra.run.smithi005.stdout:(42/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 11 MB/s | 49 MB 00:04 2024-09-15T21:56:38.066 INFO:teuthology.orchestra.run.smithi099.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-15T21:56:38.150 INFO:teuthology.orchestra.run.smithi099.stdout:(77/156): postgresql-server-13.16-1.el9.x86_64. 9.0 MB/s | 5.8 MB 00:00 2024-09-15T21:56:38.210 INFO:teuthology.orchestra.run.smithi099.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 181 kB/s | 26 kB 00:00 2024-09-15T21:56:38.233 INFO:teuthology.orchestra.run.smithi099.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 230 kB/s | 19 kB 00:00 2024-09-15T21:56:38.309 INFO:teuthology.orchestra.run.smithi099.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 464 kB/s | 35 kB 00:00 2024-09-15T21:56:38.342 INFO:teuthology.orchestra.run.smithi099.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-15T21:56:38.476 INFO:teuthology.orchestra.run.smithi005.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 204 kB/s | 253 kB 00:01 2024-09-15T21:56:38.493 INFO:teuthology.orchestra.run.smithi099.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 7.8 MB/s | 6.0 MB 00:00 2024-09-15T21:56:38.518 INFO:teuthology.orchestra.run.smithi099.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.4 MB/s | 250 kB 00:00 2024-09-15T21:56:38.583 INFO:teuthology.orchestra.run.smithi129.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.6 MB/s | 3.0 MB 00:01 2024-09-15T21:56:38.668 INFO:teuthology.orchestra.run.smithi099.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 515 kB/s | 77 kB 00:00 2024-09-15T21:56:38.702 INFO:teuthology.orchestra.run.smithi099.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 222 kB 00:00 2024-09-15T21:56:38.726 INFO:teuthology.orchestra.run.smithi005.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 154 kB/s | 106 kB 00:00 2024-09-15T21:56:38.760 INFO:teuthology.orchestra.run.smithi099.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 42 kB 00:00 2024-09-15T21:56:38.827 INFO:teuthology.orchestra.run.smithi099.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-09-15T21:56:38.903 INFO:teuthology.orchestra.run.smithi099.stdout:(88/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.6 MB/s | 5.1 MB 00:00 2024-09-15T21:56:38.918 INFO:teuthology.orchestra.run.smithi005.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 930 kB/s | 1.2 MB 00:01 2024-09-15T21:56:38.936 INFO:teuthology.orchestra.run.smithi099.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 903 kB/s | 159 kB 00:00 2024-09-15T21:56:38.995 INFO:teuthology.orchestra.run.smithi099.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 556 kB/s | 51 kB 00:00 2024-09-15T21:56:39.020 INFO:teuthology.orchestra.run.smithi099.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 643 kB/s | 54 kB 00:00 2024-09-15T21:56:39.045 INFO:teuthology.orchestra.run.smithi099.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-09-15T21:56:39.095 INFO:teuthology.orchestra.run.smithi099.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 556 kB/s | 42 kB 00:00 2024-09-15T21:56:39.121 INFO:teuthology.orchestra.run.smithi099.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 588 kB/s | 44 kB 00:00 2024-09-15T21:56:39.177 INFO:teuthology.orchestra.run.smithi005.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 193 kB/s | 135 kB 00:00 2024-09-15T21:56:39.179 INFO:teuthology.orchestra.run.smithi099.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 618 kB/s | 36 kB 00:00 2024-09-15T21:56:39.205 INFO:teuthology.orchestra.run.smithi099.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-15T21:56:39.255 INFO:teuthology.orchestra.run.smithi099.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-09-15T21:56:39.305 INFO:teuthology.orchestra.run.smithi099.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 3.0 MB/s | 304 kB 00:00 2024-09-15T21:56:39.322 INFO:teuthology.orchestra.run.smithi099.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 867 kB/s | 58 kB 00:00 2024-09-15T21:56:39.358 INFO:teuthology.orchestra.run.smithi129.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-09-15T21:56:39.389 INFO:teuthology.orchestra.run.smithi099.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.1 MB/s | 96 kB 00:00 2024-09-15T21:56:39.406 INFO:teuthology.orchestra.run.smithi099.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 763 kB/s | 64 kB 00:00 2024-09-15T21:56:39.410 INFO:teuthology.orchestra.run.smithi005.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 183 kB/s | 125 kB 00:00 2024-09-15T21:56:39.464 INFO:teuthology.orchestra.run.smithi099.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 598 kB/s | 45 kB 00:00 2024-09-15T21:56:39.677 INFO:teuthology.orchestra.run.smithi005.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 283 kB/s | 215 kB 00:00 2024-09-15T21:56:39.790 INFO:teuthology.orchestra.run.smithi099.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-15T21:56:39.791 INFO:teuthology.orchestra.run.smithi129.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 4.4 MB 00:01 2024-09-15T21:56:39.807 INFO:teuthology.orchestra.run.smithi099.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 56 kB/s | 22 kB 00:00 2024-09-15T21:56:39.858 INFO:teuthology.orchestra.run.smithi129.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 1.8 MB/s | 910 kB 00:00 2024-09-15T21:56:39.865 INFO:teuthology.orchestra.run.smithi099.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 7.2 MB/s | 548 kB 00:00 2024-09-15T21:56:39.882 INFO:teuthology.orchestra.run.smithi099.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-15T21:56:39.916 INFO:teuthology.orchestra.run.smithi099.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.8 MB/s | 308 kB 00:00 2024-09-15T21:56:39.925 INFO:teuthology.orchestra.run.smithi129.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 2.4 MB/s | 4.9 MB 00:02 2024-09-15T21:56:39.941 INFO:teuthology.orchestra.run.smithi099.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-15T21:56:39.966 INFO:teuthology.orchestra.run.smithi099.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-15T21:56:39.976 INFO:teuthology.orchestra.run.smithi129.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 748 kB/s | 137 kB 00:00 2024-09-15T21:56:39.991 INFO:teuthology.orchestra.run.smithi099.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-15T21:56:40.034 INFO:teuthology.orchestra.run.smithi099.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 3.5 MB/s | 151 kB 00:00 2024-09-15T21:56:40.059 INFO:teuthology.orchestra.run.smithi129.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-15T21:56:40.077 INFO:teuthology.orchestra.run.smithi005.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 765 kB/s | 689 kB 00:00 2024-09-15T21:56:40.109 INFO:teuthology.orchestra.run.smithi099.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-15T21:56:40.167 INFO:teuthology.orchestra.run.smithi099.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-15T21:56:40.192 INFO:teuthology.orchestra.run.smithi099.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T21:56:40.218 INFO:teuthology.orchestra.run.smithi099.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-15T21:56:40.285 INFO:teuthology.orchestra.run.smithi099.stdout:(116/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-15T21:56:40.310 INFO:teuthology.orchestra.run.smithi099.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 154 kB/s | 14 kB 00:00 2024-09-15T21:56:40.336 INFO:teuthology.orchestra.run.smithi099.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-15T21:56:40.353 INFO:teuthology.orchestra.run.smithi005.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-15T21:56:40.369 INFO:teuthology.orchestra.run.smithi099.stdout:(119/156): protobuf-compiler-3.14.0-13.el9.x86_ 954 kB/s | 863 kB 00:00 2024-09-15T21:56:40.378 INFO:teuthology.orchestra.run.smithi005.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 153 kB/s | 107 kB 00:00 2024-09-15T21:56:40.403 INFO:teuthology.orchestra.run.smithi099.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-15T21:56:40.428 INFO:teuthology.orchestra.run.smithi099.stdout:(121/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-09-15T21:56:40.462 INFO:teuthology.orchestra.run.smithi099.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-15T21:56:40.468 INFO:teuthology.orchestra.run.smithi129.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.9 MB/s | 1.0 MB 00:00 2024-09-15T21:56:40.487 INFO:teuthology.orchestra.run.smithi099.stdout:(123/156): python3-google-auth-2.34.0-1.el9.noa 1.8 MB/s | 220 kB 00:00 2024-09-15T21:56:40.512 INFO:teuthology.orchestra.run.smithi099.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-15T21:56:40.518 INFO:teuthology.orchestra.run.smithi129.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-15T21:56:40.537 INFO:teuthology.orchestra.run.smithi099.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-15T21:56:40.563 INFO:teuthology.orchestra.run.smithi099.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-15T21:56:40.588 INFO:teuthology.orchestra.run.smithi099.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-15T21:56:40.613 INFO:teuthology.orchestra.run.smithi099.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-09-15T21:56:40.638 INFO:teuthology.orchestra.run.smithi099.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T21:56:40.646 INFO:teuthology.orchestra.run.smithi005.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 38 kB/s | 22 kB 00:00 2024-09-15T21:56:40.660 INFO:teuthology.orchestra.run.smithi129.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-15T21:56:40.664 INFO:teuthology.orchestra.run.smithi099.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-15T21:56:40.689 INFO:teuthology.orchestra.run.smithi099.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-15T21:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-15T21:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-15T21:56:40.815 INFO:teuthology.orchestra.run.smithi099.stdout:(134/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.0 MB/s | 2.0 MB 00:00 2024-09-15T21:56:40.819 INFO:teuthology.orchestra.run.smithi129.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-15T21:56:40.840 INFO:teuthology.orchestra.run.smithi099.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-15T21:56:40.869 INFO:teuthology.orchestra.run.smithi099.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-15T21:56:40.870 INFO:teuthology.orchestra.run.smithi129.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-15T21:56:40.899 INFO:teuthology.orchestra.run.smithi099.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 632 kB/s | 90 kB 00:00 2024-09-15T21:56:40.919 INFO:teuthology.orchestra.run.smithi129.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-15T21:56:40.924 INFO:teuthology.orchestra.run.smithi099.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-09-15T21:56:40.929 INFO:teuthology.orchestra.run.smithi005.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-15T21:56:40.950 INFO:teuthology.orchestra.run.smithi099.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-15T21:56:40.975 INFO:teuthology.orchestra.run.smithi099.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-15T21:56:41.000 INFO:teuthology.orchestra.run.smithi099.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-15T21:56:41.033 INFO:teuthology.orchestra.run.smithi099.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2024-09-15T21:56:41.059 INFO:teuthology.orchestra.run.smithi099.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-15T21:56:41.086 INFO:teuthology.orchestra.run.smithi129.stdout:(79/156): postgresql-server-13.16-1.el9.x86_64. 4.7 MB/s | 5.8 MB 00:01 2024-09-15T21:56:41.093 INFO:teuthology.orchestra.run.smithi099.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-15T21:56:41.109 INFO:teuthology.orchestra.run.smithi099.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-15T21:56:41.111 INFO:teuthology.orchestra.run.smithi129.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 898 kB/s | 172 kB 00:00 2024-09-15T21:56:41.135 INFO:teuthology.orchestra.run.smithi099.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-15T21:56:41.137 INFO:teuthology.orchestra.run.smithi005.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 400 kB/s | 313 kB 00:00 2024-09-15T21:56:41.168 INFO:teuthology.orchestra.run.smithi099.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-15T21:56:41.187 INFO:teuthology.orchestra.run.smithi129.stdout:(81/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.3 MB/s | 6.0 MB 00:01 2024-09-15T21:56:41.204 INFO:teuthology.orchestra.run.smithi129.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 296 kB/s | 35 kB 00:00 2024-09-15T21:56:41.227 INFO:teuthology.orchestra.run.smithi099.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-15T21:56:41.254 INFO:teuthology.orchestra.run.smithi129.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-15T21:56:41.279 INFO:teuthology.orchestra.run.smithi129.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-15T21:56:41.346 INFO:teuthology.orchestra.run.smithi129.stdout:(85/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 5.1 MB 00:00 2024-09-15T21:56:41.372 INFO:teuthology.orchestra.run.smithi129.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 459 kB/s | 42 kB 00:00 2024-09-15T21:56:41.397 INFO:teuthology.orchestra.run.smithi129.stdout:(87/156): python3-packaging-20.9-5.el9.noarch.r 542 kB/s | 77 kB 00:00 2024-09-15T21:56:41.420 INFO:teuthology.orchestra.run.smithi005.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 37 kB/s | 18 kB 00:00 2024-09-15T21:56:41.422 INFO:teuthology.orchestra.run.smithi129.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-09-15T21:56:41.428 INFO:teuthology.orchestra.run.smithi099.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-15T21:56:41.447 INFO:teuthology.orchestra.run.smithi129.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-15T21:56:41.472 INFO:teuthology.orchestra.run.smithi129.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-15T21:56:41.498 INFO:teuthology.orchestra.run.smithi129.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-15T21:56:41.503 INFO:teuthology.orchestra.run.smithi099.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 9.5 MB/s | 3.2 MB 00:00 2024-09-15T21:56:41.532 INFO:teuthology.orchestra.run.smithi129.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 637 kB/s | 54 kB 00:00 2024-09-15T21:56:41.557 INFO:teuthology.orchestra.run.smithi129.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 707 kB/s | 42 kB 00:00 2024-09-15T21:56:41.591 INFO:teuthology.orchestra.run.smithi129.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 756 kB/s | 44 kB 00:00 2024-09-15T21:56:41.616 INFO:teuthology.orchestra.run.smithi129.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-15T21:56:41.638 INFO:teuthology.orchestra.run.smithi005.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-09-15T21:56:41.641 INFO:teuthology.orchestra.run.smithi129.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 712 kB/s | 36 kB 00:00 2024-09-15T21:56:41.667 INFO:teuthology.orchestra.run.smithi129.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-15T21:56:41.700 INFO:teuthology.orchestra.run.smithi129.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-15T21:56:41.721 INFO:teuthology.orchestra.run.smithi005.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-09-15T21:56:41.725 INFO:teuthology.orchestra.run.smithi129.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-09-15T21:56:41.759 INFO:teuthology.orchestra.run.smithi129.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-15T21:56:41.785 INFO:teuthology.orchestra.run.smithi129.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-15T21:56:41.810 INFO:teuthology.orchestra.run.smithi129.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 889 kB/s | 45 kB 00:00 2024-09-15T21:56:42.035 INFO:teuthology.orchestra.run.smithi129.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-15T21:56:42.070 INFO:teuthology.orchestra.run.smithi099.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 73 kB/s | 41 kB 00:00 2024-09-15T21:56:42.097 INFO:teuthology.orchestra.run.smithi129.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-09-15T21:56:42.144 INFO:teuthology.orchestra.run.smithi129.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 6.0 MB/s | 308 kB 00:00 2024-09-15T21:56:42.146 INFO:teuthology.orchestra.run.smithi005.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 221 kB/s | 160 kB 00:00 2024-09-15T21:56:42.169 INFO:teuthology.orchestra.run.smithi129.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-15T21:56:42.230 INFO:teuthology.orchestra.run.smithi005.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2024-09-15T21:56:42.312 INFO:teuthology.orchestra.run.smithi099.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-09-15T21:56:42.436 INFO:teuthology.orchestra.run.smithi129.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-15T21:56:42.437 INFO:teuthology.orchestra.run.smithi099.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-09-15T21:56:42.461 INFO:teuthology.orchestra.run.smithi129.stdout:(108/156): lua-devel-5.4.4-4.el9.x86_64.rpm 33 kB/s | 22 kB 00:00 2024-09-15T21:56:42.487 INFO:teuthology.orchestra.run.smithi129.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 531 kB/s | 26 kB 00:00 2024-09-15T21:56:42.512 INFO:teuthology.orchestra.run.smithi129.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 3.8 MB/s | 67 kB 00:00 2024-09-15T21:56:42.545 INFO:teuthology.orchestra.run.smithi129.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.7 MB/s | 151 kB 00:00 2024-09-15T21:56:42.571 INFO:teuthology.orchestra.run.smithi129.stdout:(112/156): liboath-2.6.7-2.el9.x86_64.rpm 445 kB/s | 49 kB 00:00 2024-09-15T21:56:42.589 INFO:teuthology.orchestra.run.smithi005.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 765 kB/s | 662 kB 00:00 2024-09-15T21:56:42.621 INFO:teuthology.orchestra.run.smithi129.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-15T21:56:42.646 INFO:teuthology.orchestra.run.smithi129.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T21:56:42.672 INFO:teuthology.orchestra.run.smithi129.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-15T21:56:42.697 INFO:teuthology.orchestra.run.smithi129.stdout:(116/156): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-09-15T21:56:42.723 INFO:teuthology.orchestra.run.smithi129.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-15T21:56:42.748 INFO:teuthology.orchestra.run.smithi129.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-15T21:56:42.781 INFO:teuthology.orchestra.run.smithi129.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-15T21:56:42.797 INFO:teuthology.orchestra.run.smithi005.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 43 kB/s | 24 kB 00:00 2024-09-15T21:56:42.807 INFO:teuthology.orchestra.run.smithi129.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-09-15T21:56:42.832 INFO:teuthology.orchestra.run.smithi129.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.2 MB/s | 220 kB 00:00 2024-09-15T21:56:42.854 INFO:teuthology.orchestra.run.smithi099.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 317 kB/s | 249 kB 00:00 2024-09-15T21:56:42.857 INFO:teuthology.orchestra.run.smithi129.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-15T21:56:42.883 INFO:teuthology.orchestra.run.smithi129.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-15T21:56:42.908 INFO:teuthology.orchestra.run.smithi129.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 714 kB/s | 18 kB 00:00 2024-09-15T21:56:42.933 INFO:teuthology.orchestra.run.smithi129.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 928 kB/s | 23 kB 00:00 2024-09-15T21:56:42.967 INFO:teuthology.orchestra.run.smithi129.stdout:(126/156): protobuf-compiler-3.14.0-13.el9.x86_ 746 kB/s | 863 kB 00:01 2024-09-15T21:56:42.992 INFO:teuthology.orchestra.run.smithi129.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 333 kB/s | 19 kB 00:00 2024-09-15T21:56:43.014 INFO:teuthology.orchestra.run.smithi005.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 670 kB/s | 581 kB 00:00 2024-09-15T21:56:43.042 INFO:teuthology.orchestra.run.smithi129.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-09-15T21:56:43.085 INFO:teuthology.orchestra.run.smithi129.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.3 MB/s | 2.0 MB 00:00 2024-09-15T21:56:43.110 INFO:teuthology.orchestra.run.smithi129.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 690 kB/s | 46 kB 00:00 2024-09-15T21:56:43.135 INFO:teuthology.orchestra.run.smithi129.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-15T21:56:43.156 INFO:teuthology.orchestra.run.smithi005.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 84 kB/s | 47 kB 00:00 2024-09-15T21:56:43.160 INFO:teuthology.orchestra.run.smithi129.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-15T21:56:43.186 INFO:teuthology.orchestra.run.smithi129.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 654 kB/s | 16 kB 00:00 2024-09-15T21:56:43.211 INFO:teuthology.orchestra.run.smithi129.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-15T21:56:43.236 INFO:teuthology.orchestra.run.smithi129.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-15T21:56:43.262 INFO:teuthology.orchestra.run.smithi099.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.4 MB/s | 1.4 MB 00:00 2024-09-15T21:56:43.270 INFO:teuthology.orchestra.run.smithi129.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 3.8 MB/s | 1.0 MB 00:00 2024-09-15T21:56:43.295 INFO:teuthology.orchestra.run.smithi129.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-09-15T21:56:43.320 INFO:teuthology.orchestra.run.smithi129.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-09-15T21:56:43.346 INFO:teuthology.orchestra.run.smithi129.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-15T21:56:43.371 INFO:teuthology.orchestra.run.smithi129.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-15T21:56:43.396 INFO:teuthology.orchestra.run.smithi129.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-15T21:56:43.422 INFO:teuthology.orchestra.run.smithi129.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-15T21:56:43.447 INFO:teuthology.orchestra.run.smithi129.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-15T21:56:43.472 INFO:teuthology.orchestra.run.smithi129.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-15T21:56:43.487 INFO:teuthology.orchestra.run.smithi099.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-15T21:56:43.489 INFO:teuthology.orchestra.run.smithi129.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 476 kB/s | 20 kB 00:00 2024-09-15T21:56:43.495 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:56:43.495 INFO:teuthology.orchestra.run.smithi099.stdout:Total 16 MB/s | 226 MB 00:14 2024-09-15T21:56:43.514 INFO:teuthology.orchestra.run.smithi129.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-15T21:56:43.540 INFO:teuthology.orchestra.run.smithi129.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-15T21:56:43.590 INFO:teuthology.orchestra.run.smithi129.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-15T21:56:43.657 INFO:teuthology.orchestra.run.smithi005.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 341 kB/s | 292 kB 00:00 2024-09-15T21:56:43.715 INFO:teuthology.orchestra.run.smithi005.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-15T21:56:43.740 INFO:teuthology.orchestra.run.smithi005.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 260 kB/s | 188 kB 00:00 2024-09-15T21:56:43.809 INFO:teuthology.orchestra.run.smithi129.stdout:(149/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 12 MB/s | 3.2 MB 00:00 2024-09-15T21:56:43.891 INFO:teuthology.orchestra.run.smithi129.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 7.7 MB/s | 2.3 MB 00:00 2024-09-15T21:56:43.941 INFO:teuthology.orchestra.run.smithi129.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 817 kB/s | 41 kB 00:00 2024-09-15T21:56:43.991 INFO:teuthology.orchestra.run.smithi129.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-15T21:56:44.058 INFO:teuthology.orchestra.run.smithi129.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 7.2 MB/s | 1.8 MB 00:00 2024-09-15T21:56:44.117 INFO:teuthology.orchestra.run.smithi129.stdout:(154/156): librados2-19.1.1-326.g8921c8ef.el9.x 5.7 MB/s | 3.4 MB 00:00 2024-09-15T21:56:44.159 INFO:teuthology.orchestra.run.smithi129.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 8.1 MB/s | 1.4 MB 00:00 2024-09-15T21:56:44.217 INFO:teuthology.orchestra.run.smithi129.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-09-15T21:56:44.224 INFO:teuthology.orchestra.run.smithi129.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:56:44.224 INFO:teuthology.orchestra.run.smithi129.stdout:Total 16 MB/s | 226 MB 00:13 2024-09-15T21:56:44.729 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-15T21:56:44.850 INFO:teuthology.orchestra.run.smithi005.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.1 MB/s | 4.9 MB 00:01 2024-09-15T21:56:44.882 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-15T21:56:44.882 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-15T21:56:44.933 INFO:teuthology.orchestra.run.smithi005.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.3 MB/s | 1.6 MB 00:01 2024-09-15T21:56:45.000 INFO:teuthology.orchestra.run.smithi005.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-09-15T21:56:45.453 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction check 2024-09-15T21:56:45.606 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction check succeeded. 2024-09-15T21:56:45.606 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction test 2024-09-15T21:56:45.633 INFO:teuthology.orchestra.run.smithi005.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 196 kB/s | 137 kB 00:00 2024-09-15T21:56:45.759 INFO:teuthology.orchestra.run.smithi005.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 1.0 MB/s | 910 kB 00:00 2024-09-15T21:56:46.126 INFO:teuthology.orchestra.run.smithi005.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 5.1 MB/s | 5.8 MB 00:01 2024-09-15T21:56:46.409 INFO:teuthology.orchestra.run.smithi005.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 134 kB/s | 87 kB 00:00 2024-09-15T21:56:46.706 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-15T21:56:46.706 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-15T21:56:47.018 INFO:teuthology.orchestra.run.smithi005.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 101 kB/s | 61 kB 00:00 2024-09-15T21:56:47.268 INFO:teuthology.orchestra.run.smithi005.stdout:(75/156): protobuf-3.14.0-13.el9.x86_64.rpm 632 kB/s | 1.0 MB 00:01 2024-09-15T21:56:47.401 INFO:teuthology.orchestra.run.smithi129.stdout:Transaction test succeeded. 2024-09-15T21:56:47.401 INFO:teuthology.orchestra.run.smithi129.stdout:Running transaction 2024-09-15T21:56:47.843 INFO:teuthology.orchestra.run.smithi005.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 297 kB/s | 245 kB 00:00 2024-09-15T21:56:48.085 INFO:teuthology.orchestra.run.smithi005.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 304 kB/s | 248 kB 00:00 2024-09-15T21:56:48.452 INFO:teuthology.orchestra.run.smithi005.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 42 kB/s | 26 kB 00:00 2024-09-15T21:56:48.577 INFO:teuthology.orchestra.run.smithi005.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 40 kB/s | 19 kB 00:00 2024-09-15T21:56:49.177 INFO:teuthology.orchestra.run.smithi005.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 58 kB/s | 35 kB 00:00 2024-09-15T21:56:49.253 INFO:teuthology.orchestra.run.smithi005.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 215 kB/s | 172 kB 00:00 2024-09-15T21:56:49.478 INFO:teuthology.orchestra.run.smithi005.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 1.8 MB/s | 6.0 MB 00:03 2024-09-15T21:56:50.120 INFO:teuthology.orchestra.run.smithi005.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 289 kB/s | 250 kB 00:00 2024-09-15T21:56:50.312 INFO:teuthology.orchestra.run.smithi005.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 270 kB/s | 222 kB 00:00 2024-09-15T21:56:50.804 INFO:teuthology.orchestra.run.smithi005.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 113 kB/s | 77 kB 00:00 2024-09-15T21:56:50.879 INFO:teuthology.orchestra.run.smithi005.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 75 kB/s | 42 kB 00:00 2024-09-15T21:56:51.588 INFO:teuthology.orchestra.run.smithi005.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 344 kB/s | 269 kB 00:00 2024-09-15T21:56:51.613 INFO:teuthology.orchestra.run.smithi005.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 216 kB/s | 159 kB 00:00 2024-09-15T21:56:51.680 INFO:teuthology.orchestra.run.smithi005.stdout:(89/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.0 MB/s | 5.1 MB 00:02 2024-09-15T21:56:52.247 INFO:teuthology.orchestra.run.smithi005.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 80 kB/s | 51 kB 00:00 2024-09-15T21:56:52.272 INFO:teuthology.orchestra.run.smithi005.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 91 kB/s | 54 kB 00:00 2024-09-15T21:56:52.464 INFO:teuthology.orchestra.run.smithi005.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 319 kB/s | 279 kB 00:00 2024-09-15T21:56:52.856 INFO:teuthology.orchestra.run.smithi005.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 71 kB/s | 42 kB 00:00 2024-09-15T21:56:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 75 kB/s | 44 kB 00:00 2024-09-15T21:56:53.317 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-15T21:56:53.415 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T21:56:53.453 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T21:56:53.507 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T21:56:53.549 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T21:56:53.583 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T21:56:53.625 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T21:56:53.713 INFO:teuthology.orchestra.run.smithi005.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-15T21:56:53.740 INFO:teuthology.orchestra.run.smithi005.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 53 kB/s | 36 kB 00:00 2024-09-15T21:56:53.766 INFO:teuthology.orchestra.run.smithi005.stdout:(97/156): qatlib-24.02.0-1.el9.x86_64.rpm 243 kB/s | 221 kB 00:00 2024-09-15T21:56:53.838 INFO:teuthology.orchestra.run.smithi129.stdout: Preparing : 1/1 2024-09-15T21:56:53.913 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T21:56:53.949 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T21:56:53.960 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T21:56:54.013 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T21:56:54.047 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T21:56:54.068 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:56:54.072 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T21:56:54.115 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T21:56:54.189 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:56:54.218 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:56:54.290 INFO:teuthology.orchestra.run.smithi005.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-15T21:56:54.309 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:56:54.339 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T21:56:54.374 INFO:teuthology.orchestra.run.smithi005.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 95 kB/s | 58 kB 00:00 2024-09-15T21:56:54.445 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T21:56:54.489 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T21:56:54.516 INFO:teuthology.orchestra.run.smithi005.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 392 kB/s | 304 kB 00:00 2024-09-15T21:56:54.544 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T21:56:54.557 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:56:54.572 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:56:54.676 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:56:54.679 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:56:54.698 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T21:56:54.700 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:56:54.761 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T21:56:54.782 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T21:56:54.791 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:56:54.804 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T21:56:54.821 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T21:56:54.997 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T21:56:55.009 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T21:56:55.033 INFO:teuthology.orchestra.run.smithi005.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 129 kB/s | 96 kB 00:00 2024-09-15T21:56:55.051 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T21:56:55.067 INFO:teuthology.orchestra.run.smithi005.stdout:(102/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 92 kB/s | 64 kB 00:00 2024-09-15T21:56:55.079 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:56:55.092 INFO:teuthology.orchestra.run.smithi005.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2024-09-15T21:56:55.151 INFO:teuthology.orchestra.run.smithi005.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-15T21:56:55.151 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T21:56:55.176 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:56:55.193 INFO:teuthology.orchestra.run.smithi005.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-15T21:56:55.205 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T21:56:55.218 INFO:teuthology.orchestra.run.smithi005.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-15T21:56:55.252 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T21:56:55.274 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T21:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T21:56:55.295 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T21:56:55.352 INFO:teuthology.orchestra.run.smithi005.stdout:(107/156): lua-devel-5.4.4-4.el9.x86_64.rpm 70 kB/s | 22 kB 00:00 2024-09-15T21:56:55.367 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:56:55.443 INFO:teuthology.orchestra.run.smithi005.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-15T21:56:55.444 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:56:55.468 INFO:teuthology.orchestra.run.smithi005.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-09-15T21:56:55.491 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T21:56:55.494 INFO:teuthology.orchestra.run.smithi005.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-09-15T21:56:55.528 INFO:teuthology.orchestra.run.smithi005.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-15T21:56:55.638 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T21:56:55.672 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:56:55.728 INFO:teuthology.orchestra.run.smithi005.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 648 kB/s | 151 kB 00:00 2024-09-15T21:56:55.747 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T21:56:55.787 INFO:teuthology.orchestra.run.smithi005.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.2 MB/s | 839 kB 00:00 2024-09-15T21:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T21:56:55.820 INFO:teuthology.orchestra.run.smithi005.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-09-15T21:56:55.845 INFO:teuthology.orchestra.run.smithi005.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-09-15T21:56:55.853 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:56:55.871 INFO:teuthology.orchestra.run.smithi005.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 640 kB/s | 32 kB 00:00 2024-09-15T21:56:55.880 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T21:56:55.896 INFO:teuthology.orchestra.run.smithi005.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-15T21:56:55.916 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:56:55.921 INFO:teuthology.orchestra.run.smithi005.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-15T21:56:55.924 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T21:56:55.956 INFO:teuthology.orchestra.run.smithi005.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.9 MB/s | 358 kB 00:00 2024-09-15T21:56:55.959 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T21:56:55.989 INFO:teuthology.orchestra.run.smithi005.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-15T21:56:56.031 INFO:teuthology.orchestra.run.smithi005.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-15T21:56:56.032 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T21:56:56.065 INFO:teuthology.orchestra.run.smithi005.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-15T21:56:56.076 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T21:56:56.090 INFO:teuthology.orchestra.run.smithi005.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-15T21:56:56.108 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T21:56:56.115 INFO:teuthology.orchestra.run.smithi005.stdout:(124/156): protobuf-compiler-3.14.0-13.el9.x86_ 825 kB/s | 863 kB 00:01 2024-09-15T21:56:56.132 INFO:teuthology.orchestra.run.smithi005.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-09-15T21:56:56.140 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:56:56.151 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T21:56:56.157 INFO:teuthology.orchestra.run.smithi005.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-15T21:56:56.174 INFO:teuthology.orchestra.run.smithi005.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.3 MB/s | 20 kB 00:00 2024-09-15T21:56:56.184 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T21:56:56.191 INFO:teuthology.orchestra.run.smithi005.stdout:(128/156): python3-jaraco-collections-3.0.0-8.e 306 kB/s | 23 kB 00:00 2024-09-15T21:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T21:56:56.241 INFO:teuthology.orchestra.run.smithi005.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-09-15T21:56:56.246 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T21:56:56.258 INFO:teuthology.orchestra.run.smithi005.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 692 kB/s | 46 kB 00:00 2024-09-15T21:56:56.284 INFO:teuthology.orchestra.run.smithi005.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-09-15T21:56:56.298 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T21:56:56.309 INFO:teuthology.orchestra.run.smithi005.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-15T21:56:56.334 INFO:teuthology.orchestra.run.smithi005.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-15T21:56:56.340 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T21:56:56.351 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T21:56:56.360 INFO:teuthology.orchestra.run.smithi005.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-15T21:56:56.379 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T21:56:56.382 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T21:56:56.385 INFO:teuthology.orchestra.run.smithi005.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-15T21:56:56.410 INFO:teuthology.orchestra.run.smithi005.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-15T21:56:56.417 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T21:56:56.422 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T21:56:56.435 INFO:teuthology.orchestra.run.smithi005.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-15T21:56:56.469 INFO:teuthology.orchestra.run.smithi005.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-15T21:56:56.494 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T21:56:56.494 INFO:teuthology.orchestra.run.smithi005.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-15T21:56:56.495 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T21:56:56.528 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T21:56:56.531 INFO:teuthology.orchestra.run.smithi005.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-15T21:56:56.539 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T21:56:56.556 INFO:teuthology.orchestra.run.smithi005.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-15T21:56:56.571 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T21:56:56.572 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T21:56:56.590 INFO:teuthology.orchestra.run.smithi005.stdout:(142/156): python3-kubernetes-26.1.0-2.el9.noar 2.5 MB/s | 1.0 MB 00:00 2024-09-15T21:56:56.599 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T21:56:56.615 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T21:56:56.616 INFO:teuthology.orchestra.run.smithi005.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-15T21:56:56.641 INFO:teuthology.orchestra.run.smithi005.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-15T21:56:56.648 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T21:56:56.666 INFO:teuthology.orchestra.run.smithi005.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-15T21:56:56.677 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T21:56:56.677 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T21:56:56.699 INFO:teuthology.orchestra.run.smithi005.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-15T21:56:56.731 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T21:56:56.733 INFO:teuthology.orchestra.run.smithi005.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-15T21:56:56.745 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T21:56:56.784 INFO:teuthology.orchestra.run.smithi005.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-15T21:56:56.793 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T21:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T21:56:56.825 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T21:56:56.868 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T21:56:56.937 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T21:56:56.944 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T21:56:56.979 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T21:56:56.991 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T21:56:57.022 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T21:56:57.051 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T21:56:57.084 INFO:teuthology.orchestra.run.smithi005.stdout:(149/156): librados2-19.1.1-326.g8921c8ef.el9.x 8.9 MB/s | 3.4 MB 00:00 2024-09-15T21:56:57.121 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T21:56:57.151 INFO:teuthology.orchestra.run.smithi005.stdout:(150/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 7.6 MB/s | 3.2 MB 00:00 2024-09-15T21:56:57.166 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T21:56:57.241 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T21:56:57.373 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T21:56:57.427 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T21:56:57.517 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T21:56:57.603 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T21:56:57.632 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T21:56:57.675 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T21:56:57.713 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T21:56:57.726 INFO:teuthology.orchestra.run.smithi005.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 71 kB/s | 41 kB 00:00 2024-09-15T21:56:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T21:56:57.876 INFO:teuthology.orchestra.run.smithi005.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-09-15T21:56:57.922 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T21:56:57.949 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T21:56:57.991 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T21:56:58.031 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T21:56:58.045 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T21:56:58.085 INFO:teuthology.orchestra.run.smithi005.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 1.8 MB/s | 1.8 MB 00:01 2024-09-15T21:56:58.085 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T21:56:58.118 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T21:56:58.128 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T21:56:58.150 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T21:56:58.166 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T21:56:58.186 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T21:56:58.208 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T21:56:58.241 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T21:56:58.285 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T21:56:58.328 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T21:56:58.371 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T21:56:58.371 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T21:56:58.436 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T21:56:58.491 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T21:56:58.493 INFO:teuthology.orchestra.run.smithi005.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 324 kB/s | 249 kB 00:00 2024-09-15T21:56:58.506 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T21:56:58.556 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T21:56:58.563 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T21:56:58.595 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T21:56:58.597 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T21:56:58.632 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T21:56:58.679 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T21:56:58.698 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T21:56:58.728 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T21:56:58.739 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T21:56:58.771 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T21:56:58.778 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T21:56:58.804 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T21:56:58.813 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T21:56:58.948 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T21:56:58.999 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T21:56:59.040 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T21:56:59.063 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T21:56:59.142 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T21:56:59.182 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T21:56:59.218 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T21:56:59.221 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T21:56:59.248 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T21:56:59.272 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T21:56:59.341 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T21:56:59.346 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T21:56:59.473 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T21:56:59.505 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T21:56:59.533 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T21:56:59.567 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T21:56:59.587 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T21:56:59.673 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T21:56:59.739 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T21:56:59.773 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T21:56:59.821 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T21:56:59.869 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T21:56:59.880 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T21:56:59.901 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T21:56:59.944 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T21:56:59.977 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T21:56:59.986 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T21:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T21:57:00.020 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T21:57:00.043 INFO:teuthology.orchestra.run.smithi005.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 640 kB/s | 1.4 MB 00:02 2024-09-15T21:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T21:57:00.053 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T21:57:00.096 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T21:57:00.130 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T21:57:00.140 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T21:57:00.184 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T21:57:00.215 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T21:57:00.220 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T21:57:00.269 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T21:57:00.299 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T21:57:00.317 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T21:57:00.391 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T21:57:00.404 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T21:57:00.458 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T21:57:01.108 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T21:57:01.154 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T21:57:01.188 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T21:57:01.194 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T21:57:01.239 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T21:57:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T21:57:01.292 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T21:57:01.335 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T21:57:01.388 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T21:57:01.389 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T21:57:01.436 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T21:57:01.481 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T21:57:01.483 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T21:57:01.519 INFO:teuthology.orchestra.run.smithi005.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 713 kB/s | 2.4 MB 00:03 2024-09-15T21:57:01.527 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:57:01.527 INFO:teuthology.orchestra.run.smithi005.stdout:Total 7.0 MB/s | 226 MB 00:32 2024-09-15T21:57:01.532 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T21:57:01.561 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T21:57:01.601 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T21:57:01.724 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T21:57:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T21:57:01.784 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T21:57:01.807 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T21:57:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T21:57:01.885 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T21:57:02.486 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T21:57:02.523 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T21:57:02.603 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T21:57:02.639 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T21:57:02.804 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-09-15T21:57:02.975 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-09-15T21:57:02.976 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-09-15T21:57:03.822 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T21:57:03.866 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T21:57:03.908 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T21:57:03.936 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T21:57:03.956 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T21:57:03.969 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T21:57:03.984 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T21:57:04.017 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T21:57:04.279 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T21:57:04.324 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T21:57:04.324 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T21:57:04.363 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T21:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T21:57:04.454 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T21:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T21:57:04.485 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T21:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T21:57:04.521 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T21:57:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T21:57:05.001 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T21:57:05.026 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T21:57:05.041 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T21:57:05.097 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T21:57:05.112 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T21:57:05.128 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T21:57:05.135 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T21:57:05.297 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-09-15T21:57:05.297 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-09-15T21:57:07.434 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:57:07.446 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:57:07.466 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:57:07.482 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:57:08.017 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:57:08.031 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:57:08.061 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T21:57:08.069 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T21:57:08.137 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T21:57:08.137 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T21:57:08.137 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T21:57:08.137 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:08.171 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T21:57:08.176 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T21:57:08.176 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T21:57:08.177 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T21:57:08.177 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:08.195 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T21:57:11.708 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-09-15T21:57:11.848 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T21:57:11.887 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T21:57:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T21:57:11.983 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T21:57:12.007 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T21:57:12.060 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T21:57:12.407 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T21:57:12.517 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:57:12.633 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:57:12.659 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:57:12.752 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:57:12.781 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T21:57:12.965 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T21:57:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T21:57:13.047 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:57:13.153 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:57:13.181 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T21:57:13.245 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T21:57:13.299 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T21:57:13.322 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T21:57:13.545 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T21:57:13.732 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T21:57:13.848 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T21:57:13.963 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:57:14.033 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:57:14.249 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:57:14.358 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T21:57:14.462 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T21:57:14.505 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T21:57:14.540 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T21:57:14.615 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T21:57:14.666 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T21:57:14.706 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T21:57:14.750 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T21:57:14.782 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T21:57:14.810 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T21:57:14.881 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T21:57:14.942 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T21:57:14.971 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T21:57:15.015 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T21:57:15.094 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T21:57:15.125 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T21:57:15.169 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T21:57:15.197 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T21:57:15.270 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T21:57:15.321 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T21:57:15.407 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T21:57:15.552 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T21:57:15.608 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T21:57:16.171 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T21:57:16.255 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T21:57:16.290 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T21:57:16.333 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T21:57:16.371 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T21:57:16.413 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T21:57:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T21:57:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T21:57:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T21:57:16.770 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T21:57:16.808 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T21:57:16.844 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T21:57:16.892 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T21:57:16.927 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T21:57:16.971 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T21:57:17.012 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T21:57:17.162 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T21:57:17.206 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T21:57:17.247 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T21:57:17.355 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T21:57:17.389 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T21:57:17.430 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T21:57:17.466 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T21:57:17.491 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T21:57:17.581 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T21:57:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T21:57:17.748 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T21:57:17.777 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T21:57:17.816 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T21:57:17.850 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T21:57:17.899 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T21:57:17.937 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T21:57:17.971 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T21:57:18.005 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T21:57:18.055 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T21:57:18.089 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T21:57:18.179 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T21:57:18.236 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T21:57:18.275 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T21:57:18.392 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T21:57:18.452 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T21:57:19.212 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T21:57:19.266 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T21:57:19.302 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T21:57:19.407 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T21:57:19.458 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T21:57:19.503 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T21:57:19.603 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T21:57:19.638 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T21:57:19.830 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T21:57:19.861 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T21:57:19.924 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T21:57:19.924 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /sys 2024-09-15T21:57:19.924 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /proc 2024-09-15T21:57:19.924 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /mnt 2024-09-15T21:57:19.924 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /var/tmp 2024-09-15T21:57:19.924 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /home 2024-09-15T21:57:19.924 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /root 2024-09-15T21:57:19.925 INFO:teuthology.orchestra.run.smithi129.stdout:skipping the directory /tmp 2024-09-15T21:57:19.925 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:19.940 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T21:57:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T21:57:20.068 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-09-15T21:57:20.068 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-09-15T21:57:20.068 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-09-15T21:57:20.068 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-09-15T21:57:20.068 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-09-15T21:57:20.068 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /root 2024-09-15T21:57:20.068 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-09-15T21:57:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:20.102 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T21:57:20.119 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T21:57:20.120 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:20.120 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T21:57:20.120 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T21:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T21:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T21:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:20.626 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T21:57:20.747 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T21:57:21.589 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T21:57:21.616 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T21:57:21.617 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:21.617 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T21:57:21.617 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:57:21.617 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:57:21.617 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:21.716 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T21:57:21.743 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T21:57:21.743 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:21.743 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T21:57:21.743 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:57:21.743 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:57:21.744 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:21.848 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T21:57:21.872 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T21:57:21.872 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:21.872 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T21:57:21.873 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:57:21.873 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:57:21.873 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:21.975 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T21:57:21.999 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T21:57:21.999 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:21.999 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T21:57:22.000 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:57:22.000 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:57:22.000 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:22.066 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T21:57:22.148 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T21:57:22.176 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T21:57:22.218 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T21:57:22.307 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T21:57:22.330 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T21:57:22.331 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:22.331 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T21:57:22.331 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:57:22.331 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:57:22.331 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:22.434 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T21:57:22.437 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T21:57:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T21:57:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T21:57:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:57:22.458 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:57:22.458 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:22.485 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T21:57:22.549 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T21:57:22.552 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T21:57:22.581 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T21:57:22.597 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T21:57:22.646 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T21:57:22.689 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T21:57:22.710 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T21:57:22.719 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T21:57:22.735 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T21:57:22.755 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T21:57:22.784 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T21:57:22.862 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T21:57:22.894 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T21:57:22.935 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T21:57:22.949 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T21:57:23.005 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T21:57:23.089 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T21:57:23.101 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T21:57:23.133 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T21:57:23.165 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T21:57:23.193 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T21:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T21:57:23.264 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T21:57:23.292 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T21:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T21:57:23.347 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T21:57:23.356 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T21:57:23.377 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T21:57:23.386 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T21:57:23.657 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T21:57:23.684 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T21:57:23.715 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T21:57:23.778 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T21:57:23.816 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T21:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T21:57:23.860 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T21:57:23.891 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T21:57:23.953 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T21:57:23.983 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T21:57:24.853 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T21:57:24.892 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T21:57:25.026 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T21:57:25.060 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T21:57:25.783 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:57:25.823 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:57:25.853 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T21:57:25.885 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T21:57:25.987 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T21:57:26.016 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T21:57:26.043 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T21:57:26.145 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T21:57:26.148 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T21:57:26.179 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T21:57:26.203 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T21:57:26.203 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:26.203 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T21:57:26.203 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:57:26.203 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:57:26.203 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:26.254 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T21:57:26.266 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T21:57:26.304 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T21:57:26.345 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T21:57:26.364 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T21:57:26.369 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T21:57:26.370 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:26.370 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T21:57:26.370 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:57:26.370 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:57:26.370 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:26.403 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:57:26.411 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T21:57:26.421 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T21:57:26.433 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T21:57:26.452 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T21:57:26.530 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T21:57:26.543 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T21:57:26.544 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T21:57:26.544 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T21:57:26.544 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:57:26.578 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T21:57:26.586 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T21:57:27.333 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T21:57:27.357 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T21:57:27.381 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T21:57:27.381 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:27.381 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T21:57:27.381 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:57:27.381 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:57:27.381 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:27.425 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T21:57:27.448 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T21:57:27.448 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:27.448 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T21:57:27.448 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:27.501 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T21:57:27.502 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T21:57:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T21:57:27.524 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:27.524 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T21:57:27.524 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T21:57:27.524 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T21:57:27.524 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:27.540 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T21:57:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T21:57:27.564 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:27.564 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T21:57:27.564 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:57:27.564 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:57:27.564 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:27.607 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T21:57:27.630 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T21:57:27.630 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:27.630 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T21:57:27.630 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:27.683 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T21:57:27.706 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T21:57:27.706 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:27.706 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T21:57:27.706 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T21:57:27.706 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T21:57:27.707 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:27.828 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T21:57:27.849 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T21:57:27.849 INFO:teuthology.orchestra.run.smithi129.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:27.850 INFO:teuthology.orchestra.run.smithi129.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T21:57:27.850 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:57:27.850 INFO:teuthology.orchestra.run.smithi129.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:57:27.850 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:28.008 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T21:57:28.032 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T21:57:28.033 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:28.033 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T21:57:28.033 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:57:28.033 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:57:28.033 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:32.381 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T21:57:32.421 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T21:57:32.462 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T21:57:32.495 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T21:57:32.522 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T21:57:32.586 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T21:57:32.628 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T21:57:32.677 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T21:57:32.701 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T21:57:32.717 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T21:57:32.745 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T21:57:32.752 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T21:57:32.783 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T21:57:32.813 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T21:57:32.867 INFO:teuthology.orchestra.run.smithi129.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T21:57:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T21:57:32.966 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T21:57:32.997 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T21:57:33.026 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T21:57:33.072 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T21:57:33.478 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:57:33.516 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:57:33.537 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:57:33.671 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:57:33.728 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:57:33.749 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:57:34.370 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T21:57:34.447 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T21:57:34.488 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T21:57:34.569 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T21:57:34.583 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T21:57:34.608 INFO:teuthology.orchestra.run.smithi129.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T21:57:34.634 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T21:57:34.659 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T21:57:34.660 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T21:57:34.674 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T21:57:34.715 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T21:57:34.744 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T21:57:34.744 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T21:57:34.796 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T21:57:34.828 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T21:57:34.844 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T21:57:34.853 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T21:57:34.858 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:57:34.858 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:57:34.870 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T21:57:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T21:57:34.895 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T21:57:34.989 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T21:57:34.989 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T21:57:35.046 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:57:35.047 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T21:57:35.072 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T21:57:35.097 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T21:57:35.102 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:57:35.102 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:57:35.131 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T21:57:35.197 INFO:teuthology.orchestra.run.smithi129.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:57:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:57:35.282 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T21:57:35.366 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T21:57:35.429 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:57:38.864 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T21:57:38.864 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-09-15T21:57:38.864 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-09-15T21:57:38.865 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-09-15T21:57:38.865 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-09-15T21:57:38.865 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-09-15T21:57:38.865 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-09-15T21:57:38.865 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-09-15T21:57:38.865 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:57:39.043 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T21:57:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T21:57:39.063 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:39.063 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T21:57:39.063 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:57:40.588 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T21:57:40.616 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T21:57:40.616 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:40.616 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T21:57:40.616 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:57:40.616 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:57:40.616 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:57:40.863 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T21:57:40.888 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T21:57:40.889 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:40.889 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T21:57:40.889 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:57:40.889 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:57:40.889 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:57:40.935 INFO:teuthology.orchestra.run.smithi129.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:57:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T21:57:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T21:57:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T21:57:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T21:57:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T21:57:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T21:57:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T21:57:40.936 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T21:57:40.937 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T21:57:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T21:57:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T21:57:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T21:57:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T21:57:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T21:57:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T21:57:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T21:57:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T21:57:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T21:57:40.938 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T21:57:40.939 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T21:57:40.940 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T21:57:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T21:57:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T21:57:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T21:57:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T21:57:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T21:57:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T21:57:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T21:57:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T21:57:40.941 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T21:57:40.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T21:57:40.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T21:57:40.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T21:57:40.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T21:57:40.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T21:57:40.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T21:57:40.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T21:57:40.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T21:57:40.942 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T21:57:40.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T21:57:40.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T21:57:40.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T21:57:40.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T21:57:40.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T21:57:40.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T21:57:40.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T21:57:40.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T21:57:40.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T21:57:40.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T21:57:40.943 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T21:57:40.944 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T21:57:40.945 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T21:57:40.946 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T21:57:40.947 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T21:57:40.948 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T21:57:40.949 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T21:57:40.950 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T21:57:40.950 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T21:57:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:57:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T21:57:41.151 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T21:57:41.152 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T21:57:41.153 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T21:57:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T21:57:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T21:57:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T21:57:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T21:57:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T21:57:41.165 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T21:57:41.166 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T21:57:41.167 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T21:57:41.168 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T21:57:41.169 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T21:57:41.170 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T21:57:41.171 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T21:57:41.172 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T21:57:41.173 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T21:57:41.174 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T21:57:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T21:57:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T21:57:41.175 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T21:57:41.347 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T21:57:41.372 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T21:57:41.373 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:41.373 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T21:57:41.373 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:57:41.373 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:57:41.373 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:57:41.487 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T21:57:41.544 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T21:57:41.609 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T21:57:41.802 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T21:57:41.849 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T21:57:41.929 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T21:57:42.019 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T21:57:42.161 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T21:57:42.216 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T21:57:42.248 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T21:57:42.275 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T21:57:42.766 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T21:57:42.791 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T21:57:42.831 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T21:57:42.896 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T21:57:42.922 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout:Upgraded: 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T21:57:43.394 INFO:teuthology.orchestra.run.smithi129.stdout:Installed: 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.395 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.396 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T21:57:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T21:57:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T21:57:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T21:57:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T21:57:43.397 INFO:teuthology.orchestra.run.smithi129.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T21:57:43.398 INFO:teuthology.orchestra.run.smithi129.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T21:57:43.399 INFO:teuthology.orchestra.run.smithi129.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T21:57:43.400 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T21:57:43.401 INFO:teuthology.orchestra.run.smithi129.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T21:57:43.402 INFO:teuthology.orchestra.run.smithi129.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T21:57:43.403 INFO:teuthology.orchestra.run.smithi129.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T21:57:43.403 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T21:57:43.403 INFO:teuthology.orchestra.run.smithi129.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T21:57:43.403 INFO:teuthology.orchestra.run.smithi129.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T21:57:43.403 INFO:teuthology.orchestra.run.smithi129.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T21:57:43.403 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T21:57:43.403 INFO:teuthology.orchestra.run.smithi129.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T21:57:43.403 INFO:teuthology.orchestra.run.smithi129.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T21:57:43.403 INFO:teuthology.orchestra.run.smithi129.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T21:57:43.403 INFO:teuthology.orchestra.run.smithi129.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.404 INFO:teuthology.orchestra.run.smithi129.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T21:57:43.405 INFO:teuthology.orchestra.run.smithi129.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T21:57:43.406 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.406 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.406 INFO:teuthology.orchestra.run.smithi129.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.406 INFO:teuthology.orchestra.run.smithi129.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T21:57:43.406 INFO:teuthology.orchestra.run.smithi129.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T21:57:43.406 INFO:teuthology.orchestra.run.smithi129.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T21:57:43.406 INFO:teuthology.orchestra.run.smithi129.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T21:57:43.406 INFO:teuthology.orchestra.run.smithi129.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T21:57:43.406 INFO:teuthology.orchestra.run.smithi129.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T21:57:43.407 INFO:teuthology.orchestra.run.smithi129.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T21:57:43.407 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T21:57:43.407 INFO:teuthology.orchestra.run.smithi129.stdout:Complete! 2024-09-15T21:57:43.836 DEBUG:teuthology.parallel:result is None 2024-09-15T21:57:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T21:57:43.858 INFO:teuthology.orchestra.run.smithi099.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T21:57:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T21:57:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T21:57:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.859 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.860 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:57:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T21:57:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T21:57:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T21:57:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T21:57:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T21:57:43.861 INFO:teuthology.orchestra.run.smithi099.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T21:57:43.862 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T21:57:43.863 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T21:57:43.864 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T21:57:43.865 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T21:57:43.866 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T21:57:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T21:57:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T21:57:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T21:57:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T21:57:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T21:57:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T21:57:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T21:57:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T21:57:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T21:57:43.867 INFO:teuthology.orchestra.run.smithi099.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T21:57:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T21:57:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T21:57:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T21:57:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T21:57:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T21:57:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T21:57:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T21:57:43.868 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T21:57:43.869 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T21:57:43.870 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T21:57:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T21:57:43.871 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T21:57:43.871 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-15T21:57:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T21:57:44.065 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T21:57:44.265 DEBUG:teuthology.parallel:result is None 2024-09-15T21:57:45.079 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T21:57:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T21:57:45.218 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T21:57:45.396 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T21:57:45.426 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T21:57:45.452 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T21:57:45.452 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:45.452 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T21:57:45.453 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:57:45.453 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:57:45.453 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:57:45.502 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T21:57:45.514 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T21:57:45.615 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T21:57:45.683 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T21:57:46.663 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T21:57:46.696 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T21:57:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T21:57:46.722 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:46.722 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T21:57:46.722 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:57:46.722 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:57:46.722 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:57:46.763 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T21:57:46.788 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T21:57:46.788 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:46.788 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T21:57:46.789 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:57:46.839 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T21:57:46.864 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T21:57:46.864 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:46.864 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T21:57:46.865 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T21:57:46.865 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T21:57:46.865 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:57:47.181 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T21:57:47.206 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T21:57:47.206 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:57:47.206 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T21:57:47.207 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:57:47.207 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:57:47.207 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:57:52.035 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T21:57:52.085 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T21:57:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T21:57:52.158 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T21:57:52.185 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T21:57:52.395 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T21:57:52.441 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T21:57:52.471 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T21:57:52.526 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T21:57:52.571 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T21:57:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:57:53.219 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:57:53.241 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:57:54.141 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T21:57:54.224 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T21:57:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T21:57:54.375 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T21:57:54.420 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T21:57:54.445 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T21:57:54.479 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T21:57:54.480 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T21:57:54.563 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T21:57:54.564 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T21:57:54.648 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T21:57:54.672 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T21:57:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:57:54.678 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:57:54.875 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:57:54.875 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T21:57:54.942 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T21:57:55.019 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:58:00.715 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:58:00.715 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T21:58:00.715 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T21:58:00.715 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T21:58:00.715 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T21:58:00.715 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T21:58:00.715 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T21:58:00.715 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T21:58:00.716 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T21:58:00.717 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T21:58:00.717 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T21:58:00.717 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T21:58:00.717 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T21:58:00.717 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T21:58:00.717 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T21:58:00.729 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T21:58:00.729 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T21:58:00.729 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T21:58:00.729 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T21:58:00.729 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T21:58:00.729 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T21:58:00.729 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T21:58:00.730 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T21:58:00.730 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T21:58:00.730 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T21:58:00.730 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T21:58:00.730 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T21:58:00.730 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T21:58:00.730 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T21:58:00.730 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T21:58:00.730 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T21:58:00.730 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T21:58:00.731 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T21:58:00.732 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T21:58:00.732 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T21:58:00.732 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T21:58:00.732 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T21:58:00.732 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T21:58:00.732 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T21:58:00.733 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T21:58:00.733 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T21:58:00.733 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T21:58:00.733 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T21:58:00.733 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T21:58:00.733 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T21:58:00.733 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T21:58:00.734 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T21:58:00.735 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T21:58:00.735 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T21:58:00.735 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T21:58:00.735 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T21:58:00.735 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T21:58:00.735 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T21:58:00.735 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T21:58:00.736 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T21:58:00.736 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T21:58:00.736 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T21:58:00.736 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T21:58:00.736 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T21:58:00.736 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T21:58:00.736 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T21:58:00.736 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T21:58:00.737 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T21:58:00.738 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T21:58:00.739 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T21:58:00.740 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T21:58:00.741 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T21:58:00.741 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T21:58:00.741 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T21:58:00.741 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T21:58:00.741 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T21:58:00.741 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T21:58:00.741 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T21:58:00.741 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T21:58:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:58:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:58:03.614 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-09-15T21:58:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T21:58:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T21:58:03.614 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.615 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.616 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T21:58:03.617 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T21:58:03.618 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T21:58:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T21:58:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T21:58:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T21:58:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T21:58:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T21:58:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T21:58:03.619 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T21:58:03.620 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T21:58:03.621 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T21:58:03.622 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T21:58:03.623 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.624 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T21:58:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T21:58:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T21:58:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T21:58:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T21:58:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T21:58:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T21:58:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T21:58:03.625 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T21:58:03.626 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T21:58:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T21:58:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T21:58:03.627 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T21:58:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T21:58:03.629 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:58:03.629 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-09-15T21:58:04.155 DEBUG:teuthology.parallel:result is None 2024-09-15T21:58:04.155 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:58:04.307 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T21:58:04.331 INFO:teuthology.orchestra.run.smithi005.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T21:58:04.332 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T21:58:04.332 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T21:58:04.334 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:58:04.481 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T21:58:04.508 INFO:teuthology.orchestra.run.smithi099.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T21:58:04.509 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T21:58:04.509 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T21:58:04.510 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:58:04.653 DEBUG:teuthology.orchestra.run.smithi129:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T21:58:04.679 INFO:teuthology.orchestra.run.smithi129.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T21:58:04.680 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T21:58:04.680 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T21:58:04.682 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T21:58:04.682 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T21:58:04.682 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:58:04.715 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-15T21:58:04.715 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:58:04.748 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-15T21:58:04.748 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:58:04.780 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T21:58:04.780 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T21:58:04.780 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:58:04.810 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:58:04.878 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-15T21:58:04.878 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:58:04.907 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:58:04.976 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-15T21:58:04.977 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:58:05.008 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:58:05.076 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T21:58:05.076 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T21:58:05.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:58:05.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:58:05.175 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-15T21:58:05.175 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:58:05.202 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:58:05.267 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-15T21:58:05.267 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:58:05.292 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:58:05.357 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T21:58:05.358 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T21:58:05.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:58:05.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:58:05.457 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-15T21:58:05.457 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:58:05.482 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:58:05.548 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-15T21:58:05.548 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:58:05.573 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:58:05.638 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-15T21:58:05.754 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-15T21:58:05.754 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:58:05.755 INFO:tasks.cephadm:Cluster fsid is 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:58:05.755 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-15T21:58:05.755 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.5', 'mon.b': '172.21.15.99', 'mon.c': '172.21.15.129'} 2024-09-15T21:58:05.755 INFO:tasks.cephadm:First mon is mon.a on smithi005 2024-09-15T21:58:05.756 INFO:tasks.cephadm:First mgr is x 2024-09-15T21:58:05.756 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-15T21:58:05.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname $(hostname -s) 2024-09-15T21:58:05.788 DEBUG:teuthology.orchestra.run.smithi099:> sudo hostname $(hostname -s) 2024-09-15T21:58:05.814 DEBUG:teuthology.orchestra.run.smithi129:> sudo hostname $(hostname -s) 2024-09-15T21:58:05.839 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-15T21:58:05.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:58:05.985 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-15T21:58:06.130 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:58:06.132 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-15T21:58:06.132 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-15T21:58:06.132 DEBUG:teuthology.orchestra.run.smithi005:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T21:58:06.499 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 21:58 /home/ubuntu/cephtest/cephadm 2024-09-15T21:58:06.500 DEBUG:teuthology.orchestra.run.smithi099:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T21:58:06.868 INFO:teuthology.orchestra.run.smithi099.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 21:58 /home/ubuntu/cephtest/cephadm 2024-09-15T21:58:06.868 DEBUG:teuthology.orchestra.run.smithi129:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-15T21:58:07.388 INFO:teuthology.orchestra.run.smithi129.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 15 21:58 /home/ubuntu/cephtest/cephadm 2024-09-15T21:58:07.389 DEBUG:teuthology.orchestra.run.smithi005:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T21:58:07.409 DEBUG:teuthology.orchestra.run.smithi099:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T21:58:07.426 DEBUG:teuthology.orchestra.run.smithi129:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-15T21:58:07.461 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-15T21:58:07.461 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T21:58:07.464 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T21:58:07.469 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-15T21:58:08.960 INFO:teuthology.orchestra.run.smithi129.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T21:58:09.099 INFO:teuthology.orchestra.run.smithi099.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T21:58:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T21:58:36.055 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-15T21:58:36.055 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T21:58:36.056 INFO:teuthology.orchestra.run.smithi099.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T21:58:36.056 INFO:teuthology.orchestra.run.smithi099.stdout: "repo_digests": [ 2024-09-15T21:58:36.056 INFO:teuthology.orchestra.run.smithi099.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T21:58:36.056 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-15T21:58:36.056 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-15T21:58:37.766 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-15T21:58:37.767 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T21:58:37.767 INFO:teuthology.orchestra.run.smithi129.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T21:58:37.767 INFO:teuthology.orchestra.run.smithi129.stdout: "repo_digests": [ 2024-09-15T21:58:37.767 INFO:teuthology.orchestra.run.smithi129.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T21:58:37.767 INFO:teuthology.orchestra.run.smithi129.stdout: ] 2024-09-15T21:58:37.767 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-15T21:58:39.562 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T21:58:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T21:58:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-15T21:58:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "repo_digests": [ 2024-09-15T21:58:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-15T21:58:39.562 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T21:58:39.562 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T21:58:39.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph 2024-09-15T21:58:39.627 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph 2024-09-15T21:58:39.660 DEBUG:teuthology.orchestra.run.smithi129:> sudo mkdir -p /etc/ceph 2024-09-15T21:58:39.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /etc/ceph 2024-09-15T21:58:39.719 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /etc/ceph 2024-09-15T21:58:39.749 DEBUG:teuthology.orchestra.run.smithi129:> sudo chmod 777 /etc/ceph 2024-09-15T21:58:39.778 INFO:tasks.cephadm:Writing seed config... 2024-09-15T21:58:39.779 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-15T21:58:39.780 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-15T21:58:39.781 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-15T21:58:39.782 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-15T21:58:39.783 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-15T21:58:39.783 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-15T21:58:39.783 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-15T21:58:39.783 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-15T21:58:39.783 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-15T21:58:39.783 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-15T21:58:39.783 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-15T21:58:39.783 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-15T21:58:39.784 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T21:58:39.784 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-15T21:58:39.799 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 962891f2-73ad-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-15T21:58:39.800 DEBUG:teuthology.orchestra.run.smithi005:mon.a> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.a.service 2024-09-15T21:58:39.843 DEBUG:teuthology.orchestra.run.smithi005:mgr.x> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.x.service 2024-09-15T21:58:39.886 INFO:tasks.cephadm:Bootstrapping... 2024-09-15T21:58:39.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid 962891f2-73ad-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-15T21:58:40.095 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:58:40.095 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', '962891f2-73ad-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.5', '--skip-admin-label'] 2024-09-15T21:58:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-15T21:58:40.095 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying podman|docker is present... 2024-09-15T21:58:40.119 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.2.2 2024-09-15T21:58:40.120 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying lvm2 is present... 2024-09-15T21:58:40.120 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying time synchronization is in place... 2024-09-15T21:58:40.127 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-15T21:58:40.128 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-15T21:58:40.134 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-15T21:58:40.134 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-15T21:58:40.141 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-09-15T21:58:40.147 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-09-15T21:58:40.147 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-09-15T21:58:40.147 INFO:teuthology.orchestra.run.smithi005.stdout:Repeating the final host check... 2024-09-15T21:58:40.170 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.2.2 2024-09-15T21:58:40.171 INFO:teuthology.orchestra.run.smithi005.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-15T21:58:40.171 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl is present 2024-09-15T21:58:40.171 INFO:teuthology.orchestra.run.smithi005.stdout:lvcreate is present 2024-09-15T21:58:40.177 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-15T21:58:40.177 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-15T21:58:40.184 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-15T21:58:40.184 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-15T21:58:40.190 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-09-15T21:58:40.197 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-09-15T21:58:40.197 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-09-15T21:58:40.197 INFO:teuthology.orchestra.run.smithi005.stdout:Host looks OK 2024-09-15T21:58:40.197 INFO:teuthology.orchestra.run.smithi005.stdout:Cluster fsid: 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:58:40.197 INFO:teuthology.orchestra.run.smithi005.stdout:Acquiring lock 139719307011072 on /run/cephadm/962891f2-73ad-11ef-bceb-c7b262605968.lock 2024-09-15T21:58:40.198 INFO:teuthology.orchestra.run.smithi005.stdout:Lock 139719307011072 acquired on /run/cephadm/962891f2-73ad-11ef-bceb-c7b262605968.lock 2024-09-15T21:58:40.198 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 3300 ... 2024-09-15T21:58:40.198 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 6789 ... 2024-09-15T21:58:40.198 INFO:teuthology.orchestra.run.smithi005.stdout:Base mon IP(s) is [172.21.15.5:3300, 172.21.15.5:6789], mon addrv is [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-15T21:58:40.201 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.5 metric 100 2024-09-15T21:58:40.202 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.5 metric 100 2024-09-15T21:58:40.204 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-15T21:58:40.204 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-15T21:58:40.204 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1790sec hoplimit 64 pref medium 2024-09-15T21:58:40.207 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-15T21:58:40.207 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-15T21:58:40.207 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-15T21:58:40.207 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-15T21:58:40.207 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-15T21:58:40.207 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-15T21:58:40.208 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-09-15T21:58:40.208 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-09-15T21:58:40.208 INFO:teuthology.orchestra.run.smithi005.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-15T21:58:40.208 INFO:teuthology.orchestra.run.smithi005.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-15T21:58:40.209 INFO:teuthology.orchestra.run.smithi005.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T21:58:40.647 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-15T21:58:40.647 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-15T21:58:40.648 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Getting image source signatures 2024-09-15T21:58:40.648 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-15T21:58:40.648 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-15T21:58:40.648 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-15T21:58:40.648 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-15T21:58:41.577 INFO:teuthology.orchestra.run.smithi005.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-15T21:58:41.577 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-15T21:58:41.577 INFO:teuthology.orchestra.run.smithi005.stdout:Extracting ceph user uid/gid from container image... 2024-09-15T21:58:42.448 INFO:teuthology.orchestra.run.smithi005.stdout:stat: stdout 167 167 2024-09-15T21:58:42.449 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial keys... 2024-09-15T21:58:43.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQCSWOdm6W4RLxAASaz2Own2bkqlDlBSCD41/A== 2024-09-15T21:58:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQCTWOdm6BqQLxAAfuaVI4Hjgb3ETIz4cidEFA== 2024-09-15T21:58:45.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQCUWOdmy2DPKxAAAVp1jyizopIvVMAKKWObCQ== 2024-09-15T21:58:45.376 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial monmap... 2024-09-15T21:58:46.313 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-15T21:58:46.314 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-15T21:58:46.314 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:58:46.314 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-15T21:58:46.314 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool for a [v2:172.21.15.5:3300,v1:172.21.15.5:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-15T21:58:46.314 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-09-15T21:58:46.314 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: set fsid to 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:58:46.314 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-15T21:58:46.314 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:58:46.314 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mon... 2024-09-15T21:58:47.244 INFO:teuthology.orchestra.run.smithi005.stdout:create mon.a on 2024-09-15T21:58:47.468 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-15T21:58:47.617 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-15T21:58:47.796 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-962891f2-73ad-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-962891f2-73ad-11ef-bceb-c7b262605968.target. 2024-09-15T21:58:47.796 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-962891f2-73ad-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-962891f2-73ad-11ef-bceb-c7b262605968.target. 2024-09-15T21:58:48.042 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.a 2024-09-15T21:58:48.042 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-15T21:58:48.210 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-962891f2-73ad-11ef-bceb-c7b262605968.target.wants/ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-962891f2-73ad-11ef-bceb-c7b262605968@.service. 2024-09-15T21:58:48.718 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T21:58:48.719 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-15T21:58:48.726 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T21:58:48.726 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-15T21:58:48.726 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-15T21:58:48.726 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-09-15T21:58:48.726 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon to start... 2024-09-15T21:58:48.727 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon... 2024-09-15T21:58:49.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:49 smithi005 ceph-mon[30579]: from='client.? 172.21.15.5:0/2011398189' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-15T21:58:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout cluster: 2024-09-15T21:58:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout id: 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:58:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-15T21:58:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:58:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout services: 2024-09-15T21:58:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.538728s) 2024-09-15T21:58:49.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-15T21:58:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-15T21:58:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:58:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout data: 2024-09-15T21:58:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-15T21:58:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-15T21:58:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-15T21:58:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pgs: 2024-09-15T21:58:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:58:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.134+0000 7ff5f8738640 1 Processor -- start 2024-09-15T21:58:49.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.134+0000 7ff5f8738640 1 -- start start 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.135+0000 7ff5f8738640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f0107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.135+0000 7ff5f8738640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f01080b0 con 0x7ff5f01076e0 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.135+0000 7ff5f64d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f0107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.135+0000 7ff5f64d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f0107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59014/0 (socket says 172.21.15.5:59014) 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.135+0000 7ff5f64d4640 1 -- 172.21.15.5:0/969607065 learned_addr learned my addr 172.21.15.5:0/969607065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.135+0000 7ff5f64d4640 1 -- 172.21.15.5:0/969607065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f0108890 con 0x7ff5f01076e0 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.137+0000 7ff5f64d4640 1 --2- 172.21.15.5:0/969607065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f0107ae0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009820 tx=0x7ff5ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4e61a196a13d30 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.137+0000 7ff5f54d2640 1 -- 172.21.15.5:0/969607065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec037070 con 0x7ff5f01076e0 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.137+0000 7ff5f54d2640 1 -- 172.21.15.5:0/969607065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ec032440 con 0x7ff5f01076e0 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.137+0000 7ff5f54d2640 1 -- 172.21.15.5:0/969607065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec032740 con 0x7ff5f01076e0 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.138+0000 7ff5f8738640 1 -- 172.21.15.5:0/969607065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 msgr2=0x7ff5f0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:58:49.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.138+0000 7ff5f8738640 1 --2- 172.21.15.5:0/969607065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f0107ae0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009820 tx=0x7ff5ec02a270 comp rx=0 tx=0).stop 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.138+0000 7ff5f8738640 1 -- 172.21.15.5:0/969607065 shutdown_connections 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.138+0000 7ff5f8738640 1 --2- 172.21.15.5:0/969607065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f0107ae0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.138+0000 7ff5f8738640 1 -- 172.21.15.5:0/969607065 >> 172.21.15.5:0/969607065 conn(0x7ff5f0102e90 msgr2=0x7ff5f01052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.138+0000 7ff5f8738640 1 -- 172.21.15.5:0/969607065 shutdown_connections 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.138+0000 7ff5f8738640 1 -- 172.21.15.5:0/969607065 wait complete. 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.138+0000 7ff5f8738640 1 Processor -- start 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.139+0000 7ff5f8738640 1 -- start start 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.139+0000 7ff5f8738640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f019d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.139+0000 7ff5f8738640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f0108e30 con 0x7ff5f01076e0 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.139+0000 7ff5f64d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f019d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.139+0000 7ff5f64d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f019d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59024/0 (socket says 172.21.15.5:59024) 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.139+0000 7ff5f64d4640 1 -- 172.21.15.5:0/2011398189 learned_addr learned my addr 172.21.15.5:0/2011398189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:58:49.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.139+0000 7ff5f64d4640 1 -- 172.21.15.5:0/2011398189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f019d5b0 con 0x7ff5f01076e0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.139+0000 7ff5f64d4640 1 --2- 172.21.15.5:0/2011398189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f019d070 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0090b0 tx=0x7ff5ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.140+0000 7ff5e77fe640 1 -- 172.21.15.5:0/2011398189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec041070 con 0x7ff5f01076e0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.140+0000 7ff5e77fe640 1 -- 172.21.15.5:0/2011398189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ec03c7a0 con 0x7ff5f01076e0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.140+0000 7ff5f8738640 1 -- 172.21.15.5:0/2011398189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f019d840 con 0x7ff5f01076e0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.140+0000 7ff5e77fe640 1 -- 172.21.15.5:0/2011398189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec037040 con 0x7ff5f01076e0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.140+0000 7ff5f8738640 1 -- 172.21.15.5:0/2011398189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f01a0530 con 0x7ff5f01076e0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.140+0000 7ff5e77fe640 1 -- 172.21.15.5:0/2011398189 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff5ec044070 con 0x7ff5f01076e0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.140+0000 7ff5e77fe640 1 -- 172.21.15.5:0/2011398189 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff5ec03d3f0 con 0x7ff5f01076e0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.141+0000 7ff5f8738640 1 -- 172.21.15.5:0/2011398189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f0109480 con 0x7ff5f01076e0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.143+0000 7ff5e77fe640 1 -- 172.21.15.5:0/2011398189 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff5ec046450 con 0x7ff5f01076e0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.206+0000 7ff5f8738640 1 -- 172.21.15.5:0/2011398189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7ff5f01a0a90 con 0x7ff5f01076e0 2024-09-15T21:58:49.765 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.206+0000 7ff5e77fe640 1 -- 172.21.15.5:0/2011398189 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7ff5ec046630 con 0x7ff5f01076e0 2024-09-15T21:58:49.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.207+0000 7ff5f8738640 1 -- 172.21.15.5:0/2011398189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 msgr2=0x7ff5f019d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:58:49.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.207+0000 7ff5f8738640 1 --2- 172.21.15.5:0/2011398189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f019d070 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0090b0 tx=0x7ff5ec0046c0 comp rx=0 tx=0).stop 2024-09-15T21:58:49.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.207+0000 7ff5f8738640 1 -- 172.21.15.5:0/2011398189 shutdown_connections 2024-09-15T21:58:49.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.207+0000 7ff5f8738640 1 --2- 172.21.15.5:0/2011398189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f01076e0 0x7ff5f019d070 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:58:49.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.207+0000 7ff5f8738640 1 -- 172.21.15.5:0/2011398189 >> 172.21.15.5:0/2011398189 conn(0x7ff5f0102e90 msgr2=0x7ff5f007c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:58:49.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.207+0000 7ff5f8738640 1 -- 172.21.15.5:0/2011398189 shutdown_connections 2024-09-15T21:58:49.766 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:49.207+0000 7ff5f8738640 1 -- 172.21.15.5:0/2011398189 wait complete. 2024-09-15T21:58:49.766 INFO:teuthology.orchestra.run.smithi005.stdout:mon is available 2024-09-15T21:58:49.766 INFO:teuthology.orchestra.run.smithi005.stdout:Assimilating anything we can from ceph.conf... 2024-09-15T21:58:50.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:58:50.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-09-15T21:58:50.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:58:50.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-15T21:58:50.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-15T21:58:50.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-15T21:58:50.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.143+0000 7fe1df04c640 1 Processor -- start 2024-09-15T21:58:50.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.143+0000 7fe1df04c640 1 -- start start 2024-09-15T21:58:50.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.143+0000 7fe1df04c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d8107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:58:50.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.143+0000 7fe1df04c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d8108050 con 0x7fe1d8107680 2024-09-15T21:58:50.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.143+0000 7fe1dcde8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d8107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:58:50.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.143+0000 7fe1dcde8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d8107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59030/0 (socket says 172.21.15.5:59030) 2024-09-15T21:58:50.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.143+0000 7fe1dcde8640 1 -- 172.21.15.5:0/4062369550 learned_addr learned my addr 172.21.15.5:0/4062369550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:58:50.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.144+0000 7fe1dcde8640 1 -- 172.21.15.5:0/4062369550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d8108880 con 0x7fe1d8107680 2024-09-15T21:58:50.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.144+0000 7fe1dcde8640 1 --2- 172.21.15.5:0/4062369550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d8107a80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009820 tx=0x7fe1cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f376d71c31043f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:58:50.702 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.145+0000 7fe1d77fe640 1 -- 172.21.15.5:0/4062369550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1cc037070 con 0x7fe1d8107680 2024-09-15T21:58:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.145+0000 7fe1d77fe640 1 -- 172.21.15.5:0/4062369550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1cc032440 con 0x7fe1d8107680 2024-09-15T21:58:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.145+0000 7fe1d77fe640 1 -- 172.21.15.5:0/4062369550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1cc032740 con 0x7fe1d8107680 2024-09-15T21:58:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.145+0000 7fe1df04c640 1 -- 172.21.15.5:0/4062369550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 msgr2=0x7fe1d8107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:58:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.145+0000 7fe1df04c640 1 --2- 172.21.15.5:0/4062369550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d8107a80 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc009820 tx=0x7fe1cc02a270 comp rx=0 tx=0).stop 2024-09-15T21:58:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.145+0000 7fe1df04c640 1 -- 172.21.15.5:0/4062369550 shutdown_connections 2024-09-15T21:58:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.145+0000 7fe1df04c640 1 --2- 172.21.15.5:0/4062369550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d8107a80 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:58:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.145+0000 7fe1df04c640 1 -- 172.21.15.5:0/4062369550 >> 172.21.15.5:0/4062369550 conn(0x7fe1d8102e30 msgr2=0x7fe1d8105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:58:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.145+0000 7fe1df04c640 1 -- 172.21.15.5:0/4062369550 shutdown_connections 2024-09-15T21:58:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.145+0000 7fe1df04c640 1 -- 172.21.15.5:0/4062369550 wait complete. 2024-09-15T21:58:50.703 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.146+0000 7fe1df04c640 1 Processor -- start 2024-09-15T21:58:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.146+0000 7fe1df04c640 1 -- start start 2024-09-15T21:58:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.146+0000 7fe1df04c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:58:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.146+0000 7fe1df04c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d8108e20 con 0x7fe1d8107680 2024-09-15T21:58:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.146+0000 7fe1dcde8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:58:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.146+0000 7fe1dcde8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d819ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59038/0 (socket says 172.21.15.5:59038) 2024-09-15T21:58:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.147+0000 7fe1dcde8640 1 -- 172.21.15.5:0/888639237 learned_addr learned my addr 172.21.15.5:0/888639237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:58:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.147+0000 7fe1dcde8640 1 -- 172.21.15.5:0/888639237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d819b380 con 0x7fe1d8107680 2024-09-15T21:58:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.147+0000 7fe1dcde8640 1 --2- 172.21.15.5:0/888639237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d819ae40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0090b0 tx=0x7fe1cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:58:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.147+0000 7fe1d5ffb640 1 -- 172.21.15.5:0/888639237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1cc041070 con 0x7fe1d8107680 2024-09-15T21:58:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.147+0000 7fe1d5ffb640 1 -- 172.21.15.5:0/888639237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1cc03c7a0 con 0x7fe1d8107680 2024-09-15T21:58:50.704 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.147+0000 7fe1df04c640 1 -- 172.21.15.5:0/888639237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d819b610 con 0x7fe1d8107680 2024-09-15T21:58:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.147+0000 7fe1d5ffb640 1 -- 172.21.15.5:0/888639237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1cc037040 con 0x7fe1d8107680 2024-09-15T21:58:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.147+0000 7fe1df04c640 1 -- 172.21.15.5:0/888639237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d819e300 con 0x7fe1d8107680 2024-09-15T21:58:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.148+0000 7fe1d5ffb640 1 -- 172.21.15.5:0/888639237 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe1cc044070 con 0x7fe1d8107680 2024-09-15T21:58:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.148+0000 7fe1d5ffb640 1 -- 172.21.15.5:0/888639237 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe1cc03d3f0 con 0x7fe1d8107680 2024-09-15T21:58:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.148+0000 7fe1df04c640 1 -- 172.21.15.5:0/888639237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d8109470 con 0x7fe1d8107680 2024-09-15T21:58:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.151+0000 7fe1d5ffb640 1 -- 172.21.15.5:0/888639237 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fe1cc046450 con 0x7fe1d8107680 2024-09-15T21:58:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.210+0000 7fe1df04c640 1 -- 172.21.15.5:0/888639237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe1d819f9d0 con 0x7fe1d8107680 2024-09-15T21:58:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.213+0000 7fe1d5ffb640 1 -- 172.21.15.5:0/888639237 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7fe1cc046630 con 0x7fe1d8107680 2024-09-15T21:58:50.705 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.214+0000 7fe1d5ffb640 1 -- 172.21.15.5:0/888639237 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe1cc056070 con 0x7fe1d8107680 2024-09-15T21:58:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.214+0000 7fe1df04c640 1 -- 172.21.15.5:0/888639237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 msgr2=0x7fe1d819ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:58:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.214+0000 7fe1df04c640 1 --2- 172.21.15.5:0/888639237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d819ae40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0090b0 tx=0x7fe1cc0046c0 comp rx=0 tx=0).stop 2024-09-15T21:58:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.214+0000 7fe1df04c640 1 -- 172.21.15.5:0/888639237 shutdown_connections 2024-09-15T21:58:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.214+0000 7fe1df04c640 1 --2- 172.21.15.5:0/888639237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1d8107680 0x7fe1d819ae40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:58:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.214+0000 7fe1df04c640 1 -- 172.21.15.5:0/888639237 >> 172.21.15.5:0/888639237 conn(0x7fe1d8102e30 msgr2=0x7fe1d818f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:58:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.215+0000 7fe1df04c640 1 -- 172.21.15.5:0/888639237 shutdown_connections 2024-09-15T21:58:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:50.215+0000 7fe1df04c640 1 -- 172.21.15.5:0/888639237 wait complete. 2024-09-15T21:58:50.706 INFO:teuthology.orchestra.run.smithi005.stdout:Generating new minimal ceph.conf... 2024-09-15T21:58:50.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:50 smithi005 ceph-mon[30579]: from='client.? 172.21.15.5:0/888639237' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-15T21:58:50.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:50 smithi005 ceph-mon[30579]: from='client.? 172.21.15.5:0/888639237' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-15T21:58:51.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.097+0000 7f64c3476640 1 Processor -- start 2024-09-15T21:58:51.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.097+0000 7f64c3476640 1 -- start start 2024-09-15T21:58:51.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.097+0000 7f64c3476640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc07cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:58:51.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.097+0000 7f64c3476640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc07d050 con 0x7f64bc07e660 2024-09-15T21:58:51.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.097+0000 7f64c1212640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc07cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:58:51.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.097+0000 7f64c1212640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc07cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59052/0 (socket says 172.21.15.5:59052) 2024-09-15T21:58:51.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.097+0000 7f64c1212640 1 -- 172.21.15.5:0/2969568827 learned_addr learned my addr 172.21.15.5:0/2969568827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:58:51.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.098+0000 7f64c1212640 1 -- 172.21.15.5:0/2969568827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc07d1d0 con 0x7f64bc07e660 2024-09-15T21:58:51.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.098+0000 7f64c1212640 1 --2- 172.21.15.5:0/2969568827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc07cb10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009820 tx=0x7f64b802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef742ad3a9be652 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:58:51.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.098+0000 7f64b3fff640 1 -- 172.21.15.5:0/2969568827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64b8037070 con 0x7f64bc07e660 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.098+0000 7f64b3fff640 1 -- 172.21.15.5:0/2969568827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f64b8032900 con 0x7f64bc07e660 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.099+0000 7f64b3fff640 1 -- 172.21.15.5:0/2969568827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64b80308e0 con 0x7f64bc07e660 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.099+0000 7f64c3476640 1 -- 172.21.15.5:0/2969568827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 msgr2=0x7f64bc07cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.099+0000 7f64c3476640 1 --2- 172.21.15.5:0/2969568827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc07cb10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009820 tx=0x7f64b802a300 comp rx=0 tx=0).stop 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.099+0000 7f64c3476640 1 -- 172.21.15.5:0/2969568827 shutdown_connections 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.099+0000 7f64c3476640 1 --2- 172.21.15.5:0/2969568827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc07cb10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.099+0000 7f64c3476640 1 -- 172.21.15.5:0/2969568827 >> 172.21.15.5:0/2969568827 conn(0x7f64bc102e90 msgr2=0x7f64bc1052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.099+0000 7f64c3476640 1 -- 172.21.15.5:0/2969568827 shutdown_connections 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.099+0000 7f64c3476640 1 -- 172.21.15.5:0/2969568827 wait complete. 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.100+0000 7f64c3476640 1 Processor -- start 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.100+0000 7f64c3476640 1 -- start start 2024-09-15T21:58:51.723 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.100+0000 7f64c3476640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc19d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.100+0000 7f64c3476640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10dcf0 con 0x7f64bc07e660 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.100+0000 7f64c1212640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc19d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.100+0000 7f64c1212640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc19d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59054/0 (socket says 172.21.15.5:59054) 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.100+0000 7f64c1212640 1 -- 172.21.15.5:0/4282089583 learned_addr learned my addr 172.21.15.5:0/4282089583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.100+0000 7f64c1212640 1 -- 172.21.15.5:0/4282089583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc19ddd0 con 0x7f64bc07e660 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.101+0000 7f64c1212640 1 --2- 172.21.15.5:0/4282089583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc19d890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f64b8030610 tx=0x7f64b8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.101+0000 7f64b27fc640 1 -- 172.21.15.5:0/4282089583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64b8037040 con 0x7f64bc07e660 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.101+0000 7f64b27fc640 1 -- 172.21.15.5:0/4282089583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f64b8040070 con 0x7f64bc07e660 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.101+0000 7f64c3476640 1 -- 172.21.15.5:0/4282089583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc19e060 con 0x7f64bc07e660 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.101+0000 7f64b27fc640 1 -- 172.21.15.5:0/4282089583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64b803a390 con 0x7f64bc07e660 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.101+0000 7f64c3476640 1 -- 172.21.15.5:0/4282089583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc1a0d50 con 0x7f64bc07e660 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.101+0000 7f64b27fc640 1 -- 172.21.15.5:0/4282089583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f64b803a640 con 0x7f64bc07e660 2024-09-15T21:58:51.724 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.101+0000 7f64b27fc640 1 -- 172.21.15.5:0/4282089583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f64b80457d0 con 0x7f64bc07e660 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.102+0000 7f64c3476640 1 -- 172.21.15.5:0/4282089583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64bc07cb90 con 0x7f64bc07e660 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.104+0000 7f64b27fc640 1 -- 172.21.15.5:0/4282089583 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f64b803b430 con 0x7f64bc07e660 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.163+0000 7f64c3476640 1 -- 172.21.15.5:0/4282089583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f64bc1a12d0 con 0x7f64bc07e660 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.164+0000 7f64b27fc640 1 -- 172.21.15.5:0/4282089583 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7f64b803b650 con 0x7f64bc07e660 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.164+0000 7f64c3476640 1 -- 172.21.15.5:0/4282089583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 msgr2=0x7f64bc19d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.164+0000 7f64c3476640 1 --2- 172.21.15.5:0/4282089583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc19d890 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f64b8030610 tx=0x7f64b8032670 comp rx=0 tx=0).stop 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.165+0000 7f64c3476640 1 -- 172.21.15.5:0/4282089583 shutdown_connections 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.165+0000 7f64c3476640 1 --2- 172.21.15.5:0/4282089583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64bc07e660 0x7f64bc19d890 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.165+0000 7f64c3476640 1 -- 172.21.15.5:0/4282089583 >> 172.21.15.5:0/4282089583 conn(0x7f64bc102e90 msgr2=0x7f64bc103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.165+0000 7f64c3476640 1 -- 172.21.15.5:0/4282089583 shutdown_connections 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:51.165+0000 7f64c3476640 1 -- 172.21.15.5:0/4282089583 wait complete. 2024-09-15T21:58:51.725 INFO:teuthology.orchestra.run.smithi005.stdout:Restarting the monitor... 2024-09-15T21:58:51.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:51 smithi005 ceph-mon[30579]: from='client.? 172.21.15.5:0/4282089583' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T21:58:51.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:51 smithi005 systemd[1]: Stopping Ceph mon.a for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T21:58:52.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a[30575]: 2024-09-15T21:58:51.982+0000 7f4b796cc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T21:58:52.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a[30575]: 2024-09-15T21:58:51.982+0000 7f4b796cc640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-15T21:58:52.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:52 smithi005 podman[30821]: 2024-09-15 21:58:52.126572818 +0000 UTC m=+0.242490327 container died b3b4dfb471f3f037e2e073c666d7308bcb91ee5c1a695d5f14d4c93024f10685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-15T21:58:52.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:52 smithi005 podman[30821]: 2024-09-15 21:58:52.473928416 +0000 UTC m=+0.589845929 container cleanup b3b4dfb471f3f037e2e073c666d7308bcb91ee5c1a695d5f14d4c93024f10685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-15T21:58:52.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:52 smithi005 bash[30821]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a 2024-09-15T21:58:53.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:52 smithi005 podman[30833]: 2024-09-15 21:58:52.734166338 +0000 UTC m=+0.605240571 container remove b3b4dfb471f3f037e2e073c666d7308bcb91ee5c1a695d5f14d4c93024f10685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-15T21:58:53.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:52 smithi005 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-15T21:58:53.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:52 smithi005 systemd[1]: Stopped Ceph mon.a for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T21:58:53.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:52 smithi005 systemd[1]: Starting Ceph mon.a for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T21:58:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 podman[30925]: 2024-09-15 21:58:53.064811718 +0000 UTC m=+0.107311060 container create e647ceea0e91a4de76a22cd0c39530d5dfe161dd828180b5016ad05ab14aa160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-15T21:58:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 podman[30925]: 2024-09-15 21:58:52.973952193 +0000 UTC m=+0.016451494 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:58:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 podman[30925]: 2024-09-15 21:58:53.278855184 +0000 UTC m=+0.321354485 container init e647ceea0e91a4de76a22cd0c39530d5dfe161dd828180b5016ad05ab14aa160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-09-15T21:58:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 podman[30925]: 2024-09-15 21:58:53.282216659 +0000 UTC m=+0.324715959 container start e647ceea0e91a4de76a22cd0c39530d5dfe161dd828180b5016ad05ab14aa160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T21:58:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: set uid:gid to 167:167 (ceph:ceph) 2024-09-15T21:58:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: pidfile_write: ignore empty --pid-file 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: load: jerasure load: lrc 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: RocksDB version: 7.9.2 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Git sha 0 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: DB SUMMARY 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: DB Session ID: 51LJ92I3H27K27CWA2B8 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: CURRENT file: CURRENT 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: IDENTITY file: IDENTITY 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-15T21:58:53.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105792 ; 2024-09-15T21:58:53.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.error_if_exists: 0 2024-09-15T21:58:53.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.create_if_missing: 0 2024-09-15T21:58:53.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.paranoid_checks: 1 2024-09-15T21:58:53.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-15T21:58:53.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-15T21:58:53.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-15T21:58:53.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.env: 0x55e5ee626940 2024-09-15T21:58:53.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.fs: PosixFileSystem 2024-09-15T21:58:53.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.info_log: 0x55e5efa6f6e0 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_file_opening_threads: 16 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.statistics: (nil) 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.use_fsync: 0 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_log_file_size: 0 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.keep_log_file_num: 1000 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.recycle_log_file_num: 0 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.allow_fallocate: 1 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.allow_mmap_reads: 0 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.allow_mmap_writes: 0 2024-09-15T21:58:53.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.use_direct_reads: 0 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.create_missing_column_families: 0 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.db_log_dir: 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.wal_dir: 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.advise_random_on_open: 1 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.db_write_buffer_size: 0 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.write_buffer_manager: 0x55e5efa7edc0 2024-09-15T21:58:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.rate_limiter: (nil) 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.wal_recovery_mode: 2 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.enable_thread_tracking: 0 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.enable_pipelined_write: 0 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.unordered_write: 0 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-15T21:58:53.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.row_cache: None 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.wal_filter: None 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.allow_ingest_behind: 0 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.two_write_queues: 0 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.manual_wal_flush: 0 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.wal_compression: 0 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.atomic_flush: 0 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.log_readahead_size: 0 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-15T21:58:53.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.best_efforts_recovery: 0 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.allow_data_in_errors: 0 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.db_host_id: __hostname__ 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_background_jobs: 2 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_background_compactions: -1 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_subcompactions: 1 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_total_wal_size: 0 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-15T21:58:53.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_open_files: -1 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bytes_per_sync: 0 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_readahead_size: 0 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_background_flushes: -1 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Compression algorithms supported: 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: kZSTD supported: 0 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: kXpressCompression supported: 0 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: kBZip2Compression supported: 0 2024-09-15T21:58:53.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: kLZ4Compression supported: 1 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: kZlibCompression supported: 1 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: kLZ4HCCompression supported: 1 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: kSnappyCompression supported: 1 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.merge_operator: 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_filter: None 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_filter_factory: None 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.sst_partitioner_factory: None 2024-09-15T21:58:53.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e5efa6f320) 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks: 1 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_top_level_index_and_filter: 1 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_type: 0 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_index_type: 0 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_shortening: 1 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout: checksum: 4 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout: no_block_cache: 0 2024-09-15T21:58:53.368 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache: 0x55e5efa631f0 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_name: BinnedLRUCache 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_options: 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: capacity : 536870912 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_shard_bits : 4 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: strict_capacity_limit : 0 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: high_pri_pool_ratio: 0.000 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_compressed: (nil) 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: persistent_cache: (nil) 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size: 4096 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size_deviation: 10 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_restart_interval: 16 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_block_restart_interval: 1 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: metadata_block_size: 4096 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: partition_filters: 0 2024-09-15T21:58:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout: use_delta_encoding: 1 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout: filter_policy: bloomfilter 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout: whole_key_filtering: 1 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout: verify_compression: 0 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout: read_amp_bytes_per_bit: 0 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout: format_version: 5 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_index_compression: 1 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_align: 0 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout: max_auto_readahead_size: 262144 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout: prepopulate_block_cache: 0 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout: initial_auto_readahead_size: 8192 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_file_reads_for_auto_readahead: 2 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.write_buffer_size: 33554432 2024-09-15T21:58:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_write_buffer_number: 2 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compression: NoCompression 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bottommost_compression: Disabled 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.prefix_extractor: nullptr 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.num_levels: 7 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-15T21:58:53.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compression_opts.level: 32767 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compression_opts.strategy: 0 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compression_opts.enabled: false 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-15T21:58:53.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.target_file_size_base: 67108864 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-15T21:58:53.373 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.arena_block_size: 1048576 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.disable_auto_compactions: 0 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-15T21:58:53.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.inplace_update_support: 0 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.bloom_locality: 0 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.max_successive_merges: 0 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.paranoid_file_checks: 0 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.force_consistency_checks: 1 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.report_bg_io_stats: 0 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.ttl: 2592000 2024-09-15T21:58:53.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-15T21:58:53.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-15T21:58:53.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-15T21:58:53.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.enable_blob_files: false 2024-09-15T21:58:53.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.min_blob_size: 0 2024-09-15T21:58:53.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.blob_file_size: 268435456 2024-09-15T21:58:53.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-15T21:58:53.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-15T21:58:53.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-15T21:58:53.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-15T21:58:53.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.blob_file_starting_level: 0 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 8ec1586a-205b-4173-abcd-6752f6157363 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437533309142, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437533310258, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100824, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98709, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14433, "raw_average_key_size": 49, "raw_value_size": 90884, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726437533, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8ec1586a-205b-4173-abcd-6752f6157363", "db_session_id": "51LJ92I3H27K27CWA2B8", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437533310357, "job": 1, "event": "recovery_finished"} 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e5efa90e00 2024-09-15T21:58:53.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: DB pointer 0x55e5efb8c000 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** DB Stats ** 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-09-15T21:58:53.378 INFO:journalctl@ceph.mon.a.smithi005.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: L0 2/0 100.32 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: Sum 2/0 100.32 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 135.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 135.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-15T21:58:53.379 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative compaction: 0.00 GB write, 28.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval compaction: 0.00 GB write, 28.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache BinnedLRUCache@0x55e5efa631f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: starting mon.a rank 0 at public addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] at bind addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: mon.a@-1(???) e1 preinit fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: mon.a@-1(???).mds e1 new map 2024-09-15T21:58:53.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: mon.a@-1(???).mds e1 print_map 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout: e1 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout: btime 2024-09-15T21:58:48:668797+0000 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout: legacy client fscid: -1 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout: No filesystems configured 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T21:58:53.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-15T21:58:53.385 INFO:teuthology.orchestra.run.smithi005.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-15T21:58:53.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: monmap epoch 1 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: last_changed 2024-09-15T21:58:45.697603+0000 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: created 2024-09-15T21:58:45.697603+0000 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: min_mon_release 19 (squid) 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: election_strategy: 1 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: fsmap 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: osdmap e1: 0 total, 0 up, 0 in 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 ceph-mon[30944]: mgrmap e1: no daemons active 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 bash[30925]: e647ceea0e91a4de76a22cd0c39530d5dfe161dd828180b5016ad05ab14aa160 2024-09-15T21:58:53.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:53 smithi005 systemd[1]: Started Ceph mon.a for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T21:58:55.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:58:54 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/499273841' entity='client.admin' 2024-09-15T21:58:55.529 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.797+0000 7f12d466f640 1 Processor -- start 2024-09-15T21:58:55.529 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.797+0000 7f12d466f640 1 -- start start 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.798+0000 7f12d466f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc10a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.798+0000 7f12d466f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc07bdf0 con 0x7f12cc108130 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.798+0000 7f12d240b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc10a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.798+0000 7f12d240b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc10a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46938/0 (socket says 172.21.15.5:46938) 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.798+0000 7f12d240b640 1 -- 172.21.15.5:0/1069315976 learned_addr learned my addr 172.21.15.5:0/1069315976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.798+0000 7f12d240b640 1 -- 172.21.15.5:0/1069315976 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc10aa60 con 0x7f12cc108130 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.800+0000 7f12d240b640 1 --2- 172.21.15.5:0/1069315976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc10a520 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009820 tx=0x7f12c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe4604b954e301e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.801+0000 7f12d1409640 1 -- 172.21.15.5:0/1069315976 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12c4037070 con 0x7f12cc108130 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.801+0000 7f12d1409640 1 -- 172.21.15.5:0/1069315976 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f12c4032900 con 0x7f12cc108130 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.801+0000 7f12d1409640 1 -- 172.21.15.5:0/1069315976 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12c4030850 con 0x7f12cc108130 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.801+0000 7f12d466f640 1 -- 172.21.15.5:0/1069315976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 msgr2=0x7f12cc10a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.801+0000 7f12d466f640 1 --2- 172.21.15.5:0/1069315976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc10a520 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009820 tx=0x7f12c402a270 comp rx=0 tx=0).stop 2024-09-15T21:58:55.530 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.801+0000 7f12d466f640 1 -- 172.21.15.5:0/1069315976 shutdown_connections 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.801+0000 7f12d466f640 1 --2- 172.21.15.5:0/1069315976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc10a520 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.801+0000 7f12d466f640 1 -- 172.21.15.5:0/1069315976 >> 172.21.15.5:0/1069315976 conn(0x7f12cc101d10 msgr2=0x7f12cc104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.801+0000 7f12d466f640 1 -- 172.21.15.5:0/1069315976 shutdown_connections 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.801+0000 7f12d466f640 1 -- 172.21.15.5:0/1069315976 wait complete. 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.802+0000 7f12d466f640 1 Processor -- start 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.802+0000 7f12d466f640 1 -- start start 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.802+0000 7f12d466f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc19b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.802+0000 7f12d466f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc10baa0 con 0x7f12cc108130 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.802+0000 7f12d240b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc19b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.802+0000 7f12d240b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc19b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46954/0 (socket says 172.21.15.5:46954) 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.802+0000 7f12d240b640 1 -- 172.21.15.5:0/499273841 learned_addr learned my addr 172.21.15.5:0/499273841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:58:55.531 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.803+0000 7f12d240b640 1 -- 172.21.15.5:0/499273841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc19bb80 con 0x7f12cc108130 2024-09-15T21:58:55.532 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.803+0000 7f12d240b640 1 --2- 172.21.15.5:0/499273841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc19b640 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f12c4032ce0 tx=0x7f12c4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:58:55.532 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.803+0000 7f12cb7fe640 1 -- 172.21.15.5:0/499273841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12c4042070 con 0x7f12cc108130 2024-09-15T21:58:55.532 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.803+0000 7f12cb7fe640 1 -- 172.21.15.5:0/499273841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f12c40308c0 con 0x7f12cc108130 2024-09-15T21:58:55.532 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.803+0000 7f12cb7fe640 1 -- 172.21.15.5:0/499273841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12c4037040 con 0x7f12cc108130 2024-09-15T21:58:55.532 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.803+0000 7f12d466f640 1 -- 172.21.15.5:0/499273841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc19be10 con 0x7f12cc108130 2024-09-15T21:58:55.532 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.803+0000 7f12d466f640 1 -- 172.21.15.5:0/499273841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc19eb00 con 0x7f12cc108130 2024-09-15T21:58:55.532 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.803+0000 7f12cb7fe640 1 -- 172.21.15.5:0/499273841 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f12c4045070 con 0x7f12cc108130 2024-09-15T21:58:55.532 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.803+0000 7f12cb7fe640 1 -- 172.21.15.5:0/499273841 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f12c4047b70 con 0x7f12cc108130 2024-09-15T21:58:55.532 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.804+0000 7f12d466f640 1 -- 172.21.15.5:0/499273841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12cc108590 con 0x7f12cc108130 2024-09-15T21:58:55.532 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.806+0000 7f12cb7fe640 1 -- 172.21.15.5:0/499273841 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f12c403c760 con 0x7f12cc108130 2024-09-15T21:58:55.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.866+0000 7f12d466f640 1 -- 172.21.15.5:0/499273841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f12cc19f0f0 con 0x7f12cc108130 2024-09-15T21:58:55.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.867+0000 7f12cb7fe640 1 -- 172.21.15.5:0/499273841 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f12c403c940 con 0x7f12cc108130 2024-09-15T21:58:55.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.868+0000 7f12d466f640 1 -- 172.21.15.5:0/499273841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 msgr2=0x7f12cc19b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:58:55.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.868+0000 7f12d466f640 1 --2- 172.21.15.5:0/499273841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc19b640 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f12c4032ce0 tx=0x7f12c4032d10 comp rx=0 tx=0).stop 2024-09-15T21:58:55.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.868+0000 7f12d466f640 1 -- 172.21.15.5:0/499273841 shutdown_connections 2024-09-15T21:58:55.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.868+0000 7f12d466f640 1 --2- 172.21.15.5:0/499273841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12cc108130 0x7f12cc19b640 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:58:55.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.868+0000 7f12d466f640 1 -- 172.21.15.5:0/499273841 >> 172.21.15.5:0/499273841 conn(0x7f12cc101d10 msgr2=0x7f12cc102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:58:55.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.868+0000 7f12d466f640 1 -- 172.21.15.5:0/499273841 shutdown_connections 2024-09-15T21:58:55.533 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:58:53.868+0000 7f12d466f640 1 -- 172.21.15.5:0/499273841 wait complete. 2024-09-15T21:58:55.533 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-15T21:58:55.779 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-15T21:58:55.779 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mgr... 2024-09-15T21:58:55.780 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-15T21:58:55.780 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-15T21:58:56.831 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.x 2024-09-15T21:58:56.831 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-15T21:58:57.189 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-962891f2-73ad-11ef-bceb-c7b262605968.target.wants/ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-962891f2-73ad-11ef-bceb-c7b262605968@.service. 2024-09-15T21:58:57.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:58:57 smithi005 podman[31157]: 2024-09-15 21:58:57.338061663 +0000 UTC m=+0.016060026 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:58:58.856 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:58:58 smithi005 podman[31157]: 2024-09-15 21:58:58.530149965 +0000 UTC m=+1.208148374 container create 4b0c5ec44a62dffed6f4b784c0cc9bffd5a517df2f73d8abdaef4dec5b41c4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-15T21:59:00.796 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:00 smithi005 podman[31157]: 2024-09-15 21:59:00.436588155 +0000 UTC m=+3.114586516 container init 4b0c5ec44a62dffed6f4b784c0cc9bffd5a517df2f73d8abdaef4dec5b41c4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T21:59:00.796 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:00 smithi005 podman[31157]: 2024-09-15 21:59:00.440494462 +0000 UTC m=+3.118492823 container start 4b0c5ec44a62dffed6f4b784c0cc9bffd5a517df2f73d8abdaef4dec5b41c4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-15T21:59:00.796 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:00 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:00.576+0000 7f79077bf040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T21:59:00.796 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:00 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:00.636+0000 7f79077bf040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T21:59:00.796 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:00 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:00.793+0000 7f79077bf040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T21:59:01.106 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:00 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:00.863+0000 7f79077bf040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T21:59:01.106 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:01 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:01.018+0000 7f79077bf040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:59:01.346 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T21:59:01.346 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-15T21:59:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T21:59:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-15T21:59:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-15T21:59:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-09-15T21:59:01.365 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T21:59:01.365 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-15T21:59:01.373 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T21:59:01.373 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-15T21:59:01.373 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-15T21:59:01.373 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-15T21:59:01.373 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr to start... 2024-09-15T21:59:01.373 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr... 2024-09-15T21:59:01.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:01 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:01.165+0000 7f79077bf040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T21:59:01.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:01 smithi005 bash[31157]: 4b0c5ec44a62dffed6f4b784c0cc9bffd5a517df2f73d8abdaef4dec5b41c4f3 2024-09-15T21:59:01.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:01 smithi005 systemd[1]: Started Ceph mgr.x for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T21:59:01.776 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:01 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:01.517+0000 7f79077bf040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T21:59:01.776 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:01 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:01.580+0000 7f79077bf040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T21:59:01.776 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:01 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:01.773+0000 7f79077bf040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T21:59:02.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:02 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2832427166' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T21:59:02.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:02 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:02.724+0000 7f79077bf040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T21:59:02.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:02 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:02.795+0000 7f79077bf040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T21:59:03.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:59:03.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-15T21:59:03.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T21:59:03.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T21:59:03.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:03.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T21:59:03.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T21:59:03.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:03.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T21:59:03.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:03.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T21:59:03.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T21:59:03.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T21:59:03.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T21:59:03.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T21:59:03.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T21:58:48:668797+0000", 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T21:59:03.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T21:58:48.669426+0000", 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.567+0000 7f751690c640 1 Processor -- start 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.567+0000 7f751690c640 1 -- start start 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.568+0000 7f751690c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f751010a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.568+0000 7f751690c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f751007bdd0 con 0x7f7510108150 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.568+0000 7f750ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f751010a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:03.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.568+0000 7f750ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f751010a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46988/0 (socket says 172.21.15.5:46988) 2024-09-15T21:59:03.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.568+0000 7f750ffff640 1 -- 172.21.15.5:0/3264883392 learned_addr learned my addr 172.21.15.5:0/3264883392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:03.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.568+0000 7f750ffff640 1 -- 172.21.15.5:0/3264883392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751010aa80 con 0x7f7510108150 2024-09-15T21:59:03.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.569+0000 7f750ffff640 1 --2- 172.21.15.5:0/3264883392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f751010a540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7500009820 tx=0x7f750002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527decb248871e50 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:03.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.569+0000 7f750effd640 1 -- 172.21.15.5:0/3264883392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7500037070 con 0x7f7510108150 2024-09-15T21:59:03.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.569+0000 7f750effd640 1 -- 172.21.15.5:0/3264883392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7500032900 con 0x7f7510108150 2024-09-15T21:59:03.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.569+0000 7f750effd640 1 -- 172.21.15.5:0/3264883392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7500030850 con 0x7f7510108150 2024-09-15T21:59:03.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.570+0000 7f751690c640 1 -- 172.21.15.5:0/3264883392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 msgr2=0x7f751010a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:03.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.570+0000 7f751690c640 1 --2- 172.21.15.5:0/3264883392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f751010a540 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7500009820 tx=0x7f750002a270 comp rx=0 tx=0).stop 2024-09-15T21:59:03.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.570+0000 7f751690c640 1 -- 172.21.15.5:0/3264883392 shutdown_connections 2024-09-15T21:59:03.210 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.570+0000 7f751690c640 1 --2- 172.21.15.5:0/3264883392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f751010a540 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.570+0000 7f751690c640 1 -- 172.21.15.5:0/3264883392 >> 172.21.15.5:0/3264883392 conn(0x7f7510101d10 msgr2=0x7f7510104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.570+0000 7f751690c640 1 -- 172.21.15.5:0/3264883392 shutdown_connections 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.570+0000 7f751690c640 1 -- 172.21.15.5:0/3264883392 wait complete. 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.571+0000 7f751690c640 1 Processor -- start 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.571+0000 7f751690c640 1 -- start start 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.571+0000 7f751690c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f75101079d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.571+0000 7f751690c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f751010bac0 con 0x7f7510108150 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.571+0000 7f750ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f75101079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.571+0000 7f750ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f75101079d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46990/0 (socket says 172.21.15.5:46990) 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.571+0000 7f750ffff640 1 -- 172.21.15.5:0/2832427166 learned_addr learned my addr 172.21.15.5:0/2832427166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.571+0000 7f750ffff640 1 -- 172.21.15.5:0/2832427166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7510107f10 con 0x7f7510108150 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.572+0000 7f750ffff640 1 --2- 172.21.15.5:0/2832427166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f75101079d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f75000097f0 tx=0x7f7500030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:03.211 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.572+0000 7f750d7fa640 1 -- 172.21.15.5:0/2832427166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7500037040 con 0x7f7510108150 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.572+0000 7f751690c640 1 -- 172.21.15.5:0/2832427166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7510106100 con 0x7f7510108150 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.572+0000 7f750d7fa640 1 -- 172.21.15.5:0/2832427166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7500040070 con 0x7f7510108150 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.572+0000 7f750d7fa640 1 -- 172.21.15.5:0/2832427166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f750003a390 con 0x7f7510108150 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.572+0000 7f751690c640 1 -- 172.21.15.5:0/2832427166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75101065e0 con 0x7f7510108150 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.572+0000 7f750d7fa640 1 -- 172.21.15.5:0/2832427166 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f750003a630 con 0x7f7510108150 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.572+0000 7f750d7fa640 1 -- 172.21.15.5:0/2832427166 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f75000459a0 con 0x7f7510108150 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.573+0000 7f751690c640 1 -- 172.21.15.5:0/2832427166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75101085b0 con 0x7f7510108150 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.575+0000 7f750d7fa640 1 -- 172.21.15.5:0/2832427166 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7500045460 con 0x7f7510108150 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.653+0000 7f751690c640 1 -- 172.21.15.5:0/2832427166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7510106b10 con 0x7f7510108150 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.653+0000 7f750d7fa640 1 -- 172.21.15.5:0/2832427166 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f750003b430 con 0x7f7510108150 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.654+0000 7f751690c640 1 -- 172.21.15.5:0/2832427166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 msgr2=0x7f75101079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.654+0000 7f751690c640 1 --2- 172.21.15.5:0/2832427166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f75101079d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f75000097f0 tx=0x7f7500030d80 comp rx=0 tx=0).stop 2024-09-15T21:59:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.654+0000 7f751690c640 1 -- 172.21.15.5:0/2832427166 shutdown_connections 2024-09-15T21:59:03.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.654+0000 7f751690c640 1 --2- 172.21.15.5:0/2832427166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7510108150 0x7f75101079d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:03.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.654+0000 7f751690c640 1 -- 172.21.15.5:0/2832427166 >> 172.21.15.5:0/2832427166 conn(0x7f7510101d10 msgr2=0x7f7510102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:03.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.654+0000 7f751690c640 1 -- 172.21.15.5:0/2832427166 shutdown_connections 2024-09-15T21:59:03.213 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:02.655+0000 7f751690c640 1 -- 172.21.15.5:0/2832427166 wait complete. 2024-09-15T21:59:03.213 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (1/15)... 2024-09-15T21:59:03.826 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:03 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:03.554+0000 7f79077bf040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T21:59:03.826 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:03 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:03.674+0000 7f79077bf040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T21:59:04.106 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:03 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:03.823+0000 7f79077bf040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:59:04.106 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:03 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:03.887+0000 7f79077bf040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T21:59:04.106 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:04.042+0000 7f79077bf040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T21:59:04.410 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:04.113+0000 7f79077bf040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T21:59:04.410 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:04.175+0000 7f79077bf040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T21:59:04.410 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:04.340+0000 7f79077bf040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T21:59:04.410 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:04.407+0000 7f79077bf040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T21:59:04.689 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:04.477+0000 7f79077bf040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T21:59:04.689 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:04.563+0000 7f79077bf040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T21:59:04.689 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:04.625+0000 7f79077bf040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T21:59:04.689 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:04.686+0000 7f79077bf040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T21:59:05.575 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:05.520+0000 7f79077bf040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T21:59:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: Activating manager daemon x 2024-09-15T21:59:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: mgrmap e2: x(active, starting, since 0.00352465s) 2024-09-15T21:59:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: from='mgr.14100 172.21.15.5:0/969085319' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T21:59:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: from='mgr.14100 172.21.15.5:0/969085319' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T21:59:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: from='mgr.14100 172.21.15.5:0/969085319' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T21:59:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: from='mgr.14100 172.21.15.5:0/969085319' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T21:59:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: from='mgr.14100 172.21.15.5:0/969085319' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T21:59:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: Manager daemon x is now available 2024-09-15T21:59:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: from='mgr.14100 172.21.15.5:0/969085319' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T21:59:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: from='mgr.14100 172.21.15.5:0/969085319' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T21:59:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: from='mgr.14100 172.21.15.5:0/969085319' entity='mgr.x' 2024-09-15T21:59:05.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: from='mgr.14100 172.21.15.5:0/969085319' entity='mgr.x' 2024-09-15T21:59:05.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:05 smithi005 ceph-mon[30944]: from='mgr.14100 172.21.15.5:0/969085319' entity='mgr.x' 2024-09-15T21:59:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:59:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-15T21:59:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T21:59:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T21:59:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T21:59:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T21:59:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T21:59:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T21:59:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T21:59:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T21:59:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T21:59:06.271 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T21:59:06.271 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T21:59:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T21:59:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T21:59:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T21:59:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T21:59:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T21:59:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T21:59:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T21:59:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T21:59:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T21:59:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T21:58:48:668797+0000", 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T21:59:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T21:58:48.669426+0000", 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.646+0000 7f0b97d8f640 1 Processor -- start 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.646+0000 7f0b97d8f640 1 -- start start 2024-09-15T21:59:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.647+0000 7f0b97d8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9010a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.647+0000 7f0b97d8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9007be30 con 0x7f0b901080f0 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.647+0000 7f0b95b2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9010a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.647+0000 7f0b95b2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9010a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39304/0 (socket says 172.21.15.5:39304) 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.647+0000 7f0b95b2b640 1 -- 172.21.15.5:0/4150534708 learned_addr learned my addr 172.21.15.5:0/4150534708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.647+0000 7f0b95b2b640 1 -- 172.21.15.5:0/4150534708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9010aa20 con 0x7f0b901080f0 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.648+0000 7f0b95b2b640 1 --2- 172.21.15.5:0/4150534708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9010a4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009820 tx=0x7f0b8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd4266782fb16d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.648+0000 7f0b94b29640 1 -- 172.21.15.5:0/4150534708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b8c037070 con 0x7f0b901080f0 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.648+0000 7f0b94b29640 1 -- 172.21.15.5:0/4150534708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b8c032900 con 0x7f0b901080f0 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.648+0000 7f0b94b29640 1 -- 172.21.15.5:0/4150534708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b8c030850 con 0x7f0b901080f0 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.648+0000 7f0b97d8f640 1 -- 172.21.15.5:0/4150534708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 msgr2=0x7f0b9010a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.648+0000 7f0b97d8f640 1 --2- 172.21.15.5:0/4150534708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9010a4e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009820 tx=0x7f0b8c02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.649+0000 7f0b97d8f640 1 -- 172.21.15.5:0/4150534708 shutdown_connections 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.649+0000 7f0b97d8f640 1 --2- 172.21.15.5:0/4150534708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9010a4e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.649+0000 7f0b97d8f640 1 -- 172.21.15.5:0/4150534708 >> 172.21.15.5:0/4150534708 conn(0x7f0b90101d10 msgr2=0x7f0b90104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.649+0000 7f0b97d8f640 1 -- 172.21.15.5:0/4150534708 shutdown_connections 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.649+0000 7f0b97d8f640 1 -- 172.21.15.5:0/4150534708 wait complete. 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.649+0000 7f0b97d8f640 1 Processor -- start 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.649+0000 7f0b97d8f640 1 -- start start 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.649+0000 7f0b97d8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.649+0000 7f0b97d8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9010bac0 con 0x7f0b901080f0 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.649+0000 7f0b95b2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.650+0000 7f0b95b2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9019b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39308/0 (socket says 172.21.15.5:39308) 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.650+0000 7f0b95b2b640 1 -- 172.21.15.5:0/2786546586 learned_addr learned my addr 172.21.15.5:0/2786546586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.650+0000 7f0b95b2b640 1 -- 172.21.15.5:0/2786546586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9019bb80 con 0x7f0b901080f0 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.650+0000 7f0b95b2b640 1 --2- 172.21.15.5:0/2786546586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9019b640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c032ce0 tx=0x7f0b8c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.650+0000 7f0b86ffd640 1 -- 172.21.15.5:0/2786546586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b8c042070 con 0x7f0b901080f0 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.650+0000 7f0b86ffd640 1 -- 172.21.15.5:0/2786546586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0b8c0308c0 con 0x7f0b901080f0 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.650+0000 7f0b97d8f640 1 -- 172.21.15.5:0/2786546586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9019be10 con 0x7f0b901080f0 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.650+0000 7f0b86ffd640 1 -- 172.21.15.5:0/2786546586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b8c037040 con 0x7f0b901080f0 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.650+0000 7f0b97d8f640 1 -- 172.21.15.5:0/2786546586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9019eb00 con 0x7f0b901080f0 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.651+0000 7f0b86ffd640 1 -- 172.21.15.5:0/2786546586 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 2) ==== 46217+0+0 (secure 0 0 0) 0x7f0b8c045070 con 0x7f0b901080f0 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.651+0000 7f0b97d8f640 1 -- 172.21.15.5:0/2786546586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b90108550 con 0x7f0b901080f0 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.651+0000 7f0b86ffd640 1 -- 172.21.15.5:0/2786546586 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0b8c06bef0 con 0x7f0b901080f0 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.653+0000 7f0b86ffd640 1 -- 172.21.15.5:0/2786546586 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0b8c04b170 con 0x7f0b901080f0 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.715+0000 7f0b97d8f640 1 -- 172.21.15.5:0/2786546586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0b9019f180 con 0x7f0b901080f0 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.716+0000 7f0b86ffd640 1 -- 172.21.15.5:0/2786546586 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1292 (secure 0 0 0) 0x7f0b8c03cd50 con 0x7f0b901080f0 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.716+0000 7f0b97d8f640 1 -- 172.21.15.5:0/2786546586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 msgr2=0x7f0b9019b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.716+0000 7f0b97d8f640 1 --2- 172.21.15.5:0/2786546586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9019b640 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c032ce0 tx=0x7f0b8c032d10 comp rx=0 tx=0).stop 2024-09-15T21:59:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.716+0000 7f0b97d8f640 1 -- 172.21.15.5:0/2786546586 shutdown_connections 2024-09-15T21:59:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.716+0000 7f0b97d8f640 1 --2- 172.21.15.5:0/2786546586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b901080f0 0x7f0b9019b640 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.716+0000 7f0b97d8f640 1 -- 172.21.15.5:0/2786546586 >> 172.21.15.5:0/2786546586 conn(0x7f0b90101d10 msgr2=0x7f0b901028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.717+0000 7f0b97d8f640 1 -- 172.21.15.5:0/2786546586 shutdown_connections 2024-09-15T21:59:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:05.717+0000 7f0b97d8f640 1 -- 172.21.15.5:0/2786546586 wait complete. 2024-09-15T21:59:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (2/15)... 2024-09-15T21:59:06.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:06 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2786546586' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T21:59:06.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:06 smithi005 ceph-mon[30944]: mgrmap e3: x(active, since 1.00738s) 2024-09-15T21:59:08.658 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:08 smithi005 ceph-mon[30944]: mgrmap e4: x(active, since 2s) 2024-09-15T21:59:11.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:11 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2927103937' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-15T21:59:11.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:59:11.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-15T21:59:11.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T21:59:11.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-15T21:59:11.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-15T21:59:11.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-15T21:59:11.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-15T21:59:11.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:11.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-15T21:59:11.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:11.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-15T21:59:11.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-15T21:59:11.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-15T21:59:11.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-15T21:59:11.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-15T21:59:11.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-15T21:59:11.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:11.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-15T21:59:11.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-15T21:59:11.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-15T21:58:48:668797+0000", 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-15T21:59:11.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-15T21:59:11.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-15T21:58:48.669426+0000", 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.068+0000 7f17a2b9d640 1 Processor -- start 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.069+0000 7f17a2b9d640 1 -- start start 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.069+0000 7f17a2b9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.069+0000 7f17a2b9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f179c1080b0 con 0x7f179c1076e0 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.069+0000 7f17a0939640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.069+0000 7f17a0939640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39314/0 (socket says 172.21.15.5:39314) 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.069+0000 7f17a0939640 1 -- 172.21.15.5:0/135245146 learned_addr learned my addr 172.21.15.5:0/135245146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:11.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.069+0000 7f17a0939640 1 -- 172.21.15.5:0/135245146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c108890 con 0x7f179c1076e0 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.070+0000 7f17a0939640 1 --2- 172.21.15.5:0/135245146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c107ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1790009820 tx=0x7f179002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c10628fff2c8c10 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.070+0000 7f179b7fe640 1 -- 172.21.15.5:0/135245146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1790037070 con 0x7f179c1076e0 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.070+0000 7f179b7fe640 1 -- 172.21.15.5:0/135245146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1790032900 con 0x7f179c1076e0 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.070+0000 7f179b7fe640 1 -- 172.21.15.5:0/135245146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1790030850 con 0x7f179c1076e0 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.071+0000 7f17a2b9d640 1 -- 172.21.15.5:0/135245146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 msgr2=0x7f179c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.071+0000 7f17a2b9d640 1 --2- 172.21.15.5:0/135245146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c107ae0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1790009820 tx=0x7f179002a270 comp rx=0 tx=0).stop 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.071+0000 7f17a2b9d640 1 -- 172.21.15.5:0/135245146 shutdown_connections 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.071+0000 7f17a2b9d640 1 --2- 172.21.15.5:0/135245146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c107ae0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.071+0000 7f17a2b9d640 1 -- 172.21.15.5:0/135245146 >> 172.21.15.5:0/135245146 conn(0x7f179c102e90 msgr2=0x7f179c1052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.071+0000 7f17a2b9d640 1 -- 172.21.15.5:0/135245146 shutdown_connections 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.071+0000 7f17a2b9d640 1 -- 172.21.15.5:0/135245146 wait complete. 2024-09-15T21:59:11.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.072+0000 7f17a2b9d640 1 Processor -- start 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.072+0000 7f17a2b9d640 1 -- start start 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.072+0000 7f17a2b9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.072+0000 7f17a2b9d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f179c10bc40 con 0x7f179c1076e0 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.072+0000 7f17a0939640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.072+0000 7f17a0939640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c19b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39330/0 (socket says 172.21.15.5:39330) 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.072+0000 7f17a0939640 1 -- 172.21.15.5:0/2927103937 learned_addr learned my addr 172.21.15.5:0/2927103937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.073+0000 7f17a0939640 1 -- 172.21.15.5:0/2927103937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c19bbd0 con 0x7f179c1076e0 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.073+0000 7f17a0939640 1 --2- 172.21.15.5:0/2927103937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c19b690 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1790009940 tx=0x7f1790032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.073+0000 7f1799ffb640 1 -- 172.21.15.5:0/2927103937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1790042070 con 0x7f179c1076e0 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.073+0000 7f1799ffb640 1 -- 172.21.15.5:0/2927103937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f17900308c0 con 0x7f179c1076e0 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.073+0000 7f17a2b9d640 1 -- 172.21.15.5:0/2927103937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c19be60 con 0x7f179c1076e0 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.073+0000 7f1799ffb640 1 -- 172.21.15.5:0/2927103937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1790037040 con 0x7f179c1076e0 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.073+0000 7f17a2b9d640 1 -- 172.21.15.5:0/2927103937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c19eb50 con 0x7f179c1076e0 2024-09-15T21:59:11.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.074+0000 7f1799ffb640 1 -- 172.21.15.5:0/2927103937 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f1790045070 con 0x7f179c1076e0 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.074+0000 7f17a2b9d640 1 -- 172.21.15.5:0/2927103937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179c10bdc0 con 0x7f179c1076e0 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.075+0000 7f1799ffb640 1 --2- 172.21.15.5:0/2927103937 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f178c039160 0x7f178c03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.075+0000 7f1799ffb640 1 -- 172.21.15.5:0/2927103937 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f179006c420 con 0x7f179c1076e0 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.075+0000 7f179bfff640 1 --2- 172.21.15.5:0/2927103937 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f178c039160 0x7f178c03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.075+0000 7f179bfff640 1 --2- 172.21.15.5:0/2927103937 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f178c039160 0x7f178c03b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f17880098c0 tx=0x7f1788006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.080+0000 7f1799ffb640 1 -- 172.21.15.5:0/2927103937 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f179003be10 con 0x7f179c1076e0 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.321+0000 7f17a2b9d640 1 -- 172.21.15.5:0/2927103937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f179c19f1c0 con 0x7f179c1076e0 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.321+0000 7f1799ffb640 1 -- 172.21.15.5:0/2927103937 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f179006be40 con 0x7f179c1076e0 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.325+0000 7f17a2b9d640 1 -- 172.21.15.5:0/2927103937 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f178c039160 msgr2=0x7f178c03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.325+0000 7f17a2b9d640 1 --2- 172.21.15.5:0/2927103937 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f178c039160 0x7f178c03b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f17880098c0 tx=0x7f1788006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.325+0000 7f17a2b9d640 1 -- 172.21.15.5:0/2927103937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 msgr2=0x7f179c19b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.325+0000 7f17a2b9d640 1 --2- 172.21.15.5:0/2927103937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c19b690 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1790009940 tx=0x7f1790032ce0 comp rx=0 tx=0).stop 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.325+0000 7f17a2b9d640 1 -- 172.21.15.5:0/2927103937 shutdown_connections 2024-09-15T21:59:11.922 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.325+0000 7f17a2b9d640 1 --2- 172.21.15.5:0/2927103937 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f178c039160 0x7f178c03b620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:11.923 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.325+0000 7f17a2b9d640 1 --2- 172.21.15.5:0/2927103937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f179c1076e0 0x7f179c19b690 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:11.923 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.325+0000 7f17a2b9d640 1 -- 172.21.15.5:0/2927103937 >> 172.21.15.5:0/2927103937 conn(0x7f179c102e90 msgr2=0x7f179c103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:11.923 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.325+0000 7f17a2b9d640 1 -- 172.21.15.5:0/2927103937 shutdown_connections 2024-09-15T21:59:11.923 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:11.325+0000 7f17a2b9d640 1 -- 172.21.15.5:0/2927103937 wait complete. 2024-09-15T21:59:11.923 INFO:teuthology.orchestra.run.smithi005.stdout:mgr is available 2024-09-15T21:59:12.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:12 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/462380732' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-15T21:59:13.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:59:13.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-09-15T21:59:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-15T21:59:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-15T21:59:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-15T21:59:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-15T21:59:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-15T21:59:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.282+0000 7ffa5b441640 1 Processor -- start 2024-09-15T21:59:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.282+0000 7ffa5b441640 1 -- start start 2024-09-15T21:59:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.282+0000 7ffa5b441640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5410a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.282+0000 7ffa5b441640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5407be30 con 0x7ffa541080f0 2024-09-15T21:59:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.283+0000 7ffa591dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5410a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.283+0000 7ffa591dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5410a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39334/0 (socket says 172.21.15.5:39334) 2024-09-15T21:59:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.283+0000 7ffa591dd640 1 -- 172.21.15.5:0/3781335707 learned_addr learned my addr 172.21.15.5:0/3781335707 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.283+0000 7ffa591dd640 1 -- 172.21.15.5:0/3781335707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5410aa20 con 0x7ffa541080f0 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.284+0000 7ffa591dd640 1 --2- 172.21.15.5:0/3781335707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5410a4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009820 tx=0x7ffa4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c74e4006a7a928 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.284+0000 7ffa53fff640 1 -- 172.21.15.5:0/3781335707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa4c037070 con 0x7ffa541080f0 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.284+0000 7ffa53fff640 1 -- 172.21.15.5:0/3781335707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffa4c032900 con 0x7ffa541080f0 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.284+0000 7ffa53fff640 1 -- 172.21.15.5:0/3781335707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa4c030850 con 0x7ffa541080f0 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.284+0000 7ffa5b441640 1 -- 172.21.15.5:0/3781335707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 msgr2=0x7ffa5410a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.284+0000 7ffa5b441640 1 --2- 172.21.15.5:0/3781335707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5410a4e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009820 tx=0x7ffa4c02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.285+0000 7ffa5b441640 1 -- 172.21.15.5:0/3781335707 shutdown_connections 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.285+0000 7ffa5b441640 1 --2- 172.21.15.5:0/3781335707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5410a4e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.285+0000 7ffa5b441640 1 -- 172.21.15.5:0/3781335707 >> 172.21.15.5:0/3781335707 conn(0x7ffa54101d10 msgr2=0x7ffa54104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.285+0000 7ffa5b441640 1 -- 172.21.15.5:0/3781335707 shutdown_connections 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.285+0000 7ffa5b441640 1 -- 172.21.15.5:0/3781335707 wait complete. 2024-09-15T21:59:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.285+0000 7ffa5b441640 1 Processor -- start 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.285+0000 7ffa5b441640 1 -- start start 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.285+0000 7ffa5b441640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5419b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.285+0000 7ffa5b441640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5410bac0 con 0x7ffa541080f0 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.286+0000 7ffa591dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5419b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.286+0000 7ffa591dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5419b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39348/0 (socket says 172.21.15.5:39348) 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.286+0000 7ffa591dd640 1 -- 172.21.15.5:0/462380732 learned_addr learned my addr 172.21.15.5:0/462380732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.286+0000 7ffa591dd640 1 -- 172.21.15.5:0/462380732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5419bb80 con 0x7ffa541080f0 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.286+0000 7ffa591dd640 1 --2- 172.21.15.5:0/462380732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5419b640 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0097f0 tx=0x7ffa4c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.286+0000 7ffa527fc640 1 -- 172.21.15.5:0/462380732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa4c037040 con 0x7ffa541080f0 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.286+0000 7ffa527fc640 1 -- 172.21.15.5:0/462380732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffa4c040070 con 0x7ffa541080f0 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.286+0000 7ffa5b441640 1 -- 172.21.15.5:0/462380732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5419be10 con 0x7ffa541080f0 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.286+0000 7ffa527fc640 1 -- 172.21.15.5:0/462380732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa4c03a390 con 0x7ffa541080f0 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.286+0000 7ffa5b441640 1 -- 172.21.15.5:0/462380732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5419eb00 con 0x7ffa541080f0 2024-09-15T21:59:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.287+0000 7ffa527fc640 1 -- 172.21.15.5:0/462380732 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7ffa4c032900 con 0x7ffa541080f0 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.287+0000 7ffa5b441640 1 -- 172.21.15.5:0/462380732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa54108550 con 0x7ffa541080f0 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.287+0000 7ffa527fc640 1 --2- 172.21.15.5:0/462380732 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7ffa3c0391b0 0x7ffa3c03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.287+0000 7ffa527fc640 1 -- 172.21.15.5:0/462380732 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffa4c06c340 con 0x7ffa541080f0 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.288+0000 7ffa589dc640 1 --2- 172.21.15.5:0/462380732 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7ffa3c0391b0 0x7ffa3c03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.288+0000 7ffa589dc640 1 --2- 172.21.15.5:0/462380732 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7ffa3c0391b0 0x7ffa3c03b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa480098c0 tx=0x7ffa48006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.292+0000 7ffa527fc640 1 -- 172.21.15.5:0/462380732 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa4c06c0a0 con 0x7ffa541080f0 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.472+0000 7ffa5b441640 1 -- 172.21.15.5:0/462380732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ffa541a0370 con 0x7ffa541080f0 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.473+0000 7ffa527fc640 1 -- 172.21.15.5:0/462380732 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+427 (secure 0 0 0) 0x7ffa4c03a5e0 con 0x7ffa541080f0 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.475+0000 7ffa5b441640 1 -- 172.21.15.5:0/462380732 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7ffa3c0391b0 msgr2=0x7ffa3c03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.475+0000 7ffa5b441640 1 --2- 172.21.15.5:0/462380732 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7ffa3c0391b0 0x7ffa3c03b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa480098c0 tx=0x7ffa48006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.475+0000 7ffa5b441640 1 -- 172.21.15.5:0/462380732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 msgr2=0x7ffa5419b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:13.061 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.475+0000 7ffa5b441640 1 --2- 172.21.15.5:0/462380732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5419b640 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0097f0 tx=0x7ffa4c030d80 comp rx=0 tx=0).stop 2024-09-15T21:59:13.062 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.475+0000 7ffa5b441640 1 -- 172.21.15.5:0/462380732 shutdown_connections 2024-09-15T21:59:13.062 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.475+0000 7ffa5b441640 1 --2- 172.21.15.5:0/462380732 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7ffa3c0391b0 0x7ffa3c03b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:13.062 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.475+0000 7ffa5b441640 1 --2- 172.21.15.5:0/462380732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa541080f0 0x7ffa5419b640 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:13.062 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.475+0000 7ffa5b441640 1 -- 172.21.15.5:0/462380732 >> 172.21.15.5:0/462380732 conn(0x7ffa54101d10 msgr2=0x7ffa541028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:13.062 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.476+0000 7ffa5b441640 1 -- 172.21.15.5:0/462380732 shutdown_connections 2024-09-15T21:59:13.062 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:12.476+0000 7ffa5b441640 1 -- 172.21.15.5:0/462380732 wait complete. 2024-09-15T21:59:13.062 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling cephadm module... 2024-09-15T21:59:13.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:13 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/48304230' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-15T21:59:13.926 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:13 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: ignoring --setuser ceph since I am not root 2024-09-15T21:59:13.926 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:13 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: ignoring --setgroup ceph since I am not root 2024-09-15T21:59:13.926 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:13 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:13.865+0000 7efe59919040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T21:59:13.926 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:13 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:13.924+0000 7efe59919040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T21:59:14.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:14 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:14.079+0000 7efe59919040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T21:59:14.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:14 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:14.148+0000 7efe59919040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T21:59:14.304 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:14 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:14.302+0000 7efe59919040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:59:14.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.436+0000 7fb19f096640 1 Processor -- start 2024-09-15T21:59:14.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.436+0000 7fb19f096640 1 -- start start 2024-09-15T21:59:14.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.436+0000 7fb19f096640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb198107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.436+0000 7fb19f096640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb1981080b0 con 0x7fb1981076e0 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.436+0000 7fb19ce32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb198107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.436+0000 7fb19ce32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb198107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51120/0 (socket says 172.21.15.5:51120) 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.436+0000 7fb19ce32640 1 -- 172.21.15.5:0/3432489542 learned_addr learned my addr 172.21.15.5:0/3432489542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.437+0000 7fb19ce32640 1 -- 172.21.15.5:0/3432489542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb198108890 con 0x7fb1981076e0 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.437+0000 7fb19ce32640 1 --2- 172.21.15.5:0/3432489542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb198107ae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009820 tx=0x7fb18c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c123723cf7b75196 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.438+0000 7fb1977fe640 1 -- 172.21.15.5:0/3432489542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb18c037070 con 0x7fb1981076e0 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.438+0000 7fb1977fe640 1 -- 172.21.15.5:0/3432489542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb18c032900 con 0x7fb1981076e0 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.438+0000 7fb1977fe640 1 -- 172.21.15.5:0/3432489542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb18c030850 con 0x7fb1981076e0 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.438+0000 7fb19f096640 1 -- 172.21.15.5:0/3432489542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 msgr2=0x7fb198107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.438+0000 7fb19f096640 1 --2- 172.21.15.5:0/3432489542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb198107ae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb18c009820 tx=0x7fb18c02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.438+0000 7fb19f096640 1 -- 172.21.15.5:0/3432489542 shutdown_connections 2024-09-15T21:59:14.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.438+0000 7fb19f096640 1 --2- 172.21.15.5:0/3432489542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb198107ae0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:14.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.438+0000 7fb19f096640 1 -- 172.21.15.5:0/3432489542 >> 172.21.15.5:0/3432489542 conn(0x7fb198102e90 msgr2=0x7fb1981052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:14.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.438+0000 7fb19f096640 1 -- 172.21.15.5:0/3432489542 shutdown_connections 2024-09-15T21:59:14.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.438+0000 7fb19f096640 1 -- 172.21.15.5:0/3432489542 wait complete. 2024-09-15T21:59:14.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.439+0000 7fb19f096640 1 Processor -- start 2024-09-15T21:59:14.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.439+0000 7fb19f096640 1 -- start start 2024-09-15T21:59:14.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.439+0000 7fb19f096640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb19819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:14.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.439+0000 7fb19f096640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb19810bc40 con 0x7fb1981076e0 2024-09-15T21:59:14.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.439+0000 7fb19ce32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb19819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:14.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.439+0000 7fb19ce32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb19819b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51126/0 (socket says 172.21.15.5:51126) 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.440+0000 7fb19ce32640 1 -- 172.21.15.5:0/48304230 learned_addr learned my addr 172.21.15.5:0/48304230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.440+0000 7fb19ce32640 1 -- 172.21.15.5:0/48304230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19819bbd0 con 0x7fb1981076e0 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.440+0000 7fb19ce32640 1 --2- 172.21.15.5:0/48304230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb19819b690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0097f0 tx=0x7fb18c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.440+0000 7fb195ffb640 1 -- 172.21.15.5:0/48304230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb18c037040 con 0x7fb1981076e0 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.440+0000 7fb195ffb640 1 -- 172.21.15.5:0/48304230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb18c040070 con 0x7fb1981076e0 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.440+0000 7fb19f096640 1 -- 172.21.15.5:0/48304230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb19819be60 con 0x7fb1981076e0 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.440+0000 7fb195ffb640 1 -- 172.21.15.5:0/48304230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb18c03a390 con 0x7fb1981076e0 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.440+0000 7fb19f096640 1 -- 172.21.15.5:0/48304230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb19819eb50 con 0x7fb1981076e0 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.441+0000 7fb195ffb640 1 -- 172.21.15.5:0/48304230 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fb18c03a5e0 con 0x7fb1981076e0 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.442+0000 7fb19f096640 1 -- 172.21.15.5:0/48304230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb198107b60 con 0x7fb1981076e0 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.442+0000 7fb195ffb640 1 --2- 172.21.15.5:0/48304230 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fb1880391b0 0x7fb18803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.442+0000 7fb195ffb640 1 -- 172.21.15.5:0/48304230 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb18c06c4c0 con 0x7fb1981076e0 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.442+0000 7fb197fff640 1 --2- 172.21.15.5:0/48304230 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fb1880391b0 0x7fb18803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:14.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.442+0000 7fb197fff640 1 --2- 172.21.15.5:0/48304230 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fb1880391b0 0x7fb18803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb1840098c0 tx=0x7fb184006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.447+0000 7fb195ffb640 1 -- 172.21.15.5:0/48304230 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb18c043360 con 0x7fb1981076e0 2024-09-15T21:59:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.669+0000 7fb19f096640 1 -- 172.21.15.5:0/48304230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fb19819c2e0 con 0x7fb1981076e0 2024-09-15T21:59:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.721+0000 7fb195ffb640 1 -- 172.21.15.5:0/48304230 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fb18c06c220 con 0x7fb1981076e0 2024-09-15T21:59:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.722+0000 7fb195ffb640 1 -- 172.21.15.5:0/48304230 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fb18c045460 con 0x7fb1981076e0 2024-09-15T21:59:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.723+0000 7fb19f096640 1 -- 172.21.15.5:0/48304230 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fb1880391b0 msgr2=0x7fb18803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.723+0000 7fb19f096640 1 --2- 172.21.15.5:0/48304230 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fb1880391b0 0x7fb18803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb1840098c0 tx=0x7fb184006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.723+0000 7fb19f096640 1 -- 172.21.15.5:0/48304230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 msgr2=0x7fb19819b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.723+0000 7fb19f096640 1 --2- 172.21.15.5:0/48304230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb19819b690 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb18c0097f0 tx=0x7fb18c030d80 comp rx=0 tx=0).stop 2024-09-15T21:59:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.723+0000 7fb19f096640 1 -- 172.21.15.5:0/48304230 shutdown_connections 2024-09-15T21:59:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.723+0000 7fb19f096640 1 --2- 172.21.15.5:0/48304230 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fb1880391b0 0x7fb18803b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:14.340 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.723+0000 7fb19f096640 1 --2- 172.21.15.5:0/48304230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1981076e0 0x7fb19819b690 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:14.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.723+0000 7fb19f096640 1 -- 172.21.15.5:0/48304230 >> 172.21.15.5:0/48304230 conn(0x7fb198102e90 msgr2=0x7fb198103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:14.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.723+0000 7fb19f096640 1 -- 172.21.15.5:0/48304230 shutdown_connections 2024-09-15T21:59:14.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:13.723+0000 7fb19f096640 1 -- 172.21.15.5:0/48304230 wait complete. 2024-09-15T21:59:14.599 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:14 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:14.458+0000 7efe59919040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T21:59:14.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:14 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/48304230' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-15T21:59:14.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:14 smithi005 ceph-mon[30944]: mgrmap e5: x(active, since 8s) 2024-09-15T21:59:14.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:14 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:14.792+0000 7efe59919040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T21:59:15.317 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:14 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:14.856+0000 7efe59919040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T21:59:15.318 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:15 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:15.060+0000 7efe59919040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T21:59:15.593 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-15T21:59:15.593 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-15T21:59:15.593 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T21:59:15.593 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.777+0000 7fee9e753640 1 Processor -- start 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.777+0000 7fee9e753640 1 -- start start 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.777+0000 7fee9e753640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9810a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.778+0000 7fee9e753640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee9807bdf0 con 0x7fee98108130 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.778+0000 7fee97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9810a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.778+0000 7fee97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9810a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51160/0 (socket says 172.21.15.5:51160) 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.778+0000 7fee97fff640 1 -- 172.21.15.5:0/3212996167 learned_addr learned my addr 172.21.15.5:0/3212996167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.778+0000 7fee97fff640 1 -- 172.21.15.5:0/3212996167 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9810aa60 con 0x7fee98108130 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.779+0000 7fee97fff640 1 --2- 172.21.15.5:0/3212996167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9810a520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee88009820 tx=0x7fee8802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311878a6b6d0c949 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.779+0000 7fee977fe640 1 -- 172.21.15.5:0/3212996167 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee88037070 con 0x7fee98108130 2024-09-15T21:59:15.594 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.779+0000 7fee977fe640 1 -- 172.21.15.5:0/3212996167 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fee88032900 con 0x7fee98108130 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.779+0000 7fee977fe640 1 -- 172.21.15.5:0/3212996167 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee88030850 con 0x7fee98108130 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.780+0000 7fee9e753640 1 -- 172.21.15.5:0/3212996167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 msgr2=0x7fee9810a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.780+0000 7fee9e753640 1 --2- 172.21.15.5:0/3212996167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9810a520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fee88009820 tx=0x7fee8802a270 comp rx=0 tx=0).stop 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.780+0000 7fee9e753640 1 -- 172.21.15.5:0/3212996167 shutdown_connections 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.780+0000 7fee9e753640 1 --2- 172.21.15.5:0/3212996167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9810a520 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.780+0000 7fee9e753640 1 -- 172.21.15.5:0/3212996167 >> 172.21.15.5:0/3212996167 conn(0x7fee98101d10 msgr2=0x7fee98104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.780+0000 7fee9e753640 1 -- 172.21.15.5:0/3212996167 shutdown_connections 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.780+0000 7fee9e753640 1 -- 172.21.15.5:0/3212996167 wait complete. 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.780+0000 7fee9e753640 1 Processor -- start 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.780+0000 7fee9e753640 1 -- start start 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee9e753640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee9e753640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee9810baa0 con 0x7fee98108130 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:15.595 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9819d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51174/0 (socket says 172.21.15.5:51174) 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee97fff640 1 -- 172.21.15.5:0/3529092929 learned_addr learned my addr 172.21.15.5:0/3529092929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee97fff640 1 -- 172.21.15.5:0/3529092929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9819ddb0 con 0x7fee98108130 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee97fff640 1 --2- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9819d870 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee880097f0 tx=0x7fee88030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee95ffb640 1 -- 172.21.15.5:0/3529092929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee88037040 con 0x7fee98108130 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee95ffb640 1 -- 172.21.15.5:0/3529092929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fee88040070 con 0x7fee98108130 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee9e753640 1 -- 172.21.15.5:0/3529092929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9819e040 con 0x7fee98108130 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee95ffb640 1 -- 172.21.15.5:0/3529092929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee8803a390 con 0x7fee98108130 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.781+0000 7fee9e753640 1 -- 172.21.15.5:0/3529092929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee981a0d30 con 0x7fee98108130 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.782+0000 7fee95ffb640 1 -- 172.21.15.5:0/3529092929 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fee8803a5e0 con 0x7fee98108130 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.782+0000 7fee9e753640 1 -- 172.21.15.5:0/3529092929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee98108590 con 0x7fee98108130 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.782+0000 7fee95ffb640 1 --2- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fee84039200 0x7fee8403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.783+0000 7fee8ffff640 1 -- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fee84039200 msgr2=0x7fee8403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1812503826 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.783+0000 7fee95ffb640 1 -- 172.21.15.5:0/3529092929 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee8806c590 con 0x7fee98108130 2024-09-15T21:59:15.596 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.783+0000 7fee8ffff640 1 --2- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fee84039200 0x7fee8403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.787+0000 7fee95ffb640 1 -- 172.21.15.5:0/3529092929 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee8806c2f0 con 0x7fee98108130 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.984+0000 7fee8ffff640 1 -- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fee84039200 msgr2=0x7fee8403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1812503826 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:14.984+0000 7fee8ffff640 1 --2- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fee84039200 0x7fee8403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.012+0000 7fee9e753640 1 -- 172.21.15.5:0/3529092929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fee981a0f00 con 0x7fee98108130 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.012+0000 7fee95ffb640 1 -- 172.21.15.5:0/3529092929 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fee981a0f00 con 0x7fee98108130 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.015+0000 7fee9e753640 1 -- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fee84039200 msgr2=0x7fee8403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.015+0000 7fee9e753640 1 --2- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fee84039200 0x7fee8403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.015+0000 7fee9e753640 1 -- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 msgr2=0x7fee9819d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.015+0000 7fee9e753640 1 --2- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9819d870 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fee880097f0 tx=0x7fee88030d80 comp rx=0 tx=0).stop 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.015+0000 7fee9e753640 1 -- 172.21.15.5:0/3529092929 shutdown_connections 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.015+0000 7fee9e753640 1 --2- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7fee84039200 0x7fee8403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.015+0000 7fee9e753640 1 --2- 172.21.15.5:0/3529092929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee98108130 0x7fee9819d870 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.015+0000 7fee9e753640 1 -- 172.21.15.5:0/3529092929 >> 172.21.15.5:0/3529092929 conn(0x7fee98101d10 msgr2=0x7fee98102810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:15.597 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.015+0000 7fee9e753640 1 -- 172.21.15.5:0/3529092929 shutdown_connections 2024-09-15T21:59:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.015+0000 7fee9e753640 1 -- 172.21.15.5:0/3529092929 wait complete. 2024-09-15T21:59:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-09-15T21:59:15.598 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 5... 2024-09-15T21:59:15.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:15 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3529092929' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-15T21:59:16.199 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:15 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:15.996+0000 7efe59919040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T21:59:16.199 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:16 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:16.066+0000 7efe59919040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T21:59:16.856 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:16 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:16.814+0000 7efe59919040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T21:59:17.304 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:16 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:16.935+0000 7efe59919040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T21:59:17.304 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:17.084+0000 7efe59919040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:59:17.304 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:17.147+0000 7efe59919040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T21:59:17.304 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:17.302+0000 7efe59919040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T21:59:17.602 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:17.373+0000 7efe59919040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T21:59:17.602 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:17.435+0000 7efe59919040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T21:59:17.602 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:17.600+0000 7efe59919040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T21:59:17.856 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:17.668+0000 7efe59919040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T21:59:17.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:17.737+0000 7efe59919040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T21:59:17.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:17.823+0000 7efe59919040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T21:59:18.356 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:17.885+0000 7efe59919040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T21:59:18.357 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:17.946+0000 7efe59919040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T21:59:19.106 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:18.772+0000 7efe59919040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T21:59:19.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: Active manager daemon x restarted 2024-09-15T21:59:19.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: Activating manager daemon x 2024-09-15T21:59:19.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: osdmap e2: 0 total, 0 up, 0 in 2024-09-15T21:59:19.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: mgrmap e6: x(active, starting, since 0.00358651s) 2024-09-15T21:59:19.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T21:59:19.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T21:59:19.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T21:59:19.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T21:59:19.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T21:59:19.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: Manager daemon x is now available 2024-09-15T21:59:19.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:19.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:19.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T21:59:19.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T21:59:19.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T21:59:19.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:18 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T21:59:20.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:19 smithi005 ceph-mon[30944]: Found migration_current of "None". Setting to last migration. 2024-09-15T21:59:20.083 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:19 smithi005 ceph-mon[30944]: mgrmap e7: x(active, since 1.00568s) 2024-09-15T21:59:20.368 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-15T21:59:20.368 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-15T21:59:20.368 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-15T21:59:20.368 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-15T21:59:20.368 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.991+0000 7f7250a68640 1 Processor -- start 2024-09-15T21:59:20.368 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.991+0000 7f7250a68640 1 -- start start 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.992+0000 7f7250a68640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c10a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.992+0000 7f7250a68640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f724c07be30 con 0x7f724c1080f0 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.992+0000 7f724a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c10a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.992+0000 7f724a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c10a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51180/0 (socket says 172.21.15.5:51180) 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.992+0000 7f724a59c640 1 -- 172.21.15.5:0/2384499891 learned_addr learned my addr 172.21.15.5:0/2384499891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.992+0000 7f724a59c640 1 -- 172.21.15.5:0/2384499891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c10aa20 con 0x7f724c1080f0 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.993+0000 7f724a59c640 1 --2- 172.21.15.5:0/2384499891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c10a4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f723c009820 tx=0x7f723c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee27d3d638ed8448 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.993+0000 7f724959a640 1 -- 172.21.15.5:0/2384499891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f723c037070 con 0x7f724c1080f0 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.993+0000 7f724959a640 1 -- 172.21.15.5:0/2384499891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f723c032900 con 0x7f724c1080f0 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.993+0000 7f724959a640 1 -- 172.21.15.5:0/2384499891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f723c030850 con 0x7f724c1080f0 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.994+0000 7f7250a68640 1 -- 172.21.15.5:0/2384499891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 msgr2=0x7f724c10a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:20.369 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.994+0000 7f7250a68640 1 --2- 172.21.15.5:0/2384499891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c10a4e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f723c009820 tx=0x7f723c02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.994+0000 7f7250a68640 1 -- 172.21.15.5:0/2384499891 shutdown_connections 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.994+0000 7f7250a68640 1 --2- 172.21.15.5:0/2384499891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c10a4e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.994+0000 7f7250a68640 1 -- 172.21.15.5:0/2384499891 >> 172.21.15.5:0/2384499891 conn(0x7f724c101d10 msgr2=0x7f724c104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.994+0000 7f7250a68640 1 -- 172.21.15.5:0/2384499891 shutdown_connections 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.994+0000 7f7250a68640 1 -- 172.21.15.5:0/2384499891 wait complete. 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.995+0000 7f7250a68640 1 Processor -- start 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.995+0000 7f7250a68640 1 -- start start 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.995+0000 7f7250a68640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.995+0000 7f7250a68640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f724c10bac0 con 0x7f724c1080f0 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.995+0000 7f724a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.995+0000 7f724a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c19b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51192/0 (socket says 172.21.15.5:51192) 2024-09-15T21:59:20.370 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.995+0000 7f724a59c640 1 -- 172.21.15.5:0/2317021703 learned_addr learned my addr 172.21.15.5:0/2317021703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.995+0000 7f724a59c640 1 -- 172.21.15.5:0/2317021703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724c19bb90 con 0x7f724c1080f0 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.995+0000 7f724a59c640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c19b650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f723c0097f0 tx=0x7f723c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.996+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f723c037040 con 0x7f724c1080f0 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.996+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f723c040070 con 0x7f724c1080f0 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.996+0000 7f7250a68640 1 -- 172.21.15.5:0/2317021703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f724c19be20 con 0x7f724c1080f0 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.996+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f723c03a390 con 0x7f724c1080f0 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.996+0000 7f7250a68640 1 -- 172.21.15.5:0/2317021703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f724c19eb10 con 0x7f724c1080f0 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.997+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f723c03b430 con 0x7f724c1080f0 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.997+0000 7f723b7fe640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 0x7f723403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.997+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 --> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f723403bdd0 con 0x7f7234039200 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.997+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f723c06c3f0 con 0x7f724c1080f0 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.997+0000 7f7249d9b640 1 -- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 msgr2=0x7f723403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1812503826 2024-09-15T21:59:20.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:15.997+0000 7f7249d9b640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 0x7f723403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:16.198+0000 7f7249d9b640 1 -- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 msgr2=0x7f723403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1812503826 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:16.198+0000 7f7249d9b640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 0x7f723403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:16.599+0000 7f7249d9b640 1 -- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 msgr2=0x7f723403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1812503826 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:16.599+0000 7f7249d9b640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 0x7f723403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:17.400+0000 7f7249d9b640 1 -- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 msgr2=0x7f723403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1812503826 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:17.400+0000 7f7249d9b640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 0x7f723403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:18.775+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f723c06b1f0 con 0x7f724c1080f0 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:18.775+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 msgr2=0x7f723403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:18.775+0000 7f723b7fe640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 0x7f723403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.776+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f723c040ca0 con 0x7f724c1080f0 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.777+0000 7f723b7fe640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f723403cb30 0x7f723403ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.777+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 --> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f723c049e50 con 0x7f723403cb30 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.777+0000 7f7249d9b640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f723403cb30 0x7f723403ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:20.372 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.778+0000 7f7249d9b640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f723403cb30 0x7f723403ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7240003980 tx=0x7f72400092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.778+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 <== mgr.14118 v2:172.21.15.5:6800/3785442724 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f723c049e50 con 0x7f723403cb30 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.785+0000 7f7250a68640 1 -- 172.21.15.5:0/2317021703 --> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f724c108550 con 0x7f723403cb30 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f723b7fe640 1 -- 172.21.15.5:0/2317021703 <== mgr.14118 v2:172.21.15.5:6800/3785442724 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f724c108550 con 0x7f723403cb30 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f7250a68640 1 -- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f723403cb30 msgr2=0x7f723403ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f7250a68640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f723403cb30 0x7f723403ef20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7240003980 tx=0x7f72400092b0 comp rx=0 tx=0).stop 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f7250a68640 1 -- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 msgr2=0x7f724c19b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f7250a68640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c19b650 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f723c0097f0 tx=0x7f723c030d80 comp rx=0 tx=0).stop 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f7250a68640 1 -- 172.21.15.5:0/2317021703 shutdown_connections 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f7250a68640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f723403cb30 0x7f723403ef20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f7250a68640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:6800/1812503826,v1:172.21.15.5:6801/1812503826] conn(0x7f7234039200 0x7f723403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f7250a68640 1 --2- 172.21.15.5:0/2317021703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c1080f0 0x7f724c19b650 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f7250a68640 1 -- 172.21.15.5:0/2317021703 >> 172.21.15.5:0/2317021703 conn(0x7f724c101d10 msgr2=0x7f724c102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:20.373 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f7250a68640 1 -- 172.21.15.5:0/2317021703 shutdown_connections 2024-09-15T21:59:20.374 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:19.786+0000 7f7250a68640 1 -- 172.21.15.5:0/2317021703 wait complete. 2024-09-15T21:59:20.374 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 5 is available 2024-09-15T21:59:20.374 INFO:teuthology.orchestra.run.smithi005.stdout:Setting orchestrator backend to cephadm... 2024-09-15T21:59:21.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-mon[30944]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-15T21:59:21.573 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-mon[30944]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-15T21:59:21.573 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:21.573 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:21.573 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:21.573 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T21:59:21.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.764+0000 7f6f9212c640 1 Processor -- start 2024-09-15T21:59:21.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.764+0000 7f6f9212c640 1 -- start start 2024-09-15T21:59:21.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.764+0000 7f6f9212c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c10a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:21.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.764+0000 7f6f9212c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c07be30 con 0x7f6f8c1080f0 2024-09-15T21:59:21.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.764+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c10a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.764+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c10a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51268/0 (socket says 172.21.15.5:51268) 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.764+0000 7f6f8b7fe640 1 -- 172.21.15.5:0/4037764984 learned_addr learned my addr 172.21.15.5:0/4037764984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.765+0000 7f6f8b7fe640 1 -- 172.21.15.5:0/4037764984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c10aa20 con 0x7f6f8c1080f0 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.765+0000 7f6f8b7fe640 1 --2- 172.21.15.5:0/4037764984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c10a4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009820 tx=0x7f6f7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b15d258def3ac58 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.765+0000 7f6f8affd640 1 -- 172.21.15.5:0/4037764984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f7c037070 con 0x7f6f8c1080f0 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.765+0000 7f6f8affd640 1 -- 172.21.15.5:0/4037764984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6f7c032900 con 0x7f6f8c1080f0 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.765+0000 7f6f8affd640 1 -- 172.21.15.5:0/4037764984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f7c030850 con 0x7f6f8c1080f0 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.766+0000 7f6f9212c640 1 -- 172.21.15.5:0/4037764984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 msgr2=0x7f6f8c10a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.766+0000 7f6f9212c640 1 --2- 172.21.15.5:0/4037764984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c10a4e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009820 tx=0x7f6f7c02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.766+0000 7f6f9212c640 1 -- 172.21.15.5:0/4037764984 shutdown_connections 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.766+0000 7f6f9212c640 1 --2- 172.21.15.5:0/4037764984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c10a4e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:21.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.766+0000 7f6f9212c640 1 -- 172.21.15.5:0/4037764984 >> 172.21.15.5:0/4037764984 conn(0x7f6f8c101d10 msgr2=0x7f6f8c104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.766+0000 7f6f9212c640 1 -- 172.21.15.5:0/4037764984 shutdown_connections 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.766+0000 7f6f9212c640 1 -- 172.21.15.5:0/4037764984 wait complete. 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.767+0000 7f6f9212c640 1 Processor -- start 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.767+0000 7f6f9212c640 1 -- start start 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.767+0000 7f6f9212c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c19b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.767+0000 7f6f9212c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f8c10bac0 con 0x7f6f8c1080f0 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.767+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c19b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.767+0000 7f6f8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c19b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51274/0 (socket says 172.21.15.5:51274) 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.767+0000 7f6f8b7fe640 1 -- 172.21.15.5:0/3421925072 learned_addr learned my addr 172.21.15.5:0/3421925072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.767+0000 7f6f8b7fe640 1 -- 172.21.15.5:0/3421925072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8c19bb80 con 0x7f6f8c1080f0 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.768+0000 7f6f8b7fe640 1 --2- 172.21.15.5:0/3421925072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c19b640 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c032ce0 tx=0x7f6f7c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:21.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.768+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421925072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f7c042070 con 0x7f6f8c1080f0 2024-09-15T21:59:21.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.768+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421925072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6f7c0308c0 con 0x7f6f8c1080f0 2024-09-15T21:59:21.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.768+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421925072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f7c037040 con 0x7f6f8c1080f0 2024-09-15T21:59:21.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.768+0000 7f6f9212c640 1 -- 172.21.15.5:0/3421925072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f8c19be10 con 0x7f6f8c1080f0 2024-09-15T21:59:21.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.768+0000 7f6f9212c640 1 -- 172.21.15.5:0/3421925072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f8c19eb00 con 0x7f6f8c1080f0 2024-09-15T21:59:21.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.769+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421925072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f6f7c045070 con 0x7f6f8c1080f0 2024-09-15T21:59:21.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.769+0000 7f6f9212c640 1 -- 172.21.15.5:0/3421925072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f8c108550 con 0x7f6f8c1080f0 2024-09-15T21:59:21.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.769+0000 7f6f897fa640 1 --2- 172.21.15.5:0/3421925072 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6f780390a0 0x7f6f7803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:21.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.769+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421925072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6f7c06c180 con 0x7f6f8c1080f0 2024-09-15T21:59:21.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.769+0000 7f6f83fff640 1 --2- 172.21.15.5:0/3421925072 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6f780390a0 0x7f6f7803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:21.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.770+0000 7f6f83fff640 1 --2- 172.21.15.5:0/3421925072 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6f780390a0 0x7f6f7803b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6f740098c0 tx=0x7f6f74006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:21.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.774+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421925072 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f7c04b170 con 0x7f6f8c1080f0 2024-09-15T21:59:21.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.965+0000 7f6f9212c640 1 -- 172.21.15.5:0/3421925072 --> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f6f8c102fd0 con 0x7f6f780390a0 2024-09-15T21:59:21.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.970+0000 7f6f897fa640 1 -- 172.21.15.5:0/3421925072 <== mgr.14118 v2:172.21.15.5:6800/3785442724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6f8c102fd0 con 0x7f6f780390a0 2024-09-15T21:59:21.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.972+0000 7f6f9212c640 1 -- 172.21.15.5:0/3421925072 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6f780390a0 msgr2=0x7f6f7803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:21.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.972+0000 7f6f9212c640 1 --2- 172.21.15.5:0/3421925072 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6f780390a0 0x7f6f7803b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6f740098c0 tx=0x7f6f74006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:21.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.972+0000 7f6f9212c640 1 -- 172.21.15.5:0/3421925072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 msgr2=0x7f6f8c19b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:21.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.972+0000 7f6f9212c640 1 --2- 172.21.15.5:0/3421925072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c19b640 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c032ce0 tx=0x7f6f7c032d10 comp rx=0 tx=0).stop 2024-09-15T21:59:21.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.972+0000 7f6f9212c640 1 -- 172.21.15.5:0/3421925072 shutdown_connections 2024-09-15T21:59:21.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.972+0000 7f6f9212c640 1 --2- 172.21.15.5:0/3421925072 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6f780390a0 0x7f6f7803b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:21.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.972+0000 7f6f9212c640 1 --2- 172.21.15.5:0/3421925072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f8c1080f0 0x7f6f8c19b640 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:21.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.972+0000 7f6f9212c640 1 -- 172.21.15.5:0/3421925072 >> 172.21.15.5:0/3421925072 conn(0x7f6f8c101d10 msgr2=0x7f6f8c1028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:21.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.973+0000 7f6f9212c640 1 -- 172.21.15.5:0/3421925072 shutdown_connections 2024-09-15T21:59:21.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:20.973+0000 7f6f9212c640 1 -- 172.21.15.5:0/3421925072 wait complete. 2024-09-15T21:59:21.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:21.787+0000 7efdc0c33640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:21:59:21] ENGINE Error in HTTPServer.serve 2024-09-15T21:59:21.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: Traceback (most recent call last): 2024-09-15T21:59:21.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T21:59:21.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self._connections.run(self.expiration_interval) 2024-09-15T21:59:21.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T21:59:21.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self._run(expiration_interval) 2024-09-15T21:59:21.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T21:59:21.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T21:59:21.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T21:59:21.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T21:59:21.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T21:59:21.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: s = self.context.wrap_socket( 2024-09-15T21:59:21.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T21:59:21.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: return self.sslsocket_class._create( 2024-09-15T21:59:21.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T21:59:21.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self.do_handshake() 2024-09-15T21:59:21.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T21:59:21.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self._sslobj.do_handshake() 2024-09-15T21:59:21.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T21:59:21.858 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:22.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:22 smithi005 ceph-mon[30944]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T21:59:22.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:22 smithi005 ceph-mon[30944]: mgrmap e8: x(active, since 2s) 2024-09-15T21:59:22.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:22 smithi005 ceph-mon[30944]: [15/Sep/2024:21:59:21] ENGINE Bus STARTING 2024-09-15T21:59:22.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:22 smithi005 ceph-mon[30944]: [15/Sep/2024:21:59:21] ENGINE Serving on http://172.21.15.5:8765 2024-09-15T21:59:22.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:22 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T21:59:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-15T21:59:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.949+0000 7fe5d8aa0640 1 Processor -- start 2024-09-15T21:59:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.949+0000 7fe5d8aa0640 1 -- start start 2024-09-15T21:59:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.949+0000 7fe5d8aa0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d4107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.949+0000 7fe5d8aa0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d41080b0 con 0x7fe5d41076e0 2024-09-15T21:59:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.949+0000 7fe5d259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d4107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:22.697 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.949+0000 7fe5d259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d4107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51286/0 (socket says 172.21.15.5:51286) 2024-09-15T21:59:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.949+0000 7fe5d259c640 1 -- 172.21.15.5:0/3328990483 learned_addr learned my addr 172.21.15.5:0/3328990483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.950+0000 7fe5d259c640 1 -- 172.21.15.5:0/3328990483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d4108890 con 0x7fe5d41076e0 2024-09-15T21:59:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.950+0000 7fe5d259c640 1 --2- 172.21.15.5:0/3328990483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d4107ae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009820 tx=0x7fe5c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a380d40bafc1e2ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.951+0000 7fe5d1d9b640 1 -- 172.21.15.5:0/3328990483 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5c4037070 con 0x7fe5d41076e0 2024-09-15T21:59:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.951+0000 7fe5d1d9b640 1 -- 172.21.15.5:0/3328990483 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe5c4032900 con 0x7fe5d41076e0 2024-09-15T21:59:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.951+0000 7fe5d1d9b640 1 -- 172.21.15.5:0/3328990483 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5c4030850 con 0x7fe5d41076e0 2024-09-15T21:59:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.951+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/3328990483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 msgr2=0x7fe5d4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.951+0000 7fe5d8aa0640 1 --2- 172.21.15.5:0/3328990483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d4107ae0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009820 tx=0x7fe5c402a270 comp rx=0 tx=0).stop 2024-09-15T21:59:22.698 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.951+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/3328990483 shutdown_connections 2024-09-15T21:59:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.951+0000 7fe5d8aa0640 1 --2- 172.21.15.5:0/3328990483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d4107ae0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.951+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/3328990483 >> 172.21.15.5:0/3328990483 conn(0x7fe5d4102e90 msgr2=0x7fe5d41052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.951+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/3328990483 shutdown_connections 2024-09-15T21:59:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.951+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/3328990483 wait complete. 2024-09-15T21:59:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.952+0000 7fe5d8aa0640 1 Processor -- start 2024-09-15T21:59:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.952+0000 7fe5d8aa0640 1 -- start start 2024-09-15T21:59:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.952+0000 7fe5d8aa0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.952+0000 7fe5d8aa0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d410bc40 con 0x7fe5d41076e0 2024-09-15T21:59:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.952+0000 7fe5d259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.952+0000 7fe5d259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d419b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51288/0 (socket says 172.21.15.5:51288) 2024-09-15T21:59:22.699 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.953+0000 7fe5d259c640 1 -- 172.21.15.5:0/434171874 learned_addr learned my addr 172.21.15.5:0/434171874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.953+0000 7fe5d259c640 1 -- 172.21.15.5:0/434171874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d419bbd0 con 0x7fe5d41076e0 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.953+0000 7fe5d259c640 1 --2- 172.21.15.5:0/434171874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d419b690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40097f0 tx=0x7fe5c4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.953+0000 7fe5cb7fe640 1 -- 172.21.15.5:0/434171874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5c4037040 con 0x7fe5d41076e0 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.953+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/434171874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d419be60 con 0x7fe5d41076e0 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.953+0000 7fe5cb7fe640 1 -- 172.21.15.5:0/434171874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe5c4040070 con 0x7fe5d41076e0 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.953+0000 7fe5cb7fe640 1 -- 172.21.15.5:0/434171874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5c403a390 con 0x7fe5d41076e0 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.953+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/434171874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d419eb50 con 0x7fe5d41076e0 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.954+0000 7fe5cb7fe640 1 -- 172.21.15.5:0/434171874 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fe5c403a630 con 0x7fe5d41076e0 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.954+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/434171874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d4107b60 con 0x7fe5d41076e0 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.954+0000 7fe5cb7fe640 1 --2- 172.21.15.5:0/434171874 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fe5c00391b0 0x7fe5c003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.954+0000 7fe5cb7fe640 1 -- 172.21.15.5:0/434171874 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe5c406c8c0 con 0x7fe5d41076e0 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.955+0000 7fe5cbfff640 1 --2- 172.21.15.5:0/434171874 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fe5c00391b0 0x7fe5c003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:22.700 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.955+0000 7fe5cbfff640 1 --2- 172.21.15.5:0/434171874 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fe5c00391b0 0x7fe5c003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0098c0 tx=0x7fe5bc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:21.959+0000 7fe5cb7fe640 1 -- 172.21.15.5:0/434171874 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe5c4045460 con 0x7fe5d41076e0 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.135+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/434171874 --> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fe5d4104120 con 0x7fe5c00391b0 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.136+0000 7fe5cb7fe640 1 -- 172.21.15.5:0/434171874 <== mgr.14118 v2:172.21.15.5:6800/3785442724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fe5d4104120 con 0x7fe5c00391b0 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.138+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/434171874 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fe5c00391b0 msgr2=0x7fe5c003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.138+0000 7fe5d8aa0640 1 --2- 172.21.15.5:0/434171874 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fe5c00391b0 0x7fe5c003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0098c0 tx=0x7fe5bc006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.138+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/434171874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 msgr2=0x7fe5d419b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.138+0000 7fe5d8aa0640 1 --2- 172.21.15.5:0/434171874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d419b690 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40097f0 tx=0x7fe5c4030d80 comp rx=0 tx=0).stop 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.138+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/434171874 shutdown_connections 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.138+0000 7fe5d8aa0640 1 --2- 172.21.15.5:0/434171874 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fe5c00391b0 0x7fe5c003b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.138+0000 7fe5d8aa0640 1 --2- 172.21.15.5:0/434171874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5d41076e0 0x7fe5d419b690 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.138+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/434171874 >> 172.21.15.5:0/434171874 conn(0x7fe5d4102e90 msgr2=0x7fe5d4103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.138+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/434171874 shutdown_connections 2024-09-15T21:59:22.701 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:22.138+0000 7fe5d8aa0640 1 -- 172.21.15.5:0/434171874 wait complete. 2024-09-15T21:59:22.702 INFO:teuthology.orchestra.run.smithi005.stdout:Generating ssh key... 2024-09-15T21:59:23.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-mon[30944]: [15/Sep/2024:21:59:21] ENGINE Serving on https://172.21.15.5:7150 2024-09-15T21:59:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-mon[30944]: [15/Sep/2024:21:59:21] ENGINE Bus STARTED 2024-09-15T21:59:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-mon[30944]: [15/Sep/2024:21:59:21] ENGINE Error in HTTPServer.serve 2024-09-15T21:59:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-15T21:59:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T21:59:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-15T21:59:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T21:59:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-15T21:59:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T21:59:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T21:59:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T21:59:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-mon[30944]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:23.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: Generating public/private rsa key pair. 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: Your identification has been saved in /tmp/tmp2kof1h8f/key 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: Your public key has been saved in /tmp/tmp2kof1h8f/key.pub 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: The key fingerprint is: 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: SHA256:VuGG0SoZeTTJ6eiMBlRZm4yGy4HJT/AWhB9VmKcMLLo ceph-962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: The key's randomart image is: 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: +---[RSA 3072]----+ 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: | .++o+==++. | 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: |.=+=o=o+==.. | 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: |+o=+* **..+ | 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: |..== o+ oo | 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: | .oo + .S | 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: |E o o. | 2024-09-15T21:59:23.609 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: | . | 2024-09-15T21:59:23.610 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: | | 2024-09-15T21:59:23.610 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: | | 2024-09-15T21:59:23.610 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: +----[SHA256]-----+ 2024-09-15T21:59:24.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.065+0000 7f52c60aa640 1 Processor -- start 2024-09-15T21:59:24.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.065+0000 7f52c60aa640 1 -- start start 2024-09-15T21:59:24.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.065+0000 7f52c60aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c0107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:24.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.065+0000 7f52c60aa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52c01080b0 con 0x7f52c01076e0 2024-09-15T21:59:24.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.065+0000 7f52bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c0107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.065+0000 7f52bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c0107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51302/0 (socket says 172.21.15.5:51302) 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.065+0000 7f52bf7fe640 1 -- 172.21.15.5:0/1413968211 learned_addr learned my addr 172.21.15.5:0/1413968211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.066+0000 7f52bf7fe640 1 -- 172.21.15.5:0/1413968211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c0108890 con 0x7f52c01076e0 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.066+0000 7f52bf7fe640 1 --2- 172.21.15.5:0/1413968211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c0107ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009e80 tx=0x7f52b002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb5d3d917fb5eba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.067+0000 7f52be7fc640 1 -- 172.21.15.5:0/1413968211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b0036070 con 0x7f52c01076e0 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.067+0000 7f52be7fc640 1 -- 172.21.15.5:0/1413968211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52b00318c0 con 0x7f52c01076e0 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.067+0000 7f52be7fc640 1 -- 172.21.15.5:0/1413968211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b0007ab0 con 0x7f52c01076e0 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.067+0000 7f52c60aa640 1 -- 172.21.15.5:0/1413968211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 msgr2=0x7f52c0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.067+0000 7f52c60aa640 1 --2- 172.21.15.5:0/1413968211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c0107ae0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f52b0009e80 tx=0x7f52b002a7a0 comp rx=0 tx=0).stop 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.067+0000 7f52c60aa640 1 -- 172.21.15.5:0/1413968211 shutdown_connections 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.067+0000 7f52c60aa640 1 --2- 172.21.15.5:0/1413968211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c0107ae0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.067+0000 7f52c60aa640 1 -- 172.21.15.5:0/1413968211 >> 172.21.15.5:0/1413968211 conn(0x7f52c0102e90 msgr2=0x7f52c01052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.067+0000 7f52c60aa640 1 -- 172.21.15.5:0/1413968211 shutdown_connections 2024-09-15T21:59:24.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.067+0000 7f52c60aa640 1 -- 172.21.15.5:0/1413968211 wait complete. 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.068+0000 7f52c60aa640 1 Processor -- start 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.068+0000 7f52c60aa640 1 -- start start 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.068+0000 7f52c60aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c007e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.068+0000 7f52c60aa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52c010bc40 con 0x7f52c01076e0 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.068+0000 7f52bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c007e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.068+0000 7f52bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c007e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51314/0 (socket says 172.21.15.5:51314) 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.069+0000 7f52bf7fe640 1 -- 172.21.15.5:0/1864366106 learned_addr learned my addr 172.21.15.5:0/1864366106 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.069+0000 7f52bf7fe640 1 -- 172.21.15.5:0/1864366106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c007e8c0 con 0x7f52c01076e0 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.069+0000 7f52bf7fe640 1 --2- 172.21.15.5:0/1864366106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c007e380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f52b002ad90 tx=0x7f52b0031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.069+0000 7f52bcff9640 1 -- 172.21.15.5:0/1864366106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b0036050 con 0x7f52c01076e0 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.069+0000 7f52bcff9640 1 -- 172.21.15.5:0/1864366106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f52b0042070 con 0x7f52c01076e0 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.069+0000 7f52c60aa640 1 -- 172.21.15.5:0/1864366106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c007cab0 con 0x7f52c01076e0 2024-09-15T21:59:24.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.069+0000 7f52bcff9640 1 -- 172.21.15.5:0/1864366106 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52b003d7e0 con 0x7f52c01076e0 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.069+0000 7f52c60aa640 1 -- 172.21.15.5:0/1864366106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c007cff0 con 0x7f52c01076e0 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.070+0000 7f52bcff9640 1 -- 172.21.15.5:0/1864366106 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f52b003da90 con 0x7f52c01076e0 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.070+0000 7f52c60aa640 1 -- 172.21.15.5:0/1864366106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c0107b60 con 0x7f52c01076e0 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.070+0000 7f52bcff9640 1 --2- 172.21.15.5:0/1864366106 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f52ac039200 0x7f52ac03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.070+0000 7f52bcff9640 1 -- 172.21.15.5:0/1864366106 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f52b0007e00 con 0x7f52c01076e0 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.071+0000 7f52beffd640 1 --2- 172.21.15.5:0/1864366106 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f52ac039200 0x7f52ac03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.071+0000 7f52beffd640 1 --2- 172.21.15.5:0/1864366106 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f52ac039200 0x7f52ac03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f52a80098c0 tx=0x7f52a8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.075+0000 7f52bcff9640 1 -- 172.21.15.5:0/1864366106 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52b0039de0 con 0x7f52c01076e0 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.257+0000 7f52c60aa640 1 -- 172.21.15.5:0/1864366106 --> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f52c0104120 con 0x7f52ac039200 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.436+0000 7f52bcff9640 1 -- 172.21.15.5:0/1864366106 <== mgr.14118 v2:172.21.15.5:6800/3785442724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f52c0104120 con 0x7f52ac039200 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.439+0000 7f52c60aa640 1 -- 172.21.15.5:0/1864366106 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f52ac039200 msgr2=0x7f52ac03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.439+0000 7f52c60aa640 1 --2- 172.21.15.5:0/1864366106 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f52ac039200 0x7f52ac03b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f52a80098c0 tx=0x7f52a8006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:24.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.439+0000 7f52c60aa640 1 -- 172.21.15.5:0/1864366106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 msgr2=0x7f52c007e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:24.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.439+0000 7f52c60aa640 1 --2- 172.21.15.5:0/1864366106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c007e380 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f52b002ad90 tx=0x7f52b0031ca0 comp rx=0 tx=0).stop 2024-09-15T21:59:24.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.439+0000 7f52c60aa640 1 -- 172.21.15.5:0/1864366106 shutdown_connections 2024-09-15T21:59:24.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.439+0000 7f52c60aa640 1 --2- 172.21.15.5:0/1864366106 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f52ac039200 0x7f52ac03b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:24.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.439+0000 7f52c60aa640 1 --2- 172.21.15.5:0/1864366106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c01076e0 0x7f52c007e380 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:24.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.439+0000 7f52c60aa640 1 -- 172.21.15.5:0/1864366106 >> 172.21.15.5:0/1864366106 conn(0x7f52c0102e90 msgr2=0x7f52c0103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:24.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.439+0000 7f52c60aa640 1 -- 172.21.15.5:0/1864366106 shutdown_connections 2024-09-15T21:59:24.014 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:23.439+0000 7f52c60aa640 1 -- 172.21.15.5:0/1864366106 wait complete. 2024-09-15T21:59:24.586 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:24 smithi005 ceph-mon[30944]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T21:59:24.587 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:24 smithi005 ceph-mon[30944]: Generating ssh key... 2024-09-15T21:59:25.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCbXIQVxatulaq6+BiIjbdCEKZUtHM1QP5Q2CegsUvrPFP5HKmxTmWJn4Dlab3IWDKrNM+j/H6mx1dD832gLkKb9m/PcftCgNfJIPEALrZSgQbF/hpFF9K/eXTzpBzYU7Rv0eCLVtztVapKL444GHVyh/DIeq1ZARPszP/wKgelylEdqveGkpYvcOAwdlcMuVo8oOCvcOv7jR0PX6M/eIiWOqjWsfZ/XHm6qInPro8ZDFxQ7sxZiaKiPFWdLVoOT1Sgj06bjOnL42REokOnL/m1JAXb4t0LUKLulMpthqGkCRSmydbbXmkP6107cIJpq0Lp1Ir6mZYQe+tGfObvrgSUyP4dUv5cpoRKpp20072UcZ2edw/5XjOljspa2vm7RkqIlvxrQgnb5oqJDKdQUkR4zhr3BiYbi7ZKdu4Z9Sj3zX1DAOPIJ/8fb/ntwbbim5aAddjFad76fpms8D+N6juYKnmda4V1bzSYiUKtVwTCjBWuJAUcsLzEyel7Hj2/q0= ceph-962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:59:25.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.396+0000 7fcfc736e640 1 Processor -- start 2024-09-15T21:59:25.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.396+0000 7fcfc736e640 1 -- start start 2024-09-15T21:59:25.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.397+0000 7fcfc736e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc0107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:25.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.397+0000 7fcfc736e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0108070 con 0x7fcfc01076a0 2024-09-15T21:59:25.135 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.397+0000 7fcfc510a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc0107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.397+0000 7fcfc510a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc0107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41510/0 (socket says 172.21.15.5:41510) 2024-09-15T21:59:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.397+0000 7fcfc510a640 1 -- 172.21.15.5:0/1468797444 learned_addr learned my addr 172.21.15.5:0/1468797444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.397+0000 7fcfc510a640 1 -- 172.21.15.5:0/1468797444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc01088a0 con 0x7fcfc01076a0 2024-09-15T21:59:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.398+0000 7fcfc510a640 1 --2- 172.21.15.5:0/1468797444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc0107aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009820 tx=0x7fcfbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e98f0ba1b6ea1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.398+0000 7fcfb7fff640 1 -- 172.21.15.5:0/1468797444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfbc037070 con 0x7fcfc01076a0 2024-09-15T21:59:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.398+0000 7fcfb7fff640 1 -- 172.21.15.5:0/1468797444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcfbc032900 con 0x7fcfc01076a0 2024-09-15T21:59:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.398+0000 7fcfb7fff640 1 -- 172.21.15.5:0/1468797444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfbc030850 con 0x7fcfc01076a0 2024-09-15T21:59:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.399+0000 7fcfc736e640 1 -- 172.21.15.5:0/1468797444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 msgr2=0x7fcfc0107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.399+0000 7fcfc736e640 1 --2- 172.21.15.5:0/1468797444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc0107aa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009820 tx=0x7fcfbc02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.399+0000 7fcfc736e640 1 -- 172.21.15.5:0/1468797444 shutdown_connections 2024-09-15T21:59:25.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.399+0000 7fcfc736e640 1 --2- 172.21.15.5:0/1468797444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc0107aa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:25.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.399+0000 7fcfc736e640 1 -- 172.21.15.5:0/1468797444 >> 172.21.15.5:0/1468797444 conn(0x7fcfc0102e50 msgr2=0x7fcfc0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:25.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.399+0000 7fcfc736e640 1 -- 172.21.15.5:0/1468797444 shutdown_connections 2024-09-15T21:59:25.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.399+0000 7fcfc736e640 1 -- 172.21.15.5:0/1468797444 wait complete. 2024-09-15T21:59:25.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.399+0000 7fcfc736e640 1 Processor -- start 2024-09-15T21:59:25.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.400+0000 7fcfc736e640 1 -- start start 2024-09-15T21:59:25.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.400+0000 7fcfc736e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:25.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.400+0000 7fcfc736e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc010bc50 con 0x7fcfc01076a0 2024-09-15T21:59:25.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.400+0000 7fcfc510a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:25.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.400+0000 7fcfc510a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc019b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41514/0 (socket says 172.21.15.5:41514) 2024-09-15T21:59:25.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.400+0000 7fcfc510a640 1 -- 172.21.15.5:0/2961872469 learned_addr learned my addr 172.21.15.5:0/2961872469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:25.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.400+0000 7fcfc510a640 1 -- 172.21.15.5:0/2961872469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc019bbb0 con 0x7fcfc01076a0 2024-09-15T21:59:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.400+0000 7fcfc510a640 1 --2- 172.21.15.5:0/2961872469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc019b670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0097f0 tx=0x7fcfbc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.401+0000 7fcfb67fc640 1 -- 172.21.15.5:0/2961872469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfbc037040 con 0x7fcfc01076a0 2024-09-15T21:59:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.401+0000 7fcfb67fc640 1 -- 172.21.15.5:0/2961872469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcfbc040070 con 0x7fcfc01076a0 2024-09-15T21:59:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.401+0000 7fcfc736e640 1 -- 172.21.15.5:0/2961872469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc019be40 con 0x7fcfc01076a0 2024-09-15T21:59:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.401+0000 7fcfb67fc640 1 -- 172.21.15.5:0/2961872469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcfbc03a390 con 0x7fcfc01076a0 2024-09-15T21:59:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.401+0000 7fcfc736e640 1 -- 172.21.15.5:0/2961872469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc019eb30 con 0x7fcfc01076a0 2024-09-15T21:59:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.402+0000 7fcfb67fc640 1 -- 172.21.15.5:0/2961872469 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fcfbc03b430 con 0x7fcfc01076a0 2024-09-15T21:59:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.402+0000 7fcfc736e640 1 -- 172.21.15.5:0/2961872469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc010bdd0 con 0x7fcfc01076a0 2024-09-15T21:59:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.402+0000 7fcfb67fc640 1 --2- 172.21.15.5:0/2961872469 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fcfac0391b0 0x7fcfac03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:25.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.402+0000 7fcfb67fc640 1 -- 172.21.15.5:0/2961872469 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcfbc06c7a0 con 0x7fcfc01076a0 2024-09-15T21:59:25.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.402+0000 7fcfc4909640 1 --2- 172.21.15.5:0/2961872469 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fcfac0391b0 0x7fcfac03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:25.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.402+0000 7fcfc4909640 1 --2- 172.21.15.5:0/2961872469 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fcfac0391b0 0x7fcfac03b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00098c0 tx=0x7fcfb0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:25.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.407+0000 7fcfb67fc640 1 -- 172.21.15.5:0/2961872469 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcfbc032900 con 0x7fcfc01076a0 2024-09-15T21:59:25.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.585+0000 7fcfc736e640 1 -- 172.21.15.5:0/2961872469 --> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fcfc0104050 con 0x7fcfac0391b0 2024-09-15T21:59:25.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.586+0000 7fcfb67fc640 1 -- 172.21.15.5:0/2961872469 <== mgr.14118 v2:172.21.15.5:6800/3785442724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fcfc0104050 con 0x7fcfac0391b0 2024-09-15T21:59:25.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.588+0000 7fcfc736e640 1 -- 172.21.15.5:0/2961872469 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fcfac0391b0 msgr2=0x7fcfac03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:25.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.588+0000 7fcfc736e640 1 --2- 172.21.15.5:0/2961872469 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fcfac0391b0 0x7fcfac03b670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00098c0 tx=0x7fcfb0006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:25.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.588+0000 7fcfc736e640 1 -- 172.21.15.5:0/2961872469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 msgr2=0x7fcfc019b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:25.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.588+0000 7fcfc736e640 1 --2- 172.21.15.5:0/2961872469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc019b670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0097f0 tx=0x7fcfbc030d80 comp rx=0 tx=0).stop 2024-09-15T21:59:25.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.588+0000 7fcfc736e640 1 -- 172.21.15.5:0/2961872469 shutdown_connections 2024-09-15T21:59:25.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.588+0000 7fcfc736e640 1 --2- 172.21.15.5:0/2961872469 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fcfac0391b0 0x7fcfac03b670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:25.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.588+0000 7fcfc736e640 1 --2- 172.21.15.5:0/2961872469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc01076a0 0x7fcfc019b670 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:25.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.588+0000 7fcfc736e640 1 -- 172.21.15.5:0/2961872469 >> 172.21.15.5:0/2961872469 conn(0x7fcfc0102e50 msgr2=0x7fcfc0103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:25.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.588+0000 7fcfc736e640 1 -- 172.21.15.5:0/2961872469 shutdown_connections 2024-09-15T21:59:25.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:24.588+0000 7fcfc736e640 1 -- 172.21.15.5:0/2961872469 wait complete. 2024-09-15T21:59:25.140 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-15T21:59:25.140 INFO:teuthology.orchestra.run.smithi005.stdout:Adding key to root@localhost authorized_keys... 2024-09-15T21:59:25.140 INFO:teuthology.orchestra.run.smithi005.stdout:Adding host smithi005... 2024-09-15T21:59:25.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:25 smithi005 ceph-mon[30944]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T21:59:26.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:26 smithi005 ceph-mon[30944]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T21:59:27.955 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:27 smithi005 ceph-mon[30944]: Deploying cephadm binary to smithi005 2024-09-15T21:59:30.181 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Added host 'smithi005' with addr '172.21.15.5' 2024-09-15T21:59:30.181 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.528+0000 7f7ae8778640 1 Processor -- start 2024-09-15T21:59:30.181 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.528+0000 7f7ae8778640 1 -- start start 2024-09-15T21:59:30.181 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.529+0000 7f7ae8778640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae0107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.529+0000 7f7ae8778640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae01080b0 con 0x7f7ae01076e0 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.529+0000 7f7ae6514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae0107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.529+0000 7f7ae6514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae0107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41526/0 (socket says 172.21.15.5:41526) 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.529+0000 7f7ae6514640 1 -- 172.21.15.5:0/3827477624 learned_addr learned my addr 172.21.15.5:0/3827477624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.529+0000 7f7ae6514640 1 -- 172.21.15.5:0/3827477624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae0108890 con 0x7f7ae01076e0 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.530+0000 7f7ae6514640 1 --2- 172.21.15.5:0/3827477624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae0107ae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009820 tx=0x7f7adc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708336a4aae5b4b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.530+0000 7f7ae5512640 1 -- 172.21.15.5:0/3827477624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7adc037070 con 0x7f7ae01076e0 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.530+0000 7f7ae5512640 1 -- 172.21.15.5:0/3827477624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7adc032900 con 0x7f7ae01076e0 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.530+0000 7f7ae5512640 1 -- 172.21.15.5:0/3827477624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7adc030850 con 0x7f7ae01076e0 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.531+0000 7f7ae8778640 1 -- 172.21.15.5:0/3827477624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 msgr2=0x7f7ae0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.531+0000 7f7ae8778640 1 --2- 172.21.15.5:0/3827477624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae0107ae0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009820 tx=0x7f7adc02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.531+0000 7f7ae8778640 1 -- 172.21.15.5:0/3827477624 shutdown_connections 2024-09-15T21:59:30.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.531+0000 7f7ae8778640 1 --2- 172.21.15.5:0/3827477624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae0107ae0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.531+0000 7f7ae8778640 1 -- 172.21.15.5:0/3827477624 >> 172.21.15.5:0/3827477624 conn(0x7f7ae0102e90 msgr2=0x7f7ae01052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.531+0000 7f7ae8778640 1 -- 172.21.15.5:0/3827477624 shutdown_connections 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.531+0000 7f7ae8778640 1 -- 172.21.15.5:0/3827477624 wait complete. 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.532+0000 7f7ae8778640 1 Processor -- start 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.532+0000 7f7ae8778640 1 -- start start 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.532+0000 7f7ae8778640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae007e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.532+0000 7f7ae8778640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae010bc40 con 0x7f7ae01076e0 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.532+0000 7f7ae6514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae007e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.532+0000 7f7ae6514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae007e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41542/0 (socket says 172.21.15.5:41542) 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.532+0000 7f7ae6514640 1 -- 172.21.15.5:0/2645608938 learned_addr learned my addr 172.21.15.5:0/2645608938 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.533+0000 7f7ae6514640 1 -- 172.21.15.5:0/2645608938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae007e8c0 con 0x7f7ae01076e0 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.533+0000 7f7ae6514640 1 --2- 172.21.15.5:0/2645608938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae007e380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0097f0 tx=0x7f7adc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:30.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.533+0000 7f7ad77fe640 1 -- 172.21.15.5:0/2645608938 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7adc037040 con 0x7f7ae01076e0 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.533+0000 7f7ae8778640 1 -- 172.21.15.5:0/2645608938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae007cab0 con 0x7f7ae01076e0 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.533+0000 7f7ad77fe640 1 -- 172.21.15.5:0/2645608938 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7adc040070 con 0x7f7ae01076e0 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.533+0000 7f7ad77fe640 1 -- 172.21.15.5:0/2645608938 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7adc03a390 con 0x7f7ae01076e0 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.533+0000 7f7ae8778640 1 -- 172.21.15.5:0/2645608938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae007cf90 con 0x7f7ae01076e0 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.534+0000 7f7ad77fe640 1 -- 172.21.15.5:0/2645608938 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f7adc03b430 con 0x7f7ae01076e0 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.534+0000 7f7ae8778640 1 -- 172.21.15.5:0/2645608938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae0107b60 con 0x7f7ae01076e0 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.534+0000 7f7ad77fe640 1 --2- 172.21.15.5:0/2645608938 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f7acc0391b0 0x7f7acc03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.535+0000 7f7ad77fe640 1 -- 172.21.15.5:0/2645608938 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7adc06c7a0 con 0x7f7ae01076e0 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.535+0000 7f7ae5d13640 1 --2- 172.21.15.5:0/2645608938 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f7acc0391b0 0x7f7acc03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.535+0000 7f7ae5d13640 1 --2- 172.21.15.5:0/2645608938 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f7acc0391b0 0x7f7acc03b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00098c0 tx=0x7f7ad0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.539+0000 7f7ad77fe640 1 -- 172.21.15.5:0/2645608938 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7adc032900 con 0x7f7ae01076e0 2024-09-15T21:59:30.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:25.717+0000 7f7ae8778640 1 -- 172.21.15.5:0/2645608938 --> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}) -- 0x7f7ae0104120 con 0x7f7acc0391b0 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:29.608+0000 7f7ad77fe640 1 -- 172.21.15.5:0/2645608938 <== mgr.14118 v2:172.21.15.5:6800/3785442724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7f7ae0104120 con 0x7f7acc0391b0 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:29.610+0000 7f7ae8778640 1 -- 172.21.15.5:0/2645608938 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f7acc0391b0 msgr2=0x7f7acc03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:29.610+0000 7f7ae8778640 1 --2- 172.21.15.5:0/2645608938 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f7acc0391b0 0x7f7acc03b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00098c0 tx=0x7f7ad0006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:29.610+0000 7f7ae8778640 1 -- 172.21.15.5:0/2645608938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 msgr2=0x7f7ae007e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:29.610+0000 7f7ae8778640 1 --2- 172.21.15.5:0/2645608938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae007e380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0097f0 tx=0x7f7adc030d80 comp rx=0 tx=0).stop 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:29.610+0000 7f7ae8778640 1 -- 172.21.15.5:0/2645608938 shutdown_connections 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:29.610+0000 7f7ae8778640 1 --2- 172.21.15.5:0/2645608938 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f7acc0391b0 0x7f7acc03b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:29.610+0000 7f7ae8778640 1 --2- 172.21.15.5:0/2645608938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae01076e0 0x7f7ae007e380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:29.610+0000 7f7ae8778640 1 -- 172.21.15.5:0/2645608938 >> 172.21.15.5:0/2645608938 conn(0x7f7ae0102e90 msgr2=0x7f7ae0103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:29.610+0000 7f7ae8778640 1 -- 172.21.15.5:0/2645608938 shutdown_connections 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:29.610+0000 7f7ae8778640 1 -- 172.21.15.5:0/2645608938 wait complete. 2024-09-15T21:59:30.185 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mon service... 2024-09-15T21:59:30.763 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:30 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:30.763 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:30 smithi005 ceph-mon[30944]: Added host smithi005 2024-09-15T21:59:30.763 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:30 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T21:59:32.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:31 smithi005 ceph-mon[30944]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T21:59:32.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:31 smithi005 ceph-mon[30944]: Saving service mon spec with placement count:5 2024-09-15T21:59:32.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:31 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:32.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-15T21:59:32.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.559+0000 7f61cb155640 1 Processor -- start 2024-09-15T21:59:32.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.559+0000 7f61cb155640 1 -- start start 2024-09-15T21:59:32.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.559+0000 7f61cb155640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c4107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:32.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.559+0000 7f61cb155640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61c41080b0 con 0x7f61c41076e0 2024-09-15T21:59:32.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.559+0000 7f61c8ef1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c4107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.559+0000 7f61c8ef1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c4107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41544/0 (socket says 172.21.15.5:41544) 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.559+0000 7f61c8ef1640 1 -- 172.21.15.5:0/1487683509 learned_addr learned my addr 172.21.15.5:0/1487683509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.560+0000 7f61c8ef1640 1 -- 172.21.15.5:0/1487683509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c4108890 con 0x7f61c41076e0 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.560+0000 7f61c8ef1640 1 --2- 172.21.15.5:0/1487683509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c4107ae0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009820 tx=0x7f61bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f94b9b77c21c3e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.560+0000 7f61c37fe640 1 -- 172.21.15.5:0/1487683509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61bc037070 con 0x7f61c41076e0 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.561+0000 7f61c37fe640 1 -- 172.21.15.5:0/1487683509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f61bc032900 con 0x7f61c41076e0 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.561+0000 7f61c37fe640 1 -- 172.21.15.5:0/1487683509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61bc030850 con 0x7f61c41076e0 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.561+0000 7f61cb155640 1 -- 172.21.15.5:0/1487683509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 msgr2=0x7f61c4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.561+0000 7f61cb155640 1 --2- 172.21.15.5:0/1487683509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c4107ae0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009820 tx=0x7f61bc02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.561+0000 7f61cb155640 1 -- 172.21.15.5:0/1487683509 shutdown_connections 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.561+0000 7f61cb155640 1 --2- 172.21.15.5:0/1487683509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c4107ae0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.561+0000 7f61cb155640 1 -- 172.21.15.5:0/1487683509 >> 172.21.15.5:0/1487683509 conn(0x7f61c4102e90 msgr2=0x7f61c41052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:32.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.561+0000 7f61cb155640 1 -- 172.21.15.5:0/1487683509 shutdown_connections 2024-09-15T21:59:32.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.561+0000 7f61cb155640 1 -- 172.21.15.5:0/1487683509 wait complete. 2024-09-15T21:59:32.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.562+0000 7f61cb155640 1 Processor -- start 2024-09-15T21:59:32.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.562+0000 7f61cb155640 1 -- start start 2024-09-15T21:59:32.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.562+0000 7f61cb155640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c407e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:32.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.562+0000 7f61cb155640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61c410bc40 con 0x7f61c41076e0 2024-09-15T21:59:32.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.562+0000 7f61c8ef1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c407e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:32.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.562+0000 7f61c8ef1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c407e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41556/0 (socket says 172.21.15.5:41556) 2024-09-15T21:59:32.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.562+0000 7f61c8ef1640 1 -- 172.21.15.5:0/733869711 learned_addr learned my addr 172.21.15.5:0/733869711 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:32.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.562+0000 7f61c8ef1640 1 -- 172.21.15.5:0/733869711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c407e8c0 con 0x7f61c41076e0 2024-09-15T21:59:32.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.562+0000 7f61c8ef1640 1 --2- 172.21.15.5:0/733869711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c407e380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009940 tx=0x7f61bc032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:32.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.563+0000 7f61c1ffb640 1 -- 172.21.15.5:0/733869711 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61bc042070 con 0x7f61c41076e0 2024-09-15T21:59:32.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.563+0000 7f61cb155640 1 -- 172.21.15.5:0/733869711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c407cab0 con 0x7f61c41076e0 2024-09-15T21:59:32.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.563+0000 7f61cb155640 1 -- 172.21.15.5:0/733869711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c407cff0 con 0x7f61c41076e0 2024-09-15T21:59:32.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.563+0000 7f61c1ffb640 1 -- 172.21.15.5:0/733869711 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f61bc0308c0 con 0x7f61c41076e0 2024-09-15T21:59:32.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.563+0000 7f61c1ffb640 1 -- 172.21.15.5:0/733869711 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61bc037040 con 0x7f61c41076e0 2024-09-15T21:59:32.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.564+0000 7f61c1ffb640 1 -- 172.21.15.5:0/733869711 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f61bc045070 con 0x7f61c41076e0 2024-09-15T21:59:32.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.564+0000 7f61cb155640 1 -- 172.21.15.5:0/733869711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a40052c0 con 0x7f61c41076e0 2024-09-15T21:59:32.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.564+0000 7f61c1ffb640 1 --2- 172.21.15.5:0/733869711 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f61b00391b0 0x7f61b003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:32.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.564+0000 7f61c1ffb640 1 -- 172.21.15.5:0/733869711 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f61bc06c3d0 con 0x7f61c41076e0 2024-09-15T21:59:32.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.564+0000 7f61c3fff640 1 --2- 172.21.15.5:0/733869711 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f61b00391b0 0x7f61b003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.565+0000 7f61c3fff640 1 --2- 172.21.15.5:0/733869711 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f61b00391b0 0x7f61b003b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f61b400ad30 tx=0x7f61b40093f0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.569+0000 7f61c1ffb640 1 -- 172.21.15.5:0/733869711 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61bc0392e0 con 0x7f61c41076e0 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.761+0000 7f61cb155640 1 -- 172.21.15.5:0/733869711 --> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f61a4000bf0 con 0x7f61b00391b0 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.766+0000 7f61c1ffb640 1 -- 172.21.15.5:0/733869711 <== mgr.14118 v2:172.21.15.5:6800/3785442724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f61a4000bf0 con 0x7f61b00391b0 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.768+0000 7f61cb155640 1 -- 172.21.15.5:0/733869711 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f61b00391b0 msgr2=0x7f61b003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.768+0000 7f61cb155640 1 --2- 172.21.15.5:0/733869711 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f61b00391b0 0x7f61b003b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f61b400ad30 tx=0x7f61b40093f0 comp rx=0 tx=0).stop 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.768+0000 7f61cb155640 1 -- 172.21.15.5:0/733869711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 msgr2=0x7f61c407e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.768+0000 7f61cb155640 1 --2- 172.21.15.5:0/733869711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c407e380 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f61bc009940 tx=0x7f61bc032ce0 comp rx=0 tx=0).stop 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.768+0000 7f61cb155640 1 -- 172.21.15.5:0/733869711 shutdown_connections 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.768+0000 7f61cb155640 1 --2- 172.21.15.5:0/733869711 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f61b00391b0 0x7f61b003b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.768+0000 7f61cb155640 1 --2- 172.21.15.5:0/733869711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61c41076e0 0x7f61c407e380 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.768+0000 7f61cb155640 1 -- 172.21.15.5:0/733869711 >> 172.21.15.5:0/733869711 conn(0x7f61c4102e90 msgr2=0x7f61c4103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:32.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.768+0000 7f61cb155640 1 -- 172.21.15.5:0/733869711 shutdown_connections 2024-09-15T21:59:32.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:30.768+0000 7f61cb155640 1 -- 172.21.15.5:0/733869711 wait complete. 2024-09-15T21:59:32.294 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mgr service... 2024-09-15T21:59:33.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:32 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:33.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:32 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:33.916 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-15T21:59:33.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.687+0000 7f8cd4234640 1 Processor -- start 2024-09-15T21:59:33.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.687+0000 7f8cd4234640 1 -- start start 2024-09-15T21:59:33.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.687+0000 7f8cd4234640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc07cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:33.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.687+0000 7f8cd4234640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc07d050 con 0x7f8ccc07e660 2024-09-15T21:59:33.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.688+0000 7f8cd1fd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc07cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:33.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.688+0000 7f8cd1fd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc07cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41566/0 (socket says 172.21.15.5:41566) 2024-09-15T21:59:33.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.688+0000 7f8cd1fd0640 1 -- 172.21.15.5:0/1530249252 learned_addr learned my addr 172.21.15.5:0/1530249252 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:33.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.688+0000 7f8cd1fd0640 1 -- 172.21.15.5:0/1530249252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc07d1d0 con 0x7f8ccc07e660 2024-09-15T21:59:33.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.689+0000 7f8cd1fd0640 1 --2- 172.21.15.5:0/1530249252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc07cb10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009820 tx=0x7f8cc802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ae1a1ec5b1d34a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:33.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.689+0000 7f8cd0fce640 1 -- 172.21.15.5:0/1530249252 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cc8037070 con 0x7f8ccc07e660 2024-09-15T21:59:33.917 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.689+0000 7f8cd0fce640 1 -- 172.21.15.5:0/1530249252 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc8032920 con 0x7f8ccc07e660 2024-09-15T21:59:33.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.689+0000 7f8cd0fce640 1 -- 172.21.15.5:0/1530249252 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cc8030890 con 0x7f8ccc07e660 2024-09-15T21:59:33.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.690+0000 7f8cd4234640 1 -- 172.21.15.5:0/1530249252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 msgr2=0x7f8ccc07cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:33.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.690+0000 7f8cd4234640 1 --2- 172.21.15.5:0/1530249252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc07cb10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009820 tx=0x7f8cc802a300 comp rx=0 tx=0).stop 2024-09-15T21:59:33.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.690+0000 7f8cd4234640 1 -- 172.21.15.5:0/1530249252 shutdown_connections 2024-09-15T21:59:33.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.690+0000 7f8cd4234640 1 --2- 172.21.15.5:0/1530249252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc07cb10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:33.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.690+0000 7f8cd4234640 1 -- 172.21.15.5:0/1530249252 >> 172.21.15.5:0/1530249252 conn(0x7f8ccc102e90 msgr2=0x7f8ccc1052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:33.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.690+0000 7f8cd4234640 1 -- 172.21.15.5:0/1530249252 shutdown_connections 2024-09-15T21:59:33.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.690+0000 7f8cd4234640 1 -- 172.21.15.5:0/1530249252 wait complete. 2024-09-15T21:59:33.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.691+0000 7f8cd4234640 1 Processor -- start 2024-09-15T21:59:33.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.691+0000 7f8cd4234640 1 -- start start 2024-09-15T21:59:33.918 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.691+0000 7f8cd4234640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:33.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.691+0000 7f8cd4234640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc10dcf0 con 0x7f8ccc07e660 2024-09-15T21:59:33.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.691+0000 7f8cd1fd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:33.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.691+0000 7f8cd1fd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc19d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41576/0 (socket says 172.21.15.5:41576) 2024-09-15T21:59:33.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.691+0000 7f8cd1fd0640 1 -- 172.21.15.5:0/3403671290 learned_addr learned my addr 172.21.15.5:0/3403671290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:33.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.691+0000 7f8cd1fd0640 1 -- 172.21.15.5:0/3403671290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc19ddf0 con 0x7f8ccc07e660 2024-09-15T21:59:33.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.691+0000 7f8cd1fd0640 1 --2- 172.21.15.5:0/3403671290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc19d8b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80305a0 tx=0x7f8cc8030d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:33.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.692+0000 7f8cc2ffd640 1 -- 172.21.15.5:0/3403671290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cc8037040 con 0x7f8ccc07e660 2024-09-15T21:59:33.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.692+0000 7f8cc2ffd640 1 -- 172.21.15.5:0/3403671290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cc8040070 con 0x7f8ccc07e660 2024-09-15T21:59:33.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.692+0000 7f8cd4234640 1 -- 172.21.15.5:0/3403671290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc19e080 con 0x7f8ccc07e660 2024-09-15T21:59:33.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.692+0000 7f8cc2ffd640 1 -- 172.21.15.5:0/3403671290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cc803a390 con 0x7f8ccc07e660 2024-09-15T21:59:33.919 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.692+0000 7f8cd4234640 1 -- 172.21.15.5:0/3403671290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc1a0d70 con 0x7f8ccc07e660 2024-09-15T21:59:33.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.693+0000 7f8cc2ffd640 1 -- 172.21.15.5:0/3403671290 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f8cc803a640 con 0x7f8ccc07e660 2024-09-15T21:59:33.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.693+0000 7f8cd4234640 1 -- 172.21.15.5:0/3403671290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ccc07cb90 con 0x7f8ccc07e660 2024-09-15T21:59:33.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.693+0000 7f8cc2ffd640 1 --2- 172.21.15.5:0/3403671290 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f8cb8039200 0x7f8cb803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:33.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.693+0000 7f8cc2ffd640 1 -- 172.21.15.5:0/3403671290 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8cc806c820 con 0x7f8ccc07e660 2024-09-15T21:59:33.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.693+0000 7f8cd17cf640 1 --2- 172.21.15.5:0/3403671290 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f8cb8039200 0x7f8cb803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:33.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.694+0000 7f8cd17cf640 1 --2- 172.21.15.5:0/3403671290 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f8cb8039200 0x7f8cb803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0098c0 tx=0x7f8cbc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:33.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.698+0000 7f8cc2ffd640 1 -- 172.21.15.5:0/3403671290 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cc8041270 con 0x7f8ccc07e660 2024-09-15T21:59:33.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.908+0000 7f8cd4234640 1 -- 172.21.15.5:0/3403671290 --> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8ccc104110 con 0x7f8cb8039200 2024-09-15T21:59:33.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.912+0000 7f8cc2ffd640 1 -- 172.21.15.5:0/3403671290 <== mgr.14118 v2:172.21.15.5:6800/3785442724 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8ccc104110 con 0x7f8cb8039200 2024-09-15T21:59:33.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.914+0000 7f8cd4234640 1 -- 172.21.15.5:0/3403671290 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f8cb8039200 msgr2=0x7f8cb803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:33.920 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.914+0000 7f8cd4234640 1 --2- 172.21.15.5:0/3403671290 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f8cb8039200 0x7f8cb803b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0098c0 tx=0x7f8cbc006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:33.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.914+0000 7f8cd4234640 1 -- 172.21.15.5:0/3403671290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 msgr2=0x7f8ccc19d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:33.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.914+0000 7f8cd4234640 1 --2- 172.21.15.5:0/3403671290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc19d8b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80305a0 tx=0x7f8cc8030d70 comp rx=0 tx=0).stop 2024-09-15T21:59:33.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.914+0000 7f8cd4234640 1 -- 172.21.15.5:0/3403671290 shutdown_connections 2024-09-15T21:59:33.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.914+0000 7f8cd4234640 1 --2- 172.21.15.5:0/3403671290 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f8cb8039200 0x7f8cb803b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:33.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.914+0000 7f8cd4234640 1 --2- 172.21.15.5:0/3403671290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07e660 0x7f8ccc19d8b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:33.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.914+0000 7f8cd4234640 1 -- 172.21.15.5:0/3403671290 >> 172.21.15.5:0/3403671290 conn(0x7f8ccc102e90 msgr2=0x7f8ccc103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:33.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.914+0000 7f8cd4234640 1 -- 172.21.15.5:0/3403671290 shutdown_connections 2024-09-15T21:59:33.921 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:32.914+0000 7f8cd4234640 1 -- 172.21.15.5:0/3403671290 wait complete. 2024-09-15T21:59:34.181 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:33 smithi005 ceph-mon[30944]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T21:59:34.181 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:33 smithi005 ceph-mon[30944]: Saving service mgr spec with placement count:2 2024-09-15T21:59:34.181 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:33 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:35.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.357+0000 7fb2ca0da640 1 Processor -- start 2024-09-15T21:59:35.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.357+0000 7fb2ca0da640 1 -- start start 2024-09-15T21:59:35.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.357+0000 7fb2ca0da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c4107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:35.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.357+0000 7fb2ca0da640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c41080b0 con 0x7fb2c41076e0 2024-09-15T21:59:35.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.357+0000 7fb2c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c4107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.357+0000 7fb2c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c4107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53572/0 (socket says 172.21.15.5:53572) 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.357+0000 7fb2c37fe640 1 -- 172.21.15.5:0/1075241084 learned_addr learned my addr 172.21.15.5:0/1075241084 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.358+0000 7fb2c37fe640 1 -- 172.21.15.5:0/1075241084 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c4108890 con 0x7fb2c41076e0 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.358+0000 7fb2c37fe640 1 --2- 172.21.15.5:0/1075241084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c4107ae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009820 tx=0x7fb2b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7707ea4b756261d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.359+0000 7fb2c27fc640 1 -- 172.21.15.5:0/1075241084 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b4037070 con 0x7fb2c41076e0 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.359+0000 7fb2c27fc640 1 -- 172.21.15.5:0/1075241084 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b4032920 con 0x7fb2c41076e0 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.359+0000 7fb2c27fc640 1 -- 172.21.15.5:0/1075241084 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b4030810 con 0x7fb2c41076e0 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.359+0000 7fb2ca0da640 1 -- 172.21.15.5:0/1075241084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 msgr2=0x7fb2c4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.359+0000 7fb2ca0da640 1 --2- 172.21.15.5:0/1075241084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c4107ae0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009820 tx=0x7fb2b402a270 comp rx=0 tx=0).stop 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.359+0000 7fb2ca0da640 1 -- 172.21.15.5:0/1075241084 shutdown_connections 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.359+0000 7fb2ca0da640 1 --2- 172.21.15.5:0/1075241084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c4107ae0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:35.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.359+0000 7fb2ca0da640 1 -- 172.21.15.5:0/1075241084 >> 172.21.15.5:0/1075241084 conn(0x7fb2c4102e90 msgr2=0x7fb2c41052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.359+0000 7fb2ca0da640 1 -- 172.21.15.5:0/1075241084 shutdown_connections 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.359+0000 7fb2ca0da640 1 -- 172.21.15.5:0/1075241084 wait complete. 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.360+0000 7fb2ca0da640 1 Processor -- start 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.360+0000 7fb2ca0da640 1 -- start start 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.360+0000 7fb2ca0da640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.360+0000 7fb2ca0da640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c410bc40 con 0x7fb2c41076e0 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.360+0000 7fb2c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.360+0000 7fb2c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c419b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53584/0 (socket says 172.21.15.5:53584) 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.360+0000 7fb2c37fe640 1 -- 172.21.15.5:0/2961545213 learned_addr learned my addr 172.21.15.5:0/2961545213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.361+0000 7fb2c37fe640 1 -- 172.21.15.5:0/2961545213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c419bbd0 con 0x7fb2c41076e0 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.361+0000 7fb2c37fe640 1 --2- 172.21.15.5:0/2961545213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c419b690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4030560 tx=0x7fb2b4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:35.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.361+0000 7fb2c0ff9640 1 -- 172.21.15.5:0/2961545213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b4037040 con 0x7fb2c41076e0 2024-09-15T21:59:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.361+0000 7fb2c0ff9640 1 -- 172.21.15.5:0/2961545213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2b4040070 con 0x7fb2c41076e0 2024-09-15T21:59:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.361+0000 7fb2ca0da640 1 -- 172.21.15.5:0/2961545213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c419be60 con 0x7fb2c41076e0 2024-09-15T21:59:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.361+0000 7fb2c0ff9640 1 -- 172.21.15.5:0/2961545213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb2b403a740 con 0x7fb2c41076e0 2024-09-15T21:59:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.361+0000 7fb2ca0da640 1 -- 172.21.15.5:0/2961545213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c419eb50 con 0x7fb2c41076e0 2024-09-15T21:59:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.362+0000 7fb2c0ff9640 1 -- 172.21.15.5:0/2961545213 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb2b403a8e0 con 0x7fb2c41076e0 2024-09-15T21:59:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.362+0000 7fb2ca0da640 1 -- 172.21.15.5:0/2961545213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c410bdc0 con 0x7fb2c41076e0 2024-09-15T21:59:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.362+0000 7fb2c0ff9640 1 --2- 172.21.15.5:0/2961545213 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fb2ac039200 0x7fb2ac03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.362+0000 7fb2c0ff9640 1 -- 172.21.15.5:0/2961545213 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb2b406c8d0 con 0x7fb2c41076e0 2024-09-15T21:59:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.362+0000 7fb2c2ffd640 1 --2- 172.21.15.5:0/2961545213 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fb2ac039200 0x7fb2ac03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.363+0000 7fb2c2ffd640 1 --2- 172.21.15.5:0/2961545213 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fb2ac039200 0x7fb2ac03b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80098c0 tx=0x7fb2b8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:35.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.367+0000 7fb2c0ff9640 1 -- 172.21.15.5:0/2961545213 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2b4041270 con 0x7fb2c41076e0 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.547+0000 7fb2ca0da640 1 -- 172.21.15.5:0/2961545213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fb2c4107b60 con 0x7fb2c41076e0 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.549+0000 7fb2c0ff9640 1 -- 172.21.15.5:0/2961545213 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fb2b406c220 con 0x7fb2c41076e0 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.552+0000 7fb2ca0da640 1 -- 172.21.15.5:0/2961545213 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fb2ac039200 msgr2=0x7fb2ac03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.552+0000 7fb2ca0da640 1 --2- 172.21.15.5:0/2961545213 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fb2ac039200 0x7fb2ac03b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80098c0 tx=0x7fb2b8006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.552+0000 7fb2ca0da640 1 -- 172.21.15.5:0/2961545213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 msgr2=0x7fb2c419b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.552+0000 7fb2ca0da640 1 --2- 172.21.15.5:0/2961545213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c419b690 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4030560 tx=0x7fb2b4032670 comp rx=0 tx=0).stop 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.552+0000 7fb2ca0da640 1 -- 172.21.15.5:0/2961545213 shutdown_connections 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.552+0000 7fb2ca0da640 1 --2- 172.21.15.5:0/2961545213 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7fb2ac039200 0x7fb2ac03b6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.552+0000 7fb2ca0da640 1 --2- 172.21.15.5:0/2961545213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2c41076e0 0x7fb2c419b690 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.552+0000 7fb2ca0da640 1 -- 172.21.15.5:0/2961545213 >> 172.21.15.5:0/2961545213 conn(0x7fb2c4102e90 msgr2=0x7fb2c4103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.552+0000 7fb2ca0da640 1 -- 172.21.15.5:0/2961545213 shutdown_connections 2024-09-15T21:59:35.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:34.552+0000 7fb2ca0da640 1 -- 172.21.15.5:0/2961545213 wait complete. 2024-09-15T21:59:35.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:34 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:35.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:34 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2961545213' entity='client.admin' 2024-09-15T21:59:36.420 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.525+0000 7f15ba55f640 1 Processor -- start 2024-09-15T21:59:36.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.525+0000 7f15ba55f640 1 -- start start 2024-09-15T21:59:36.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.526+0000 7f15ba55f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b4107ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:36.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.526+0000 7f15ba55f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4108080 con 0x7f15b41076b0 2024-09-15T21:59:36.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.526+0000 7f15b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b4107ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:36.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.526+0000 7f15b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b4107ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53594/0 (socket says 172.21.15.5:53594) 2024-09-15T21:59:36.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.526+0000 7f15b3fff640 1 -- 172.21.15.5:0/4063329173 learned_addr learned my addr 172.21.15.5:0/4063329173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:36.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.526+0000 7f15b3fff640 1 -- 172.21.15.5:0/4063329173 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b41088b0 con 0x7f15b41076b0 2024-09-15T21:59:36.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.527+0000 7f15b3fff640 1 --2- 172.21.15.5:0/4063329173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b4107ab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009820 tx=0x7f15a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e59239e633bd07 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:36.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.527+0000 7f15b2ffd640 1 -- 172.21.15.5:0/4063329173 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15a4037070 con 0x7f15b41076b0 2024-09-15T21:59:36.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.527+0000 7f15b2ffd640 1 -- 172.21.15.5:0/4063329173 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15a4032920 con 0x7f15b41076b0 2024-09-15T21:59:36.421 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.527+0000 7f15b2ffd640 1 -- 172.21.15.5:0/4063329173 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15a4030810 con 0x7f15b41076b0 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.528+0000 7f15ba55f640 1 -- 172.21.15.5:0/4063329173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 msgr2=0x7f15b4107ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.528+0000 7f15ba55f640 1 --2- 172.21.15.5:0/4063329173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b4107ab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f15a4009820 tx=0x7f15a402a270 comp rx=0 tx=0).stop 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.528+0000 7f15ba55f640 1 -- 172.21.15.5:0/4063329173 shutdown_connections 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.528+0000 7f15ba55f640 1 --2- 172.21.15.5:0/4063329173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b4107ab0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.528+0000 7f15ba55f640 1 -- 172.21.15.5:0/4063329173 >> 172.21.15.5:0/4063329173 conn(0x7f15b4102e80 msgr2=0x7f15b41052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.528+0000 7f15ba55f640 1 -- 172.21.15.5:0/4063329173 shutdown_connections 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.528+0000 7f15ba55f640 1 -- 172.21.15.5:0/4063329173 wait complete. 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.529+0000 7f15ba55f640 1 Processor -- start 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.529+0000 7f15ba55f640 1 -- start start 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.529+0000 7f15ba55f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.529+0000 7f15ba55f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15b410bc60 con 0x7f15b41076b0 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.529+0000 7f15b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:36.422 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.529+0000 7f15b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b419b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53598/0 (socket says 172.21.15.5:53598) 2024-09-15T21:59:36.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.529+0000 7f15b3fff640 1 -- 172.21.15.5:0/4292563826 learned_addr learned my addr 172.21.15.5:0/4292563826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:36.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.530+0000 7f15b3fff640 1 -- 172.21.15.5:0/4292563826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b419bbe0 con 0x7f15b41076b0 2024-09-15T21:59:36.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.530+0000 7f15b3fff640 1 --2- 172.21.15.5:0/4292563826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b419b6a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f15a4030560 tx=0x7f15a4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:36.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.530+0000 7f15b17fa640 1 -- 172.21.15.5:0/4292563826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15a4037040 con 0x7f15b41076b0 2024-09-15T21:59:36.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.530+0000 7f15b17fa640 1 -- 172.21.15.5:0/4292563826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15a4040070 con 0x7f15b41076b0 2024-09-15T21:59:36.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.530+0000 7f15ba55f640 1 -- 172.21.15.5:0/4292563826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b419be70 con 0x7f15b41076b0 2024-09-15T21:59:36.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.530+0000 7f15b17fa640 1 -- 172.21.15.5:0/4292563826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15a403a740 con 0x7f15b41076b0 2024-09-15T21:59:36.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.530+0000 7f15ba55f640 1 -- 172.21.15.5:0/4292563826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b419eb60 con 0x7f15b41076b0 2024-09-15T21:59:36.423 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.531+0000 7f15b17fa640 1 -- 172.21.15.5:0/4292563826 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f15a403a9b0 con 0x7f15b41076b0 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.531+0000 7f15b17fa640 1 --2- 172.21.15.5:0/4292563826 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f15a00391b0 0x7f15a003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.531+0000 7f15ba55f640 1 -- 172.21.15.5:0/4292563826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b410bde0 con 0x7f15b41076b0 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.531+0000 7f15b17fa640 1 -- 172.21.15.5:0/4292563826 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f15a406d8f0 con 0x7f15b41076b0 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.532+0000 7f15b37fe640 1 --2- 172.21.15.5:0/4292563826 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f15a00391b0 0x7f15a003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.532+0000 7f15b37fe640 1 --2- 172.21.15.5:0/4292563826 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f15a00391b0 0x7f15a003b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f159c0098c0 tx=0x7f159c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.536+0000 7f15b17fa640 1 -- 172.21.15.5:0/4292563826 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15a4041270 con 0x7f15b41076b0 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.731+0000 7f15ba55f640 1 -- 172.21.15.5:0/4292563826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f15b4107b30 con 0x7f15b41076b0 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.734+0000 7f15b17fa640 1 -- 172.21.15.5:0/4292563826 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f15a405d0c0 con 0x7f15b41076b0 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.736+0000 7f15ba55f640 1 -- 172.21.15.5:0/4292563826 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f15a00391b0 msgr2=0x7f15a003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.736+0000 7f15ba55f640 1 --2- 172.21.15.5:0/4292563826 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f15a00391b0 0x7f15a003b670 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f159c0098c0 tx=0x7f159c006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.736+0000 7f15ba55f640 1 -- 172.21.15.5:0/4292563826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 msgr2=0x7f15b419b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:36.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.736+0000 7f15ba55f640 1 --2- 172.21.15.5:0/4292563826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b419b6a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f15a4030560 tx=0x7f15a4032670 comp rx=0 tx=0).stop 2024-09-15T21:59:36.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.736+0000 7f15ba55f640 1 -- 172.21.15.5:0/4292563826 shutdown_connections 2024-09-15T21:59:36.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.736+0000 7f15ba55f640 1 --2- 172.21.15.5:0/4292563826 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f15a00391b0 0x7f15a003b670 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:36.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.736+0000 7f15ba55f640 1 --2- 172.21.15.5:0/4292563826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15b41076b0 0x7f15b419b6a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:36.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.736+0000 7f15ba55f640 1 -- 172.21.15.5:0/4292563826 >> 172.21.15.5:0/4292563826 conn(0x7f15b4102e80 msgr2=0x7f15b4103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:36.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.736+0000 7f15ba55f640 1 -- 172.21.15.5:0/4292563826 shutdown_connections 2024-09-15T21:59:36.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:35.736+0000 7f15ba55f640 1 -- 172.21.15.5:0/4292563826 wait complete. 2024-09-15T21:59:36.425 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling the dashboard module... 2024-09-15T21:59:36.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:36 smithi005 ceph-mon[30944]: from='mgr.14118 172.21.15.5:0/4091965934' entity='mgr.x' 2024-09-15T21:59:36.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:36 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4292563826' entity='client.admin' 2024-09-15T21:59:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:37 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/104701807' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-15T21:59:37.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:37 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: ignoring --setuser ceph since I am not root 2024-09-15T21:59:37.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:37 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: ignoring --setgroup ceph since I am not root 2024-09-15T21:59:37.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:37 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:37.724+0000 7f0b55a08040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T21:59:37.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:37 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:37.783+0000 7f0b55a08040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T21:59:38.301 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:37 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:37.942+0000 7f0b55a08040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T21:59:38.301 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:38 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:38.011+0000 7f0b55a08040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T21:59:38.301 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:38 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:38.166+0000 7f0b55a08040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:59:38.553 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.857+0000 7f6b614f2640 1 Processor -- start 2024-09-15T21:59:38.554 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:38 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:38.315+0000 7f0b55a08040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T21:59:38.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.857+0000 7f6b614f2640 1 -- start start 2024-09-15T21:59:38.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.857+0000 7f6b614f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c10a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:38.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.857+0000 7f6b614f2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c07bdf0 con 0x7f6b5c108130 2024-09-15T21:59:38.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.857+0000 7f6b5affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c10a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:38.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.857+0000 7f6b5affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c10a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53614/0 (socket says 172.21.15.5:53614) 2024-09-15T21:59:38.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.857+0000 7f6b5affd640 1 -- 172.21.15.5:0/1102401289 learned_addr learned my addr 172.21.15.5:0/1102401289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:38.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.857+0000 7f6b5affd640 1 -- 172.21.15.5:0/1102401289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c10aa60 con 0x7f6b5c108130 2024-09-15T21:59:38.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.858+0000 7f6b5affd640 1 --2- 172.21.15.5:0/1102401289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c10a520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c009820 tx=0x7f6b4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2935209b990bad04 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:38.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.858+0000 7f6b59ffb640 1 -- 172.21.15.5:0/1102401289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b4c037070 con 0x7f6b5c108130 2024-09-15T21:59:38.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.858+0000 7f6b59ffb640 1 -- 172.21.15.5:0/1102401289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b4c032920 con 0x7f6b5c108130 2024-09-15T21:59:38.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.858+0000 7f6b59ffb640 1 -- 172.21.15.5:0/1102401289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b4c030810 con 0x7f6b5c108130 2024-09-15T21:59:38.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.859+0000 7f6b614f2640 1 -- 172.21.15.5:0/1102401289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 msgr2=0x7f6b5c10a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:38.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.859+0000 7f6b614f2640 1 --2- 172.21.15.5:0/1102401289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c10a520 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c009820 tx=0x7f6b4c02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:38.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.859+0000 7f6b614f2640 1 -- 172.21.15.5:0/1102401289 shutdown_connections 2024-09-15T21:59:38.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.859+0000 7f6b614f2640 1 --2- 172.21.15.5:0/1102401289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c10a520 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:38.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.859+0000 7f6b614f2640 1 -- 172.21.15.5:0/1102401289 >> 172.21.15.5:0/1102401289 conn(0x7f6b5c101d10 msgr2=0x7f6b5c104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:38.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.859+0000 7f6b614f2640 1 -- 172.21.15.5:0/1102401289 shutdown_connections 2024-09-15T21:59:38.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.859+0000 7f6b614f2640 1 -- 172.21.15.5:0/1102401289 wait complete. 2024-09-15T21:59:38.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.860+0000 7f6b614f2640 1 Processor -- start 2024-09-15T21:59:38.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.860+0000 7f6b614f2640 1 -- start start 2024-09-15T21:59:38.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.860+0000 7f6b614f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:38.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.860+0000 7f6b614f2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b5c10bae0 con 0x7f6b5c108130 2024-09-15T21:59:38.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.860+0000 7f6b5affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:38.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.860+0000 7f6b5affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c19b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53618/0 (socket says 172.21.15.5:53618) 2024-09-15T21:59:38.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.860+0000 7f6b5affd640 1 -- 172.21.15.5:0/104701807 learned_addr learned my addr 172.21.15.5:0/104701807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:38.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.860+0000 7f6b5affd640 1 -- 172.21.15.5:0/104701807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b5c19bba0 con 0x7f6b5c108130 2024-09-15T21:59:38.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.860+0000 7f6b5affd640 1 --2- 172.21.15.5:0/104701807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c19b660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c030380 tx=0x7f6b4c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:38.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.861+0000 7f6b43fff640 1 -- 172.21.15.5:0/104701807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b4c037040 con 0x7f6b5c108130 2024-09-15T21:59:38.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.861+0000 7f6b43fff640 1 -- 172.21.15.5:0/104701807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b4c040070 con 0x7f6b5c108130 2024-09-15T21:59:38.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.861+0000 7f6b43fff640 1 -- 172.21.15.5:0/104701807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b4c03a7b0 con 0x7f6b5c108130 2024-09-15T21:59:38.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.861+0000 7f6b614f2640 1 -- 172.21.15.5:0/104701807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b5c19be30 con 0x7f6b5c108130 2024-09-15T21:59:38.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.861+0000 7f6b614f2640 1 -- 172.21.15.5:0/104701807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b5c19eb20 con 0x7f6b5c108130 2024-09-15T21:59:38.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.862+0000 7f6b43fff640 1 -- 172.21.15.5:0/104701807 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f6b4c03b420 con 0x7f6b5c108130 2024-09-15T21:59:38.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.862+0000 7f6b614f2640 1 -- 172.21.15.5:0/104701807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b5c108590 con 0x7f6b5c108130 2024-09-15T21:59:38.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.862+0000 7f6b43fff640 1 --2- 172.21.15.5:0/104701807 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6b440391b0 0x7f6b4403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:38.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.862+0000 7f6b43fff640 1 -- 172.21.15.5:0/104701807 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6b4c06c8e0 con 0x7f6b5c108130 2024-09-15T21:59:38.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.862+0000 7f6b5a7fc640 1 --2- 172.21.15.5:0/104701807 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6b440391b0 0x7f6b4403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:38.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.862+0000 7f6b5a7fc640 1 --2- 172.21.15.5:0/104701807 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6b440391b0 0x7f6b4403b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6b500098c0 tx=0x7f6b50006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:38.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:36.867+0000 7f6b43fff640 1 -- 172.21.15.5:0/104701807 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b4c043360 con 0x7f6b5c108130 2024-09-15T21:59:38.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.093+0000 7f6b614f2640 1 -- 172.21.15.5:0/104701807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f6b5c19f110 con 0x7f6b5c108130 2024-09-15T21:59:38.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.570+0000 7f6b43fff640 1 -- 172.21.15.5:0/104701807 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f6b4c040220 con 0x7f6b5c108130 2024-09-15T21:59:38.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.571+0000 7f6b43fff640 1 -- 172.21.15.5:0/104701807 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f6b4c0313a0 con 0x7f6b5c108130 2024-09-15T21:59:38.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.573+0000 7f6b614f2640 1 -- 172.21.15.5:0/104701807 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6b440391b0 msgr2=0x7f6b4403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:38.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.573+0000 7f6b614f2640 1 --2- 172.21.15.5:0/104701807 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6b440391b0 0x7f6b4403b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6b500098c0 tx=0x7f6b50006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:38.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.573+0000 7f6b614f2640 1 -- 172.21.15.5:0/104701807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 msgr2=0x7f6b5c19b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:38.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.573+0000 7f6b614f2640 1 --2- 172.21.15.5:0/104701807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c19b660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c030380 tx=0x7f6b4c032670 comp rx=0 tx=0).stop 2024-09-15T21:59:38.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.574+0000 7f6b614f2640 1 -- 172.21.15.5:0/104701807 shutdown_connections 2024-09-15T21:59:38.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.574+0000 7f6b614f2640 1 --2- 172.21.15.5:0/104701807 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f6b440391b0 0x7f6b4403b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:38.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.574+0000 7f6b614f2640 1 --2- 172.21.15.5:0/104701807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b5c108130 0x7f6b5c19b660 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:38.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.574+0000 7f6b614f2640 1 -- 172.21.15.5:0/104701807 >> 172.21.15.5:0/104701807 conn(0x7f6b5c101d10 msgr2=0x7f6b5c102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:38.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.574+0000 7f6b614f2640 1 -- 172.21.15.5:0/104701807 shutdown_connections 2024-09-15T21:59:38.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:37.574+0000 7f6b614f2640 1 -- 172.21.15.5:0/104701807 wait complete. 2024-09-15T21:59:38.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:38 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/104701807' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-15T21:59:38.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:38 smithi005 ceph-mon[30944]: mgrmap e9: x(active, since 18s) 2024-09-15T21:59:38.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:38 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:38.653+0000 7f0b55a08040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T21:59:38.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:38 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:38.717+0000 7f0b55a08040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T21:59:39.148 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:38 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:38.918+0000 7f0b55a08040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T21:59:39.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:39 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4115244811' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-15T21:59:39.785 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-15T21:59:39.792 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-15T21:59:39.792 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-15T21:59:39.792 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-15T21:59:39.792 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-15T21:59:39.792 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-15T21:59:39.792 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.939+0000 7f1cb89a0640 1 Processor -- start 2024-09-15T21:59:39.792 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.939+0000 7f1cb89a0640 1 -- start start 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.940+0000 7f1cb89a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb410a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.940+0000 7f1cb89a0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb407bdf0 con 0x7f1cb4108130 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.940+0000 7f1cb259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb410a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.940+0000 7f1cb259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb410a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53640/0 (socket says 172.21.15.5:53640) 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.940+0000 7f1cb259c640 1 -- 172.21.15.5:0/523338190 learned_addr learned my addr 172.21.15.5:0/523338190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.940+0000 7f1cb259c640 1 -- 172.21.15.5:0/523338190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb410aa60 con 0x7f1cb4108130 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.941+0000 7f1cb259c640 1 --2- 172.21.15.5:0/523338190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb410a520 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009820 tx=0x7f1ca402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0cd235f548d23f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.941+0000 7f1cb159a640 1 -- 172.21.15.5:0/523338190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ca4037070 con 0x7f1cb4108130 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.941+0000 7f1cb159a640 1 -- 172.21.15.5:0/523338190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca4032920 con 0x7f1cb4108130 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.941+0000 7f1cb159a640 1 -- 172.21.15.5:0/523338190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ca4030810 con 0x7f1cb4108130 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.942+0000 7f1cb89a0640 1 -- 172.21.15.5:0/523338190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 msgr2=0x7f1cb410a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.942+0000 7f1cb89a0640 1 --2- 172.21.15.5:0/523338190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb410a520 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009820 tx=0x7f1ca402a270 comp rx=0 tx=0).stop 2024-09-15T21:59:39.793 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.943+0000 7f1cb89a0640 1 -- 172.21.15.5:0/523338190 shutdown_connections 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.943+0000 7f1cb89a0640 1 --2- 172.21.15.5:0/523338190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb410a520 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.943+0000 7f1cb89a0640 1 -- 172.21.15.5:0/523338190 >> 172.21.15.5:0/523338190 conn(0x7f1cb4101d10 msgr2=0x7f1cb4104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.943+0000 7f1cb89a0640 1 -- 172.21.15.5:0/523338190 shutdown_connections 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.943+0000 7f1cb89a0640 1 -- 172.21.15.5:0/523338190 wait complete. 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.944+0000 7f1cb89a0640 1 Processor -- start 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.944+0000 7f1cb89a0640 1 -- start start 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.944+0000 7f1cb89a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb41079b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.944+0000 7f1cb89a0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb410bae0 con 0x7f1cb4108130 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.944+0000 7f1cb259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb41079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.944+0000 7f1cb259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb41079b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53650/0 (socket says 172.21.15.5:53650) 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.944+0000 7f1cb259c640 1 -- 172.21.15.5:0/4115244811 learned_addr learned my addr 172.21.15.5:0/4115244811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.944+0000 7f1cb259c640 1 -- 172.21.15.5:0/4115244811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb4107ef0 con 0x7f1cb4108130 2024-09-15T21:59:39.794 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.944+0000 7f1cb259c640 1 --2- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb41079b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4030380 tx=0x7f1ca4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.945+0000 7f1ca37fe640 1 -- 172.21.15.5:0/4115244811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ca4037040 con 0x7f1cb4108130 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.945+0000 7f1ca37fe640 1 -- 172.21.15.5:0/4115244811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca4040070 con 0x7f1cb4108130 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.945+0000 7f1ca37fe640 1 -- 172.21.15.5:0/4115244811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1ca403a7b0 con 0x7f1cb4108130 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.945+0000 7f1cb89a0640 1 -- 172.21.15.5:0/4115244811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb41060e0 con 0x7f1cb4108130 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.945+0000 7f1cb89a0640 1 -- 172.21.15.5:0/4115244811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb4106620 con 0x7f1cb4108130 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.946+0000 7f1ca37fe640 1 -- 172.21.15.5:0/4115244811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f1ca403aa30 con 0x7f1cb4108130 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.946+0000 7f1cb89a0640 1 -- 172.21.15.5:0/4115244811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb4108590 con 0x7f1cb4108130 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.946+0000 7f1ca37fe640 1 --2- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f1c9c039200 0x7f1c9c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.946+0000 7f1ca37fe640 1 -- 172.21.15.5:0/4115244811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1ca406c970 con 0x7f1cb4108130 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.946+0000 7f1cb1d9b640 1 -- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f1c9c039200 msgr2=0x7f1c9c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3785442724 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.946+0000 7f1cb1d9b640 1 --2- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f1c9c039200 0x7f1c9c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:38.951+0000 7f1ca37fe640 1 -- 172.21.15.5:0/4115244811 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ca4041270 con 0x7f1cb4108130 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.147+0000 7f1cb1d9b640 1 -- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f1c9c039200 msgr2=0x7f1c9c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3785442724 2024-09-15T21:59:39.795 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.147+0000 7f1cb1d9b640 1 --2- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f1c9c039200 0x7f1c9c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.174+0000 7f1cb89a0640 1 -- 172.21.15.5:0/4115244811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1cb41068a0 con 0x7f1cb4108130 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.174+0000 7f1ca37fe640 1 -- 172.21.15.5:0/4115244811 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f1ca403ae70 con 0x7f1cb4108130 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.176+0000 7f1cb89a0640 1 -- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f1c9c039200 msgr2=0x7f1c9c03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.176+0000 7f1cb89a0640 1 --2- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f1c9c039200 0x7f1c9c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.176+0000 7f1cb89a0640 1 -- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 msgr2=0x7f1cb41079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.176+0000 7f1cb89a0640 1 --2- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb41079b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4030380 tx=0x7f1ca4032670 comp rx=0 tx=0).stop 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.176+0000 7f1cb89a0640 1 -- 172.21.15.5:0/4115244811 shutdown_connections 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.176+0000 7f1cb89a0640 1 --2- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f1c9c039200 0x7f1c9c03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.176+0000 7f1cb89a0640 1 --2- 172.21.15.5:0/4115244811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb4108130 0x7f1cb41079b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.176+0000 7f1cb89a0640 1 -- 172.21.15.5:0/4115244811 >> 172.21.15.5:0/4115244811 conn(0x7f1cb4101d10 msgr2=0x7f1cb4102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.176+0000 7f1cb89a0640 1 -- 172.21.15.5:0/4115244811 shutdown_connections 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:39.176+0000 7f1cb89a0640 1 -- 172.21.15.5:0/4115244811 wait complete. 2024-09-15T21:59:39.796 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-09-15T21:59:39.797 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 9... 2024-09-15T21:59:40.034 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:39 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:39.887+0000 7f0b55a08040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T21:59:40.034 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:39 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:39.961+0000 7f0b55a08040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T21:59:40.993 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:40 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:40.719+0000 7f0b55a08040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T21:59:40.993 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:40 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:40.840+0000 7f0b55a08040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T21:59:41.283 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:40 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:40.990+0000 7f0b55a08040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:59:41.283 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:41.054+0000 7f0b55a08040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T21:59:41.283 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:41.209+0000 7f0b55a08040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T21:59:41.283 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:41.280+0000 7f0b55a08040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T21:59:41.579 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:41.343+0000 7f0b55a08040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T21:59:41.579 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:41.509+0000 7f0b55a08040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T21:59:41.580 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:41.577+0000 7f0b55a08040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T21:59:41.856 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:41.646+0000 7f0b55a08040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T21:59:41.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:41.733+0000 7f0b55a08040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T21:59:41.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:41.795+0000 7f0b55a08040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T21:59:42.356 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:41.856+0000 7f0b55a08040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T21:59:42.939 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:42.682+0000 7f0b55a08040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T21:59:42.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: Active manager daemon x restarted 2024-09-15T21:59:42.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: Activating manager daemon x 2024-09-15T21:59:42.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: osdmap e3: 0 total, 0 up, 0 in 2024-09-15T21:59:42.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: mgrmap e10: x(active, starting, since 0.00314164s) 2024-09-15T21:59:42.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T21:59:42.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T21:59:42.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T21:59:42.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T21:59:42.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T21:59:42.941 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: Manager daemon x is now available 2024-09-15T21:59:42.941 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T21:59:42.941 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T21:59:42.941 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T21:59:44.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-15T21:59:44.625 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.200+0000 7f349680e640 1 Processor -- start 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.200+0000 7f349680e640 1 -- start start 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.200+0000 7f349680e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349010a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.200+0000 7f349680e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f349007be30 con 0x7f34901080f0 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.200+0000 7f348ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349010a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.201+0000 7f348ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349010a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53658/0 (socket says 172.21.15.5:53658) 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.201+0000 7f348ffff640 1 -- 172.21.15.5:0/1332558288 learned_addr learned my addr 172.21.15.5:0/1332558288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.201+0000 7f348ffff640 1 -- 172.21.15.5:0/1332558288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349010aa20 con 0x7f34901080f0 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.201+0000 7f348ffff640 1 --2- 172.21.15.5:0/1332558288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349010a4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3480009820 tx=0x7f348002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c119311135673879 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:44.626 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.202+0000 7f348effd640 1 -- 172.21.15.5:0/1332558288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3480037070 con 0x7f34901080f0 2024-09-15T21:59:44.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.202+0000 7f348effd640 1 -- 172.21.15.5:0/1332558288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3480032920 con 0x7f34901080f0 2024-09-15T21:59:44.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.202+0000 7f348effd640 1 -- 172.21.15.5:0/1332558288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3480030810 con 0x7f34901080f0 2024-09-15T21:59:44.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.203+0000 7f349680e640 1 -- 172.21.15.5:0/1332558288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 msgr2=0x7f349010a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:44.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.203+0000 7f349680e640 1 --2- 172.21.15.5:0/1332558288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349010a4e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3480009820 tx=0x7f348002a270 comp rx=0 tx=0).stop 2024-09-15T21:59:44.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.203+0000 7f349680e640 1 -- 172.21.15.5:0/1332558288 shutdown_connections 2024-09-15T21:59:44.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.203+0000 7f349680e640 1 --2- 172.21.15.5:0/1332558288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349010a4e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:44.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.203+0000 7f349680e640 1 -- 172.21.15.5:0/1332558288 >> 172.21.15.5:0/1332558288 conn(0x7f3490101d10 msgr2=0x7f3490104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:44.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.203+0000 7f349680e640 1 -- 172.21.15.5:0/1332558288 shutdown_connections 2024-09-15T21:59:44.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.203+0000 7f349680e640 1 -- 172.21.15.5:0/1332558288 wait complete. 2024-09-15T21:59:44.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.203+0000 7f349680e640 1 Processor -- start 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.203+0000 7f349680e640 1 -- start start 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.204+0000 7f349680e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.204+0000 7f349680e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f349010bb00 con 0x7f34901080f0 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.204+0000 7f348ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.204+0000 7f348ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349019b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53666/0 (socket says 172.21.15.5:53666) 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.204+0000 7f348ffff640 1 -- 172.21.15.5:0/3800778211 learned_addr learned my addr 172.21.15.5:0/3800778211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.204+0000 7f348ffff640 1 -- 172.21.15.5:0/3800778211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f349019bbb0 con 0x7f34901080f0 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.204+0000 7f348ffff640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349019b670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3480030380 tx=0x7f3480032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.205+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3480037040 con 0x7f34901080f0 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.205+0000 7f349680e640 1 -- 172.21.15.5:0/3800778211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f349019be40 con 0x7f34901080f0 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.205+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3480040070 con 0x7f34901080f0 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.205+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f348003a7b0 con 0x7f34901080f0 2024-09-15T21:59:44.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.205+0000 7f349680e640 1 -- 172.21.15.5:0/3800778211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f349019eb30 con 0x7f34901080f0 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.206+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f348003a9e0 con 0x7f34901080f0 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.206+0000 7f348d7fa640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 0x7f347803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.206+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 --> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f347803be30 con 0x7f3478039260 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.206+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f348006d960 con 0x7f34901080f0 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.206+0000 7f348f7fe640 1 -- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 msgr2=0x7f347803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3785442724 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.206+0000 7f348f7fe640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 0x7f347803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.407+0000 7f348f7fe640 1 -- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 msgr2=0x7f347803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3785442724 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.407+0000 7f348f7fe640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 0x7f347803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.807+0000 7f348f7fe640 1 -- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 msgr2=0x7f347803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3785442724 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:40.807+0000 7f348f7fe640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 0x7f347803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:41.608+0000 7f348f7fe640 1 -- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 msgr2=0x7f347803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3785442724 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:41.608+0000 7f348f7fe640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 0x7f347803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:59:44.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:42.685+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f348006c410 con 0x7f34901080f0 2024-09-15T21:59:44.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:42.685+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 msgr2=0x7f347803b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:59:44.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:42.685+0000 7f348d7fa640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 0x7f347803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:44.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.687+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f3480045460 con 0x7f34901080f0 2024-09-15T21:59:44.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.687+0000 7f348d7fa640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f347803cd20 0x7f347803f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:44.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.687+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f347803be30 con 0x7f347803cd20 2024-09-15T21:59:44.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.687+0000 7f348f7fe640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f347803cd20 0x7f347803f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:44.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.688+0000 7f348f7fe640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f347803cd20 0x7f347803f110 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3484003980 tx=0x7f34840092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:44.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.689+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f347803be30 con 0x7f347803cd20 2024-09-15T21:59:44.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.696+0000 7f349680e640 1 -- 172.21.15.5:0/3800778211 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3490108550 con 0x7f347803cd20 2024-09-15T21:59:44.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.696+0000 7f348d7fa640 1 -- 172.21.15.5:0/3800778211 <== mgr.14150 v2:172.21.15.5:6800/801026306 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3490108550 con 0x7f347803cd20 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.697+0000 7f349680e640 1 -- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f347803cd20 msgr2=0x7f347803f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.697+0000 7f349680e640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f347803cd20 0x7f347803f110 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3484003980 tx=0x7f34840092b0 comp rx=0 tx=0).stop 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.697+0000 7f349680e640 1 -- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 msgr2=0x7f349019b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.697+0000 7f349680e640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349019b670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3480030380 tx=0x7f3480032670 comp rx=0 tx=0).stop 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.697+0000 7f349680e640 1 -- 172.21.15.5:0/3800778211 shutdown_connections 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.697+0000 7f349680e640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/3785442724,v1:172.21.15.5:6801/3785442724] conn(0x7f3478039260 0x7f347803b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.697+0000 7f349680e640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f347803cd20 0x7f347803f110 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.697+0000 7f349680e640 1 --2- 172.21.15.5:0/3800778211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34901080f0 0x7f349019b670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.697+0000 7f349680e640 1 -- 172.21.15.5:0/3800778211 >> 172.21.15.5:0/3800778211 conn(0x7f3490101d10 msgr2=0x7f3490102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.697+0000 7f349680e640 1 -- 172.21.15.5:0/3800778211 shutdown_connections 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:43.697+0000 7f349680e640 1 -- 172.21.15.5:0/3800778211 wait complete. 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 9 is available 2024-09-15T21:59:44.631 INFO:teuthology.orchestra.run.smithi005.stdout:Generating a dashboard self-signed certificate... 2024-09-15T21:59:44.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:44 smithi005 ceph-mon[30944]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-15T21:59:44.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:44 smithi005 ceph-mon[30944]: mgrmap e11: x(active, since 1.00622s) 2024-09-15T21:59:44.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:44 smithi005 ceph-mon[30944]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-15T21:59:44.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:45.253 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:45.051+0000 7f0abb4d1640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:21:59:45] ENGINE Error in HTTPServer.serve 2024-09-15T21:59:45.253 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: Traceback (most recent call last): 2024-09-15T21:59:45.253 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T21:59:45.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self._connections.run(self.expiration_interval) 2024-09-15T21:59:45.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T21:59:45.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self._run(expiration_interval) 2024-09-15T21:59:45.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T21:59:45.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T21:59:45.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T21:59:45.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T21:59:45.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T21:59:45.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: s = self.context.wrap_socket( 2024-09-15T21:59:45.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T21:59:45.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: return self.sslsocket_class._create( 2024-09-15T21:59:45.255 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T21:59:45.255 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self.do_handshake() 2024-09-15T21:59:45.255 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T21:59:45.255 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self._sslobj.do_handshake() 2024-09-15T21:59:45.255 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T21:59:45.255 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 21:59:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T21:59:45.948 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.056+0000 7f5b029d9640 1 Processor -- start 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.057+0000 7f5b029d9640 1 -- start start 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.057+0000 7f5b029d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc10a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.057+0000 7f5b029d9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc07bdf0 con 0x7f5afc108130 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.057+0000 7f5afbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc10a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.057+0000 7f5afbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc10a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50596/0 (socket says 172.21.15.5:50596) 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.057+0000 7f5afbfff640 1 -- 172.21.15.5:0/2252605016 learned_addr learned my addr 172.21.15.5:0/2252605016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.058+0000 7f5afbfff640 1 -- 172.21.15.5:0/2252605016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc10aa60 con 0x7f5afc108130 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.058+0000 7f5afbfff640 1 --2- 172.21.15.5:0/2252605016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc10a520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009820 tx=0x7f5aec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8965bfc6c88e5b5b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.058+0000 7f5afb7fe640 1 -- 172.21.15.5:0/2252605016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec037070 con 0x7f5afc108130 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.058+0000 7f5afb7fe640 1 -- 172.21.15.5:0/2252605016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aec032920 con 0x7f5afc108130 2024-09-15T21:59:45.949 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.058+0000 7f5afb7fe640 1 -- 172.21.15.5:0/2252605016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec030810 con 0x7f5afc108130 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.059+0000 7f5b029d9640 1 -- 172.21.15.5:0/2252605016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 msgr2=0x7f5afc10a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.059+0000 7f5b029d9640 1 --2- 172.21.15.5:0/2252605016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc10a520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009820 tx=0x7f5aec02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.059+0000 7f5b029d9640 1 -- 172.21.15.5:0/2252605016 shutdown_connections 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.059+0000 7f5b029d9640 1 --2- 172.21.15.5:0/2252605016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc10a520 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.059+0000 7f5b029d9640 1 -- 172.21.15.5:0/2252605016 >> 172.21.15.5:0/2252605016 conn(0x7f5afc101d10 msgr2=0x7f5afc104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.059+0000 7f5b029d9640 1 -- 172.21.15.5:0/2252605016 shutdown_connections 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.059+0000 7f5b029d9640 1 -- 172.21.15.5:0/2252605016 wait complete. 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.060+0000 7f5b029d9640 1 Processor -- start 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.060+0000 7f5b029d9640 1 -- start start 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.060+0000 7f5b029d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.060+0000 7f5b029d9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc10bae0 con 0x7f5afc108130 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.060+0000 7f5afbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.060+0000 7f5afbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc19b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50608/0 (socket says 172.21.15.5:50608) 2024-09-15T21:59:45.950 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.060+0000 7f5afbfff640 1 -- 172.21.15.5:0/2918851290 learned_addr learned my addr 172.21.15.5:0/2918851290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.060+0000 7f5afbfff640 1 -- 172.21.15.5:0/2918851290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc19bba0 con 0x7f5afc108130 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.060+0000 7f5afbfff640 1 --2- 172.21.15.5:0/2918851290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc19b660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5aec030380 tx=0x7f5aec032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.061+0000 7f5af9ffb640 1 -- 172.21.15.5:0/2918851290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec037040 con 0x7f5afc108130 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.061+0000 7f5af9ffb640 1 -- 172.21.15.5:0/2918851290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aec040070 con 0x7f5afc108130 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.061+0000 7f5b029d9640 1 -- 172.21.15.5:0/2918851290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5afc19be30 con 0x7f5afc108130 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.061+0000 7f5af9ffb640 1 -- 172.21.15.5:0/2918851290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5aec03a7b0 con 0x7f5afc108130 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.061+0000 7f5b029d9640 1 -- 172.21.15.5:0/2918851290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5afc19eb20 con 0x7f5afc108130 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.062+0000 7f5af9ffb640 1 -- 172.21.15.5:0/2918851290 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f5aec03b420 con 0x7f5afc108130 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.062+0000 7f5af9ffb640 1 --2- 172.21.15.5:0/2918851290 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5ae8039140 0x7f5ae803b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.062+0000 7f5b029d9640 1 -- 172.21.15.5:0/2918851290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5afc108590 con 0x7f5afc108130 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.062+0000 7f5af9ffb640 1 -- 172.21.15.5:0/2918851290 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5aec06c7a0 con 0x7f5afc108130 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.062+0000 7f5af3fff640 1 --2- 172.21.15.5:0/2918851290 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5ae8039140 0x7f5ae803b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.063+0000 7f5af3fff640 1 --2- 172.21.15.5:0/2918851290 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5ae8039140 0x7f5ae803b600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098c0 tx=0x7f5ae4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.067+0000 7f5af9ffb640 1 -- 172.21.15.5:0/2918851290 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5aec043360 con 0x7f5afc108130 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.069+0000 7f5af9ffb640 1 -- 172.21.15.5:0/2918851290 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f5aec03bc00 con 0x7f5afc108130 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.250+0000 7f5b029d9640 1 -- 172.21.15.5:0/2918851290 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f5afc102f80 con 0x7f5ae8039140 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.365+0000 7f5af9ffb640 1 -- 172.21.15.5:0/2918851290 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f5afc102f80 con 0x7f5ae8039140 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.367+0000 7f5b029d9640 1 -- 172.21.15.5:0/2918851290 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5ae8039140 msgr2=0x7f5ae803b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.367+0000 7f5b029d9640 1 --2- 172.21.15.5:0/2918851290 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5ae8039140 0x7f5ae803b600 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ae40098c0 tx=0x7f5ae4006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.367+0000 7f5b029d9640 1 -- 172.21.15.5:0/2918851290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 msgr2=0x7f5afc19b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.367+0000 7f5b029d9640 1 --2- 172.21.15.5:0/2918851290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc19b660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5aec030380 tx=0x7f5aec032670 comp rx=0 tx=0).stop 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.367+0000 7f5b029d9640 1 -- 172.21.15.5:0/2918851290 shutdown_connections 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.367+0000 7f5b029d9640 1 --2- 172.21.15.5:0/2918851290 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5ae8039140 0x7f5ae803b600 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.367+0000 7f5b029d9640 1 --2- 172.21.15.5:0/2918851290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5afc108130 0x7f5afc19b660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.367+0000 7f5b029d9640 1 -- 172.21.15.5:0/2918851290 >> 172.21.15.5:0/2918851290 conn(0x7f5afc101d10 msgr2=0x7f5afc102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:45.952 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.367+0000 7f5b029d9640 1 -- 172.21.15.5:0/2918851290 shutdown_connections 2024-09-15T21:59:45.953 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:45.367+0000 7f5b029d9640 1 -- 172.21.15.5:0/2918851290 wait complete. 2024-09-15T21:59:45.953 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial admin user... 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:46 smithi005 ceph-mon[30944]: [15/Sep/2024:21:59:44] ENGINE Bus STARTING 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:46 smithi005 ceph-mon[30944]: [15/Sep/2024:21:59:44] ENGINE Serving on http://172.21.15.5:8765 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:46 smithi005 ceph-mon[30944]: [15/Sep/2024:21:59:45] ENGINE Serving on https://172.21.15.5:7150 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:46 smithi005 ceph-mon[30944]: [15/Sep/2024:21:59:45] ENGINE Bus STARTED 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:46 smithi005 ceph-mon[30944]: [15/Sep/2024:21:59:45] ENGINE Error in HTTPServer.serve 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T21:59:46.343 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:46 smithi005 ceph-mon[30944]: mgrmap e12: x(active, since 2s) 2024-09-15T21:59:46.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:46.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:46 smithi005 ceph-mon[30944]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T21:59:46.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:46.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:47.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:47 smithi005 ceph-mon[30944]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T21:59:47.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:47 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:48.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$mMLlIym5TNbMpa8hNOHk.uaipolEM7Rt8ltxwnLq6Vjp2i2r28qZq", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726437586, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-15T21:59:48.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.375+0000 7f2a77c2d640 1 Processor -- start 2024-09-15T21:59:48.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.375+0000 7f2a77c2d640 1 -- start start 2024-09-15T21:59:48.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.375+0000 7f2a77c2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a70107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:48.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.375+0000 7f2a77c2d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a701080b0 con 0x7f2a701076e0 2024-09-15T21:59:48.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.375+0000 7f2a759c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a70107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:48.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.375+0000 7f2a759c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a70107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50620/0 (socket says 172.21.15.5:50620) 2024-09-15T21:59:48.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.375+0000 7f2a759c9640 1 -- 172.21.15.5:0/217760648 learned_addr learned my addr 172.21.15.5:0/217760648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:48.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.376+0000 7f2a759c9640 1 -- 172.21.15.5:0/217760648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a70108890 con 0x7f2a701076e0 2024-09-15T21:59:48.112 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.376+0000 7f2a759c9640 1 --2- 172.21.15.5:0/217760648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a70107ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009820 tx=0x7f2a6c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523ab641503d8e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.377+0000 7f2a749c7640 1 -- 172.21.15.5:0/217760648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c037070 con 0x7f2a701076e0 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.377+0000 7f2a749c7640 1 -- 172.21.15.5:0/217760648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6c032920 con 0x7f2a701076e0 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.377+0000 7f2a749c7640 1 -- 172.21.15.5:0/217760648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c030810 con 0x7f2a701076e0 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.377+0000 7f2a77c2d640 1 -- 172.21.15.5:0/217760648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 msgr2=0x7f2a70107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.377+0000 7f2a77c2d640 1 --2- 172.21.15.5:0/217760648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a70107ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009820 tx=0x7f2a6c02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.377+0000 7f2a77c2d640 1 -- 172.21.15.5:0/217760648 shutdown_connections 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.377+0000 7f2a77c2d640 1 --2- 172.21.15.5:0/217760648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a70107ae0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.377+0000 7f2a77c2d640 1 -- 172.21.15.5:0/217760648 >> 172.21.15.5:0/217760648 conn(0x7f2a70102e90 msgr2=0x7f2a701052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.377+0000 7f2a77c2d640 1 -- 172.21.15.5:0/217760648 shutdown_connections 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.377+0000 7f2a77c2d640 1 -- 172.21.15.5:0/217760648 wait complete. 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.378+0000 7f2a77c2d640 1 Processor -- start 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.378+0000 7f2a77c2d640 1 -- start start 2024-09-15T21:59:48.113 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.378+0000 7f2a77c2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a7007e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.378+0000 7f2a77c2d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7010bc40 con 0x7f2a701076e0 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.378+0000 7f2a759c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a7007e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.378+0000 7f2a759c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a7007e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50622/0 (socket says 172.21.15.5:50622) 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.378+0000 7f2a759c9640 1 -- 172.21.15.5:0/2275918823 learned_addr learned my addr 172.21.15.5:0/2275918823 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.379+0000 7f2a759c9640 1 -- 172.21.15.5:0/2275918823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7007e8c0 con 0x7f2a701076e0 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.379+0000 7f2a759c9640 1 --2- 172.21.15.5:0/2275918823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a7007e380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c030560 tx=0x7f2a6c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.379+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2275918823 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c037040 con 0x7f2a701076e0 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.379+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2275918823 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a6c040070 con 0x7f2a701076e0 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.379+0000 7f2a77c2d640 1 -- 172.21.15.5:0/2275918823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7007cab0 con 0x7f2a701076e0 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.379+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2275918823 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c03a740 con 0x7f2a701076e0 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.379+0000 7f2a77c2d640 1 -- 172.21.15.5:0/2275918823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7007cff0 con 0x7f2a701076e0 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.380+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2275918823 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f2a6c03b420 con 0x7f2a701076e0 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.380+0000 7f2a77c2d640 1 -- 172.21.15.5:0/2275918823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7010bdc0 con 0x7f2a701076e0 2024-09-15T21:59:48.114 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.380+0000 7f2a66ffd640 1 --2- 172.21.15.5:0/2275918823 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2a5c039200 0x7f2a5c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.380+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2275918823 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2a6c06c990 con 0x7f2a701076e0 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.380+0000 7f2a751c8640 1 --2- 172.21.15.5:0/2275918823 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2a5c039200 0x7f2a5c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.381+0000 7f2a751c8640 1 --2- 172.21.15.5:0/2275918823 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2a5c039200 0x7f2a5c03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2a600098c0 tx=0x7f2a60006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.385+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2275918823 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a6c04a200 con 0x7f2a701076e0 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.589+0000 7f2a77c2d640 1 -- 172.21.15.5:0/2275918823 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f2a70107b60 con 0x7f2a5c039200 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.838+0000 7f2a66ffd640 1 -- 172.21.15.5:0/2275918823 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f2a70107b60 con 0x7f2a5c039200 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.840+0000 7f2a77c2d640 1 -- 172.21.15.5:0/2275918823 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2a5c039200 msgr2=0x7f2a5c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.840+0000 7f2a77c2d640 1 --2- 172.21.15.5:0/2275918823 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2a5c039200 0x7f2a5c03b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2a600098c0 tx=0x7f2a60006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.840+0000 7f2a77c2d640 1 -- 172.21.15.5:0/2275918823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 msgr2=0x7f2a7007e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.840+0000 7f2a77c2d640 1 --2- 172.21.15.5:0/2275918823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a7007e380 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c030560 tx=0x7f2a6c032670 comp rx=0 tx=0).stop 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.840+0000 7f2a77c2d640 1 -- 172.21.15.5:0/2275918823 shutdown_connections 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.840+0000 7f2a77c2d640 1 --2- 172.21.15.5:0/2275918823 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2a5c039200 0x7f2a5c03b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.840+0000 7f2a77c2d640 1 --2- 172.21.15.5:0/2275918823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a701076e0 0x7f2a7007e380 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.840+0000 7f2a77c2d640 1 -- 172.21.15.5:0/2275918823 >> 172.21.15.5:0/2275918823 conn(0x7f2a70102e90 msgr2=0x7f2a70103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:48.115 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.840+0000 7f2a77c2d640 1 -- 172.21.15.5:0/2275918823 shutdown_connections 2024-09-15T21:59:48.116 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:46.840+0000 7f2a77c2d640 1 -- 172.21.15.5:0/2275918823 wait complete. 2024-09-15T21:59:48.116 INFO:teuthology.orchestra.run.smithi005.stdout:Fetching dashboard port number... 2024-09-15T21:59:48.958 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:48 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1637974583' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-15T21:59:49.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 8443 2024-09-15T21:59:49.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.485+0000 7f98d0dc7640 1 Processor -- start 2024-09-15T21:59:49.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.486+0000 7f98d0dc7640 1 -- start start 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.486+0000 7f98d0dc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc10a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.486+0000 7f98d0dc7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc07c670 con 0x7f98cc108150 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.486+0000 7f98ca59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc10a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.486+0000 7f98ca59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc10a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50636/0 (socket says 172.21.15.5:50636) 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.486+0000 7f98ca59c640 1 -- 172.21.15.5:0/1899548235 learned_addr learned my addr 172.21.15.5:0/1899548235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.486+0000 7f98ca59c640 1 -- 172.21.15.5:0/1899548235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc10aa80 con 0x7f98cc108150 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.487+0000 7f98ca59c640 1 --2- 172.21.15.5:0/1899548235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc10a540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009820 tx=0x7f98bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1642c8ed9c1ff5a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.487+0000 7f98c959a640 1 -- 172.21.15.5:0/1899548235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98bc037070 con 0x7f98cc108150 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.487+0000 7f98c959a640 1 -- 172.21.15.5:0/1899548235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98bc032920 con 0x7f98cc108150 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.487+0000 7f98c959a640 1 -- 172.21.15.5:0/1899548235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98bc030810 con 0x7f98cc108150 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.488+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1899548235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 msgr2=0x7f98cc10a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.488+0000 7f98d0dc7640 1 --2- 172.21.15.5:0/1899548235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc10a540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009820 tx=0x7f98bc02a270 comp rx=0 tx=0).stop 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.488+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1899548235 shutdown_connections 2024-09-15T21:59:49.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.488+0000 7f98d0dc7640 1 --2- 172.21.15.5:0/1899548235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc10a540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.488+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1899548235 >> 172.21.15.5:0/1899548235 conn(0x7f98cc101db0 msgr2=0x7f98cc1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.488+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1899548235 shutdown_connections 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.488+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1899548235 wait complete. 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.489+0000 7f98d0dc7640 1 Processor -- start 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.489+0000 7f98d0dc7640 1 -- start start 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.489+0000 7f98d0dc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.489+0000 7f98d0dc7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc10bb00 con 0x7f98cc108150 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.489+0000 7f98ca59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.489+0000 7f98ca59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc19b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50646/0 (socket says 172.21.15.5:50646) 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.489+0000 7f98ca59c640 1 -- 172.21.15.5:0/1637974583 learned_addr learned my addr 172.21.15.5:0/1637974583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.489+0000 7f98ca59c640 1 -- 172.21.15.5:0/1637974583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc19bbb0 con 0x7f98cc108150 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.489+0000 7f98ca59c640 1 --2- 172.21.15.5:0/1637974583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc19b670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0097f0 tx=0x7f98bc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:49.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.490+0000 7f98bb7fe640 1 -- 172.21.15.5:0/1637974583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98bc037040 con 0x7f98cc108150 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.490+0000 7f98bb7fe640 1 -- 172.21.15.5:0/1637974583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98bc030cc0 con 0x7f98cc108150 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.490+0000 7f98bb7fe640 1 -- 172.21.15.5:0/1637974583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98bc03a6b0 con 0x7f98cc108150 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.490+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1637974583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc19be40 con 0x7f98cc108150 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.490+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1637974583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc19eb30 con 0x7f98cc108150 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.491+0000 7f98bb7fe640 1 -- 172.21.15.5:0/1637974583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f98bc03baf0 con 0x7f98cc108150 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.491+0000 7f98bb7fe640 1 --2- 172.21.15.5:0/1637974583 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f98b4039260 0x7f98b403b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.491+0000 7f98bb7fe640 1 -- 172.21.15.5:0/1637974583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f98bc06c8b0 con 0x7f98cc108150 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.491+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1637974583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98cc1085b0 con 0x7f98cc108150 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.491+0000 7f98c9d9b640 1 --2- 172.21.15.5:0/1637974583 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f98b4039260 0x7f98b403b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.491+0000 7f98c9d9b640 1 --2- 172.21.15.5:0/1637974583 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f98b4039260 0x7f98b403b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f98c00098c0 tx=0x7f98c0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.496+0000 7f98bb7fe640 1 -- 172.21.15.5:0/1637974583 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98bc042e10 con 0x7f98cc108150 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.671+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1637974583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f98cc19eec0 con 0x7f98cc108150 2024-09-15T21:59:49.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.671+0000 7f98bb7fe640 1 -- 172.21.15.5:0/1637974583 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f98bc03a850 con 0x7f98cc108150 2024-09-15T21:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.673+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1637974583 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f98b4039260 msgr2=0x7f98b403b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.673+0000 7f98d0dc7640 1 --2- 172.21.15.5:0/1637974583 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f98b4039260 0x7f98b403b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f98c00098c0 tx=0x7f98c0006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.673+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1637974583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 msgr2=0x7f98cc19b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.673+0000 7f98d0dc7640 1 --2- 172.21.15.5:0/1637974583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc19b670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f98bc0097f0 tx=0x7f98bc0327b0 comp rx=0 tx=0).stop 2024-09-15T21:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.674+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1637974583 shutdown_connections 2024-09-15T21:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.674+0000 7f98d0dc7640 1 --2- 172.21.15.5:0/1637974583 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f98b4039260 0x7f98b403b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.674+0000 7f98d0dc7640 1 --2- 172.21.15.5:0/1637974583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98cc108150 0x7f98cc19b670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.674+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1637974583 >> 172.21.15.5:0/1637974583 conn(0x7f98cc101db0 msgr2=0x7f98cc102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.674+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1637974583 shutdown_connections 2024-09-15T21:59:49.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:48.674+0000 7f98d0dc7640 1 -- 172.21.15.5:0/1637974583 wait complete. 2024-09-15T21:59:49.235 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-15T21:59:49.235 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-15T21:59:49.243 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-15T21:59:49.243 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-15T21:59:49.243 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-15T21:59:49.243 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-15T21:59:49.245 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph Dashboard is now available at: 2024-09-15T21:59:49.245 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:59:49.245 INFO:teuthology.orchestra.run.smithi005.stdout: URL: https://smithi005.front.sepia.ceph.com:8443/ 2024-09-15T21:59:49.245 INFO:teuthology.orchestra.run.smithi005.stdout: User: admin 2024-09-15T21:59:49.246 INFO:teuthology.orchestra.run.smithi005.stdout: Password: w1o6zdnrhw 2024-09-15T21:59:49.246 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:59:49.246 INFO:teuthology.orchestra.run.smithi005.stdout:Saving cluster configuration to /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config directory 2024-09-15T21:59:50.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:49 smithi005 ceph-mon[30944]: mgrmap e13: x(active, since 6s) 2024-09-15T21:59:50.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.676+0000 7f96e389d640 1 Processor -- start 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.676+0000 7f96e389d640 1 -- start start 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.676+0000 7f96e389d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc107ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.676+0000 7f96e389d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc1080b0 con 0x7f96dc1076e0 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.677+0000 7f96e1639640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc107ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.677+0000 7f96e1639640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc107ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50650/0 (socket says 172.21.15.5:50650) 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.677+0000 7f96e1639640 1 -- 172.21.15.5:0/3878304643 learned_addr learned my addr 172.21.15.5:0/3878304643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.677+0000 7f96e1639640 1 -- 172.21.15.5:0/3878304643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc108890 con 0x7f96dc1076e0 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.678+0000 7f96e1639640 1 --2- 172.21.15.5:0/3878304643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc107ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009820 tx=0x7f96d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b192b0bf64e5563 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.678+0000 7f96dbfff640 1 -- 172.21.15.5:0/3878304643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d4037070 con 0x7f96dc1076e0 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.678+0000 7f96dbfff640 1 -- 172.21.15.5:0/3878304643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d4032920 con 0x7f96dc1076e0 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.678+0000 7f96dbfff640 1 -- 172.21.15.5:0/3878304643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d4030810 con 0x7f96dc1076e0 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.679+0000 7f96e389d640 1 -- 172.21.15.5:0/3878304643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 msgr2=0x7f96dc107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.679+0000 7f96e389d640 1 --2- 172.21.15.5:0/3878304643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc107ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009820 tx=0x7f96d402a270 comp rx=0 tx=0).stop 2024-09-15T21:59:50.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.679+0000 7f96e389d640 1 -- 172.21.15.5:0/3878304643 shutdown_connections 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.679+0000 7f96e389d640 1 --2- 172.21.15.5:0/3878304643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc107ae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.679+0000 7f96e389d640 1 -- 172.21.15.5:0/3878304643 >> 172.21.15.5:0/3878304643 conn(0x7f96dc102e90 msgr2=0x7f96dc1052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.679+0000 7f96e389d640 1 -- 172.21.15.5:0/3878304643 shutdown_connections 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.679+0000 7f96e389d640 1 -- 172.21.15.5:0/3878304643 wait complete. 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.679+0000 7f96e389d640 1 Processor -- start 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.680+0000 7f96e389d640 1 -- start start 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.680+0000 7f96e389d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.680+0000 7f96e389d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96dc10bc40 con 0x7f96dc1076e0 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.680+0000 7f96e1639640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.680+0000 7f96e1639640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc19d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50660/0 (socket says 172.21.15.5:50660) 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.680+0000 7f96e1639640 1 -- 172.21.15.5:0/2791010929 learned_addr learned my addr 172.21.15.5:0/2791010929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.680+0000 7f96e1639640 1 -- 172.21.15.5:0/2791010929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96dc19de30 con 0x7f96dc1076e0 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.680+0000 7f96e1639640 1 --2- 172.21.15.5:0/2791010929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc19d8f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96d40097f0 tx=0x7f96d40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:50.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.680+0000 7f96da7fc640 1 -- 172.21.15.5:0/2791010929 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d4037040 con 0x7f96dc1076e0 2024-09-15T21:59:50.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.681+0000 7f96da7fc640 1 -- 172.21.15.5:0/2791010929 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96d4030cc0 con 0x7f96dc1076e0 2024-09-15T21:59:50.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.681+0000 7f96da7fc640 1 -- 172.21.15.5:0/2791010929 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d403a640 con 0x7f96dc1076e0 2024-09-15T21:59:50.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.681+0000 7f96e389d640 1 -- 172.21.15.5:0/2791010929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96dc19e0c0 con 0x7f96dc1076e0 2024-09-15T21:59:50.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.681+0000 7f96e389d640 1 -- 172.21.15.5:0/2791010929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96dc1a0db0 con 0x7f96dc1076e0 2024-09-15T21:59:50.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.681+0000 7f96da7fc640 1 -- 172.21.15.5:0/2791010929 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f96d403b3f0 con 0x7f96dc1076e0 2024-09-15T21:59:50.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.682+0000 7f96e389d640 1 -- 172.21.15.5:0/2791010929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96dc10bdc0 con 0x7f96dc1076e0 2024-09-15T21:59:50.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.682+0000 7f96da7fc640 1 --2- 172.21.15.5:0/2791010929 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f96c4039300 0x7f96c403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:50.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.682+0000 7f96e0e38640 1 --2- 172.21.15.5:0/2791010929 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f96c4039300 0x7f96c403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.682+0000 7f96da7fc640 1 -- 172.21.15.5:0/2791010929 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f96d406c9a0 con 0x7f96dc1076e0 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.682+0000 7f96e0e38640 1 --2- 172.21.15.5:0/2791010929 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f96c4039300 0x7f96c403b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f96d00098c0 tx=0x7f96d0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.687+0000 7f96da7fc640 1 -- 172.21.15.5:0/2791010929 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96d4041890 con 0x7f96dc1076e0 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.954+0000 7f96e389d640 1 -- 172.21.15.5:0/2791010929 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f96dc1a0f80 con 0x7f96dc1076e0 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.956+0000 7f96da7fc640 1 -- 172.21.15.5:0/2791010929 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f96d40371e0 con 0x7f96dc1076e0 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.958+0000 7f96e389d640 1 -- 172.21.15.5:0/2791010929 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f96c4039300 msgr2=0x7f96c403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.958+0000 7f96e389d640 1 --2- 172.21.15.5:0/2791010929 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f96c4039300 0x7f96c403b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f96d00098c0 tx=0x7f96d0006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.958+0000 7f96e389d640 1 -- 172.21.15.5:0/2791010929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 msgr2=0x7f96dc19d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.958+0000 7f96e389d640 1 --2- 172.21.15.5:0/2791010929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc19d8f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f96d40097f0 tx=0x7f96d40327b0 comp rx=0 tx=0).stop 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.958+0000 7f96e389d640 1 -- 172.21.15.5:0/2791010929 shutdown_connections 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.958+0000 7f96e389d640 1 --2- 172.21.15.5:0/2791010929 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f96c4039300 0x7f96c403b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:50.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.958+0000 7f96e389d640 1 --2- 172.21.15.5:0/2791010929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96dc1076e0 0x7f96dc19d8f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.958+0000 7f96e389d640 1 -- 172.21.15.5:0/2791010929 >> 172.21.15.5:0/2791010929 conn(0x7f96dc102e90 msgr2=0x7f96dc1039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.959+0000 7f96e389d640 1 -- 172.21.15.5:0/2791010929 shutdown_connections 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-15T21:59:49.959+0000 7f96e389d640 1 -- 172.21.15.5:0/2791010929 wait complete. 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout:Or, if you are only running a single cluster on this host: 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: ceph telemetry on 2024-09-15T21:59:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:59:50.571 INFO:teuthology.orchestra.run.smithi005.stdout:For more information see: 2024-09-15T21:59:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:59:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-15T21:59:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T21:59:50.571 INFO:teuthology.orchestra.run.smithi005.stdout:Bootstrap complete. 2024-09-15T21:59:50.610 INFO:tasks.cephadm:Fetching config... 2024-09-15T21:59:50.610 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T21:59:50.610 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-15T21:59:50.628 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-15T21:59:50.628 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T21:59:50.628 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-15T21:59:50.684 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-15T21:59:50.684 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T21:59:50.684 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-15T21:59:50.750 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-15T21:59:50.750 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T21:59:50.751 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-15T21:59:50.806 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-15T21:59:50.806 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCbXIQVxatulaq6+BiIjbdCEKZUtHM1QP5Q2CegsUvrPFP5HKmxTmWJn4Dlab3IWDKrNM+j/H6mx1dD832gLkKb9m/PcftCgNfJIPEALrZSgQbF/hpFF9K/eXTzpBzYU7Rv0eCLVtztVapKL444GHVyh/DIeq1ZARPszP/wKgelylEdqveGkpYvcOAwdlcMuVo8oOCvcOv7jR0PX6M/eIiWOqjWsfZ/XHm6qInPro8ZDFxQ7sxZiaKiPFWdLVoOT1Sgj06bjOnL42REokOnL/m1JAXb4t0LUKLulMpthqGkCRSmydbbXmkP6107cIJpq0Lp1Ir6mZYQe+tGfObvrgSUyP4dUv5cpoRKpp20072UcZ2edw/5XjOljspa2vm7RkqIlvxrQgnb5oqJDKdQUkR4zhr3BiYbi7ZKdu4Z9Sj3zX1DAOPIJ/8fb/ntwbbim5aAddjFad76fpms8D+N6juYKnmda4V1bzSYiUKtVwTCjBWuJAUcsLzEyel7Hj2/q0= ceph-962891f2-73ad-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T21:59:50.895 INFO:teuthology.orchestra.run.smithi005.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCbXIQVxatulaq6+BiIjbdCEKZUtHM1QP5Q2CegsUvrPFP5HKmxTmWJn4Dlab3IWDKrNM+j/H6mx1dD832gLkKb9m/PcftCgNfJIPEALrZSgQbF/hpFF9K/eXTzpBzYU7Rv0eCLVtztVapKL444GHVyh/DIeq1ZARPszP/wKgelylEdqveGkpYvcOAwdlcMuVo8oOCvcOv7jR0PX6M/eIiWOqjWsfZ/XHm6qInPro8ZDFxQ7sxZiaKiPFWdLVoOT1Sgj06bjOnL42REokOnL/m1JAXb4t0LUKLulMpthqGkCRSmydbbXmkP6107cIJpq0Lp1Ir6mZYQe+tGfObvrgSUyP4dUv5cpoRKpp20072UcZ2edw/5XjOljspa2vm7RkqIlvxrQgnb5oqJDKdQUkR4zhr3BiYbi7ZKdu4Z9Sj3zX1DAOPIJ/8fb/ntwbbim5aAddjFad76fpms8D+N6juYKnmda4V1bzSYiUKtVwTCjBWuJAUcsLzEyel7Hj2/q0= ceph-962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:59:50.907 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCbXIQVxatulaq6+BiIjbdCEKZUtHM1QP5Q2CegsUvrPFP5HKmxTmWJn4Dlab3IWDKrNM+j/H6mx1dD832gLkKb9m/PcftCgNfJIPEALrZSgQbF/hpFF9K/eXTzpBzYU7Rv0eCLVtztVapKL444GHVyh/DIeq1ZARPszP/wKgelylEdqveGkpYvcOAwdlcMuVo8oOCvcOv7jR0PX6M/eIiWOqjWsfZ/XHm6qInPro8ZDFxQ7sxZiaKiPFWdLVoOT1Sgj06bjOnL42REokOnL/m1JAXb4t0LUKLulMpthqGkCRSmydbbXmkP6107cIJpq0Lp1Ir6mZYQe+tGfObvrgSUyP4dUv5cpoRKpp20072UcZ2edw/5XjOljspa2vm7RkqIlvxrQgnb5oqJDKdQUkR4zhr3BiYbi7ZKdu4Z9Sj3zX1DAOPIJ/8fb/ntwbbim5aAddjFad76fpms8D+N6juYKnmda4V1bzSYiUKtVwTCjBWuJAUcsLzEyel7Hj2/q0= ceph-962891f2-73ad-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T21:59:50.949 INFO:teuthology.orchestra.run.smithi099.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCbXIQVxatulaq6+BiIjbdCEKZUtHM1QP5Q2CegsUvrPFP5HKmxTmWJn4Dlab3IWDKrNM+j/H6mx1dD832gLkKb9m/PcftCgNfJIPEALrZSgQbF/hpFF9K/eXTzpBzYU7Rv0eCLVtztVapKL444GHVyh/DIeq1ZARPszP/wKgelylEdqveGkpYvcOAwdlcMuVo8oOCvcOv7jR0PX6M/eIiWOqjWsfZ/XHm6qInPro8ZDFxQ7sxZiaKiPFWdLVoOT1Sgj06bjOnL42REokOnL/m1JAXb4t0LUKLulMpthqGkCRSmydbbXmkP6107cIJpq0Lp1Ir6mZYQe+tGfObvrgSUyP4dUv5cpoRKpp20072UcZ2edw/5XjOljspa2vm7RkqIlvxrQgnb5oqJDKdQUkR4zhr3BiYbi7ZKdu4Z9Sj3zX1DAOPIJ/8fb/ntwbbim5aAddjFad76fpms8D+N6juYKnmda4V1bzSYiUKtVwTCjBWuJAUcsLzEyel7Hj2/q0= ceph-962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:59:50.961 DEBUG:teuthology.orchestra.run.smithi129:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCbXIQVxatulaq6+BiIjbdCEKZUtHM1QP5Q2CegsUvrPFP5HKmxTmWJn4Dlab3IWDKrNM+j/H6mx1dD832gLkKb9m/PcftCgNfJIPEALrZSgQbF/hpFF9K/eXTzpBzYU7Rv0eCLVtztVapKL444GHVyh/DIeq1ZARPszP/wKgelylEdqveGkpYvcOAwdlcMuVo8oOCvcOv7jR0PX6M/eIiWOqjWsfZ/XHm6qInPro8ZDFxQ7sxZiaKiPFWdLVoOT1Sgj06bjOnL42REokOnL/m1JAXb4t0LUKLulMpthqGkCRSmydbbXmkP6107cIJpq0Lp1Ir6mZYQe+tGfObvrgSUyP4dUv5cpoRKpp20072UcZ2edw/5XjOljspa2vm7RkqIlvxrQgnb5oqJDKdQUkR4zhr3BiYbi7ZKdu4Z9Sj3zX1DAOPIJ/8fb/ntwbbim5aAddjFad76fpms8D+N6juYKnmda4V1bzSYiUKtVwTCjBWuJAUcsLzEyel7Hj2/q0= ceph-962891f2-73ad-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-15T21:59:51.002 INFO:teuthology.orchestra.run.smithi129.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDCbXIQVxatulaq6+BiIjbdCEKZUtHM1QP5Q2CegsUvrPFP5HKmxTmWJn4Dlab3IWDKrNM+j/H6mx1dD832gLkKb9m/PcftCgNfJIPEALrZSgQbF/hpFF9K/eXTzpBzYU7Rv0eCLVtztVapKL444GHVyh/DIeq1ZARPszP/wKgelylEdqveGkpYvcOAwdlcMuVo8oOCvcOv7jR0PX6M/eIiWOqjWsfZ/XHm6qInPro8ZDFxQ7sxZiaKiPFWdLVoOT1Sgj06bjOnL42REokOnL/m1JAXb4t0LUKLulMpthqGkCRSmydbbXmkP6107cIJpq0Lp1Ir6mZYQe+tGfObvrgSUyP4dUv5cpoRKpp20072UcZ2edw/5XjOljspa2vm7RkqIlvxrQgnb5oqJDKdQUkR4zhr3BiYbi7ZKdu4Z9Sj3zX1DAOPIJ/8fb/ntwbbim5aAddjFad76fpms8D+N6juYKnmda4V1bzSYiUKtVwTCjBWuJAUcsLzEyel7Hj2/q0= ceph-962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T21:59:51.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-15T21:59:51.269 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T21:59:51.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:50 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2791010929' entity='client.admin' 2024-09-15T21:59:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.782+0000 7f5c239af640 1 -- 172.21.15.5:0/1160877741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c1020d0 msgr2=0x7f5c1c1044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.782+0000 7f5c239af640 1 --2- 172.21.15.5:0/1160877741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c1020d0 0x7f5c1c1044c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c18009880 tx=0x7f5c1802a5d0 comp rx=0 tx=0).stop 2024-09-15T21:59:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.782+0000 7f5c239af640 1 -- 172.21.15.5:0/1160877741 shutdown_connections 2024-09-15T21:59:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.782+0000 7f5c239af640 1 --2- 172.21.15.5:0/1160877741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c1020d0 0x7f5c1c1044c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.782+0000 7f5c239af640 1 -- 172.21.15.5:0/1160877741 >> 172.21.15.5:0/1160877741 conn(0x7f5c1c0fbcd0 msgr2=0x7f5c1c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.782+0000 7f5c239af640 1 -- 172.21.15.5:0/1160877741 shutdown_connections 2024-09-15T21:59:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.782+0000 7f5c239af640 1 -- 172.21.15.5:0/1160877741 wait complete. 2024-09-15T21:59:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.783+0000 7f5c239af640 1 Processor -- start 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.783+0000 7f5c239af640 1 -- start start 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.783+0000 7f5c239af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c1020d0 0x7f5c1c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.783+0000 7f5c239af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c1c107670 con 0x7f5c1c1020d0 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.783+0000 7f5c2174b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c1020d0 0x7f5c1c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.783+0000 7f5c2174b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c1020d0 0x7f5c1c198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50672/0 (socket says 172.21.15.5:50672) 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.783+0000 7f5c2174b640 1 -- 172.21.15.5:0/3253474086 learned_addr learned my addr 172.21.15.5:0/3253474086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.783+0000 7f5c2174b640 1 -- 172.21.15.5:0/3253474086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c1c199470 con 0x7f5c1c1020d0 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.783+0000 7f5c2174b640 1 --2- 172.21.15.5:0/3253474086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c1020d0 0x7f5c1c198f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c18004f60 tx=0x7f5c180048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.784+0000 7f5c127fc640 1 -- 172.21.15.5:0/3253474086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c1803f070 con 0x7f5c1c1020d0 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.784+0000 7f5c127fc640 1 -- 172.21.15.5:0/3253474086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c18003ad0 con 0x7f5c1c1020d0 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.784+0000 7f5c127fc640 1 -- 172.21.15.5:0/3253474086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5c18035030 con 0x7f5c1c1020d0 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.784+0000 7f5c239af640 1 -- 172.21.15.5:0/3253474086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c1c199760 con 0x7f5c1c1020d0 2024-09-15T21:59:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.784+0000 7f5c239af640 1 -- 172.21.15.5:0/3253474086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c1c19c3f0 con 0x7f5c1c1020d0 2024-09-15T21:59:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.785+0000 7f5c127fc640 1 -- 172.21.15.5:0/3253474086 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f5c18042070 con 0x7f5c1c1020d0 2024-09-15T21:59:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.785+0000 7f5c239af640 1 -- 172.21.15.5:0/3253474086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c1c107df0 con 0x7f5c1c1020d0 2024-09-15T21:59:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.785+0000 7f5c127fc640 1 --2- 172.21.15.5:0/3253474086 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c08039300 0x7f5c0803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.785+0000 7f5c127fc640 1 -- 172.21.15.5:0/3253474086 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5c1806d000 con 0x7f5c1c1020d0 2024-09-15T21:59:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.785+0000 7f5c20f4a640 1 --2- 172.21.15.5:0/3253474086 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c08039300 0x7f5c0803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.785+0000 7f5c20f4a640 1 --2- 172.21.15.5:0/3253474086 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c08039300 0x7f5c0803b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0098c0 tx=0x7f5c0c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.790+0000 7f5c127fc640 1 -- 172.21.15.5:0/3253474086 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c1804a260 con 0x7f5c1c1020d0 2024-09-15T21:59:51.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.972+0000 7f5c239af640 1 -- 172.21.15.5:0/3253474086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f5c1c02d9a0 con 0x7f5c1c1020d0 2024-09-15T21:59:51.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.974+0000 7f5c127fc640 1 -- 172.21.15.5:0/3253474086 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f5c18040ca0 con 0x7f5c1c1020d0 2024-09-15T21:59:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.977+0000 7f5c239af640 1 -- 172.21.15.5:0/3253474086 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c08039300 msgr2=0x7f5c0803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.977+0000 7f5c239af640 1 --2- 172.21.15.5:0/3253474086 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c08039300 0x7f5c0803b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c0098c0 tx=0x7f5c0c006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.977+0000 7f5c239af640 1 -- 172.21.15.5:0/3253474086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c1020d0 msgr2=0x7f5c1c198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.977+0000 7f5c239af640 1 --2- 172.21.15.5:0/3253474086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c1020d0 0x7f5c1c198f30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c18004f60 tx=0x7f5c180048f0 comp rx=0 tx=0).stop 2024-09-15T21:59:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.977+0000 7f5c239af640 1 -- 172.21.15.5:0/3253474086 shutdown_connections 2024-09-15T21:59:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.977+0000 7f5c239af640 1 --2- 172.21.15.5:0/3253474086 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c08039300 0x7f5c0803b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.977+0000 7f5c239af640 1 --2- 172.21.15.5:0/3253474086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c1c1020d0 0x7f5c1c198f30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.977+0000 7f5c239af640 1 -- 172.21.15.5:0/3253474086 >> 172.21.15.5:0/3253474086 conn(0x7f5c1c0fbcd0 msgr2=0x7f5c1c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.977+0000 7f5c239af640 1 -- 172.21.15.5:0/3253474086 shutdown_connections 2024-09-15T21:59:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:51.977+0000 7f5c239af640 1 -- 172.21.15.5:0/3253474086 wait complete. 2024-09-15T21:59:53.299 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:52 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3253474086' entity='client.admin' 2024-09-15T21:59:53.329 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-15T21:59:53.330 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-15T21:59:54.180 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T21:59:54.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.614+0000 7fa71e2e3640 1 -- 172.21.15.5:0/4210093343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa718102190 msgr2=0x7fa718104580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:54.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.614+0000 7fa71e2e3640 1 --2- 172.21.15.5:0/4210093343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa718102190 0x7fa718104580 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa708009880 tx=0x7fa70802a5d0 comp rx=0 tx=0).stop 2024-09-15T21:59:54.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.614+0000 7fa71e2e3640 1 -- 172.21.15.5:0/4210093343 shutdown_connections 2024-09-15T21:59:54.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.614+0000 7fa71e2e3640 1 --2- 172.21.15.5:0/4210093343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa718102190 0x7fa718104580 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:54.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.614+0000 7fa71e2e3640 1 -- 172.21.15.5:0/4210093343 >> 172.21.15.5:0/4210093343 conn(0x7fa7180fbcd0 msgr2=0x7fa7180fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.614+0000 7fa71e2e3640 1 -- 172.21.15.5:0/4210093343 shutdown_connections 2024-09-15T21:59:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.614+0000 7fa71e2e3640 1 -- 172.21.15.5:0/4210093343 wait complete. 2024-09-15T21:59:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.615+0000 7fa71e2e3640 1 Processor -- start 2024-09-15T21:59:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.615+0000 7fa71e2e3640 1 -- start start 2024-09-15T21:59:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.615+0000 7fa71e2e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa718102190 0x7fa718196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.615+0000 7fa71e2e3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7181076d0 con 0x7fa718102190 2024-09-15T21:59:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.615+0000 7fa717fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa718102190 0x7fa718196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.616+0000 7fa717fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa718102190 0x7fa718196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37912/0 (socket says 172.21.15.5:37912) 2024-09-15T21:59:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.616+0000 7fa717fff640 1 -- 172.21.15.5:0/773994850 learned_addr learned my addr 172.21.15.5:0/773994850 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:54.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.616+0000 7fa717fff640 1 -- 172.21.15.5:0/773994850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7181972d0 con 0x7fa718102190 2024-09-15T21:59:54.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.616+0000 7fa717fff640 1 --2- 172.21.15.5:0/773994850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa718102190 0x7fa718196d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa708004f60 tx=0x7fa7080048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:54.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.616+0000 7fa7157fa640 1 -- 172.21.15.5:0/773994850 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa70803f070 con 0x7fa718102190 2024-09-15T21:59:54.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.616+0000 7fa7157fa640 1 -- 172.21.15.5:0/773994850 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa708003ad0 con 0x7fa718102190 2024-09-15T21:59:54.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.616+0000 7fa71e2e3640 1 -- 172.21.15.5:0/773994850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa718197560 con 0x7fa718102190 2024-09-15T21:59:54.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.616+0000 7fa7157fa640 1 -- 172.21.15.5:0/773994850 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa708035030 con 0x7fa718102190 2024-09-15T21:59:54.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.616+0000 7fa71e2e3640 1 -- 172.21.15.5:0/773994850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71819a210 con 0x7fa718102190 2024-09-15T21:59:54.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.617+0000 7fa7157fa640 1 -- 172.21.15.5:0/773994850 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fa708042070 con 0x7fa718102190 2024-09-15T21:59:54.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.617+0000 7fa7157fa640 1 --2- 172.21.15.5:0/773994850 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7000392a0 0x7fa70003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:54.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.617+0000 7fa7157fa640 1 -- 172.21.15.5:0/773994850 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa70806d0c0 con 0x7fa718102190 2024-09-15T21:59:54.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.617+0000 7fa7177fe640 1 --2- 172.21.15.5:0/773994850 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7000392a0 0x7fa70003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:54.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.618+0000 7fa71e2e3640 1 -- 172.21.15.5:0/773994850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa718107e50 con 0x7fa718102190 2024-09-15T21:59:54.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.618+0000 7fa7177fe640 1 --2- 172.21.15.5:0/773994850 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7000392a0 0x7fa70003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0098c0 tx=0x7fa70c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:54.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.622+0000 7fa7157fa640 1 -- 172.21.15.5:0/773994850 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa708039350 con 0x7fa718102190 2024-09-15T21:59:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.803+0000 7fa71e2e3640 1 -- 172.21.15.5:0/773994850 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fa718062630 con 0x7fa7000392a0 2024-09-15T21:59:54.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.806+0000 7fa7157fa640 1 -- 172.21.15.5:0/773994850 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa718062630 con 0x7fa7000392a0 2024-09-15T21:59:54.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.809+0000 7fa71e2e3640 1 -- 172.21.15.5:0/773994850 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7000392a0 msgr2=0x7fa70003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:54.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.809+0000 7fa71e2e3640 1 --2- 172.21.15.5:0/773994850 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7000392a0 0x7fa70003b760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0098c0 tx=0x7fa70c006db0 comp rx=0 tx=0).stop 2024-09-15T21:59:54.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.809+0000 7fa71e2e3640 1 -- 172.21.15.5:0/773994850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa718102190 msgr2=0x7fa718196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:54.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.809+0000 7fa71e2e3640 1 --2- 172.21.15.5:0/773994850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa718102190 0x7fa718196d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa708004f60 tx=0x7fa7080048f0 comp rx=0 tx=0).stop 2024-09-15T21:59:54.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.809+0000 7fa71e2e3640 1 -- 172.21.15.5:0/773994850 shutdown_connections 2024-09-15T21:59:54.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.809+0000 7fa71e2e3640 1 --2- 172.21.15.5:0/773994850 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7000392a0 0x7fa70003b760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:54.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.809+0000 7fa71e2e3640 1 --2- 172.21.15.5:0/773994850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa718102190 0x7fa718196d90 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:54.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.809+0000 7fa71e2e3640 1 -- 172.21.15.5:0/773994850 >> 172.21.15.5:0/773994850 conn(0x7fa7180fbcd0 msgr2=0x7fa7180fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:54.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.809+0000 7fa71e2e3640 1 -- 172.21.15.5:0/773994850 shutdown_connections 2024-09-15T21:59:54.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:54.809+0000 7fa71e2e3640 1 -- 172.21.15.5:0/773994850 wait complete. 2024-09-15T21:59:55.604 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:55.604 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:55.604 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi005", "name": "osd_memory_target"}]: dispatch 2024-09-15T21:59:55.604 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:55.604 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T21:59:55.604 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:55.604 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:55.605 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T21:59:55.605 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T21:59:55.605 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T21:59:55.635 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi099 2024-09-15T21:59:55.635 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-15T21:59:55.636 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.conf 2024-09-15T21:59:55.657 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-15T21:59:55.657 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T21:59:55.714 INFO:tasks.cephadm:Adding host smithi099 to orchestrator... 2024-09-15T21:59:55.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch host add smithi099 2024-09-15T21:59:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T21:59:56.349 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:56 smithi005 ceph-mon[30944]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T21:59:56.349 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:56 smithi005 ceph-mon[30944]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-15T21:59:56.349 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:56 smithi005 ceph-mon[30944]: Updating smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T21:59:56.349 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:56 smithi005 ceph-mon[30944]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-15T21:59:56.349 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:56 smithi005 ceph-mon[30944]: Updating smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T21:59:56.350 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:56 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:56.350 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:56 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:56.350 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:56 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T21:59:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.432+0000 7f99fb755640 1 -- 172.21.15.5:0/2261430969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f4106540 msgr2=0x7f99f4106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:59:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.432+0000 7f99fb755640 1 --2- 172.21.15.5:0/2261430969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f4106540 0x7f99f4106920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009810 tx=0x7f99f002a540 comp rx=0 tx=0).stop 2024-09-15T21:59:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.433+0000 7f99fb755640 1 -- 172.21.15.5:0/2261430969 shutdown_connections 2024-09-15T21:59:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.433+0000 7f99fb755640 1 --2- 172.21.15.5:0/2261430969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f4106540 0x7f99f4106920 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:59:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.433+0000 7f99fb755640 1 -- 172.21.15.5:0/2261430969 >> 172.21.15.5:0/2261430969 conn(0x7f99f4077310 msgr2=0x7f99f40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:59:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.433+0000 7f99fb755640 1 -- 172.21.15.5:0/2261430969 shutdown_connections 2024-09-15T21:59:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.433+0000 7f99fb755640 1 -- 172.21.15.5:0/2261430969 wait complete. 2024-09-15T21:59:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.433+0000 7f99fb755640 1 Processor -- start 2024-09-15T21:59:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.433+0000 7f99fb755640 1 -- start start 2024-09-15T21:59:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99fb755640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f4106540 0x7f99f40fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99fb755640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f99f410ab70 con 0x7f99f4106540 2024-09-15T21:59:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99f94f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f4106540 0x7f99f40fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99f94f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f4106540 0x7f99f40fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37928/0 (socket says 172.21.15.5:37928) 2024-09-15T21:59:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99f94f1640 1 -- 172.21.15.5:0/479069749 learned_addr learned my addr 172.21.15.5:0/479069749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T21:59:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99f94f1640 1 -- 172.21.15.5:0/479069749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f40ff290 con 0x7f99f4106540 2024-09-15T21:59:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99f94f1640 1 --2- 172.21.15.5:0/479069749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f4106540 0x7f99f40fd3a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99f0030940 tx=0x7f99f0005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99ea7fc640 1 -- 172.21.15.5:0/479069749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f0040070 con 0x7f99f4106540 2024-09-15T21:59:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99fb755640 1 -- 172.21.15.5:0/479069749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f40fd8e0 con 0x7f99f4106540 2024-09-15T21:59:56.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99ea7fc640 1 -- 172.21.15.5:0/479069749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99f0004400 con 0x7f99f4106540 2024-09-15T21:59:56.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99ea7fc640 1 -- 172.21.15.5:0/479069749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f0004950 con 0x7f99f4106540 2024-09-15T21:59:56.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.434+0000 7f99fb755640 1 -- 172.21.15.5:0/479069749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f40fdda0 con 0x7f99f4106540 2024-09-15T21:59:56.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.435+0000 7f99ea7fc640 1 -- 172.21.15.5:0/479069749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f99f0039040 con 0x7f99f4106540 2024-09-15T21:59:56.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.435+0000 7f99fb755640 1 -- 172.21.15.5:0/479069749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f4101bb0 con 0x7f99f4106540 2024-09-15T21:59:56.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.435+0000 7f99ea7fc640 1 --2- 172.21.15.5:0/479069749 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f99e00392a0 0x7f99e003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:59:56.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.436+0000 7f99ea7fc640 1 -- 172.21.15.5:0/479069749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f99f006d3f0 con 0x7f99f4106540 2024-09-15T21:59:56.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.436+0000 7f99f8cf0640 1 --2- 172.21.15.5:0/479069749 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f99e00392a0 0x7f99e003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:59:56.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.436+0000 7f99f8cf0640 1 --2- 172.21.15.5:0/479069749 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f99e00392a0 0x7f99e003b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f99e40098c0 tx=0x7f99e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:59:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.440+0000 7f99ea7fc640 1 -- 172.21.15.5:0/479069749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99f0005010 con 0x7f99f4106540 2024-09-15T21:59:56.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T21:59:56.621+0000 7f99fb755640 1 -- 172.21.15.5:0/479069749 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}) -- 0x7f99f40fe940 con 0x7f99e00392a0 2024-09-15T21:59:57.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:57 smithi005 ceph-mon[30944]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi099", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T21:59:59.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 21:59:58 smithi005 ceph-mon[30944]: Deploying cephadm binary to smithi099 2024-09-15T22:00:00.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:00.413+0000 7f99ea7fc640 1 -- 172.21.15.5:0/479069749 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f99f40fe940 con 0x7f99e00392a0 2024-09-15T22:00:00.415 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi099' with addr '172.21.15.99' 2024-09-15T22:00:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:00.415+0000 7f99fb755640 1 -- 172.21.15.5:0/479069749 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f99e00392a0 msgr2=0x7f99e003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:00.415+0000 7f99fb755640 1 --2- 172.21.15.5:0/479069749 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f99e00392a0 0x7f99e003b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f99e40098c0 tx=0x7f99e4006db0 comp rx=0 tx=0).stop 2024-09-15T22:00:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:00.415+0000 7f99fb755640 1 -- 172.21.15.5:0/479069749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f4106540 msgr2=0x7f99f40fd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:00.415+0000 7f99fb755640 1 --2- 172.21.15.5:0/479069749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f4106540 0x7f99f40fd3a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f99f0030940 tx=0x7f99f0005df0 comp rx=0 tx=0).stop 2024-09-15T22:00:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:00.415+0000 7f99fb755640 1 -- 172.21.15.5:0/479069749 shutdown_connections 2024-09-15T22:00:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:00.415+0000 7f99fb755640 1 --2- 172.21.15.5:0/479069749 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f99e00392a0 0x7f99e003b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:00.415+0000 7f99fb755640 1 --2- 172.21.15.5:0/479069749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f4106540 0x7f99f40fd3a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:00.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:00.415+0000 7f99fb755640 1 -- 172.21.15.5:0/479069749 >> 172.21.15.5:0/479069749 conn(0x7f99f4077310 msgr2=0x7f99f40696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:00.415+0000 7f99fb755640 1 -- 172.21.15.5:0/479069749 shutdown_connections 2024-09-15T22:00:00.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:00.415+0000 7f99fb755640 1 -- 172.21.15.5:0/479069749 wait complete. 2024-09-15T22:00:00.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-15T22:00:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:00:01.605 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:01 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:01.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:01 smithi005 ceph-mon[30944]: Added host smithi099 2024-09-15T22:00:01.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:01 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:01.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:01 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:01.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:01 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.629+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1292152854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41027b0 msgr2=0x7f17d4102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.629+0000 7f17d9eb4640 1 --2- 172.21.15.5:0/1292152854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41027b0 0x7f17d4102b90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009880 tx=0x7f17c402a670 comp rx=0 tx=0).stop 2024-09-15T22:00:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.629+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1292152854 shutdown_connections 2024-09-15T22:00:01.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.629+0000 7f17d9eb4640 1 --2- 172.21.15.5:0/1292152854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41027b0 0x7f17d4102b90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.629+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1292152854 >> 172.21.15.5:0/1292152854 conn(0x7f17d40fc1b0 msgr2=0x7f17d40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.629+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1292152854 shutdown_connections 2024-09-15T22:00:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.629+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1292152854 wait complete. 2024-09-15T22:00:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.629+0000 7f17d9eb4640 1 Processor -- start 2024-09-15T22:00:01.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.629+0000 7f17d9eb4640 1 -- start start 2024-09-15T22:00:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d9eb4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41027b0 0x7f17d41971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d9eb4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17d410bcb0 con 0x7f17d41027b0 2024-09-15T22:00:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41027b0 0x7f17d41971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41027b0 0x7f17d41971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37938/0 (socket says 172.21.15.5:37938) 2024-09-15T22:00:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d37fe640 1 -- 172.21.15.5:0/1656684485 learned_addr learned my addr 172.21.15.5:0/1656684485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:00:01.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d37fe640 1 -- 172.21.15.5:0/1656684485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d4197700 con 0x7f17d41027b0 2024-09-15T22:00:01.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d37fe640 1 --2- 172.21.15.5:0/1656684485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41027b0 0x7f17d41971c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009230 tx=0x7f17c4003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:01.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d17fa640 1 -- 172.21.15.5:0/1656684485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c403f070 con 0x7f17d41027b0 2024-09-15T22:00:01.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d17fa640 1 -- 172.21.15.5:0/1656684485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c403bd40 con 0x7f17d41027b0 2024-09-15T22:00:01.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1656684485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d4197990 con 0x7f17d41027b0 2024-09-15T22:00:01.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d17fa640 1 -- 172.21.15.5:0/1656684485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c4035030 con 0x7f17d41027b0 2024-09-15T22:00:01.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.630+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1656684485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d4197df0 con 0x7f17d41027b0 2024-09-15T22:00:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.631+0000 7f17d17fa640 1 -- 172.21.15.5:0/1656684485 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f17c4042070 con 0x7f17d41027b0 2024-09-15T22:00:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.631+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1656684485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d406a750 con 0x7f17d41027b0 2024-09-15T22:00:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.632+0000 7f17d17fa640 1 --2- 172.21.15.5:0/1656684485 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f17c00392a0 0x7f17c003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.632+0000 7f17d17fa640 1 -- 172.21.15.5:0/1656684485 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f17c406cf30 con 0x7f17d41027b0 2024-09-15T22:00:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.632+0000 7f17cbfff640 1 --2- 172.21.15.5:0/1656684485 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f17c00392a0 0x7f17c003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.632+0000 7f17cbfff640 1 --2- 172.21.15.5:0/1656684485 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f17c00392a0 0x7f17c003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0098c0 tx=0x7f17bc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.637+0000 7f17d17fa640 1 -- 172.21.15.5:0/1656684485 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17c4030d90 con 0x7f17d41027b0 2024-09-15T22:00:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.819+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1656684485 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17d4198520 con 0x7f17c00392a0 2024-09-15T22:00:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.820+0000 7f17d17fa640 1 -- 172.21.15.5:0/1656684485 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+158 (secure 0 0 0) 0x7f17d4198520 con 0x7f17c00392a0 2024-09-15T22:00:01.822 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:00:01.822 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}] 2024-09-15T22:00:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.822+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1656684485 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f17c00392a0 msgr2=0x7f17c003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.822+0000 7f17d9eb4640 1 --2- 172.21.15.5:0/1656684485 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f17c00392a0 0x7f17c003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0098c0 tx=0x7f17bc006db0 comp rx=0 tx=0).stop 2024-09-15T22:00:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.822+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1656684485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41027b0 msgr2=0x7f17d41971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.822+0000 7f17d9eb4640 1 --2- 172.21.15.5:0/1656684485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41027b0 0x7f17d41971c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009230 tx=0x7f17c4003d80 comp rx=0 tx=0).stop 2024-09-15T22:00:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.822+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1656684485 shutdown_connections 2024-09-15T22:00:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.822+0000 7f17d9eb4640 1 --2- 172.21.15.5:0/1656684485 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f17c00392a0 0x7f17c003b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.822+0000 7f17d9eb4640 1 --2- 172.21.15.5:0/1656684485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17d41027b0 0x7f17d41971c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.822+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1656684485 >> 172.21.15.5:0/1656684485 conn(0x7f17d40fc1b0 msgr2=0x7f17d4105570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.823+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1656684485 shutdown_connections 2024-09-15T22:00:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:01.823+0000 7f17d9eb4640 1 -- 172.21.15.5:0/1656684485 wait complete. 2024-09-15T22:00:02.363 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi129 2024-09-15T22:00:02.363 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-15T22:00:02.363 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.conf 2024-09-15T22:00:02.383 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-15T22:00:02.383 DEBUG:teuthology.orchestra.run.smithi129:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:00:02.441 INFO:tasks.cephadm:Adding host smithi129 to orchestrator... 2024-09-15T22:00:02.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch host add smithi129 2024-09-15T22:00:02.693 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:00:03.045 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:03.045 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:02 smithi005 ceph-mon[30944]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:00:03.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.128+0000 7fa65ff1f640 1 -- 172.21.15.5:0/3693935243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa658106560 msgr2=0x7fa658106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.128+0000 7fa65ff1f640 1 --2- 172.21.15.5:0/3693935243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa658106560 0x7fa658106940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa654009880 tx=0x7fa65402a5d0 comp rx=0 tx=0).stop 2024-09-15T22:00:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.128+0000 7fa65ff1f640 1 -- 172.21.15.5:0/3693935243 shutdown_connections 2024-09-15T22:00:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.128+0000 7fa65ff1f640 1 --2- 172.21.15.5:0/3693935243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa658106560 0x7fa658106940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.128+0000 7fa65ff1f640 1 -- 172.21.15.5:0/3693935243 >> 172.21.15.5:0/3693935243 conn(0x7fa6580fc1b0 msgr2=0x7fa6580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.128+0000 7fa65ff1f640 1 -- 172.21.15.5:0/3693935243 shutdown_connections 2024-09-15T22:00:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.128+0000 7fa65ff1f640 1 -- 172.21.15.5:0/3693935243 wait complete. 2024-09-15T22:00:03.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.129+0000 7fa65ff1f640 1 Processor -- start 2024-09-15T22:00:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.129+0000 7fa65ff1f640 1 -- start start 2024-09-15T22:00:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.129+0000 7fa65ff1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa658106560 0x7fa6581993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.129+0000 7fa65ff1f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa65810ab90 con 0x7fa658106560 2024-09-15T22:00:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.129+0000 7fa65dcbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa658106560 0x7fa6581993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.129+0000 7fa65dcbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa658106560 0x7fa6581993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37962/0 (socket says 172.21.15.5:37962) 2024-09-15T22:00:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.129+0000 7fa65dcbb640 1 -- 172.21.15.5:0/1853685004 learned_addr learned my addr 172.21.15.5:0/1853685004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:00:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.129+0000 7fa65dcbb640 1 -- 172.21.15.5:0/1853685004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa658199910 con 0x7fa658106560 2024-09-15T22:00:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.130+0000 7fa65dcbb640 1 --2- 172.21.15.5:0/1853685004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa658106560 0x7fa6581993d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa654009230 tx=0x7fa654003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:03.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.130+0000 7fa64effd640 1 -- 172.21.15.5:0/1853685004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa65403f070 con 0x7fa658106560 2024-09-15T22:00:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.130+0000 7fa64effd640 1 -- 172.21.15.5:0/1853685004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65403bc80 con 0x7fa658106560 2024-09-15T22:00:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.130+0000 7fa65ff1f640 1 -- 172.21.15.5:0/1853685004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa658199ba0 con 0x7fa658106560 2024-09-15T22:00:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.130+0000 7fa64effd640 1 -- 172.21.15.5:0/1853685004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa654035030 con 0x7fa658106560 2024-09-15T22:00:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.130+0000 7fa65ff1f640 1 -- 172.21.15.5:0/1853685004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65819a000 con 0x7fa658106560 2024-09-15T22:00:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.131+0000 7fa64effd640 1 -- 172.21.15.5:0/1853685004 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fa654042070 con 0x7fa658106560 2024-09-15T22:00:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.131+0000 7fa64effd640 1 --2- 172.21.15.5:0/1853685004 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa6440392a0 0x7fa64403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.131+0000 7fa65ff1f640 1 -- 172.21.15.5:0/1853685004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa658101bd0 con 0x7fa658106560 2024-09-15T22:00:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.131+0000 7fa64effd640 1 -- 172.21.15.5:0/1853685004 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa65406d020 con 0x7fa658106560 2024-09-15T22:00:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.131+0000 7fa65d4ba640 1 --2- 172.21.15.5:0/1853685004 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa6440392a0 0x7fa64403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:03.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.132+0000 7fa65d4ba640 1 --2- 172.21.15.5:0/1853685004 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa6440392a0 0x7fa64403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6480098c0 tx=0x7fa648006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:03.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.136+0000 7fa64effd640 1 -- 172.21.15.5:0/1853685004 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6540372d0 con 0x7fa658106560 2024-09-15T22:00:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:03.315+0000 7fa65ff1f640 1 -- 172.21.15.5:0/1853685004 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}) -- 0x7fa65819a730 con 0x7fa6440392a0 2024-09-15T22:00:04.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:03 smithi005 ceph-mon[30944]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:04.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:03 smithi005 ceph-mon[30944]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi129", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:00:05.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: Deploying cephadm binary to smithi129 2024-09-15T22:00:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi099", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:00:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-15T22:00:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:06 smithi005 ceph-mon[30944]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:06 smithi005 ceph-mon[30944]: Updating smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:06 smithi005 ceph-mon[30944]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:00:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:06 smithi005 ceph-mon[30944]: Updating smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:00:07.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:07.194+0000 7fa64effd640 1 -- 172.21.15.5:0/1853685004 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fa65819a730 con 0x7fa6440392a0 2024-09-15T22:00:07.197 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi129' with addr '172.21.15.129' 2024-09-15T22:00:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:07.197+0000 7fa65ff1f640 1 -- 172.21.15.5:0/1853685004 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa6440392a0 msgr2=0x7fa64403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:07.197+0000 7fa65ff1f640 1 --2- 172.21.15.5:0/1853685004 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa6440392a0 0x7fa64403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa6480098c0 tx=0x7fa648006db0 comp rx=0 tx=0).stop 2024-09-15T22:00:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:07.197+0000 7fa65ff1f640 1 -- 172.21.15.5:0/1853685004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa658106560 msgr2=0x7fa6581993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:07.197+0000 7fa65ff1f640 1 --2- 172.21.15.5:0/1853685004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa658106560 0x7fa6581993d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa654009230 tx=0x7fa654003d40 comp rx=0 tx=0).stop 2024-09-15T22:00:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:07.197+0000 7fa65ff1f640 1 -- 172.21.15.5:0/1853685004 shutdown_connections 2024-09-15T22:00:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:07.197+0000 7fa65ff1f640 1 --2- 172.21.15.5:0/1853685004 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa6440392a0 0x7fa64403b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:07.197+0000 7fa65ff1f640 1 --2- 172.21.15.5:0/1853685004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa658106560 0x7fa6581993d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:07.197+0000 7fa65ff1f640 1 -- 172.21.15.5:0/1853685004 >> 172.21.15.5:0/1853685004 conn(0x7fa6580fc1b0 msgr2=0x7fa6580fcac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:07.197+0000 7fa65ff1f640 1 -- 172.21.15.5:0/1853685004 shutdown_connections 2024-09-15T22:00:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:07.197+0000 7fa65ff1f640 1 -- 172.21.15.5:0/1853685004 wait complete. 2024-09-15T22:00:07.821 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-15T22:00:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:00:08.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:08 smithi005 ceph-mon[30944]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:08.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:08 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:08.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:08 smithi005 ceph-mon[30944]: Added host smithi129 2024-09-15T22:00:08.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:08 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:08.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:08 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:08.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:08 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.501+0000 7f0da22c2640 1 -- 172.21.15.5:0/3916507252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d9c106540 msgr2=0x7f0d9c106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.501+0000 7f0da22c2640 1 --2- 172.21.15.5:0/3916507252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d9c106540 0x7f0d9c106920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009810 tx=0x7f0d8c02a540 comp rx=0 tx=0).stop 2024-09-15T22:00:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.501+0000 7f0da22c2640 1 -- 172.21.15.5:0/3916507252 shutdown_connections 2024-09-15T22:00:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.501+0000 7f0da22c2640 1 --2- 172.21.15.5:0/3916507252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d9c106540 0x7f0d9c106920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.501+0000 7f0da22c2640 1 -- 172.21.15.5:0/3916507252 >> 172.21.15.5:0/3916507252 conn(0x7f0d9c077310 msgr2=0x7f0d9c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.501+0000 7f0da22c2640 1 -- 172.21.15.5:0/3916507252 shutdown_connections 2024-09-15T22:00:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.501+0000 7f0da22c2640 1 -- 172.21.15.5:0/3916507252 wait complete. 2024-09-15T22:00:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.501+0000 7f0da22c2640 1 Processor -- start 2024-09-15T22:00:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.501+0000 7f0da22c2640 1 -- start start 2024-09-15T22:00:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0da22c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d9c106540 0x7f0d9c102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0da22c2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10ab70 con 0x7f0d9c106540 2024-09-15T22:00:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0d9bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d9c106540 0x7f0d9c102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0d9bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d9c106540 0x7f0d9c102100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60458/0 (socket says 172.21.15.5:60458) 2024-09-15T22:00:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0d9bfff640 1 -- 172.21.15.5:0/1058735767 learned_addr learned my addr 172.21.15.5:0/1058735767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:00:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0d9bfff640 1 -- 172.21.15.5:0/1058735767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c103ff0 con 0x7f0d9c106540 2024-09-15T22:00:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0d9bfff640 1 --2- 172.21.15.5:0/1058735767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d9c106540 0x7f0d9c102100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0045e0 tx=0x7f0d8c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0d99ffb640 1 -- 172.21.15.5:0/1058735767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c030e70 con 0x7f0d9c106540 2024-09-15T22:00:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0d99ffb640 1 -- 172.21.15.5:0/1058735767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d8c0318e0 con 0x7f0d9c106540 2024-09-15T22:00:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0da22c2640 1 -- 172.21.15.5:0/1058735767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c102640 con 0x7f0d9c106540 2024-09-15T22:00:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0d99ffb640 1 -- 172.21.15.5:0/1058735767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c0049c0 con 0x7f0d9c106540 2024-09-15T22:00:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.502+0000 7f0da22c2640 1 -- 172.21.15.5:0/1058735767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c102b00 con 0x7f0d9c106540 2024-09-15T22:00:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.503+0000 7f0d99ffb640 1 -- 172.21.15.5:0/1058735767 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f0d8c039040 con 0x7f0d9c106540 2024-09-15T22:00:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.503+0000 7f0da22c2640 1 -- 172.21.15.5:0/1058735767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c101bb0 con 0x7f0d9c106540 2024-09-15T22:00:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.503+0000 7f0d99ffb640 1 --2- 172.21.15.5:0/1058735767 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0d88039350 0x7f0d8803b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.504+0000 7f0d99ffb640 1 -- 172.21.15.5:0/1058735767 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0d8c06d630 con 0x7f0d9c106540 2024-09-15T22:00:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.504+0000 7f0d93fff640 1 --2- 172.21.15.5:0/1058735767 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0d88039350 0x7f0d8803b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.504+0000 7f0d93fff640 1 --2- 172.21.15.5:0/1058735767 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0d88039350 0x7f0d8803b810 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0d840098c0 tx=0x7f0d84006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.508+0000 7f0d99ffb640 1 -- 172.21.15.5:0/1058735767 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d8c040030 con 0x7f0d9c106540 2024-09-15T22:00:08.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.701+0000 7f0da22c2640 1 -- 172.21.15.5:0/1058735767 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0d9c1036a0 con 0x7f0d88039350 2024-09-15T22:00:08.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.703+0000 7f0d99ffb640 1 -- 172.21.15.5:0/1058735767 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f0d9c1036a0 con 0x7f0d88039350 2024-09-15T22:00:08.704 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:00:08.704 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.99", "hostname": "smithi099", "labels": [], "status": ""}, {"addr": "172.21.15.129", "hostname": "smithi129", "labels": [], "status": ""}] 2024-09-15T22:00:08.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.705+0000 7f0da22c2640 1 -- 172.21.15.5:0/1058735767 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0d88039350 msgr2=0x7f0d8803b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.705+0000 7f0da22c2640 1 --2- 172.21.15.5:0/1058735767 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0d88039350 0x7f0d8803b810 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0d840098c0 tx=0x7f0d84006db0 comp rx=0 tx=0).stop 2024-09-15T22:00:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.705+0000 7f0da22c2640 1 -- 172.21.15.5:0/1058735767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d9c106540 msgr2=0x7f0d9c102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.705+0000 7f0da22c2640 1 --2- 172.21.15.5:0/1058735767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d9c106540 0x7f0d9c102100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0045e0 tx=0x7f0d8c004610 comp rx=0 tx=0).stop 2024-09-15T22:00:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.705+0000 7f0da22c2640 1 -- 172.21.15.5:0/1058735767 shutdown_connections 2024-09-15T22:00:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.705+0000 7f0da22c2640 1 --2- 172.21.15.5:0/1058735767 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0d88039350 0x7f0d8803b810 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.705+0000 7f0da22c2640 1 --2- 172.21.15.5:0/1058735767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d9c106540 0x7f0d9c102100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.705+0000 7f0da22c2640 1 -- 172.21.15.5:0/1058735767 >> 172.21.15.5:0/1058735767 conn(0x7f0d9c077310 msgr2=0x7f0d9c0fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.705+0000 7f0da22c2640 1 -- 172.21.15.5:0/1058735767 shutdown_connections 2024-09-15T22:00:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:08.705+0000 7f0da22c2640 1 -- 172.21.15.5:0/1058735767 wait complete. 2024-09-15T22:00:09.295 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-15T22:00:09.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-15T22:00:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:00:09.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:09 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:09.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.992+0000 7f8fe1d90640 1 -- 172.21.15.5:0/242315750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fdc1027f0 msgr2=0x7f8fdc102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:09.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.992+0000 7f8fe1d90640 1 --2- 172.21.15.5:0/242315750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fdc1027f0 0x7f8fdc102bd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009810 tx=0x7f8fcc02a540 comp rx=0 tx=0).stop 2024-09-15T22:00:09.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.992+0000 7f8fe1d90640 1 -- 172.21.15.5:0/242315750 shutdown_connections 2024-09-15T22:00:09.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.992+0000 7f8fe1d90640 1 --2- 172.21.15.5:0/242315750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fdc1027f0 0x7f8fdc102bd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.992+0000 7f8fe1d90640 1 -- 172.21.15.5:0/242315750 >> 172.21.15.5:0/242315750 conn(0x7f8fdc077310 msgr2=0x7f8fdc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.992+0000 7f8fe1d90640 1 -- 172.21.15.5:0/242315750 shutdown_connections 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.992+0000 7f8fe1d90640 1 -- 172.21.15.5:0/242315750 wait complete. 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.992+0000 7f8fe1d90640 1 Processor -- start 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.993+0000 7f8fe1d90640 1 -- start start 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.993+0000 7f8fe1d90640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fdc1027f0 0x7f8fdc100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.993+0000 7f8fe1d90640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8fdc10bc80 con 0x7f8fdc1027f0 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.993+0000 7f8fdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fdc1027f0 0x7f8fdc100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.993+0000 7f8fdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fdc1027f0 0x7f8fdc100450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60476/0 (socket says 172.21.15.5:60476) 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.993+0000 7f8fdb7fe640 1 -- 172.21.15.5:0/96898022 learned_addr learned my addr 172.21.15.5:0/96898022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.993+0000 7f8fdb7fe640 1 -- 172.21.15.5:0/96898022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fdc102340 con 0x7f8fdc1027f0 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.993+0000 7f8fdb7fe640 1 --2- 172.21.15.5:0/96898022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fdc1027f0 0x7f8fdc100450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0045e0 tx=0x7f8fcc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:09.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.994+0000 7f8fd8ff9640 1 -- 172.21.15.5:0/96898022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fcc030e70 con 0x7f8fdc1027f0 2024-09-15T22:00:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.994+0000 7f8fe1d90640 1 -- 172.21.15.5:0/96898022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fdc100990 con 0x7f8fdc1027f0 2024-09-15T22:00:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.994+0000 7f8fd8ff9640 1 -- 172.21.15.5:0/96898022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fcc0318e0 con 0x7f8fdc1027f0 2024-09-15T22:00:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.994+0000 7f8fd8ff9640 1 -- 172.21.15.5:0/96898022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fcc0049c0 con 0x7f8fdc1027f0 2024-09-15T22:00:09.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.994+0000 7f8fe1d90640 1 -- 172.21.15.5:0/96898022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fdc100e50 con 0x7f8fdc1027f0 2024-09-15T22:00:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.995+0000 7f8fd8ff9640 1 -- 172.21.15.5:0/96898022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f8fcc039040 con 0x7f8fdc1027f0 2024-09-15T22:00:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.995+0000 7f8fd8ff9640 1 --2- 172.21.15.5:0/96898022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fc40392a0 0x7f8fc403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.995+0000 7f8fd8ff9640 1 -- 172.21.15.5:0/96898022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8fcc06d5c0 con 0x7f8fdc1027f0 2024-09-15T22:00:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.995+0000 7f8fe1d90640 1 -- 172.21.15.5:0/96898022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fdc069ed0 con 0x7f8fdc1027f0 2024-09-15T22:00:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.995+0000 7f8fdaffd640 1 --2- 172.21.15.5:0/96898022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fc40392a0 0x7f8fc403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:09.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:09.996+0000 7f8fdaffd640 1 --2- 172.21.15.5:0/96898022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fc40392a0 0x7f8fc403b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00098c0 tx=0x7f8fd0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:10.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.000+0000 7f8fd8ff9640 1 -- 172.21.15.5:0/96898022 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fcc040030 con 0x7f8fdc1027f0 2024-09-15T22:00:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.182+0000 7f8fe1d90640 1 -- 172.21.15.5:0/96898022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f8fdc101980 con 0x7f8fdc1027f0 2024-09-15T22:00:10.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.549+0000 7f8fd8ff9640 1 -- 172.21.15.5:0/96898022 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f8fcc037d70 con 0x7f8fdc1027f0 2024-09-15T22:00:10.550 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-09-15T22:00:10.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.551+0000 7f8fe1d90640 1 -- 172.21.15.5:0/96898022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fc40392a0 msgr2=0x7f8fc403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:10.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.551+0000 7f8fe1d90640 1 --2- 172.21.15.5:0/96898022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fc40392a0 0x7f8fc403b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00098c0 tx=0x7f8fd0006db0 comp rx=0 tx=0).stop 2024-09-15T22:00:10.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.551+0000 7f8fe1d90640 1 -- 172.21.15.5:0/96898022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fdc1027f0 msgr2=0x7f8fdc100450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:10.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.551+0000 7f8fe1d90640 1 --2- 172.21.15.5:0/96898022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fdc1027f0 0x7f8fdc100450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc0045e0 tx=0x7f8fcc004610 comp rx=0 tx=0).stop 2024-09-15T22:00:10.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.551+0000 7f8fe1d90640 1 -- 172.21.15.5:0/96898022 shutdown_connections 2024-09-15T22:00:10.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.551+0000 7f8fe1d90640 1 --2- 172.21.15.5:0/96898022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fc40392a0 0x7f8fc403b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:10.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.551+0000 7f8fe1d90640 1 --2- 172.21.15.5:0/96898022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fdc1027f0 0x7f8fdc100450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:10.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.551+0000 7f8fe1d90640 1 -- 172.21.15.5:0/96898022 >> 172.21.15.5:0/96898022 conn(0x7f8fdc077310 msgr2=0x7f8fdc105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:10.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.552+0000 7f8fe1d90640 1 -- 172.21.15.5:0/96898022 shutdown_connections 2024-09-15T22:00:10.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:10.552+0000 7f8fe1d90640 1 -- 172.21.15.5:0/96898022 wait complete. 2024-09-15T22:00:10.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:10 smithi005 ceph-mon[30944]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:10.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:10 smithi005 ceph-mon[30944]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:00:10.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:10 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/96898022' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-15T22:00:11.228 INFO:tasks.cephadm:Adding mon.a on smithi005 2024-09-15T22:00:11.228 INFO:tasks.cephadm:Adding mon.b on smithi099 2024-09-15T22:00:11.229 INFO:tasks.cephadm:Adding mon.c on smithi129 2024-09-15T22:00:11.229 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi005:172.21.15.5=a;smithi099:172.21.15.99=b;smithi129:172.21.15.129=c' 2024-09-15T22:00:11.477 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-15T22:00:11.522 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-15T22:00:11.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:11 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/96898022' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-15T22:00:11.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:11 smithi005 ceph-mon[30944]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:11.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:11.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:11.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:11.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:11.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:00:11.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:11.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:12.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.972+0000 7ff22a4d9640 1 -- 172.21.15.129:0/2581621885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff224102770 msgr2=0x7ff224102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:12.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.972+0000 7ff22a4d9640 1 --2- 172.21.15.129:0/2581621885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff224102770 0x7ff224102b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff214009880 tx=0x7ff21402a670 comp rx=0 tx=0).stop 2024-09-15T22:00:12.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.972+0000 7ff22a4d9640 1 -- 172.21.15.129:0/2581621885 shutdown_connections 2024-09-15T22:00:12.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.972+0000 7ff22a4d9640 1 --2- 172.21.15.129:0/2581621885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff224102770 0x7ff224102b50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:12.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.972+0000 7ff22a4d9640 1 -- 172.21.15.129:0/2581621885 >> 172.21.15.129:0/2581621885 conn(0x7ff2240fc190 msgr2=0x7ff2240fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:12.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.972+0000 7ff22a4d9640 1 -- 172.21.15.129:0/2581621885 shutdown_connections 2024-09-15T22:00:12.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.972+0000 7ff22a4d9640 1 -- 172.21.15.129:0/2581621885 wait complete. 2024-09-15T22:00:12.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.973+0000 7ff22a4d9640 1 Processor -- start 2024-09-15T22:00:12.003 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.973+0000 7ff22a4d9640 1 -- start start 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.973+0000 7ff22a4d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff224102770 0x7ff224197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.973+0000 7ff22a4d9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff22410bce0 con 0x7ff224102770 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.973+0000 7ff223fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff224102770 0x7ff224197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.974+0000 7ff223fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff224102770 0x7ff224197220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.129:53720/0 (socket says 172.21.15.129:53720) 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.974+0000 7ff223fff640 1 -- 172.21.15.129:0/1862070593 learned_addr learned my addr 172.21.15.129:0/1862070593 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.974+0000 7ff223fff640 1 -- 172.21.15.129:0/1862070593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff224197760 con 0x7ff224102770 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.974+0000 7ff223fff640 1 --2- 172.21.15.129:0/1862070593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff224102770 0x7ff224197220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff214009230 tx=0x7ff214003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.975+0000 7ff2217fa640 1 -- 172.21.15.129:0/1862070593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff21403f070 con 0x7ff224102770 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.975+0000 7ff22a4d9640 1 -- 172.21.15.129:0/1862070593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2241979f0 con 0x7ff224102770 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.975+0000 7ff22a4d9640 1 -- 172.21.15.129:0/1862070593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff224197e50 con 0x7ff224102770 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.975+0000 7ff2217fa640 1 -- 172.21.15.129:0/1862070593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff21403bc80 con 0x7ff224102770 2024-09-15T22:00:12.004 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.975+0000 7ff2217fa640 1 -- 172.21.15.129:0/1862070593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff214049060 con 0x7ff224102770 2024-09-15T22:00:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.976+0000 7ff2217fa640 1 -- 172.21.15.129:0/1862070593 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7ff214042070 con 0x7ff224102770 2024-09-15T22:00:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.976+0000 7ff22a4d9640 1 -- 172.21.15.129:0/1862070593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff22406a7b0 con 0x7ff224102770 2024-09-15T22:00:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.976+0000 7ff2217fa640 1 --2- 172.21.15.129:0/1862070593 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff20c0392a0 0x7ff20c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.976+0000 7ff2217fa640 1 -- 172.21.15.129:0/1862070593 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff21406d1b0 con 0x7ff224102770 2024-09-15T22:00:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.976+0000 7ff2237fe640 1 --2- 172.21.15.129:0/1862070593 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff20c0392a0 0x7ff20c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.977+0000 7ff2237fe640 1 --2- 172.21.15.129:0/1862070593 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff20c0392a0 0x7ff20c03b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff2180098c0 tx=0x7ff218006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:12.005 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:11.982+0000 7ff2217fa640 1 -- 172.21.15.129:0/1862070593 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff214031df0 con 0x7ff224102770 2024-09-15T22:00:12.166 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.164+0000 7ff22a4d9640 1 -- 172.21.15.129:0/1862070593 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi099:172.21.15.99=b;smithi129:172.21.15.129=c", "target": ["mon-mgr", ""]}) -- 0x7ff224198630 con 0x7ff20c0392a0 2024-09-15T22:00:12.171 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.169+0000 7ff2217fa640 1 -- 172.21.15.129:0/1862070593 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff2180094d0 con 0x7ff20c0392a0 2024-09-15T22:00:12.171 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled mon update... 2024-09-15T22:00:12.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.171+0000 7ff22a4d9640 1 -- 172.21.15.129:0/1862070593 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff20c0392a0 msgr2=0x7ff20c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:12.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.171+0000 7ff22a4d9640 1 --2- 172.21.15.129:0/1862070593 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff20c0392a0 0x7ff20c03b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff2180098c0 tx=0x7ff218006db0 comp rx=0 tx=0).stop 2024-09-15T22:00:12.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.171+0000 7ff22a4d9640 1 -- 172.21.15.129:0/1862070593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff224102770 msgr2=0x7ff224197220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:12.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.171+0000 7ff22a4d9640 1 --2- 172.21.15.129:0/1862070593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff224102770 0x7ff224197220 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff214009230 tx=0x7ff214003d40 comp rx=0 tx=0).stop 2024-09-15T22:00:12.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.171+0000 7ff22a4d9640 1 -- 172.21.15.129:0/1862070593 shutdown_connections 2024-09-15T22:00:12.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.171+0000 7ff22a4d9640 1 --2- 172.21.15.129:0/1862070593 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff20c0392a0 0x7ff20c03b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:12.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.171+0000 7ff22a4d9640 1 --2- 172.21.15.129:0/1862070593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff224102770 0x7ff224197220 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:12.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.171+0000 7ff22a4d9640 1 -- 172.21.15.129:0/1862070593 >> 172.21.15.129:0/1862070593 conn(0x7ff2240fc190 msgr2=0x7ff224105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:12.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.171+0000 7ff22a4d9640 1 -- 172.21.15.129:0/1862070593 shutdown_connections 2024-09-15T22:00:12.173 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:12.171+0000 7ff22a4d9640 1 -- 172.21.15.129:0/1862070593 wait complete. 2024-09-15T22:00:12.716 DEBUG:teuthology.orchestra.run.smithi099:mon.b> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.b.service 2024-09-15T22:00:12.719 DEBUG:teuthology.orchestra.run.smithi129:mon.c> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.c.service 2024-09-15T22:00:12.721 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:00:12.722 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:00:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-15T22:00:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:00:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:12.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:12.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:12.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:12.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:12.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:13.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:13 smithi005 ceph-mon[30944]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:00:14.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:13 smithi005 ceph-mon[30944]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi099:172.21.15.99=b;smithi129:172.21.15.129=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:00:14.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:13 smithi005 ceph-mon[30944]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi099:172.21.15.99=b;smithi129:172.21.15.129=c;count:3 2024-09-15T22:00:14.013 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:13 smithi005 ceph-mon[30944]: Deploying daemon mon.c on smithi129 2024-09-15T22:00:14.376 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:14.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.820+0000 7fbd7dee6640 1 -- 172.21.15.129:0/156758573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd781065c0 msgr2=0x7fbd781069a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:14.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.820+0000 7fbd7dee6640 1 --2- 172.21.15.129:0/156758573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd781065c0 0x7fbd781069a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009880 tx=0x7fbd6802a5d0 comp rx=0 tx=0).stop 2024-09-15T22:00:14.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.820+0000 7fbd7dee6640 1 -- 172.21.15.129:0/156758573 shutdown_connections 2024-09-15T22:00:14.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.820+0000 7fbd7dee6640 1 --2- 172.21.15.129:0/156758573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd781065c0 0x7fbd781069a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:14.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.820+0000 7fbd7dee6640 1 -- 172.21.15.129:0/156758573 >> 172.21.15.129:0/156758573 conn(0x7fbd780fb8a0 msgr2=0x7fbd780fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:14.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.820+0000 7fbd7dee6640 1 -- 172.21.15.129:0/156758573 shutdown_connections 2024-09-15T22:00:14.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.820+0000 7fbd7dee6640 1 -- 172.21.15.129:0/156758573 wait complete. 2024-09-15T22:00:14.853 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.820+0000 7fbd7dee6640 1 Processor -- start 2024-09-15T22:00:14.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.821+0000 7fbd7dee6640 1 -- start start 2024-09-15T22:00:14.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.821+0000 7fbd7dee6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd781065c0 0x7fbd78199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:14.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.821+0000 7fbd7dee6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd7810abd0 con 0x7fbd781065c0 2024-09-15T22:00:14.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.821+0000 7fbd777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd781065c0 0x7fbd78199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:14.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.821+0000 7fbd777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd781065c0 0x7fbd78199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.129:53726/0 (socket says 172.21.15.129:53726) 2024-09-15T22:00:14.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.821+0000 7fbd777fe640 1 -- 172.21.15.129:0/3322775667 learned_addr learned my addr 172.21.15.129:0/3322775667 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-15T22:00:14.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.821+0000 7fbd777fe640 1 -- 172.21.15.129:0/3322775667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd78199850 con 0x7fbd781065c0 2024-09-15T22:00:14.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.822+0000 7fbd777fe640 1 --2- 172.21.15.129:0/3322775667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd781065c0 0x7fbd78199310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009230 tx=0x7fbd68003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:14.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.822+0000 7fbd74ff9640 1 -- 172.21.15.129:0/3322775667 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd6803f070 con 0x7fbd781065c0 2024-09-15T22:00:14.854 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.822+0000 7fbd74ff9640 1 -- 172.21.15.129:0/3322775667 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd6803bc80 con 0x7fbd781065c0 2024-09-15T22:00:14.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.822+0000 7fbd7dee6640 1 -- 172.21.15.129:0/3322775667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd78199ae0 con 0x7fbd781065c0 2024-09-15T22:00:14.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.822+0000 7fbd74ff9640 1 -- 172.21.15.129:0/3322775667 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd68035030 con 0x7fbd781065c0 2024-09-15T22:00:14.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.822+0000 7fbd7dee6640 1 -- 172.21.15.129:0/3322775667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd7819dbe0 con 0x7fbd781065c0 2024-09-15T22:00:14.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.823+0000 7fbd74ff9640 1 -- 172.21.15.129:0/3322775667 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fbd68042070 con 0x7fbd781065c0 2024-09-15T22:00:14.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.823+0000 7fbd7dee6640 1 -- 172.21.15.129:0/3322775667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd78101c10 con 0x7fbd781065c0 2024-09-15T22:00:14.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.823+0000 7fbd74ff9640 1 --2- 172.21.15.129:0/3322775667 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbd600392a0 0x7fbd6003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:14.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.823+0000 7fbd74ff9640 1 -- 172.21.15.129:0/3322775667 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbd6806cfd0 con 0x7fbd781065c0 2024-09-15T22:00:14.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.823+0000 7fbd76ffd640 1 --2- 172.21.15.129:0/3322775667 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbd600392a0 0x7fbd6003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:14.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.824+0000 7fbd76ffd640 1 --2- 172.21.15.129:0/3322775667 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbd600392a0 0x7fbd6003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0098c0 tx=0x7fbd6c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:14.855 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:14.829+0000 7fbd74ff9640 1 -- 172.21.15.129:0/3322775667 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd680372d0 con 0x7fbd781065c0 2024-09-15T22:00:14.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:14 smithi005 ceph-mon[30944]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:15.063 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.062+0000 7fbd7dee6640 1 -- 172.21.15.129:0/3322775667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fbd781069a0 con 0x7fbd781065c0 2024-09-15T22:00:15.064 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.062+0000 7fbd74ff9640 1 -- 172.21.15.129:0/3322775667 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7fbd68031570 con 0x7fbd781065c0 2024-09-15T22:00:15.064 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T22:00:15.064 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":1,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","modified":"2024-09-15T21:58:45.697603Z","created":"2024-09-15T21:58:45.697603Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-15T22:00:15.064 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 1 2024-09-15T22:00:15.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.065+0000 7fbd7dee6640 1 -- 172.21.15.129:0/3322775667 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbd600392a0 msgr2=0x7fbd6003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:15.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.065+0000 7fbd7dee6640 1 --2- 172.21.15.129:0/3322775667 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbd600392a0 0x7fbd6003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0098c0 tx=0x7fbd6c006db0 comp rx=0 tx=0).stop 2024-09-15T22:00:15.066 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.065+0000 7fbd7dee6640 1 -- 172.21.15.129:0/3322775667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd781065c0 msgr2=0x7fbd78199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:15.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.065+0000 7fbd7dee6640 1 --2- 172.21.15.129:0/3322775667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd781065c0 0x7fbd78199310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009230 tx=0x7fbd68003d40 comp rx=0 tx=0).stop 2024-09-15T22:00:15.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.065+0000 7fbd7dee6640 1 -- 172.21.15.129:0/3322775667 shutdown_connections 2024-09-15T22:00:15.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.065+0000 7fbd7dee6640 1 --2- 172.21.15.129:0/3322775667 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbd600392a0 0x7fbd6003b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:15.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.065+0000 7fbd7dee6640 1 --2- 172.21.15.129:0/3322775667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd781065c0 0x7fbd78199310 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:15.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.065+0000 7fbd7dee6640 1 -- 172.21.15.129:0/3322775667 >> 172.21.15.129:0/3322775667 conn(0x7fbd780fb8a0 msgr2=0x7fbd780fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:15.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.065+0000 7fbd7dee6640 1 -- 172.21.15.129:0/3322775667 shutdown_connections 2024-09-15T22:00:15.067 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:15.065+0000 7fbd7dee6640 1 -- 172.21.15.129:0/3322775667 wait complete. 2024-09-15T22:00:15.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:15 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/3322775667' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:16.755 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:00:16.755 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:00:16.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:16 smithi005 ceph-mon[30944]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 podman[35283]: 2024-09-15 22:00:17.159526706 +0000 UTC m=+0.126709269 container create 43f1b4bee469bb60e422e7f59c2b956af3e55cef66ebc218d46d80baf77ea757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-15T22:00:17.681 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:00:17.789 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 podman[35283]: 2024-09-15 22:00:17.453463591 +0000 UTC m=+0.420646116 container init 43f1b4bee469bb60e422e7f59c2b956af3e55cef66ebc218d46d80baf77ea757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-15T22:00:17.790 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 podman[35283]: 2024-09-15 22:00:17.456658894 +0000 UTC m=+0.423841416 container start 43f1b4bee469bb60e422e7f59c2b956af3e55cef66ebc218d46d80baf77ea757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-15T22:00:17.790 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: set uid:gid to 167:167 (ceph:ceph) 2024-09-15T22:00:17.790 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-15T22:00:17.790 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: pidfile_write: ignore empty --pid-file 2024-09-15T22:00:17.790 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: load: jerasure load: lrc 2024-09-15T22:00:17.790 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: RocksDB version: 7.9.2 2024-09-15T22:00:17.790 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Git sha 0 2024-09-15T22:00:17.790 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-15T22:00:17.790 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: DB SUMMARY 2024-09-15T22:00:17.791 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: DB Session ID: 6S2BOFD77Q02JPBDC9QN 2024-09-15T22:00:17.791 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: CURRENT file: CURRENT 2024-09-15T22:00:17.791 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: IDENTITY file: IDENTITY 2024-09-15T22:00:17.791 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-15T22:00:17.791 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-15T22:00:17.791 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-15T22:00:17.791 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.error_if_exists: 0 2024-09-15T22:00:17.791 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.create_if_missing: 0 2024-09-15T22:00:17.791 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.paranoid_checks: 1 2024-09-15T22:00:17.791 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-15T22:00:17.791 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-15T22:00:17.792 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-15T22:00:17.792 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.env: 0x5587370ac940 2024-09-15T22:00:17.792 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.fs: PosixFileSystem 2024-09-15T22:00:17.792 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.info_log: 0x5587383856e0 2024-09-15T22:00:17.792 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_file_opening_threads: 16 2024-09-15T22:00:17.794 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.statistics: (nil) 2024-09-15T22:00:17.794 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.use_fsync: 0 2024-09-15T22:00:17.794 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_log_file_size: 0 2024-09-15T22:00:17.794 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-15T22:00:17.794 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-15T22:00:17.794 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.keep_log_file_num: 1000 2024-09-15T22:00:17.794 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.recycle_log_file_num: 0 2024-09-15T22:00:17.794 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.allow_fallocate: 1 2024-09-15T22:00:17.794 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.allow_mmap_reads: 0 2024-09-15T22:00:17.794 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.allow_mmap_writes: 0 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.use_direct_reads: 0 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.create_missing_column_families: 0 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.db_log_dir: 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.wal_dir: 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.advise_random_on_open: 1 2024-09-15T22:00:17.795 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.db_write_buffer_size: 0 2024-09-15T22:00:17.796 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.write_buffer_manager: 0x558738394dc0 2024-09-15T22:00:17.796 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-15T22:00:17.796 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-15T22:00:17.796 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-15T22:00:17.796 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.rate_limiter: (nil) 2024-09-15T22:00:17.796 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-15T22:00:17.796 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.wal_recovery_mode: 2 2024-09-15T22:00:17.796 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.enable_thread_tracking: 0 2024-09-15T22:00:17.796 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.enable_pipelined_write: 0 2024-09-15T22:00:17.796 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.unordered_write: 0 2024-09-15T22:00:17.796 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.row_cache: None 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.wal_filter: None 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.allow_ingest_behind: 0 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.two_write_queues: 0 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.manual_wal_flush: 0 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.wal_compression: 0 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.atomic_flush: 0 2024-09-15T22:00:17.797 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-15T22:00:17.798 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-15T22:00:17.798 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-15T22:00:17.798 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.log_readahead_size: 0 2024-09-15T22:00:17.798 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-15T22:00:17.798 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.best_efforts_recovery: 0 2024-09-15T22:00:17.798 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-15T22:00:17.798 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-15T22:00:17.798 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.allow_data_in_errors: 0 2024-09-15T22:00:17.798 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.db_host_id: __hostname__ 2024-09-15T22:00:17.798 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-15T22:00:17.798 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_background_jobs: 2 2024-09-15T22:00:17.799 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_background_compactions: -1 2024-09-15T22:00:17.799 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_subcompactions: 1 2024-09-15T22:00:17.799 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-15T22:00:17.799 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-15T22:00:17.799 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-15T22:00:17.799 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_total_wal_size: 0 2024-09-15T22:00:17.799 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-15T22:00:17.799 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-15T22:00:17.799 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-15T22:00:17.799 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-15T22:00:17.799 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_open_files: -1 2024-09-15T22:00:17.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bytes_per_sync: 0 2024-09-15T22:00:17.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-15T22:00:17.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-15T22:00:17.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_readahead_size: 0 2024-09-15T22:00:17.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_background_flushes: -1 2024-09-15T22:00:17.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Compression algorithms supported: 2024-09-15T22:00:17.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: kZSTD supported: 0 2024-09-15T22:00:17.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: kXpressCompression supported: 0 2024-09-15T22:00:17.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: kBZip2Compression supported: 0 2024-09-15T22:00:17.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-15T22:00:17.800 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: kLZ4Compression supported: 1 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: kZlibCompression supported: 1 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: kLZ4HCCompression supported: 1 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: kSnappyCompression supported: 1 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.merge_operator: 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_filter: None 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_filter_factory: None 2024-09-15T22:00:17.801 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.sst_partitioner_factory: None 2024-09-15T22:00:17.802 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-15T22:00:17.802 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-15T22:00:17.802 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558738385320) 2024-09-15T22:00:17.802 INFO:journalctl@ceph.mon.c.smithi129.stdout: cache_index_and_filter_blocks: 1 2024-09-15T22:00:17.802 INFO:journalctl@ceph.mon.c.smithi129.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-15T22:00:17.802 INFO:journalctl@ceph.mon.c.smithi129.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-15T22:00:17.802 INFO:journalctl@ceph.mon.c.smithi129.stdout: pin_top_level_index_and_filter: 1 2024-09-15T22:00:17.802 INFO:journalctl@ceph.mon.c.smithi129.stdout: index_type: 0 2024-09-15T22:00:17.802 INFO:journalctl@ceph.mon.c.smithi129.stdout: data_block_index_type: 0 2024-09-15T22:00:17.802 INFO:journalctl@ceph.mon.c.smithi129.stdout: index_shortening: 1 2024-09-15T22:00:17.802 INFO:journalctl@ceph.mon.c.smithi129.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-15T22:00:17.803 INFO:journalctl@ceph.mon.c.smithi129.stdout: checksum: 4 2024-09-15T22:00:17.803 INFO:journalctl@ceph.mon.c.smithi129.stdout: no_block_cache: 0 2024-09-15T22:00:17.803 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_cache: 0x5587383791f0 2024-09-15T22:00:17.803 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_cache_name: BinnedLRUCache 2024-09-15T22:00:17.803 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_cache_options: 2024-09-15T22:00:17.803 INFO:journalctl@ceph.mon.c.smithi129.stdout: capacity : 536870912 2024-09-15T22:00:17.803 INFO:journalctl@ceph.mon.c.smithi129.stdout: num_shard_bits : 4 2024-09-15T22:00:17.803 INFO:journalctl@ceph.mon.c.smithi129.stdout: strict_capacity_limit : 0 2024-09-15T22:00:17.803 INFO:journalctl@ceph.mon.c.smithi129.stdout: high_pri_pool_ratio: 0.000 2024-09-15T22:00:17.803 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_cache_compressed: (nil) 2024-09-15T22:00:17.803 INFO:journalctl@ceph.mon.c.smithi129.stdout: persistent_cache: (nil) 2024-09-15T22:00:17.804 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_size: 4096 2024-09-15T22:00:17.804 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_size_deviation: 10 2024-09-15T22:00:17.804 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_restart_interval: 16 2024-09-15T22:00:17.804 INFO:journalctl@ceph.mon.c.smithi129.stdout: index_block_restart_interval: 1 2024-09-15T22:00:17.804 INFO:journalctl@ceph.mon.c.smithi129.stdout: metadata_block_size: 4096 2024-09-15T22:00:17.804 INFO:journalctl@ceph.mon.c.smithi129.stdout: partition_filters: 0 2024-09-15T22:00:17.804 INFO:journalctl@ceph.mon.c.smithi129.stdout: use_delta_encoding: 1 2024-09-15T22:00:17.804 INFO:journalctl@ceph.mon.c.smithi129.stdout: filter_policy: bloomfilter 2024-09-15T22:00:17.804 INFO:journalctl@ceph.mon.c.smithi129.stdout: whole_key_filtering: 1 2024-09-15T22:00:17.804 INFO:journalctl@ceph.mon.c.smithi129.stdout: verify_compression: 0 2024-09-15T22:00:17.804 INFO:journalctl@ceph.mon.c.smithi129.stdout: read_amp_bytes_per_bit: 0 2024-09-15T22:00:17.805 INFO:journalctl@ceph.mon.c.smithi129.stdout: format_version: 5 2024-09-15T22:00:17.805 INFO:journalctl@ceph.mon.c.smithi129.stdout: enable_index_compression: 1 2024-09-15T22:00:17.805 INFO:journalctl@ceph.mon.c.smithi129.stdout: block_align: 0 2024-09-15T22:00:17.805 INFO:journalctl@ceph.mon.c.smithi129.stdout: max_auto_readahead_size: 262144 2024-09-15T22:00:17.805 INFO:journalctl@ceph.mon.c.smithi129.stdout: prepopulate_block_cache: 0 2024-09-15T22:00:17.805 INFO:journalctl@ceph.mon.c.smithi129.stdout: initial_auto_readahead_size: 8192 2024-09-15T22:00:17.805 INFO:journalctl@ceph.mon.c.smithi129.stdout: num_file_reads_for_auto_readahead: 2 2024-09-15T22:00:17.805 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.write_buffer_size: 33554432 2024-09-15T22:00:17.805 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_write_buffer_number: 2 2024-09-15T22:00:17.805 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compression: NoCompression 2024-09-15T22:00:17.805 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bottommost_compression: Disabled 2024-09-15T22:00:17.806 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.prefix_extractor: nullptr 2024-09-15T22:00:17.806 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-15T22:00:17.806 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.num_levels: 7 2024-09-15T22:00:17.806 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-15T22:00:17.806 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-15T22:00:17.806 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-15T22:00:17.806 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-15T22:00:17.806 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-15T22:00:17.806 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-15T22:00:17.806 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-15T22:00:17.806 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compression_opts.level: 32767 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compression_opts.strategy: 0 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-15T22:00:17.807 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compression_opts.enabled: false 2024-09-15T22:00:17.808 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-15T22:00:17.808 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-15T22:00:17.808 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-15T22:00:17.808 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-15T22:00:17.808 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.target_file_size_base: 67108864 2024-09-15T22:00:17.808 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-15T22:00:17.808 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-15T22:00:17.808 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-15T22:00:17.808 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-15T22:00:17.808 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-15T22:00:17.808 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-15T22:00:17.809 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-15T22:00:17.809 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-15T22:00:17.809 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-15T22:00:17.809 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-15T22:00:17.809 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-15T22:00:17.809 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-15T22:00:17.809 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-15T22:00:17.809 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-15T22:00:17.809 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.arena_block_size: 1048576 2024-09-15T22:00:17.809 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-15T22:00:17.809 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.disable_auto_compactions: 0 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-15T22:00:17.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.inplace_update_support: 0 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.bloom_locality: 0 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.max_successive_merges: 0 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.paranoid_file_checks: 0 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.force_consistency_checks: 1 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.report_bg_io_stats: 0 2024-09-15T22:00:17.811 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.ttl: 2592000 2024-09-15T22:00:17.812 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-15T22:00:17.812 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-15T22:00:17.812 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-15T22:00:17.812 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.enable_blob_files: false 2024-09-15T22:00:17.812 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.min_blob_size: 0 2024-09-15T22:00:17.812 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.blob_file_size: 268435456 2024-09-15T22:00:17.812 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-15T22:00:17.812 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-15T22:00:17.812 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-15T22:00:17.812 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-15T22:00:17.813 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-15T22:00:17.813 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.blob_file_starting_level: 0 2024-09-15T22:00:17.813 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-15T22:00:17.813 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-15T22:00:17.813 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-15T22:00:17.813 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 12deb50f-f1f2-4c8b-8835-38643bf11be5 2024-09-15T22:00:17.813 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437617483277, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-15T22:00:17.813 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-15T22:00:17.813 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437617483685, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726437617, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "12deb50f-f1f2-4c8b-8835-38643bf11be5", "db_session_id": "6S2BOFD77Q02JPBDC9QN", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-15T22:00:17.813 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726437617483778, "job": 1, "event": "recovery_finished"} 2024-09-15T22:00:17.813 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-15T22:00:17.814 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-15T22:00:17.814 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5587383a6e00 2024-09-15T22:00:17.814 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: DB pointer 0x5587384a2000 2024-09-15T22:00:17.814 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-15T22:00:17.814 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-15T22:00:17.814 INFO:journalctl@ceph.mon.c.smithi129.stdout: ** DB Stats ** 2024-09-15T22:00:17.814 INFO:journalctl@ceph.mon.c.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:00:17.814 INFO:journalctl@ceph.mon.c.smithi129.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-15T22:00:17.814 INFO:journalctl@ceph.mon.c.smithi129.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:00:17.814 INFO:journalctl@ceph.mon.c.smithi129.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:00:17.814 INFO:journalctl@ceph.mon.c.smithi129.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-15T22:00:17.815 INFO:journalctl@ceph.mon.c.smithi129.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-15T22:00:17.815 INFO:journalctl@ceph.mon.c.smithi129.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-15T22:00:17.815 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-09-15T22:00:17.815 INFO:journalctl@ceph.mon.c.smithi129.stdout: ** Compaction Stats [default] ** 2024-09-15T22:00:17.815 INFO:journalctl@ceph.mon.c.smithi129.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:00:17.815 INFO:journalctl@ceph.mon.c.smithi129.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-15T22:00:17.815 INFO:journalctl@ceph.mon.c.smithi129.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:00:17.815 INFO:journalctl@ceph.mon.c.smithi129.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:00:17.815 INFO:journalctl@ceph.mon.c.smithi129.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:00:17.815 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-09-15T22:00:17.815 INFO:journalctl@ceph.mon.c.smithi129.stdout: ** Compaction Stats [default] ** 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-15T22:00:17.816 INFO:journalctl@ceph.mon.c.smithi129.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout: Block cache BinnedLRUCache@0x5587383791f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: using public_addr v2:172.21.15.129:0/0 -> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: starting mon.c rank -1 at public addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] at bind addrs [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(???) e0 preinit fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:17.817 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).mds e1 new map 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout: e1 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout: btime 2024-09-15T21:58:48:668797+0000 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout: legacy client fscid: -1 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout: 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout: No filesystems configured 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-15T22:00:17.818 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: Added host smithi129 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.819 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.820 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.820 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:17.820 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:00:17.820 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/96898022' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-15T22:00:17.820 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/96898022' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-15T22:00:17.820 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:17.820 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.820 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.820 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.820 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.820 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:17.821 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:17.822 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:17.822 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:17.822 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:17.822 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:17.822 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:00:17.822 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi099:172.21.15.99=b;smithi129:172.21.15.129=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:00:17.822 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi099:172.21.15.99=b;smithi129:172.21.15.129=c;count:3 2024-09-15T22:00:17.822 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: Deploying daemon mon.c on smithi129 2024-09-15T22:00:17.822 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:17.822 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/3322775667' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:17.822 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:17.823 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 ceph-mon[35314]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-15T22:00:17.823 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 bash[35283]: 43f1b4bee469bb60e422e7f59c2b956af3e55cef66ebc218d46d80baf77ea757 2024-09-15T22:00:17.823 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:17 smithi129 systemd[1]: Started Ceph mon.c for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:00:22.551 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.547+0000 7fc0075ab640 1 -- 172.21.15.129:0/2775330390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec003530 msgr2=0x7fbfec0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:22.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.547+0000 7fc0075ab640 1 --2- 172.21.15.129:0/2775330390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec003530 0x7fbfec0059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc000102af0 tx=0x7fbff002c490 comp rx=0 tx=0).stop 2024-09-15T22:00:22.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.547+0000 7fc0075ab640 1 -- 172.21.15.129:0/2775330390 shutdown_connections 2024-09-15T22:00:22.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.547+0000 7fc0075ab640 1 --2- 172.21.15.129:0/2775330390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec003530 0x7fbfec0059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:22.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.547+0000 7fc0075ab640 1 -- 172.21.15.129:0/2775330390 >> 172.21.15.129:0/2775330390 conn(0x7fc0000fd320 msgr2=0x7fc0000ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:22.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.547+0000 7fc0075ab640 1 -- 172.21.15.129:0/2775330390 shutdown_connections 2024-09-15T22:00:22.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.547+0000 7fc0075ab640 1 -- 172.21.15.129:0/2775330390 wait complete. 2024-09-15T22:00:22.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.548+0000 7fc0075ab640 1 Processor -- start 2024-09-15T22:00:22.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.548+0000 7fc0075ab640 1 -- start start 2024-09-15T22:00:22.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc0075ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec003530 0x7fc0001972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:22.552 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc0075ab640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0001977e0 0x7fc00019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc0075ab640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc000109780 con 0x7fbfec003530 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc0075ab640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc000109600 con 0x7fc0001977e0 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc004b46640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0001977e0 0x7fc00019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc004b46640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0001977e0 0x7fc00019bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52794/0 (socket says 172.21.15.129:52794) 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc005347640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec003530 0x7fc0001972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc004b46640 1 -- 172.21.15.129:0/4294935711 learned_addr learned my addr 172.21.15.129:0/4294935711 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc004b46640 1 -- 172.21.15.129:0/4294935711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0001977e0 msgr2=0x7fc00019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc004b46640 1 -- 172.21.15.129:0/4294935711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0001977e0 msgr2=0x7fc00019bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc004b46640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0001977e0 0x7fc00019bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc004b46640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0001977e0 0x7fc00019bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc005347640 1 -- 172.21.15.129:0/4294935711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0001977e0 msgr2=0x7fc00019bb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:00:22.553 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc005347640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0001977e0 0x7fc00019bb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:22.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc005347640 1 -- 172.21.15.129:0/4294935711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc000197e00 con 0x7fbfec003530 2024-09-15T22:00:22.554 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.549+0000 7fc005347640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec003530 0x7fc0001972a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbff002ca80 tx=0x7fbff00339a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:22.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.550+0000 7fbff67fc640 1 -- 172.21.15.129:0/4294935711 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbff00329e0 con 0x7fbfec003530 2024-09-15T22:00:22.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.550+0000 7fc0075ab640 1 -- 172.21.15.129:0/4294935711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00019c0b0 con 0x7fbfec003530 2024-09-15T22:00:22.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.550+0000 7fbff67fc640 1 -- 172.21.15.129:0/4294935711 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff0005520 con 0x7fbfec003530 2024-09-15T22:00:22.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.550+0000 7fbff67fc640 1 -- 172.21.15.129:0/4294935711 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fbff0004bc0 con 0x7fbfec003530 2024-09-15T22:00:22.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.550+0000 7fc0075ab640 1 -- 172.21.15.129:0/4294935711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00019c5c0 con 0x7fbfec003530 2024-09-15T22:00:22.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.551+0000 7fbff67fc640 1 -- 172.21.15.129:0/4294935711 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fbff0043020 con 0x7fbfec003530 2024-09-15T22:00:22.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.551+0000 7fc0075ab640 1 -- 172.21.15.129:0/4294935711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc000102da0 con 0x7fbfec003530 2024-09-15T22:00:22.555 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.551+0000 7fbff67fc640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfe4039040 0x7fbfe403b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:22.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.551+0000 7fbff67fc640 1 -- 172.21.15.129:0/4294935711 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbff0041020 con 0x7fbfec003530 2024-09-15T22:00:22.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.551+0000 7fc004b46640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfe4039040 0x7fbfe403b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:22.556 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.552+0000 7fc004b46640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfe4039040 0x7fbfe403b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009100 tx=0x7fbffc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:22.558 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.556+0000 7fbff67fc640 1 -- 172.21.15.129:0/4294935711 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbff004a020 con 0x7fbfec003530 2024-09-15T22:00:22.790 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.788+0000 7fc0075ab640 1 -- 172.21.15.129:0/4294935711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc000190f60 con 0x7fbfec003530 2024-09-15T22:00:22.791 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.788+0000 7fbff67fc640 1 -- 172.21.15.129:0/4294935711 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fbff00491b0 con 0x7fbfec003530 2024-09-15T22:00:22.791 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T22:00:22.791 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":2,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","modified":"2024-09-15T22:00:17.509153Z","created":"2024-09-15T21:58:45.697603Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T22:00:22.791 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 2 2024-09-15T22:00:22.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.790+0000 7fc0075ab640 1 -- 172.21.15.129:0/4294935711 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfe4039040 msgr2=0x7fbfe403b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:22.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.790+0000 7fc0075ab640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfe4039040 0x7fbfe403b500 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009100 tx=0x7fbffc008040 comp rx=0 tx=0).stop 2024-09-15T22:00:22.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.790+0000 7fc0075ab640 1 -- 172.21.15.129:0/4294935711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec003530 msgr2=0x7fc0001972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:22.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.790+0000 7fc0075ab640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec003530 0x7fc0001972a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbff002ca80 tx=0x7fbff00339a0 comp rx=0 tx=0).stop 2024-09-15T22:00:22.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.790+0000 7fc0075ab640 1 -- 172.21.15.129:0/4294935711 shutdown_connections 2024-09-15T22:00:22.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.790+0000 7fc0075ab640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0001977e0 0x7fc00019bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:22.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.790+0000 7fc0075ab640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfe4039040 0x7fbfe403b500 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:22.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.790+0000 7fc0075ab640 1 --2- 172.21.15.129:0/4294935711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfec003530 0x7fc0001972a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:22.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.790+0000 7fc0075ab640 1 -- 172.21.15.129:0/4294935711 >> 172.21.15.129:0/4294935711 conn(0x7fc0000fd320 msgr2=0x7fc0000ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:22.793 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.790+0000 7fc0075ab640 1 -- 172.21.15.129:0/4294935711 shutdown_connections 2024-09-15T22:00:22.794 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:22.790+0000 7fc0075ab640 1 -- 172.21.15.129:0/4294935711 wait complete. 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: mon.a calling monitor election 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: mon.c calling monitor election 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.803 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: monmap epoch 2 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: last_changed 2024-09-15T22:00:17.509153+0000 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: created 2024-09-15T21:58:45.697603+0000 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: min_mon_release 19 (squid) 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: election_strategy: 1 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: fsmap 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: mgrmap e13: x(active, since 39s) 2024-09-15T22:00:22.804 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: overall HEALTH_OK 2024-09-15T22:00:22.805 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:22.805 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:22.805 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:22.805 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:22.805 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:22.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:22.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:00:22.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: mon.a calling monitor election 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: mon.c calling monitor election 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: monmap epoch 2 2024-09-15T22:00:22.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: last_changed 2024-09-15T22:00:17.509153+0000 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: created 2024-09-15T21:58:45.697603+0000 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: min_mon_release 19 (squid) 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: election_strategy: 1 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: fsmap 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: mgrmap e13: x(active, since 39s) 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: overall HEALTH_OK 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:22.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:22.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:23.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:23 smithi129 ceph-mon[35314]: Deploying daemon mon.b on smithi099 2024-09-15T22:00:23.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:23 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/4294935711' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:23.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:23 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:23.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:23 smithi005 ceph-mon[30944]: Deploying daemon mon.b on smithi099 2024-09-15T22:00:23.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:23 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/4294935711' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:23.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:23 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:24.571 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:00:24.571 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:00:24.824 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:00:24.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:24 smithi129 ceph-mon[35314]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:24.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:24 smithi005 ceph-mon[30944]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:25.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.227+0000 7fa7f506d640 1 -- 172.21.15.129:0/3440269000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f0107660 msgr2=0x7fa7f0107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:25.232 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.227+0000 7fa7f506d640 1 --2- 172.21.15.129:0/3440269000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f0107660 0x7fa7f0107a40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0009880 tx=0x7fa7e002a5d0 comp rx=0 tx=0).stop 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.227+0000 7fa7f506d640 1 -- 172.21.15.129:0/3440269000 shutdown_connections 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.227+0000 7fa7f506d640 1 --2- 172.21.15.129:0/3440269000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f0107660 0x7fa7f0107a40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.227+0000 7fa7f506d640 1 -- 172.21.15.129:0/3440269000 >> 172.21.15.129:0/3440269000 conn(0x7fa7f00fd320 msgr2=0x7fa7f00ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.227+0000 7fa7f506d640 1 -- 172.21.15.129:0/3440269000 shutdown_connections 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.227+0000 7fa7f506d640 1 -- 172.21.15.129:0/3440269000 wait complete. 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.228+0000 7fa7f506d640 1 Processor -- start 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.228+0000 7fa7f506d640 1 -- start start 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.228+0000 7fa7f506d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f0107660 0x7fa7f01995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.228+0000 7fa7f506d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0199b20 0x7fa7f019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.228+0000 7fa7f506d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010bd20 con 0x7fa7f0107660 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.228+0000 7fa7f506d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f010bba0 con 0x7fa7f0199b20 2024-09-15T22:00:25.233 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.228+0000 7fa7eed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f0107660 0x7fa7f01995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.228+0000 7fa7ee59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0199b20 0x7fa7f019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.228+0000 7fa7ee59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0199b20 0x7fa7f019deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:52806/0 (socket says 172.21.15.129:52806) 2024-09-15T22:00:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7ee59c640 1 -- 172.21.15.129:0/2047962914 learned_addr learned my addr 172.21.15.129:0/2047962914 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-15T22:00:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7ee59c640 1 -- 172.21.15.129:0/2047962914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0199b20 msgr2=0x7fa7f019deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-15T22:00:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7ee59c640 1 -- 172.21.15.129:0/2047962914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0199b20 msgr2=0x7fa7f019deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:00:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7ee59c640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0199b20 0x7fa7f019deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:00:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7ee59c640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0199b20 0x7fa7f019deb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:00:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7eed9d640 1 -- 172.21.15.129:0/2047962914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0199b20 msgr2=0x7fa7f019deb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:00:25.234 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7eed9d640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0199b20 0x7fa7f019deb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:25.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7eed9d640 1 -- 172.21.15.129:0/2047962914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7f019a140 con 0x7fa7f0107660 2024-09-15T22:00:25.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7eed9d640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f0107660 0x7fa7f01995e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0006010 tx=0x7fa7e0032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:25.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7d7fff640 1 -- 172.21.15.129:0/2047962914 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa7e0035070 con 0x7fa7f0107660 2024-09-15T22:00:25.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7f506d640 1 -- 172.21.15.129:0/2047962914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7f019e3f0 con 0x7fa7f0107660 2024-09-15T22:00:25.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7d7fff640 1 -- 172.21.15.129:0/2047962914 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7e0032e40 con 0x7fa7f0107660 2024-09-15T22:00:25.235 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7d7fff640 1 -- 172.21.15.129:0/2047962914 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa7e003bb50 con 0x7fa7f0107660 2024-09-15T22:00:25.236 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.229+0000 7fa7f506d640 1 -- 172.21.15.129:0/2047962914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7f019e930 con 0x7fa7f0107660 2024-09-15T22:00:25.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.231+0000 7fa7d7fff640 1 -- 172.21.15.129:0/2047962914 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fa7e0042030 con 0x7fa7f0107660 2024-09-15T22:00:25.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.231+0000 7fa7f506d640 1 -- 172.21.15.129:0/2047962914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7f0102da0 con 0x7fa7f0107660 2024-09-15T22:00:25.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.231+0000 7fa7d7fff640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7d8039040 0x7fa7d803b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:25.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.231+0000 7fa7d7fff640 1 -- 172.21.15.129:0/2047962914 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa7e0040070 con 0x7fa7f0107660 2024-09-15T22:00:25.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.231+0000 7fa7ee59c640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7d8039040 0x7fa7d803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:25.237 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.232+0000 7fa7ee59c640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7d8039040 0x7fa7d803b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009100 tx=0x7fa7e4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:25.238 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.236+0000 7fa7d7fff640 1 -- 172.21.15.129:0/2047962914 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7e003ac70 con 0x7fa7f0107660 2024-09-15T22:00:25.464 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.461+0000 7fa7f506d640 1 -- 172.21.15.129:0/2047962914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa7f019eca0 con 0x7fa7f0107660 2024-09-15T22:00:25.465 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.462+0000 7fa7d7fff640 1 -- 172.21.15.129:0/2047962914 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fa7e003b460 con 0x7fa7f0107660 2024-09-15T22:00:25.465 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T22:00:25.465 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":2,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","modified":"2024-09-15T22:00:17.509153Z","created":"2024-09-15T21:58:45.697603Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T22:00:25.465 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 2 2024-09-15T22:00:25.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.464+0000 7fa7f506d640 1 -- 172.21.15.129:0/2047962914 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7d8039040 msgr2=0x7fa7d803b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:25.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.464+0000 7fa7f506d640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7d8039040 0x7fa7d803b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa7e4009100 tx=0x7fa7e4008040 comp rx=0 tx=0).stop 2024-09-15T22:00:25.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.464+0000 7fa7f506d640 1 -- 172.21.15.129:0/2047962914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f0107660 msgr2=0x7fa7f01995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:25.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.464+0000 7fa7f506d640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f0107660 0x7fa7f01995e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0006010 tx=0x7fa7e0032c50 comp rx=0 tx=0).stop 2024-09-15T22:00:25.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.464+0000 7fa7f506d640 1 -- 172.21.15.129:0/2047962914 shutdown_connections 2024-09-15T22:00:25.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.464+0000 7fa7f506d640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7f0199b20 0x7fa7f019deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:25.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.464+0000 7fa7f506d640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7d8039040 0x7fa7d803b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:25.467 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.464+0000 7fa7f506d640 1 --2- 172.21.15.129:0/2047962914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7f0107660 0x7fa7f01995e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:25.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.464+0000 7fa7f506d640 1 -- 172.21.15.129:0/2047962914 >> 172.21.15.129:0/2047962914 conn(0x7fa7f00fd320 msgr2=0x7fa7f0074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:25.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.464+0000 7fa7f506d640 1 -- 172.21.15.129:0/2047962914 shutdown_connections 2024-09-15T22:00:25.468 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:25.464+0000 7fa7f506d640 1 -- 172.21.15.129:0/2047962914 wait complete. 2024-09-15T22:00:25.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:25 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/2047962914' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:25.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:25 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/2047962914' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:26.151 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(???) e0 preinit fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:26.151 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).mds e1 new map 2024-09-15T22:00:26.151 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-15T22:00:26.151 INFO:journalctl@ceph.mon.b.smithi099.stdout: e1 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout: btime 2024-09-15T21:58:48:668797+0000 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout: legacy client fscid: -1 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout: 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout: No filesystems configured 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-15T22:00:26.152 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: Added host smithi129 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:00:26.153 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/96898022' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-15T22:00:26.154 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/96898022' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-15T22:00:26.154 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:26.154 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.154 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.154 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.154 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.154 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi129", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:00:26.154 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:26.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:26.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:26.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-15T22:00:26.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:26.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:00:26.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:26.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:26.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:26.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi099:172.21.15.99=b;smithi129:172.21.15.129=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi099:172.21.15.99=b;smithi129:172.21.15.129=c;count:3 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: Deploying daemon mon.c on smithi129 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/3322775667' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:26.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:00:26.594 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:26.594 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.a calling monitor election 2024-09-15T22:00:26.594 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:26.594 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:26.594 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:26.594 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.c calling monitor election 2024-09-15T22:00:26.595 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:26.595 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:26.595 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:26.595 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:26.595 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:00:26.595 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: monmap epoch 2 2024-09-15T22:00:26.596 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:26.596 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: last_changed 2024-09-15T22:00:17.509153+0000 2024-09-15T22:00:26.596 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: created 2024-09-15T21:58:45.697603+0000 2024-09-15T22:00:26.596 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: min_mon_release 19 (squid) 2024-09-15T22:00:26.596 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: election_strategy: 1 2024-09-15T22:00:26.596 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-15T22:00:26.596 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-09-15T22:00:26.596 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: fsmap 2024-09-15T22:00:26.596 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mgrmap e13: x(active, since 39s) 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: overall HEALTH_OK 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: Deploying daemon mon.b on smithi099 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/4294935711' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/2047962914' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 ceph-mon[34892]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-15T22:00:26.597 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 bash[34873]: 4f087ea6937a3f4ba6b1bffaff90fe2c4bd09aebd0487f60adb962b479feeda8 2024-09-15T22:00:26.598 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:26 smithi099 systemd[1]: Started Ceph mon.b for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:00:27.139 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-15T22:00:27.140 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-15T22:00:27.404 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:00:31.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.365+0000 7f922af79640 1 -- 172.21.15.129:0/2620827510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9214003530 msgr2=0x7f92140059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:31.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.365+0000 7f922af79640 1 --2- 172.21.15.129:0/2620827510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9214003530 0x7f92140059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f922406aa60 tx=0x7f921802c3f0 comp rx=0 tx=0).stop 2024-09-15T22:00:31.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.365+0000 7f922af79640 1 -- 172.21.15.129:0/2620827510 shutdown_connections 2024-09-15T22:00:31.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.365+0000 7f922af79640 1 --2- 172.21.15.129:0/2620827510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9214003530 0x7f92140059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:31.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.365+0000 7f922af79640 1 -- 172.21.15.129:0/2620827510 >> 172.21.15.129:0/2620827510 conn(0x7f92240fc220 msgr2=0x7f92240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:31.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.365+0000 7f922af79640 1 -- 172.21.15.129:0/2620827510 shutdown_connections 2024-09-15T22:00:31.372 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.365+0000 7f922af79640 1 -- 172.21.15.129:0/2620827510 wait complete. 2024-09-15T22:00:31.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.366+0000 7f922af79640 1 Processor -- start 2024-09-15T22:00:31.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.366+0000 7f922af79640 1 -- start start 2024-09-15T22:00:31.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.366+0000 7f922af79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9214003530 0x7f9224199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:31.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.366+0000 7f922af79640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9224199ad0 0x7f922419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:31.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.366+0000 7f922af79640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f922419e3a0 0x7f922419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:31.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.366+0000 7f922af79640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9224109780 con 0x7f922419e3a0 2024-09-15T22:00:31.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.366+0000 7f922af79640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9224109600 con 0x7f9214003530 2024-09-15T22:00:31.373 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.366+0000 7f922af79640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9224109900 con 0x7f9224199ad0 2024-09-15T22:00:31.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9223fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9224199ad0 0x7f922419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:31.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9223fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9224199ad0 0x7f922419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:60074/0 (socket says 172.21.15.129:60074) 2024-09-15T22:00:31.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9223fff640 1 -- 172.21.15.129:0/1668759795 learned_addr learned my addr 172.21.15.129:0/1668759795 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-15T22:00:31.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9228d15640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9214003530 0x7f9224199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:31.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9229516640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f922419e3a0 0x7f922419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:31.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9223fff640 1 -- 172.21.15.129:0/1668759795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9224199ad0 msgr2=0x7f922419de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-15T22:00:31.374 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9223fff640 1 -- 172.21.15.129:0/1668759795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9224199ad0 msgr2=0x7f922419de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:00:31.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9223fff640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9224199ad0 0x7f922419de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:00:31.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9228d15640 1 -- 172.21.15.129:0/1668759795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9214003530 msgr2=0x7f9224199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-15T22:00:31.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9228d15640 1 -- 172.21.15.129:0/1668759795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9214003530 msgr2=0x7f9224199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:00:31.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9228d15640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9214003530 0x7f9224199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:00:31.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9228d15640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9214003530 0x7f9224199590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:00:31.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9223fff640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9224199ad0 0x7f922419de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:00:31.375 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9229516640 1 -- 172.21.15.129:0/1668759795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9224199ad0 msgr2=0x7f922419de60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:00:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9229516640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9224199ad0 0x7f922419de60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9229516640 1 -- 172.21.15.129:0/1668759795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9214003530 msgr2=0x7f9224199590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:00:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9229516640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9214003530 0x7f9224199590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9229516640 1 -- 172.21.15.129:0/1668759795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92241a6110 con 0x7f922419e3a0 2024-09-15T22:00:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.367+0000 7f9229516640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f922419e3a0 0x7f922419e850 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f921c00bd50 tx=0x7f921c00be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.368+0000 7f9221ffb640 1 -- 172.21.15.129:0/1668759795 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c019070 con 0x7f922419e3a0 2024-09-15T22:00:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.368+0000 7f9221ffb640 1 -- 172.21.15.129:0/1668759795 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f921c014920 con 0x7f922419e3a0 2024-09-15T22:00:31.376 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.368+0000 7f9221ffb640 1 -- 172.21.15.129:0/1668759795 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c012990 con 0x7f922419e3a0 2024-09-15T22:00:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.368+0000 7f922af79640 1 -- 172.21.15.129:0/1668759795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92241a6340 con 0x7f922419e3a0 2024-09-15T22:00:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.368+0000 7f922af79640 1 -- 172.21.15.129:0/1668759795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92241a6850 con 0x7f922419e3a0 2024-09-15T22:00:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.369+0000 7f9221ffb640 1 -- 172.21.15.129:0/1668759795 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f921c01c070 con 0x7f922419e3a0 2024-09-15T22:00:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.369+0000 7f9221ffb640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9208039730 0x7f920803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.369+0000 7f9221ffb640 1 -- 172.21.15.129:0/1668759795 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f921c04c1e0 con 0x7f922419e3a0 2024-09-15T22:00:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.369+0000 7f922af79640 1 -- 172.21.15.129:0/1668759795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f922410a890 con 0x7f922419e3a0 2024-09-15T22:00:31.377 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.369+0000 7f9228d15640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9208039730 0x7f920803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:31.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.370+0000 7f9228d15640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9208039730 0x7f920803bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9218014cc0 tx=0x7f9218005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:31.378 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.374+0000 7f9221ffb640 1 -- 172.21.15.129:0/1668759795 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f921c011df0 con 0x7f922419e3a0 2024-09-15T22:00:31.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: mon.c calling monitor election 2024-09-15T22:00:31.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:00:31.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: mon.a calling monitor election 2024-09-15T22:00:31.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: monmap epoch 3 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: last_changed 2024-09-15T22:00:26.154916+0000 2024-09-15T22:00:31.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: created 2024-09-15T21:58:45.697603+0000 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: min_mon_release 19 (squid) 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: election_strategy: 1 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: fsmap 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: mgrmap e13: x(active, since 48s) 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: overall HEALTH_OK 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:31.597 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:31.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: mon.c calling monitor election 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: mon.a calling monitor election 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: monmap epoch 3 2024-09-15T22:00:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: last_changed 2024-09-15T22:00:26.154916+0000 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: created 2024-09-15T21:58:45.697603+0000 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: min_mon_release 19 (squid) 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: election_strategy: 1 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: fsmap 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: mgrmap e13: x(active, since 48s) 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: overall HEALTH_OK 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:31.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:31.609 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:31.609 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:31.619 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.617+0000 7f922af79640 1 -- 172.21.15.129:0/1668759795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9224106d30 con 0x7f922419e3a0 2024-09-15T22:00:31.620 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.618+0000 7f9221ffb640 1 -- 172.21.15.129:0/1668759795 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7f921c013ba0 con 0x7f922419e3a0 2024-09-15T22:00:31.621 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T22:00:31.621 INFO:teuthology.orchestra.run.smithi129.stdout:{"epoch":3,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","modified":"2024-09-15T22:00:26.154916Z","created":"2024-09-15T21:58:45.697603Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:3300","nonce":0},{"type":"v1","addr":"172.21.15.129:6789","nonce":0}]},"addr":"172.21.15.129:6789/0","public_addr":"172.21.15.129:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:3300","nonce":0},{"type":"v1","addr":"172.21.15.99:6789","nonce":0}]},"addr":"172.21.15.99:6789/0","public_addr":"172.21.15.99:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-15T22:00:31.621 INFO:teuthology.orchestra.run.smithi129.stderr:dumped monmap epoch 3 2024-09-15T22:00:31.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 -- 172.21.15.129:0/1668759795 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9208039730 msgr2=0x7f920803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:31.622 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9208039730 0x7f920803bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9218014cc0 tx=0x7f9218005f60 comp rx=0 tx=0).stop 2024-09-15T22:00:31.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 -- 172.21.15.129:0/1668759795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f922419e3a0 msgr2=0x7f922419e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:31.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f922419e3a0 0x7f922419e850 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f921c00bd50 tx=0x7f921c00be40 comp rx=0 tx=0).stop 2024-09-15T22:00:31.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 -- 172.21.15.129:0/1668759795 shutdown_connections 2024-09-15T22:00:31.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9208039730 0x7f920803bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:31.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f922419e3a0 0x7f922419e850 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:31.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9224199ad0 0x7f922419de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:31.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 --2- 172.21.15.129:0/1668759795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9214003530 0x7f9224199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:31.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 -- 172.21.15.129:0/1668759795 >> 172.21.15.129:0/1668759795 conn(0x7f92240fc220 msgr2=0x7f92240fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:31.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 -- 172.21.15.129:0/1668759795 shutdown_connections 2024-09-15T22:00:31.623 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:31.620+0000 7f922af79640 1 -- 172.21.15.129:0/1668759795 wait complete. 2024-09-15T22:00:32.218 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-15T22:00:32.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-15T22:00:32.485 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:00:32.501 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:32 smithi129 ceph-mon[35314]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:32.501 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:32.501 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:32 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/1668759795' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:32.501 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:32.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:32 smithi005 ceph-mon[30944]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:32.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:32.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:32 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/1668759795' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:32.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.114+0000 7efe30e19640 1 -- 172.21.15.5:0/1264846204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2c106540 msgr2=0x7efe2c106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:33.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.114+0000 7efe30e19640 1 --2- 172.21.15.5:0/1264846204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2c106540 0x7efe2c106920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009810 tx=0x7efe1c02a540 comp rx=0 tx=0).stop 2024-09-15T22:00:33.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.114+0000 7efe30e19640 1 -- 172.21.15.5:0/1264846204 shutdown_connections 2024-09-15T22:00:33.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.114+0000 7efe30e19640 1 --2- 172.21.15.5:0/1264846204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2c106540 0x7efe2c106920 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:33.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.114+0000 7efe30e19640 1 -- 172.21.15.5:0/1264846204 >> 172.21.15.5:0/1264846204 conn(0x7efe2c077310 msgr2=0x7efe2c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:33.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.114+0000 7efe30e19640 1 -- 172.21.15.5:0/1264846204 shutdown_connections 2024-09-15T22:00:33.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.115+0000 7efe30e19640 1 -- 172.21.15.5:0/1264846204 wait complete. 2024-09-15T22:00:33.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.115+0000 7efe30e19640 1 Processor -- start 2024-09-15T22:00:33.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.115+0000 7efe30e19640 1 -- start start 2024-09-15T22:00:33.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe30e19640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2c106540 0x7efe2c109ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe30e19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe2c10a420 0x7efe2c10a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe30e19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe2c1a3510 0x7efe2c1a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe30e19640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c06a410 con 0x7efe2c106540 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe30e19640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c06a290 con 0x7efe2c1a3510 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe30e19640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efe2c06a590 con 0x7efe2c10a420 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2c106540 0x7efe2c109ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2c106540 0x7efe2c109ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45902/0 (socket says 172.21.15.5:45902) 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2a59c640 1 -- 172.21.15.5:0/3602866682 learned_addr learned my addr 172.21.15.5:0/3602866682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe29d9b640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe2c10a420 0x7efe2c10a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2ad9d640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe2c1a3510 0x7efe2c1a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2ad9d640 1 -- 172.21.15.5:0/3602866682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe2c1a3510 msgr2=0x7efe2c1a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2ad9d640 1 -- 172.21.15.5:0/3602866682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe2c1a3510 msgr2=0x7efe2c1a5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T22:00:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2ad9d640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe2c1a3510 0x7efe2c1a5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2a59c640 1 -- 172.21.15.5:0/3602866682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe2c10a420 msgr2=0x7efe2c10a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2a59c640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe2c10a420 0x7efe2c10a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2ad9d640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe2c1a3510 0x7efe2c1a5900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2a59c640 1 -- 172.21.15.5:0/3602866682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe2c1a3510 msgr2=0x7efe2c1a5900 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2a59c640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe2c1a3510 0x7efe2c1a5900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.116+0000 7efe2a59c640 1 -- 172.21.15.5:0/3602866682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2c1a5f70 con 0x7efe2c106540 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.117+0000 7efe2a59c640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2c106540 0x7efe2c109ee0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005040 tx=0x7efe1c005070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.117+0000 7efe1b7fe640 1 -- 172.21.15.5:0/3602866682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c040070 con 0x7efe2c106540 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.117+0000 7efe1b7fe640 1 -- 172.21.15.5:0/3602866682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe1c0056a0 con 0x7efe2c106540 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.117+0000 7efe30e19640 1 -- 172.21.15.5:0/3602866682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe2c1a6200 con 0x7efe2c106540 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.117+0000 7efe1b7fe640 1 -- 172.21.15.5:0/3602866682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1c045460 con 0x7efe2c106540 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.117+0000 7efe30e19640 1 -- 172.21.15.5:0/3602866682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe2c1a6730 con 0x7efe2c106540 2024-09-15T22:00:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.118+0000 7efe1b7fe640 1 -- 172.21.15.5:0/3602866682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7efe1c039040 con 0x7efe2c106540 2024-09-15T22:00:33.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.118+0000 7efe30e19640 1 -- 172.21.15.5:0/3602866682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe2c101bb0 con 0x7efe2c106540 2024-09-15T22:00:33.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.118+0000 7efe1b7fe640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efe14039760 0x7efe1403bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:33.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.118+0000 7efe1b7fe640 1 -- 172.21.15.5:0/3602866682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe1c06d7e0 con 0x7efe2c106540 2024-09-15T22:00:33.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.118+0000 7efe29d9b640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efe14039760 0x7efe1403bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:33.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.119+0000 7efe29d9b640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efe14039760 0x7efe1403bc20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efe20009a30 tx=0x7efe20009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:33.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.123+0000 7efe1b7fe640 1 -- 172.21.15.5:0/3602866682 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe1c03a3b0 con 0x7efe2c106540 2024-09-15T22:00:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.310+0000 7efe30e19640 1 -- 172.21.15.5:0/3602866682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efe2c1a6990 con 0x7efe2c106540 2024-09-15T22:00:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.311+0000 7efe1b7fe640 1 -- 172.21.15.5:0/3602866682 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7efe1c042dd0 con 0x7efe2c106540 2024-09-15T22:00:33.313 INFO:teuthology.orchestra.run.smithi005.stdout:# minimal ceph.conf for 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:33.314 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-09-15T22:00:33.314 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:33.314 INFO:teuthology.orchestra.run.smithi005.stdout: mon_host = [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.313+0000 7efe30e19640 1 -- 172.21.15.5:0/3602866682 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efe14039760 msgr2=0x7efe1403bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.313+0000 7efe30e19640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efe14039760 0x7efe1403bc20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efe20009a30 tx=0x7efe20009190 comp rx=0 tx=0).stop 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.313+0000 7efe30e19640 1 -- 172.21.15.5:0/3602866682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2c106540 msgr2=0x7efe2c109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.313+0000 7efe30e19640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2c106540 0x7efe2c109ee0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005040 tx=0x7efe1c005070 comp rx=0 tx=0).stop 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.313+0000 7efe30e19640 1 -- 172.21.15.5:0/3602866682 shutdown_connections 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.313+0000 7efe30e19640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe2c1a3510 0x7efe2c1a5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.313+0000 7efe30e19640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe2c10a420 0x7efe2c10a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.313+0000 7efe30e19640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efe14039760 0x7efe1403bc20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.313+0000 7efe30e19640 1 --2- 172.21.15.5:0/3602866682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe2c106540 0x7efe2c109ee0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.313+0000 7efe30e19640 1 -- 172.21.15.5:0/3602866682 >> 172.21.15.5:0/3602866682 conn(0x7efe2c077310 msgr2=0x7efe2c0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.313+0000 7efe30e19640 1 -- 172.21.15.5:0/3602866682 shutdown_connections 2024-09-15T22:00:33.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:33.314+0000 7efe30e19640 1 -- 172.21.15.5:0/3602866682 wait complete. 2024-09-15T22:00:33.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: mon.c calling monitor election 2024-09-15T22:00:33.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:00:33.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: mon.a calling monitor election 2024-09-15T22:00:33.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:33.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: monmap epoch 3 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: last_changed 2024-09-15T22:00:26.154916+0000 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: created 2024-09-15T21:58:45.697603+0000 2024-09-15T22:00:33.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: min_mon_release 19 (squid) 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: election_strategy: 1 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: fsmap 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: mgrmap e13: x(active, since 48s) 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: overall HEALTH_OK 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:33.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:33.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/1668759795' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-15T22:00:33.593 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:33.962 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-15T22:00:33.962 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T22:00:33.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:00:33.991 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T22:00:33.991 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:00:34.057 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-15T22:00:34.057 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:00:34.086 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-15T22:00:34.087 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:00:34.152 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-15T22:00:34.152 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.conf 2024-09-15T22:00:34.181 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-15T22:00:34.181 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:00:34.244 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: mon.b calling monitor election 2024-09-15T22:00:34.244 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:34.244 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-15T22:00:34.244 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-15T22:00:34.244 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-15T22:00:34.244 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: mon.b calling monitor election 2024-09-15T22:00:34.244 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: mon.c calling monitor election 2024-09-15T22:00:34.244 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: mon.a calling monitor election 2024-09-15T22:00:34.244 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:00:34.244 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: monmap epoch 3 2024-09-15T22:00:34.244 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: last_changed 2024-09-15T22:00:26.154916+0000 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: created 2024-09-15T21:58:45.697603+0000 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: min_mon_release 19 (squid) 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: election_strategy: 1 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: fsmap 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: mgrmap e13: x(active, since 50s) 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: overall HEALTH_OK 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: Updating smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:34.245 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: Updating smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3602866682' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:00:34.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:34.247 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:00:34.247 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:34.247 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:33 smithi129 ceph-mon[35314]: Reconfiguring daemon mon.a on smithi005 2024-09-15T22:00:34.248 INFO:tasks.cephadm:Adding mgr.x on smithi005 2024-09-15T22:00:34.248 INFO:tasks.cephadm:Adding mgr.y on smithi099 2024-09-15T22:00:34.248 INFO:tasks.cephadm:Adding mgr.z on smithi129 2024-09-15T22:00:34.248 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi005=x;smithi099=y;smithi129=z' 2024-09-15T22:00:34.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: mon.b calling monitor election 2024-09-15T22:00:34.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:34.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-15T22:00:34.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-15T22:00:34.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-15T22:00:34.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: mon.b calling monitor election 2024-09-15T22:00:34.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: mon.c calling monitor election 2024-09-15T22:00:34.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: mon.a calling monitor election 2024-09-15T22:00:34.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:00:34.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: monmap epoch 3 2024-09-15T22:00:34.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:34.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: last_changed 2024-09-15T22:00:26.154916+0000 2024-09-15T22:00:34.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: created 2024-09-15T21:58:45.697603+0000 2024-09-15T22:00:34.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: min_mon_release 19 (squid) 2024-09-15T22:00:34.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: election_strategy: 1 2024-09-15T22:00:34.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-15T22:00:34.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-09-15T22:00:34.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-15T22:00:34.266 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: fsmap 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: mgrmap e13: x(active, since 50s) 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: overall HEALTH_OK 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: Updating smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: Updating smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3602866682' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.268 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.268 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.268 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.268 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.268 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:00:34.268 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:34.268 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:00:34.268 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:34.268 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:33 smithi005 ceph-mon[30944]: Reconfiguring daemon mon.a on smithi005 2024-09-15T22:00:34.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: mon.b calling monitor election 2024-09-15T22:00:34.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:34.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-15T22:00:34.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-15T22:00:34.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-15T22:00:34.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: mon.b calling monitor election 2024-09-15T22:00:34.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: mon.c calling monitor election 2024-09-15T22:00:34.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: mon.a calling monitor election 2024-09-15T22:00:34.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-15T22:00:34.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: monmap epoch 3 2024-09-15T22:00:34.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: last_changed 2024-09-15T22:00:26.154916+0000 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: created 2024-09-15T21:58:45.697603+0000 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: min_mon_release 19 (squid) 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: election_strategy: 1 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: 1: [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] mon.c 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: 2: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: fsmap 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: osdmap e4: 0 total, 0 up, 0 in 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: mgrmap e13: x(active, since 50s) 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: overall HEALTH_OK 2024-09-15T22:00:34.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: Updating smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: Updating smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3602866682' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.344 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:34.344 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: Reconfiguring mon.a (unknown last config time)... 2024-09-15T22:00:34.344 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:34.344 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:00:34.344 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:34.344 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:33 smithi099 ceph-mon[34892]: Reconfiguring daemon mon.a on smithi005 2024-09-15T22:00:34.523 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:00:34.922 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.918+0000 7fe7232d6640 1 -- 172.21.15.129:0/291628712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c076220 msgr2=0x7fe71c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:34.923 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.918+0000 7fe7232d6640 1 --2- 172.21.15.129:0/291628712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c076220 0x7fe71c074680 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe718009880 tx=0x7fe71802a5d0 comp rx=0 tx=0).stop 2024-09-15T22:00:34.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.919+0000 7fe7232d6640 1 -- 172.21.15.129:0/291628712 shutdown_connections 2024-09-15T22:00:34.924 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.919+0000 7fe7232d6640 1 --2- 172.21.15.129:0/291628712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c076220 0x7fe71c074680 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.919+0000 7fe7232d6640 1 -- 172.21.15.129:0/291628712 >> 172.21.15.129:0/291628712 conn(0x7fe71c0fce90 msgr2=0x7fe71c0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.919+0000 7fe7232d6640 1 -- 172.21.15.129:0/291628712 shutdown_connections 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.919+0000 7fe7232d6640 1 -- 172.21.15.129:0/291628712 wait complete. 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.919+0000 7fe7232d6640 1 Processor -- start 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.919+0000 7fe7232d6640 1 -- start start 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.920+0000 7fe7232d6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c076220 0x7fe71c19d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.920+0000 7fe7232d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c19db00 0x7fe71c1a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.920+0000 7fe7232d6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71c19df80 0x7fe71c1a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.920+0000 7fe7232d6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c10de40 con 0x7fe71c19db00 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.920+0000 7fe7232d6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c10dcc0 con 0x7fe71c19df80 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.920+0000 7fe7232d6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c10dfc0 con 0x7fe71c076220 2024-09-15T22:00:34.925 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.920+0000 7fe721873640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71c19df80 0x7fe71c1a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe721873640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71c19df80 0x7fe71c1a5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.129:46612/0 (socket says 172.21.15.129:46612) 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe721873640 1 -- 172.21.15.129:0/721617495 learned_addr learned my addr 172.21.15.129:0/721617495 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe720871640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c19db00 0x7fe71c1a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe721072640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c076220 0x7fe71c19d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe721873640 1 -- 172.21.15.129:0/721617495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c076220 msgr2=0x7fe71c19d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe721873640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c076220 0x7fe71c19d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe721873640 1 -- 172.21.15.129:0/721617495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c19db00 msgr2=0x7fe71c1a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe721873640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c19db00 0x7fe71c1a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe721873640 1 -- 172.21.15.129:0/721617495 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c1a59c0 con 0x7fe71c19df80 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe720871640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c19db00 0x7fe71c1a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe721072640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c076220 0x7fe71c19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe721873640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71c19df80 0x7fe71c1a5100 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe71400ba50 tx=0x7fe71400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:34.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe7127fc640 1 -- 172.21.15.129:0/721617495 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714019070 con 0x7fe71c19df80 2024-09-15T22:00:34.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe7232d6640 1 -- 172.21.15.129:0/721617495 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c1a5cb0 con 0x7fe71c19df80 2024-09-15T22:00:34.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe7127fc640 1 -- 172.21.15.129:0/721617495 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe71400cbe0 con 0x7fe71c19df80 2024-09-15T22:00:34.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.921+0000 7fe7127fc640 1 -- 172.21.15.129:0/721617495 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140076a0 con 0x7fe71c19df80 2024-09-15T22:00:34.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.922+0000 7fe7232d6640 1 -- 172.21.15.129:0/721617495 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c1a61a0 con 0x7fe71c19df80 2024-09-15T22:00:34.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.922+0000 7fe7127fc640 1 -- 172.21.15.129:0/721617495 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7fe71401c070 con 0x7fe71c19df80 2024-09-15T22:00:34.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.922+0000 7fe7232d6640 1 -- 172.21.15.129:0/721617495 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe71c196bc0 con 0x7fe71c19df80 2024-09-15T22:00:34.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.923+0000 7fe7127fc640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe708039730 0x7fe70803bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:34.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.923+0000 7fe7127fc640 1 -- 172.21.15.129:0/721617495 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe71404be70 con 0x7fe71c19df80 2024-09-15T22:00:34.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.923+0000 7fe721072640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe708039730 0x7fe70803bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:34.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.923+0000 7fe721072640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe708039730 0x7fe70803bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe718005ab0 tx=0x7fe7180315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:34.930 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:34.929+0000 7fe7127fc640 1 -- 172.21.15.129:0/721617495 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7140112a0 con 0x7fe71c19df80 2024-09-15T22:00:35.110 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.108+0000 7fe7232d6640 1 -- 172.21.15.129:0/721617495 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi099=y;smithi129=z", "target": ["mon-mgr", ""]}) -- 0x7fe71c0626c0 con 0x7fe708039730 2024-09-15T22:00:35.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.113+0000 7fe7127fc640 1 -- 172.21.15.129:0/721617495 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe71c0626c0 con 0x7fe708039730 2024-09-15T22:00:35.115 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled mgr update... 2024-09-15T22:00:35.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 -- 172.21.15.129:0/721617495 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe708039730 msgr2=0x7fe70803bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:35.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe708039730 0x7fe70803bbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe718005ab0 tx=0x7fe7180315d0 comp rx=0 tx=0).stop 2024-09-15T22:00:35.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 -- 172.21.15.129:0/721617495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71c19df80 msgr2=0x7fe71c1a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:35.117 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71c19df80 0x7fe71c1a5100 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe71400ba50 tx=0x7fe71400bf10 comp rx=0 tx=0).stop 2024-09-15T22:00:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 -- 172.21.15.129:0/721617495 shutdown_connections 2024-09-15T22:00:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe71c19df80 0x7fe71c1a5100 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe708039730 0x7fe70803bbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe71c19db00 0x7fe71c1a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 --2- 172.21.15.129:0/721617495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe71c076220 0x7fe71c19d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 -- 172.21.15.129:0/721617495 >> 172.21.15.129:0/721617495 conn(0x7fe71c0fce90 msgr2=0x7fe71c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 -- 172.21.15.129:0/721617495 shutdown_connections 2024-09-15T22:00:35.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:00:35.115+0000 7fe7232d6640 1 -- 172.21.15.129:0/721617495 wait complete. 2024-09-15T22:00:35.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:34 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:35.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:34 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:35.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:34 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:00:35.703 DEBUG:teuthology.orchestra.run.smithi099:mgr.y> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.y.service 2024-09-15T22:00:35.705 DEBUG:teuthology.orchestra.run.smithi129:mgr.z> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.z.service 2024-09-15T22:00:35.707 INFO:tasks.cephadm:Deploying OSDs... 2024-09-15T22:00:35.707 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T22:00:35.708 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:00:35.726 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:00:35.727 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-09-15T22:00:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:00:35.783 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:35.784 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-15T22:00:35.784 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:35.784 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:35.784 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-15 21:59:53.328745242 +0000 2024-09-15T22:00:35.784 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-15 21:55:34.297041825 +0000 2024-09-15T22:00:35.784 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-15 21:55:34.297041825 +0000 2024-09-15T22:00:35.784 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-15 21:55:34.297041825 +0000 2024-09-15T22:00:35.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:00:35.852 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-15T22:00:35.852 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-15T22:00:35.852 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000122108 s, 4.2 MB/s 2024-09-15T22:00:35.853 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:00:35.912 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-09-15T22:00:35.969 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:00:35.970 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:35.970 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-15T22:00:35.970 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:35.970 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:35.970 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-15 21:59:53.362744558 +0000 2024-09-15T22:00:35.970 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-15 21:55:34.625035227 +0000 2024-09-15T22:00:35.970 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-15 21:55:34.625035227 +0000 2024-09-15T22:00:35.970 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-15 21:55:34.625035227 +0000 2024-09-15T22:00:35.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-15T22:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-15T22:00:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000121847 s, 4.2 MB/s 2024-09-15T22:00:36.042 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:00:36.100 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-09-15T22:00:36.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:36 smithi005 ceph-mon[30944]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:36.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:36 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:36.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:36 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:36.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:36 smithi005 ceph-mon[30944]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:00:36.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:36 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:36.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:36 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:00:36.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:36 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:36.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:36 smithi005 ceph-mon[30944]: Reconfiguring daemon mon.b on smithi099 2024-09-15T22:00:36.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:36 smithi005 ceph-mon[30944]: from='client.24101 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi099=y;smithi129=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:00:36.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:36 smithi005 ceph-mon[30944]: Saving service mgr spec with placement smithi005=x;smithi099=y;smithi129=z;count:3 2024-09-15T22:00:36.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:36 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:36.124 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:00:36.124 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:36.124 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-15T22:00:36.124 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:36.124 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:36.124 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-15 21:59:53.389744014 +0000 2024-09-15T22:00:36.124 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-15 21:55:34.957028548 +0000 2024-09-15T22:00:36.124 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-15 21:55:34.957028548 +0000 2024-09-15T22:00:36.124 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-15 21:55:34.957028548 +0000 2024-09-15T22:00:36.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:00:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-15T22:00:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-15T22:00:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000140179 s, 3.7 MB/s 2024-09-15T22:00:36.191 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:00:36.249 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-09-15T22:00:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:00:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:36.306 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-15T22:00:36.306 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:36.306 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:36.306 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-15 21:59:53.417743451 +0000 2024-09-15T22:00:36.306 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-15 21:55:35.276022130 +0000 2024-09-15T22:00:36.306 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-15 21:55:35.276022130 +0000 2024-09-15T22:00:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-15 21:55:35.276022130 +0000 2024-09-15T22:00:36.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:00:36.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:36 smithi099 ceph-mon[34892]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:36.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:36 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:36.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:36 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:36.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:36 smithi099 ceph-mon[34892]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:00:36.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:36 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:36.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:36 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:00:36.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:36 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:36.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:36 smithi099 ceph-mon[34892]: Reconfiguring daemon mon.b on smithi099 2024-09-15T22:00:36.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:36 smithi099 ceph-mon[34892]: from='client.24101 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi099=y;smithi129=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:00:36.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:36 smithi099 ceph-mon[34892]: Saving service mgr spec with placement smithi005=x;smithi099=y;smithi129=z;count:3 2024-09-15T22:00:36.336 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:36 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:36.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:36 smithi129 ceph-mon[35314]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:36.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:36 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:36.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:36 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:36.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:36 smithi129 ceph-mon[35314]: Reconfiguring mon.b (monmap changed)... 2024-09-15T22:00:36.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:36 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:36.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:36 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:00:36.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:36 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:36.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:36 smithi129 ceph-mon[35314]: Reconfiguring daemon mon.b on smithi099 2024-09-15T22:00:36.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:36 smithi129 ceph-mon[35314]: from='client.24101 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi099=y;smithi129=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:00:36.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:36 smithi129 ceph-mon[35314]: Saving service mgr spec with placement smithi005=x;smithi099=y;smithi129=z;count:3 2024-09-15T22:00:36.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:36 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-15T22:00:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-15T22:00:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000123284 s, 4.2 MB/s 2024-09-15T22:00:36.373 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:00:36.431 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-15T22:00:36.431 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:00:36.450 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:00:36.450 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-09-15T22:00:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:00:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:36.506 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-15T22:00:36.506 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:36.506 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:36.506 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-15 22:00:25.599118395 +0000 2024-09-15T22:00:36.506 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-15 21:55:48.785703486 +0000 2024-09-15T22:00:36.506 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-15 21:55:48.785703486 +0000 2024-09-15T22:00:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-15 21:55:48.785703486 +0000 2024-09-15T22:00:36.507 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:00:36.570 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-15T22:00:36.570 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-15T22:00:36.570 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000275697 s, 1.9 MB/s 2024-09-15T22:00:36.571 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:00:36.627 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-09-15T22:00:36.682 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:00:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:36.683 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-15T22:00:36.683 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:36.683 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:36.683 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-15 22:00:25.600118373 +0000 2024-09-15T22:00:36.683 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-15 21:55:49.114695886 +0000 2024-09-15T22:00:36.683 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-15 21:55:49.114695886 +0000 2024-09-15T22:00:36.683 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-15 21:55:49.114695886 +0000 2024-09-15T22:00:36.683 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:00:36.746 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-15T22:00:36.746 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-15T22:00:36.746 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000218246 s, 2.3 MB/s 2024-09-15T22:00:36.747 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:00:36.803 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-09-15T22:00:36.858 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:00:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:36.859 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-15T22:00:36.859 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:36.859 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:36.859 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-15 22:00:25.600118373 +0000 2024-09-15T22:00:36.859 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-15 21:55:49.453688055 +0000 2024-09-15T22:00:36.859 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-15 21:55:49.453688055 +0000 2024-09-15T22:00:36.859 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-15 21:55:49.453688055 +0000 2024-09-15T22:00:36.859 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:00:36.921 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-15T22:00:36.922 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-15T22:00:36.922 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000209502 s, 2.4 MB/s 2024-09-15T22:00:36.923 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:00:36.979 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-09-15T22:00:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:00:37.034 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:37.034 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-15T22:00:37.034 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:37.035 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:37.035 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-15 22:00:25.600118373 +0000 2024-09-15T22:00:37.035 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-15 21:55:49.775680616 +0000 2024-09-15T22:00:37.035 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-15 21:55:49.775680616 +0000 2024-09-15T22:00:37.035 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-15 21:55:49.775680616 +0000 2024-09-15T22:00:37.035 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:00:37.097 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-15T22:00:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-15T22:00:37.098 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000232875 s, 2.2 MB/s 2024-09-15T22:00:37.099 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:00:37.156 DEBUG:teuthology.orchestra.run.smithi129:> set -ex 2024-09-15T22:00:37.156 DEBUG:teuthology.orchestra.run.smithi129:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T22:00:37.173 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T22:00:37.173 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_1 2024-09-15T22:00:37.229 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T22:00:37.229 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:37.229 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-15T22:00:37.229 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:37.229 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:37.229 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-15 22:00:16.832090674 +0000 2024-09-15T22:00:37.230 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-15 21:55:47.444439422 +0000 2024-09-15T22:00:37.230 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-15 21:55:47.444439422 +0000 2024-09-15T22:00:37.230 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-15 21:55:47.444439422 +0000 2024-09-15T22:00:37.230 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T22:00:37.294 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-15T22:00:37.294 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-15T22:00:37.294 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000268312 s, 1.9 MB/s 2024-09-15T22:00:37.295 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T22:00:37.351 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_2 2024-09-15T22:00:37.406 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T22:00:37.407 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:37.407 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-15T22:00:37.407 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:37.407 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:37.407 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-15 22:00:16.832090674 +0000 2024-09-15T22:00:37.407 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-15 21:55:47.773431958 +0000 2024-09-15T22:00:37.407 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-15 21:55:47.773431958 +0000 2024-09-15T22:00:37.407 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-15 21:55:47.773431958 +0000 2024-09-15T22:00:37.407 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T22:00:37.474 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-15T22:00:37.474 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-15T22:00:37.474 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000174539 s, 2.9 MB/s 2024-09-15T22:00:37.474 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T22:00:37.530 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_3 2024-09-15T22:00:37.585 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T22:00:37.585 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:37.585 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-15T22:00:37.585 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:37.586 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:37.586 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-15 22:00:16.832090674 +0000 2024-09-15T22:00:37.586 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-15 21:55:48.093424699 +0000 2024-09-15T22:00:37.586 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-15 21:55:48.093424699 +0000 2024-09-15T22:00:37.586 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-15 21:55:48.093424699 +0000 2024-09-15T22:00:37.586 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T22:00:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:37 smithi129 ceph-mon[35314]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:00:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:00:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:37 smithi129 ceph-mon[35314]: Reconfiguring daemon mon.c on smithi129 2024-09-15T22:00:37.617 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-15T22:00:37.617 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-15T22:00:37.618 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.000202418 s, 2.5 MB/s 2024-09-15T22:00:37.619 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T22:00:37.675 DEBUG:teuthology.orchestra.run.smithi129:> stat /dev/vg_nvme/lv_4 2024-09-15T22:00:37.731 INFO:teuthology.orchestra.run.smithi129.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T22:00:37.731 INFO:teuthology.orchestra.run.smithi129.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T22:00:37.731 INFO:teuthology.orchestra.run.smithi129.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-15T22:00:37.731 INFO:teuthology.orchestra.run.smithi129.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T22:00:37.731 INFO:teuthology.orchestra.run.smithi129.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T22:00:37.731 INFO:teuthology.orchestra.run.smithi129.stdout:Access: 2024-09-15 22:00:16.833090652 +0000 2024-09-15T22:00:37.731 INFO:teuthology.orchestra.run.smithi129.stdout:Modify: 2024-09-15 21:55:48.412417463 +0000 2024-09-15T22:00:37.732 INFO:teuthology.orchestra.run.smithi129.stdout:Change: 2024-09-15 21:55:48.412417463 +0000 2024-09-15T22:00:37.732 INFO:teuthology.orchestra.run.smithi129.stdout: Birth: 2024-09-15 21:55:48.412417463 +0000 2024-09-15T22:00:37.732 DEBUG:teuthology.orchestra.run.smithi129:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T22:00:37.794 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records in 2024-09-15T22:00:37.794 INFO:teuthology.orchestra.run.smithi129.stderr:1+0 records out 2024-09-15T22:00:37.794 INFO:teuthology.orchestra.run.smithi129.stderr:512 bytes copied, 0.0001925 s, 2.7 MB/s 2024-09-15T22:00:37.795 DEBUG:teuthology.orchestra.run.smithi129:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T22:00:37.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:37.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:37.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:37 smithi099 ceph-mon[34892]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:00:37.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:37.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:00:37.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:37.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:37 smithi099 ceph-mon[34892]: Reconfiguring daemon mon.c on smithi129 2024-09-15T22:00:37.851 INFO:tasks.cephadm:Deploying osd.0 on smithi005 with /dev/vg_nvme/lv_4... 2024-09-15T22:00:37.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:00:37.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:37.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:37.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:37 smithi005 ceph-mon[30944]: Reconfiguring mon.c (monmap changed)... 2024-09-15T22:00:37.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-15T22:00:37.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-15T22:00:37.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:37 smithi005 ceph-mon[30944]: Reconfiguring daemon mon.c on smithi129 2024-09-15T22:00:38.108 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:00:38.435 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:38 smithi129 ceph-mon[35314]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:38.435 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:38.436 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:38.436 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:38.436 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:38.436 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:38.436 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:38.436 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:00:38.436 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:00:38.436 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:00:38.436 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:38.482 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:38 smithi005 ceph-mon[30944]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:38.482 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:38.482 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:38.482 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:38.482 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:38.482 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:38.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:38.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:00:38.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:00:38.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:00:38.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:38 smithi099 ceph-mon[34892]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:00:38.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:00:38.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:00:38.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:39.649 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:39 smithi129 ceph-mon[35314]: Deploying daemon mgr.z on smithi129 2024-09-15T22:00:39.649 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:39 smithi129 systemd[1]: Starting Ceph mgr.z for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:00:39.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:39 smithi005 ceph-mon[30944]: Deploying daemon mgr.z on smithi129 2024-09-15T22:00:39.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:39 smithi099 ceph-mon[34892]: Deploying daemon mgr.z on smithi129 2024-09-15T22:00:39.926 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:39 smithi129 podman[37079]: 2024-09-15 22:00:39.647913045 +0000 UTC m=+0.111492730 container create 1ca9519e701b5ef611240d93727f7751deeb240a6758b6a8c245b39e56ac5ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-15T22:00:39.926 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:39 smithi129 podman[37079]: 2024-09-15 22:00:39.55228644 +0000 UTC m=+0.015866140 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:00:39.927 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:39 smithi129 podman[37079]: 2024-09-15 22:00:39.833563339 +0000 UTC m=+0.297143021 container init 1ca9519e701b5ef611240d93727f7751deeb240a6758b6a8c245b39e56ac5ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-15T22:00:39.927 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:39 smithi129 podman[37079]: 2024-09-15 22:00:39.836261717 +0000 UTC m=+0.299841400 container start 1ca9519e701b5ef611240d93727f7751deeb240a6758b6a8c245b39e56ac5ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:00:39.927 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:39 smithi129 ceph-mgr[37098]: -- 172.21.15.129:0/3389807613 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a6c3e2f0e0 con 0x55a6c3e0d000 2024-09-15T22:00:39.927 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:39 smithi129 bash[37079]: 1ca9519e701b5ef611240d93727f7751deeb240a6758b6a8c245b39e56ac5ea2 2024-09-15T22:00:40.234 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:39 smithi129 systemd[1]: Started Ceph mgr.z for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:00:40.234 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:39 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:39.961+0000 7f4a26183040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:00:40.234 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:40.022+0000 7f4a26183040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:00:40.234 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:40.168+0000 7f4a26183040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:00:40.234 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:40.232+0000 7f4a26183040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:00:40.392 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:00:40.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_4 2024-09-15T22:00:40.517 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:40.377+0000 7f4a26183040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:00:40.517 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-mon[35314]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:40.517 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:40.517 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:40.517 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:40.517 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:00:40.517 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:00:40.518 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:00:40.518 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:40.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:40 smithi005 ceph-mon[30944]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:40.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:40.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:40.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:40.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:00:40.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:00:40.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:00:40.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:00:40.729 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:40 smithi099 ceph-mon[34892]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:40.729 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:40.729 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:40.729 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:40.729 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:00:40.729 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-15T22:00:40.729 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:00:40.729 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:40.828 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:40.515+0000 7f4a26183040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:00:41.095 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:40.827+0000 7f4a26183040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:00:41.095 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:40.887+0000 7f4a26183040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:00:41.095 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:41.067+0000 7f4a26183040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:00:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.125+0000 7f731d0cd640 1 -- 172.21.15.5:0/1577088398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73181037c0 msgr2=0x7f731810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.125+0000 7f731d0cd640 1 --2- 172.21.15.5:0/1577088398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73181037c0 0x7f731810a050 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7310009ca0 tx=0x7f731002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:00:41.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.125+0000 7f731d0cd640 1 -- 172.21.15.5:0/1577088398 shutdown_connections 2024-09-15T22:00:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.125+0000 7f731d0cd640 1 --2- 172.21.15.5:0/1577088398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73181037c0 0x7f731810a050 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.125+0000 7f731d0cd640 1 --2- 172.21.15.5:0/1577088398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7318102e00 0x7f7318103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.125+0000 7f731d0cd640 1 --2- 172.21.15.5:0/1577088398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7318101c00 0x7f7318102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.125+0000 7f731d0cd640 1 -- 172.21.15.5:0/1577088398 >> 172.21.15.5:0/1577088398 conn(0x7f73180fd390 msgr2=0x7f73180ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:00:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.125+0000 7f731d0cd640 1 -- 172.21.15.5:0/1577088398 shutdown_connections 2024-09-15T22:00:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.125+0000 7f731d0cd640 1 -- 172.21.15.5:0/1577088398 wait complete. 2024-09-15T22:00:41.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.126+0000 7f731d0cd640 1 Processor -- start 2024-09-15T22:00:41.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.126+0000 7f731d0cd640 1 -- start start 2024-09-15T22:00:41.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731d0cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7318101c00 0x7f7318075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731d0cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318102e00 0x7f73180745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731d0cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73181037c0 0x7f7318074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731d0cd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f731810d930 con 0x7f7318102e00 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731d0cd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f731810d7b0 con 0x7f73181037c0 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731d0cd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f731810dab0 con 0x7f7318101c00 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318102e00 0x7f73180745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f7316d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7318101c00 0x7f7318075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318102e00 0x7f73180745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50938/0 (socket says 172.21.15.5:50938) 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f7316d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7318101c00 0x7f7318075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53918/0 (socket says 172.21.15.5:53918) 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731659c640 1 -- 172.21.15.5:0/1169918126 learned_addr learned my addr 172.21.15.5:0/1169918126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731759e640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73181037c0 0x7f7318074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:41.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731659c640 1 -- 172.21.15.5:0/1169918126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7318101c00 msgr2=0x7f7318075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731659c640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7318101c00 0x7f7318075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731659c640 1 -- 172.21.15.5:0/1169918126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73181037c0 msgr2=0x7f7318074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731659c640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73181037c0 0x7f7318074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731659c640 1 -- 172.21.15.5:0/1169918126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73180753e0 con 0x7f7318102e00 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731659c640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318102e00 0x7f73180745e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f730c00b4a0 tx=0x7f730c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f72fffff640 1 -- 172.21.15.5:0/1169918126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c014070 con 0x7f7318102e00 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f72fffff640 1 -- 172.21.15.5:0/1169918126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f730c010af0 con 0x7f7318102e00 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.127+0000 7f731d0cd640 1 -- 172.21.15.5:0/1169918126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73181a2060 con 0x7f7318102e00 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.128+0000 7f72fffff640 1 -- 172.21.15.5:0/1169918126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730c017d10 con 0x7f7318102e00 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.128+0000 7f731d0cd640 1 -- 172.21.15.5:0/1169918126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73181a25c0 con 0x7f7318102e00 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.128+0000 7f72fffff640 1 -- 172.21.15.5:0/1169918126 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46516+0+0 (secure 0 0 0) 0x7f730c017070 con 0x7f7318102e00 2024-09-15T22:00:41.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.129+0000 7f731d0cd640 1 -- 172.21.15.5:0/1169918126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7318108f30 con 0x7f7318102e00 2024-09-15T22:00:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.129+0000 7f72fffff640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73000394d0 0x7f730003b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:00:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.129+0000 7f72fffff640 1 -- 172.21.15.5:0/1169918126 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f730c04b9d0 con 0x7f7318102e00 2024-09-15T22:00:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.129+0000 7f7316d9d640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73000394d0 0x7f730003b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:00:41.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.129+0000 7f7316d9d640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73000394d0 0x7f730003b990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7308005fd0 tx=0x7f730800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:00:41.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.134+0000 7f72fffff640 1 -- 172.21.15.5:0/1169918126 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f730c020300 con 0x7f7318102e00 2024-09-15T22:00:41.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:41.317+0000 7f731d0cd640 1 -- 172.21.15.5:0/1169918126 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f7318062630 con 0x7f73000394d0 2024-09-15T22:00:41.626 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:41 smithi099 ceph-mon[34892]: Deploying daemon mgr.y on smithi099 2024-09-15T22:00:41.626 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:41 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:00:41.627 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:41 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:00:41.627 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:41 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:41.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:41 smithi005 ceph-mon[30944]: Deploying daemon mgr.y on smithi099 2024-09-15T22:00:41.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:41 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:00:41.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:41 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:00:41.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:41 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:41.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:41 smithi129 ceph-mon[35314]: Deploying daemon mgr.y on smithi099 2024-09-15T22:00:41.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:41 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:00:41.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:41 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:00:41.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:41 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:42.001 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:41 smithi099 systemd[1]: Starting Ceph mgr.y for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:00:42.287 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 podman[36245]: 2024-09-15 22:00:41.902425752 +0000 UTC m=+0.015429578 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:00:42.288 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 podman[36245]: 2024-09-15 22:00:42.000198102 +0000 UTC m=+0.113201940 container create d366fade87e0218028b575a13be9663c2d9fe7a6c0fa06afe1735326013b9213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-15T22:00:42.288 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 podman[36245]: 2024-09-15 22:00:42.202660067 +0000 UTC m=+0.315663894 container init d366fade87e0218028b575a13be9663c2d9fe7a6c0fa06afe1735326013b9213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-15T22:00:42.288 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 podman[36245]: 2024-09-15 22:00:42.205424599 +0000 UTC m=+0.318428428 container start d366fade87e0218028b575a13be9663c2d9fe7a6c0fa06afe1735326013b9213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-15T22:00:42.288 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-mgr[36265]: -- 172.21.15.99:0/1573291590 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dc92c0d0e0 con 0x55dc92beb000 2024-09-15T22:00:42.288 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 bash[36245]: d366fade87e0218028b575a13be9663c2d9fe7a6c0fa06afe1735326013b9213 2024-09-15T22:00:42.345 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:41.946+0000 7f4a26183040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:00:42.345 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:42.012+0000 7f4a26183040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:00:42.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-mon[34892]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:42.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-mon[34892]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:00:42.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.562 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.562 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 systemd[1]: Started Ceph mgr.y for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:00:42.562 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:42.337+0000 7f74dea74040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:00:42.562 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:42.404+0000 7f74dea74040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:00:42.562 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:42.559+0000 7f74dea74040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:00:42.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:42 smithi005 ceph-mon[30944]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:42.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:42 smithi005 ceph-mon[30944]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:00:42.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.747 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:42 smithi129 ceph-mon[35314]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:42.747 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:42 smithi129 ceph-mon[35314]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:00:42.747 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:42 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.748 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:42 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.748 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:42 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.748 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:42 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:42.840 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:42.627+0000 7f74dea74040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:00:42.840 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:42.785+0000 7f74dea74040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:00:43.027 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:42.745+0000 7f4a26183040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:00:43.027 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:42.882+0000 7f4a26183040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:00:43.027 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:43.025+0000 7f4a26183040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:00:43.263 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:42.937+0000 7f74dea74040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:00:43.263 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:43 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:43.261+0000 7f74dea74040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:00:43.316 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:43.086+0000 7f4a26183040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:00:43.316 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:43.246+0000 7f4a26183040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:00:43.316 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:43.314+0000 7f4a26183040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:00:43.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:43 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:43.521 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:43 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:43.521 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:43 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:43.324+0000 7f74dea74040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:00:43.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:43.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:43.595 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:43.375+0000 7f4a26183040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:00:43.595 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:43.535+0000 7f4a26183040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:00:43.721 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:43 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:43.722 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:43 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:43.839 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:43 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:43.519+0000 7f74dea74040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:00:43.880 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:43.607+0000 7f4a26183040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:00:43.880 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:43.675+0000 7f4a26183040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:00:43.880 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:43.760+0000 7f4a26183040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:00:43.880 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:43.819+0000 7f4a26183040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:00:43.880 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:43.878+0000 7f4a26183040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:00:44.840 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:44.400+0000 7f74dea74040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:00:44.841 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:44.466+0000 7f74dea74040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:00:44.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:44.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1867279240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f"}]: dispatch 2024-09-15T22:00:44.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1867279240' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f"}]': finished 2024-09-15T22:00:44.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:00:44.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:00:44.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:44.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:44.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:00:44.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:00:44.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:44.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:44 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1563811713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:00:44.845 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:00:44.656+0000 7f4a26183040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:00:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1867279240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f"}]: dispatch 2024-09-15T22:00:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1867279240' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f"}]': finished 2024-09-15T22:00:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:00:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:00:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:44.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:44.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:00:44.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:00:44.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:44.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:44 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1563811713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:00:44.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:44.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1867279240' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f"}]: dispatch 2024-09-15T22:00:44.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1867279240' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f"}]': finished 2024-09-15T22:00:44.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: osdmap e5: 1 total, 0 up, 1 in 2024-09-15T22:00:44.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:00:44.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:44.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:44.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:44.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-15T22:00:44.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-15T22:00:44.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:44.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:44 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1563811713' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:00:45.435 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:45.179+0000 7f74dea74040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:00:45.436 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:45.293+0000 7f74dea74040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:00:45.436 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:45.433+0000 7f74dea74040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:00:45.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:45.508+0000 7f72fffff640 1 -- 172.21.15.5:0/1169918126 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 14) ==== 92407+0+0 (secure 0 0 0) 0x7f730c00fc00 con 0x7f7318102e00 2024-09-15T22:00:45.712 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-mon[34892]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:00:45.712 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-mon[34892]: Reconfiguring daemon mgr.x on smithi005 2024-09-15T22:00:45.712 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-mon[34892]: Standby manager daemon z started 2024-09-15T22:00:45.712 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:00:45.712 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:00:45.712 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:00:45.712 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:00:45.712 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:45.494+0000 7f74dea74040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:00:45.713 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:45.642+0000 7f74dea74040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:00:45.713 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:45.710+0000 7f74dea74040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:00:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:45 smithi129 ceph-mon[35314]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:00:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:45 smithi129 ceph-mon[35314]: Reconfiguring daemon mgr.x on smithi005 2024-09-15T22:00:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:45 smithi129 ceph-mon[35314]: Standby manager daemon z started 2024-09-15T22:00:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:45 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:00:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:45 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:00:45.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:45 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:00:45.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:45 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:00:45.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:45 smithi005 ceph-mon[30944]: Reconfiguring mgr.x (unknown last config time)... 2024-09-15T22:00:45.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:45 smithi005 ceph-mon[30944]: Reconfiguring daemon mgr.x on smithi005 2024-09-15T22:00:45.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:45 smithi005 ceph-mon[30944]: Standby manager daemon z started 2024-09-15T22:00:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:45 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:00:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:45 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:00:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:45 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:00:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:45 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.129:0/94464131' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:00:45.989 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:45.769+0000 7f74dea74040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:00:45.989 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:45.924+0000 7f74dea74040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:00:45.989 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:45.987+0000 7f74dea74040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:00:46.255 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:46 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:46.053+0000 7f74dea74040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:00:46.255 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:46 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:46.135+0000 7f74dea74040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:00:46.255 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:46 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:46.194+0000 7f74dea74040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:00:46.255 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:46 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:46.253+0000 7f74dea74040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:00:46.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:46 smithi099 ceph-mon[34892]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:46 smithi099 ceph-mon[34892]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-15T22:00:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:46 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:00:46.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:46 smithi129 ceph-mon[35314]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:46.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:46 smithi129 ceph-mon[35314]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-15T22:00:46.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:46 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:00:46.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:46 smithi005 ceph-mon[30944]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:46 smithi005 ceph-mon[30944]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-15T22:00:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:00:47.340 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:00:47 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:00:47.039+0000 7f74dea74040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:00:47.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:00:47.519+0000 7f72fffff640 1 -- 172.21.15.5:0/1169918126 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7f730c00eba0 con 0x7f7318102e00 2024-09-15T22:00:47.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:47 smithi099 ceph-mon[34892]: Standby manager daemon y started 2024-09-15T22:00:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:47 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:00:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:47 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:00:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:47 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:00:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:47 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:00:47.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:47 smithi129 ceph-mon[35314]: Standby manager daemon y started 2024-09-15T22:00:47.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:47 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:00:47.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:47 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:00:47.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:47 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:00:47.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:47 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:00:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:47 smithi005 ceph-mon[30944]: Standby manager daemon y started 2024-09-15T22:00:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:47 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:00:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:47 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:00:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:47 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:00:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:47 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.99:0/552924379' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:00:48.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:48 smithi005 ceph-mon[30944]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:48.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:48 smithi005 ceph-mon[30944]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-15T22:00:48.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:00:48.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:48 smithi099 ceph-mon[34892]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:48 smithi099 ceph-mon[34892]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-15T22:00:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:00:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:48 smithi129 ceph-mon[35314]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:48 smithi129 ceph-mon[35314]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-15T22:00:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:00:50.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:50 smithi005 ceph-mon[30944]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:50.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:50 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:50.818 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:50 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:50.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:50 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:50.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:50 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:50.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:50 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:50.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:50 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:50.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:50 smithi099 ceph-mon[34892]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:50.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:50 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:50.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:50 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:50.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:50 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:50.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:50 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:50.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:50 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:50.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:50 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:50 smithi129 ceph-mon[35314]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:50 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:50 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:50 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:50 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:50 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:00:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:50 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:52.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:52 smithi005 ceph-mon[30944]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:52.827 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:00:52.828 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:52.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:52 smithi099 ceph-mon[34892]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:52.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:00:52.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:52.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:52 smithi129 ceph-mon[35314]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:52.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-15T22:00:52.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:00:53.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:53 smithi005 ceph-mon[30944]: Deploying daemon osd.0 on smithi005 2024-09-15T22:00:53.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:53 smithi099 ceph-mon[34892]: Deploying daemon osd.0 on smithi005 2024-09-15T22:00:53.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:53 smithi129 ceph-mon[35314]: Deploying daemon osd.0 on smithi005 2024-09-15T22:00:54.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:54 smithi099 ceph-mon[34892]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:54.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:54 smithi129 ceph-mon[35314]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:54.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:54 smithi005 ceph-mon[30944]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:56 smithi099 ceph-mon[34892]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:56 smithi129 ceph-mon[35314]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:56.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:56 smithi005 ceph-mon[30944]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:58.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:58 smithi099 ceph-mon[34892]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:00:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:58 smithi129 ceph-mon[35314]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:00:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:58.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:58 smithi005 ceph-mon[30944]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:00:58.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:00:58.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:00:58.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:00:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:00.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:00 smithi005 ceph-mon[30944]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:00.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:00 smithi099 ceph-mon[34892]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:00.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:00 smithi129 ceph-mon[35314]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:02.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:02 smithi005 ceph-mon[30944]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:02.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:02.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:02.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:02.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:02.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:02.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:02 smithi099 ceph-mon[34892]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:02 smithi129 ceph-mon[35314]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.586+0000 7f72fffff640 1 -- 172.21.15.5:0/1169918126 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7318062630 con 0x7f73000394d0 2024-09-15T22:01:02.588 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 0 on host 'smithi005' 2024-09-15T22:01:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 -- 172.21.15.5:0/1169918126 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73000394d0 msgr2=0x7f730003b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73000394d0 0x7f730003b990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7308005fd0 tx=0x7f730800b410 comp rx=0 tx=0).stop 2024-09-15T22:01:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 -- 172.21.15.5:0/1169918126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318102e00 msgr2=0x7f73180745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318102e00 0x7f73180745e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f730c00b4a0 tx=0x7f730c00b960 comp rx=0 tx=0).stop 2024-09-15T22:01:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 -- 172.21.15.5:0/1169918126 shutdown_connections 2024-09-15T22:01:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73181037c0 0x7f7318074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73000394d0 0x7f730003b990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318102e00 0x7f73180745e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 --2- 172.21.15.5:0/1169918126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7318101c00 0x7f7318075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 -- 172.21.15.5:0/1169918126 >> 172.21.15.5:0/1169918126 conn(0x7f73180fd390 msgr2=0x7f7318106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:01:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 -- 172.21.15.5:0/1169918126 shutdown_connections 2024-09-15T22:01:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:02.588+0000 7f731d0cd640 1 -- 172.21.15.5:0/1169918126 wait complete. 2024-09-15T22:01:03.103 DEBUG:teuthology.orchestra.run.smithi005:osd.0> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.0.service 2024-09-15T22:01:03.105 INFO:tasks.cephadm:Deploying osd.1 on smithi099 with /dev/vg_nvme/lv_4... 2024-09-15T22:01:03.105 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:01:03.337 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:03.338 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:03.338 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:03.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:03.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:03.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:03.342 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:01:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:04.482 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:01:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0[44440]: 2024-09-15T22:01:04.235+0000 7f8be97ae640 -1 osd.0 0 log_to_monitors true 2024-09-15T22:01:04.784 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:04 smithi005 ceph-mon[30944]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:04.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:04 smithi005 ceph-mon[30944]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:01:04.818 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:04 smithi099 ceph-mon[34892]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:04.818 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:04 smithi099 ceph-mon[34892]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:01:04.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:04 smithi129 ceph-mon[35314]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:04.868 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:04 smithi129 ceph-mon[35314]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-15T22:01:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:05 smithi005 ceph-mon[30944]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:01:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:05 smithi005 ceph-mon[30944]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:01:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:05 smithi005 ceph-mon[30944]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:01:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:05 smithi005 ceph-mon[30944]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:01:05.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:05 smithi099 ceph-mon[34892]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:01:05.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:05 smithi099 ceph-mon[34892]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:01:05.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:05 smithi099 ceph-mon[34892]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:01:05.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:05 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:05.861 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:05 smithi099 ceph-mon[34892]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:01:05.863 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T22:01:05.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_4 2024-09-15T22:01:06.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:05 smithi129 ceph-mon[35314]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-15T22:01:06.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:05 smithi129 ceph-mon[35314]: osdmap e6: 1 total, 0 up, 1 in 2024-09-15T22:01:06.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:05 smithi129 ceph-mon[35314]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:01:06.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:05 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:06.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:05 smithi129 ceph-mon[35314]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:01:06.127 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:01:06.489 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0[44440]: 2024-09-15T22:01:06.151+0000 7f8be5731640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T22:01:06.489 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0[44440]: 2024-09-15T22:01:06.155+0000 7f8bdd551640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:01:06.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.644+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/1121574862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b0101e50 msgr2=0x7fe3b01086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:06.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.644+0000 7fe3b6cd1640 1 --2- 172.21.15.99:0/1121574862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b0101e50 0x7fe3b01086e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe3a8009ca0 tx=0x7fe3a802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:01:06.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.644+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/1121574862 shutdown_connections 2024-09-15T22:01:06.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.644+0000 7fe3b6cd1640 1 --2- 172.21.15.99:0/1121574862 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b0101e50 0x7fe3b01086e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:06.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.644+0000 7fe3b6cd1640 1 --2- 172.21.15.99:0/1121574862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b0101490 0x7fe3b0101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:06.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.644+0000 7fe3b6cd1640 1 --2- 172.21.15.99:0/1121574862 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3b0100290 0x7fe3b0100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:06.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.644+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/1121574862 >> 172.21.15.99:0/1121574862 conn(0x7fe3b00fba40 msgr2=0x7fe3b00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:01:06.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.644+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/1121574862 shutdown_connections 2024-09-15T22:01:06.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.644+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/1121574862 wait complete. 2024-09-15T22:01:06.656 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b6cd1640 1 Processor -- start 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b6cd1640 1 -- start start 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b6cd1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b0100290 0x7fe3b0197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b6cd1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3b0101490 0x7fe3b0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b6cd1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b0101e50 0x7fe3b019f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b6cd1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b010b6b0 con 0x7fe3b0101e50 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b6cd1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b010b530 con 0x7fe3b0100290 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b6cd1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b010b830 con 0x7fe3b0101490 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b4a6d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b0100290 0x7fe3b0197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b4a6d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b0100290 0x7fe3b0197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:58550/0 (socket says 172.21.15.99:58550) 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b4a6d640 1 -- 172.21.15.99:0/3127056791 learned_addr learned my addr 172.21.15.99:0/3127056791 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3affff640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3b0101490 0x7fe3b0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:06.657 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b526e640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b0101e50 0x7fe3b019f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b4a6d640 1 -- 172.21.15.99:0/3127056791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3b0101490 msgr2=0x7fe3b0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b4a6d640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3b0101490 0x7fe3b0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b4a6d640 1 -- 172.21.15.99:0/3127056791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b0101e50 msgr2=0x7fe3b019f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b4a6d640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b0101e50 0x7fe3b019f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.645+0000 7fe3b4a6d640 1 -- 172.21.15.99:0/3127056791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b019fb30 con 0x7fe3b0100290 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.646+0000 7fe3b4a6d640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b0100290 0x7fe3b0197f00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400dbc0 tx=0x7fe3a400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.646+0000 7fe3adffb640 1 -- 172.21.15.99:0/3127056791 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4004220 con 0x7fe3b0100290 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.646+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/3127056791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b019fe20 con 0x7fe3b0100290 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.646+0000 7fe3adffb640 1 -- 172.21.15.99:0/3127056791 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3a400ae50 con 0x7fe3b0100290 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.646+0000 7fe3adffb640 1 -- 172.21.15.99:0/3127056791 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a400e850 con 0x7fe3b0100290 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.646+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/3127056791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b01a02d0 con 0x7fe3b0100290 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.647+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/3127056791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b0107690 con 0x7fe3b0100290 2024-09-15T22:01:06.658 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.648+0000 7fe3adffb640 1 -- 172.21.15.99:0/3127056791 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7fe3a400e9f0 con 0x7fe3b0100290 2024-09-15T22:01:06.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.648+0000 7fe3adffb640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3a00a31e0 0x7fe3a00a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:06.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.649+0000 7fe3adffb640 1 -- 172.21.15.99:0/3127056791 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fe3a40cdd50 con 0x7fe3b0100290 2024-09-15T22:01:06.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.649+0000 7fe3affff640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3a00a31e0 0x7fe3a00a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:06.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.649+0000 7fe3affff640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3a00a31e0 0x7fe3a00a56a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0199420 tx=0x7fe39c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:01:06.659 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.653+0000 7fe3adffb640 1 -- 172.21.15.99:0/3127056791 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3a409c600 con 0x7fe3b0100290 2024-09-15T22:01:06.832 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:06.830+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/3127056791 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe3b0062630 con 0x7fe3a00a31e0 2024-09-15T22:01:06.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:06.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:01:06.832 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:06.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:06.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' 2024-09-15T22:01:06.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:06.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:06.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:01:06.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:06.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:06.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:06.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:06.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:06.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:06.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:01:06.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' 2024-09-15T22:01:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:01:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:07.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:07.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: osdmap e7: 1 total, 0 up, 1 in 2024-09-15T22:01:07.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:07.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:07.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: from='osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865]' entity='osd.0' 2024-09-15T22:01:07.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:07.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:07.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:01:07.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:07.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:07.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:07.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:07.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:07 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:01:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:07 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:01:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:07 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi005 2024-09-15T22:01:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:07 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-15T22:01:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:01:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:01:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:07 smithi099 ceph-mon[34892]: osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865] boot 2024-09-15T22:01:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:07 smithi099 ceph-mon[34892]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:01:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:08.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:07 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:01:08.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:07 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:01:08.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:07 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi005 2024-09-15T22:01:08.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:07 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-15T22:01:08.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:01:08.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:01:08.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:08.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:07 smithi129 ceph-mon[35314]: osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865] boot 2024-09-15T22:01:08.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:07 smithi129 ceph-mon[35314]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:01:08.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:08.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:07 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:01:08.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:07 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:01:08.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:07 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi005 2024-09-15T22:01:08.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:07 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-15T22:01:08.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:01:08.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:01:08.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:08.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:07 smithi005 ceph-mon[30944]: osd.0 [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865] boot 2024-09-15T22:01:08.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:07 smithi005 ceph-mon[30944]: osdmap e8: 1 total, 1 up, 1 in 2024-09-15T22:01:08.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:01:09.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:08 smithi005 ceph-mon[30944]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:09.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:08 smithi005 ceph-mon[30944]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:01:09.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:08 smithi005 ceph-mon[30944]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:01:09.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:08 smithi099 ceph-mon[34892]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:09.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:08 smithi099 ceph-mon[34892]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:01:09.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:08 smithi099 ceph-mon[34892]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:01:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:08 smithi129 ceph-mon[35314]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-15T22:01:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:08 smithi129 ceph-mon[35314]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:01:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:08 smithi129 ceph-mon[35314]: osdmap e9: 1 total, 1 up, 1 in 2024-09-15T22:01:10.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:09 smithi099 ceph-mon[34892]: from='client.? 172.21.15.99:0/1869552577' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5"}]: dispatch 2024-09-15T22:01:10.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:09 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5"}]: dispatch 2024-09-15T22:01:10.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:09 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5"}]': finished 2024-09-15T22:01:10.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:09 smithi099 ceph-mon[34892]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:01:10.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:09 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:10.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:09 smithi129 ceph-mon[35314]: from='client.? 172.21.15.99:0/1869552577' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5"}]: dispatch 2024-09-15T22:01:10.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:09 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5"}]: dispatch 2024-09-15T22:01:10.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:09 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5"}]': finished 2024-09-15T22:01:10.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:09 smithi129 ceph-mon[35314]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:01:10.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:09 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:10.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:09 smithi005 ceph-mon[30944]: from='client.? 172.21.15.99:0/1869552577' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5"}]: dispatch 2024-09-15T22:01:10.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:09 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5"}]: dispatch 2024-09-15T22:01:10.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:09 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5"}]': finished 2024-09-15T22:01:10.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:09 smithi005 ceph-mon[30944]: osdmap e10: 2 total, 1 up, 2 in 2024-09-15T22:01:10.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:09 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:11.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:10 smithi099 ceph-mon[34892]: pgmap v42: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:11.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:10 smithi099 ceph-mon[34892]: from='client.? 172.21.15.99:0/2608092984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:01:11.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:10 smithi129 ceph-mon[35314]: pgmap v42: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:11.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:10 smithi129 ceph-mon[35314]: from='client.? 172.21.15.99:0/2608092984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:01:11.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:10 smithi005 ceph-mon[30944]: pgmap v42: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:11.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:10 smithi005 ceph-mon[30944]: from='client.? 172.21.15.99:0/2608092984' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:01:13.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:12 smithi099 ceph-mon[34892]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:12 smithi129 ceph-mon[35314]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:13.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:12 smithi005 ceph-mon[30944]: pgmap v44: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:14.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:14 smithi099 ceph-mon[34892]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:15.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:14 smithi129 ceph-mon[35314]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:15.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:14 smithi005 ceph-mon[30944]: pgmap v45: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:16.667 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:16 smithi099 ceph-mon[34892]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:16.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:16 smithi005 ceph-mon[30944]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:17.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:16 smithi129 ceph-mon[35314]: pgmap v46: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:17.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:17 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:01:17.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:17 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:18.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:01:18.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:18.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:17 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-15T22:01:18.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:17 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:18.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:18 smithi099 ceph-mon[34892]: Deploying daemon osd.1 on smithi099 2024-09-15T22:01:18.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:18 smithi099 ceph-mon[34892]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:19.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:18 smithi129 ceph-mon[35314]: Deploying daemon osd.1 on smithi099 2024-09-15T22:01:19.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:18 smithi129 ceph-mon[35314]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:19.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:18 smithi005 ceph-mon[30944]: Deploying daemon osd.1 on smithi099 2024-09-15T22:01:19.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:18 smithi005 ceph-mon[30944]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:21.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:20 smithi099 ceph-mon[34892]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:21.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:20 smithi129 ceph-mon[35314]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:21.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:20 smithi005 ceph-mon[30944]: pgmap v48: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:22.733 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:22 smithi099 ceph-mon[34892]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:22.897 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:22 smithi129 ceph-mon[35314]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:23.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:22 smithi005 ceph-mon[30944]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:24.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:23 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:24.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:23 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:24.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:23 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:24.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:23 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:24.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:23 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:24.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:23 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:24.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:23 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:24.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:23 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:24.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:23 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:24.798 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:24 smithi099 ceph-mon[34892]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:25.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:24 smithi129 ceph-mon[35314]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:25.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:24 smithi005 ceph-mon[30944]: pgmap v50: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:26.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:26 smithi099 ceph-mon[34892]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:26.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:26.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:26.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:26.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:26.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:26 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:26.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:26 smithi005 ceph-mon[30944]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:26.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:26 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:26.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:26 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:26.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:26 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:26.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:26 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:26.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:26 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:26.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:26 smithi129 ceph-mon[35314]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:26.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:26 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:26.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:26 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:26.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:26 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:26.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:26 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:26.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:26 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:26.950 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.948+0000 7fe3adffb640 1 -- 172.21.15.99:0/3127056791 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe3b0062630 con 0x7fe3a00a31e0 2024-09-15T22:01:26.950 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 1 on host 'smithi099' 2024-09-15T22:01:26.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/3127056791 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3a00a31e0 msgr2=0x7fe3a00a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:26.952 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3a00a31e0 0x7fe3a00a56a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0199420 tx=0x7fe39c005f90 comp rx=0 tx=0).stop 2024-09-15T22:01:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/3127056791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b0100290 msgr2=0x7fe3b0197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b0100290 0x7fe3b0197f00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400dbc0 tx=0x7fe3a400a620 comp rx=0 tx=0).stop 2024-09-15T22:01:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/3127056791 shutdown_connections 2024-09-15T22:01:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3a00a31e0 0x7fe3a00a56a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3b0101e50 0x7fe3b019f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3b0101490 0x7fe3b0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 --2- 172.21.15.99:0/3127056791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3b0100290 0x7fe3b0197f00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/3127056791 >> 172.21.15.99:0/3127056791 conn(0x7fe3b00fba40 msgr2=0x7fe3b0104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:01:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/3127056791 shutdown_connections 2024-09-15T22:01:26.953 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:01:26.951+0000 7fe3b6cd1640 1 -- 172.21.15.99:0/3127056791 wait complete. 2024-09-15T22:01:27.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:27.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:27.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:27.551 DEBUG:teuthology.orchestra.run.smithi099:osd.1> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.1.service 2024-09-15T22:01:27.553 INFO:tasks.cephadm:Deploying osd.2 on smithi129 with /dev/vg_nvme/lv_4... 2024-09-15T22:01:27.553 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-15T22:01:27.803 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:01:27.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:27.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:27.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:27.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:27.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:27.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:28.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:28 smithi129 ceph-mon[35314]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:28.717 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:28 smithi099 ceph-mon[34892]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:28.718 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:01:28 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1[39339]: 2024-09-15T22:01:28.587+0000 7fb4b13c1640 -1 osd.1 0 log_to_monitors true 2024-09-15T22:01:28.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:28 smithi005 ceph-mon[30944]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:29.496 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:29 smithi129 ceph-mon[35314]: from='osd.1 [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:01:29.496 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:29 smithi129 ceph-mon[35314]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:01:29.620 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:29 smithi099 ceph-mon[34892]: from='osd.1 [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:01:29.620 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:29 smithi099 ceph-mon[34892]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:01:29.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:29 smithi005 ceph-mon[30944]: from='osd.1 [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:01:29.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:29 smithi005 ceph-mon[30944]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-15T22:01:30.193 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T22:01:30.219 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_4 2024-09-15T22:01:30.468 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:01:30.495 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:30.495 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:01:30.495 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: from='osd.1 [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:01:30.495 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:01:30.496 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:30.496 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:01:30.496 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.496 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.496 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:01:30.496 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.496 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:30.496 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:30.496 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:30 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:30.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:01:30.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: from='osd.1 [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:01:30.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:01:30.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:30.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:01:30.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:01:30.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:30.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:30.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:30 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:30.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-15T22:01:30.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: from='osd.1 [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:01:30.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: osdmap e11: 2 total, 1 up, 2 in 2024-09-15T22:01:30.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:30.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:01:30.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:01:30.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:30.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:30.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:30 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:30.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.918+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/858285367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4b0102690 msgr2=0x7fe4b0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:30.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.918+0000 7fe4b7fd6640 1 --2- 172.21.15.129:0/858285367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4b0102690 0x7fe4b0108f20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8009ca0 tx=0x7fe4a802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:01:30.926 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.918+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/858285367 shutdown_connections 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.918+0000 7fe4b7fd6640 1 --2- 172.21.15.129:0/858285367 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4b0102690 0x7fe4b0108f20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.918+0000 7fe4b7fd6640 1 --2- 172.21.15.129:0/858285367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4b0101cd0 0x7fe4b0102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.918+0000 7fe4b7fd6640 1 --2- 172.21.15.129:0/858285367 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b0100ad0 0x7fe4b0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.918+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/858285367 >> 172.21.15.129:0/858285367 conn(0x7fe4b00fc280 msgr2=0x7fe4b00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/858285367 shutdown_connections 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/858285367 wait complete. 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b7fd6640 1 Processor -- start 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b7fd6640 1 -- start start 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b7fd6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b0100ad0 0x7fe4b019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b7fd6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4b0101cd0 0x7fe4b019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b7fd6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4b0102690 0x7fe4b01a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:30.927 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b7fd6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b010c750 con 0x7fe4b0102690 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b7fd6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b010c5d0 con 0x7fe4b0100ad0 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b7fd6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b010c8d0 con 0x7fe4b0101cd0 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b5571640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4b0101cd0 0x7fe4b019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.919+0000 7fe4b5d72640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b0100ad0 0x7fe4b019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4b5571640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4b0101cd0 0x7fe4b019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:35364/0 (socket says 172.21.15.129:35364) 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4b5571640 1 -- 172.21.15.129:0/1098893189 learned_addr learned my addr 172.21.15.129:0/1098893189 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4b6573640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4b0102690 0x7fe4b01a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4b5571640 1 -- 172.21.15.129:0/1098893189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b0100ad0 msgr2=0x7fe4b019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4b5571640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b0100ad0 0x7fe4b019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4b5571640 1 -- 172.21.15.129:0/1098893189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4b0102690 msgr2=0x7fe4b01a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4b5571640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4b0102690 0x7fe4b01a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:30.928 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4b5571640 1 -- 172.21.15.129:0/1098893189 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b01a1c70 con 0x7fe4b0101cd0 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4b5571640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4b0101cd0 0x7fe4b019a5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000c920 tx=0x7fe4a000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1098893189 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0007c70 con 0x7fe4b0101cd0 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1098893189 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a0007e10 con 0x7fe4b0101cd0 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1098893189 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0018890 con 0x7fe4b0101cd0 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/1098893189 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b01a1f60 con 0x7fe4b0101cd0 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.920+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/1098893189 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b01a2460 con 0x7fe4b0101cd0 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.921+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/1098893189 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b01a26f0 con 0x7fe4b0101cd0 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.923+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1098893189 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7fe4a0018a30 con 0x7fe4b0101cd0 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.923+0000 7fe4a6ffd640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe49c0a3260 0x7fe49c0a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.923+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1098893189 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fe4a00ce270 con 0x7fe4b0101cd0 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.923+0000 7fe4b5d72640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe49c0a3260 0x7fe49c0a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:30.929 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.924+0000 7fe4b5d72640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe49c0a3260 0x7fe49c0a5720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0101b30 tx=0x7fe4ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:01:30.932 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:30.931+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1098893189 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe4a0014030 con 0x7fe4b0101cd0 2024-09-15T22:01:31.115 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:31.113+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/1098893189 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe4b01a28c0 con 0x7fe49c0a3260 2024-09-15T22:01:31.339 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1[39339]: 2024-09-15T22:01:31.024+0000 7fb4ad344640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T22:01:31.340 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1[39339]: 2024-09-15T22:01:31.027+0000 7fb4a5164640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:01:31.633 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:31 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi099 2024-09-15T22:01:31.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:31 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi099 to 17088M 2024-09-15T22:01:31.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:31 smithi129 ceph-mon[35314]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:01:31.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:31 smithi129 ceph-mon[35314]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:01:31.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:31.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:31.634 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:31 smithi129 ceph-mon[35314]: from='osd.1 ' entity='osd.1' 2024-09-15T22:01:31.635 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:01:31.635 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:01:31.635 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:31.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi099 2024-09-15T22:01:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi099 to 17088M 2024-09-15T22:01:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-mon[34892]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:01:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-mon[34892]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:01:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-mon[34892]: from='osd.1 ' entity='osd.1' 2024-09-15T22:01:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:01:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:01:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:31 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:31.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:31 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi099 2024-09-15T22:01:31.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:31 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi099 to 17088M 2024-09-15T22:01:31.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:31 smithi005 ceph-mon[30944]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:01:31.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:31 smithi005 ceph-mon[30944]: osdmap e12: 2 total, 1 up, 2 in 2024-09-15T22:01:31.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:31.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:31.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:31 smithi005 ceph-mon[30944]: from='osd.1 ' entity='osd.1' 2024-09-15T22:01:31.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:01:31.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:01:31.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:32.646 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:32 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:01:32.647 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:32 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:01:32.647 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:32 smithi129 ceph-mon[35314]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:32.647 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:32 smithi129 ceph-mon[35314]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:01:32.647 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:32.647 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:32 smithi129 ceph-mon[35314]: osd.1 [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880] boot 2024-09-15T22:01:32.647 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:32 smithi129 ceph-mon[35314]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:01:32.647 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:32 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:01:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:32 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:01:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:32 smithi099 ceph-mon[34892]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:32 smithi099 ceph-mon[34892]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:01:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:32 smithi099 ceph-mon[34892]: osd.1 [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880] boot 2024-09-15T22:01:32.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:32 smithi099 ceph-mon[34892]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:01:32.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:32.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:32 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:01:32.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:32 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:01:32.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:32 smithi005 ceph-mon[30944]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-09-15T22:01:32.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:32 smithi005 ceph-mon[30944]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:01:32.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:32.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:32 smithi005 ceph-mon[30944]: osd.1 [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880] boot 2024-09-15T22:01:32.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:32 smithi005 ceph-mon[30944]: osdmap e13: 2 total, 2 up, 2 in 2024-09-15T22:01:32.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:01:33.647 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:33 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/3752606838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cf9aa4e-8865-471c-a841-4be5386a3fcd"}]: dispatch 2024-09-15T22:01:33.648 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:33 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cf9aa4e-8865-471c-a841-4be5386a3fcd"}]: dispatch 2024-09-15T22:01:33.648 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:33 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cf9aa4e-8865-471c-a841-4be5386a3fcd"}]': finished 2024-09-15T22:01:33.648 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:33 smithi129 ceph-mon[35314]: osdmap e14: 3 total, 2 up, 3 in 2024-09-15T22:01:33.648 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:33.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:33 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/3752606838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cf9aa4e-8865-471c-a841-4be5386a3fcd"}]: dispatch 2024-09-15T22:01:33.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:33 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cf9aa4e-8865-471c-a841-4be5386a3fcd"}]: dispatch 2024-09-15T22:01:33.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:33 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cf9aa4e-8865-471c-a841-4be5386a3fcd"}]': finished 2024-09-15T22:01:33.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:33 smithi099 ceph-mon[34892]: osdmap e14: 3 total, 2 up, 3 in 2024-09-15T22:01:33.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:33.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:33 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/3752606838' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cf9aa4e-8865-471c-a841-4be5386a3fcd"}]: dispatch 2024-09-15T22:01:33.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:33 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0cf9aa4e-8865-471c-a841-4be5386a3fcd"}]: dispatch 2024-09-15T22:01:33.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:33 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0cf9aa4e-8865-471c-a841-4be5386a3fcd"}]': finished 2024-09-15T22:01:33.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:33 smithi005 ceph-mon[30944]: osdmap e14: 3 total, 2 up, 3 in 2024-09-15T22:01:33.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:34.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:34 smithi099 ceph-mon[34892]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:34.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:34 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/1241357315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:01:34.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:34 smithi129 ceph-mon[35314]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:34.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:34 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/1241357315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:01:34.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:34 smithi005 ceph-mon[30944]: pgmap v58: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:34.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:34 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/1241357315' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:01:36.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:36 smithi099 ceph-mon[34892]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:36.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:36 smithi129 ceph-mon[35314]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:36.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:36 smithi005 ceph-mon[30944]: pgmap v60: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:38.765 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:38 smithi129 ceph-mon[35314]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:38.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:38 smithi099 ceph-mon[34892]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:38.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:38 smithi005 ceph-mon[30944]: pgmap v61: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:40.555 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:40 smithi129 ceph-mon[35314]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:40.659 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:40 smithi005 ceph-mon[30944]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:40.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:40 smithi099 ceph-mon[34892]: pgmap v62: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:42.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:42 smithi099 ceph-mon[34892]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:42.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:42 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:01:42.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:42 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:42.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:42 smithi129 ceph-mon[35314]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:42.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:42 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:01:42.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:42 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:42.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:42 smithi005 ceph-mon[30944]: pgmap v63: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:42.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-15T22:01:42.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:43.529 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:43 smithi129 ceph-mon[35314]: Deploying daemon osd.2 on smithi129 2024-09-15T22:01:43.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:43 smithi099 ceph-mon[34892]: Deploying daemon osd.2 on smithi129 2024-09-15T22:01:43.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:43 smithi005 ceph-mon[30944]: Deploying daemon osd.2 on smithi129 2024-09-15T22:01:44.785 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:44 smithi129 ceph-mon[35314]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:44.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:44 smithi099 ceph-mon[34892]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:44.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:44 smithi005 ceph-mon[30944]: pgmap v64: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:46.698 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:46 smithi129 ceph-mon[35314]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:46 smithi099 ceph-mon[34892]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:46.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:46 smithi005 ceph-mon[30944]: pgmap v65: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:48.688 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:48 smithi129 ceph-mon[35314]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:48.688 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:48.688 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:48.688 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:48 smithi099 ceph-mon[34892]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:48.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:48 smithi005 ceph-mon[30944]: pgmap v66: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:48.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:48.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:48.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:50.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:50 smithi099 ceph-mon[34892]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:50 smithi129 ceph-mon[35314]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:50.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:50 smithi005 ceph-mon[30944]: pgmap v67: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:51.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:51 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:51.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:51 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:51.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:51 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:51.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:51 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:51.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:51 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:51.607 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.605+0000 7fe4a6ffd640 1 -- 172.21.15.129:0/1098893189 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe4b01a28c0 con 0x7fe49c0a3260 2024-09-15T22:01:51.607 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 2 on host 'smithi129' 2024-09-15T22:01:51.608 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/1098893189 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe49c0a3260 msgr2=0x7fe49c0a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe49c0a3260 0x7fe49c0a5720 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0101b30 tx=0x7fe4ac005f90 comp rx=0 tx=0).stop 2024-09-15T22:01:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/1098893189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4b0101cd0 msgr2=0x7fe4b019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4b0101cd0 0x7fe4b019a5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000c920 tx=0x7fe4a000cde0 comp rx=0 tx=0).stop 2024-09-15T22:01:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/1098893189 shutdown_connections 2024-09-15T22:01:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe49c0a3260 0x7fe49c0a5720 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4b0102690 0x7fe4b01a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4b0101cd0 0x7fe4b019a5f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 --2- 172.21.15.129:0/1098893189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4b0100ad0 0x7fe4b019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/1098893189 >> 172.21.15.129:0/1098893189 conn(0x7fe4b00fc280 msgr2=0x7fe4b00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:01:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/1098893189 shutdown_connections 2024-09-15T22:01:51.609 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:01:51.607+0000 7fe4b7fd6640 1 -- 172.21.15.129:0/1098893189 wait complete. 2024-09-15T22:01:51.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:51 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:51.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:51 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:51.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:51 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:51.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:51 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:51.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:51 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:51.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:51 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:51.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:51 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:51.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:51 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:51.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:51 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:51.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:51 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:52.240 DEBUG:teuthology.orchestra.run.smithi129:osd.2> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.2.service 2024-09-15T22:01:52.242 INFO:tasks.cephadm:Deploying osd.3 on smithi005 with /dev/vg_nvme/lv_3... 2024-09-15T22:01:52.242 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:01:52.494 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:01:52.531 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:52 smithi129 ceph-mon[35314]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:52.532 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:52.532 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:52.532 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:52.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:52 smithi005 ceph-mon[30944]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:52.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:52.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:52.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:52.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:52 smithi099 ceph-mon[34892]: pgmap v68: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:52.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:01:52.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:52.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:53.845 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:01:53 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2[40242]: 2024-09-15T22:01:53.709+0000 7f62ced14640 -1 osd.2 0 log_to_monitors true 2024-09-15T22:01:54.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:54 smithi005 ceph-mon[30944]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:54.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:54 smithi005 ceph-mon[30944]: from='osd.2 [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:01:54.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:54 smithi005 ceph-mon[30944]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:01:54.658 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:54 smithi129 ceph-mon[35314]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:54.658 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:54 smithi129 ceph-mon[35314]: from='osd.2 [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:01:54.658 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:54 smithi129 ceph-mon[35314]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:01:54.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:54 smithi099 ceph-mon[34892]: pgmap v69: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:54.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:54 smithi099 ceph-mon[34892]: from='osd.2 [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:01:54.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:54 smithi099 ceph-mon[34892]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-15T22:01:55.256 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:01:55.283 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_3 2024-09-15T22:01:55.537 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:01:55.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:01:55.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:01:55.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:55.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: from='osd.2 [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:01:55.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:01:55.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:55.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:55.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:01:55.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:55.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:55.609 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:55.609 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:01:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:01:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: from='osd.2 [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:01:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:01:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:01:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:55.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:55.845 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2[40242]: 2024-09-15T22:01:55.474+0000 7f62cac97640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T22:01:55.845 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2[40242]: 2024-09-15T22:01:55.477+0000 7f62c6abf640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:01:55.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-15T22:01:55.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: osdmap e15: 3 total, 2 up, 3 in 2024-09-15T22:01:55.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:55.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: from='osd.2 [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:01:55.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:01:55.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:55.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:55.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:01:55.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:55.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:55.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:01:55.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:01:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.007+0000 7fbf8a544640 1 -- 172.21.15.5:0/2973926033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf841026a0 msgr2=0x7fbf84104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.007+0000 7fbf8a544640 1 --2- 172.21.15.5:0/2973926033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf841026a0 0x7fbf84104a90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbf74009900 tx=0x7fbf7402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:01:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.007+0000 7fbf8a544640 1 -- 172.21.15.5:0/2973926033 shutdown_connections 2024-09-15T22:01:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.007+0000 7fbf8a544640 1 --2- 172.21.15.5:0/2973926033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf84107900 0x7fbf84109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.007+0000 7fbf8a544640 1 --2- 172.21.15.5:0/2973926033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf84104fd0 0x7fbf841073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.007+0000 7fbf8a544640 1 --2- 172.21.15.5:0/2973926033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf841026a0 0x7fbf84104a90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:56.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.007+0000 7fbf8a544640 1 -- 172.21.15.5:0/2973926033 >> 172.21.15.5:0/2973926033 conn(0x7fbf840fc2a0 msgr2=0x7fbf840fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:01:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.007+0000 7fbf8a544640 1 -- 172.21.15.5:0/2973926033 shutdown_connections 2024-09-15T22:01:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.007+0000 7fbf8a544640 1 -- 172.21.15.5:0/2973926033 wait complete. 2024-09-15T22:01:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.007+0000 7fbf8a544640 1 Processor -- start 2024-09-15T22:01:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.007+0000 7fbf8a544640 1 -- start start 2024-09-15T22:01:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf8a544640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf841026a0 0x7fbf84102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf8a544640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf84104fd0 0x7fbf84100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:56.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf8a544640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf84107900 0x7fbf84100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf8a544640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8410d900 con 0x7fbf841026a0 2024-09-15T22:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf8a544640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8410d780 con 0x7fbf84107900 2024-09-15T22:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf8a544640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8410da80 con 0x7fbf84104fd0 2024-09-15T22:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf841026a0 0x7fbf84102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf841026a0 0x7fbf84102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52480/0 (socket says 172.21.15.5:52480) 2024-09-15T22:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf83fff640 1 -- 172.21.15.5:0/128687461 learned_addr learned my addr 172.21.15.5:0/128687461 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf88ae1640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf84107900 0x7fbf84100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:56.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf837fe640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf84104fd0 0x7fbf84100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf88ae1640 1 -- 172.21.15.5:0/128687461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf84104fd0 msgr2=0x7fbf84100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf88ae1640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf84104fd0 0x7fbf84100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf88ae1640 1 -- 172.21.15.5:0/128687461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf841026a0 msgr2=0x7fbf84102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:01:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf88ae1640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf841026a0 0x7fbf84102000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:01:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf88ae1640 1 -- 172.21.15.5:0/128687461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf84101450 con 0x7fbf84107900 2024-09-15T22:01:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf83fff640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf841026a0 0x7fbf84102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:01:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf88ae1640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf84107900 0x7fbf84100b90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009fd0 tx=0x7fbf7c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:01:56.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.008+0000 7fbf817fa640 1 -- 172.21.15.5:0/128687461 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c00ce20 con 0x7fbf84107900 2024-09-15T22:01:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.009+0000 7fbf817fa640 1 -- 172.21.15.5:0/128687461 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf7c005020 con 0x7fbf84107900 2024-09-15T22:01:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.009+0000 7fbf8a544640 1 -- 172.21.15.5:0/128687461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf841a20f0 con 0x7fbf84107900 2024-09-15T22:01:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.009+0000 7fbf817fa640 1 -- 172.21.15.5:0/128687461 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c010720 con 0x7fbf84107900 2024-09-15T22:01:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.009+0000 7fbf8a544640 1 -- 172.21.15.5:0/128687461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf841a25b0 con 0x7fbf84107900 2024-09-15T22:01:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.009+0000 7fbf8a544640 1 -- 172.21.15.5:0/128687461 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf840fdd90 con 0x7fbf84107900 2024-09-15T22:01:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.011+0000 7fbf817fa640 1 -- 172.21.15.5:0/128687461 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 15) ==== 138298+0+0 (secure 0 0 0) 0x7fbf7c005540 con 0x7fbf84107900 2024-09-15T22:01:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.012+0000 7fbf817fa640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbf6c0a31b0 0x7fbf6c0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:01:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.012+0000 7fbf83fff640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbf6c0a31b0 0x7fbf6c0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:01:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.012+0000 7fbf817fa640 1 -- 172.21.15.5:0/128687461 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2571+0+0 (secure 0 0 0) 0x7fbf7c0ce830 con 0x7fbf84107900 2024-09-15T22:01:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.012+0000 7fbf83fff640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbf6c0a31b0 0x7fbf6c0a5670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbf740096c0 tx=0x7fbf740315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:01:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.015+0000 7fbf817fa640 1 -- 172.21.15.5:0/128687461 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf7c09cc40 con 0x7fbf84107900 2024-09-15T22:01:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:56.198+0000 7fbf8a544640 1 -- 172.21.15.5:0/128687461 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbf84062630 con 0x7fbf6c0a31b0 2024-09-15T22:01:56.748 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:56 smithi005 ceph-mon[30944]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:56.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:56 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi129 2024-09-15T22:01:56.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:56 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi129 to 17088M 2024-09-15T22:01:56.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:56 smithi005 ceph-mon[30944]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:01:56.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:56 smithi005 ceph-mon[30944]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:01:56.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:56 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:56.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:56 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:56.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:56 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:01:56.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:56 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:01:56.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:56 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:56 smithi099 ceph-mon[34892]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:56 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi129 2024-09-15T22:01:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:56 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi129 to 17088M 2024-09-15T22:01:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:56 smithi099 ceph-mon[34892]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:01:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:56 smithi099 ceph-mon[34892]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:01:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:56 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:56 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:56 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:01:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:56 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:01:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:56 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:56 smithi129 ceph-mon[35314]: pgmap v71: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:56 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi129 2024-09-15T22:01:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:56 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi129 to 17088M 2024-09-15T22:01:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:56 smithi129 ceph-mon[35314]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:01:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:56 smithi129 ceph-mon[35314]: osdmap e16: 3 total, 2 up, 3 in 2024-09-15T22:01:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:56 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:56 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:56 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:01:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:56 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:01:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:56 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:01:57.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:57 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:01:57.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:57 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:01:57.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:57 smithi005 ceph-mon[30944]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:01:57.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:57 smithi005 ceph-mon[30944]: osd.2 [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483] boot 2024-09-15T22:01:57.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:57 smithi005 ceph-mon[30944]: osdmap e17: 3 total, 3 up, 3 in 2024-09-15T22:01:57.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:57 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:57.530 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:57 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:01:57.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:57 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:01:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:57 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:01:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:57 smithi099 ceph-mon[34892]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:01:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:57 smithi099 ceph-mon[34892]: osd.2 [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483] boot 2024-09-15T22:01:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:57 smithi099 ceph-mon[34892]: osdmap e17: 3 total, 3 up, 3 in 2024-09-15T22:01:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:57 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:57 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:01:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:57 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:01:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:57 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:01:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:57 smithi129 ceph-mon[35314]: from='client.24188 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:01:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:57 smithi129 ceph-mon[35314]: osd.2 [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483] boot 2024-09-15T22:01:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:57 smithi129 ceph-mon[35314]: osdmap e17: 3 total, 3 up, 3 in 2024-09-15T22:01:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:57 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:01:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:57 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:01:58.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:58 smithi005 ceph-mon[30944]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:58.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:01:58.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:58 smithi005 ceph-mon[30944]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:01:58.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:01:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:58 smithi099 ceph-mon[34892]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:01:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:58 smithi099 ceph-mon[34892]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:01:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:01:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:58 smithi129 ceph-mon[35314]: pgmap v74: 0 pgs: ; 0 B data, 65 MiB used, 179 GiB / 179 GiB avail 2024-09-15T22:01:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-15T22:01:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:58 smithi129 ceph-mon[35314]: osdmap e18: 3 total, 3 up, 3 in 2024-09-15T22:01:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:01:59.107 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:01:59 smithi005 sudo[49186]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:01:59.107 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:01:59 smithi005 sudo[49186]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:01:59.107 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:01:59 smithi005 sudo[49186]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:01:59.107 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:01:59 smithi005 sudo[49186]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:01:59.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:01:59.574+0000 7fbf817fa640 1 -- 172.21.15.5:0/128687461 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fbf7c098bb0 con 0x7fbf84107900 2024-09-15T22:01:59.743 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:59 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:01:59.743 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:59 smithi099 ceph-mon[34892]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T22:01:59.743 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:59 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4101782396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fcbf74d-7a3d-471e-9447-688e10d0c350"}]: dispatch 2024-09-15T22:01:59.743 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:59 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4101782396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fcbf74d-7a3d-471e-9447-688e10d0c350"}]': finished 2024-09-15T22:01:59.743 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:59 smithi099 ceph-mon[34892]: osdmap e20: 4 total, 3 up, 4 in 2024-09-15T22:01:59.743 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:59 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:01:59.743 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:01:59 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2844310910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:01:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:59 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:01:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:59 smithi129 ceph-mon[35314]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T22:01:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:59 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4101782396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fcbf74d-7a3d-471e-9447-688e10d0c350"}]: dispatch 2024-09-15T22:01:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:59 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4101782396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fcbf74d-7a3d-471e-9447-688e10d0c350"}]': finished 2024-09-15T22:01:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:59 smithi129 ceph-mon[35314]: osdmap e20: 4 total, 3 up, 4 in 2024-09-15T22:01:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:59 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:01:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:01:59 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2844310910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:01:59.856 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:01:59 smithi005 sudo[49211]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:01:59.857 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:01:59 smithi005 sudo[49211]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:01:59.857 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:01:59 smithi005 sudo[49211]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:01:59.857 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:01:59 smithi005 sudo[49211]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:01:59.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:59 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:01:59.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:59 smithi005 ceph-mon[30944]: osdmap e19: 3 total, 3 up, 3 in 2024-09-15T22:01:59.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:59 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4101782396' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0fcbf74d-7a3d-471e-9447-688e10d0c350"}]: dispatch 2024-09-15T22:01:59.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:59 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4101782396' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0fcbf74d-7a3d-471e-9447-688e10d0c350"}]': finished 2024-09-15T22:01:59.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:59 smithi005 ceph-mon[30944]: osdmap e20: 4 total, 3 up, 4 in 2024-09-15T22:01:59.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:59 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:01:59.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:01:59 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2844310910' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:01:59.998 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:01:59 smithi099 sudo[43198]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:01:59.998 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:01:59 smithi099 sudo[43198]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:01:59.998 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:01:59 smithi099 sudo[43198]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:01:59.998 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:01:59 smithi099 sudo[43198]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:02:00.339 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:01:59 smithi099 sudo[43201]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:02:00.340 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:01:59 smithi099 sudo[43201]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:02:00.340 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:01:59 smithi099 sudo[43201]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:02:00.340 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:02:00 smithi099 sudo[43201]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:02:00.525 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:00 smithi129 ceph-mon[35314]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:00.525 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:00 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:00.525 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:00 smithi129 ceph-mon[35314]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:02:00.525 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:00 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:00.525 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:02:00 smithi129 sudo[43868]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:02:00.525 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:02:00 smithi129 sudo[43868]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:02:00.525 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:02:00 smithi129 sudo[43868]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:02:00.525 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:02:00 smithi129 sudo[43868]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:02:00.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:00 smithi099 ceph-mon[34892]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:00.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:00 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:00.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:00 smithi099 ceph-mon[34892]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:02:00.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:00 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:00.845 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:02:00 smithi129 sudo[43871]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:02:00.845 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:02:00 smithi129 sudo[43871]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:02:00.845 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:02:00 smithi129 sudo[43871]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:02:00.845 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:02:00 smithi129 sudo[43871]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:02:00.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:00 smithi005 ceph-mon[30944]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:00.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:00 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:00.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:00 smithi005 ceph-mon[30944]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-15T22:02:00.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:00 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:01.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:00 smithi005 sudo[49802]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:02:01.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:00 smithi005 sudo[49802]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:02:01.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:00 smithi005 sudo[49802]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:02:01.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:01 smithi005 sudo[49802]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:02:01.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:01 smithi005 sudo[49989]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:02:01.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:01 smithi005 sudo[49989]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:02:01.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:01 smithi005 sudo[49989]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:02:01.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:01 smithi005 sudo[49989]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:02:01.942 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:01 smithi129 sudo[43874]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:02:01.943 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:01 smithi129 sudo[43874]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:02:01.943 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:01 smithi129 sudo[43874]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:02:01.943 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:01 smithi129 sudo[43874]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:02:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:01 smithi129 sudo[43877]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:02:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:01 smithi129 sudo[43877]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:02:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:01 smithi129 sudo[43877]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:02:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:01 smithi129 sudo[43877]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:02:02.441 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 sudo[43204]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-15T22:02:02.441 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 sudo[43204]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:02:02.441 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 sudo[43204]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:02:02.442 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 sudo[43204]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:02:02.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:02.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:02 smithi005 ceph-mon[30944]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:02:02.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:02 smithi005 ceph-mon[30944]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:02:02.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:02:02.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:02:02.741 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:02:02.742 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:02:02.742 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:02:02.742 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 sudo[43207]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-15T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 sudo[43207]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-15T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 sudo[43207]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-15T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 sudo[43207]: pam_unix(sudo:session): session closed for user root 2024-09-15T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 ceph-mon[34892]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:02:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 ceph-mon[34892]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:02:02.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:02:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:02 smithi129 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:02:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:02 smithi129 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:02:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:02:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:02:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:02:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:02:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:02:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:02:03.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:03 smithi005 ceph-mon[30944]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:03.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:03 smithi005 ceph-mon[30944]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:02:03.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:03 smithi005 ceph-mon[30944]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:02:03.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:03 smithi005 ceph-mon[30944]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:02:03.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:02:03.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:02:03.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:02:03.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:03 smithi005 ceph-mon[30944]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:03 smithi099 ceph-mon[34892]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:03 smithi099 ceph-mon[34892]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:03 smithi099 ceph-mon[34892]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:03 smithi099 ceph-mon[34892]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:02:03.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:03 smithi099 ceph-mon[34892]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:02:03.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:03 smithi129 ceph-mon[35314]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:03.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:03 smithi129 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:02:03.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:03 smithi129 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:02:03.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:03 smithi129 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-15T22:02:03.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:02:03.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:02:03.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:02:03.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:03 smithi129 ceph-mon[35314]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-15T22:02:04.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:04 smithi099 ceph-mon[34892]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:04.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:04 smithi129 ceph-mon[35314]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:04.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:04 smithi005 ceph-mon[30944]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:06.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:06 smithi005 ceph-mon[30944]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:06.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:02:06.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:06.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:06 smithi099 ceph-mon[34892]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:06.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:02:06.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:06 smithi129 ceph-mon[35314]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-15T22:02:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:07.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:07 smithi005 ceph-mon[30944]: Deploying daemon osd.3 on smithi005 2024-09-15T22:02:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:07 smithi099 ceph-mon[34892]: Deploying daemon osd.3 on smithi005 2024-09-15T22:02:07.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:07 smithi129 ceph-mon[35314]: Deploying daemon osd.3 on smithi005 2024-09-15T22:02:08.773 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:08 smithi005 ceph-mon[30944]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:08.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:08 smithi099 ceph-mon[34892]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:08.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:08 smithi129 ceph-mon[35314]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:10 smithi099 ceph-mon[34892]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-15T22:02:10.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:10 smithi129 ceph-mon[35314]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-15T22:02:10.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:10 smithi005 ceph-mon[30944]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-15T22:02:12.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:12 smithi005 ceph-mon[30944]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-15T22:02:12.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:12 smithi099 ceph-mon[34892]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-15T22:02:12.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:12 smithi129 ceph-mon[35314]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-15T22:02:13.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:13 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:13 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:13.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:13 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:13 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:13 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:13 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:13.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:13 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:13.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:13 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:13.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:13 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:14.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:14 smithi005 ceph-mon[30944]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-15T22:02:14.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:14 smithi099 ceph-mon[34892]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-15T22:02:14.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:14 smithi129 ceph-mon[35314]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-15T22:02:16.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:16 smithi005 ceph-mon[30944]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:02:16.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:16 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:16.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:16 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:16.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:16 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:16.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:16 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:16.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:16 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.758+0000 7fbf817fa640 1 -- 172.21.15.5:0/128687461 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf84062630 con 0x7fbf6c0a31b0 2024-09-15T22:02:16.761 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 3 on host 'smithi005' 2024-09-15T22:02:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 -- 172.21.15.5:0/128687461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbf6c0a31b0 msgr2=0x7fbf6c0a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbf6c0a31b0 0x7fbf6c0a5670 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbf740096c0 tx=0x7fbf740315b0 comp rx=0 tx=0).stop 2024-09-15T22:02:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 -- 172.21.15.5:0/128687461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf84107900 msgr2=0x7fbf84100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf84107900 0x7fbf84100b90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c009fd0 tx=0x7fbf7c00ef10 comp rx=0 tx=0).stop 2024-09-15T22:02:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 -- 172.21.15.5:0/128687461 shutdown_connections 2024-09-15T22:02:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbf84107900 0x7fbf84100b90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbf84104fd0 0x7fbf84100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbf6c0a31b0 0x7fbf6c0a5670 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 --2- 172.21.15.5:0/128687461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf841026a0 0x7fbf84102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 -- 172.21.15.5:0/128687461 >> 172.21.15.5:0/128687461 conn(0x7fbf840fc2a0 msgr2=0x7fbf84103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 -- 172.21.15.5:0/128687461 shutdown_connections 2024-09-15T22:02:16.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:02:16.760+0000 7fbf8a544640 1 -- 172.21.15.5:0/128687461 wait complete. 2024-09-15T22:02:16.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:16 smithi099 ceph-mon[34892]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:02:16.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:16 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:16.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:16 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:16.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:16 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:16.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:16 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:16.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:16 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:16.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:16 smithi129 ceph-mon[35314]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:02:16.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:16 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:16.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:16 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:16.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:16 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:16.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:16 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:16.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:16 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:17.426 DEBUG:teuthology.orchestra.run.smithi005:osd.3> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.3.service 2024-09-15T22:02:17.428 INFO:tasks.cephadm:Deploying osd.4 on smithi099 with /dev/vg_nvme/lv_3... 2024-09-15T22:02:17.428 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:02:17.649 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:17 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:17.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:17 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:17.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:17 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:17.683 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:02:17.803 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:17 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:17.803 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:17 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:17.804 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:17 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:17.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:17.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:17.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:17 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:18.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:18 smithi005 ceph-mon[30944]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:02:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:18 smithi005 ceph-mon[30944]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:02:18.607 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:02:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3[51213]: 2024-09-15T22:02:18.331+0000 7f8ba0f0b640 -1 osd.3 0 log_to_monitors true 2024-09-15T22:02:18.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:18 smithi099 ceph-mon[34892]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:02:18.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:18 smithi099 ceph-mon[34892]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:02:18.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:18 smithi129 ceph-mon[35314]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-15T22:02:18.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:18 smithi129 ceph-mon[35314]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-15T22:02:19.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:19 smithi099 ceph-mon[34892]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:02:19.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:19 smithi099 ceph-mon[34892]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:02:19.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:19 smithi099 ceph-mon[34892]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:02:19.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:19 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:19.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:19 smithi129 ceph-mon[35314]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:02:19.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:19 smithi129 ceph-mon[35314]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:02:19.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:19 smithi129 ceph-mon[35314]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:02:19.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:19 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:19.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:19 smithi005 ceph-mon[30944]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-15T22:02:19.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:19 smithi005 ceph-mon[30944]: osdmap e21: 4 total, 3 up, 4 in 2024-09-15T22:02:19.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:19 smithi005 ceph-mon[30944]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:02:19.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:19 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:20.254 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3[51213]: 2024-09-15T22:02:20.099+0000 7f8b9ce8e640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T22:02:20.254 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3[51213]: 2024-09-15T22:02:20.104+0000 7f8b94cae640 -1 osd.3 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:02:20.374 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T22:02:20.401 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_3 2024-09-15T22:02:20.651 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:02:20.775 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:20.776 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:20.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616] boot 2024-09-15T22:02:20.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: osdmap e23: 4 total, 4 up, 4 in 2024-09-15T22:02:20.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:20.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:20.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:02:20.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:02:20.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616] boot 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: osdmap e23: 4 total, 4 up, 4 in 2024-09-15T22:02:20.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: osdmap e22: 4 total, 3 up, 4 in 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616]' entity='osd.3' 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:20.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:20.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:20.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: osd.3 [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616] boot 2024-09-15T22:02:20.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: osdmap e23: 4 total, 4 up, 4 in 2024-09-15T22:02:20.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:02:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.092+0000 7fa1d697a640 1 -- 172.21.15.99:0/1876528726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1d0101cd0 msgr2=0x7fa1d0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:21.112 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.092+0000 7fa1d697a640 1 --2- 172.21.15.99:0/1876528726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1d0101cd0 0x7fa1d0102150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009900 tx=0x7fa1c402a410 comp rx=0 tx=0).stop 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.092+0000 7fa1d697a640 1 -- 172.21.15.99:0/1876528726 shutdown_connections 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.092+0000 7fa1d697a640 1 --2- 172.21.15.99:0/1876528726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1d0102690 0x7fa1d0108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.092+0000 7fa1d697a640 1 --2- 172.21.15.99:0/1876528726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1d0101cd0 0x7fa1d0102150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.092+0000 7fa1d697a640 1 --2- 172.21.15.99:0/1876528726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1d0100ad0 0x7fa1d0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.092+0000 7fa1d697a640 1 -- 172.21.15.99:0/1876528726 >> 172.21.15.99:0/1876528726 conn(0x7fa1d00fc280 msgr2=0x7fa1d00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.092+0000 7fa1d697a640 1 -- 172.21.15.99:0/1876528726 shutdown_connections 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.092+0000 7fa1d697a640 1 -- 172.21.15.99:0/1876528726 wait complete. 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1d697a640 1 Processor -- start 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1d697a640 1 -- start start 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1d697a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1d0100ad0 0x7fa1d006aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1d697a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1d0101cd0 0x7fa1d019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1d697a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1d0102690 0x7fa1d01a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:21.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1d697a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010c750 con 0x7fa1d0102690 2024-09-15T22:02:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1d697a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010c5d0 con 0x7fa1d0100ad0 2024-09-15T22:02:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1d697a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d010c8d0 con 0x7fa1d0101cd0 2024-09-15T22:02:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1cffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1d0100ad0 0x7fa1d006aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1cffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1d0100ad0 0x7fa1d006aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:56542/0 (socket says 172.21.15.99:56542) 2024-09-15T22:02:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1cffff640 1 -- 172.21.15.99:0/656714857 learned_addr learned my addr 172.21.15.99:0/656714857 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-15T22:02:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.093+0000 7fa1d4f17640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1d0102690 0x7fa1d01a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1cf7fe640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1d0101cd0 0x7fa1d019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1cffff640 1 -- 172.21.15.99:0/656714857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1d0101cd0 msgr2=0x7fa1d019f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1cffff640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1d0101cd0 0x7fa1d019f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1cffff640 1 -- 172.21.15.99:0/656714857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1d0102690 msgr2=0x7fa1d01a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:21.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1cffff640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1d0102690 0x7fa1d01a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1cffff640 1 -- 172.21.15.99:0/656714857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d01a1d70 con 0x7fa1d0100ad0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1cffff640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1d0100ad0 0x7fa1d006aad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000d950 tx=0x7fa1c000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1cd7fa640 1 -- 172.21.15.99:0/656714857 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0017070 con 0x7fa1d0100ad0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1d697a640 1 -- 172.21.15.99:0/656714857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d01a1fa0 con 0x7fa1d0100ad0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1cd7fa640 1 -- 172.21.15.99:0/656714857 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1c0009c30 con 0x7fa1d0100ad0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1cd7fa640 1 -- 172.21.15.99:0/656714857 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c000f600 con 0x7fa1d0100ad0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.094+0000 7fa1d697a640 1 -- 172.21.15.99:0/656714857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d01a2450 con 0x7fa1d0100ad0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.095+0000 7fa1d697a640 1 -- 172.21.15.99:0/656714857 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d0100ed0 con 0x7fa1d0100ad0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.096+0000 7fa1cd7fa640 1 -- 172.21.15.99:0/656714857 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fa1c001a070 con 0x7fa1d0100ad0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.097+0000 7fa1cd7fa640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1b80a3270 0x7fa1b80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.097+0000 7fa1cd7fa640 1 -- 172.21.15.99:0/656714857 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3702+0+0 (secure 0 0 0) 0x7fa1c00ce140 con 0x7fa1d0100ad0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.097+0000 7fa1cf7fe640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1b80a3270 0x7fa1b80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.098+0000 7fa1cf7fe640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1b80a3270 0x7fa1b80a5730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:21.115 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.106+0000 7fa1cd7fa640 1 -- 172.21.15.99:0/656714857 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1c009c060 con 0x7fa1d0100ad0 2024-09-15T22:02:21.284 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:21.283+0000 7fa1d697a640 1 -- 172.21.15.99:0/656714857 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa1d0062630 con 0x7fa1b80a3270 2024-09-15T22:02:21.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:21 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:02:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:21 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:02:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:21 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi005 2024-09-15T22:02:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:21 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-15T22:02:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:21 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:02:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:21 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:02:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:21 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:21 smithi099 ceph-mon[34892]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T22:02:21.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:21 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:02:21.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:21 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:02:21.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:21 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi005 2024-09-15T22:02:21.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:21 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-15T22:02:21.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:21 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:02:21.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:21 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:02:21.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:21 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:21.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:21 smithi129 ceph-mon[35314]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T22:02:21.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:21 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:02:21.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:21 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:02:21.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:21 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi005 2024-09-15T22:02:21.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:21 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-15T22:02:21.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:21 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:02:21.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:21 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:02:21.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:21 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:21.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:21 smithi005 ceph-mon[30944]: osdmap e24: 4 total, 4 up, 4 in 2024-09-15T22:02:22.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:22 smithi129 ceph-mon[35314]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:22.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:22 smithi129 ceph-mon[35314]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:22.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:22 smithi129 ceph-mon[35314]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:02:22.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:22 smithi005 ceph-mon[30944]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:22.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:22 smithi005 ceph-mon[30944]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:22.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:22 smithi005 ceph-mon[30944]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:02:22.943 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:22 smithi099 ceph-mon[34892]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:22.943 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:22 smithi099 ceph-mon[34892]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:22.943 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:22 smithi099 ceph-mon[34892]: osdmap e25: 4 total, 4 up, 4 in 2024-09-15T22:02:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:24 smithi099 ceph-mon[34892]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:24 smithi099 ceph-mon[34892]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:02:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:24 smithi099 ceph-mon[34892]: from='client.? 172.21.15.99:0/2582613014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "511bc0a8-e4f3-48e2-824b-7afd2ead90e4"}]: dispatch 2024-09-15T22:02:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:24 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "511bc0a8-e4f3-48e2-824b-7afd2ead90e4"}]: dispatch 2024-09-15T22:02:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:24 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "511bc0a8-e4f3-48e2-824b-7afd2ead90e4"}]': finished 2024-09-15T22:02:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:24 smithi099 ceph-mon[34892]: osdmap e27: 5 total, 4 up, 5 in 2024-09-15T22:02:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:24 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:24 smithi099 ceph-mon[34892]: from='client.? 172.21.15.99:0/2471226329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:02:24.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:24 smithi005 ceph-mon[30944]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:24.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:24 smithi005 ceph-mon[30944]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:02:24.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:24 smithi005 ceph-mon[30944]: from='client.? 172.21.15.99:0/2582613014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "511bc0a8-e4f3-48e2-824b-7afd2ead90e4"}]: dispatch 2024-09-15T22:02:24.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:24 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "511bc0a8-e4f3-48e2-824b-7afd2ead90e4"}]: dispatch 2024-09-15T22:02:24.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:24 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "511bc0a8-e4f3-48e2-824b-7afd2ead90e4"}]': finished 2024-09-15T22:02:24.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:24 smithi005 ceph-mon[30944]: osdmap e27: 5 total, 4 up, 5 in 2024-09-15T22:02:24.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:24 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:24.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:24 smithi005 ceph-mon[30944]: from='client.? 172.21.15.99:0/2471226329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:02:24.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:24 smithi129 ceph-mon[35314]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:24.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:24 smithi129 ceph-mon[35314]: osdmap e26: 4 total, 4 up, 4 in 2024-09-15T22:02:24.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:24 smithi129 ceph-mon[35314]: from='client.? 172.21.15.99:0/2582613014' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "511bc0a8-e4f3-48e2-824b-7afd2ead90e4"}]: dispatch 2024-09-15T22:02:24.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:24 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "511bc0a8-e4f3-48e2-824b-7afd2ead90e4"}]: dispatch 2024-09-15T22:02:24.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:24 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "511bc0a8-e4f3-48e2-824b-7afd2ead90e4"}]': finished 2024-09-15T22:02:24.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:24 smithi129 ceph-mon[35314]: osdmap e27: 5 total, 4 up, 5 in 2024-09-15T22:02:24.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:24 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:24.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:24 smithi129 ceph-mon[35314]: from='client.? 172.21.15.99:0/2471226329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:02:26.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:26 smithi005 ceph-mon[30944]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:26.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:26 smithi099 ceph-mon[34892]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:26.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:26 smithi129 ceph-mon[35314]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:28.777 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:28 smithi099 ceph-mon[34892]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:28.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:28 smithi129 ceph-mon[35314]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:28.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:28 smithi005 ceph-mon[30944]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:30.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:30 smithi099 ceph-mon[34892]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-15T22:02:30.848 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:30 smithi129 ceph-mon[35314]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-15T22:02:30.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:30 smithi005 ceph-mon[30944]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-09-15T22:02:32.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:32 smithi099 ceph-mon[34892]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:02:32.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:02:32.579 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:32.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:32 smithi129 ceph-mon[35314]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:02:32.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:02:32.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:32.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:32 smithi005 ceph-mon[30944]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:02:32.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-15T22:02:32.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:33.561 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:33 smithi099 ceph-mon[34892]: Deploying daemon osd.4 on smithi099 2024-09-15T22:02:33.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:33 smithi129 ceph-mon[35314]: Deploying daemon osd.4 on smithi099 2024-09-15T22:02:33.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:33 smithi005 ceph-mon[30944]: Deploying daemon osd.4 on smithi099 2024-09-15T22:02:34.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:34 smithi099 ceph-mon[34892]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-15T22:02:34.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:34 smithi129 ceph-mon[35314]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-15T22:02:34.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:34 smithi005 ceph-mon[30944]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-15T22:02:36.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:36 smithi099 ceph-mon[34892]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:02:36.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:36 smithi129 ceph-mon[35314]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:02:36.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:36 smithi005 ceph-mon[30944]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-15T22:02:38.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:38 smithi099 ceph-mon[34892]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:02:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:38 smithi129 ceph-mon[35314]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:02:38.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:38 smithi005 ceph-mon[30944]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:02:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:39 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:39 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:39 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:39.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:39 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:39.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:39 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:39.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:39 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:39.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:39 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:39.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:39 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:39.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:39 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:40.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:40 smithi005 ceph-mon[30944]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:02:40.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:40 smithi099 ceph-mon[34892]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:02:40.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:40 smithi129 ceph-mon[35314]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:02:42.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:42 smithi099 ceph-mon[34892]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:42.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:42 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:42.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:42 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:42.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:42 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:42.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:42 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:42.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:42 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:42.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:42 smithi005 ceph-mon[30944]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:42.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:42.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:42.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:42.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:42.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:42 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:42.418 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.414+0000 7fa1cd7fa640 1 -- 172.21.15.99:0/656714857 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa1d0062630 con 0x7fa1b80a3270 2024-09-15T22:02:42.418 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 4 on host 'smithi099' 2024-09-15T22:02:42.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 -- 172.21.15.99:0/656714857 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1b80a3270 msgr2=0x7fa1b80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:42.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1b80a3270 0x7fa1b80a5730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4000c00 tx=0x7fa1c40315d0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 -- 172.21.15.99:0/656714857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1d0100ad0 msgr2=0x7fa1d006aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:42.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1d0100ad0 0x7fa1d006aad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa1c000d950 tx=0x7fa1c000de10 comp rx=0 tx=0).stop 2024-09-15T22:02:42.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 -- 172.21.15.99:0/656714857 shutdown_connections 2024-09-15T22:02:42.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1b80a3270 0x7fa1b80a5730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.419 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1d0102690 0x7fa1d01a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1d0101cd0 0x7fa1d019f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 --2- 172.21.15.99:0/656714857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1d0100ad0 0x7fa1d006aad0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:42.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 -- 172.21.15.99:0/656714857 >> 172.21.15.99:0/656714857 conn(0x7fa1d00fc280 msgr2=0x7fa1d01050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:42.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 -- 172.21.15.99:0/656714857 shutdown_connections 2024-09-15T22:02:42.420 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:02:42.416+0000 7fa1d697a640 1 -- 172.21.15.99:0/656714857 wait complete. 2024-09-15T22:02:42.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:42 smithi129 ceph-mon[35314]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:42.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:42 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:42.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:42 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:42.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:42 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:42.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:42 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:42.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:42 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:43.081 DEBUG:teuthology.orchestra.run.smithi099:osd.4> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.4.service 2024-09-15T22:02:43.083 INFO:tasks.cephadm:Deploying osd.5 on smithi129 with /dev/vg_nvme/lv_3... 2024-09-15T22:02:43.083 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-15T22:02:43.249 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:43 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:43.249 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:43 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:43.249 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:43 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:43.334 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:02:43.454 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:43 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:43.455 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:43 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:43.455 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:43 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:43.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:43 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:02:43.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:43 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:43.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:43 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:44.522 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:44 smithi099 ceph-mon[34892]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:44.712 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:44 smithi129 ceph-mon[35314]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:44.840 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:02:44 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4[45975]: 2024-09-15T22:02:44.518+0000 7f103cf7b640 -1 osd.4 0 log_to_monitors true 2024-09-15T22:02:44.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:44 smithi005 ceph-mon[30944]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:45.518 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:45 smithi099 ceph-mon[34892]: from='osd.4 [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:02:45.518 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:45 smithi099 ceph-mon[34892]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:02:45.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:45 smithi129 ceph-mon[35314]: from='osd.4 [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:02:45.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:45 smithi129 ceph-mon[35314]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:02:45.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:45 smithi005 ceph-mon[30944]: from='osd.4 [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:02:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:45 smithi005 ceph-mon[30944]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-15T22:02:45.939 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T22:02:45.966 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_3 2024-09-15T22:02:46.211 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:02:46.586 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='osd.4 [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:46.587 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:46.588 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:46 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.674+0000 7f7875837640 1 -- 172.21.15.129:0/143992060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7870102690 msgr2=0x7f7870108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:46.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.674+0000 7f7875837640 1 --2- 172.21.15.129:0/143992060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7870102690 0x7f7870108f20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7868009900 tx=0x7f786802a410 comp rx=0 tx=0).stop 2024-09-15T22:02:46.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.674+0000 7f7875837640 1 -- 172.21.15.129:0/143992060 shutdown_connections 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.674+0000 7f7875837640 1 --2- 172.21.15.129:0/143992060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7870102690 0x7f7870108f20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.674+0000 7f7875837640 1 --2- 172.21.15.129:0/143992060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7870101cd0 0x7f7870102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.674+0000 7f7875837640 1 --2- 172.21.15.129:0/143992060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7870100ad0 0x7f7870100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.674+0000 7f7875837640 1 -- 172.21.15.129:0/143992060 >> 172.21.15.129:0/143992060 conn(0x7f78700fc280 msgr2=0x7f78700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.674+0000 7f7875837640 1 -- 172.21.15.129:0/143992060 shutdown_connections 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.674+0000 7f7875837640 1 -- 172.21.15.129:0/143992060 wait complete. 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f7875837640 1 Processor -- start 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f7875837640 1 -- start start 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f7875837640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7870100ad0 0x7f7870196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f7875837640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7870101cd0 0x7f78701972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f7875837640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7870102690 0x7f787019e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f7875837640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f787010c750 con 0x7f7870102690 2024-09-15T22:02:46.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f7875837640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f787010c5d0 con 0x7f7870100ad0 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f7875837640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f787010c8d0 con 0x7f7870101cd0 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7870101cd0 0x7f78701972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7870101cd0 0x7f78701972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:50316/0 (socket says 172.21.15.129:50316) 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786e7fc640 1 -- 172.21.15.129:0/2960555140 learned_addr learned my addr 172.21.15.129:0/2960555140 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786effd640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7870100ad0 0x7f7870196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786f7fe640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7870102690 0x7f787019e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786e7fc640 1 -- 172.21.15.129:0/2960555140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7870100ad0 msgr2=0x7f7870196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786e7fc640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7870100ad0 0x7f7870196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786e7fc640 1 -- 172.21.15.129:0/2960555140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7870102690 msgr2=0x7f787019e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786e7fc640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7870102690 0x7f787019e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786e7fc640 1 -- 172.21.15.129:0/2960555140 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787006a070 con 0x7f7870101cd0 2024-09-15T22:02:46.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786effd640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7870100ad0 0x7f7870196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.675+0000 7f786f7fe640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7870102690 0x7f787019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.676+0000 7f786e7fc640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7870101cd0 0x7f78701972d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f785800d950 tx=0x7f785800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.676+0000 7f7874835640 1 -- 172.21.15.129:0/2960555140 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7858017070 con 0x7f7870101cd0 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.676+0000 7f7874835640 1 -- 172.21.15.129:0/2960555140 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7858009c30 con 0x7f7870101cd0 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.676+0000 7f7875837640 1 -- 172.21.15.129:0/2960555140 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787006a360 con 0x7f7870101cd0 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.676+0000 7f7874835640 1 -- 172.21.15.129:0/2960555140 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f785800f6b0 con 0x7f7870101cd0 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.676+0000 7f7875837640 1 -- 172.21.15.129:0/2960555140 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f787006a8a0 con 0x7f7870101cd0 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.677+0000 7f7875837640 1 -- 172.21.15.129:0/2960555140 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7870107e20 con 0x7f7870101cd0 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.678+0000 7f7874835640 1 -- 172.21.15.129:0/2960555140 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f78580109d0 con 0x7f7870101cd0 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.679+0000 7f7874835640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f785c0a3240 0x7f785c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.679+0000 7f7874835640 1 -- 172.21.15.129:0/2960555140 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3821+0+0 (secure 0 0 0) 0x7f78580ce410 con 0x7f7870101cd0 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.680+0000 7f786effd640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f785c0a3240 0x7f785c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:02:46.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.680+0000 7f786effd640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f785c0a3240 0x7f785c0a5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7870198270 tx=0x7f7860009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:02:46.719 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.684+0000 7f7874835640 1 -- 172.21.15.129:0/2960555140 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7858009de0 con 0x7f7870101cd0 2024-09-15T22:02:46.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='osd.4 [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:46.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:46 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-15T22:02:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: osdmap e28: 5 total, 4 up, 5 in 2024-09-15T22:02:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='osd.4 [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:02:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:02:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:46.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:02:46.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:46.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:02:46.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:46 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:02:46.865 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:02:46.863+0000 7f7875837640 1 -- 172.21.15.129:0/2960555140 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f787002d830 con 0x7f785c0a3240 2024-09-15T22:02:47.435 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4[45975]: 2024-09-15T22:02:47.091+0000 7f1038efe640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T22:02:47.435 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4[45975]: 2024-09-15T22:02:47.096+0000 7f1030d1e640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:02:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:47 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi099 2024-09-15T22:02:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:47 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi099 to 8544M 2024-09-15T22:02:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:47 smithi129 ceph-mon[35314]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:02:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:47 smithi129 ceph-mon[35314]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:02:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:47 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:47 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:47 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:02:47.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:47 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:02:47.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:47 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:47.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:47 smithi129 ceph-mon[35314]: from='osd.4 ' entity='osd.4' 2024-09-15T22:02:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi099 2024-09-15T22:02:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi099 to 8544M 2024-09-15T22:02:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-mon[34892]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:02:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-mon[34892]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:02:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:02:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:02:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:47 smithi099 ceph-mon[34892]: from='osd.4 ' entity='osd.4' 2024-09-15T22:02:47.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:47 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi099 2024-09-15T22:02:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:47 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi099 to 8544M 2024-09-15T22:02:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:47 smithi005 ceph-mon[30944]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:02:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:47 smithi005 ceph-mon[30944]: osdmap e29: 5 total, 4 up, 5 in 2024-09-15T22:02:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:47 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:47 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:47 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:02:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:47 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:02:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:47 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:47.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:47 smithi005 ceph-mon[30944]: from='osd.4 ' entity='osd.4' 2024-09-15T22:02:48.519 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:48 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:02:48.519 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:48 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:02:48.519 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:48 smithi129 ceph-mon[35314]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:48.519 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:48 smithi129 ceph-mon[35314]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:48.519 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:48.519 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:48 smithi129 ceph-mon[35314]: osd.4 [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025] boot 2024-09-15T22:02:48.519 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:48 smithi129 ceph-mon[35314]: osdmap e30: 5 total, 5 up, 5 in 2024-09-15T22:02:48.519 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:48 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:02:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:48 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:02:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:48 smithi099 ceph-mon[34892]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:48 smithi099 ceph-mon[34892]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:48.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:48 smithi099 ceph-mon[34892]: osd.4 [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025] boot 2024-09-15T22:02:48.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:48 smithi099 ceph-mon[34892]: osdmap e30: 5 total, 5 up, 5 in 2024-09-15T22:02:48.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:48.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:48 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:02:48.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:48 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:02:48.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:48 smithi005 ceph-mon[30944]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-09-15T22:02:48.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:48 smithi005 ceph-mon[30944]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:02:48.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:48.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:48 smithi005 ceph-mon[30944]: osd.4 [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025] boot 2024-09-15T22:02:48.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:48 smithi005 ceph-mon[30944]: osdmap e30: 5 total, 5 up, 5 in 2024-09-15T22:02:48.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:02:49.815 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:49 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/1665767963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5524a904-d0c6-4276-9e11-2dcf0e6ec037"}]: dispatch 2024-09-15T22:02:49.815 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:49 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5524a904-d0c6-4276-9e11-2dcf0e6ec037"}]: dispatch 2024-09-15T22:02:49.815 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:49 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5524a904-d0c6-4276-9e11-2dcf0e6ec037"}]': finished 2024-09-15T22:02:49.815 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:49 smithi129 ceph-mon[35314]: osdmap e31: 6 total, 5 up, 6 in 2024-09-15T22:02:49.815 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:49 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:02:49.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:49 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/1665767963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5524a904-d0c6-4276-9e11-2dcf0e6ec037"}]: dispatch 2024-09-15T22:02:49.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:49 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5524a904-d0c6-4276-9e11-2dcf0e6ec037"}]: dispatch 2024-09-15T22:02:49.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:49 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5524a904-d0c6-4276-9e11-2dcf0e6ec037"}]': finished 2024-09-15T22:02:49.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:49 smithi099 ceph-mon[34892]: osdmap e31: 6 total, 5 up, 6 in 2024-09-15T22:02:49.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:49 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:02:49.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:49 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/1665767963' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5524a904-d0c6-4276-9e11-2dcf0e6ec037"}]: dispatch 2024-09-15T22:02:49.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:49 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5524a904-d0c6-4276-9e11-2dcf0e6ec037"}]: dispatch 2024-09-15T22:02:49.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:49 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5524a904-d0c6-4276-9e11-2dcf0e6ec037"}]': finished 2024-09-15T22:02:49.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:49 smithi005 ceph-mon[30944]: osdmap e31: 6 total, 5 up, 6 in 2024-09-15T22:02:49.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:49 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:02:50.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:50 smithi099 ceph-mon[34892]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:50.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:50 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/2167670222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:02:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:50 smithi129 ceph-mon[35314]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:50 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/2167670222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:02:50.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:50 smithi005 ceph-mon[30944]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:50.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:50 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/2167670222' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:02:52.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:52 smithi099 ceph-mon[34892]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:52.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:52 smithi129 ceph-mon[35314]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:52.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:52 smithi005 ceph-mon[30944]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:54.772 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:54 smithi129 ceph-mon[35314]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:54.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:54 smithi099 ceph-mon[34892]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:54.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:54 smithi005 ceph-mon[30944]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:56.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:56 smithi129 ceph-mon[35314]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:56.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:56 smithi099 ceph-mon[34892]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:56.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:56 smithi005 ceph-mon[30944]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:57.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:57 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:02:57.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:57 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:57.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:57 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:02:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:57 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:57.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:57 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-15T22:02:57.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:57 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:02:58.771 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:58 smithi129 ceph-mon[35314]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:58.771 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:02:58 smithi129 ceph-mon[35314]: Deploying daemon osd.5 on smithi129 2024-09-15T22:02:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:58 smithi099 ceph-mon[34892]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:02:58 smithi099 ceph-mon[34892]: Deploying daemon osd.5 on smithi129 2024-09-15T22:02:58.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:58 smithi005 ceph-mon[30944]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:02:58.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:02:58 smithi005 ceph-mon[30944]: Deploying daemon osd.5 on smithi129 2024-09-15T22:03:00.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:00 smithi099 ceph-mon[34892]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:00.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:00 smithi129 ceph-mon[35314]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:00.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:00 smithi005 ceph-mon[30944]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:02.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:02 smithi129 ceph-mon[35314]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:02 smithi099 ceph-mon[34892]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:02.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:02 smithi005 ceph-mon[30944]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:04.588 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:04 smithi129 ceph-mon[35314]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:04.588 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:04 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:04.588 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:04 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:04.588 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:04 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:04.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:04 smithi099 ceph-mon[34892]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:04.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:04 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:04.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:04 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:04.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:04 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:04.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:04 smithi005 ceph-mon[30944]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:04.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:04 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:04.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:04 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:04.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:04 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:06.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:06 smithi099 ceph-mon[34892]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:06 smithi129 ceph-mon[35314]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:06.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:06 smithi005 ceph-mon[30944]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:07.994 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:07.994 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:07.994 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:07.994 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:07.994 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:08.034 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 5 on host 'smithi129' 2024-09-15T22:03:08.034 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.032+0000 7f7874835640 1 -- 172.21.15.129:0/2960555140 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f787002d830 con 0x7f785c0a3240 2024-09-15T22:03:08.036 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 -- 172.21.15.129:0/2960555140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f785c0a3240 msgr2=0x7f785c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:08.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f785c0a3240 0x7f785c0a5700 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7870198270 tx=0x7f7860009500 comp rx=0 tx=0).stop 2024-09-15T22:03:08.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 -- 172.21.15.129:0/2960555140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7870101cd0 msgr2=0x7f78701972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:08.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7870101cd0 0x7f78701972d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f785800d950 tx=0x7f785800de10 comp rx=0 tx=0).stop 2024-09-15T22:03:08.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 -- 172.21.15.129:0/2960555140 shutdown_connections 2024-09-15T22:03:08.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f785c0a3240 0x7f785c0a5700 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:08.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7870102690 0x7f787019e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:08.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7870101cd0 0x7f78701972d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:08.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 --2- 172.21.15.129:0/2960555140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7870100ad0 0x7f7870196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:08.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 -- 172.21.15.129:0/2960555140 >> 172.21.15.129:0/2960555140 conn(0x7f78700fc280 msgr2=0x7f7870104ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:08.037 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 -- 172.21.15.129:0/2960555140 shutdown_connections 2024-09-15T22:03:08.038 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:03:08.035+0000 7f7875837640 1 -- 172.21.15.129:0/2960555140 wait complete. 2024-09-15T22:03:08.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:08.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:08.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:08.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:08.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:08.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:08.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:08.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:08.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:08.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:08.673 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:08 smithi129 ceph-mon[35314]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:08.673 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:08 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:08.673 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:08 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:08.673 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:08 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:08.699 DEBUG:teuthology.orchestra.run.smithi129:osd.5> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.5.service 2024-09-15T22:03:08.701 INFO:tasks.cephadm:Deploying osd.6 on smithi005 with /dev/vg_nvme/lv_2... 2024-09-15T22:03:08.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:03:08.969 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:09.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:08 smithi099 ceph-mon[34892]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:09.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:08 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:09.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:08 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:09.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:08 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:09.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:08 smithi005 ceph-mon[30944]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:09.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:08 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:09.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:08 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:09.090 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:08 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:10.047 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:09 smithi129 ceph-mon[35314]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:10.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:09 smithi099 ceph-mon[34892]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:10.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:09 smithi005 ceph-mon[30944]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:10.316 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:03:10 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:03:10.180+0000 7f7b890c1640 -1 osd.5 0 log_to_monitors true 2024-09-15T22:03:10.934 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:10 smithi005 ceph-mon[30944]: from='osd.5 [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:03:10.934 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:10 smithi005 ceph-mon[30944]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:03:11.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:10 smithi099 ceph-mon[34892]: from='osd.5 [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:03:11.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:10 smithi099 ceph-mon[34892]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:03:11.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:10 smithi129 ceph-mon[35314]: from='osd.5 [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:03:11.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:10 smithi129 ceph-mon[35314]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-15T22:03:11.619 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:03:11.649 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_2 2024-09-15T22:03:11.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:03:11.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: osdmap e32: 6 total, 5 up, 6 in 2024-09-15T22:03:11.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:11.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='osd.5 [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:03:11.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:03:11.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:11.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi129 2024-09-15T22:03:11.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:11.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:11.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:11.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:11.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi129 to 8544M 2024-09-15T22:03:11.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:11.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:11.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:11.848 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:11 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:11.910 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:03:12.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:03:12.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: osdmap e32: 6 total, 5 up, 6 in 2024-09-15T22:03:12.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:12.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='osd.5 [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:03:12.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:03:12.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:12.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi129 2024-09-15T22:03:12.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:12.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:12.033 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:12.033 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:12.033 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi129 to 8544M 2024-09-15T22:03:12.033 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:12.033 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:12.033 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:12.033 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:11 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:12.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-15T22:03:12.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: osdmap e32: 6 total, 5 up, 6 in 2024-09-15T22:03:12.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:12.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='osd.5 [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:03:12.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:03:12.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:12.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi129 2024-09-15T22:03:12.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:12.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:12.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:12.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:12.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi129 to 8544M 2024-09-15T22:03:12.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:12.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:12.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:12.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:11 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.370+0000 7fe19a260640 1 -- 172.21.15.5:0/108227975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194102e10 msgr2=0x7fe194103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.370+0000 7fe19a260640 1 --2- 172.21.15.5:0/108227975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194102e10 0x7fe194103290 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe188009900 tx=0x7fe18802a410 comp rx=0 tx=0).stop 2024-09-15T22:03:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.370+0000 7fe19a260640 1 -- 172.21.15.5:0/108227975 shutdown_connections 2024-09-15T22:03:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.370+0000 7fe19a260640 1 --2- 172.21.15.5:0/108227975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe1941037d0 0x7fe19410a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.370+0000 7fe19a260640 1 --2- 172.21.15.5:0/108227975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe194102e10 0x7fe194103290 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.370+0000 7fe19a260640 1 --2- 172.21.15.5:0/108227975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe194101c10 0x7fe194102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.370+0000 7fe19a260640 1 -- 172.21.15.5:0/108227975 >> 172.21.15.5:0/108227975 conn(0x7fe1940fd3c0 msgr2=0x7fe1940ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.370+0000 7fe19a260640 1 -- 172.21.15.5:0/108227975 shutdown_connections 2024-09-15T22:03:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.370+0000 7fe19a260640 1 -- 172.21.15.5:0/108227975 wait complete. 2024-09-15T22:03:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.371+0000 7fe19a260640 1 Processor -- start 2024-09-15T22:03:12.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.371+0000 7fe19a260640 1 -- start start 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe19a260640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe194101c10 0x7fe194197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe19a260640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe194102e10 0x7fe1941984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe19a260640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1941037d0 0x7fe19419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe19a260640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe19410d910 con 0x7fe1941037d0 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe19a260640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe19410d790 con 0x7fe194101c10 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe19a260640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe19410da90 con 0x7fe194102e10 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe193fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1941037d0 0x7fe19419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe193fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1941037d0 0x7fe19419f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34892/0 (socket says 172.21.15.5:34892) 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe192ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe194102e10 0x7fe1941984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe193fff640 1 -- 172.21.15.5:0/3026191779 learned_addr learned my addr 172.21.15.5:0/3026191779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe1937fe640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe194101c10 0x7fe194197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:12.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe193fff640 1 -- 172.21.15.5:0/3026191779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe194102e10 msgr2=0x7fe1941984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe193fff640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe194102e10 0x7fe1941984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe193fff640 1 -- 172.21.15.5:0/3026191779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe194101c10 msgr2=0x7fe194197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe193fff640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe194101c10 0x7fe194197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe193fff640 1 -- 172.21.15.5:0/3026191779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19419fb90 con 0x7fe1941037d0 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe1937fe640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe194101c10 0x7fe194197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe193fff640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1941037d0 0x7fe19419f520 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00eaa0 tx=0x7fe18c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe190ff9640 1 -- 172.21.15.5:0/3026191779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c00cdb0 con 0x7fe1941037d0 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe190ff9640 1 -- 172.21.15.5:0/3026191779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe18c005020 con 0x7fe1941037d0 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe19a260640 1 -- 172.21.15.5:0/3026191779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19419fe80 con 0x7fe1941037d0 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.372+0000 7fe190ff9640 1 -- 172.21.15.5:0/3026191779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0116b0 con 0x7fe1941037d0 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.373+0000 7fe19a260640 1 -- 172.21.15.5:0/3026191779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1941a0360 con 0x7fe1941037d0 2024-09-15T22:03:12.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.373+0000 7fe19a260640 1 -- 172.21.15.5:0/3026191779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe194109060 con 0x7fe1941037d0 2024-09-15T22:03:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.375+0000 7fe190ff9640 1 -- 172.21.15.5:0/3026191779 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fe18c005560 con 0x7fe1941037d0 2024-09-15T22:03:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.375+0000 7fe190ff9640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe17c0a3270 0x7fe17c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.375+0000 7fe190ff9640 1 -- 172.21.15.5:0/3026191779 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4253+0+0 (secure 0 0 0) 0x7fe18c0cec00 con 0x7fe1941037d0 2024-09-15T22:03:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.375+0000 7fe1937fe640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe17c0a3270 0x7fe17c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.376+0000 7fe1937fe640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe17c0a3270 0x7fe17c0a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe184006fd0 tx=0x7fe184008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:12.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.380+0000 7fe190ff9640 1 -- 172.21.15.5:0/3026191779 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe18c016030 con 0x7fe1941037d0 2024-09-15T22:03:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:12.565+0000 7fe19a260640 1 -- 172.21.15.5:0/3026191779 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe194062630 con 0x7fe17c0a3270 2024-09-15T22:03:12.730 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:03:12 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:03:12.389+0000 7f7b85044640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T22:03:12.730 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:03:12 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:03:12.394+0000 7f7b7ce64640 -1 osd.5 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:03:12.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:12 smithi005 ceph-mon[30944]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:03:12.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:12 smithi005 ceph-mon[30944]: osdmap e33: 6 total, 5 up, 6 in 2024-09-15T22:03:12.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:12.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:12.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:12 smithi005 ceph-mon[30944]: from='osd.5 ' entity='osd.5' 2024-09-15T22:03:12.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:12 smithi005 ceph-mon[30944]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:12.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:03:12.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:03:12.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:13.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:12 smithi099 ceph-mon[34892]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:03:13.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:12 smithi099 ceph-mon[34892]: osdmap e33: 6 total, 5 up, 6 in 2024-09-15T22:03:13.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:12 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:13.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:12 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:13.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:12 smithi099 ceph-mon[34892]: from='osd.5 ' entity='osd.5' 2024-09-15T22:03:13.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:12 smithi099 ceph-mon[34892]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:13.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:12 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:03:13.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:12 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:03:13.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:12 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:12 smithi129 ceph-mon[35314]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:03:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:12 smithi129 ceph-mon[35314]: osdmap e33: 6 total, 5 up, 6 in 2024-09-15T22:03:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:12 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:12 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:12 smithi129 ceph-mon[35314]: from='osd.5 ' entity='osd.5' 2024-09-15T22:03:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:12 smithi129 ceph-mon[35314]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:12 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:03:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:12 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:03:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:12 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:13.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:13 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:03:13.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:13 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:03:13.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:13 smithi005 ceph-mon[30944]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:13.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:13 smithi005 ceph-mon[30944]: osd.5 [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949] boot 2024-09-15T22:03:13.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:13 smithi005 ceph-mon[30944]: osdmap e34: 6 total, 6 up, 6 in 2024-09-15T22:03:13.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:13 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:13.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:13 smithi005 ceph-mon[30944]: osdmap e35: 6 total, 6 up, 6 in 2024-09-15T22:03:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:13 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:03:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:13 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:03:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:13 smithi099 ceph-mon[34892]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:13 smithi099 ceph-mon[34892]: osd.5 [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949] boot 2024-09-15T22:03:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:13 smithi099 ceph-mon[34892]: osdmap e34: 6 total, 6 up, 6 in 2024-09-15T22:03:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:13 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:13 smithi099 ceph-mon[34892]: osdmap e35: 6 total, 6 up, 6 in 2024-09-15T22:03:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:13 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:03:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:13 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:03:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:13 smithi129 ceph-mon[35314]: pgmap v129: 1 pgs: 1 peering; 577 KiB data, 163 MiB used, 447 GiB / 447 GiB avail 2024-09-15T22:03:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:13 smithi129 ceph-mon[35314]: osd.5 [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949] boot 2024-09-15T22:03:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:13 smithi129 ceph-mon[35314]: osdmap e34: 6 total, 6 up, 6 in 2024-09-15T22:03:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:13 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:03:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:13 smithi129 ceph-mon[35314]: osdmap e35: 6 total, 6 up, 6 in 2024-09-15T22:03:15.592 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:15 smithi005 ceph-mon[30944]: osdmap e36: 6 total, 6 up, 6 in 2024-09-15T22:03:15.592 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:15 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4019109263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5b89d06-6596-4cd8-9083-3e023d156603"}]: dispatch 2024-09-15T22:03:15.592 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:15 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4019109263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5b89d06-6596-4cd8-9083-3e023d156603"}]': finished 2024-09-15T22:03:15.592 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:15 smithi005 ceph-mon[30944]: osdmap e37: 7 total, 6 up, 7 in 2024-09-15T22:03:15.592 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:15 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:15.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:15 smithi099 ceph-mon[34892]: osdmap e36: 6 total, 6 up, 6 in 2024-09-15T22:03:15.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:15 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4019109263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5b89d06-6596-4cd8-9083-3e023d156603"}]: dispatch 2024-09-15T22:03:15.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:15 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4019109263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5b89d06-6596-4cd8-9083-3e023d156603"}]': finished 2024-09-15T22:03:15.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:15 smithi099 ceph-mon[34892]: osdmap e37: 7 total, 6 up, 7 in 2024-09-15T22:03:15.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:15 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:15.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:15 smithi129 ceph-mon[35314]: osdmap e36: 6 total, 6 up, 6 in 2024-09-15T22:03:15.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:15 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4019109263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a5b89d06-6596-4cd8-9083-3e023d156603"}]: dispatch 2024-09-15T22:03:15.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:15 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4019109263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a5b89d06-6596-4cd8-9083-3e023d156603"}]': finished 2024-09-15T22:03:15.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:15 smithi129 ceph-mon[35314]: osdmap e37: 7 total, 6 up, 7 in 2024-09-15T22:03:15.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:15 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:16.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:16 smithi099 ceph-mon[34892]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:16.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:16 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3652391879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:03:16.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:16 smithi099 ceph-mon[34892]: osdmap e38: 7 total, 6 up, 7 in 2024-09-15T22:03:16.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:16 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:16.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:16 smithi129 ceph-mon[35314]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:16.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:16 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3652391879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:03:16.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:16 smithi129 ceph-mon[35314]: osdmap e38: 7 total, 6 up, 7 in 2024-09-15T22:03:16.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:16 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:16.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:16 smithi005 ceph-mon[30944]: pgmap v132: 1 pgs: 1 peering; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:16.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:16 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3652391879' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:03:16.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:16 smithi005 ceph-mon[30944]: osdmap e38: 7 total, 6 up, 7 in 2024-09-15T22:03:16.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:16 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:18.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:18 smithi099 ceph-mon[34892]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:18.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:18 smithi129 ceph-mon[35314]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:18.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:18 smithi005 ceph-mon[30944]: pgmap v135: 1 pgs: 1 peering; 577 KiB data, 590 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:20.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:20 smithi005 ceph-mon[30944]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-15T22:03:20.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:20 smithi099 ceph-mon[34892]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-15T22:03:20.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:20 smithi129 ceph-mon[35314]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-09-15T22:03:22.647 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:22 smithi129 ceph-mon[35314]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-15T22:03:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:22 smithi005 ceph-mon[30944]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-15T22:03:22.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:22 smithi099 ceph-mon[34892]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-15T22:03:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:23 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:03:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:23 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:23.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:23 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:03:23.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:23 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:23.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:23 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-15T22:03:23.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:23 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:24.634 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:24 smithi005 ceph-mon[30944]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:03:24.634 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:24 smithi005 ceph-mon[30944]: Deploying daemon osd.6 on smithi005 2024-09-15T22:03:24.741 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:24 smithi099 ceph-mon[34892]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:03:24.741 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:24 smithi099 ceph-mon[34892]: Deploying daemon osd.6 on smithi005 2024-09-15T22:03:24.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:24 smithi129 ceph-mon[35314]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-15T22:03:24.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:24 smithi129 ceph-mon[35314]: Deploying daemon osd.6 on smithi005 2024-09-15T22:03:26.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:26 smithi005 ceph-mon[30944]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T22:03:26.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:26 smithi099 ceph-mon[34892]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T22:03:26.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:26 smithi129 ceph-mon[35314]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-15T22:03:28.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:28 smithi005 ceph-mon[30944]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-15T22:03:28.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:28 smithi099 ceph-mon[34892]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-15T22:03:28.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:28 smithi129 ceph-mon[35314]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-15T22:03:29.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:29.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:29.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:29.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:29.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:29.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:29.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:29.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:29.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:30.765 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:30 smithi005 ceph-mon[30944]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:03:30.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:30 smithi099 ceph-mon[34892]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:03:30.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:30 smithi129 ceph-mon[35314]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:03:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:32 smithi005 ceph-mon[30944]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:32.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:32.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:32.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:32.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:32 smithi099 ceph-mon[34892]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:32.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:32.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:32 smithi129 ceph-mon[35314]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:32.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:32.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:32.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:32.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:32.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.064+0000 7fe190ff9640 1 -- 172.21.15.5:0/3026191779 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe194062630 con 0x7fe17c0a3270 2024-09-15T22:03:33.066 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 6 on host 'smithi005' 2024-09-15T22:03:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 -- 172.21.15.5:0/3026191779 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe17c0a3270 msgr2=0x7fe17c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe17c0a3270 0x7fe17c0a5730 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe184006fd0 tx=0x7fe184008040 comp rx=0 tx=0).stop 2024-09-15T22:03:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 -- 172.21.15.5:0/3026191779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1941037d0 msgr2=0x7fe19419f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1941037d0 0x7fe19419f520 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00eaa0 tx=0x7fe18c00ef60 comp rx=0 tx=0).stop 2024-09-15T22:03:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 -- 172.21.15.5:0/3026191779 shutdown_connections 2024-09-15T22:03:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe17c0a3270 0x7fe17c0a5730 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1941037d0 0x7fe19419f520 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe194102e10 0x7fe1941984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 --2- 172.21.15.5:0/3026191779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe194101c10 0x7fe194197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 -- 172.21.15.5:0/3026191779 >> 172.21.15.5:0/3026191779 conn(0x7fe1940fd3c0 msgr2=0x7fe194106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 -- 172.21.15.5:0/3026191779 shutdown_connections 2024-09-15T22:03:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:03:33.066+0000 7fe19a260640 1 -- 172.21.15.5:0/3026191779 wait complete. 2024-09-15T22:03:33.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:33.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:33.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:33 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:33.741 DEBUG:teuthology.orchestra.run.smithi005:osd.6> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.6.service 2024-09-15T22:03:33.744 INFO:tasks.cephadm:Deploying osd.7 on smithi099 with /dev/vg_nvme/lv_2... 2024-09-15T22:03:33.744 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:03:33.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:33.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:33.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:33 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:33.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:33.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:33.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:33 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:33.996 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:03:34.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:34 smithi099 ceph-mon[34892]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:34.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:34 smithi129 ceph-mon[35314]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:34.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:34 smithi005 ceph-mon[30944]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:35.106 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:03:34 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:03:34.848+0000 7fd249058640 -1 osd.6 0 log_to_monitors true 2024-09-15T22:03:35.475 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:35 smithi099 ceph-mon[34892]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:03:35.524 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:35 smithi005 ceph-mon[30944]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:03:35.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:35 smithi129 ceph-mon[35314]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-15T22:03:36.747 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:03:36 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:03:36.418+0000 7fd244fdb640 -1 osd.6 0 waiting for initial osdmap 2024-09-15T22:03:36.747 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:03:36 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:03:36.425+0000 7fd240e03640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:03:36.747 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:36 smithi005 ceph-mon[30944]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:36.748 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:36 smithi005 ceph-mon[30944]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:03:36.748 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:36 smithi005 ceph-mon[30944]: osdmap e39: 7 total, 6 up, 7 in 2024-09-15T22:03:36.748 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:36 smithi005 ceph-mon[30944]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:03:36.748 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:36 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:36.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:36 smithi099 ceph-mon[34892]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:36.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:36 smithi099 ceph-mon[34892]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:03:36.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:36 smithi099 ceph-mon[34892]: osdmap e39: 7 total, 6 up, 7 in 2024-09-15T22:03:36.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:36 smithi099 ceph-mon[34892]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:03:36.771 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:36 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:36.773 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T22:03:36.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_2 2024-09-15T22:03:36.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:36 smithi129 ceph-mon[35314]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:36.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:36 smithi129 ceph-mon[35314]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-15T22:03:36.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:36 smithi129 ceph-mon[35314]: osdmap e39: 7 total, 6 up, 7 in 2024-09-15T22:03:36.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:36 smithi129 ceph-mon[35314]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:03:36.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:36 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:37.096 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:03:37.474 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:03:37.474 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T22:03:37.474 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:37.474 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:37.474 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:37.474 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:37.475 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:37.475 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:37.475 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:37.475 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:37.475 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:37.475 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:37.475 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:37.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.555+0000 7f2e0f196640 1 -- 172.21.15.99:0/4233455428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e081037c0 msgr2=0x7f2e0810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:37.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.555+0000 7f2e0f196640 1 --2- 172.21.15.99:0/4233455428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e081037c0 0x7f2e0810a050 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2e00009ca0 tx=0x7f2e0002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:03:37.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.555+0000 7f2e0f196640 1 -- 172.21.15.99:0/4233455428 shutdown_connections 2024-09-15T22:03:37.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.555+0000 7f2e0f196640 1 --2- 172.21.15.99:0/4233455428 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e081037c0 0x7f2e0810a050 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:37.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.555+0000 7f2e0f196640 1 --2- 172.21.15.99:0/4233455428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e08102e00 0x7f2e08103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:37.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.555+0000 7f2e0f196640 1 --2- 172.21.15.99:0/4233455428 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e08101c00 0x7f2e08102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:37.574 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.555+0000 7f2e0f196640 1 -- 172.21.15.99:0/4233455428 >> 172.21.15.99:0/4233455428 conn(0x7f2e080fd390 msgr2=0x7f2e080ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:37.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.555+0000 7f2e0f196640 1 -- 172.21.15.99:0/4233455428 shutdown_connections 2024-09-15T22:03:37.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.555+0000 7f2e0f196640 1 -- 172.21.15.99:0/4233455428 wait complete. 2024-09-15T22:03:37.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e0f196640 1 Processor -- start 2024-09-15T22:03:37.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e0f196640 1 -- start start 2024-09-15T22:03:37.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e0f196640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e08101c00 0x7f2e08075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:37.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e0f196640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e08102e00 0x7f2e080745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:37.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e0f196640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e081037c0 0x7f2e08074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:37.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e0f196640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810d900 con 0x7f2e08101c00 2024-09-15T22:03:37.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e0f196640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810d780 con 0x7f2e08102e00 2024-09-15T22:03:37.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e0f196640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810da80 con 0x7f2e081037c0 2024-09-15T22:03:37.575 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e07fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e08102e00 0x7f2e080745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:37.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e07fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e08102e00 0x7f2e080745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:33346/0 (socket says 172.21.15.99:33346) 2024-09-15T22:03:37.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e07fff640 1 -- 172.21.15.99:0/1299202753 learned_addr learned my addr 172.21.15.99:0/1299202753 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-15T22:03:37.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e0cf32640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e08101c00 0x7f2e08075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:37.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e0d733640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e081037c0 0x7f2e08074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:37.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e07fff640 1 -- 172.21.15.99:0/1299202753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e081037c0 msgr2=0x7f2e08074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:37.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e07fff640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e081037c0 0x7f2e08074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:37.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e07fff640 1 -- 172.21.15.99:0/1299202753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e08101c00 msgr2=0x7f2e08075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:37.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e07fff640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e08101c00 0x7f2e08075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:37.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e07fff640 1 -- 172.21.15.99:0/1299202753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e080753e0 con 0x7f2e08102e00 2024-09-15T22:03:37.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.556+0000 7f2e0d733640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e081037c0 0x7f2e08074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:03:37.576 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.557+0000 7f2e07fff640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e08102e00 0x7f2e080745e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2df400c890 tx=0x7f2df400cc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.557+0000 7f2e05ffb640 1 -- 172.21.15.99:0/1299202753 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4004220 con 0x7f2e08102e00 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.557+0000 7f2e05ffb640 1 -- 172.21.15.99:0/1299202753 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2df40043c0 con 0x7f2e08102e00 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.557+0000 7f2e0f196640 1 -- 172.21.15.99:0/1299202753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e081a20b0 con 0x7f2e08102e00 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.557+0000 7f2e05ffb640 1 -- 172.21.15.99:0/1299202753 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4004a00 con 0x7f2e08102e00 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.557+0000 7f2e0f196640 1 -- 172.21.15.99:0/1299202753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e081a2640 con 0x7f2e08102e00 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.558+0000 7f2e0f196640 1 -- 172.21.15.99:0/1299202753 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e08108fb0 con 0x7f2e08102e00 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.559+0000 7f2e05ffb640 1 -- 172.21.15.99:0/1299202753 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f2df40079b0 con 0x7f2e08102e00 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.560+0000 7f2e05ffb640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2df80a3240 0x7f2df80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.560+0000 7f2e05ffb640 1 -- 172.21.15.99:0/1299202753 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4965+0+0 (secure 0 0 0) 0x7f2df40ce8d0 con 0x7f2e08102e00 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.560+0000 7f2e0cf32640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2df80a3240 0x7f2df80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.560+0000 7f2e0cf32640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2df80a3240 0x7f2df80a5700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e08102c60 tx=0x7f2dfc009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:03:37.577 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.564+0000 7f2e05ffb640 1 -- 172.21.15.99:0/1299202753 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2df4014030 con 0x7f2e08102e00 2024-09-15T22:03:37.739 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:37.737+0000 7f2e0f196640 1 -- 172.21.15.99:0/1299202753 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2e08062630 con 0x7f2df80a3240 2024-09-15T22:03:37.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:03:37.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T22:03:37.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:37.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:37.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:37.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:37.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:37.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:37.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:37.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:37.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:37.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:37.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:37.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: osdmap e40: 7 total, 6 up, 7 in 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:38.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:38 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:03:38.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:38 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:03:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:38 smithi099 ceph-mon[34892]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:38 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi005 2024-09-15T22:03:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:38 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-15T22:03:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:38 smithi099 ceph-mon[34892]: osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223] boot 2024-09-15T22:03:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:38 smithi099 ceph-mon[34892]: osdmap e41: 7 total, 7 up, 7 in 2024-09-15T22:03:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:03:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:03:38.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:38 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:03:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:38 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:03:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:38 smithi129 ceph-mon[35314]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:38 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi005 2024-09-15T22:03:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:38 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-15T22:03:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:38 smithi129 ceph-mon[35314]: osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223] boot 2024-09-15T22:03:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:38 smithi129 ceph-mon[35314]: osdmap e41: 7 total, 7 up, 7 in 2024-09-15T22:03:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:03:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:03:38.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:38.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:38 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:03:38.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:38 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:03:38.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:38 smithi005 ceph-mon[30944]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 536 GiB / 536 GiB avail 2024-09-15T22:03:38.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:38 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi005 2024-09-15T22:03:38.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:38 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-15T22:03:38.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:38 smithi005 ceph-mon[30944]: osd.6 [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223] boot 2024-09-15T22:03:38.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:38 smithi005 ceph-mon[30944]: osdmap e41: 7 total, 7 up, 7 in 2024-09-15T22:03:38.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:03:38.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:03:38.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:03:38.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:39.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:39 smithi099 ceph-mon[34892]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:39.778 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:39 smithi099 ceph-mon[34892]: osdmap e42: 7 total, 7 up, 7 in 2024-09-15T22:03:39.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:39 smithi129 ceph-mon[35314]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:39.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:39 smithi129 ceph-mon[35314]: osdmap e42: 7 total, 7 up, 7 in 2024-09-15T22:03:39.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:39 smithi005 ceph-mon[30944]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:03:39.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:39 smithi005 ceph-mon[30944]: osdmap e42: 7 total, 7 up, 7 in 2024-09-15T22:03:40.683 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:40 smithi099 ceph-mon[34892]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:40.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:40 smithi099 ceph-mon[34892]: from='client.? 172.21.15.99:0/311907085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e8598f5-b1c0-4196-8668-780ad8f79e76"}]: dispatch 2024-09-15T22:03:40.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:40 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e8598f5-b1c0-4196-8668-780ad8f79e76"}]: dispatch 2024-09-15T22:03:40.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:40 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e8598f5-b1c0-4196-8668-780ad8f79e76"}]': finished 2024-09-15T22:03:40.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:40 smithi099 ceph-mon[34892]: osdmap e43: 8 total, 7 up, 8 in 2024-09-15T22:03:40.684 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:03:40.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:40 smithi005 ceph-mon[30944]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:40.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:40 smithi005 ceph-mon[30944]: from='client.? 172.21.15.99:0/311907085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e8598f5-b1c0-4196-8668-780ad8f79e76"}]: dispatch 2024-09-15T22:03:40.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:40 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e8598f5-b1c0-4196-8668-780ad8f79e76"}]: dispatch 2024-09-15T22:03:40.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:40 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e8598f5-b1c0-4196-8668-780ad8f79e76"}]': finished 2024-09-15T22:03:40.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:40 smithi005 ceph-mon[30944]: osdmap e43: 8 total, 7 up, 8 in 2024-09-15T22:03:40.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:03:40.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:40 smithi129 ceph-mon[35314]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:40.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:40 smithi129 ceph-mon[35314]: from='client.? 172.21.15.99:0/311907085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e8598f5-b1c0-4196-8668-780ad8f79e76"}]: dispatch 2024-09-15T22:03:40.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:40 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e8598f5-b1c0-4196-8668-780ad8f79e76"}]: dispatch 2024-09-15T22:03:40.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:40 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e8598f5-b1c0-4196-8668-780ad8f79e76"}]': finished 2024-09-15T22:03:40.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:40 smithi129 ceph-mon[35314]: osdmap e43: 8 total, 7 up, 8 in 2024-09-15T22:03:40.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:03:41.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:41 smithi099 ceph-mon[34892]: from='client.? 172.21.15.99:0/3319512976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:03:41.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:41 smithi129 ceph-mon[35314]: from='client.? 172.21.15.99:0/3319512976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:03:41.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:41 smithi005 ceph-mon[30944]: from='client.? 172.21.15.99:0/3319512976' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:03:42.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:42 smithi099 ceph-mon[34892]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:42.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:42 smithi129 ceph-mon[35314]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:42.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:42 smithi005 ceph-mon[30944]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:44.722 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:44 smithi099 ceph-mon[34892]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:44 smithi129 ceph-mon[35314]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:44.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:44 smithi005 ceph-mon[30944]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:46.747 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:46 smithi099 ceph-mon[34892]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:46.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:46 smithi129 ceph-mon[35314]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:46.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:46 smithi005 ceph-mon[30944]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:48.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:48 smithi099 ceph-mon[34892]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:48.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:03:48.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:48 smithi129 ceph-mon[35314]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:03:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:48.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:48 smithi005 ceph-mon[30944]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:48.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-15T22:03:48.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:49.721 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:49 smithi099 ceph-mon[34892]: Deploying daemon osd.7 on smithi099 2024-09-15T22:03:49.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:49 smithi129 ceph-mon[35314]: Deploying daemon osd.7 on smithi099 2024-09-15T22:03:49.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:49 smithi005 ceph-mon[30944]: Deploying daemon osd.7 on smithi099 2024-09-15T22:03:50.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:50 smithi099 ceph-mon[34892]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:50 smithi129 ceph-mon[35314]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:50.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:50 smithi005 ceph-mon[30944]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:52.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:52 smithi099 ceph-mon[34892]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:52.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:52 smithi129 ceph-mon[35314]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:52.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:52 smithi005 ceph-mon[30944]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:54.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:54 smithi099 ceph-mon[34892]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:54.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:54 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:54.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:54 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:54.834 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:54 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:54.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:54 smithi129 ceph-mon[35314]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:54.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:54 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:54.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:54 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:54.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:54 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:54.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:54 smithi005 ceph-mon[30944]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:54.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:54 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:54.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:54 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:54.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:54 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:56.833 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:56 smithi099 ceph-mon[34892]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:56 smithi129 ceph-mon[35314]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:56.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:56 smithi005 ceph-mon[30944]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:58.221 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.219+0000 7f2e05ffb640 1 -- 172.21.15.99:0/1299202753 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2e08062630 con 0x7f2df80a3240 2024-09-15T22:03:58.221 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 7 on host 'smithi099' 2024-09-15T22:03:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 -- 172.21.15.99:0/1299202753 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2df80a3240 msgr2=0x7f2df80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2df80a3240 0x7f2df80a5700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e08102c60 tx=0x7f2dfc009110 comp rx=0 tx=0).stop 2024-09-15T22:03:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 -- 172.21.15.99:0/1299202753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e08102e00 msgr2=0x7f2e080745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:03:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e08102e00 0x7f2e080745e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2df400c890 tx=0x7f2df400cc90 comp rx=0 tx=0).stop 2024-09-15T22:03:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 -- 172.21.15.99:0/1299202753 shutdown_connections 2024-09-15T22:03:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2e081037c0 0x7f2e08074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2e08102e00 0x7f2e080745e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2df80a3240 0x7f2df80a5700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 --2- 172.21.15.99:0/1299202753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2e08101c00 0x7f2e08075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:03:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 -- 172.21.15.99:0/1299202753 >> 172.21.15.99:0/1299202753 conn(0x7f2e080fd390 msgr2=0x7f2e08106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:03:58.224 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 -- 172.21.15.99:0/1299202753 shutdown_connections 2024-09-15T22:03:58.225 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:03:58.222+0000 7f2e0f196640 1 -- 172.21.15.99:0/1299202753 wait complete. 2024-09-15T22:03:58.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:58 smithi099 ceph-mon[34892]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:58.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:58.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:58.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:58.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:03:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:58 smithi129 ceph-mon[35314]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:58.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:58.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:58.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:58.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:03:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:58 smithi005 ceph-mon[30944]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:03:58.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:03:58.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:03:58.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:03:58.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:58.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:03:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:03:59.036 DEBUG:teuthology.orchestra.run.smithi099:osd.7> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.7.service 2024-09-15T22:03:59.042 INFO:tasks.cephadm:Deploying osd.8 on smithi129 with /dev/vg_nvme/lv_2... 2024-09-15T22:03:59.042 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-15T22:03:59.297 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:04:00.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:00 smithi099 ceph-mon[34892]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:04:00.566 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:00 smithi129 ceph-mon[35314]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:04:00.589 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:04:00 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:04:00.477+0000 7f0d913ec640 -1 osd.7 0 log_to_monitors true 2024-09-15T22:04:00.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:00 smithi005 ceph-mon[30944]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:04:01.466 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:01 smithi099 ceph-mon[34892]: from='osd.7 [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:04:01.467 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:01 smithi099 ceph-mon[34892]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:04:01.524 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:01 smithi129 ceph-mon[35314]: from='osd.7 [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:04:01.524 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:01 smithi129 ceph-mon[35314]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:04:01.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:01 smithi005 ceph-mon[30944]: from='osd.7 [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:04:01.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:01 smithi005 ceph-mon[30944]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-15T22:04:02.003 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T22:04:02.028 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_2 2024-09-15T22:04:02.277 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:04:02.532 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:04:02.532 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:04:02.532 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: osdmap e44: 8 total, 7 up, 8 in 2024-09-15T22:04:02.532 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:02.532 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='osd.7 [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:04:02.532 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:04:02.532 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.533 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.533 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:02.533 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:02.533 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:02.533 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.533 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:02.533 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:02.533 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:02 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.589 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:04:02.273+0000 7f0d8d36f640 -1 osd.7 0 waiting for initial osdmap 2024-09-15T22:04:02.590 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:04:02.281+0000 7f0d89197640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:04:02.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:04:02.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:04:02.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: osdmap e44: 8 total, 7 up, 8 in 2024-09-15T22:04:02.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:02.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='osd.7 [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:04:02.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:04:02.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:02.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:02.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:02.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:02.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:02.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:02 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: osdmap e44: 8 total, 7 up, 8 in 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='osd.7 [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:02.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:02.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:02 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.752+0000 7fca22662640 1 -- 172.21.15.129:0/3330614972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca1c10d830 msgr2=0x7fca1c10dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.752+0000 7fca22662640 1 --2- 172.21.15.129:0/3330614972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca1c10d830 0x7fca1c10dcb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fca04009900 tx=0x7fca0402a410 comp rx=0 tx=0).stop 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.752+0000 7fca22662640 1 -- 172.21.15.129:0/3330614972 shutdown_connections 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.752+0000 7fca22662640 1 --2- 172.21.15.129:0/3330614972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca1c10e1f0 0x7fca1c114a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.752+0000 7fca22662640 1 --2- 172.21.15.129:0/3330614972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca1c10d830 0x7fca1c10dcb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.752+0000 7fca22662640 1 --2- 172.21.15.129:0/3330614972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c10c640 0x7fca1c10ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.752+0000 7fca22662640 1 -- 172.21.15.129:0/3330614972 >> 172.21.15.129:0/3330614972 conn(0x7fca1c06c860 msgr2=0x7fca1c06cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.752+0000 7fca22662640 1 -- 172.21.15.129:0/3330614972 shutdown_connections 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.752+0000 7fca22662640 1 -- 172.21.15.129:0/3330614972 wait complete. 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca22662640 1 Processor -- start 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca22662640 1 -- start start 2024-09-15T22:04:02.762 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca22662640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca1c10c640 0x7fca1c1a2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca22662640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca1c10d830 0x7fca1c1a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca22662640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c10e1f0 0x7fca1c1a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca22662640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c118290 con 0x7fca1c10e1f0 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca22662640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c118110 con 0x7fca1c10c640 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca22662640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c118410 con 0x7fca1c10d830 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca13fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca1c10d830 0x7fca1c1a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca13fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca1c10d830 0x7fca1c1a2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:55060/0 (socket says 172.21.15.129:55060) 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca13fff640 1 -- 172.21.15.129:0/3777092188 learned_addr learned my addr 172.21.15.129:0/3777092188 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca1bfff640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca1c10c640 0x7fca1c1a2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca20bff640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c10e1f0 0x7fca1c1a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca13fff640 1 -- 172.21.15.129:0/3777092188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca1c10c640 msgr2=0x7fca1c1a2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:02.763 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca13fff640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca1c10c640 0x7fca1c1a2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca13fff640 1 -- 172.21.15.129:0/3777092188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c10e1f0 msgr2=0x7fca1c1a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca13fff640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c10e1f0 0x7fca1c1a9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca13fff640 1 -- 172.21.15.129:0/3777092188 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c1aa480 con 0x7fca1c10d830 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.753+0000 7fca13fff640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca1c10d830 0x7fca1c1a2d90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fca04009230 tx=0x7fca04004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.754+0000 7fca19ffb640 1 -- 172.21.15.129:0/3777092188 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0403f030 con 0x7fca1c10d830 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.754+0000 7fca22662640 1 -- 172.21.15.129:0/3777092188 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1c1aa710 con 0x7fca1c10d830 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.754+0000 7fca19ffb640 1 -- 172.21.15.129:0/3777092188 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca04003ce0 con 0x7fca1c10d830 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.754+0000 7fca19ffb640 1 -- 172.21.15.129:0/3777092188 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04035070 con 0x7fca1c10d830 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.754+0000 7fca22662640 1 -- 172.21.15.129:0/3777092188 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1c1aabf0 con 0x7fca1c10d830 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.755+0000 7fca22662640 1 -- 172.21.15.129:0/3777092188 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca1c1139e0 con 0x7fca1c10d830 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.756+0000 7fca19ffb640 1 -- 172.21.15.129:0/3777092188 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fca04042030 con 0x7fca1c10d830 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.756+0000 7fca19ffb640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fca080a3270 0x7fca080a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:02.764 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.757+0000 7fca19ffb640 1 -- 172.21.15.129:0/3777092188 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5117+0+0 (secure 0 0 0) 0x7fca040ef330 con 0x7fca1c10d830 2024-09-15T22:04:02.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.757+0000 7fca1bfff640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fca080a3270 0x7fca080a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:02.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.757+0000 7fca1bfff640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fca080a3270 0x7fca080a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fca1c10d340 tx=0x7fca0c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:04:02.765 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.761+0000 7fca19ffb640 1 -- 172.21.15.129:0/3777092188 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca040bccb0 con 0x7fca1c10d830 2024-09-15T22:04:02.939 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:02.937+0000 7fca22662640 1 -- 172.21.15.129:0/3777092188 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fca1c062630 con 0x7fca080a3270 2024-09-15T22:04:03.581 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:03 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi099 2024-09-15T22:04:03.581 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:03 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi099 to 5696M 2024-09-15T22:04:03.581 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:03 smithi129 ceph-mon[35314]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:04:03.581 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:03 smithi129 ceph-mon[35314]: osdmap e45: 8 total, 7 up, 8 in 2024-09-15T22:04:03.581 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:03.581 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:03.581 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:03.581 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:03.581 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:03.582 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:03 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:03 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi099 2024-09-15T22:04:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:03 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi099 to 5696M 2024-09-15T22:04:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:03 smithi099 ceph-mon[34892]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:04:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:03 smithi099 ceph-mon[34892]: osdmap e45: 8 total, 7 up, 8 in 2024-09-15T22:04:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:03 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:03.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:03 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi099 2024-09-15T22:04:03.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:03 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi099 to 5696M 2024-09-15T22:04:03.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:03 smithi005 ceph-mon[30944]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:04:03.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:03 smithi005 ceph-mon[30944]: osdmap e45: 8 total, 7 up, 8 in 2024-09-15T22:04:03.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:03.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:03.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:03.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:03.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:03.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:03 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:04 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:04:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:04 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:04:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:04 smithi129 ceph-mon[35314]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:04:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:04 smithi129 ceph-mon[35314]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:04 smithi129 ceph-mon[35314]: osd.7 [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883] boot 2024-09-15T22:04:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:04 smithi129 ceph-mon[35314]: osdmap e46: 8 total, 8 up, 8 in 2024-09-15T22:04:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:04 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:04 smithi129 ceph-mon[35314]: osdmap e47: 8 total, 8 up, 8 in 2024-09-15T22:04:04.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:04 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:04:04.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:04 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:04:04.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:04 smithi099 ceph-mon[34892]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:04:04.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:04 smithi099 ceph-mon[34892]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:04.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:04 smithi099 ceph-mon[34892]: osd.7 [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883] boot 2024-09-15T22:04:04.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:04 smithi099 ceph-mon[34892]: osdmap e46: 8 total, 8 up, 8 in 2024-09-15T22:04:04.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:04 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:04.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:04 smithi099 ceph-mon[34892]: osdmap e47: 8 total, 8 up, 8 in 2024-09-15T22:04:04.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:04 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:04:04.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:04 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:04:04.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:04 smithi005 ceph-mon[30944]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 222 MiB used, 626 GiB / 626 GiB avail 2024-09-15T22:04:04.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:04 smithi005 ceph-mon[30944]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:04.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:04 smithi005 ceph-mon[30944]: osd.7 [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883] boot 2024-09-15T22:04:04.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:04 smithi005 ceph-mon[30944]: osdmap e46: 8 total, 8 up, 8 in 2024-09-15T22:04:04.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:04 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:04:04.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:04 smithi005 ceph-mon[30944]: osdmap e47: 8 total, 8 up, 8 in 2024-09-15T22:04:05.635 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:05 smithi129 ceph-mon[35314]: osdmap e48: 8 total, 8 up, 8 in 2024-09-15T22:04:05.822 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:05 smithi099 ceph-mon[34892]: osdmap e48: 8 total, 8 up, 8 in 2024-09-15T22:04:05.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:05 smithi005 ceph-mon[30944]: osdmap e48: 8 total, 8 up, 8 in 2024-09-15T22:04:06.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:06 smithi099 ceph-mon[34892]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:06.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:06 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/2358253635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5048f4-2e10-4e65-a7e9-130180439ad9"}]: dispatch 2024-09-15T22:04:06.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:06 smithi099 ceph-mon[34892]: osdmap e49: 8 total, 8 up, 8 in 2024-09-15T22:04:06.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:06 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5048f4-2e10-4e65-a7e9-130180439ad9"}]: dispatch 2024-09-15T22:04:06.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:06 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae5048f4-2e10-4e65-a7e9-130180439ad9"}]': finished 2024-09-15T22:04:06.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:06 smithi099 ceph-mon[34892]: osdmap e50: 9 total, 8 up, 9 in 2024-09-15T22:04:06.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:06 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:06.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:06 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/3185871692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:06 smithi129 ceph-mon[35314]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:06 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/2358253635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5048f4-2e10-4e65-a7e9-130180439ad9"}]: dispatch 2024-09-15T22:04:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:06 smithi129 ceph-mon[35314]: osdmap e49: 8 total, 8 up, 8 in 2024-09-15T22:04:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:06 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5048f4-2e10-4e65-a7e9-130180439ad9"}]: dispatch 2024-09-15T22:04:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:06 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae5048f4-2e10-4e65-a7e9-130180439ad9"}]': finished 2024-09-15T22:04:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:06 smithi129 ceph-mon[35314]: osdmap e50: 9 total, 8 up, 9 in 2024-09-15T22:04:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:06 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:06.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:06 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/3185871692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:06.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:06 smithi005 ceph-mon[30944]: pgmap v169: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:06.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:06 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/2358253635' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5048f4-2e10-4e65-a7e9-130180439ad9"}]: dispatch 2024-09-15T22:04:06.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:06 smithi005 ceph-mon[30944]: osdmap e49: 8 total, 8 up, 8 in 2024-09-15T22:04:06.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:06 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ae5048f4-2e10-4e65-a7e9-130180439ad9"}]: dispatch 2024-09-15T22:04:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:06 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae5048f4-2e10-4e65-a7e9-130180439ad9"}]': finished 2024-09-15T22:04:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:06 smithi005 ceph-mon[30944]: osdmap e50: 9 total, 8 up, 9 in 2024-09-15T22:04:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:06 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:06.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:06 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/3185871692' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:07.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:07 smithi099 ceph-mon[34892]: osdmap e51: 9 total, 8 up, 9 in 2024-09-15T22:04:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:07.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:07 smithi129 ceph-mon[35314]: osdmap e51: 9 total, 8 up, 9 in 2024-09-15T22:04:07.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:07.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:07 smithi005 ceph-mon[30944]: osdmap e51: 9 total, 8 up, 9 in 2024-09-15T22:04:07.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:08.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:08 smithi099 ceph-mon[34892]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:08.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:08 smithi129 ceph-mon[35314]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:08.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:08 smithi005 ceph-mon[30944]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:10.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:10 smithi129 ceph-mon[35314]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:10.658 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:10 smithi005 ceph-mon[30944]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:10.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:10 smithi099 ceph-mon[34892]: pgmap v174: 1 pgs: 1 unknown; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:12.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:12 smithi129 ceph-mon[35314]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:12.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:12 smithi099 ceph-mon[34892]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:12 smithi005 ceph-mon[30944]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:14 smithi129 ceph-mon[35314]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:14 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:04:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:14 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:14.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:14 smithi129 ceph-mon[35314]: Deploying daemon osd.8 on smithi129 2024-09-15T22:04:14.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:14 smithi099 ceph-mon[34892]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:14.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:14 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:04:14.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:14 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:14.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:14 smithi099 ceph-mon[34892]: Deploying daemon osd.8 on smithi129 2024-09-15T22:04:14.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:14 smithi005 ceph-mon[30944]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:14.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:14 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-15T22:04:14.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:14 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:14.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:14 smithi005 ceph-mon[30944]: Deploying daemon osd.8 on smithi129 2024-09-15T22:04:16.714 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:16 smithi129 ceph-mon[35314]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:16.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:16 smithi099 ceph-mon[34892]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:16.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:16 smithi005 ceph-mon[30944]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:18.509 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:18 smithi129 ceph-mon[35314]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:18.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:18 smithi099 ceph-mon[34892]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:18.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:18 smithi005 ceph-mon[30944]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:20.546 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:20 smithi129 ceph-mon[35314]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:20.546 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:20.546 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:20.546 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:20.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:20 smithi099 ceph-mon[34892]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:20.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:20.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:20.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:20.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:20 smithi005 ceph-mon[30944]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:20.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:20.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:20.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:22.810 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:22 smithi129 ceph-mon[35314]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:22.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:22 smithi099 ceph-mon[34892]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:22.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:22 smithi005 ceph-mon[30944]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:23.116 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.114+0000 7fca19ffb640 1 -- 172.21.15.129:0/3777092188 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fca1c062630 con 0x7fca080a3270 2024-09-15T22:04:23.116 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 8 on host 'smithi129' 2024-09-15T22:04:23.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 -- 172.21.15.129:0/3777092188 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fca080a3270 msgr2=0x7fca080a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:23.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fca080a3270 0x7fca080a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fca1c10d340 tx=0x7fca0c005f90 comp rx=0 tx=0).stop 2024-09-15T22:04:23.118 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 -- 172.21.15.129:0/3777092188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca1c10d830 msgr2=0x7fca1c1a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:23.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca1c10d830 0x7fca1c1a2d90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fca04009230 tx=0x7fca04004360 comp rx=0 tx=0).stop 2024-09-15T22:04:23.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 -- 172.21.15.129:0/3777092188 shutdown_connections 2024-09-15T22:04:23.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fca080a3270 0x7fca080a5730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:23.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca1c10e1f0 0x7fca1c1a9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:23.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca1c10d830 0x7fca1c1a2d90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:23.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 --2- 172.21.15.129:0/3777092188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca1c10c640 0x7fca1c1a2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:23.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 -- 172.21.15.129:0/3777092188 >> 172.21.15.129:0/3777092188 conn(0x7fca1c06c860 msgr2=0x7fca1c110bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:04:23.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 -- 172.21.15.129:0/3777092188 shutdown_connections 2024-09-15T22:04:23.119 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:04:23.117+0000 7fca22662640 1 -- 172.21.15.129:0/3777092188 wait complete. 2024-09-15T22:04:23.735 DEBUG:teuthology.orchestra.run.smithi129:osd.8> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.8.service 2024-09-15T22:04:23.738 INFO:tasks.cephadm:Deploying osd.9 on smithi005 with /dev/vg_nvme/lv_1... 2024-09-15T22:04:23.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:04:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:04:24.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:24 smithi005 ceph-mon[30944]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:24.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:24 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:24.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:24 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:24.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:24 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:24.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:24 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:24.401 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:24 smithi129 ceph-mon[35314]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:24.402 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:24 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:24.402 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:24 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:24.402 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:24 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:24.402 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:24 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:24 smithi099 ceph-mon[34892]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:24 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:24 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:24 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:24 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:25.846 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:04:25 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:04:25.621+0000 7fc5db091640 -1 osd.8 0 log_to_monitors true 2024-09-15T22:04:26.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:26 smithi005 ceph-mon[30944]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:26.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:26 smithi005 ceph-mon[30944]: from='osd.8 [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:04:26.397 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:26 smithi005 ceph-mon[30944]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:04:26.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:26 smithi099 ceph-mon[34892]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:26.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:26 smithi099 ceph-mon[34892]: from='osd.8 [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:04:26.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:26 smithi099 ceph-mon[34892]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:04:26.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:26 smithi129 ceph-mon[35314]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:26.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:26 smithi129 ceph-mon[35314]: from='osd.8 [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:04:26.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:26 smithi129 ceph-mon[35314]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-15T22:04:26.922 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:04:26.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_1 2024-09-15T22:04:27.223 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:04:27.345 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:04:27.124+0000 7fc5d7014640 -1 osd.8 0 waiting for initial osdmap 2024-09-15T22:04:27.345 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:04:27.133+0000 7fc5d2e3c640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:04:27.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:04:27.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='osd.8 [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:27.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:27.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:27.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:27 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:04:27.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T22:04:27.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:27.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='osd.8 [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:04:27.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:04:27.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:27.348 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:27 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-15T22:04:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: osdmap e52: 9 total, 8 up, 9 in 2024-09-15T22:04:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='osd.8 [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:04:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:04:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:27.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:27.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:27.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:27.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:27.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:27.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:27.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:27.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:27 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:27.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.859+0000 7f63b75b5640 1 -- 172.21.15.5:0/2674990928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63b01026a0 msgr2=0x7f63b0104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:27.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.859+0000 7f63b75b5640 1 --2- 172.21.15.5:0/2674990928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63b01026a0 0x7f63b0104a90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f63ac009900 tx=0x7f63ac02a410 comp rx=0 tx=0).stop 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.859+0000 7f63b75b5640 1 -- 172.21.15.5:0/2674990928 shutdown_connections 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.859+0000 7f63b75b5640 1 --2- 172.21.15.5:0/2674990928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63b0107900 0x7f63b0109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.859+0000 7f63b75b5640 1 --2- 172.21.15.5:0/2674990928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b0104fd0 0x7f63b01073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.859+0000 7f63b75b5640 1 --2- 172.21.15.5:0/2674990928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63b01026a0 0x7f63b0104a90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.859+0000 7f63b75b5640 1 -- 172.21.15.5:0/2674990928 >> 172.21.15.5:0/2674990928 conn(0x7f63b00fc2a0 msgr2=0x7f63b00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.859+0000 7f63b75b5640 1 -- 172.21.15.5:0/2674990928 shutdown_connections 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.859+0000 7f63b75b5640 1 -- 172.21.15.5:0/2674990928 wait complete. 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b75b5640 1 Processor -- start 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b75b5640 1 -- start start 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b75b5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b01026a0 0x7f63b0197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b75b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63b0104fd0 0x7f63b01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:27.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b75b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63b0107900 0x7f63b019f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b75b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010d920 con 0x7f63b0104fd0 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b75b5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010d7a0 con 0x7f63b01026a0 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b75b5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010daa0 con 0x7f63b0107900 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b4b50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63b0104fd0 0x7f63b01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b4b50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63b0104fd0 0x7f63b01984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56652/0 (socket says 172.21.15.5:56652) 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b4b50640 1 -- 172.21.15.5:0/3669131263 learned_addr learned my addr 172.21.15.5:0/3669131263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b5351640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b01026a0 0x7f63b0197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.860+0000 7f63b5b52640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63b0107900 0x7f63b019f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63b4b50640 1 -- 172.21.15.5:0/3669131263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63b0107900 msgr2=0x7f63b019f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63b4b50640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63b0107900 0x7f63b019f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63b4b50640 1 -- 172.21.15.5:0/3669131263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b01026a0 msgr2=0x7f63b0197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:27.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63b4b50640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b01026a0 0x7f63b0197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:27.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63b4b50640 1 -- 172.21.15.5:0/3669131263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b019fbe0 con 0x7f63b0104fd0 2024-09-15T22:04:27.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63b5b52640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63b0107900 0x7f63b019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:04:27.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63b4b50640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63b0104fd0 0x7f63b01984f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f63a000d770 tx=0x7f63a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:04:27.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63a67fc640 1 -- 172.21.15.5:0/3669131263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0017070 con 0x7f63b0104fd0 2024-09-15T22:04:27.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63a67fc640 1 -- 172.21.15.5:0/3669131263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a0009c30 con 0x7f63b0104fd0 2024-09-15T22:04:27.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63b75b5640 1 -- 172.21.15.5:0/3669131263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b019fed0 con 0x7f63b0104fd0 2024-09-15T22:04:27.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63a67fc640 1 -- 172.21.15.5:0/3669131263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a000f620 con 0x7f63b0104fd0 2024-09-15T22:04:27.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.861+0000 7f63b75b5640 1 -- 172.21.15.5:0/3669131263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a0400 con 0x7f63b0104fd0 2024-09-15T22:04:27.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.862+0000 7f63b75b5640 1 -- 172.21.15.5:0/3669131263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b00fdd90 con 0x7f63b0104fd0 2024-09-15T22:04:27.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.864+0000 7f63a67fc640 1 -- 172.21.15.5:0/3669131263 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f63a001a070 con 0x7f63b0104fd0 2024-09-15T22:04:27.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.865+0000 7f63a67fc640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f639c0a3270 0x7f639c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:27.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.865+0000 7f63b5351640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f639c0a3270 0x7f639c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:27.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.865+0000 7f63a67fc640 1 -- 172.21.15.5:0/3669131263 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5549+0+0 (secure 0 0 0) 0x7f63a00ce8f0 con 0x7f63b0104fd0 2024-09-15T22:04:27.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.865+0000 7f63b5351640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f639c0a3270 0x7f639c0a5730 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0314e0 tx=0x7f63ac000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:04:27.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:27.868+0000 7f63a67fc640 1 -- 172.21.15.5:0/3669131263 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63a009c0e0 con 0x7f63b0104fd0 2024-09-15T22:04:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:28.053+0000 7f63b75b5640 1 -- 172.21.15.5:0/3669131263 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f63b0062630 con 0x7f639c0a3270 2024-09-15T22:04:28.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:28 smithi005 ceph-mon[30944]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:28.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:28 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi129 2024-09-15T22:04:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:28 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi129 to 5696M 2024-09-15T22:04:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:28 smithi005 ceph-mon[30944]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:04:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:28 smithi005 ceph-mon[30944]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T22:04:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:28.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:28 smithi099 ceph-mon[34892]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:28.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:28 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi129 2024-09-15T22:04:28.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:28 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi129 to 5696M 2024-09-15T22:04:28.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:28 smithi099 ceph-mon[34892]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:04:28.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:28 smithi099 ceph-mon[34892]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T22:04:28.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:28.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:28.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:28.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:28.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:28.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:28 smithi129 ceph-mon[35314]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-09-15T22:04:28.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:28 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi129 2024-09-15T22:04:28.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:28 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi129 to 5696M 2024-09-15T22:04:28.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:28 smithi129 ceph-mon[35314]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:04:28.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:28 smithi129 ceph-mon[35314]: osdmap e53: 9 total, 8 up, 9 in 2024-09-15T22:04:28.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:28.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:28.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:28.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:28.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:29 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:04:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:29 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:04:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:29 smithi099 ceph-mon[34892]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:29 smithi099 ceph-mon[34892]: osd.8 [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431] boot 2024-09-15T22:04:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:29 smithi099 ceph-mon[34892]: osdmap e54: 9 total, 9 up, 9 in 2024-09-15T22:04:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:29 smithi099 ceph-mon[34892]: osdmap e55: 9 total, 9 up, 9 in 2024-09-15T22:04:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:29 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:04:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:29 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:04:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:29 smithi129 ceph-mon[35314]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:29 smithi129 ceph-mon[35314]: osd.8 [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431] boot 2024-09-15T22:04:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:29 smithi129 ceph-mon[35314]: osdmap e54: 9 total, 9 up, 9 in 2024-09-15T22:04:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:29 smithi129 ceph-mon[35314]: osdmap e55: 9 total, 9 up, 9 in 2024-09-15T22:04:29.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:29 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:04:29.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:29 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:04:29.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:29 smithi005 ceph-mon[30944]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:29.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:29 smithi005 ceph-mon[30944]: osd.8 [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431] boot 2024-09-15T22:04:29.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:29 smithi005 ceph-mon[30944]: osdmap e54: 9 total, 9 up, 9 in 2024-09-15T22:04:29.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:04:29.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:29 smithi005 ceph-mon[30944]: osdmap e55: 9 total, 9 up, 9 in 2024-09-15T22:04:30.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:30 smithi005 ceph-mon[30944]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:30.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:30 smithi005 ceph-mon[30944]: osdmap e56: 9 total, 9 up, 9 in 2024-09-15T22:04:30.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:30 smithi099 ceph-mon[34892]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:30.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:30 smithi099 ceph-mon[34892]: osdmap e56: 9 total, 9 up, 9 in 2024-09-15T22:04:30.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:30 smithi129 ceph-mon[35314]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:30.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:30 smithi129 ceph-mon[35314]: osdmap e56: 9 total, 9 up, 9 in 2024-09-15T22:04:31.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:31 smithi005 ceph-mon[30944]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T22:04:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:31 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3614724513' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93835974-5099-475b-86ce-057cf6614dff"}]: dispatch 2024-09-15T22:04:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:31 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3614724513' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93835974-5099-475b-86ce-057cf6614dff"}]': finished 2024-09-15T22:04:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:31 smithi005 ceph-mon[30944]: osdmap e58: 10 total, 9 up, 10 in 2024-09-15T22:04:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:31 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/494004455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:31 smithi099 ceph-mon[34892]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T22:04:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:31 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3614724513' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93835974-5099-475b-86ce-057cf6614dff"}]: dispatch 2024-09-15T22:04:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:31 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3614724513' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93835974-5099-475b-86ce-057cf6614dff"}]': finished 2024-09-15T22:04:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:31 smithi099 ceph-mon[34892]: osdmap e58: 10 total, 9 up, 10 in 2024-09-15T22:04:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:31 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:31.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:31 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/494004455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:31.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:31 smithi129 ceph-mon[35314]: osdmap e57: 9 total, 9 up, 9 in 2024-09-15T22:04:31.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:31 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3614724513' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93835974-5099-475b-86ce-057cf6614dff"}]: dispatch 2024-09-15T22:04:31.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:31 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3614724513' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93835974-5099-475b-86ce-057cf6614dff"}]': finished 2024-09-15T22:04:31.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:31 smithi129 ceph-mon[35314]: osdmap e58: 10 total, 9 up, 10 in 2024-09-15T22:04:31.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:31.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:31 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/494004455' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:32.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:32 smithi099 ceph-mon[34892]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:32.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:32 smithi129 ceph-mon[35314]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:32.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:32 smithi005 ceph-mon[30944]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:34.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:34 smithi099 ceph-mon[34892]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:34.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:34 smithi129 ceph-mon[35314]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:34.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:34 smithi005 ceph-mon[30944]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:36.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:36 smithi005 ceph-mon[30944]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:04:36.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:36 smithi099 ceph-mon[34892]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:04:36.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:36 smithi129 ceph-mon[35314]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:04:38.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:38 smithi005 ceph-mon[30944]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:04:38.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:38 smithi099 ceph-mon[34892]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:04:38.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:38 smithi129 ceph-mon[35314]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-15T22:04:39.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:39 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:04:39.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:39 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:39.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:39 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:04:39.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:39 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:39.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:39 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-15T22:04:39.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:39 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:40.831 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:40 smithi005 ceph-mon[30944]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-15T22:04:40.832 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:40 smithi005 ceph-mon[30944]: Deploying daemon osd.9 on smithi005 2024-09-15T22:04:40.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:40 smithi099 ceph-mon[34892]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-15T22:04:40.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:40 smithi099 ceph-mon[34892]: Deploying daemon osd.9 on smithi005 2024-09-15T22:04:40.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:40 smithi129 ceph-mon[35314]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-15T22:04:40.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:40 smithi129 ceph-mon[35314]: Deploying daemon osd.9 on smithi005 2024-09-15T22:04:42.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:42 smithi005 ceph-mon[30944]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:04:42.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:42 smithi099 ceph-mon[34892]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:04:42.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:42 smithi129 ceph-mon[35314]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:04:44.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:44 smithi005 ceph-mon[30944]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:04:44.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:44 smithi099 ceph-mon[34892]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:04:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:44 smithi129 ceph-mon[35314]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:04:45.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:45 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:45.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:45 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:45.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:45 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:45.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:45 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:45.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:45 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:45.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:45 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:45 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:45 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:45 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:46.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:46 smithi005 ceph-mon[30944]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:04:46.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:46 smithi099 ceph-mon[34892]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:04:46.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:46 smithi129 ceph-mon[35314]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-15T22:04:48.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:48 smithi099 ceph-mon[34892]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:48 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:48 smithi129 ceph-mon[35314]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:48 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:48.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:48 smithi005 ceph-mon[30944]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:48.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:48.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:48.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:48.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:48.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:48 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:48.927 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 9 on host 'smithi005' 2024-09-15T22:04:48.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.924+0000 7f63a67fc640 1 -- 172.21.15.5:0/3669131263 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f63b0062630 con 0x7f639c0a3270 2024-09-15T22:04:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 -- 172.21.15.5:0/3669131263 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f639c0a3270 msgr2=0x7f639c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f639c0a3270 0x7f639c0a5730 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0314e0 tx=0x7f63ac000bc0 comp rx=0 tx=0).stop 2024-09-15T22:04:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 -- 172.21.15.5:0/3669131263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63b0104fd0 msgr2=0x7f63b01984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:48.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63b0104fd0 0x7f63b01984f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f63a000d770 tx=0x7f63a000dc30 comp rx=0 tx=0).stop 2024-09-15T22:04:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 -- 172.21.15.5:0/3669131263 shutdown_connections 2024-09-15T22:04:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63b0107900 0x7f63b019f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f639c0a3270 0x7f639c0a5730 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63b0104fd0 0x7f63b01984f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 --2- 172.21.15.5:0/3669131263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63b01026a0 0x7f63b0197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 -- 172.21.15.5:0/3669131263 >> 172.21.15.5:0/3669131263 conn(0x7f63b00fc2a0 msgr2=0x7f63b0103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:04:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 -- 172.21.15.5:0/3669131263 shutdown_connections 2024-09-15T22:04:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:04:48.927+0000 7f63b75b5640 1 -- 172.21.15.5:0/3669131263 wait complete. 2024-09-15T22:04:49.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:49 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:49.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:49 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:49.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:49 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:49.692 DEBUG:teuthology.orchestra.run.smithi005:osd.9> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.9.service 2024-09-15T22:04:49.695 INFO:tasks.cephadm:Deploying osd.10 on smithi099 with /dev/vg_nvme/lv_1... 2024-09-15T22:04:49.695 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:04:49.723 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:49 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:49.723 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:49 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:49.723 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:49 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:49.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:49 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:04:49.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:49 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:49.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:49 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:49.948 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:04:50.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:50 smithi099 ceph-mon[34892]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:50.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:50 smithi005 ceph-mon[30944]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:50 smithi129 ceph-mon[35314]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:50.856 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:04:50 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:04:50.785+0000 7f2c58794640 -1 osd.9 0 log_to_monitors true 2024-09-15T22:04:51.465 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:51 smithi005 ceph-mon[30944]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:04:51.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:51 smithi099 ceph-mon[34892]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:04:51.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:51 smithi129 ceph-mon[35314]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-15T22:04:52.382 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-15T22:04:52.408 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi099:vg_nvme/lv_1 2024-09-15T22:04:52.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:52 smithi005 ceph-mon[30944]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:52.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:52 smithi005 ceph-mon[30944]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:04:52.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:52 smithi005 ceph-mon[30944]: osdmap e59: 10 total, 9 up, 10 in 2024-09-15T22:04:52.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:52 smithi005 ceph-mon[30944]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:04:52.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:52.687 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:52 smithi099 ceph-mon[34892]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:52.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:52 smithi099 ceph-mon[34892]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:04:52.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:52 smithi099 ceph-mon[34892]: osdmap e59: 10 total, 9 up, 10 in 2024-09-15T22:04:52.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:52 smithi099 ceph-mon[34892]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:04:52.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:52.698 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:04:52.745 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:52 smithi129 ceph-mon[35314]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:52.745 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:52 smithi129 ceph-mon[35314]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-15T22:04:52.745 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:52 smithi129 ceph-mon[35314]: osdmap e59: 10 total, 9 up, 10 in 2024-09-15T22:04:52.746 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:52 smithi129 ceph-mon[35314]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-15T22:04:52.746 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:53.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.151+0000 7fb630835640 1 -- 172.21.15.99:0/3571395045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62c102690 msgr2=0x7fb62c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:53.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.151+0000 7fb630835640 1 --2- 172.21.15.99:0/3571395045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62c102690 0x7fb62c108f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009ca0 tx=0x7fb61c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:04:53.175 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.151+0000 7fb630835640 1 -- 172.21.15.99:0/3571395045 shutdown_connections 2024-09-15T22:04:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.151+0000 7fb630835640 1 --2- 172.21.15.99:0/3571395045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62c102690 0x7fb62c108f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.151+0000 7fb630835640 1 --2- 172.21.15.99:0/3571395045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb62c101cd0 0x7fb62c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.151+0000 7fb630835640 1 --2- 172.21.15.99:0/3571395045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb62c100ad0 0x7fb62c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.151+0000 7fb630835640 1 -- 172.21.15.99:0/3571395045 >> 172.21.15.99:0/3571395045 conn(0x7fb62c0fc280 msgr2=0x7fb62c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:04:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.151+0000 7fb630835640 1 -- 172.21.15.99:0/3571395045 shutdown_connections 2024-09-15T22:04:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.151+0000 7fb630835640 1 -- 172.21.15.99:0/3571395045 wait complete. 2024-09-15T22:04:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb630835640 1 Processor -- start 2024-09-15T22:04:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb630835640 1 -- start start 2024-09-15T22:04:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb630835640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62c100ad0 0x7fb62c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb630835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb62c101cd0 0x7fb62c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:53.176 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb630835640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb62c102690 0x7fb62c19f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb630835640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c10c750 con 0x7fb62c101cd0 2024-09-15T22:04:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb630835640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c10c5d0 con 0x7fb62c100ad0 2024-09-15T22:04:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb630835640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb62c10c8d0 con 0x7fb62c102690 2024-09-15T22:04:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb62a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62c100ad0 0x7fb62c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb62a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62c100ad0 0x7fb62c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.99:55952/0 (socket says 172.21.15.99:55952) 2024-09-15T22:04:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb62a59c640 1 -- 172.21.15.99:0/2901919680 learned_addr learned my addr 172.21.15.99:0/2901919680 (peer_addr_for_me v2:172.21.15.99:0/0) 2024-09-15T22:04:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb629d9b640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb62c101cd0 0x7fb62c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb62ad9d640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb62c102690 0x7fb62c19f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb62a59c640 1 -- 172.21.15.99:0/2901919680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb62c102690 msgr2=0x7fb62c19f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:53.177 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb62a59c640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb62c102690 0x7fb62c19f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb62a59c640 1 -- 172.21.15.99:0/2901919680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb62c101cd0 msgr2=0x7fb62c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:04:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb62a59c640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb62c101cd0 0x7fb62c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:04:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.152+0000 7fb62a59c640 1 -- 172.21.15.99:0/2901919680 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62c19fad0 con 0x7fb62c100ad0 2024-09-15T22:04:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.153+0000 7fb62a59c640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62c100ad0 0x7fb62c197ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb62400e9c0 tx=0x7fb62400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:04:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.153+0000 7fb61b7fe640 1 -- 172.21.15.99:0/2901919680 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62400cd80 con 0x7fb62c100ad0 2024-09-15T22:04:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.153+0000 7fb630835640 1 -- 172.21.15.99:0/2901919680 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62c19fdc0 con 0x7fb62c100ad0 2024-09-15T22:04:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.153+0000 7fb61b7fe640 1 -- 172.21.15.99:0/2901919680 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6240035f0 con 0x7fb62c100ad0 2024-09-15T22:04:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.153+0000 7fb61b7fe640 1 -- 172.21.15.99:0/2901919680 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624010690 con 0x7fb62c100ad0 2024-09-15T22:04:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.153+0000 7fb630835640 1 -- 172.21.15.99:0/2901919680 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb62c1a0220 con 0x7fb62c100ad0 2024-09-15T22:04:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.154+0000 7fb630835640 1 -- 172.21.15.99:0/2901919680 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb62c1a04b0 con 0x7fb62c100ad0 2024-09-15T22:04:53.178 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.155+0000 7fb61b7fe640 1 -- 172.21.15.99:0/2901919680 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fb6240037a0 con 0x7fb62c100ad0 2024-09-15T22:04:53.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.156+0000 7fb61b7fe640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fb6140a3270 0x7fb6140a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:04:53.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.156+0000 7fb61b7fe640 1 -- 172.21.15.99:0/2901919680 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5981+0+0 (secure 0 0 0) 0x7fb6240cf3f0 con 0x7fb62c100ad0 2024-09-15T22:04:53.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.156+0000 7fb629d9b640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fb6140a3270 0x7fb6140a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:04:53.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.157+0000 7fb629d9b640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fb6140a3270 0x7fb6140a5730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb62c1993c0 tx=0x7fb620005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:04:53.179 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.159+0000 7fb61b7fe640 1 -- 172.21.15.99:0/2901919680 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb62409cac0 con 0x7fb62c100ad0 2024-09-15T22:04:53.342 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:04:53.341+0000 7fb630835640 1 -- 172.21.15.99:0/2901919680 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb62c06a680 con 0x7fb6140a3270 2024-09-15T22:04:53.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:04:52 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:04:52.984+0000 7f2c54717640 -1 osd.9 0 waiting for initial osdmap 2024-09-15T22:04:53.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:04:52 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:04:52.995+0000 7f2c4c537640 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:04:53.687 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:04:53.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: osdmap e60: 10 total, 9 up, 10 in 2024-09-15T22:04:53.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:53.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:53.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.688 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.689 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.689 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:53.689 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:53.689 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.689 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' 2024-09-15T22:04:53.689 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:53.689 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:53.689 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:53 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:53.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:04:53.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: osdmap e60: 10 total, 9 up, 10 in 2024-09-15T22:04:53.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:53.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:53.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:53.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:53.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:53 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:53.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-15T22:04:53.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: osdmap e60: 10 total, 9 up, 10 in 2024-09-15T22:04:53.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:53.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:53.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:04:53.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:53.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:04:53.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:04:53.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355]' entity='osd.9' 2024-09-15T22:04:53.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:04:53.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:04:53.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:53 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:04:54.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:54 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:04:54.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:54 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:04:54.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:54 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi005 2024-09-15T22:04:54.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:54 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-15T22:04:54.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:54 smithi099 ceph-mon[34892]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:54.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:54 smithi099 ceph-mon[34892]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:54.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:54 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:54.796 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:54 smithi099 ceph-mon[34892]: osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355] boot 2024-09-15T22:04:54.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:54 smithi099 ceph-mon[34892]: osdmap e61: 10 total, 10 up, 10 in 2024-09-15T22:04:54.797 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:54 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:54.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:54 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:04:54.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:54 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:04:54.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:54 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi005 2024-09-15T22:04:54.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:54 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-15T22:04:54.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:54 smithi129 ceph-mon[35314]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:54.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:54 smithi129 ceph-mon[35314]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:54.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:54 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:54.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:54 smithi129 ceph-mon[35314]: osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355] boot 2024-09-15T22:04:54.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:54 smithi129 ceph-mon[35314]: osdmap e61: 10 total, 10 up, 10 in 2024-09-15T22:04:54.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:54 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:54.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:54 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:04:54.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:54 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:04:54.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:54 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi005 2024-09-15T22:04:54.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:54 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-15T22:04:54.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:54 smithi005 ceph-mon[30944]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-09-15T22:04:54.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:54 smithi005 ceph-mon[30944]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi099:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:04:54.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:54 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:54.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:54 smithi005 ceph-mon[30944]: osd.9 [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355] boot 2024-09-15T22:04:54.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:54 smithi005 ceph-mon[30944]: osdmap e61: 10 total, 10 up, 10 in 2024-09-15T22:04:54.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:54 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:04:56.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:56 smithi099 ceph-mon[34892]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:04:56.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:56 smithi099 ceph-mon[34892]: osdmap e62: 10 total, 10 up, 10 in 2024-09-15T22:04:56.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.99:0/480126184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d019941f-3a11-449b-8e5d-dac790b1530f"}]: dispatch 2024-09-15T22:04:56.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:56 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d019941f-3a11-449b-8e5d-dac790b1530f"}]: dispatch 2024-09-15T22:04:56.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:56 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d019941f-3a11-449b-8e5d-dac790b1530f"}]': finished 2024-09-15T22:04:56.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:56 smithi099 ceph-mon[34892]: osdmap e63: 11 total, 10 up, 11 in 2024-09-15T22:04:56.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:56 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:04:56.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.99:0/791976443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:56 smithi129 ceph-mon[35314]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:04:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:56 smithi129 ceph-mon[35314]: osdmap e62: 10 total, 10 up, 10 in 2024-09-15T22:04:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.99:0/480126184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d019941f-3a11-449b-8e5d-dac790b1530f"}]: dispatch 2024-09-15T22:04:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:56 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d019941f-3a11-449b-8e5d-dac790b1530f"}]: dispatch 2024-09-15T22:04:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:56 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d019941f-3a11-449b-8e5d-dac790b1530f"}]': finished 2024-09-15T22:04:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:56 smithi129 ceph-mon[35314]: osdmap e63: 11 total, 10 up, 11 in 2024-09-15T22:04:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:56 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:04:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.99:0/791976443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:56.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:56 smithi005 ceph-mon[30944]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:04:56.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:56 smithi005 ceph-mon[30944]: osdmap e62: 10 total, 10 up, 10 in 2024-09-15T22:04:56.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.99:0/480126184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d019941f-3a11-449b-8e5d-dac790b1530f"}]: dispatch 2024-09-15T22:04:56.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:56 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d019941f-3a11-449b-8e5d-dac790b1530f"}]: dispatch 2024-09-15T22:04:56.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:56 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d019941f-3a11-449b-8e5d-dac790b1530f"}]': finished 2024-09-15T22:04:56.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:56 smithi005 ceph-mon[30944]: osdmap e63: 11 total, 10 up, 11 in 2024-09-15T22:04:56.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:56 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:04:56.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.99:0/791976443' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:04:58.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:04:58 smithi099 ceph-mon[34892]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:04:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:04:58 smithi129 ceph-mon[35314]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:04:58.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:04:58 smithi005 ceph-mon[30944]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:00.805 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:00 smithi099 ceph-mon[34892]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:00.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:00 smithi129 ceph-mon[35314]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:00.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:00 smithi005 ceph-mon[30944]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 312 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:02.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:02 smithi099 ceph-mon[34892]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:02 smithi129 ceph-mon[35314]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:02.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:02 smithi005 ceph-mon[30944]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:04.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:04 smithi099 ceph-mon[34892]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:04.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:04 smithi129 ceph-mon[35314]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:04.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:04 smithi005 ceph-mon[30944]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:05.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:05 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:05:05.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:05 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:05.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:05 smithi099 ceph-mon[34892]: Deploying daemon osd.10 on smithi099 2024-09-15T22:05:05.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:05 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:05:05.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:05 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:05.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:05 smithi129 ceph-mon[35314]: Deploying daemon osd.10 on smithi099 2024-09-15T22:05:05.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-15T22:05:05.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:05.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:05 smithi005 ceph-mon[30944]: Deploying daemon osd.10 on smithi099 2024-09-15T22:05:06.573 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:06 smithi099 ceph-mon[34892]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:06.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:06 smithi129 ceph-mon[35314]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:06.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:06 smithi005 ceph-mon[30944]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:08.822 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:08 smithi099 ceph-mon[34892]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:08.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:08 smithi129 ceph-mon[35314]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:08.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:08 smithi005 ceph-mon[30944]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:10.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:10 smithi099 ceph-mon[34892]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:10 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:10 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:10.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:10 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:10.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:10 smithi129 ceph-mon[35314]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:10.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:10 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:10.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:10 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:10.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:10 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:10.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:10 smithi005 ceph-mon[30944]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:10.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:10 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:10.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:10 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:10.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:10 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:12.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:12 smithi099 ceph-mon[34892]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:12.844 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:12 smithi129 ceph-mon[35314]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:12.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:12 smithi005 ceph-mon[30944]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:13.929 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.927+0000 7fb61b7fe640 1 -- 172.21.15.99:0/2901919680 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb62c06a680 con 0x7fb6140a3270 2024-09-15T22:05:13.930 INFO:teuthology.orchestra.run.smithi099.stdout:Created osd(s) 10 on host 'smithi099' 2024-09-15T22:05:13.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.929+0000 7fb630835640 1 -- 172.21.15.99:0/2901919680 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fb6140a3270 msgr2=0x7fb6140a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:13.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.929+0000 7fb630835640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fb6140a3270 0x7fb6140a5730 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb62c1993c0 tx=0x7fb620005f90 comp rx=0 tx=0).stop 2024-09-15T22:05:13.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.929+0000 7fb630835640 1 -- 172.21.15.99:0/2901919680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62c100ad0 msgr2=0x7fb62c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:13.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.929+0000 7fb630835640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62c100ad0 0x7fb62c197ef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb62400e9c0 tx=0x7fb62400ee80 comp rx=0 tx=0).stop 2024-09-15T22:05:13.931 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.930+0000 7fb630835640 1 -- 172.21.15.99:0/2901919680 shutdown_connections 2024-09-15T22:05:13.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.930+0000 7fb630835640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb62c102690 0x7fb62c19f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:13.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.930+0000 7fb630835640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fb6140a3270 0x7fb6140a5730 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:13.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.930+0000 7fb630835640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb62c101cd0 0x7fb62c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:13.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.930+0000 7fb630835640 1 --2- 172.21.15.99:0/2901919680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb62c100ad0 0x7fb62c197ef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:13.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.930+0000 7fb630835640 1 -- 172.21.15.99:0/2901919680 >> 172.21.15.99:0/2901919680 conn(0x7fb62c0fc280 msgr2=0x7fb62c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:13.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.930+0000 7fb630835640 1 -- 172.21.15.99:0/2901919680 shutdown_connections 2024-09-15T22:05:13.932 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-15T22:05:13.930+0000 7fb630835640 1 -- 172.21.15.99:0/2901919680 wait complete. 2024-09-15T22:05:14.584 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:14 smithi099 ceph-mon[34892]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:14.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:14 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:14 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:14 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:14.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:14 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:14.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:14 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:14 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:14.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:14 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.585 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:14 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:14 smithi129 ceph-mon[35314]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:14 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:14 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:14 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:14 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:14 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:14 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:14 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:14 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:14 smithi005 ceph-mon[30944]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:14.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:14 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:14 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:14 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:14.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:14 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:14.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:14 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:14 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:14.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:14 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:14 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:14.610 DEBUG:teuthology.orchestra.run.smithi099:osd.10> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.10.service 2024-09-15T22:05:14.613 INFO:tasks.cephadm:Deploying osd.11 on smithi129 with /dev/vg_nvme/lv_1... 2024-09-15T22:05:14.613 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-15T22:05:14.866 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:05:16.294 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:16 smithi129 ceph-mon[35314]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:16.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:16 smithi099 ceph-mon[34892]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:16.591 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:05:16 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:05:16.228+0000 7f49f2d58640 -1 osd.10 0 log_to_monitors true 2024-09-15T22:05:16.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:16 smithi005 ceph-mon[30944]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:17.413 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:17 smithi099 ceph-mon[34892]: from='osd.10 [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:05:17.413 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:17 smithi099 ceph-mon[34892]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:05:17.563 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:17 smithi129 ceph-mon[35314]: from='osd.10 [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:05:17.564 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:17 smithi129 ceph-mon[35314]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:05:17.575 INFO:teuthology.orchestra.run.smithi129.stdout: 2024-09-15T22:05:17.604 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi129:vg_nvme/lv_1 2024-09-15T22:05:17.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:17 smithi005 ceph-mon[30944]: from='osd.10 [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:05:17.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:17 smithi005 ceph-mon[30944]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-15T22:05:17.850 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:05:18.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.303+0000 7fcece455640 1 -- 172.21.15.129:0/3349858593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcec8101cd0 msgr2=0x7fcec8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:18.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.303+0000 7fcece455640 1 --2- 172.21.15.129:0/3349858593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcec8101cd0 0x7fcec8102150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009900 tx=0x7fcebc02a410 comp rx=0 tx=0).stop 2024-09-15T22:05:18.317 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.303+0000 7fcece455640 1 -- 172.21.15.129:0/3349858593 shutdown_connections 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.303+0000 7fcece455640 1 --2- 172.21.15.129:0/3349858593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec8102690 0x7fcec8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.303+0000 7fcece455640 1 --2- 172.21.15.129:0/3349858593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcec8101cd0 0x7fcec8102150 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.303+0000 7fcece455640 1 --2- 172.21.15.129:0/3349858593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec8100ad0 0x7fcec8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.303+0000 7fcece455640 1 -- 172.21.15.129:0/3349858593 >> 172.21.15.129:0/3349858593 conn(0x7fcec80fc280 msgr2=0x7fcec80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.303+0000 7fcece455640 1 -- 172.21.15.129:0/3349858593 shutdown_connections 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.303+0000 7fcece455640 1 -- 172.21.15.129:0/3349858593 wait complete. 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.303+0000 7fcece455640 1 Processor -- start 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcece455640 1 -- start start 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcece455640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec8100ad0 0x7fcec8196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcece455640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec8101cd0 0x7fcec8197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcece455640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcec8102690 0x7fcec819e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:18.318 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcece455640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcec810c750 con 0x7fcec8101cd0 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcece455640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcec810c5d0 con 0x7fcec8100ad0 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcece455640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcec810c8d0 con 0x7fcec8102690 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcecc9f2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcec8102690 0x7fcec819e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcecc9f2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcec8102690 0x7fcec819e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:59716/0 (socket says 172.21.15.129:59716) 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcecc9f2640 1 -- 172.21.15.129:0/2490881616 learned_addr learned my addr 172.21.15.129:0/2490881616 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcec7fff640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec8100ad0 0x7fcec8196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcec77fe640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec8101cd0 0x7fcec8197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcecc9f2640 1 -- 172.21.15.129:0/2490881616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec8100ad0 msgr2=0x7fcec8196e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcecc9f2640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec8100ad0 0x7fcec8196e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcecc9f2640 1 -- 172.21.15.129:0/2490881616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec8101cd0 msgr2=0x7fcec8197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcecc9f2640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec8101cd0 0x7fcec8197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcecc9f2640 1 -- 172.21.15.129:0/2490881616 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec819ea40 con 0x7fcec8102690 2024-09-15T22:05:18.319 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.304+0000 7fcec77fe640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec8101cd0 0x7fcec8197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.305+0000 7fcecc9f2640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcec8102690 0x7fcec819e3d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcec000cad0 tx=0x7fcec000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.305+0000 7fcec57fa640 1 -- 172.21.15.129:0/2490881616 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec000ae20 con 0x7fcec8102690 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.305+0000 7fcec57fa640 1 -- 172.21.15.129:0/2490881616 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcec0005020 con 0x7fcec8102690 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.305+0000 7fcec57fa640 1 -- 172.21.15.129:0/2490881616 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec000f6f0 con 0x7fcec8102690 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.305+0000 7fcece455640 1 -- 172.21.15.129:0/2490881616 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec819ed30 con 0x7fcec8102690 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.305+0000 7fcece455640 1 -- 172.21.15.129:0/2490881616 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec819f1e0 con 0x7fcec8102690 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.306+0000 7fcece455640 1 -- 172.21.15.129:0/2490881616 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcec8107f40 con 0x7fcec8102690 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.307+0000 7fcec57fa640 1 -- 172.21.15.129:0/2490881616 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fcec00051d0 con 0x7fcec8102690 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.308+0000 7fcec57fa640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fceb00a32f0 0x7fceb00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.308+0000 7fcec57fa640 1 -- 172.21.15.129:0/2490881616 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6414+0+0 (secure 0 0 0) 0x7fcec00cf530 con 0x7fcec8102690 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.308+0000 7fcec7fff640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fceb00a32f0 0x7fceb00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.309+0000 7fcec7fff640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fceb00a32f0 0x7fceb00a57b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcec8101b30 tx=0x7fceb8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:18.320 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.311+0000 7fcec57fa640 1 -- 172.21.15.129:0/2490881616 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcec0014030 con 0x7fcec8102690 2024-09-15T22:05:18.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:18.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='osd.10 [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: osdmap e64: 11 total, 10 up, 11 in 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi099 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi099 to 4272M 2024-09-15T22:05:18.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:18.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:18.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:18.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:18.496 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:18.495+0000 7fcece455640 1 -- 172.21.15.129:0/2490881616 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcec8062630 con 0x7fceb00a32f0 2024-09-15T22:05:18.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:05:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='osd.10 [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:05:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: osdmap e64: 11 total, 10 up, 11 in 2024-09-15T22:05:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:05:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi099 2024-09-15T22:05:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi099 to 4272M 2024-09-15T22:05:18.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:18.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:18.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:18.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:18.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:18.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='osd.10 [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: osdmap e64: 11 total, 10 up, 11 in 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]: dispatch 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi099 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi099 to 4272M 2024-09-15T22:05:18.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:18.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:18.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:18.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:19.248 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:05:18.903+0000 7f49eecdb640 -1 osd.10 0 waiting for initial osdmap 2024-09-15T22:05:19.248 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:05:18 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:05:18.913+0000 7f49e6afb640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: osdmap e65: 11 total, 10 up, 11 in 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: osdmap e66: 11 total, 10 up, 11 in 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:05:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:19.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:19 smithi099 ceph-mon[34892]: from='osd.10 ' entity='osd.10' 2024-09-15T22:05:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:05:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:05:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:05:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: osdmap e65: 11 total, 10 up, 11 in 2024-09-15T22:05:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: osdmap e66: 11 total, 10 up, 11 in 2024-09-15T22:05:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:05:19.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:05:19.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:19.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:19 smithi129 ceph-mon[35314]: from='osd.10 ' entity='osd.10' 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi099", "root=default"]}]': finished 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: osdmap e65: 11 total, 10 up, 11 in 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: osdmap e66: 11 total, 10 up, 11 in 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi129:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:19 smithi005 ceph-mon[30944]: from='osd.10 ' entity='osd.10' 2024-09-15T22:05:20.571 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:20 smithi129 ceph-mon[35314]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:20.571 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:20.571 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:20 smithi129 ceph-mon[35314]: osd.10 [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498] boot 2024-09-15T22:05:20.571 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:20 smithi129 ceph-mon[35314]: osdmap e67: 11 total, 11 up, 11 in 2024-09-15T22:05:20.571 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:20 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:20.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:20 smithi099 ceph-mon[34892]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:20.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:20.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:20 smithi099 ceph-mon[34892]: osd.10 [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498] boot 2024-09-15T22:05:20.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:20 smithi099 ceph-mon[34892]: osdmap e67: 11 total, 11 up, 11 in 2024-09-15T22:05:20.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:20 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:20.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:20 smithi005 ceph-mon[30944]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 313 MiB used, 894 GiB / 894 GiB avail 2024-09-15T22:05:20.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:20.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:20 smithi005 ceph-mon[30944]: osd.10 [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498] boot 2024-09-15T22:05:20.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:20 smithi005 ceph-mon[30944]: osdmap e67: 11 total, 11 up, 11 in 2024-09-15T22:05:20.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:20 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:05:21.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:21 smithi129 ceph-mon[35314]: osdmap e68: 11 total, 11 up, 11 in 2024-09-15T22:05:21.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:21 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/1388702800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cff3d48-81e6-4748-96bb-c20ba054f287"}]: dispatch 2024-09-15T22:05:21.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:21 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cff3d48-81e6-4748-96bb-c20ba054f287"}]: dispatch 2024-09-15T22:05:21.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:21 smithi129 ceph-mon[35314]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cff3d48-81e6-4748-96bb-c20ba054f287"}]': finished 2024-09-15T22:05:21.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:21 smithi129 ceph-mon[35314]: osdmap e69: 12 total, 11 up, 12 in 2024-09-15T22:05:21.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:21 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:21.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:21 smithi099 ceph-mon[34892]: osdmap e68: 11 total, 11 up, 11 in 2024-09-15T22:05:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:21 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/1388702800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cff3d48-81e6-4748-96bb-c20ba054f287"}]: dispatch 2024-09-15T22:05:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:21 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cff3d48-81e6-4748-96bb-c20ba054f287"}]: dispatch 2024-09-15T22:05:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:21 smithi099 ceph-mon[34892]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cff3d48-81e6-4748-96bb-c20ba054f287"}]': finished 2024-09-15T22:05:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:21 smithi099 ceph-mon[34892]: osdmap e69: 12 total, 11 up, 12 in 2024-09-15T22:05:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:21 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:21.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:21 smithi005 ceph-mon[30944]: osdmap e68: 11 total, 11 up, 11 in 2024-09-15T22:05:21.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:21 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/1388702800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cff3d48-81e6-4748-96bb-c20ba054f287"}]: dispatch 2024-09-15T22:05:21.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:21 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cff3d48-81e6-4748-96bb-c20ba054f287"}]: dispatch 2024-09-15T22:05:21.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:21 smithi005 ceph-mon[30944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cff3d48-81e6-4748-96bb-c20ba054f287"}]': finished 2024-09-15T22:05:21.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:21 smithi005 ceph-mon[30944]: osdmap e69: 12 total, 11 up, 12 in 2024-09-15T22:05:21.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:21 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:22.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:22 smithi099 ceph-mon[34892]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:22.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:22 smithi099 ceph-mon[34892]: from='client.? 172.21.15.129:0/2731699622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:05:22.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:22 smithi099 ceph-mon[34892]: osdmap e70: 12 total, 11 up, 12 in 2024-09-15T22:05:22.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:22 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:22.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:22 smithi129 ceph-mon[35314]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:22.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:22 smithi129 ceph-mon[35314]: from='client.? 172.21.15.129:0/2731699622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:05:22.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:22 smithi129 ceph-mon[35314]: osdmap e70: 12 total, 11 up, 12 in 2024-09-15T22:05:22.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:22 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:22.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:22 smithi005 ceph-mon[30944]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:22.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:22 smithi005 ceph-mon[30944]: from='client.? 172.21.15.129:0/2731699622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-15T22:05:22.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:22 smithi005 ceph-mon[30944]: osdmap e70: 12 total, 11 up, 12 in 2024-09-15T22:05:22.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:22 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:24 smithi099 ceph-mon[34892]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:24 smithi099 ceph-mon[34892]: osdmap e71: 12 total, 11 up, 12 in 2024-09-15T22:05:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:24 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:24 smithi129 ceph-mon[35314]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:24 smithi129 ceph-mon[35314]: osdmap e71: 12 total, 11 up, 12 in 2024-09-15T22:05:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:24 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:24.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:24 smithi005 ceph-mon[30944]: pgmap v230: 1 pgs: 1 remapped+peering; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:24.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:24 smithi005 ceph-mon[30944]: osdmap e71: 12 total, 11 up, 12 in 2024-09-15T22:05:24.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:24 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:26.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:26 smithi129 ceph-mon[35314]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-15T22:05:26.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:26 smithi099 ceph-mon[34892]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-15T22:05:26.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:26 smithi005 ceph-mon[30944]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-09-15T22:05:28.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:28 smithi099 ceph-mon[34892]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:05:28.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:28 smithi129 ceph-mon[35314]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:05:28.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:28 smithi005 ceph-mon[30944]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-15T22:05:30.315 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:30 smithi129 ceph-mon[35314]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-15T22:05:30.315 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:30 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:05:30.315 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:30 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:30.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:30 smithi099 ceph-mon[34892]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-15T22:05:30.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:30 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:05:30.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:30 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:30.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:30 smithi005 ceph-mon[30944]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-15T22:05:30.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:30 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-15T22:05:30.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:30 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:31.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:31 smithi099 ceph-mon[34892]: Deploying daemon osd.11 on smithi129 2024-09-15T22:05:31.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:31 smithi129 ceph-mon[35314]: Deploying daemon osd.11 on smithi129 2024-09-15T22:05:31.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:31 smithi005 ceph-mon[30944]: Deploying daemon osd.11 on smithi129 2024-09-15T22:05:32.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:32 smithi099 ceph-mon[34892]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-15T22:05:32.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:32 smithi129 ceph-mon[35314]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-15T22:05:32.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:32 smithi005 ceph-mon[30944]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-15T22:05:34.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:34 smithi099 ceph-mon[34892]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:05:34.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:34 smithi129 ceph-mon[35314]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:05:34.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:34 smithi005 ceph-mon[30944]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-15T22:05:36.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:36 smithi099 ceph-mon[34892]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T22:05:36.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:36 smithi129 ceph-mon[35314]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T22:05:36.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:36 smithi005 ceph-mon[30944]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-15T22:05:37.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:37.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:37.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:37.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:37.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:37.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:37.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:37.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:37.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:38.285 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:38 smithi129 ceph-mon[35314]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:38.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:38 smithi099 ceph-mon[34892]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:38.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:38 smithi005 ceph-mon[30944]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:39.679 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.676+0000 7fcec57fa640 1 -- 172.21.15.129:0/2490881616 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fcec8062630 con 0x7fceb00a32f0 2024-09-15T22:05:39.679 INFO:teuthology.orchestra.run.smithi129.stdout:Created osd(s) 11 on host 'smithi129' 2024-09-15T22:05:39.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.678+0000 7fcece455640 1 -- 172.21.15.129:0/2490881616 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fceb00a32f0 msgr2=0x7fceb00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:39.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.678+0000 7fcece455640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fceb00a32f0 0x7fceb00a57b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcec8101b30 tx=0x7fceb8005f90 comp rx=0 tx=0).stop 2024-09-15T22:05:39.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.678+0000 7fcece455640 1 -- 172.21.15.129:0/2490881616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcec8102690 msgr2=0x7fcec819e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:39.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.678+0000 7fcece455640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcec8102690 0x7fcec819e3d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcec000cad0 tx=0x7fcec000cf90 comp rx=0 tx=0).stop 2024-09-15T22:05:39.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.678+0000 7fcece455640 1 -- 172.21.15.129:0/2490881616 shutdown_connections 2024-09-15T22:05:39.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.678+0000 7fcece455640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcec8102690 0x7fcec819e3d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:39.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.678+0000 7fcece455640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fceb00a32f0 0x7fceb00a57b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:39.681 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.678+0000 7fcece455640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec8101cd0 0x7fcec8197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:39.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.678+0000 7fcece455640 1 --2- 172.21.15.129:0/2490881616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcec8100ad0 0x7fcec8196e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:39.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.678+0000 7fcece455640 1 -- 172.21.15.129:0/2490881616 >> 172.21.15.129:0/2490881616 conn(0x7fcec80fc280 msgr2=0x7fcec8105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:39.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.679+0000 7fcece455640 1 -- 172.21.15.129:0/2490881616 shutdown_connections 2024-09-15T22:05:39.682 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:39.679+0000 7fcece455640 1 -- 172.21.15.129:0/2490881616 wait complete. 2024-09-15T22:05:40.389 DEBUG:teuthology.orchestra.run.smithi129:osd.11> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.11.service 2024-09-15T22:05:40.391 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-15T22:05:40.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T22:05:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:05:40.936 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:40 smithi005 ceph-mon[30944]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:40.936 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:40.936 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:40.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:40.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:40 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:40.944 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:40 smithi129 ceph-mon[35314]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:40.944 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:40.944 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:40.944 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:40.944 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:40 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:40 smithi099 ceph-mon[34892]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:41.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:40 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.162+0000 7fabba26f640 1 -- 172.21.15.5:0/3165066395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb4076220 msgr2=0x7fabb4074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.162+0000 7fabba26f640 1 --2- 172.21.15.5:0/3165066395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb4076220 0x7fabb4074680 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7faba4009900 tx=0x7faba402a410 comp rx=0 tx=0).stop 2024-09-15T22:05:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.162+0000 7fabba26f640 1 -- 172.21.15.5:0/3165066395 shutdown_connections 2024-09-15T22:05:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.162+0000 7fabba26f640 1 --2- 172.21.15.5:0/3165066395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb4109a70 0x7fabb410bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:41.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.162+0000 7fabba26f640 1 --2- 172.21.15.5:0/3165066395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb4074c50 0x7fabb40750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.162+0000 7fabba26f640 1 --2- 172.21.15.5:0/3165066395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb4076220 0x7fabb4074680 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.162+0000 7fabba26f640 1 -- 172.21.15.5:0/3165066395 >> 172.21.15.5:0/3165066395 conn(0x7fabb40fd3c0 msgr2=0x7fabb40ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.162+0000 7fabba26f640 1 -- 172.21.15.5:0/3165066395 shutdown_connections 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.162+0000 7fabba26f640 1 -- 172.21.15.5:0/3165066395 wait complete. 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.162+0000 7fabba26f640 1 Processor -- start 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.162+0000 7fabba26f640 1 -- start start 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabba26f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb4074c50 0x7fabb419a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabba26f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb4076220 0x7fabb419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabba26f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb4109a70 0x7fabb41a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabba26f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabb410fa70 con 0x7fabb4109a70 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabba26f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fabb410f8f0 con 0x7fabb4074c50 2024-09-15T22:05:41.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabba26f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fabb410fbf0 con 0x7fabb4076220 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabb880c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb4109a70 0x7fabb41a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabb880c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb4109a70 0x7fabb41a1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54890/0 (socket says 172.21.15.5:54890) 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabb880c640 1 -- 172.21.15.5:0/3498283875 learned_addr learned my addr 172.21.15.5:0/3498283875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabb3fff640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb4074c50 0x7fabb419a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabb37fe640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb4076220 0x7fabb419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabb880c640 1 -- 172.21.15.5:0/3498283875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb4076220 msgr2=0x7fabb419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabb880c640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb4076220 0x7fabb419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabb880c640 1 -- 172.21.15.5:0/3498283875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb4074c50 msgr2=0x7fabb419a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabb880c640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb4074c50 0x7fabb419a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.163+0000 7fabb880c640 1 -- 172.21.15.5:0/3498283875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb41a1cf0 con 0x7fabb4109a70 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.164+0000 7fabb880c640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb4109a70 0x7fabb41a1680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fabac00eb10 tx=0x7fabac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:41.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.164+0000 7fabb17fa640 1 -- 172.21.15.5:0/3498283875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac004b20 con 0x7fabb4109a70 2024-09-15T22:05:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.164+0000 7fabba26f640 1 -- 172.21.15.5:0/3498283875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb41a1fe0 con 0x7fabb4109a70 2024-09-15T22:05:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.164+0000 7fabb17fa640 1 -- 172.21.15.5:0/3498283875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabac004cc0 con 0x7fabb4109a70 2024-09-15T22:05:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.164+0000 7fabb17fa640 1 -- 172.21.15.5:0/3498283875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabac010760 con 0x7fabb4109a70 2024-09-15T22:05:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.164+0000 7fabba26f640 1 -- 172.21.15.5:0/3498283875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb41a2490 con 0x7fabb4109a70 2024-09-15T22:05:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.165+0000 7fabba26f640 1 -- 172.21.15.5:0/3498283875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb41087f0 con 0x7fabb4109a70 2024-09-15T22:05:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.168+0000 7fabb17fa640 1 -- 172.21.15.5:0/3498283875 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fabac010900 con 0x7fabb4109a70 2024-09-15T22:05:41.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.169+0000 7fabb17fa640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab9c0a3270 0x7fab9c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.169+0000 7fabb17fa640 1 -- 172.21.15.5:0/3498283875 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7fabac0cfa30 con 0x7fabb4109a70 2024-09-15T22:05:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.170+0000 7fabb3fff640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab9c0a3270 0x7fab9c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.170+0000 7fabb3fff640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab9c0a3270 0x7fab9c0a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faba4005ab0 tx=0x7faba4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:41.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.172+0000 7fabb17fa640 1 -- 172.21.15.5:0/3498283875 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabac0541b0 con 0x7fabb4109a70 2024-09-15T22:05:41.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.350+0000 7fabba26f640 1 -- 172.21.15.5:0/3498283875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fabb402da10 con 0x7fabb4109a70 2024-09-15T22:05:41.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.350+0000 7fabb17fa640 1 -- 172.21.15.5:0/3498283875 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7fabac09cd30 con 0x7fabb4109a70 2024-09-15T22:05:41.352 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:05:41.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.351+0000 7fabba26f640 1 -- 172.21.15.5:0/3498283875 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab9c0a3270 msgr2=0x7fab9c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:41.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.351+0000 7fabba26f640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab9c0a3270 0x7fab9c0a5730 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7faba4005ab0 tx=0x7faba4000bc0 comp rx=0 tx=0).stop 2024-09-15T22:05:41.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.351+0000 7fabba26f640 1 -- 172.21.15.5:0/3498283875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb4109a70 msgr2=0x7fabb41a1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:41.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.351+0000 7fabba26f640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb4109a70 0x7fabb41a1680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fabac00eb10 tx=0x7fabac00c2d0 comp rx=0 tx=0).stop 2024-09-15T22:05:41.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.351+0000 7fabba26f640 1 -- 172.21.15.5:0/3498283875 shutdown_connections 2024-09-15T22:05:41.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.351+0000 7fabba26f640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab9c0a3270 0x7fab9c0a5730 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:41.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.351+0000 7fabba26f640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabb4109a70 0x7fabb41a1680 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:41.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.351+0000 7fabba26f640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabb4076220 0x7fabb419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:41.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.351+0000 7fabba26f640 1 --2- 172.21.15.5:0/3498283875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabb4074c50 0x7fabb419a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:41.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.351+0000 7fabba26f640 1 -- 172.21.15.5:0/3498283875 >> 172.21.15.5:0/3498283875 conn(0x7fabb40fd3c0 msgr2=0x7fabb41059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:41.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.352+0000 7fabba26f640 1 -- 172.21.15.5:0/3498283875 shutdown_connections 2024-09-15T22:05:41.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:41.352+0000 7fabba26f640 1 -- 172.21.15.5:0/3498283875 wait complete. 2024-09-15T22:05:41.977 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:41 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3498283875' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:05:42.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:41 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3498283875' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:05:42.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:41 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3498283875' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:05:42.181 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726437919,"num_in_osds":12,"osd_in_since":1726437921,"num_remapped_pgs":0} 2024-09-15T22:05:42.279 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:05:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:05:42.029+0000 7f3cca711640 -1 osd.11 0 log_to_monitors true 2024-09-15T22:05:42.992 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:42 smithi129 ceph-mon[35314]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:42.992 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:42 smithi129 ceph-mon[35314]: from='osd.11 [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:05:42.992 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:42 smithi129 ceph-mon[35314]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:05:43.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:42 smithi099 ceph-mon[34892]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:43.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:42 smithi099 ceph-mon[34892]: from='osd.11 [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:05:43.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:42 smithi099 ceph-mon[34892]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:05:43.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:42 smithi005 ceph-mon[30944]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:43.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:42 smithi005 ceph-mon[30944]: from='osd.11 [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:05:43.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:42 smithi005 ceph-mon[30944]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-15T22:05:43.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T22:05:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:05:43.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:43 smithi005 ceph-mon[30944]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:05:43.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:43 smithi005 ceph-mon[30944]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T22:05:43.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:43 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:43.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:43 smithi005 ceph-mon[30944]: from='osd.11 [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:05:43.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:43 smithi005 ceph-mon[30944]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:05:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.931+0000 7f198ebb3640 1 -- 172.21.15.5:0/780369584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1988104ff0 msgr2=0x7f19881073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.931+0000 7f198ebb3640 1 --2- 172.21.15.5:0/780369584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1988104ff0 0x7f19881073e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1974009900 tx=0x7f197402a410 comp rx=0 tx=0).stop 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.931+0000 7f198ebb3640 1 -- 172.21.15.5:0/780369584 shutdown_connections 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.931+0000 7f198ebb3640 1 --2- 172.21.15.5:0/780369584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1988107920 0x7f1988109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.931+0000 7f198ebb3640 1 --2- 172.21.15.5:0/780369584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1988104ff0 0x7f19881073e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.931+0000 7f198ebb3640 1 --2- 172.21.15.5:0/780369584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19881026c0 0x7f1988104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.931+0000 7f198ebb3640 1 -- 172.21.15.5:0/780369584 >> 172.21.15.5:0/780369584 conn(0x7f19880fc280 msgr2=0x7f19880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.931+0000 7f198ebb3640 1 -- 172.21.15.5:0/780369584 shutdown_connections 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.931+0000 7f198ebb3640 1 -- 172.21.15.5:0/780369584 wait complete. 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.932+0000 7f198ebb3640 1 Processor -- start 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.932+0000 7f198ebb3640 1 -- start start 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.932+0000 7f198ebb3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19881026c0 0x7f1988197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.932+0000 7f198ebb3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1988104ff0 0x7f19881984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.932+0000 7f198ebb3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1988107920 0x7f198819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.932+0000 7f198ebb3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f198810d850 con 0x7f1988104ff0 2024-09-15T22:05:43.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.932+0000 7f198ebb3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f198810d6d0 con 0x7f1988107920 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.932+0000 7f198ebb3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f198810d9d0 con 0x7f19881026c0 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.932+0000 7f1987fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1988104ff0 0x7f19881984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f198c94f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19881026c0 0x7f1988197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f1987fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1988104ff0 0x7f19881984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45360/0 (socket says 172.21.15.5:45360) 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f1987fff640 1 -- 172.21.15.5:0/796822817 learned_addr learned my addr 172.21.15.5:0/796822817 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f1987fff640 1 -- 172.21.15.5:0/796822817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19881026c0 msgr2=0x7f1988197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f1987fff640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19881026c0 0x7f1988197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f1987fff640 1 -- 172.21.15.5:0/796822817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1988107920 msgr2=0x7f198819f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f1987fff640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1988107920 0x7f198819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f1987fff640 1 -- 172.21.15.5:0/796822817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198819fbb0 con 0x7f1988104ff0 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f198c94f640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19881026c0 0x7f1988197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:05:43.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f1987fff640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1988104ff0 0x7f19881984c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1974031610 tx=0x7f1974004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f1985ffb640 1 -- 172.21.15.5:0/796822817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197403f030 con 0x7f1988104ff0 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f1985ffb640 1 -- 172.21.15.5:0/796822817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1974003ad0 con 0x7f1988104ff0 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f198ebb3640 1 -- 172.21.15.5:0/796822817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198819fe40 con 0x7f1988104ff0 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.933+0000 7f1985ffb640 1 -- 172.21.15.5:0/796822817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974035070 con 0x7f1988104ff0 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.934+0000 7f198ebb3640 1 -- 172.21.15.5:0/796822817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19881a0290 con 0x7f1988104ff0 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.935+0000 7f198ebb3640 1 -- 172.21.15.5:0/796822817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19880fdd70 con 0x7f1988104ff0 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.936+0000 7f1985ffb640 1 -- 172.21.15.5:0/796822817 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f1974042030 con 0x7f1988104ff0 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.937+0000 7f1985ffb640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f19780a3270 0x7f19780a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.937+0000 7f198c94f640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f19780a3270 0x7f19780a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.937+0000 7f1985ffb640 1 -- 172.21.15.5:0/796822817 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6847+0+0 (secure 0 0 0) 0x7f19740efd10 con 0x7f1988104ff0 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.937+0000 7f198c94f640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f19780a3270 0x7f19780a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f197c009fc0 tx=0x7f197c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:43.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:43.940+0000 7f1985ffb640 1 -- 172.21.15.5:0/796822817 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19740bcf50 con 0x7f1988104ff0 2024-09-15T22:05:44.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:43 smithi099 ceph-mon[34892]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:05:44.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:43 smithi099 ceph-mon[34892]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T22:05:44.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:43 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:44.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:43 smithi099 ceph-mon[34892]: from='osd.11 [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:05:44.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:43 smithi099 ceph-mon[34892]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:05:44.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:43 smithi129 ceph-mon[35314]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-15T22:05:44.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:43 smithi129 ceph-mon[35314]: osdmap e72: 12 total, 11 up, 12 in 2024-09-15T22:05:44.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:43 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:44.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:43 smithi129 ceph-mon[35314]: from='osd.11 [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:05:44.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:43 smithi129 ceph-mon[35314]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]: dispatch 2024-09-15T22:05:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.121+0000 7f198ebb3640 1 -- 172.21.15.5:0/796822817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f198802da10 con 0x7f1988104ff0 2024-09-15T22:05:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.121+0000 7f1985ffb640 1 -- 172.21.15.5:0/796822817 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f19740bc850 con 0x7f1988104ff0 2024-09-15T22:05:44.123 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 -- 172.21.15.5:0/796822817 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f19780a3270 msgr2=0x7f19780a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f19780a3270 0x7f19780a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f197c009fc0 tx=0x7f197c005f90 comp rx=0 tx=0).stop 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 -- 172.21.15.5:0/796822817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1988104ff0 msgr2=0x7f19881984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1988104ff0 0x7f19881984c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1974031610 tx=0x7f1974004270 comp rx=0 tx=0).stop 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 -- 172.21.15.5:0/796822817 shutdown_connections 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1988107920 0x7f198819f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f19780a3270 0x7f19780a5730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1988104ff0 0x7f19881984c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 --2- 172.21.15.5:0/796822817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19881026c0 0x7f1988197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 -- 172.21.15.5:0/796822817 >> 172.21.15.5:0/796822817 conn(0x7f19880fc280 msgr2=0x7f1988103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 -- 172.21.15.5:0/796822817 shutdown_connections 2024-09-15T22:05:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:44.123+0000 7f198ebb3640 1 -- 172.21.15.5:0/796822817 wait complete. 2024-09-15T22:05:44.690 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:05:44.346+0000 7f3cc6694640 -1 osd.11 0 waiting for initial osdmap 2024-09-15T22:05:44.690 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:05:44.358+0000 7f3cbe4b4640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T22:05:44.936 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:44.936 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: Detected new or changed devices on smithi129 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi129 to 4272M 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:44.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:44.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:44.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:44.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:44.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:44.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:44.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/796822817' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:05:44.938 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:44 smithi005 ceph-mon[30944]: from='osd.11 ' entity='osd.11' 2024-09-15T22:05:44.971 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726437919,"num_in_osds":12,"osd_in_since":1726437921,"num_remapped_pgs":0} 2024-09-15T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: Detected new or changed devices on smithi129 2024-09-15T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi129 to 4272M 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/796822817' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:05:45.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:44 smithi099 ceph-mon[34892]: from='osd.11 ' entity='osd.11' 2024-09-15T22:05:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi129", "root=default"]}]': finished 2024-09-15T22:05:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: osdmap e73: 12 total, 11 up, 12 in 2024-09-15T22:05:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: Detected new or changed devices on smithi129 2024-09-15T22:05:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi129 to 4272M 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:45.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:45.097 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:45.097 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/796822817' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:05:45.097 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:44 smithi129 ceph-mon[35314]: from='osd.11 ' entity='osd.11' 2024-09-15T22:05:45.972 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-15T22:05:46.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:45 smithi005 ceph-mon[30944]: purged_snaps scrub starts 2024-09-15T22:05:46.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:45 smithi005 ceph-mon[30944]: purged_snaps scrub ok 2024-09-15T22:05:46.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:45 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:46.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:45 smithi005 ceph-mon[30944]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:46.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:45 smithi005 ceph-mon[30944]: osd.11 [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789] boot 2024-09-15T22:05:46.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:45 smithi005 ceph-mon[30944]: osdmap e74: 12 total, 12 up, 12 in 2024-09-15T22:05:46.002 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:45 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:46.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:45 smithi099 ceph-mon[34892]: purged_snaps scrub starts 2024-09-15T22:05:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:45 smithi099 ceph-mon[34892]: purged_snaps scrub ok 2024-09-15T22:05:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:45 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:45 smithi099 ceph-mon[34892]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:45 smithi099 ceph-mon[34892]: osd.11 [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789] boot 2024-09-15T22:05:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:45 smithi099 ceph-mon[34892]: osdmap e74: 12 total, 12 up, 12 in 2024-09-15T22:05:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:45 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:45 smithi129 ceph-mon[35314]: purged_snaps scrub starts 2024-09-15T22:05:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:45 smithi129 ceph-mon[35314]: purged_snaps scrub ok 2024-09-15T22:05:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:45 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:45 smithi129 ceph-mon[35314]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 983 GiB / 983 GiB avail 2024-09-15T22:05:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:45 smithi129 ceph-mon[35314]: osd.11 [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789] boot 2024-09-15T22:05:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:45 smithi129 ceph-mon[35314]: osdmap e74: 12 total, 12 up, 12 in 2024-09-15T22:05:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:45 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:05:46.239 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:05:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.703+0000 7fc7a30e0640 1 -- 172.21.15.5:0/1007153270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79c102690 msgr2=0x7fc79c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.703+0000 7fc7a30e0640 1 --2- 172.21.15.5:0/1007153270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79c102690 0x7fc79c108f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc794009ca0 tx=0x7fc79402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:05:46.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.704+0000 7fc7a30e0640 1 -- 172.21.15.5:0/1007153270 shutdown_connections 2024-09-15T22:05:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.704+0000 7fc7a30e0640 1 --2- 172.21.15.5:0/1007153270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79c102690 0x7fc79c108f20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.704+0000 7fc7a30e0640 1 --2- 172.21.15.5:0/1007153270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc79c101cd0 0x7fc79c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.704+0000 7fc7a30e0640 1 --2- 172.21.15.5:0/1007153270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc79c100ad0 0x7fc79c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.704+0000 7fc7a30e0640 1 -- 172.21.15.5:0/1007153270 >> 172.21.15.5:0/1007153270 conn(0x7fc79c0fc280 msgr2=0x7fc79c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.704+0000 7fc7a30e0640 1 -- 172.21.15.5:0/1007153270 shutdown_connections 2024-09-15T22:05:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.704+0000 7fc7a30e0640 1 -- 172.21.15.5:0/1007153270 wait complete. 2024-09-15T22:05:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.704+0000 7fc7a30e0640 1 Processor -- start 2024-09-15T22:05:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.704+0000 7fc7a30e0640 1 -- start start 2024-09-15T22:05:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a30e0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc79c100ad0 0x7fc79c104c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a30e0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc79c101cd0 0x7fc79c103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:46.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a30e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79c102690 0x7fc79c1037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a30e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c10c730 con 0x7fc79c102690 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a30e0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c10c5b0 con 0x7fc79c100ad0 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a30e0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c10c8b0 con 0x7fc79c101cd0 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a167d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79c102690 0x7fc79c1037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a167d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79c102690 0x7fc79c1037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45380/0 (socket says 172.21.15.5:45380) 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a167d640 1 -- 172.21.15.5:0/2387849660 learned_addr learned my addr 172.21.15.5:0/2387849660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a0e7c640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc79c100ad0 0x7fc79c104c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc79bfff640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc79c101cd0 0x7fc79c103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a167d640 1 -- 172.21.15.5:0/2387849660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc79c101cd0 msgr2=0x7fc79c103270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a167d640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc79c101cd0 0x7fc79c103270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a167d640 1 -- 172.21.15.5:0/2387849660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc79c100ad0 msgr2=0x7fc79c104c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:46.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a167d640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc79c100ad0 0x7fc79c104c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a167d640 1 -- 172.21.15.5:0/2387849660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c104070 con 0x7fc79c102690 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc79bfff640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc79c101cd0 0x7fc79c103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a167d640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79c102690 0x7fc79c1037b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc794005e30 tx=0x7fc794031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc799ffb640 1 -- 172.21.15.5:0/2387849660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794003c30 con 0x7fc79c102690 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.705+0000 7fc7a30e0640 1 -- 172.21.15.5:0/2387849660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79c1a1fa0 con 0x7fc79c102690 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.706+0000 7fc799ffb640 1 -- 172.21.15.5:0/2387849660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc794003dd0 con 0x7fc79c102690 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.706+0000 7fc799ffb640 1 -- 172.21.15.5:0/2387849660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc79402fde0 con 0x7fc79c102690 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.706+0000 7fc7a30e0640 1 -- 172.21.15.5:0/2387849660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79c1a24a0 con 0x7fc79c102690 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.707+0000 7fc7a30e0640 1 -- 172.21.15.5:0/2387849660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc79c1a2790 con 0x7fc79c102690 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.708+0000 7fc799ffb640 1 -- 172.21.15.5:0/2387849660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fc794004050 con 0x7fc79c102690 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.709+0000 7fc799ffb640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc78c0a32c0 0x7fc78c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:46.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.709+0000 7fc7a0e7c640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc78c0a32c0 0x7fc78c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.709+0000 7fc799ffb640 1 -- 172.21.15.5:0/2387849660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fc7940f0a00 con 0x7fc79c102690 2024-09-15T22:05:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.709+0000 7fc7a0e7c640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc78c0a32c0 0x7fc78c0a5780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc79c101b30 tx=0x7fc79000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:46.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.712+0000 7fc799ffb640 1 -- 172.21.15.5:0/2387849660 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7940bdb20 con 0x7fc79c102690 2024-09-15T22:05:46.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.888+0000 7fc7a30e0640 1 -- 172.21.15.5:0/2387849660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc79c1089e0 con 0x7fc79c102690 2024-09-15T22:05:46.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.889+0000 7fc799ffb640 1 -- 172.21.15.5:0/2387849660 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fc7940bd420 con 0x7fc79c102690 2024-09-15T22:05:46.890 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:05:46.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 -- 172.21.15.5:0/2387849660 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc78c0a32c0 msgr2=0x7fc78c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc78c0a32c0 0x7fc78c0a5780 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc79c101b30 tx=0x7fc79000b390 comp rx=0 tx=0).stop 2024-09-15T22:05:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 -- 172.21.15.5:0/2387849660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79c102690 msgr2=0x7fc79c1037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79c102690 0x7fc79c1037b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc794005e30 tx=0x7fc794031ee0 comp rx=0 tx=0).stop 2024-09-15T22:05:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 -- 172.21.15.5:0/2387849660 shutdown_connections 2024-09-15T22:05:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc78c0a32c0 0x7fc78c0a5780 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc79c102690 0x7fc79c1037b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc79c101cd0 0x7fc79c103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 --2- 172.21.15.5:0/2387849660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc79c100ad0 0x7fc79c104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 -- 172.21.15.5:0/2387849660 >> 172.21.15.5:0/2387849660 conn(0x7fc79c0fc280 msgr2=0x7fc79c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 -- 172.21.15.5:0/2387849660 shutdown_connections 2024-09-15T22:05:46.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:46.891+0000 7fc7a30e0640 1 -- 172.21.15.5:0/2387849660 wait complete. 2024-09-15T22:05:47.604 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1726437945,"num_in_osds":12,"osd_in_since":1726437921,"num_remapped_pgs":0} 2024-09-15T22:05:47.605 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:05:47.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:47 smithi005 ceph-mon[30944]: osdmap e75: 12 total, 12 up, 12 in 2024-09-15T22:05:47.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:47 smithi005 ceph-mon[30944]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:47.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:47 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2387849660' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:05:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:05:48.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:47 smithi099 ceph-mon[34892]: osdmap e75: 12 total, 12 up, 12 in 2024-09-15T22:05:48.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:47 smithi099 ceph-mon[34892]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:48.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:47 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2387849660' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:05:48.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:47 smithi129 ceph-mon[35314]: osdmap e75: 12 total, 12 up, 12 in 2024-09-15T22:05:48.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:47 smithi129 ceph-mon[35314]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:48.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:47 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2387849660' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-15T22:05:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.361+0000 7f711817d640 1 -- 172.21.15.5:0/3723028543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7110100a80 msgr2=0x7f7110100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.361+0000 7f711817d640 1 --2- 172.21.15.5:0/3723028543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7110100a80 0x7f7110100ee0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7100009930 tx=0x7f710002a420 comp rx=0 tx=0).stop 2024-09-15T22:05:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.361+0000 7f711817d640 1 -- 172.21.15.5:0/3723028543 shutdown_connections 2024-09-15T22:05:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.361+0000 7f711817d640 1 --2- 172.21.15.5:0/3723028543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7110101420 0x7f711010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.361+0000 7f711817d640 1 --2- 172.21.15.5:0/3723028543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7110100a80 0x7f7110100ee0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.361+0000 7f711817d640 1 --2- 172.21.15.5:0/3723028543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7110106a80 0x7f7110106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.361+0000 7f711817d640 1 -- 172.21.15.5:0/3723028543 >> 172.21.15.5:0/3723028543 conn(0x7f71100fc7a0 msgr2=0x7f71100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.361+0000 7f711817d640 1 -- 172.21.15.5:0/3723028543 shutdown_connections 2024-09-15T22:05:48.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.361+0000 7f711817d640 1 -- 172.21.15.5:0/3723028543 wait complete. 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f711817d640 1 Processor -- start 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f711817d640 1 -- start start 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f711817d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7110100a80 0x7f711010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f711817d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7110101420 0x7f711010e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f711817d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7110106a80 0x7f711010ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f711817d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7110111f60 con 0x7f7110100a80 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f711817d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7110111de0 con 0x7f7110101420 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f711817d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71101120e0 con 0x7f7110106a80 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f7115f19640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7110100a80 0x7f711010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f7115f19640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7110100a80 0x7f711010e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45390/0 (socket says 172.21.15.5:45390) 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f7115f19640 1 -- 172.21.15.5:0/1042781261 learned_addr learned my addr 172.21.15.5:0/1042781261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f7115718640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7110101420 0x7f711010e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:48.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f711671a640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7110106a80 0x7f711010ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f7115f19640 1 -- 172.21.15.5:0/1042781261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7110106a80 msgr2=0x7f711010ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f7115f19640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7110106a80 0x7f711010ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f7115f19640 1 -- 172.21.15.5:0/1042781261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7110101420 msgr2=0x7f711010e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f7115f19640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7110101420 0x7f711010e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f7115f19640 1 -- 172.21.15.5:0/1042781261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71101a68c0 con 0x7f7110100a80 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f7115718640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7110101420 0x7f711010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.362+0000 7f7115f19640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7110100a80 0x7f711010e3e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f710c00d950 tx=0x7f710c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.363+0000 7f7106ffd640 1 -- 172.21.15.5:0/1042781261 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c017030 con 0x7f7110100a80 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.363+0000 7f711817d640 1 -- 172.21.15.5:0/1042781261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71101a6bb0 con 0x7f7110100a80 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.363+0000 7f7106ffd640 1 -- 172.21.15.5:0/1042781261 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f710c004020 con 0x7f7110100a80 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.363+0000 7f7106ffd640 1 -- 172.21.15.5:0/1042781261 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c0047a0 con 0x7f7110100a80 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.363+0000 7f711817d640 1 -- 172.21.15.5:0/1042781261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71101a70c0 con 0x7f7110100a80 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.364+0000 7f711817d640 1 -- 172.21.15.5:0/1042781261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7110100ee0 con 0x7f7110100a80 2024-09-15T22:05:48.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.365+0000 7f7106ffd640 1 -- 172.21.15.5:0/1042781261 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f710c0041d0 con 0x7f7110100a80 2024-09-15T22:05:48.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.366+0000 7f7106ffd640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70fc0a3240 0x7f70fc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:48.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.366+0000 7f7115718640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70fc0a3240 0x7f70fc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:48.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.366+0000 7f7106ffd640 1 -- 172.21.15.5:0/1042781261 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f710c0cf4f0 con 0x7f7110100a80 2024-09-15T22:05:48.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.366+0000 7f7115718640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70fc0a3240 0x7f70fc0a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7100000c00 tx=0x7f7100005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:48.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.369+0000 7f7106ffd640 1 -- 172.21.15.5:0/1042781261 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f710c09c6b0 con 0x7f7110100a80 2024-09-15T22:05:48.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.544+0000 7f711817d640 1 -- 172.21.15.5:0/1042781261 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f711010f910 con 0x7f7110100a80 2024-09-15T22:05:48.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.545+0000 7f7106ffd640 1 -- 172.21.15.5:0/1042781261 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18364 (secure 0 0 0) 0x7f710c0ba7d0 con 0x7f7110100a80 2024-09-15T22:05:48.547 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:05:48.547 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:05:46.696599+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:05:48.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 -- 172.21.15.5:0/1042781261 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70fc0a3240 msgr2=0x7f70fc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:48.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70fc0a3240 0x7f70fc0a5700 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7100000c00 tx=0x7f7100005a20 comp rx=0 tx=0).stop 2024-09-15T22:05:48.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 -- 172.21.15.5:0/1042781261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7110100a80 msgr2=0x7f711010e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:48.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7110100a80 0x7f711010e3e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f710c00d950 tx=0x7f710c00de10 comp rx=0 tx=0).stop 2024-09-15T22:05:48.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 -- 172.21.15.5:0/1042781261 shutdown_connections 2024-09-15T22:05:48.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7110106a80 0x7f711010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:48.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7110101420 0x7f711010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:48.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70fc0a3240 0x7f70fc0a5700 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:48.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 --2- 172.21.15.5:0/1042781261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7110100a80 0x7f711010e3e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:48.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 -- 172.21.15.5:0/1042781261 >> 172.21.15.5:0/1042781261 conn(0x7f71100fc7a0 msgr2=0x7f71100fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:48.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 -- 172.21.15.5:0/1042781261 shutdown_connections 2024-09-15T22:05:48.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:48.548+0000 7f711817d640 1 -- 172.21.15.5:0/1042781261 wait complete. 2024-09-15T22:05:48.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:48 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1042781261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:05:49.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:48 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1042781261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:05:49.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:48 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1042781261' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:05:49.333 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T22:01:57.184697+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-15T22:05:49.334 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-15T22:05:49.590 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:05:49.989 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:49 smithi005 ceph-mon[30944]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:50.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:49 smithi099 ceph-mon[34892]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:50.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:49 smithi129 ceph-mon[35314]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:50.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.073+0000 7f957cbc7640 1 -- 172.21.15.5:0/2100577919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578100e20 msgr2=0x7f9578101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.073+0000 7f957cbc7640 1 --2- 172.21.15.5:0/2100577919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578100e20 0x7f9578101200 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9568009900 tx=0x7f956802a480 comp rx=0 tx=0).stop 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.073+0000 7f957cbc7640 1 -- 172.21.15.5:0/2100577919 shutdown_connections 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.073+0000 7f957cbc7640 1 --2- 172.21.15.5:0/2100577919 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9578102ef0 0x7f957810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.073+0000 7f957cbc7640 1 --2- 172.21.15.5:0/2100577919 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9578102570 0x7f95781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.073+0000 7f957cbc7640 1 --2- 172.21.15.5:0/2100577919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578100e20 0x7f9578101200 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.073+0000 7f957cbc7640 1 -- 172.21.15.5:0/2100577919 >> 172.21.15.5:0/2100577919 conn(0x7f95780fc7d0 msgr2=0x7f95780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.073+0000 7f957cbc7640 1 -- 172.21.15.5:0/2100577919 shutdown_connections 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.073+0000 7f957cbc7640 1 -- 172.21.15.5:0/2100577919 wait complete. 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.074+0000 7f957cbc7640 1 Processor -- start 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.074+0000 7f957cbc7640 1 -- start start 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.074+0000 7f957cbc7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578100e20 0x7f9578197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.074+0000 7f957cbc7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9578102570 0x7f9578197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.074+0000 7f957cbc7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9578102ef0 0x7f957819bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.074+0000 7f957cbc7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9578110fd0 con 0x7f9578100e20 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.074+0000 7f957cbc7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9578110e50 con 0x7f9578102ef0 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.074+0000 7f957cbc7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9578111150 con 0x7f9578102570 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f957659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578100e20 0x7f9578197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f957659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578100e20 0x7f9578197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45402/0 (socket says 172.21.15.5:45402) 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f957659c640 1 -- 172.21.15.5:0/32107978 learned_addr learned my addr 172.21.15.5:0/32107978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f956ffff640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9578102570 0x7f9578197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f9576d9d640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9578102ef0 0x7f957819bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f957659c640 1 -- 172.21.15.5:0/32107978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9578102570 msgr2=0x7f9578197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f957659c640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9578102570 0x7f9578197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f957659c640 1 -- 172.21.15.5:0/32107978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9578102ef0 msgr2=0x7f957819bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f957659c640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9578102ef0 0x7f957819bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f957659c640 1 -- 172.21.15.5:0/32107978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f957819c2e0 con 0x7f9578100e20 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f9576d9d640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9578102ef0 0x7f957819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f957659c640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578100e20 0x7f9578197330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f956802aa70 tx=0x7f9568003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f956f7fe640 1 -- 172.21.15.5:0/32107978 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956803f070 con 0x7f9578100e20 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f956f7fe640 1 -- 172.21.15.5:0/32107978 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9568031c20 con 0x7f9578100e20 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f957cbc7640 1 -- 172.21.15.5:0/32107978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f957819c570 con 0x7f9578100e20 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f956f7fe640 1 -- 172.21.15.5:0/32107978 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9568035030 con 0x7f9578100e20 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.075+0000 7f957cbc7640 1 -- 172.21.15.5:0/32107978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9578069f70 con 0x7f9578100e20 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.076+0000 7f957cbc7640 1 -- 172.21.15.5:0/32107978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95781021b0 con 0x7f9578100e20 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.077+0000 7f956f7fe640 1 -- 172.21.15.5:0/32107978 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f9568042070 con 0x7f9578100e20 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.078+0000 7f956f7fe640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f95640a32c0 0x7f95640a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.078+0000 7f956ffff640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f95640a32c0 0x7f95640a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.078+0000 7f956f7fe640 1 -- 172.21.15.5:0/32107978 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f95680efc10 con 0x7f9578100e20 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.079+0000 7f956ffff640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f95640a32c0 0x7f95640a5780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f95780fec10 tx=0x7f956000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:50.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.082+0000 7f956f7fe640 1 -- 172.21.15.5:0/32107978 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95680bcd30 con 0x7f9578100e20 2024-09-15T22:05:50.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.256+0000 7f957cbc7640 1 -- 172.21.15.5:0/32107978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9578198690 con 0x7f9578100e20 2024-09-15T22:05:50.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.257+0000 7f956f7fe640 1 -- 172.21.15.5:0/32107978 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7f95680bc630 con 0x7f9578100e20 2024-09-15T22:05:50.258 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-15T22:05:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 -- 172.21.15.5:0/32107978 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f95640a32c0 msgr2=0x7f95640a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f95640a32c0 0x7f95640a5780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f95780fec10 tx=0x7f956000a2d0 comp rx=0 tx=0).stop 2024-09-15T22:05:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 -- 172.21.15.5:0/32107978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578100e20 msgr2=0x7f9578197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578100e20 0x7f9578197330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f956802aa70 tx=0x7f9568003b60 comp rx=0 tx=0).stop 2024-09-15T22:05:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 -- 172.21.15.5:0/32107978 shutdown_connections 2024-09-15T22:05:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9578102ef0 0x7f957819bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9578102570 0x7f9578197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f95640a32c0 0x7f95640a5780 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:50.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 --2- 172.21.15.5:0/32107978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9578100e20 0x7f9578197330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 -- 172.21.15.5:0/32107978 >> 172.21.15.5:0/32107978 conn(0x7f95780fc7d0 msgr2=0x7f9578106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 -- 172.21.15.5:0/32107978 shutdown_connections 2024-09-15T22:05:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:50.259+0000 7f957cbc7640 1 -- 172.21.15.5:0/32107978 wait complete. 2024-09-15T22:05:50.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:50 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/32107978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:05:50.975 INFO:tasks.cephadm:Adding mds.a on smithi005 2024-09-15T22:05:50.975 INFO:tasks.cephadm:Adding mds.d on smithi005 2024-09-15T22:05:50.975 INFO:tasks.cephadm:Adding mds.g on smithi005 2024-09-15T22:05:50.975 INFO:tasks.cephadm:Adding mds.j on smithi005 2024-09-15T22:05:50.975 INFO:tasks.cephadm:Adding mds.b on smithi099 2024-09-15T22:05:50.975 INFO:tasks.cephadm:Adding mds.e on smithi099 2024-09-15T22:05:50.975 INFO:tasks.cephadm:Adding mds.h on smithi099 2024-09-15T22:05:50.975 INFO:tasks.cephadm:Adding mds.k on smithi099 2024-09-15T22:05:50.975 INFO:tasks.cephadm:Adding mds.c on smithi129 2024-09-15T22:05:50.976 INFO:tasks.cephadm:Adding mds.f on smithi129 2024-09-15T22:05:50.976 INFO:tasks.cephadm:Adding mds.i on smithi129 2024-09-15T22:05:50.976 INFO:tasks.cephadm:Adding mds.l on smithi129 2024-09-15T22:05:50.976 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l' 2024-09-15T22:05:51.005 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:50 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/32107978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:05:51.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:50 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/32107978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:05:51.232 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:05:51.714 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.694+0000 7fbfd863a640 1 -- 172.21.15.129:0/3444918811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfd0101d10 msgr2=0x7fbfd0102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:51.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.694+0000 7fbfd863a640 1 --2- 172.21.15.129:0/3444918811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfd0101d10 0x7fbfd0102190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009900 tx=0x7fbfc002a410 comp rx=0 tx=0).stop 2024-09-15T22:05:51.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.694+0000 7fbfd863a640 1 -- 172.21.15.129:0/3444918811 shutdown_connections 2024-09-15T22:05:51.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.694+0000 7fbfd863a640 1 --2- 172.21.15.129:0/3444918811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfd01026d0 0x7fbfd0108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:51.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.694+0000 7fbfd863a640 1 --2- 172.21.15.129:0/3444918811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfd0101d10 0x7fbfd0102190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:51.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.694+0000 7fbfd863a640 1 --2- 172.21.15.129:0/3444918811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd0100b10 0x7fbfd0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:51.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.694+0000 7fbfd863a640 1 -- 172.21.15.129:0/3444918811 >> 172.21.15.129:0/3444918811 conn(0x7fbfd00fc2a0 msgr2=0x7fbfd00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:51.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.694+0000 7fbfd863a640 1 -- 172.21.15.129:0/3444918811 shutdown_connections 2024-09-15T22:05:51.715 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.694+0000 7fbfd863a640 1 -- 172.21.15.129:0/3444918811 wait complete. 2024-09-15T22:05:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd863a640 1 Processor -- start 2024-09-15T22:05:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd863a640 1 -- start start 2024-09-15T22:05:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd863a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfd0100b10 0x7fbfd0196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd863a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd0101d10 0x7fbfd0197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd863a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfd01026d0 0x7fbfd019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd863a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd010c810 con 0x7fbfd0100b10 2024-09-15T22:05:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd863a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd010c690 con 0x7fbfd0101d10 2024-09-15T22:05:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd863a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd010c990 con 0x7fbfd01026d0 2024-09-15T22:05:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd6bd7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfd01026d0 0x7fbfd019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd6bd7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfd01026d0 0x7fbfd019e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.129:34360/0 (socket says 172.21.15.129:34360) 2024-09-15T22:05:51.716 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd6bd7640 1 -- 172.21.15.129:0/2545294465 learned_addr learned my addr 172.21.15.129:0/2545294465 (peer_addr_for_me v2:172.21.15.129:0/0) 2024-09-15T22:05:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd63d6640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfd0100b10 0x7fbfd0196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd6bd7640 1 -- 172.21.15.129:0/2545294465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd0101d10 msgr2=0x7fbfd0197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd6bd7640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd0101d10 0x7fbfd0197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd6bd7640 1 -- 172.21.15.129:0/2545294465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfd0100b10 msgr2=0x7fbfd0196e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd6bd7640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfd0100b10 0x7fbfd0196e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.695+0000 7fbfd6bd7640 1 -- 172.21.15.129:0/2545294465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd019ea60 con 0x7fbfd01026d0 2024-09-15T22:05:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.696+0000 7fbfd6bd7640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfd01026d0 0x7fbfd019e3f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800cce0 tx=0x7fbfc800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.696+0000 7fbfc77fe640 1 -- 172.21.15.129:0/2545294465 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc800f450 con 0x7fbfd01026d0 2024-09-15T22:05:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.696+0000 7fbfd863a640 1 -- 172.21.15.129:0/2545294465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd019ed50 con 0x7fbfd01026d0 2024-09-15T22:05:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.696+0000 7fbfc77fe640 1 -- 172.21.15.129:0/2545294465 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc800fad0 con 0x7fbfd01026d0 2024-09-15T22:05:51.717 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.696+0000 7fbfc77fe640 1 -- 172.21.15.129:0/2545294465 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8018c10 con 0x7fbfd01026d0 2024-09-15T22:05:51.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.696+0000 7fbfd863a640 1 -- 172.21.15.129:0/2545294465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd019f2c0 con 0x7fbfd01026d0 2024-09-15T22:05:51.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.697+0000 7fbfd863a640 1 -- 172.21.15.129:0/2545294465 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd0107ee0 con 0x7fbfd01026d0 2024-09-15T22:05:51.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.698+0000 7fbfc77fe640 1 -- 172.21.15.129:0/2545294465 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fbfc800f5f0 con 0x7fbfd01026d0 2024-09-15T22:05:51.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.698+0000 7fbfc77fe640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfbc0a3270 0x7fbfbc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:51.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.699+0000 7fbfc77fe640 1 -- 172.21.15.129:0/2545294465 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fbfc80cfc20 con 0x7fbfd01026d0 2024-09-15T22:05:51.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.699+0000 7fbfd63d6640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfbc0a3270 0x7fbfbc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:51.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.699+0000 7fbfd63d6640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfbc0a3270 0x7fbfbc0a5730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0101b70 tx=0x7fbfcc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:51.718 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.702+0000 7fbfc77fe640 1 -- 172.21.15.129:0/2545294465 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbfc8014030 con 0x7fbfd01026d0 2024-09-15T22:05:51.882 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.879+0000 7fbfd863a640 1 -- 172.21.15.129:0/2545294465 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l", "target": ["mon-mgr", ""]}) -- 0x7fbfd0062730 con 0x7fbfbc0a3270 2024-09-15T22:05:51.882 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:51 smithi129 ceph-mon[35314]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:51.889 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.887+0000 7fbfc77fe640 1 -- 172.21.15.129:0/2545294465 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fbfd0062730 con 0x7fbfbc0a3270 2024-09-15T22:05:51.889 INFO:teuthology.orchestra.run.smithi129.stdout:Scheduled mds.all update... 2024-09-15T22:05:51.891 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 -- 172.21.15.129:0/2545294465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfbc0a3270 msgr2=0x7fbfbc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:51.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfbc0a3270 0x7fbfbc0a5730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0101b70 tx=0x7fbfcc008040 comp rx=0 tx=0).stop 2024-09-15T22:05:51.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 -- 172.21.15.129:0/2545294465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfd01026d0 msgr2=0x7fbfd019e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:51.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfd01026d0 0x7fbfd019e3f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800cce0 tx=0x7fbfc800a620 comp rx=0 tx=0).stop 2024-09-15T22:05:51.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 -- 172.21.15.129:0/2545294465 shutdown_connections 2024-09-15T22:05:51.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbfd01026d0 0x7fbfd019e3f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:51.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbfd0101d10 0x7fbfd0197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:51.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbfbc0a3270 0x7fbfbc0a5730 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:51.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 --2- 172.21.15.129:0/2545294465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbfd0100b10 0x7fbfd0196e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:51.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 -- 172.21.15.129:0/2545294465 >> 172.21.15.129:0/2545294465 conn(0x7fbfd00fc2a0 msgr2=0x7fbfd01050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:51.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 -- 172.21.15.129:0/2545294465 shutdown_connections 2024-09-15T22:05:51.892 INFO:teuthology.orchestra.run.smithi129.stderr:2024-09-15T22:05:51.889+0000 7fbfd863a640 1 -- 172.21.15.129:0/2545294465 wait complete. 2024-09-15T22:05:52.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:51 smithi099 ceph-mon[34892]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:52.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:51 smithi005 ceph-mon[30944]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:52.701 DEBUG:teuthology.orchestra.run.smithi005:mds.a> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.a.service 2024-09-15T22:05:52.704 DEBUG:teuthology.orchestra.run.smithi005:mds.d> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.d.service 2024-09-15T22:05:52.747 DEBUG:teuthology.orchestra.run.smithi005:mds.g> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.g.service 2024-09-15T22:05:52.790 DEBUG:teuthology.orchestra.run.smithi005:mds.j> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.j.service 2024-09-15T22:05:52.833 DEBUG:teuthology.orchestra.run.smithi099:mds.b> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.b.service 2024-09-15T22:05:52.836 DEBUG:teuthology.orchestra.run.smithi099:mds.e> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.e.service 2024-09-15T22:05:52.879 DEBUG:teuthology.orchestra.run.smithi099:mds.h> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.h.service 2024-09-15T22:05:52.923 DEBUG:teuthology.orchestra.run.smithi099:mds.k> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.k.service 2024-09-15T22:05:52.965 DEBUG:teuthology.orchestra.run.smithi129:mds.c> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.c.service 2024-09-15T22:05:52.967 DEBUG:teuthology.orchestra.run.smithi129:mds.f> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.f.service 2024-09-15T22:05:52.988 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:52 smithi129 ceph-mon[35314]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:52.988 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:52 smithi129 ceph-mon[35314]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l;count:12 2024-09-15T22:05:52.989 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:52.989 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:52.989 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:52.989 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:52.989 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:52.989 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:05:52.989 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:05:52.989 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:52 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:52.989 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:52 smithi129 ceph-mon[35314]: Deploying daemon mds.l on smithi129 2024-09-15T22:05:52.990 DEBUG:teuthology.orchestra.run.smithi129:mds.i> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.i.service 2024-09-15T22:05:53.033 DEBUG:teuthology.orchestra.run.smithi129:mds.l> sudo journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.l.service 2024-09-15T22:05:53.076 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-15T22:05:53.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:05:53.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:52 smithi005 ceph-mon[30944]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:53.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:52 smithi005 ceph-mon[30944]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l;count:12 2024-09-15T22:05:53.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:53.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:53.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:53.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:53.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:53.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:05:53.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:05:53.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:52 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:53.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:52 smithi005 ceph-mon[30944]: Deploying daemon mds.l on smithi129 2024-09-15T22:05:53.335 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:05:53.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:52 smithi099 ceph-mon[34892]: from='client.24406 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:05:53.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:52 smithi099 ceph-mon[34892]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l;count:12 2024-09-15T22:05:53.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:05:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:05:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:05:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:05:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:52 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:52 smithi099 ceph-mon[34892]: Deploying daemon mds.l on smithi129 2024-09-15T22:05:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.829+0000 7f405e190640 1 -- 172.21.15.5:0/1647893126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058101de0 msgr2=0x7f40581021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.829+0000 7f405e190640 1 --2- 172.21.15.5:0/1647893126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058101de0 0x7f40581021c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4048009900 tx=0x7f404802a410 comp rx=0 tx=0).stop 2024-09-15T22:05:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.829+0000 7f405e190640 1 -- 172.21.15.5:0/1647893126 shutdown_connections 2024-09-15T22:05:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.829+0000 7f405e190640 1 --2- 172.21.15.5:0/1647893126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4058106f10 0x7f405810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.829+0000 7f405e190640 1 --2- 172.21.15.5:0/1647893126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4058102790 0x7f40581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.829+0000 7f405e190640 1 --2- 172.21.15.5:0/1647893126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058101de0 0x7f40581021c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.829+0000 7f405e190640 1 -- 172.21.15.5:0/1647893126 >> 172.21.15.5:0/1647893126 conn(0x7f40580fd8c0 msgr2=0x7f40580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.829+0000 7f405e190640 1 -- 172.21.15.5:0/1647893126 shutdown_connections 2024-09-15T22:05:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.829+0000 7f405e190640 1 -- 172.21.15.5:0/1647893126 wait complete. 2024-09-15T22:05:53.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.830+0000 7f405e190640 1 Processor -- start 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.830+0000 7f405e190640 1 -- start start 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.830+0000 7f405e190640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4058101de0 0x7f40580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.830+0000 7f405e190640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4058102790 0x7f4058074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.830+0000 7f405e190640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058106f10 0x7f4058074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.830+0000 7f405e190640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4058111fc0 con 0x7f4058106f10 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.830+0000 7f405e190640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4058111e40 con 0x7f4058102790 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.830+0000 7f405e190640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4058112140 con 0x7f4058101de0 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4057fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058106f10 0x7f4058074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4057fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058106f10 0x7f4058074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45652/0 (socket says 172.21.15.5:45652) 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4057fff640 1 -- 172.21.15.5:0/3561827483 learned_addr learned my addr 172.21.15.5:0/3561827483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f40577fe640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4058101de0 0x7f40580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4056ffd640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4058102790 0x7f4058074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4057fff640 1 -- 172.21.15.5:0/3561827483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4058101de0 msgr2=0x7f40580744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4057fff640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4058101de0 0x7f40580744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4057fff640 1 -- 172.21.15.5:0/3561827483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4058102790 msgr2=0x7f4058074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4057fff640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4058102790 0x7f4058074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4057fff640 1 -- 172.21.15.5:0/3561827483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40581a6920 con 0x7f4058106f10 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4057fff640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058106f10 0x7f4058074f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f405000eaa0 tx=0x7f405000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4054ff9640 1 -- 172.21.15.5:0/3561827483 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405000cdb0 con 0x7f4058106f10 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4054ff9640 1 -- 172.21.15.5:0/3561827483 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4050005020 con 0x7f4058106f10 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f405e190640 1 -- 172.21.15.5:0/3561827483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40581a6c10 con 0x7f4058106f10 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f4054ff9640 1 -- 172.21.15.5:0/3561827483 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40500116b0 con 0x7f4058106f10 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.831+0000 7f405e190640 1 -- 172.21.15.5:0/3561827483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40581a7150 con 0x7f4058106f10 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.832+0000 7f405e190640 1 -- 172.21.15.5:0/3561827483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40581021c0 con 0x7f4058106f10 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.833+0000 7f4054ff9640 1 -- 172.21.15.5:0/3561827483 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f4050005560 con 0x7f4058106f10 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.834+0000 7f4054ff9640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f40400a32c0 0x7f40400a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.834+0000 7f40577fe640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f40400a32c0 0x7f40400a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:53.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.834+0000 7f4054ff9640 1 -- 172.21.15.5:0/3561827483 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f40500cf550 con 0x7f4058106f10 2024-09-15T22:05:53.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.835+0000 7f40577fe640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f40400a32c0 0x7f40400a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4048000c00 tx=0x7f4048005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:53.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:53.838+0000 7f4054ff9640 1 -- 172.21.15.5:0/3561827483 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4050016030 con 0x7f4058106f10 2024-09-15T22:05:54.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.050+0000 7f405e190640 1 -- 172.21.15.5:0/3561827483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f405810a0f0 con 0x7f4058106f10 2024-09-15T22:05:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.050+0000 7f4054ff9640 1 -- 172.21.15.5:0/3561827483 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f405009c0c0 con 0x7f4058106f10 2024-09-15T22:05:54.052 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:05:54.052 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":1,"btime":"2024-09-15T21:58:48:668797+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-15T22:05:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 1 2024-09-15T22:05:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 -- 172.21.15.5:0/3561827483 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f40400a32c0 msgr2=0x7f40400a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f40400a32c0 0x7f40400a5780 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4048000c00 tx=0x7f4048005b50 comp rx=0 tx=0).stop 2024-09-15T22:05:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 -- 172.21.15.5:0/3561827483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058106f10 msgr2=0x7f4058074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058106f10 0x7f4058074f70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f405000eaa0 tx=0x7f405000ef60 comp rx=0 tx=0).stop 2024-09-15T22:05:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 -- 172.21.15.5:0/3561827483 shutdown_connections 2024-09-15T22:05:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f40400a32c0 0x7f40400a5780 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4058106f10 0x7f4058074f70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4058102790 0x7f4058074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 --2- 172.21.15.5:0/3561827483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4058101de0 0x7f40580744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 -- 172.21.15.5:0/3561827483 >> 172.21.15.5:0/3561827483 conn(0x7f40580fd8c0 msgr2=0x7f40580febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:54.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 -- 172.21.15.5:0/3561827483 shutdown_connections 2024-09-15T22:05:54.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:54.053+0000 7f405e190640 1 -- 172.21.15.5:0/3561827483 wait complete. 2024-09-15T22:05:54.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:54 smithi005 ceph-mon[30944]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:54.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:54 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3561827483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:05:54.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:54 smithi099 ceph-mon[34892]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:54.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:54 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3561827483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:05:54.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:54 smithi129 ceph-mon[35314]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:54.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:54 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3561827483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:05:54.594 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:05:54 smithi129 systemd[1]: Starting Ceph mds.l for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:05:54.594 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:05:54 smithi129 podman[64322]: 2024-09-15 22:05:54.495221932 +0000 UTC m=+0.015558733 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:05:54.845 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:05:54 smithi129 podman[64322]: 2024-09-15 22:05:54.620005537 +0000 UTC m=+0.140342336 container create e98fd1bf7eed02365a983f1f9df51606521b26a48733437c55daf7fdee34d6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-15T22:05:55.345 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:05:54 smithi129 podman[64322]: 2024-09-15 22:05:54.866282506 +0000 UTC m=+0.386619311 container init e98fd1bf7eed02365a983f1f9df51606521b26a48733437c55daf7fdee34d6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-l, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-15T22:05:55.345 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:05:54 smithi129 podman[64322]: 2024-09-15 22:05:54.869004729 +0000 UTC m=+0.389341528 container start e98fd1bf7eed02365a983f1f9df51606521b26a48733437c55daf7fdee34d6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:05:55.345 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:05:54 smithi129 ceph-mds[64341]: -- 172.21.15.129:0/1392296304 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5607e3af3860 con 0x5607e3ad1400 2024-09-15T22:05:55.345 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:05:54 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-l[64337]: starting mds.l at 2024-09-15T22:05:55.345 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:05:54 smithi129 bash[64322]: e98fd1bf7eed02365a983f1f9df51606521b26a48733437c55daf7fdee34d6c9 2024-09-15T22:05:55.345 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:05:54 smithi129 systemd[1]: Started Ceph mds.l for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:05:56.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:55 smithi099 ceph-mon[34892]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:56.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:55 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.129:6832/3381054167,v1:172.21.15.129:6833/3381054167] up:boot 2024-09-15T22:05:56.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:55 smithi099 ceph-mon[34892]: fsmap 1 up:standby 2024-09-15T22:05:56.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:05:56.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:56.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:56.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:56.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:05:56.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:05:56.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:55 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:56.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:55 smithi099 ceph-mon[34892]: Deploying daemon mds.k on smithi099 2024-09-15T22:05:56.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:55 smithi129 ceph-mon[35314]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:56.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:55 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.129:6832/3381054167,v1:172.21.15.129:6833/3381054167] up:boot 2024-09-15T22:05:56.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:55 smithi129 ceph-mon[35314]: fsmap 1 up:standby 2024-09-15T22:05:56.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:05:56.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:56.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:56.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:56.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:05:56.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:05:56.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:55 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:56.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:55 smithi129 ceph-mon[35314]: Deploying daemon mds.k on smithi099 2024-09-15T22:05:56.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:55 smithi005 ceph-mon[30944]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:56.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:55 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.129:6832/3381054167,v1:172.21.15.129:6833/3381054167] up:boot 2024-09-15T22:05:56.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:55 smithi005 ceph-mon[30944]: fsmap 1 up:standby 2024-09-15T22:05:56.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:05:56.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:56.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:56.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:56.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:05:56.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:05:56.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:55 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:56.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:55 smithi005 ceph-mon[30944]: Deploying daemon mds.k on smithi099 2024-09-15T22:05:56.829 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:05:57.099 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:05:57.246 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:05:57 smithi099 systemd[1]: Starting Ceph mds.k for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:05:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.572+0000 7f87483e2640 1 -- 172.21.15.5:0/2087638562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8740100a80 msgr2=0x7f8740100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.572+0000 7f87483e2640 1 --2- 172.21.15.5:0/2087638562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8740100a80 0x7f8740100ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8734009930 tx=0x7f873402a420 comp rx=0 tx=0).stop 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.572+0000 7f87483e2640 1 -- 172.21.15.5:0/2087638562 shutdown_connections 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.572+0000 7f87483e2640 1 --2- 172.21.15.5:0/2087638562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8740101420 0x7f874010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.572+0000 7f87483e2640 1 --2- 172.21.15.5:0/2087638562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8740100a80 0x7f8740100ee0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.572+0000 7f87483e2640 1 --2- 172.21.15.5:0/2087638562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8740106a80 0x7f8740106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.572+0000 7f87483e2640 1 -- 172.21.15.5:0/2087638562 >> 172.21.15.5:0/2087638562 conn(0x7f87400fc7a0 msgr2=0x7f87400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.572+0000 7f87483e2640 1 -- 172.21.15.5:0/2087638562 shutdown_connections 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.572+0000 7f87483e2640 1 -- 172.21.15.5:0/2087638562 wait complete. 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f87483e2640 1 Processor -- start 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f87483e2640 1 -- start start 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f87483e2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8740100a80 0x7f87401983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f87483e2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8740101420 0x7f8740198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f87483e2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8740106a80 0x7f874019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f87483e2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8740110e80 con 0x7f8740100a80 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f87483e2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8740110d00 con 0x7f8740101420 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f87483e2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8740111000 con 0x7f8740106a80 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874617e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8740100a80 0x7f87401983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874617e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8740100a80 0x7f87401983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45676/0 (socket says 172.21.15.5:45676) 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874617e640 1 -- 172.21.15.5:0/3721919702 learned_addr learned my addr 172.21.15.5:0/3721919702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874597d640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8740101420 0x7f8740198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874697f640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8740106a80 0x7f874019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874617e640 1 -- 172.21.15.5:0/3721919702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8740106a80 msgr2=0x7f874019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874617e640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8740106a80 0x7f874019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874617e640 1 -- 172.21.15.5:0/3721919702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8740101420 msgr2=0x7f8740198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874617e640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8740101420 0x7f8740198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874617e640 1 -- 172.21.15.5:0/3721919702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f874019d380 con 0x7f8740100a80 2024-09-15T22:05:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874697f640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8740106a80 0x7f874019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.573+0000 7f874617e640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8740100a80 0x7f87401983d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f873000cce0 tx=0x7f873000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.574+0000 7f873f7fe640 1 -- 172.21.15.5:0/3721919702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873000f650 con 0x7f8740100a80 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.574+0000 7f873f7fe640 1 -- 172.21.15.5:0/3721919702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f873000fcd0 con 0x7f8740100a80 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.574+0000 7f87483e2640 1 -- 172.21.15.5:0/3721919702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f874019d670 con 0x7f8740100a80 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.574+0000 7f873f7fe640 1 -- 172.21.15.5:0/3721919702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8730018c70 con 0x7f8740100a80 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.574+0000 7f87483e2640 1 -- 172.21.15.5:0/3721919702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87401a4f50 con 0x7f8740100a80 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.575+0000 7f87483e2640 1 -- 172.21.15.5:0/3721919702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8740100ee0 con 0x7f8740100a80 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.576+0000 7f873f7fe640 1 -- 172.21.15.5:0/3721919702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f873000f7f0 con 0x7f8740100a80 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.577+0000 7f873f7fe640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f87280a32c0 0x7f87280a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.577+0000 7f874597d640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f87280a32c0 0x7f87280a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.577+0000 7f873f7fe640 1 -- 172.21.15.5:0/3721919702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f87300cf8c0 con 0x7f8740100a80 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.577+0000 7f874597d640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f87280a32c0 0x7f87280a5780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8734009900 tx=0x7f87340058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:05:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.581+0000 7f873f7fe640 1 -- 172.21.15.5:0/3721919702 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8730014030 con 0x7f8740100a80 2024-09-15T22:05:57.589 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:05:57 smithi099 podman[64058]: 2024-09-15 22:05:57.148062181 +0000 UTC m=+0.016059531 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:05:57.589 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:05:57 smithi099 podman[64058]: 2024-09-15 22:05:57.276504015 +0000 UTC m=+0.144501406 container create 7f7f5a520440cd1c6c78bc890cd4266fe4c4358b094f80a6322623cab9982e7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-15T22:05:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.797+0000 7f87483e2640 1 -- 172.21.15.5:0/3721919702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8740199730 con 0x7f8740100a80 2024-09-15T22:05:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.798+0000 7f873f7fe640 1 -- 172.21.15.5:0/3721919702 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f873009c9e0 con 0x7f8740100a80 2024-09-15T22:05:57.799 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:05:57.800 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"btime":"2024-09-15T22:05:54:897137+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-15T22:05:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-09-15T22:05:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 -- 172.21.15.5:0/3721919702 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f87280a32c0 msgr2=0x7f87280a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f87280a32c0 0x7f87280a5780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8734009900 tx=0x7f87340058d0 comp rx=0 tx=0).stop 2024-09-15T22:05:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 -- 172.21.15.5:0/3721919702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8740100a80 msgr2=0x7f87401983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:05:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8740100a80 0x7f87401983d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f873000cce0 tx=0x7f873000a620 comp rx=0 tx=0).stop 2024-09-15T22:05:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 -- 172.21.15.5:0/3721919702 shutdown_connections 2024-09-15T22:05:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8740106a80 0x7f874019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8740101420 0x7f8740198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f87280a32c0 0x7f87280a5780 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 --2- 172.21.15.5:0/3721919702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8740100a80 0x7f87401983d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:05:57.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 -- 172.21.15.5:0/3721919702 >> 172.21.15.5:0/3721919702 conn(0x7f87400fc7a0 msgr2=0x7f874006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:05:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 -- 172.21.15.5:0/3721919702 shutdown_connections 2024-09-15T22:05:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:05:57.800+0000 7f87483e2640 1 -- 172.21.15.5:0/3721919702 wait complete. 2024-09-15T22:05:57.903 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:05:57 smithi099 podman[64058]: 2024-09-15 22:05:57.621560705 +0000 UTC m=+0.489558058 container init 7f7f5a520440cd1c6c78bc890cd4266fe4c4358b094f80a6322623cab9982e7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-k, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:05:57.903 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:05:57 smithi099 podman[64058]: 2024-09-15 22:05:57.624702503 +0000 UTC m=+0.492699862 container start 7f7f5a520440cd1c6c78bc890cd4266fe4c4358b094f80a6322623cab9982e7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-k, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-15T22:05:57.903 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:05:57 smithi099 ceph-mds[64077]: -- 172.21.15.99:0/2212464305 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5609749a7860 con 0x560974985000 2024-09-15T22:05:57.903 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:05:57 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-k[64073]: starting mds.k at 2024-09-15T22:05:57.904 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:05:57 smithi099 bash[64058]: 7f7f5a520440cd1c6c78bc890cd4266fe4c4358b094f80a6322623cab9982e7b 2024-09-15T22:05:57.904 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:05:57 smithi099 systemd[1]: Started Ceph mds.k for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:05:58.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:57 smithi005 ceph-mon[30944]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:58.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:57 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:58.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:57 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:58.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:57 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:58.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:57 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:05:58.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:57 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:05:58.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:57 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:58.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:57 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3721919702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:05:58.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:57 smithi099 ceph-mon[34892]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:57 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:57 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:57 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:57 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:05:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:57 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:05:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:57 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:57 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3721919702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:05:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:57 smithi129 ceph-mon[35314]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:05:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:57 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:57 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:57 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:05:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:57 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:05:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:57 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:05:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:57 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:05:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:57 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3721919702' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:05:59.083 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:58 smithi005 ceph-mon[30944]: Deploying daemon mds.j on smithi005 2024-09-15T22:05:59.083 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:58 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.99:6832/1006057528,v1:172.21.15.99:6833/1006057528] up:boot 2024-09-15T22:05:59.083 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:58 smithi005 ceph-mon[30944]: fsmap 2 up:standby 2024-09-15T22:05:59.083 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:58 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:05:59.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:58 smithi099 ceph-mon[34892]: Deploying daemon mds.j on smithi005 2024-09-15T22:05:59.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:58 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.99:6832/1006057528,v1:172.21.15.99:6833/1006057528] up:boot 2024-09-15T22:05:59.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:58 smithi099 ceph-mon[34892]: fsmap 2 up:standby 2024-09-15T22:05:59.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:58 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:05:59.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:58 smithi129 ceph-mon[35314]: Deploying daemon mds.j on smithi005 2024-09-15T22:05:59.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:58 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.99:6832/1006057528,v1:172.21.15.99:6833/1006057528] up:boot 2024-09-15T22:05:59.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:58 smithi129 ceph-mon[35314]: fsmap 2 up:standby 2024-09-15T22:05:59.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:58 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:06:00.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:05:59 smithi099 ceph-mon[34892]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:00.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:05:59 smithi005 ceph-mon[30944]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:00.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:05:59 smithi129 ceph-mon[35314]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:00.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:06:00.558 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:06:00 smithi005 systemd[1]: Starting Ceph mds.j for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:06:00.830 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:06:00 smithi005 podman[69810]: 2024-09-15 22:06:00.460309863 +0000 UTC m=+0.016510909 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:06:00.830 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:06:00 smithi005 podman[69810]: 2024-09-15 22:06:00.572005208 +0000 UTC m=+0.128206242 container create f40634a25b83278242096d6568d96cc0f125bfa28e7c3cde39d6a72d5cea7f27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-j, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-15T22:06:00.830 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:06:00 smithi005 podman[69810]: 2024-09-15 22:06:00.800103544 +0000 UTC m=+0.356304589 container init f40634a25b83278242096d6568d96cc0f125bfa28e7c3cde39d6a72d5cea7f27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:06:00.830 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:06:00 smithi005 podman[69810]: 2024-09-15 22:06:00.804054992 +0000 UTC m=+0.360256028 container start f40634a25b83278242096d6568d96cc0f125bfa28e7c3cde39d6a72d5cea7f27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, ceph=True) 2024-09-15T22:06:00.830 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:06:00 smithi005 ceph-mds[69860]: -- 172.21.15.5:0/1153364319 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a8d093860 con 0x558a8d070800 2024-09-15T22:06:00.995 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:01.106 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:06:00 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-j[69843]: starting mds.j at 2024-09-15T22:06:01.107 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:06:00 smithi005 bash[69810]: f40634a25b83278242096d6568d96cc0f125bfa28e7c3cde39d6a72d5cea7f27 2024-09-15T22:06:01.107 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:06:00 smithi005 systemd[1]: Started Ceph mds.j for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:06:01.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.513+0000 7f58a36cd640 1 -- 172.21.15.5:0/3795758761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c101440 msgr2=0x7f589c10df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:01.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.513+0000 7f58a36cd640 1 --2- 172.21.15.5:0/3795758761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c101440 0x7f589c10df30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5890009900 tx=0x7f589002a410 comp rx=0 tx=0).stop 2024-09-15T22:06:01.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.513+0000 7f58a36cd640 1 -- 172.21.15.5:0/3795758761 shutdown_connections 2024-09-15T22:06:01.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.513+0000 7f58a36cd640 1 --2- 172.21.15.5:0/3795758761 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f589c10e500 0x7f589c110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.513+0000 7f58a36cd640 1 --2- 172.21.15.5:0/3795758761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c101440 0x7f589c10df30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.513+0000 7f58a36cd640 1 --2- 172.21.15.5:0/3795758761 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f589c100b20 0x7f589c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.513+0000 7f58a36cd640 1 -- 172.21.15.5:0/3795758761 >> 172.21.15.5:0/3795758761 conn(0x7f589c0fc7d0 msgr2=0x7f589c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:01.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.513+0000 7f58a36cd640 1 -- 172.21.15.5:0/3795758761 shutdown_connections 2024-09-15T22:06:01.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.513+0000 7f58a36cd640 1 -- 172.21.15.5:0/3795758761 wait complete. 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a36cd640 1 Processor -- start 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a36cd640 1 -- start start 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a36cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c100b20 0x7f589c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a36cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f589c101440 0x7f589c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a36cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f589c10e500 0x7f589c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a36cd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f589c114270 con 0x7f589c100b20 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a36cd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f589c1140f0 con 0x7f589c101440 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a36cd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f589c1143f0 con 0x7f589c10e500 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a1469640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c100b20 0x7f589c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a1469640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c100b20 0x7f589c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45704/0 (socket says 172.21.15.5:45704) 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a1469640 1 -- 172.21.15.5:0/2820095191 learned_addr learned my addr 172.21.15.5:0/2820095191 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:01.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a0c68640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f589c101440 0x7f589c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:01.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.514+0000 7f58a1c6a640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f589c10e500 0x7f589c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:01.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f58a1469640 1 -- 172.21.15.5:0/2820095191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f589c10e500 msgr2=0x7f589c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:01.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f58a1469640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f589c10e500 0x7f589c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f58a1469640 1 -- 172.21.15.5:0/2820095191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f589c101440 msgr2=0x7f589c19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:01.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f58a1469640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f589c101440 0x7f589c19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f58a1469640 1 -- 172.21.15.5:0/2820095191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589c19f700 con 0x7f589c100b20 2024-09-15T22:06:01.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f58a1c6a640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f589c10e500 0x7f589c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:01.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f58a0c68640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f589c101440 0x7f589c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:06:01.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f58a1469640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c100b20 0x7f589c19a750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f588c00cce0 tx=0x7f588c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:01.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f589a7fc640 1 -- 172.21.15.5:0/2820095191 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c004b20 con 0x7f589c100b20 2024-09-15T22:06:01.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f589a7fc640 1 -- 172.21.15.5:0/2820095191 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f588c004cc0 con 0x7f589c100b20 2024-09-15T22:06:01.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f58a36cd640 1 -- 172.21.15.5:0/2820095191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589c19f9f0 con 0x7f589c100b20 2024-09-15T22:06:01.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f589a7fc640 1 -- 172.21.15.5:0/2820095191 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c00f7d0 con 0x7f589c100b20 2024-09-15T22:06:01.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.515+0000 7f58a36cd640 1 -- 172.21.15.5:0/2820095191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589c1a72d0 con 0x7f589c100b20 2024-09-15T22:06:01.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.516+0000 7f58a36cd640 1 -- 172.21.15.5:0/2820095191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f589c100f00 con 0x7f589c100b20 2024-09-15T22:06:01.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.517+0000 7f589a7fc640 1 -- 172.21.15.5:0/2820095191 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f588c00f970 con 0x7f589c100b20 2024-09-15T22:06:01.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.518+0000 7f589a7fc640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f58840a32c0 0x7f58840a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:01.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.518+0000 7f58a0c68640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f58840a32c0 0x7f58840a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:01.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.518+0000 7f589a7fc640 1 -- 172.21.15.5:0/2820095191 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f588c0cf930 con 0x7f589c100b20 2024-09-15T22:06:01.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.519+0000 7f58a0c68640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f58840a32c0 0x7f58840a5780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f5890005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:01.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.521+0000 7f589a7fc640 1 -- 172.21.15.5:0/2820095191 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f588c014030 con 0x7f589c100b20 2024-09-15T22:06:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.759+0000 7f58a36cd640 1 -- 172.21.15.5:0/2820095191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f589c19bab0 con 0x7f589c100b20 2024-09-15T22:06:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.759+0000 7f589a7fc640 1 -- 172.21.15.5:0/2820095191 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3079 (secure 0 0 0) 0x7f588c09cb70 con 0x7f589c100b20 2024-09-15T22:06:01.761 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:01.761 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"btime":"2024-09-15T22:06:00:917294+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-15T22:06:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2024-09-15T22:06:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.761+0000 7f58a36cd640 1 -- 172.21.15.5:0/2820095191 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f58840a32c0 msgr2=0x7f58840a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.761+0000 7f58a36cd640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f58840a32c0 0x7f58840a5780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f5890005a20 comp rx=0 tx=0).stop 2024-09-15T22:06:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.761+0000 7f58a36cd640 1 -- 172.21.15.5:0/2820095191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c100b20 msgr2=0x7f589c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.761+0000 7f58a36cd640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c100b20 0x7f589c19a750 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f588c00cce0 tx=0x7f588c00a620 comp rx=0 tx=0).stop 2024-09-15T22:06:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.761+0000 7f58a36cd640 1 -- 172.21.15.5:0/2820095191 shutdown_connections 2024-09-15T22:06:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.761+0000 7f58a36cd640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f589c10e500 0x7f589c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.761+0000 7f58a36cd640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f589c101440 0x7f589c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.761+0000 7f58a36cd640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f58840a32c0 0x7f58840a5780 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.761+0000 7f58a36cd640 1 --2- 172.21.15.5:0/2820095191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f589c100b20 0x7f589c19a750 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.761+0000 7f58a36cd640 1 -- 172.21.15.5:0/2820095191 >> 172.21.15.5:0/2820095191 conn(0x7f589c0fc7d0 msgr2=0x7f589c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.762+0000 7f58a36cd640 1 -- 172.21.15.5:0/2820095191 shutdown_connections 2024-09-15T22:06:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:01.762+0000 7f58a36cd640 1 -- 172.21.15.5:0/2820095191 wait complete. 2024-09-15T22:06:02.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:02.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.5:6834/93159772,v1:172.21.15.5:6835/93159772] up:boot 2024-09-15T22:06:02.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: fsmap 3 up:standby 2024-09-15T22:06:02.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:06:02.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:02.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:02.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:02.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:02.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:02.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:02.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: Deploying daemon mds.d on smithi005 2024-09-15T22:06:02.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:01 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2820095191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.5:6834/93159772,v1:172.21.15.5:6835/93159772] up:boot 2024-09-15T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: fsmap 3 up:standby 2024-09-15T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: Deploying daemon mds.d on smithi005 2024-09-15T22:06:02.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:01 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2820095191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.5:6834/93159772,v1:172.21.15.5:6835/93159772] up:boot 2024-09-15T22:06:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: fsmap 3 up:standby 2024-09-15T22:06:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:06:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:02.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:02.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:02.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:02.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:02.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: Deploying daemon mds.d on smithi005 2024-09-15T22:06:02.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:01 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2820095191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:03.606 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:06:03 smithi005 systemd[1]: Starting Ceph mds.d for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:06:03.927 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:06:03 smithi005 podman[70220]: 2024-09-15 22:06:03.76807416 +0000 UTC m=+0.016639531 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:06:03.927 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:06:03 smithi005 podman[70220]: 2024-09-15 22:06:03.869863886 +0000 UTC m=+0.118429240 container create ab9a334f36f0c3fc13d7c8515d28800d1327a3f6327bd05fc5387b9836e58535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-15T22:06:04.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:03 smithi005 ceph-mon[30944]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:04.224 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:06:04 smithi005 podman[70220]: 2024-09-15 22:06:04.130977119 +0000 UTC m=+0.379542483 container init ab9a334f36f0c3fc13d7c8515d28800d1327a3f6327bd05fc5387b9836e58535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e) 2024-09-15T22:06:04.224 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:06:04 smithi005 podman[70220]: 2024-09-15 22:06:04.133844384 +0000 UTC m=+0.382409739 container start ab9a334f36f0c3fc13d7c8515d28800d1327a3f6327bd05fc5387b9836e58535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-15T22:06:04.224 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:06:04 smithi005 ceph-mds[70239]: -- 172.21.15.5:0/1164422863 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56032904d860 con 0x56032902b000 2024-09-15T22:06:04.224 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:06:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-d[70235]: starting mds.d at 2024-09-15T22:06:04.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:03 smithi099 ceph-mon[34892]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:03 smithi129 ceph-mon[35314]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:04.606 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:06:04 smithi005 bash[70220]: ab9a334f36f0c3fc13d7c8515d28800d1327a3f6327bd05fc5387b9836e58535 2024-09-15T22:06:04.607 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:06:04 smithi005 systemd[1]: Started Ceph mds.d for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:06:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:05 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:05 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:05 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:05 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:05 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:05 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:05 smithi099 ceph-mon[34892]: Deploying daemon mds.e on smithi099 2024-09-15T22:06:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:05 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.5:6836/2795083273,v1:172.21.15.5:6837/2795083273] up:boot 2024-09-15T22:06:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:05 smithi099 ceph-mon[34892]: fsmap 4 up:standby 2024-09-15T22:06:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:05 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:06:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:05 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:05 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:05 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:05 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:05.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:05 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:05.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:05 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:05.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:05 smithi129 ceph-mon[35314]: Deploying daemon mds.e on smithi099 2024-09-15T22:06:05.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:05 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.5:6836/2795083273,v1:172.21.15.5:6837/2795083273] up:boot 2024-09-15T22:06:05.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:05 smithi129 ceph-mon[35314]: fsmap 4 up:standby 2024-09-15T22:06:05.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:05 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:06:05.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:05.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:05.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:05.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:05.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:05.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:05.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:05 smithi005 ceph-mon[30944]: Deploying daemon mds.e on smithi099 2024-09-15T22:06:05.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:05 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.5:6836/2795083273,v1:172.21.15.5:6837/2795083273] up:boot 2024-09-15T22:06:05.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:05 smithi005 ceph-mon[30944]: fsmap 4 up:standby 2024-09-15T22:06:05.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:05 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:06:05.683 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:06:05.958 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:06.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:06 smithi099 ceph-mon[34892]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:06.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:06 smithi005 ceph-mon[30944]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:06.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.463+0000 7faf8af3d640 1 -- 172.21.15.5:0/2804858523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf84102ef0 msgr2=0x7faf8410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.463+0000 7faf8af3d640 1 --2- 172.21.15.5:0/2804858523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf84102ef0 0x7faf8410da40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faf7c009ca0 tx=0x7faf7c02a950 comp rx=0 tx=0).stop 2024-09-15T22:06:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.463+0000 7faf8af3d640 1 -- 172.21.15.5:0/2804858523 shutdown_connections 2024-09-15T22:06:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.463+0000 7faf8af3d640 1 --2- 172.21.15.5:0/2804858523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf84102ef0 0x7faf8410da40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.463+0000 7faf8af3d640 1 --2- 172.21.15.5:0/2804858523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf84102570 0x7faf841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.463+0000 7faf8af3d640 1 --2- 172.21.15.5:0/2804858523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf84100e20 0x7faf84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.463+0000 7faf8af3d640 1 -- 172.21.15.5:0/2804858523 >> 172.21.15.5:0/2804858523 conn(0x7faf840fc7d0 msgr2=0x7faf840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.463+0000 7faf8af3d640 1 -- 172.21.15.5:0/2804858523 shutdown_connections 2024-09-15T22:06:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.463+0000 7faf8af3d640 1 -- 172.21.15.5:0/2804858523 wait complete. 2024-09-15T22:06:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf8af3d640 1 Processor -- start 2024-09-15T22:06:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf8af3d640 1 -- start start 2024-09-15T22:06:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf8af3d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf84100e20 0x7faf84199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:06.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf8af3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf84102570 0x7faf84199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:06.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf8af3d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf84102ef0 0x7faf841a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:06.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf8af3d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf84110fd0 con 0x7faf84102570 2024-09-15T22:06:06.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf8af3d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf84110e50 con 0x7faf84102ef0 2024-09-15T22:06:06.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf8af3d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf84111150 con 0x7faf84100e20 2024-09-15T22:06:06.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf84102570 0x7faf84199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:06.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf84102570 0x7faf84199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43414/0 (socket says 172.21.15.5:43414) 2024-09-15T22:06:06.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf7bfff640 1 -- 172.21.15.5:0/3160428872 learned_addr learned my addr 172.21.15.5:0/3160428872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:06.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf894da640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf84102ef0 0x7faf841a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:06.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf88cd9640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf84100e20 0x7faf84199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:06.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf7bfff640 1 -- 172.21.15.5:0/3160428872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf84100e20 msgr2=0x7faf84199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:06.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf7bfff640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf84100e20 0x7faf84199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:06.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf7bfff640 1 -- 172.21.15.5:0/3160428872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf84102ef0 msgr2=0x7faf841a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:06.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf7bfff640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf84102ef0 0x7faf841a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:06.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf7bfff640 1 -- 172.21.15.5:0/3160428872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf841a4900 con 0x7faf84102570 2024-09-15T22:06:06.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf88cd9640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf84100e20 0x7faf84199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:06.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.464+0000 7faf7bfff640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf84102570 0x7faf84199880 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf7400e9f0 tx=0x7faf7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:06.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.465+0000 7faf79ffb640 1 -- 172.21.15.5:0/3160428872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf7400cd80 con 0x7faf84102570 2024-09-15T22:06:06.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.465+0000 7faf79ffb640 1 -- 172.21.15.5:0/3160428872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf74010910 con 0x7faf84102570 2024-09-15T22:06:06.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.465+0000 7faf8af3d640 1 -- 172.21.15.5:0/3160428872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf841a4ad0 con 0x7faf84102570 2024-09-15T22:06:06.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.465+0000 7faf79ffb640 1 -- 172.21.15.5:0/3160428872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf74018990 con 0x7faf84102570 2024-09-15T22:06:06.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.465+0000 7faf8af3d640 1 -- 172.21.15.5:0/3160428872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf841a4fe0 con 0x7faf84102570 2024-09-15T22:06:06.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.466+0000 7faf8af3d640 1 -- 172.21.15.5:0/3160428872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf84101200 con 0x7faf84102570 2024-09-15T22:06:06.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.467+0000 7faf79ffb640 1 -- 172.21.15.5:0/3160428872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7faf74010ac0 con 0x7faf84102570 2024-09-15T22:06:06.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.468+0000 7faf79ffb640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf6c0a3240 0x7faf6c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:06.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.468+0000 7faf88cd9640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf6c0a3240 0x7faf6c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:06.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.468+0000 7faf79ffb640 1 -- 172.21.15.5:0/3160428872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7faf740cf850 con 0x7faf84102570 2024-09-15T22:06:06.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.469+0000 7faf88cd9640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf6c0a3240 0x7faf6c0a5700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faf800058c0 tx=0x7faf80005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:06.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.473+0000 7faf79ffb640 1 -- 172.21.15.5:0/3160428872 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf7409c970 con 0x7faf84102570 2024-09-15T22:06:06.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:06 smithi129 ceph-mon[35314]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.688+0000 7faf8af3d640 1 -- 172.21.15.5:0/3160428872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf841a5210 con 0x7faf84102570 2024-09-15T22:06:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.689+0000 7faf79ffb640 1 -- 172.21.15.5:0/3160428872 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3901 (secure 0 0 0) 0x7faf7409c270 con 0x7faf84102570 2024-09-15T22:06:06.694 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:06.694 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"btime":"2024-09-15T22:06:04:926437+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-15T22:06:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 5 2024-09-15T22:06:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 -- 172.21.15.5:0/3160428872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf6c0a3240 msgr2=0x7faf6c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf6c0a3240 0x7faf6c0a5700 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7faf800058c0 tx=0x7faf80005d30 comp rx=0 tx=0).stop 2024-09-15T22:06:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 -- 172.21.15.5:0/3160428872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf84102570 msgr2=0x7faf84199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf84102570 0x7faf84199880 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7faf7400e9f0 tx=0x7faf7400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:06:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 -- 172.21.15.5:0/3160428872 shutdown_connections 2024-09-15T22:06:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf84102ef0 0x7faf841a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf6c0a3240 0x7faf6c0a5700 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf84102570 0x7faf84199880 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 --2- 172.21.15.5:0/3160428872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf84100e20 0x7faf84199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 -- 172.21.15.5:0/3160428872 >> 172.21.15.5:0/3160428872 conn(0x7faf840fc7d0 msgr2=0x7faf84106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 -- 172.21.15.5:0/3160428872 shutdown_connections 2024-09-15T22:06:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:06.691+0000 7faf8af3d640 1 -- 172.21.15.5:0/3160428872 wait complete. 2024-09-15T22:06:06.730 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:06:06 smithi099 podman[64359]: 2024-09-15 22:06:06.70598742 +0000 UTC m=+0.360482459 container start 3c51429a29f34d2fb032cbfc67a4c0580aa5073dacfdc19bdcd78c6ff1f886b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-15T22:06:06.730 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:06:06 smithi099 ceph-mds[64378]: -- 172.21.15.99:0/3115937550 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55efb24d1860 con 0x55efb24af400 2024-09-15T22:06:07.089 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:06:06 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-e[64374]: starting mds.e at 2024-09-15T22:06:07.090 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:06:06 smithi099 bash[64359]: 3c51429a29f34d2fb032cbfc67a4c0580aa5073dacfdc19bdcd78c6ff1f886b8 2024-09-15T22:06:07.090 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:06:06 smithi099 systemd[1]: Started Ceph mds.e for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:06:07.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:07 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3160428872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:07.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:07.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:07.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:07.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:07.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:07.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:07 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:07.550 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:07 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3160428872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:07.550 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:07.550 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:07.550 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:07.550 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:07.550 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:07.550 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:07 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:07 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3160428872' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:07 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:08.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:08 smithi099 ceph-mon[34892]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:08.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:08 smithi099 ceph-mon[34892]: Deploying daemon mds.f on smithi129 2024-09-15T22:06:08.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:08 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.99:6834/157351762,v1:172.21.15.99:6835/157351762] up:boot 2024-09-15T22:06:08.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:08 smithi099 ceph-mon[34892]: fsmap 5 up:standby 2024-09-15T22:06:08.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:08 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:06:08.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:08 smithi129 ceph-mon[35314]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:08.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:08 smithi129 ceph-mon[35314]: Deploying daemon mds.f on smithi129 2024-09-15T22:06:08.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:08 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.99:6834/157351762,v1:172.21.15.99:6835/157351762] up:boot 2024-09-15T22:06:08.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:08 smithi129 ceph-mon[35314]: fsmap 5 up:standby 2024-09-15T22:06:08.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:08 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:06:08.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:08 smithi005 ceph-mon[30944]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:08.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:08 smithi005 ceph-mon[30944]: Deploying daemon mds.f on smithi129 2024-09-15T22:06:08.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:08 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.99:6834/157351762,v1:172.21.15.99:6835/157351762] up:boot 2024-09-15T22:06:08.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:08 smithi005 ceph-mon[30944]: fsmap 5 up:standby 2024-09-15T22:06:08.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:08 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:06:09.037 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:06:08 smithi129 systemd[1]: Starting Ceph mds.f for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:06:09.287 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:06:09 smithi129 podman[64623]: 2024-09-15 22:06:08.938223997 +0000 UTC m=+0.015631807 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:06:09.287 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:06:09 smithi129 podman[64623]: 2024-09-15 22:06:09.055838935 +0000 UTC m=+0.133246738 container create 44bb71243fa874eef6486747a9814378487b2d933f48cb4f2ad82d0e2a7b36a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-15T22:06:09.287 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:06:09 smithi129 podman[64623]: 2024-09-15 22:06:09.283723431 +0000 UTC m=+0.361131231 container init 44bb71243fa874eef6486747a9814378487b2d933f48cb4f2ad82d0e2a7b36a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:06:09.288 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:06:09 smithi129 podman[64623]: 2024-09-15 22:06:09.286596897 +0000 UTC m=+0.364004697 container start 44bb71243fa874eef6486747a9814378487b2d933f48cb4f2ad82d0e2a7b36a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-15T22:06:09.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:06:09.595 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:06:09 smithi129 ceph-mds[64642]: -- 172.21.15.129:0/2443961867 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561406413860 con 0x5614063f0800 2024-09-15T22:06:09.595 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:06:09 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f[64638]: starting mds.f at 2024-09-15T22:06:09.595 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:06:09 smithi129 bash[64623]: 44bb71243fa874eef6486747a9814378487b2d933f48cb4f2ad82d0e2a7b36a2 2024-09-15T22:06:09.595 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:06:09 smithi129 systemd[1]: Started Ceph mds.f for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:06:09.711 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:10.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.243+0000 7fe9b3d08640 1 -- 172.21.15.5:0/1571703906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac102570 msgr2=0x7fe9ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:10.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.243+0000 7fe9b3d08640 1 --2- 172.21.15.5:0/1571703906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac102570 0x7fe9ac1029b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0009ca0 tx=0x7fe9a002a970 comp rx=0 tx=0).stop 2024-09-15T22:06:10.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.243+0000 7fe9b3d08640 1 -- 172.21.15.5:0/1571703906 shutdown_connections 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.243+0000 7fe9b3d08640 1 --2- 172.21.15.5:0/1571703906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac102ef0 0x7fe9ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.243+0000 7fe9b3d08640 1 --2- 172.21.15.5:0/1571703906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac102570 0x7fe9ac1029b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.243+0000 7fe9b3d08640 1 --2- 172.21.15.5:0/1571703906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9ac100e20 0x7fe9ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.243+0000 7fe9b3d08640 1 -- 172.21.15.5:0/1571703906 >> 172.21.15.5:0/1571703906 conn(0x7fe9ac0fc7d0 msgr2=0x7fe9ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.243+0000 7fe9b3d08640 1 -- 172.21.15.5:0/1571703906 shutdown_connections 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.243+0000 7fe9b3d08640 1 -- 172.21.15.5:0/1571703906 wait complete. 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b3d08640 1 Processor -- start 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b3d08640 1 -- start start 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b3d08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac100e20 0x7fe9ac198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b3d08640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9ac102570 0x7fe9ac198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b3d08640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac102ef0 0x7fe9ac19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b3d08640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac110fc0 con 0x7fe9ac100e20 2024-09-15T22:06:10.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b3d08640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac110e40 con 0x7fe9ac102570 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b3d08640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac111140 con 0x7fe9ac102ef0 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b1aa4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac100e20 0x7fe9ac198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b1aa4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac100e20 0x7fe9ac198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43430/0 (socket says 172.21.15.5:43430) 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b1aa4640 1 -- 172.21.15.5:0/3414783800 learned_addr learned my addr 172.21.15.5:0/3414783800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b12a3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9ac102570 0x7fe9ac198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.244+0000 7fe9b22a5640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac102ef0 0x7fe9ac19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9b1aa4640 1 -- 172.21.15.5:0/3414783800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac102ef0 msgr2=0x7fe9ac19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9b1aa4640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac102ef0 0x7fe9ac19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9b1aa4640 1 -- 172.21.15.5:0/3414783800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9ac102570 msgr2=0x7fe9ac198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9b1aa4640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9ac102570 0x7fe9ac198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9b1aa4640 1 -- 172.21.15.5:0/3414783800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac19d5e0 con 0x7fe9ac100e20 2024-09-15T22:06:10.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9b12a3640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9ac102570 0x7fe9ac198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9b1aa4640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac100e20 0x7fe9ac198630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00e9f0 tx=0x7fe99c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9aaffd640 1 -- 172.21.15.5:0/3414783800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c00cd80 con 0x7fe9ac100e20 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9b3d08640 1 -- 172.21.15.5:0/3414783800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac19d8d0 con 0x7fe9ac100e20 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9aaffd640 1 -- 172.21.15.5:0/3414783800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe99c010910 con 0x7fe9ac100e20 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9aaffd640 1 -- 172.21.15.5:0/3414783800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c018990 con 0x7fe9ac100e20 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.245+0000 7fe9b3d08640 1 -- 172.21.15.5:0/3414783800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac1a51b0 con 0x7fe9ac100e20 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.246+0000 7fe9b3d08640 1 -- 172.21.15.5:0/3414783800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac101200 con 0x7fe9ac100e20 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.248+0000 7fe9aaffd640 1 -- 172.21.15.5:0/3414783800 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fe99c010ac0 con 0x7fe9ac100e20 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.249+0000 7fe9aaffd640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe9940a3240 0x7fe9940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.249+0000 7fe9b12a3640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe9940a3240 0x7fe9940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.249+0000 7fe9aaffd640 1 -- 172.21.15.5:0/3414783800 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe99c0cf8d0 con 0x7fe9ac100e20 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.249+0000 7fe9b12a3640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe9940a3240 0x7fe9940a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac100030 tx=0x7fe9a003c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:10.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.254+0000 7fe9aaffd640 1 -- 172.21.15.5:0/3414783800 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe99c09c9f0 con 0x7fe9ac100e20 2024-09-15T22:06:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.466+0000 7fe9b3d08640 1 -- 172.21.15.5:0/3414783800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9ac10a0a0 con 0x7fe9ac100e20 2024-09-15T22:06:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.467+0000 7fe9aaffd640 1 -- 172.21.15.5:0/3414783800 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5551 (secure 0 0 0) 0x7fe99c09c2f0 con 0x7fe9ac100e20 2024-09-15T22:06:10.469 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:10.469 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"btime":"2024-09-15T22:06:10:298795+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-15T22:06:10.469 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2024-09-15T22:06:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.470+0000 7fe9b3d08640 1 -- 172.21.15.5:0/3414783800 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe9940a3240 msgr2=0x7fe9940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.470+0000 7fe9b3d08640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe9940a3240 0x7fe9940a5700 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac100030 tx=0x7fe9a003c040 comp rx=0 tx=0).stop 2024-09-15T22:06:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.470+0000 7fe9b3d08640 1 -- 172.21.15.5:0/3414783800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac100e20 msgr2=0x7fe9ac198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.470+0000 7fe9b3d08640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac100e20 0x7fe9ac198630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00e9f0 tx=0x7fe99c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:06:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.471+0000 7fe9b3d08640 1 -- 172.21.15.5:0/3414783800 shutdown_connections 2024-09-15T22:06:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.471+0000 7fe9b3d08640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe9ac102ef0 0x7fe9ac19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.471+0000 7fe9b3d08640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe9ac102570 0x7fe9ac198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.471+0000 7fe9b3d08640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe9940a3240 0x7fe9940a5700 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.471+0000 7fe9b3d08640 1 --2- 172.21.15.5:0/3414783800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9ac100e20 0x7fe9ac198630 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.471+0000 7fe9b3d08640 1 -- 172.21.15.5:0/3414783800 >> 172.21.15.5:0/3414783800 conn(0x7fe9ac0fc7d0 msgr2=0x7fe9ac06a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.471+0000 7fe9b3d08640 1 -- 172.21.15.5:0/3414783800 shutdown_connections 2024-09-15T22:06:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:10.471+0000 7fe9b3d08640 1 -- 172.21.15.5:0/3414783800 wait complete. 2024-09-15T22:06:10.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:10 smithi129 ceph-mon[35314]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:10.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:10 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:10.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:10 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:10.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:10 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:10.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:10 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:10.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:10 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:10.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:10 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:10.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:10 smithi129 ceph-mon[35314]: Deploying daemon mds.c on smithi129 2024-09-15T22:06:10.567 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:10 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.129:6834/3511675647,v1:172.21.15.129:6835/3511675647] up:boot 2024-09-15T22:06:10.568 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:10 smithi129 ceph-mon[35314]: fsmap 6 up:standby 2024-09-15T22:06:10.568 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:10 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:06:10.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:10 smithi099 ceph-mon[34892]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:10 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:10 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:10 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:10 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:10 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:10 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:10 smithi099 ceph-mon[34892]: Deploying daemon mds.c on smithi129 2024-09-15T22:06:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:10 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.129:6834/3511675647,v1:172.21.15.129:6835/3511675647] up:boot 2024-09-15T22:06:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:10 smithi099 ceph-mon[34892]: fsmap 6 up:standby 2024-09-15T22:06:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:10 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:06:10.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:10 smithi005 ceph-mon[30944]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:10.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:10 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:10.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:10 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:10.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:10 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:10.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:10 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:10.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:10 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:10.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:10 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:10.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:10 smithi005 ceph-mon[30944]: Deploying daemon mds.c on smithi129 2024-09-15T22:06:10.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:10 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.129:6834/3511675647,v1:172.21.15.129:6835/3511675647] up:boot 2024-09-15T22:06:10.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:10 smithi005 ceph-mon[30944]: fsmap 6 up:standby 2024-09-15T22:06:10.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:10 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:06:11.440 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:06:11 smithi129 systemd[1]: Starting Ceph mds.c for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:06:11.714 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:11 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3414783800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:11.714 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:06:11 smithi129 podman[64926]: 2024-09-15 22:06:11.469550894 +0000 UTC m=+0.015760105 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:06:11.714 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:06:11 smithi129 podman[64926]: 2024-09-15 22:06:11.594084706 +0000 UTC m=+0.140293909 container create d13e9bfd1eefb51f920fc826c063fea7885232658bfb857bd7a2e3200ec15855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-15T22:06:11.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:11 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3414783800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:11.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:11 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3414783800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:12.095 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:06:11 smithi129 podman[64926]: 2024-09-15 22:06:11.838425963 +0000 UTC m=+0.384635166 container init d13e9bfd1eefb51f920fc826c063fea7885232658bfb857bd7a2e3200ec15855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-15T22:06:12.095 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:06:11 smithi129 podman[64926]: 2024-09-15 22:06:11.841183609 +0000 UTC m=+0.387392812 container start d13e9bfd1eefb51f920fc826c063fea7885232658bfb857bd7a2e3200ec15855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:06:12.095 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:06:11 smithi129 ceph-mds[64945]: -- 172.21.15.129:0/1459966631 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5611edda3860 con 0x5611edd80800 2024-09-15T22:06:12.095 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:06:11 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c[64941]: starting mds.c at 2024-09-15T22:06:12.095 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:06:11 smithi129 bash[64926]: d13e9bfd1eefb51f920fc826c063fea7885232658bfb857bd7a2e3200ec15855 2024-09-15T22:06:12.095 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:06:11 smithi129 systemd[1]: Started Ceph mds.c for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:06:12.749 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:12 smithi099 ceph-mon[34892]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:12.749 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:12 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:12.749 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:12 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:12.750 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:12 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:12.750 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:12 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:12.750 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:12 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:12.750 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:12 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:12.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:12 smithi129 ceph-mon[35314]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:12.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:12 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:12.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:12 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:12.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:12 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:12.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:12 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:12.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:12 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:12.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:12 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:12.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:12 smithi005 ceph-mon[30944]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:12.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:12.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:12 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:13.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:06:13.449 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:13.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:13 smithi005 ceph-mon[30944]: Deploying daemon mds.h on smithi099 2024-09-15T22:06:13.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:13 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.129:6836/3535375231,v1:172.21.15.129:6837/3535375231] up:boot 2024-09-15T22:06:13.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:13 smithi005 ceph-mon[30944]: fsmap 7 up:standby 2024-09-15T22:06:13.571 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:13 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:06:13.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:13 smithi099 ceph-mon[34892]: Deploying daemon mds.h on smithi099 2024-09-15T22:06:13.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:13 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.129:6836/3535375231,v1:172.21.15.129:6837/3535375231] up:boot 2024-09-15T22:06:13.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:13 smithi099 ceph-mon[34892]: fsmap 7 up:standby 2024-09-15T22:06:13.790 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:13 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:06:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:13 smithi129 ceph-mon[35314]: Deploying daemon mds.h on smithi099 2024-09-15T22:06:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:13 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.129:6836/3535375231,v1:172.21.15.129:6837/3535375231] up:boot 2024-09-15T22:06:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:13 smithi129 ceph-mon[35314]: fsmap 7 up:standby 2024-09-15T22:06:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:13 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:06:14.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.042+0000 7f7046366640 1 -- 172.21.15.5:0/3614373222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7040101280 msgr2=0x7f704010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:14.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.042+0000 7f7046366640 1 --2- 172.21.15.5:0/3614373222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7040101280 0x7f704010df40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7028009900 tx=0x7f702802a410 comp rx=0 tx=0).stop 2024-09-15T22:06:14.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.042+0000 7f7046366640 1 -- 172.21.15.5:0/3614373222 shutdown_connections 2024-09-15T22:06:14.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.042+0000 7f7046366640 1 --2- 172.21.15.5:0/3614373222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f704010e480 0x7f7040110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:14.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.042+0000 7f7046366640 1 --2- 172.21.15.5:0/3614373222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7040101280 0x7f704010df40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:14.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.042+0000 7f7046366640 1 --2- 172.21.15.5:0/3614373222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70401008d0 0x7f7040100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:14.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.042+0000 7f7046366640 1 -- 172.21.15.5:0/3614373222 >> 172.21.15.5:0/3614373222 conn(0x7f70400fc7a0 msgr2=0x7f70400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:14.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.042+0000 7f7046366640 1 -- 172.21.15.5:0/3614373222 shutdown_connections 2024-09-15T22:06:14.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.042+0000 7f7046366640 1 -- 172.21.15.5:0/3614373222 wait complete. 2024-09-15T22:06:14.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.043+0000 7f7046366640 1 Processor -- start 2024-09-15T22:06:14.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.043+0000 7f7046366640 1 -- start start 2024-09-15T22:06:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.043+0000 7f7046366640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70401008d0 0x7f704019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.043+0000 7f7046366640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7040101280 0x7f704019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.043+0000 7f7046366640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704010e480 0x7f704019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.043+0000 7f7046366640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70401141a0 con 0x7f70401008d0 2024-09-15T22:06:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.043+0000 7f7046366640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7040114020 con 0x7f704010e480 2024-09-15T22:06:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.043+0000 7f7046366640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7040114320 con 0x7f7040101280 2024-09-15T22:06:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.043+0000 7f703ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70401008d0 0x7f704019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.043+0000 7f703ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70401008d0 0x7f704019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52234/0 (socket says 172.21.15.5:52234) 2024-09-15T22:06:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.043+0000 7f703ffff640 1 -- 172.21.15.5:0/3978697078 learned_addr learned my addr 172.21.15.5:0/3978697078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:14.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f7037fff640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7040101280 0x7f704019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f7044903640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704010e480 0x7f704019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f703ffff640 1 -- 172.21.15.5:0/3978697078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7040101280 msgr2=0x7f704019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f703ffff640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7040101280 0x7f704019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f703ffff640 1 -- 172.21.15.5:0/3978697078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704010e480 msgr2=0x7f704019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f703ffff640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704010e480 0x7f704019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f703ffff640 1 -- 172.21.15.5:0/3978697078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704019f6d0 con 0x7f70401008d0 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f7037fff640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7040101280 0x7f704019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f703ffff640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70401008d0 0x7f704019a720 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f703000d950 tx=0x7f703000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f703dffb640 1 -- 172.21.15.5:0/3978697078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030017030 con 0x7f70401008d0 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f703dffb640 1 -- 172.21.15.5:0/3978697078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7030004020 con 0x7f70401008d0 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f703dffb640 1 -- 172.21.15.5:0/3978697078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70300047a0 con 0x7f70401008d0 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f7046366640 1 -- 172.21.15.5:0/3978697078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704019f9c0 con 0x7f70401008d0 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.044+0000 7f7046366640 1 -- 172.21.15.5:0/3978697078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70401a72a0 con 0x7f70401008d0 2024-09-15T22:06:14.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.045+0000 7f7046366640 1 -- 172.21.15.5:0/3978697078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7040100cb0 con 0x7f70401008d0 2024-09-15T22:06:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.046+0000 7f703dffb640 1 -- 172.21.15.5:0/3978697078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f70300041d0 con 0x7f70401008d0 2024-09-15T22:06:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.047+0000 7f703dffb640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f702c0a3240 0x7f702c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.047+0000 7f7037fff640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f702c0a3240 0x7f702c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:14.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.048+0000 7f703dffb640 1 -- 172.21.15.5:0/3978697078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f70300cef10 con 0x7f70401008d0 2024-09-15T22:06:14.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.048+0000 7f7037fff640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f702c0a3240 0x7f702c0a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f704010ed50 tx=0x7f7028005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:14.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.052+0000 7f703dffb640 1 -- 172.21.15.5:0/3978697078 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f703009c0d0 con 0x7f70401008d0 2024-09-15T22:06:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.271+0000 7f7046366640 1 -- 172.21.15.5:0/3978697078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7040109670 con 0x7f70401008d0 2024-09-15T22:06:14.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.271+0000 7f703dffb640 1 -- 172.21.15.5:0/3978697078 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6379 (secure 0 0 0) 0x7f703009b9d0 con 0x7f70401008d0 2024-09-15T22:06:14.273 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"btime":"2024-09-15T22:06:12:438981+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-15T22:06:14.274 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-09-15T22:06:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 -- 172.21.15.5:0/3978697078 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f702c0a3240 msgr2=0x7f702c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f702c0a3240 0x7f702c0a5700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f704010ed50 tx=0x7f7028005a20 comp rx=0 tx=0).stop 2024-09-15T22:06:14.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 -- 172.21.15.5:0/3978697078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70401008d0 msgr2=0x7f704019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70401008d0 0x7f704019a720 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f703000d950 tx=0x7f703000de10 comp rx=0 tx=0).stop 2024-09-15T22:06:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 -- 172.21.15.5:0/3978697078 shutdown_connections 2024-09-15T22:06:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f704010e480 0x7f704019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7040101280 0x7f704019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f702c0a3240 0x7f702c0a5700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 --2- 172.21.15.5:0/3978697078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70401008d0 0x7f704019a720 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 -- 172.21.15.5:0/3978697078 >> 172.21.15.5:0/3978697078 conn(0x7f70400fc7a0 msgr2=0x7f70400feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 -- 172.21.15.5:0/3978697078 shutdown_connections 2024-09-15T22:06:14.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:14.274+0000 7f7046366640 1 -- 172.21.15.5:0/3978697078 wait complete. 2024-09-15T22:06:14.340 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:06:14 smithi099 podman[64664]: 2024-09-15 22:06:14.281726051 +0000 UTC m=+0.130595605 container create 6f5ba94cdc815f3e2f349cbe1b59350d1c40302e47f53106809aa46f37c7fa3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=) 2024-09-15T22:06:14.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:14 smithi005 ceph-mon[30944]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:14.720 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:14 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3978697078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:14.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:14 smithi099 ceph-mon[34892]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:14.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:14 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3978697078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:14.840 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:06:14 smithi099 podman[64664]: 2024-09-15 22:06:14.527082514 +0000 UTC m=+0.375952079 container init 6f5ba94cdc815f3e2f349cbe1b59350d1c40302e47f53106809aa46f37c7fa3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-15T22:06:14.840 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:06:14 smithi099 podman[64664]: 2024-09-15 22:06:14.530177811 +0000 UTC m=+0.379047369 container start 6f5ba94cdc815f3e2f349cbe1b59350d1c40302e47f53106809aa46f37c7fa3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-15T22:06:14.840 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:06:14 smithi099 ceph-mds[64683]: -- 172.21.15.99:0/50536676 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5621e7197860 con 0x5621e7175400 2024-09-15T22:06:14.840 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:06:14 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h[64679]: starting mds.h at 2024-09-15T22:06:14.840 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:06:14 smithi099 bash[64664]: 6f5ba94cdc815f3e2f349cbe1b59350d1c40302e47f53106809aa46f37c7fa3c 2024-09-15T22:06:14.840 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:06:14 smithi099 systemd[1]: Started Ceph mds.h for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:06:14.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:14 smithi129 ceph-mon[35314]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:14.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:14 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3978697078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:15 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:15 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:15 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:15 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:15 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:15 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:15 smithi099 ceph-mon[34892]: Deploying daemon mds.i on smithi129 2024-09-15T22:06:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:15 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] up:boot 2024-09-15T22:06:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:15 smithi099 ceph-mon[34892]: fsmap 8 up:standby 2024-09-15T22:06:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:15 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:06:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:15 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:15 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:15 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:15 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:15 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:15 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:15 smithi129 ceph-mon[35314]: Deploying daemon mds.i on smithi129 2024-09-15T22:06:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:15 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] up:boot 2024-09-15T22:06:16.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:15 smithi129 ceph-mon[35314]: fsmap 8 up:standby 2024-09-15T22:06:16.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:15 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:06:16.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:15 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:16.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:15 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:16.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:15 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:16.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:15 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:16.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:15 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:16.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:15 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:16.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:15 smithi005 ceph-mon[30944]: Deploying daemon mds.i on smithi129 2024-09-15T22:06:16.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:15 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] up:boot 2024-09-15T22:06:16.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:15 smithi005 ceph-mon[30944]: fsmap 8 up:standby 2024-09-15T22:06:16.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:15 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:06:16.680 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:06:16 smithi129 systemd[1]: Starting Ceph mds.i for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:06:16.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:06:16.956 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:16 smithi005 ceph-mon[30944]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:16.959 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:16 smithi129 ceph-mon[35314]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:16.959 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:06:16 smithi129 podman[65228]: 2024-09-15 22:06:16.690999911 +0000 UTC m=+0.016021674 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:06:16.959 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:06:16 smithi129 podman[65228]: 2024-09-15 22:06:16.824306105 +0000 UTC m=+0.149327861 container create 54daa7354c84a78d28f77acb2ac7826bec264c8aabe37039e0bba273c0f7efec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e) 2024-09-15T22:06:17.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:16 smithi099 ceph-mon[34892]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:17.198 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:17.345 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:06:17 smithi129 podman[65228]: 2024-09-15 22:06:17.077045288 +0000 UTC m=+0.402067057 container init 54daa7354c84a78d28f77acb2ac7826bec264c8aabe37039e0bba273c0f7efec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e) 2024-09-15T22:06:17.345 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:06:17 smithi129 podman[65228]: 2024-09-15 22:06:17.080284945 +0000 UTC m=+0.405306701 container start 54daa7354c84a78d28f77acb2ac7826bec264c8aabe37039e0bba273c0f7efec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-15T22:06:17.345 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:06:17 smithi129 ceph-mds[65249]: -- 172.21.15.129:0/195959737 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564505379c20 con 0x564505356800 2024-09-15T22:06:17.345 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:06:17 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i[65245]: starting mds.i at 2024-09-15T22:06:17.345 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:06:17 smithi129 bash[65228]: 54daa7354c84a78d28f77acb2ac7826bec264c8aabe37039e0bba273c0f7efec 2024-09-15T22:06:17.346 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:06:17 smithi129 systemd[1]: Started Ceph mds.i for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:06:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.806+0000 7f618a291640 1 -- 172.21.15.5:0/3820444987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184103740 msgr2=0x7f6184107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.806+0000 7f618a291640 1 --2- 172.21.15.5:0/3820444987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184103740 0x7f6184107b80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6178009930 tx=0x7f617802a420 comp rx=0 tx=0).stop 2024-09-15T22:06:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.806+0000 7f618a291640 1 -- 172.21.15.5:0/3820444987 shutdown_connections 2024-09-15T22:06:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.806+0000 7f618a291640 1 --2- 172.21.15.5:0/3820444987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6184108290 0x7f618410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.806+0000 7f618a291640 1 --2- 172.21.15.5:0/3820444987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184103740 0x7f6184107b80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.806+0000 7f618a291640 1 --2- 172.21.15.5:0/3820444987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6184102d90 0x7f6184103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.806+0000 7f618a291640 1 -- 172.21.15.5:0/3820444987 >> 172.21.15.5:0/3820444987 conn(0x7f61840fc7d0 msgr2=0x7f61840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.806+0000 7f618a291640 1 -- 172.21.15.5:0/3820444987 shutdown_connections 2024-09-15T22:06:17.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.806+0000 7f618a291640 1 -- 172.21.15.5:0/3820444987 wait complete. 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.806+0000 7f618a291640 1 Processor -- start 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f618a291640 1 -- start start 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f618a291640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6184102d90 0x7f61841985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f618a291640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184103740 0x7f6184198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f618a291640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6184108290 0x7f618419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f618a291640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6184112090 con 0x7f6184103740 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f618a291640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6184111f10 con 0x7f6184102d90 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f618a291640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6184112210 con 0x7f6184108290 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f61837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184103740 0x7f6184198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f61837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184103740 0x7f6184198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52250/0 (socket says 172.21.15.5:52250) 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f61837fe640 1 -- 172.21.15.5:0/2885415465 learned_addr learned my addr 172.21.15.5:0/2885415465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f6183fff640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6184102d90 0x7f61841985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f618882e640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6184108290 0x7f618419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:17.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f61837fe640 1 -- 172.21.15.5:0/2885415465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6184108290 msgr2=0x7f618419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f61837fe640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6184108290 0x7f618419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f61837fe640 1 -- 172.21.15.5:0/2885415465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6184102d90 msgr2=0x7f61841985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f61837fe640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6184102d90 0x7f61841985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f61837fe640 1 -- 172.21.15.5:0/2885415465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618419d5a0 con 0x7f6184103740 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f618882e640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6184108290 0x7f618419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.807+0000 7f6183fff640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6184102d90 0x7f61841985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.808+0000 7f61837fe640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184103740 0x7f6184198b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6178005940 tx=0x7f61780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.808+0000 7f61817fa640 1 -- 172.21.15.5:0/2885415465 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617803f030 con 0x7f6184103740 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.808+0000 7f61817fa640 1 -- 172.21.15.5:0/2885415465 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6178003ce0 con 0x7f6184103740 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.808+0000 7f618a291640 1 -- 172.21.15.5:0/2885415465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f618419d830 con 0x7f6184103740 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.808+0000 7f61817fa640 1 -- 172.21.15.5:0/2885415465 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6178035070 con 0x7f6184103740 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.808+0000 7f618a291640 1 -- 172.21.15.5:0/2885415465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61841a5170 con 0x7f6184103740 2024-09-15T22:06:17.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.809+0000 7f618a291640 1 -- 172.21.15.5:0/2885415465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f618406a7f0 con 0x7f6184103740 2024-09-15T22:06:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.810+0000 7f61817fa640 1 -- 172.21.15.5:0/2885415465 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f6178042030 con 0x7f6184103740 2024-09-15T22:06:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.811+0000 7f61817fa640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f616c0a3240 0x7f616c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.811+0000 7f6183fff640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f616c0a3240 0x7f616c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.811+0000 7f61817fa640 1 -- 172.21.15.5:0/2885415465 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f61780efb30 con 0x7f6184103740 2024-09-15T22:06:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.811+0000 7f6183fff640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f616c0a3240 0x7f616c0a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6174009b40 tx=0x7f6174005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:17.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:17.815+0000 7f61817fa640 1 -- 172.21.15.5:0/2885415465 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61780bcc50 con 0x7f6184103740 2024-09-15T22:06:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.029+0000 7f618a291640 1 -- 172.21.15.5:0/2885415465 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f618410a2a0 con 0x7f6184103740 2024-09-15T22:06:18.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.030+0000 7f61817fa640 1 -- 172.21.15.5:0/2885415465 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8034 (secure 0 0 0) 0x7f61780bc550 con 0x7f6184103740 2024-09-15T22:06:18.033 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:18.033 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"btime":"2024-09-15T22:06:17:678895+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-15T22:06:18.033 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 10 2024-09-15T22:06:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.032+0000 7f618a291640 1 -- 172.21.15.5:0/2885415465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f616c0a3240 msgr2=0x7f616c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.032+0000 7f618a291640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f616c0a3240 0x7f616c0a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6174009b40 tx=0x7f6174005e50 comp rx=0 tx=0).stop 2024-09-15T22:06:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.032+0000 7f618a291640 1 -- 172.21.15.5:0/2885415465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184103740 msgr2=0x7f6184198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.032+0000 7f618a291640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184103740 0x7f6184198b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6178005940 tx=0x7f61780042f0 comp rx=0 tx=0).stop 2024-09-15T22:06:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.032+0000 7f618a291640 1 -- 172.21.15.5:0/2885415465 shutdown_connections 2024-09-15T22:06:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.032+0000 7f618a291640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6184108290 0x7f618419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.032+0000 7f618a291640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f616c0a3240 0x7f616c0a5700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.032+0000 7f618a291640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6184103740 0x7f6184198b30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.032+0000 7f618a291640 1 --2- 172.21.15.5:0/2885415465 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6184102d90 0x7f61841985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.032+0000 7f618a291640 1 -- 172.21.15.5:0/2885415465 >> 172.21.15.5:0/2885415465 conn(0x7f61840fc7d0 msgr2=0x7f6184106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.032+0000 7f618a291640 1 -- 172.21.15.5:0/2885415465 shutdown_connections 2024-09-15T22:06:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:18.033+0000 7f618a291640 1 -- 172.21.15.5:0/2885415465 wait complete. 2024-09-15T22:06:18.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:18.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:18.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:18.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:18.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:18.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:18.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:18.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: Deploying daemon mds.g on smithi005 2024-09-15T22:06:18.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] up:boot 2024-09-15T22:06:18.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: fsmap 9 up:standby 2024-09-15T22:06:18.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:06:18.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:18 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2885415465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: Deploying daemon mds.g on smithi005 2024-09-15T22:06:18.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] up:boot 2024-09-15T22:06:18.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: fsmap 9 up:standby 2024-09-15T22:06:18.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:06:18.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:18 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2885415465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:18.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:18.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:18.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:18.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:18.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:18.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:18.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:18.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: Deploying daemon mds.g on smithi005 2024-09-15T22:06:18.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] up:boot 2024-09-15T22:06:18.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: fsmap 9 up:standby 2024-09-15T22:06:18.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:06:18.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:18 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2885415465' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:20.234 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:06:20 smithi005 systemd[1]: Starting Ceph mds.g for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:06:20.508 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:20 smithi005 ceph-mon[30944]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:20.509 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:06:20 smithi005 podman[70991]: 2024-09-15 22:06:20.272239124 +0000 UTC m=+0.016442942 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:06:20.509 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:06:20 smithi005 podman[70991]: 2024-09-15 22:06:20.383507502 +0000 UTC m=+0.127711362 container create 99ebb54bd236fa17df78451f76ef44a0067f6505b3cc5cb3893e165a1231edef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:06:20.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:20 smithi099 ceph-mon[34892]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:20.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:20 smithi129 ceph-mon[35314]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:20.786 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:06:20.812 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:06:20 smithi005 podman[70991]: 2024-09-15 22:06:20.627971804 +0000 UTC m=+0.372175621 container init 99ebb54bd236fa17df78451f76ef44a0067f6505b3cc5cb3893e165a1231edef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-09-15T22:06:20.812 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:06:20 smithi005 podman[70991]: 2024-09-15 22:06:20.630838989 +0000 UTC m=+0.375042807 container start 99ebb54bd236fa17df78451f76ef44a0067f6505b3cc5cb3893e165a1231edef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-g, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-15T22:06:20.813 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:06:20 smithi005 ceph-mds[71010]: -- 172.21.15.5:0/1488132671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x556f5855a680 con 0x556f5852e800 2024-09-15T22:06:20.813 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:06:20 smithi005 ceph-mds[71010]: -- 172.21.15.5:0/1488132671 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f58551860 con 0x556f5852e800 2024-09-15T22:06:20.813 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:06:20 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-g[71006]: starting mds.g at 2024-09-15T22:06:20.813 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:06:20 smithi005 bash[70991]: 99ebb54bd236fa17df78451f76ef44a0067f6505b3cc5cb3893e165a1231edef 2024-09-15T22:06:20.813 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:06:20 smithi005 systemd[1]: Started Ceph mds.g for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:06:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:21.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.506+0000 7efce6583640 1 -- 172.21.15.5:0/2901885217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0106a80 msgr2=0x7efce0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.506+0000 7efce6583640 1 --2- 172.21.15.5:0/2901885217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0106a80 0x7efce0106e60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009930 tx=0x7efcd002a420 comp rx=0 tx=0).stop 2024-09-15T22:06:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.506+0000 7efce6583640 1 -- 172.21.15.5:0/2901885217 shutdown_connections 2024-09-15T22:06:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.506+0000 7efce6583640 1 --2- 172.21.15.5:0/2901885217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0101420 0x7efce010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.506+0000 7efce6583640 1 --2- 172.21.15.5:0/2901885217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efce0100a80 0x7efce0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.506+0000 7efce6583640 1 --2- 172.21.15.5:0/2901885217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0106a80 0x7efce0106e60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.506+0000 7efce6583640 1 -- 172.21.15.5:0/2901885217 >> 172.21.15.5:0/2901885217 conn(0x7efce00fc7a0 msgr2=0x7efce00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.506+0000 7efce6583640 1 -- 172.21.15.5:0/2901885217 shutdown_connections 2024-09-15T22:06:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.506+0000 7efce6583640 1 -- 172.21.15.5:0/2901885217 wait complete. 2024-09-15T22:06:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efce6583640 1 Processor -- start 2024-09-15T22:06:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efce6583640 1 -- start start 2024-09-15T22:06:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efce6583640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0100a80 0x7efce01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:21.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efce6583640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efce0101420 0x7efce01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:21.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efce6583640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0106a80 0x7efce01a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:21.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efce6583640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efce0110ef0 con 0x7efce0100a80 2024-09-15T22:06:21.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efce6583640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efce0110d70 con 0x7efce0106a80 2024-09-15T22:06:21.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efce6583640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efce0111070 con 0x7efce0101420 2024-09-15T22:06:21.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efcdffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0100a80 0x7efce01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:21.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efcdffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0100a80 0x7efce01993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52296/0 (socket says 172.21.15.5:52296) 2024-09-15T22:06:21.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efcdffff640 1 -- 172.21.15.5:0/3758099582 learned_addr learned my addr 172.21.15.5:0/3758099582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:21.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efce4b20640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0106a80 0x7efce01a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:21.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.507+0000 7efcd7fff640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efce0101420 0x7efce01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:21.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efcdffff640 1 -- 172.21.15.5:0/3758099582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efce0101420 msgr2=0x7efce01998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efcdffff640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efce0101420 0x7efce01998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efcdffff640 1 -- 172.21.15.5:0/3758099582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0106a80 msgr2=0x7efce01a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efcdffff640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0106a80 0x7efce01a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efcdffff640 1 -- 172.21.15.5:0/3758099582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce01a4960 con 0x7efce0100a80 2024-09-15T22:06:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efce4b20640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0106a80 0x7efce01a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efcd7fff640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efce0101420 0x7efce01998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efcdffff640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0100a80 0x7efce01993a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efcd002a940 tx=0x7efcd0003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efcddffb640 1 -- 172.21.15.5:0/3758099582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd003f070 con 0x7efce0100a80 2024-09-15T22:06:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efce6583640 1 -- 172.21.15.5:0/3758099582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce0197190 con 0x7efce0100a80 2024-09-15T22:06:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efcddffb640 1 -- 172.21.15.5:0/3758099582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd003b460 con 0x7efce0100a80 2024-09-15T22:06:21.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efcddffb640 1 -- 172.21.15.5:0/3758099582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0035030 con 0x7efce0100a80 2024-09-15T22:06:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.508+0000 7efce6583640 1 -- 172.21.15.5:0/3758099582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce01976c0 con 0x7efce0100a80 2024-09-15T22:06:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.509+0000 7efce6583640 1 -- 172.21.15.5:0/3758099582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce01021c0 con 0x7efce0100a80 2024-09-15T22:06:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.510+0000 7efcddffb640 1 -- 172.21.15.5:0/3758099582 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7efcd0042070 con 0x7efce0100a80 2024-09-15T22:06:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.511+0000 7efcddffb640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efccc0a3240 0x7efccc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.511+0000 7efcd7fff640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efccc0a3240 0x7efccc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.511+0000 7efcddffb640 1 -- 172.21.15.5:0/3758099582 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7efcd00efe20 con 0x7efce0100a80 2024-09-15T22:06:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.511+0000 7efcd7fff640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efccc0a3240 0x7efccc0a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efce006b2e0 tx=0x7efcc8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:21.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.515+0000 7efcddffb640 1 -- 172.21.15.5:0/3758099582 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcd00bcf40 con 0x7efce0100a80 2024-09-15T22:06:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.726+0000 7efce6583640 1 -- 172.21.15.5:0/3758099582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efce006ad80 con 0x7efce0100a80 2024-09-15T22:06:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.727+0000 7efcddffb640 1 -- 172.21.15.5:0/3758099582 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8857 (secure 0 0 0) 0x7efcd00bc840 con 0x7efce0100a80 2024-09-15T22:06:21.730 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:21.730 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":11,"btime":"2024-09-15T22:06:21:232725+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-15T22:06:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 11 2024-09-15T22:06:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 -- 172.21.15.5:0/3758099582 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efccc0a3240 msgr2=0x7efccc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efccc0a3240 0x7efccc0a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efce006b2e0 tx=0x7efcc8005d30 comp rx=0 tx=0).stop 2024-09-15T22:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 -- 172.21.15.5:0/3758099582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0100a80 msgr2=0x7efce01993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0100a80 0x7efce01993a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7efcd002a940 tx=0x7efcd0003c00 comp rx=0 tx=0).stop 2024-09-15T22:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 -- 172.21.15.5:0/3758099582 shutdown_connections 2024-09-15T22:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0106a80 0x7efce01a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efce0101420 0x7efce01998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efccc0a3240 0x7efccc0a5700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 --2- 172.21.15.5:0/3758099582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0100a80 0x7efce01993a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 -- 172.21.15.5:0/3758099582 >> 172.21.15.5:0/3758099582 conn(0x7efce00fc7a0 msgr2=0x7efce00fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 -- 172.21.15.5:0/3758099582 shutdown_connections 2024-09-15T22:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:21.729+0000 7efce6583640 1 -- 172.21.15.5:0/3758099582 wait complete. 2024-09-15T22:06:22.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:22.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:22.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:22.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:22.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:22.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:22.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:22.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: Deploying daemon mds.a on smithi005 2024-09-15T22:06:22.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.5:6838/2796220965,v1:172.21.15.5:6839/2796220965] up:boot 2024-09-15T22:06:22.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: fsmap 10 up:standby 2024-09-15T22:06:22.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:06:22.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:21 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3758099582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:22.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:22.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:22.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:22.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:22.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:22.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:22.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:22.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: Deploying daemon mds.a on smithi005 2024-09-15T22:06:22.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.5:6838/2796220965,v1:172.21.15.5:6839/2796220965] up:boot 2024-09-15T22:06:22.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: fsmap 10 up:standby 2024-09-15T22:06:22.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:06:22.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:21 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3758099582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:22.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:22.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:22.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:22.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:22.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:22.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:22.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:22.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: Deploying daemon mds.a on smithi005 2024-09-15T22:06:22.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.5:6838/2796220965,v1:172.21.15.5:6839/2796220965] up:boot 2024-09-15T22:06:22.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: fsmap 10 up:standby 2024-09-15T22:06:22.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:06:22.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:21 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3758099582' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:24.025 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:23 smithi005 ceph-mon[30944]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:24.025 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:06:23 smithi005 systemd[1]: Starting Ceph mds.a for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:06:24.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:23 smithi099 ceph-mon[34892]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:24.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:23 smithi129 ceph-mon[35314]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:24.284 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:06:24 smithi005 podman[71410]: 2024-09-15 22:06:23.926398738 +0000 UTC m=+0.016625600 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:06:24.284 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:06:24 smithi005 podman[71410]: 2024-09-15 22:06:24.030530843 +0000 UTC m=+0.120757705 container create 18bcf8c0d601b76db0a774a8e98b2a783715988251938914fd3adbcb336198b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:06:24.284 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:06:24 smithi005 podman[71410]: 2024-09-15 22:06:24.283644871 +0000 UTC m=+0.373871727 container init 18bcf8c0d601b76db0a774a8e98b2a783715988251938914fd3adbcb336198b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-15T22:06:24.407 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:06:24.606 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:06:24 smithi005 podman[71410]: 2024-09-15 22:06:24.286589429 +0000 UTC m=+0.376816286 container start 18bcf8c0d601b76db0a774a8e98b2a783715988251938914fd3adbcb336198b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:06:24.607 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:06:24 smithi005 ceph-mds[71430]: -- 172.21.15.5:0/3495510324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc4ccfb860 con 0x55fc4ccd9000 2024-09-15T22:06:24.607 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:06:24 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-a[71425]: starting mds.a at 2024-09-15T22:06:24.607 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:06:24 smithi005 bash[71410]: 18bcf8c0d601b76db0a774a8e98b2a783715988251938914fd3adbcb336198b4 2024-09-15T22:06:24.607 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:06:24 smithi005 systemd[1]: Started Ceph mds.a for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:06:24.685 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.169+0000 7fcc74dc1640 1 -- 172.21.15.5:0/3365748657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100a80 msgr2=0x7fcc70100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.169+0000 7fcc74dc1640 1 --2- 172.21.15.5:0/3365748657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100a80 0x7fcc70100ee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009930 tx=0x7fcc5802a420 comp rx=0 tx=0).stop 2024-09-15T22:06:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.169+0000 7fcc74dc1640 1 -- 172.21.15.5:0/3365748657 shutdown_connections 2024-09-15T22:06:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.169+0000 7fcc74dc1640 1 --2- 172.21.15.5:0/3365748657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc70101420 0x7fcc7010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.169+0000 7fcc74dc1640 1 --2- 172.21.15.5:0/3365748657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100a80 0x7fcc70100ee0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.169+0000 7fcc74dc1640 1 --2- 172.21.15.5:0/3365748657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc70106a80 0x7fcc70106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.169+0000 7fcc74dc1640 1 -- 172.21.15.5:0/3365748657 >> 172.21.15.5:0/3365748657 conn(0x7fcc700fc7a0 msgr2=0x7fcc700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.169+0000 7fcc74dc1640 1 -- 172.21.15.5:0/3365748657 shutdown_connections 2024-09-15T22:06:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.169+0000 7fcc74dc1640 1 -- 172.21.15.5:0/3365748657 wait complete. 2024-09-15T22:06:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.170+0000 7fcc74dc1640 1 Processor -- start 2024-09-15T22:06:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.170+0000 7fcc74dc1640 1 -- start start 2024-09-15T22:06:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc74dc1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100a80 0x7fcc70199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc74dc1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc70101420 0x7fcc701997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc74dc1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc70106a80 0x7fcc701a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc74dc1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70110e40 con 0x7fcc70100a80 2024-09-15T22:06:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc74dc1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70110cc0 con 0x7fcc70101420 2024-09-15T22:06:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc74dc1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcc70110fc0 con 0x7fcc70106a80 2024-09-15T22:06:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc6e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100a80 0x7fcc70199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc6e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100a80 0x7fcc70199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54818/0 (socket says 172.21.15.5:54818) 2024-09-15T22:06:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc6e59c640 1 -- 172.21.15.5:0/2861997656 learned_addr learned my addr 172.21.15.5:0/2861997656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc6ed9d640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc70106a80 0x7fcc701a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc67fff640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc70101420 0x7fcc701997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:25.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc6e59c640 1 -- 172.21.15.5:0/2861997656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc70106a80 msgr2=0x7fcc701a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc6e59c640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc70106a80 0x7fcc701a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc6e59c640 1 -- 172.21.15.5:0/2861997656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc70101420 msgr2=0x7fcc701997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc6e59c640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc70101420 0x7fcc701997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc6e59c640 1 -- 172.21.15.5:0/2861997656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc701a4830 con 0x7fcc70100a80 2024-09-15T22:06:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc6e59c640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100a80 0x7fcc70199270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000c920 tx=0x7fcc6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc677fe640 1 -- 172.21.15.5:0/2861997656 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60007c70 con 0x7fcc70100a80 2024-09-15T22:06:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc74dc1640 1 -- 172.21.15.5:0/2861997656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc701a4a00 con 0x7fcc70100a80 2024-09-15T22:06:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc677fe640 1 -- 172.21.15.5:0/2861997656 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc60007e10 con 0x7fcc70100a80 2024-09-15T22:06:25.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.171+0000 7fcc677fe640 1 -- 172.21.15.5:0/2861997656 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60018890 con 0x7fcc70100a80 2024-09-15T22:06:25.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.172+0000 7fcc74dc1640 1 -- 172.21.15.5:0/2861997656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc701a4ec0 con 0x7fcc70100a80 2024-09-15T22:06:25.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.173+0000 7fcc74dc1640 1 -- 172.21.15.5:0/2861997656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc70100ee0 con 0x7fcc70100a80 2024-09-15T22:06:25.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.174+0000 7fcc677fe640 1 -- 172.21.15.5:0/2861997656 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fcc60018a50 con 0x7fcc70100a80 2024-09-15T22:06:25.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.174+0000 7fcc677fe640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcc5c0a3240 0x7fcc5c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:25.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.175+0000 7fcc67fff640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcc5c0a3240 0x7fcc5c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:25.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.175+0000 7fcc677fe640 1 -- 172.21.15.5:0/2861997656 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fcc600cf730 con 0x7fcc70100a80 2024-09-15T22:06:25.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.175+0000 7fcc67fff640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcc5c0a3240 0x7fcc5c0a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009900 tx=0x7fcc580058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:25.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.178+0000 7fcc677fe640 1 -- 172.21.15.5:0/2861997656 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc60014030 con 0x7fcc70100a80 2024-09-15T22:06:25.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.388+0000 7fcc74dc1640 1 -- 172.21.15.5:0/2861997656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc70109040 con 0x7fcc70100a80 2024-09-15T22:06:25.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.389+0000 7fcc677fe640 1 -- 172.21.15.5:0/2861997656 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9680 (secure 0 0 0) 0x7fcc6009c980 con 0x7fcc70100a80 2024-09-15T22:06:25.392 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:25.392 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":12,"btime":"2024-09-15T22:06:24:772278+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-15T22:06:25.392 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 12 2024-09-15T22:06:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 -- 172.21.15.5:0/2861997656 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcc5c0a3240 msgr2=0x7fcc5c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcc5c0a3240 0x7fcc5c0a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009900 tx=0x7fcc580058d0 comp rx=0 tx=0).stop 2024-09-15T22:06:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 -- 172.21.15.5:0/2861997656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100a80 msgr2=0x7fcc70199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:25.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100a80 0x7fcc70199270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000c920 tx=0x7fcc6000cde0 comp rx=0 tx=0).stop 2024-09-15T22:06:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 -- 172.21.15.5:0/2861997656 shutdown_connections 2024-09-15T22:06:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc70106a80 0x7fcc701a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc70101420 0x7fcc701997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcc5c0a3240 0x7fcc5c0a5700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 --2- 172.21.15.5:0/2861997656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc70100a80 0x7fcc70199270 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 -- 172.21.15.5:0/2861997656 >> 172.21.15.5:0/2861997656 conn(0x7fcc700fc7a0 msgr2=0x7fcc700fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 -- 172.21.15.5:0/2861997656 shutdown_connections 2024-09-15T22:06:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:25.391+0000 7fcc74dc1640 1 -- 172.21.15.5:0/2861997656 wait complete. 2024-09-15T22:06:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:25 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:25 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:25 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:25 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:25 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:25 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:25 smithi099 ceph-mon[34892]: Deploying daemon mds.b on smithi099 2024-09-15T22:06:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:25 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.5:6840/3376413003,v1:172.21.15.5:6841/3376413003] up:boot 2024-09-15T22:06:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:25 smithi099 ceph-mon[34892]: fsmap 11 up:standby 2024-09-15T22:06:25.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:25 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:06:25.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:25 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2861997656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:25 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:25 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:25 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:25 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:25.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:25 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:25.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:25 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:25.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:25 smithi005 ceph-mon[30944]: Deploying daemon mds.b on smithi099 2024-09-15T22:06:25.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:25 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.5:6840/3376413003,v1:172.21.15.5:6841/3376413003] up:boot 2024-09-15T22:06:25.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:25 smithi005 ceph-mon[30944]: fsmap 11 up:standby 2024-09-15T22:06:25.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:25 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:06:25.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:25 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2861997656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:25.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:25 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:25.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:25 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:25.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:25 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:25.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:25 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-15T22:06:25.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:25 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-15T22:06:25.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:25 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:25.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:25 smithi129 ceph-mon[35314]: Deploying daemon mds.b on smithi099 2024-09-15T22:06:25.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:25 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.5:6840/3376413003,v1:172.21.15.5:6841/3376413003] up:boot 2024-09-15T22:06:25.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:25 smithi129 ceph-mon[35314]: fsmap 11 up:standby 2024-09-15T22:06:25.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:25 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:06:25.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:25 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2861997656' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:26.565 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:26 smithi099 ceph-mon[34892]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:26.840 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:06:26 smithi099 podman[64976]: 2024-09-15 22:06:26.797763086 +0000 UTC m=+0.137228275 container create 51a3dc6feead20303ee11e71e434d74e9b55d0a409e378bbe8a8f08489541854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731) 2024-09-15T22:06:26.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:26 smithi129 ceph-mon[35314]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:26.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:26 smithi005 ceph-mon[30944]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:27.340 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:06:27 smithi099 podman[64976]: 2024-09-15 22:06:27.025430928 +0000 UTC m=+0.364896078 container init 51a3dc6feead20303ee11e71e434d74e9b55d0a409e378bbe8a8f08489541854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-15T22:06:27.340 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:06:27 smithi099 podman[64976]: 2024-09-15 22:06:27.028415776 +0000 UTC m=+0.367880946 container start 51a3dc6feead20303ee11e71e434d74e9b55d0a409e378bbe8a8f08489541854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-15T22:06:27.340 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:06:27 smithi099 ceph-mds[64995]: -- 172.21.15.99:0/816029415 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd52cad860 con 0x55dd52c8b400 2024-09-15T22:06:27.340 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:06:27 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b[64991]: starting mds.b at 2024-09-15T22:06:27.340 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:06:27 smithi099 bash[64976]: 51a3dc6feead20303ee11e71e434d74e9b55d0a409e378bbe8a8f08489541854 2024-09-15T22:06:27.340 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:06:27 smithi099 systemd[1]: Started Ceph mds.b for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:06:28.118 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:06:28.206 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:28 smithi099 ceph-mon[34892]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:28.206 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.206 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.206 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.206 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.206 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.207 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:28.207 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:28 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] up:boot 2024-09-15T22:06:28.207 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:28 smithi099 ceph-mon[34892]: fsmap 12 up:standby 2024-09-15T22:06:28.207 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:06:28.207 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:28 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:28 smithi129 ceph-mon[35314]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:28.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:28.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:28 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] up:boot 2024-09-15T22:06:28.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:28 smithi129 ceph-mon[35314]: fsmap 12 up:standby 2024-09-15T22:06:28.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:06:28.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:28 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:28 smithi005 ceph-mon[30944]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:28.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:28 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] up:boot 2024-09-15T22:06:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:28 smithi005 ceph-mon[30944]: fsmap 12 up:standby 2024-09-15T22:06:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:06:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:28 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:28.466 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.964+0000 7f6455483640 1 -- 172.21.15.5:0/468627431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f645010c170 msgr2=0x7f645010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.964+0000 7f6455483640 1 --2- 172.21.15.5:0/468627431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f645010c170 0x7f645010e610 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6448009ca0 tx=0x7f644802a950 comp rx=0 tx=0).stop 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.964+0000 7f6455483640 1 -- 172.21.15.5:0/468627431 shutdown_connections 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.964+0000 7f6455483640 1 --2- 172.21.15.5:0/468627431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f645010c170 0x7f645010e610 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.964+0000 7f6455483640 1 --2- 172.21.15.5:0/468627431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450103660 0x7f645010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.964+0000 7f6455483640 1 --2- 172.21.15.5:0/468627431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6450102d40 0x7f6450103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.964+0000 7f6455483640 1 -- 172.21.15.5:0/468627431 >> 172.21.15.5:0/468627431 conn(0x7f64500fc7a0 msgr2=0x7f64500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.964+0000 7f6455483640 1 -- 172.21.15.5:0/468627431 shutdown_connections 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.964+0000 7f6455483640 1 -- 172.21.15.5:0/468627431 wait complete. 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.965+0000 7f6455483640 1 Processor -- start 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.965+0000 7f6455483640 1 -- start start 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.965+0000 7f6455483640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6450102d40 0x7f6450198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.965+0000 7f6455483640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450103660 0x7f6450198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.965+0000 7f6455483640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f645010c170 0x7f645019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.965+0000 7f6455483640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6450111f30 con 0x7f645010c170 2024-09-15T22:06:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.965+0000 7f6455483640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6450111db0 con 0x7f6450102d40 2024-09-15T22:06:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.965+0000 7f6455483640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f64501120b0 con 0x7f6450103660 2024-09-15T22:06:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.965+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f645010c170 0x7f645019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.965+0000 7f644f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f645010c170 0x7f645019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54836/0 (socket says 172.21.15.5:54836) 2024-09-15T22:06:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.965+0000 7f644f7fe640 1 -- 172.21.15.5:0/503757540 learned_addr learned my addr 172.21.15.5:0/503757540 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f6447fff640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450103660 0x7f6450198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f644effd640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6450102d40 0x7f6450198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f644f7fe640 1 -- 172.21.15.5:0/503757540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450103660 msgr2=0x7f6450198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f644f7fe640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450103660 0x7f6450198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f644f7fe640 1 -- 172.21.15.5:0/503757540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6450102d40 msgr2=0x7f6450198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f644f7fe640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6450102d40 0x7f6450198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f644f7fe640 1 -- 172.21.15.5:0/503757540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645019d3f0 con 0x7f645010c170 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f644f7fe640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f645010c170 0x7f645019cd10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f644802ae70 tx=0x7f644802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f644cff9640 1 -- 172.21.15.5:0/503757540 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448034040 con 0x7f645010c170 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f644cff9640 1 -- 172.21.15.5:0/503757540 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6448030920 con 0x7f645010c170 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f6455483640 1 -- 172.21.15.5:0/503757540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645019d680 con 0x7f645010c170 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f644cff9640 1 -- 172.21.15.5:0/503757540 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448045480 con 0x7f645010c170 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.966+0000 7f6455483640 1 -- 172.21.15.5:0/503757540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64501a4fc0 con 0x7f645010c170 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.967+0000 7f6455483640 1 -- 172.21.15.5:0/503757540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645006b070 con 0x7f645010c170 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.968+0000 7f644cff9640 1 -- 172.21.15.5:0/503757540 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f6448037020 con 0x7f645010c170 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.969+0000 7f644cff9640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f643c0a3240 0x7f643c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.969+0000 7f644effd640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f643c0a3240 0x7f643c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.969+0000 7f644cff9640 1 -- 172.21.15.5:0/503757540 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f64480f0390 con 0x7f645010c170 2024-09-15T22:06:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.969+0000 7f644effd640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f643c0a3240 0x7f643c0a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f644000bbe0 tx=0x7f644000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:28.973+0000 7f644cff9640 1 -- 172.21.15.5:0/503757540 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64480bd4b0 con 0x7f645010c170 2024-09-15T22:06:29.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.206+0000 7f6455483640 1 -- 172.21.15.5:0/503757540 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64501071e0 con 0x7f645010c170 2024-09-15T22:06:29.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.206+0000 7f644cff9640 1 -- 172.21.15.5:0/503757540 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10506 (secure 0 0 0) 0x7f64480bcdb0 con 0x7f645010c170 2024-09-15T22:06:29.209 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:29.209 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"btime":"2024-09-15T22:06:27:470352+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24491,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-15T22:06:29.209 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 13 2024-09-15T22:06:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 -- 172.21.15.5:0/503757540 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f643c0a3240 msgr2=0x7f643c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f643c0a3240 0x7f643c0a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f644000bbe0 tx=0x7f644000b390 comp rx=0 tx=0).stop 2024-09-15T22:06:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 -- 172.21.15.5:0/503757540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f645010c170 msgr2=0x7f645019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f645010c170 0x7f645019cd10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f644802ae70 tx=0x7f644802f5b0 comp rx=0 tx=0).stop 2024-09-15T22:06:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 -- 172.21.15.5:0/503757540 shutdown_connections 2024-09-15T22:06:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f643c0a3240 0x7f643c0a5700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f645010c170 0x7f645019cd10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6450103660 0x7f6450198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 --2- 172.21.15.5:0/503757540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6450102d40 0x7f6450198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:29.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 -- 172.21.15.5:0/503757540 >> 172.21.15.5:0/503757540 conn(0x7f64500fc7a0 msgr2=0x7f64500fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 -- 172.21.15.5:0/503757540 shutdown_connections 2024-09-15T22:06:29.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:29.209+0000 7f6455483640 1 -- 172.21.15.5:0/503757540 wait complete. 2024-09-15T22:06:29.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:29.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:29.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:29.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:29.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:29.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/503757540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:29.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:29.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:29.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:29.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:29.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:29.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:29 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:29.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-15T22:06:30.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:30.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/503757540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:30.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:30.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:30.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.091 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:29 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:30.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:30.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/503757540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:30.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:30.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:30.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:30.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:29 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:06:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.347+0000 7ff30a916640 1 -- 172.21.15.5:0/2982713014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3041082a0 msgr2=0x7ff30410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.347+0000 7ff30a916640 1 --2- 172.21.15.5:0/2982713014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3041082a0 0x7ff30410ec40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009cd0 tx=0x7ff2fc02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.347+0000 7ff30a916640 1 -- 172.21.15.5:0/2982713014 shutdown_connections 2024-09-15T22:06:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.347+0000 7ff30a916640 1 --2- 172.21.15.5:0/2982713014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3041082a0 0x7ff30410ec40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.347+0000 7ff30a916640 1 --2- 172.21.15.5:0/2982713014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3041036e0 0x7ff304107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.347+0000 7ff30a916640 1 --2- 172.21.15.5:0/2982713014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff304102d30 0x7ff304103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.347+0000 7ff30a916640 1 -- 172.21.15.5:0/2982713014 >> 172.21.15.5:0/2982713014 conn(0x7ff3040fc7d0 msgr2=0x7ff3040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.347+0000 7ff30a916640 1 -- 172.21.15.5:0/2982713014 shutdown_connections 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.347+0000 7ff30a916640 1 -- 172.21.15.5:0/2982713014 wait complete. 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.347+0000 7ff30a916640 1 Processor -- start 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.347+0000 7ff30a916640 1 -- start start 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff30a916640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff304102d30 0x7ff304198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff30a916640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3041036e0 0x7ff304198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff30a916640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3041082a0 0x7ff30419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff30a916640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041120a0 con 0x7ff304102d30 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff30a916640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff304111f20 con 0x7ff3041036e0 2024-09-15T22:06:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff30a916640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff304112220 con 0x7ff3041082a0 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff303fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff304102d30 0x7ff304198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff303fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff304102d30 0x7ff304198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54856/0 (socket says 172.21.15.5:54856) 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff303fff640 1 -- 172.21.15.5:0/913017116 learned_addr learned my addr 172.21.15.5:0/913017116 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff3037fe640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3041036e0 0x7ff304198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff308eb3640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3041082a0 0x7ff30419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff303fff640 1 -- 172.21.15.5:0/913017116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3041082a0 msgr2=0x7ff30419ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff303fff640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3041082a0 0x7ff30419ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff303fff640 1 -- 172.21.15.5:0/913017116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3041036e0 msgr2=0x7ff304198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff303fff640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3041036e0 0x7ff304198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff303fff640 1 -- 172.21.15.5:0/913017116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30419d520 con 0x7ff304102d30 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff303fff640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff304102d30 0x7ff304198570 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400e9f0 tx=0x7ff2f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.348+0000 7ff3017fa640 1 -- 172.21.15.5:0/913017116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400cd80 con 0x7ff304102d30 2024-09-15T22:06:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.349+0000 7ff30a916640 1 -- 172.21.15.5:0/913017116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30419d810 con 0x7ff304102d30 2024-09-15T22:06:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.349+0000 7ff3017fa640 1 -- 172.21.15.5:0/913017116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f4005500 con 0x7ff304102d30 2024-09-15T22:06:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.349+0000 7ff3017fa640 1 -- 172.21.15.5:0/913017116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4010690 con 0x7ff304102d30 2024-09-15T22:06:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.349+0000 7ff30a916640 1 -- 172.21.15.5:0/913017116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3041a50f0 con 0x7ff304102d30 2024-09-15T22:06:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.349+0000 7ff30a916640 1 -- 172.21.15.5:0/913017116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff304103110 con 0x7ff304102d30 2024-09-15T22:06:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.351+0000 7ff3017fa640 1 -- 172.21.15.5:0/913017116 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7ff2f40046c0 con 0x7ff304102d30 2024-09-15T22:06:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.351+0000 7ff3017fa640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff2ec0a3240 0x7ff2ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.351+0000 7ff3037fe640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff2ec0a3240 0x7ff2ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.351+0000 7ff3017fa640 1 -- 172.21.15.5:0/913017116 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7ff2f40cf900 con 0x7ff304102d30 2024-09-15T22:06:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.352+0000 7ff3037fe640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff2ec0a3240 0x7ff2ec0a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3040fede0 tx=0x7ff2f800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.355+0000 7ff3017fa640 1 -- 172.21.15.5:0/913017116 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2f409cac0 con 0x7ff304102d30 2024-09-15T22:06:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.561+0000 7ff30a916640 1 -- 172.21.15.5:0/913017116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff3041998d0 con 0x7ff304102d30 2024-09-15T22:06:31.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.562+0000 7ff3017fa640 1 -- 172.21.15.5:0/913017116 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7ff2f409c3c0 con 0x7ff304102d30 2024-09-15T22:06:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:31.564 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-09-15T22:06:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:06:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-09-15T22:06:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-09-15T22:06:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:06:31.564 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-09-15T22:06:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.564+0000 7ff30a916640 1 -- 172.21.15.5:0/913017116 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff2ec0a3240 msgr2=0x7ff2ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.564+0000 7ff30a916640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff2ec0a3240 0x7ff2ec0a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff3040fede0 tx=0x7ff2f800b390 comp rx=0 tx=0).stop 2024-09-15T22:06:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.564+0000 7ff30a916640 1 -- 172.21.15.5:0/913017116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff304102d30 msgr2=0x7ff304198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.564+0000 7ff30a916640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff304102d30 0x7ff304198570 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400e9f0 tx=0x7ff2f400edf0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.564+0000 7ff30a916640 1 -- 172.21.15.5:0/913017116 shutdown_connections 2024-09-15T22:06:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.564+0000 7ff30a916640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3041082a0 0x7ff30419ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.564+0000 7ff30a916640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3041036e0 0x7ff304198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.564+0000 7ff30a916640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff2ec0a3240 0x7ff2ec0a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.564+0000 7ff30a916640 1 --2- 172.21.15.5:0/913017116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff304102d30 0x7ff304198570 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.564+0000 7ff30a916640 1 -- 172.21.15.5:0/913017116 >> 172.21.15.5:0/913017116 conn(0x7ff3040fc7d0 msgr2=0x7ff304105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.565+0000 7ff30a916640 1 -- 172.21.15.5:0/913017116 shutdown_connections 2024-09-15T22:06:31.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:31.565+0000 7ff30a916640 1 -- 172.21.15.5:0/913017116 wait complete. 2024-09-15T22:06:32.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:32 smithi005 ceph-mon[30944]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:32.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:32 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/913017116' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:06:32.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:32.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:32.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:32 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.277 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-15T22:06:32.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-15T22:06:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:32.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:32 smithi099 ceph-mon[34892]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:32.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:32 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/913017116' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:06:32.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:32.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:32.591 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:32 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:32 smithi129 ceph-mon[35314]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:32.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:32 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/913017116' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-15T22:06:32.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:32.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:06:32.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:06:32.596 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:32 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:06:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:32.999+0000 7ff9b6114640 1 -- 172.21.15.5:0/2602647058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b0102e00 msgr2=0x7ff9b01031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:32.999+0000 7ff9b6114640 1 --2- 172.21.15.5:0/2602647058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b0102e00 0x7ff9b01031e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009900 tx=0x7ff9a002a4a0 comp rx=0 tx=0).stop 2024-09-15T22:06:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:32.999+0000 7ff9b6114640 1 -- 172.21.15.5:0/2602647058 shutdown_connections 2024-09-15T22:06:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:32.999+0000 7ff9b6114640 1 --2- 172.21.15.5:0/2602647058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9b010c1a0 0x7ff9b010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:32.999+0000 7ff9b6114640 1 --2- 172.21.15.5:0/2602647058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9b0103720 0x7ff9b010bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:32.999+0000 7ff9b6114640 1 --2- 172.21.15.5:0/2602647058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b0102e00 0x7ff9b01031e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:32.999+0000 7ff9b6114640 1 -- 172.21.15.5:0/2602647058 >> 172.21.15.5:0/2602647058 conn(0x7ff9b00fc7a0 msgr2=0x7ff9b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:32.999+0000 7ff9b6114640 1 -- 172.21.15.5:0/2602647058 shutdown_connections 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:32.999+0000 7ff9b6114640 1 -- 172.21.15.5:0/2602647058 wait complete. 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9b6114640 1 Processor -- start 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9b6114640 1 -- start start 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9b6114640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9b0102e00 0x7ff9b01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9b6114640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9b0103720 0x7ff9b0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9b6114640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b010c1a0 0x7ff9b019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9b6114640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b0112050 con 0x7ff9b010c1a0 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9b6114640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b0111ed0 con 0x7ff9b0102e00 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9b6114640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b01121d0 con 0x7ff9b0103720 2024-09-15T22:06:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b010c1a0 0x7ff9b019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b010c1a0 0x7ff9b019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54880/0 (socket says 172.21.15.5:54880) 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9affff640 1 -- 172.21.15.5:0/1188051022 learned_addr learned my addr 172.21.15.5:0/1188051022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.000+0000 7ff9af7fe640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9b0102e00 0x7ff9b01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9aeffd640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9b0103720 0x7ff9b0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9affff640 1 -- 172.21.15.5:0/1188051022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9b0103720 msgr2=0x7ff9b0198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9affff640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9b0103720 0x7ff9b0198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9affff640 1 -- 172.21.15.5:0/1188051022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9b0102e00 msgr2=0x7ff9b01985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9affff640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9b0102e00 0x7ff9b01985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9affff640 1 -- 172.21.15.5:0/1188051022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b019d580 con 0x7ff9b010c1a0 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9aeffd640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9b0103720 0x7ff9b0198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9af7fe640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9b0102e00 0x7ff9b01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9affff640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b010c1a0 0x7ff9b019cea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800b990 tx=0x7ff9a800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9acff9640 1 -- 172.21.15.5:0/1188051022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8019070 con 0x7ff9b010c1a0 2024-09-15T22:06:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9acff9640 1 -- 172.21.15.5:0/1188051022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9a800cba0 con 0x7ff9b010c1a0 2024-09-15T22:06:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9b6114640 1 -- 172.21.15.5:0/1188051022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b019d870 con 0x7ff9b010c1a0 2024-09-15T22:06:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9acff9640 1 -- 172.21.15.5:0/1188051022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8007650 con 0x7ff9b010c1a0 2024-09-15T22:06:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.001+0000 7ff9b6114640 1 -- 172.21.15.5:0/1188051022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b01a5150 con 0x7ff9b010c1a0 2024-09-15T22:06:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.002+0000 7ff9b6114640 1 -- 172.21.15.5:0/1188051022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b0103260 con 0x7ff9b010c1a0 2024-09-15T22:06:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.003+0000 7ff9acff9640 1 -- 172.21.15.5:0/1188051022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7ff9a801c070 con 0x7ff9b010c1a0 2024-09-15T22:06:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.004+0000 7ff9acff9640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff9980a32c0 0x7ff9980a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.004+0000 7ff9acff9640 1 -- 172.21.15.5:0/1188051022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7ff9a80cea80 con 0x7ff9b010c1a0 2024-09-15T22:06:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.004+0000 7ff9af7fe640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff9980a32c0 0x7ff9980a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.004+0000 7ff9af7fe640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff9980a32c0 0x7ff9980a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0005940 tx=0x7ff9a00058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.008+0000 7ff9acff9640 1 -- 172.21.15.5:0/1188051022 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9a809bc40 con 0x7ff9b010c1a0 2024-09-15T22:06:33.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:33.184+0000 7ff9b6114640 1 -- 172.21.15.5:0/1188051022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff9b002d7c0 con 0x7ff9b010c1a0 2024-09-15T22:06:34.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.117+0000 7ff9acff9640 1 -- 172.21.15.5:0/1188051022 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7ff9a809b540 con 0x7ff9b010c1a0 2024-09-15T22:06:34.121 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 -- 172.21.15.5:0/1188051022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff9980a32c0 msgr2=0x7ff9980a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff9980a32c0 0x7ff9980a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0005940 tx=0x7ff9a00058b0 comp rx=0 tx=0).stop 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 -- 172.21.15.5:0/1188051022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b010c1a0 msgr2=0x7ff9b019cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b010c1a0 0x7ff9b019cea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800b990 tx=0x7ff9a800be50 comp rx=0 tx=0).stop 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 -- 172.21.15.5:0/1188051022 shutdown_connections 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff9980a32c0 0x7ff9980a5780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9b010c1a0 0x7ff9b019cea0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff9b0103720 0x7ff9b0198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 --2- 172.21.15.5:0/1188051022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff9b0102e00 0x7ff9b01985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 -- 172.21.15.5:0/1188051022 >> 172.21.15.5:0/1188051022 conn(0x7ff9b00fc7a0 msgr2=0x7ff9b00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 -- 172.21.15.5:0/1188051022 shutdown_connections 2024-09-15T22:06:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:34.120+0000 7ff9b6114640 1 -- 172.21.15.5:0/1188051022 wait complete. 2024-09-15T22:06:34.427 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:34 smithi005 ceph-mon[30944]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:34.427 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:34 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1188051022' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:06:34.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:34 smithi099 ceph-mon[34892]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:34.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:34 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1188051022' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:06:34.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:34 smithi129 ceph-mon[35314]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:34.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:34 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1188051022' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-15T22:06:34.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-15T22:06:35.017 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:35.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:35 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1188051022' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:06:35.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:35 smithi005 ceph-mon[30944]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T22:06:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.545+0000 7f0a37e8c640 1 -- 172.21.15.5:0/3414391868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30102d40 msgr2=0x7f0a30103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.545+0000 7f0a37e8c640 1 --2- 172.21.15.5:0/3414391868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30102d40 0x7f0a30103120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c009900 tx=0x7f0a2c02a480 comp rx=0 tx=0).stop 2024-09-15T22:06:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.545+0000 7f0a37e8c640 1 -- 172.21.15.5:0/3414391868 shutdown_connections 2024-09-15T22:06:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.545+0000 7f0a37e8c640 1 --2- 172.21.15.5:0/3414391868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3010c170 0x7f0a3010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:35.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.545+0000 7f0a37e8c640 1 --2- 172.21.15.5:0/3414391868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a30103660 0x7f0a3010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.545+0000 7f0a37e8c640 1 --2- 172.21.15.5:0/3414391868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30102d40 0x7f0a30103120 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.545+0000 7f0a37e8c640 1 -- 172.21.15.5:0/3414391868 >> 172.21.15.5:0/3414391868 conn(0x7f0a300fc7a0 msgr2=0x7f0a300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.545+0000 7f0a37e8c640 1 -- 172.21.15.5:0/3414391868 shutdown_connections 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.545+0000 7f0a37e8c640 1 -- 172.21.15.5:0/3414391868 wait complete. 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a37e8c640 1 Processor -- start 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a37e8c640 1 -- start start 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a37e8c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a30102d40 0x7f0a301983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a37e8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30103660 0x7f0a30198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a37e8c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3010c170 0x7f0a3019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a37e8c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a30111f70 con 0x7f0a30103660 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a37e8c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a30111df0 con 0x7f0a3010c170 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a37e8c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a301120f0 con 0x7f0a30102d40 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a35427640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30103660 0x7f0a30198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:35.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a35427640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30103660 0x7f0a30198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46946/0 (socket says 172.21.15.5:46946) 2024-09-15T22:06:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a35427640 1 -- 172.21.15.5:0/4218061911 learned_addr learned my addr 172.21.15.5:0/4218061911 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a35c28640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a30102d40 0x7f0a301983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a36429640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3010c170 0x7f0a3019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a35427640 1 -- 172.21.15.5:0/4218061911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a30102d40 msgr2=0x7f0a301983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a35427640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a30102d40 0x7f0a301983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a35427640 1 -- 172.21.15.5:0/4218061911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3010c170 msgr2=0x7f0a3019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.546+0000 7f0a35427640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3010c170 0x7f0a3019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.547+0000 7f0a35427640 1 -- 172.21.15.5:0/4218061911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3019d380 con 0x7f0a30103660 2024-09-15T22:06:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.547+0000 7f0a36429640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3010c170 0x7f0a3019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.547+0000 7f0a35c28640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a30102d40 0x7f0a301983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:06:35.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.547+0000 7f0a35427640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30103660 0x7f0a30198910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c970 tx=0x7f0a2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.547+0000 7f0a26ffd640 1 -- 172.21.15.5:0/4218061911 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20007bb0 con 0x7f0a30103660 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.547+0000 7f0a37e8c640 1 -- 172.21.15.5:0/4218061911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3019d670 con 0x7f0a30103660 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.547+0000 7f0a26ffd640 1 -- 172.21.15.5:0/4218061911 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a20007d50 con 0x7f0a30103660 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.547+0000 7f0a26ffd640 1 -- 172.21.15.5:0/4218061911 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20018820 con 0x7f0a30103660 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.547+0000 7f0a37e8c640 1 -- 172.21.15.5:0/4218061911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a301a4f50 con 0x7f0a30103660 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.548+0000 7f0a37e8c640 1 -- 172.21.15.5:0/4218061911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a30103120 con 0x7f0a30103660 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.549+0000 7f0a26ffd640 1 -- 172.21.15.5:0/4218061911 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f0a200189c0 con 0x7f0a30103660 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.550+0000 7f0a26ffd640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0a1c0a3240 0x7f0a1c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.550+0000 7f0a35c28640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0a1c0a3240 0x7f0a1c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.550+0000 7f0a26ffd640 1 -- 172.21.15.5:0/4218061911 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7f0a200cfa80 con 0x7f0a30103660 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.551+0000 7f0a35c28640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0a1c0a3240 0x7f0a1c0a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c005ab0 tx=0x7f0a2c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:35.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.554+0000 7f0a26ffd640 1 -- 172.21.15.5:0/4218061911 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a20014030 con 0x7f0a30103660 2024-09-15T22:06:35.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:35 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1188051022' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:06:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:35 smithi099 ceph-mon[34892]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T22:06:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:35 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1188051022' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-15T22:06:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:35 smithi129 ceph-mon[35314]: osdmap e76: 12 total, 12 up, 12 in 2024-09-15T22:06:35.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:35.729+0000 7f0a37e8c640 1 -- 172.21.15.5:0/4218061911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f0a3002d7c0 con 0x7f0a30103660 2024-09-15T22:06:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.163+0000 7f0a26ffd640 1 -- 172.21.15.5:0/4218061911 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v78) ==== 171+0+0 (secure 0 0 0) 0x7f0a2009cac0 con 0x7f0a30103660 2024-09-15T22:06:36.164 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-09-15T22:06:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 -- 172.21.15.5:0/4218061911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0a1c0a3240 msgr2=0x7f0a1c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0a1c0a3240 0x7f0a1c0a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c005ab0 tx=0x7f0a2c005a20 comp rx=0 tx=0).stop 2024-09-15T22:06:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 -- 172.21.15.5:0/4218061911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30103660 msgr2=0x7f0a30198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30103660 0x7f0a30198910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c970 tx=0x7f0a2000ce30 comp rx=0 tx=0).stop 2024-09-15T22:06:36.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 -- 172.21.15.5:0/4218061911 shutdown_connections 2024-09-15T22:06:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a3010c170 0x7f0a3019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f0a1c0a3240 0x7f0a1c0a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30103660 0x7f0a30198910 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000c970 tx=0x7f0a2000ce30 comp rx=0 tx=0).stop 2024-09-15T22:06:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 --2- 172.21.15.5:0/4218061911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a30102d40 0x7f0a301983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 -- 172.21.15.5:0/4218061911 >> 172.21.15.5:0/4218061911 conn(0x7f0a300fc7a0 msgr2=0x7f0a300fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 -- 172.21.15.5:0/4218061911 shutdown_connections 2024-09-15T22:06:36.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:36.165+0000 7f0a37e8c640 1 -- 172.21.15.5:0/4218061911 wait complete. 2024-09-15T22:06:36.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:36 smithi005 ceph-mon[30944]: pgmap v272: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:36.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:36 smithi005 ceph-mon[30944]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:06:36.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:36 smithi005 ceph-mon[30944]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T22:06:36.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:36 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4218061911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-15T22:06:36.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:36 smithi099 ceph-mon[34892]: pgmap v272: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:36.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:36 smithi099 ceph-mon[34892]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:06:36.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:36 smithi099 ceph-mon[34892]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T22:06:36.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:36 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4218061911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-15T22:06:36.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:36 smithi129 ceph-mon[35314]: pgmap v272: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:36.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:36 smithi129 ceph-mon[35314]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:06:36.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:36 smithi129 ceph-mon[35314]: osdmap e77: 12 total, 12 up, 12 in 2024-09-15T22:06:36.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:36 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4218061911' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-15T22:06:36.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-15T22:06:37.158 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:37.446 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:37 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4218061911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-15T22:06:37.446 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:37 smithi005 ceph-mon[30944]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:06:37.446 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:37 smithi005 ceph-mon[30944]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:06:37.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:37 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4218061911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-15T22:06:37.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:37 smithi099 ceph-mon[34892]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:06:37.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:37 smithi099 ceph-mon[34892]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:06:37.598 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:37 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4218061911' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-15T22:06:37.598 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:37 smithi129 ceph-mon[35314]: osdmap e78: 12 total, 12 up, 12 in 2024-09-15T22:06:37.598 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:37 smithi129 ceph-mon[35314]: osdmap e79: 12 total, 12 up, 12 in 2024-09-15T22:06:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.632+0000 7f22852ab640 1 -- 172.21.15.5:0/202885980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2280101de0 msgr2=0x7f22801021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.632+0000 7f22852ab640 1 --2- 172.21.15.5:0/202885980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2280101de0 0x7f22801021c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2270009900 tx=0x7f227002a410 comp rx=0 tx=0).stop 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.632+0000 7f22852ab640 1 -- 172.21.15.5:0/202885980 shutdown_connections 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.632+0000 7f22852ab640 1 --2- 172.21.15.5:0/202885980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2280106f10 0x7f228010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.632+0000 7f22852ab640 1 --2- 172.21.15.5:0/202885980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2280102790 0x7f22801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.632+0000 7f22852ab640 1 --2- 172.21.15.5:0/202885980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2280101de0 0x7f22801021c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.632+0000 7f22852ab640 1 -- 172.21.15.5:0/202885980 >> 172.21.15.5:0/202885980 conn(0x7f22800fd8c0 msgr2=0x7f22800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.632+0000 7f22852ab640 1 -- 172.21.15.5:0/202885980 shutdown_connections 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.632+0000 7f22852ab640 1 -- 172.21.15.5:0/202885980 wait complete. 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.633+0000 7f22852ab640 1 Processor -- start 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.633+0000 7f22852ab640 1 -- start start 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.633+0000 7f22852ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2280101de0 0x7f22801985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.633+0000 7f22852ab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2280102790 0x7f2280198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:37.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.633+0000 7f22852ab640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2280106f10 0x7f228019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.633+0000 7f22852ab640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2280112060 con 0x7f2280101de0 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.633+0000 7f22852ab640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2280111ee0 con 0x7f2280102790 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.633+0000 7f22852ab640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f22801121e0 con 0x7f2280106f10 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.633+0000 7f227effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2280101de0 0x7f22801985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.633+0000 7f227effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2280101de0 0x7f22801985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46966/0 (socket says 172.21.15.5:46966) 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.633+0000 7f227effd640 1 -- 172.21.15.5:0/3668504567 learned_addr learned my addr 172.21.15.5:0/3668504567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f227f7fe640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2280106f10 0x7f228019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f227e7fc640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2280102790 0x7f2280198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f227effd640 1 -- 172.21.15.5:0/3668504567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2280106f10 msgr2=0x7f228019ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f227effd640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2280106f10 0x7f228019ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f227effd640 1 -- 172.21.15.5:0/3668504567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2280102790 msgr2=0x7f2280198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:37.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f227effd640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2280102790 0x7f2280198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f227effd640 1 -- 172.21.15.5:0/3668504567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228019d570 con 0x7f2280101de0 2024-09-15T22:06:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f227e7fc640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2280102790 0x7f2280198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f227effd640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2280101de0 0x7f22801985c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2270006010 tx=0x7f2270003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f2267fff640 1 -- 172.21.15.5:0/3668504567 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227003f070 con 0x7f2280101de0 2024-09-15T22:06:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f22852ab640 1 -- 172.21.15.5:0/3668504567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228019d800 con 0x7f2280101de0 2024-09-15T22:06:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f2267fff640 1 -- 172.21.15.5:0/3668504567 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2270031c90 con 0x7f2280101de0 2024-09-15T22:06:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f2267fff640 1 -- 172.21.15.5:0/3668504567 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2270035030 con 0x7f2280101de0 2024-09-15T22:06:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.634+0000 7f22852ab640 1 -- 172.21.15.5:0/3668504567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22801a5140 con 0x7f2280101de0 2024-09-15T22:06:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.635+0000 7f22852ab640 1 -- 172.21.15.5:0/3668504567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22801032c0 con 0x7f2280101de0 2024-09-15T22:06:37.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.637+0000 7f2267fff640 1 -- 172.21.15.5:0/3668504567 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f2270042070 con 0x7f2280101de0 2024-09-15T22:06:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.637+0000 7f2267fff640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f22680a3240 0x7f22680a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.637+0000 7f2267fff640 1 -- 172.21.15.5:0/3668504567 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f22700f0530 con 0x7f2280101de0 2024-09-15T22:06:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.637+0000 7f227e7fc640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f22680a3240 0x7f22680a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.638+0000 7f227e7fc640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f22680a3240 0x7f22680a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2274006fd0 tx=0x7f2274008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:37.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.641+0000 7f2267fff640 1 -- 172.21.15.5:0/3668504567 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22700bd400 con 0x7f2280101de0 2024-09-15T22:06:37.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:37.853+0000 7f22852ab640 1 -- 172.21.15.5:0/3668504567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2280102240 con 0x7f2280101de0 2024-09-15T22:06:38.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.187+0000 7f2267fff640 1 -- 172.21.15.5:0/3668504567 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f2280102240 con 0x7f2280101de0 2024-09-15T22:06:38.189 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-09-15T22:06:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.190+0000 7f22852ab640 1 -- 172.21.15.5:0/3668504567 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f22680a3240 msgr2=0x7f22680a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.190+0000 7f22852ab640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f22680a3240 0x7f22680a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2274006fd0 tx=0x7f2274008040 comp rx=0 tx=0).stop 2024-09-15T22:06:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.190+0000 7f22852ab640 1 -- 172.21.15.5:0/3668504567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2280101de0 msgr2=0x7f22801985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.190+0000 7f22852ab640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2280101de0 0x7f22801985c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2270006010 tx=0x7f2270003ba0 comp rx=0 tx=0).stop 2024-09-15T22:06:38.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.190+0000 7f22852ab640 1 -- 172.21.15.5:0/3668504567 shutdown_connections 2024-09-15T22:06:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.190+0000 7f22852ab640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2280106f10 0x7f228019ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.190+0000 7f22852ab640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2280102790 0x7f2280198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.190+0000 7f22852ab640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f22680a3240 0x7f22680a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.190+0000 7f22852ab640 1 --2- 172.21.15.5:0/3668504567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2280101de0 0x7f22801985c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.191+0000 7f22852ab640 1 -- 172.21.15.5:0/3668504567 >> 172.21.15.5:0/3668504567 conn(0x7f22800fd8c0 msgr2=0x7f22800fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.191+0000 7f22852ab640 1 -- 172.21.15.5:0/3668504567 shutdown_connections 2024-09-15T22:06:38.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:38.191+0000 7f22852ab640 1 -- 172.21.15.5:0/3668504567 wait complete. 2024-09-15T22:06:38.467 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:38 smithi099 ceph-mon[34892]: pgmap v275: 97 pgs: 6 creating+peering, 6 active+clean, 85 unknown; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:38.467 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:38 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3668504567' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:06:38.467 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:38 smithi099 ceph-mon[34892]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:06:38.530 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:38 smithi005 ceph-mon[30944]: pgmap v275: 97 pgs: 6 creating+peering, 6 active+clean, 85 unknown; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:38.530 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:38 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3668504567' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:06:38.530 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:38 smithi005 ceph-mon[30944]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:06:38.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:38 smithi129 ceph-mon[35314]: pgmap v275: 97 pgs: 6 creating+peering, 6 active+clean, 85 unknown; 577 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:38.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:38 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3668504567' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-15T22:06:38.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:38 smithi129 ceph-mon[35314]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-15T22:06:38.855 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-15T22:06:38.855 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-15T22:06:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:39 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3668504567' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:06:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:39 smithi005 ceph-mon[30944]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:06:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:39 smithi005 ceph-mon[30944]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:06:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:39 smithi005 ceph-mon[30944]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:06:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:39 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] up:active 2024-09-15T22:06:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:39 smithi005 ceph-mon[30944]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:06:39.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:39 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3668504567' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:06:39.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:39 smithi099 ceph-mon[34892]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:06:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:39 smithi099 ceph-mon[34892]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:06:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:39 smithi099 ceph-mon[34892]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:06:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:39 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] up:active 2024-09-15T22:06:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:39 smithi099 ceph-mon[34892]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:06:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:39 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3668504567' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-15T22:06:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:39 smithi129 ceph-mon[35314]: osdmap e80: 12 total, 12 up, 12 in 2024-09-15T22:06:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:39 smithi129 ceph-mon[35314]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-15T22:06:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:39 smithi129 ceph-mon[35314]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-15T22:06:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:39 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] up:active 2024-09-15T22:06:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:39 smithi129 ceph-mon[35314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:06:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.642+0000 7f463f228640 1 -- 172.21.15.5:0/735008780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463810c2f0 msgr2=0x7f463810e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.642+0000 7f463f228640 1 --2- 172.21.15.5:0/735008780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463810c2f0 0x7f463810e790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4630009ca0 tx=0x7f463002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:06:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.642+0000 7f463f228640 1 -- 172.21.15.5:0/735008780 shutdown_connections 2024-09-15T22:06:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.642+0000 7f463f228640 1 --2- 172.21.15.5:0/735008780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463810c2f0 0x7f463810e790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.642+0000 7f463f228640 1 --2- 172.21.15.5:0/735008780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46381037a0 0x7f463810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:39.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.642+0000 7f463f228640 1 --2- 172.21.15.5:0/735008780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4638102df0 0x7f46381031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.642+0000 7f463f228640 1 -- 172.21.15.5:0/735008780 >> 172.21.15.5:0/735008780 conn(0x7f46380fc7d0 msgr2=0x7f46380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.642+0000 7f463f228640 1 -- 172.21.15.5:0/735008780 shutdown_connections 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.642+0000 7f463f228640 1 -- 172.21.15.5:0/735008780 wait complete. 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463f228640 1 Processor -- start 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463f228640 1 -- start start 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463f228640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4638102df0 0x7f4638100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463f228640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46381037a0 0x7f4638100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463f228640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463810c2f0 0x7f46381014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463f228640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4638112110 con 0x7f463810c2f0 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463f228640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4638111f90 con 0x7f4638102df0 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463f228640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4638112290 con 0x7f46381037a0 2024-09-15T22:06:39.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463d7c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463810c2f0 0x7f46381014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463d7c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463810c2f0 0x7f46381014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46996/0 (socket says 172.21.15.5:46996) 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463d7c5640 1 -- 172.21.15.5:0/1490350349 learned_addr learned my addr 172.21.15.5:0/1490350349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.643+0000 7f463cfc4640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4638102df0 0x7f4638100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f4637fff640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46381037a0 0x7f4638100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f463d7c5640 1 -- 172.21.15.5:0/1490350349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46381037a0 msgr2=0x7f4638100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f463d7c5640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46381037a0 0x7f4638100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f463d7c5640 1 -- 172.21.15.5:0/1490350349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4638102df0 msgr2=0x7f4638100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f463d7c5640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4638102df0 0x7f4638100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f463d7c5640 1 -- 172.21.15.5:0/1490350349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46381aaf40 con 0x7f463810c2f0 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f463d7c5640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463810c2f0 0x7f46381014d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4630031a20 tx=0x7f4630031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f4635ffb640 1 -- 172.21.15.5:0/1490350349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630003c30 con 0x7f463810c2f0 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f463f228640 1 -- 172.21.15.5:0/1490350349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46381ab170 con 0x7f463810c2f0 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f4635ffb640 1 -- 172.21.15.5:0/1490350349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4630003dd0 con 0x7f463810c2f0 2024-09-15T22:06:39.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f4635ffb640 1 -- 172.21.15.5:0/1490350349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463002fcf0 con 0x7f463810c2f0 2024-09-15T22:06:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.644+0000 7f463f228640 1 -- 172.21.15.5:0/1490350349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46381ab700 con 0x7f463810c2f0 2024-09-15T22:06:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.645+0000 7f463f228640 1 -- 172.21.15.5:0/1490350349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463806a790 con 0x7f463810c2f0 2024-09-15T22:06:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.646+0000 7f4635ffb640 1 -- 172.21.15.5:0/1490350349 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f4630004050 con 0x7f463810c2f0 2024-09-15T22:06:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.647+0000 7f4635ffb640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46280a3240 0x7f46280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.647+0000 7f463cfc4640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46280a3240 0x7f46280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.647+0000 7f4635ffb640 1 -- 172.21.15.5:0/1490350349 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f46300f0cf0 con 0x7f463810c2f0 2024-09-15T22:06:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.648+0000 7f463cfc4640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46280a3240 0x7f46280a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f462c0058c0 tx=0x7f462c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.650+0000 7f4635ffb640 1 -- 172.21.15.5:0/1490350349 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46300bdbf0 con 0x7f463810c2f0 2024-09-15T22:06:39.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:39.833+0000 7f463f228640 1 -- 172.21.15.5:0/1490350349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f4638101f80 con 0x7f463810c2f0 2024-09-15T22:06:40.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:40 smithi099 ceph-mon[34892]: pgmap v278: 97 pgs: 6 creating+peering, 62 active+clean, 29 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-09-15T22:06:40.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:40 smithi099 ceph-mon[34892]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:06:40.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:40 smithi099 ceph-mon[34892]: Cluster is now healthy 2024-09-15T22:06:40.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:40 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1490350349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-15T22:06:40.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:40 smithi129 ceph-mon[35314]: pgmap v278: 97 pgs: 6 creating+peering, 62 active+clean, 29 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-09-15T22:06:40.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:40 smithi129 ceph-mon[35314]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:06:40.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:40 smithi129 ceph-mon[35314]: Cluster is now healthy 2024-09-15T22:06:40.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:40 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1490350349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-15T22:06:40.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:40 smithi005 ceph-mon[30944]: pgmap v278: 97 pgs: 6 creating+peering, 62 active+clean, 29 unknown; 577 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-09-15T22:06:40.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:40 smithi005 ceph-mon[30944]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-15T22:06:40.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:40 smithi005 ceph-mon[30944]: Cluster is now healthy 2024-09-15T22:06:40.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:40 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1490350349' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-15T22:06:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.623+0000 7f4635ffb640 1 -- 172.21.15.5:0/1490350349 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v81) ==== 157+0+0 (secure 0 0 0) 0x7f463002fe90 con 0x7f463810c2f0 2024-09-15T22:06:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 -- 172.21.15.5:0/1490350349 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46280a3240 msgr2=0x7f46280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46280a3240 0x7f46280a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f462c0058c0 tx=0x7f462c00a250 comp rx=0 tx=0).stop 2024-09-15T22:06:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 -- 172.21.15.5:0/1490350349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463810c2f0 msgr2=0x7f46381014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463810c2f0 0x7f46381014d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4630031a20 tx=0x7f4630031a50 comp rx=0 tx=0).stop 2024-09-15T22:06:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 -- 172.21.15.5:0/1490350349 shutdown_connections 2024-09-15T22:06:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46280a3240 0x7f46280a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463810c2f0 0x7f46381014d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46381037a0 0x7f4638100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 --2- 172.21.15.5:0/1490350349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4638102df0 0x7f4638100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 -- 172.21.15.5:0/1490350349 >> 172.21.15.5:0/1490350349 conn(0x7f46380fc7d0 msgr2=0x7f46380fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 -- 172.21.15.5:0/1490350349 shutdown_connections 2024-09-15T22:06:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:40.627+0000 7f463f228640 1 -- 172.21.15.5:0/1490350349 wait complete. 2024-09-15T22:06:41.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-15T22:06:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:41.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:41 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1490350349' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-15T22:06:41.930 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:41 smithi005 ceph-mon[30944]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:06:42.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:41 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1490350349' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-15T22:06:42.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:41 smithi099 ceph-mon[34892]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:06:42.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:41 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1490350349' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-15T22:06:42.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:41 smithi129 ceph-mon[35314]: osdmap e81: 12 total, 12 up, 12 in 2024-09-15T22:06:42.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.165+0000 7f5626d17640 1 -- 172.21.15.5:0/2319712681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620102790 msgr2=0x7f56201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:42.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.165+0000 7f5626d17640 1 --2- 172.21.15.5:0/2319712681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620102790 0x7f56201067e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f560c009900 tx=0x7f560c02a410 comp rx=0 tx=0).stop 2024-09-15T22:06:42.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.165+0000 7f5626d17640 1 -- 172.21.15.5:0/2319712681 shutdown_connections 2024-09-15T22:06:42.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.165+0000 7f5626d17640 1 --2- 172.21.15.5:0/2319712681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5620106f10 0x7f562010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.165+0000 7f5626d17640 1 --2- 172.21.15.5:0/2319712681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620102790 0x7f56201067e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.165+0000 7f5626d17640 1 --2- 172.21.15.5:0/2319712681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620101de0 0x7f56201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.165+0000 7f5626d17640 1 -- 172.21.15.5:0/2319712681 >> 172.21.15.5:0/2319712681 conn(0x7f56200fd8c0 msgr2=0x7f56200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.165+0000 7f5626d17640 1 -- 172.21.15.5:0/2319712681 shutdown_connections 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.165+0000 7f5626d17640 1 -- 172.21.15.5:0/2319712681 wait complete. 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.166+0000 7f5626d17640 1 Processor -- start 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.166+0000 7f5626d17640 1 -- start start 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.166+0000 7f5626d17640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620101de0 0x7f56200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.166+0000 7f5626d17640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620102790 0x7f5620074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.166+0000 7f5626d17640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5620106f10 0x7f5620074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.166+0000 7f5626d17640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5620112010 con 0x7f5620101de0 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.166+0000 7f5626d17640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5620111e90 con 0x7f5620102790 2024-09-15T22:06:42.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.166+0000 7f5626d17640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5620112190 con 0x7f5620106f10 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.166+0000 7f5624ab3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620101de0 0x7f56200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.166+0000 7f5624ab3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620101de0 0x7f56200744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47014/0 (socket says 172.21.15.5:47014) 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.166+0000 7f5624ab3640 1 -- 172.21.15.5:0/366910704 learned_addr learned my addr 172.21.15.5:0/366910704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.167+0000 7f5624ab3640 1 -- 172.21.15.5:0/366910704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5620106f10 msgr2=0x7f5620074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.167+0000 7f5624ab3640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5620106f10 0x7f5620074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.167+0000 7f5624ab3640 1 -- 172.21.15.5:0/366910704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620102790 msgr2=0x7f5620074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.167+0000 7f5624ab3640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620102790 0x7f5620074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.167+0000 7f5624ab3640 1 -- 172.21.15.5:0/366910704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56201a6a70 con 0x7f5620101de0 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.167+0000 7f5624ab3640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620101de0 0x7f56200744f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f561400b4a0 tx=0x7f561400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.167+0000 7f561dffb640 1 -- 172.21.15.5:0/366910704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614014070 con 0x7f5620101de0 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.167+0000 7f561dffb640 1 -- 172.21.15.5:0/366910704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5614010af0 con 0x7f5620101de0 2024-09-15T22:06:42.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.167+0000 7f5626d17640 1 -- 172.21.15.5:0/366910704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56201a6d60 con 0x7f5620101de0 2024-09-15T22:06:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.167+0000 7f561dffb640 1 -- 172.21.15.5:0/366910704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614017d10 con 0x7f5620101de0 2024-09-15T22:06:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.167+0000 7f5626d17640 1 -- 172.21.15.5:0/366910704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56201a7240 con 0x7f5620101de0 2024-09-15T22:06:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.168+0000 7f5626d17640 1 -- 172.21.15.5:0/366910704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5620102240 con 0x7f5620101de0 2024-09-15T22:06:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.169+0000 7f561dffb640 1 -- 172.21.15.5:0/366910704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f5614017070 con 0x7f5620101de0 2024-09-15T22:06:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.170+0000 7f561dffb640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f56100a3240 0x7f56100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.170+0000 7f561ffff640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f56100a3240 0x7f56100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.170+0000 7f561dffb640 1 -- 172.21.15.5:0/366910704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8277+0+0 (secure 0 0 0) 0x7f56140cedc0 con 0x7f5620101de0 2024-09-15T22:06:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.171+0000 7f561ffff640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f56100a3240 0x7f56100a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f560c02a930 tx=0x7f560c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:42.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.175+0000 7f561dffb640 1 -- 172.21.15.5:0/366910704 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f561409ba50 con 0x7f5620101de0 2024-09-15T22:06:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:42.352+0000 7f5626d17640 1 -- 172.21.15.5:0/366910704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f562002d7c0 con 0x7f5620101de0 2024-09-15T22:06:42.839 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:06:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b[64991]: 2024-09-15T22:06:42.482+0000 7f7475f12640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-15T22:06:43.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:43 smithi099 ceph-mon[34892]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 886 B/s wr, 3 op/s 2024-09-15T22:06:43.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:43 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/366910704' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:06:43.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:43 smithi129 ceph-mon[35314]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 886 B/s wr, 3 op/s 2024-09-15T22:06:43.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:43 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/366910704' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:06:43.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:43 smithi005 ceph-mon[30944]: pgmap v280: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 886 B/s wr, 3 op/s 2024-09-15T22:06:43.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:43 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/366910704' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-15T22:06:43.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.633+0000 7f561dffb640 1 -- 172.21.15.5:0/366910704 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v83) ==== 228+0+0 (secure 0 0 0) 0x7f561409b350 con 0x7f5620101de0 2024-09-15T22:06:43.634 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' created 2024-09-15T22:06:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.636+0000 7f5626d17640 1 -- 172.21.15.5:0/366910704 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f56100a3240 msgr2=0x7f56100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.636+0000 7f5626d17640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f56100a3240 0x7f56100a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f560c02a930 tx=0x7f560c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:06:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.636+0000 7f5626d17640 1 -- 172.21.15.5:0/366910704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620101de0 msgr2=0x7f56200744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.636+0000 7f5626d17640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620101de0 0x7f56200744f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f561400b4a0 tx=0x7f561400b960 comp rx=0 tx=0).stop 2024-09-15T22:06:43.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.637+0000 7f5626d17640 1 -- 172.21.15.5:0/366910704 shutdown_connections 2024-09-15T22:06:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.637+0000 7f5626d17640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5620106f10 0x7f5620074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.637+0000 7f5626d17640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5620102790 0x7f5620074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.637+0000 7f5626d17640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f56100a3240 0x7f56100a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.637+0000 7f5626d17640 1 --2- 172.21.15.5:0/366910704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5620101de0 0x7f56200744f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.637+0000 7f5626d17640 1 -- 172.21.15.5:0/366910704 >> 172.21.15.5:0/366910704 conn(0x7f56200fd8c0 msgr2=0x7f56200fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.638+0000 7f5626d17640 1 -- 172.21.15.5:0/366910704 shutdown_connections 2024-09-15T22:06:43.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:43.638+0000 7f5626d17640 1 -- 172.21.15.5:0/366910704 wait complete. 2024-09-15T22:06:44.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 4 op/s 2024-09-15T22:06:44.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:06:44.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]: dispatch 2024-09-15T22:06:44.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/366910704' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]': finished 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-09-15T22:06:44.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-15T22:06:44.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-15T22:06:44.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-15T22:06:44.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-09-15T22:06:44.323 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:44 smithi005 ceph-mon[30944]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:06:44.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 4 op/s 2024-09-15T22:06:44.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:06:44.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]: dispatch 2024-09-15T22:06:44.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-09-15T22:06:44.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-09-15T22:06:44.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-09-15T22:06:44.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-09-15T22:06:44.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-15T22:06:44.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/366910704' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]': finished 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-09-15T22:06:44.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:44 smithi099 ceph-mon[34892]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:06:44.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 4 op/s 2024-09-15T22:06:44.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: osdmap e82: 12 total, 12 up, 12 in 2024-09-15T22:06:44.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]: dispatch 2024-09-15T22:06:44.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]: dispatch 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]: dispatch 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]: dispatch 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]: dispatch 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/366910704' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11, 8, 6]}]': finished 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4]}]': finished 2024-09-15T22:06:44.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [3, 6]}]': finished 2024-09-15T22:06:44.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 7]}]': finished 2024-09-15T22:06:44.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-15T22:06:44.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-15T22:06:44.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-09-15T22:06:44.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [1, 9]}]': finished 2024-09-15T22:06:44.347 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:44 smithi129 ceph-mon[35314]: osdmap e83: 12 total, 12 up, 12 in 2024-09-15T22:06:44.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-15T22:06:44.624 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.161+0000 7f200e83f640 1 -- 172.21.15.5:0/331045412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2008106a80 msgr2=0x7f2008106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.161+0000 7f200e83f640 1 --2- 172.21.15.5:0/331045412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2008106a80 0x7f2008106e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009930 tx=0x7f1ff802a420 comp rx=0 tx=0).stop 2024-09-15T22:06:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.161+0000 7f200e83f640 1 -- 172.21.15.5:0/331045412 shutdown_connections 2024-09-15T22:06:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.161+0000 7f200e83f640 1 --2- 172.21.15.5:0/331045412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2008101420 0x7f200810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.161+0000 7f200e83f640 1 --2- 172.21.15.5:0/331045412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2008100a80 0x7f2008100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.161+0000 7f200e83f640 1 --2- 172.21.15.5:0/331045412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2008106a80 0x7f2008106e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.161+0000 7f200e83f640 1 -- 172.21.15.5:0/331045412 >> 172.21.15.5:0/331045412 conn(0x7f20080fc7a0 msgr2=0x7f20080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.162+0000 7f200e83f640 1 -- 172.21.15.5:0/331045412 shutdown_connections 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.162+0000 7f200e83f640 1 -- 172.21.15.5:0/331045412 wait complete. 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.162+0000 7f200e83f640 1 Processor -- start 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.162+0000 7f200e83f640 1 -- start start 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200e83f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2008100a80 0x7f2008198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200e83f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2008101420 0x7f2008198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200e83f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2008106a80 0x7f200819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200e83f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2008110e80 con 0x7f2008106a80 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200e83f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2008110d00 con 0x7f2008100a80 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200e83f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2008111000 con 0x7f2008101420 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200cddc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2008106a80 0x7f200819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200cddc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2008106a80 0x7f200819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38748/0 (socket says 172.21.15.5:38748) 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200cddc640 1 -- 172.21.15.5:0/1373088799 learned_addr learned my addr 172.21.15.5:0/1373088799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f2007fff640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2008100a80 0x7f2008198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f20077fe640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2008101420 0x7f2008198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200cddc640 1 -- 172.21.15.5:0/1373088799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2008101420 msgr2=0x7f2008198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200cddc640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2008101420 0x7f2008198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200cddc640 1 -- 172.21.15.5:0/1373088799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2008100a80 msgr2=0x7f2008198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200cddc640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2008100a80 0x7f2008198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200cddc640 1 -- 172.21.15.5:0/1373088799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200819d3f0 con 0x7f2008106a80 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.163+0000 7f200cddc640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2008106a80 0x7f200819cd10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f200000eaa0 tx=0x7f200000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.164+0000 7f20057fa640 1 -- 172.21.15.5:0/1373088799 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200000cdb0 con 0x7f2008106a80 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.164+0000 7f20057fa640 1 -- 172.21.15.5:0/1373088799 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2000005020 con 0x7f2008106a80 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.164+0000 7f200e83f640 1 -- 172.21.15.5:0/1373088799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200819d6e0 con 0x7f2008106a80 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.164+0000 7f20057fa640 1 -- 172.21.15.5:0/1373088799 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20000116b0 con 0x7f2008106a80 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.164+0000 7f200e83f640 1 -- 172.21.15.5:0/1373088799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20081a4fc0 con 0x7f2008106a80 2024-09-15T22:06:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.165+0000 7f200e83f640 1 -- 172.21.15.5:0/1373088799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2008100ee0 con 0x7f2008106a80 2024-09-15T22:06:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.166+0000 7f20057fa640 1 -- 172.21.15.5:0/1373088799 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f2000005560 con 0x7f2008106a80 2024-09-15T22:06:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.167+0000 7f20057fa640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff00a3240 0x7f1ff00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.167+0000 7f2007fff640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff00a3240 0x7f1ff00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.167+0000 7f20057fa640 1 -- 172.21.15.5:0/1373088799 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 9039+0+0 (secure 0 0 0) 0x7f20000d0200 con 0x7f2008106a80 2024-09-15T22:06:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.167+0000 7f2007fff640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff00a3240 0x7f1ff00a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009900 tx=0x7f1ff80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:45.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.170+0000 7f20057fa640 1 -- 172.21.15.5:0/1373088799 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2000016030 con 0x7f2008106a80 2024-09-15T22:06:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.347+0000 7f200e83f640 1 -- 172.21.15.5:0/1373088799 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f20081997a0 con 0x7f2008106a80 2024-09-15T22:06:45.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.639+0000 7f20057fa640 1 -- 172.21.15.5:0/1373088799 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f20081997a0 con 0x7f2008106a80 2024-09-15T22:06:45.641 INFO:teuthology.orchestra.run.smithi005.stderr:set pool 4 allow_ec_overwrites to true 2024-09-15T22:06:45.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 -- 172.21.15.5:0/1373088799 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff00a3240 msgr2=0x7f1ff00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:45.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff00a3240 0x7f1ff00a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009900 tx=0x7f1ff80058d0 comp rx=0 tx=0).stop 2024-09-15T22:06:45.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 -- 172.21.15.5:0/1373088799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2008106a80 msgr2=0x7f200819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2008106a80 0x7f200819cd10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f200000eaa0 tx=0x7f200000ef60 comp rx=0 tx=0).stop 2024-09-15T22:06:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 -- 172.21.15.5:0/1373088799 shutdown_connections 2024-09-15T22:06:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff00a3240 0x7f1ff00a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2008106a80 0x7f200819cd10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2008101420 0x7f2008198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 --2- 172.21.15.5:0/1373088799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2008100a80 0x7f2008198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 -- 172.21.15.5:0/1373088799 >> 172.21.15.5:0/1373088799 conn(0x7f20080fc7a0 msgr2=0x7f200806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 -- 172.21.15.5:0/1373088799 shutdown_connections 2024-09-15T22:06:45.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:45.643+0000 7f200e83f640 1 -- 172.21.15.5:0/1373088799 wait complete. 2024-09-15T22:06:46.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:45 smithi005 ceph-mon[30944]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:06:46.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:45 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1373088799' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-15T22:06:46.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:45 smithi099 ceph-mon[34892]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:06:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:45 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1373088799' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-15T22:06:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:45 smithi129 ceph-mon[35314]: osdmap e84: 12 total, 12 up, 12 in 2024-09-15T22:06:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:45 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1373088799' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-15T22:06:46.365 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-15T22:06:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:46.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:46 smithi005 ceph-mon[30944]: pgmap v285: 129 pgs: 5 creating+peering, 3 peering, 27 unknown, 94 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-15T22:06:46.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:46 smithi005 ceph-mon[30944]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:06:46.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:46 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1373088799' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-15T22:06:46.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:46 smithi005 ceph-mon[30944]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:06:47.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:46 smithi099 ceph-mon[34892]: pgmap v285: 129 pgs: 5 creating+peering, 3 peering, 27 unknown, 94 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-15T22:06:47.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:46 smithi099 ceph-mon[34892]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:06:47.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:46 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1373088799' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-15T22:06:47.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:46 smithi099 ceph-mon[34892]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:06:47.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:46 smithi129 ceph-mon[35314]: pgmap v285: 129 pgs: 5 creating+peering, 3 peering, 27 unknown, 94 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 6 op/s 2024-09-15T22:06:47.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:46 smithi129 ceph-mon[35314]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-15T22:06:47.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:46 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1373088799' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-15T22:06:47.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:46 smithi129 ceph-mon[35314]: osdmap e85: 12 total, 12 up, 12 in 2024-09-15T22:06:47.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.140+0000 7f2d992ba640 1 -- 172.21.15.5:0/2243477408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9410c150 msgr2=0x7f2d9410e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:47.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.140+0000 7f2d992ba640 1 --2- 172.21.15.5:0/2243477408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9410c150 0x7f2d9410e5f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009ca0 tx=0x7f2d8c02a950 comp rx=0 tx=0).stop 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.140+0000 7f2d992ba640 1 -- 172.21.15.5:0/2243477408 shutdown_connections 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.140+0000 7f2d992ba640 1 --2- 172.21.15.5:0/2243477408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d9410c150 0x7f2d9410e5f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.140+0000 7f2d992ba640 1 --2- 172.21.15.5:0/2243477408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d94103640 0x7f2d9410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.140+0000 7f2d992ba640 1 --2- 172.21.15.5:0/2243477408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d94102d20 0x7f2d94103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.140+0000 7f2d992ba640 1 -- 172.21.15.5:0/2243477408 >> 172.21.15.5:0/2243477408 conn(0x7f2d940fc780 msgr2=0x7f2d940feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.141+0000 7f2d992ba640 1 -- 172.21.15.5:0/2243477408 shutdown_connections 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.141+0000 7f2d992ba640 1 -- 172.21.15.5:0/2243477408 wait complete. 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.141+0000 7f2d992ba640 1 Processor -- start 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.141+0000 7f2d992ba640 1 -- start start 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d992ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d94102d20 0x7f2d9419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d992ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d94103640 0x7f2d9419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d992ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d9410c150 0x7f2d9419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d992ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94111f90 con 0x7f2d94103640 2024-09-15T22:06:47.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d992ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94111e10 con 0x7f2d94102d20 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d992ba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2d94112110 con 0x7f2d9410c150 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d927fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d94103640 0x7f2d9419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d927fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d94103640 0x7f2d9419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38776/0 (socket says 172.21.15.5:38776) 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d927fc640 1 -- 172.21.15.5:0/2139611887 learned_addr learned my addr 172.21.15.5:0/2139611887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d92ffd640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d94102d20 0x7f2d9419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d937fe640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d9410c150 0x7f2d9419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d927fc640 1 -- 172.21.15.5:0/2139611887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d9410c150 msgr2=0x7f2d9419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d927fc640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d9410c150 0x7f2d9419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d927fc640 1 -- 172.21.15.5:0/2139611887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d94102d20 msgr2=0x7f2d9419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d927fc640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d94102d20 0x7f2d9419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d927fc640 1 -- 172.21.15.5:0/2139611887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d9419f640 con 0x7f2d94103640 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d927fc640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d94103640 0x7f2d9419abd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c940 tx=0x7f2d8800cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d7bfff640 1 -- 172.21.15.5:0/2139611887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88004220 con 0x7f2d94103640 2024-09-15T22:06:47.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d7bfff640 1 -- 172.21.15.5:0/2139611887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d880043c0 con 0x7f2d94103640 2024-09-15T22:06:47.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d992ba640 1 -- 172.21.15.5:0/2139611887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d9419f930 con 0x7f2d94103640 2024-09-15T22:06:47.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.142+0000 7f2d7bfff640 1 -- 172.21.15.5:0/2139611887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88005650 con 0x7f2d94103640 2024-09-15T22:06:47.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.143+0000 7f2d992ba640 1 -- 172.21.15.5:0/2139611887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d941a7210 con 0x7f2d94103640 2024-09-15T22:06:47.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.144+0000 7f2d992ba640 1 -- 172.21.15.5:0/2139611887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d94103100 con 0x7f2d94103640 2024-09-15T22:06:47.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.145+0000 7f2d7bfff640 1 -- 172.21.15.5:0/2139611887 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f2d880079b0 con 0x7f2d94103640 2024-09-15T22:06:47.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.146+0000 7f2d7bfff640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2d7c0a32c0 0x7f2d7c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:47.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.146+0000 7f2d92ffd640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2d7c0a32c0 0x7f2d7c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:47.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.146+0000 7f2d7bfff640 1 -- 172.21.15.5:0/2139611887 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9039+0+0 (secure 0 0 0) 0x7f2d880cf7d0 con 0x7f2d94103640 2024-09-15T22:06:47.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.146+0000 7f2d92ffd640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2d7c0a32c0 0x7f2d7c0a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009990 tx=0x7f2d84009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:47.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.150+0000 7f2d7bfff640 1 -- 172.21.15.5:0/2139611887 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d88014030 con 0x7f2d94103640 2024-09-15T22:06:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.362+0000 7f2d992ba640 1 -- 172.21.15.5:0/2139611887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f2d941a7510 con 0x7f2d94103640 2024-09-15T22:06:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.648+0000 7f2d7bfff640 1 -- 172.21.15.5:0/2139611887 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7f2d88084e20 con 0x7f2d94103640 2024-09-15T22:06:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:added data pool 4 to fsmap 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.650+0000 7f2d992ba640 1 -- 172.21.15.5:0/2139611887 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2d7c0a32c0 msgr2=0x7f2d7c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.650+0000 7f2d992ba640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2d7c0a32c0 0x7f2d7c0a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2d84009990 tx=0x7f2d84009110 comp rx=0 tx=0).stop 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.650+0000 7f2d992ba640 1 -- 172.21.15.5:0/2139611887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d94103640 msgr2=0x7f2d9419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.650+0000 7f2d992ba640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d94103640 0x7f2d9419abd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c940 tx=0x7f2d8800cd40 comp rx=0 tx=0).stop 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.650+0000 7f2d992ba640 1 -- 172.21.15.5:0/2139611887 shutdown_connections 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.650+0000 7f2d992ba640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2d9410c150 0x7f2d9419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.650+0000 7f2d992ba640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2d7c0a32c0 0x7f2d7c0a5780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.650+0000 7f2d992ba640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d94103640 0x7f2d9419abd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.650+0000 7f2d992ba640 1 --2- 172.21.15.5:0/2139611887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2d94102d20 0x7f2d9419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.650+0000 7f2d992ba640 1 -- 172.21.15.5:0/2139611887 >> 172.21.15.5:0/2139611887 conn(0x7f2d940fc780 msgr2=0x7f2d941008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.650+0000 7f2d992ba640 1 -- 172.21.15.5:0/2139611887 shutdown_connections 2024-09-15T22:06:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:47.651+0000 7f2d992ba640 1 -- 172.21.15.5:0/2139611887 wait complete. 2024-09-15T22:06:47.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:47 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2139611887' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-15T22:06:48.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:47 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2139611887' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-15T22:06:48.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:47 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2139611887' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-15T22:06:48.291 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:06:48.556 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:48.967 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:48 smithi005 ceph-mon[30944]: pgmap v287: 129 pgs: 5 creating+peering, 3 peering, 22 unknown, 99 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 739 B/s wr, 6 op/s 2024-09-15T22:06:48.967 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:48 smithi005 ceph-mon[30944]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:06:48.967 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:48 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2139611887' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-15T22:06:48.967 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:48 smithi005 ceph-mon[30944]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:06:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.047+0000 7fa7a45db640 1 -- 172.21.15.5:0/1266880420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c10c180 msgr2=0x7fa79c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.047+0000 7fa7a45db640 1 --2- 172.21.15.5:0/1266880420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c10c180 0x7fa79c10e600 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa794009ca0 tx=0x7fa79402a950 comp rx=0 tx=0).stop 2024-09-15T22:06:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.047+0000 7fa7a45db640 1 -- 172.21.15.5:0/1266880420 shutdown_connections 2024-09-15T22:06:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.047+0000 7fa7a45db640 1 --2- 172.21.15.5:0/1266880420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c10c180 0x7fa79c10e600 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.047+0000 7fa7a45db640 1 --2- 172.21.15.5:0/1266880420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c1036c0 0x7fa79c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:49.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.047+0000 7fa7a45db640 1 --2- 172.21.15.5:0/1266880420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c102da0 0x7fa79c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.047+0000 7fa7a45db640 1 -- 172.21.15.5:0/1266880420 >> 172.21.15.5:0/1266880420 conn(0x7fa79c0fc7a0 msgr2=0x7fa79c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.047+0000 7fa7a45db640 1 -- 172.21.15.5:0/1266880420 shutdown_connections 2024-09-15T22:06:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.047+0000 7fa7a45db640 1 -- 172.21.15.5:0/1266880420 wait complete. 2024-09-15T22:06:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.048+0000 7fa7a45db640 1 Processor -- start 2024-09-15T22:06:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.048+0000 7fa7a45db640 1 -- start start 2024-09-15T22:06:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.048+0000 7fa7a45db640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c102da0 0x7fa79c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.048+0000 7fa7a45db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c1036c0 0x7fa79c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.048+0000 7fa7a45db640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c10c180 0x7fa79c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.048+0000 7fa7a45db640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c111f60 con 0x7fa79c1036c0 2024-09-15T22:06:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.048+0000 7fa7a45db640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c111de0 con 0x7fa79c10c180 2024-09-15T22:06:49.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.048+0000 7fa7a45db640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1120e0 con 0x7fa79c102da0 2024-09-15T22:06:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.048+0000 7fa7a1b76640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c1036c0 0x7fa79c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.048+0000 7fa7a1b76640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c1036c0 0x7fa79c198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38798/0 (socket says 172.21.15.5:38798) 2024-09-15T22:06:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.048+0000 7fa7a1b76640 1 -- 172.21.15.5:0/244404758 learned_addr learned my addr 172.21.15.5:0/244404758 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7a2377640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c102da0 0x7fa79c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7a2b78640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c10c180 0x7fa79c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7a1b76640 1 -- 172.21.15.5:0/244404758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c102da0 msgr2=0x7fa79c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7a1b76640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c102da0 0x7fa79c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7a1b76640 1 -- 172.21.15.5:0/244404758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c10c180 msgr2=0x7fa79c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7a1b76640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c10c180 0x7fa79c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:49.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7a1b76640 1 -- 172.21.15.5:0/244404758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c19d390 con 0x7fa79c1036c0 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7a2b78640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c10c180 0x7fa79c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7a1b76640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c1036c0 0x7fa79c198920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00d950 tx=0x7fa78c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7937fe640 1 -- 172.21.15.5:0/244404758 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c017070 con 0x7fa79c1036c0 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7937fe640 1 -- 172.21.15.5:0/244404758 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa78c009c30 con 0x7fa79c1036c0 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7a45db640 1 -- 172.21.15.5:0/244404758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c19d680 con 0x7fa79c1036c0 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7937fe640 1 -- 172.21.15.5:0/244404758 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00f6b0 con 0x7fa79c1036c0 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.049+0000 7fa7a45db640 1 -- 172.21.15.5:0/244404758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c1a4f60 con 0x7fa79c1036c0 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.050+0000 7fa7a45db640 1 -- 172.21.15.5:0/244404758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa79c103180 con 0x7fa79c1036c0 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.051+0000 7fa7937fe640 1 -- 172.21.15.5:0/244404758 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fa78c0109d0 con 0x7fa79c1036c0 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.052+0000 7fa7937fe640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7880a3240 0x7fa7880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.052+0000 7fa7a2377640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7880a3240 0x7fa7880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:49.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.052+0000 7fa7937fe640 1 -- 172.21.15.5:0/244404758 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7fa78c0cf8b0 con 0x7fa79c1036c0 2024-09-15T22:06:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.053+0000 7fa7a2377640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7880a3240 0x7fa7880a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa798005fd0 tx=0x7fa798005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:49.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.055+0000 7fa7937fe640 1 -- 172.21.15.5:0/244404758 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa78c009de0 con 0x7fa79c1036c0 2024-09-15T22:06:49.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:48 smithi099 ceph-mon[34892]: pgmap v287: 129 pgs: 5 creating+peering, 3 peering, 22 unknown, 99 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 739 B/s wr, 6 op/s 2024-09-15T22:06:49.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:48 smithi099 ceph-mon[34892]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:06:49.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:48 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2139611887' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-15T22:06:49.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:48 smithi099 ceph-mon[34892]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:06:49.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:48 smithi129 ceph-mon[35314]: pgmap v287: 129 pgs: 5 creating+peering, 3 peering, 22 unknown, 99 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 739 B/s wr, 6 op/s 2024-09-15T22:06:49.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:48 smithi129 ceph-mon[35314]: osdmap e86: 12 total, 12 up, 12 in 2024-09-15T22:06:49.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:48 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2139611887' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-15T22:06:49.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:48 smithi129 ceph-mon[35314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:06:49.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.271+0000 7fa7a45db640 1 -- 172.21.15.5:0/244404758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa79c107280 con 0x7fa79c1036c0 2024-09-15T22:06:49.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.272+0000 7fa7937fe640 1 -- 172.21.15.5:0/244404758 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11824 (secure 0 0 0) 0x7fa78c09c290 con 0x7fa79c1036c0 2024-09-15T22:06:49.273 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:49.273 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"btime":"2024-09-15T22:06:47:642362+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:06:47.363272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-09-15T22:06:49.274 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 16 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.274+0000 7fa7a45db640 1 -- 172.21.15.5:0/244404758 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7880a3240 msgr2=0x7fa7880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.274+0000 7fa7a45db640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7880a3240 0x7fa7880a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa798005fd0 tx=0x7fa798005850 comp rx=0 tx=0).stop 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.274+0000 7fa7a45db640 1 -- 172.21.15.5:0/244404758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c1036c0 msgr2=0x7fa79c198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.274+0000 7fa7a45db640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c1036c0 0x7fa79c198920 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa78c00d950 tx=0x7fa78c00de10 comp rx=0 tx=0).stop 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.274+0000 7fa7a45db640 1 -- 172.21.15.5:0/244404758 shutdown_connections 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.274+0000 7fa7a45db640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c10c180 0x7fa79c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.274+0000 7fa7a45db640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa7880a3240 0x7fa7880a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.274+0000 7fa7a45db640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c1036c0 0x7fa79c198920 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.274+0000 7fa7a45db640 1 --2- 172.21.15.5:0/244404758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c102da0 0x7fa79c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.274+0000 7fa7a45db640 1 -- 172.21.15.5:0/244404758 >> 172.21.15.5:0/244404758 conn(0x7fa79c0fc7a0 msgr2=0x7fa79c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.275+0000 7fa7a45db640 1 -- 172.21.15.5:0/244404758 shutdown_connections 2024-09-15T22:06:49.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:49.275+0000 7fa7a45db640 1 -- 172.21.15.5:0/244404758 wait complete. 2024-09-15T22:06:49.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:49 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/244404758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:49.961 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:06:50.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:49 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/244404758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:50.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:49 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/244404758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.722+0000 7f793312a640 1 -- 172.21.15.5:0/2707475422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f792c10c170 msgr2=0x7f792c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.722+0000 7f793312a640 1 --2- 172.21.15.5:0/2707475422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f792c10c170 0x7f792c10e610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7924009ca0 tx=0x7f792402a950 comp rx=0 tx=0).stop 2024-09-15T22:06:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.722+0000 7f793312a640 1 -- 172.21.15.5:0/2707475422 shutdown_connections 2024-09-15T22:06:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.722+0000 7f793312a640 1 --2- 172.21.15.5:0/2707475422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f792c10c170 0x7f792c10e610 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.722+0000 7f793312a640 1 --2- 172.21.15.5:0/2707475422 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f792c103660 0x7f792c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.722+0000 7f793312a640 1 --2- 172.21.15.5:0/2707475422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f792c102d40 0x7f792c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.722+0000 7f793312a640 1 -- 172.21.15.5:0/2707475422 >> 172.21.15.5:0/2707475422 conn(0x7f792c0fc7a0 msgr2=0x7f792c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:50.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.722+0000 7f793312a640 1 -- 172.21.15.5:0/2707475422 shutdown_connections 2024-09-15T22:06:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.722+0000 7f793312a640 1 -- 172.21.15.5:0/2707475422 wait complete. 2024-09-15T22:06:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f793312a640 1 Processor -- start 2024-09-15T22:06:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f793312a640 1 -- start start 2024-09-15T22:06:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f793312a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f792c102d40 0x7f792c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f793312a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f792c103660 0x7f792c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f793312a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f792c10c170 0x7f792c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f793312a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f792c112030 con 0x7f792c102d40 2024-09-15T22:06:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f793312a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f792c111eb0 con 0x7f792c10c170 2024-09-15T22:06:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f793312a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f792c1121b0 con 0x7f792c103660 2024-09-15T22:06:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f7930ec6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f792c102d40 0x7f792c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:50.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f7930ec6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f792c102d40 0x7f792c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38812/0 (socket says 172.21.15.5:38812) 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f7930ec6640 1 -- 172.21.15.5:0/46637599 learned_addr learned my addr 172.21.15.5:0/46637599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f792bfff640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f792c103660 0x7f792c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f79316c7640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f792c10c170 0x7f792c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f7930ec6640 1 -- 172.21.15.5:0/46637599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f792c103660 msgr2=0x7f792c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.723+0000 7f7930ec6640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f792c103660 0x7f792c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.724+0000 7f7930ec6640 1 -- 172.21.15.5:0/46637599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f792c10c170 msgr2=0x7f792c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.724+0000 7f7930ec6640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f792c10c170 0x7f792c101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.724+0000 7f7930ec6640 1 -- 172.21.15.5:0/46637599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792c1a69f0 con 0x7f792c102d40 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.724+0000 7f79316c7640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f792c10c170 0x7f792c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.724+0000 7f7930ec6640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f792c102d40 0x7f792c1009a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f791c00b4a0 tx=0x7f791c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.724+0000 7f7929ffb640 1 -- 172.21.15.5:0/46637599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c014070 con 0x7f792c102d40 2024-09-15T22:06:50.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.724+0000 7f7929ffb640 1 -- 172.21.15.5:0/46637599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f791c010af0 con 0x7f792c102d40 2024-09-15T22:06:50.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.724+0000 7f793312a640 1 -- 172.21.15.5:0/46637599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792c1a6ce0 con 0x7f792c102d40 2024-09-15T22:06:50.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.724+0000 7f7929ffb640 1 -- 172.21.15.5:0/46637599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791c017d10 con 0x7f792c102d40 2024-09-15T22:06:50.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.724+0000 7f793312a640 1 -- 172.21.15.5:0/46637599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f792c1a71c0 con 0x7f792c102d40 2024-09-15T22:06:50.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.725+0000 7f793312a640 1 -- 172.21.15.5:0/46637599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f792c1031a0 con 0x7f792c102d40 2024-09-15T22:06:50.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.726+0000 7f7929ffb640 1 -- 172.21.15.5:0/46637599 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f791c017070 con 0x7f792c102d40 2024-09-15T22:06:50.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.727+0000 7f7929ffb640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f79140a3240 0x7f79140a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:50.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.727+0000 7f792bfff640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f79140a3240 0x7f79140a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:50.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.727+0000 7f7929ffb640 1 -- 172.21.15.5:0/46637599 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f791c0cf0d0 con 0x7f792c102d40 2024-09-15T22:06:50.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.727+0000 7f792bfff640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f79140a3240 0x7f79140a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f792c102430 tx=0x7f7920008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:50.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.731+0000 7f7929ffb640 1 -- 172.21.15.5:0/46637599 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f791c09ba50 con 0x7f792c102d40 2024-09-15T22:06:50.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.908+0000 7f793312a640 1 -- 172.21.15.5:0/46637599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f792c02d7c0 con 0x7f792c102d40 2024-09-15T22:06:50.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:50 smithi005 ceph-mon[30944]: pgmap v289: 129 pgs: 5 creating+peering, 3 peering, 121 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:50.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:50 smithi005 ceph-mon[30944]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-15T22:06:50.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:50 smithi005 ceph-mon[30944]: Cluster is now healthy 2024-09-15T22:06:50.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.910+0000 7f7929ffb640 1 -- 172.21.15.5:0/46637599 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24549 (secure 0 0 0) 0x7f791c09b350 con 0x7f792c102d40 2024-09-15T22:06:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:50.912 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:06:47.363102+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 -- 172.21.15.5:0/46637599 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f79140a3240 msgr2=0x7f79140a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f79140a3240 0x7f79140a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f792c102430 tx=0x7f7920008040 comp rx=0 tx=0).stop 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 -- 172.21.15.5:0/46637599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f792c102d40 msgr2=0x7f792c1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f792c102d40 0x7f792c1009a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f791c00b4a0 tx=0x7f791c00b960 comp rx=0 tx=0).stop 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 -- 172.21.15.5:0/46637599 shutdown_connections 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f792c10c170 0x7f792c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f792c103660 0x7f792c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f79140a3240 0x7f79140a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 --2- 172.21.15.5:0/46637599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f792c102d40 0x7f792c1009a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 -- 172.21.15.5:0/46637599 >> 172.21.15.5:0/46637599 conn(0x7f792c0fc7a0 msgr2=0x7f792c0fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 -- 172.21.15.5:0/46637599 shutdown_connections 2024-09-15T22:06:50.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:50.912+0000 7f793312a640 1 -- 172.21.15.5:0/46637599 wait complete. 2024-09-15T22:06:51.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:50 smithi099 ceph-mon[34892]: pgmap v289: 129 pgs: 5 creating+peering, 3 peering, 121 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:51.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:50 smithi099 ceph-mon[34892]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-15T22:06:51.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:50 smithi099 ceph-mon[34892]: Cluster is now healthy 2024-09-15T22:06:51.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:50 smithi129 ceph-mon[35314]: pgmap v289: 129 pgs: 5 creating+peering, 3 peering, 121 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:06:51.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:50 smithi129 ceph-mon[35314]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-15T22:06:51.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:50 smithi129 ceph-mon[35314]: Cluster is now healthy 2024-09-15T22:06:51.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:06:51.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:51 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/46637599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:06:51.905 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:52.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:51 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/46637599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:06:52.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:51 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/46637599' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:06:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.414+0000 7fd34bfe2640 1 -- 172.21.15.5:0/1062053546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3441036e0 msgr2=0x7fd344107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.414+0000 7fd34bfe2640 1 --2- 172.21.15.5:0/1062053546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3441036e0 0x7fd344107b90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd338009930 tx=0x7fd33802a420 comp rx=0 tx=0).stop 2024-09-15T22:06:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.414+0000 7fd34bfe2640 1 -- 172.21.15.5:0/1062053546 shutdown_connections 2024-09-15T22:06:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.414+0000 7fd34bfe2640 1 --2- 172.21.15.5:0/1062053546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3441082a0 0x7fd34410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.414+0000 7fd34bfe2640 1 --2- 172.21.15.5:0/1062053546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3441036e0 0x7fd344107b90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.414+0000 7fd34bfe2640 1 --2- 172.21.15.5:0/1062053546 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd344102d30 0x7fd344103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.414+0000 7fd34bfe2640 1 -- 172.21.15.5:0/1062053546 >> 172.21.15.5:0/1062053546 conn(0x7fd3440fc7d0 msgr2=0x7fd3440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.414+0000 7fd34bfe2640 1 -- 172.21.15.5:0/1062053546 shutdown_connections 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.414+0000 7fd34bfe2640 1 -- 172.21.15.5:0/1062053546 wait complete. 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.414+0000 7fd34bfe2640 1 Processor -- start 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34bfe2640 1 -- start start 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34bfe2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd344102d30 0x7fd3441985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34bfe2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3441036e0 0x7fd344198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34bfe2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3441082a0 0x7fd34419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34bfe2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3441120c0 con 0x7fd3441082a0 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34bfe2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd344111f40 con 0x7fd3441036e0 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34bfe2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd344112240 con 0x7fd344102d30 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34a57f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3441082a0 0x7fd34419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34a57f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3441082a0 0x7fd34419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38838/0 (socket says 172.21.15.5:38838) 2024-09-15T22:06:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34a57f640 1 -- 172.21.15.5:0/28474872 learned_addr learned my addr 172.21.15.5:0/28474872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34957d640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3441036e0 0x7fd344198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34a57f640 1 -- 172.21.15.5:0/28474872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd344102d30 msgr2=0x7fd3441985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd349d7e640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd344102d30 0x7fd3441985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34a57f640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd344102d30 0x7fd3441985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34a57f640 1 -- 172.21.15.5:0/28474872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3441036e0 msgr2=0x7fd344198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34a57f640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3441036e0 0x7fd344198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34a57f640 1 -- 172.21.15.5:0/28474872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34419d560 con 0x7fd3441082a0 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd34957d640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3441036e0 0x7fd344198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.415+0000 7fd349d7e640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd344102d30 0x7fd3441985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.416+0000 7fd34a57f640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3441082a0 0x7fd34419ce80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00cce0 tx=0x7fd33c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.416+0000 7fd336ffd640 1 -- 172.21.15.5:0/28474872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c00f450 con 0x7fd3441082a0 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.416+0000 7fd336ffd640 1 -- 172.21.15.5:0/28474872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd33c00fad0 con 0x7fd3441082a0 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.416+0000 7fd34bfe2640 1 -- 172.21.15.5:0/28474872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34419d850 con 0x7fd3441082a0 2024-09-15T22:06:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.416+0000 7fd336ffd640 1 -- 172.21.15.5:0/28474872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c018c10 con 0x7fd3441082a0 2024-09-15T22:06:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.416+0000 7fd34bfe2640 1 -- 172.21.15.5:0/28474872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3441a5130 con 0x7fd3441082a0 2024-09-15T22:06:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.417+0000 7fd34bfe2640 1 -- 172.21.15.5:0/28474872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd344103110 con 0x7fd3441082a0 2024-09-15T22:06:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.418+0000 7fd336ffd640 1 -- 172.21.15.5:0/28474872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fd33c00f5f0 con 0x7fd3441082a0 2024-09-15T22:06:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.419+0000 7fd336ffd640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd3300a32c0 0x7fd3300a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.419+0000 7fd349d7e640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd3300a32c0 0x7fd3300a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.419+0000 7fd336ffd640 1 -- 172.21.15.5:0/28474872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7fd33c0d0340 con 0x7fd3441082a0 2024-09-15T22:06:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.419+0000 7fd349d7e640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd3300a32c0 0x7fd3300a5780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd340006fd0 tx=0x7fd340008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.423+0000 7fd336ffd640 1 -- 172.21.15.5:0/28474872 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd33c014030 con 0x7fd3441082a0 2024-09-15T22:06:52.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.598+0000 7fd34bfe2640 1 -- 172.21.15.5:0/28474872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd34410a2b0 con 0x7fd3441082a0 2024-09-15T22:06:52.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.600+0000 7fd336ffd640 1 -- 172.21.15.5:0/28474872 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24549 (secure 0 0 0) 0x7fd33c09cd20 con 0x7fd3441082a0 2024-09-15T22:06:52.601 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:52.601 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:06:47.363102+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.602+0000 7fd34bfe2640 1 -- 172.21.15.5:0/28474872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd3300a32c0 msgr2=0x7fd3300a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.602+0000 7fd34bfe2640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd3300a32c0 0x7fd3300a5780 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd340006fd0 tx=0x7fd340008040 comp rx=0 tx=0).stop 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.602+0000 7fd34bfe2640 1 -- 172.21.15.5:0/28474872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3441082a0 msgr2=0x7fd34419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.602+0000 7fd34bfe2640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3441082a0 0x7fd34419ce80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00cce0 tx=0x7fd33c00a620 comp rx=0 tx=0).stop 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.602+0000 7fd34bfe2640 1 -- 172.21.15.5:0/28474872 shutdown_connections 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.602+0000 7fd34bfe2640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd3300a32c0 0x7fd3300a5780 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.602+0000 7fd34bfe2640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3441082a0 0x7fd34419ce80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.602+0000 7fd34bfe2640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd3441036e0 0x7fd344198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.602+0000 7fd34bfe2640 1 --2- 172.21.15.5:0/28474872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd344102d30 0x7fd3441985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.602+0000 7fd34bfe2640 1 -- 172.21.15.5:0/28474872 >> 172.21.15.5:0/28474872 conn(0x7fd3440fc7d0 msgr2=0x7fd344105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.602+0000 7fd34bfe2640 1 -- 172.21.15.5:0/28474872 shutdown_connections 2024-09-15T22:06:52.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:52.603+0000 7fd34bfe2640 1 -- 172.21.15.5:0/28474872 wait complete. 2024-09-15T22:06:52.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:52 smithi005 ceph-mon[30944]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s, 0 objects/s recovering 2024-09-15T22:06:52.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:52 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/28474872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:06:52.996 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:52 smithi129 ceph-mon[35314]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s, 0 objects/s recovering 2024-09-15T22:06:52.997 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:52 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/28474872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:06:53.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:52 smithi099 ceph-mon[34892]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 94 B/s, 0 objects/s recovering 2024-09-15T22:06:53.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:52 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/28474872' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:06:53.405 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:06:53.671 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.163+0000 7f8cf00ff640 1 -- 172.21.15.5:0/2036683064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce8101420 msgr2=0x7f8ce810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.163+0000 7f8cf00ff640 1 --2- 172.21.15.5:0/2036683064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce8101420 0x7f8ce810d950 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0009cd0 tx=0x7f8ce002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:06:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.163+0000 7f8cf00ff640 1 -- 172.21.15.5:0/2036683064 shutdown_connections 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.163+0000 7f8cf00ff640 1 --2- 172.21.15.5:0/2036683064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce8101420 0x7f8ce810d950 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.163+0000 7f8cf00ff640 1 --2- 172.21.15.5:0/2036683064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ce8100a80 0x7f8ce8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.163+0000 7f8cf00ff640 1 --2- 172.21.15.5:0/2036683064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce8106a80 0x7f8ce8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.163+0000 7f8cf00ff640 1 -- 172.21.15.5:0/2036683064 >> 172.21.15.5:0/2036683064 conn(0x7f8ce80fc7a0 msgr2=0x7f8ce80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.163+0000 7f8cf00ff640 1 -- 172.21.15.5:0/2036683064 shutdown_connections 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.163+0000 7f8cf00ff640 1 -- 172.21.15.5:0/2036683064 wait complete. 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cf00ff640 1 Processor -- start 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cf00ff640 1 -- start start 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cf00ff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ce8100a80 0x7f8ce81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cf00ff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce8101420 0x7f8ce8198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cf00ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce8106a80 0x7f8ce819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cf00ff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce8110e80 con 0x7f8ce8106a80 2024-09-15T22:06:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cf00ff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce8110d00 con 0x7f8ce8101420 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cf00ff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce8111000 con 0x7f8ce8100a80 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cee69c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce8106a80 0x7f8ce819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cee69c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce8106a80 0x7f8ce819ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35956/0 (socket says 172.21.15.5:35956) 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cee69c640 1 -- 172.21.15.5:0/633140072 learned_addr learned my addr 172.21.15.5:0/633140072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.164+0000 7f8cede9b640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ce8100a80 0x7f8ce81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8ced69a640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce8101420 0x7f8ce8198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8cee69c640 1 -- 172.21.15.5:0/633140072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ce8100a80 msgr2=0x7f8ce81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8cee69c640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ce8100a80 0x7f8ce81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8cee69c640 1 -- 172.21.15.5:0/633140072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce8101420 msgr2=0x7f8ce8198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8cee69c640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce8101420 0x7f8ce8198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8cee69c640 1 -- 172.21.15.5:0/633140072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce819d560 con 0x7f8ce8106a80 2024-09-15T22:06:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8cee69c640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce8106a80 0x7f8ce819ce80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0005cc0 tx=0x7f8ce0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8cdeffd640 1 -- 172.21.15.5:0/633140072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0004040 con 0x7f8ce8106a80 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8cf00ff640 1 -- 172.21.15.5:0/633140072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce819d7f0 con 0x7f8ce8106a80 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8cdeffd640 1 -- 172.21.15.5:0/633140072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ce00046c0 con 0x7f8ce8106a80 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8cdeffd640 1 -- 172.21.15.5:0/633140072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0046480 con 0x7f8ce8106a80 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.165+0000 7f8cf00ff640 1 -- 172.21.15.5:0/633140072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce81a5130 con 0x7f8ce8106a80 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.166+0000 7f8cf00ff640 1 -- 172.21.15.5:0/633140072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce81021c0 con 0x7f8ce8106a80 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.167+0000 7f8cdeffd640 1 -- 172.21.15.5:0/633140072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f8ce00305a0 con 0x7f8ce8106a80 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.168+0000 7f8cdeffd640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8cd40a3240 0x7f8cd40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.168+0000 7f8cede9b640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8cd40a3240 0x7f8cd40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.168+0000 7f8cdeffd640 1 -- 172.21.15.5:0/633140072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f8ce002e030 con 0x7f8ce8106a80 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.169+0000 7f8cede9b640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8cd40a3240 0x7f8cd40a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400bb60 tx=0x7f8ce400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.172+0000 7f8cdeffd640 1 -- 172.21.15.5:0/633140072 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ce004b060 con 0x7f8ce8106a80 2024-09-15T22:06:54.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.383+0000 7f8cf00ff640 1 -- 172.21.15.5:0/633140072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ce810a000 con 0x7f8ce8106a80 2024-09-15T22:06:54.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.383+0000 7f8cdeffd640 1 -- 172.21.15.5:0/633140072 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11824 (secure 0 0 0) 0x7f8ce00bdda0 con 0x7f8ce8106a80 2024-09-15T22:06:54.385 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:54.386 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"btime":"2024-09-15T22:06:47:642362+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24442,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24451,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24466,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24467,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:06:47.363272+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24491},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24491,"qdb_cluster":[24491]},"id":1}]} 2024-09-15T22:06:54.386 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 16 2024-09-15T22:06:54.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 -- 172.21.15.5:0/633140072 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8cd40a3240 msgr2=0x7f8cd40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:54.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8cd40a3240 0x7f8cd40a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8ce400bb60 tx=0x7f8ce400b390 comp rx=0 tx=0).stop 2024-09-15T22:06:54.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 -- 172.21.15.5:0/633140072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce8106a80 msgr2=0x7f8ce819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:54.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce8106a80 0x7f8ce819ce80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0005cc0 tx=0x7f8ce0003970 comp rx=0 tx=0).stop 2024-09-15T22:06:54.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 -- 172.21.15.5:0/633140072 shutdown_connections 2024-09-15T22:06:54.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8cd40a3240 0x7f8cd40a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:54.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce8106a80 0x7f8ce819ce80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:54.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8ce8101420 0x7f8ce8198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:54.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 --2- 172.21.15.5:0/633140072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8ce8100a80 0x7f8ce81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:54.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 -- 172.21.15.5:0/633140072 >> 172.21.15.5:0/633140072 conn(0x7f8ce80fc7a0 msgr2=0x7f8ce806a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:54.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 -- 172.21.15.5:0/633140072 shutdown_connections 2024-09-15T22:06:54.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:54.386+0000 7f8cf00ff640 1 -- 172.21.15.5:0/633140072 wait complete. 2024-09-15T22:06:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:54 smithi099 ceph-mon[34892]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-09-15T22:06:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:54 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/633140072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:55.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:54 smithi129 ceph-mon[35314]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-09-15T22:06:55.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:54 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/633140072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:55.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:54 smithi005 ceph-mon[30944]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 72 B/s, 0 objects/s recovering 2024-09-15T22:06:55.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:54 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/633140072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:06:55.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:06:55.505 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.998+0000 7f899743f640 1 -- 172.21.15.5:0/1512185152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8990106f10 msgr2=0x7f899010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.998+0000 7f899743f640 1 --2- 172.21.15.5:0/1512185152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8990106f10 0x7f899010ea90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8988009ca0 tx=0x7f898802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.999+0000 7f899743f640 1 -- 172.21.15.5:0/1512185152 shutdown_connections 2024-09-15T22:06:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.999+0000 7f899743f640 1 --2- 172.21.15.5:0/1512185152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8990106f10 0x7f899010ea90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.999+0000 7f899743f640 1 --2- 172.21.15.5:0/1512185152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8990102790 0x7f89901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.999+0000 7f899743f640 1 --2- 172.21.15.5:0/1512185152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8990101de0 0x7f89901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.999+0000 7f899743f640 1 -- 172.21.15.5:0/1512185152 >> 172.21.15.5:0/1512185152 conn(0x7f89900fd8c0 msgr2=0x7f89900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:56.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.999+0000 7f899743f640 1 -- 172.21.15.5:0/1512185152 shutdown_connections 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.999+0000 7f899743f640 1 -- 172.21.15.5:0/1512185152 wait complete. 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.999+0000 7f899743f640 1 Processor -- start 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.999+0000 7f899743f640 1 -- start start 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:55.999+0000 7f899743f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8990101de0 0x7f89900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f899743f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8990102790 0x7f8990074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f899743f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8990106f10 0x7f8990074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f899743f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8990111f70 con 0x7f8990106f10 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f899743f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8990111df0 con 0x7f8990102790 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f899743f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f89901120f0 con 0x7f8990101de0 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89959dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8990106f10 0x7f8990074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89959dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8990106f10 0x7f8990074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35982/0 (socket says 172.21.15.5:35982) 2024-09-15T22:06:56.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89959dc640 1 -- 172.21.15.5:0/943716140 learned_addr learned my addr 172.21.15.5:0/943716140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89949da640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8990102790 0x7f8990074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89959dc640 1 -- 172.21.15.5:0/943716140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8990101de0 msgr2=0x7f89900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89959dc640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8990101de0 0x7f89900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89959dc640 1 -- 172.21.15.5:0/943716140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8990102790 msgr2=0x7f8990074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89959dc640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8990102790 0x7f8990074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89959dc640 1 -- 172.21.15.5:0/943716140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89901a6890 con 0x7f8990106f10 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89959dc640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8990106f10 0x7f8990074f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8988031a20 tx=0x7f8988031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89867fc640 1 -- 172.21.15.5:0/943716140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8988003c30 con 0x7f8990106f10 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89867fc640 1 -- 172.21.15.5:0/943716140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8988003dd0 con 0x7f8990106f10 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.000+0000 7f89867fc640 1 -- 172.21.15.5:0/943716140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898802fcf0 con 0x7f8990106f10 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.001+0000 7f899743f640 1 -- 172.21.15.5:0/943716140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89901a6b20 con 0x7f8990106f10 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.001+0000 7f899743f640 1 -- 172.21.15.5:0/943716140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89901a7010 con 0x7f8990106f10 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.001+0000 7f899743f640 1 -- 172.21.15.5:0/943716140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89901021c0 con 0x7f8990106f10 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.003+0000 7f89867fc640 1 -- 172.21.15.5:0/943716140 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f8988004b40 con 0x7f8990106f10 2024-09-15T22:06:56.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.003+0000 7f89867fc640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f897c0a3240 0x7f897c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:56.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.004+0000 7f89951db640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f897c0a3240 0x7f897c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:56.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.004+0000 7f89867fc640 1 -- 172.21.15.5:0/943716140 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f89880f1170 con 0x7f8990106f10 2024-09-15T22:06:56.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.004+0000 7f89951db640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f897c0a3240 0x7f897c0a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f898c005fd0 tx=0x7f898c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:56.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.007+0000 7f89867fc640 1 -- 172.21.15.5:0/943716140 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f89880bdb80 con 0x7f8990106f10 2024-09-15T22:06:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.185+0000 7f899743f640 1 -- 172.21.15.5:0/943716140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8990075a20 con 0x7f8990106f10 2024-09-15T22:06:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.187+0000 7f89867fc640 1 -- 172.21.15.5:0/943716140 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24549 (secure 0 0 0) 0x7f8990075a20 con 0x7f8990106f10 2024-09-15T22:06:56.189 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:06:56.189 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:06:47.363102+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:06:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.189+0000 7f899743f640 1 -- 172.21.15.5:0/943716140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f897c0a3240 msgr2=0x7f897c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.189+0000 7f899743f640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f897c0a3240 0x7f897c0a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f898c005fd0 tx=0x7f898c005850 comp rx=0 tx=0).stop 2024-09-15T22:06:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.189+0000 7f899743f640 1 -- 172.21.15.5:0/943716140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8990106f10 msgr2=0x7f8990074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.189+0000 7f899743f640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8990106f10 0x7f8990074f70 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8988031a20 tx=0x7f8988031a50 comp rx=0 tx=0).stop 2024-09-15T22:06:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.189+0000 7f899743f640 1 -- 172.21.15.5:0/943716140 shutdown_connections 2024-09-15T22:06:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.189+0000 7f899743f640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f897c0a3240 0x7f897c0a5700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.189+0000 7f899743f640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8990106f10 0x7f8990074f70 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.189+0000 7f899743f640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8990102790 0x7f8990074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.189+0000 7f899743f640 1 --2- 172.21.15.5:0/943716140 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8990101de0 0x7f89900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.189+0000 7f899743f640 1 -- 172.21.15.5:0/943716140 >> 172.21.15.5:0/943716140 conn(0x7f89900fd8c0 msgr2=0x7f89900feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.190+0000 7f899743f640 1 -- 172.21.15.5:0/943716140 shutdown_connections 2024-09-15T22:06:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:56.190+0000 7f899743f640 1 -- 172.21.15.5:0/943716140 wait complete. 2024-09-15T22:06:56.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:56 smithi005 ceph-mon[30944]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 63 B/s, 0 objects/s recovering 2024-09-15T22:06:56.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/943716140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:06:56.971 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-15T22:06:37.854436+0000', 'modified': '2024-09-15T22:06:47.363272+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24491}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.99:6839/3763383986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 3763383986}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 3763383986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24491, 'qdb_cluster': [24491]} 2024-09-15T22:06:56.971 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-15T22:06:56.971 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:06:57.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:56 smithi099 ceph-mon[34892]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 63 B/s, 0 objects/s recovering 2024-09-15T22:06:57.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/943716140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:06:57.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:56 smithi129 ceph-mon[35314]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 63 B/s, 0 objects/s recovering 2024-09-15T22:06:57.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/943716140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:06:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:57.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.736+0000 7f2cdd6be640 1 -- 172.21.15.5:0/590374197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd8102790 msgr2=0x7f2cd81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.736+0000 7f2cdd6be640 1 --2- 172.21.15.5:0/590374197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd8102790 0x7f2cd81067e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009900 tx=0x7f2ccc02a410 comp rx=0 tx=0).stop 2024-09-15T22:06:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.736+0000 7f2cdd6be640 1 -- 172.21.15.5:0/590374197 shutdown_connections 2024-09-15T22:06:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.736+0000 7f2cdd6be640 1 --2- 172.21.15.5:0/590374197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cd8106f10 0x7f2cd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.736+0000 7f2cdd6be640 1 --2- 172.21.15.5:0/590374197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd8102790 0x7f2cd81067e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.736+0000 7f2cdd6be640 1 --2- 172.21.15.5:0/590374197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd8101de0 0x7f2cd81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.736+0000 7f2cdd6be640 1 -- 172.21.15.5:0/590374197 >> 172.21.15.5:0/590374197 conn(0x7f2cd80fd8c0 msgr2=0x7f2cd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.736+0000 7f2cdd6be640 1 -- 172.21.15.5:0/590374197 shutdown_connections 2024-09-15T22:06:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.736+0000 7f2cdd6be640 1 -- 172.21.15.5:0/590374197 wait complete. 2024-09-15T22:06:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.736+0000 7f2cdd6be640 1 Processor -- start 2024-09-15T22:06:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.736+0000 7f2cdd6be640 1 -- start start 2024-09-15T22:06:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cdd6be640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cd8101de0 0x7f2cd8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cdd6be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd8102790 0x7f2cd81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cdd6be640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd8106f10 0x7f2cd819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cdd6be640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd8111fc0 con 0x7f2cd8102790 2024-09-15T22:06:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cdd6be640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd8111e40 con 0x7f2cd8106f10 2024-09-15T22:06:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cdd6be640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd8112140 con 0x7f2cd8101de0 2024-09-15T22:06:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cd67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd8102790 0x7f2cd81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cd67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd8102790 0x7f2cd81989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35998/0 (socket says 172.21.15.5:35998) 2024-09-15T22:06:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cd67fc640 1 -- 172.21.15.5:0/1390378383 learned_addr learned my addr 172.21.15.5:0/1390378383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cd8101de0 0x7f2cd8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cd77fe640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd8106f10 0x7f2cd819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cd67fc640 1 -- 172.21.15.5:0/1390378383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cd8101de0 msgr2=0x7f2cd8198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cd67fc640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cd8101de0 0x7f2cd8198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cd67fc640 1 -- 172.21.15.5:0/1390378383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd8106f10 msgr2=0x7f2cd819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cd67fc640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd8106f10 0x7f2cd819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cd67fc640 1 -- 172.21.15.5:0/1390378383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd819d410 con 0x7f2cd8102790 2024-09-15T22:06:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cd67fc640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd8102790 0x7f2cd81989a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc004770 tx=0x7f2ccc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cbffff640 1 -- 172.21.15.5:0/1390378383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc03f070 con 0x7f2cd8102790 2024-09-15T22:06:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.737+0000 7f2cdd6be640 1 -- 172.21.15.5:0/1390378383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd819d6a0 con 0x7f2cd8102790 2024-09-15T22:06:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.738+0000 7f2cbffff640 1 -- 172.21.15.5:0/1390378383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ccc031ca0 con 0x7f2cd8102790 2024-09-15T22:06:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.738+0000 7f2cbffff640 1 -- 172.21.15.5:0/1390378383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc035030 con 0x7f2cd8102790 2024-09-15T22:06:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.738+0000 7f2cdd6be640 1 -- 172.21.15.5:0/1390378383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd81a4fe0 con 0x7f2cd8102790 2024-09-15T22:06:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.740+0000 7f2cbffff640 1 -- 172.21.15.5:0/1390378383 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f2ccc042070 con 0x7f2cd8102790 2024-09-15T22:06:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.740+0000 7f2cdfa97740 1 Processor -- start 2024-09-15T22:06:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.740+0000 7f2cdfa97740 1 -- start start 2024-09-15T22:06:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.740+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26109d0 0x562ef2610db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.740+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26112f0 0x562ef261b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.740+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef261bbb0 0x562ef261dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.740+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562ef2577090 con 0x562ef26112f0 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.740+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562ef251ea70 con 0x562ef26109d0 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.740+0000 7f2cbffff640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2cc00a3240 0x7f2cc00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.740+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ef2573050 con 0x562ef261bbb0 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.740+0000 7f2cd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26112f0 0x562ef261b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26112f0 0x562ef261b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36010/0 (socket says 172.21.15.5:36010) 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/1887274142 learned_addr learned my addr 172.21.15.5:0/1887274142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd77fe640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2cc00a3240 0x7f2cc00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd67fc640 1 --2- 172.21.15.5:0/1887274142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef261bbb0 0x562ef261dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd77fe640 1 --2- 172.21.15.5:0/1887274142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26109d0 0x562ef2610db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cbffff640 1 -- 172.21.15.5:0/1390378383 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f2ccc0f0440 con 0x7f2cd8102790 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/1887274142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef261bbb0 msgr2=0x562ef261dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/1887274142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef261bbb0 0x562ef261dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/1887274142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26109d0 msgr2=0x562ef2610db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/1887274142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26109d0 0x562ef2610db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/1887274142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef252b130 con 0x562ef26112f0 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd77fe640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2cc00a3240 0x7f2cc00a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005cc0 tx=0x7f2cd000a340 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/1887274142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26112f0 0x562ef261b670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800bcd0 tx=0x7f2cc80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21949df541caad90 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cbd7fa640 1 -- 172.21.15.5:0/1887274142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8005020 con 0x562ef26112f0 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cbd7fa640 1 -- 172.21.15.5:0/1887274142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc80047a0 con 0x562ef26112f0 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.741+0000 7f2cbd7fa640 1 -- 172.21.15.5:0/1887274142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8011720 con 0x562ef26112f0 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.742+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1887274142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26112f0 msgr2=0x562ef261b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.742+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1887274142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26112f0 0x562ef261b670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800bcd0 tx=0x7f2cc80107a0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.742+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1887274142 shutdown_connections 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.742+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1887274142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef261bbb0 0x562ef261dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.742+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1887274142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26112f0 0x562ef261b670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.742+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1887274142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26109d0 0x562ef2610db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.742+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1887274142 >> 172.21.15.5:0/1887274142 conn(0x562ef260cf60 msgr2=0x562ef260d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.742+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1887274142 shutdown_connections 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.742+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1887274142 wait complete. 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.742+0000 7f2cdfa97740 1 -- start start 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cdfa97740 1 Processor -- start 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef261b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26112f0 0x562ef26187c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef261bbb0 0x562ef2618d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562ef261e780 con 0x562ef26109d0 2024-09-15T22:06:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562ef2531f20 con 0x562ef26112f0 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ef261e900 con 0x562ef261bbb0 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cdfa97740 10 client.0 _ll_register_callbacks cb 0x562ef24fb030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef261b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef261b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36018/0 (socket says 172.21.15.5:36018) 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/1733507847 learned_addr learned my addr 172.21.15.5:0/1733507847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cd67fc640 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26112f0 0x562ef26187c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cd77fe640 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef261bbb0 0x562ef2618d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/1733507847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef261bbb0 msgr2=0x562ef2618d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef261bbb0 0x562ef2618d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/1733507847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26112f0 msgr2=0x562ef26187c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26112f0 0x562ef26187c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/1733507847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef22e82b0 con 0x562ef26109d0 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef261b470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800bca0 tx=0x7f2cc800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf753e8657108b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:57.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.743+0000 7f2cbaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1733507847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8004b60 con 0x562ef26109d0 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1733507847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc80047a0 con 0x562ef26109d0 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1733507847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f2cc8004020 con 0x562ef26109d0 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1733507847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f2cc802f050 con 0x562ef26109d0 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cdfa97740 10 client.14700 fetch_fsmap learned FSMap version 16 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1733507847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x562ef262a0d0 con 0x562ef26109d0 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1733507847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc80047a0 con 0x562ef26109d0 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1733507847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f2cc80036f0 con 0x562ef26109d0 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cdfa97740 10 client.14700 fetch_fsmap finished waiting for FSMap version 16 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cdfa97740 10 client.14700 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cdfa97740 20 client.14700 populate_metadata read hostname 'smithi005' 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562ef262c350 0x562ef262d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cdfa97740 4 client.14700 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.744+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1733507847 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ef2584600 con 0x562ef262c350 2024-09-15T22:06:57.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.745+0000 7f2cd67fc640 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562ef262c350 0x562ef262d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.745+0000 7f2cd67fc640 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562ef262c350 0x562ef262d7c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44aad7f7c5e76113 server_cookie=abdde5383a72a7ab in_seq=0 out_seq=0 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.745+0000 7f2cbaff5640 10 client.14700 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1733507847 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ef2584600 con 0x562ef262c350 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cbaff5640 10 client.14700 handle_command_reply: tid=0 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 1 client.14700 shutdown 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1733507847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 msgr2=0x562ef261b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef261b470 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800bca0 tx=0x7f2cc800bfd0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1733507847 shutdown_connections 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef261bbb0 0x562ef2618d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562ef262c350 0x562ef262d7c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26112f0 0x562ef26187c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1733507847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef261b470 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cbaff5640 0 client.14700 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:06:57.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1733507847 >> 172.21.15.5:0/1733507847 conn(0x562ef260cf60 msgr2=0x562ef2612c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1733507847 shutdown_connections 2024-09-15T22:06:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1733507847 wait complete. 2024-09-15T22:06:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.746+0000 7f2cdfa97740 20 client.14700 trim_cache size 0 max 16384 2024-09-15T22:06:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cdfa97740 1 Processor -- start 2024-09-15T22:06:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cdfa97740 1 -- start start 2024-09-15T22:06:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef262d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 0x562ef261a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef261bbb0 0x562ef2615a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562ef262bb00 con 0x562ef26109d0 2024-09-15T22:06:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562ef262c1d0 con 0x562ef261bbb0 2024-09-15T22:06:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ef262c8a0 con 0x562ef26112f0 2024-09-15T22:06:57.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef262d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef262d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36034/0 (socket says 172.21.15.5:36034) 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/3659312451 learned_addr learned my addr 172.21.15.5:0/3659312451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cd67fc640 1 --2- 172.21.15.5:0/3659312451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 0x562ef261a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/3659312451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 msgr2=0x562ef261a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/3659312451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 0x562ef261a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/3659312451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef261bbb0 msgr2=0x562ef2615a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cd77fe640 1 --2- 172.21.15.5:0/3659312451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef261bbb0 0x562ef2615a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/3659312451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef261bbb0 0x562ef2615a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cd6ffd640 1 -- 172.21.15.5:0/3659312451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ef261b090 con 0x562ef26109d0 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.765+0000 7f2cd67fc640 1 --2- 172.21.15.5:0/3659312451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 0x562ef261a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.766+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/3659312451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef262d7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80107e0 tx=0x7f2cc80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356420fd2bd3fbd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.766+0000 7f2cbcff9640 1 -- 172.21.15.5:0/3659312451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc801f070 con 0x562ef26109d0 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.766+0000 7f2cbcff9640 1 -- 172.21.15.5:0/3659312451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc8004780 con 0x562ef26109d0 2024-09-15T22:06:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.766+0000 7f2cbcff9640 1 -- 172.21.15.5:0/3659312451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8026470 con 0x562ef26109d0 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.766+0000 7f2cdfa97740 1 -- 172.21.15.5:0/3659312451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 msgr2=0x562ef262d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.766+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/3659312451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef262d7c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2cc80107e0 tx=0x7f2cc80058d0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.766+0000 7f2cdfa97740 1 -- 172.21.15.5:0/3659312451 shutdown_connections 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.766+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/3659312451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef261bbb0 0x562ef2615a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.766+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/3659312451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 0x562ef261a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.766+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/3659312451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef26109d0 0x562ef262d7c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.766+0000 7f2cdfa97740 1 -- 172.21.15.5:0/3659312451 >> 172.21.15.5:0/3659312451 conn(0x562ef260cf60 msgr2=0x562ef2619f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cdfa97740 1 -- 172.21.15.5:0/3659312451 shutdown_connections 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cdfa97740 1 -- 172.21.15.5:0/3659312451 wait complete. 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cdfa97740 1 -- start start 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cdfa97740 1 Processor -- start 2024-09-15T22:06:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26109d0 0x562ef2631100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 0x562ef2631640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cdfa97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef261bbb0 0x562ef268ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562ef262d640 con 0x562ef261bbb0 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562ef262cf70 con 0x562ef26109d0 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cdfa97740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562ef262dd10 con 0x562ef26112f0 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cdfa97740 10 client.0 _ll_register_callbacks cb 0x7f2cd0013540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cd77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef261bbb0 0x562ef268ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cd77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef261bbb0 0x562ef268ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36038/0 (socket says 172.21.15.5:36038) 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.767+0000 7f2cd77fe640 1 -- 172.21.15.5:0/1036993816 learned_addr learned my addr 172.21.15.5:0/1036993816 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26109d0 0x562ef2631100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cd67fc640 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 0x562ef2631640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cd77fe640 1 -- 172.21.15.5:0/1036993816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 msgr2=0x562ef2631640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:06:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cd77fe640 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 0x562ef2631640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cd77fe640 1 -- 172.21.15.5:0/1036993816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26109d0 msgr2=0x562ef2631100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cd77fe640 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26109d0 0x562ef2631100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cd77fe640 1 -- 172.21.15.5:0/1036993816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ef268f0c0 con 0x562ef261bbb0 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cd67fc640 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 0x562ef2631640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cd77fe640 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef261bbb0 0x562ef268ea20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000eaa0 tx=0x7f2cd0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3748a77e296585 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cbaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1036993816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0011b40 con 0x562ef261bbb0 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1036993816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f2cd0011ce0 con 0x562ef261bbb0 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1036993816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc8004780 con 0x562ef261bbb0 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1036993816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f2cd0021440 con 0x562ef261bbb0 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cdfa97740 10 client.14706 fetch_fsmap learned FSMap version 16 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1036993816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x562ef2696440 con 0x562ef261bbb0 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.768+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1036993816 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc8004780 con 0x562ef261bbb0 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.769+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1036993816 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7f2cd002a5f0 con 0x562ef261bbb0 2024-09-15T22:06:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.769+0000 7f2cdfa97740 10 client.14706 fetch_fsmap finished waiting for FSMap version 16 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.769+0000 7f2cdfa97740 10 client.14706 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.769+0000 7f2cdfa97740 20 client.14706 populate_metadata read hostname 'smithi005' 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.769+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562ef269b090 0x562ef2699da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.769+0000 7f2cdfa97740 4 client.14706 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.769+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1036993816 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "status"}) -- 0x562ef257eb10 con 0x562ef269b090 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.769+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562ef269b090 0x562ef2699da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.769+0000 7f2cbaff5640 10 client.14706 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.769+0000 7f2cd6ffd640 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562ef269b090 0x562ef2699da0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37971d22e5981170 server_cookie=24dcab1c13597334 in_seq=0 out_seq=0 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cbaff5640 1 -- 172.21.15.5:0/1036993816 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x562ef257eb10 con 0x562ef269b090 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cbaff5640 10 client.14706 handle_command_reply: tid=0 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 1 client.14706 shutdown 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1036993816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef261bbb0 msgr2=0x562ef268ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef261bbb0 0x562ef268ea20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2cd000eaa0 tx=0x7f2cd0004ab0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1036993816 shutdown_connections 2024-09-15T22:06:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ef261bbb0 0x562ef268ea20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562ef26112f0 0x562ef2631640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562ef269b090 0x562ef2699da0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 1 --2- 172.21.15.5:0/1036993816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562ef26109d0 0x562ef2631100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cbaff5640 0 client.14706 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1036993816 >> 172.21.15.5:0/1036993816 conn(0x562ef260cf60 msgr2=0x562ef260c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1036993816 shutdown_connections 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 1 -- 172.21.15.5:0/1036993816 wait complete. 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdfa97740 20 client.14706 trim_cache size 0 max 16384 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-15T22:06:57.776 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24491, 2024-09-15T22:06:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-15T22:06:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-15T22:06:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-15T22:06:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 19.580101181, 2024-09-15T22:06:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 16, 2024-09-15T22:06:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 86, 2024-09-15T22:06:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:06:57.777 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 30.715820955000002 2024-09-15T22:06:57.777 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdd6be640 1 -- 172.21.15.5:0/1390378383 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2cc00a3240 msgr2=0x7f2cc00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdd6be640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2cc00a3240 0x7f2cc00a5700 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0005cc0 tx=0x7f2cd000a340 comp rx=0 tx=0).stop 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdd6be640 1 -- 172.21.15.5:0/1390378383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd8102790 msgr2=0x7f2cd81989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.770+0000 7f2cdd6be640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd8102790 0x7f2cd81989a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc004770 tx=0x7f2ccc003c10 comp rx=0 tx=0).stop 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.771+0000 7f2cdd6be640 1 -- 172.21.15.5:0/1390378383 shutdown_connections 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.771+0000 7f2cdd6be640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2cd8106f10 0x7f2cd819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.771+0000 7f2cdd6be640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f2cc00a3240 0x7f2cc00a5700 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.771+0000 7f2cdd6be640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2cd8102790 0x7f2cd81989a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.771+0000 7f2cdd6be640 1 --2- 172.21.15.5:0/1390378383 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2cd8101de0 0x7f2cd8198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.771+0000 7f2cdd6be640 1 -- 172.21.15.5:0/1390378383 >> 172.21.15.5:0/1390378383 conn(0x7f2cd80fd8c0 msgr2=0x7f2cd80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.771+0000 7f2cdd6be640 1 -- 172.21.15.5:0/1390378383 shutdown_connections 2024-09-15T22:06:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:57.771+0000 7f2cdd6be640 1 -- 172.21.15.5:0/1390378383 wait complete. 2024-09-15T22:06:58.540 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T22:06:58.540 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:06:58.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-15T22:06:58.801 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:06:58.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:58 smithi005 ceph-mon[30944]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 58 B/s, 0 objects/s recovering 2024-09-15T22:06:59.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:58 smithi099 ceph-mon[34892]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 58 B/s, 0 objects/s recovering 2024-09-15T22:06:59.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:58 smithi129 ceph-mon[35314]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 58 B/s, 0 objects/s recovering 2024-09-15T22:06:59.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.294+0000 7f42e7371640 1 -- 172.21.15.5:0/2226034445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e0100e20 msgr2=0x7f42e0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:59.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.294+0000 7f42e7371640 1 --2- 172.21.15.5:0/2226034445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e0100e20 0x7f42e0101200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009900 tx=0x7f42dc02a480 comp rx=0 tx=0).stop 2024-09-15T22:06:59.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.294+0000 7f42e7371640 1 -- 172.21.15.5:0/2226034445 shutdown_connections 2024-09-15T22:06:59.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.294+0000 7f42e7371640 1 --2- 172.21.15.5:0/2226034445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42e0102ef0 0x7f42e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:59.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.294+0000 7f42e7371640 1 --2- 172.21.15.5:0/2226034445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e0102570 0x7f42e01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.294+0000 7f42e7371640 1 --2- 172.21.15.5:0/2226034445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e0100e20 0x7f42e0101200 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.294+0000 7f42e7371640 1 -- 172.21.15.5:0/2226034445 >> 172.21.15.5:0/2226034445 conn(0x7f42e00fc7d0 msgr2=0x7f42e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.294+0000 7f42e7371640 1 -- 172.21.15.5:0/2226034445 shutdown_connections 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.294+0000 7f42e7371640 1 -- 172.21.15.5:0/2226034445 wait complete. 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e7371640 1 Processor -- start 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e7371640 1 -- start start 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e7371640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42e0100e20 0x7f42e0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e7371640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e0102570 0x7f42e0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e7371640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e0102ef0 0x7f42e019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e7371640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42e0110fa0 con 0x7f42e0102ef0 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e7371640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42e0110e20 con 0x7f42e0100e20 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e7371640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f42e0111120 con 0x7f42e0102570 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e590e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e0102ef0 0x7f42e019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:59.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e590e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e0102ef0 0x7f42e019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36070/0 (socket says 172.21.15.5:36070) 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e590e640 1 -- 172.21.15.5:0/3168426911 learned_addr learned my addr 172.21.15.5:0/3168426911 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.295+0000 7f42e510d640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42e0100e20 0x7f42e0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42e490c640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e0102570 0x7f42e0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42e590e640 1 -- 172.21.15.5:0/3168426911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e0102570 msgr2=0x7f42e0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42e590e640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e0102570 0x7f42e0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42e590e640 1 -- 172.21.15.5:0/3168426911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42e0100e20 msgr2=0x7f42e0198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42e590e640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42e0100e20 0x7f42e0198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42e590e640 1 -- 172.21.15.5:0/3168426911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42e019d540 con 0x7f42e0102ef0 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42e490c640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e0102570 0x7f42e0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42e590e640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e0102ef0 0x7f42e019ce60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42d800eaa0 tx=0x7f42d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42d67fc640 1 -- 172.21.15.5:0/3168426911 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d800cdb0 con 0x7f42e0102ef0 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42e7371640 1 -- 172.21.15.5:0/3168426911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42e019d830 con 0x7f42e0102ef0 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42d67fc640 1 -- 172.21.15.5:0/3168426911 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42d8005020 con 0x7f42e0102ef0 2024-09-15T22:06:59.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42d67fc640 1 -- 172.21.15.5:0/3168426911 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42d80116b0 con 0x7f42e0102ef0 2024-09-15T22:06:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.296+0000 7f42e7371640 1 -- 172.21.15.5:0/3168426911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42e01a5110 con 0x7f42e0102ef0 2024-09-15T22:06:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.297+0000 7f42e7371640 1 -- 172.21.15.5:0/3168426911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42e0101200 con 0x7f42e0102ef0 2024-09-15T22:06:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.298+0000 7f42d67fc640 1 -- 172.21.15.5:0/3168426911 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f42d8005560 con 0x7f42e0102ef0 2024-09-15T22:06:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.299+0000 7f42d67fc640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f42cc0a3240 0x7f42cc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:06:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.299+0000 7f42e510d640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f42cc0a3240 0x7f42cc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:06:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.299+0000 7f42d67fc640 1 -- 172.21.15.5:0/3168426911 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f42d80d02f0 con 0x7f42e0102ef0 2024-09-15T22:06:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.299+0000 7f42e510d640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f42cc0a3240 0x7f42cc0a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009230 tx=0x7f42dc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:06:59.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.302+0000 7f42d67fc640 1 -- 172.21.15.5:0/3168426911 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f42d8016030 con 0x7f42e0102ef0 2024-09-15T22:06:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.514+0000 7f42e7371640 1 -- 172.21.15.5:0/3168426911 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f42e01998f0 con 0x7f42e0102ef0 2024-09-15T22:06:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.515+0000 7f42d67fc640 1 -- 172.21.15.5:0/3168426911 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7f42e01998f0 con 0x7f42e0102ef0 2024-09-15T22:06:59.516 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-15T22:06:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.516+0000 7f42e7371640 1 -- 172.21.15.5:0/3168426911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f42cc0a3240 msgr2=0x7f42cc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.516+0000 7f42e7371640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f42cc0a3240 0x7f42cc0a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009230 tx=0x7f42dc0058d0 comp rx=0 tx=0).stop 2024-09-15T22:06:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.517+0000 7f42e7371640 1 -- 172.21.15.5:0/3168426911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e0102ef0 msgr2=0x7f42e019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:06:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.517+0000 7f42e7371640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e0102ef0 0x7f42e019ce60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f42d800eaa0 tx=0x7f42d800ef60 comp rx=0 tx=0).stop 2024-09-15T22:06:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.517+0000 7f42e7371640 1 -- 172.21.15.5:0/3168426911 shutdown_connections 2024-09-15T22:06:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.517+0000 7f42e7371640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f42cc0a3240 0x7f42cc0a5700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.517+0000 7f42e7371640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42e0102ef0 0x7f42e019ce60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.517+0000 7f42e7371640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f42e0102570 0x7f42e0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.517+0000 7f42e7371640 1 --2- 172.21.15.5:0/3168426911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f42e0100e20 0x7f42e0198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:06:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.517+0000 7f42e7371640 1 -- 172.21.15.5:0/3168426911 >> 172.21.15.5:0/3168426911 conn(0x7f42e00fc7d0 msgr2=0x7f42e006a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:06:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.517+0000 7f42e7371640 1 -- 172.21.15.5:0/3168426911 shutdown_connections 2024-09-15T22:06:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:06:59.517+0000 7f42e7371640 1 -- 172.21.15.5:0/3168426911 wait complete. 2024-09-15T22:06:59.826 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:06:59 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3168426911' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:07:00.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:06:59 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3168426911' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:07:00.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:06:59 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3168426911' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:07:00.143 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-15T22:07:00.143 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T22:07:00.143 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-09-15T22:07:00.143 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-15T22:07:00.143 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:07:00.143 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-15T22:07:00.143 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T22:07:00.144 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T22:07:00.144 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-15T22:07:00.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:00.144 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-09-15T22:07:00.162 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T22:07:00.162 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T22:07:00.162 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T22:07:00.162 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:07:00.162 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-09-15T22:07:00.162 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:07:00.162 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:07:00.162 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-09-15T22:07:00.162 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-09-15T22:07:00.162 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42300sec preferred_lft 42300sec 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:07:00.163 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-09-15T22:07:00.164 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-15T22:07:00.164 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:07:00.164 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-15T22:07:00.164 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-09-15T22:07:00.164 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-09-15T22:07:00.164 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-09-15T22:07:00.164 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-15T22:07:00.165 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-15T22:07:00.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:07:00.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:07:00.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:00.388 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T22:07:00.414 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-09-15T22:07:00.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:00.416 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-09-15T22:07:00.501 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-09-15T22:07:00.501 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T22:07:00.501 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-15T22:07:00.501 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-15T22:07:00.501 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T22:07:00.502 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:07:00.502 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-15T22:07:00.502 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-15T22:07:00.502 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-15T22:07:00.502 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-15T22:07:00.502 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-15T22:07:00.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:07:00.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:07:00.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:00.813 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-15T22:07:00.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:00.830 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-09-15T22:07:00.886 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:07:00.886 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-15T22:07:00.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:07:00.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-15T22:07:00.886 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-15T22:07:00.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:07:00.983 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:00 smithi005 ceph-mon[30944]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 52 B/s, 0 objects/s recovering 2024-09-15T22:07:00.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:07:01.006 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-15T22:07:01.006 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:07:01.006 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-15T22:07:01.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-15T22:07:01.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-15T22:07:01.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-15T22:07:01.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-15T22:07:01.006 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-15T22:07:01.006 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-15T22:07:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:07:01.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:00 smithi099 ceph-mon[34892]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 52 B/s, 0 objects/s recovering 2024-09-15T22:07:01.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:00 smithi129 ceph-mon[35314]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 52 B/s, 0 objects/s recovering 2024-09-15T22:07:01.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:07:01.133 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:07:01.133 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-15T22:07:01.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-09-15T22:07:01.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-09-15T22:07:01.133 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-15T22:07:01.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:07:01.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:07:01.215 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-15T22:07:01.215 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:07:01.215 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:07:01.231 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-15T22:07:01.231 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:07:01.231 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:07:01.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-09-15T22:07:01.354 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-09-15T22:07:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-09-15T22:07:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-09-15T22:07:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-09-15T22:07:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-09-15T22:07:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-09-15T22:07:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-09-15T22:07:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/d9919bd38868efac62a64ced842c98781a4aae485929fecdae89356d25da0873/merged 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/19d0969efd62277363b779c05f709445a592d6c592ccd59be6dbb88da18969e9/merged 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/0 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/64af27383b4f441589ed7612b3ba45ec6fc96d746a524155f81c2d8e284c0f5f/merged 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/d97a7ceee00496b3797785aa6ce4f4070837939fa21d9674db481c66615acc84/merged 2024-09-15T22:07:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/978c9fcf9c52816845c791885727ecc2c74000b21868153ab31ca52c5bccd86a/merged 2024-09-15T22:07:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/a570665ea4a27305dae5929dff31c77ab727d8d1df05f43c89e531e1766f5a3b/merged 2024-09-15T22:07:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/ba3339e07f8d43b190ef167a5236b880b21b42e7d214855c4396818ad5a5ac2a/merged 2024-09-15T22:07:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/a4a75b5048794f23e1c6fa0203ff67adcd417d7b37fac8c5162f4b2cfb10847a/merged 2024-09-15T22:07:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/71cfd08d78763dd853953867d14af836bb60b34579aa22d90e990207fe5aa31f/merged 2024-09-15T22:07:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/e4e7ff231bdd6c49e7410ea3cada0bf0a8743529212e2c163d31f0fe5c0af3ff/merged 2024-09-15T22:07:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-09-15T22:07:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:07:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:07:01.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:01.414 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-09-15T22:07:01.467 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-15T22:07:01.468 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-15T22:07:01.511 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-09-15T22:07:01.536 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-09-15T22:07:01.593 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-09-15T22:07:01.594 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/d9919bd38868efac62a64ced842c98781a4aae485929fecdae89356d25da0873/merged 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/19d0969efd62277363b779c05f709445a592d6c592ccd59be6dbb88da18969e9/merged 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/0 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/64af27383b4f441589ed7612b3ba45ec6fc96d746a524155f81c2d8e284c0f5f/merged 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/d97a7ceee00496b3797785aa6ce4f4070837939fa21d9674db481c66615acc84/merged 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/978c9fcf9c52816845c791885727ecc2c74000b21868153ab31ca52c5bccd86a/merged 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/a570665ea4a27305dae5929dff31c77ab727d8d1df05f43c89e531e1766f5a3b/merged 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/ba3339e07f8d43b190ef167a5236b880b21b42e7d214855c4396818ad5a5ac2a/merged 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/a4a75b5048794f23e1c6fa0203ff67adcd417d7b37fac8c5162f4b2cfb10847a/merged 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/71cfd08d78763dd853953867d14af836bb60b34579aa22d90e990207fe5aa31f/merged 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/e4e7ff231bdd6c49e7410ea3cada0bf0a8743529212e2c163d31f0fe5c0af3ff/merged 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-09-15T22:07:01.595 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:07:01.596 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:07:01.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:01.596 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-09-15T22:07:01.622 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:2024-09-15T22:07:01.621+0000 7f67f5322480 -1 init, newargv = 0x555d9a9f3b10 newargc=15 2024-09-15T22:07:01.622 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[74606]: starting ceph client 2024-09-15T22:07:01.628 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[74606]: starting fuse 2024-09-15T22:07:01.639 INFO:teuthology.orchestra.run.smithi005.stdout:120 2024-09-15T22:07:01.639 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-15T22:07:01.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2024-09-15T22:07:01.639 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2024-09-15T22:07:01.639 DEBUG:teuthology.orchestra.run.smithi005:> import re 2024-09-15T22:07:01.639 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-09-15T22:07:01.639 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2024-09-15T22:07:01.639 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-15T22:07:01.640 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-15T22:07:01.641 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2024-09-15T22:07:01.641 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2024-09-15T22:07:01.641 DEBUG:teuthology.orchestra.run.smithi005:> return f 2024-09-15T22:07:01.641 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-15T22:07:01.641 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-15T22:07:01.641 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.admin")) 2024-09-15T22:07:01.641 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-09-15T22:07:01.759 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.admin.74606.asok 2024-09-15T22:07:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:07:01.770 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74606.asok 2024-09-15T22:07:01.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:01.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74606.asok status 2024-09-15T22:07:01.907 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:07:01.907 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2024-09-15T22:07:01.907 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-15T22:07:01.907 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-15T22:07:01.907 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "admin", 2024-09-15T22:07:01.907 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2024-09-15T22:07:01.907 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-15T22:07:01.907 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "74606", 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/" 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 14718, 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 14718 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-09-15T22:07:01.908 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1422879311 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1422879311 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.14718 192.168.0.1:0/1422879311", 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/1422879311", 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 1, 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 16, 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 86, 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2024-09-15T22:07:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2024-09-15T22:07:01.910 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:07:01.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:01.916 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2024-09-15T22:07:01.916 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:07:01.932 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2024-09-15T22:07:01.932 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:07:01.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:01.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:07:01.999 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-15T22:07:02.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:07:02.113 INFO:teuthology.orchestra.run.smithi005.stdout:# file: . 2024-09-15T22:07:02.113 INFO:teuthology.orchestra.run.smithi005.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-15T22:07:02.113 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:07:02.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:07:02.119 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T22:07:02.119 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:07:02.137 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-15T22:07:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:02.842 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:02 smithi005 ceph-mon[30944]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 48 B/s, 0 objects/s recovering 2024-09-15T22:07:02.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.925+0000 7f14d7c79640 1 -- 172.21.15.5:0/3603463591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14d0101280 msgr2=0x7f14d010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.925+0000 7f14d7c79640 1 --2- 172.21.15.5:0/3603463591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14d0101280 0x7f14d010df40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009900 tx=0x7f14c002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 -- 172.21.15.5:0/3603463591 shutdown_connections 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 --2- 172.21.15.5:0/3603463591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d010e480 0x7f14d0110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 --2- 172.21.15.5:0/3603463591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14d0101280 0x7f14d010df40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 --2- 172.21.15.5:0/3603463591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14d01008d0 0x7f14d0100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 -- 172.21.15.5:0/3603463591 >> 172.21.15.5:0/3603463591 conn(0x7f14d00fc7a0 msgr2=0x7f14d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 -- 172.21.15.5:0/3603463591 shutdown_connections 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 -- 172.21.15.5:0/3603463591 wait complete. 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 Processor -- start 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 -- start start 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d01008d0 0x7f14d0102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14d0101280 0x7f14d01030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14d010e480 0x7f14d0103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14d0114190 con 0x7f14d010e480 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14d0114010 con 0x7f14d01008d0 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.926+0000 7f14d7c79640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f14d0114310 con 0x7f14d0101280 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d6216640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14d010e480 0x7f14d0103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d6216640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14d010e480 0x7f14d0103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36092/0 (socket says 172.21.15.5:36092) 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d6216640 1 -- 172.21.15.5:0/2972048418 learned_addr learned my addr 172.21.15.5:0/2972048418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d5214640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14d0101280 0x7f14d01030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d5a15640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d01008d0 0x7f14d0102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d6216640 1 -- 172.21.15.5:0/2972048418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14d0101280 msgr2=0x7f14d01030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d6216640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14d0101280 0x7f14d01030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d6216640 1 -- 172.21.15.5:0/2972048418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d01008d0 msgr2=0x7f14d0102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d6216640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d01008d0 0x7f14d0102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d6216640 1 -- 172.21.15.5:0/2972048418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d01aad90 con 0x7f14d010e480 2024-09-15T22:07:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d6216640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14d010e480 0x7f14d0103630 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14c800cd10 tx=0x7f14c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14c6ffd640 1 -- 172.21.15.5:0/2972048418 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c800f450 con 0x7f14d010e480 2024-09-15T22:07:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14c6ffd640 1 -- 172.21.15.5:0/2972048418 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c800fad0 con 0x7f14d010e480 2024-09-15T22:07:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14c6ffd640 1 -- 172.21.15.5:0/2972048418 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8018bb0 con 0x7f14d010e480 2024-09-15T22:07:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d7c79640 1 -- 172.21.15.5:0/2972048418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d01ab080 con 0x7f14d010e480 2024-09-15T22:07:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.927+0000 7f14d7c79640 1 -- 172.21.15.5:0/2972048418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d01ab590 con 0x7f14d010e480 2024-09-15T22:07:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.928+0000 7f14d7c79640 1 -- 172.21.15.5:0/2972048418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d006b070 con 0x7f14d010e480 2024-09-15T22:07:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.931+0000 7f14c6ffd640 1 -- 172.21.15.5:0/2972048418 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f14c800f5f0 con 0x7f14d010e480 2024-09-15T22:07:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.931+0000 7f14c6ffd640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f14bc0a32c0 0x7f14bc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.931+0000 7f14d5a15640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f14bc0a32c0 0x7f14bc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.931+0000 7f14c6ffd640 1 -- 172.21.15.5:0/2972048418 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f14c80d01e0 con 0x7f14d010e480 2024-09-15T22:07:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.932+0000 7f14d5a15640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f14bc0a32c0 0x7f14bc0a5780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14cc005fd0 tx=0x7f14cc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:02.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:02.935+0000 7f14c6ffd640 1 -- 172.21.15.5:0/2972048418 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14c8014030 con 0x7f14d010e480 2024-09-15T22:07:03.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:02 smithi099 ceph-mon[34892]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 48 B/s, 0 objects/s recovering 2024-09-15T22:07:03.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:02 smithi129 ceph-mon[35314]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 48 B/s, 0 objects/s recovering 2024-09-15T22:07:03.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.126+0000 7f14d7c79640 1 -- 172.21.15.5:0/2972048418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f14d0100cb0 con 0x7f14d010e480 2024-09-15T22:07:03.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.126+0000 7f14c6ffd640 1 -- 172.21.15.5:0/2972048418 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+582 (secure 0 0 0) 0x7f14c809cc40 con 0x7f14d010e480 2024-09-15T22:07:03.128 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3146025957 2024-09-16T21:59:42.685255+0000 2024-09-15T22:07:03.128 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3880791045 2024-09-16T21:59:42.685255+0000 2024-09-15T22:07:03.128 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3326533830 2024-09-16T21:59:42.685255+0000 2024-09-15T22:07:03.128 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/3785442724 2024-09-16T21:59:42.685255+0000 2024-09-15T22:07:03.128 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1445520321 2024-09-16T21:59:18.773927+0000 2024-09-15T22:07:03.128 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2658926924 2024-09-16T21:59:18.773927+0000 2024-09-15T22:07:03.128 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4013564474 2024-09-16T21:59:18.773927+0000 2024-09-15T22:07:03.128 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/3785442724 2024-09-16T21:59:42.685255+0000 2024-09-15T22:07:03.129 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1812503826 2024-09-16T21:59:18.773927+0000 2024-09-15T22:07:03.129 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1812503826 2024-09-16T21:59:18.773927+0000 2024-09-15T22:07:03.129 INFO:teuthology.orchestra.run.smithi005.stderr:listed 10 entries 2024-09-15T22:07:03.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.128+0000 7f14d7c79640 1 -- 172.21.15.5:0/2972048418 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f14bc0a32c0 msgr2=0x7f14bc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:03.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.128+0000 7f14d7c79640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f14bc0a32c0 0x7f14bc0a5780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14cc005fd0 tx=0x7f14cc005850 comp rx=0 tx=0).stop 2024-09-15T22:07:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.128+0000 7f14d7c79640 1 -- 172.21.15.5:0/2972048418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14d010e480 msgr2=0x7f14d0103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.128+0000 7f14d7c79640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14d010e480 0x7f14d0103630 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14c800cd10 tx=0x7f14c800a620 comp rx=0 tx=0).stop 2024-09-15T22:07:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.129+0000 7f14d7c79640 1 -- 172.21.15.5:0/2972048418 shutdown_connections 2024-09-15T22:07:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.129+0000 7f14d7c79640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f14bc0a32c0 0x7f14bc0a5780 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.129+0000 7f14d7c79640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14d010e480 0x7f14d0103630 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.129+0000 7f14d7c79640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14d0101280 0x7f14d01030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.129+0000 7f14d7c79640 1 --2- 172.21.15.5:0/2972048418 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14d01008d0 0x7f14d0102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.129+0000 7f14d7c79640 1 -- 172.21.15.5:0/2972048418 >> 172.21.15.5:0/2972048418 conn(0x7f14d00fc7a0 msgr2=0x7f14d00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.129+0000 7f14d7c79640 1 -- 172.21.15.5:0/2972048418 shutdown_connections 2024-09-15T22:07:03.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:03.129+0000 7f14d7c79640 1 -- 172.21.15.5:0/2972048418 wait complete. 2024-09-15T22:07:03.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:03 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2972048418' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:07:03.924 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T22:07:03.924 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:07:03.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-15T22:07:04.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:03 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2972048418' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:07:04.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:03 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2972048418' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:07:04.245 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.716+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/33473967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f68102d40 msgr2=0x7f9f68103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.716+0000 7f9f6ecc9640 1 --2- 172.21.15.5:0/33473967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f68102d40 0x7f9f68103120 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009ca0 tx=0x7f9f5802a950 comp rx=0 tx=0).stop 2024-09-15T22:07:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.716+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/33473967 shutdown_connections 2024-09-15T22:07:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.716+0000 7f9f6ecc9640 1 --2- 172.21.15.5:0/33473967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f6810c170 0x7f9f6810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.716+0000 7f9f6ecc9640 1 --2- 172.21.15.5:0/33473967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f68103660 0x7f9f6810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.716+0000 7f9f6ecc9640 1 --2- 172.21.15.5:0/33473967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f68102d40 0x7f9f68103120 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.716+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/33473967 >> 172.21.15.5:0/33473967 conn(0x7f9f680fc7a0 msgr2=0x7f9f680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.716+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/33473967 shutdown_connections 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.716+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/33473967 wait complete. 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.717+0000 7f9f6ecc9640 1 Processor -- start 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.717+0000 7f9f6ecc9640 1 -- start start 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f6ecc9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f68102d40 0x7f9f681983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f6ecc9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f68103660 0x7f9f68198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f6ecc9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6810c170 0x7f9f6819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f6ecc9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9f68111f70 con 0x7f9f68103660 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f6ecc9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9f68111df0 con 0x7f9f68102d40 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f6ecc9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9f681120f0 con 0x7f9f6810c170 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f67fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f68103660 0x7f9f68198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f67fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f68103660 0x7f9f68198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52238/0 (socket says 172.21.15.5:52238) 2024-09-15T22:07:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f67fff640 1 -- 172.21.15.5:0/3907846888 learned_addr learned my addr 172.21.15.5:0/3907846888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f6ca65640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f68102d40 0x7f9f681983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f6d266640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6810c170 0x7f9f6819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f67fff640 1 -- 172.21.15.5:0/3907846888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6810c170 msgr2=0x7f9f6819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f67fff640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6810c170 0x7f9f6819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f67fff640 1 -- 172.21.15.5:0/3907846888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f68102d40 msgr2=0x7f9f681983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f67fff640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f68102d40 0x7f9f681983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f67fff640 1 -- 172.21.15.5:0/3907846888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6819d390 con 0x7f9f68103660 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f6ca65640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f68102d40 0x7f9f681983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f6d266640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6810c170 0x7f9f6819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f67fff640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f68103660 0x7f9f68198920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000d740 tx=0x7f9f5000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f65ffb640 1 -- 172.21.15.5:0/3907846888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50017070 con 0x7f9f68103660 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/3907846888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6819d680 con 0x7f9f68103660 2024-09-15T22:07:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f65ffb640 1 -- 172.21.15.5:0/3907846888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f5000de50 con 0x7f9f68103660 2024-09-15T22:07:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.718+0000 7f9f65ffb640 1 -- 172.21.15.5:0/3907846888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5000f620 con 0x7f9f68103660 2024-09-15T22:07:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.719+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/3907846888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f681a4f60 con 0x7f9f68103660 2024-09-15T22:07:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.720+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/3907846888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f68103120 con 0x7f9f68103660 2024-09-15T22:07:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.721+0000 7f9f65ffb640 1 -- 172.21.15.5:0/3907846888 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f9f50009c30 con 0x7f9f68103660 2024-09-15T22:07:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.722+0000 7f9f65ffb640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9f540a3240 0x7f9f540a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.722+0000 7f9f6ca65640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9f540a3240 0x7f9f540a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.722+0000 7f9f65ffb640 1 -- 172.21.15.5:0/3907846888 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f9f500cf900 con 0x7f9f68103660 2024-09-15T22:07:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.722+0000 7f9f6ca65640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9f540a3240 0x7f9f540a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9f58006010 tx=0x7f9f58005f80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.726+0000 7f9f65ffb640 1 -- 172.21.15.5:0/3907846888 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f5009c2e0 con 0x7f9f68103660 2024-09-15T22:07:04.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.919+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/3907846888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9f6802d7c0 con 0x7f9f68103660 2024-09-15T22:07:04.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:04 smithi005 ceph-mon[30944]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:07:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.919+0000 7f9f65ffb640 1 -- 172.21.15.5:0/3907846888 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v86) ==== 81+0+582 (secure 0 0 0) 0x7f9f5009bbe0 con 0x7f9f68103660 2024-09-15T22:07:04.921 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3146025957 2024-09-16T21:59:42.685255+0000 2024-09-15T22:07:04.921 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3880791045 2024-09-16T21:59:42.685255+0000 2024-09-15T22:07:04.921 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3326533830 2024-09-16T21:59:42.685255+0000 2024-09-15T22:07:04.921 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/3785442724 2024-09-16T21:59:42.685255+0000 2024-09-15T22:07:04.922 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1445520321 2024-09-16T21:59:18.773927+0000 2024-09-15T22:07:04.922 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2658926924 2024-09-16T21:59:18.773927+0000 2024-09-15T22:07:04.922 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4013564474 2024-09-16T21:59:18.773927+0000 2024-09-15T22:07:04.922 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/3785442724 2024-09-16T21:59:42.685255+0000 2024-09-15T22:07:04.922 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1812503826 2024-09-16T21:59:18.773927+0000 2024-09-15T22:07:04.922 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1812503826 2024-09-16T21:59:18.773927+0000 2024-09-15T22:07:04.922 INFO:teuthology.orchestra.run.smithi005.stderr:listed 10 entries 2024-09-15T22:07:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.921+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/3907846888 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9f540a3240 msgr2=0x7f9f540a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.921+0000 7f9f6ecc9640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9f540a3240 0x7f9f540a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9f58006010 tx=0x7f9f58005f80 comp rx=0 tx=0).stop 2024-09-15T22:07:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.921+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/3907846888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f68103660 msgr2=0x7f9f68198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.921+0000 7f9f6ecc9640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f68103660 0x7f9f68198920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000d740 tx=0x7f9f5000dc00 comp rx=0 tx=0).stop 2024-09-15T22:07:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.921+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/3907846888 shutdown_connections 2024-09-15T22:07:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.921+0000 7f9f6ecc9640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9f6810c170 0x7f9f6819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.921+0000 7f9f6ecc9640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f9f540a3240 0x7f9f540a5700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.921+0000 7f9f6ecc9640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f68103660 0x7f9f68198920 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.921+0000 7f9f6ecc9640 1 --2- 172.21.15.5:0/3907846888 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9f68102d40 0x7f9f681983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.921+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/3907846888 >> 172.21.15.5:0/3907846888 conn(0x7f9f680fc7a0 msgr2=0x7f9f680fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.922+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/3907846888 shutdown_connections 2024-09-15T22:07:04.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:04.922+0000 7f9f6ecc9640 1 -- 172.21.15.5:0/3907846888 wait complete. 2024-09-15T22:07:05.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:04 smithi099 ceph-mon[34892]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:07:05.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:04 smithi129 ceph-mon[35314]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:07:05.728 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2024-09-15T22:07:05.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:05.729 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-15T22:07:05.756 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[74606]: fuse finished with error 0 and tester_r 0 2024-09-15T22:07:05.758 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T22:07:05.805 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-09-15T22:07:05.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:07:05.805 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-15T22:07:05.829 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:07:06.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:05 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3907846888' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:07:06.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:05 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3907846888' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:07:06.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:05 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3907846888' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:07:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.582+0000 7f26ce79d640 1 -- 172.21.15.5:0/2328584221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c81034b0 msgr2=0x7f26c810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.582+0000 7f26ce79d640 1 --2- 172.21.15.5:0/2328584221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c81034b0 0x7f26c810b9a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009900 tx=0x7f26bc02a480 comp rx=0 tx=0).stop 2024-09-15T22:07:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.582+0000 7f26ce79d640 1 -- 172.21.15.5:0/2328584221 shutdown_connections 2024-09-15T22:07:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.582+0000 7f26ce79d640 1 --2- 172.21.15.5:0/2328584221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26c810bfc0 0x7f26c810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.582+0000 7f26ce79d640 1 --2- 172.21.15.5:0/2328584221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c81034b0 0x7f26c810b9a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.582+0000 7f26ce79d640 1 --2- 172.21.15.5:0/2328584221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26c8102b90 0x7f26c8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.582+0000 7f26ce79d640 1 -- 172.21.15.5:0/2328584221 >> 172.21.15.5:0/2328584221 conn(0x7f26c80fc630 msgr2=0x7f26c80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.582+0000 7f26ce79d640 1 -- 172.21.15.5:0/2328584221 shutdown_connections 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.582+0000 7f26ce79d640 1 -- 172.21.15.5:0/2328584221 wait complete. 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26ce79d640 1 Processor -- start 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26ce79d640 1 -- start start 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26ce79d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26c8102b90 0x7f26c81982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26ce79d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c81034b0 0x7f26c8198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26ce79d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26c810bfc0 0x7f26c819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26ce79d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26c8111e00 con 0x7f26c81034b0 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26ce79d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26c8111c80 con 0x7f26c810bfc0 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26ce79d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f26c8111f80 con 0x7f26c8102b90 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26ccf9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c81034b0 0x7f26c8198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26ccf9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c81034b0 0x7f26c8198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52246/0 (socket says 172.21.15.5:52246) 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26ccf9a640 1 -- 172.21.15.5:0/96629895 learned_addr learned my addr 172.21.15.5:0/96629895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.583+0000 7f26cd79b640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26c8102b90 0x7f26c81982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26cdf9c640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26c810bfc0 0x7f26c819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26ccf9a640 1 -- 172.21.15.5:0/96629895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26c8102b90 msgr2=0x7f26c81982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26ccf9a640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26c8102b90 0x7f26c81982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26ccf9a640 1 -- 172.21.15.5:0/96629895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26c810bfc0 msgr2=0x7f26c819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26ccf9a640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26c810bfc0 0x7f26c819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26ccf9a640 1 -- 172.21.15.5:0/96629895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26c819d270 con 0x7f26c81034b0 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26cdf9c640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26c810bfc0 0x7f26c819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26ccf9a640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c81034b0 0x7f26c8198800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f26bc006010 tx=0x7f26bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26ba7fc640 1 -- 172.21.15.5:0/96629895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc03f030 con 0x7f26c81034b0 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26ba7fc640 1 -- 172.21.15.5:0/96629895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26bc003ce0 con 0x7f26c81034b0 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26ce79d640 1 -- 172.21.15.5:0/96629895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26c819d500 con 0x7f26c81034b0 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26ba7fc640 1 -- 172.21.15.5:0/96629895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26bc035070 con 0x7f26c81034b0 2024-09-15T22:07:06.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.584+0000 7f26ce79d640 1 -- 172.21.15.5:0/96629895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26c81a4e40 con 0x7f26c81034b0 2024-09-15T22:07:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.585+0000 7f26ce79d640 1 -- 172.21.15.5:0/96629895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26c806b070 con 0x7f26c81034b0 2024-09-15T22:07:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.586+0000 7f26ba7fc640 1 -- 172.21.15.5:0/96629895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f26bc042030 con 0x7f26c81034b0 2024-09-15T22:07:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.587+0000 7f26ba7fc640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f26b40a3240 0x7f26b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.587+0000 7f26ba7fc640 1 -- 172.21.15.5:0/96629895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f26bc0efd70 con 0x7f26c81034b0 2024-09-15T22:07:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.587+0000 7f26cd79b640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f26b40a3240 0x7f26b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.588+0000 7f26cd79b640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f26b40a3240 0x7f26b40a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f26c400bbe0 tx=0x7f26c400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.591+0000 7f26ba7fc640 1 -- 172.21.15.5:0/96629895 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26bc0bc810 con 0x7f26c81034b0 2024-09-15T22:07:06.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.769+0000 7f26ce79d640 1 -- 172.21.15.5:0/96629895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26c8102f70 con 0x7f26c81034b0 2024-09-15T22:07:06.772 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:06 smithi005 ceph-mon[30944]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-15T22:07:06.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.770+0000 7f26ba7fc640 1 -- 172.21.15.5:0/96629895 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24549 (secure 0 0 0) 0x7f26bc0bc110 con 0x7f26c81034b0 2024-09-15T22:07:06.773 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:07:06.773 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:06:47.363102+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:07:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 -- 172.21.15.5:0/96629895 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f26b40a3240 msgr2=0x7f26b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f26b40a3240 0x7f26b40a5700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f26c400bbe0 tx=0x7f26c400b390 comp rx=0 tx=0).stop 2024-09-15T22:07:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 -- 172.21.15.5:0/96629895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c81034b0 msgr2=0x7f26c8198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:06.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c81034b0 0x7f26c8198800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f26bc006010 tx=0x7f26bc0042f0 comp rx=0 tx=0).stop 2024-09-15T22:07:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 -- 172.21.15.5:0/96629895 shutdown_connections 2024-09-15T22:07:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26c810bfc0 0x7f26c819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f26b40a3240 0x7f26b40a5700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26c81034b0 0x7f26c8198800 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 --2- 172.21.15.5:0/96629895 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26c8102b90 0x7f26c81982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 -- 172.21.15.5:0/96629895 >> 172.21.15.5:0/96629895 conn(0x7f26c80fc630 msgr2=0x7f26c80fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 -- 172.21.15.5:0/96629895 shutdown_connections 2024-09-15T22:07:06.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:06.773+0000 7f26ce79d640 1 -- 172.21.15.5:0/96629895 wait complete. 2024-09-15T22:07:07.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:06 smithi099 ceph-mon[34892]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-15T22:07:07.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:06 smithi129 ceph-mon[35314]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-09-15T22:07:07.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:07:07.847 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:07.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:07 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/96629895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:08.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:07 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/96629895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:08.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:07 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/96629895' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:08.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.332+0000 7f55c62e8640 1 -- 172.21.15.5:0/477727093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c0102790 msgr2=0x7f55c01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:08.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.332+0000 7f55c62e8640 1 --2- 172.21.15.5:0/477727093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c0102790 0x7f55c01067e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009900 tx=0x7f55a802a410 comp rx=0 tx=0).stop 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.332+0000 7f55c62e8640 1 -- 172.21.15.5:0/477727093 shutdown_connections 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.332+0000 7f55c62e8640 1 --2- 172.21.15.5:0/477727093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c0106f10 0x7f55c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.332+0000 7f55c62e8640 1 --2- 172.21.15.5:0/477727093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c0102790 0x7f55c01067e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.332+0000 7f55c62e8640 1 --2- 172.21.15.5:0/477727093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c0101de0 0x7f55c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.332+0000 7f55c62e8640 1 -- 172.21.15.5:0/477727093 >> 172.21.15.5:0/477727093 conn(0x7f55c00fd8c0 msgr2=0x7f55c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.332+0000 7f55c62e8640 1 -- 172.21.15.5:0/477727093 shutdown_connections 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.332+0000 7f55c62e8640 1 -- 172.21.15.5:0/477727093 wait complete. 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55c62e8640 1 Processor -- start 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55c62e8640 1 -- start start 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55c62e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c0101de0 0x7f55c0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55c62e8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c0102790 0x7f55c01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:08.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55c62e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c0106f10 0x7f55c019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55c62e8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55c0111f70 con 0x7f55c0106f10 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55c62e8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55c0111df0 con 0x7f55c0101de0 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55c62e8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55c01120f0 con 0x7f55c0102790 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55c4885640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c0106f10 0x7f55c019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55c4885640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c0106f10 0x7f55c019cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52268/0 (socket says 172.21.15.5:52268) 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55c4885640 1 -- 172.21.15.5:0/3160040194 learned_addr learned my addr 172.21.15.5:0/3160040194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55b7fff640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c0102790 0x7f55c01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.333+0000 7f55bffff640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c0101de0 0x7f55c0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.334+0000 7f55c4885640 1 -- 172.21.15.5:0/3160040194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c0102790 msgr2=0x7f55c01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.334+0000 7f55c4885640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c0102790 0x7f55c01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.334+0000 7f55c4885640 1 -- 172.21.15.5:0/3160040194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c0101de0 msgr2=0x7f55c0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.334+0000 7f55c4885640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c0101de0 0x7f55c0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:08.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.334+0000 7f55c4885640 1 -- 172.21.15.5:0/3160040194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c019d440 con 0x7f55c0106f10 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.334+0000 7f55c4885640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c0106f10 0x7f55c019cd60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f55b800eaa0 tx=0x7f55b800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.334+0000 7f55bdffb640 1 -- 172.21.15.5:0/3160040194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b800cdb0 con 0x7f55c0106f10 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.334+0000 7f55bdffb640 1 -- 172.21.15.5:0/3160040194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b8005020 con 0x7f55c0106f10 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.334+0000 7f55c62e8640 1 -- 172.21.15.5:0/3160040194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c019d730 con 0x7f55c0106f10 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.334+0000 7f55bdffb640 1 -- 172.21.15.5:0/3160040194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b80116b0 con 0x7f55c0106f10 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.334+0000 7f55c62e8640 1 -- 172.21.15.5:0/3160040194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c01a5010 con 0x7f55c0106f10 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.335+0000 7f55c62e8640 1 -- 172.21.15.5:0/3160040194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c01021c0 con 0x7f55c0106f10 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.336+0000 7f55bdffb640 1 -- 172.21.15.5:0/3160040194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f55b8005560 con 0x7f55c0106f10 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.337+0000 7f55bdffb640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55ac0a3240 0x7f55ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.337+0000 7f55bdffb640 1 -- 172.21.15.5:0/3160040194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f55b80cfee0 con 0x7f55c0106f10 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.337+0000 7f55bffff640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55ac0a3240 0x7f55ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.337+0000 7f55bffff640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55ac0a3240 0x7f55ac0a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f55b0005fd0 tx=0x7f55b0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:08.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.341+0000 7f55bdffb640 1 -- 172.21.15.5:0/3160040194 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55b8016030 con 0x7f55c0106f10 2024-09-15T22:07:08.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.517+0000 7f55c62e8640 1 -- 172.21.15.5:0/3160040194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55c0109b60 con 0x7f55c0106f10 2024-09-15T22:07:08.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.519+0000 7f55bdffb640 1 -- 172.21.15.5:0/3160040194 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24549 (secure 0 0 0) 0x7f55b809c8c0 con 0x7f55c0106f10 2024-09-15T22:07:08.522 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:07:08.522 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:06:47.363102+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:07:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.521+0000 7f55c62e8640 1 -- 172.21.15.5:0/3160040194 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55ac0a3240 msgr2=0x7f55ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.521+0000 7f55c62e8640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55ac0a3240 0x7f55ac0a5700 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f55b0005fd0 tx=0x7f55b0005850 comp rx=0 tx=0).stop 2024-09-15T22:07:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.521+0000 7f55c62e8640 1 -- 172.21.15.5:0/3160040194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c0106f10 msgr2=0x7f55c019cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.521+0000 7f55c62e8640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c0106f10 0x7f55c019cd60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f55b800eaa0 tx=0x7f55b800ef60 comp rx=0 tx=0).stop 2024-09-15T22:07:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.521+0000 7f55c62e8640 1 -- 172.21.15.5:0/3160040194 shutdown_connections 2024-09-15T22:07:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.521+0000 7f55c62e8640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55ac0a3240 0x7f55ac0a5700 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.521+0000 7f55c62e8640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55c0106f10 0x7f55c019cd60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.521+0000 7f55c62e8640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55c0102790 0x7f55c01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.521+0000 7f55c62e8640 1 --2- 172.21.15.5:0/3160040194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55c0101de0 0x7f55c0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.521+0000 7f55c62e8640 1 -- 172.21.15.5:0/3160040194 >> 172.21.15.5:0/3160040194 conn(0x7f55c00fd8c0 msgr2=0x7f55c00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:08.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.521+0000 7f55c62e8640 1 -- 172.21.15.5:0/3160040194 shutdown_connections 2024-09-15T22:07:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:08.522+0000 7f55c62e8640 1 -- 172.21.15.5:0/3160040194 wait complete. 2024-09-15T22:07:08.840 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:08 smithi005 ceph-mon[30944]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:07:08.840 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:08 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3160040194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:09.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:08 smithi099 ceph-mon[34892]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:07:09.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:08 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3160040194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:08 smithi129 ceph-mon[35314]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:07:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:08 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3160040194' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:09.228 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-15T22:07:09.484 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:09.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.936+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3288124061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa178102d40 msgr2=0x7fa178103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.936+0000 7fa17c8bc640 1 --2- 172.21.15.5:0/3288124061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa178102d40 0x7fa178103120 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa160009900 tx=0x7fa16002a480 comp rx=0 tx=0).stop 2024-09-15T22:07:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.936+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3288124061 shutdown_connections 2024-09-15T22:07:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.936+0000 7fa17c8bc640 1 --2- 172.21.15.5:0/3288124061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa17810c170 0x7fa17810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.936+0000 7fa17c8bc640 1 --2- 172.21.15.5:0/3288124061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa178103660 0x7fa17810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.936+0000 7fa17c8bc640 1 --2- 172.21.15.5:0/3288124061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa178102d40 0x7fa178103120 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.936+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3288124061 >> 172.21.15.5:0/3288124061 conn(0x7fa1780fc7a0 msgr2=0x7fa1780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.936+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3288124061 shutdown_connections 2024-09-15T22:07:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.936+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3288124061 wait complete. 2024-09-15T22:07:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.937+0000 7fa17c8bc640 1 Processor -- start 2024-09-15T22:07:09.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.937+0000 7fa17c8bc640 1 -- start start 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.937+0000 7fa17c8bc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa178102d40 0x7fa178198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.937+0000 7fa17c8bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa178103660 0x7fa1781989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.937+0000 7fa17c8bc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa17810c170 0x7fa17819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.937+0000 7fa17c8bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa178111fb0 con 0x7fa178103660 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.937+0000 7fa17c8bc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa178111e30 con 0x7fa17810c170 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.937+0000 7fa17c8bc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa178112130 con 0x7fa178102d40 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.937+0000 7fa175d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa178103660 0x7fa1781989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa175d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa178103660 0x7fa1781989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52298/0 (socket says 172.21.15.5:52298) 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa175d9b640 1 -- 172.21.15.5:0/3091218830 learned_addr learned my addr 172.21.15.5:0/3091218830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa176d9d640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa17810c170 0x7fa17819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa17659c640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa178102d40 0x7fa178198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:09.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa175d9b640 1 -- 172.21.15.5:0/3091218830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa178102d40 msgr2=0x7fa178198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa175d9b640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa178102d40 0x7fa178198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa175d9b640 1 -- 172.21.15.5:0/3091218830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa17810c170 msgr2=0x7fa17819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa175d9b640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa17810c170 0x7fa17819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa175d9b640 1 -- 172.21.15.5:0/3091218830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17819d420 con 0x7fa178103660 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa175d9b640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa178103660 0x7fa1781989b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa16800c970 tx=0x7fa16800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa16f7fe640 1 -- 172.21.15.5:0/3091218830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa168007bb0 con 0x7fa178103660 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa16f7fe640 1 -- 172.21.15.5:0/3091218830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa168007d50 con 0x7fa178103660 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa16f7fe640 1 -- 172.21.15.5:0/3091218830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa168018820 con 0x7fa178103660 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3091218830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17819d710 con 0x7fa178103660 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.938+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3091218830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1781a4ff0 con 0x7fa178103660 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.939+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3091218830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa178103120 con 0x7fa178103660 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.940+0000 7fa16f7fe640 1 -- 172.21.15.5:0/3091218830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7fa1680189c0 con 0x7fa178103660 2024-09-15T22:07:09.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.942+0000 7fa16f7fe640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1640a3240 0x7fa1640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.942+0000 7fa17659c640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1640a3240 0x7fa1640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.942+0000 7fa16f7fe640 1 -- 172.21.15.5:0/3091218830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7fa1680cfd10 con 0x7fa178103660 2024-09-15T22:07:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.942+0000 7fa17659c640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1640a3240 0x7fa1640a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1600096c0 tx=0x7fa160031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:09.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:09.946+0000 7fa16f7fe640 1 -- 172.21.15.5:0/3091218830 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa168014030 con 0x7fa178103660 2024-09-15T22:07:10.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.160+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3091218830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa178107210 con 0x7fa178103660 2024-09-15T22:07:10.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.700+0000 7fa16f7fe640 1 -- 172.21.15.5:0/3091218830 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fa16809c6f0 con 0x7fa178103660 2024-09-15T22:07:10.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3091218830 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1640a3240 msgr2=0x7fa1640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:10.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1640a3240 0x7fa1640a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1600096c0 tx=0x7fa160031490 comp rx=0 tx=0).stop 2024-09-15T22:07:10.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3091218830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa178103660 msgr2=0x7fa1781989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:10.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa178103660 0x7fa1781989b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa16800c970 tx=0x7fa16800ce30 comp rx=0 tx=0).stop 2024-09-15T22:07:10.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3091218830 shutdown_connections 2024-09-15T22:07:10.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa17810c170 0x7fa17819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:10.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa1640a3240 0x7fa1640a5700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:10.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa178103660 0x7fa1781989b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:10.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 --2- 172.21.15.5:0/3091218830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa178102d40 0x7fa178198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:10.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3091218830 >> 172.21.15.5:0/3091218830 conn(0x7fa1780fc7a0 msgr2=0x7fa1780fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:10.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3091218830 shutdown_connections 2024-09-15T22:07:10.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:10.703+0000 7fa17c8bc640 1 -- 172.21.15.5:0/3091218830 wait complete. 2024-09-15T22:07:10.950 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:10 smithi005 ceph-mon[30944]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:07:10.950 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:10 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3091218830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:07:11.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:10 smithi099 ceph-mon[34892]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:07:11.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:10 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3091218830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:07:11.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:10 smithi129 ceph-mon[35314]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-15T22:07:11.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:10 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3091218830' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-15T22:07:11.494 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-15T22:07:11.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-15T22:07:11.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:11 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3091218830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:07:11.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:11 smithi005 ceph-mon[30944]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:07:11.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:11 smithi005 ceph-mon[30944]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-15T22:07:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:12.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:11 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3091218830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:07:12.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:11 smithi099 ceph-mon[34892]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:07:12.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:11 smithi099 ceph-mon[34892]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-15T22:07:12.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:11 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3091218830' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-15T22:07:12.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:11 smithi129 ceph-mon[35314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:07:12.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:11 smithi129 ceph-mon[35314]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-15T22:07:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.239+0000 7f244d421640 1 -- 172.21.15.5:0/1960662838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2448106f10 msgr2=0x7f244810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.239+0000 7f244d421640 1 --2- 172.21.15.5:0/1960662838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2448106f10 0x7f244810ea90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2438009ca0 tx=0x7f243802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:07:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.239+0000 7f244d421640 1 -- 172.21.15.5:0/1960662838 shutdown_connections 2024-09-15T22:07:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.239+0000 7f244d421640 1 --2- 172.21.15.5:0/1960662838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2448106f10 0x7f244810ea90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.239+0000 7f244d421640 1 --2- 172.21.15.5:0/1960662838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2448102790 0x7f24481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.239+0000 7f244d421640 1 --2- 172.21.15.5:0/1960662838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2448101de0 0x7f24481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.239+0000 7f244d421640 1 -- 172.21.15.5:0/1960662838 >> 172.21.15.5:0/1960662838 conn(0x7f24480fd8c0 msgr2=0x7f24480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.239+0000 7f244d421640 1 -- 172.21.15.5:0/1960662838 shutdown_connections 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.239+0000 7f244d421640 1 -- 172.21.15.5:0/1960662838 wait complete. 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.239+0000 7f244d421640 1 Processor -- start 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.239+0000 7f244d421640 1 -- start start 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f244d421640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2448101de0 0x7f244819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f244d421640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2448102790 0x7f244819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f244d421640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2448106f10 0x7f244819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f244d421640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2448111f70 con 0x7f2448106f10 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f244d421640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2448111df0 con 0x7f2448101de0 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f244d421640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f24481120f0 con 0x7f2448102790 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f24477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2448106f10 0x7f244819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f24477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2448106f10 0x7f244819eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52314/0 (socket says 172.21.15.5:52314) 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f24477fe640 1 -- 172.21.15.5:0/513725669 learned_addr learned my addr 172.21.15.5:0/513725669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f24467fc640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2448102790 0x7f244819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f2446ffd640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2448101de0 0x7f244819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f24477fe640 1 -- 172.21.15.5:0/513725669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2448102790 msgr2=0x7f244819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f24477fe640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2448102790 0x7f244819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f24477fe640 1 -- 172.21.15.5:0/513725669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2448101de0 msgr2=0x7f244819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f24477fe640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2448101de0 0x7f244819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f24477fe640 1 -- 172.21.15.5:0/513725669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244819f5c0 con 0x7f2448106f10 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.240+0000 7f24477fe640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2448106f10 0x7f244819eee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2438031a60 tx=0x7f2438031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.241+0000 7f243ffff640 1 -- 172.21.15.5:0/513725669 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438003c30 con 0x7f2448106f10 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.241+0000 7f244d421640 1 -- 172.21.15.5:0/513725669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f244819f850 con 0x7f2448106f10 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.241+0000 7f243ffff640 1 -- 172.21.15.5:0/513725669 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2438003dd0 con 0x7f2448106f10 2024-09-15T22:07:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.241+0000 7f243ffff640 1 -- 172.21.15.5:0/513725669 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24380047e0 con 0x7f2448106f10 2024-09-15T22:07:12.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.241+0000 7f244d421640 1 -- 172.21.15.5:0/513725669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24481a7190 con 0x7f2448106f10 2024-09-15T22:07:12.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.242+0000 7f244d421640 1 -- 172.21.15.5:0/513725669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24481021c0 con 0x7f2448106f10 2024-09-15T22:07:12.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.243+0000 7f243ffff640 1 -- 172.21.15.5:0/513725669 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f243802f790 con 0x7f2448106f10 2024-09-15T22:07:12.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.243+0000 7f243ffff640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24340a3240 0x7f24340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:12.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.244+0000 7f2446ffd640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24340a3240 0x7f24340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:12.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.244+0000 7f243ffff640 1 -- 172.21.15.5:0/513725669 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f24380f1060 con 0x7f2448106f10 2024-09-15T22:07:12.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.244+0000 7f2446ffd640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24340a3240 0x7f24340a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f24400058c0 tx=0x7f244000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:12.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.248+0000 7f243ffff640 1 -- 172.21.15.5:0/513725669 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24380bda40 con 0x7f2448106f10 2024-09-15T22:07:12.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.459+0000 7f244d421640 1 -- 172.21.15.5:0/513725669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f24481a7360 con 0x7f2448106f10 2024-09-15T22:07:12.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.711+0000 7f243ffff640 1 -- 172.21.15.5:0/513725669 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f24380bd340 con 0x7f2448106f10 2024-09-15T22:07:12.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:12 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/513725669' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:07:12.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 -- 172.21.15.5:0/513725669 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24340a3240 msgr2=0x7f24340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:12.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24340a3240 0x7f24340a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f24400058c0 tx=0x7f244000a4c0 comp rx=0 tx=0).stop 2024-09-15T22:07:12.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 -- 172.21.15.5:0/513725669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2448106f10 msgr2=0x7f244819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:12.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2448106f10 0x7f244819eee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2438031a60 tx=0x7f2438031a90 comp rx=0 tx=0).stop 2024-09-15T22:07:12.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 -- 172.21.15.5:0/513725669 shutdown_connections 2024-09-15T22:07:12.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24340a3240 0x7f24340a5700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:12.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2448106f10 0x7f244819eee0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:12.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2448102790 0x7f244819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:12.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 --2- 172.21.15.5:0/513725669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2448101de0 0x7f244819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:12.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 -- 172.21.15.5:0/513725669 >> 172.21.15.5:0/513725669 conn(0x7f24480fd8c0 msgr2=0x7f24480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:12.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 -- 172.21.15.5:0/513725669 shutdown_connections 2024-09-15T22:07:12.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:12.714+0000 7f244d421640 1 -- 172.21.15.5:0/513725669 wait complete. 2024-09-15T22:07:13.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:12 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/513725669' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:07:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:12 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/513725669' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-15T22:07:13.507 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-15T22:07:13.772 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:13.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:13 smithi005 ceph-mon[30944]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:07:13.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:13 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/513725669' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-15T22:07:13.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:13 smithi005 ceph-mon[30944]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:07:13.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:13 smithi005 ceph-mon[30944]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:07:13.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:13 smithi005 ceph-mon[30944]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-15T22:07:13.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:13 smithi005 ceph-mon[30944]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-15T22:07:13.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:13 smithi005 ceph-mon[30944]: pgmap v301: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-15T22:07:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:13 smithi099 ceph-mon[34892]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:07:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:13 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/513725669' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-15T22:07:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:13 smithi099 ceph-mon[34892]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:07:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:13 smithi099 ceph-mon[34892]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:07:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:13 smithi099 ceph-mon[34892]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-15T22:07:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:13 smithi099 ceph-mon[34892]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-15T22:07:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:13 smithi099 ceph-mon[34892]: pgmap v301: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-15T22:07:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:13 smithi129 ceph-mon[35314]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-15T22:07:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:13 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/513725669' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-15T22:07:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:13 smithi129 ceph-mon[35314]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-15T22:07:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:13 smithi129 ceph-mon[35314]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-15T22:07:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:13 smithi129 ceph-mon[35314]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-15T22:07:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:13 smithi129 ceph-mon[35314]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-15T22:07:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:13 smithi129 ceph-mon[35314]: pgmap v301: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-09-15T22:07:14.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.247+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3616631730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c4c102da0 msgr2=0x7f5c4c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:14.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.247+0000 7f5c50bf3640 1 --2- 172.21.15.5:0/3616631730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c4c102da0 0x7f5c4c103180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c009900 tx=0x7f5c3c02a480 comp rx=0 tx=0).stop 2024-09-15T22:07:14.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.247+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3616631730 shutdown_connections 2024-09-15T22:07:14.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.247+0000 7f5c50bf3640 1 --2- 172.21.15.5:0/3616631730 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c4c10c180 0x7f5c4c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:14.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.247+0000 7f5c50bf3640 1 --2- 172.21.15.5:0/3616631730 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c1036c0 0x7f5c4c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:14.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.247+0000 7f5c50bf3640 1 --2- 172.21.15.5:0/3616631730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c4c102da0 0x7f5c4c103180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.247+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3616631730 >> 172.21.15.5:0/3616631730 conn(0x7f5c4c0fc7a0 msgr2=0x7f5c4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.247+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3616631730 shutdown_connections 2024-09-15T22:07:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.247+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3616631730 wait complete. 2024-09-15T22:07:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c50bf3640 1 Processor -- start 2024-09-15T22:07:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c50bf3640 1 -- start start 2024-09-15T22:07:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c50bf3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c4c102da0 0x7f5c4c1a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c50bf3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c1036c0 0x7f5c4c1a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c50bf3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c4c10c180 0x7f5c4c10adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:14.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c50bf3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c111fb0 con 0x7f5c4c10c180 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c50bf3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c111e30 con 0x7f5c4c1036c0 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c50bf3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c4c112130 con 0x7f5c4c102da0 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c4ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c4c10c180 0x7f5c4c10adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c4ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c4c10c180 0x7f5c4c10adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54048/0 (socket says 172.21.15.5:54048) 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c4ad9d640 1 -- 172.21.15.5:0/3307482494 learned_addr learned my addr 172.21.15.5:0/3307482494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.248+0000 7f5c4a59c640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c4c102da0 0x7f5c4c1a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c41d9b640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c1036c0 0x7f5c4c1a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c4ad9d640 1 -- 172.21.15.5:0/3307482494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c4c102da0 msgr2=0x7f5c4c1a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c4ad9d640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c4c102da0 0x7f5c4c1a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c4ad9d640 1 -- 172.21.15.5:0/3307482494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c1036c0 msgr2=0x7f5c4c1a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c4ad9d640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c1036c0 0x7f5c4c1a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:14.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c4ad9d640 1 -- 172.21.15.5:0/3307482494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c4c10b630 con 0x7f5c4c10c180 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c4ad9d640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c4c10c180 0x7f5c4c10adc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400cad0 tx=0x7f5c4400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c43fff640 1 -- 172.21.15.5:0/3307482494 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4400ae20 con 0x7f5c4c10c180 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c43fff640 1 -- 172.21.15.5:0/3307482494 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c44005020 con 0x7f5c4c10c180 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c43fff640 1 -- 172.21.15.5:0/3307482494 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4400f6f0 con 0x7f5c4c10c180 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3307482494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c4c10b920 con 0x7f5c4c10c180 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.249+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3307482494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c4c1ab700 con 0x7f5c4c10c180 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.250+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3307482494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c4c06b010 con 0x7f5c4c10c180 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.251+0000 7f5c43fff640 1 -- 172.21.15.5:0/3307482494 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f5c440051d0 con 0x7f5c4c10c180 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.252+0000 7f5c43fff640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c380a3240 0x7f5c380a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.252+0000 7f5c4a59c640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c380a3240 0x7f5c380a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.252+0000 7f5c43fff640 1 -- 172.21.15.5:0/3307482494 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f5c440cfbd0 con 0x7f5c4c10c180 2024-09-15T22:07:14.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.253+0000 7f5c4a59c640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c380a3240 0x7f5c380a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c000c00 tx=0x7f5c3c005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:14.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.256+0000 7f5c43fff640 1 -- 172.21.15.5:0/3307482494 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c44014030 con 0x7f5c4c10c180 2024-09-15T22:07:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.470+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3307482494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f5c4c1ab8d0 con 0x7f5c4c10c180 2024-09-15T22:07:14.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.726+0000 7f5c43fff640 1 -- 172.21.15.5:0/3307482494 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f5c4409bf60 con 0x7f5c4c10c180 2024-09-15T22:07:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3307482494 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c380a3240 msgr2=0x7f5c380a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c380a3240 0x7f5c380a5700 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c000c00 tx=0x7f5c3c005df0 comp rx=0 tx=0).stop 2024-09-15T22:07:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3307482494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c4c10c180 msgr2=0x7f5c4c10adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c4c10c180 0x7f5c4c10adc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400cad0 tx=0x7f5c4400cf90 comp rx=0 tx=0).stop 2024-09-15T22:07:14.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3307482494 shutdown_connections 2024-09-15T22:07:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5c380a3240 0x7f5c380a5700 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c4c10c180 0x7f5c4c10adc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c4c1036c0 0x7f5c4c1a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 --2- 172.21.15.5:0/3307482494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c4c102da0 0x7f5c4c1a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3307482494 >> 172.21.15.5:0/3307482494 conn(0x7f5c4c0fc7a0 msgr2=0x7f5c4c101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3307482494 shutdown_connections 2024-09-15T22:07:14.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:14.728+0000 7f5c50bf3640 1 -- 172.21.15.5:0/3307482494 wait complete. 2024-09-15T22:07:15.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:14 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] up:active 2024-09-15T22:07:15.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:14 smithi005 ceph-mon[30944]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:07:15.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:14 smithi005 ceph-mon[30944]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:07:15.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:14 smithi005 ceph-mon[30944]: Cluster is now healthy 2024-09-15T22:07:15.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:14 smithi005 ceph-mon[30944]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-15T22:07:15.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:14 smithi005 ceph-mon[30944]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-15T22:07:15.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:14 smithi005 ceph-mon[30944]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-15T22:07:15.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:14 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3307482494' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:07:15.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:14 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] up:active 2024-09-15T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:14 smithi099 ceph-mon[34892]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:14 smithi099 ceph-mon[34892]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:14 smithi099 ceph-mon[34892]: Cluster is now healthy 2024-09-15T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:14 smithi099 ceph-mon[34892]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-15T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:14 smithi099 ceph-mon[34892]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-15T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:14 smithi099 ceph-mon[34892]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-15T22:07:15.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:14 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3307482494' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:07:15.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:14 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] up:active 2024-09-15T22:07:15.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:14 smithi129 ceph-mon[35314]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-15T22:07:15.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:14 smithi129 ceph-mon[35314]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-15T22:07:15.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:14 smithi129 ceph-mon[35314]: Cluster is now healthy 2024-09-15T22:07:15.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:14 smithi129 ceph-mon[35314]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-15T22:07:15.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:14 smithi129 ceph-mon[35314]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-09-15T22:07:15.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:14 smithi129 ceph-mon[35314]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-15T22:07:15.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:14 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3307482494' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-15T22:07:15.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-15T22:07:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:16.055 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:15 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3307482494' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:07:16.055 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:15 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] up:active 2024-09-15T22:07:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:15 smithi005 ceph-mon[30944]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-15T22:07:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:15 smithi005 ceph-mon[30944]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:07:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:15 smithi005 ceph-mon[30944]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2024-09-15T22:07:16.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:15 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3307482494' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:07:16.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:15 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] up:active 2024-09-15T22:07:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:15 smithi099 ceph-mon[34892]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-15T22:07:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:15 smithi099 ceph-mon[34892]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:07:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:15 smithi099 ceph-mon[34892]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2024-09-15T22:07:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:15 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3307482494' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-15T22:07:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:15 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] up:active 2024-09-15T22:07:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:15 smithi129 ceph-mon[35314]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-09-15T22:07:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:15 smithi129 ceph-mon[35314]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:07:16.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:15 smithi129 ceph-mon[35314]: pgmap v302: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 3 op/s 2024-09-15T22:07:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.140+0000 7f73c08b5640 1 -- 172.21.15.5:0/274026765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73bc102570 msgr2=0x7f73bc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:16.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.140+0000 7f73c08b5640 1 --2- 172.21.15.5:0/274026765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73bc102570 0x7f73bc1029b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f73b0009900 tx=0x7f73b002a480 comp rx=0 tx=0).stop 2024-09-15T22:07:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.140+0000 7f73c08b5640 1 -- 172.21.15.5:0/274026765 shutdown_connections 2024-09-15T22:07:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.140+0000 7f73c08b5640 1 --2- 172.21.15.5:0/274026765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73bc102ef0 0x7f73bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.140+0000 7f73c08b5640 1 --2- 172.21.15.5:0/274026765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73bc102570 0x7f73bc1029b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.140+0000 7f73c08b5640 1 --2- 172.21.15.5:0/274026765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73bc100e20 0x7f73bc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.140+0000 7f73c08b5640 1 -- 172.21.15.5:0/274026765 >> 172.21.15.5:0/274026765 conn(0x7f73bc0fc7d0 msgr2=0x7f73bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.140+0000 7f73c08b5640 1 -- 172.21.15.5:0/274026765 shutdown_connections 2024-09-15T22:07:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.140+0000 7f73c08b5640 1 -- 172.21.15.5:0/274026765 wait complete. 2024-09-15T22:07:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73c08b5640 1 Processor -- start 2024-09-15T22:07:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73c08b5640 1 -- start start 2024-09-15T22:07:16.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73c08b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73bc100e20 0x7f73bc197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73c08b5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73bc102570 0x7f73bc197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73c08b5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73bc102ef0 0x7f73bc19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73c08b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc110f50 con 0x7f73bc100e20 2024-09-15T22:07:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73c08b5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc110dd0 con 0x7f73bc102570 2024-09-15T22:07:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73c08b5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f73bc1110d0 con 0x7f73bc102ef0 2024-09-15T22:07:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73ba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73bc100e20 0x7f73bc197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73ba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73bc100e20 0x7f73bc197430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54070/0 (socket says 172.21.15.5:54070) 2024-09-15T22:07:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73ba59c640 1 -- 172.21.15.5:0/1074835397 learned_addr learned my addr 172.21.15.5:0/1074835397 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.141+0000 7f73bad9d640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73bc102ef0 0x7f73bc19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73b9d9b640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73bc102570 0x7f73bc197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73ba59c640 1 -- 172.21.15.5:0/1074835397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73bc102ef0 msgr2=0x7f73bc19bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73ba59c640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73bc102ef0 0x7f73bc19bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73ba59c640 1 -- 172.21.15.5:0/1074835397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73bc102570 msgr2=0x7f73bc197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73ba59c640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73bc102570 0x7f73bc197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73ba59c640 1 -- 172.21.15.5:0/1074835397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73bc19c3e0 con 0x7f73bc100e20 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73ba59c640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73bc100e20 0x7f73bc197430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00d950 tx=0x7f73ac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73ab7fe640 1 -- 172.21.15.5:0/1074835397 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac017030 con 0x7f73bc100e20 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73c08b5640 1 -- 172.21.15.5:0/1074835397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73bc19c6d0 con 0x7f73bc100e20 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73ab7fe640 1 -- 172.21.15.5:0/1074835397 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73ac004020 con 0x7f73bc100e20 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73ab7fe640 1 -- 172.21.15.5:0/1074835397 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ac0047a0 con 0x7f73bc100e20 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.142+0000 7f73c08b5640 1 -- 172.21.15.5:0/1074835397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73bc1a3fb0 con 0x7f73bc100e20 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.143+0000 7f73c08b5640 1 -- 172.21.15.5:0/1074835397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73bc101200 con 0x7f73bc100e20 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.144+0000 7f73ab7fe640 1 -- 172.21.15.5:0/1074835397 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f73ac0041d0 con 0x7f73bc100e20 2024-09-15T22:07:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.145+0000 7f73ab7fe640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73a40a3240 0x7f73a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.145+0000 7f73b9d9b640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73a40a3240 0x7f73a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.145+0000 7f73ab7fe640 1 -- 172.21.15.5:0/1074835397 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f73ac0cfdb0 con 0x7f73bc100e20 2024-09-15T22:07:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.146+0000 7f73b9d9b640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73a40a3240 0x7f73a40a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0ffda0 tx=0x7f73b0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.149+0000 7f73ab7fe640 1 -- 172.21.15.5:0/1074835397 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73ac09c730 con 0x7f73bc100e20 2024-09-15T22:07:16.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.362+0000 7f73c08b5640 1 -- 172.21.15.5:0/1074835397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f73bc109090 con 0x7f73bc100e20 2024-09-15T22:07:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.739+0000 7f73ab7fe640 1 -- 172.21.15.5:0/1074835397 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f73ac09c030 con 0x7f73bc100e20 2024-09-15T22:07:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.741+0000 7f73c08b5640 1 -- 172.21.15.5:0/1074835397 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73a40a3240 msgr2=0x7f73a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.741+0000 7f73c08b5640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73a40a3240 0x7f73a40a5700 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0ffda0 tx=0x7f73b0005a20 comp rx=0 tx=0).stop 2024-09-15T22:07:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.742+0000 7f73c08b5640 1 -- 172.21.15.5:0/1074835397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73bc100e20 msgr2=0x7f73bc197430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.742+0000 7f73c08b5640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73bc100e20 0x7f73bc197430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f73ac00d950 tx=0x7f73ac00de10 comp rx=0 tx=0).stop 2024-09-15T22:07:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.742+0000 7f73c08b5640 1 -- 172.21.15.5:0/1074835397 shutdown_connections 2024-09-15T22:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.742+0000 7f73c08b5640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f73bc102ef0 0x7f73bc19bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.742+0000 7f73c08b5640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f73bc102570 0x7f73bc197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.742+0000 7f73c08b5640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f73a40a3240 0x7f73a40a5700 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.742+0000 7f73c08b5640 1 --2- 172.21.15.5:0/1074835397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73bc100e20 0x7f73bc197430 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.742+0000 7f73c08b5640 1 -- 172.21.15.5:0/1074835397 >> 172.21.15.5:0/1074835397 conn(0x7f73bc0fc7d0 msgr2=0x7f73bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.742+0000 7f73c08b5640 1 -- 172.21.15.5:0/1074835397 shutdown_connections 2024-09-15T22:07:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:16.742+0000 7f73c08b5640 1 -- 172.21.15.5:0/1074835397 wait complete. 2024-09-15T22:07:17.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:16 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1074835397' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:07:17.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:16 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1074835397' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:07:17.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:16 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1074835397' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-15T22:07:17.577 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-15T22:07:17.578 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-15T22:07:17.748 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:07:17 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b[64991]: 2024-09-15T22:07:17.484+0000 7f7475f12640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-15T22:07:17.748 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:07:17 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b[64991]: 2024-09-15T22:07:17.484+0000 7f7475f12640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-15T22:07:17.843 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:17.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:17 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1074835397' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:07:17.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:17 smithi005 ceph-mon[30944]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:07:17.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:17 smithi005 ceph-mon[30944]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-15T22:07:18.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:17 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1074835397' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:07:18.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:17 smithi099 ceph-mon[34892]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:07:18.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:17 smithi099 ceph-mon[34892]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-15T22:07:18.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:17 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1074835397' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-15T22:07:18.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:17 smithi129 ceph-mon[35314]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:07:18.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:17 smithi129 ceph-mon[35314]: pgmap v303: 129 pgs: 129 active+clean; 582 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.4 KiB/s wr, 5 op/s 2024-09-15T22:07:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.322+0000 7f5525212640 1 -- 172.21.15.5:0/3383219906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5520102790 msgr2=0x7f55201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.322+0000 7f5525212640 1 --2- 172.21.15.5:0/3383219906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5520102790 0x7f55201067e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5508009900 tx=0x7f550802a410 comp rx=0 tx=0).stop 2024-09-15T22:07:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.322+0000 7f5525212640 1 -- 172.21.15.5:0/3383219906 shutdown_connections 2024-09-15T22:07:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.322+0000 7f5525212640 1 --2- 172.21.15.5:0/3383219906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5520106f10 0x7f552010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.322+0000 7f5525212640 1 --2- 172.21.15.5:0/3383219906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5520102790 0x7f55201067e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.322+0000 7f5525212640 1 --2- 172.21.15.5:0/3383219906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5520101de0 0x7f55201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.322+0000 7f5525212640 1 -- 172.21.15.5:0/3383219906 >> 172.21.15.5:0/3383219906 conn(0x7f55200fd8c0 msgr2=0x7f55200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.322+0000 7f5525212640 1 -- 172.21.15.5:0/3383219906 shutdown_connections 2024-09-15T22:07:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.322+0000 7f5525212640 1 -- 172.21.15.5:0/3383219906 wait complete. 2024-09-15T22:07:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.323+0000 7f5525212640 1 Processor -- start 2024-09-15T22:07:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.323+0000 7f5525212640 1 -- start start 2024-09-15T22:07:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.323+0000 7f5525212640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5520101de0 0x7f55200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.323+0000 7f5525212640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5520102790 0x7f5520074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.323+0000 7f5525212640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5520106f10 0x7f5520074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.323+0000 7f5525212640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5520111f90 con 0x7f5520102790 2024-09-15T22:07:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.323+0000 7f5525212640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5520111e10 con 0x7f5520106f10 2024-09-15T22:07:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.323+0000 7f5525212640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5520112110 con 0x7f5520101de0 2024-09-15T22:07:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f5517fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5520102790 0x7f5520074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f5517fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5520102790 0x7f5520074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54076/0 (socket says 172.21.15.5:54076) 2024-09-15T22:07:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f5517fff640 1 -- 172.21.15.5:0/3468918255 learned_addr learned my addr 172.21.15.5:0/3468918255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f551f59e640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5520106f10 0x7f5520074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f551ed9d640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5520101de0 0x7f55200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f5517fff640 1 -- 172.21.15.5:0/3468918255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5520101de0 msgr2=0x7f55200744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f5517fff640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5520101de0 0x7f55200744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f5517fff640 1 -- 172.21.15.5:0/3468918255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5520106f10 msgr2=0x7f5520074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f5517fff640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5520106f10 0x7f5520074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f5517fff640 1 -- 172.21.15.5:0/3468918255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55201a68e0 con 0x7f5520102790 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f551f59e640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5520106f10 0x7f5520074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f5517fff640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5520102790 0x7f5520074a30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5508005ab0 tx=0x7f55080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f551cd99640 1 -- 172.21.15.5:0/3468918255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550803f030 con 0x7f5520102790 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f551cd99640 1 -- 172.21.15.5:0/3468918255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5508003ce0 con 0x7f5520102790 2024-09-15T22:07:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f5525212640 1 -- 172.21.15.5:0/3468918255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55201a6b70 con 0x7f5520102790 2024-09-15T22:07:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f551cd99640 1 -- 172.21.15.5:0/3468918255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508035070 con 0x7f5520102790 2024-09-15T22:07:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.324+0000 7f5525212640 1 -- 172.21.15.5:0/3468918255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55201a7050 con 0x7f5520102790 2024-09-15T22:07:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.325+0000 7f5525212640 1 -- 172.21.15.5:0/3468918255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55201021c0 con 0x7f5520102790 2024-09-15T22:07:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.327+0000 7f551cd99640 1 -- 172.21.15.5:0/3468918255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 138354+0+0 (secure 0 0 0) 0x7f5508042030 con 0x7f5520102790 2024-09-15T22:07:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.327+0000 7f551cd99640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f550c0a3240 0x7f550c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.327+0000 7f551cd99640 1 -- 172.21.15.5:0/3468918255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f55080f0610 con 0x7f5520102790 2024-09-15T22:07:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.327+0000 7f551ed9d640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f550c0a3240 0x7f550c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.328+0000 7f551ed9d640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f550c0a3240 0x7f550c0a5700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5510005e10 tx=0x7f5510005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:18.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.331+0000 7f551cd99640 1 -- 172.21.15.5:0/3468918255 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55080bcff0 con 0x7f5520102790 2024-09-15T22:07:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.530+0000 7f5525212640 1 -- 172.21.15.5:0/3468918255 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5520062630 con 0x7f550c0a3240 2024-09-15T22:07:18.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.557+0000 7f551cd99640 1 -- 172.21.15.5:0/3468918255 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5520062630 con 0x7f550c0a3240 2024-09-15T22:07:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 -- 172.21.15.5:0/3468918255 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f550c0a3240 msgr2=0x7f550c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f550c0a3240 0x7f550c0a5700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5510005e10 tx=0x7f5510005da0 comp rx=0 tx=0).stop 2024-09-15T22:07:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 -- 172.21.15.5:0/3468918255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5520102790 msgr2=0x7f5520074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5520102790 0x7f5520074a30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5508005ab0 tx=0x7f55080042f0 comp rx=0 tx=0).stop 2024-09-15T22:07:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 -- 172.21.15.5:0/3468918255 shutdown_connections 2024-09-15T22:07:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5520106f10 0x7f5520074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f550c0a3240 0x7f550c0a5700 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5520102790 0x7f5520074a30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 --2- 172.21.15.5:0/3468918255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5520101de0 0x7f55200744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 -- 172.21.15.5:0/3468918255 >> 172.21.15.5:0/3468918255 conn(0x7f55200fd8c0 msgr2=0x7f55200fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 -- 172.21.15.5:0/3468918255 shutdown_connections 2024-09-15T22:07:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:18.559+0000 7f5525212640 1 -- 172.21.15.5:0/3468918255 wait complete. 2024-09-15T22:07:18.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:18 smithi005 ceph-mon[30944]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:18.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:07:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:07:18.536+0000 7f0ac159d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:07:18.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:07:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:07:18.536+0000 7f0ac159d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:07:18.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:07:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:07:18.536+0000 7f0ac159d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:07:18.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:07:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:07:18.536+0000 7f0ac159d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:07:18.857 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:07:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:07:18.536+0000 7f0ac159d640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:07:19.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:18 smithi099 ceph-mon[34892]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:19.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:18 smithi129 ceph-mon[35314]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:19.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-15T22:07:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:20.036 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:19 smithi005 ceph-mon[30944]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-15T22:07:20.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:19 smithi099 ceph-mon[34892]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-15T22:07:20.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:19 smithi129 ceph-mon[35314]: pgmap v304: 129 pgs: 129 active+clean; 584 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 2.1 KiB/s wr, 9 op/s 2024-09-15T22:07:20.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.123+0000 7ff3a1776640 1 -- 172.21.15.5:0/3509615369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c101420 msgr2=0x7ff39c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.123+0000 7ff3a1776640 1 --2- 172.21.15.5:0/3509615369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c101420 0x7ff39c10d950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff394009cd0 tx=0x7ff39402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:07:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.123+0000 7ff3a1776640 1 -- 172.21.15.5:0/3509615369 shutdown_connections 2024-09-15T22:07:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.123+0000 7ff3a1776640 1 --2- 172.21.15.5:0/3509615369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c101420 0x7ff39c10d950 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.123+0000 7ff3a1776640 1 --2- 172.21.15.5:0/3509615369 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff39c100a80 0x7ff39c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.123+0000 7ff3a1776640 1 --2- 172.21.15.5:0/3509615369 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39c106a80 0x7ff39c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.123+0000 7ff3a1776640 1 -- 172.21.15.5:0/3509615369 >> 172.21.15.5:0/3509615369 conn(0x7ff39c0fc7a0 msgr2=0x7ff39c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.123+0000 7ff3a1776640 1 -- 172.21.15.5:0/3509615369 shutdown_connections 2024-09-15T22:07:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.123+0000 7ff3a1776640 1 -- 172.21.15.5:0/3509615369 wait complete. 2024-09-15T22:07:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff3a1776640 1 Processor -- start 2024-09-15T22:07:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff3a1776640 1 -- start start 2024-09-15T22:07:20.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff3a1776640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff39c100a80 0x7ff39c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff3a1776640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c101420 0x7ff39c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff3a1776640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39c106a80 0x7ff39c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff3a1776640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c110e80 con 0x7ff39c101420 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff3a1776640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c110d00 con 0x7ff39c106a80 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff3a1776640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c111000 con 0x7ff39c100a80 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff393fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c101420 0x7ff39c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff393fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c101420 0x7ff39c19ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54106/0 (socket says 172.21.15.5:54106) 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff393fff640 1 -- 172.21.15.5:0/3566930135 learned_addr learned my addr 172.21.15.5:0/3566930135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff39affd640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff39c100a80 0x7ff39c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff39b7fe640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39c106a80 0x7ff39c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff393fff640 1 -- 172.21.15.5:0/3566930135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff39c100a80 msgr2=0x7ff39c19a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff393fff640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff39c100a80 0x7ff39c19a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:20.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff393fff640 1 -- 172.21.15.5:0/3566930135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39c106a80 msgr2=0x7ff39c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff393fff640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39c106a80 0x7ff39c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.124+0000 7ff393fff640 1 -- 172.21.15.5:0/3566930135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39c19f670 con 0x7ff39c101420 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.125+0000 7ff39b7fe640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39c106a80 0x7ff39c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.125+0000 7ff393fff640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c101420 0x7ff39c19ac00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff38400e9f0 tx=0x7ff38400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.125+0000 7ff398ff9640 1 -- 172.21.15.5:0/3566930135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38400cd80 con 0x7ff39c101420 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.125+0000 7ff398ff9640 1 -- 172.21.15.5:0/3566930135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff384005500 con 0x7ff39c101420 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.125+0000 7ff3a1776640 1 -- 172.21.15.5:0/3566930135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39c19f960 con 0x7ff39c101420 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.125+0000 7ff398ff9640 1 -- 172.21.15.5:0/3566930135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384010690 con 0x7ff39c101420 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.125+0000 7ff3a1776640 1 -- 172.21.15.5:0/3566930135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff39c1a7240 con 0x7ff39c101420 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.126+0000 7ff3a1776640 1 -- 172.21.15.5:0/3566930135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff39c100ee0 con 0x7ff39c101420 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.127+0000 7ff398ff9640 1 -- 172.21.15.5:0/3566930135 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ff3840046c0 con 0x7ff39c101420 2024-09-15T22:07:20.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.128+0000 7ff398ff9640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff3880a3350 0x7ff3880a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:20.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.128+0000 7ff39affd640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff3880a3350 0x7ff3880a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:20.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.128+0000 7ff398ff9640 1 -- 172.21.15.5:0/3566930135 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7ff3840d03f0 con 0x7ff39c101420 2024-09-15T22:07:20.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.129+0000 7ff39affd640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff3880a3350 0x7ff3880a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00bb60 tx=0x7ff38c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:20.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.131+0000 7ff398ff9640 1 -- 172.21.15.5:0/3566930135 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff3840541b0 con 0x7ff39c101420 2024-09-15T22:07:20.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.334+0000 7ff3a1776640 1 -- 172.21.15.5:0/3566930135 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff39c06aea0 con 0x7ff3880a3350 2024-09-15T22:07:20.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.360+0000 7ff398ff9640 1 -- 172.21.15.5:0/3566930135 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff39c06aea0 con 0x7ff3880a3350 2024-09-15T22:07:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.362+0000 7ff3a1776640 1 -- 172.21.15.5:0/3566930135 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff3880a3350 msgr2=0x7ff3880a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.362+0000 7ff3a1776640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff3880a3350 0x7ff3880a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00bb60 tx=0x7ff38c00b390 comp rx=0 tx=0).stop 2024-09-15T22:07:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.362+0000 7ff3a1776640 1 -- 172.21.15.5:0/3566930135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c101420 msgr2=0x7ff39c19ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.362+0000 7ff3a1776640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c101420 0x7ff39c19ac00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff38400e9f0 tx=0x7ff38400edf0 comp rx=0 tx=0).stop 2024-09-15T22:07:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.363+0000 7ff3a1776640 1 -- 172.21.15.5:0/3566930135 shutdown_connections 2024-09-15T22:07:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.363+0000 7ff3a1776640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff39c106a80 0x7ff39c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.363+0000 7ff3a1776640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff3880a3350 0x7ff3880a5810 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.363+0000 7ff3a1776640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c101420 0x7ff39c19ac00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.363+0000 7ff3a1776640 1 --2- 172.21.15.5:0/3566930135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff39c100a80 0x7ff39c19a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.363+0000 7ff3a1776640 1 -- 172.21.15.5:0/3566930135 >> 172.21.15.5:0/3566930135 conn(0x7ff39c0fc7a0 msgr2=0x7ff39c06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.363+0000 7ff3a1776640 1 -- 172.21.15.5:0/3566930135 shutdown_connections 2024-09-15T22:07:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:20.363+0000 7ff3a1776640 1 -- 172.21.15.5:0/3566930135 wait complete. 2024-09-15T22:07:21.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:20 smithi005 ceph-mon[30944]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:07:21.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:20 smithi005 ceph-mon[30944]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:21.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-15T22:07:21.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:20 smithi099 ceph-mon[34892]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:07:21.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:20 smithi099 ceph-mon[34892]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:21.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:20 smithi129 ceph-mon[35314]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-15T22:07:21.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:20 smithi129 ceph-mon[35314]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.822+0000 7ff67583e640 1 -- 172.21.15.5:0/1173312978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67010c170 msgr2=0x7ff67010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.822+0000 7ff67583e640 1 --2- 172.21.15.5:0/1173312978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67010c170 0x7ff67010e610 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff668009ca0 tx=0x7ff66802a950 comp rx=0 tx=0).stop 2024-09-15T22:07:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.822+0000 7ff67583e640 1 -- 172.21.15.5:0/1173312978 shutdown_connections 2024-09-15T22:07:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.822+0000 7ff67583e640 1 --2- 172.21.15.5:0/1173312978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67010c170 0x7ff67010e610 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.822+0000 7ff67583e640 1 --2- 172.21.15.5:0/1173312978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff670103660 0x7ff67010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.822+0000 7ff67583e640 1 --2- 172.21.15.5:0/1173312978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff670102d40 0x7ff670103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.822+0000 7ff67583e640 1 -- 172.21.15.5:0/1173312978 >> 172.21.15.5:0/1173312978 conn(0x7ff6700fc7a0 msgr2=0x7ff6700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.822+0000 7ff67583e640 1 -- 172.21.15.5:0/1173312978 shutdown_connections 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.822+0000 7ff67583e640 1 -- 172.21.15.5:0/1173312978 wait complete. 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.822+0000 7ff67583e640 1 Processor -- start 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff67583e640 1 -- start start 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff67583e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff670102d40 0x7ff6701009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff67583e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff670103660 0x7ff670100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff67583e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67010c170 0x7ff670101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff67583e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff670111f70 con 0x7ff67010c170 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff67583e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff670111df0 con 0x7ff670103660 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff67583e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701120f0 con 0x7ff670102d40 2024-09-15T22:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67010c170 0x7ff670101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67010c170 0x7ff670101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54126/0 (socket says 172.21.15.5:54126) 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66f7fe640 1 -- 172.21.15.5:0/999965860 learned_addr learned my addr 172.21.15.5:0/999965860 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66effd640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff670102d40 0x7ff6701009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66e7fc640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff670103660 0x7ff670100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66f7fe640 1 -- 172.21.15.5:0/999965860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff670102d40 msgr2=0x7ff6701009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66f7fe640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff670102d40 0x7ff6701009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66f7fe640 1 -- 172.21.15.5:0/999965860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff670103660 msgr2=0x7ff670100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66f7fe640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff670103660 0x7ff670100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66f7fe640 1 -- 172.21.15.5:0/999965860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6701a68d0 con 0x7ff67010c170 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66effd640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff670102d40 0x7ff6701009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66e7fc640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff670103660 0x7ff670100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.823+0000 7ff66f7fe640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67010c170 0x7ff670101420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff66802ae70 tx=0x7ff66802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.824+0000 7ff67483c640 1 -- 172.21.15.5:0/999965860 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668034040 con 0x7ff67010c170 2024-09-15T22:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.824+0000 7ff67483c640 1 -- 172.21.15.5:0/999965860 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff668030920 con 0x7ff67010c170 2024-09-15T22:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.824+0000 7ff67483c640 1 -- 172.21.15.5:0/999965860 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668045480 con 0x7ff67010c170 2024-09-15T22:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.824+0000 7ff67583e640 1 -- 172.21.15.5:0/999965860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6701a6b60 con 0x7ff67010c170 2024-09-15T22:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.824+0000 7ff67583e640 1 -- 172.21.15.5:0/999965860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6701a7040 con 0x7ff67010c170 2024-09-15T22:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.825+0000 7ff67583e640 1 -- 172.21.15.5:0/999965860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff670103120 con 0x7ff67010c170 2024-09-15T22:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.826+0000 7ff67483c640 1 -- 172.21.15.5:0/999965860 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ff668037020 con 0x7ff67010c170 2024-09-15T22:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.827+0000 7ff67483c640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff6580a32d0 0x7ff6580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.827+0000 7ff66effd640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff6580a32d0 0x7ff6580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.827+0000 7ff67483c640 1 -- 172.21.15.5:0/999965860 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7ff6680f0cf0 con 0x7ff67010c170 2024-09-15T22:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.828+0000 7ff66effd640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff6580a32d0 0x7ff6580a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6600058c0 tx=0x7ff660005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:21.831+0000 7ff67483c640 1 -- 172.21.15.5:0/999965860 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff6680bd680 con 0x7ff67010c170 2024-09-15T22:07:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.033+0000 7ff67583e640 1 -- 172.21.15.5:0/999965860 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7ff6700feac0 con 0x7ff6580a32d0 2024-09-15T22:07:22.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:21 smithi005 ceph-mon[30944]: pgmap v305: 129 pgs: 129 active+clean; 590 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-15T22:07:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.038+0000 7ff67483c640 1 -- 172.21.15.5:0/999965860 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff6700feac0 con 0x7ff6580a32d0 2024-09-15T22:07:22.039 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_0/08129853-f913-48a6-8297-46fd9ea9cdfb 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 -- 172.21.15.5:0/999965860 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff6580a32d0 msgr2=0x7ff6580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff6580a32d0 0x7ff6580a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff6600058c0 tx=0x7ff660005850 comp rx=0 tx=0).stop 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 -- 172.21.15.5:0/999965860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67010c170 msgr2=0x7ff670101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67010c170 0x7ff670101420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff66802ae70 tx=0x7ff66802f5b0 comp rx=0 tx=0).stop 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 -- 172.21.15.5:0/999965860 shutdown_connections 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff6580a32d0 0x7ff6580a5790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff67010c170 0x7ff670101420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff670103660 0x7ff670100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 --2- 172.21.15.5:0/999965860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff670102d40 0x7ff6701009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 -- 172.21.15.5:0/999965860 >> 172.21.15.5:0/999965860 conn(0x7ff6700fc7a0 msgr2=0x7ff6700fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 -- 172.21.15.5:0/999965860 shutdown_connections 2024-09-15T22:07:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:22.040+0000 7ff67583e640 1 -- 172.21.15.5:0/999965860 wait complete. 2024-09-15T22:07:22.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:21 smithi099 ceph-mon[34892]: pgmap v305: 129 pgs: 129 active+clean; 590 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-15T22:07:22.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:21 smithi129 ceph-mon[35314]: pgmap v305: 129 pgs: 129 active+clean; 590 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-15T22:07:22.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:22 smithi005 ceph-mon[30944]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:22.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-15T22:07:23.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:22 smithi099 ceph-mon[34892]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:23.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:22 smithi129 ceph-mon[35314]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:23.097 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.568+0000 7f44422ba640 1 -- 172.21.15.5:0/1301341162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443c101420 msgr2=0x7f443c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.568+0000 7f44422ba640 1 --2- 172.21.15.5:0/1301341162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443c101420 0x7f443c10d950 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4434009cd0 tx=0x7f443402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.568+0000 7f44422ba640 1 -- 172.21.15.5:0/1301341162 shutdown_connections 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.568+0000 7f44422ba640 1 --2- 172.21.15.5:0/1301341162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443c101420 0x7f443c10d950 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.568+0000 7f44422ba640 1 --2- 172.21.15.5:0/1301341162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f443c100a80 0x7f443c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.568+0000 7f44422ba640 1 --2- 172.21.15.5:0/1301341162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f443c106a80 0x7f443c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.568+0000 7f44422ba640 1 -- 172.21.15.5:0/1301341162 >> 172.21.15.5:0/1301341162 conn(0x7f443c0fc7a0 msgr2=0x7f443c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.568+0000 7f44422ba640 1 -- 172.21.15.5:0/1301341162 shutdown_connections 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.568+0000 7f44422ba640 1 -- 172.21.15.5:0/1301341162 wait complete. 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f44422ba640 1 Processor -- start 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f44422ba640 1 -- start start 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f44422ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443c100a80 0x7f443c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f44422ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f443c101420 0x7f443c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f44422ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f443c106a80 0x7f443c1a4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f44422ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f443c110e80 con 0x7f443c100a80 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f44422ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f443c110d00 con 0x7f443c106a80 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f44422ba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f443c111000 con 0x7f443c101420 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f443bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443c100a80 0x7f443c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f443bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443c100a80 0x7f443c199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52942/0 (socket says 172.21.15.5:52942) 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f443bfff640 1 -- 172.21.15.5:0/998659361 learned_addr learned my addr 172.21.15.5:0/998659361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f443b7fe640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f443c101420 0x7f443c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.569+0000 7f4440857640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f443c106a80 0x7f443c1a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.570+0000 7f443bfff640 1 -- 172.21.15.5:0/998659361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f443c101420 msgr2=0x7f443c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.570+0000 7f443bfff640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f443c101420 0x7f443c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.570+0000 7f443bfff640 1 -- 172.21.15.5:0/998659361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f443c106a80 msgr2=0x7f443c1a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.570+0000 7f443bfff640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f443c106a80 0x7f443c1a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.570+0000 7f443bfff640 1 -- 172.21.15.5:0/998659361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f443c1a4740 con 0x7f443c100a80 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.570+0000 7f443bfff640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443c100a80 0x7f443c199180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f442c00d950 tx=0x7f442c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.570+0000 7f44397fa640 1 -- 172.21.15.5:0/998659361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c017070 con 0x7f443c100a80 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.570+0000 7f44422ba640 1 -- 172.21.15.5:0/998659361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f443c1a4910 con 0x7f443c100a80 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.570+0000 7f44397fa640 1 -- 172.21.15.5:0/998659361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f442c009c30 con 0x7f443c100a80 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.570+0000 7f44397fa640 1 -- 172.21.15.5:0/998659361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c00f6b0 con 0x7f443c100a80 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.570+0000 7f44422ba640 1 -- 172.21.15.5:0/998659361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443c1a4ea0 con 0x7f443c100a80 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.571+0000 7f44422ba640 1 -- 172.21.15.5:0/998659361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f443c100ee0 con 0x7f443c100a80 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.572+0000 7f44397fa640 1 -- 172.21.15.5:0/998659361 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f442c0109d0 con 0x7f443c100a80 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.573+0000 7f44397fa640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f44240a32d0 0x7f44240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.573+0000 7f443b7fe640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f44240a32d0 0x7f44240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.573+0000 7f44397fa640 1 -- 172.21.15.5:0/998659361 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f442c0cfa90 con 0x7f443c100a80 2024-09-15T22:07:23.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.573+0000 7f443b7fe640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f44240a32d0 0x7f44240a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4430005e10 tx=0x7f4430005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:23.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.577+0000 7f44397fa640 1 -- 172.21.15.5:0/998659361 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f442c009de0 con 0x7f443c100a80 2024-09-15T22:07:23.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.780+0000 7f44422ba640 1 -- 172.21.15.5:0/998659361 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f443c0fe550 con 0x7f44240a32d0 2024-09-15T22:07:23.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.805+0000 7f44397fa640 1 -- 172.21.15.5:0/998659361 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f443c0fe550 con 0x7f44240a32d0 2024-09-15T22:07:23.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 -- 172.21.15.5:0/998659361 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f44240a32d0 msgr2=0x7f44240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:23.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f44240a32d0 0x7f44240a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4430005e10 tx=0x7f4430005da0 comp rx=0 tx=0).stop 2024-09-15T22:07:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 -- 172.21.15.5:0/998659361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443c100a80 msgr2=0x7f443c199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443c100a80 0x7f443c199180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f442c00d950 tx=0x7f442c00de10 comp rx=0 tx=0).stop 2024-09-15T22:07:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 -- 172.21.15.5:0/998659361 shutdown_connections 2024-09-15T22:07:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f443c106a80 0x7f443c1a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f443c101420 0x7f443c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f44240a32d0 0x7f44240a5790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 --2- 172.21.15.5:0/998659361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f443c100a80 0x7f443c199180 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 -- 172.21.15.5:0/998659361 >> 172.21.15.5:0/998659361 conn(0x7f443c0fc7a0 msgr2=0x7f443c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 -- 172.21.15.5:0/998659361 shutdown_connections 2024-09-15T22:07:23.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:23.807+0000 7f44422ba640 1 -- 172.21.15.5:0/998659361 wait complete. 2024-09-15T22:07:24.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:23 smithi099 ceph-mon[34892]: pgmap v306: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-15T22:07:24.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:23 smithi129 ceph-mon[35314]: pgmap v306: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-15T22:07:24.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:23 smithi005 ceph-mon[30944]: pgmap v306: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-15T22:07:24.527 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-15T22:07:24.788 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:25.034 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:24 smithi005 ceph-mon[30944]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:25.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:24 smithi099 ceph-mon[34892]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:25.094 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:24 smithi129 ceph-mon[35314]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.257+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/2283212457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48106f10 msgr2=0x7f1a4810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.257+0000 7f1a4ef0d640 1 --2- 172.21.15.5:0/2283212457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48106f10 0x7f1a4810ea90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009ca0 tx=0x7f1a4002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:07:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.257+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/2283212457 shutdown_connections 2024-09-15T22:07:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.257+0000 7f1a4ef0d640 1 --2- 172.21.15.5:0/2283212457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48106f10 0x7f1a4810ea90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.257+0000 7f1a4ef0d640 1 --2- 172.21.15.5:0/2283212457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48102790 0x7f1a481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.257+0000 7f1a4ef0d640 1 --2- 172.21.15.5:0/2283212457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a48101de0 0x7f1a481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.257+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/2283212457 >> 172.21.15.5:0/2283212457 conn(0x7f1a480fd8c0 msgr2=0x7f1a480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.257+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/2283212457 shutdown_connections 2024-09-15T22:07:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.257+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/2283212457 wait complete. 2024-09-15T22:07:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a4ef0d640 1 Processor -- start 2024-09-15T22:07:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a4ef0d640 1 -- start start 2024-09-15T22:07:25.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a4ef0d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48101de0 0x7f1a48198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a4ef0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48102790 0x7f1a481989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a4ef0d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a48106f10 0x7f1a4819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a4ef0d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48111f90 con 0x7f1a48102790 2024-09-15T22:07:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a4ef0d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48111e10 con 0x7f1a48101de0 2024-09-15T22:07:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a4ef0d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48112110 con 0x7f1a48106f10 2024-09-15T22:07:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48102790 0x7f1a481989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48102790 0x7f1a481989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52966/0 (socket says 172.21.15.5:52966) 2024-09-15T22:07:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a47fff640 1 -- 172.21.15.5:0/655525287 learned_addr learned my addr 172.21.15.5:0/655525287 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a4cca9640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48101de0 0x7f1a48198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a4d4aa640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a48106f10 0x7f1a4819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a47fff640 1 -- 172.21.15.5:0/655525287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a48106f10 msgr2=0x7f1a4819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a47fff640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a48106f10 0x7f1a4819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a47fff640 1 -- 172.21.15.5:0/655525287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48101de0 msgr2=0x7f1a48198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a47fff640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48101de0 0x7f1a48198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.258+0000 7f1a47fff640 1 -- 172.21.15.5:0/655525287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4819d410 con 0x7f1a48102790 2024-09-15T22:07:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.259+0000 7f1a4cca9640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48101de0 0x7f1a48198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:07:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.259+0000 7f1a47fff640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48102790 0x7f1a481989a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400d950 tx=0x7f1a3400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.259+0000 7f1a45ffb640 1 -- 172.21.15.5:0/655525287 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34017070 con 0x7f1a48102790 2024-09-15T22:07:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.259+0000 7f1a45ffb640 1 -- 172.21.15.5:0/655525287 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a34009c30 con 0x7f1a48102790 2024-09-15T22:07:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.259+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/655525287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4819d700 con 0x7f1a48102790 2024-09-15T22:07:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.259+0000 7f1a45ffb640 1 -- 172.21.15.5:0/655525287 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400f6b0 con 0x7f1a48102790 2024-09-15T22:07:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.259+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/655525287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a481a4fe0 con 0x7f1a48102790 2024-09-15T22:07:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.260+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/655525287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a481021c0 con 0x7f1a48102790 2024-09-15T22:07:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.261+0000 7f1a45ffb640 1 -- 172.21.15.5:0/655525287 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f1a340109d0 con 0x7f1a48102790 2024-09-15T22:07:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.262+0000 7f1a45ffb640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1a380a3350 0x7f1a380a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.262+0000 7f1a45ffb640 1 -- 172.21.15.5:0/655525287 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f1a340cfa80 con 0x7f1a48102790 2024-09-15T22:07:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.262+0000 7f1a4cca9640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1a380a3350 0x7f1a380a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.262+0000 7f1a4cca9640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1a380a3350 0x7f1a380a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009930 tx=0x7f1a3c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.266+0000 7f1a45ffb640 1 -- 172.21.15.5:0/655525287 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a34009de0 con 0x7f1a48102790 2024-09-15T22:07:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.470+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/655525287 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1a4810a150 con 0x7f1a380a3350 2024-09-15T22:07:25.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.474+0000 7f1a45ffb640 1 -- 172.21.15.5:0/655525287 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1a4810a150 con 0x7f1a380a3350 2024-09-15T22:07:25.476 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_1/b72084f4-a7c4-4312-ae50-6ba93e80f47a 2024-09-15T22:07:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/655525287 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1a380a3350 msgr2=0x7f1a380a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1a380a3350 0x7f1a380a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009930 tx=0x7f1a3c008040 comp rx=0 tx=0).stop 2024-09-15T22:07:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/655525287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48102790 msgr2=0x7f1a481989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48102790 0x7f1a481989a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400d950 tx=0x7f1a3400de10 comp rx=0 tx=0).stop 2024-09-15T22:07:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/655525287 shutdown_connections 2024-09-15T22:07:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a48106f10 0x7f1a4819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1a380a3350 0x7f1a380a5810 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48102790 0x7f1a481989a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:25.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 --2- 172.21.15.5:0/655525287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48101de0 0x7f1a48198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:25.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/655525287 >> 172.21.15.5:0/655525287 conn(0x7f1a480fd8c0 msgr2=0x7f1a480fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:25.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/655525287 shutdown_connections 2024-09-15T22:07:25.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:25.476+0000 7f1a4ef0d640 1 -- 172.21.15.5:0/655525287 wait complete. 2024-09-15T22:07:26.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:25 smithi099 ceph-mon[34892]: pgmap v307: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-09-15T22:07:26.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:25 smithi099 ceph-mon[34892]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:26.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:25 smithi129 ceph-mon[35314]: pgmap v307: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-09-15T22:07:26.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:25 smithi129 ceph-mon[35314]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:26.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:25 smithi005 ceph-mon[30944]: pgmap v307: 129 pgs: 129 active+clean; 591 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 3.1 KiB/s wr, 13 op/s 2024-09-15T22:07:26.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:25 smithi005 ceph-mon[30944]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:07:26.163 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:07:26.418 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.877+0000 7faf81858640 1 -- 172.21.15.5:0/2319569267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c106f20 msgr2=0x7faf7c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.877+0000 7faf81858640 1 --2- 172.21.15.5:0/2319569267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c106f20 0x7faf7c10eaa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf74009900 tx=0x7faf7402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:07:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.877+0000 7faf81858640 1 -- 172.21.15.5:0/2319569267 shutdown_connections 2024-09-15T22:07:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.877+0000 7faf81858640 1 --2- 172.21.15.5:0/2319569267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c106f20 0x7faf7c10eaa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.877+0000 7faf81858640 1 --2- 172.21.15.5:0/2319569267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf7c1027a0 0x7faf7c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.877+0000 7faf81858640 1 --2- 172.21.15.5:0/2319569267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7c101df0 0x7faf7c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.877+0000 7faf81858640 1 -- 172.21.15.5:0/2319569267 >> 172.21.15.5:0/2319569267 conn(0x7faf7c0fd8f0 msgr2=0x7faf7c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.877+0000 7faf81858640 1 -- 172.21.15.5:0/2319569267 shutdown_connections 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.877+0000 7faf81858640 1 -- 172.21.15.5:0/2319569267 wait complete. 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.877+0000 7faf81858640 1 Processor -- start 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf81858640 1 -- start start 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf81858640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf7c101df0 0x7faf7c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf81858640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c1027a0 0x7faf7c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf81858640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7c106f20 0x7faf7c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf81858640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c111f80 con 0x7faf7c1027a0 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf81858640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c111e00 con 0x7faf7c106f20 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf81858640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faf7c112100 con 0x7faf7c101df0 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf7a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c1027a0 0x7faf7c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf7a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c1027a0 0x7faf7c198800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52976/0 (socket says 172.21.15.5:52976) 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf7a7fc640 1 -- 172.21.15.5:0/1742898001 learned_addr learned my addr 172.21.15.5:0/1742898001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:26.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf7b7fe640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7c106f20 0x7faf7c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf7a7fc640 1 -- 172.21.15.5:0/1742898001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf7c101df0 msgr2=0x7faf7c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf7a7fc640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf7c101df0 0x7faf7c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf7a7fc640 1 -- 172.21.15.5:0/1742898001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7c106f20 msgr2=0x7faf7c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf7a7fc640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7c106f20 0x7faf7c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf7a7fc640 1 -- 172.21.15.5:0/1742898001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7c19d270 con 0x7faf7c1027a0 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.878+0000 7faf7a7fc640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c1027a0 0x7faf7c198800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c920 tx=0x7faf7000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.879+0000 7faf80856640 1 -- 172.21.15.5:0/1742898001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70007c70 con 0x7faf7c1027a0 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.879+0000 7faf81858640 1 -- 172.21.15.5:0/1742898001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7c19d560 con 0x7faf7c1027a0 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.879+0000 7faf80856640 1 -- 172.21.15.5:0/1742898001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf70007e10 con 0x7faf7c1027a0 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.879+0000 7faf80856640 1 -- 172.21.15.5:0/1742898001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf70018890 con 0x7faf7c1027a0 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.879+0000 7faf81858640 1 -- 172.21.15.5:0/1742898001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7c1a4e40 con 0x7faf7c1027a0 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.880+0000 7faf81858640 1 -- 172.21.15.5:0/1742898001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7c1021d0 con 0x7faf7c1027a0 2024-09-15T22:07:26.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.881+0000 7faf80856640 1 -- 172.21.15.5:0/1742898001 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7faf70018a70 con 0x7faf7c1027a0 2024-09-15T22:07:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.881+0000 7faf80856640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf640a3350 0x7faf640a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.882+0000 7faf7affd640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf640a3350 0x7faf640a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.882+0000 7faf80856640 1 -- 172.21.15.5:0/1742898001 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7faf700cfea0 con 0x7faf7c1027a0 2024-09-15T22:07:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.882+0000 7faf7affd640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf640a3350 0x7faf640a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf6c005fd0 tx=0x7faf6c0071c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:26.886+0000 7faf80856640 1 -- 172.21.15.5:0/1742898001 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf70014030 con 0x7faf7c1027a0 2024-09-15T22:07:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.096+0000 7faf81858640 1 -- 172.21.15.5:0/1742898001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf7c1a51a0 con 0x7faf7c1027a0 2024-09-15T22:07:27.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.097+0000 7faf80856640 1 -- 172.21.15.5:0/1742898001 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11897 (secure 0 0 0) 0x7faf7009c6c0 con 0x7faf7c1027a0 2024-09-15T22:07:27.099 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:07:27.099 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"btime":"2024-09-15T22:07:16:736368+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:07:16.364125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24466,24467,24491]},"id":1}]} 2024-09-15T22:07:27.099 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 24 2024-09-15T22:07:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 -- 172.21.15.5:0/1742898001 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf640a3350 msgr2=0x7faf640a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf640a3350 0x7faf640a5810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faf6c005fd0 tx=0x7faf6c0071c0 comp rx=0 tx=0).stop 2024-09-15T22:07:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 -- 172.21.15.5:0/1742898001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c1027a0 msgr2=0x7faf7c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c1027a0 0x7faf7c198800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c920 tx=0x7faf7000cde0 comp rx=0 tx=0).stop 2024-09-15T22:07:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 -- 172.21.15.5:0/1742898001 shutdown_connections 2024-09-15T22:07:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faf7c106f20 0x7faf7c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7faf640a3350 0x7faf640a5810 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf7c1027a0 0x7faf7c198800 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 --2- 172.21.15.5:0/1742898001 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faf7c101df0 0x7faf7c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 -- 172.21.15.5:0/1742898001 >> 172.21.15.5:0/1742898001 conn(0x7faf7c0fd8f0 msgr2=0x7faf7c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:27.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 -- 172.21.15.5:0/1742898001 shutdown_connections 2024-09-15T22:07:27.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:27.099+0000 7faf81858640 1 -- 172.21.15.5:0/1742898001 wait complete. 2024-09-15T22:07:27.776 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:07:28.032 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:28.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:27 smithi005 ceph-mon[30944]: pgmap v308: 129 pgs: 129 active+clean; 623 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-15T22:07:28.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:27 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1742898001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:07:28.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:27 smithi099 ceph-mon[34892]: pgmap v308: 129 pgs: 129 active+clean; 623 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-15T22:07:28.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:27 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1742898001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:07:28.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:27 smithi129 ceph-mon[35314]: pgmap v308: 129 pgs: 129 active+clean; 623 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-09-15T22:07:28.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:27 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1742898001' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:07:28.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.467+0000 7f1bb4872640 1 -- 172.21.15.5:0/2257705893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb0106f10 msgr2=0x7f1bb010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:28.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.467+0000 7f1bb4872640 1 --2- 172.21.15.5:0/2257705893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb0106f10 0x7f1bb010ea90 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8009ca0 tx=0x7f1ba802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:07:28.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.467+0000 7f1bb4872640 1 -- 172.21.15.5:0/2257705893 shutdown_connections 2024-09-15T22:07:28.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.467+0000 7f1bb4872640 1 --2- 172.21.15.5:0/2257705893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb0106f10 0x7f1bb010ea90 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:28.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.467+0000 7f1bb4872640 1 --2- 172.21.15.5:0/2257705893 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb0102790 0x7f1bb01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:28.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.467+0000 7f1bb4872640 1 --2- 172.21.15.5:0/2257705893 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bb0101de0 0x7f1bb01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:28.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.467+0000 7f1bb4872640 1 -- 172.21.15.5:0/2257705893 >> 172.21.15.5:0/2257705893 conn(0x7f1bb00fd8c0 msgr2=0x7f1bb00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:28.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.467+0000 7f1bb4872640 1 -- 172.21.15.5:0/2257705893 shutdown_connections 2024-09-15T22:07:28.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.467+0000 7f1bb4872640 1 -- 172.21.15.5:0/2257705893 wait complete. 2024-09-15T22:07:28.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.468+0000 7f1bb4872640 1 Processor -- start 2024-09-15T22:07:28.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.468+0000 7f1bb4872640 1 -- start start 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.468+0000 7f1bb4872640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb0101de0 0x7f1bb00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.468+0000 7f1bb4872640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bb0102790 0x7f1bb0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.468+0000 7f1bb4872640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb0106f10 0x7f1bb0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.468+0000 7f1bb4872640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0112010 con 0x7f1bb0101de0 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.468+0000 7f1bb4872640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0111e90 con 0x7f1bb0102790 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.468+0000 7f1bb4872640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb0112190 con 0x7f1bb0106f10 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.468+0000 7f1bae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb0101de0 0x7f1bb00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.468+0000 7f1bae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb0101de0 0x7f1bb00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52994/0 (socket says 172.21.15.5:52994) 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.468+0000 7f1bae59c640 1 -- 172.21.15.5:0/94582056 learned_addr learned my addr 172.21.15.5:0/94582056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1baed9d640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb0106f10 0x7f1bb0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1badd9b640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bb0102790 0x7f1bb0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:28.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1bae59c640 1 -- 172.21.15.5:0/94582056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb0106f10 msgr2=0x7f1bb0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1bae59c640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb0106f10 0x7f1bb0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1bae59c640 1 -- 172.21.15.5:0/94582056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bb0102790 msgr2=0x7f1bb0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1bae59c640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bb0102790 0x7f1bb0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1bae59c640 1 -- 172.21.15.5:0/94582056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb01a6ab0 con 0x7f1bb0101de0 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1baed9d640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb0106f10 0x7f1bb0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1bae59c640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb0101de0 0x7f1bb00744f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000b4a0 tx=0x7f1ba000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1b9f7fe640 1 -- 172.21.15.5:0/94582056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0014070 con 0x7f1bb0101de0 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1b9f7fe640 1 -- 172.21.15.5:0/94582056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ba0010af0 con 0x7f1bb0101de0 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1bb4872640 1 -- 172.21.15.5:0/94582056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb01a6da0 con 0x7f1bb0101de0 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1b9f7fe640 1 -- 172.21.15.5:0/94582056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0017d10 con 0x7f1bb0101de0 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.469+0000 7f1bb4872640 1 -- 172.21.15.5:0/94582056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb01a7250 con 0x7f1bb0101de0 2024-09-15T22:07:28.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.470+0000 7f1bb4872640 1 -- 172.21.15.5:0/94582056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb0102240 con 0x7f1bb0101de0 2024-09-15T22:07:28.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.472+0000 7f1b9f7fe640 1 -- 172.21.15.5:0/94582056 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f1ba0017070 con 0x7f1bb0101de0 2024-09-15T22:07:28.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.473+0000 7f1b9f7fe640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1b980a32d0 0x7f1b980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:28.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.473+0000 7f1badd9b640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1b980a32d0 0x7f1b980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:28.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.473+0000 7f1b9f7fe640 1 -- 172.21.15.5:0/94582056 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f1ba00cf0e0 con 0x7f1bb0101de0 2024-09-15T22:07:28.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.473+0000 7f1badd9b640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1b980a32d0 0x7f1b980a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0075f80 tx=0x7f1ba4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:28.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.477+0000 7f1b9f7fe640 1 -- 172.21.15.5:0/94582056 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ba009ba70 con 0x7f1bb0101de0 2024-09-15T22:07:28.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.657+0000 7f1bb4872640 1 -- 172.21.15.5:0/94582056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1bb0075a20 con 0x7f1bb0101de0 2024-09-15T22:07:28.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.659+0000 7f1b9f7fe640 1 -- 172.21.15.5:0/94582056 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24549 (secure 0 0 0) 0x7f1ba009b370 con 0x7f1bb0101de0 2024-09-15T22:07:28.660 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:07:28.660 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:06:47.363102+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:07:28.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 -- 172.21.15.5:0/94582056 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1b980a32d0 msgr2=0x7f1b980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:28.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1b980a32d0 0x7f1b980a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0075f80 tx=0x7f1ba4008040 comp rx=0 tx=0).stop 2024-09-15T22:07:28.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 -- 172.21.15.5:0/94582056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb0101de0 msgr2=0x7f1bb00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:28.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb0101de0 0x7f1bb00744f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000b4a0 tx=0x7f1ba000b960 comp rx=0 tx=0).stop 2024-09-15T22:07:28.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 -- 172.21.15.5:0/94582056 shutdown_connections 2024-09-15T22:07:28.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1bb0106f10 0x7f1bb0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:28.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1bb0102790 0x7f1bb0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:28.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1b980a32d0 0x7f1b980a5790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:28.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 --2- 172.21.15.5:0/94582056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb0101de0 0x7f1bb00744f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:28.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 -- 172.21.15.5:0/94582056 >> 172.21.15.5:0/94582056 conn(0x7f1bb00fd8c0 msgr2=0x7f1bb00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:28.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 -- 172.21.15.5:0/94582056 shutdown_connections 2024-09-15T22:07:28.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:28.661+0000 7f1bb4872640 1 -- 172.21.15.5:0/94582056 wait complete. 2024-09-15T22:07:29.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:28 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/94582056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:29.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:28 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/94582056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:29.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:28 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/94582056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:29.356 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T22:07:29.356 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:07:29.614 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:29.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:29 smithi005 ceph-mon[30944]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2024-09-15T22:07:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.076+0000 7fdfe6d10640 1 -- 172.21.15.5:0/617287264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfe0101420 msgr2=0x7fdfe010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.076+0000 7fdfe6d10640 1 --2- 172.21.15.5:0/617287264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfe0101420 0x7fdfe010d950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8009cd0 tx=0x7fdfd802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.076+0000 7fdfe6d10640 1 -- 172.21.15.5:0/617287264 shutdown_connections 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.076+0000 7fdfe6d10640 1 --2- 172.21.15.5:0/617287264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfe0101420 0x7fdfe010d950 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.076+0000 7fdfe6d10640 1 --2- 172.21.15.5:0/617287264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfe0100a80 0x7fdfe0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.076+0000 7fdfe6d10640 1 --2- 172.21.15.5:0/617287264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0106a80 0x7fdfe0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.076+0000 7fdfe6d10640 1 -- 172.21.15.5:0/617287264 >> 172.21.15.5:0/617287264 conn(0x7fdfe00fc7a0 msgr2=0x7fdfe00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.076+0000 7fdfe6d10640 1 -- 172.21.15.5:0/617287264 shutdown_connections 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.076+0000 7fdfe6d10640 1 -- 172.21.15.5:0/617287264 wait complete. 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe6d10640 1 Processor -- start 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe6d10640 1 -- start start 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe6d10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfe0100a80 0x7fdfe010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe6d10640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfe0101420 0x7fdfe010e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe6d10640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0106a80 0x7fdfe010ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe6d10640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0112020 con 0x7fdfe0100a80 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe6d10640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe0111ea0 con 0x7fdfe0106a80 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe6d10640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe01121a0 con 0x7fdfe0101420 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe4aac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfe0100a80 0x7fdfe010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe4aac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfe0100a80 0x7fdfe010e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53000/0 (socket says 172.21.15.5:53000) 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe4aac640 1 -- 172.21.15.5:0/1481660619 learned_addr learned my addr 172.21.15.5:0/1481660619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfdffff640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfe0101420 0x7fdfe010e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe52ad640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0106a80 0x7fdfe010ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe4aac640 1 -- 172.21.15.5:0/1481660619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfe0101420 msgr2=0x7fdfe010e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe4aac640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfe0101420 0x7fdfe010e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe4aac640 1 -- 172.21.15.5:0/1481660619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0106a80 msgr2=0x7fdfe010ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe4aac640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0106a80 0x7fdfe010ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.077+0000 7fdfe4aac640 1 -- 172.21.15.5:0/1481660619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe01a6a60 con 0x7fdfe0100a80 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.078+0000 7fdfe4aac640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfe0100a80 0x7fdfe010e3e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400b4a0 tx=0x7fdfd400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.078+0000 7fdfddffb640 1 -- 172.21.15.5:0/1481660619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4014070 con 0x7fdfe0100a80 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.078+0000 7fdfe6d10640 1 -- 172.21.15.5:0/1481660619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe01a6d50 con 0x7fdfe0100a80 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.078+0000 7fdfddffb640 1 -- 172.21.15.5:0/1481660619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd4010af0 con 0x7fdfe0100a80 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.078+0000 7fdfddffb640 1 -- 172.21.15.5:0/1481660619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4017d10 con 0x7fdfe0100a80 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.078+0000 7fdfe6d10640 1 -- 172.21.15.5:0/1481660619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe01a7230 con 0x7fdfe0100a80 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.079+0000 7fdfe6d10640 1 -- 172.21.15.5:0/1481660619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe0100f60 con 0x7fdfe0100a80 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.080+0000 7fdfddffb640 1 -- 172.21.15.5:0/1481660619 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fdfd4017070 con 0x7fdfe0100a80 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.081+0000 7fdfddffb640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdfd00a32d0 0x7fdfd00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.081+0000 7fdfdffff640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdfd00a32d0 0x7fdfd00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.081+0000 7fdfddffb640 1 -- 172.21.15.5:0/1481660619 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7fdfd40cf150 con 0x7fdfe0100a80 2024-09-15T22:07:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.081+0000 7fdfdffff640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdfd00a32d0 0x7fdfd00a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdfe010fe70 tx=0x7fdfcc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.085+0000 7fdfddffb640 1 -- 172.21.15.5:0/1481660619 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdfd409ba80 con 0x7fdfe0100a80 2024-09-15T22:07:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.257+0000 7fdfe6d10640 1 -- 172.21.15.5:0/1481660619 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfe0062630 con 0x7fdfd00a32d0 2024-09-15T22:07:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.269+0000 7fdfddffb640 1 -- 172.21.15.5:0/1481660619 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340337 (secure 0 0 0) 0x7fdfe0062630 con 0x7fdfd00a32d0 2024-09-15T22:07:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:07:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-15T22:07:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 -- 172.21.15.5:0/1481660619 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdfd00a32d0 msgr2=0x7fdfd00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdfd00a32d0 0x7fdfd00a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdfe010fe70 tx=0x7fdfcc008040 comp rx=0 tx=0).stop 2024-09-15T22:07:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 -- 172.21.15.5:0/1481660619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfe0100a80 msgr2=0x7fdfe010e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfe0100a80 0x7fdfe010e3e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400b4a0 tx=0x7fdfd400b960 comp rx=0 tx=0).stop 2024-09-15T22:07:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 -- 172.21.15.5:0/1481660619 shutdown_connections 2024-09-15T22:07:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdfe0106a80 0x7fdfe010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdfe0101420 0x7fdfe010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdfd00a32d0 0x7fdfd00a5790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 --2- 172.21.15.5:0/1481660619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfe0100a80 0x7fdfe010e3e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 -- 172.21.15.5:0/1481660619 >> 172.21.15.5:0/1481660619 conn(0x7fdfe00fc7a0 msgr2=0x7fdfe00fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 -- 172.21.15.5:0/1481660619 shutdown_connections 2024-09-15T22:07:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:30.272+0000 7fdfe6d10640 1 -- 172.21.15.5:0/1481660619 wait complete. 2024-09-15T22:07:30.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:29 smithi099 ceph-mon[34892]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2024-09-15T22:07:30.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:29 smithi129 ceph-mon[35314]: pgmap v309: 129 pgs: 129 active+clean; 623 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.3 KiB/s wr, 12 op/s 2024-09-15T22:07:31.058 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-15T22:07:28.798815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638343,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":236,"num_read_kb":220,"num_write":340,"num_write_kb":3034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387676,"kb_used_data":13468,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554148,"statfs":{"total":1151940427776,"available":1151543447552,"internally_reserved":0,"allocated":13791232,"data_stored":7700241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41927,"num_objects":18,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":95,"num_read_kb":94,"num_write":56,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003116"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.737074+0000","last_change":"2024-09-15T22:06:44.668504+0000","last_active":"2024-09-15T22:07:13.737074+0000","last_peered":"2024-09-15T22:07:13.737074+0000","last_clean":"2024-09-15T22:07:13.737074+0000","last_became_active":"2024-09-15T22:06:44.661198+0000","last_became_peered":"2024-09-15T22:06:44.661198+0000","last_unstale":"2024-09-15T22:07:13.737074+0000","last_undegraded":"2024-09-15T22:07:13.737074+0000","last_fullsized":"2024-09-15T22:07:13.737074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:33.749045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645317+0000","last_change":"2024-09-15T22:06:37.184096+0000","last_active":"2024-09-15T22:06:47.645317+0000","last_peered":"2024-09-15T22:06:47.645317+0000","last_clean":"2024-09-15T22:06:47.645317+0000","last_became_active":"2024-09-15T22:06:37.183758+0000","last_became_peered":"2024-09-15T22:06:37.183758+0000","last_unstale":"2024-09-15T22:06:47.645317+0000","last_undegraded":"2024-09-15T22:06:47.645317+0000","last_fullsized":"2024-09-15T22:06:47.645317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:04.210551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849243+0000","last_change":"2024-09-15T22:06:44.653042+0000","last_active":"2024-09-15T22:06:47.849243+0000","last_peered":"2024-09-15T22:06:47.849243+0000","last_clean":"2024-09-15T22:06:47.849243+0000","last_became_active":"2024-09-15T22:06:44.652658+0000","last_became_peered":"2024-09-15T22:06:44.652658+0000","last_unstale":"2024-09-15T22:06:47.849243+0000","last_undegraded":"2024-09-15T22:06:47.849243+0000","last_fullsized":"2024-09-15T22:06:47.849243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:59.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645466+0000","last_change":"2024-09-15T22:06:35.355392+0000","last_active":"2024-09-15T22:06:47.645466+0000","last_peered":"2024-09-15T22:06:47.645466+0000","last_clean":"2024-09-15T22:06:47.645466+0000","last_became_active":"2024-09-15T22:06:35.355042+0000","last_became_peered":"2024-09-15T22:06:35.355042+0000","last_unstale":"2024-09-15T22:06:47.645466+0000","last_undegraded":"2024-09-15T22:06:47.645466+0000","last_fullsized":"2024-09-15T22:06:47.645466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:05.186937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851100+0000","last_change":"2024-09-15T22:06:37.184421+0000","last_active":"2024-09-15T22:06:47.851100+0000","last_peered":"2024-09-15T22:06:47.851100+0000","last_clean":"2024-09-15T22:06:47.851100+0000","last_became_active":"2024-09-15T22:06:37.184092+0000","last_became_peered":"2024-09-15T22:06:37.184092+0000","last_unstale":"2024-09-15T22:06:47.851100+0000","last_undegraded":"2024-09-15T22:06:47.851100+0000","last_fullsized":"2024-09-15T22:06:47.851100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:41.080981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646238+0000","last_change":"2024-09-15T22:06:44.656525+0000","last_active":"2024-09-15T22:06:47.646238+0000","last_peered":"2024-09-15T22:06:47.646238+0000","last_clean":"2024-09-15T22:06:47.646238+0000","last_became_active":"2024-09-15T22:06:44.656131+0000","last_became_peered":"2024-09-15T22:06:44.656131+0000","last_unstale":"2024-09-15T22:06:47.646238+0000","last_undegraded":"2024-09-15T22:06:47.646238+0000","last_fullsized":"2024-09-15T22:06:47.646238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:10:17.056927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.734100+0000","last_change":"2024-09-15T22:06:35.134606+0000","last_active":"2024-09-15T22:07:13.734100+0000","last_peered":"2024-09-15T22:07:13.734100+0000","last_clean":"2024-09-15T22:07:13.734100+0000","last_became_active":"2024-09-15T22:06:35.134263+0000","last_became_peered":"2024-09-15T22:06:35.134263+0000","last_unstale":"2024-09-15T22:07:13.734100+0000","last_undegraded":"2024-09-15T22:07:13.734100+0000","last_fullsized":"2024-09-15T22:07:13.734100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:45.008935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851556+0000","last_change":"2024-09-15T22:06:37.177556+0000","last_active":"2024-09-15T22:06:47.851556+0000","last_peered":"2024-09-15T22:06:47.851556+0000","last_clean":"2024-09-15T22:06:47.851556+0000","last_became_active":"2024-09-15T22:06:37.177161+0000","last_became_peered":"2024-09-15T22:06:37.177161+0000","last_unstale":"2024-09-15T22:06:47.851556+0000","last_undegraded":"2024-09-15T22:06:47.851556+0000","last_fullsized":"2024-09-15T22:06:47.851556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:30.192728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646502+0000","last_change":"2024-09-15T22:06:44.659097+0000","last_active":"2024-09-15T22:06:47.646502+0000","last_peered":"2024-09-15T22:06:47.646502+0000","last_clean":"2024-09-15T22:06:47.646502+0000","last_became_active":"2024-09-15T22:06:44.658626+0000","last_became_peered":"2024-09-15T22:06:44.658626+0000","last_unstale":"2024-09-15T22:06:47.646502+0000","last_undegraded":"2024-09-15T22:06:47.646502+0000","last_fullsized":"2024-09-15T22:06:47.646502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:01.308004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645834+0000","last_change":"2024-09-15T22:06:35.136326+0000","last_active":"2024-09-15T22:06:47.645834+0000","last_peered":"2024-09-15T22:06:47.645834+0000","last_clean":"2024-09-15T22:06:47.645834+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:06:47.645834+0000","last_undegraded":"2024-09-15T22:06:47.645834+0000","last_fullsized":"2024-09-15T22:06:47.645834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:28.675832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850874+0000","last_change":"2024-09-15T22:06:37.192939+0000","last_active":"2024-09-15T22:06:47.850874+0000","last_peered":"2024-09-15T22:06:47.850874+0000","last_clean":"2024-09-15T22:06:47.850874+0000","last_became_active":"2024-09-15T22:06:37.192616+0000","last_became_peered":"2024-09-15T22:06:37.192616+0000","last_unstale":"2024-09-15T22:06:47.850874+0000","last_undegraded":"2024-09-15T22:06:47.850874+0000","last_fullsized":"2024-09-15T22:06:47.850874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:03.615203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646361+0000","last_change":"2024-09-15T22:06:44.654741+0000","last_active":"2024-09-15T22:06:47.646361+0000","last_peered":"2024-09-15T22:06:47.646361+0000","last_clean":"2024-09-15T22:06:47.646361+0000","last_became_active":"2024-09-15T22:06:44.654362+0000","last_became_peered":"2024-09-15T22:06:44.654362+0000","last_unstale":"2024-09-15T22:06:47.646361+0000","last_undegraded":"2024-09-15T22:06:47.646361+0000","last_fullsized":"2024-09-15T22:06:47.646361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:59.269948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851179+0000","last_change":"2024-09-15T22:06:37.186385+0000","last_active":"2024-09-15T22:06:47.851179+0000","last_peered":"2024-09-15T22:06:47.851179+0000","last_clean":"2024-09-15T22:06:47.851179+0000","last_became_active":"2024-09-15T22:06:37.185588+0000","last_became_peered":"2024-09-15T22:06:37.185588+0000","last_unstale":"2024-09-15T22:06:47.851179+0000","last_undegraded":"2024-09-15T22:06:47.851179+0000","last_fullsized":"2024-09-15T22:06:47.851179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:22.523083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.725378+0000","last_change":"2024-09-15T22:06:35.137383+0000","last_active":"2024-09-15T22:07:12.725378+0000","last_peered":"2024-09-15T22:07:12.725378+0000","last_clean":"2024-09-15T22:07:12.725378+0000","last_became_active":"2024-09-15T22:06:35.137022+0000","last_became_peered":"2024-09-15T22:06:35.137022+0000","last_unstale":"2024-09-15T22:07:12.725378+0000","last_undegraded":"2024-09-15T22:07:12.725378+0000","last_fullsized":"2024-09-15T22:07:12.725378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.863155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646734+0000","last_change":"2024-09-15T22:06:44.653807+0000","last_active":"2024-09-15T22:06:47.646734+0000","last_peered":"2024-09-15T22:06:47.646734+0000","last_clean":"2024-09-15T22:06:47.646734+0000","last_became_active":"2024-09-15T22:06:44.653406+0000","last_became_peered":"2024-09-15T22:06:44.653406+0000","last_unstale":"2024-09-15T22:06:47.646734+0000","last_undegraded":"2024-09-15T22:06:47.646734+0000","last_fullsized":"2024-09-15T22:06:47.646734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:12.018208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645073+0000","last_change":"2024-09-15T22:06:37.186899+0000","last_active":"2024-09-15T22:06:47.645073+0000","last_peered":"2024-09-15T22:06:47.645073+0000","last_clean":"2024-09-15T22:06:47.645073+0000","last_became_active":"2024-09-15T22:06:37.183782+0000","last_became_peered":"2024-09-15T22:06:37.183782+0000","last_unstale":"2024-09-15T22:06:47.645073+0000","last_undegraded":"2024-09-15T22:06:47.645073+0000","last_fullsized":"2024-09-15T22:06:47.645073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:39.269186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850326+0000","last_change":"2024-09-15T22:06:44.652686+0000","last_active":"2024-09-15T22:06:47.850326+0000","last_peered":"2024-09-15T22:06:47.850326+0000","last_clean":"2024-09-15T22:06:47.850326+0000","last_became_active":"2024-09-15T22:06:44.652306+0000","last_became_peered":"2024-09-15T22:06:44.652306+0000","last_unstale":"2024-09-15T22:06:47.850326+0000","last_undegraded":"2024-09-15T22:06:47.850326+0000","last_fullsized":"2024-09-15T22:06:47.850326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:48.454535+0000","last_change":"2024-09-15T22:06:44.655424+0000","last_active":"2024-09-15T22:06:48.454535+0000","last_peered":"2024-09-15T22:06:48.454535+0000","last_clean":"2024-09-15T22:06:48.454535+0000","last_became_active":"2024-09-15T22:06:44.654959+0000","last_became_peered":"2024-09-15T22:06:44.654959+0000","last_unstale":"2024-09-15T22:06:48.454535+0000","last_undegraded":"2024-09-15T22:06:48.454535+0000","last_fullsized":"2024-09-15T22:06:48.454535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:07.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850289+0000","last_change":"2024-09-15T22:06:37.188424+0000","last_active":"2024-09-15T22:06:47.850289+0000","last_peered":"2024-09-15T22:06:47.850289+0000","last_clean":"2024-09-15T22:06:47.850289+0000","last_became_active":"2024-09-15T22:06:37.188082+0000","last_became_peered":"2024-09-15T22:06:37.188082+0000","last_unstale":"2024-09-15T22:06:47.850289+0000","last_undegraded":"2024-09-15T22:06:47.850289+0000","last_fullsized":"2024-09-15T22:06:47.850289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:07.159341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.729202+0000","last_change":"2024-09-15T22:06:35.134237+0000","last_active":"2024-09-15T22:07:12.729202+0000","last_peered":"2024-09-15T22:07:12.729202+0000","last_clean":"2024-09-15T22:07:12.729202+0000","last_became_active":"2024-09-15T22:06:35.133885+0000","last_became_peered":"2024-09-15T22:06:35.133885+0000","last_unstale":"2024-09-15T22:07:12.729202+0000","last_undegraded":"2024-09-15T22:07:12.729202+0000","last_fullsized":"2024-09-15T22:07:12.729202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:07:35.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850431+0000","last_change":"2024-09-15T22:06:44.664578+0000","last_active":"2024-09-15T22:06:47.850431+0000","last_peered":"2024-09-15T22:06:47.850431+0000","last_clean":"2024-09-15T22:06:47.850431+0000","last_became_active":"2024-09-15T22:06:44.664170+0000","last_became_peered":"2024-09-15T22:06:44.664170+0000","last_unstale":"2024-09-15T22:06:47.850431+0000","last_undegraded":"2024-09-15T22:06:47.850431+0000","last_fullsized":"2024-09-15T22:06:47.850431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:41.443816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645413+0000","last_change":"2024-09-15T22:06:37.186107+0000","last_active":"2024-09-15T22:06:47.645413+0000","last_peered":"2024-09-15T22:06:47.645413+0000","last_clean":"2024-09-15T22:06:47.645413+0000","last_became_active":"2024-09-15T22:06:37.185757+0000","last_became_peered":"2024-09-15T22:06:37.185757+0000","last_unstale":"2024-09-15T22:06:47.645413+0000","last_undegraded":"2024-09-15T22:06:47.645413+0000","last_fullsized":"2024-09-15T22:06:47.645413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:14.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.726066+0000","last_change":"2024-09-15T22:06:35.356281+0000","last_active":"2024-09-15T22:07:12.726066+0000","last_peered":"2024-09-15T22:07:12.726066+0000","last_clean":"2024-09-15T22:07:12.726066+0000","last_became_active":"2024-09-15T22:06:35.355922+0000","last_became_peered":"2024-09-15T22:06:35.355922+0000","last_unstale":"2024-09-15T22:07:12.726066+0000","last_undegraded":"2024-09-15T22:07:12.726066+0000","last_fullsized":"2024-09-15T22:07:12.726066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:06.107462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646959+0000","last_change":"2024-09-15T22:06:44.659230+0000","last_active":"2024-09-15T22:06:47.646959+0000","last_peered":"2024-09-15T22:06:47.646959+0000","last_clean":"2024-09-15T22:06:47.646959+0000","last_became_active":"2024-09-15T22:06:44.658649+0000","last_became_peered":"2024-09-15T22:06:44.658649+0000","last_unstale":"2024-09-15T22:06:47.646959+0000","last_undegraded":"2024-09-15T22:06:47.646959+0000","last_fullsized":"2024-09-15T22:06:47.646959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:14:21.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850818+0000","last_change":"2024-09-15T22:06:44.649394+0000","last_active":"2024-09-15T22:06:47.850818+0000","last_peered":"2024-09-15T22:06:47.850818+0000","last_clean":"2024-09-15T22:06:47.850818+0000","last_became_active":"2024-09-15T22:06:44.649043+0000","last_became_peered":"2024-09-15T22:06:44.649043+0000","last_unstale":"2024-09-15T22:06:47.850818+0000","last_undegraded":"2024-09-15T22:06:47.850818+0000","last_fullsized":"2024-09-15T22:06:47.850818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:56:28.335869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.726312+0000","last_change":"2024-09-15T22:06:35.265970+0000","last_active":"2024-09-15T22:07:12.726312+0000","last_peered":"2024-09-15T22:07:12.726312+0000","last_clean":"2024-09-15T22:07:12.726312+0000","last_became_active":"2024-09-15T22:06:35.265628+0000","last_became_peered":"2024-09-15T22:06:35.265628+0000","last_unstale":"2024-09-15T22:07:12.726312+0000","last_undegraded":"2024-09-15T22:07:12.726312+0000","last_fullsized":"2024-09-15T22:07:12.726312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:41.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849607+0000","last_change":"2024-09-15T22:06:44.649652+0000","last_active":"2024-09-15T22:06:47.849607+0000","last_peered":"2024-09-15T22:06:47.849607+0000","last_clean":"2024-09-15T22:06:47.849607+0000","last_became_active":"2024-09-15T22:06:44.649112+0000","last_became_peered":"2024-09-15T22:06:44.649112+0000","last_unstale":"2024-09-15T22:06:47.849607+0000","last_undegraded":"2024-09-15T22:06:47.849607+0000","last_fullsized":"2024-09-15T22:06:47.849607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:53.789320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645620+0000","last_change":"2024-09-15T22:06:37.190701+0000","last_active":"2024-09-15T22:06:47.645620+0000","last_peered":"2024-09-15T22:06:47.645620+0000","last_clean":"2024-09-15T22:06:47.645620+0000","last_became_active":"2024-09-15T22:06:37.190430+0000","last_became_peered":"2024-09-15T22:06:37.190430+0000","last_unstale":"2024-09-15T22:06:47.645620+0000","last_undegraded":"2024-09-15T22:06:47.645620+0000","last_fullsized":"2024-09-15T22:06:47.645620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:14.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":39,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:22.202815+0000","last_change":"2024-09-15T22:06:35.133899+0000","last_active":"2024-09-15T22:07:22.202815+0000","last_peered":"2024-09-15T22:07:22.202815+0000","last_clean":"2024-09-15T22:07:22.202815+0000","last_became_active":"2024-09-15T22:06:35.133556+0000","last_became_peered":"2024-09-15T22:06:35.133556+0000","last_unstale":"2024-09-15T22:07:22.202815+0000","last_undegraded":"2024-09-15T22:07:22.202815+0000","last_fullsized":"2024-09-15T22:07:22.202815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:01.692416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.644792+0000","last_change":"2024-09-15T22:06:44.655280+0000","last_active":"2024-09-15T22:06:47.644792+0000","last_peered":"2024-09-15T22:06:47.644792+0000","last_clean":"2024-09-15T22:06:47.644792+0000","last_became_active":"2024-09-15T22:06:44.654883+0000","last_became_peered":"2024-09-15T22:06:44.654883+0000","last_unstale":"2024-09-15T22:06:47.644792+0000","last_undegraded":"2024-09-15T22:06:47.644792+0000","last_fullsized":"2024-09-15T22:06:47.644792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.074035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851064+0000","last_change":"2024-09-15T22:06:37.182006+0000","last_active":"2024-09-15T22:06:47.851064+0000","last_peered":"2024-09-15T22:06:47.851064+0000","last_clean":"2024-09-15T22:06:47.851064+0000","last_became_active":"2024-09-15T22:06:37.181693+0000","last_became_peered":"2024-09-15T22:06:37.181693+0000","last_unstale":"2024-09-15T22:06:47.851064+0000","last_undegraded":"2024-09-15T22:06:47.851064+0000","last_fullsized":"2024-09-15T22:06:47.851064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:23.483930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.724002+0000","last_change":"2024-09-15T22:06:35.355854+0000","last_active":"2024-09-15T22:07:12.724002+0000","last_peered":"2024-09-15T22:07:12.724002+0000","last_clean":"2024-09-15T22:07:12.724002+0000","last_became_active":"2024-09-15T22:06:35.355486+0000","last_became_peered":"2024-09-15T22:06:35.355486+0000","last_unstale":"2024-09-15T22:07:12.724002+0000","last_undegraded":"2024-09-15T22:07:12.724002+0000","last_fullsized":"2024-09-15T22:07:12.724002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:36.708476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850529+0000","last_change":"2024-09-15T22:06:44.660523+0000","last_active":"2024-09-15T22:06:47.850529+0000","last_peered":"2024-09-15T22:06:47.850529+0000","last_clean":"2024-09-15T22:06:47.850529+0000","last_became_active":"2024-09-15T22:06:44.660113+0000","last_became_peered":"2024-09-15T22:06:44.660113+0000","last_unstale":"2024-09-15T22:06:47.850529+0000","last_undegraded":"2024-09-15T22:06:47.850529+0000","last_fullsized":"2024-09-15T22:06:47.850529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:42.164558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850182+0000","last_change":"2024-09-15T22:06:37.183649+0000","last_active":"2024-09-15T22:06:47.850182+0000","last_peered":"2024-09-15T22:06:47.850182+0000","last_clean":"2024-09-15T22:06:47.850182+0000","last_became_active":"2024-09-15T22:06:37.183296+0000","last_became_peered":"2024-09-15T22:06:37.183296+0000","last_unstale":"2024-09-15T22:06:47.850182+0000","last_undegraded":"2024-09-15T22:06:47.850182+0000","last_fullsized":"2024-09-15T22:06:47.850182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:22.010288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850466+0000","last_change":"2024-09-15T22:06:35.355068+0000","last_active":"2024-09-15T22:06:47.850466+0000","last_peered":"2024-09-15T22:06:47.850466+0000","last_clean":"2024-09-15T22:06:47.850466+0000","last_became_active":"2024-09-15T22:06:35.354728+0000","last_became_peered":"2024-09-15T22:06:35.354728+0000","last_unstale":"2024-09-15T22:06:47.850466+0000","last_undegraded":"2024-09-15T22:06:47.850466+0000","last_fullsized":"2024-09-15T22:06:47.850466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:35.479290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645591+0000","last_change":"2024-09-15T22:06:44.653849+0000","last_active":"2024-09-15T22:06:47.645591+0000","last_peered":"2024-09-15T22:06:47.645591+0000","last_clean":"2024-09-15T22:06:47.645591+0000","last_became_active":"2024-09-15T22:06:44.653419+0000","last_became_peered":"2024-09-15T22:06:44.653419+0000","last_unstale":"2024-09-15T22:06:47.645591+0000","last_undegraded":"2024-09-15T22:06:47.645591+0000","last_fullsized":"2024-09-15T22:06:47.645591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:04.131553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850416+0000","last_change":"2024-09-15T22:06:37.191438+0000","last_active":"2024-09-15T22:06:47.850416+0000","last_peered":"2024-09-15T22:06:47.850416+0000","last_clean":"2024-09-15T22:06:47.850416+0000","last_became_active":"2024-09-15T22:06:37.191088+0000","last_became_peered":"2024-09-15T22:06:37.191088+0000","last_unstale":"2024-09-15T22:06:47.850416+0000","last_undegraded":"2024-09-15T22:06:47.850416+0000","last_fullsized":"2024-09-15T22:06:47.850416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:46:36.078872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851041+0000","last_change":"2024-09-15T22:06:35.135866+0000","last_active":"2024-09-15T22:06:47.851041+0000","last_peered":"2024-09-15T22:06:47.851041+0000","last_clean":"2024-09-15T22:06:47.851041+0000","last_became_active":"2024-09-15T22:06:35.135524+0000","last_became_peered":"2024-09-15T22:06:35.135524+0000","last_unstale":"2024-09-15T22:06:47.851041+0000","last_undegraded":"2024-09-15T22:06:47.851041+0000","last_fullsized":"2024-09-15T22:06:47.851041+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850057+0000","last_change":"2024-09-15T22:06:44.649943+0000","last_active":"2024-09-15T22:06:47.850057+0000","last_peered":"2024-09-15T22:06:47.850057+0000","last_clean":"2024-09-15T22:06:47.850057+0000","last_became_active":"2024-09-15T22:06:44.649579+0000","last_became_peered":"2024-09-15T22:06:44.649579+0000","last_unstale":"2024-09-15T22:06:47.850057+0000","last_undegraded":"2024-09-15T22:06:47.850057+0000","last_fullsized":"2024-09-15T22:06:47.850057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:18.390608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849866+0000","last_change":"2024-09-15T22:06:37.189988+0000","last_active":"2024-09-15T22:06:47.849866+0000","last_peered":"2024-09-15T22:06:47.849866+0000","last_clean":"2024-09-15T22:06:47.849866+0000","last_became_active":"2024-09-15T22:06:37.189319+0000","last_became_peered":"2024-09-15T22:06:37.189319+0000","last_unstale":"2024-09-15T22:06:47.849866+0000","last_undegraded":"2024-09-15T22:06:47.849866+0000","last_fullsized":"2024-09-15T22:06:47.849866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:08.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:26.756488+0000","last_change":"2024-09-15T22:06:35.355371+0000","last_active":"2024-09-15T22:07:26.756488+0000","last_peered":"2024-09-15T22:07:26.756488+0000","last_clean":"2024-09-15T22:07:26.756488+0000","last_became_active":"2024-09-15T22:06:35.354687+0000","last_became_peered":"2024-09-15T22:06:35.354687+0000","last_unstale":"2024-09-15T22:07:26.756488+0000","last_undegraded":"2024-09-15T22:07:26.756488+0000","last_fullsized":"2024-09-15T22:07:26.756488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:27.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851369+0000","last_change":"2024-09-15T22:06:44.657929+0000","last_active":"2024-09-15T22:06:47.851369+0000","last_peered":"2024-09-15T22:06:47.851369+0000","last_clean":"2024-09-15T22:06:47.851369+0000","last_became_active":"2024-09-15T22:06:44.657519+0000","last_became_peered":"2024-09-15T22:06:44.657519+0000","last_unstale":"2024-09-15T22:06:47.851369+0000","last_undegraded":"2024-09-15T22:06:47.851369+0000","last_fullsized":"2024-09-15T22:06:47.851369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:12.313877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849506+0000","last_change":"2024-09-15T22:06:37.193288+0000","last_active":"2024-09-15T22:06:47.849506+0000","last_peered":"2024-09-15T22:06:47.849506+0000","last_clean":"2024-09-15T22:06:47.849506+0000","last_became_active":"2024-09-15T22:06:37.193007+0000","last_became_peered":"2024-09-15T22:06:37.193007+0000","last_unstale":"2024-09-15T22:06:47.849506+0000","last_undegraded":"2024-09-15T22:06:47.849506+0000","last_fullsized":"2024-09-15T22:06:47.849506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:45.956206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.734429+0000","last_change":"2024-09-15T22:06:35.138726+0000","last_active":"2024-09-15T22:07:13.734429+0000","last_peered":"2024-09-15T22:07:13.734429+0000","last_clean":"2024-09-15T22:07:13.734429+0000","last_became_active":"2024-09-15T22:06:35.138390+0000","last_became_peered":"2024-09-15T22:06:35.138390+0000","last_unstale":"2024-09-15T22:07:13.734429+0000","last_undegraded":"2024-09-15T22:07:13.734429+0000","last_fullsized":"2024-09-15T22:07:13.734429+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:51.336171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851139+0000","last_change":"2024-09-15T22:06:44.656811+0000","last_active":"2024-09-15T22:06:47.851139+0000","last_peered":"2024-09-15T22:06:47.851139+0000","last_clean":"2024-09-15T22:06:47.851139+0000","last_became_active":"2024-09-15T22:06:44.656381+0000","last_became_peered":"2024-09-15T22:06:44.656381+0000","last_unstale":"2024-09-15T22:06:47.851139+0000","last_undegraded":"2024-09-15T22:06:47.851139+0000","last_fullsized":"2024-09-15T22:06:47.851139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:15.219637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645187+0000","last_change":"2024-09-15T22:06:37.176478+0000","last_active":"2024-09-15T22:06:47.645187+0000","last_peered":"2024-09-15T22:06:47.645187+0000","last_clean":"2024-09-15T22:06:47.645187+0000","last_became_active":"2024-09-15T22:06:37.176171+0000","last_became_peered":"2024-09-15T22:06:37.176171+0000","last_unstale":"2024-09-15T22:06:47.645187+0000","last_undegraded":"2024-09-15T22:06:47.645187+0000","last_fullsized":"2024-09-15T22:06:47.645187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:14.735811+0000","last_change":"2024-09-15T22:06:35.134651+0000","last_active":"2024-09-15T22:07:14.735811+0000","last_peered":"2024-09-15T22:07:14.735811+0000","last_clean":"2024-09-15T22:07:14.735811+0000","last_became_active":"2024-09-15T22:06:35.134185+0000","last_became_peered":"2024-09-15T22:06:35.134185+0000","last_unstale":"2024-09-15T22:07:14.735811+0000","last_undegraded":"2024-09-15T22:07:14.735811+0000","last_fullsized":"2024-09-15T22:07:14.735811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:56.385046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851362+0000","last_change":"2024-09-15T22:05:23.048068+0000","last_active":"2024-09-15T22:06:47.851362+0000","last_peered":"2024-09-15T22:06:47.851362+0000","last_clean":"2024-09-15T22:06:47.851362+0000","last_became_active":"2024-09-15T22:05:23.047692+0000","last_became_peered":"2024-09-15T22:05:23.047692+0000","last_unstale":"2024-09-15T22:06:47.851362+0000","last_undegraded":"2024-09-15T22:06:47.851362+0000","last_fullsized":"2024-09-15T22:06:47.851362+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_clean_scrub_stamp":"2024-09-15T22:01:57.477629+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850994+0000","last_change":"2024-09-15T22:06:44.657213+0000","last_active":"2024-09-15T22:06:47.850994+0000","last_peered":"2024-09-15T22:06:47.850994+0000","last_clean":"2024-09-15T22:06:47.850994+0000","last_became_active":"2024-09-15T22:06:44.656815+0000","last_became_peered":"2024-09-15T22:06:44.656815+0000","last_unstale":"2024-09-15T22:06:47.850994+0000","last_undegraded":"2024-09-15T22:06:47.850994+0000","last_fullsized":"2024-09-15T22:06:47.850994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:26.525992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851058+0000","last_change":"2024-09-15T22:06:37.184467+0000","last_active":"2024-09-15T22:06:47.851058+0000","last_peered":"2024-09-15T22:06:47.851058+0000","last_clean":"2024-09-15T22:06:47.851058+0000","last_became_active":"2024-09-15T22:06:37.183864+0000","last_became_peered":"2024-09-15T22:06:37.183864+0000","last_unstale":"2024-09-15T22:06:47.851058+0000","last_undegraded":"2024-09-15T22:06:47.851058+0000","last_fullsized":"2024-09-15T22:06:47.851058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:35.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.737313+0000","last_change":"2024-09-15T22:06:44.651594+0000","last_active":"2024-09-15T22:07:13.737313+0000","last_peered":"2024-09-15T22:07:13.737313+0000","last_clean":"2024-09-15T22:07:13.737313+0000","last_became_active":"2024-09-15T22:06:44.650393+0000","last_became_peered":"2024-09-15T22:06:44.650393+0000","last_unstale":"2024-09-15T22:07:13.737313+0000","last_undegraded":"2024-09-15T22:07:13.737313+0000","last_fullsized":"2024-09-15T22:07:13.737313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:13.599744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851420+0000","last_change":"2024-09-15T22:06:44.660789+0000","last_active":"2024-09-15T22:06:47.851420+0000","last_peered":"2024-09-15T22:06:47.851420+0000","last_clean":"2024-09-15T22:06:47.851420+0000","last_became_active":"2024-09-15T22:06:44.660331+0000","last_became_peered":"2024-09-15T22:06:44.660331+0000","last_unstale":"2024-09-15T22:06:47.851420+0000","last_undegraded":"2024-09-15T22:06:47.851420+0000","last_fullsized":"2024-09-15T22:06:47.851420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:04:45.562795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850373+0000","last_change":"2024-09-15T22:06:37.185697+0000","last_active":"2024-09-15T22:06:47.850373+0000","last_peered":"2024-09-15T22:06:47.850373+0000","last_clean":"2024-09-15T22:06:47.850373+0000","last_became_active":"2024-09-15T22:06:37.185207+0000","last_became_peered":"2024-09-15T22:06:37.185207+0000","last_unstale":"2024-09-15T22:06:47.850373+0000","last_undegraded":"2024-09-15T22:06:47.850373+0000","last_fullsized":"2024-09-15T22:06:47.850373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:39.005273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851200+0000","last_change":"2024-09-15T22:06:35.135176+0000","last_active":"2024-09-15T22:06:47.851200+0000","last_peered":"2024-09-15T22:06:47.851200+0000","last_clean":"2024-09-15T22:06:47.851200+0000","last_became_active":"2024-09-15T22:06:35.134649+0000","last_became_peered":"2024-09-15T22:06:35.134649+0000","last_unstale":"2024-09-15T22:06:47.851200+0000","last_undegraded":"2024-09-15T22:06:47.851200+0000","last_fullsized":"2024-09-15T22:06:47.851200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:33.780162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850486+0000","last_change":"2024-09-15T22:06:44.656972+0000","last_active":"2024-09-15T22:06:47.850486+0000","last_peered":"2024-09-15T22:06:47.850486+0000","last_clean":"2024-09-15T22:06:47.850486+0000","last_became_active":"2024-09-15T22:06:44.656528+0000","last_became_peered":"2024-09-15T22:06:44.656528+0000","last_unstale":"2024-09-15T22:06:47.850486+0000","last_undegraded":"2024-09-15T22:06:47.850486+0000","last_fullsized":"2024-09-15T22:06:47.850486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:09.164596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850793+0000","last_change":"2024-09-15T22:06:37.186118+0000","last_active":"2024-09-15T22:06:47.850793+0000","last_peered":"2024-09-15T22:06:47.850793+0000","last_clean":"2024-09-15T22:06:47.850793+0000","last_became_active":"2024-09-15T22:06:37.185118+0000","last_became_peered":"2024-09-15T22:06:37.185118+0000","last_unstale":"2024-09-15T22:06:47.850793+0000","last_undegraded":"2024-09-15T22:06:47.850793+0000","last_fullsized":"2024-09-15T22:06:47.850793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:53.357045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851347+0000","last_change":"2024-09-15T22:06:35.133341+0000","last_active":"2024-09-15T22:06:47.851347+0000","last_peered":"2024-09-15T22:06:47.851347+0000","last_clean":"2024-09-15T22:06:47.851347+0000","last_became_active":"2024-09-15T22:06:35.132961+0000","last_became_peered":"2024-09-15T22:06:35.132961+0000","last_unstale":"2024-09-15T22:06:47.851347+0000","last_undegraded":"2024-09-15T22:06:47.851347+0000","last_fullsized":"2024-09-15T22:06:47.851347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:39.579470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849894+0000","last_change":"2024-09-15T22:06:44.659148+0000","last_active":"2024-09-15T22:06:47.849894+0000","last_peered":"2024-09-15T22:06:47.849894+0000","last_clean":"2024-09-15T22:06:47.849894+0000","last_became_active":"2024-09-15T22:06:44.658750+0000","last_became_peered":"2024-09-15T22:06:44.658750+0000","last_unstale":"2024-09-15T22:06:47.849894+0000","last_undegraded":"2024-09-15T22:06:47.849894+0000","last_fullsized":"2024-09-15T22:06:47.849894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:48.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850697+0000","last_change":"2024-09-15T22:06:37.194066+0000","last_active":"2024-09-15T22:06:47.850697+0000","last_peered":"2024-09-15T22:06:47.850697+0000","last_clean":"2024-09-15T22:06:47.850697+0000","last_became_active":"2024-09-15T22:06:37.193738+0000","last_became_peered":"2024-09-15T22:06:37.193738+0000","last_unstale":"2024-09-15T22:06:47.850697+0000","last_undegraded":"2024-09-15T22:06:47.850697+0000","last_fullsized":"2024-09-15T22:06:47.850697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.107188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:14.731656+0000","last_change":"2024-09-15T22:06:35.136470+0000","last_active":"2024-09-15T22:07:14.731656+0000","last_peered":"2024-09-15T22:07:14.731656+0000","last_clean":"2024-09-15T22:07:14.731656+0000","last_became_active":"2024-09-15T22:06:35.135953+0000","last_became_peered":"2024-09-15T22:06:35.135953+0000","last_unstale":"2024-09-15T22:07:14.731656+0000","last_undegraded":"2024-09-15T22:07:14.731656+0000","last_fullsized":"2024-09-15T22:07:14.731656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:44.548566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850399+0000","last_change":"2024-09-15T22:06:44.653682+0000","last_active":"2024-09-15T22:06:47.850399+0000","last_peered":"2024-09-15T22:06:47.850399+0000","last_clean":"2024-09-15T22:06:47.850399+0000","last_became_active":"2024-09-15T22:06:44.653303+0000","last_became_peered":"2024-09-15T22:06:44.653303+0000","last_unstale":"2024-09-15T22:06:47.850399+0000","last_undegraded":"2024-09-15T22:06:47.850399+0000","last_fullsized":"2024-09-15T22:06:47.850399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:07.847971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850520+0000","last_change":"2024-09-15T22:06:44.658436+0000","last_active":"2024-09-15T22:06:47.850520+0000","last_peered":"2024-09-15T22:06:47.850520+0000","last_clean":"2024-09-15T22:06:47.850520+0000","last_became_active":"2024-09-15T22:06:44.658037+0000","last_became_peered":"2024-09-15T22:06:44.658037+0000","last_unstale":"2024-09-15T22:06:47.850520+0000","last_undegraded":"2024-09-15T22:06:47.850520+0000","last_fullsized":"2024-09-15T22:06:47.850520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:34:20.369249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.735911+0000","last_change":"2024-09-15T22:06:35.358018+0000","last_active":"2024-09-15T22:07:13.735911+0000","last_peered":"2024-09-15T22:07:13.735911+0000","last_clean":"2024-09-15T22:07:13.735911+0000","last_became_active":"2024-09-15T22:06:35.357666+0000","last_became_peered":"2024-09-15T22:06:35.357666+0000","last_unstale":"2024-09-15T22:07:13.735911+0000","last_undegraded":"2024-09-15T22:07:13.735911+0000","last_fullsized":"2024-09-15T22:07:13.735911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:34.089113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851395+0000","last_change":"2024-09-15T22:06:44.656807+0000","last_active":"2024-09-15T22:06:47.851395+0000","last_peered":"2024-09-15T22:06:47.851395+0000","last_clean":"2024-09-15T22:06:47.851395+0000","last_became_active":"2024-09-15T22:06:44.656409+0000","last_became_peered":"2024-09-15T22:06:44.656409+0000","last_unstale":"2024-09-15T22:06:47.851395+0000","last_undegraded":"2024-09-15T22:06:47.851395+0000","last_fullsized":"2024-09-15T22:06:47.851395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850035+0000","last_change":"2024-09-15T22:06:37.180615+0000","last_active":"2024-09-15T22:06:47.850035+0000","last_peered":"2024-09-15T22:06:47.850035+0000","last_clean":"2024-09-15T22:06:47.850035+0000","last_became_active":"2024-09-15T22:06:37.180306+0000","last_became_peered":"2024-09-15T22:06:37.180306+0000","last_unstale":"2024-09-15T22:06:47.850035+0000","last_undegraded":"2024-09-15T22:06:47.850035+0000","last_fullsized":"2024-09-15T22:06:47.850035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:31.145128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:26.756977+0000","last_change":"2024-09-15T22:06:35.136192+0000","last_active":"2024-09-15T22:07:26.756977+0000","last_peered":"2024-09-15T22:07:26.756977+0000","last_clean":"2024-09-15T22:07:26.756977+0000","last_became_active":"2024-09-15T22:06:35.135608+0000","last_became_peered":"2024-09-15T22:06:35.135608+0000","last_unstale":"2024-09-15T22:07:26.756977+0000","last_undegraded":"2024-09-15T22:07:26.756977+0000","last_fullsized":"2024-09-15T22:07:26.756977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:48.472717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850695+0000","last_change":"2024-09-15T22:06:44.664763+0000","last_active":"2024-09-15T22:06:47.850695+0000","last_peered":"2024-09-15T22:06:47.850695+0000","last_clean":"2024-09-15T22:06:47.850695+0000","last_became_active":"2024-09-15T22:06:44.664410+0000","last_became_peered":"2024-09-15T22:06:44.664410+0000","last_unstale":"2024-09-15T22:06:47.850695+0000","last_undegraded":"2024-09-15T22:06:47.850695+0000","last_fullsized":"2024-09-15T22:06:47.850695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:14.364387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851090+0000","last_change":"2024-09-15T22:06:44.652747+0000","last_active":"2024-09-15T22:06:47.851090+0000","last_peered":"2024-09-15T22:06:47.851090+0000","last_clean":"2024-09-15T22:06:47.851090+0000","last_became_active":"2024-09-15T22:06:44.652368+0000","last_became_peered":"2024-09-15T22:06:44.652368+0000","last_unstale":"2024-09-15T22:06:47.851090+0000","last_undegraded":"2024-09-15T22:06:47.851090+0000","last_fullsized":"2024-09-15T22:06:47.851090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:51.047549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.729356+0000","last_change":"2024-09-15T22:06:35.136311+0000","last_active":"2024-09-15T22:07:12.729356+0000","last_peered":"2024-09-15T22:07:12.729356+0000","last_clean":"2024-09-15T22:07:12.729356+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:07:12.729356+0000","last_undegraded":"2024-09-15T22:07:12.729356+0000","last_fullsized":"2024-09-15T22:07:12.729356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:36:07.138056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850773+0000","last_change":"2024-09-15T22:06:44.653747+0000","last_active":"2024-09-15T22:06:47.850773+0000","last_peered":"2024-09-15T22:06:47.850773+0000","last_clean":"2024-09-15T22:06:47.850773+0000","last_became_active":"2024-09-15T22:06:44.653357+0000","last_became_peered":"2024-09-15T22:06:44.653357+0000","last_unstale":"2024-09-15T22:06:47.850773+0000","last_undegraded":"2024-09-15T22:06:47.850773+0000","last_fullsized":"2024-09-15T22:06:47.850773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.113924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850070+0000","last_change":"2024-09-15T22:06:37.178308+0000","last_active":"2024-09-15T22:06:47.850070+0000","last_peered":"2024-09-15T22:06:47.850070+0000","last_clean":"2024-09-15T22:06:47.850070+0000","last_became_active":"2024-09-15T22:06:37.177920+0000","last_became_peered":"2024-09-15T22:06:37.177920+0000","last_unstale":"2024-09-15T22:06:47.850070+0000","last_undegraded":"2024-09-15T22:06:47.850070+0000","last_fullsized":"2024-09-15T22:06:47.850070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.735092+0000","last_change":"2024-09-15T22:06:35.136132+0000","last_active":"2024-09-15T22:07:13.735092+0000","last_peered":"2024-09-15T22:07:13.735092+0000","last_clean":"2024-09-15T22:07:13.735092+0000","last_became_active":"2024-09-15T22:06:35.135789+0000","last_became_peered":"2024-09-15T22:06:35.135789+0000","last_unstale":"2024-09-15T22:07:13.735092+0000","last_undegraded":"2024-09-15T22:07:13.735092+0000","last_fullsized":"2024-09-15T22:07:13.735092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:01.692885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850765+0000","last_change":"2024-09-15T22:06:44.658920+0000","last_active":"2024-09-15T22:06:47.850765+0000","last_peered":"2024-09-15T22:06:47.850765+0000","last_clean":"2024-09-15T22:06:47.850765+0000","last_became_active":"2024-09-15T22:06:44.658434+0000","last_became_peered":"2024-09-15T22:06:44.658434+0000","last_unstale":"2024-09-15T22:06:47.850765+0000","last_undegraded":"2024-09-15T22:06:47.850765+0000","last_fullsized":"2024-09-15T22:06:47.850765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:37.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645847+0000","last_change":"2024-09-15T22:06:37.184958+0000","last_active":"2024-09-15T22:06:47.645847+0000","last_peered":"2024-09-15T22:06:47.645847+0000","last_clean":"2024-09-15T22:06:47.645847+0000","last_became_active":"2024-09-15T22:06:37.184648+0000","last_became_peered":"2024-09-15T22:06:37.184648+0000","last_unstale":"2024-09-15T22:06:47.645847+0000","last_undegraded":"2024-09-15T22:06:47.645847+0000","last_fullsized":"2024-09-15T22:06:47.645847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:28.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'26","reported_seq":84,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:24.755589+0000","last_change":"2024-09-15T22:06:35.138533+0000","last_active":"2024-09-15T22:07:24.755589+0000","last_peered":"2024-09-15T22:07:24.755589+0000","last_clean":"2024-09-15T22:07:24.755589+0000","last_became_active":"2024-09-15T22:06:35.138142+0000","last_became_peered":"2024-09-15T22:06:35.138142+0000","last_unstale":"2024-09-15T22:07:24.755589+0000","last_undegraded":"2024-09-15T22:07:24.755589+0000","last_fullsized":"2024-09-15T22:07:24.755589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:32.950036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":56,"num_write":27,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850060+0000","last_change":"2024-09-15T22:06:44.659773+0000","last_active":"2024-09-15T22:06:47.850060+0000","last_peered":"2024-09-15T22:06:47.850060+0000","last_clean":"2024-09-15T22:06:47.850060+0000","last_became_active":"2024-09-15T22:06:44.659374+0000","last_became_peered":"2024-09-15T22:06:44.659374+0000","last_unstale":"2024-09-15T22:06:47.850060+0000","last_undegraded":"2024-09-15T22:06:47.850060+0000","last_fullsized":"2024-09-15T22:06:47.850060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:13.491116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851697+0000","last_change":"2024-09-15T22:06:37.194297+0000","last_active":"2024-09-15T22:06:47.851697+0000","last_peered":"2024-09-15T22:06:47.851697+0000","last_clean":"2024-09-15T22:06:47.851697+0000","last_became_active":"2024-09-15T22:06:37.193986+0000","last_became_peered":"2024-09-15T22:06:37.193986+0000","last_unstale":"2024-09-15T22:06:47.851697+0000","last_undegraded":"2024-09-15T22:06:47.851697+0000","last_fullsized":"2024-09-15T22:06:47.851697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:27.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.728943+0000","last_change":"2024-09-15T22:06:35.356537+0000","last_active":"2024-09-15T22:07:12.728943+0000","last_peered":"2024-09-15T22:07:12.728943+0000","last_clean":"2024-09-15T22:07:12.728943+0000","last_became_active":"2024-09-15T22:06:35.355997+0000","last_became_peered":"2024-09-15T22:06:35.355997+0000","last_unstale":"2024-09-15T22:07:12.728943+0000","last_undegraded":"2024-09-15T22:07:12.728943+0000","last_fullsized":"2024-09-15T22:07:12.728943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:11.563083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646787+0000","last_change":"2024-09-15T22:06:44.654579+0000","last_active":"2024-09-15T22:06:47.646787+0000","last_peered":"2024-09-15T22:06:47.646787+0000","last_clean":"2024-09-15T22:06:47.646787+0000","last_became_active":"2024-09-15T22:06:44.654187+0000","last_became_peered":"2024-09-15T22:06:44.654187+0000","last_unstale":"2024-09-15T22:06:47.646787+0000","last_undegraded":"2024-09-15T22:06:47.646787+0000","last_fullsized":"2024-09-15T22:06:47.646787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:15.617528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851150+0000","last_change":"2024-09-15T22:06:44.661533+0000","last_active":"2024-09-15T22:06:47.851150+0000","last_peered":"2024-09-15T22:06:47.851150+0000","last_clean":"2024-09-15T22:06:47.851150+0000","last_became_active":"2024-09-15T22:06:44.661193+0000","last_became_peered":"2024-09-15T22:06:44.661193+0000","last_unstale":"2024-09-15T22:06:47.851150+0000","last_undegraded":"2024-09-15T22:06:47.851150+0000","last_fullsized":"2024-09-15T22:06:47.851150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:06.559720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.727278+0000","last_change":"2024-09-15T22:06:35.140578+0000","last_active":"2024-09-15T22:07:12.727278+0000","last_peered":"2024-09-15T22:07:12.727278+0000","last_clean":"2024-09-15T22:07:12.727278+0000","last_became_active":"2024-09-15T22:06:35.139818+0000","last_became_peered":"2024-09-15T22:06:35.139818+0000","last_unstale":"2024-09-15T22:07:12.727278+0000","last_undegraded":"2024-09-15T22:07:12.727278+0000","last_fullsized":"2024-09-15T22:07:12.727278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:59.806964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850716+0000","last_change":"2024-09-15T22:06:44.654755+0000","last_active":"2024-09-15T22:06:47.850716+0000","last_peered":"2024-09-15T22:06:47.850716+0000","last_clean":"2024-09-15T22:06:47.850716+0000","last_became_active":"2024-09-15T22:06:44.654390+0000","last_became_peered":"2024-09-15T22:06:44.654390+0000","last_unstale":"2024-09-15T22:06:47.850716+0000","last_undegraded":"2024-09-15T22:06:47.850716+0000","last_fullsized":"2024-09-15T22:06:47.850716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:35.781878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851689+0000","last_change":"2024-09-15T22:06:37.182457+0000","last_active":"2024-09-15T22:06:47.851689+0000","last_peered":"2024-09-15T22:06:47.851689+0000","last_clean":"2024-09-15T22:06:47.851689+0000","last_became_active":"2024-09-15T22:06:37.182106+0000","last_became_peered":"2024-09-15T22:06:37.182106+0000","last_unstale":"2024-09-15T22:06:47.851689+0000","last_undegraded":"2024-09-15T22:06:47.851689+0000","last_fullsized":"2024-09-15T22:06:47.851689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:57.560894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":44,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:23.752831+0000","last_change":"2024-09-15T22:06:35.140590+0000","last_active":"2024-09-15T22:07:23.752831+0000","last_peered":"2024-09-15T22:07:23.752831+0000","last_clean":"2024-09-15T22:07:23.752831+0000","last_became_active":"2024-09-15T22:06:35.140198+0000","last_became_peered":"2024-09-15T22:06:35.140198+0000","last_unstale":"2024-09-15T22:07:23.752831+0000","last_undegraded":"2024-09-15T22:07:23.752831+0000","last_fullsized":"2024-09-15T22:07:23.752831+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:46.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.644921+0000","last_change":"2024-09-15T22:06:44.660188+0000","last_active":"2024-09-15T22:06:47.644921+0000","last_peered":"2024-09-15T22:06:47.644921+0000","last_clean":"2024-09-15T22:06:47.644921+0000","last_became_active":"2024-09-15T22:06:44.659808+0000","last_became_peered":"2024-09-15T22:06:44.659808+0000","last_unstale":"2024-09-15T22:06:47.644921+0000","last_undegraded":"2024-09-15T22:06:47.644921+0000","last_fullsized":"2024-09-15T22:06:47.644921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:39.473765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850703+0000","last_change":"2024-09-15T22:06:44.647967+0000","last_active":"2024-09-15T22:06:47.850703+0000","last_peered":"2024-09-15T22:06:47.850703+0000","last_clean":"2024-09-15T22:06:47.850703+0000","last_became_active":"2024-09-15T22:06:44.647608+0000","last_became_peered":"2024-09-15T22:06:44.647608+0000","last_unstale":"2024-09-15T22:06:47.850703+0000","last_undegraded":"2024-09-15T22:06:47.850703+0000","last_fullsized":"2024-09-15T22:06:47.850703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:06.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850435+0000","last_change":"2024-09-15T22:06:37.187020+0000","last_active":"2024-09-15T22:06:47.850435+0000","last_peered":"2024-09-15T22:06:47.850435+0000","last_clean":"2024-09-15T22:06:47.850435+0000","last_became_active":"2024-09-15T22:06:37.186564+0000","last_became_peered":"2024-09-15T22:06:37.186564+0000","last_unstale":"2024-09-15T22:06:47.850435+0000","last_undegraded":"2024-09-15T22:06:47.850435+0000","last_fullsized":"2024-09-15T22:06:47.850435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:20.586763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850053+0000","last_change":"2024-09-15T22:06:35.135126+0000","last_active":"2024-09-15T22:06:47.850053+0000","last_peered":"2024-09-15T22:06:47.850053+0000","last_clean":"2024-09-15T22:06:47.850053+0000","last_became_active":"2024-09-15T22:06:35.134783+0000","last_became_peered":"2024-09-15T22:06:35.134783+0000","last_unstale":"2024-09-15T22:06:47.850053+0000","last_undegraded":"2024-09-15T22:06:47.850053+0000","last_fullsized":"2024-09-15T22:06:47.850053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:31.910988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850222+0000","last_change":"2024-09-15T22:06:44.656981+0000","last_active":"2024-09-15T22:06:47.850222+0000","last_peered":"2024-09-15T22:06:47.850222+0000","last_clean":"2024-09-15T22:06:47.850222+0000","last_became_active":"2024-09-15T22:06:44.656477+0000","last_became_peered":"2024-09-15T22:06:44.656477+0000","last_unstale":"2024-09-15T22:06:47.850222+0000","last_undegraded":"2024-09-15T22:06:47.850222+0000","last_fullsized":"2024-09-15T22:06:47.850222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:06.153490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.644921+0000","last_change":"2024-09-15T22:06:37.183034+0000","last_active":"2024-09-15T22:06:47.644921+0000","last_peered":"2024-09-15T22:06:47.644921+0000","last_clean":"2024-09-15T22:06:47.644921+0000","last_became_active":"2024-09-15T22:06:37.182734+0000","last_became_peered":"2024-09-15T22:06:37.182734+0000","last_unstale":"2024-09-15T22:06:47.644921+0000","last_undegraded":"2024-09-15T22:06:47.644921+0000","last_fullsized":"2024-09-15T22:06:47.644921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:19.352000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.736235+0000","last_change":"2024-09-15T22:06:35.134954+0000","last_active":"2024-09-15T22:07:13.736235+0000","last_peered":"2024-09-15T22:07:13.736235+0000","last_clean":"2024-09-15T22:07:13.736235+0000","last_became_active":"2024-09-15T22:06:35.134583+0000","last_became_peered":"2024-09-15T22:06:35.134583+0000","last_unstale":"2024-09-15T22:07:13.736235+0000","last_undegraded":"2024-09-15T22:07:13.736235+0000","last_fullsized":"2024-09-15T22:07:13.736235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:56.082123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.735087+0000","last_change":"2024-09-15T22:06:35.139229+0000","last_active":"2024-09-15T22:07:13.735087+0000","last_peered":"2024-09-15T22:07:13.735087+0000","last_clean":"2024-09-15T22:07:13.735087+0000","last_became_active":"2024-09-15T22:06:35.138834+0000","last_became_peered":"2024-09-15T22:06:35.138834+0000","last_unstale":"2024-09-15T22:07:13.735087+0000","last_undegraded":"2024-09-15T22:07:13.735087+0000","last_fullsized":"2024-09-15T22:07:13.735087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:05.877404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850737+0000","last_change":"2024-09-15T22:06:37.192212+0000","last_active":"2024-09-15T22:06:47.850737+0000","last_peered":"2024-09-15T22:06:47.850737+0000","last_clean":"2024-09-15T22:06:47.850737+0000","last_became_active":"2024-09-15T22:06:37.191872+0000","last_became_peered":"2024-09-15T22:06:37.191872+0000","last_unstale":"2024-09-15T22:06:47.850737+0000","last_undegraded":"2024-09-15T22:06:47.850737+0000","last_fullsized":"2024-09-15T22:06:47.850737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:32.845794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849819+0000","last_change":"2024-09-15T22:06:44.661976+0000","last_active":"2024-09-15T22:06:47.849819+0000","last_peered":"2024-09-15T22:06:47.849819+0000","last_clean":"2024-09-15T22:06:47.849819+0000","last_became_active":"2024-09-15T22:06:44.661169+0000","last_became_peered":"2024-09-15T22:06:44.661169+0000","last_unstale":"2024-09-15T22:06:47.849819+0000","last_undegraded":"2024-09-15T22:06:47.849819+0000","last_fullsized":"2024-09-15T22:06:47.849819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:03.930064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:25.755869+0000","last_change":"2024-09-15T22:06:35.133846+0000","last_active":"2024-09-15T22:07:25.755869+0000","last_peered":"2024-09-15T22:07:25.755869+0000","last_clean":"2024-09-15T22:07:25.755869+0000","last_became_active":"2024-09-15T22:06:35.133516+0000","last_became_peered":"2024-09-15T22:06:35.133516+0000","last_unstale":"2024-09-15T22:07:25.755869+0000","last_undegraded":"2024-09-15T22:07:25.755869+0000","last_fullsized":"2024-09-15T22:07:25.755869+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:29.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645631+0000","last_change":"2024-09-15T22:06:37.182904+0000","last_active":"2024-09-15T22:06:47.645631+0000","last_peered":"2024-09-15T22:06:47.645631+0000","last_clean":"2024-09-15T22:06:47.645631+0000","last_became_active":"2024-09-15T22:06:37.182563+0000","last_became_peered":"2024-09-15T22:06:37.182563+0000","last_unstale":"2024-09-15T22:06:47.645631+0000","last_undegraded":"2024-09-15T22:06:47.645631+0000","last_fullsized":"2024-09-15T22:06:47.645631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850024+0000","last_change":"2024-09-15T22:06:44.660997+0000","last_active":"2024-09-15T22:06:47.850024+0000","last_peered":"2024-09-15T22:06:47.850024+0000","last_clean":"2024-09-15T22:06:47.850024+0000","last_became_active":"2024-09-15T22:06:44.660577+0000","last_became_peered":"2024-09-15T22:06:44.660577+0000","last_unstale":"2024-09-15T22:06:47.850024+0000","last_undegraded":"2024-09-15T22:06:47.850024+0000","last_fullsized":"2024-09-15T22:06:47.850024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:56.006891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850176+0000","last_change":"2024-09-15T22:06:37.182077+0000","last_active":"2024-09-15T22:06:47.850176+0000","last_peered":"2024-09-15T22:06:47.850176+0000","last_clean":"2024-09-15T22:06:47.850176+0000","last_became_active":"2024-09-15T22:06:37.181622+0000","last_became_peered":"2024-09-15T22:06:37.181622+0000","last_unstale":"2024-09-15T22:06:47.850176+0000","last_undegraded":"2024-09-15T22:06:47.850176+0000","last_fullsized":"2024-09-15T22:06:47.850176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:07.586406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849694+0000","last_change":"2024-09-15T22:06:44.653845+0000","last_active":"2024-09-15T22:06:47.849694+0000","last_peered":"2024-09-15T22:06:47.849694+0000","last_clean":"2024-09-15T22:06:47.849694+0000","last_became_active":"2024-09-15T22:06:44.653483+0000","last_became_peered":"2024-09-15T22:06:44.653483+0000","last_unstale":"2024-09-15T22:06:47.849694+0000","last_undegraded":"2024-09-15T22:06:47.849694+0000","last_fullsized":"2024-09-15T22:06:47.849694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:08:07.822279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849449+0000","last_change":"2024-09-15T22:06:37.187476+0000","last_active":"2024-09-15T22:06:47.849449+0000","last_peered":"2024-09-15T22:06:47.849449+0000","last_clean":"2024-09-15T22:06:47.849449+0000","last_became_active":"2024-09-15T22:06:37.187047+0000","last_became_peered":"2024-09-15T22:06:37.187047+0000","last_unstale":"2024-09-15T22:06:47.849449+0000","last_undegraded":"2024-09-15T22:06:47.849449+0000","last_fullsized":"2024-09-15T22:06:47.849449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.220834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850466+0000","last_change":"2024-09-15T22:06:37.190583+0000","last_active":"2024-09-15T22:06:47.850466+0000","last_peered":"2024-09-15T22:06:47.850466+0000","last_clean":"2024-09-15T22:06:47.850466+0000","last_became_active":"2024-09-15T22:06:37.190212+0000","last_became_peered":"2024-09-15T22:06:37.190212+0000","last_unstale":"2024-09-15T22:06:47.850466+0000","last_undegraded":"2024-09-15T22:06:47.850466+0000","last_fullsized":"2024-09-15T22:06:47.850466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:48.574188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850185+0000","last_change":"2024-09-15T22:06:37.192508+0000","last_active":"2024-09-15T22:06:47.850185+0000","last_peered":"2024-09-15T22:06:47.850185+0000","last_clean":"2024-09-15T22:06:47.850185+0000","last_became_active":"2024-09-15T22:06:37.192117+0000","last_became_peered":"2024-09-15T22:06:37.192117+0000","last_unstale":"2024-09-15T22:06:47.850185+0000","last_undegraded":"2024-09-15T22:06:47.850185+0000","last_fullsized":"2024-09-15T22:06:47.850185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:41.355966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850817+0000","last_change":"2024-09-15T22:06:37.183377+0000","last_active":"2024-09-15T22:06:47.850817+0000","last_peered":"2024-09-15T22:06:47.850817+0000","last_clean":"2024-09-15T22:06:47.850817+0000","last_became_active":"2024-09-15T22:06:37.183072+0000","last_became_peered":"2024-09-15T22:06:37.183072+0000","last_unstale":"2024-09-15T22:06:47.850817+0000","last_undegraded":"2024-09-15T22:06:47.850817+0000","last_fullsized":"2024-09-15T22:06:47.850817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.318702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850033+0000","last_change":"2024-09-15T22:06:37.185348+0000","last_active":"2024-09-15T22:06:47.850033+0000","last_peered":"2024-09-15T22:06:47.850033+0000","last_clean":"2024-09-15T22:06:47.850033+0000","last_became_active":"2024-09-15T22:06:37.184934+0000","last_became_peered":"2024-09-15T22:06:37.184934+0000","last_unstale":"2024-09-15T22:06:47.850033+0000","last_undegraded":"2024-09-15T22:06:47.850033+0000","last_fullsized":"2024-09-15T22:06:47.850033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:57.564860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645548+0000","last_change":"2024-09-15T22:06:37.192152+0000","last_active":"2024-09-15T22:06:47.645548+0000","last_peered":"2024-09-15T22:06:47.645548+0000","last_clean":"2024-09-15T22:06:47.645548+0000","last_became_active":"2024-09-15T22:06:37.191836+0000","last_became_peered":"2024-09-15T22:06:37.191836+0000","last_unstale":"2024-09-15T22:06:47.645548+0000","last_undegraded":"2024-09-15T22:06:47.645548+0000","last_fullsized":"2024-09-15T22:06:47.645548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:31.781110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849758+0000","last_change":"2024-09-15T22:06:37.179996+0000","last_active":"2024-09-15T22:06:47.849758+0000","last_peered":"2024-09-15T22:06:47.849758+0000","last_clean":"2024-09-15T22:06:47.849758+0000","last_became_active":"2024-09-15T22:06:37.179646+0000","last_became_peered":"2024-09-15T22:06:37.179646+0000","last_unstale":"2024-09-15T22:06:47.849758+0000","last_undegraded":"2024-09-15T22:06:47.849758+0000","last_fullsized":"2024-09-15T22:06:47.849758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:35.240468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.644930+0000","last_change":"2024-09-15T22:06:37.182992+0000","last_active":"2024-09-15T22:06:47.644930+0000","last_peered":"2024-09-15T22:06:47.644930+0000","last_clean":"2024-09-15T22:06:47.644930+0000","last_became_active":"2024-09-15T22:06:37.182676+0000","last_became_peered":"2024-09-15T22:06:37.182676+0000","last_unstale":"2024-09-15T22:06:47.644930+0000","last_undegraded":"2024-09-15T22:06:47.644930+0000","last_fullsized":"2024-09-15T22:06:47.644930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:41.524600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851507+0000","last_change":"2024-09-15T22:06:37.180107+0000","last_active":"2024-09-15T22:06:47.851507+0000","last_peered":"2024-09-15T22:06:47.851507+0000","last_clean":"2024-09-15T22:06:47.851507+0000","last_became_active":"2024-09-15T22:06:37.179785+0000","last_became_peered":"2024-09-15T22:06:37.179785+0000","last_unstale":"2024-09-15T22:06:47.851507+0000","last_undegraded":"2024-09-15T22:06:47.851507+0000","last_fullsized":"2024-09-15T22:06:47.851507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:46.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850135+0000","last_change":"2024-09-15T22:06:37.177406+0000","last_active":"2024-09-15T22:06:47.850135+0000","last_peered":"2024-09-15T22:06:47.850135+0000","last_clean":"2024-09-15T22:06:47.850135+0000","last_became_active":"2024-09-15T22:06:37.177093+0000","last_became_peered":"2024-09-15T22:06:37.177093+0000","last_unstale":"2024-09-15T22:06:47.850135+0000","last_undegraded":"2024-09-15T22:06:47.850135+0000","last_fullsized":"2024-09-15T22:06:47.850135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:34.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850683+0000","last_change":"2024-09-15T22:06:37.192314+0000","last_active":"2024-09-15T22:06:47.850683+0000","last_peered":"2024-09-15T22:06:47.850683+0000","last_clean":"2024-09-15T22:06:47.850683+0000","last_became_active":"2024-09-15T22:06:37.191958+0000","last_became_peered":"2024-09-15T22:06:37.191958+0000","last_unstale":"2024-09-15T22:06:47.850683+0000","last_undegraded":"2024-09-15T22:06:47.850683+0000","last_fullsized":"2024-09-15T22:06:47.850683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:47.763742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850196+0000","last_change":"2024-09-15T22:06:37.190212+0000","last_active":"2024-09-15T22:06:47.850196+0000","last_peered":"2024-09-15T22:06:47.850196+0000","last_clean":"2024-09-15T22:06:47.850196+0000","last_became_active":"2024-09-15T22:06:37.189881+0000","last_became_peered":"2024-09-15T22:06:37.189881+0000","last_unstale":"2024-09-15T22:06:47.850196+0000","last_undegraded":"2024-09-15T22:06:47.850196+0000","last_fullsized":"2024-09-15T22:06:47.850196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:59.943574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850343+0000","last_change":"2024-09-15T22:06:37.187967+0000","last_active":"2024-09-15T22:06:47.850343+0000","last_peered":"2024-09-15T22:06:47.850343+0000","last_clean":"2024-09-15T22:06:47.850343+0000","last_became_active":"2024-09-15T22:06:37.187596+0000","last_became_peered":"2024-09-15T22:06:37.187596+0000","last_unstale":"2024-09-15T22:06:47.850343+0000","last_undegraded":"2024-09-15T22:06:47.850343+0000","last_fullsized":"2024-09-15T22:06:47.850343+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:00.042784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850681+0000","last_change":"2024-09-15T22:06:37.189434+0000","last_active":"2024-09-15T22:06:47.850681+0000","last_peered":"2024-09-15T22:06:47.850681+0000","last_clean":"2024-09-15T22:06:47.850681+0000","last_became_active":"2024-09-15T22:06:37.189053+0000","last_became_peered":"2024-09-15T22:06:37.189053+0000","last_unstale":"2024-09-15T22:06:47.850681+0000","last_undegraded":"2024-09-15T22:06:47.850681+0000","last_fullsized":"2024-09-15T22:06:47.850681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:51.640495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850378+0000","last_change":"2024-09-15T22:06:37.179229+0000","last_active":"2024-09-15T22:06:47.850378+0000","last_peered":"2024-09-15T22:06:47.850378+0000","last_clean":"2024-09-15T22:06:47.850378+0000","last_became_active":"2024-09-15T22:06:37.178828+0000","last_became_peered":"2024-09-15T22:06:37.178828+0000","last_unstale":"2024-09-15T22:06:47.850378+0000","last_undegraded":"2024-09-15T22:06:47.850378+0000","last_fullsized":"2024-09-15T22:06:47.850378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:25.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849946+0000","last_change":"2024-09-15T22:06:37.193095+0000","last_active":"2024-09-15T22:06:47.849946+0000","last_peered":"2024-09-15T22:06:47.849946+0000","last_clean":"2024-09-15T22:06:47.849946+0000","last_became_active":"2024-09-15T22:06:37.192801+0000","last_became_peered":"2024-09-15T22:06:37.192801+0000","last_unstale":"2024-09-15T22:06:47.849946+0000","last_undegraded":"2024-09-15T22:06:47.849946+0000","last_fullsized":"2024-09-15T22:06:47.849946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:12.521510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850198+0000","last_change":"2024-09-15T22:06:37.182332+0000","last_active":"2024-09-15T22:06:47.850198+0000","last_peered":"2024-09-15T22:06:47.850198+0000","last_clean":"2024-09-15T22:06:47.850198+0000","last_became_active":"2024-09-15T22:06:37.182001+0000","last_became_peered":"2024-09-15T22:06:37.182001+0000","last_unstale":"2024-09-15T22:06:47.850198+0000","last_undegraded":"2024-09-15T22:06:47.850198+0000","last_fullsized":"2024-09-15T22:06:47.850198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:52.230452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850037+0000","last_change":"2024-09-15T22:06:37.180584+0000","last_active":"2024-09-15T22:06:47.850037+0000","last_peered":"2024-09-15T22:06:47.850037+0000","last_clean":"2024-09-15T22:06:47.850037+0000","last_became_active":"2024-09-15T22:06:37.179341+0000","last_became_peered":"2024-09-15T22:06:37.179341+0000","last_unstale":"2024-09-15T22:06:47.850037+0000","last_undegraded":"2024-09-15T22:06:47.850037+0000","last_fullsized":"2024-09-15T22:06:47.850037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.610835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646833+0000","last_change":"2024-09-15T22:06:37.189500+0000","last_active":"2024-09-15T22:06:47.646833+0000","last_peered":"2024-09-15T22:06:47.646833+0000","last_clean":"2024-09-15T22:06:47.646833+0000","last_became_active":"2024-09-15T22:06:37.189137+0000","last_became_peered":"2024-09-15T22:06:37.189137+0000","last_unstale":"2024-09-15T22:06:47.646833+0000","last_undegraded":"2024-09-15T22:06:47.646833+0000","last_fullsized":"2024-09-15T22:06:47.646833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:27.199854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850938+0000","last_change":"2024-09-15T22:06:37.182618+0000","last_active":"2024-09-15T22:06:47.850938+0000","last_peered":"2024-09-15T22:06:47.850938+0000","last_clean":"2024-09-15T22:06:47.850938+0000","last_became_active":"2024-09-15T22:06:37.182257+0000","last_became_peered":"2024-09-15T22:06:37.182257+0000","last_unstale":"2024-09-15T22:06:47.850938+0000","last_undegraded":"2024-09-15T22:06:47.850938+0000","last_fullsized":"2024-09-15T22:06:47.850938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:17.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851071+0000","last_change":"2024-09-15T22:06:37.193036+0000","last_active":"2024-09-15T22:06:47.851071+0000","last_peered":"2024-09-15T22:06:47.851071+0000","last_clean":"2024-09-15T22:06:47.851071+0000","last_became_active":"2024-09-15T22:06:37.192668+0000","last_became_peered":"2024-09-15T22:06:37.192668+0000","last_unstale":"2024-09-15T22:06:47.851071+0000","last_undegraded":"2024-09-15T22:06:47.851071+0000","last_fullsized":"2024-09-15T22:06:47.851071+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:05.436045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850535+0000","last_change":"2024-09-15T22:06:44.659503+0000","last_active":"2024-09-15T22:06:47.850535+0000","last_peered":"2024-09-15T22:06:47.850535+0000","last_clean":"2024-09-15T22:06:47.850535+0000","last_became_active":"2024-09-15T22:06:44.659155+0000","last_became_peered":"2024-09-15T22:06:44.659155+0000","last_unstale":"2024-09-15T22:06:47.850535+0000","last_undegraded":"2024-09-15T22:06:47.850535+0000","last_fullsized":"2024-09-15T22:06:47.850535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:41:04.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.644915+0000","last_change":"2024-09-15T22:06:37.178386+0000","last_active":"2024-09-15T22:06:47.644915+0000","last_peered":"2024-09-15T22:06:47.644915+0000","last_clean":"2024-09-15T22:06:47.644915+0000","last_became_active":"2024-09-15T22:06:37.178083+0000","last_became_peered":"2024-09-15T22:06:37.178083+0000","last_unstale":"2024-09-15T22:06:47.644915+0000","last_undegraded":"2024-09-15T22:06:47.644915+0000","last_fullsized":"2024-09-15T22:06:47.644915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:35.350698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851249+0000","last_change":"2024-09-15T22:06:37.186741+0000","last_active":"2024-09-15T22:06:47.851249+0000","last_peered":"2024-09-15T22:06:47.851249+0000","last_clean":"2024-09-15T22:06:47.851249+0000","last_became_active":"2024-09-15T22:06:37.185740+0000","last_became_peered":"2024-09-15T22:06:37.185740+0000","last_unstale":"2024-09-15T22:06:47.851249+0000","last_undegraded":"2024-09-15T22:06:47.851249+0000","last_fullsized":"2024-09-15T22:06:47.851249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.762407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850187+0000","last_change":"2024-09-15T22:06:37.189552+0000","last_active":"2024-09-15T22:06:47.850187+0000","last_peered":"2024-09-15T22:06:47.850187+0000","last_clean":"2024-09-15T22:06:47.850187+0000","last_became_active":"2024-09-15T22:06:37.189197+0000","last_became_peered":"2024-09-15T22:06:37.189197+0000","last_unstale":"2024-09-15T22:06:47.850187+0000","last_undegraded":"2024-09-15T22:06:47.850187+0000","last_fullsized":"2024-09-15T22:06:47.850187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:54.268720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645537+0000","last_change":"2024-09-15T22:06:37.186917+0000","last_active":"2024-09-15T22:06:47.645537+0000","last_peered":"2024-09-15T22:06:47.645537+0000","last_clean":"2024-09-15T22:06:47.645537+0000","last_became_active":"2024-09-15T22:06:37.186571+0000","last_became_peered":"2024-09-15T22:06:37.186571+0000","last_unstale":"2024-09-15T22:06:47.645537+0000","last_undegraded":"2024-09-15T22:06:47.645537+0000","last_fullsized":"2024-09-15T22:06:47.645537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:02.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646824+0000","last_change":"2024-09-15T22:06:37.190960+0000","last_active":"2024-09-15T22:06:47.646824+0000","last_peered":"2024-09-15T22:06:47.646824+0000","last_clean":"2024-09-15T22:06:47.646824+0000","last_became_active":"2024-09-15T22:06:37.190650+0000","last_became_peered":"2024-09-15T22:06:37.190650+0000","last_unstale":"2024-09-15T22:06:47.646824+0000","last_undegraded":"2024-09-15T22:06:47.646824+0000","last_fullsized":"2024-09-15T22:06:47.646824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:34.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851201+0000","last_change":"2024-09-15T22:06:37.186718+0000","last_active":"2024-09-15T22:06:47.851201+0000","last_peered":"2024-09-15T22:06:47.851201+0000","last_clean":"2024-09-15T22:06:47.851201+0000","last_became_active":"2024-09-15T22:06:37.186424+0000","last_became_peered":"2024-09-15T22:06:37.186424+0000","last_unstale":"2024-09-15T22:06:47.851201+0000","last_undegraded":"2024-09-15T22:06:47.851201+0000","last_fullsized":"2024-09-15T22:06:47.851201+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:53.266007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850897+0000","last_change":"2024-09-15T22:06:37.185522+0000","last_active":"2024-09-15T22:06:47.850897+0000","last_peered":"2024-09-15T22:06:47.850897+0000","last_clean":"2024-09-15T22:06:47.850897+0000","last_became_active":"2024-09-15T22:06:37.185192+0000","last_became_peered":"2024-09-15T22:06:37.185192+0000","last_unstale":"2024-09-15T22:06:47.850897+0000","last_undegraded":"2024-09-15T22:06:47.850897+0000","last_fullsized":"2024-09-15T22:06:47.850897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:12.868324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850539+0000","last_change":"2024-09-15T22:06:37.189553+0000","last_active":"2024-09-15T22:06:47.850539+0000","last_peered":"2024-09-15T22:06:47.850539+0000","last_clean":"2024-09-15T22:06:47.850539+0000","last_became_active":"2024-09-15T22:06:37.189270+0000","last_became_peered":"2024-09-15T22:06:37.189270+0000","last_unstale":"2024-09-15T22:06:47.850539+0000","last_undegraded":"2024-09-15T22:06:47.850539+0000","last_fullsized":"2024-09-15T22:06:47.850539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:34.157077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47741,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":100,"num_read_kb":101,"num_write":157,"num_write_kb":94,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":385024,"data_stored":161349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579927,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":483279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808860,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32792,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712360,"statfs":{"total":95995035648,"available":95961456640,"internally_reserved":0,"allocated":1646592,"data_stored":1114285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005089,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":483515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234021,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":482853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495659,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1075643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659184,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":478691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888117,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1593344,"data_stored":1074934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018939,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":484316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247871,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":478885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444101,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":532170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":531591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":480079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":52990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":53569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":45316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:07:31.065 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:07:31.325 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:31.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:30 smithi099 ceph-mon[34892]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:07:31.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:30 smithi129 ceph-mon[35314]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:07:31.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:30 smithi005 ceph-mon[30944]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:07:31.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.822+0000 7fbbb68e3640 1 -- 172.21.15.5:0/550910854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb0102570 msgr2=0x7fbbb01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:31.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.822+0000 7fbbb68e3640 1 --2- 172.21.15.5:0/550910854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb0102570 0x7fbbb01029b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009900 tx=0x7fbba402a480 comp rx=0 tx=0).stop 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.822+0000 7fbbb68e3640 1 -- 172.21.15.5:0/550910854 shutdown_connections 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.822+0000 7fbbb68e3640 1 --2- 172.21.15.5:0/550910854 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbb0102ef0 0x7fbbb010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.822+0000 7fbbb68e3640 1 --2- 172.21.15.5:0/550910854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb0102570 0x7fbbb01029b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.822+0000 7fbbb68e3640 1 --2- 172.21.15.5:0/550910854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb0100e20 0x7fbbb0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.822+0000 7fbbb68e3640 1 -- 172.21.15.5:0/550910854 >> 172.21.15.5:0/550910854 conn(0x7fbbb00fc7d0 msgr2=0x7fbbb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.822+0000 7fbbb68e3640 1 -- 172.21.15.5:0/550910854 shutdown_connections 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.822+0000 7fbbb68e3640 1 -- 172.21.15.5:0/550910854 wait complete. 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.823+0000 7fbbb68e3640 1 Processor -- start 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.823+0000 7fbbb68e3640 1 -- start start 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb68e3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbb0100e20 0x7fbbb01973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb68e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb0102570 0x7fbbb0197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb68e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb0102ef0 0x7fbbb019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:31.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb68e3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0110f80 con 0x7fbbb0102ef0 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb68e3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0110e00 con 0x7fbbb0102570 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb68e3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb0111100 con 0x7fbbb0100e20 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb4e80640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb0102ef0 0x7fbbb019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb4e80640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb0102ef0 0x7fbbb019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53006/0 (socket says 172.21.15.5:53006) 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb4e80640 1 -- 172.21.15.5:0/3921735213 learned_addr learned my addr 172.21.15.5:0/3921735213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbaffff640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbb0100e20 0x7fbbb01973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbaf7fe640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb0102570 0x7fbbb0197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb4e80640 1 -- 172.21.15.5:0/3921735213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbb0100e20 msgr2=0x7fbbb01973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb4e80640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbb0100e20 0x7fbbb01973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb4e80640 1 -- 172.21.15.5:0/3921735213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb0102570 msgr2=0x7fbbb0197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb4e80640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb0102570 0x7fbbb0197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbb4e80640 1 -- 172.21.15.5:0/3921735213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb019c370 con 0x7fbbb0102ef0 2024-09-15T22:07:31.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbaffff640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbb0100e20 0x7fbbb01973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.824+0000 7fbbaf7fe640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb0102570 0x7fbbb0197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.825+0000 7fbbb4e80640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb0102ef0 0x7fbbb019bc90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbba800eaa0 tx=0x7fbba800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.825+0000 7fbbad7fa640 1 -- 172.21.15.5:0/3921735213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba800cdb0 con 0x7fbbb0102ef0 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.825+0000 7fbbad7fa640 1 -- 172.21.15.5:0/3921735213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbba8005020 con 0x7fbbb0102ef0 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.825+0000 7fbbb68e3640 1 -- 172.21.15.5:0/3921735213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb019c660 con 0x7fbbb0102ef0 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.825+0000 7fbbad7fa640 1 -- 172.21.15.5:0/3921735213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba80116b0 con 0x7fbbb0102ef0 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.825+0000 7fbbb68e3640 1 -- 172.21.15.5:0/3921735213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb01a3f40 con 0x7fbbb0102ef0 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.826+0000 7fbbb68e3640 1 -- 172.21.15.5:0/3921735213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb0101200 con 0x7fbbb0102ef0 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.827+0000 7fbbad7fa640 1 -- 172.21.15.5:0/3921735213 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fbba8005560 con 0x7fbbb0102ef0 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.828+0000 7fbbad7fa640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbb980a3350 0x7fbb980a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.828+0000 7fbbad7fa640 1 -- 172.21.15.5:0/3921735213 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7fbba80d02d0 con 0x7fbbb0102ef0 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.828+0000 7fbbaffff640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbb980a3350 0x7fbb980a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:31.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.829+0000 7fbbaffff640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbb980a3350 0x7fbb980a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009fc0 tx=0x7fbba0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:31.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:31.831+0000 7fbbad7fa640 1 -- 172.21.15.5:0/3921735213 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbba8016030 con 0x7fbbb0102ef0 2024-09-15T22:07:32.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.024+0000 7fbbb68e3640 1 -- 172.21.15.5:0/3921735213 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbbb0106fb0 con 0x7fbb980a3350 2024-09-15T22:07:32.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:31 smithi005 ceph-mon[30944]: pgmap v310: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-15T22:07:32.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:31 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:32.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.038+0000 7fbbad7fa640 1 -- 172.21.15.5:0/3921735213 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340337 (secure 0 0 0) 0x7fbbb0106fb0 con 0x7fbb980a3350 2024-09-15T22:07:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:07:32.044 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-15T22:07:32.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 -- 172.21.15.5:0/3921735213 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbb980a3350 msgr2=0x7fbb980a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbb980a3350 0x7fbb980a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009fc0 tx=0x7fbba0005f90 comp rx=0 tx=0).stop 2024-09-15T22:07:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 -- 172.21.15.5:0/3921735213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb0102ef0 msgr2=0x7fbbb019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb0102ef0 0x7fbbb019bc90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbba800eaa0 tx=0x7fbba800ef60 comp rx=0 tx=0).stop 2024-09-15T22:07:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 -- 172.21.15.5:0/3921735213 shutdown_connections 2024-09-15T22:07:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fbb980a3350 0x7fbb980a5810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbb0102ef0 0x7fbbb019bc90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbbb0102570 0x7fbbb0197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 --2- 172.21.15.5:0/3921735213 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbbb0100e20 0x7fbbb01973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 -- 172.21.15.5:0/3921735213 >> 172.21.15.5:0/3921735213 conn(0x7fbbb00fc7d0 msgr2=0x7fbbb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 -- 172.21.15.5:0/3921735213 shutdown_connections 2024-09-15T22:07:32.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:32.043+0000 7fbbb68e3640 1 -- 172.21.15.5:0/3921735213 wait complete. 2024-09-15T22:07:32.330 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:31 smithi129 ceph-mon[35314]: pgmap v310: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-15T22:07:32.331 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:31 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:32.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:31 smithi099 ceph-mon[34892]: pgmap v310: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.3 KiB/s wr, 9 op/s 2024-09-15T22:07:32.335 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:31 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:07:32.736 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-15T22:07:30.799503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647406,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":252,"num_read_kb":239,"num_write":350,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1181318,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387692,"kb_used_data":13484,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554132,"statfs":{"total":1151940427776,"available":1151543431168,"internally_reserved":0,"allocated":13807616,"data_stored":7717496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49584,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":92,"num_write":35,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003390"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.737074+0000","last_change":"2024-09-15T22:06:44.668504+0000","last_active":"2024-09-15T22:07:13.737074+0000","last_peered":"2024-09-15T22:07:13.737074+0000","last_clean":"2024-09-15T22:07:13.737074+0000","last_became_active":"2024-09-15T22:06:44.661198+0000","last_became_peered":"2024-09-15T22:06:44.661198+0000","last_unstale":"2024-09-15T22:07:13.737074+0000","last_undegraded":"2024-09-15T22:07:13.737074+0000","last_fullsized":"2024-09-15T22:07:13.737074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:33.749045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,4],"acting":[9,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645317+0000","last_change":"2024-09-15T22:06:37.184096+0000","last_active":"2024-09-15T22:06:47.645317+0000","last_peered":"2024-09-15T22:06:47.645317+0000","last_clean":"2024-09-15T22:06:47.645317+0000","last_became_active":"2024-09-15T22:06:37.183758+0000","last_became_peered":"2024-09-15T22:06:37.183758+0000","last_unstale":"2024-09-15T22:06:47.645317+0000","last_undegraded":"2024-09-15T22:06:47.645317+0000","last_fullsized":"2024-09-15T22:06:47.645317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:04.210551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849243+0000","last_change":"2024-09-15T22:06:44.653042+0000","last_active":"2024-09-15T22:06:47.849243+0000","last_peered":"2024-09-15T22:06:47.849243+0000","last_clean":"2024-09-15T22:06:47.849243+0000","last_became_active":"2024-09-15T22:06:44.652658+0000","last_became_peered":"2024-09-15T22:06:44.652658+0000","last_unstale":"2024-09-15T22:06:47.849243+0000","last_undegraded":"2024-09-15T22:06:47.849243+0000","last_fullsized":"2024-09-15T22:06:47.849243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:59.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645466+0000","last_change":"2024-09-15T22:06:35.355392+0000","last_active":"2024-09-15T22:06:47.645466+0000","last_peered":"2024-09-15T22:06:47.645466+0000","last_clean":"2024-09-15T22:06:47.645466+0000","last_became_active":"2024-09-15T22:06:35.355042+0000","last_became_peered":"2024-09-15T22:06:35.355042+0000","last_unstale":"2024-09-15T22:06:47.645466+0000","last_undegraded":"2024-09-15T22:06:47.645466+0000","last_fullsized":"2024-09-15T22:06:47.645466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:05.186937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851100+0000","last_change":"2024-09-15T22:06:37.184421+0000","last_active":"2024-09-15T22:06:47.851100+0000","last_peered":"2024-09-15T22:06:47.851100+0000","last_clean":"2024-09-15T22:06:47.851100+0000","last_became_active":"2024-09-15T22:06:37.184092+0000","last_became_peered":"2024-09-15T22:06:37.184092+0000","last_unstale":"2024-09-15T22:06:47.851100+0000","last_undegraded":"2024-09-15T22:06:47.851100+0000","last_fullsized":"2024-09-15T22:06:47.851100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:41.080981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646238+0000","last_change":"2024-09-15T22:06:44.656525+0000","last_active":"2024-09-15T22:06:47.646238+0000","last_peered":"2024-09-15T22:06:47.646238+0000","last_clean":"2024-09-15T22:06:47.646238+0000","last_became_active":"2024-09-15T22:06:44.656131+0000","last_became_peered":"2024-09-15T22:06:44.656131+0000","last_unstale":"2024-09-15T22:06:47.646238+0000","last_undegraded":"2024-09-15T22:06:47.646238+0000","last_fullsized":"2024-09-15T22:06:47.646238+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:10:17.056927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.734100+0000","last_change":"2024-09-15T22:06:35.134606+0000","last_active":"2024-09-15T22:07:13.734100+0000","last_peered":"2024-09-15T22:07:13.734100+0000","last_clean":"2024-09-15T22:07:13.734100+0000","last_became_active":"2024-09-15T22:06:35.134263+0000","last_became_peered":"2024-09-15T22:06:35.134263+0000","last_unstale":"2024-09-15T22:07:13.734100+0000","last_undegraded":"2024-09-15T22:07:13.734100+0000","last_fullsized":"2024-09-15T22:07:13.734100+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:45.008935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851556+0000","last_change":"2024-09-15T22:06:37.177556+0000","last_active":"2024-09-15T22:06:47.851556+0000","last_peered":"2024-09-15T22:06:47.851556+0000","last_clean":"2024-09-15T22:06:47.851556+0000","last_became_active":"2024-09-15T22:06:37.177161+0000","last_became_peered":"2024-09-15T22:06:37.177161+0000","last_unstale":"2024-09-15T22:06:47.851556+0000","last_undegraded":"2024-09-15T22:06:47.851556+0000","last_fullsized":"2024-09-15T22:06:47.851556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:30.192728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646502+0000","last_change":"2024-09-15T22:06:44.659097+0000","last_active":"2024-09-15T22:06:47.646502+0000","last_peered":"2024-09-15T22:06:47.646502+0000","last_clean":"2024-09-15T22:06:47.646502+0000","last_became_active":"2024-09-15T22:06:44.658626+0000","last_became_peered":"2024-09-15T22:06:44.658626+0000","last_unstale":"2024-09-15T22:06:47.646502+0000","last_undegraded":"2024-09-15T22:06:47.646502+0000","last_fullsized":"2024-09-15T22:06:47.646502+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:01.308004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645834+0000","last_change":"2024-09-15T22:06:35.136326+0000","last_active":"2024-09-15T22:06:47.645834+0000","last_peered":"2024-09-15T22:06:47.645834+0000","last_clean":"2024-09-15T22:06:47.645834+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:06:47.645834+0000","last_undegraded":"2024-09-15T22:06:47.645834+0000","last_fullsized":"2024-09-15T22:06:47.645834+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:28.675832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850874+0000","last_change":"2024-09-15T22:06:37.192939+0000","last_active":"2024-09-15T22:06:47.850874+0000","last_peered":"2024-09-15T22:06:47.850874+0000","last_clean":"2024-09-15T22:06:47.850874+0000","last_became_active":"2024-09-15T22:06:37.192616+0000","last_became_peered":"2024-09-15T22:06:37.192616+0000","last_unstale":"2024-09-15T22:06:47.850874+0000","last_undegraded":"2024-09-15T22:06:47.850874+0000","last_fullsized":"2024-09-15T22:06:47.850874+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:03.615203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646361+0000","last_change":"2024-09-15T22:06:44.654741+0000","last_active":"2024-09-15T22:06:47.646361+0000","last_peered":"2024-09-15T22:06:47.646361+0000","last_clean":"2024-09-15T22:06:47.646361+0000","last_became_active":"2024-09-15T22:06:44.654362+0000","last_became_peered":"2024-09-15T22:06:44.654362+0000","last_unstale":"2024-09-15T22:06:47.646361+0000","last_undegraded":"2024-09-15T22:06:47.646361+0000","last_fullsized":"2024-09-15T22:06:47.646361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:59.269948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851179+0000","last_change":"2024-09-15T22:06:37.186385+0000","last_active":"2024-09-15T22:06:47.851179+0000","last_peered":"2024-09-15T22:06:47.851179+0000","last_clean":"2024-09-15T22:06:47.851179+0000","last_became_active":"2024-09-15T22:06:37.185588+0000","last_became_peered":"2024-09-15T22:06:37.185588+0000","last_unstale":"2024-09-15T22:06:47.851179+0000","last_undegraded":"2024-09-15T22:06:47.851179+0000","last_fullsized":"2024-09-15T22:06:47.851179+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:22.523083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.725378+0000","last_change":"2024-09-15T22:06:35.137383+0000","last_active":"2024-09-15T22:07:12.725378+0000","last_peered":"2024-09-15T22:07:12.725378+0000","last_clean":"2024-09-15T22:07:12.725378+0000","last_became_active":"2024-09-15T22:06:35.137022+0000","last_became_peered":"2024-09-15T22:06:35.137022+0000","last_unstale":"2024-09-15T22:07:12.725378+0000","last_undegraded":"2024-09-15T22:07:12.725378+0000","last_fullsized":"2024-09-15T22:07:12.725378+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.863155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646734+0000","last_change":"2024-09-15T22:06:44.653807+0000","last_active":"2024-09-15T22:06:47.646734+0000","last_peered":"2024-09-15T22:06:47.646734+0000","last_clean":"2024-09-15T22:06:47.646734+0000","last_became_active":"2024-09-15T22:06:44.653406+0000","last_became_peered":"2024-09-15T22:06:44.653406+0000","last_unstale":"2024-09-15T22:06:47.646734+0000","last_undegraded":"2024-09-15T22:06:47.646734+0000","last_fullsized":"2024-09-15T22:06:47.646734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:12.018208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645073+0000","last_change":"2024-09-15T22:06:37.186899+0000","last_active":"2024-09-15T22:06:47.645073+0000","last_peered":"2024-09-15T22:06:47.645073+0000","last_clean":"2024-09-15T22:06:47.645073+0000","last_became_active":"2024-09-15T22:06:37.183782+0000","last_became_peered":"2024-09-15T22:06:37.183782+0000","last_unstale":"2024-09-15T22:06:47.645073+0000","last_undegraded":"2024-09-15T22:06:47.645073+0000","last_fullsized":"2024-09-15T22:06:47.645073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:39.269186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850326+0000","last_change":"2024-09-15T22:06:44.652686+0000","last_active":"2024-09-15T22:06:47.850326+0000","last_peered":"2024-09-15T22:06:47.850326+0000","last_clean":"2024-09-15T22:06:47.850326+0000","last_became_active":"2024-09-15T22:06:44.652306+0000","last_became_peered":"2024-09-15T22:06:44.652306+0000","last_unstale":"2024-09-15T22:06:47.850326+0000","last_undegraded":"2024-09-15T22:06:47.850326+0000","last_fullsized":"2024-09-15T22:06:47.850326+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:48.454535+0000","last_change":"2024-09-15T22:06:44.655424+0000","last_active":"2024-09-15T22:06:48.454535+0000","last_peered":"2024-09-15T22:06:48.454535+0000","last_clean":"2024-09-15T22:06:48.454535+0000","last_became_active":"2024-09-15T22:06:44.654959+0000","last_became_peered":"2024-09-15T22:06:44.654959+0000","last_unstale":"2024-09-15T22:06:48.454535+0000","last_undegraded":"2024-09-15T22:06:48.454535+0000","last_fullsized":"2024-09-15T22:06:48.454535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:07.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850289+0000","last_change":"2024-09-15T22:06:37.188424+0000","last_active":"2024-09-15T22:06:47.850289+0000","last_peered":"2024-09-15T22:06:47.850289+0000","last_clean":"2024-09-15T22:06:47.850289+0000","last_became_active":"2024-09-15T22:06:37.188082+0000","last_became_peered":"2024-09-15T22:06:37.188082+0000","last_unstale":"2024-09-15T22:06:47.850289+0000","last_undegraded":"2024-09-15T22:06:47.850289+0000","last_fullsized":"2024-09-15T22:06:47.850289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:07.159341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.729202+0000","last_change":"2024-09-15T22:06:35.134237+0000","last_active":"2024-09-15T22:07:12.729202+0000","last_peered":"2024-09-15T22:07:12.729202+0000","last_clean":"2024-09-15T22:07:12.729202+0000","last_became_active":"2024-09-15T22:06:35.133885+0000","last_became_peered":"2024-09-15T22:06:35.133885+0000","last_unstale":"2024-09-15T22:07:12.729202+0000","last_undegraded":"2024-09-15T22:07:12.729202+0000","last_fullsized":"2024-09-15T22:07:12.729202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:07:35.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850431+0000","last_change":"2024-09-15T22:06:44.664578+0000","last_active":"2024-09-15T22:06:47.850431+0000","last_peered":"2024-09-15T22:06:47.850431+0000","last_clean":"2024-09-15T22:06:47.850431+0000","last_became_active":"2024-09-15T22:06:44.664170+0000","last_became_peered":"2024-09-15T22:06:44.664170+0000","last_unstale":"2024-09-15T22:06:47.850431+0000","last_undegraded":"2024-09-15T22:06:47.850431+0000","last_fullsized":"2024-09-15T22:06:47.850431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:41.443816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645413+0000","last_change":"2024-09-15T22:06:37.186107+0000","last_active":"2024-09-15T22:06:47.645413+0000","last_peered":"2024-09-15T22:06:47.645413+0000","last_clean":"2024-09-15T22:06:47.645413+0000","last_became_active":"2024-09-15T22:06:37.185757+0000","last_became_peered":"2024-09-15T22:06:37.185757+0000","last_unstale":"2024-09-15T22:06:47.645413+0000","last_undegraded":"2024-09-15T22:06:47.645413+0000","last_fullsized":"2024-09-15T22:06:47.645413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:14.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.726066+0000","last_change":"2024-09-15T22:06:35.356281+0000","last_active":"2024-09-15T22:07:12.726066+0000","last_peered":"2024-09-15T22:07:12.726066+0000","last_clean":"2024-09-15T22:07:12.726066+0000","last_became_active":"2024-09-15T22:06:35.355922+0000","last_became_peered":"2024-09-15T22:06:35.355922+0000","last_unstale":"2024-09-15T22:07:12.726066+0000","last_undegraded":"2024-09-15T22:07:12.726066+0000","last_fullsized":"2024-09-15T22:07:12.726066+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:06.107462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646959+0000","last_change":"2024-09-15T22:06:44.659230+0000","last_active":"2024-09-15T22:06:47.646959+0000","last_peered":"2024-09-15T22:06:47.646959+0000","last_clean":"2024-09-15T22:06:47.646959+0000","last_became_active":"2024-09-15T22:06:44.658649+0000","last_became_peered":"2024-09-15T22:06:44.658649+0000","last_unstale":"2024-09-15T22:06:47.646959+0000","last_undegraded":"2024-09-15T22:06:47.646959+0000","last_fullsized":"2024-09-15T22:06:47.646959+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:14:21.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850818+0000","last_change":"2024-09-15T22:06:44.649394+0000","last_active":"2024-09-15T22:06:47.850818+0000","last_peered":"2024-09-15T22:06:47.850818+0000","last_clean":"2024-09-15T22:06:47.850818+0000","last_became_active":"2024-09-15T22:06:44.649043+0000","last_became_peered":"2024-09-15T22:06:44.649043+0000","last_unstale":"2024-09-15T22:06:47.850818+0000","last_undegraded":"2024-09-15T22:06:47.850818+0000","last_fullsized":"2024-09-15T22:06:47.850818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:56:28.335869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":34,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.726312+0000","last_change":"2024-09-15T22:06:35.265970+0000","last_active":"2024-09-15T22:07:12.726312+0000","last_peered":"2024-09-15T22:07:12.726312+0000","last_clean":"2024-09-15T22:07:12.726312+0000","last_became_active":"2024-09-15T22:06:35.265628+0000","last_became_peered":"2024-09-15T22:06:35.265628+0000","last_unstale":"2024-09-15T22:07:12.726312+0000","last_undegraded":"2024-09-15T22:07:12.726312+0000","last_fullsized":"2024-09-15T22:07:12.726312+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:41.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849607+0000","last_change":"2024-09-15T22:06:44.649652+0000","last_active":"2024-09-15T22:06:47.849607+0000","last_peered":"2024-09-15T22:06:47.849607+0000","last_clean":"2024-09-15T22:06:47.849607+0000","last_became_active":"2024-09-15T22:06:44.649112+0000","last_became_peered":"2024-09-15T22:06:44.649112+0000","last_unstale":"2024-09-15T22:06:47.849607+0000","last_undegraded":"2024-09-15T22:06:47.849607+0000","last_fullsized":"2024-09-15T22:06:47.849607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:53.789320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645620+0000","last_change":"2024-09-15T22:06:37.190701+0000","last_active":"2024-09-15T22:06:47.645620+0000","last_peered":"2024-09-15T22:06:47.645620+0000","last_clean":"2024-09-15T22:06:47.645620+0000","last_became_active":"2024-09-15T22:06:37.190430+0000","last_became_peered":"2024-09-15T22:06:37.190430+0000","last_unstale":"2024-09-15T22:06:47.645620+0000","last_undegraded":"2024-09-15T22:06:47.645620+0000","last_fullsized":"2024-09-15T22:06:47.645620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:14.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:27.204544+0000","last_change":"2024-09-15T22:06:35.133899+0000","last_active":"2024-09-15T22:07:27.204544+0000","last_peered":"2024-09-15T22:07:27.204544+0000","last_clean":"2024-09-15T22:07:27.204544+0000","last_became_active":"2024-09-15T22:06:35.133556+0000","last_became_peered":"2024-09-15T22:06:35.133556+0000","last_unstale":"2024-09-15T22:07:27.204544+0000","last_undegraded":"2024-09-15T22:07:27.204544+0000","last_fullsized":"2024-09-15T22:07:27.204544+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:01.692416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.644792+0000","last_change":"2024-09-15T22:06:44.655280+0000","last_active":"2024-09-15T22:06:47.644792+0000","last_peered":"2024-09-15T22:06:47.644792+0000","last_clean":"2024-09-15T22:06:47.644792+0000","last_became_active":"2024-09-15T22:06:44.654883+0000","last_became_peered":"2024-09-15T22:06:44.654883+0000","last_unstale":"2024-09-15T22:06:47.644792+0000","last_undegraded":"2024-09-15T22:06:47.644792+0000","last_fullsized":"2024-09-15T22:06:47.644792+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.074035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851064+0000","last_change":"2024-09-15T22:06:37.182006+0000","last_active":"2024-09-15T22:06:47.851064+0000","last_peered":"2024-09-15T22:06:47.851064+0000","last_clean":"2024-09-15T22:06:47.851064+0000","last_became_active":"2024-09-15T22:06:37.181693+0000","last_became_peered":"2024-09-15T22:06:37.181693+0000","last_unstale":"2024-09-15T22:06:47.851064+0000","last_undegraded":"2024-09-15T22:06:47.851064+0000","last_fullsized":"2024-09-15T22:06:47.851064+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:23.483930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.724002+0000","last_change":"2024-09-15T22:06:35.355854+0000","last_active":"2024-09-15T22:07:12.724002+0000","last_peered":"2024-09-15T22:07:12.724002+0000","last_clean":"2024-09-15T22:07:12.724002+0000","last_became_active":"2024-09-15T22:06:35.355486+0000","last_became_peered":"2024-09-15T22:06:35.355486+0000","last_unstale":"2024-09-15T22:07:12.724002+0000","last_undegraded":"2024-09-15T22:07:12.724002+0000","last_fullsized":"2024-09-15T22:07:12.724002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:36.708476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850529+0000","last_change":"2024-09-15T22:06:44.660523+0000","last_active":"2024-09-15T22:06:47.850529+0000","last_peered":"2024-09-15T22:06:47.850529+0000","last_clean":"2024-09-15T22:06:47.850529+0000","last_became_active":"2024-09-15T22:06:44.660113+0000","last_became_peered":"2024-09-15T22:06:44.660113+0000","last_unstale":"2024-09-15T22:06:47.850529+0000","last_undegraded":"2024-09-15T22:06:47.850529+0000","last_fullsized":"2024-09-15T22:06:47.850529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:42.164558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850182+0000","last_change":"2024-09-15T22:06:37.183649+0000","last_active":"2024-09-15T22:06:47.850182+0000","last_peered":"2024-09-15T22:06:47.850182+0000","last_clean":"2024-09-15T22:06:47.850182+0000","last_became_active":"2024-09-15T22:06:37.183296+0000","last_became_peered":"2024-09-15T22:06:37.183296+0000","last_unstale":"2024-09-15T22:06:47.850182+0000","last_undegraded":"2024-09-15T22:06:47.850182+0000","last_fullsized":"2024-09-15T22:06:47.850182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:22.010288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850466+0000","last_change":"2024-09-15T22:06:35.355068+0000","last_active":"2024-09-15T22:06:47.850466+0000","last_peered":"2024-09-15T22:06:47.850466+0000","last_clean":"2024-09-15T22:06:47.850466+0000","last_became_active":"2024-09-15T22:06:35.354728+0000","last_became_peered":"2024-09-15T22:06:35.354728+0000","last_unstale":"2024-09-15T22:06:47.850466+0000","last_undegraded":"2024-09-15T22:06:47.850466+0000","last_fullsized":"2024-09-15T22:06:47.850466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:35.479290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645591+0000","last_change":"2024-09-15T22:06:44.653849+0000","last_active":"2024-09-15T22:06:47.645591+0000","last_peered":"2024-09-15T22:06:47.645591+0000","last_clean":"2024-09-15T22:06:47.645591+0000","last_became_active":"2024-09-15T22:06:44.653419+0000","last_became_peered":"2024-09-15T22:06:44.653419+0000","last_unstale":"2024-09-15T22:06:47.645591+0000","last_undegraded":"2024-09-15T22:06:47.645591+0000","last_fullsized":"2024-09-15T22:06:47.645591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:04.131553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850416+0000","last_change":"2024-09-15T22:06:37.191438+0000","last_active":"2024-09-15T22:06:47.850416+0000","last_peered":"2024-09-15T22:06:47.850416+0000","last_clean":"2024-09-15T22:06:47.850416+0000","last_became_active":"2024-09-15T22:06:37.191088+0000","last_became_peered":"2024-09-15T22:06:37.191088+0000","last_unstale":"2024-09-15T22:06:47.850416+0000","last_undegraded":"2024-09-15T22:06:47.850416+0000","last_fullsized":"2024-09-15T22:06:47.850416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:46:36.078872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851041+0000","last_change":"2024-09-15T22:06:35.135866+0000","last_active":"2024-09-15T22:06:47.851041+0000","last_peered":"2024-09-15T22:06:47.851041+0000","last_clean":"2024-09-15T22:06:47.851041+0000","last_became_active":"2024-09-15T22:06:35.135524+0000","last_became_peered":"2024-09-15T22:06:35.135524+0000","last_unstale":"2024-09-15T22:06:47.851041+0000","last_undegraded":"2024-09-15T22:06:47.851041+0000","last_fullsized":"2024-09-15T22:06:47.851041+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850057+0000","last_change":"2024-09-15T22:06:44.649943+0000","last_active":"2024-09-15T22:06:47.850057+0000","last_peered":"2024-09-15T22:06:47.850057+0000","last_clean":"2024-09-15T22:06:47.850057+0000","last_became_active":"2024-09-15T22:06:44.649579+0000","last_became_peered":"2024-09-15T22:06:44.649579+0000","last_unstale":"2024-09-15T22:06:47.850057+0000","last_undegraded":"2024-09-15T22:06:47.850057+0000","last_fullsized":"2024-09-15T22:06:47.850057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:18.390608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849866+0000","last_change":"2024-09-15T22:06:37.189988+0000","last_active":"2024-09-15T22:06:47.849866+0000","last_peered":"2024-09-15T22:06:47.849866+0000","last_clean":"2024-09-15T22:06:47.849866+0000","last_became_active":"2024-09-15T22:06:37.189319+0000","last_became_peered":"2024-09-15T22:06:37.189319+0000","last_unstale":"2024-09-15T22:06:47.849866+0000","last_undegraded":"2024-09-15T22:06:47.849866+0000","last_fullsized":"2024-09-15T22:06:47.849866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:08.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":48,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:26.756488+0000","last_change":"2024-09-15T22:06:35.355371+0000","last_active":"2024-09-15T22:07:26.756488+0000","last_peered":"2024-09-15T22:07:26.756488+0000","last_clean":"2024-09-15T22:07:26.756488+0000","last_became_active":"2024-09-15T22:06:35.354687+0000","last_became_peered":"2024-09-15T22:06:35.354687+0000","last_unstale":"2024-09-15T22:07:26.756488+0000","last_undegraded":"2024-09-15T22:07:26.756488+0000","last_fullsized":"2024-09-15T22:07:26.756488+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:27.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851369+0000","last_change":"2024-09-15T22:06:44.657929+0000","last_active":"2024-09-15T22:06:47.851369+0000","last_peered":"2024-09-15T22:06:47.851369+0000","last_clean":"2024-09-15T22:06:47.851369+0000","last_became_active":"2024-09-15T22:06:44.657519+0000","last_became_peered":"2024-09-15T22:06:44.657519+0000","last_unstale":"2024-09-15T22:06:47.851369+0000","last_undegraded":"2024-09-15T22:06:47.851369+0000","last_fullsized":"2024-09-15T22:06:47.851369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:12.313877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849506+0000","last_change":"2024-09-15T22:06:37.193288+0000","last_active":"2024-09-15T22:06:47.849506+0000","last_peered":"2024-09-15T22:06:47.849506+0000","last_clean":"2024-09-15T22:06:47.849506+0000","last_became_active":"2024-09-15T22:06:37.193007+0000","last_became_peered":"2024-09-15T22:06:37.193007+0000","last_unstale":"2024-09-15T22:06:47.849506+0000","last_undegraded":"2024-09-15T22:06:47.849506+0000","last_fullsized":"2024-09-15T22:06:47.849506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:45.956206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.734429+0000","last_change":"2024-09-15T22:06:35.138726+0000","last_active":"2024-09-15T22:07:13.734429+0000","last_peered":"2024-09-15T22:07:13.734429+0000","last_clean":"2024-09-15T22:07:13.734429+0000","last_became_active":"2024-09-15T22:06:35.138390+0000","last_became_peered":"2024-09-15T22:06:35.138390+0000","last_unstale":"2024-09-15T22:07:13.734429+0000","last_undegraded":"2024-09-15T22:07:13.734429+0000","last_fullsized":"2024-09-15T22:07:13.734429+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:51.336171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851139+0000","last_change":"2024-09-15T22:06:44.656811+0000","last_active":"2024-09-15T22:06:47.851139+0000","last_peered":"2024-09-15T22:06:47.851139+0000","last_clean":"2024-09-15T22:06:47.851139+0000","last_became_active":"2024-09-15T22:06:44.656381+0000","last_became_peered":"2024-09-15T22:06:44.656381+0000","last_unstale":"2024-09-15T22:06:47.851139+0000","last_undegraded":"2024-09-15T22:06:47.851139+0000","last_fullsized":"2024-09-15T22:06:47.851139+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:15.219637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645187+0000","last_change":"2024-09-15T22:06:37.176478+0000","last_active":"2024-09-15T22:06:47.645187+0000","last_peered":"2024-09-15T22:06:47.645187+0000","last_clean":"2024-09-15T22:06:47.645187+0000","last_became_active":"2024-09-15T22:06:37.176171+0000","last_became_peered":"2024-09-15T22:06:37.176171+0000","last_unstale":"2024-09-15T22:06:47.645187+0000","last_undegraded":"2024-09-15T22:06:47.645187+0000","last_fullsized":"2024-09-15T22:06:47.645187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:14.735811+0000","last_change":"2024-09-15T22:06:35.134651+0000","last_active":"2024-09-15T22:07:14.735811+0000","last_peered":"2024-09-15T22:07:14.735811+0000","last_clean":"2024-09-15T22:07:14.735811+0000","last_became_active":"2024-09-15T22:06:35.134185+0000","last_became_peered":"2024-09-15T22:06:35.134185+0000","last_unstale":"2024-09-15T22:07:14.735811+0000","last_undegraded":"2024-09-15T22:07:14.735811+0000","last_fullsized":"2024-09-15T22:07:14.735811+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:56.385046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":351,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851362+0000","last_change":"2024-09-15T22:05:23.048068+0000","last_active":"2024-09-15T22:06:47.851362+0000","last_peered":"2024-09-15T22:06:47.851362+0000","last_clean":"2024-09-15T22:06:47.851362+0000","last_became_active":"2024-09-15T22:05:23.047692+0000","last_became_peered":"2024-09-15T22:05:23.047692+0000","last_unstale":"2024-09-15T22:06:47.851362+0000","last_undegraded":"2024-09-15T22:06:47.851362+0000","last_fullsized":"2024-09-15T22:06:47.851362+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_clean_scrub_stamp":"2024-09-15T22:01:57.477629+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850994+0000","last_change":"2024-09-15T22:06:44.657213+0000","last_active":"2024-09-15T22:06:47.850994+0000","last_peered":"2024-09-15T22:06:47.850994+0000","last_clean":"2024-09-15T22:06:47.850994+0000","last_became_active":"2024-09-15T22:06:44.656815+0000","last_became_peered":"2024-09-15T22:06:44.656815+0000","last_unstale":"2024-09-15T22:06:47.850994+0000","last_undegraded":"2024-09-15T22:06:47.850994+0000","last_fullsized":"2024-09-15T22:06:47.850994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:26.525992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851058+0000","last_change":"2024-09-15T22:06:37.184467+0000","last_active":"2024-09-15T22:06:47.851058+0000","last_peered":"2024-09-15T22:06:47.851058+0000","last_clean":"2024-09-15T22:06:47.851058+0000","last_became_active":"2024-09-15T22:06:37.183864+0000","last_became_peered":"2024-09-15T22:06:37.183864+0000","last_unstale":"2024-09-15T22:06:47.851058+0000","last_undegraded":"2024-09-15T22:06:47.851058+0000","last_fullsized":"2024-09-15T22:06:47.851058+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:35.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.737313+0000","last_change":"2024-09-15T22:06:44.651594+0000","last_active":"2024-09-15T22:07:13.737313+0000","last_peered":"2024-09-15T22:07:13.737313+0000","last_clean":"2024-09-15T22:07:13.737313+0000","last_became_active":"2024-09-15T22:06:44.650393+0000","last_became_peered":"2024-09-15T22:06:44.650393+0000","last_unstale":"2024-09-15T22:07:13.737313+0000","last_undegraded":"2024-09-15T22:07:13.737313+0000","last_fullsized":"2024-09-15T22:07:13.737313+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:13.599744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851420+0000","last_change":"2024-09-15T22:06:44.660789+0000","last_active":"2024-09-15T22:06:47.851420+0000","last_peered":"2024-09-15T22:06:47.851420+0000","last_clean":"2024-09-15T22:06:47.851420+0000","last_became_active":"2024-09-15T22:06:44.660331+0000","last_became_peered":"2024-09-15T22:06:44.660331+0000","last_unstale":"2024-09-15T22:06:47.851420+0000","last_undegraded":"2024-09-15T22:06:47.851420+0000","last_fullsized":"2024-09-15T22:06:47.851420+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:04:45.562795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850373+0000","last_change":"2024-09-15T22:06:37.185697+0000","last_active":"2024-09-15T22:06:47.850373+0000","last_peered":"2024-09-15T22:06:47.850373+0000","last_clean":"2024-09-15T22:06:47.850373+0000","last_became_active":"2024-09-15T22:06:37.185207+0000","last_became_peered":"2024-09-15T22:06:37.185207+0000","last_unstale":"2024-09-15T22:06:47.850373+0000","last_undegraded":"2024-09-15T22:06:47.850373+0000","last_fullsized":"2024-09-15T22:06:47.850373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:39.005273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851200+0000","last_change":"2024-09-15T22:06:35.135176+0000","last_active":"2024-09-15T22:06:47.851200+0000","last_peered":"2024-09-15T22:06:47.851200+0000","last_clean":"2024-09-15T22:06:47.851200+0000","last_became_active":"2024-09-15T22:06:35.134649+0000","last_became_peered":"2024-09-15T22:06:35.134649+0000","last_unstale":"2024-09-15T22:06:47.851200+0000","last_undegraded":"2024-09-15T22:06:47.851200+0000","last_fullsized":"2024-09-15T22:06:47.851200+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:33.780162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850486+0000","last_change":"2024-09-15T22:06:44.656972+0000","last_active":"2024-09-15T22:06:47.850486+0000","last_peered":"2024-09-15T22:06:47.850486+0000","last_clean":"2024-09-15T22:06:47.850486+0000","last_became_active":"2024-09-15T22:06:44.656528+0000","last_became_peered":"2024-09-15T22:06:44.656528+0000","last_unstale":"2024-09-15T22:06:47.850486+0000","last_undegraded":"2024-09-15T22:06:47.850486+0000","last_fullsized":"2024-09-15T22:06:47.850486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:09.164596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850793+0000","last_change":"2024-09-15T22:06:37.186118+0000","last_active":"2024-09-15T22:06:47.850793+0000","last_peered":"2024-09-15T22:06:47.850793+0000","last_clean":"2024-09-15T22:06:47.850793+0000","last_became_active":"2024-09-15T22:06:37.185118+0000","last_became_peered":"2024-09-15T22:06:37.185118+0000","last_unstale":"2024-09-15T22:06:47.850793+0000","last_undegraded":"2024-09-15T22:06:47.850793+0000","last_fullsized":"2024-09-15T22:06:47.850793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:53.357045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851347+0000","last_change":"2024-09-15T22:06:35.133341+0000","last_active":"2024-09-15T22:06:47.851347+0000","last_peered":"2024-09-15T22:06:47.851347+0000","last_clean":"2024-09-15T22:06:47.851347+0000","last_became_active":"2024-09-15T22:06:35.132961+0000","last_became_peered":"2024-09-15T22:06:35.132961+0000","last_unstale":"2024-09-15T22:06:47.851347+0000","last_undegraded":"2024-09-15T22:06:47.851347+0000","last_fullsized":"2024-09-15T22:06:47.851347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:39.579470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849894+0000","last_change":"2024-09-15T22:06:44.659148+0000","last_active":"2024-09-15T22:06:47.849894+0000","last_peered":"2024-09-15T22:06:47.849894+0000","last_clean":"2024-09-15T22:06:47.849894+0000","last_became_active":"2024-09-15T22:06:44.658750+0000","last_became_peered":"2024-09-15T22:06:44.658750+0000","last_unstale":"2024-09-15T22:06:47.849894+0000","last_undegraded":"2024-09-15T22:06:47.849894+0000","last_fullsized":"2024-09-15T22:06:47.849894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:48.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850697+0000","last_change":"2024-09-15T22:06:37.194066+0000","last_active":"2024-09-15T22:06:47.850697+0000","last_peered":"2024-09-15T22:06:47.850697+0000","last_clean":"2024-09-15T22:06:47.850697+0000","last_became_active":"2024-09-15T22:06:37.193738+0000","last_became_peered":"2024-09-15T22:06:37.193738+0000","last_unstale":"2024-09-15T22:06:47.850697+0000","last_undegraded":"2024-09-15T22:06:47.850697+0000","last_fullsized":"2024-09-15T22:06:47.850697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.107188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:14.731656+0000","last_change":"2024-09-15T22:06:35.136470+0000","last_active":"2024-09-15T22:07:14.731656+0000","last_peered":"2024-09-15T22:07:14.731656+0000","last_clean":"2024-09-15T22:07:14.731656+0000","last_became_active":"2024-09-15T22:06:35.135953+0000","last_became_peered":"2024-09-15T22:06:35.135953+0000","last_unstale":"2024-09-15T22:07:14.731656+0000","last_undegraded":"2024-09-15T22:07:14.731656+0000","last_fullsized":"2024-09-15T22:07:14.731656+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:44.548566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850399+0000","last_change":"2024-09-15T22:06:44.653682+0000","last_active":"2024-09-15T22:06:47.850399+0000","last_peered":"2024-09-15T22:06:47.850399+0000","last_clean":"2024-09-15T22:06:47.850399+0000","last_became_active":"2024-09-15T22:06:44.653303+0000","last_became_peered":"2024-09-15T22:06:44.653303+0000","last_unstale":"2024-09-15T22:06:47.850399+0000","last_undegraded":"2024-09-15T22:06:47.850399+0000","last_fullsized":"2024-09-15T22:06:47.850399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:07.847971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850520+0000","last_change":"2024-09-15T22:06:44.658436+0000","last_active":"2024-09-15T22:06:47.850520+0000","last_peered":"2024-09-15T22:06:47.850520+0000","last_clean":"2024-09-15T22:06:47.850520+0000","last_became_active":"2024-09-15T22:06:44.658037+0000","last_became_peered":"2024-09-15T22:06:44.658037+0000","last_unstale":"2024-09-15T22:06:47.850520+0000","last_undegraded":"2024-09-15T22:06:47.850520+0000","last_fullsized":"2024-09-15T22:06:47.850520+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:34:20.369249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":46,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.735911+0000","last_change":"2024-09-15T22:06:35.358018+0000","last_active":"2024-09-15T22:07:13.735911+0000","last_peered":"2024-09-15T22:07:13.735911+0000","last_clean":"2024-09-15T22:07:13.735911+0000","last_became_active":"2024-09-15T22:06:35.357666+0000","last_became_peered":"2024-09-15T22:06:35.357666+0000","last_unstale":"2024-09-15T22:07:13.735911+0000","last_undegraded":"2024-09-15T22:07:13.735911+0000","last_fullsized":"2024-09-15T22:07:13.735911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:34.089113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851395+0000","last_change":"2024-09-15T22:06:44.656807+0000","last_active":"2024-09-15T22:06:47.851395+0000","last_peered":"2024-09-15T22:06:47.851395+0000","last_clean":"2024-09-15T22:06:47.851395+0000","last_became_active":"2024-09-15T22:06:44.656409+0000","last_became_peered":"2024-09-15T22:06:44.656409+0000","last_unstale":"2024-09-15T22:06:47.851395+0000","last_undegraded":"2024-09-15T22:06:47.851395+0000","last_fullsized":"2024-09-15T22:06:47.851395+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850035+0000","last_change":"2024-09-15T22:06:37.180615+0000","last_active":"2024-09-15T22:06:47.850035+0000","last_peered":"2024-09-15T22:06:47.850035+0000","last_clean":"2024-09-15T22:06:47.850035+0000","last_became_active":"2024-09-15T22:06:37.180306+0000","last_became_peered":"2024-09-15T22:06:37.180306+0000","last_unstale":"2024-09-15T22:06:47.850035+0000","last_undegraded":"2024-09-15T22:06:47.850035+0000","last_fullsized":"2024-09-15T22:06:47.850035+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:31.145128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":51,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:26.756977+0000","last_change":"2024-09-15T22:06:35.136192+0000","last_active":"2024-09-15T22:07:26.756977+0000","last_peered":"2024-09-15T22:07:26.756977+0000","last_clean":"2024-09-15T22:07:26.756977+0000","last_became_active":"2024-09-15T22:06:35.135608+0000","last_became_peered":"2024-09-15T22:06:35.135608+0000","last_unstale":"2024-09-15T22:07:26.756977+0000","last_undegraded":"2024-09-15T22:07:26.756977+0000","last_fullsized":"2024-09-15T22:07:26.756977+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:48.472717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850695+0000","last_change":"2024-09-15T22:06:44.664763+0000","last_active":"2024-09-15T22:06:47.850695+0000","last_peered":"2024-09-15T22:06:47.850695+0000","last_clean":"2024-09-15T22:06:47.850695+0000","last_became_active":"2024-09-15T22:06:44.664410+0000","last_became_peered":"2024-09-15T22:06:44.664410+0000","last_unstale":"2024-09-15T22:06:47.850695+0000","last_undegraded":"2024-09-15T22:06:47.850695+0000","last_fullsized":"2024-09-15T22:06:47.850695+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:14.364387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851090+0000","last_change":"2024-09-15T22:06:44.652747+0000","last_active":"2024-09-15T22:06:47.851090+0000","last_peered":"2024-09-15T22:06:47.851090+0000","last_clean":"2024-09-15T22:06:47.851090+0000","last_became_active":"2024-09-15T22:06:44.652368+0000","last_became_peered":"2024-09-15T22:06:44.652368+0000","last_unstale":"2024-09-15T22:06:47.851090+0000","last_undegraded":"2024-09-15T22:06:47.851090+0000","last_fullsized":"2024-09-15T22:06:47.851090+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:51.047549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.729356+0000","last_change":"2024-09-15T22:06:35.136311+0000","last_active":"2024-09-15T22:07:12.729356+0000","last_peered":"2024-09-15T22:07:12.729356+0000","last_clean":"2024-09-15T22:07:12.729356+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:07:12.729356+0000","last_undegraded":"2024-09-15T22:07:12.729356+0000","last_fullsized":"2024-09-15T22:07:12.729356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:36:07.138056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850773+0000","last_change":"2024-09-15T22:06:44.653747+0000","last_active":"2024-09-15T22:06:47.850773+0000","last_peered":"2024-09-15T22:06:47.850773+0000","last_clean":"2024-09-15T22:06:47.850773+0000","last_became_active":"2024-09-15T22:06:44.653357+0000","last_became_peered":"2024-09-15T22:06:44.653357+0000","last_unstale":"2024-09-15T22:06:47.850773+0000","last_undegraded":"2024-09-15T22:06:47.850773+0000","last_fullsized":"2024-09-15T22:06:47.850773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.113924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850070+0000","last_change":"2024-09-15T22:06:37.178308+0000","last_active":"2024-09-15T22:06:47.850070+0000","last_peered":"2024-09-15T22:06:47.850070+0000","last_clean":"2024-09-15T22:06:47.850070+0000","last_became_active":"2024-09-15T22:06:37.177920+0000","last_became_peered":"2024-09-15T22:06:37.177920+0000","last_unstale":"2024-09-15T22:06:47.850070+0000","last_undegraded":"2024-09-15T22:06:47.850070+0000","last_fullsized":"2024-09-15T22:06:47.850070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":37,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.735092+0000","last_change":"2024-09-15T22:06:35.136132+0000","last_active":"2024-09-15T22:07:13.735092+0000","last_peered":"2024-09-15T22:07:13.735092+0000","last_clean":"2024-09-15T22:07:13.735092+0000","last_became_active":"2024-09-15T22:06:35.135789+0000","last_became_peered":"2024-09-15T22:06:35.135789+0000","last_unstale":"2024-09-15T22:07:13.735092+0000","last_undegraded":"2024-09-15T22:07:13.735092+0000","last_fullsized":"2024-09-15T22:07:13.735092+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:01.692885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850765+0000","last_change":"2024-09-15T22:06:44.658920+0000","last_active":"2024-09-15T22:06:47.850765+0000","last_peered":"2024-09-15T22:06:47.850765+0000","last_clean":"2024-09-15T22:06:47.850765+0000","last_became_active":"2024-09-15T22:06:44.658434+0000","last_became_peered":"2024-09-15T22:06:44.658434+0000","last_unstale":"2024-09-15T22:06:47.850765+0000","last_undegraded":"2024-09-15T22:06:47.850765+0000","last_fullsized":"2024-09-15T22:06:47.850765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:37.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645847+0000","last_change":"2024-09-15T22:06:37.184958+0000","last_active":"2024-09-15T22:06:47.645847+0000","last_peered":"2024-09-15T22:06:47.645847+0000","last_clean":"2024-09-15T22:06:47.645847+0000","last_became_active":"2024-09-15T22:06:37.184648+0000","last_became_peered":"2024-09-15T22:06:37.184648+0000","last_unstale":"2024-09-15T22:06:47.645847+0000","last_undegraded":"2024-09-15T22:06:47.645847+0000","last_fullsized":"2024-09-15T22:06:47.645847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:28.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":99,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:29.763379+0000","last_change":"2024-09-15T22:06:35.138533+0000","last_active":"2024-09-15T22:07:29.763379+0000","last_peered":"2024-09-15T22:07:29.763379+0000","last_clean":"2024-09-15T22:07:29.763379+0000","last_became_active":"2024-09-15T22:06:35.138142+0000","last_became_peered":"2024-09-15T22:06:35.138142+0000","last_unstale":"2024-09-15T22:07:29.763379+0000","last_undegraded":"2024-09-15T22:07:29.763379+0000","last_fullsized":"2024-09-15T22:07:29.763379+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:32.950036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52408,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":70,"num_write":31,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850060+0000","last_change":"2024-09-15T22:06:44.659773+0000","last_active":"2024-09-15T22:06:47.850060+0000","last_peered":"2024-09-15T22:06:47.850060+0000","last_clean":"2024-09-15T22:06:47.850060+0000","last_became_active":"2024-09-15T22:06:44.659374+0000","last_became_peered":"2024-09-15T22:06:44.659374+0000","last_unstale":"2024-09-15T22:06:47.850060+0000","last_undegraded":"2024-09-15T22:06:47.850060+0000","last_fullsized":"2024-09-15T22:06:47.850060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:13.491116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851697+0000","last_change":"2024-09-15T22:06:37.194297+0000","last_active":"2024-09-15T22:06:47.851697+0000","last_peered":"2024-09-15T22:06:47.851697+0000","last_clean":"2024-09-15T22:06:47.851697+0000","last_became_active":"2024-09-15T22:06:37.193986+0000","last_became_peered":"2024-09-15T22:06:37.193986+0000","last_unstale":"2024-09-15T22:06:47.851697+0000","last_undegraded":"2024-09-15T22:06:47.851697+0000","last_fullsized":"2024-09-15T22:06:47.851697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:27.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.728943+0000","last_change":"2024-09-15T22:06:35.356537+0000","last_active":"2024-09-15T22:07:12.728943+0000","last_peered":"2024-09-15T22:07:12.728943+0000","last_clean":"2024-09-15T22:07:12.728943+0000","last_became_active":"2024-09-15T22:06:35.355997+0000","last_became_peered":"2024-09-15T22:06:35.355997+0000","last_unstale":"2024-09-15T22:07:12.728943+0000","last_undegraded":"2024-09-15T22:07:12.728943+0000","last_fullsized":"2024-09-15T22:07:12.728943+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:11.563083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646787+0000","last_change":"2024-09-15T22:06:44.654579+0000","last_active":"2024-09-15T22:06:47.646787+0000","last_peered":"2024-09-15T22:06:47.646787+0000","last_clean":"2024-09-15T22:06:47.646787+0000","last_became_active":"2024-09-15T22:06:44.654187+0000","last_became_peered":"2024-09-15T22:06:44.654187+0000","last_unstale":"2024-09-15T22:06:47.646787+0000","last_undegraded":"2024-09-15T22:06:47.646787+0000","last_fullsized":"2024-09-15T22:06:47.646787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:15.617528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851150+0000","last_change":"2024-09-15T22:06:44.661533+0000","last_active":"2024-09-15T22:06:47.851150+0000","last_peered":"2024-09-15T22:06:47.851150+0000","last_clean":"2024-09-15T22:06:47.851150+0000","last_became_active":"2024-09-15T22:06:44.661193+0000","last_became_peered":"2024-09-15T22:06:44.661193+0000","last_unstale":"2024-09-15T22:06:47.851150+0000","last_undegraded":"2024-09-15T22:06:47.851150+0000","last_fullsized":"2024-09-15T22:06:47.851150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:06.559720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:12.727278+0000","last_change":"2024-09-15T22:06:35.140578+0000","last_active":"2024-09-15T22:07:12.727278+0000","last_peered":"2024-09-15T22:07:12.727278+0000","last_clean":"2024-09-15T22:07:12.727278+0000","last_became_active":"2024-09-15T22:06:35.139818+0000","last_became_peered":"2024-09-15T22:06:35.139818+0000","last_unstale":"2024-09-15T22:07:12.727278+0000","last_undegraded":"2024-09-15T22:07:12.727278+0000","last_fullsized":"2024-09-15T22:07:12.727278+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:59.806964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850716+0000","last_change":"2024-09-15T22:06:44.654755+0000","last_active":"2024-09-15T22:06:47.850716+0000","last_peered":"2024-09-15T22:06:47.850716+0000","last_clean":"2024-09-15T22:06:47.850716+0000","last_became_active":"2024-09-15T22:06:44.654390+0000","last_became_peered":"2024-09-15T22:06:44.654390+0000","last_unstale":"2024-09-15T22:06:47.850716+0000","last_undegraded":"2024-09-15T22:06:47.850716+0000","last_fullsized":"2024-09-15T22:06:47.850716+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:35.781878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851689+0000","last_change":"2024-09-15T22:06:37.182457+0000","last_active":"2024-09-15T22:06:47.851689+0000","last_peered":"2024-09-15T22:06:47.851689+0000","last_clean":"2024-09-15T22:06:47.851689+0000","last_became_active":"2024-09-15T22:06:37.182106+0000","last_became_peered":"2024-09-15T22:06:37.182106+0000","last_unstale":"2024-09-15T22:06:47.851689+0000","last_undegraded":"2024-09-15T22:06:47.851689+0000","last_fullsized":"2024-09-15T22:06:47.851689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:57.560894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":49,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:28.759821+0000","last_change":"2024-09-15T22:06:35.140590+0000","last_active":"2024-09-15T22:07:28.759821+0000","last_peered":"2024-09-15T22:07:28.759821+0000","last_clean":"2024-09-15T22:07:28.759821+0000","last_became_active":"2024-09-15T22:06:35.140198+0000","last_became_peered":"2024-09-15T22:06:35.140198+0000","last_unstale":"2024-09-15T22:07:28.759821+0000","last_undegraded":"2024-09-15T22:07:28.759821+0000","last_fullsized":"2024-09-15T22:07:28.759821+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:46.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.644921+0000","last_change":"2024-09-15T22:06:44.660188+0000","last_active":"2024-09-15T22:06:47.644921+0000","last_peered":"2024-09-15T22:06:47.644921+0000","last_clean":"2024-09-15T22:06:47.644921+0000","last_became_active":"2024-09-15T22:06:44.659808+0000","last_became_peered":"2024-09-15T22:06:44.659808+0000","last_unstale":"2024-09-15T22:06:47.644921+0000","last_undegraded":"2024-09-15T22:06:47.644921+0000","last_fullsized":"2024-09-15T22:06:47.644921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:39.473765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850703+0000","last_change":"2024-09-15T22:06:44.647967+0000","last_active":"2024-09-15T22:06:47.850703+0000","last_peered":"2024-09-15T22:06:47.850703+0000","last_clean":"2024-09-15T22:06:47.850703+0000","last_became_active":"2024-09-15T22:06:44.647608+0000","last_became_peered":"2024-09-15T22:06:44.647608+0000","last_unstale":"2024-09-15T22:06:47.850703+0000","last_undegraded":"2024-09-15T22:06:47.850703+0000","last_fullsized":"2024-09-15T22:06:47.850703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:06.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850435+0000","last_change":"2024-09-15T22:06:37.187020+0000","last_active":"2024-09-15T22:06:47.850435+0000","last_peered":"2024-09-15T22:06:47.850435+0000","last_clean":"2024-09-15T22:06:47.850435+0000","last_became_active":"2024-09-15T22:06:37.186564+0000","last_became_peered":"2024-09-15T22:06:37.186564+0000","last_unstale":"2024-09-15T22:06:47.850435+0000","last_undegraded":"2024-09-15T22:06:47.850435+0000","last_fullsized":"2024-09-15T22:06:47.850435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:20.586763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850053+0000","last_change":"2024-09-15T22:06:35.135126+0000","last_active":"2024-09-15T22:06:47.850053+0000","last_peered":"2024-09-15T22:06:47.850053+0000","last_clean":"2024-09-15T22:06:47.850053+0000","last_became_active":"2024-09-15T22:06:35.134783+0000","last_became_peered":"2024-09-15T22:06:35.134783+0000","last_unstale":"2024-09-15T22:06:47.850053+0000","last_undegraded":"2024-09-15T22:06:47.850053+0000","last_fullsized":"2024-09-15T22:06:47.850053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:31.910988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850222+0000","last_change":"2024-09-15T22:06:44.656981+0000","last_active":"2024-09-15T22:06:47.850222+0000","last_peered":"2024-09-15T22:06:47.850222+0000","last_clean":"2024-09-15T22:06:47.850222+0000","last_became_active":"2024-09-15T22:06:44.656477+0000","last_became_peered":"2024-09-15T22:06:44.656477+0000","last_unstale":"2024-09-15T22:06:47.850222+0000","last_undegraded":"2024-09-15T22:06:47.850222+0000","last_fullsized":"2024-09-15T22:06:47.850222+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:06.153490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.644921+0000","last_change":"2024-09-15T22:06:37.183034+0000","last_active":"2024-09-15T22:06:47.644921+0000","last_peered":"2024-09-15T22:06:47.644921+0000","last_clean":"2024-09-15T22:06:47.644921+0000","last_became_active":"2024-09-15T22:06:37.182734+0000","last_became_peered":"2024-09-15T22:06:37.182734+0000","last_unstale":"2024-09-15T22:06:47.644921+0000","last_undegraded":"2024-09-15T22:06:47.644921+0000","last_fullsized":"2024-09-15T22:06:47.644921+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:19.352000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":32,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.736235+0000","last_change":"2024-09-15T22:06:35.134954+0000","last_active":"2024-09-15T22:07:13.736235+0000","last_peered":"2024-09-15T22:07:13.736235+0000","last_clean":"2024-09-15T22:07:13.736235+0000","last_became_active":"2024-09-15T22:06:35.134583+0000","last_became_peered":"2024-09-15T22:06:35.134583+0000","last_unstale":"2024-09-15T22:07:13.736235+0000","last_undegraded":"2024-09-15T22:07:13.736235+0000","last_fullsized":"2024-09-15T22:07:13.736235+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:56.082123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:13.735087+0000","last_change":"2024-09-15T22:06:35.139229+0000","last_active":"2024-09-15T22:07:13.735087+0000","last_peered":"2024-09-15T22:07:13.735087+0000","last_clean":"2024-09-15T22:07:13.735087+0000","last_became_active":"2024-09-15T22:06:35.138834+0000","last_became_peered":"2024-09-15T22:06:35.138834+0000","last_unstale":"2024-09-15T22:07:13.735087+0000","last_undegraded":"2024-09-15T22:07:13.735087+0000","last_fullsized":"2024-09-15T22:07:13.735087+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:05.877404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850737+0000","last_change":"2024-09-15T22:06:37.192212+0000","last_active":"2024-09-15T22:06:47.850737+0000","last_peered":"2024-09-15T22:06:47.850737+0000","last_clean":"2024-09-15T22:06:47.850737+0000","last_became_active":"2024-09-15T22:06:37.191872+0000","last_became_peered":"2024-09-15T22:06:37.191872+0000","last_unstale":"2024-09-15T22:06:47.850737+0000","last_undegraded":"2024-09-15T22:06:47.850737+0000","last_fullsized":"2024-09-15T22:06:47.850737+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:32.845794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849819+0000","last_change":"2024-09-15T22:06:44.661976+0000","last_active":"2024-09-15T22:06:47.849819+0000","last_peered":"2024-09-15T22:06:47.849819+0000","last_clean":"2024-09-15T22:06:47.849819+0000","last_became_active":"2024-09-15T22:06:44.661169+0000","last_became_peered":"2024-09-15T22:06:44.661169+0000","last_unstale":"2024-09-15T22:06:47.849819+0000","last_undegraded":"2024-09-15T22:06:47.849819+0000","last_fullsized":"2024-09-15T22:06:47.849819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:25:03.930064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":52,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:07:25.755869+0000","last_change":"2024-09-15T22:06:35.133846+0000","last_active":"2024-09-15T22:07:25.755869+0000","last_peered":"2024-09-15T22:07:25.755869+0000","last_clean":"2024-09-15T22:07:25.755869+0000","last_became_active":"2024-09-15T22:06:35.133516+0000","last_became_peered":"2024-09-15T22:06:35.133516+0000","last_unstale":"2024-09-15T22:07:25.755869+0000","last_undegraded":"2024-09-15T22:07:25.755869+0000","last_fullsized":"2024-09-15T22:07:25.755869+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:29.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645631+0000","last_change":"2024-09-15T22:06:37.182904+0000","last_active":"2024-09-15T22:06:47.645631+0000","last_peered":"2024-09-15T22:06:47.645631+0000","last_clean":"2024-09-15T22:06:47.645631+0000","last_became_active":"2024-09-15T22:06:37.182563+0000","last_became_peered":"2024-09-15T22:06:37.182563+0000","last_unstale":"2024-09-15T22:06:47.645631+0000","last_undegraded":"2024-09-15T22:06:47.645631+0000","last_fullsized":"2024-09-15T22:06:47.645631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850024+0000","last_change":"2024-09-15T22:06:44.660997+0000","last_active":"2024-09-15T22:06:47.850024+0000","last_peered":"2024-09-15T22:06:47.850024+0000","last_clean":"2024-09-15T22:06:47.850024+0000","last_became_active":"2024-09-15T22:06:44.660577+0000","last_became_peered":"2024-09-15T22:06:44.660577+0000","last_unstale":"2024-09-15T22:06:47.850024+0000","last_undegraded":"2024-09-15T22:06:47.850024+0000","last_fullsized":"2024-09-15T22:06:47.850024+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:56.006891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850176+0000","last_change":"2024-09-15T22:06:37.182077+0000","last_active":"2024-09-15T22:06:47.850176+0000","last_peered":"2024-09-15T22:06:47.850176+0000","last_clean":"2024-09-15T22:06:47.850176+0000","last_became_active":"2024-09-15T22:06:37.181622+0000","last_became_peered":"2024-09-15T22:06:37.181622+0000","last_unstale":"2024-09-15T22:06:47.850176+0000","last_undegraded":"2024-09-15T22:06:47.850176+0000","last_fullsized":"2024-09-15T22:06:47.850176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:07.586406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849694+0000","last_change":"2024-09-15T22:06:44.653845+0000","last_active":"2024-09-15T22:06:47.849694+0000","last_peered":"2024-09-15T22:06:47.849694+0000","last_clean":"2024-09-15T22:06:47.849694+0000","last_became_active":"2024-09-15T22:06:44.653483+0000","last_became_peered":"2024-09-15T22:06:44.653483+0000","last_unstale":"2024-09-15T22:06:47.849694+0000","last_undegraded":"2024-09-15T22:06:47.849694+0000","last_fullsized":"2024-09-15T22:06:47.849694+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:08:07.822279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849449+0000","last_change":"2024-09-15T22:06:37.187476+0000","last_active":"2024-09-15T22:06:47.849449+0000","last_peered":"2024-09-15T22:06:47.849449+0000","last_clean":"2024-09-15T22:06:47.849449+0000","last_became_active":"2024-09-15T22:06:37.187047+0000","last_became_peered":"2024-09-15T22:06:37.187047+0000","last_unstale":"2024-09-15T22:06:47.849449+0000","last_undegraded":"2024-09-15T22:06:47.849449+0000","last_fullsized":"2024-09-15T22:06:47.849449+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.220834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850466+0000","last_change":"2024-09-15T22:06:37.190583+0000","last_active":"2024-09-15T22:06:47.850466+0000","last_peered":"2024-09-15T22:06:47.850466+0000","last_clean":"2024-09-15T22:06:47.850466+0000","last_became_active":"2024-09-15T22:06:37.190212+0000","last_became_peered":"2024-09-15T22:06:37.190212+0000","last_unstale":"2024-09-15T22:06:47.850466+0000","last_undegraded":"2024-09-15T22:06:47.850466+0000","last_fullsized":"2024-09-15T22:06:47.850466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:48.574188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850185+0000","last_change":"2024-09-15T22:06:37.192508+0000","last_active":"2024-09-15T22:06:47.850185+0000","last_peered":"2024-09-15T22:06:47.850185+0000","last_clean":"2024-09-15T22:06:47.850185+0000","last_became_active":"2024-09-15T22:06:37.192117+0000","last_became_peered":"2024-09-15T22:06:37.192117+0000","last_unstale":"2024-09-15T22:06:47.850185+0000","last_undegraded":"2024-09-15T22:06:47.850185+0000","last_fullsized":"2024-09-15T22:06:47.850185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:41.355966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850817+0000","last_change":"2024-09-15T22:06:37.183377+0000","last_active":"2024-09-15T22:06:47.850817+0000","last_peered":"2024-09-15T22:06:47.850817+0000","last_clean":"2024-09-15T22:06:47.850817+0000","last_became_active":"2024-09-15T22:06:37.183072+0000","last_became_peered":"2024-09-15T22:06:37.183072+0000","last_unstale":"2024-09-15T22:06:47.850817+0000","last_undegraded":"2024-09-15T22:06:47.850817+0000","last_fullsized":"2024-09-15T22:06:47.850817+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.318702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850033+0000","last_change":"2024-09-15T22:06:37.185348+0000","last_active":"2024-09-15T22:06:47.850033+0000","last_peered":"2024-09-15T22:06:47.850033+0000","last_clean":"2024-09-15T22:06:47.850033+0000","last_became_active":"2024-09-15T22:06:37.184934+0000","last_became_peered":"2024-09-15T22:06:37.184934+0000","last_unstale":"2024-09-15T22:06:47.850033+0000","last_undegraded":"2024-09-15T22:06:47.850033+0000","last_fullsized":"2024-09-15T22:06:47.850033+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:57.564860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645548+0000","last_change":"2024-09-15T22:06:37.192152+0000","last_active":"2024-09-15T22:06:47.645548+0000","last_peered":"2024-09-15T22:06:47.645548+0000","last_clean":"2024-09-15T22:06:47.645548+0000","last_became_active":"2024-09-15T22:06:37.191836+0000","last_became_peered":"2024-09-15T22:06:37.191836+0000","last_unstale":"2024-09-15T22:06:47.645548+0000","last_undegraded":"2024-09-15T22:06:47.645548+0000","last_fullsized":"2024-09-15T22:06:47.645548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:31.781110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849758+0000","last_change":"2024-09-15T22:06:37.179996+0000","last_active":"2024-09-15T22:06:47.849758+0000","last_peered":"2024-09-15T22:06:47.849758+0000","last_clean":"2024-09-15T22:06:47.849758+0000","last_became_active":"2024-09-15T22:06:37.179646+0000","last_became_peered":"2024-09-15T22:06:37.179646+0000","last_unstale":"2024-09-15T22:06:47.849758+0000","last_undegraded":"2024-09-15T22:06:47.849758+0000","last_fullsized":"2024-09-15T22:06:47.849758+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:35.240468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.644930+0000","last_change":"2024-09-15T22:06:37.182992+0000","last_active":"2024-09-15T22:06:47.644930+0000","last_peered":"2024-09-15T22:06:47.644930+0000","last_clean":"2024-09-15T22:06:47.644930+0000","last_became_active":"2024-09-15T22:06:37.182676+0000","last_became_peered":"2024-09-15T22:06:37.182676+0000","last_unstale":"2024-09-15T22:06:47.644930+0000","last_undegraded":"2024-09-15T22:06:47.644930+0000","last_fullsized":"2024-09-15T22:06:47.644930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:41.524600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851507+0000","last_change":"2024-09-15T22:06:37.180107+0000","last_active":"2024-09-15T22:06:47.851507+0000","last_peered":"2024-09-15T22:06:47.851507+0000","last_clean":"2024-09-15T22:06:47.851507+0000","last_became_active":"2024-09-15T22:06:37.179785+0000","last_became_peered":"2024-09-15T22:06:37.179785+0000","last_unstale":"2024-09-15T22:06:47.851507+0000","last_undegraded":"2024-09-15T22:06:47.851507+0000","last_fullsized":"2024-09-15T22:06:47.851507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:46.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850135+0000","last_change":"2024-09-15T22:06:37.177406+0000","last_active":"2024-09-15T22:06:47.850135+0000","last_peered":"2024-09-15T22:06:47.850135+0000","last_clean":"2024-09-15T22:06:47.850135+0000","last_became_active":"2024-09-15T22:06:37.177093+0000","last_became_peered":"2024-09-15T22:06:37.177093+0000","last_unstale":"2024-09-15T22:06:47.850135+0000","last_undegraded":"2024-09-15T22:06:47.850135+0000","last_fullsized":"2024-09-15T22:06:47.850135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:34.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850683+0000","last_change":"2024-09-15T22:06:37.192314+0000","last_active":"2024-09-15T22:06:47.850683+0000","last_peered":"2024-09-15T22:06:47.850683+0000","last_clean":"2024-09-15T22:06:47.850683+0000","last_became_active":"2024-09-15T22:06:37.191958+0000","last_became_peered":"2024-09-15T22:06:37.191958+0000","last_unstale":"2024-09-15T22:06:47.850683+0000","last_undegraded":"2024-09-15T22:06:47.850683+0000","last_fullsized":"2024-09-15T22:06:47.850683+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:47.763742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850196+0000","last_change":"2024-09-15T22:06:37.190212+0000","last_active":"2024-09-15T22:06:47.850196+0000","last_peered":"2024-09-15T22:06:47.850196+0000","last_clean":"2024-09-15T22:06:47.850196+0000","last_became_active":"2024-09-15T22:06:37.189881+0000","last_became_peered":"2024-09-15T22:06:37.189881+0000","last_unstale":"2024-09-15T22:06:47.850196+0000","last_undegraded":"2024-09-15T22:06:47.850196+0000","last_fullsized":"2024-09-15T22:06:47.850196+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:59.943574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850343+0000","last_change":"2024-09-15T22:06:37.187967+0000","last_active":"2024-09-15T22:06:47.850343+0000","last_peered":"2024-09-15T22:06:47.850343+0000","last_clean":"2024-09-15T22:06:47.850343+0000","last_became_active":"2024-09-15T22:06:37.187596+0000","last_became_peered":"2024-09-15T22:06:37.187596+0000","last_unstale":"2024-09-15T22:06:47.850343+0000","last_undegraded":"2024-09-15T22:06:47.850343+0000","last_fullsized":"2024-09-15T22:06:47.850343+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:00.042784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850681+0000","last_change":"2024-09-15T22:06:37.189434+0000","last_active":"2024-09-15T22:06:47.850681+0000","last_peered":"2024-09-15T22:06:47.850681+0000","last_clean":"2024-09-15T22:06:47.850681+0000","last_became_active":"2024-09-15T22:06:37.189053+0000","last_became_peered":"2024-09-15T22:06:37.189053+0000","last_unstale":"2024-09-15T22:06:47.850681+0000","last_undegraded":"2024-09-15T22:06:47.850681+0000","last_fullsized":"2024-09-15T22:06:47.850681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:51.640495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850378+0000","last_change":"2024-09-15T22:06:37.179229+0000","last_active":"2024-09-15T22:06:47.850378+0000","last_peered":"2024-09-15T22:06:47.850378+0000","last_clean":"2024-09-15T22:06:47.850378+0000","last_became_active":"2024-09-15T22:06:37.178828+0000","last_became_peered":"2024-09-15T22:06:37.178828+0000","last_unstale":"2024-09-15T22:06:47.850378+0000","last_undegraded":"2024-09-15T22:06:47.850378+0000","last_fullsized":"2024-09-15T22:06:47.850378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:25.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.849946+0000","last_change":"2024-09-15T22:06:37.193095+0000","last_active":"2024-09-15T22:06:47.849946+0000","last_peered":"2024-09-15T22:06:47.849946+0000","last_clean":"2024-09-15T22:06:47.849946+0000","last_became_active":"2024-09-15T22:06:37.192801+0000","last_became_peered":"2024-09-15T22:06:37.192801+0000","last_unstale":"2024-09-15T22:06:47.849946+0000","last_undegraded":"2024-09-15T22:06:47.849946+0000","last_fullsized":"2024-09-15T22:06:47.849946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:12.521510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850198+0000","last_change":"2024-09-15T22:06:37.182332+0000","last_active":"2024-09-15T22:06:47.850198+0000","last_peered":"2024-09-15T22:06:47.850198+0000","last_clean":"2024-09-15T22:06:47.850198+0000","last_became_active":"2024-09-15T22:06:37.182001+0000","last_became_peered":"2024-09-15T22:06:37.182001+0000","last_unstale":"2024-09-15T22:06:47.850198+0000","last_undegraded":"2024-09-15T22:06:47.850198+0000","last_fullsized":"2024-09-15T22:06:47.850198+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:52.230452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850037+0000","last_change":"2024-09-15T22:06:37.180584+0000","last_active":"2024-09-15T22:06:47.850037+0000","last_peered":"2024-09-15T22:06:47.850037+0000","last_clean":"2024-09-15T22:06:47.850037+0000","last_became_active":"2024-09-15T22:06:37.179341+0000","last_became_peered":"2024-09-15T22:06:37.179341+0000","last_unstale":"2024-09-15T22:06:47.850037+0000","last_undegraded":"2024-09-15T22:06:47.850037+0000","last_fullsized":"2024-09-15T22:06:47.850037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.610835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646833+0000","last_change":"2024-09-15T22:06:37.189500+0000","last_active":"2024-09-15T22:06:47.646833+0000","last_peered":"2024-09-15T22:06:47.646833+0000","last_clean":"2024-09-15T22:06:47.646833+0000","last_became_active":"2024-09-15T22:06:37.189137+0000","last_became_peered":"2024-09-15T22:06:37.189137+0000","last_unstale":"2024-09-15T22:06:47.646833+0000","last_undegraded":"2024-09-15T22:06:47.646833+0000","last_fullsized":"2024-09-15T22:06:47.646833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:27.199854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850938+0000","last_change":"2024-09-15T22:06:37.182618+0000","last_active":"2024-09-15T22:06:47.850938+0000","last_peered":"2024-09-15T22:06:47.850938+0000","last_clean":"2024-09-15T22:06:47.850938+0000","last_became_active":"2024-09-15T22:06:37.182257+0000","last_became_peered":"2024-09-15T22:06:37.182257+0000","last_unstale":"2024-09-15T22:06:47.850938+0000","last_undegraded":"2024-09-15T22:06:47.850938+0000","last_fullsized":"2024-09-15T22:06:47.850938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:17.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851071+0000","last_change":"2024-09-15T22:06:37.193036+0000","last_active":"2024-09-15T22:06:47.851071+0000","last_peered":"2024-09-15T22:06:47.851071+0000","last_clean":"2024-09-15T22:06:47.851071+0000","last_became_active":"2024-09-15T22:06:37.192668+0000","last_became_peered":"2024-09-15T22:06:37.192668+0000","last_unstale":"2024-09-15T22:06:47.851071+0000","last_undegraded":"2024-09-15T22:06:47.851071+0000","last_fullsized":"2024-09-15T22:06:47.851071+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:05.436045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850535+0000","last_change":"2024-09-15T22:06:44.659503+0000","last_active":"2024-09-15T22:06:47.850535+0000","last_peered":"2024-09-15T22:06:47.850535+0000","last_clean":"2024-09-15T22:06:47.850535+0000","last_became_active":"2024-09-15T22:06:44.659155+0000","last_became_peered":"2024-09-15T22:06:44.659155+0000","last_unstale":"2024-09-15T22:06:47.850535+0000","last_undegraded":"2024-09-15T22:06:47.850535+0000","last_fullsized":"2024-09-15T22:06:47.850535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:41:04.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.644915+0000","last_change":"2024-09-15T22:06:37.178386+0000","last_active":"2024-09-15T22:06:47.644915+0000","last_peered":"2024-09-15T22:06:47.644915+0000","last_clean":"2024-09-15T22:06:47.644915+0000","last_became_active":"2024-09-15T22:06:37.178083+0000","last_became_peered":"2024-09-15T22:06:37.178083+0000","last_unstale":"2024-09-15T22:06:47.644915+0000","last_undegraded":"2024-09-15T22:06:47.644915+0000","last_fullsized":"2024-09-15T22:06:47.644915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:35.350698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851249+0000","last_change":"2024-09-15T22:06:37.186741+0000","last_active":"2024-09-15T22:06:47.851249+0000","last_peered":"2024-09-15T22:06:47.851249+0000","last_clean":"2024-09-15T22:06:47.851249+0000","last_became_active":"2024-09-15T22:06:37.185740+0000","last_became_peered":"2024-09-15T22:06:37.185740+0000","last_unstale":"2024-09-15T22:06:47.851249+0000","last_undegraded":"2024-09-15T22:06:47.851249+0000","last_fullsized":"2024-09-15T22:06:47.851249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.762407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850187+0000","last_change":"2024-09-15T22:06:37.189552+0000","last_active":"2024-09-15T22:06:47.850187+0000","last_peered":"2024-09-15T22:06:47.850187+0000","last_clean":"2024-09-15T22:06:47.850187+0000","last_became_active":"2024-09-15T22:06:37.189197+0000","last_became_peered":"2024-09-15T22:06:37.189197+0000","last_unstale":"2024-09-15T22:06:47.850187+0000","last_undegraded":"2024-09-15T22:06:47.850187+0000","last_fullsized":"2024-09-15T22:06:47.850187+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:54.268720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.645537+0000","last_change":"2024-09-15T22:06:37.186917+0000","last_active":"2024-09-15T22:06:47.645537+0000","last_peered":"2024-09-15T22:06:47.645537+0000","last_clean":"2024-09-15T22:06:47.645537+0000","last_became_active":"2024-09-15T22:06:37.186571+0000","last_became_peered":"2024-09-15T22:06:37.186571+0000","last_unstale":"2024-09-15T22:06:47.645537+0000","last_undegraded":"2024-09-15T22:06:47.645537+0000","last_fullsized":"2024-09-15T22:06:47.645537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:02.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.646824+0000","last_change":"2024-09-15T22:06:37.190960+0000","last_active":"2024-09-15T22:06:47.646824+0000","last_peered":"2024-09-15T22:06:47.646824+0000","last_clean":"2024-09-15T22:06:47.646824+0000","last_became_active":"2024-09-15T22:06:37.190650+0000","last_became_peered":"2024-09-15T22:06:37.190650+0000","last_unstale":"2024-09-15T22:06:47.646824+0000","last_undegraded":"2024-09-15T22:06:47.646824+0000","last_fullsized":"2024-09-15T22:06:47.646824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:34.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.851201+0000","last_change":"2024-09-15T22:06:37.186718+0000","last_active":"2024-09-15T22:06:47.851201+0000","last_peered":"2024-09-15T22:06:47.851201+0000","last_clean":"2024-09-15T22:06:47.851201+0000","last_became_active":"2024-09-15T22:06:37.186424+0000","last_became_peered":"2024-09-15T22:06:37.186424+0000","last_unstale":"2024-09-15T22:06:47.851201+0000","last_undegraded":"2024-09-15T22:06:47.851201+0000","last_fullsized":"2024-09-15T22:06:47.851201+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:53.266007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850897+0000","last_change":"2024-09-15T22:06:37.185522+0000","last_active":"2024-09-15T22:06:47.850897+0000","last_peered":"2024-09-15T22:06:47.850897+0000","last_clean":"2024-09-15T22:06:47.850897+0000","last_became_active":"2024-09-15T22:06:37.185192+0000","last_became_peered":"2024-09-15T22:06:37.185192+0000","last_unstale":"2024-09-15T22:06:47.850897+0000","last_undegraded":"2024-09-15T22:06:47.850897+0000","last_fullsized":"2024-09-15T22:06:47.850897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:12.868324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-15T22:06:47.850539+0000","last_change":"2024-09-15T22:06:37.189553+0000","last_active":"2024-09-15T22:06:47.850539+0000","last_peered":"2024-09-15T22:06:47.850539+0000","last_clean":"2024-09-15T22:06:47.850539+0000","last_became_active":"2024-09-15T22:06:37.189270+0000","last_became_peered":"2024-09-15T22:06:37.189270+0000","last_unstale":"2024-09-15T22:06:47.850539+0000","last_undegraded":"2024-09-15T22:06:47.850539+0000","last_fullsized":"2024-09-15T22:06:47.850539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:34.157077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":56804,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":116,"num_read_kb":120,"num_write":167,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":170412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579928,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":483279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808861,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32800,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712352,"statfs":{"total":95995035648,"available":95961448448,"internally_reserved":0,"allocated":1654784,"data_stored":1123348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005090,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":487611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234021,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":482853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495660,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1585152,"data_stored":1079739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659184,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":478691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888117,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1593344,"data_stored":1074934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018939,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":484316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247872,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36304,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708848,"statfs":{"total":95995035648,"available":95957860352,"internally_reserved":0,"allocated":983040,"data_stored":478885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444101,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":532170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1032192,"data_stored":531591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":480079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":52990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":53569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:07:32.742 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T22:07:32.743 INFO:tasks.cephadm:Setting up client nodes... 2024-09-15T22:07:32.744 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-15T22:07:33.079 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:32 smithi005 ceph-mon[30944]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:07:33.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:32 smithi099 ceph-mon[34892]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:07:33.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:32 smithi129 ceph-mon[35314]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:07:33.462 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.943+0000 7f120415f640 1 -- 172.21.15.5:0/927094403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11fc100dd0 msgr2=0x7f11fc101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.943+0000 7f120415f640 1 --2- 172.21.15.5:0/927094403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11fc100dd0 0x7f11fc101230 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f11ec009900 tx=0x7f11ec02a410 comp rx=0 tx=0).stop 2024-09-15T22:07:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 -- 172.21.15.5:0/927094403 shutdown_connections 2024-09-15T22:07:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 --2- 172.21.15.5:0/927094403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11fc101770 0x7f11fc101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 --2- 172.21.15.5:0/927094403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11fc100dd0 0x7f11fc101230 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 --2- 172.21.15.5:0/927094403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11fc100490 0x7f11fc100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 -- 172.21.15.5:0/927094403 >> 172.21.15.5:0/927094403 conn(0x7f11fc0fc2a0 msgr2=0x7f11fc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 -- 172.21.15.5:0/927094403 shutdown_connections 2024-09-15T22:07:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 -- 172.21.15.5:0/927094403 wait complete. 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 Processor -- start 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 -- start start 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11fc100490 0x7f11fc19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11fc100dd0 0x7f11fc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11fc101770 0x7f11fc1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc10fa60 con 0x7f11fc101770 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc10f8e0 con 0x7f11fc100dd0 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.944+0000 7f120415f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f11fc10fbe0 con 0x7f11fc100490 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f12026fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11fc101770 0x7f11fc1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f12026fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11fc101770 0x7f11fc1a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51746/0 (socket says 172.21.15.5:51746) 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f1201efb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11fc100490 0x7f11fc19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f12026fc640 1 -- 172.21.15.5:0/705280152 learned_addr learned my addr 172.21.15.5:0/705280152 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f12016fa640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11fc100dd0 0x7f11fc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f12026fc640 1 -- 172.21.15.5:0/705280152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11fc100490 msgr2=0x7f11fc19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f12026fc640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11fc100490 0x7f11fc19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f12026fc640 1 -- 172.21.15.5:0/705280152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11fc100dd0 msgr2=0x7f11fc19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f12026fc640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11fc100dd0 0x7f11fc19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f12026fc640 1 -- 172.21.15.5:0/705280152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11fc1a1d10 con 0x7f11fc101770 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f1201efb640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11fc100490 0x7f11fc19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f12026fc640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11fc101770 0x7f11fc1a16a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009fd0 tx=0x7f11f400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f11f2ffd640 1 -- 172.21.15.5:0/705280152 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4017070 con 0x7f11fc101770 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f11f2ffd640 1 -- 172.21.15.5:0/705280152 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f4003e50 con 0x7f11fc101770 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.945+0000 7f120415f640 1 -- 172.21.15.5:0/705280152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11fc1a1fa0 con 0x7f11fc101770 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.946+0000 7f11f2ffd640 1 -- 172.21.15.5:0/705280152 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4010460 con 0x7f11fc101770 2024-09-15T22:07:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.946+0000 7f120415f640 1 -- 172.21.15.5:0/705280152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11fc1a24b0 con 0x7f11fc101770 2024-09-15T22:07:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.946+0000 7f120415f640 1 -- 172.21.15.5:0/705280152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11fc101230 con 0x7f11fc101770 2024-09-15T22:07:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.948+0000 7f11f2ffd640 1 -- 172.21.15.5:0/705280152 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f11f4003970 con 0x7f11fc101770 2024-09-15T22:07:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.948+0000 7f11f2ffd640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f11e80a3380 0x7f11e80a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.949+0000 7f1201efb640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f11e80a3380 0x7f11e80a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.949+0000 7f11f2ffd640 1 -- 172.21.15.5:0/705280152 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f11f40cf980 con 0x7f11fc101770 2024-09-15T22:07:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.949+0000 7f1201efb640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f11e80a3380 0x7f11e80a5840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f11f800afb0 tx=0x7f11f800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:33.953+0000 7f11f2ffd640 1 -- 172.21.15.5:0/705280152 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11f409c200 con 0x7f11fc101770 2024-09-15T22:07:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.204+0000 7f120415f640 1 -- 172.21.15.5:0/705280152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f11fc02d860 con 0x7f11fc101770 2024-09-15T22:07:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.208+0000 7f11f2ffd640 1 -- 172.21.15.5:0/705280152 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f11fc02d860 con 0x7f11fc101770 2024-09-15T22:07:34.210 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-09-15T22:07:34.218 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCmWudmLGQ+DBAAwtZsvU3Y8ZC90SE4rt86Wg== 2024-09-15T22:07:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.210+0000 7f120415f640 1 -- 172.21.15.5:0/705280152 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f11e80a3380 msgr2=0x7f11e80a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.210+0000 7f120415f640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f11e80a3380 0x7f11e80a5840 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f11f800afb0 tx=0x7f11f800a2d0 comp rx=0 tx=0).stop 2024-09-15T22:07:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.210+0000 7f120415f640 1 -- 172.21.15.5:0/705280152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11fc101770 msgr2=0x7f11fc1a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.210+0000 7f120415f640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11fc101770 0x7f11fc1a16a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009fd0 tx=0x7f11f400def0 comp rx=0 tx=0).stop 2024-09-15T22:07:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.210+0000 7f120415f640 1 -- 172.21.15.5:0/705280152 shutdown_connections 2024-09-15T22:07:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.210+0000 7f120415f640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f11e80a3380 0x7f11e80a5840 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.210+0000 7f120415f640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11fc101770 0x7f11fc1a16a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.210+0000 7f120415f640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f11fc100dd0 0x7f11fc19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.210+0000 7f120415f640 1 --2- 172.21.15.5:0/705280152 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f11fc100490 0x7f11fc19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.210+0000 7f120415f640 1 -- 172.21.15.5:0/705280152 >> 172.21.15.5:0/705280152 conn(0x7f11fc0fc2a0 msgr2=0x7f11fc0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.210+0000 7f120415f640 1 -- 172.21.15.5:0/705280152 shutdown_connections 2024-09-15T22:07:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:34.211+0000 7f120415f640 1 -- 172.21.15.5:0/705280152 wait complete. 2024-09-15T22:07:34.567 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:34 smithi005 ceph-mon[30944]: pgmap v311: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-15T22:07:34.567 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:34 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.567 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:34 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.567 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:34 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.567 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:34 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:34 smithi099 ceph-mon[34892]: pgmap v311: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-15T22:07:34.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:34 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:34 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:34 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:34 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:34 smithi129 ceph-mon[35314]: pgmap v311: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-09-15T22:07:34.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:34 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:34 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:34 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:34 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:34.937 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T22:07:34.937 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-15T22:07:34.937 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T22:07:34.977 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T22:07:34.978 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-15T22:07:34.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-15T22:07:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:35.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:35 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:35.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:35 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/705280152' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:07:35.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:35 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:35.522 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:35 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/705280152' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:07:35.522 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:35 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:35.522 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:35 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:35.522 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:35 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:35 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:35 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/705280152' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:07:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:35 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:35 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/705280152' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:07:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:35 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:35 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:35 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:35 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:35 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/705280152' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-15T22:07:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:35 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:35 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/705280152' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-15T22:07:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:35 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:07:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:35 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:07:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:35 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:07:35.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.729+0000 7eff76426640 1 -- 172.21.15.5:0/1070818968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff70102ef0 msgr2=0x7eff7010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.729+0000 7eff76426640 1 --2- 172.21.15.5:0/1070818968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff70102ef0 0x7eff7010da40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7eff68009ca0 tx=0x7eff6802a950 comp rx=0 tx=0).stop 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.729+0000 7eff76426640 1 -- 172.21.15.5:0/1070818968 shutdown_connections 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.729+0000 7eff76426640 1 --2- 172.21.15.5:0/1070818968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff70102ef0 0x7eff7010da40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.729+0000 7eff76426640 1 --2- 172.21.15.5:0/1070818968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff70102570 0x7eff701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.729+0000 7eff76426640 1 --2- 172.21.15.5:0/1070818968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff70100e20 0x7eff70101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.729+0000 7eff76426640 1 -- 172.21.15.5:0/1070818968 >> 172.21.15.5:0/1070818968 conn(0x7eff700fc7d0 msgr2=0x7eff700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.729+0000 7eff76426640 1 -- 172.21.15.5:0/1070818968 shutdown_connections 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.729+0000 7eff76426640 1 -- 172.21.15.5:0/1070818968 wait complete. 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.729+0000 7eff76426640 1 Processor -- start 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff76426640 1 -- start start 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff76426640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff70100e20 0x7eff7019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff76426640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff70102570 0x7eff7019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:35.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff76426640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff70102ef0 0x7eff7019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff76426640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff70110f80 con 0x7eff70102ef0 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff76426640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7eff70110e00 con 0x7eff70100e20 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff76426640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7eff70111100 con 0x7eff70102570 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff749c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff70102ef0 0x7eff7019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff749c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff70102ef0 0x7eff7019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51772/0 (socket says 172.21.15.5:51772) 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff749c3640 1 -- 172.21.15.5:0/1658157500 learned_addr learned my addr 172.21.15.5:0/1658157500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff6ffff640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff70100e20 0x7eff7019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff67fff640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff70102570 0x7eff7019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff749c3640 1 -- 172.21.15.5:0/1658157500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff70102570 msgr2=0x7eff7019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff749c3640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff70102570 0x7eff7019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff749c3640 1 -- 172.21.15.5:0/1658157500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff70100e20 msgr2=0x7eff7019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff749c3640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff70100e20 0x7eff7019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:35.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff749c3640 1 -- 172.21.15.5:0/1658157500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7019f600 con 0x7eff70102ef0 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff67fff640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff70102570 0x7eff7019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.730+0000 7eff749c3640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff70102ef0 0x7eff7019ef20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7eff6802ae70 tx=0x7eff6802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.731+0000 7eff6dffb640 1 -- 172.21.15.5:0/1658157500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68034040 con 0x7eff70102ef0 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.731+0000 7eff6dffb640 1 -- 172.21.15.5:0/1658157500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff68030920 con 0x7eff70102ef0 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.731+0000 7eff76426640 1 -- 172.21.15.5:0/1658157500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7019f890 con 0x7eff70102ef0 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.731+0000 7eff6dffb640 1 -- 172.21.15.5:0/1658157500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68045480 con 0x7eff70102ef0 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.731+0000 7eff76426640 1 -- 172.21.15.5:0/1658157500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff701a71d0 con 0x7eff70102ef0 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.732+0000 7eff76426640 1 -- 172.21.15.5:0/1658157500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff701021b0 con 0x7eff70102ef0 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.733+0000 7eff6dffb640 1 -- 172.21.15.5:0/1658157500 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7eff68037020 con 0x7eff70102ef0 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.733+0000 7eff6dffb640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7eff5c0a32d0 0x7eff5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.734+0000 7eff6ffff640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7eff5c0a32d0 0x7eff5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.734+0000 7eff6dffb640 1 -- 172.21.15.5:0/1658157500 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7eff680f0c00 con 0x7eff70102ef0 2024-09-15T22:07:35.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.734+0000 7eff6ffff640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7eff5c0a32d0 0x7eff5c0a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff6000bbe0 tx=0x7eff6000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:35.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.738+0000 7eff6dffb640 1 -- 172.21.15.5:0/1658157500 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff680bd590 con 0x7eff70102ef0 2024-09-15T22:07:35.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.963+0000 7eff76426640 1 -- 172.21.15.5:0/1658157500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7eff701a73a0 con 0x7eff70102ef0 2024-09-15T22:07:35.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.967+0000 7eff6dffb640 1 -- 172.21.15.5:0/1658157500 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264037 (secure 0 0 0) 0x7eff680bce90 con 0x7eff70102ef0 2024-09-15T22:07:35.973 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:07:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.970+0000 7eff76426640 1 -- 172.21.15.5:0/1658157500 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7eff5c0a32d0 msgr2=0x7eff5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.970+0000 7eff76426640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7eff5c0a32d0 0x7eff5c0a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff6000bbe0 tx=0x7eff6000b390 comp rx=0 tx=0).stop 2024-09-15T22:07:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.970+0000 7eff76426640 1 -- 172.21.15.5:0/1658157500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff70102ef0 msgr2=0x7eff7019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:35.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.970+0000 7eff76426640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff70102ef0 0x7eff7019ef20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7eff6802ae70 tx=0x7eff6802f5d0 comp rx=0 tx=0).stop 2024-09-15T22:07:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.970+0000 7eff76426640 1 -- 172.21.15.5:0/1658157500 shutdown_connections 2024-09-15T22:07:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.970+0000 7eff76426640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7eff5c0a32d0 0x7eff5c0a5790 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.970+0000 7eff76426640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff70102ef0 0x7eff7019ef20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.970+0000 7eff76426640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7eff70102570 0x7eff7019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.970+0000 7eff76426640 1 --2- 172.21.15.5:0/1658157500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7eff70100e20 0x7eff7019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.970+0000 7eff76426640 1 -- 172.21.15.5:0/1658157500 >> 172.21.15.5:0/1658157500 conn(0x7eff700fc7d0 msgr2=0x7eff7006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:35.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.971+0000 7eff76426640 1 -- 172.21.15.5:0/1658157500 shutdown_connections 2024-09-15T22:07:35.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:35.971+0000 7eff76426640 1 -- 172.21.15.5:0/1658157500 wait complete. 2024-09-15T22:07:36.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:36 smithi005 ceph-mon[30944]: pgmap v312: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-15T22:07:36.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:36 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1658157500' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:07:36.700 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":801026306},{"type":"v1","addr":"172.21.15.5:6801","nonce":801026306}]},"active_addr":"172.21.15.5:6801/801026306","active_change":"2024-09-15T21:59:42.685363+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.5:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2453332511}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2083481614}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1408571851}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":878276657}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1000612363}]}]} 2024-09-15T22:07:36.706 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-15T22:07:36.707 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-15T22:07:36.707 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:07:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:37.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:36 smithi099 ceph-mon[34892]: pgmap v312: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-15T22:07:37.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:36 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1658157500' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:07:37.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:36 smithi129 ceph-mon[35314]: pgmap v312: 129 pgs: 129 active+clean; 632 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.4 KiB/s wr, 8 op/s 2024-09-15T22:07:37.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:36 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1658157500' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-15T22:07:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.489+0000 7fa678c92640 1 -- 172.21.15.5:0/2972362728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674106f50 msgr2=0x7fa67410eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.489+0000 7fa678c92640 1 --2- 172.21.15.5:0/2972362728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674106f50 0x7fa67410eaf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009cd0 tx=0x7fa66c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:07:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.489+0000 7fa678c92640 1 -- 172.21.15.5:0/2972362728 shutdown_connections 2024-09-15T22:07:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.489+0000 7fa678c92640 1 --2- 172.21.15.5:0/2972362728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674106f50 0x7fa67410eaf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.489+0000 7fa678c92640 1 --2- 172.21.15.5:0/2972362728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa674100d00 0x7fa674106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.489+0000 7fa678c92640 1 --2- 172.21.15.5:0/2972362728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa674100350 0x7fa674100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.489+0000 7fa678c92640 1 -- 172.21.15.5:0/2972362728 >> 172.21.15.5:0/2972362728 conn(0x7fa674077210 msgr2=0x7fa6740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.489+0000 7fa678c92640 1 -- 172.21.15.5:0/2972362728 shutdown_connections 2024-09-15T22:07:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.489+0000 7fa678c92640 1 -- 172.21.15.5:0/2972362728 wait complete. 2024-09-15T22:07:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.489+0000 7fa678c92640 1 Processor -- start 2024-09-15T22:07:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa678c92640 1 -- start start 2024-09-15T22:07:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa678c92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674100350 0x7fa674198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa678c92640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa674100d00 0x7fa674198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa678c92640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa674106f50 0x7fa67419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa678c92640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa674111fa0 con 0x7fa674100350 2024-09-15T22:07:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa678c92640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa674111e20 con 0x7fa674106f50 2024-09-15T22:07:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa678c92640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa674112120 con 0x7fa674100d00 2024-09-15T22:07:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa67259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674100350 0x7fa674198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa67259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674100350 0x7fa674198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51790/0 (socket says 172.21.15.5:51790) 2024-09-15T22:07:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa67259c640 1 -- 172.21.15.5:0/986316407 learned_addr learned my addr 172.21.15.5:0/986316407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa672d9d640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa674106f50 0x7fa67419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa671d9b640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa674100d00 0x7fa674198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa67259c640 1 -- 172.21.15.5:0/986316407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa674100d00 msgr2=0x7fa674198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa67259c640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa674100d00 0x7fa674198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa67259c640 1 -- 172.21.15.5:0/986316407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa674106f50 msgr2=0x7fa67419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa67259c640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa674106f50 0x7fa67419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa67259c640 1 -- 172.21.15.5:0/986316407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67419d530 con 0x7fa674100350 2024-09-15T22:07:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa672d9d640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa674106f50 0x7fa67419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa671d9b640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa674100d00 0x7fa674198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.490+0000 7fa67259c640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674100350 0x7fa674198580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa66400d950 tx=0x7fa66400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.491+0000 7fa6637fe640 1 -- 172.21.15.5:0/986316407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664017030 con 0x7fa674100350 2024-09-15T22:07:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.491+0000 7fa6637fe640 1 -- 172.21.15.5:0/986316407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa664004020 con 0x7fa674100350 2024-09-15T22:07:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.491+0000 7fa678c92640 1 -- 172.21.15.5:0/986316407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67419d820 con 0x7fa674100350 2024-09-15T22:07:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.491+0000 7fa6637fe640 1 -- 172.21.15.5:0/986316407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6640047a0 con 0x7fa674100350 2024-09-15T22:07:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.491+0000 7fa678c92640 1 -- 172.21.15.5:0/986316407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6741a5100 con 0x7fa674100350 2024-09-15T22:07:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.492+0000 7fa678c92640 1 -- 172.21.15.5:0/986316407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa674100730 con 0x7fa674100350 2024-09-15T22:07:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.493+0000 7fa6637fe640 1 -- 172.21.15.5:0/986316407 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fa6640041d0 con 0x7fa674100350 2024-09-15T22:07:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.493+0000 7fa6637fe640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa65c0a32d0 0x7fa65c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.494+0000 7fa671d9b640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa65c0a32d0 0x7fa65c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.494+0000 7fa6637fe640 1 -- 172.21.15.5:0/986316407 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7fa6640cf880 con 0x7fa674100350 2024-09-15T22:07:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.494+0000 7fa671d9b640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa65c0a32d0 0x7fa65c0a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa674104900 tx=0x7fa668005cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.498+0000 7fa6637fe640 1 -- 172.21.15.5:0/986316407 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa66409c100 con 0x7fa674100350 2024-09-15T22:07:37.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.677+0000 7fa678c92640 1 -- 172.21.15.5:0/986316407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa67410a120 con 0x7fa674100350 2024-09-15T22:07:37.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.679+0000 7fa6637fe640 1 -- 172.21.15.5:0/986316407 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24549 (secure 0 0 0) 0x7fa66409ba00 con 0x7fa674100350 2024-09-15T22:07:37.681 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:07:37.681 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:06:47.363102+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:07:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.681+0000 7fa678c92640 1 -- 172.21.15.5:0/986316407 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa65c0a32d0 msgr2=0x7fa65c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.681+0000 7fa678c92640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa65c0a32d0 0x7fa65c0a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa674104900 tx=0x7fa668005cb0 comp rx=0 tx=0).stop 2024-09-15T22:07:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.681+0000 7fa678c92640 1 -- 172.21.15.5:0/986316407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674100350 msgr2=0x7fa674198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.681+0000 7fa678c92640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674100350 0x7fa674198580 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa66400d950 tx=0x7fa66400de10 comp rx=0 tx=0).stop 2024-09-15T22:07:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.681+0000 7fa678c92640 1 -- 172.21.15.5:0/986316407 shutdown_connections 2024-09-15T22:07:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.681+0000 7fa678c92640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa674106f50 0x7fa67419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.681+0000 7fa678c92640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa674100d00 0x7fa674198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.681+0000 7fa678c92640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa65c0a32d0 0x7fa65c0a5790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:37.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.681+0000 7fa678c92640 1 --2- 172.21.15.5:0/986316407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674100350 0x7fa674198580 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.681+0000 7fa678c92640 1 -- 172.21.15.5:0/986316407 >> 172.21.15.5:0/986316407 conn(0x7fa674077210 msgr2=0x7fa6740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.682+0000 7fa678c92640 1 -- 172.21.15.5:0/986316407 shutdown_connections 2024-09-15T22:07:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:37.682+0000 7fa678c92640 1 -- 172.21.15.5:0/986316407 wait complete. 2024-09-15T22:07:38.342 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-15T22:07:38.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:07:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:38.855 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:38 smithi005 ceph-mon[30944]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-15T22:07:38.855 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:38 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/986316407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.063+0000 7ffb163be640 1 -- 172.21.15.5:0/4213488089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1010c170 msgr2=0x7ffb1010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.063+0000 7ffb163be640 1 --2- 172.21.15.5:0/4213488089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1010c170 0x7ffb1010e610 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009ca0 tx=0x7ffb0802a950 comp rx=0 tx=0).stop 2024-09-15T22:07:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.063+0000 7ffb163be640 1 -- 172.21.15.5:0/4213488089 shutdown_connections 2024-09-15T22:07:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.063+0000 7ffb163be640 1 --2- 172.21.15.5:0/4213488089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1010c170 0x7ffb1010e610 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.063+0000 7ffb163be640 1 --2- 172.21.15.5:0/4213488089 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb10103660 0x7ffb1010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.063+0000 7ffb163be640 1 --2- 172.21.15.5:0/4213488089 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb10102d40 0x7ffb10103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.063+0000 7ffb163be640 1 -- 172.21.15.5:0/4213488089 >> 172.21.15.5:0/4213488089 conn(0x7ffb100fc7a0 msgr2=0x7ffb100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.063+0000 7ffb163be640 1 -- 172.21.15.5:0/4213488089 shutdown_connections 2024-09-15T22:07:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.063+0000 7ffb163be640 1 -- 172.21.15.5:0/4213488089 wait complete. 2024-09-15T22:07:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.064+0000 7ffb163be640 1 Processor -- start 2024-09-15T22:07:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.064+0000 7ffb163be640 1 -- start start 2024-09-15T22:07:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.064+0000 7ffb163be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb10102d40 0x7ffb101009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.064+0000 7ffb163be640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb10103660 0x7ffb10100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.064+0000 7ffb163be640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb1010c170 0x7ffb10101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.064+0000 7ffb163be640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10111f30 con 0x7ffb10102d40 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.064+0000 7ffb163be640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10111db0 con 0x7ffb10103660 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.064+0000 7ffb163be640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101120b0 con 0x7ffb1010c170 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb10102d40 0x7ffb101009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb10102d40 0x7ffb101009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51806/0 (socket says 172.21.15.5:51806) 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0ffff640 1 -- 172.21.15.5:0/2118526930 learned_addr learned my addr 172.21.15.5:0/2118526930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0f7fe640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb10103660 0x7ffb10100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0ffff640 1 -- 172.21.15.5:0/2118526930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb1010c170 msgr2=0x7ffb10101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb1495b640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb1010c170 0x7ffb10101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0ffff640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb1010c170 0x7ffb10101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0ffff640 1 -- 172.21.15.5:0/2118526930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb10103660 msgr2=0x7ffb10100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0ffff640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb10103660 0x7ffb10100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0ffff640 1 -- 172.21.15.5:0/2118526930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb101a6890 con 0x7ffb10102d40 2024-09-15T22:07:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0ffff640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb10102d40 0x7ffb101009a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c890 tx=0x7ffb0000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0d7fa640 1 -- 172.21.15.5:0/2118526930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00004220 con 0x7ffb10102d40 2024-09-15T22:07:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0d7fa640 1 -- 172.21.15.5:0/2118526930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb000043c0 con 0x7ffb10102d40 2024-09-15T22:07:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb163be640 1 -- 172.21.15.5:0/2118526930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb101a6b80 con 0x7ffb10102d40 2024-09-15T22:07:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb0d7fa640 1 -- 172.21.15.5:0/2118526930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00004a00 con 0x7ffb10102d40 2024-09-15T22:07:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.065+0000 7ffb163be640 1 -- 172.21.15.5:0/2118526930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb101a70c0 con 0x7ffb10102d40 2024-09-15T22:07:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.066+0000 7ffb163be640 1 -- 172.21.15.5:0/2118526930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb10103120 con 0x7ffb10102d40 2024-09-15T22:07:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.068+0000 7ffb0d7fa640 1 -- 172.21.15.5:0/2118526930 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ffb000079b0 con 0x7ffb10102d40 2024-09-15T22:07:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.069+0000 7ffb0d7fa640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ffaf80a32d0 0x7ffaf80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.069+0000 7ffb0f7fe640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ffaf80a32d0 0x7ffaf80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.069+0000 7ffb0d7fa640 1 -- 172.21.15.5:0/2118526930 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7ffb000cfeb0 con 0x7ffb10102d40 2024-09-15T22:07:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.069+0000 7ffb0f7fe640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ffaf80a32d0 0x7ffaf80a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb10102430 tx=0x7ffb0400b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:39.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.072+0000 7ffb0d7fa640 1 -- 172.21.15.5:0/2118526930 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb00014030 con 0x7ffb10102d40 2024-09-15T22:07:39.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:38 smithi099 ceph-mon[34892]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-15T22:07:39.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:38 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/986316407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:39.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:38 smithi129 ceph-mon[35314]: pgmap v313: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-15T22:07:39.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:38 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/986316407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:39.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.249+0000 7ffb163be640 1 -- 172.21.15.5:0/2118526930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb10101ed0 con 0x7ffb10102d40 2024-09-15T22:07:39.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.251+0000 7ffb0d7fa640 1 -- 172.21.15.5:0/2118526930 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24549 (secure 0 0 0) 0x7ffb0009c730 con 0x7ffb10102d40 2024-09-15T22:07:39.253 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:07:39.253 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":86,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:06:47.363102+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:07:39.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 -- 172.21.15.5:0/2118526930 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ffaf80a32d0 msgr2=0x7ffaf80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:39.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ffaf80a32d0 0x7ffaf80a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ffb10102430 tx=0x7ffb0400b2e0 comp rx=0 tx=0).stop 2024-09-15T22:07:39.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 -- 172.21.15.5:0/2118526930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb10102d40 msgr2=0x7ffb101009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:39.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb10102d40 0x7ffb101009a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffb0000c890 tx=0x7ffb0000cd50 comp rx=0 tx=0).stop 2024-09-15T22:07:39.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 -- 172.21.15.5:0/2118526930 shutdown_connections 2024-09-15T22:07:39.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffb1010c170 0x7ffb10101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:39.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffb10103660 0x7ffb10100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:39.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ffaf80a32d0 0x7ffaf80a5790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:39.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 --2- 172.21.15.5:0/2118526930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb10102d40 0x7ffb101009a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:39.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 -- 172.21.15.5:0/2118526930 >> 172.21.15.5:0/2118526930 conn(0x7ffb100fc7a0 msgr2=0x7ffb100fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:39.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 -- 172.21.15.5:0/2118526930 shutdown_connections 2024-09-15T22:07:39.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:39.253+0000 7ffb163be640 1 -- 172.21.15.5:0/2118526930 wait complete. 2024-09-15T22:07:39.847 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:39 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2118526930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:39.879 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-15T22:07:39.879 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-15T22:07:39.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-15T22:07:39.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-15T22:07:39.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-15T22:07:39.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-15T22:07:39.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-15T22:07:39.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-15T22:07:39.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-15T22:07:39.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-15T22:07:39.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-15T22:07:39.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-15T22:07:40.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:39 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2118526930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:40.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:39 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2118526930' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:07:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:40.644 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:40.644 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:40.650 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:40.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:40 smithi005 ceph-mon[30944]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-15T22:07:41.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:40 smithi099 ceph-mon[34892]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-15T22:07:41.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:40 smithi129 ceph-mon[35314]: pgmap v314: 129 pgs: 129 active+clean; 641 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-15T22:07:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.472+0000 7f4e01d38640 1 -- 172.21.15.5:0/603422107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc100aa0 msgr2=0x7f4dfc10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.472+0000 7f4e01d38640 1 --2- 172.21.15.5:0/603422107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc100aa0 0x7f4dfc10ea20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4df4009ca0 tx=0x7f4df402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:07:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.472+0000 7f4e01d38640 1 -- 172.21.15.5:0/603422107 shutdown_connections 2024-09-15T22:07:41.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.472+0000 7f4e01d38640 1 --2- 172.21.15.5:0/603422107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc100aa0 0x7f4dfc10ea20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.472+0000 7f4e01d38640 1 --2- 172.21.15.5:0/603422107 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dfc100100 0x7f4dfc100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.472+0000 7f4e01d38640 1 --2- 172.21.15.5:0/603422107 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dfc106b80 0x7f4dfc106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.472+0000 7f4e01d38640 1 -- 172.21.15.5:0/603422107 >> 172.21.15.5:0/603422107 conn(0x7f4dfc077230 msgr2=0x7f4dfc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.472+0000 7f4e01d38640 1 -- 172.21.15.5:0/603422107 shutdown_connections 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.472+0000 7f4e01d38640 1 -- 172.21.15.5:0/603422107 wait complete. 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.473+0000 7f4e01d38640 1 Processor -- start 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.473+0000 7f4e01d38640 1 -- start start 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.473+0000 7f4e01d38640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dfc100100 0x7f4dfc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.473+0000 7f4e01d38640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dfc100aa0 0x7f4dfc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.473+0000 7f4e01d38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc106b80 0x7f4dfc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.473+0000 7f4e01d38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc112070 con 0x7f4dfc106b80 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.473+0000 7f4e01d38640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc111ef0 con 0x7f4dfc100100 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.473+0000 7f4e01d38640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4dfc1121f0 con 0x7f4dfc100aa0 2024-09-15T22:07:41.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.473+0000 7f4dfbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc106b80 0x7f4dfc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.473+0000 7f4dfbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc106b80 0x7f4dfc19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51820/0 (socket says 172.21.15.5:51820) 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.473+0000 7f4dfbfff640 1 -- 172.21.15.5:0/170468912 learned_addr learned my addr 172.21.15.5:0/170468912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4df2ffd640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dfc100aa0 0x7f4dfc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4dfb7fe640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dfc100100 0x7f4dfc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4dfbfff640 1 -- 172.21.15.5:0/170468912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dfc100aa0 msgr2=0x7f4dfc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4dfbfff640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4dfc100aa0 0x7f4dfc198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4dfbfff640 1 -- 172.21.15.5:0/170468912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dfc100100 msgr2=0x7f4dfc198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4dfbfff640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dfc100100 0x7f4dfc198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4dfbfff640 1 -- 172.21.15.5:0/170468912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dfc19d520 con 0x7f4dfc106b80 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4dfb7fe640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4dfc100100 0x7f4dfc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4dfbfff640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc106b80 0x7f4dfc19ce40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4df4031f80 tx=0x7f4df4031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4df97fa640 1 -- 172.21.15.5:0/170468912 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4003c30 con 0x7f4dfc106b80 2024-09-15T22:07:41.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4e01d38640 1 -- 172.21.15.5:0/170468912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dfc19d7b0 con 0x7f4dfc106b80 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4df97fa640 1 -- 172.21.15.5:0/170468912 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4df4003dd0 con 0x7f4dfc106b80 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4df97fa640 1 -- 172.21.15.5:0/170468912 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df40047d0 con 0x7f4dfc106b80 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.474+0000 7f4e01d38640 1 -- 172.21.15.5:0/170468912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dfc1a50f0 con 0x7f4dfc106b80 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.475+0000 7f4e01d38640 1 -- 172.21.15.5:0/170468912 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4dfc062900 con 0x7f4dfc106b80 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.477+0000 7f4df97fa640 1 -- 172.21.15.5:0/170468912 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f4df402f710 con 0x7f4dfc106b80 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.478+0000 7f4df97fa640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4de80a32d0 0x7f4de80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.478+0000 7f4df97fa640 1 -- 172.21.15.5:0/170468912 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f4df40f13d0 con 0x7f4dfc106b80 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.478+0000 7f4dfb7fe640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4de80a32d0 0x7f4de80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.478+0000 7f4df97fa640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355] conn(0x7f4de80ae420 0x7f4de80b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.479+0000 7f4df97fa640 1 -- 172.21.15.5:0/170468912 --> [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4df40020b0 con 0x7f4de80ae420 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.479+0000 7f4df97fa640 1 -- 172.21.15.5:0/170468912 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f4df40f1780 con 0x7f4dfc106b80 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.479+0000 7f4df2ffd640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355] conn(0x7f4de80ae420 0x7f4de80b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.479+0000 7f4dfb7fe640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4de80a32d0 0x7f4de80a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0058c0 tx=0x7f4dec005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:41.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.479+0000 7f4df2ffd640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355] conn(0x7f4de80ae420 0x7f4de80b08c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:41.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.480+0000 7f4df97fa640 1 -- 172.21.15.5:0/170468912 <== osd.9 v2:172.21.15.5:6826/3480349355 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4df40020b0 con 0x7f4de80ae420 2024-09-15T22:07:41.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.494+0000 7f4e01d38640 1 -- 172.21.15.5:0/170468912 --> [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4dfc062d00 con 0x7f4de80ae420 2024-09-15T22:07:41.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.494+0000 7f4df97fa640 1 -- 172.21.15.5:0/170468912 <== osd.9 v2:172.21.15.5:6826/3480349355 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4dfc062d00 con 0x7f4de80ae420 2024-09-15T22:07:41.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.495+0000 7f4e01d38640 1 -- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355] conn(0x7f4de80ae420 msgr2=0x7f4de80b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:41.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.495+0000 7f4e01d38640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355] conn(0x7f4de80ae420 0x7f4de80b08c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:41.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.495+0000 7f4e01d38640 1 -- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4de80a32d0 msgr2=0x7f4de80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:41.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.495+0000 7f4e01d38640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4de80a32d0 0x7f4de80a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0058c0 tx=0x7f4dec005d30 comp rx=0 tx=0).stop 2024-09-15T22:07:41.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.495+0000 7f4e01d38640 1 -- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc106b80 msgr2=0x7f4dfc19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:41.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.495+0000 7f4e01d38640 1 --2- 172.21.15.5:0/170468912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dfc106b80 0x7f4dfc19ce40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4df4031f80 tx=0x7f4df4031a60 comp rx=0 tx=0).stop 2024-09-15T22:07:41.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.495+0000 7f4dfbfff640 1 -- 172.21.15.5:0/170468912 reap_dead start 2024-09-15T22:07:41.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.495+0000 7f4e01d38640 1 -- 172.21.15.5:0/170468912 shutdown_connections 2024-09-15T22:07:41.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.495+0000 7f4e01d38640 1 -- 172.21.15.5:0/170468912 >> 172.21.15.5:0/170468912 conn(0x7f4dfc077230 msgr2=0x7f4dfc104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:41.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.495+0000 7f4e01d38640 1 -- 172.21.15.5:0/170468912 shutdown_connections 2024-09-15T22:07:41.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:41.495+0000 7f4e01d38640 1 -- 172.21.15.5:0/170468912 wait complete. 2024-09-15T22:07:42.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.184+0000 7f23dfd65640 1 -- 172.21.15.5:0/2767725596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8102ef0 msgr2=0x7f23d810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.184+0000 7f23dfd65640 1 --2- 172.21.15.5:0/2767725596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8102ef0 0x7f23d810da40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f23d0009ca0 tx=0x7f23d002a950 comp rx=0 tx=0).stop 2024-09-15T22:07:42.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.184+0000 7f23dfd65640 1 -- 172.21.15.5:0/2767725596 shutdown_connections 2024-09-15T22:07:42.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.184+0000 7f23dfd65640 1 --2- 172.21.15.5:0/2767725596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8102ef0 0x7f23d810da40 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.184+0000 7f23dfd65640 1 --2- 172.21.15.5:0/2767725596 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23d8102570 0x7f23d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.184+0000 7f23dfd65640 1 --2- 172.21.15.5:0/2767725596 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23d8100e20 0x7f23d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.184+0000 7f23dfd65640 1 -- 172.21.15.5:0/2767725596 >> 172.21.15.5:0/2767725596 conn(0x7f23d80fc7d0 msgr2=0x7f23d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:42.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.184+0000 7f23dfd65640 1 -- 172.21.15.5:0/2767725596 shutdown_connections 2024-09-15T22:07:42.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.184+0000 7f23dfd65640 1 -- 172.21.15.5:0/2767725596 wait complete. 2024-09-15T22:07:42.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23dfd65640 1 Processor -- start 2024-09-15T22:07:42.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23dfd65640 1 -- start start 2024-09-15T22:07:42.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23dfd65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8100e20 0x7f23d8198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:42.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23dfd65640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23d8102570 0x7f23d8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:42.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23dfd65640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23d8102ef0 0x7f23d819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:42.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23dfd65640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f23d8110fd0 con 0x7f23d8100e20 2024-09-15T22:07:42.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23dfd65640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23d8110e50 con 0x7f23d8102ef0 2024-09-15T22:07:42.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23dfd65640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23d8111150 con 0x7f23d8102570 2024-09-15T22:07:42.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23ddb01640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8100e20 0x7f23d8198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:42.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23ddb01640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8100e20 0x7f23d8198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51832/0 (socket says 172.21.15.5:51832) 2024-09-15T22:07:42.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23ddb01640 1 -- 172.21.15.5:0/1338602188 learned_addr learned my addr 172.21.15.5:0/1338602188 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23dd300640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23d8102570 0x7f23d8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23ddb01640 1 -- 172.21.15.5:0/1338602188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23d8102570 msgr2=0x7f23d8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23ddb01640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23d8102570 0x7f23d8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23ddb01640 1 -- 172.21.15.5:0/1338602188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23d8102ef0 msgr2=0x7f23d819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23de302640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23d8102ef0 0x7f23d819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23ddb01640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23d8102ef0 0x7f23d819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23ddb01640 1 -- 172.21.15.5:0/1338602188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23d819d3b0 con 0x7f23d8100e20 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.185+0000 7f23de302640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23d8102ef0 0x7f23d819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.186+0000 7f23ddb01640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8100e20 0x7f23d8198400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f23d400c920 tx=0x7f23d400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.186+0000 7f23caffd640 1 -- 172.21.15.5:0/1338602188 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4007c70 con 0x7f23d8100e20 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.186+0000 7f23dfd65640 1 -- 172.21.15.5:0/1338602188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23d819d6a0 con 0x7f23d8100e20 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.186+0000 7f23caffd640 1 -- 172.21.15.5:0/1338602188 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23d4007e10 con 0x7f23d8100e20 2024-09-15T22:07:42.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.186+0000 7f23caffd640 1 -- 172.21.15.5:0/1338602188 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4018890 con 0x7f23d8100e20 2024-09-15T22:07:42.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.186+0000 7f23dfd65640 1 -- 172.21.15.5:0/1338602188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23d81a4f80 con 0x7f23d8100e20 2024-09-15T22:07:42.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.187+0000 7f23dfd65640 1 -- 172.21.15.5:0/1338602188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23d8062940 con 0x7f23d8100e20 2024-09-15T22:07:42.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.188+0000 7f23caffd640 1 -- 172.21.15.5:0/1338602188 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f23d4018a30 con 0x7f23d8100e20 2024-09-15T22:07:42.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.189+0000 7f23caffd640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f23c40a32d0 0x7f23c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:42.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.189+0000 7f23dd300640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f23c40a32d0 0x7f23c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:42.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.189+0000 7f23caffd640 1 -- 172.21.15.5:0/1338602188 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f23d40d03f0 con 0x7f23d8100e20 2024-09-15T22:07:42.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.189+0000 7f23caffd640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025] conn(0x7f23c40ae420 0x7f23c40b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:42.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.189+0000 7f23caffd640 1 -- 172.21.15.5:0/1338602188 --> [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23c40b0f90 con 0x7f23c40ae420 2024-09-15T22:07:42.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.189+0000 7f23caffd640 1 -- 172.21.15.5:0/1338602188 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f23d40d0840 con 0x7f23d8100e20 2024-09-15T22:07:42.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.189+0000 7f23dd300640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f23c40a32d0 0x7f23c40a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f23cc005fd0 tx=0x7f23cc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:42.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.190+0000 7f23de302640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025] conn(0x7f23c40ae420 0x7f23c40b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:42.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.190+0000 7f23de302640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025] conn(0x7f23c40ae420 0x7f23c40b08c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:42.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.191+0000 7f23caffd640 1 -- 172.21.15.5:0/1338602188 <== osd.4 v2:172.21.15.99:6808/79942025 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23c40b0f90 con 0x7f23c40ae420 2024-09-15T22:07:42.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.205+0000 7f23dfd65640 1 -- 172.21.15.5:0/1338602188 --> [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f23d8062d70 con 0x7f23c40ae420 2024-09-15T22:07:42.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.206+0000 7f23caffd640 1 -- 172.21.15.5:0/1338602188 <== osd.4 v2:172.21.15.99:6808/79942025 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f23d8062d70 con 0x7f23c40ae420 2024-09-15T22:07:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.206+0000 7f23dfd65640 1 -- 172.21.15.5:0/1338602188 >> [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025] conn(0x7f23c40ae420 msgr2=0x7f23c40b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.206+0000 7f23dfd65640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025] conn(0x7f23c40ae420 0x7f23c40b08c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.207+0000 7f23dfd65640 1 -- 172.21.15.5:0/1338602188 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f23c40a32d0 msgr2=0x7f23c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.207+0000 7f23dfd65640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f23c40a32d0 0x7f23c40a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f23cc005fd0 tx=0x7f23cc005850 comp rx=0 tx=0).stop 2024-09-15T22:07:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.207+0000 7f23dfd65640 1 -- 172.21.15.5:0/1338602188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8100e20 msgr2=0x7f23d8198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.207+0000 7f23dfd65640 1 --2- 172.21.15.5:0/1338602188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23d8100e20 0x7f23d8198400 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f23d400c920 tx=0x7f23d400cde0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.207+0000 7f23de302640 1 -- 172.21.15.5:0/1338602188 reap_dead start 2024-09-15T22:07:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.207+0000 7f23dfd65640 1 -- 172.21.15.5:0/1338602188 shutdown_connections 2024-09-15T22:07:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.207+0000 7f23dfd65640 1 -- 172.21.15.5:0/1338602188 >> 172.21.15.5:0/1338602188 conn(0x7f23d80fc7d0 msgr2=0x7f23d806a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.207+0000 7f23dfd65640 1 -- 172.21.15.5:0/1338602188 shutdown_connections 2024-09-15T22:07:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.207+0000 7f23dfd65640 1 -- 172.21.15.5:0/1338602188 wait complete. 2024-09-15T22:07:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.615+0000 7f7553ac5640 1 -- 172.21.15.5:0/1970703832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f754c10e480 msgr2=0x7f754c110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.615+0000 7f7553ac5640 1 --2- 172.21.15.5:0/1970703832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f754c10e480 0x7f754c110870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7544009ca0 tx=0x7f754402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.615+0000 7f7553ac5640 1 -- 172.21.15.5:0/1970703832 shutdown_connections 2024-09-15T22:07:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.615+0000 7f7553ac5640 1 --2- 172.21.15.5:0/1970703832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f754c10e480 0x7f754c110870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.615+0000 7f7553ac5640 1 --2- 172.21.15.5:0/1970703832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c101280 0x7f754c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.615+0000 7f7553ac5640 1 --2- 172.21.15.5:0/1970703832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f754c1008d0 0x7f754c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.615+0000 7f7553ac5640 1 -- 172.21.15.5:0/1970703832 >> 172.21.15.5:0/1970703832 conn(0x7f754c0fc7a0 msgr2=0x7f754c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.615+0000 7f7553ac5640 1 -- 172.21.15.5:0/1970703832 shutdown_connections 2024-09-15T22:07:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.615+0000 7f7553ac5640 1 -- 172.21.15.5:0/1970703832 wait complete. 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.616+0000 7f7553ac5640 1 Processor -- start 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.616+0000 7f7553ac5640 1 -- start start 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.616+0000 7f7553ac5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f754c1008d0 0x7f754c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.616+0000 7f7553ac5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f754c101280 0x7f754c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.616+0000 7f7553ac5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c10e480 0x7f754c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.616+0000 7f7553ac5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1141b0 con 0x7f754c101280 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.616+0000 7f7553ac5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f754c114030 con 0x7f754c1008d0 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.616+0000 7f7553ac5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f754c114330 con 0x7f754c10e480 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7551060640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f754c101280 0x7f754c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7551060640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f754c101280 0x7f754c19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51848/0 (socket says 172.21.15.5:51848) 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7551060640 1 -- 172.21.15.5:0/1013131330 learned_addr learned my addr 172.21.15.5:0/1013131330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:42.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7551861640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f754c1008d0 0x7f754c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7552062640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c10e480 0x7f754c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7551060640 1 -- 172.21.15.5:0/1013131330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c10e480 msgr2=0x7f754c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7551060640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c10e480 0x7f754c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7551060640 1 -- 172.21.15.5:0/1013131330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f754c1008d0 msgr2=0x7f754c19a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7551060640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f754c1008d0 0x7f754c19a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7551060640 1 -- 172.21.15.5:0/1013131330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c19f5a0 con 0x7f754c101280 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7552062640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f754c10e480 0x7f754c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7551060640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f754c101280 0x7f754c19ab30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f753c00d950 tx=0x7f753c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7542ffd640 1 -- 172.21.15.5:0/1013131330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c017070 con 0x7f754c101280 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7553ac5640 1 -- 172.21.15.5:0/1013131330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c19f890 con 0x7f754c101280 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7542ffd640 1 -- 172.21.15.5:0/1013131330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f753c009c30 con 0x7f754c101280 2024-09-15T22:07:42.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7542ffd640 1 -- 172.21.15.5:0/1013131330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753c00f6b0 con 0x7f754c101280 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.617+0000 7f7553ac5640 1 -- 172.21.15.5:0/1013131330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c1a7170 con 0x7f754c101280 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.618+0000 7f7553ac5640 1 -- 172.21.15.5:0/1013131330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f754c10fa70 con 0x7f754c101280 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.619+0000 7f7542ffd640 1 -- 172.21.15.5:0/1013131330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f753c0109d0 con 0x7f754c101280 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.620+0000 7f7542ffd640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f75380a32d0 0x7f75380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.620+0000 7f7542ffd640 1 -- 172.21.15.5:0/1013131330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f753c0cfce0 con 0x7f754c101280 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.620+0000 7f7551861640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f75380a32d0 0x7f75380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.620+0000 7f7542ffd640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431] conn(0x7f75380ae420 0x7f75380b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.620+0000 7f7542ffd640 1 -- 172.21.15.5:0/1013131330 --> [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f753c024e50 con 0x7f75380ae420 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.620+0000 7f7542ffd640 1 -- 172.21.15.5:0/1013131330 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f753c0d0130 con 0x7f754c101280 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.621+0000 7f7551861640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f75380a32d0 0x7f75380a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f754800bbe0 tx=0x7f754800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.621+0000 7f7552062640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431] conn(0x7f75380ae420 0x7f75380b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.621+0000 7f7552062640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431] conn(0x7f75380ae420 0x7f75380b08c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.622+0000 7f7542ffd640 1 -- 172.21.15.5:0/1013131330 <== osd.8 v2:172.21.15.129:6816/2292146431 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f753c024e50 con 0x7f75380ae420 2024-09-15T22:07:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.636+0000 7f7553ac5640 1 -- 172.21.15.5:0/1013131330 --> [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f754c06b070 con 0x7f75380ae420 2024-09-15T22:07:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.636+0000 7f7542ffd640 1 -- 172.21.15.5:0/1013131330 <== osd.8 v2:172.21.15.129:6816/2292146431 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f754c06b070 con 0x7f75380ae420 2024-09-15T22:07:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.636+0000 7f7553ac5640 1 -- 172.21.15.5:0/1013131330 >> [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431] conn(0x7f75380ae420 msgr2=0x7f75380b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.636+0000 7f7553ac5640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431] conn(0x7f75380ae420 0x7f75380b08c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.636+0000 7f7553ac5640 1 -- 172.21.15.5:0/1013131330 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f75380a32d0 msgr2=0x7f75380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.636+0000 7f7553ac5640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f75380a32d0 0x7f75380a5790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f754800bbe0 tx=0x7f754800b390 comp rx=0 tx=0).stop 2024-09-15T22:07:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.636+0000 7f7553ac5640 1 -- 172.21.15.5:0/1013131330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f754c101280 msgr2=0x7f754c19ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.636+0000 7f7553ac5640 1 --2- 172.21.15.5:0/1013131330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f754c101280 0x7f754c19ab30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f753c00d950 tx=0x7f753c00de10 comp rx=0 tx=0).stop 2024-09-15T22:07:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.636+0000 7f7552062640 1 -- 172.21.15.5:0/1013131330 reap_dead start 2024-09-15T22:07:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.637+0000 7f7553ac5640 1 -- 172.21.15.5:0/1013131330 shutdown_connections 2024-09-15T22:07:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.637+0000 7f7553ac5640 1 -- 172.21.15.5:0/1013131330 >> 172.21.15.5:0/1013131330 conn(0x7f754c0fc7a0 msgr2=0x7f754c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.637+0000 7f7553ac5640 1 -- 172.21.15.5:0/1013131330 shutdown_connections 2024-09-15T22:07:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:42.637+0000 7f7553ac5640 1 -- 172.21.15.5:0/1013131330 wait complete. 2024-09-15T22:07:42.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:42 smithi005 ceph-mon[30944]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-15T22:07:43.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:42 smithi099 ceph-mon[34892]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-15T22:07:43.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:42 smithi129 ceph-mon[35314]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-15T22:07:44.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.203+0000 7f65dcc56640 1 -- 172.21.15.5:0/1958267746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8101de0 msgr2=0x7f65d81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.203+0000 7f65dcc56640 1 --2- 172.21.15.5:0/1958267746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8101de0 0x7f65d81021c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009900 tx=0x7f65c802a410 comp rx=0 tx=0).stop 2024-09-15T22:07:44.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.203+0000 7f65dcc56640 1 -- 172.21.15.5:0/1958267746 shutdown_connections 2024-09-15T22:07:44.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.203+0000 7f65dcc56640 1 --2- 172.21.15.5:0/1958267746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d8106f10 0x7f65d810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.203+0000 7f65dcc56640 1 --2- 172.21.15.5:0/1958267746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65d8102790 0x7f65d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.203+0000 7f65dcc56640 1 --2- 172.21.15.5:0/1958267746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8101de0 0x7f65d81021c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.203+0000 7f65dcc56640 1 -- 172.21.15.5:0/1958267746 >> 172.21.15.5:0/1958267746 conn(0x7f65d80fd8c0 msgr2=0x7f65d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:44.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.203+0000 7f65dcc56640 1 -- 172.21.15.5:0/1958267746 shutdown_connections 2024-09-15T22:07:44.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.203+0000 7f65dcc56640 1 -- 172.21.15.5:0/1958267746 wait complete. 2024-09-15T22:07:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.203+0000 7f65dcc56640 1 Processor -- start 2024-09-15T22:07:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65dcc56640 1 -- start start 2024-09-15T22:07:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65dcc56640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65d8101de0 0x7f65d8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65dcc56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8102790 0x7f65d8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65dcc56640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d8106f10 0x7f65d819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65dcc56640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8111f70 con 0x7f65d8102790 2024-09-15T22:07:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65dcc56640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f65d8111df0 con 0x7f65d8106f10 2024-09-15T22:07:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65dcc56640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f65d81120f0 con 0x7f65d8101de0 2024-09-15T22:07:44.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8102790 0x7f65d8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8102790 0x7f65d8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56952/0 (socket says 172.21.15.5:56952) 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d5d9b640 1 -- 172.21.15.5:0/3541760712 learned_addr learned my addr 172.21.15.5:0/3541760712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d6d9d640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d8106f10 0x7f65d819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d659c640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65d8101de0 0x7f65d8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d5d9b640 1 -- 172.21.15.5:0/3541760712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65d8101de0 msgr2=0x7f65d8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d5d9b640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f65d8101de0 0x7f65d8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d5d9b640 1 -- 172.21.15.5:0/3541760712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d8106f10 msgr2=0x7f65d819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d5d9b640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d8106f10 0x7f65d819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d5d9b640 1 -- 172.21.15.5:0/3541760712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65d819d3f0 con 0x7f65d8102790 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d6d9d640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f65d8106f10 0x7f65d819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.204+0000 7f65d5d9b640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8102790 0x7f65d8198980 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00e9f0 tx=0x7f65cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.205+0000 7f65c77fe640 1 -- 172.21.15.5:0/3541760712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc00cd80 con 0x7f65d8102790 2024-09-15T22:07:44.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.205+0000 7f65c77fe640 1 -- 172.21.15.5:0/3541760712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65cc005500 con 0x7f65d8102790 2024-09-15T22:07:44.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.205+0000 7f65dcc56640 1 -- 172.21.15.5:0/3541760712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65d819d6e0 con 0x7f65d8102790 2024-09-15T22:07:44.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.205+0000 7f65c77fe640 1 -- 172.21.15.5:0/3541760712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65cc010690 con 0x7f65d8102790 2024-09-15T22:07:44.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.205+0000 7f65dcc56640 1 -- 172.21.15.5:0/3541760712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65d81a4fc0 con 0x7f65d8102790 2024-09-15T22:07:44.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.206+0000 7f65dcc56640 1 -- 172.21.15.5:0/3541760712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65d8062980 con 0x7f65d8102790 2024-09-15T22:07:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.207+0000 7f65c77fe640 1 -- 172.21.15.5:0/3541760712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f65cc0046c0 con 0x7f65d8102790 2024-09-15T22:07:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.208+0000 7f65c77fe640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f65c00a3350 0x7f65c00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.208+0000 7f65d659c640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f65c00a3350 0x7f65c00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.208+0000 7f65c77fe640 1 -- 172.21.15.5:0/3541760712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f65cc0d0710 con 0x7f65d8102790 2024-09-15T22:07:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.208+0000 7f65c77fe640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498] conn(0x7f65c00ae4a0 0x7f65c00b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.208+0000 7f65c77fe640 1 -- 172.21.15.5:0/3541760712 --> [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65cc01aec0 con 0x7f65c00ae4a0 2024-09-15T22:07:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.208+0000 7f65c77fe640 1 -- 172.21.15.5:0/3541760712 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f65cc0d0b60 con 0x7f65d8102790 2024-09-15T22:07:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.208+0000 7f65d659c640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f65c00a3350 0x7f65c00a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65c8005ab0 tx=0x7f65c8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:44.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.208+0000 7f65d6d9d640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498] conn(0x7f65c00ae4a0 0x7f65c00b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:44.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.209+0000 7f65d6d9d640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498] conn(0x7f65c00ae4a0 0x7f65c00b0940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:44.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.210+0000 7f65c77fe640 1 -- 172.21.15.5:0/3541760712 <== osd.10 v2:172.21.15.99:6824/154036498 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65cc01aec0 con 0x7f65c00ae4a0 2024-09-15T22:07:44.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.224+0000 7f65dcc56640 1 -- 172.21.15.5:0/3541760712 --> [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65d8062e00 con 0x7f65c00ae4a0 2024-09-15T22:07:44.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.224+0000 7f65c77fe640 1 -- 172.21.15.5:0/3541760712 <== osd.10 v2:172.21.15.99:6824/154036498 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f65d8062e00 con 0x7f65c00ae4a0 2024-09-15T22:07:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.224+0000 7f65dcc56640 1 -- 172.21.15.5:0/3541760712 >> [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498] conn(0x7f65c00ae4a0 msgr2=0x7f65c00b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.224+0000 7f65dcc56640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498] conn(0x7f65c00ae4a0 0x7f65c00b0940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.224+0000 7f65dcc56640 1 -- 172.21.15.5:0/3541760712 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f65c00a3350 msgr2=0x7f65c00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.224+0000 7f65dcc56640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f65c00a3350 0x7f65c00a5810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f65c8005ab0 tx=0x7f65c8005a20 comp rx=0 tx=0).stop 2024-09-15T22:07:44.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.224+0000 7f65dcc56640 1 -- 172.21.15.5:0/3541760712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8102790 msgr2=0x7f65d8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.224+0000 7f65dcc56640 1 --2- 172.21.15.5:0/3541760712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65d8102790 0x7f65d8198980 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00e9f0 tx=0x7f65cc00edf0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.224+0000 7f65d6d9d640 1 -- 172.21.15.5:0/3541760712 reap_dead start 2024-09-15T22:07:44.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.225+0000 7f65dcc56640 1 -- 172.21.15.5:0/3541760712 shutdown_connections 2024-09-15T22:07:44.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.225+0000 7f65dcc56640 1 -- 172.21.15.5:0/3541760712 >> 172.21.15.5:0/3541760712 conn(0x7f65d80fd8c0 msgr2=0x7f65d80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:44.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.225+0000 7f65dcc56640 1 -- 172.21.15.5:0/3541760712 shutdown_connections 2024-09-15T22:07:44.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:44.225+0000 7f65dcc56640 1 -- 172.21.15.5:0/3541760712 wait complete. 2024-09-15T22:07:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.685+0000 7f49e1639640 1 -- 172.21.15.5:0/3792299641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49dc101de0 msgr2=0x7f49dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.685+0000 7f49e1639640 1 --2- 172.21.15.5:0/3792299641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49dc101de0 0x7f49dc1021c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009900 tx=0x7f49cc02a410 comp rx=0 tx=0).stop 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.685+0000 7f49e1639640 1 -- 172.21.15.5:0/3792299641 shutdown_connections 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.685+0000 7f49e1639640 1 --2- 172.21.15.5:0/3792299641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49dc106f10 0x7f49dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.685+0000 7f49e1639640 1 --2- 172.21.15.5:0/3792299641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49dc102790 0x7f49dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.685+0000 7f49e1639640 1 --2- 172.21.15.5:0/3792299641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49dc101de0 0x7f49dc1021c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.685+0000 7f49e1639640 1 -- 172.21.15.5:0/3792299641 >> 172.21.15.5:0/3792299641 conn(0x7f49dc0fd8c0 msgr2=0x7f49dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.685+0000 7f49e1639640 1 -- 172.21.15.5:0/3792299641 shutdown_connections 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.685+0000 7f49e1639640 1 -- 172.21.15.5:0/3792299641 wait complete. 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.685+0000 7f49e1639640 1 Processor -- start 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.685+0000 7f49e1639640 1 -- start start 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49e1639640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49dc101de0 0x7f49dc1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49e1639640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49dc102790 0x7f49dc198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:44.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49e1639640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49dc106f10 0x7f49dc19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49e1639640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc111fe0 con 0x7f49dc101de0 2024-09-15T22:07:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49e1639640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc111e60 con 0x7f49dc102790 2024-09-15T22:07:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49e1639640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc112160 con 0x7f49dc106f10 2024-09-15T22:07:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49daffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49dc101de0 0x7f49dc1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49daffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49dc101de0 0x7f49dc1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56946/0 (socket says 172.21.15.5:56946) 2024-09-15T22:07:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49daffd640 1 -- 172.21.15.5:0/2822419494 learned_addr learned my addr 172.21.15.5:0/2822419494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49db7fe640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49dc106f10 0x7f49dc19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49da7fc640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49dc102790 0x7f49dc198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49daffd640 1 -- 172.21.15.5:0/2822419494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49dc106f10 msgr2=0x7f49dc19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49daffd640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f49dc106f10 0x7f49dc19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49daffd640 1 -- 172.21.15.5:0/2822419494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49dc102790 msgr2=0x7f49dc198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49daffd640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f49dc102790 0x7f49dc198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49daffd640 1 -- 172.21.15.5:0/2822419494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc19d390 con 0x7f49dc101de0 2024-09-15T22:07:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.686+0000 7f49daffd640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49dc101de0 0x7f49dc1983e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f49cc006010 tx=0x7f49cc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.687+0000 7f49d3fff640 1 -- 172.21.15.5:0/2822419494 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc03f070 con 0x7f49dc101de0 2024-09-15T22:07:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.687+0000 7f49d3fff640 1 -- 172.21.15.5:0/2822419494 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49cc031c90 con 0x7f49dc101de0 2024-09-15T22:07:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.687+0000 7f49e1639640 1 -- 172.21.15.5:0/2822419494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc19d620 con 0x7f49dc101de0 2024-09-15T22:07:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.687+0000 7f49d3fff640 1 -- 172.21.15.5:0/2822419494 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc035030 con 0x7f49dc101de0 2024-09-15T22:07:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.687+0000 7f49e1639640 1 -- 172.21.15.5:0/2822419494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc1a4f60 con 0x7f49dc101de0 2024-09-15T22:07:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.687+0000 7f49e1639640 1 -- 172.21.15.5:0/2822419494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49dc062980 con 0x7f49dc101de0 2024-09-15T22:07:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.689+0000 7f49d3fff640 1 -- 172.21.15.5:0/2822419494 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f49cc042070 con 0x7f49dc101de0 2024-09-15T22:07:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.689+0000 7f49d3fff640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f49c80a32d0 0x7f49c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.689+0000 7f49da7fc640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f49c80a32d0 0x7f49c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.689+0000 7f49d3fff640 1 -- 172.21.15.5:0/2822419494 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9071+0+0 (secure 0 0 0) 0x7f49cc0f0bb0 con 0x7f49dc101de0 2024-09-15T22:07:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.690+0000 7f49d3fff640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223] conn(0x7f49c80ae420 0x7f49c80b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.690+0000 7f49d3fff640 1 -- 172.21.15.5:0/2822419494 --> [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49cc038e90 con 0x7f49c80ae420 2024-09-15T22:07:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.690+0000 7f49d3fff640 1 -- 172.21.15.5:0/2822419494 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f49cc0f1000 con 0x7f49dc101de0 2024-09-15T22:07:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.690+0000 7f49db7fe640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223] conn(0x7f49c80ae420 0x7f49c80b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.690+0000 7f49da7fc640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f49c80a32d0 0x7f49c80a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005e10 tx=0x7f49c4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.690+0000 7f49db7fe640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223] conn(0x7f49c80ae420 0x7f49c80b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.691+0000 7f49d3fff640 1 -- 172.21.15.5:0/2822419494 <== osd.6 v2:172.21.15.5:6818/1626090223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49cc038e90 con 0x7f49c80ae420 2024-09-15T22:07:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.704+0000 7f49e1639640 1 -- 172.21.15.5:0/2822419494 --> [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49dc062e00 con 0x7f49c80ae420 2024-09-15T22:07:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49d3fff640 1 -- 172.21.15.5:0/2822419494 <== osd.6 v2:172.21.15.5:6818/1626090223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f49dc062e00 con 0x7f49c80ae420 2024-09-15T22:07:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49e1639640 1 -- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223] conn(0x7f49c80ae420 msgr2=0x7f49c80b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49e1639640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223] conn(0x7f49c80ae420 0x7f49c80b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49e1639640 1 -- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f49c80a32d0 msgr2=0x7f49c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49e1639640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f49c80a32d0 0x7f49c80a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f49c4005e10 tx=0x7f49c4005da0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49e1639640 1 -- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49dc101de0 msgr2=0x7f49dc1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49e1639640 1 --2- 172.21.15.5:0/2822419494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49dc101de0 0x7f49dc1983e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f49cc006010 tx=0x7f49cc003ba0 comp rx=0 tx=0).stop 2024-09-15T22:07:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49db7fe640 1 -- 172.21.15.5:0/2822419494 reap_dead start 2024-09-15T22:07:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49e1639640 1 -- 172.21.15.5:0/2822419494 shutdown_connections 2024-09-15T22:07:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49e1639640 1 -- 172.21.15.5:0/2822419494 >> 172.21.15.5:0/2822419494 conn(0x7f49dc0fd8c0 msgr2=0x7f49dc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49e1639640 1 -- 172.21.15.5:0/2822419494 shutdown_connections 2024-09-15T22:07:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:43.705+0000 7f49e1639640 1 -- 172.21.15.5:0/2822419494 wait complete. 2024-09-15T22:07:45.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:44 smithi005 ceph-mon[30944]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-15T22:07:45.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4, 7, 11]}]: dispatch 2024-09-15T22:07:45.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-15T22:07:45.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-15T22:07:45.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-15T22:07:45.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:44 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2024-09-15T22:07:45.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:44 smithi099 ceph-mon[34892]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-15T22:07:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4, 7, 11]}]: dispatch 2024-09-15T22:07:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-15T22:07:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-15T22:07:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-15T22:07:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:44 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2024-09-15T22:07:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:44 smithi129 ceph-mon[35314]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-15T22:07:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4, 7, 11]}]: dispatch 2024-09-15T22:07:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]: dispatch 2024-09-15T22:07:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-15T22:07:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-15T22:07:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:44 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]: dispatch 2024-09-15T22:07:45.188 INFO:teuthology.orchestra.run.smithi005.stdout:231928234024 2024-09-15T22:07:45.188 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-15T22:07:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.564+0000 7f66ebdaf640 1 -- 172.21.15.5:0/3721355922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4102790 msgr2=0x7f66e41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.564+0000 7f66ebdaf640 1 --2- 172.21.15.5:0/3721355922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4102790 0x7f66e41067e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009900 tx=0x7f66d402a410 comp rx=0 tx=0).stop 2024-09-15T22:07:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.564+0000 7f66ebdaf640 1 -- 172.21.15.5:0/3721355922 shutdown_connections 2024-09-15T22:07:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.564+0000 7f66ebdaf640 1 --2- 172.21.15.5:0/3721355922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66e4106f10 0x7f66e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.564+0000 7f66ebdaf640 1 --2- 172.21.15.5:0/3721355922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4102790 0x7f66e41067e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.564+0000 7f66ebdaf640 1 --2- 172.21.15.5:0/3721355922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66e4101de0 0x7f66e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.564+0000 7f66ebdaf640 1 -- 172.21.15.5:0/3721355922 >> 172.21.15.5:0/3721355922 conn(0x7f66e40fd8c0 msgr2=0x7f66e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.564+0000 7f66ebdaf640 1 -- 172.21.15.5:0/3721355922 shutdown_connections 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.564+0000 7f66ebdaf640 1 -- 172.21.15.5:0/3721355922 wait complete. 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.564+0000 7f66ebdaf640 1 Processor -- start 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.564+0000 7f66ebdaf640 1 -- start start 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ebdaf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66e4101de0 0x7f66e4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ebdaf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66e4102790 0x7f66e41989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ebdaf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4106f10 0x7f66e419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ebdaf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4111f90 con 0x7f66e4106f10 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ebdaf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4111e10 con 0x7f66e4101de0 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ebdaf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4112110 con 0x7f66e4102790 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ea34c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4106f10 0x7f66e419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ea34c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4106f10 0x7f66e419cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56962/0 (socket says 172.21.15.5:56962) 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ea34c640 1 -- 172.21.15.5:0/907920977 learned_addr learned my addr 172.21.15.5:0/907920977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:45.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66e934a640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66e4102790 0x7f66e41989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:45.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66e9b4b640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66e4101de0 0x7f66e4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:45.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ea34c640 1 -- 172.21.15.5:0/907920977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66e4102790 msgr2=0x7f66e41989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:45.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ea34c640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f66e4102790 0x7f66e41989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:45.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ea34c640 1 -- 172.21.15.5:0/907920977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66e4101de0 msgr2=0x7f66e4198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:45.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ea34c640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f66e4101de0 0x7f66e4198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:45.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ea34c640 1 -- 172.21.15.5:0/907920977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e419d440 con 0x7f66e4106f10 2024-09-15T22:07:45.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ea34c640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4106f10 0x7f66e419cd60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00b970 tx=0x7f66dc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:45.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66daffd640 1 -- 172.21.15.5:0/907920977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc019070 con 0x7f66e4106f10 2024-09-15T22:07:45.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66daffd640 1 -- 172.21.15.5:0/907920977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66dc00cb40 con 0x7f66e4106f10 2024-09-15T22:07:45.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ebdaf640 1 -- 172.21.15.5:0/907920977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e419d730 con 0x7f66e4106f10 2024-09-15T22:07:45.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66daffd640 1 -- 172.21.15.5:0/907920977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc007650 con 0x7f66e4106f10 2024-09-15T22:07:45.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.565+0000 7f66ebdaf640 1 -- 172.21.15.5:0/907920977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e41a5010 con 0x7f66e4106f10 2024-09-15T22:07:45.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.566+0000 7f66ebdaf640 1 -- 172.21.15.5:0/907920977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f66e4062980 con 0x7f66e4106f10 2024-09-15T22:07:45.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.568+0000 7f66daffd640 1 -- 172.21.15.5:0/907920977 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f66dc01c070 con 0x7f66e4106f10 2024-09-15T22:07:45.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.568+0000 7f66daffd640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f66d00a32d0 0x7f66d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:45.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.568+0000 7f66daffd640 1 -- 172.21.15.5:0/907920977 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9195+0+0 (secure 0 0 0) 0x7f66dc0cfbf0 con 0x7f66e4106f10 2024-09-15T22:07:45.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.568+0000 7f66e9b4b640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f66d00a32d0 0x7f66d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:45.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.568+0000 7f66daffd640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616] conn(0x7f66d00ae620 0x7f66d00b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:45.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.568+0000 7f66daffd640 1 -- 172.21.15.5:0/907920977 --> [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f66dc0047d0 con 0x7f66d00ae620 2024-09-15T22:07:45.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.568+0000 7f66daffd640 1 -- 172.21.15.5:0/907920977 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f66dc0d0040 con 0x7f66e4106f10 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.569+0000 7f66e934a640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616] conn(0x7f66d00ae620 0x7f66d00b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.569+0000 7f66e9b4b640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f66d00a32d0 0x7f66d00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f66e0005fd0 tx=0x7f66e0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.569+0000 7f66e934a640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616] conn(0x7f66d00ae620 0x7f66d00b0ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.570+0000 7f66daffd640 1 -- 172.21.15.5:0/907920977 <== osd.3 v2:172.21.15.5:6810/1557519616 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f66dc0047d0 con 0x7f66d00ae620 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ebdaf640 1 -- 172.21.15.5:0/907920977 --> [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f66e4062e00 con 0x7f66d00ae620 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66daffd640 1 -- 172.21.15.5:0/907920977 <== osd.3 v2:172.21.15.5:6810/1557519616 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f66e4062e00 con 0x7f66d00ae620 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ebdaf640 1 -- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616] conn(0x7f66d00ae620 msgr2=0x7f66d00b0ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ebdaf640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616] conn(0x7f66d00ae620 0x7f66d00b0ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ebdaf640 1 -- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f66d00a32d0 msgr2=0x7f66d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ebdaf640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f66d00a32d0 0x7f66d00a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f66e0005fd0 tx=0x7f66e0005850 comp rx=0 tx=0).stop 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ebdaf640 1 -- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4106f10 msgr2=0x7f66e419cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ebdaf640 1 --2- 172.21.15.5:0/907920977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66e4106f10 0x7f66e419cd60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00b970 tx=0x7f66dc00be30 comp rx=0 tx=0).stop 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ea34c640 1 -- 172.21.15.5:0/907920977 reap_dead start 2024-09-15T22:07:45.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ebdaf640 1 -- 172.21.15.5:0/907920977 shutdown_connections 2024-09-15T22:07:45.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ebdaf640 1 -- 172.21.15.5:0/907920977 >> 172.21.15.5:0/907920977 conn(0x7f66e40fd8c0 msgr2=0x7f66e40fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:45.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ebdaf640 1 -- 172.21.15.5:0/907920977 shutdown_connections 2024-09-15T22:07:45.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:45.584+0000 7f66ebdaf640 1 -- 172.21.15.5:0/907920977 wait complete. 2024-09-15T22:07:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:45 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4, 7, 11]}]': finished 2024-09-15T22:07:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:45 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-15T22:07:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:45 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-15T22:07:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:45 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-15T22:07:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:45 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2024-09-15T22:07:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:45 smithi005 ceph-mon[30944]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:07:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:45 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4, 7, 11]}]': finished 2024-09-15T22:07:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:45 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-15T22:07:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:45 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-15T22:07:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:45 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-15T22:07:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:45 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2024-09-15T22:07:46.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:45 smithi099 ceph-mon[34892]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:07:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:45 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 4, 7, 11]}]': finished 2024-09-15T22:07:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:45 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [9, 11]}]': finished 2024-09-15T22:07:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:45 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-15T22:07:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:45 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-15T22:07:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:45 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [5, 8]}]': finished 2024-09-15T22:07:46.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:45 smithi129 ceph-mon[35314]: osdmap e87: 12 total, 12 up, 12 in 2024-09-15T22:07:46.286 INFO:teuthology.orchestra.run.smithi005.stdout:176093659188 2024-09-15T22:07:46.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-15T22:07:46.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:46 smithi005 ceph-mon[30944]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-15T22:07:46.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:46 smithi005 ceph-mon[30944]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:07:47.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:46 smithi099 ceph-mon[34892]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-15T22:07:47.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:46 smithi099 ceph-mon[34892]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:07:47.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:46 smithi129 ceph-mon[35314]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-15T22:07:47.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:46 smithi129 ceph-mon[35314]: osdmap e88: 12 total, 12 up, 12 in 2024-09-15T22:07:47.481 INFO:teuthology.orchestra.run.smithi005.stdout:128849018942 2024-09-15T22:07:47.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-15T22:07:47.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.648+0000 7f52d42c3640 1 -- 172.21.15.5:0/2315592018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc102ef0 msgr2=0x7f52cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:47.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.648+0000 7f52d42c3640 1 --2- 172.21.15.5:0/2315592018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc102ef0 0x7f52cc10da40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009ca0 tx=0x7f52c402a950 comp rx=0 tx=0).stop 2024-09-15T22:07:47.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.648+0000 7f52d42c3640 1 -- 172.21.15.5:0/2315592018 shutdown_connections 2024-09-15T22:07:47.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.648+0000 7f52d42c3640 1 --2- 172.21.15.5:0/2315592018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc102ef0 0x7f52cc10da40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.648+0000 7f52d42c3640 1 --2- 172.21.15.5:0/2315592018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52cc102570 0x7f52cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.648+0000 7f52d42c3640 1 --2- 172.21.15.5:0/2315592018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52cc100e20 0x7f52cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.648+0000 7f52d42c3640 1 -- 172.21.15.5:0/2315592018 >> 172.21.15.5:0/2315592018 conn(0x7f52cc0fc7d0 msgr2=0x7f52cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.649+0000 7f52d42c3640 1 -- 172.21.15.5:0/2315592018 shutdown_connections 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.649+0000 7f52d42c3640 1 -- 172.21.15.5:0/2315592018 wait complete. 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.649+0000 7f52d42c3640 1 Processor -- start 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.649+0000 7f52d42c3640 1 -- start start 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.649+0000 7f52d42c3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52cc100e20 0x7f52cc19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.649+0000 7f52d42c3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52cc102570 0x7f52cc19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.649+0000 7f52d42c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc102ef0 0x7f52cc19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.649+0000 7f52d42c3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc110fa0 con 0x7f52cc102ef0 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.649+0000 7f52d42c3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc110e20 con 0x7f52cc102570 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.649+0000 7f52d42c3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc111120 con 0x7f52cc100e20 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d2860640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc102ef0 0x7f52cc19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:47.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d2860640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc102ef0 0x7f52cc19efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56984/0 (socket says 172.21.15.5:56984) 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d2860640 1 -- 172.21.15.5:0/1119866849 learned_addr learned my addr 172.21.15.5:0/1119866849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d185e640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52cc102570 0x7f52cc19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d2860640 1 -- 172.21.15.5:0/1119866849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52cc100e20 msgr2=0x7f52cc19a6d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d2860640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52cc100e20 0x7f52cc19a6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d2860640 1 -- 172.21.15.5:0/1119866849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52cc102570 msgr2=0x7f52cc19ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d2860640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52cc102570 0x7f52cc19ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d2860640 1 -- 172.21.15.5:0/1119866849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc19f680 con 0x7f52cc102ef0 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d2860640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc102ef0 0x7f52cc19efa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f52c4006010 tx=0x7f52c402f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52c37fe640 1 -- 172.21.15.5:0/1119866849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4034040 con 0x7f52cc102ef0 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52c37fe640 1 -- 172.21.15.5:0/1119866849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52c4003e00 con 0x7f52cc102ef0 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52c37fe640 1 -- 172.21.15.5:0/1119866849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4045480 con 0x7f52cc102ef0 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d42c3640 1 -- 172.21.15.5:0/1119866849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc19f910 con 0x7f52cc102ef0 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.650+0000 7f52d42c3640 1 -- 172.21.15.5:0/1119866849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc1a7250 con 0x7f52cc102ef0 2024-09-15T22:07:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.651+0000 7f52d42c3640 1 -- 172.21.15.5:0/1119866849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52cc106b20 con 0x7f52cc102ef0 2024-09-15T22:07:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.653+0000 7f52c37fe640 1 -- 172.21.15.5:0/1119866849 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f52c4037020 con 0x7f52cc102ef0 2024-09-15T22:07:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.653+0000 7f52c37fe640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f52b80a32d0 0x7f52b80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.653+0000 7f52c37fe640 1 -- 172.21.15.5:0/1119866849 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f52c40f0fa0 con 0x7f52cc102ef0 2024-09-15T22:07:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.653+0000 7f52d205f640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f52b80a32d0 0x7f52b80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.654+0000 7f52c37fe640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949] conn(0x7f52b80ae620 0x7f52b80b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.654+0000 7f52c37fe640 1 -- 172.21.15.5:0/1119866849 --> [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52b80b1190 con 0x7f52b80ae620 2024-09-15T22:07:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.654+0000 7f52c37fe640 1 -- 172.21.15.5:0/1119866849 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f52c40f13f0 con 0x7f52cc102ef0 2024-09-15T22:07:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.654+0000 7f52d205f640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f52b80a32d0 0x7f52b80a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f52c800a890 tx=0x7f52c8005b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.654+0000 7f52d185e640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949] conn(0x7f52b80ae620 0x7f52b80b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.655+0000 7f52d185e640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949] conn(0x7f52b80ae620 0x7f52b80b0ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.656+0000 7f52c37fe640 1 -- 172.21.15.5:0/1119866849 <== osd.5 v2:172.21.15.129:6808/3964498949 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52b80b1190 con 0x7f52b80ae620 2024-09-15T22:07:47.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.669+0000 7f52d42c3640 1 -- 172.21.15.5:0/1119866849 --> [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f52cc101200 con 0x7f52b80ae620 2024-09-15T22:07:47.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.670+0000 7f52c37fe640 1 -- 172.21.15.5:0/1119866849 <== osd.5 v2:172.21.15.129:6808/3964498949 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f52cc101200 con 0x7f52b80ae620 2024-09-15T22:07:47.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.670+0000 7f52d42c3640 1 -- 172.21.15.5:0/1119866849 >> [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949] conn(0x7f52b80ae620 msgr2=0x7f52b80b0ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:47.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.670+0000 7f52d42c3640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949] conn(0x7f52b80ae620 0x7f52b80b0ac0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.670+0000 7f52d42c3640 1 -- 172.21.15.5:0/1119866849 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f52b80a32d0 msgr2=0x7f52b80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:47.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.670+0000 7f52d42c3640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f52b80a32d0 0x7f52b80a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f52c800a890 tx=0x7f52c8005b80 comp rx=0 tx=0).stop 2024-09-15T22:07:47.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.670+0000 7f52d42c3640 1 -- 172.21.15.5:0/1119866849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc102ef0 msgr2=0x7f52cc19efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:47.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.670+0000 7f52d42c3640 1 --2- 172.21.15.5:0/1119866849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52cc102ef0 0x7f52cc19efa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f52c4006010 tx=0x7f52c402f6f0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.670+0000 7f52d2860640 1 -- 172.21.15.5:0/1119866849 reap_dead start 2024-09-15T22:07:47.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.671+0000 7f52d42c3640 1 -- 172.21.15.5:0/1119866849 shutdown_connections 2024-09-15T22:07:47.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.671+0000 7f52d42c3640 1 -- 172.21.15.5:0/1119866849 >> 172.21.15.5:0/1119866849 conn(0x7f52cc0fc7d0 msgr2=0x7f52cc06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:47.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.671+0000 7f52d42c3640 1 -- 172.21.15.5:0/1119866849 shutdown_connections 2024-09-15T22:07:47.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.671+0000 7f52d42c3640 1 -- 172.21.15.5:0/1119866849 wait complete. 2024-09-15T22:07:47.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:47 smithi005 ceph-mon[30944]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-15T22:07:47.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.854+0000 7efce50c2640 1 -- 172.21.15.5:0/403465129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0103760 msgr2=0x7efce0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:47.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.854+0000 7efce50c2640 1 --2- 172.21.15.5:0/403465129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0103760 0x7efce0107ba0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009930 tx=0x7efcd402a420 comp rx=0 tx=0).stop 2024-09-15T22:07:47.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.854+0000 7efce50c2640 1 -- 172.21.15.5:0/403465129 shutdown_connections 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.854+0000 7efce50c2640 1 --2- 172.21.15.5:0/403465129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efce01082e0 0x7efce010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.854+0000 7efce50c2640 1 --2- 172.21.15.5:0/403465129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0103760 0x7efce0107ba0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.854+0000 7efce50c2640 1 --2- 172.21.15.5:0/403465129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0102db0 0x7efce0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.854+0000 7efce50c2640 1 -- 172.21.15.5:0/403465129 >> 172.21.15.5:0/403465129 conn(0x7efce00fc7d0 msgr2=0x7efce00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.854+0000 7efce50c2640 1 -- 172.21.15.5:0/403465129 shutdown_connections 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.854+0000 7efce50c2640 1 -- 172.21.15.5:0/403465129 wait complete. 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.854+0000 7efce50c2640 1 Processor -- start 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efce50c2640 1 -- start start 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efce50c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0102db0 0x7efce01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efce50c2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0103760 0x7efce0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efce50c2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efce01082e0 0x7efce019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efce50c2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efce0112070 con 0x7efce0102db0 2024-09-15T22:07:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efce50c2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efce0111ef0 con 0x7efce0103760 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efce50c2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efce01121f0 con 0x7efce01082e0 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efcded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0102db0 0x7efce01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efcded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0102db0 0x7efce01985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57008/0 (socket says 172.21.15.5:57008) 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efcded9d640 1 -- 172.21.15.5:0/2130427793 learned_addr learned my addr 172.21.15.5:0/2130427793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efcde59c640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0103760 0x7efce0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efcded9d640 1 -- 172.21.15.5:0/2130427793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efce01082e0 msgr2=0x7efce019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efcded9d640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efce01082e0 0x7efce019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efcded9d640 1 -- 172.21.15.5:0/2130427793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0103760 msgr2=0x7efce0198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efcded9d640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0103760 0x7efce0198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efcded9d640 1 -- 172.21.15.5:0/2130427793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce019d580 con 0x7efce0102db0 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.855+0000 7efcde59c640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efce0103760 0x7efce0198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.856+0000 7efcded9d640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0102db0 0x7efce01985d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efcd000d950 tx=0x7efcd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.856+0000 7efcc7fff640 1 -- 172.21.15.5:0/2130427793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0017030 con 0x7efce0102db0 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.856+0000 7efce50c2640 1 -- 172.21.15.5:0/2130427793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce019d870 con 0x7efce0102db0 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.856+0000 7efcc7fff640 1 -- 172.21.15.5:0/2130427793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd0004020 con 0x7efce0102db0 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.856+0000 7efcc7fff640 1 -- 172.21.15.5:0/2130427793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd00047a0 con 0x7efce0102db0 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.856+0000 7efce50c2640 1 -- 172.21.15.5:0/2130427793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce01a5150 con 0x7efce0102db0 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.857+0000 7efce50c2640 1 -- 172.21.15.5:0/2130427793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efce0062980 con 0x7efce0102db0 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.858+0000 7efcc7fff640 1 -- 172.21.15.5:0/2130427793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7efcd00041d0 con 0x7efce0102db0 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.859+0000 7efcc7fff640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efcc80a32d0 0x7efcc80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.859+0000 7efcc7fff640 1 -- 172.21.15.5:0/2130427793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7efcd00cffd0 con 0x7efce0102db0 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.859+0000 7efcde59c640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efcc80a32d0 0x7efcc80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.859+0000 7efcc7fff640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789] conn(0x7efcc80ae620 0x7efcc80b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.859+0000 7efcc7fff640 1 -- 172.21.15.5:0/2130427793 --> [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efcc80b1190 con 0x7efcc80ae620 2024-09-15T22:07:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.859+0000 7efcc7fff640 1 -- 172.21.15.5:0/2130427793 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7efcd00d0420 con 0x7efce0102db0 2024-09-15T22:07:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.860+0000 7efcde59c640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efcc80a32d0 0x7efcc80a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efcd402a940 tx=0x7efcd40040e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.860+0000 7efcdf59e640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789] conn(0x7efcc80ae620 0x7efcc80b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.860+0000 7efcdf59e640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789] conn(0x7efcc80ae620 0x7efcc80b0ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.861+0000 7efcc7fff640 1 -- 172.21.15.5:0/2130427793 <== osd.11 v2:172.21.15.129:6824/4002757789 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efcc80b1190 con 0x7efcc80ae620 2024-09-15T22:07:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.875+0000 7efce50c2640 1 -- 172.21.15.5:0/2130427793 --> [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efce0062e00 con 0x7efcc80ae620 2024-09-15T22:07:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.875+0000 7efcc7fff640 1 -- 172.21.15.5:0/2130427793 <== osd.11 v2:172.21.15.129:6824/4002757789 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efce0062e00 con 0x7efcc80ae620 2024-09-15T22:07:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.876+0000 7efce50c2640 1 -- 172.21.15.5:0/2130427793 >> [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789] conn(0x7efcc80ae620 msgr2=0x7efcc80b0ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.876+0000 7efce50c2640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789] conn(0x7efcc80ae620 0x7efcc80b0ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.876+0000 7efce50c2640 1 -- 172.21.15.5:0/2130427793 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efcc80a32d0 msgr2=0x7efcc80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.876+0000 7efce50c2640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7efcc80a32d0 0x7efcc80a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efcd402a940 tx=0x7efcd40040e0 comp rx=0 tx=0).stop 2024-09-15T22:07:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.876+0000 7efce50c2640 1 -- 172.21.15.5:0/2130427793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0102db0 msgr2=0x7efce01985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.876+0000 7efce50c2640 1 --2- 172.21.15.5:0/2130427793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efce0102db0 0x7efce01985d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efcd000d950 tx=0x7efcd000de10 comp rx=0 tx=0).stop 2024-09-15T22:07:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.876+0000 7efcdf59e640 1 -- 172.21.15.5:0/2130427793 reap_dead start 2024-09-15T22:07:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.876+0000 7efce50c2640 1 -- 172.21.15.5:0/2130427793 shutdown_connections 2024-09-15T22:07:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.876+0000 7efce50c2640 1 -- 172.21.15.5:0/2130427793 >> 172.21.15.5:0/2130427793 conn(0x7efce00fc7d0 msgr2=0x7efce0106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.876+0000 7efce50c2640 1 -- 172.21.15.5:0/2130427793 shutdown_connections 2024-09-15T22:07:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:47.876+0000 7efce50c2640 1 -- 172.21.15.5:0/2130427793 wait complete. 2024-09-15T22:07:48.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:47 smithi099 ceph-mon[34892]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-15T22:07:48.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:47 smithi129 ceph-mon[35314]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-15T22:07:48.528 INFO:teuthology.orchestra.run.smithi005.stdout:317827579932 2024-09-15T22:07:48.529 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-15T22:07:48.832 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:48 smithi005 ceph-mon[30944]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 5 op/s 2024-09-15T22:07:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:49.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:48 smithi099 ceph-mon[34892]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 5 op/s 2024-09-15T22:07:49.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:48 smithi129 ceph-mon[35314]: pgmap v320: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 383 B/s wr, 5 op/s 2024-09-15T22:07:49.263 INFO:teuthology.orchestra.run.smithi005.stdout:146028888121 2024-09-15T22:07:49.263 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-15T22:07:49.663 INFO:teuthology.orchestra.run.smithi005.stdout:261993005093 2024-09-15T22:07:49.663 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-15T22:07:49.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.836+0000 7ff298156640 1 -- 172.21.15.5:0/3601591157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff290102ef0 msgr2=0x7ff29010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.836+0000 7ff298156640 1 --2- 172.21.15.5:0/3601591157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff290102ef0 0x7ff29010da40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff288009ca0 tx=0x7ff28802a950 comp rx=0 tx=0).stop 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.836+0000 7ff298156640 1 -- 172.21.15.5:0/3601591157 shutdown_connections 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.836+0000 7ff298156640 1 --2- 172.21.15.5:0/3601591157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff290102ef0 0x7ff29010da40 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.836+0000 7ff298156640 1 --2- 172.21.15.5:0/3601591157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290102570 0x7ff2901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.836+0000 7ff298156640 1 --2- 172.21.15.5:0/3601591157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff290100e20 0x7ff290101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.836+0000 7ff298156640 1 -- 172.21.15.5:0/3601591157 >> 172.21.15.5:0/3601591157 conn(0x7ff2900fc7d0 msgr2=0x7ff2900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.836+0000 7ff298156640 1 -- 172.21.15.5:0/3601591157 shutdown_connections 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.836+0000 7ff298156640 1 -- 172.21.15.5:0/3601591157 wait complete. 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff298156640 1 Processor -- start 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff298156640 1 -- start start 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff298156640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff290100e20 0x7ff290198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:49.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff298156640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290102570 0x7ff290198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff298156640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff290102ef0 0x7ff29019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff298156640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff290110f80 con 0x7ff290100e20 2024-09-15T22:07:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff298156640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff290110e00 con 0x7ff290102570 2024-09-15T22:07:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff298156640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff290111100 con 0x7ff290102ef0 2024-09-15T22:07:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff295ef2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff290100e20 0x7ff290198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff295ef2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff290100e20 0x7ff290198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57046/0 (socket says 172.21.15.5:57046) 2024-09-15T22:07:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff295ef2640 1 -- 172.21.15.5:0/2814342273 learned_addr learned my addr 172.21.15.5:0/2814342273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff2956f1640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290102570 0x7ff290198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff2966f3640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff290102ef0 0x7ff29019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff295ef2640 1 -- 172.21.15.5:0/2814342273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff290102ef0 msgr2=0x7ff29019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff295ef2640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff290102ef0 0x7ff29019cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff295ef2640 1 -- 172.21.15.5:0/2814342273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290102570 msgr2=0x7ff290198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.837+0000 7ff295ef2640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290102570 0x7ff290198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.838+0000 7ff295ef2640 1 -- 172.21.15.5:0/2814342273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29019d4b0 con 0x7ff290100e20 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.838+0000 7ff2956f1640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff290102570 0x7ff290198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.838+0000 7ff2966f3640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff290102ef0 0x7ff29019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.838+0000 7ff295ef2640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff290100e20 0x7ff290198500 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00d770 tx=0x7ff28c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.838+0000 7ff286ffd640 1 -- 172.21.15.5:0/2814342273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c017070 con 0x7ff290100e20 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.838+0000 7ff286ffd640 1 -- 172.21.15.5:0/2814342273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff28c005500 con 0x7ff290100e20 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.838+0000 7ff286ffd640 1 -- 172.21.15.5:0/2814342273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28c004e00 con 0x7ff290100e20 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.838+0000 7ff298156640 1 -- 172.21.15.5:0/2814342273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff29019d7a0 con 0x7ff290100e20 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.838+0000 7ff298156640 1 -- 172.21.15.5:0/2814342273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2901a5080 con 0x7ff290100e20 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.839+0000 7ff298156640 1 -- 172.21.15.5:0/2814342273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff290062980 con 0x7ff290100e20 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.840+0000 7ff286ffd640 1 -- 172.21.15.5:0/2814342273 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ff28c004fa0 con 0x7ff290100e20 2024-09-15T22:07:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.841+0000 7ff286ffd640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff27c0a32d0 0x7ff27c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:49.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.841+0000 7ff286ffd640 1 -- 172.21.15.5:0/2814342273 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7ff28c0cfd80 con 0x7ff290100e20 2024-09-15T22:07:49.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.841+0000 7ff2956f1640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff27c0a32d0 0x7ff27c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:49.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.841+0000 7ff286ffd640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865] conn(0x7ff27c0ae620 0x7ff27c0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:49.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.841+0000 7ff286ffd640 1 -- 172.21.15.5:0/2814342273 --> [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff28c001d30 con 0x7ff27c0ae620 2024-09-15T22:07:49.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.841+0000 7ff286ffd640 1 -- 172.21.15.5:0/2814342273 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7ff28c0d01d0 con 0x7ff290100e20 2024-09-15T22:07:49.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.842+0000 7ff2966f3640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865] conn(0x7ff27c0ae620 0x7ff27c0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:49.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.842+0000 7ff2956f1640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff27c0a32d0 0x7ff27c0a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff290105210 tx=0x7ff28000b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:49.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.842+0000 7ff2966f3640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865] conn(0x7ff27c0ae620 0x7ff27c0b0ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:49.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.843+0000 7ff286ffd640 1 -- 172.21.15.5:0/2814342273 <== osd.0 v2:172.21.15.5:6802/3136788865 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff28c001d30 con 0x7ff27c0ae620 2024-09-15T22:07:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.857+0000 7ff298156640 1 -- 172.21.15.5:0/2814342273 --> [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff290062e00 con 0x7ff27c0ae620 2024-09-15T22:07:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.857+0000 7ff286ffd640 1 -- 172.21.15.5:0/2814342273 <== osd.0 v2:172.21.15.5:6802/3136788865 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff290062e00 con 0x7ff27c0ae620 2024-09-15T22:07:49.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.858+0000 7ff298156640 1 -- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865] conn(0x7ff27c0ae620 msgr2=0x7ff27c0b0ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.858+0000 7ff298156640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865] conn(0x7ff27c0ae620 0x7ff27c0b0ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.858+0000 7ff298156640 1 -- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff27c0a32d0 msgr2=0x7ff27c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.858+0000 7ff298156640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff27c0a32d0 0x7ff27c0a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff290105210 tx=0x7ff28000b5f0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.858+0000 7ff298156640 1 -- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff290100e20 msgr2=0x7ff290198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.858+0000 7ff298156640 1 --2- 172.21.15.5:0/2814342273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff290100e20 0x7ff290198500 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00d770 tx=0x7ff28c00dc30 comp rx=0 tx=0).stop 2024-09-15T22:07:49.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.858+0000 7ff2966f3640 1 -- 172.21.15.5:0/2814342273 reap_dead start 2024-09-15T22:07:49.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.858+0000 7ff298156640 1 -- 172.21.15.5:0/2814342273 shutdown_connections 2024-09-15T22:07:49.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.858+0000 7ff298156640 1 -- 172.21.15.5:0/2814342273 >> 172.21.15.5:0/2814342273 conn(0x7ff2900fc7d0 msgr2=0x7ff29006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:49.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.858+0000 7ff298156640 1 -- 172.21.15.5:0/2814342273 shutdown_connections 2024-09-15T22:07:49.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.859+0000 7ff298156640 1 -- 172.21.15.5:0/2814342273 wait complete. 2024-09-15T22:07:49.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.022+0000 7f63c9a61640 1 -- 172.21.15.5:0/3163478797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c4101dd0 msgr2=0x7f63c41021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.022+0000 7f63c9a61640 1 --2- 172.21.15.5:0/3163478797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c4101dd0 0x7f63c41021b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009900 tx=0x7f63b402a410 comp rx=0 tx=0).stop 2024-09-15T22:07:49.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.023+0000 7f63c9a61640 1 -- 172.21.15.5:0/3163478797 shutdown_connections 2024-09-15T22:07:49.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.023+0000 7f63c9a61640 1 --2- 172.21.15.5:0/3163478797 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63c4106f00 0x7f63c410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.023+0000 7f63c9a61640 1 --2- 172.21.15.5:0/3163478797 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63c4102780 0x7f63c41067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.023+0000 7f63c9a61640 1 --2- 172.21.15.5:0/3163478797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c4101dd0 0x7f63c41021b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.023+0000 7f63c9a61640 1 -- 172.21.15.5:0/3163478797 >> 172.21.15.5:0/3163478797 conn(0x7f63c40fd8b0 msgr2=0x7f63c40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:49.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.023+0000 7f63c9a61640 1 -- 172.21.15.5:0/3163478797 shutdown_connections 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.023+0000 7f63c9a61640 1 -- 172.21.15.5:0/3163478797 wait complete. 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.023+0000 7f63c9a61640 1 Processor -- start 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c9a61640 1 -- start start 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c9a61640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63c4101dd0 0x7f63c4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c9a61640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63c4102780 0x7f63c4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c9a61640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c4106f00 0x7f63c419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c9a61640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63c4111ff0 con 0x7f63c4106f00 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c9a61640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f63c4111e70 con 0x7f63c4101dd0 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c9a61640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f63c4112170 con 0x7f63c4102780 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c4106f00 0x7f63c419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c4106f00 0x7f63c419ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57032/0 (socket says 172.21.15.5:57032) 2024-09-15T22:07:49.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c37fe640 1 -- 172.21.15.5:0/709230704 learned_addr learned my addr 172.21.15.5:0/709230704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c2ffd640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63c4101dd0 0x7f63c4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c27fc640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63c4102780 0x7f63c4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c37fe640 1 -- 172.21.15.5:0/709230704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63c4102780 msgr2=0x7f63c4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c37fe640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f63c4102780 0x7f63c4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c37fe640 1 -- 172.21.15.5:0/709230704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63c4101dd0 msgr2=0x7f63c4198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c37fe640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f63c4101dd0 0x7f63c4198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.024+0000 7f63c37fe640 1 -- 172.21.15.5:0/709230704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c419d4e0 con 0x7f63c4106f00 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.025+0000 7f63c37fe640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c4106f00 0x7f63c419ce00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00eaa0 tx=0x7f63bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.025+0000 7f63c8a5f640 1 -- 172.21.15.5:0/709230704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00cdb0 con 0x7f63c4106f00 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.025+0000 7f63c8a5f640 1 -- 172.21.15.5:0/709230704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63bc005020 con 0x7f63c4106f00 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.025+0000 7f63c9a61640 1 -- 172.21.15.5:0/709230704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c419d7d0 con 0x7f63c4106f00 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.025+0000 7f63c8a5f640 1 -- 172.21.15.5:0/709230704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc0116b0 con 0x7f63c4106f00 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.025+0000 7f63c9a61640 1 -- 172.21.15.5:0/709230704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c41a50b0 con 0x7f63c4106f00 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.026+0000 7f63c9a61640 1 -- 172.21.15.5:0/709230704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63c4062900 con 0x7f63c4106f00 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.027+0000 7f63c8a5f640 1 -- 172.21.15.5:0/709230704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f63bc005560 con 0x7f63c4106f00 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.028+0000 7f63c8a5f640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f63b00a32d0 0x7f63b00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.028+0000 7f63c2ffd640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f63b00a32d0 0x7f63b00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.028+0000 7f63c8a5f640 1 -- 172.21.15.5:0/709230704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f63bc0d0500 con 0x7f63c4106f00 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.028+0000 7f63c8a5f640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880] conn(0x7f63b00ae620 0x7f63b00b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.028+0000 7f63c8a5f640 1 -- 172.21.15.5:0/709230704 --> [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f63bc014ec0 con 0x7f63b00ae620 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.028+0000 7f63c8a5f640 1 -- 172.21.15.5:0/709230704 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f63bc0d08b0 con 0x7f63c4106f00 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.028+0000 7f63c2ffd640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f63b00a32d0 0x7f63b00a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f63b4000c00 tx=0x7f63b4005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.029+0000 7f63c27fc640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880] conn(0x7f63b00ae620 0x7f63b00b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.029+0000 7f63c27fc640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880] conn(0x7f63b00ae620 0x7f63b00b0ac0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.030+0000 7f63c8a5f640 1 -- 172.21.15.5:0/709230704 <== osd.1 v2:172.21.15.99:6800/2888502880 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f63bc014ec0 con 0x7f63b00ae620 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.044+0000 7f63c9a61640 1 -- 172.21.15.5:0/709230704 --> [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f63c4062d80 con 0x7f63b00ae620 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.044+0000 7f63c8a5f640 1 -- 172.21.15.5:0/709230704 <== osd.1 v2:172.21.15.99:6800/2888502880 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f63c4062d80 con 0x7f63b00ae620 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.045+0000 7f63c9a61640 1 -- 172.21.15.5:0/709230704 >> [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880] conn(0x7f63b00ae620 msgr2=0x7f63b00b0ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.045+0000 7f63c9a61640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880] conn(0x7f63b00ae620 0x7f63b00b0ac0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.045+0000 7f63c9a61640 1 -- 172.21.15.5:0/709230704 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f63b00a32d0 msgr2=0x7f63b00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.045+0000 7f63c9a61640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f63b00a32d0 0x7f63b00a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f63b4000c00 tx=0x7f63b4005b50 comp rx=0 tx=0).stop 2024-09-15T22:07:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.045+0000 7f63c9a61640 1 -- 172.21.15.5:0/709230704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c4106f00 msgr2=0x7f63c419ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.045+0000 7f63c9a61640 1 --2- 172.21.15.5:0/709230704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c4106f00 0x7f63c419ce00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00eaa0 tx=0x7f63bc00ef60 comp rx=0 tx=0).stop 2024-09-15T22:07:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.045+0000 7f63c37fe640 1 -- 172.21.15.5:0/709230704 reap_dead start 2024-09-15T22:07:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.045+0000 7f63c9a61640 1 -- 172.21.15.5:0/709230704 shutdown_connections 2024-09-15T22:07:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.045+0000 7f63c9a61640 1 -- 172.21.15.5:0/709230704 >> 172.21.15.5:0/709230704 conn(0x7f63c40fd8b0 msgr2=0x7f63c40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.045+0000 7f63c9a61640 1 -- 172.21.15.5:0/709230704 shutdown_connections 2024-09-15T22:07:49.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:49.045+0000 7f63c9a61640 1 -- 172.21.15.5:0/709230704 wait complete. 2024-09-15T22:07:50.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:50 smithi005 ceph-mon[30944]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:07:50.871 INFO:teuthology.orchestra.run.smithi005.stdout:287762808864 2024-09-15T22:07:50.871 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-15T22:07:51.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:50 smithi099 ceph-mon[34892]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:07:51.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:50 smithi129 ceph-mon[35314]: pgmap v321: 129 pgs: 1 peering, 128 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:07:51.306 INFO:teuthology.orchestra.run.smithi005.stdout:34359738452 2024-09-15T22:07:51.306 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-15T22:07:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:98784247875 2024-09-15T22:07:51.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-15T22:07:52.026 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:51 smithi005 ceph-mon[30944]: pgmap v322: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-09-15T22:07:52.026 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:51 smithi005 ceph-mon[30944]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-15T22:07:52.027 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:51 smithi005 ceph-mon[30944]: Cluster is now healthy 2024-09-15T22:07:52.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:51 smithi099 ceph-mon[34892]: pgmap v322: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-09-15T22:07:52.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:51 smithi099 ceph-mon[34892]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-15T22:07:52.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:51 smithi099 ceph-mon[34892]: Cluster is now healthy 2024-09-15T22:07:52.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:51 smithi129 ceph-mon[35314]: pgmap v322: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 72 B/s, 0 objects/s recovering 2024-09-15T22:07:52.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:51 smithi129 ceph-mon[35314]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-15T22:07:52.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:51 smithi129 ceph-mon[35314]: Cluster is now healthy 2024-09-15T22:07:52.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.491+0000 7f6529566640 1 -- 172.21.15.5:0/2005963495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652410c170 msgr2=0x7f652410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.491+0000 7f6529566640 1 --2- 172.21.15.5:0/2005963495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652410c170 0x7f652410e610 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f651c009900 tx=0x7f651c02a480 comp rx=0 tx=0).stop 2024-09-15T22:07:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.491+0000 7f6529566640 1 -- 172.21.15.5:0/2005963495 shutdown_connections 2024-09-15T22:07:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.491+0000 7f6529566640 1 --2- 172.21.15.5:0/2005963495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652410c170 0x7f652410e610 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.491+0000 7f6529566640 1 --2- 172.21.15.5:0/2005963495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6524103660 0x7f652410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.491+0000 7f6529566640 1 --2- 172.21.15.5:0/2005963495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524102d40 0x7f6524103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.491+0000 7f6529566640 1 -- 172.21.15.5:0/2005963495 >> 172.21.15.5:0/2005963495 conn(0x7f65240fc7a0 msgr2=0x7f65240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.491+0000 7f6529566640 1 -- 172.21.15.5:0/2005963495 shutdown_connections 2024-09-15T22:07:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.491+0000 7f6529566640 1 -- 172.21.15.5:0/2005963495 wait complete. 2024-09-15T22:07:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.491+0000 7f6529566640 1 Processor -- start 2024-09-15T22:07:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f6529566640 1 -- start start 2024-09-15T22:07:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f6529566640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524102d40 0x7f65241983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f6529566640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6524103660 0x7f6524198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f6529566640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652410c170 0x7f652419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f6529566640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6524111fb0 con 0x7f652410c170 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f6529566640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6524111e30 con 0x7f6524102d40 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f6529566640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6524112130 con 0x7f6524103660 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f65237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652410c170 0x7f652419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f65237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652410c170 0x7f652419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57058/0 (socket says 172.21.15.5:57058) 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f65237fe640 1 -- 172.21.15.5:0/3616650137 learned_addr learned my addr 172.21.15.5:0/3616650137 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f6522ffd640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524102d40 0x7f65241983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f65227fc640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6524103660 0x7f6524198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f65237fe640 1 -- 172.21.15.5:0/3616650137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6524103660 msgr2=0x7f6524198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f65237fe640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6524103660 0x7f6524198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f65237fe640 1 -- 172.21.15.5:0/3616650137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524102d40 msgr2=0x7f65241983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f65237fe640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6524102d40 0x7f65241983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f65237fe640 1 -- 172.21.15.5:0/3616650137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652419d380 con 0x7f652410c170 2024-09-15T22:07:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.492+0000 7f65227fc640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6524103660 0x7f6524198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:07:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.493+0000 7f65237fe640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652410c170 0x7f652419cca0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f651c009230 tx=0x7f651c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.493+0000 7f650bfff640 1 -- 172.21.15.5:0/3616650137 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c03f030 con 0x7f652410c170 2024-09-15T22:07:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.493+0000 7f650bfff640 1 -- 172.21.15.5:0/3616650137 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f651c003ce0 con 0x7f652410c170 2024-09-15T22:07:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.493+0000 7f6529566640 1 -- 172.21.15.5:0/3616650137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f652419d610 con 0x7f652410c170 2024-09-15T22:07:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.493+0000 7f650bfff640 1 -- 172.21.15.5:0/3616650137 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c035070 con 0x7f652410c170 2024-09-15T22:07:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.493+0000 7f6529566640 1 -- 172.21.15.5:0/3616650137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65241a4f50 con 0x7f652410c170 2024-09-15T22:07:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.494+0000 7f6529566640 1 -- 172.21.15.5:0/3616650137 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65240fee10 con 0x7f652410c170 2024-09-15T22:07:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.495+0000 7f650bfff640 1 -- 172.21.15.5:0/3616650137 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f651c042030 con 0x7f652410c170 2024-09-15T22:07:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.496+0000 7f650bfff640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f650c0a32d0 0x7f650c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.496+0000 7f6522ffd640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f650c0a32d0 0x7f650c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.496+0000 7f650bfff640 1 -- 172.21.15.5:0/3616650137 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f651c0f07b0 con 0x7f652410c170 2024-09-15T22:07:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.496+0000 7f650bfff640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483] conn(0x7f650c0ae620 0x7f650c0b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.496+0000 7f650bfff640 1 -- 172.21.15.5:0/3616650137 --> [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f650c0b1190 con 0x7f650c0ae620 2024-09-15T22:07:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.496+0000 7f650bfff640 1 -- 172.21.15.5:0/3616650137 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f651c0f0c00 con 0x7f652410c170 2024-09-15T22:07:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.496+0000 7f6522ffd640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f650c0a32d0 0x7f650c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6514007ec0 tx=0x7f65140075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.496+0000 7f65227fc640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483] conn(0x7f650c0ae620 0x7f650c0b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.497+0000 7f65227fc640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483] conn(0x7f650c0ae620 0x7f650c0b0ac0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.498+0000 7f650bfff640 1 -- 172.21.15.5:0/3616650137 <== osd.2 v2:172.21.15.129:6800/2902807483 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f650c0b1190 con 0x7f650c0ae620 2024-09-15T22:07:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.512+0000 7f6529566640 1 -- 172.21.15.5:0/3616650137 --> [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f652406b070 con 0x7f650c0ae620 2024-09-15T22:07:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f650bfff640 1 -- 172.21.15.5:0/3616650137 <== osd.2 v2:172.21.15.129:6800/2902807483 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f652406b070 con 0x7f650c0ae620 2024-09-15T22:07:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f6529566640 1 -- 172.21.15.5:0/3616650137 >> [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483] conn(0x7f650c0ae620 msgr2=0x7f650c0b0ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f6529566640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483] conn(0x7f650c0ae620 0x7f650c0b0ac0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f6529566640 1 -- 172.21.15.5:0/3616650137 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f650c0a32d0 msgr2=0x7f650c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f6529566640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f650c0a32d0 0x7f650c0a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6514007ec0 tx=0x7f65140075b0 comp rx=0 tx=0).stop 2024-09-15T22:07:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f6529566640 1 -- 172.21.15.5:0/3616650137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652410c170 msgr2=0x7f652419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f6529566640 1 --2- 172.21.15.5:0/3616650137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f652410c170 0x7f652419cca0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f651c009230 tx=0x7f651c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:07:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f65237fe640 1 -- 172.21.15.5:0/3616650137 reap_dead start 2024-09-15T22:07:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f6529566640 1 -- 172.21.15.5:0/3616650137 shutdown_connections 2024-09-15T22:07:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f6529566640 1 -- 172.21.15.5:0/3616650137 >> 172.21.15.5:0/3616650137 conn(0x7f65240fc7a0 msgr2=0x7f65240fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f6529566640 1 -- 172.21.15.5:0/3616650137 shutdown_connections 2024-09-15T22:07:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:52.513+0000 7f6529566640 1 -- 172.21.15.5:0/3616650137 wait complete. 2024-09-15T22:07:53.217 INFO:teuthology.orchestra.run.smithi005.stdout:55834574927 2024-09-15T22:07:53.218 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-15T22:07:53.709 INFO:teuthology.orchestra.run.smithi005.stdout:73014444107 2024-09-15T22:07:53.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-15T22:07:54.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:53 smithi005 ceph-mon[30944]: pgmap v323: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 71 B/s, 0 objects/s recovering 2024-09-15T22:07:54.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:53.999+0000 7fb48f5ab640 1 -- 172.21.15.5:0/3149666891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb488106f50 msgr2=0x7fb48810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:54.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:53.999+0000 7fb48f5ab640 1 --2- 172.21.15.5:0/3149666891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb488106f50 0x7fb48810eaf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb480009930 tx=0x7fb48002a420 comp rx=0 tx=0).stop 2024-09-15T22:07:54.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:53.999+0000 7fb48f5ab640 1 -- 172.21.15.5:0/3149666891 shutdown_connections 2024-09-15T22:07:54.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:53.999+0000 7fb48f5ab640 1 --2- 172.21.15.5:0/3149666891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb488106f50 0x7fb48810eaf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:54.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:53.999+0000 7fb48f5ab640 1 --2- 172.21.15.5:0/3149666891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb488100ca0 0x7fb488106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:54.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:53.999+0000 7fb48f5ab640 1 --2- 172.21.15.5:0/3149666891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb4881002f0 0x7fb4881006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:53.999+0000 7fb48f5ab640 1 -- 172.21.15.5:0/3149666891 >> 172.21.15.5:0/3149666891 conn(0x7fb488077210 msgr2=0x7fb4880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:53.999+0000 7fb48f5ab640 1 -- 172.21.15.5:0/3149666891 shutdown_connections 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:53.999+0000 7fb48f5ab640 1 -- 172.21.15.5:0/3149666891 wait complete. 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.000+0000 7fb48f5ab640 1 Processor -- start 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.000+0000 7fb48f5ab640 1 -- start start 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.000+0000 7fb48f5ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4881002f0 0x7fb48819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.000+0000 7fb48f5ab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb488100ca0 0x7fb48819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.000+0000 7fb48f5ab640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb488106f50 0x7fb48819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.000+0000 7fb48f5ab640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb488112070 con 0x7fb4881002f0 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.000+0000 7fb48f5ab640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb488111ef0 con 0x7fb488100ca0 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.000+0000 7fb48f5ab640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb4881121f0 con 0x7fb488106f50 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.000+0000 7fb48d347640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4881002f0 0x7fb48819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:54.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.000+0000 7fb48d347640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4881002f0 0x7fb48819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50482/0 (socket says 172.21.15.5:50482) 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.000+0000 7fb48d347640 1 -- 172.21.15.5:0/1389430012 learned_addr learned my addr 172.21.15.5:0/1389430012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb48db48640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb488106f50 0x7fb48819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb48cb46640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb488100ca0 0x7fb48819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb48d347640 1 -- 172.21.15.5:0/1389430012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb488106f50 msgr2=0x7fb48819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb48d347640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb488106f50 0x7fb48819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb48d347640 1 -- 172.21.15.5:0/1389430012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb488100ca0 msgr2=0x7fb48819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb48d347640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb488100ca0 0x7fb48819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb48d347640 1 -- 172.21.15.5:0/1389430012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48819f6f0 con 0x7fb4881002f0 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb48d347640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4881002f0 0x7fb48819a740 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb48400ea80 tx=0x7fb48400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb47e7fc640 1 -- 172.21.15.5:0/1389430012 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48400ce40 con 0x7fb4881002f0 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb48f5ab640 1 -- 172.21.15.5:0/1389430012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48819f9e0 con 0x7fb4881002f0 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb48f5ab640 1 -- 172.21.15.5:0/1389430012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4881a72c0 con 0x7fb4881002f0 2024-09-15T22:07:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb47e7fc640 1 -- 172.21.15.5:0/1389430012 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb484005020 con 0x7fb4881002f0 2024-09-15T22:07:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.001+0000 7fb47e7fc640 1 -- 172.21.15.5:0/1389430012 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484011760 con 0x7fb4881002f0 2024-09-15T22:07:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.002+0000 7fb48f5ab640 1 -- 172.21.15.5:0/1389430012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb46c001030 con 0x7fb4881002f0 2024-09-15T22:07:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.003+0000 7fb47e7fc640 1 -- 172.21.15.5:0/1389430012 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fb484004a40 con 0x7fb4881002f0 2024-09-15T22:07:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.004+0000 7fb47e7fc640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fb4740a3350 0x7fb4740a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.004+0000 7fb48cb46640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fb4740a3350 0x7fb4740a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.004+0000 7fb47e7fc640 1 -- 172.21.15.5:0/1389430012 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fb4840d0580 con 0x7fb4881002f0 2024-09-15T22:07:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.004+0000 7fb47e7fc640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883] conn(0x7fb4740ae6a0 0x7fb4740b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.004+0000 7fb47e7fc640 1 -- 172.21.15.5:0/1389430012 --> [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb484026ea0 con 0x7fb4740ae6a0 2024-09-15T22:07:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.004+0000 7fb47e7fc640 1 -- 172.21.15.5:0/1389430012 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fb4840d09d0 con 0x7fb4881002f0 2024-09-15T22:07:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.005+0000 7fb48cb46640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fb4740a3350 0x7fb4740a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb488106200 tx=0x7fb478008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.005+0000 7fb48db48640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883] conn(0x7fb4740ae6a0 0x7fb4740b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.005+0000 7fb48db48640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883] conn(0x7fb4740ae6a0 0x7fb4740b0b40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.006+0000 7fb47e7fc640 1 -- 172.21.15.5:0/1389430012 <== osd.7 v2:172.21.15.99:6816/521420883 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb484026ea0 con 0x7fb4740ae6a0 2024-09-15T22:07:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.022+0000 7fb48f5ab640 1 -- 172.21.15.5:0/1389430012 --> [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb46c002dd0 con 0x7fb4740ae6a0 2024-09-15T22:07:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.022+0000 7fb47e7fc640 1 -- 172.21.15.5:0/1389430012 <== osd.7 v2:172.21.15.99:6816/521420883 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb46c002dd0 con 0x7fb4740ae6a0 2024-09-15T22:07:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.022+0000 7fb48f5ab640 1 -- 172.21.15.5:0/1389430012 >> [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883] conn(0x7fb4740ae6a0 msgr2=0x7fb4740b0b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.022+0000 7fb48f5ab640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883] conn(0x7fb4740ae6a0 0x7fb4740b0b40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.023+0000 7fb48f5ab640 1 -- 172.21.15.5:0/1389430012 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fb4740a3350 msgr2=0x7fb4740a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.023+0000 7fb48f5ab640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fb4740a3350 0x7fb4740a5810 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb488106200 tx=0x7fb478008040 comp rx=0 tx=0).stop 2024-09-15T22:07:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.023+0000 7fb48f5ab640 1 -- 172.21.15.5:0/1389430012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4881002f0 msgr2=0x7fb48819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.023+0000 7fb48f5ab640 1 --2- 172.21.15.5:0/1389430012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4881002f0 0x7fb48819a740 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb48400ea80 tx=0x7fb48400ef40 comp rx=0 tx=0).stop 2024-09-15T22:07:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.023+0000 7fb48db48640 1 -- 172.21.15.5:0/1389430012 reap_dead start 2024-09-15T22:07:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.023+0000 7fb48f5ab640 1 -- 172.21.15.5:0/1389430012 shutdown_connections 2024-09-15T22:07:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.023+0000 7fb48f5ab640 1 -- 172.21.15.5:0/1389430012 >> 172.21.15.5:0/1389430012 conn(0x7fb488077210 msgr2=0x7fb488105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.023+0000 7fb48f5ab640 1 -- 172.21.15.5:0/1389430012 shutdown_connections 2024-09-15T22:07:54.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:54.023+0000 7fb48f5ab640 1 -- 172.21.15.5:0/1389430012 wait complete. 2024-09-15T22:07:54.046 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:54.052 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:54.176 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:53 smithi129 ceph-mon[35314]: pgmap v323: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 71 B/s, 0 objects/s recovering 2024-09-15T22:07:54.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:53 smithi099 ceph-mon[34892]: pgmap v323: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4 op/s; 71 B/s, 0 objects/s recovering 2024-09-15T22:07:54.730 INFO:teuthology.orchestra.run.smithi005.stdout:197568495665 2024-09-15T22:07:54.730 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-15T22:07:54.985 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:54.985 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:54.985 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:54.987 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:54.990 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:54.992 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:07:55.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:55 smithi005 ceph-mon[30944]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 58 B/s, 0 objects/s recovering 2024-09-15T22:07:56.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.005+0000 7f8fd6271640 1 -- 172.21.15.5:0/705241519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd0102ef0 msgr2=0x7f8fd010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.005+0000 7f8fd6271640 1 --2- 172.21.15.5:0/705241519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd0102ef0 0x7f8fd010da40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009ca0 tx=0x7f8fc802a950 comp rx=0 tx=0).stop 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.006+0000 7f8fd6271640 1 -- 172.21.15.5:0/705241519 shutdown_connections 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.006+0000 7f8fd6271640 1 --2- 172.21.15.5:0/705241519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd0102ef0 0x7f8fd010da40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.006+0000 7f8fd6271640 1 --2- 172.21.15.5:0/705241519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0102570 0x7f8fd01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.006+0000 7f8fd6271640 1 --2- 172.21.15.5:0/705241519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 0x7f8fd0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.006+0000 7f8fd6271640 1 -- 172.21.15.5:0/705241519 >> 172.21.15.5:0/705241519 conn(0x7f8fd00fc7d0 msgr2=0x7f8fd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.006+0000 7f8fd6271640 1 -- 172.21.15.5:0/705241519 shutdown_connections 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.006+0000 7f8fd6271640 1 -- 172.21.15.5:0/705241519 wait complete. 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.006+0000 7f8fd6271640 1 Processor -- start 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.006+0000 7f8fd6271640 1 -- start start 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fd6271640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 0x7f8fd0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fd6271640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd0102570 0x7f8fd0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fd6271640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fd6271640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0110f80 con 0x7f8fd0102570 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fd6271640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0110e00 con 0x7f8fd0100e20 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fd6271640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd0111100 con 0x7f8fd0102ef0 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd0102570 0x7f8fd0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fcf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd0102570 0x7f8fd0198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50532/0 (socket says 172.21.15.5:50532) 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fcf7fe640 1 -- 172.21.15.5:0/786924903 learned_addr learned my addr 172.21.15.5:0/786924903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fcffff640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 0x7f8fd0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fd480e640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fcf7fe640 1 -- 172.21.15.5:0/786924903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0102ef0 msgr2=0x7f8fd019ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fcf7fe640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fcf7fe640 1 -- 172.21.15.5:0/786924903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 msgr2=0x7f8fd0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fcf7fe640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 0x7f8fd0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fcf7fe640 1 -- 172.21.15.5:0/786924903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd019d3b0 con 0x7f8fd0102570 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.007+0000 7f8fcf7fe640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd0102570 0x7f8fd0198940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400c890 tx=0x7f8fc400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.008+0000 7f8fcd7fa640 1 -- 172.21.15.5:0/786924903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4004220 con 0x7f8fd0102570 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.008+0000 7f8fd6271640 1 -- 172.21.15.5:0/786924903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd019d6a0 con 0x7f8fd0102570 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.008+0000 7f8fcd7fa640 1 -- 172.21.15.5:0/786924903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc40043c0 con 0x7f8fd0102570 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.008+0000 7f8fcd7fa640 1 -- 172.21.15.5:0/786924903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4004a00 con 0x7f8fd0102570 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.008+0000 7f8fd6271640 1 -- 172.21.15.5:0/786924903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd01a4f80 con 0x7f8fd0102570 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.009+0000 7f8fd6271640 1 -- 172.21.15.5:0/786924903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fd0101200 con 0x7f8fd0102570 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.010+0000 7f8fcd7fa640 1 -- 172.21.15.5:0/786924903 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f8fc40079b0 con 0x7f8fd0102570 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.011+0000 7f8fcd7fa640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fb80a3350 0x7f8fb80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.011+0000 7f8fcffff640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fb80a3350 0x7f8fb80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.011+0000 7f8fcd7fa640 1 -- 172.21.15.5:0/786924903 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f8fc40cfe40 con 0x7f8fd0102570 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.011+0000 7f8fcffff640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fb80a3350 0x7f8fb80a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000abe0 tx=0x7f8fc000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:56.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.018+0000 7f8fcd7fa640 1 -- 172.21.15.5:0/786924903 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8fc4014030 con 0x7f8fd0102570 2024-09-15T22:07:56.142 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:55 smithi099 ceph-mon[34892]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 58 B/s, 0 objects/s recovering 2024-09-15T22:07:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.199+0000 7f8fd6271640 1 -- 172.21.15.5:0/786924903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8fd010a0d0 con 0x7f8fd0102570 2024-09-15T22:07:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.199+0000 7f8fcd7fa640 1 -- 172.21.15.5:0/786924903 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8fc409c6c0 con 0x7f8fd0102570 2024-09-15T22:07:56.202 INFO:teuthology.orchestra.run.smithi005.stdout:34359738453 2024-09-15T22:07:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.201+0000 7f8fd6271640 1 -- 172.21.15.5:0/786924903 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fb80a3350 msgr2=0x7f8fb80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.201+0000 7f8fd6271640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fb80a3350 0x7f8fb80a5810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000abe0 tx=0x7f8fc000a2d0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.201+0000 7f8fd6271640 1 -- 172.21.15.5:0/786924903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd0102570 msgr2=0x7f8fd0198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.201+0000 7f8fd6271640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd0102570 0x7f8fd0198940 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400c890 tx=0x7f8fc400cc90 comp rx=0 tx=0).stop 2024-09-15T22:07:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.202+0000 7f8fd6271640 1 -- 172.21.15.5:0/786924903 shutdown_connections 2024-09-15T22:07:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.202+0000 7f8fd6271640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8fd0102ef0 0x7f8fd019ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.202+0000 7f8fd6271640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8fb80a3350 0x7f8fb80a5810 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.202+0000 7f8fd6271640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8fd0102570 0x7f8fd0198940 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.202+0000 7f8fd6271640 1 --2- 172.21.15.5:0/786924903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8fd0100e20 0x7f8fd0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.202+0000 7f8fd6271640 1 -- 172.21.15.5:0/786924903 >> 172.21.15.5:0/786924903 conn(0x7f8fd00fc7d0 msgr2=0x7f8fd006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.202+0000 7f8fd6271640 1 -- 172.21.15.5:0/786924903 shutdown_connections 2024-09-15T22:07:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.202+0000 7f8fd6271640 1 -- 172.21.15.5:0/786924903 wait complete. 2024-09-15T22:07:56.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:55 smithi129 ceph-mon[35314]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 4 op/s; 58 B/s, 0 objects/s recovering 2024-09-15T22:07:56.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.468+0000 7f6edcd1f640 1 -- 172.21.15.5:0/2724528777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed810c170 msgr2=0x7f6ed810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.468+0000 7f6edcd1f640 1 --2- 172.21.15.5:0/2724528777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed810c170 0x7f6ed810e610 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0009ca0 tx=0x7f6ed002a950 comp rx=0 tx=0).stop 2024-09-15T22:07:56.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.468+0000 7f6edcd1f640 1 -- 172.21.15.5:0/2724528777 shutdown_connections 2024-09-15T22:07:56.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.468+0000 7f6edcd1f640 1 --2- 172.21.15.5:0/2724528777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed810c170 0x7f6ed810e610 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.468+0000 7f6edcd1f640 1 --2- 172.21.15.5:0/2724528777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed8103660 0x7f6ed810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.468+0000 7f6edcd1f640 1 --2- 172.21.15.5:0/2724528777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed8102d40 0x7f6ed8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.468+0000 7f6edcd1f640 1 -- 172.21.15.5:0/2724528777 >> 172.21.15.5:0/2724528777 conn(0x7f6ed80fc7a0 msgr2=0x7f6ed80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:56.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.468+0000 7f6edcd1f640 1 -- 172.21.15.5:0/2724528777 shutdown_connections 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.468+0000 7f6edcd1f640 1 -- 172.21.15.5:0/2724528777 wait complete. 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6edcd1f640 1 Processor -- start 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6edcd1f640 1 -- start start 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6edcd1f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed8102d40 0x7f6ed81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6edcd1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed8103660 0x7f6ed8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6edcd1f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed810c170 0x7f6ed8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6edcd1f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8111fb0 con 0x7f6ed8103660 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6edcd1f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8111e30 con 0x7f6ed8102d40 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6edcd1f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8112130 con 0x7f6ed810c170 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6ed5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed8103660 0x7f6ed8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6ed5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed8103660 0x7f6ed8100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50550/0 (socket says 172.21.15.5:50550) 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6ed5d9b640 1 -- 172.21.15.5:0/3953716913 learned_addr learned my addr 172.21.15.5:0/3953716913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:56.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.469+0000 7f6ed659c640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed8102d40 0x7f6ed81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6ed6d9d640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed810c170 0x7f6ed8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6ed5d9b640 1 -- 172.21.15.5:0/3953716913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed810c170 msgr2=0x7f6ed8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6ed5d9b640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed810c170 0x7f6ed8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6ed5d9b640 1 -- 172.21.15.5:0/3953716913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed8102d40 msgr2=0x7f6ed81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6ed5d9b640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed8102d40 0x7f6ed81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6ed5d9b640 1 -- 172.21.15.5:0/3953716913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed81a6930 con 0x7f6ed8103660 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6ed6d9d640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed810c170 0x7f6ed8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6ed5d9b640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed8103660 0x7f6ed8100ee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00b3f0 tx=0x7f6ecc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6ec77fe640 1 -- 172.21.15.5:0/3953716913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc015020 con 0x7f6ed8103660 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6ec77fe640 1 -- 172.21.15.5:0/3953716913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ecc011d10 con 0x7f6ed8103660 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6ec77fe640 1 -- 172.21.15.5:0/3953716913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc00f2e0 con 0x7f6ed8103660 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6edcd1f640 1 -- 172.21.15.5:0/3953716913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed81a6c20 con 0x7f6ed8103660 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.470+0000 7f6edcd1f640 1 -- 172.21.15.5:0/3953716913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed81a7160 con 0x7f6ed8103660 2024-09-15T22:07:56.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.472+0000 7f6edcd1f640 1 -- 172.21.15.5:0/3953716913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed8103120 con 0x7f6ed8103660 2024-09-15T22:07:56.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.473+0000 7f6ec77fe640 1 -- 172.21.15.5:0/3953716913 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f6ecc018020 con 0x7f6ed8103660 2024-09-15T22:07:56.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.473+0000 7f6ec77fe640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ec00a32d0 0x7f6ec00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.473+0000 7f6ed659c640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ec00a32d0 0x7f6ec00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.473+0000 7f6ec77fe640 1 -- 172.21.15.5:0/3953716913 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f6ecc0cf590 con 0x7f6ed8103660 2024-09-15T22:07:56.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.474+0000 7f6ed659c640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ec00a32d0 0x7f6ec00a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800bbe0 tx=0x7f6ec800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:56.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.477+0000 7f6ec77fe640 1 -- 172.21.15.5:0/3953716913 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ecc01f020 con 0x7f6ed8103660 2024-09-15T22:07:56.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.661+0000 7f6edcd1f640 1 -- 172.21.15.5:0/3953716913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6ed8107210 con 0x7f6ed8103660 2024-09-15T22:07:56.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.661+0000 7f6ec77fe640 1 -- 172.21.15.5:0/3953716913 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6ecc09bea0 con 0x7f6ed8103660 2024-09-15T22:07:56.664 INFO:teuthology.orchestra.run.smithi005.stdout:98784247877 2024-09-15T22:07:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.663+0000 7f6edcd1f640 1 -- 172.21.15.5:0/3953716913 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ec00a32d0 msgr2=0x7f6ec00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.663+0000 7f6edcd1f640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ec00a32d0 0x7f6ec00a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800bbe0 tx=0x7f6ec800b390 comp rx=0 tx=0).stop 2024-09-15T22:07:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.663+0000 7f6edcd1f640 1 -- 172.21.15.5:0/3953716913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed8103660 msgr2=0x7f6ed8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.663+0000 7f6edcd1f640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed8103660 0x7f6ed8100ee0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00b3f0 tx=0x7f6ecc00b8b0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.663+0000 7f6edcd1f640 1 -- 172.21.15.5:0/3953716913 shutdown_connections 2024-09-15T22:07:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.663+0000 7f6edcd1f640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6ed810c170 0x7f6ed8101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.663+0000 7f6edcd1f640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ec00a32d0 0x7f6ec00a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.663+0000 7f6edcd1f640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ed8103660 0x7f6ed8100ee0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.663+0000 7f6edcd1f640 1 --2- 172.21.15.5:0/3953716913 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6ed8102d40 0x7f6ed81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.663+0000 7f6edcd1f640 1 -- 172.21.15.5:0/3953716913 >> 172.21.15.5:0/3953716913 conn(0x7f6ed80fc7a0 msgr2=0x7f6ed80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.664+0000 7f6edcd1f640 1 -- 172.21.15.5:0/3953716913 shutdown_connections 2024-09-15T22:07:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.664+0000 7f6edcd1f640 1 -- 172.21.15.5:0/3953716913 wait complete. 2024-09-15T22:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.700+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/3755298493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc08100e20 msgr2=0x7fcc08101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.700+0000 7fcc0d1c0640 1 --2- 172.21.15.5:0/3755298493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc08100e20 0x7fcc08101200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009900 tx=0x7fcbf802a480 comp rx=0 tx=0).stop 2024-09-15T22:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.700+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/3755298493 shutdown_connections 2024-09-15T22:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.700+0000 7fcc0d1c0640 1 --2- 172.21.15.5:0/3755298493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc08102ef0 0x7fcc0810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.700+0000 7fcc0d1c0640 1 --2- 172.21.15.5:0/3755298493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc08102570 0x7fcc081029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.700+0000 7fcc0d1c0640 1 --2- 172.21.15.5:0/3755298493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc08100e20 0x7fcc08101200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.700+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/3755298493 >> 172.21.15.5:0/3755298493 conn(0x7fcc080fc7d0 msgr2=0x7fcc080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.700+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/3755298493 shutdown_connections 2024-09-15T22:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.700+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/3755298493 wait complete. 2024-09-15T22:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.700+0000 7fcc0d1c0640 1 Processor -- start 2024-09-15T22:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc0d1c0640 1 -- start start 2024-09-15T22:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc0d1c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc08100e20 0x7fcc080fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc0d1c0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc08102570 0x7fcc080fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc0d1c0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc08102ef0 0x7fcc080fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc0d1c0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc08110f50 con 0x7fcc08100e20 2024-09-15T22:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc0d1c0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcc08110dd0 con 0x7fcc08102570 2024-09-15T22:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc0d1c0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcc081110d0 con 0x7fcc08102ef0 2024-09-15T22:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc06d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc08100e20 0x7fcc080fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc06d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc08100e20 0x7fcc080fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50574/0 (socket says 172.21.15.5:50574) 2024-09-15T22:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc06d9d640 1 -- 172.21.15.5:0/1448528974 learned_addr learned my addr 172.21.15.5:0/1448528974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc0659c640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc08102570 0x7fcc080fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc0759e640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc08102ef0 0x7fcc080fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc06d9d640 1 -- 172.21.15.5:0/1448528974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc08102ef0 msgr2=0x7fcc080fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc06d9d640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc08102ef0 0x7fcc080fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc06d9d640 1 -- 172.21.15.5:0/1448528974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc08102570 msgr2=0x7fcc080fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc06d9d640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc08102570 0x7fcc080fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc06d9d640 1 -- 172.21.15.5:0/1448528974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc081a6a20 con 0x7fcc08100e20 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.701+0000 7fcc0659c640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc08102570 0x7fcc080fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.702+0000 7fcc06d9d640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc08100e20 0x7fcc080fcc80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8006010 tx=0x7fcbf8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.702+0000 7fcbfffff640 1 -- 172.21.15.5:0/1448528974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf803f070 con 0x7fcc08100e20 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.702+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/1448528974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc081a6cb0 con 0x7fcc08100e20 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.702+0000 7fcbfffff640 1 -- 172.21.15.5:0/1448528974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbf8031c30 con 0x7fcc08100e20 2024-09-15T22:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.702+0000 7fcbfffff640 1 -- 172.21.15.5:0/1448528974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8035030 con 0x7fcc08100e20 2024-09-15T22:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.702+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/1448528974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc081a7190 con 0x7fcc08100e20 2024-09-15T22:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.703+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/1448528974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc08101200 con 0x7fcc08100e20 2024-09-15T22:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.704+0000 7fcbfffff640 1 -- 172.21.15.5:0/1448528974 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fcbf8042070 con 0x7fcc08100e20 2024-09-15T22:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.705+0000 7fcbfffff640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcbf40a32d0 0x7fcbf40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.705+0000 7fcbfffff640 1 -- 172.21.15.5:0/1448528974 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fcbf80f0b70 con 0x7fcc08100e20 2024-09-15T22:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.705+0000 7fcc0659c640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcbf40a32d0 0x7fcbf40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.706+0000 7fcc0659c640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcbf40a32d0 0x7fcbf40a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc080fe710 tx=0x7fcbf000b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.709+0000 7fcbfffff640 1 -- 172.21.15.5:0/1448528974 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcbf8031de0 con 0x7fcc08100e20 2024-09-15T22:07:56.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.890+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/1448528974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fcc080fe1b0 con 0x7fcc08100e20 2024-09-15T22:07:56.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.890+0000 7fcbfffff640 1 -- 172.21.15.5:0/1448528974 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcc080fe1b0 con 0x7fcc08100e20 2024-09-15T22:07:56.892 INFO:teuthology.orchestra.run.smithi005.stdout:317827579934 2024-09-15T22:07:56.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/1448528974 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcbf40a32d0 msgr2=0x7fcbf40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcbf40a32d0 0x7fcbf40a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcc080fe710 tx=0x7fcbf000b5f0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/1448528974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc08100e20 msgr2=0x7fcc080fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc08100e20 0x7fcc080fcc80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8006010 tx=0x7fcbf8003c10 comp rx=0 tx=0).stop 2024-09-15T22:07:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/1448528974 shutdown_connections 2024-09-15T22:07:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcc08102ef0 0x7fcc080fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcc08102570 0x7fcc080fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcbf40a32d0 0x7fcbf40a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 --2- 172.21.15.5:0/1448528974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc08100e20 0x7fcc080fcc80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/1448528974 >> 172.21.15.5:0/1448528974 conn(0x7fcc080fc7d0 msgr2=0x7fcc0806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/1448528974 shutdown_connections 2024-09-15T22:07:56.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.892+0000 7fcc0d1c0640 1 -- 172.21.15.5:0/1448528974 wait complete. 2024-09-15T22:07:57.269 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1201797031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:07:57.269 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/786924903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:07:57.269 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3953716913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:07:57.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1201797031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:07:57.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/786924903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:07:57.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3953716913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:07:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1201797031' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-15T22:07:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/786924903' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-15T22:07:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3953716913' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-15T22:07:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.351+0000 7f3f00033640 1 -- 172.21.15.5:0/1922686258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef8100b30 msgr2=0x7f3ef8100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.351+0000 7f3f00033640 1 --2- 172.21.15.5:0/1922686258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef8100b30 0x7f3ef8100f10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009900 tx=0x7f3ef402a470 comp rx=0 tx=0).stop 2024-09-15T22:07:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.351+0000 7f3f00033640 1 -- 172.21.15.5:0/1922686258 shutdown_connections 2024-09-15T22:07:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.351+0000 7f3f00033640 1 --2- 172.21.15.5:0/1922686258 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ef8105f40 0x7f3ef81063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.351+0000 7f3f00033640 1 --2- 172.21.15.5:0/1922686258 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef8101450 0x7f3ef8105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.351+0000 7f3f00033640 1 --2- 172.21.15.5:0/1922686258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef8100b30 0x7f3ef8100f10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.351+0000 7f3f00033640 1 -- 172.21.15.5:0/1922686258 >> 172.21.15.5:0/1922686258 conn(0x7f3ef80fc7a0 msgr2=0x7f3ef80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.351+0000 7f3f00033640 1 -- 172.21.15.5:0/1922686258 shutdown_connections 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.351+0000 7f3f00033640 1 -- 172.21.15.5:0/1922686258 wait complete. 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3f00033640 1 Processor -- start 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3f00033640 1 -- start start 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3f00033640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef8100b30 0x7f3ef81a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3f00033640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ef8101450 0x7f3ef81a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3f00033640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef8105f40 0x7f3ef819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3f00033640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef8118570 con 0x7f3ef8100b30 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3f00033640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81183f0 con 0x7f3ef8105f40 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3f00033640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef81186f0 con 0x7f3ef8101450 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3efddcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef8100b30 0x7f3ef81a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3efddcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef8100b30 0x7f3ef81a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50592/0 (socket says 172.21.15.5:50592) 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3efddcf640 1 -- 172.21.15.5:0/358233628 learned_addr learned my addr 172.21.15.5:0/358233628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3efe5d0640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef8105f40 0x7f3ef819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.352+0000 7f3efd5ce640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ef8101450 0x7f3ef81a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3efddcf640 1 -- 172.21.15.5:0/358233628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ef8101450 msgr2=0x7f3ef81a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3efddcf640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ef8101450 0x7f3ef81a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3efddcf640 1 -- 172.21.15.5:0/358233628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef8105f40 msgr2=0x7f3ef819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3efddcf640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef8105f40 0x7f3ef819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3efddcf640 1 -- 172.21.15.5:0/358233628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef819f000 con 0x7f3ef8100b30 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3efd5ce640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ef8101450 0x7f3ef81a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3efddcf640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef8100b30 0x7f3ef81a4710 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4006010 tx=0x7f3ef40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3eeeffd640 1 -- 172.21.15.5:0/358233628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef403f030 con 0x7f3ef8100b30 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3eeeffd640 1 -- 172.21.15.5:0/358233628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ef4003ce0 con 0x7f3ef8100b30 2024-09-15T22:07:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3eeeffd640 1 -- 172.21.15.5:0/358233628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4035070 con 0x7f3ef8100b30 2024-09-15T22:07:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3f00033640 1 -- 172.21.15.5:0/358233628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef819f290 con 0x7f3ef8100b30 2024-09-15T22:07:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.353+0000 7f3f00033640 1 -- 172.21.15.5:0/358233628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef81ab5e0 con 0x7f3ef8100b30 2024-09-15T22:07:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.354+0000 7f3f00033640 1 -- 172.21.15.5:0/358233628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ef8105920 con 0x7f3ef8100b30 2024-09-15T22:07:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.355+0000 7f3eeeffd640 1 -- 172.21.15.5:0/358233628 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f3ef4042030 con 0x7f3ef8100b30 2024-09-15T22:07:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.356+0000 7f3eeeffd640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ee40a32d0 0x7f3ee40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.356+0000 7f3efd5ce640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ee40a32d0 0x7f3ee40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.356+0000 7f3eeeffd640 1 -- 172.21.15.5:0/358233628 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f3ef40f08c0 con 0x7f3ef8100b30 2024-09-15T22:07:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.357+0000 7f3efd5ce640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ee40a32d0 0x7f3ee40a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80fec40 tx=0x7f3ee8005dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.360+0000 7f3eeeffd640 1 -- 172.21.15.5:0/358233628 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ef40bd180 con 0x7f3ef8100b30 2024-09-15T22:07:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.541+0000 7f3f00033640 1 -- 172.21.15.5:0/358233628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f3ef81ab810 con 0x7f3ef8100b30 2024-09-15T22:07:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.541+0000 7f3eeeffd640 1 -- 172.21.15.5:0/358233628 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3ef40bca80 con 0x7f3ef8100b30 2024-09-15T22:07:57.544 INFO:teuthology.orchestra.run.smithi005.stdout:287762808867 2024-09-15T22:07:57.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.543+0000 7f3f00033640 1 -- 172.21.15.5:0/358233628 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ee40a32d0 msgr2=0x7f3ee40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:57.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.544+0000 7f3f00033640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ee40a32d0 0x7f3ee40a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3ef80fec40 tx=0x7f3ee8005dd0 comp rx=0 tx=0).stop 2024-09-15T22:07:57.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.544+0000 7f3f00033640 1 -- 172.21.15.5:0/358233628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef8100b30 msgr2=0x7f3ef81a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:57.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.544+0000 7f3f00033640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef8100b30 0x7f3ef81a4710 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4006010 tx=0x7f3ef40042f0 comp rx=0 tx=0).stop 2024-09-15T22:07:57.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.544+0000 7f3f00033640 1 -- 172.21.15.5:0/358233628 shutdown_connections 2024-09-15T22:07:57.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.544+0000 7f3f00033640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ef8105f40 0x7f3ef819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:57.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.544+0000 7f3f00033640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ef8101450 0x7f3ef81a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:57.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.544+0000 7f3f00033640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ee40a32d0 0x7f3ee40a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.544+0000 7f3f00033640 1 --2- 172.21.15.5:0/358233628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ef8100b30 0x7f3ef81a4710 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.544+0000 7f3f00033640 1 -- 172.21.15.5:0/358233628 >> 172.21.15.5:0/358233628 conn(0x7f3ef80fc7a0 msgr2=0x7f3ef810f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.544+0000 7f3f00033640 1 -- 172.21.15.5:0/358233628 shutdown_connections 2024-09-15T22:07:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:57.544+0000 7f3f00033640 1 -- 172.21.15.5:0/358233628 wait complete. 2024-09-15T22:07:57.909 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579932 got 317827579934 for osd.11 2024-09-15T22:07:57.910 DEBUG:teuthology.parallel:result is None 2024-09-15T22:07:58.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:57 smithi005 ceph-mon[30944]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 52 B/s, 0 objects/s recovering 2024-09-15T22:07:58.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:57 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1448528974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:07:58.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:57 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/358233628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:07:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:57 smithi099 ceph-mon[34892]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 52 B/s, 0 objects/s recovering 2024-09-15T22:07:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:57 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1448528974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:07:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:57 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/358233628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:07:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:57 smithi129 ceph-mon[35314]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 52 B/s, 0 objects/s recovering 2024-09-15T22:07:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:57 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1448528974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-15T22:07:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:57 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/358233628' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-15T22:07:58.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.737+0000 7f7006c50640 1 -- 172.21.15.5:0/248831719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7000106f10 msgr2=0x7f700010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:58.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.737+0000 7f7006c50640 1 --2- 172.21.15.5:0/248831719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7000106f10 0x7f700010ea90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009ca0 tx=0x7f6ff802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.737+0000 7f7006c50640 1 -- 172.21.15.5:0/248831719 shutdown_connections 2024-09-15T22:07:58.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.737+0000 7f7006c50640 1 --2- 172.21.15.5:0/248831719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7000106f10 0x7f700010ea90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.737+0000 7f7006c50640 1 --2- 172.21.15.5:0/248831719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7000102790 0x7f70001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.737+0000 7f7006c50640 1 --2- 172.21.15.5:0/248831719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7000101de0 0x7f70001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.737+0000 7f7006c50640 1 -- 172.21.15.5:0/248831719 >> 172.21.15.5:0/248831719 conn(0x7f70000fd8c0 msgr2=0x7f70000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.737+0000 7f7006c50640 1 -- 172.21.15.5:0/248831719 shutdown_connections 2024-09-15T22:07:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.737+0000 7f7006c50640 1 -- 172.21.15.5:0/248831719 wait complete. 2024-09-15T22:07:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f7006c50640 1 Processor -- start 2024-09-15T22:07:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f7006c50640 1 -- start start 2024-09-15T22:07:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f7006c50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7000101de0 0x7f7000198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f7006c50640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7000102790 0x7f7000198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f7006c50640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7000106f10 0x7f700019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f7006c50640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7000111f90 con 0x7f7000101de0 2024-09-15T22:07:58.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f7006c50640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7000111e10 con 0x7f7000106f10 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f7006c50640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7000112110 con 0x7f7000102790 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f70049ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7000101de0 0x7f7000198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f70049ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7000101de0 0x7f7000198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50616/0 (socket says 172.21.15.5:50616) 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f70049ec640 1 -- 172.21.15.5:0/3594258509 learned_addr learned my addr 172.21.15.5:0/3594258509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.738+0000 7f70051ed640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7000106f10 0x7f700019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.739+0000 7f70049ec640 1 -- 172.21.15.5:0/3594258509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7000102790 msgr2=0x7f7000198a90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.739+0000 7f70049ec640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7000102790 0x7f7000198a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.739+0000 7f70049ec640 1 -- 172.21.15.5:0/3594258509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7000106f10 msgr2=0x7f700019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.739+0000 7f70049ec640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7000106f10 0x7f700019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.739+0000 7f70049ec640 1 -- 172.21.15.5:0/3594258509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700019d500 con 0x7f7000101de0 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.739+0000 7f70049ec640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7000101de0 0x7f7000198550 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000d950 tx=0x7f6ff000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.739+0000 7f6ffdffb640 1 -- 172.21.15.5:0/3594258509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0017070 con 0x7f7000101de0 2024-09-15T22:07:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.739+0000 7f6ffdffb640 1 -- 172.21.15.5:0/3594258509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ff0009c30 con 0x7f7000101de0 2024-09-15T22:07:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.739+0000 7f7006c50640 1 -- 172.21.15.5:0/3594258509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700019d7f0 con 0x7f7000101de0 2024-09-15T22:07:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.739+0000 7f6ffdffb640 1 -- 172.21.15.5:0/3594258509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000f6b0 con 0x7f7000101de0 2024-09-15T22:07:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.739+0000 7f7006c50640 1 -- 172.21.15.5:0/3594258509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70001a50d0 con 0x7f7000101de0 2024-09-15T22:07:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.740+0000 7f7006c50640 1 -- 172.21.15.5:0/3594258509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70001021c0 con 0x7f7000101de0 2024-09-15T22:07:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.741+0000 7f6ffdffb640 1 -- 172.21.15.5:0/3594258509 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f6ff00109d0 con 0x7f7000101de0 2024-09-15T22:07:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.742+0000 7f6ffdffb640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6fec0a3350 0x7f6fec0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.742+0000 7f6ffffff640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6fec0a3350 0x7f6fec0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.742+0000 7f6ffdffb640 1 -- 172.21.15.5:0/3594258509 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f6ff00cfb00 con 0x7f7000101de0 2024-09-15T22:07:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.743+0000 7f6ffffff640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6fec0a3350 0x7f6fec0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f70001087b0 tx=0x7f6fe8006c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.747+0000 7f6ffdffb640 1 -- 172.21.15.5:0/3594258509 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ff0009de0 con 0x7f7000101de0 2024-09-15T22:07:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.915+0000 7f67e79cc640 1 -- 172.21.15.5:0/420645550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67e0102dd0 msgr2=0x7f67e01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.915+0000 7f67e79cc640 1 --2- 172.21.15.5:0/420645550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67e0102dd0 0x7f67e01031b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009930 tx=0x7f67d802a420 comp rx=0 tx=0).stop 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.915+0000 7f67e79cc640 1 -- 172.21.15.5:0/420645550 shutdown_connections 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.915+0000 7f67e79cc640 1 --2- 172.21.15.5:0/420645550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67e0108300 0x7f67e010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.915+0000 7f67e79cc640 1 --2- 172.21.15.5:0/420645550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e0103780 0x7f67e0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.915+0000 7f67e79cc640 1 --2- 172.21.15.5:0/420645550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67e0102dd0 0x7f67e01031b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.915+0000 7f67e79cc640 1 -- 172.21.15.5:0/420645550 >> 172.21.15.5:0/420645550 conn(0x7f67e00fc7d0 msgr2=0x7f67e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.915+0000 7f67e79cc640 1 -- 172.21.15.5:0/420645550 shutdown_connections 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.915+0000 7f67e79cc640 1 -- 172.21.15.5:0/420645550 wait complete. 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.915+0000 7f67e79cc640 1 Processor -- start 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.915+0000 7f67e79cc640 1 -- start start 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e79cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67e0102dd0 0x7f67e01985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e79cc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e0103780 0x7f67e0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e79cc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67e0108300 0x7f67e019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e79cc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67e0112110 con 0x7f67e0102dd0 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e79cc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67e0111f90 con 0x7f67e0103780 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e79cc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67e0112290 con 0x7f67e0108300 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e5768640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67e0102dd0 0x7f67e01985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e5768640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67e0102dd0 0x7f67e01985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50510/0 (socket says 172.21.15.5:50510) 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e5768640 1 -- 172.21.15.5:0/1201797031 learned_addr learned my addr 172.21.15.5:0/1201797031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e4f67640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e0103780 0x7f67e0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e5768640 1 -- 172.21.15.5:0/1201797031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67e0108300 msgr2=0x7f67e019cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e5768640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67e0108300 0x7f67e019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e5768640 1 -- 172.21.15.5:0/1201797031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e0103780 msgr2=0x7f67e0198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e5768640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e0103780 0x7f67e0198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e5768640 1 -- 172.21.15.5:0/1201797031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e019d580 con 0x7f67e0102dd0 2024-09-15T22:07:58.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.916+0000 7f67e5768640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67e0102dd0 0x7f67e01985d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009a50 tx=0x7f67d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:58.861 INFO:teuthology.orchestra.run.smithi005.stdout:231928234027 2024-09-15T22:07:58.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.917+0000 7f67de7fc640 1 -- 172.21.15.5:0/1201797031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d803f030 con 0x7f67e0102dd0 2024-09-15T22:07:58.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.917+0000 7f67e79cc640 1 -- 172.21.15.5:0/1201797031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e019d810 con 0x7f67e0102dd0 2024-09-15T22:07:58.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.917+0000 7f67de7fc640 1 -- 172.21.15.5:0/1201797031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67d8003ce0 con 0x7f67e0102dd0 2024-09-15T22:07:58.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.917+0000 7f67de7fc640 1 -- 172.21.15.5:0/1201797031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8035070 con 0x7f67e0102dd0 2024-09-15T22:07:58.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.917+0000 7f67e79cc640 1 -- 172.21.15.5:0/1201797031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e01a5150 con 0x7f67e0102dd0 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.918+0000 7f67e79cc640 1 -- 172.21.15.5:0/1201797031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e01031b0 con 0x7f67e0102dd0 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.919+0000 7f67de7fc640 1 -- 172.21.15.5:0/1201797031 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f67d8042030 con 0x7f67e0102dd0 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.920+0000 7f67de7fc640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f67c80a32d0 0x7f67c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.920+0000 7f67de7fc640 1 -- 172.21.15.5:0/1201797031 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f67d80f04d0 con 0x7f67e0102dd0 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.920+0000 7f67e4f67640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f67c80a32d0 0x7f67c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.920+0000 7f67e4f67640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f67c80a32d0 0x7f67c80a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005e10 tx=0x7f67d4005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:55.924+0000 7f67de7fc640 1 -- 172.21.15.5:0/1201797031 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67d80bccd0 con 0x7f67e0102dd0 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.108+0000 7f67e79cc640 1 -- 172.21.15.5:0/1201797031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f67e010a290 con 0x7f67e0102dd0 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.108+0000 7f67de7fc640 1 -- 172.21.15.5:0/1201797031 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f67d80bc5d0 con 0x7f67e0102dd0 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 -- 172.21.15.5:0/1201797031 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f67c80a32d0 msgr2=0x7f67c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f67c80a32d0 0x7f67c80a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f67d4005e10 tx=0x7f67d4005da0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 -- 172.21.15.5:0/1201797031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67e0102dd0 msgr2=0x7f67e01985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:58.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67e0102dd0 0x7f67e01985d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009a50 tx=0x7f67d8004270 comp rx=0 tx=0).stop 2024-09-15T22:07:58.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 -- 172.21.15.5:0/1201797031 shutdown_connections 2024-09-15T22:07:58.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f67e0108300 0x7f67e019cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f67e0103780 0x7f67e0198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f67c80a32d0 0x7f67c80a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 --2- 172.21.15.5:0/1201797031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67e0102dd0 0x7f67e01985d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 -- 172.21.15.5:0/1201797031 >> 172.21.15.5:0/1201797031 conn(0x7f67e00fc7d0 msgr2=0x7f67e01063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:58.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 -- 172.21.15.5:0/1201797031 shutdown_connections 2024-09-15T22:07:58.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:56.110+0000 7f67e79cc640 1 -- 172.21.15.5:0/1201797031 wait complete. 2024-09-15T22:07:58.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.924+0000 7f7006c50640 1 -- 172.21.15.5:0/3594258509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f700010a150 con 0x7f7000101de0 2024-09-15T22:07:58.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.924+0000 7f6ffdffb640 1 -- 172.21.15.5:0/3594258509 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6ff009c300 con 0x7f7000101de0 2024-09-15T22:07:58.926 INFO:teuthology.orchestra.run.smithi005.stdout:261993005096 2024-09-15T22:07:58.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 -- 172.21.15.5:0/3594258509 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6fec0a3350 msgr2=0x7f6fec0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:58.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6fec0a3350 0x7f6fec0a5810 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f70001087b0 tx=0x7f6fe8006c20 comp rx=0 tx=0).stop 2024-09-15T22:07:58.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 -- 172.21.15.5:0/3594258509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7000101de0 msgr2=0x7f7000198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:07:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7000101de0 0x7f7000198550 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000d950 tx=0x7f6ff000de10 comp rx=0 tx=0).stop 2024-09-15T22:07:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 -- 172.21.15.5:0/3594258509 shutdown_connections 2024-09-15T22:07:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7000106f10 0x7f700019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7000102790 0x7f7000198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6fec0a3350 0x7f6fec0a5810 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 --2- 172.21.15.5:0/3594258509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7000101de0 0x7f7000198550 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:07:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 -- 172.21.15.5:0/3594258509 >> 172.21.15.5:0/3594258509 conn(0x7f70000fd8c0 msgr2=0x7f70000fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:07:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 -- 172.21.15.5:0/3594258509 shutdown_connections 2024-09-15T22:07:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:07:58.926+0000 7f7006c50640 1 -- 172.21.15.5:0/3594258509 wait complete. 2024-09-15T22:07:59.509 INFO:tasks.cephadm.ceph_manager.ceph:need seq 98784247875 got 98784247877 for osd.3 2024-09-15T22:07:59.509 DEBUG:teuthology.parallel:result is None 2024-09-15T22:07:59.942 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234024 got 231928234027 for osd.8 2024-09-15T22:07:59.942 DEBUG:teuthology.parallel:result is None 2024-09-15T22:08:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.109+0000 7f06386b0640 1 -- 172.21.15.5:0/3366956802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0630102790 msgr2=0x7f06301067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.109+0000 7f06386b0640 1 --2- 172.21.15.5:0/3366956802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0630102790 0x7f06301067e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0620009900 tx=0x7f062002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.109+0000 7f06386b0640 1 -- 172.21.15.5:0/3366956802 shutdown_connections 2024-09-15T22:08:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.109+0000 7f06386b0640 1 --2- 172.21.15.5:0/3366956802 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630106f10 0x7f063010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.109+0000 7f06386b0640 1 --2- 172.21.15.5:0/3366956802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0630102790 0x7f06301067e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.109+0000 7f06386b0640 1 --2- 172.21.15.5:0/3366956802 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0630101de0 0x7f06301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.109+0000 7f06386b0640 1 -- 172.21.15.5:0/3366956802 >> 172.21.15.5:0/3366956802 conn(0x7f06300fd8c0 msgr2=0x7f06300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.109+0000 7f06386b0640 1 -- 172.21.15.5:0/3366956802 shutdown_connections 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.109+0000 7f06386b0640 1 -- 172.21.15.5:0/3366956802 wait complete. 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f06386b0640 1 Processor -- start 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f06386b0640 1 -- start start 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f06386b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0630101de0 0x7f06300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f06386b0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630102790 0x7f0630074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f06386b0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0630106f10 0x7f0630074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f06386b0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0630111f30 con 0x7f0630101de0 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f06386b0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0630111db0 con 0x7f0630102790 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f06386b0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06301120b0 con 0x7f0630106f10 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f063644c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0630101de0 0x7f06300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f063644c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0630101de0 0x7f06300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50638/0 (socket says 172.21.15.5:50638) 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f063644c640 1 -- 172.21.15.5:0/267801581 learned_addr learned my addr 172.21.15.5:0/267801581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f0635c4b640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630102790 0x7f0630074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f063644c640 1 -- 172.21.15.5:0/267801581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0630106f10 msgr2=0x7f0630074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f0636c4d640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0630106f10 0x7f0630074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f063644c640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0630106f10 0x7f0630074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f063644c640 1 -- 172.21.15.5:0/267801581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630102790 msgr2=0x7f0630074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f063644c640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630102790 0x7f0630074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f063644c640 1 -- 172.21.15.5:0/267801581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06301a68b0 con 0x7f0630101de0 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.110+0000 7f0636c4d640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0630106f10 0x7f0630074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.111+0000 7f063644c640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0630101de0 0x7f06300744f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c920 tx=0x7f062c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.111+0000 7f06277fe640 1 -- 172.21.15.5:0/267801581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c007c70 con 0x7f0630101de0 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.111+0000 7f06277fe640 1 -- 172.21.15.5:0/267801581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f062c007e10 con 0x7f0630101de0 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.111+0000 7f06386b0640 1 -- 172.21.15.5:0/267801581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06301a6ba0 con 0x7f0630101de0 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.111+0000 7f06277fe640 1 -- 172.21.15.5:0/267801581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c018890 con 0x7f0630101de0 2024-09-15T22:08:00.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.111+0000 7f06386b0640 1 -- 172.21.15.5:0/267801581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06301a70e0 con 0x7f0630101de0 2024-09-15T22:08:00.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.112+0000 7f06386b0640 1 -- 172.21.15.5:0/267801581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06301021c0 con 0x7f0630101de0 2024-09-15T22:08:00.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.113+0000 7f06277fe640 1 -- 172.21.15.5:0/267801581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f062c018a70 con 0x7f0630101de0 2024-09-15T22:08:00.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.114+0000 7f06277fe640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f061c0a32d0 0x7f061c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:00.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.114+0000 7f0635c4b640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f061c0a32d0 0x7f061c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:00.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.114+0000 7f06277fe640 1 -- 172.21.15.5:0/267801581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f062c0d0060 con 0x7f0630101de0 2024-09-15T22:08:00.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.115+0000 7f0635c4b640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f061c0a32d0 0x7f061c0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0630075f80 tx=0x7f0620005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:00.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.118+0000 7f06277fe640 1 -- 172.21.15.5:0/267801581 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f062c014030 con 0x7f0630101de0 2024-09-15T22:08:00.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:59 smithi005 ceph-mon[30944]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-09-15T22:08:00.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:07:59 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3594258509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:08:00.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.296+0000 7f06386b0640 1 -- 172.21.15.5:0/267801581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f0630075a20 con 0x7f0630101de0 2024-09-15T22:08:00.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.296+0000 7f06277fe640 1 -- 172.21.15.5:0/267801581 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f062c00de20 con 0x7f0630101de0 2024-09-15T22:08:00.299 INFO:teuthology.orchestra.run.smithi005.stdout:176093659191 2024-09-15T22:08:00.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 -- 172.21.15.5:0/267801581 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f061c0a32d0 msgr2=0x7f061c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:00.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f061c0a32d0 0x7f061c0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0630075f80 tx=0x7f0620005a20 comp rx=0 tx=0).stop 2024-09-15T22:08:00.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 -- 172.21.15.5:0/267801581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0630101de0 msgr2=0x7f06300744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:00.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0630101de0 0x7f06300744f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f062c00c920 tx=0x7f062c00cde0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 -- 172.21.15.5:0/267801581 shutdown_connections 2024-09-15T22:08:00.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0630106f10 0x7f0630074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0630102790 0x7f0630074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f061c0a32d0 0x7f061c0a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 --2- 172.21.15.5:0/267801581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0630101de0 0x7f06300744f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 -- 172.21.15.5:0/267801581 >> 172.21.15.5:0/267801581 conn(0x7f06300fd8c0 msgr2=0x7f06300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:00.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 -- 172.21.15.5:0/267801581 shutdown_connections 2024-09-15T22:08:00.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.299+0000 7f06386b0640 1 -- 172.21.15.5:0/267801581 wait complete. 2024-09-15T22:08:00.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:59 smithi099 ceph-mon[34892]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-09-15T22:08:00.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:07:59 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3594258509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:08:00.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:59 smithi129 ceph-mon[35314]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-09-15T22:08:00.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:07:59 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3594258509' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-15T22:08:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.873+0000 7f4faa077640 1 -- 172.21.15.5:0/402297878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4102790 msgr2=0x7f4fa41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.873+0000 7f4faa077640 1 --2- 172.21.15.5:0/402297878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4102790 0x7f4fa41067e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009900 tx=0x7f4f9802a410 comp rx=0 tx=0).stop 2024-09-15T22:08:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.873+0000 7f4faa077640 1 -- 172.21.15.5:0/402297878 shutdown_connections 2024-09-15T22:08:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.873+0000 7f4faa077640 1 --2- 172.21.15.5:0/402297878 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fa4106f10 0x7f4fa410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.873+0000 7f4faa077640 1 --2- 172.21.15.5:0/402297878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4102790 0x7f4fa41067e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.873+0000 7f4faa077640 1 --2- 172.21.15.5:0/402297878 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa4101de0 0x7f4fa41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.873+0000 7f4faa077640 1 -- 172.21.15.5:0/402297878 >> 172.21.15.5:0/402297878 conn(0x7f4fa40fd8c0 msgr2=0x7f4fa40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.873+0000 7f4faa077640 1 -- 172.21.15.5:0/402297878 shutdown_connections 2024-09-15T22:08:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.873+0000 7f4faa077640 1 -- 172.21.15.5:0/402297878 wait complete. 2024-09-15T22:08:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4faa077640 1 Processor -- start 2024-09-15T22:08:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4faa077640 1 -- start start 2024-09-15T22:08:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4faa077640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fa4101de0 0x7f4fa4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4faa077640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa4102790 0x7f4fa4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4faa077640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4106f10 0x7f4fa419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4faa077640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4111fe0 con 0x7f4fa4106f10 2024-09-15T22:08:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4faa077640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4111e60 con 0x7f4fa4101de0 2024-09-15T22:08:00.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4faa077640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4112160 con 0x7f4fa4102790 2024-09-15T22:08:00.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4fa3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4106f10 0x7f4fa419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:00.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4fa3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4106f10 0x7f4fa419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50642/0 (socket says 172.21.15.5:50642) 2024-09-15T22:08:00.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4fa3fff640 1 -- 172.21.15.5:0/1613115661 learned_addr learned my addr 172.21.15.5:0/1613115661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:00.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.874+0000 7f4fa37fe640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fa4101de0 0x7f4fa4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:00.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.875+0000 7f4fa3fff640 1 -- 172.21.15.5:0/1613115661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa4102790 msgr2=0x7f4fa4198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:00.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.875+0000 7f4fa3fff640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa4102790 0x7f4fa4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.875+0000 7f4fa3fff640 1 -- 172.21.15.5:0/1613115661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fa4101de0 msgr2=0x7f4fa4198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:00.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.875+0000 7f4fa3fff640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fa4101de0 0x7f4fa4198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:00.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.875+0000 7f4fa3fff640 1 -- 172.21.15.5:0/1613115661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa419d4d0 con 0x7f4fa4106f10 2024-09-15T22:08:00.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.875+0000 7f4fa3fff640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4106f10 0x7f4fa419cdf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00c970 tx=0x7f4f9c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:00.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.877+0000 7f4fa0ff9640 1 -- 172.21.15.5:0/1613115661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c007bb0 con 0x7f4fa4106f10 2024-09-15T22:08:00.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.877+0000 7f4fa0ff9640 1 -- 172.21.15.5:0/1613115661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f9c007d50 con 0x7f4fa4106f10 2024-09-15T22:08:00.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.877+0000 7f4faa077640 1 -- 172.21.15.5:0/1613115661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa419d7c0 con 0x7f4fa4106f10 2024-09-15T22:08:00.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.877+0000 7f4fa0ff9640 1 -- 172.21.15.5:0/1613115661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f9c018800 con 0x7f4fa4106f10 2024-09-15T22:08:00.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.877+0000 7f4faa077640 1 -- 172.21.15.5:0/1613115661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa41a50a0 con 0x7f4fa4106f10 2024-09-15T22:08:00.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.878+0000 7f4faa077640 1 -- 172.21.15.5:0/1613115661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa41021c0 con 0x7f4fa4106f10 2024-09-15T22:08:00.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.879+0000 7f4fa0ff9640 1 -- 172.21.15.5:0/1613115661 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f4f9c00fa50 con 0x7f4fa4106f10 2024-09-15T22:08:00.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.880+0000 7f4fa0ff9640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4f8c0a32d0 0x7f4f8c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:00.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.880+0000 7f4fa0ff9640 1 -- 172.21.15.5:0/1613115661 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f4f9c0cfeb0 con 0x7f4fa4106f10 2024-09-15T22:08:00.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.880+0000 7f4fa37fe640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4f8c0a32d0 0x7f4f8c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:00.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.880+0000 7f4fa37fe640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4f8c0a32d0 0x7f4f8c0a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400a400 tx=0x7f4f9400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:00.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:00.884+0000 7f4fa0ff9640 1 -- 172.21.15.5:0/1613115661 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f9c014030 con 0x7f4fa4106f10 2024-09-15T22:08:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.059+0000 7f4faa077640 1 -- 172.21.15.5:0/1613115661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4fa410a0f0 con 0x7f4fa4106f10 2024-09-15T22:08:01.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:00 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/267801581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:08:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.060+0000 7f4fa0ff9640 1 -- 172.21.15.5:0/1613115661 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4f9c09c6b0 con 0x7f4fa4106f10 2024-09-15T22:08:01.063 INFO:teuthology.orchestra.run.smithi005.stdout:73014444108 2024-09-15T22:08:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 -- 172.21.15.5:0/1613115661 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4f8c0a32d0 msgr2=0x7f4f8c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4f8c0a32d0 0x7f4f8c0a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f9400a400 tx=0x7f4f9400d040 comp rx=0 tx=0).stop 2024-09-15T22:08:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 -- 172.21.15.5:0/1613115661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4106f10 msgr2=0x7f4fa419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4106f10 0x7f4fa419cdf0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c00c970 tx=0x7f4f9c00ce30 comp rx=0 tx=0).stop 2024-09-15T22:08:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 -- 172.21.15.5:0/1613115661 shutdown_connections 2024-09-15T22:08:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4f8c0a32d0 0x7f4f8c0a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fa4106f10 0x7f4fa419cdf0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4fa4102790 0x7f4fa4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 --2- 172.21.15.5:0/1613115661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4fa4101de0 0x7f4fa4198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 -- 172.21.15.5:0/1613115661 >> 172.21.15.5:0/1613115661 conn(0x7f4fa40fd8c0 msgr2=0x7f4fa40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 -- 172.21.15.5:0/1613115661 shutdown_connections 2024-09-15T22:08:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:01.062+0000 7f4faa077640 1 -- 172.21.15.5:0/1613115661 wait complete. 2024-09-15T22:08:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:00 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/267801581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:08:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:00 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/267801581' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-15T22:08:01.863 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738452 got 34359738453 for osd.0 2024-09-15T22:08:01.863 DEBUG:teuthology.parallel:result is None 2024-09-15T22:08:02.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:01 smithi005 ceph-mon[30944]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-09-15T22:08:02.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:01 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1613115661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:08:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:01 smithi099 ceph-mon[34892]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-09-15T22:08:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:01 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1613115661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:08:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:01 smithi129 ceph-mon[35314]: pgmap v327: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s; 48 B/s, 0 objects/s recovering 2024-09-15T22:08:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:01 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1613115661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-15T22:08:02.520 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005093 got 261993005096 for osd.9 2024-09-15T22:08:02.520 DEBUG:teuthology.parallel:result is None 2024-09-15T22:08:02.890 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:02.913 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659188 got 176093659191 for osd.6 2024-09-15T22:08:02.914 DEBUG:teuthology.parallel:result is None 2024-09-15T22:08:03.316 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808864 got 287762808867 for osd.10 2024-09-15T22:08:03.316 DEBUG:teuthology.parallel:result is None 2024-09-15T22:08:03.324 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:03.324 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:03.808 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444107 got 73014444108 for osd.2 2024-09-15T22:08:03.809 DEBUG:teuthology.parallel:result is None 2024-09-15T22:08:04.033 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:04.034 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:03 smithi005 ceph-mon[30944]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:04.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:03 smithi099 ceph-mon[34892]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:03 smithi129 ceph-mon[35314]: pgmap v328: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.453+0000 7fae3fb26640 1 -- 172.21.15.5:0/954491938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae381008d0 msgr2=0x7fae38100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.453+0000 7fae3fb26640 1 --2- 172.21.15.5:0/954491938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae381008d0 0x7fae38100cb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fae34009900 tx=0x7fae3402a410 comp rx=0 tx=0).stop 2024-09-15T22:08:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.453+0000 7fae3fb26640 1 -- 172.21.15.5:0/954491938 shutdown_connections 2024-09-15T22:08:04.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.453+0000 7fae3fb26640 1 --2- 172.21.15.5:0/954491938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae3810e480 0x7fae38110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.453+0000 7fae3fb26640 1 --2- 172.21.15.5:0/954491938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae38101280 0x7fae3810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.453+0000 7fae3fb26640 1 --2- 172.21.15.5:0/954491938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae381008d0 0x7fae38100cb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.453+0000 7fae3fb26640 1 -- 172.21.15.5:0/954491938 >> 172.21.15.5:0/954491938 conn(0x7fae380fc7a0 msgr2=0x7fae380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.453+0000 7fae3fb26640 1 -- 172.21.15.5:0/954491938 shutdown_connections 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.453+0000 7fae3fb26640 1 -- 172.21.15.5:0/954491938 wait complete. 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.454+0000 7fae3fb26640 1 Processor -- start 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.454+0000 7fae3fb26640 1 -- start start 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.454+0000 7fae3fb26640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae381008d0 0x7fae3819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.454+0000 7fae3fb26640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae38101280 0x7fae3819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.454+0000 7fae3fb26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3810e480 0x7fae3819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.454+0000 7fae3fb26640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae381141e0 con 0x7fae3810e480 2024-09-15T22:08:04.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.454+0000 7fae3fb26640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae38114060 con 0x7fae38101280 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.454+0000 7fae3fb26640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae38114360 con 0x7fae381008d0 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.454+0000 7fae3e0c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3810e480 0x7fae3819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.454+0000 7fae3e0c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3810e480 0x7fae3819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57154/0 (socket says 172.21.15.5:57154) 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.454+0000 7fae3e0c3640 1 -- 172.21.15.5:0/3841844934 learned_addr learned my addr 172.21.15.5:0/3841844934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae3d8c2640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae381008d0 0x7fae3819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae3e0c3640 1 -- 172.21.15.5:0/3841844934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae381008d0 msgr2=0x7fae3819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae3e0c3640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae381008d0 0x7fae3819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae3e0c3640 1 -- 172.21.15.5:0/3841844934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae38101280 msgr2=0x7fae3819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae3e0c3640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae38101280 0x7fae3819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae3e0c3640 1 -- 172.21.15.5:0/3841844934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3819f630 con 0x7fae3810e480 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae3e0c3640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3810e480 0x7fae3819ef50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fae30009fd0 tx=0x7fae3000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae2effd640 1 -- 172.21.15.5:0/3841844934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30004d20 con 0x7fae3810e480 2024-09-15T22:08:04.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae2effd640 1 -- 172.21.15.5:0/3841844934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae30005020 con 0x7fae3810e480 2024-09-15T22:08:04.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae3fb26640 1 -- 172.21.15.5:0/3841844934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3819f920 con 0x7fae3810e480 2024-09-15T22:08:04.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae2effd640 1 -- 172.21.15.5:0/3841844934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae300116d0 con 0x7fae3810e480 2024-09-15T22:08:04.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.455+0000 7fae3fb26640 1 -- 172.21.15.5:0/3841844934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae381a7200 con 0x7fae3810e480 2024-09-15T22:08:04.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.456+0000 7fae3fb26640 1 -- 172.21.15.5:0/3841844934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae38100cb0 con 0x7fae3810e480 2024-09-15T22:08:04.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.457+0000 7fae2effd640 1 -- 172.21.15.5:0/3841844934 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fae300051d0 con 0x7fae3810e480 2024-09-15T22:08:04.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.458+0000 7fae2effd640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fae240a32d0 0x7fae240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:04.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.458+0000 7fae2effd640 1 -- 172.21.15.5:0/3841844934 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fae300d04d0 con 0x7fae3810e480 2024-09-15T22:08:04.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.458+0000 7fae3d8c2640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fae240a32d0 0x7fae240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:04.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.458+0000 7fae3d8c2640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fae240a32d0 0x7fae240a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae34000c00 tx=0x7fae34005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:04.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.463+0000 7fae2effd640 1 -- 172.21.15.5:0/3841844934 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae30016030 con 0x7fae3810e480 2024-09-15T22:08:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.645+0000 7fae3fb26640 1 -- 172.21.15.5:0/3841844934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fae38109080 con 0x7fae3810e480 2024-09-15T22:08:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.646+0000 7fae2effd640 1 -- 172.21.15.5:0/3841844934 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fae300bc030 con 0x7fae3810e480 2024-09-15T22:08:04.648 INFO:teuthology.orchestra.run.smithi005.stdout:128849018946 2024-09-15T22:08:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 -- 172.21.15.5:0/3841844934 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fae240a32d0 msgr2=0x7fae240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fae240a32d0 0x7fae240a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fae34000c00 tx=0x7fae34005a20 comp rx=0 tx=0).stop 2024-09-15T22:08:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 -- 172.21.15.5:0/3841844934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3810e480 msgr2=0x7fae3819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3810e480 0x7fae3819ef50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fae30009fd0 tx=0x7fae3000ee60 comp rx=0 tx=0).stop 2024-09-15T22:08:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 -- 172.21.15.5:0/3841844934 shutdown_connections 2024-09-15T22:08:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fae240a32d0 0x7fae240a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3810e480 0x7fae3819ef50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae38101280 0x7fae3819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 --2- 172.21.15.5:0/3841844934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae381008d0 0x7fae3819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 -- 172.21.15.5:0/3841844934 >> 172.21.15.5:0/3841844934 conn(0x7fae380fc7a0 msgr2=0x7fae380feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 -- 172.21.15.5:0/3841844934 shutdown_connections 2024-09-15T22:08:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.648+0000 7fae3fb26640 1 -- 172.21.15.5:0/3841844934 wait complete. 2024-09-15T22:08:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.669+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/575260848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1ac10e4d0 msgr2=0x7fc1ac1108c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.669+0000 7fc1b1f3b640 1 --2- 172.21.15.5:0/575260848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1ac10e4d0 0x7fc1ac1108c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4009ca0 tx=0x7fc1a402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.669+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/575260848 shutdown_connections 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.669+0000 7fc1b1f3b640 1 --2- 172.21.15.5:0/575260848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1ac10e4d0 0x7fc1ac1108c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.669+0000 7fc1b1f3b640 1 --2- 172.21.15.5:0/575260848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1ac1012d0 0x7fc1ac10df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.669+0000 7fc1b1f3b640 1 --2- 172.21.15.5:0/575260848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac100920 0x7fc1ac100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.669+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/575260848 >> 172.21.15.5:0/575260848 conn(0x7fc1ac0fc7d0 msgr2=0x7fc1ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.669+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/575260848 shutdown_connections 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.669+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/575260848 wait complete. 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.670+0000 7fc1b1f3b640 1 Processor -- start 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.670+0000 7fc1b1f3b640 1 -- start start 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.670+0000 7fc1b1f3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1ac100920 0x7fc1ac19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.670+0000 7fc1b1f3b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1ac1012d0 0x7fc1ac19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.670+0000 7fc1b1f3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac10e4d0 0x7fc1ac19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.670+0000 7fc1b1f3b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac1141a0 con 0x7fc1ac100920 2024-09-15T22:08:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.670+0000 7fc1b1f3b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac114020 con 0x7fc1ac10e4d0 2024-09-15T22:08:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.670+0000 7fc1b1f3b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc1ac114320 con 0x7fc1ac1012d0 2024-09-15T22:08:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.670+0000 7fc1ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1ac100920 0x7fc1ac19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.670+0000 7fc1ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1ac100920 0x7fc1ac19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57174/0 (socket says 172.21.15.5:57174) 2024-09-15T22:08:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.670+0000 7fc1ab7fe640 1 -- 172.21.15.5:0/3700738403 learned_addr learned my addr 172.21.15.5:0/3700738403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1abfff640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac10e4d0 0x7fc1ac19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1a2ffd640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1ac1012d0 0x7fc1ac19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1ab7fe640 1 -- 172.21.15.5:0/3700738403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1ac1012d0 msgr2=0x7fc1ac19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1ab7fe640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1ac1012d0 0x7fc1ac19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1ab7fe640 1 -- 172.21.15.5:0/3700738403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac10e4d0 msgr2=0x7fc1ac19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1ab7fe640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac10e4d0 0x7fc1ac19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1ab7fe640 1 -- 172.21.15.5:0/3700738403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1ac19f630 con 0x7fc1ac100920 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1ab7fe640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1ac100920 0x7fc1ac19a680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e9f0 tx=0x7fc19c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1a97fa640 1 -- 172.21.15.5:0/3700738403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00cd80 con 0x7fc1ac100920 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/3700738403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1ac19f920 con 0x7fc1ac100920 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1a97fa640 1 -- 172.21.15.5:0/3700738403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc19c010910 con 0x7fc1ac100920 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1a97fa640 1 -- 172.21.15.5:0/3700738403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0188f0 con 0x7fc1ac100920 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.671+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/3700738403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1ac1a7200 con 0x7fc1ac100920 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.672+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/3700738403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1ac100d00 con 0x7fc1ac100920 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.674+0000 7fc1a97fa640 1 -- 172.21.15.5:0/3700738403 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fc19c010ac0 con 0x7fc1ac100920 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.675+0000 7fc1a97fa640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc18c0a32d0 0x7fc18c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.675+0000 7fc1a97fa640 1 -- 172.21.15.5:0/3700738403 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fc19c0d06d0 con 0x7fc1ac100920 2024-09-15T22:08:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.675+0000 7fc1a2ffd640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc18c0a32d0 0x7fc18c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.675+0000 7fc1a2ffd640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc18c0a32d0 0x7fc18c0a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac102300 tx=0x7fc194008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.681+0000 7fc1a97fa640 1 -- 172.21.15.5:0/3700738403 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc19c09ced0 con 0x7fc1ac100920 2024-09-15T22:08:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.861+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/3700738403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc1ac109660 con 0x7fc1ac100920 2024-09-15T22:08:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.862+0000 7fc1a97fa640 1 -- 172.21.15.5:0/3700738403 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc19c09c7d0 con 0x7fc1ac100920 2024-09-15T22:08:04.864 INFO:teuthology.orchestra.run.smithi005.stdout:146028888125 2024-09-15T22:08:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/3700738403 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc18c0a32d0 msgr2=0x7fc18c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc18c0a32d0 0x7fc18c0a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac102300 tx=0x7fc194008040 comp rx=0 tx=0).stop 2024-09-15T22:08:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/3700738403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1ac100920 msgr2=0x7fc1ac19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1ac100920 0x7fc1ac19a680 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e9f0 tx=0x7fc19c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/3700738403 shutdown_connections 2024-09-15T22:08:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc1ac10e4d0 0x7fc1ac19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc1ac1012d0 0x7fc1ac19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fc18c0a32d0 0x7fc18c0a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 --2- 172.21.15.5:0/3700738403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1ac100920 0x7fc1ac19a680 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/3700738403 >> 172.21.15.5:0/3700738403 conn(0x7fc1ac0fc7d0 msgr2=0x7fc1ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/3700738403 shutdown_connections 2024-09-15T22:08:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:04.864+0000 7fc1b1f3b640 1 -- 172.21.15.5:0/3700738403 wait complete. 2024-09-15T22:08:05.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:04 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3841844934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:08:05.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:04 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3700738403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:08:05.121 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:04 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3841844934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:08:05.121 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:04 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3700738403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:08:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:04 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3841844934' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-15T22:08:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:04 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3700738403' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-15T22:08:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.751+0000 7fe790634640 1 -- 172.21.15.5:0/3956461288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe78810e4d0 msgr2=0x7fe7881108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.751+0000 7fe790634640 1 --2- 172.21.15.5:0/3956461288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe78810e4d0 0x7fe7881108e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe780009ca0 tx=0x7fe78002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:08:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.751+0000 7fe790634640 1 -- 172.21.15.5:0/3956461288 shutdown_connections 2024-09-15T22:08:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.751+0000 7fe790634640 1 --2- 172.21.15.5:0/3956461288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe78810e4d0 0x7fe7881108e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.751+0000 7fe790634640 1 --2- 172.21.15.5:0/3956461288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe788101360 0x7fe78810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.751+0000 7fe790634640 1 --2- 172.21.15.5:0/3956461288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7881009b0 0x7fe788100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.751+0000 7fe790634640 1 -- 172.21.15.5:0/3956461288 >> 172.21.15.5:0/3956461288 conn(0x7fe7880fc7d0 msgr2=0x7fe7880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.751+0000 7fe790634640 1 -- 172.21.15.5:0/3956461288 shutdown_connections 2024-09-15T22:08:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.751+0000 7fe790634640 1 -- 172.21.15.5:0/3956461288 wait complete. 2024-09-15T22:08:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe790634640 1 Processor -- start 2024-09-15T22:08:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe790634640 1 -- start start 2024-09-15T22:08:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe790634640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7881009b0 0x7fe78819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe790634640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788101360 0x7fe78819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe790634640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe78810e4d0 0x7fe78819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe790634640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe788114250 con 0x7fe788101360 2024-09-15T22:08:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe790634640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7881140d0 con 0x7fe7881009b0 2024-09-15T22:08:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe790634640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7881143d0 con 0x7fe78810e4d0 2024-09-15T22:08:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe78dbcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788101360 0x7fe78819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe78dbcf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788101360 0x7fe78819ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57204/0 (socket says 172.21.15.5:57204) 2024-09-15T22:08:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe78dbcf640 1 -- 172.21.15.5:0/9878831 learned_addr learned my addr 172.21.15.5:0/9878831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.752+0000 7fe78e3d0640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7881009b0 0x7fe78819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe78ebd1640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe78810e4d0 0x7fe78819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe78dbcf640 1 -- 172.21.15.5:0/9878831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe78810e4d0 msgr2=0x7fe78819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe78dbcf640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe78810e4d0 0x7fe78819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe78dbcf640 1 -- 172.21.15.5:0/9878831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7881009b0 msgr2=0x7fe78819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe78dbcf640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7881009b0 0x7fe78819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe78dbcf640 1 -- 172.21.15.5:0/9878831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78819f6e0 con 0x7fe788101360 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe78e3d0640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7881009b0 0x7fe78819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe78ebd1640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe78810e4d0 0x7fe78819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe78dbcf640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788101360 0x7fe78819ac70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00e9f0 tx=0x7fe77c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe77b7fe640 1 -- 172.21.15.5:0/9878831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00cd80 con 0x7fe788101360 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe77b7fe640 1 -- 172.21.15.5:0/9878831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe77c010910 con 0x7fe788101360 2024-09-15T22:08:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe77b7fe640 1 -- 172.21.15.5:0/9878831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77c018990 con 0x7fe788101360 2024-09-15T22:08:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe790634640 1 -- 172.21.15.5:0/9878831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78819f9d0 con 0x7fe788101360 2024-09-15T22:08:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.753+0000 7fe790634640 1 -- 172.21.15.5:0/9878831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7881a72b0 con 0x7fe788101360 2024-09-15T22:08:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.754+0000 7fe790634640 1 -- 172.21.15.5:0/9878831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe788100d90 con 0x7fe788101360 2024-09-15T22:08:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.756+0000 7fe77b7fe640 1 -- 172.21.15.5:0/9878831 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fe77c010ac0 con 0x7fe788101360 2024-09-15T22:08:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.756+0000 7fe77b7fe640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe7740a32d0 0x7fe7740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.756+0000 7fe77b7fe640 1 -- 172.21.15.5:0/9878831 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fe77c0d05a0 con 0x7fe788101360 2024-09-15T22:08:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.756+0000 7fe78e3d0640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe7740a32d0 0x7fe7740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.757+0000 7fe78e3d0640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe7740a32d0 0x7fe7740a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe78400bfd0 tx=0x7fe78400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.759+0000 7fe77b7fe640 1 -- 172.21.15.5:0/9878831 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe77c09cda0 con 0x7fe788101360 2024-09-15T22:08:05.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.941+0000 7fe790634640 1 -- 172.21.15.5:0/9878831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fe788109690 con 0x7fe788101360 2024-09-15T22:08:05.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.941+0000 7fe77b7fe640 1 -- 172.21.15.5:0/9878831 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe77c09c6a0 con 0x7fe788101360 2024-09-15T22:08:05.944 INFO:teuthology.orchestra.run.smithi005.stdout:197568495668 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 -- 172.21.15.5:0/9878831 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe7740a32d0 msgr2=0x7fe7740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe7740a32d0 0x7fe7740a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe78400bfd0 tx=0x7fe78400b390 comp rx=0 tx=0).stop 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 -- 172.21.15.5:0/9878831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788101360 msgr2=0x7fe78819ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788101360 0x7fe78819ac70 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe77c00e9f0 tx=0x7fe77c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 -- 172.21.15.5:0/9878831 shutdown_connections 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe78810e4d0 0x7fe78819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe7740a32d0 0x7fe7740a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe788101360 0x7fe78819ac70 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 --2- 172.21.15.5:0/9878831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7881009b0 0x7fe78819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 -- 172.21.15.5:0/9878831 >> 172.21.15.5:0/9878831 conn(0x7fe7880fc7d0 msgr2=0x7fe7880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 -- 172.21.15.5:0/9878831 shutdown_connections 2024-09-15T22:08:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.943+0000 7fe790634640 1 -- 172.21.15.5:0/9878831 wait complete. 2024-09-15T22:08:06.100 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:05 smithi005 ceph-mon[30944]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:06.100 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:05 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4168440222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:08:06.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:05 smithi099 ceph-mon[34892]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:06.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:05 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4168440222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:08:06.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:05 smithi129 ceph-mon[35314]: pgmap v329: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:06.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:05 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4168440222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-15T22:08:06.447 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888121 got 146028888125 for osd.5 2024-09-15T22:08:06.447 DEBUG:teuthology.parallel:result is None 2024-09-15T22:08:06.776 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018942 got 128849018946 for osd.4 2024-09-15T22:08:06.776 DEBUG:teuthology.parallel:result is None 2024-09-15T22:08:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.656+0000 7f366d598640 1 -- 172.21.15.5:0/1835338959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3668074f50 msgr2=0x7f366810df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.656+0000 7f366d598640 1 --2- 172.21.15.5:0/1835338959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3668074f50 0x7f366810df30 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3650009900 tx=0x7f365002a410 comp rx=0 tx=0).stop 2024-09-15T22:08:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.656+0000 7f366d598640 1 -- 172.21.15.5:0/1835338959 shutdown_connections 2024-09-15T22:08:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.656+0000 7f366d598640 1 --2- 172.21.15.5:0/1835338959 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f366810e470 0x7f36681108a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.656+0000 7f366d598640 1 --2- 172.21.15.5:0/1835338959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3668074f50 0x7f366810df30 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.656+0000 7f366d598640 1 --2- 172.21.15.5:0/1835338959 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36680745a0 0x7f3668074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.657+0000 7f366d598640 1 -- 172.21.15.5:0/1835338959 >> 172.21.15.5:0/1835338959 conn(0x7f36680fd8d0 msgr2=0x7f36680ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.657+0000 7f366d598640 1 -- 172.21.15.5:0/1835338959 shutdown_connections 2024-09-15T22:08:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.657+0000 7f366d598640 1 -- 172.21.15.5:0/1835338959 wait complete. 2024-09-15T22:08:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.657+0000 7f366d598640 1 Processor -- start 2024-09-15T22:08:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.657+0000 7f366d598640 1 -- start start 2024-09-15T22:08:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.657+0000 7f366d598640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36680745a0 0x7f366819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.657+0000 7f366d598640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668074f50 0x7f366819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f366d598640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366810e470 0x7f366819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f366d598640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36681141d0 con 0x7f366810e470 2024-09-15T22:08:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f366d598640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3668114050 con 0x7f36680745a0 2024-09-15T22:08:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f366d598640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3668114350 con 0x7f3668074f50 2024-09-15T22:08:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f36677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366810e470 0x7f366819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f36677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366810e470 0x7f366819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57182/0 (socket says 172.21.15.5:57182) 2024-09-15T22:08:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f36677fe640 1 -- 172.21.15.5:0/4168440222 learned_addr learned my addr 172.21.15.5:0/4168440222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:07.100 INFO:teuthology.orchestra.run.smithi005.stdout:55834574930 2024-09-15T22:08:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f3666ffd640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36680745a0 0x7f366819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f36667fc640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668074f50 0x7f366819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f36677fe640 1 -- 172.21.15.5:0/4168440222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668074f50 msgr2=0x7f366819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f36677fe640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668074f50 0x7f366819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f36677fe640 1 -- 172.21.15.5:0/4168440222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36680745a0 msgr2=0x7f366819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f36677fe640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36680745a0 0x7f366819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f36677fe640 1 -- 172.21.15.5:0/4168440222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366819f5e0 con 0x7f366810e470 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f3666ffd640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36680745a0 0x7f366819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f36677fe640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366810e470 0x7f366819ef00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f365800eaa0 tx=0x7f365800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f365ffff640 1 -- 172.21.15.5:0/4168440222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365800cdb0 con 0x7f366810e470 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f365ffff640 1 -- 172.21.15.5:0/4168440222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3658005020 con 0x7f366810e470 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f366d598640 1 -- 172.21.15.5:0/4168440222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366819f8d0 con 0x7f366810e470 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f365ffff640 1 -- 172.21.15.5:0/4168440222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36580116b0 con 0x7f366810e470 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.658+0000 7f366d598640 1 -- 172.21.15.5:0/4168440222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36681a71b0 con 0x7f366810e470 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.659+0000 7f366d598640 1 -- 172.21.15.5:0/4168440222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3668074980 con 0x7f366810e470 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.661+0000 7f365ffff640 1 -- 172.21.15.5:0/4168440222 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f3658005560 con 0x7f366810e470 2024-09-15T22:08:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.661+0000 7f365ffff640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36540a32d0 0x7f36540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.661+0000 7f3666ffd640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36540a32d0 0x7f36540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.661+0000 7f365ffff640 1 -- 172.21.15.5:0/4168440222 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f36580d05d0 con 0x7f366810e470 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.662+0000 7f3666ffd640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36540a32d0 0x7f36540a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3660009c20 tx=0x7f3660005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.664+0000 7f365ffff640 1 -- 172.21.15.5:0/4168440222 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3658016030 con 0x7f366810e470 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.849+0000 7f366d598640 1 -- 172.21.15.5:0/4168440222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f366819b990 con 0x7f366810e470 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.850+0000 7f365ffff640 1 -- 172.21.15.5:0/4168440222 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f366819b990 con 0x7f366810e470 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 -- 172.21.15.5:0/4168440222 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36540a32d0 msgr2=0x7f36540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36540a32d0 0x7f36540a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3660009c20 tx=0x7f3660005f90 comp rx=0 tx=0).stop 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 -- 172.21.15.5:0/4168440222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366810e470 msgr2=0x7f366819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366810e470 0x7f366819ef00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f365800eaa0 tx=0x7f365800ef60 comp rx=0 tx=0).stop 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 -- 172.21.15.5:0/4168440222 shutdown_connections 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36540a32d0 0x7f36540a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f366810e470 0x7f366819ef00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3668074f50 0x7f366819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 --2- 172.21.15.5:0/4168440222 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36680745a0 0x7f366819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 -- 172.21.15.5:0/4168440222 >> 172.21.15.5:0/4168440222 conn(0x7f36680fd8d0 msgr2=0x7f36680fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 -- 172.21.15.5:0/4168440222 shutdown_connections 2024-09-15T22:08:07.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:05.852+0000 7f366d598640 1 -- 172.21.15.5:0/4168440222 wait complete. 2024-09-15T22:08:07.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:06 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/9878831' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:08:07.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:06 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/9878831' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:08:07.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:06 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/9878831' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-15T22:08:07.781 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495665 got 197568495668 for osd.7 2024-09-15T22:08:07.781 DEBUG:teuthology.parallel:result is None 2024-09-15T22:08:08.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:07 smithi005 ceph-mon[30944]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:08.251 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574930 for osd.1 2024-09-15T22:08:08.251 DEBUG:teuthology.parallel:result is None 2024-09-15T22:08:08.251 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T22:08:08.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:08:08.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:07 smithi099 ceph-mon[34892]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:08.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:07 smithi129 ceph-mon[35314]: pgmap v330: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:08.518 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.007+0000 7ff0bfff0640 1 -- 172.21.15.5:0/3027487677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b81082a0 msgr2=0x7ff0b810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.007+0000 7ff0bfff0640 1 --2- 172.21.15.5:0/3027487677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b81082a0 0x7ff0b810ec40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009cd0 tx=0x7ff0b002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:08:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.007+0000 7ff0bfff0640 1 -- 172.21.15.5:0/3027487677 shutdown_connections 2024-09-15T22:08:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.007+0000 7ff0bfff0640 1 --2- 172.21.15.5:0/3027487677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b81082a0 0x7ff0b810ec40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.007+0000 7ff0bfff0640 1 --2- 172.21.15.5:0/3027487677 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0b81036e0 0x7ff0b8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.007+0000 7ff0bfff0640 1 --2- 172.21.15.5:0/3027487677 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b8102d30 0x7ff0b8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.007+0000 7ff0bfff0640 1 -- 172.21.15.5:0/3027487677 >> 172.21.15.5:0/3027487677 conn(0x7ff0b80fc7d0 msgr2=0x7ff0b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.007+0000 7ff0bfff0640 1 -- 172.21.15.5:0/3027487677 shutdown_connections 2024-09-15T22:08:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.007+0000 7ff0bfff0640 1 -- 172.21.15.5:0/3027487677 wait complete. 2024-09-15T22:08:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0bfff0640 1 Processor -- start 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0bfff0640 1 -- start start 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0bfff0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b8102d30 0x7ff0b8100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0bfff0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0b81036e0 0x7ff0b8100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0bfff0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b81082a0 0x7ff0b8101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0bfff0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b81120a0 con 0x7ff0b81082a0 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0bfff0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b8111f20 con 0x7ff0b8102d30 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0bfff0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b8112220 con 0x7ff0b81036e0 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0be58d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b81082a0 0x7ff0b8101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0be58d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b81082a0 0x7ff0b8101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57220/0 (socket says 172.21.15.5:57220) 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0be58d640 1 -- 172.21.15.5:0/2392803757 learned_addr learned my addr 172.21.15.5:0/2392803757 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0bd58b640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0b81036e0 0x7ff0b8100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0bdd8c640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b8102d30 0x7ff0b8100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0be58d640 1 -- 172.21.15.5:0/2392803757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0b81036e0 msgr2=0x7ff0b8100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0be58d640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0b81036e0 0x7ff0b8100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0be58d640 1 -- 172.21.15.5:0/2392803757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b8102d30 msgr2=0x7ff0b8100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0be58d640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b8102d30 0x7ff0b8100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0be58d640 1 -- 172.21.15.5:0/2392803757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b81a6a20 con 0x7ff0b81082a0 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0bd58b640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0b81036e0 0x7ff0b8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.008+0000 7ff0be58d640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b81082a0 0x7ff0b8101410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0005cc0 tx=0x7ff0b0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.009+0000 7ff0aeffd640 1 -- 172.21.15.5:0/2392803757 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0004040 con 0x7ff0b81082a0 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.009+0000 7ff0bfff0640 1 -- 172.21.15.5:0/2392803757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b81a6cb0 con 0x7ff0b81082a0 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.009+0000 7ff0aeffd640 1 -- 172.21.15.5:0/2392803757 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0b00046c0 con 0x7ff0b81082a0 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.009+0000 7ff0aeffd640 1 -- 172.21.15.5:0/2392803757 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0046480 con 0x7ff0b81082a0 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.009+0000 7ff0bfff0640 1 -- 172.21.15.5:0/2392803757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b81a7240 con 0x7ff0b81082a0 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.010+0000 7ff0bfff0640 1 -- 172.21.15.5:0/2392803757 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b8103110 con 0x7ff0b81082a0 2024-09-15T22:08:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.011+0000 7ff0aeffd640 1 -- 172.21.15.5:0/2392803757 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7ff0b00305a0 con 0x7ff0b81082a0 2024-09-15T22:08:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.012+0000 7ff0aeffd640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff0a40a32d0 0x7ff0a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.012+0000 7ff0aeffd640 1 -- 172.21.15.5:0/2392803757 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7ff0b0074080 con 0x7ff0b81082a0 2024-09-15T22:08:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.012+0000 7ff0bdd8c640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff0a40a32d0 0x7ff0a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.012+0000 7ff0bdd8c640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff0a40a32d0 0x7ff0a40a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40058c0 tx=0x7ff0b400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.015+0000 7ff0aeffd640 1 -- 172.21.15.5:0/2392803757 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0b004b060 con 0x7ff0b81082a0 2024-09-15T22:08:09.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.188+0000 7ff0bfff0640 1 -- 172.21.15.5:0/2392803757 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff0b8062630 con 0x7ff0a40a32d0 2024-09-15T22:08:09.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.197+0000 7ff0aeffd640 1 -- 172.21.15.5:0/2392803757 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340359 (secure 0 0 0) 0x7ff0b8062630 con 0x7ff0a40a32d0 2024-09-15T22:08:09.202 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:09.203 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 -- 172.21.15.5:0/2392803757 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff0a40a32d0 msgr2=0x7ff0a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff0a40a32d0 0x7ff0a40a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40058c0 tx=0x7ff0b400a4c0 comp rx=0 tx=0).stop 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 -- 172.21.15.5:0/2392803757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b81082a0 msgr2=0x7ff0b8101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b81082a0 0x7ff0b8101410 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0005cc0 tx=0x7ff0b0003970 comp rx=0 tx=0).stop 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 -- 172.21.15.5:0/2392803757 shutdown_connections 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7ff0a40a32d0 0x7ff0a40a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b81082a0 0x7ff0b8101410 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0b81036e0 0x7ff0b8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 --2- 172.21.15.5:0/2392803757 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0b8102d30 0x7ff0b8100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 -- 172.21.15.5:0/2392803757 >> 172.21.15.5:0/2392803757 conn(0x7ff0b80fc7d0 msgr2=0x7ff0b8105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 -- 172.21.15.5:0/2392803757 shutdown_connections 2024-09-15T22:08:09.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:09.200+0000 7ff0bfff0640 1 -- 172.21.15.5:0/2392803757 wait complete. 2024-09-15T22:08:10.029 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-15T22:08:08.809184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657074,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":479,"num_read_kb":360,"num_write":357,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":286,"ondisk_log_size":286,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388168,"kb_used_data":13892,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553656,"statfs":{"total":1151940427776,"available":1151542943744,"internally_reserved":0,"allocated":14225408,"data_stored":7974296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003010"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.226073+0000","last_change":"2024-09-15T22:07:46.226073+0000","last_active":"2024-09-15T22:07:46.226073+0000","last_peered":"2024-09-15T22:07:46.226073+0000","last_clean":"2024-09-15T22:07:46.226073+0000","last_became_active":"2024-09-15T22:07:46.219250+0000","last_became_peered":"2024-09-15T22:07:46.219250+0000","last_unstale":"2024-09-15T22:07:46.226073+0000","last_undegraded":"2024-09-15T22:07:46.226073+0000","last_fullsized":"2024-09-15T22:07:46.226073+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:56.324959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,4],"acting":[9,11,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811629+0000","last_change":"2024-09-15T22:06:37.184096+0000","last_active":"2024-09-15T22:07:45.811629+0000","last_peered":"2024-09-15T22:07:45.811629+0000","last_clean":"2024-09-15T22:07:45.811629+0000","last_became_active":"2024-09-15T22:06:37.183758+0000","last_became_peered":"2024-09-15T22:06:37.183758+0000","last_unstale":"2024-09-15T22:07:45.811629+0000","last_undegraded":"2024-09-15T22:07:45.811629+0000","last_fullsized":"2024-09-15T22:07:45.811629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:04.210551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630641+0000","last_change":"2024-09-15T22:06:44.653042+0000","last_active":"2024-09-15T22:07:45.630641+0000","last_peered":"2024-09-15T22:07:45.630641+0000","last_clean":"2024-09-15T22:07:45.630641+0000","last_became_active":"2024-09-15T22:06:44.652658+0000","last_became_peered":"2024-09-15T22:06:44.652658+0000","last_unstale":"2024-09-15T22:07:45.630641+0000","last_undegraded":"2024-09-15T22:07:45.630641+0000","last_fullsized":"2024-09-15T22:07:45.630641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:59.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811815+0000","last_change":"2024-09-15T22:06:35.355392+0000","last_active":"2024-09-15T22:07:45.811815+0000","last_peered":"2024-09-15T22:07:45.811815+0000","last_clean":"2024-09-15T22:07:45.811815+0000","last_became_active":"2024-09-15T22:06:35.355042+0000","last_became_peered":"2024-09-15T22:06:35.355042+0000","last_unstale":"2024-09-15T22:07:45.811815+0000","last_undegraded":"2024-09-15T22:07:45.811815+0000","last_fullsized":"2024-09-15T22:07:45.811815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:05.186937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217282+0000","last_change":"2024-09-15T22:06:37.184421+0000","last_active":"2024-09-15T22:07:46.217282+0000","last_peered":"2024-09-15T22:07:46.217282+0000","last_clean":"2024-09-15T22:07:46.217282+0000","last_became_active":"2024-09-15T22:06:37.184092+0000","last_became_peered":"2024-09-15T22:06:37.184092+0000","last_unstale":"2024-09-15T22:07:46.217282+0000","last_undegraded":"2024-09-15T22:07:46.217282+0000","last_fullsized":"2024-09-15T22:07:46.217282+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:41.080981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.968640+0000","last_change":"2024-09-15T22:07:45.968640+0000","last_active":"2024-09-15T22:07:45.968640+0000","last_peered":"2024-09-15T22:07:45.968640+0000","last_clean":"2024-09-15T22:07:45.968640+0000","last_became_active":"2024-09-15T22:07:45.968320+0000","last_became_peered":"2024-09-15T22:07:45.968320+0000","last_unstale":"2024-09-15T22:07:45.968640+0000","last_undegraded":"2024-09-15T22:07:45.968640+0000","last_fullsized":"2024-09-15T22:07:45.968640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:28.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.967279+0000","last_change":"2024-09-15T22:06:35.134606+0000","last_active":"2024-09-15T22:07:45.967279+0000","last_peered":"2024-09-15T22:07:45.967279+0000","last_clean":"2024-09-15T22:07:45.967279+0000","last_became_active":"2024-09-15T22:06:35.134263+0000","last_became_peered":"2024-09-15T22:06:35.134263+0000","last_unstale":"2024-09-15T22:07:45.967279+0000","last_undegraded":"2024-09-15T22:07:45.967279+0000","last_fullsized":"2024-09-15T22:07:45.967279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:45.008935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216998+0000","last_change":"2024-09-15T22:06:37.177556+0000","last_active":"2024-09-15T22:07:46.216998+0000","last_peered":"2024-09-15T22:07:46.216998+0000","last_clean":"2024-09-15T22:07:46.216998+0000","last_became_active":"2024-09-15T22:06:37.177161+0000","last_became_peered":"2024-09-15T22:06:37.177161+0000","last_unstale":"2024-09-15T22:07:46.216998+0000","last_undegraded":"2024-09-15T22:07:46.216998+0000","last_fullsized":"2024-09-15T22:07:46.216998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:30.192728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629350+0000","last_change":"2024-09-15T22:06:44.659097+0000","last_active":"2024-09-15T22:07:45.629350+0000","last_peered":"2024-09-15T22:07:45.629350+0000","last_clean":"2024-09-15T22:07:45.629350+0000","last_became_active":"2024-09-15T22:06:44.658626+0000","last_became_peered":"2024-09-15T22:06:44.658626+0000","last_unstale":"2024-09-15T22:07:45.629350+0000","last_undegraded":"2024-09-15T22:07:45.629350+0000","last_fullsized":"2024-09-15T22:07:45.629350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:01.308004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.967234+0000","last_change":"2024-09-15T22:06:35.136326+0000","last_active":"2024-09-15T22:07:45.967234+0000","last_peered":"2024-09-15T22:07:45.967234+0000","last_clean":"2024-09-15T22:07:45.967234+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:07:45.967234+0000","last_undegraded":"2024-09-15T22:07:45.967234+0000","last_fullsized":"2024-09-15T22:07:45.967234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:28.675832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632320+0000","last_change":"2024-09-15T22:06:37.192939+0000","last_active":"2024-09-15T22:07:45.632320+0000","last_peered":"2024-09-15T22:07:45.632320+0000","last_clean":"2024-09-15T22:07:45.632320+0000","last_became_active":"2024-09-15T22:06:37.192616+0000","last_became_peered":"2024-09-15T22:06:37.192616+0000","last_unstale":"2024-09-15T22:07:45.632320+0000","last_undegraded":"2024-09-15T22:07:45.632320+0000","last_fullsized":"2024-09-15T22:07:45.632320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:03.615203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629078+0000","last_change":"2024-09-15T22:06:44.654741+0000","last_active":"2024-09-15T22:07:45.629078+0000","last_peered":"2024-09-15T22:07:45.629078+0000","last_clean":"2024-09-15T22:07:45.629078+0000","last_became_active":"2024-09-15T22:06:44.654362+0000","last_became_peered":"2024-09-15T22:06:44.654362+0000","last_unstale":"2024-09-15T22:07:45.629078+0000","last_undegraded":"2024-09-15T22:07:45.629078+0000","last_fullsized":"2024-09-15T22:07:45.629078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:59.269948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631095+0000","last_change":"2024-09-15T22:06:37.186385+0000","last_active":"2024-09-15T22:07:45.631095+0000","last_peered":"2024-09-15T22:07:45.631095+0000","last_clean":"2024-09-15T22:07:45.631095+0000","last_became_active":"2024-09-15T22:06:37.185588+0000","last_became_peered":"2024-09-15T22:06:37.185588+0000","last_unstale":"2024-09-15T22:07:45.631095+0000","last_undegraded":"2024-09-15T22:07:45.631095+0000","last_fullsized":"2024-09-15T22:07:45.631095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:22.523083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966841+0000","last_change":"2024-09-15T22:06:35.137383+0000","last_active":"2024-09-15T22:07:45.966841+0000","last_peered":"2024-09-15T22:07:45.966841+0000","last_clean":"2024-09-15T22:07:45.966841+0000","last_became_active":"2024-09-15T22:06:35.137022+0000","last_became_peered":"2024-09-15T22:06:35.137022+0000","last_unstale":"2024-09-15T22:07:45.966841+0000","last_undegraded":"2024-09-15T22:07:45.966841+0000","last_fullsized":"2024-09-15T22:07:45.966841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.863155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629442+0000","last_change":"2024-09-15T22:06:44.653807+0000","last_active":"2024-09-15T22:07:45.629442+0000","last_peered":"2024-09-15T22:07:45.629442+0000","last_clean":"2024-09-15T22:07:45.629442+0000","last_became_active":"2024-09-15T22:06:44.653406+0000","last_became_peered":"2024-09-15T22:06:44.653406+0000","last_unstale":"2024-09-15T22:07:45.629442+0000","last_undegraded":"2024-09-15T22:07:45.629442+0000","last_fullsized":"2024-09-15T22:07:45.629442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:12.018208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811325+0000","last_change":"2024-09-15T22:06:37.186899+0000","last_active":"2024-09-15T22:07:45.811325+0000","last_peered":"2024-09-15T22:07:45.811325+0000","last_clean":"2024-09-15T22:07:45.811325+0000","last_became_active":"2024-09-15T22:06:37.183782+0000","last_became_peered":"2024-09-15T22:06:37.183782+0000","last_unstale":"2024-09-15T22:07:45.811325+0000","last_undegraded":"2024-09-15T22:07:45.811325+0000","last_fullsized":"2024-09-15T22:07:45.811325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:39.269186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217605+0000","last_change":"2024-09-15T22:06:44.652686+0000","last_active":"2024-09-15T22:07:46.217605+0000","last_peered":"2024-09-15T22:07:46.217605+0000","last_clean":"2024-09-15T22:07:46.217605+0000","last_became_active":"2024-09-15T22:06:44.652306+0000","last_became_peered":"2024-09-15T22:06:44.652306+0000","last_unstale":"2024-09-15T22:07:46.217605+0000","last_undegraded":"2024-09-15T22:07:46.217605+0000","last_fullsized":"2024-09-15T22:07:46.217605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629036+0000","last_change":"2024-09-15T22:06:44.655424+0000","last_active":"2024-09-15T22:07:45.629036+0000","last_peered":"2024-09-15T22:07:45.629036+0000","last_clean":"2024-09-15T22:07:45.629036+0000","last_became_active":"2024-09-15T22:06:44.654959+0000","last_became_peered":"2024-09-15T22:06:44.654959+0000","last_unstale":"2024-09-15T22:07:45.629036+0000","last_undegraded":"2024-09-15T22:07:45.629036+0000","last_fullsized":"2024-09-15T22:07:45.629036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:07.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217592+0000","last_change":"2024-09-15T22:06:37.188424+0000","last_active":"2024-09-15T22:07:46.217592+0000","last_peered":"2024-09-15T22:07:46.217592+0000","last_clean":"2024-09-15T22:07:46.217592+0000","last_became_active":"2024-09-15T22:06:37.188082+0000","last_became_peered":"2024-09-15T22:06:37.188082+0000","last_unstale":"2024-09-15T22:07:46.217592+0000","last_undegraded":"2024-09-15T22:07:46.217592+0000","last_fullsized":"2024-09-15T22:07:46.217592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:07.159341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811407+0000","last_change":"2024-09-15T22:06:35.134237+0000","last_active":"2024-09-15T22:07:45.811407+0000","last_peered":"2024-09-15T22:07:45.811407+0000","last_clean":"2024-09-15T22:07:45.811407+0000","last_became_active":"2024-09-15T22:06:35.133885+0000","last_became_peered":"2024-09-15T22:06:35.133885+0000","last_unstale":"2024-09-15T22:07:45.811407+0000","last_undegraded":"2024-09-15T22:07:45.811407+0000","last_fullsized":"2024-09-15T22:07:45.811407+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:07:35.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627325+0000","last_change":"2024-09-15T22:06:44.664578+0000","last_active":"2024-09-15T22:07:45.627325+0000","last_peered":"2024-09-15T22:07:45.627325+0000","last_clean":"2024-09-15T22:07:45.627325+0000","last_became_active":"2024-09-15T22:06:44.664170+0000","last_became_peered":"2024-09-15T22:06:44.664170+0000","last_unstale":"2024-09-15T22:07:45.627325+0000","last_undegraded":"2024-09-15T22:07:45.627325+0000","last_fullsized":"2024-09-15T22:07:45.627325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:41.443816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966606+0000","last_change":"2024-09-15T22:06:37.186107+0000","last_active":"2024-09-15T22:07:45.966606+0000","last_peered":"2024-09-15T22:07:45.966606+0000","last_clean":"2024-09-15T22:07:45.966606+0000","last_became_active":"2024-09-15T22:06:37.185757+0000","last_became_peered":"2024-09-15T22:06:37.185757+0000","last_unstale":"2024-09-15T22:07:45.966606+0000","last_undegraded":"2024-09-15T22:07:45.966606+0000","last_fullsized":"2024-09-15T22:07:45.966606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:14.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216800+0000","last_change":"2024-09-15T22:06:35.356281+0000","last_active":"2024-09-15T22:07:46.216800+0000","last_peered":"2024-09-15T22:07:46.216800+0000","last_clean":"2024-09-15T22:07:46.216800+0000","last_became_active":"2024-09-15T22:06:35.355922+0000","last_became_peered":"2024-09-15T22:06:35.355922+0000","last_unstale":"2024-09-15T22:07:46.216800+0000","last_undegraded":"2024-09-15T22:07:46.216800+0000","last_fullsized":"2024-09-15T22:07:46.216800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:06.107462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629586+0000","last_change":"2024-09-15T22:06:44.659230+0000","last_active":"2024-09-15T22:07:45.629586+0000","last_peered":"2024-09-15T22:07:45.629586+0000","last_clean":"2024-09-15T22:07:45.629586+0000","last_became_active":"2024-09-15T22:06:44.658649+0000","last_became_peered":"2024-09-15T22:06:44.658649+0000","last_unstale":"2024-09-15T22:07:45.629586+0000","last_undegraded":"2024-09-15T22:07:45.629586+0000","last_fullsized":"2024-09-15T22:07:45.629586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:14:21.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216942+0000","last_change":"2024-09-15T22:06:44.649394+0000","last_active":"2024-09-15T22:07:46.216942+0000","last_peered":"2024-09-15T22:07:46.216942+0000","last_clean":"2024-09-15T22:07:46.216942+0000","last_became_active":"2024-09-15T22:06:44.649043+0000","last_became_peered":"2024-09-15T22:06:44.649043+0000","last_unstale":"2024-09-15T22:07:46.216942+0000","last_undegraded":"2024-09-15T22:07:46.216942+0000","last_fullsized":"2024-09-15T22:07:46.216942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:56:28.335869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629894+0000","last_change":"2024-09-15T22:06:35.265970+0000","last_active":"2024-09-15T22:07:45.629894+0000","last_peered":"2024-09-15T22:07:45.629894+0000","last_clean":"2024-09-15T22:07:45.629894+0000","last_became_active":"2024-09-15T22:06:35.265628+0000","last_became_peered":"2024-09-15T22:06:35.265628+0000","last_unstale":"2024-09-15T22:07:45.629894+0000","last_undegraded":"2024-09-15T22:07:45.629894+0000","last_fullsized":"2024-09-15T22:07:45.629894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:41.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630784+0000","last_change":"2024-09-15T22:06:44.649652+0000","last_active":"2024-09-15T22:07:45.630784+0000","last_peered":"2024-09-15T22:07:45.630784+0000","last_clean":"2024-09-15T22:07:45.630784+0000","last_became_active":"2024-09-15T22:06:44.649112+0000","last_became_peered":"2024-09-15T22:06:44.649112+0000","last_unstale":"2024-09-15T22:07:45.630784+0000","last_undegraded":"2024-09-15T22:07:45.630784+0000","last_fullsized":"2024-09-15T22:07:45.630784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:53.789320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811736+0000","last_change":"2024-09-15T22:06:37.190701+0000","last_active":"2024-09-15T22:07:45.811736+0000","last_peered":"2024-09-15T22:07:45.811736+0000","last_clean":"2024-09-15T22:07:45.811736+0000","last_became_active":"2024-09-15T22:06:37.190430+0000","last_became_peered":"2024-09-15T22:06:37.190430+0000","last_unstale":"2024-09-15T22:07:45.811736+0000","last_undegraded":"2024-09-15T22:07:45.811736+0000","last_fullsized":"2024-09-15T22:07:45.811736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:14.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'14","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627467+0000","last_change":"2024-09-15T22:06:35.133899+0000","last_active":"2024-09-15T22:07:45.627467+0000","last_peered":"2024-09-15T22:07:45.627467+0000","last_clean":"2024-09-15T22:07:45.627467+0000","last_became_active":"2024-09-15T22:06:35.133556+0000","last_became_peered":"2024-09-15T22:06:35.133556+0000","last_unstale":"2024-09-15T22:07:45.627467+0000","last_undegraded":"2024-09-15T22:07:45.627467+0000","last_fullsized":"2024-09-15T22:07:45.627467+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:01.692416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811382+0000","last_change":"2024-09-15T22:06:44.655280+0000","last_active":"2024-09-15T22:07:45.811382+0000","last_peered":"2024-09-15T22:07:45.811382+0000","last_clean":"2024-09-15T22:07:45.811382+0000","last_became_active":"2024-09-15T22:06:44.654883+0000","last_became_peered":"2024-09-15T22:06:44.654883+0000","last_unstale":"2024-09-15T22:07:45.811382+0000","last_undegraded":"2024-09-15T22:07:45.811382+0000","last_fullsized":"2024-09-15T22:07:45.811382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.074035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786557+0000","last_change":"2024-09-15T22:06:37.182006+0000","last_active":"2024-09-15T22:07:45.786557+0000","last_peered":"2024-09-15T22:07:45.786557+0000","last_clean":"2024-09-15T22:07:45.786557+0000","last_became_active":"2024-09-15T22:06:37.181693+0000","last_became_peered":"2024-09-15T22:06:37.181693+0000","last_unstale":"2024-09-15T22:07:45.786557+0000","last_undegraded":"2024-09-15T22:07:45.786557+0000","last_fullsized":"2024-09-15T22:07:45.786557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:23.483930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.967289+0000","last_change":"2024-09-15T22:06:35.355854+0000","last_active":"2024-09-15T22:07:45.967289+0000","last_peered":"2024-09-15T22:07:45.967289+0000","last_clean":"2024-09-15T22:07:45.967289+0000","last_became_active":"2024-09-15T22:06:35.355486+0000","last_became_peered":"2024-09-15T22:06:35.355486+0000","last_unstale":"2024-09-15T22:07:45.967289+0000","last_undegraded":"2024-09-15T22:07:45.967289+0000","last_fullsized":"2024-09-15T22:07:45.967289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:36.708476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217530+0000","last_change":"2024-09-15T22:06:44.660523+0000","last_active":"2024-09-15T22:07:46.217530+0000","last_peered":"2024-09-15T22:07:46.217530+0000","last_clean":"2024-09-15T22:07:46.217530+0000","last_became_active":"2024-09-15T22:06:44.660113+0000","last_became_peered":"2024-09-15T22:06:44.660113+0000","last_unstale":"2024-09-15T22:07:46.217530+0000","last_undegraded":"2024-09-15T22:07:46.217530+0000","last_fullsized":"2024-09-15T22:07:46.217530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:42.164558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627691+0000","last_change":"2024-09-15T22:06:37.183649+0000","last_active":"2024-09-15T22:07:45.627691+0000","last_peered":"2024-09-15T22:07:45.627691+0000","last_clean":"2024-09-15T22:07:45.627691+0000","last_became_active":"2024-09-15T22:06:37.183296+0000","last_became_peered":"2024-09-15T22:06:37.183296+0000","last_unstale":"2024-09-15T22:07:45.627691+0000","last_undegraded":"2024-09-15T22:07:45.627691+0000","last_fullsized":"2024-09-15T22:07:45.627691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:22.010288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627247+0000","last_change":"2024-09-15T22:06:35.355068+0000","last_active":"2024-09-15T22:07:45.627247+0000","last_peered":"2024-09-15T22:07:45.627247+0000","last_clean":"2024-09-15T22:07:45.627247+0000","last_became_active":"2024-09-15T22:06:35.354728+0000","last_became_peered":"2024-09-15T22:06:35.354728+0000","last_unstale":"2024-09-15T22:07:45.627247+0000","last_undegraded":"2024-09-15T22:07:45.627247+0000","last_fullsized":"2024-09-15T22:07:45.627247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:35.479290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966634+0000","last_change":"2024-09-15T22:06:44.653849+0000","last_active":"2024-09-15T22:07:45.966634+0000","last_peered":"2024-09-15T22:07:45.966634+0000","last_clean":"2024-09-15T22:07:45.966634+0000","last_became_active":"2024-09-15T22:06:44.653419+0000","last_became_peered":"2024-09-15T22:06:44.653419+0000","last_unstale":"2024-09-15T22:07:45.966634+0000","last_undegraded":"2024-09-15T22:07:45.966634+0000","last_fullsized":"2024-09-15T22:07:45.966634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:04.131553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627824+0000","last_change":"2024-09-15T22:06:37.191438+0000","last_active":"2024-09-15T22:07:45.627824+0000","last_peered":"2024-09-15T22:07:45.627824+0000","last_clean":"2024-09-15T22:07:45.627824+0000","last_became_active":"2024-09-15T22:06:37.191088+0000","last_became_peered":"2024-09-15T22:06:37.191088+0000","last_unstale":"2024-09-15T22:07:45.627824+0000","last_undegraded":"2024-09-15T22:07:45.627824+0000","last_fullsized":"2024-09-15T22:07:45.627824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:46:36.078872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631155+0000","last_change":"2024-09-15T22:06:35.135866+0000","last_active":"2024-09-15T22:07:45.631155+0000","last_peered":"2024-09-15T22:07:45.631155+0000","last_clean":"2024-09-15T22:07:45.631155+0000","last_became_active":"2024-09-15T22:06:35.135524+0000","last_became_peered":"2024-09-15T22:06:35.135524+0000","last_unstale":"2024-09-15T22:07:45.631155+0000","last_undegraded":"2024-09-15T22:07:45.631155+0000","last_fullsized":"2024-09-15T22:07:45.631155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786408+0000","last_change":"2024-09-15T22:06:44.649943+0000","last_active":"2024-09-15T22:07:45.786408+0000","last_peered":"2024-09-15T22:07:45.786408+0000","last_clean":"2024-09-15T22:07:45.786408+0000","last_became_active":"2024-09-15T22:06:44.649579+0000","last_became_peered":"2024-09-15T22:06:44.649579+0000","last_unstale":"2024-09-15T22:07:45.786408+0000","last_undegraded":"2024-09-15T22:07:45.786408+0000","last_fullsized":"2024-09-15T22:07:45.786408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:18.390608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631303+0000","last_change":"2024-09-15T22:06:37.189988+0000","last_active":"2024-09-15T22:07:45.631303+0000","last_peered":"2024-09-15T22:07:45.631303+0000","last_clean":"2024-09-15T22:07:45.631303+0000","last_became_active":"2024-09-15T22:06:37.189319+0000","last_became_peered":"2024-09-15T22:06:37.189319+0000","last_unstale":"2024-09-15T22:07:45.631303+0000","last_undegraded":"2024-09-15T22:07:45.631303+0000","last_fullsized":"2024-09-15T22:07:45.631303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:08.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:08:06.818441+0000","last_change":"2024-09-15T22:06:35.355371+0000","last_active":"2024-09-15T22:08:06.818441+0000","last_peered":"2024-09-15T22:08:06.818441+0000","last_clean":"2024-09-15T22:08:06.818441+0000","last_became_active":"2024-09-15T22:06:35.354687+0000","last_became_peered":"2024-09-15T22:06:35.354687+0000","last_unstale":"2024-09-15T22:08:06.818441+0000","last_undegraded":"2024-09-15T22:08:06.818441+0000","last_fullsized":"2024-09-15T22:08:06.818441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:27.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786580+0000","last_change":"2024-09-15T22:06:44.657929+0000","last_active":"2024-09-15T22:07:45.786580+0000","last_peered":"2024-09-15T22:07:45.786580+0000","last_clean":"2024-09-15T22:07:45.786580+0000","last_became_active":"2024-09-15T22:06:44.657519+0000","last_became_peered":"2024-09-15T22:06:44.657519+0000","last_unstale":"2024-09-15T22:07:45.786580+0000","last_undegraded":"2024-09-15T22:07:45.786580+0000","last_fullsized":"2024-09-15T22:07:45.786580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:12.313877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.785906+0000","last_change":"2024-09-15T22:06:37.193288+0000","last_active":"2024-09-15T22:07:45.785906+0000","last_peered":"2024-09-15T22:07:45.785906+0000","last_clean":"2024-09-15T22:07:45.785906+0000","last_became_active":"2024-09-15T22:06:37.193007+0000","last_became_peered":"2024-09-15T22:06:37.193007+0000","last_unstale":"2024-09-15T22:07:45.785906+0000","last_undegraded":"2024-09-15T22:07:45.785906+0000","last_fullsized":"2024-09-15T22:07:45.785906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:45.956206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630622+0000","last_change":"2024-09-15T22:06:35.138726+0000","last_active":"2024-09-15T22:07:45.630622+0000","last_peered":"2024-09-15T22:07:45.630622+0000","last_clean":"2024-09-15T22:07:45.630622+0000","last_became_active":"2024-09-15T22:06:35.138390+0000","last_became_peered":"2024-09-15T22:06:35.138390+0000","last_unstale":"2024-09-15T22:07:45.630622+0000","last_undegraded":"2024-09-15T22:07:45.630622+0000","last_fullsized":"2024-09-15T22:07:45.630622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:51.336171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.219720+0000","last_change":"2024-09-15T22:07:46.219720+0000","last_active":"2024-09-15T22:07:46.219720+0000","last_peered":"2024-09-15T22:07:46.219720+0000","last_clean":"2024-09-15T22:07:46.219720+0000","last_became_active":"2024-09-15T22:07:46.219303+0000","last_became_peered":"2024-09-15T22:07:46.219303+0000","last_unstale":"2024-09-15T22:07:46.219720+0000","last_undegraded":"2024-09-15T22:07:46.219720+0000","last_fullsized":"2024-09-15T22:07:46.219720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:03.596795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811307+0000","last_change":"2024-09-15T22:06:37.176478+0000","last_active":"2024-09-15T22:07:45.811307+0000","last_peered":"2024-09-15T22:07:45.811307+0000","last_clean":"2024-09-15T22:07:45.811307+0000","last_became_active":"2024-09-15T22:06:37.176171+0000","last_became_peered":"2024-09-15T22:06:37.176171+0000","last_unstale":"2024-09-15T22:07:45.811307+0000","last_undegraded":"2024-09-15T22:07:45.811307+0000","last_fullsized":"2024-09-15T22:07:45.811307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629800+0000","last_change":"2024-09-15T22:06:35.134651+0000","last_active":"2024-09-15T22:07:45.629800+0000","last_peered":"2024-09-15T22:07:45.629800+0000","last_clean":"2024-09-15T22:07:45.629800+0000","last_became_active":"2024-09-15T22:06:35.134185+0000","last_became_peered":"2024-09-15T22:06:35.134185+0000","last_unstale":"2024-09-15T22:07:45.629800+0000","last_undegraded":"2024-09-15T22:07:45.629800+0000","last_fullsized":"2024-09-15T22:07:45.629800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:56.385046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630799+0000","last_change":"2024-09-15T22:05:23.048068+0000","last_active":"2024-09-15T22:07:45.630799+0000","last_peered":"2024-09-15T22:07:45.630799+0000","last_clean":"2024-09-15T22:07:45.630799+0000","last_became_active":"2024-09-15T22:05:23.047692+0000","last_became_peered":"2024-09-15T22:05:23.047692+0000","last_unstale":"2024-09-15T22:07:45.630799+0000","last_undegraded":"2024-09-15T22:07:45.630799+0000","last_fullsized":"2024-09-15T22:07:45.630799+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_clean_scrub_stamp":"2024-09-15T22:01:57.477629+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631277+0000","last_change":"2024-09-15T22:06:44.657213+0000","last_active":"2024-09-15T22:07:45.631277+0000","last_peered":"2024-09-15T22:07:45.631277+0000","last_clean":"2024-09-15T22:07:45.631277+0000","last_became_active":"2024-09-15T22:06:44.656815+0000","last_became_peered":"2024-09-15T22:06:44.656815+0000","last_unstale":"2024-09-15T22:07:45.631277+0000","last_undegraded":"2024-09-15T22:07:45.631277+0000","last_fullsized":"2024-09-15T22:07:45.631277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:26.525992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631397+0000","last_change":"2024-09-15T22:06:37.184467+0000","last_active":"2024-09-15T22:07:45.631397+0000","last_peered":"2024-09-15T22:07:45.631397+0000","last_clean":"2024-09-15T22:07:45.631397+0000","last_became_active":"2024-09-15T22:06:37.183864+0000","last_became_peered":"2024-09-15T22:06:37.183864+0000","last_unstale":"2024-09-15T22:07:45.631397+0000","last_undegraded":"2024-09-15T22:07:45.631397+0000","last_fullsized":"2024-09-15T22:07:45.631397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:35.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216954+0000","last_change":"2024-09-15T22:06:44.651594+0000","last_active":"2024-09-15T22:07:46.216954+0000","last_peered":"2024-09-15T22:07:46.216954+0000","last_clean":"2024-09-15T22:07:46.216954+0000","last_became_active":"2024-09-15T22:06:44.650393+0000","last_became_peered":"2024-09-15T22:06:44.650393+0000","last_unstale":"2024-09-15T22:07:46.216954+0000","last_undegraded":"2024-09-15T22:07:46.216954+0000","last_fullsized":"2024-09-15T22:07:46.216954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:13.599744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216738+0000","last_change":"2024-09-15T22:06:44.660789+0000","last_active":"2024-09-15T22:07:46.216738+0000","last_peered":"2024-09-15T22:07:46.216738+0000","last_clean":"2024-09-15T22:07:46.216738+0000","last_became_active":"2024-09-15T22:06:44.660331+0000","last_became_peered":"2024-09-15T22:06:44.660331+0000","last_unstale":"2024-09-15T22:07:46.216738+0000","last_undegraded":"2024-09-15T22:07:46.216738+0000","last_fullsized":"2024-09-15T22:07:46.216738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:04:45.562795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631098+0000","last_change":"2024-09-15T22:06:37.185697+0000","last_active":"2024-09-15T22:07:45.631098+0000","last_peered":"2024-09-15T22:07:45.631098+0000","last_clean":"2024-09-15T22:07:45.631098+0000","last_became_active":"2024-09-15T22:06:37.185207+0000","last_became_peered":"2024-09-15T22:06:37.185207+0000","last_unstale":"2024-09-15T22:07:45.631098+0000","last_undegraded":"2024-09-15T22:07:45.631098+0000","last_fullsized":"2024-09-15T22:07:45.631098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:39.005273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632692+0000","last_change":"2024-09-15T22:06:35.135176+0000","last_active":"2024-09-15T22:07:45.632692+0000","last_peered":"2024-09-15T22:07:45.632692+0000","last_clean":"2024-09-15T22:07:45.632692+0000","last_became_active":"2024-09-15T22:06:35.134649+0000","last_became_peered":"2024-09-15T22:06:35.134649+0000","last_unstale":"2024-09-15T22:07:45.632692+0000","last_undegraded":"2024-09-15T22:07:45.632692+0000","last_fullsized":"2024-09-15T22:07:45.632692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:33.780162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216836+0000","last_change":"2024-09-15T22:06:44.656972+0000","last_active":"2024-09-15T22:07:46.216836+0000","last_peered":"2024-09-15T22:07:46.216836+0000","last_clean":"2024-09-15T22:07:46.216836+0000","last_became_active":"2024-09-15T22:06:44.656528+0000","last_became_peered":"2024-09-15T22:06:44.656528+0000","last_unstale":"2024-09-15T22:07:46.216836+0000","last_undegraded":"2024-09-15T22:07:46.216836+0000","last_fullsized":"2024-09-15T22:07:46.216836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:09.164596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630362+0000","last_change":"2024-09-15T22:06:37.186118+0000","last_active":"2024-09-15T22:07:45.630362+0000","last_peered":"2024-09-15T22:07:45.630362+0000","last_clean":"2024-09-15T22:07:45.630362+0000","last_became_active":"2024-09-15T22:06:37.185118+0000","last_became_peered":"2024-09-15T22:06:37.185118+0000","last_unstale":"2024-09-15T22:07:45.630362+0000","last_undegraded":"2024-09-15T22:07:45.630362+0000","last_fullsized":"2024-09-15T22:07:45.630362+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:53.357045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786815+0000","last_change":"2024-09-15T22:06:35.133341+0000","last_active":"2024-09-15T22:07:45.786815+0000","last_peered":"2024-09-15T22:07:45.786815+0000","last_clean":"2024-09-15T22:07:45.786815+0000","last_became_active":"2024-09-15T22:06:35.132961+0000","last_became_peered":"2024-09-15T22:06:35.132961+0000","last_unstale":"2024-09-15T22:07:45.786815+0000","last_undegraded":"2024-09-15T22:07:45.786815+0000","last_fullsized":"2024-09-15T22:07:45.786815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:39.579470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630891+0000","last_change":"2024-09-15T22:06:44.659148+0000","last_active":"2024-09-15T22:07:45.630891+0000","last_peered":"2024-09-15T22:07:45.630891+0000","last_clean":"2024-09-15T22:07:45.630891+0000","last_became_active":"2024-09-15T22:06:44.658750+0000","last_became_peered":"2024-09-15T22:06:44.658750+0000","last_unstale":"2024-09-15T22:07:45.630891+0000","last_undegraded":"2024-09-15T22:07:45.630891+0000","last_fullsized":"2024-09-15T22:07:45.630891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:48.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.218042+0000","last_change":"2024-09-15T22:06:37.194066+0000","last_active":"2024-09-15T22:07:46.218042+0000","last_peered":"2024-09-15T22:07:46.218042+0000","last_clean":"2024-09-15T22:07:46.218042+0000","last_became_active":"2024-09-15T22:06:37.193738+0000","last_became_peered":"2024-09-15T22:06:37.193738+0000","last_unstale":"2024-09-15T22:07:46.218042+0000","last_undegraded":"2024-09-15T22:07:46.218042+0000","last_fullsized":"2024-09-15T22:07:46.218042+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.107188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.967402+0000","last_change":"2024-09-15T22:06:35.136470+0000","last_active":"2024-09-15T22:07:45.967402+0000","last_peered":"2024-09-15T22:07:45.967402+0000","last_clean":"2024-09-15T22:07:45.967402+0000","last_became_active":"2024-09-15T22:06:35.135953+0000","last_became_peered":"2024-09-15T22:06:35.135953+0000","last_unstale":"2024-09-15T22:07:45.967402+0000","last_undegraded":"2024-09-15T22:07:45.967402+0000","last_fullsized":"2024-09-15T22:07:45.967402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:44.548566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631818+0000","last_change":"2024-09-15T22:06:44.653682+0000","last_active":"2024-09-15T22:07:45.631818+0000","last_peered":"2024-09-15T22:07:45.631818+0000","last_clean":"2024-09-15T22:07:45.631818+0000","last_became_active":"2024-09-15T22:06:44.653303+0000","last_became_peered":"2024-09-15T22:06:44.653303+0000","last_unstale":"2024-09-15T22:07:45.631818+0000","last_undegraded":"2024-09-15T22:07:45.631818+0000","last_fullsized":"2024-09-15T22:07:45.631818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:07.847971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631929+0000","last_change":"2024-09-15T22:06:44.658436+0000","last_active":"2024-09-15T22:07:45.631929+0000","last_peered":"2024-09-15T22:07:45.631929+0000","last_clean":"2024-09-15T22:07:45.631929+0000","last_became_active":"2024-09-15T22:06:44.658037+0000","last_became_peered":"2024-09-15T22:06:44.658037+0000","last_unstale":"2024-09-15T22:07:45.631929+0000","last_undegraded":"2024-09-15T22:07:45.631929+0000","last_fullsized":"2024-09-15T22:07:45.631929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:34:20.369249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966897+0000","last_change":"2024-09-15T22:06:35.358018+0000","last_active":"2024-09-15T22:07:45.966897+0000","last_peered":"2024-09-15T22:07:45.966897+0000","last_clean":"2024-09-15T22:07:45.966897+0000","last_became_active":"2024-09-15T22:06:35.357666+0000","last_became_peered":"2024-09-15T22:06:35.357666+0000","last_unstale":"2024-09-15T22:07:45.966897+0000","last_undegraded":"2024-09-15T22:07:45.966897+0000","last_fullsized":"2024-09-15T22:07:45.966897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:34.089113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786697+0000","last_change":"2024-09-15T22:06:44.656807+0000","last_active":"2024-09-15T22:07:45.786697+0000","last_peered":"2024-09-15T22:07:45.786697+0000","last_clean":"2024-09-15T22:07:45.786697+0000","last_became_active":"2024-09-15T22:06:44.656409+0000","last_became_peered":"2024-09-15T22:06:44.656409+0000","last_unstale":"2024-09-15T22:07:45.786697+0000","last_undegraded":"2024-09-15T22:07:45.786697+0000","last_fullsized":"2024-09-15T22:07:45.786697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630373+0000","last_change":"2024-09-15T22:06:37.180615+0000","last_active":"2024-09-15T22:07:45.630373+0000","last_peered":"2024-09-15T22:07:45.630373+0000","last_clean":"2024-09-15T22:07:45.630373+0000","last_became_active":"2024-09-15T22:06:37.180306+0000","last_became_peered":"2024-09-15T22:06:37.180306+0000","last_unstale":"2024-09-15T22:07:45.630373+0000","last_undegraded":"2024-09-15T22:07:45.630373+0000","last_fullsized":"2024-09-15T22:07:45.630373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:31.145128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:08:06.819117+0000","last_change":"2024-09-15T22:06:35.136192+0000","last_active":"2024-09-15T22:08:06.819117+0000","last_peered":"2024-09-15T22:08:06.819117+0000","last_clean":"2024-09-15T22:08:06.819117+0000","last_became_active":"2024-09-15T22:06:35.135608+0000","last_became_peered":"2024-09-15T22:06:35.135608+0000","last_unstale":"2024-09-15T22:08:06.819117+0000","last_undegraded":"2024-09-15T22:08:06.819117+0000","last_fullsized":"2024-09-15T22:08:06.819117+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:48.472717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627405+0000","last_change":"2024-09-15T22:06:44.664763+0000","last_active":"2024-09-15T22:07:45.627405+0000","last_peered":"2024-09-15T22:07:45.627405+0000","last_clean":"2024-09-15T22:07:45.627405+0000","last_became_active":"2024-09-15T22:06:44.664410+0000","last_became_peered":"2024-09-15T22:06:44.664410+0000","last_unstale":"2024-09-15T22:07:45.627405+0000","last_undegraded":"2024-09-15T22:07:45.627405+0000","last_fullsized":"2024-09-15T22:07:45.627405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:14.364387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631277+0000","last_change":"2024-09-15T22:06:44.652747+0000","last_active":"2024-09-15T22:07:45.631277+0000","last_peered":"2024-09-15T22:07:45.631277+0000","last_clean":"2024-09-15T22:07:45.631277+0000","last_became_active":"2024-09-15T22:06:44.652368+0000","last_became_peered":"2024-09-15T22:06:44.652368+0000","last_unstale":"2024-09-15T22:07:45.631277+0000","last_undegraded":"2024-09-15T22:07:45.631277+0000","last_fullsized":"2024-09-15T22:07:45.631277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:51.047549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632221+0000","last_change":"2024-09-15T22:06:35.136311+0000","last_active":"2024-09-15T22:07:45.632221+0000","last_peered":"2024-09-15T22:07:45.632221+0000","last_clean":"2024-09-15T22:07:45.632221+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:07:45.632221+0000","last_undegraded":"2024-09-15T22:07:45.632221+0000","last_fullsized":"2024-09-15T22:07:45.632221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:36:07.138056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217545+0000","last_change":"2024-09-15T22:06:44.653747+0000","last_active":"2024-09-15T22:07:46.217545+0000","last_peered":"2024-09-15T22:07:46.217545+0000","last_clean":"2024-09-15T22:07:46.217545+0000","last_became_active":"2024-09-15T22:06:44.653357+0000","last_became_peered":"2024-09-15T22:06:44.653357+0000","last_unstale":"2024-09-15T22:07:46.217545+0000","last_undegraded":"2024-09-15T22:07:46.217545+0000","last_fullsized":"2024-09-15T22:07:46.217545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.113924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786191+0000","last_change":"2024-09-15T22:06:37.178308+0000","last_active":"2024-09-15T22:07:45.786191+0000","last_peered":"2024-09-15T22:07:45.786191+0000","last_clean":"2024-09-15T22:07:45.786191+0000","last_became_active":"2024-09-15T22:06:37.177920+0000","last_became_peered":"2024-09-15T22:06:37.177920+0000","last_unstale":"2024-09-15T22:07:45.786191+0000","last_undegraded":"2024-09-15T22:07:45.786191+0000","last_fullsized":"2024-09-15T22:07:45.786191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631690+0000","last_change":"2024-09-15T22:06:35.136132+0000","last_active":"2024-09-15T22:07:45.631690+0000","last_peered":"2024-09-15T22:07:45.631690+0000","last_clean":"2024-09-15T22:07:45.631690+0000","last_became_active":"2024-09-15T22:06:35.135789+0000","last_became_peered":"2024-09-15T22:06:35.135789+0000","last_unstale":"2024-09-15T22:07:45.631690+0000","last_undegraded":"2024-09-15T22:07:45.631690+0000","last_fullsized":"2024-09-15T22:07:45.631690+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:01.692885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632214+0000","last_change":"2024-09-15T22:06:44.658920+0000","last_active":"2024-09-15T22:07:45.632214+0000","last_peered":"2024-09-15T22:07:45.632214+0000","last_clean":"2024-09-15T22:07:45.632214+0000","last_became_active":"2024-09-15T22:06:44.658434+0000","last_became_peered":"2024-09-15T22:06:44.658434+0000","last_unstale":"2024-09-15T22:07:45.632214+0000","last_undegraded":"2024-09-15T22:07:45.632214+0000","last_fullsized":"2024-09-15T22:07:45.632214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:37.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.967121+0000","last_change":"2024-09-15T22:06:37.184958+0000","last_active":"2024-09-15T22:07:45.967121+0000","last_peered":"2024-09-15T22:07:45.967121+0000","last_clean":"2024-09-15T22:07:45.967121+0000","last_became_active":"2024-09-15T22:06:37.184648+0000","last_became_peered":"2024-09-15T22:06:37.184648+0000","last_unstale":"2024-09-15T22:07:45.967121+0000","last_undegraded":"2024-09-15T22:07:45.967121+0000","last_fullsized":"2024-09-15T22:07:45.967121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:28.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'33","reported_seq":178,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:08:04.816164+0000","last_change":"2024-09-15T22:06:35.138533+0000","last_active":"2024-09-15T22:08:04.816164+0000","last_peered":"2024-09-15T22:08:04.816164+0000","last_clean":"2024-09-15T22:08:04.816164+0000","last_became_active":"2024-09-15T22:06:35.138142+0000","last_became_peered":"2024-09-15T22:06:35.138142+0000","last_unstale":"2024-09-15T22:08:04.816164+0000","last_undegraded":"2024-09-15T22:08:04.816164+0000","last_fullsized":"2024-09-15T22:08:04.816164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:32.950036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":111,"num_read_kb":116,"num_write":34,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.218222+0000","last_change":"2024-09-15T22:07:46.218222+0000","last_active":"2024-09-15T22:07:46.218222+0000","last_peered":"2024-09-15T22:07:46.218222+0000","last_clean":"2024-09-15T22:07:46.218222+0000","last_became_active":"2024-09-15T22:07:46.217903+0000","last_became_peered":"2024-09-15T22:07:46.217903+0000","last_unstale":"2024-09-15T22:07:46.218222+0000","last_undegraded":"2024-09-15T22:07:46.218222+0000","last_fullsized":"2024-09-15T22:07:46.218222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:17.730288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217066+0000","last_change":"2024-09-15T22:06:37.194297+0000","last_active":"2024-09-15T22:07:46.217066+0000","last_peered":"2024-09-15T22:07:46.217066+0000","last_clean":"2024-09-15T22:07:46.217066+0000","last_became_active":"2024-09-15T22:06:37.193986+0000","last_became_peered":"2024-09-15T22:06:37.193986+0000","last_unstale":"2024-09-15T22:07:46.217066+0000","last_undegraded":"2024-09-15T22:07:46.217066+0000","last_fullsized":"2024-09-15T22:07:46.217066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:27.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632496+0000","last_change":"2024-09-15T22:06:35.356537+0000","last_active":"2024-09-15T22:07:45.632496+0000","last_peered":"2024-09-15T22:07:45.632496+0000","last_clean":"2024-09-15T22:07:45.632496+0000","last_became_active":"2024-09-15T22:06:35.355997+0000","last_became_peered":"2024-09-15T22:06:35.355997+0000","last_unstale":"2024-09-15T22:07:45.632496+0000","last_undegraded":"2024-09-15T22:07:45.632496+0000","last_fullsized":"2024-09-15T22:07:45.632496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:11.563083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629628+0000","last_change":"2024-09-15T22:06:44.654579+0000","last_active":"2024-09-15T22:07:45.629628+0000","last_peered":"2024-09-15T22:07:45.629628+0000","last_clean":"2024-09-15T22:07:45.629628+0000","last_became_active":"2024-09-15T22:06:44.654187+0000","last_became_peered":"2024-09-15T22:06:44.654187+0000","last_unstale":"2024-09-15T22:07:45.629628+0000","last_undegraded":"2024-09-15T22:07:45.629628+0000","last_fullsized":"2024-09-15T22:07:45.629628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:15.617528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217296+0000","last_change":"2024-09-15T22:06:44.661533+0000","last_active":"2024-09-15T22:07:46.217296+0000","last_peered":"2024-09-15T22:07:46.217296+0000","last_clean":"2024-09-15T22:07:46.217296+0000","last_became_active":"2024-09-15T22:06:44.661193+0000","last_became_peered":"2024-09-15T22:06:44.661193+0000","last_unstale":"2024-09-15T22:07:46.217296+0000","last_undegraded":"2024-09-15T22:07:46.217296+0000","last_fullsized":"2024-09-15T22:07:46.217296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:06.559720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629989+0000","last_change":"2024-09-15T22:06:35.140578+0000","last_active":"2024-09-15T22:07:45.629989+0000","last_peered":"2024-09-15T22:07:45.629989+0000","last_clean":"2024-09-15T22:07:45.629989+0000","last_became_active":"2024-09-15T22:06:35.139818+0000","last_became_peered":"2024-09-15T22:06:35.139818+0000","last_unstale":"2024-09-15T22:07:45.629989+0000","last_undegraded":"2024-09-15T22:07:45.629989+0000","last_fullsized":"2024-09-15T22:07:45.629989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:59.806964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.626672+0000","last_change":"2024-09-15T22:06:44.654755+0000","last_active":"2024-09-15T22:07:45.626672+0000","last_peered":"2024-09-15T22:07:45.626672+0000","last_clean":"2024-09-15T22:07:45.626672+0000","last_became_active":"2024-09-15T22:06:44.654390+0000","last_became_peered":"2024-09-15T22:06:44.654390+0000","last_unstale":"2024-09-15T22:07:45.626672+0000","last_undegraded":"2024-09-15T22:07:45.626672+0000","last_fullsized":"2024-09-15T22:07:45.626672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:35.781878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217329+0000","last_change":"2024-09-15T22:06:37.182457+0000","last_active":"2024-09-15T22:07:46.217329+0000","last_peered":"2024-09-15T22:07:46.217329+0000","last_clean":"2024-09-15T22:07:46.217329+0000","last_became_active":"2024-09-15T22:06:37.182106+0000","last_became_peered":"2024-09-15T22:06:37.182106+0000","last_unstale":"2024-09-15T22:07:46.217329+0000","last_undegraded":"2024-09-15T22:07:46.217329+0000","last_fullsized":"2024-09-15T22:07:46.217329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:57.560894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":88,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:08:03.814699+0000","last_change":"2024-09-15T22:06:35.140590+0000","last_active":"2024-09-15T22:08:03.814699+0000","last_peered":"2024-09-15T22:08:03.814699+0000","last_clean":"2024-09-15T22:08:03.814699+0000","last_became_active":"2024-09-15T22:06:35.140198+0000","last_became_peered":"2024-09-15T22:06:35.140198+0000","last_unstale":"2024-09-15T22:08:03.814699+0000","last_undegraded":"2024-09-15T22:08:03.814699+0000","last_fullsized":"2024-09-15T22:08:03.814699+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:46.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811193+0000","last_change":"2024-09-15T22:06:44.660188+0000","last_active":"2024-09-15T22:07:45.811193+0000","last_peered":"2024-09-15T22:07:45.811193+0000","last_clean":"2024-09-15T22:07:45.811193+0000","last_became_active":"2024-09-15T22:06:44.659808+0000","last_became_peered":"2024-09-15T22:06:44.659808+0000","last_unstale":"2024-09-15T22:07:45.811193+0000","last_undegraded":"2024-09-15T22:07:45.811193+0000","last_fullsized":"2024-09-15T22:07:45.811193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:39.473765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216787+0000","last_change":"2024-09-15T22:06:44.647967+0000","last_active":"2024-09-15T22:07:46.216787+0000","last_peered":"2024-09-15T22:07:46.216787+0000","last_clean":"2024-09-15T22:07:46.216787+0000","last_became_active":"2024-09-15T22:06:44.647608+0000","last_became_peered":"2024-09-15T22:06:44.647608+0000","last_unstale":"2024-09-15T22:07:46.216787+0000","last_undegraded":"2024-09-15T22:07:46.216787+0000","last_fullsized":"2024-09-15T22:07:46.216787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:06.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.626850+0000","last_change":"2024-09-15T22:06:37.187020+0000","last_active":"2024-09-15T22:07:45.626850+0000","last_peered":"2024-09-15T22:07:45.626850+0000","last_clean":"2024-09-15T22:07:45.626850+0000","last_became_active":"2024-09-15T22:06:37.186564+0000","last_became_peered":"2024-09-15T22:06:37.186564+0000","last_unstale":"2024-09-15T22:07:45.626850+0000","last_undegraded":"2024-09-15T22:07:45.626850+0000","last_fullsized":"2024-09-15T22:07:45.626850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:20.586763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631462+0000","last_change":"2024-09-15T22:06:35.135126+0000","last_active":"2024-09-15T22:07:45.631462+0000","last_peered":"2024-09-15T22:07:45.631462+0000","last_clean":"2024-09-15T22:07:45.631462+0000","last_became_active":"2024-09-15T22:06:35.134783+0000","last_became_peered":"2024-09-15T22:06:35.134783+0000","last_unstale":"2024-09-15T22:07:45.631462+0000","last_undegraded":"2024-09-15T22:07:45.631462+0000","last_fullsized":"2024-09-15T22:07:45.631462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:31.910988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.218056+0000","last_change":"2024-09-15T22:06:44.656981+0000","last_active":"2024-09-15T22:07:46.218056+0000","last_peered":"2024-09-15T22:07:46.218056+0000","last_clean":"2024-09-15T22:07:46.218056+0000","last_became_active":"2024-09-15T22:06:44.656477+0000","last_became_peered":"2024-09-15T22:06:44.656477+0000","last_unstale":"2024-09-15T22:07:46.218056+0000","last_undegraded":"2024-09-15T22:07:46.218056+0000","last_fullsized":"2024-09-15T22:07:46.218056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:06.153490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811588+0000","last_change":"2024-09-15T22:06:37.183034+0000","last_active":"2024-09-15T22:07:45.811588+0000","last_peered":"2024-09-15T22:07:45.811588+0000","last_clean":"2024-09-15T22:07:45.811588+0000","last_became_active":"2024-09-15T22:06:37.182734+0000","last_became_peered":"2024-09-15T22:06:37.182734+0000","last_unstale":"2024-09-15T22:07:45.811588+0000","last_undegraded":"2024-09-15T22:07:45.811588+0000","last_fullsized":"2024-09-15T22:07:45.811588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:19.352000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.785911+0000","last_change":"2024-09-15T22:06:35.134954+0000","last_active":"2024-09-15T22:07:45.785911+0000","last_peered":"2024-09-15T22:07:45.785911+0000","last_clean":"2024-09-15T22:07:45.785911+0000","last_became_active":"2024-09-15T22:06:35.134583+0000","last_became_peered":"2024-09-15T22:06:35.134583+0000","last_unstale":"2024-09-15T22:07:45.785911+0000","last_undegraded":"2024-09-15T22:07:45.785911+0000","last_fullsized":"2024-09-15T22:07:45.785911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:56.082123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.628342+0000","last_change":"2024-09-15T22:06:35.139229+0000","last_active":"2024-09-15T22:07:45.628342+0000","last_peered":"2024-09-15T22:07:45.628342+0000","last_clean":"2024-09-15T22:07:45.628342+0000","last_became_active":"2024-09-15T22:06:35.138834+0000","last_became_peered":"2024-09-15T22:06:35.138834+0000","last_unstale":"2024-09-15T22:07:45.628342+0000","last_undegraded":"2024-09-15T22:07:45.628342+0000","last_fullsized":"2024-09-15T22:07:45.628342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:05.877404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632504+0000","last_change":"2024-09-15T22:06:37.192212+0000","last_active":"2024-09-15T22:07:45.632504+0000","last_peered":"2024-09-15T22:07:45.632504+0000","last_clean":"2024-09-15T22:07:45.632504+0000","last_became_active":"2024-09-15T22:06:37.191872+0000","last_became_peered":"2024-09-15T22:06:37.191872+0000","last_unstale":"2024-09-15T22:07:45.632504+0000","last_undegraded":"2024-09-15T22:07:45.632504+0000","last_fullsized":"2024-09-15T22:07:45.632504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:32.845794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.968398+0000","last_change":"2024-09-15T22:07:45.968398+0000","last_active":"2024-09-15T22:07:45.968398+0000","last_peered":"2024-09-15T22:07:45.968398+0000","last_clean":"2024-09-15T22:07:45.968398+0000","last_became_active":"2024-09-15T22:07:45.967912+0000","last_became_peered":"2024-09-15T22:07:45.967912+0000","last_unstale":"2024-09-15T22:07:45.968398+0000","last_undegraded":"2024-09-15T22:07:45.968398+0000","last_fullsized":"2024-09-15T22:07:45.968398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:59.909290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:08:05.818159+0000","last_change":"2024-09-15T22:06:35.133846+0000","last_active":"2024-09-15T22:08:05.818159+0000","last_peered":"2024-09-15T22:08:05.818159+0000","last_clean":"2024-09-15T22:08:05.818159+0000","last_became_active":"2024-09-15T22:06:35.133516+0000","last_became_peered":"2024-09-15T22:06:35.133516+0000","last_unstale":"2024-09-15T22:08:05.818159+0000","last_undegraded":"2024-09-15T22:08:05.818159+0000","last_fullsized":"2024-09-15T22:08:05.818159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:29.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966752+0000","last_change":"2024-09-15T22:06:37.182904+0000","last_active":"2024-09-15T22:07:45.966752+0000","last_peered":"2024-09-15T22:07:45.966752+0000","last_clean":"2024-09-15T22:07:45.966752+0000","last_became_active":"2024-09-15T22:06:37.182563+0000","last_became_peered":"2024-09-15T22:06:37.182563+0000","last_unstale":"2024-09-15T22:07:45.966752+0000","last_undegraded":"2024-09-15T22:07:45.966752+0000","last_fullsized":"2024-09-15T22:07:45.966752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631298+0000","last_change":"2024-09-15T22:06:44.660997+0000","last_active":"2024-09-15T22:07:45.631298+0000","last_peered":"2024-09-15T22:07:45.631298+0000","last_clean":"2024-09-15T22:07:45.631298+0000","last_became_active":"2024-09-15T22:06:44.660577+0000","last_became_peered":"2024-09-15T22:06:44.660577+0000","last_unstale":"2024-09-15T22:07:45.631298+0000","last_undegraded":"2024-09-15T22:07:45.631298+0000","last_fullsized":"2024-09-15T22:07:45.631298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:56.006891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217462+0000","last_change":"2024-09-15T22:06:37.182077+0000","last_active":"2024-09-15T22:07:46.217462+0000","last_peered":"2024-09-15T22:07:46.217462+0000","last_clean":"2024-09-15T22:07:46.217462+0000","last_became_active":"2024-09-15T22:06:37.181622+0000","last_became_peered":"2024-09-15T22:06:37.181622+0000","last_unstale":"2024-09-15T22:07:46.217462+0000","last_undegraded":"2024-09-15T22:07:46.217462+0000","last_fullsized":"2024-09-15T22:07:46.217462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:07.586406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631109+0000","last_change":"2024-09-15T22:06:44.653845+0000","last_active":"2024-09-15T22:07:45.631109+0000","last_peered":"2024-09-15T22:07:45.631109+0000","last_clean":"2024-09-15T22:07:45.631109+0000","last_became_active":"2024-09-15T22:06:44.653483+0000","last_became_peered":"2024-09-15T22:06:44.653483+0000","last_unstale":"2024-09-15T22:07:45.631109+0000","last_undegraded":"2024-09-15T22:07:45.631109+0000","last_fullsized":"2024-09-15T22:07:45.631109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:08:07.822279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630930+0000","last_change":"2024-09-15T22:06:37.187476+0000","last_active":"2024-09-15T22:07:45.630930+0000","last_peered":"2024-09-15T22:07:45.630930+0000","last_clean":"2024-09-15T22:07:45.630930+0000","last_became_active":"2024-09-15T22:06:37.187047+0000","last_became_peered":"2024-09-15T22:06:37.187047+0000","last_unstale":"2024-09-15T22:07:45.630930+0000","last_undegraded":"2024-09-15T22:07:45.630930+0000","last_fullsized":"2024-09-15T22:07:45.630930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.220834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216953+0000","last_change":"2024-09-15T22:06:37.190583+0000","last_active":"2024-09-15T22:07:46.216953+0000","last_peered":"2024-09-15T22:07:46.216953+0000","last_clean":"2024-09-15T22:07:46.216953+0000","last_became_active":"2024-09-15T22:06:37.190212+0000","last_became_peered":"2024-09-15T22:06:37.190212+0000","last_unstale":"2024-09-15T22:07:46.216953+0000","last_undegraded":"2024-09-15T22:07:46.216953+0000","last_fullsized":"2024-09-15T22:07:46.216953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:48.574188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786533+0000","last_change":"2024-09-15T22:06:37.192508+0000","last_active":"2024-09-15T22:07:45.786533+0000","last_peered":"2024-09-15T22:07:45.786533+0000","last_clean":"2024-09-15T22:07:45.786533+0000","last_became_active":"2024-09-15T22:06:37.192117+0000","last_became_peered":"2024-09-15T22:06:37.192117+0000","last_unstale":"2024-09-15T22:07:45.786533+0000","last_undegraded":"2024-09-15T22:07:45.786533+0000","last_fullsized":"2024-09-15T22:07:45.786533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:41.355966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217986+0000","last_change":"2024-09-15T22:06:37.183377+0000","last_active":"2024-09-15T22:07:46.217986+0000","last_peered":"2024-09-15T22:07:46.217986+0000","last_clean":"2024-09-15T22:07:46.217986+0000","last_became_active":"2024-09-15T22:06:37.183072+0000","last_became_peered":"2024-09-15T22:06:37.183072+0000","last_unstale":"2024-09-15T22:07:46.217986+0000","last_undegraded":"2024-09-15T22:07:46.217986+0000","last_fullsized":"2024-09-15T22:07:46.217986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.318702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631812+0000","last_change":"2024-09-15T22:06:37.185348+0000","last_active":"2024-09-15T22:07:45.631812+0000","last_peered":"2024-09-15T22:07:45.631812+0000","last_clean":"2024-09-15T22:07:45.631812+0000","last_became_active":"2024-09-15T22:06:37.184934+0000","last_became_peered":"2024-09-15T22:06:37.184934+0000","last_unstale":"2024-09-15T22:07:45.631812+0000","last_undegraded":"2024-09-15T22:07:45.631812+0000","last_fullsized":"2024-09-15T22:07:45.631812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:57.564860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811690+0000","last_change":"2024-09-15T22:06:37.192152+0000","last_active":"2024-09-15T22:07:45.811690+0000","last_peered":"2024-09-15T22:07:45.811690+0000","last_clean":"2024-09-15T22:07:45.811690+0000","last_became_active":"2024-09-15T22:06:37.191836+0000","last_became_peered":"2024-09-15T22:06:37.191836+0000","last_unstale":"2024-09-15T22:07:45.811690+0000","last_undegraded":"2024-09-15T22:07:45.811690+0000","last_fullsized":"2024-09-15T22:07:45.811690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:31.781110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631207+0000","last_change":"2024-09-15T22:06:37.179996+0000","last_active":"2024-09-15T22:07:45.631207+0000","last_peered":"2024-09-15T22:07:45.631207+0000","last_clean":"2024-09-15T22:07:45.631207+0000","last_became_active":"2024-09-15T22:06:37.179646+0000","last_became_peered":"2024-09-15T22:06:37.179646+0000","last_unstale":"2024-09-15T22:07:45.631207+0000","last_undegraded":"2024-09-15T22:07:45.631207+0000","last_fullsized":"2024-09-15T22:07:45.631207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:35.240468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811293+0000","last_change":"2024-09-15T22:06:37.182992+0000","last_active":"2024-09-15T22:07:45.811293+0000","last_peered":"2024-09-15T22:07:45.811293+0000","last_clean":"2024-09-15T22:07:45.811293+0000","last_became_active":"2024-09-15T22:06:37.182676+0000","last_became_peered":"2024-09-15T22:06:37.182676+0000","last_unstale":"2024-09-15T22:07:45.811293+0000","last_undegraded":"2024-09-15T22:07:45.811293+0000","last_fullsized":"2024-09-15T22:07:45.811293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:41.524600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216932+0000","last_change":"2024-09-15T22:06:37.180107+0000","last_active":"2024-09-15T22:07:46.216932+0000","last_peered":"2024-09-15T22:07:46.216932+0000","last_clean":"2024-09-15T22:07:46.216932+0000","last_became_active":"2024-09-15T22:06:37.179785+0000","last_became_peered":"2024-09-15T22:06:37.179785+0000","last_unstale":"2024-09-15T22:07:46.216932+0000","last_undegraded":"2024-09-15T22:07:46.216932+0000","last_fullsized":"2024-09-15T22:07:46.216932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:46.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786319+0000","last_change":"2024-09-15T22:06:37.177406+0000","last_active":"2024-09-15T22:07:45.786319+0000","last_peered":"2024-09-15T22:07:45.786319+0000","last_clean":"2024-09-15T22:07:45.786319+0000","last_became_active":"2024-09-15T22:06:37.177093+0000","last_became_peered":"2024-09-15T22:06:37.177093+0000","last_unstale":"2024-09-15T22:07:45.786319+0000","last_undegraded":"2024-09-15T22:07:45.786319+0000","last_fullsized":"2024-09-15T22:07:45.786319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:34.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217704+0000","last_change":"2024-09-15T22:06:37.192314+0000","last_active":"2024-09-15T22:07:46.217704+0000","last_peered":"2024-09-15T22:07:46.217704+0000","last_clean":"2024-09-15T22:07:46.217704+0000","last_became_active":"2024-09-15T22:06:37.191958+0000","last_became_peered":"2024-09-15T22:06:37.191958+0000","last_unstale":"2024-09-15T22:07:46.217704+0000","last_undegraded":"2024-09-15T22:07:46.217704+0000","last_fullsized":"2024-09-15T22:07:46.217704+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:47.763742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631408+0000","last_change":"2024-09-15T22:06:37.190212+0000","last_active":"2024-09-15T22:07:45.631408+0000","last_peered":"2024-09-15T22:07:45.631408+0000","last_clean":"2024-09-15T22:07:45.631408+0000","last_became_active":"2024-09-15T22:06:37.189881+0000","last_became_peered":"2024-09-15T22:06:37.189881+0000","last_unstale":"2024-09-15T22:07:45.631408+0000","last_undegraded":"2024-09-15T22:07:45.631408+0000","last_fullsized":"2024-09-15T22:07:45.631408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:59.943574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627394+0000","last_change":"2024-09-15T22:06:37.187967+0000","last_active":"2024-09-15T22:07:45.627394+0000","last_peered":"2024-09-15T22:07:45.627394+0000","last_clean":"2024-09-15T22:07:45.627394+0000","last_became_active":"2024-09-15T22:06:37.187596+0000","last_became_peered":"2024-09-15T22:06:37.187596+0000","last_unstale":"2024-09-15T22:07:45.627394+0000","last_undegraded":"2024-09-15T22:07:45.627394+0000","last_fullsized":"2024-09-15T22:07:45.627394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:00.042784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.218545+0000","last_change":"2024-09-15T22:06:37.189434+0000","last_active":"2024-09-15T22:07:46.218545+0000","last_peered":"2024-09-15T22:07:46.218545+0000","last_clean":"2024-09-15T22:07:46.218545+0000","last_became_active":"2024-09-15T22:06:37.189053+0000","last_became_peered":"2024-09-15T22:06:37.189053+0000","last_unstale":"2024-09-15T22:07:46.218545+0000","last_undegraded":"2024-09-15T22:07:46.218545+0000","last_fullsized":"2024-09-15T22:07:46.218545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:51.640495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216636+0000","last_change":"2024-09-15T22:06:37.179229+0000","last_active":"2024-09-15T22:07:46.216636+0000","last_peered":"2024-09-15T22:07:46.216636+0000","last_clean":"2024-09-15T22:07:46.216636+0000","last_became_active":"2024-09-15T22:06:37.178828+0000","last_became_peered":"2024-09-15T22:06:37.178828+0000","last_unstale":"2024-09-15T22:07:46.216636+0000","last_undegraded":"2024-09-15T22:07:46.216636+0000","last_fullsized":"2024-09-15T22:07:46.216636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:25.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216741+0000","last_change":"2024-09-15T22:06:37.193095+0000","last_active":"2024-09-15T22:07:46.216741+0000","last_peered":"2024-09-15T22:07:46.216741+0000","last_clean":"2024-09-15T22:07:46.216741+0000","last_became_active":"2024-09-15T22:06:37.192801+0000","last_became_peered":"2024-09-15T22:06:37.192801+0000","last_unstale":"2024-09-15T22:07:46.216741+0000","last_undegraded":"2024-09-15T22:07:46.216741+0000","last_fullsized":"2024-09-15T22:07:46.216741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:12.521510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216619+0000","last_change":"2024-09-15T22:06:37.182332+0000","last_active":"2024-09-15T22:07:46.216619+0000","last_peered":"2024-09-15T22:07:46.216619+0000","last_clean":"2024-09-15T22:07:46.216619+0000","last_became_active":"2024-09-15T22:06:37.182001+0000","last_became_peered":"2024-09-15T22:06:37.182001+0000","last_unstale":"2024-09-15T22:07:46.216619+0000","last_undegraded":"2024-09-15T22:07:46.216619+0000","last_fullsized":"2024-09-15T22:07:46.216619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:52.230452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786312+0000","last_change":"2024-09-15T22:06:37.180584+0000","last_active":"2024-09-15T22:07:45.786312+0000","last_peered":"2024-09-15T22:07:45.786312+0000","last_clean":"2024-09-15T22:07:45.786312+0000","last_became_active":"2024-09-15T22:06:37.179341+0000","last_became_peered":"2024-09-15T22:06:37.179341+0000","last_unstale":"2024-09-15T22:07:45.786312+0000","last_undegraded":"2024-09-15T22:07:45.786312+0000","last_fullsized":"2024-09-15T22:07:45.786312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.610835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629971+0000","last_change":"2024-09-15T22:06:37.189500+0000","last_active":"2024-09-15T22:07:45.629971+0000","last_peered":"2024-09-15T22:07:45.629971+0000","last_clean":"2024-09-15T22:07:45.629971+0000","last_became_active":"2024-09-15T22:06:37.189137+0000","last_became_peered":"2024-09-15T22:06:37.189137+0000","last_unstale":"2024-09-15T22:07:45.629971+0000","last_undegraded":"2024-09-15T22:07:45.629971+0000","last_fullsized":"2024-09-15T22:07:45.629971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:27.199854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631325+0000","last_change":"2024-09-15T22:06:37.182618+0000","last_active":"2024-09-15T22:07:45.631325+0000","last_peered":"2024-09-15T22:07:45.631325+0000","last_clean":"2024-09-15T22:07:45.631325+0000","last_became_active":"2024-09-15T22:06:37.182257+0000","last_became_peered":"2024-09-15T22:06:37.182257+0000","last_unstale":"2024-09-15T22:07:45.631325+0000","last_undegraded":"2024-09-15T22:07:45.631325+0000","last_fullsized":"2024-09-15T22:07:45.631325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:17.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.628425+0000","last_change":"2024-09-15T22:06:37.193036+0000","last_active":"2024-09-15T22:07:45.628425+0000","last_peered":"2024-09-15T22:07:45.628425+0000","last_clean":"2024-09-15T22:07:45.628425+0000","last_became_active":"2024-09-15T22:06:37.192668+0000","last_became_peered":"2024-09-15T22:06:37.192668+0000","last_unstale":"2024-09-15T22:07:45.628425+0000","last_undegraded":"2024-09-15T22:07:45.628425+0000","last_fullsized":"2024-09-15T22:07:45.628425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:05.436045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627388+0000","last_change":"2024-09-15T22:06:44.659503+0000","last_active":"2024-09-15T22:07:45.627388+0000","last_peered":"2024-09-15T22:07:45.627388+0000","last_clean":"2024-09-15T22:07:45.627388+0000","last_became_active":"2024-09-15T22:06:44.659155+0000","last_became_peered":"2024-09-15T22:06:44.659155+0000","last_unstale":"2024-09-15T22:07:45.627388+0000","last_undegraded":"2024-09-15T22:07:45.627388+0000","last_fullsized":"2024-09-15T22:07:45.627388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:41:04.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811547+0000","last_change":"2024-09-15T22:06:37.178386+0000","last_active":"2024-09-15T22:07:45.811547+0000","last_peered":"2024-09-15T22:07:45.811547+0000","last_clean":"2024-09-15T22:07:45.811547+0000","last_became_active":"2024-09-15T22:06:37.178083+0000","last_became_peered":"2024-09-15T22:06:37.178083+0000","last_unstale":"2024-09-15T22:07:45.811547+0000","last_undegraded":"2024-09-15T22:07:45.811547+0000","last_fullsized":"2024-09-15T22:07:45.811547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:35.350698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630748+0000","last_change":"2024-09-15T22:06:37.186741+0000","last_active":"2024-09-15T22:07:45.630748+0000","last_peered":"2024-09-15T22:07:45.630748+0000","last_clean":"2024-09-15T22:07:45.630748+0000","last_became_active":"2024-09-15T22:06:37.185740+0000","last_became_peered":"2024-09-15T22:06:37.185740+0000","last_unstale":"2024-09-15T22:07:45.630748+0000","last_undegraded":"2024-09-15T22:07:45.630748+0000","last_fullsized":"2024-09-15T22:07:45.630748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.762407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.628239+0000","last_change":"2024-09-15T22:06:37.189552+0000","last_active":"2024-09-15T22:07:45.628239+0000","last_peered":"2024-09-15T22:07:45.628239+0000","last_clean":"2024-09-15T22:07:45.628239+0000","last_became_active":"2024-09-15T22:06:37.189197+0000","last_became_peered":"2024-09-15T22:06:37.189197+0000","last_unstale":"2024-09-15T22:07:45.628239+0000","last_undegraded":"2024-09-15T22:07:45.628239+0000","last_fullsized":"2024-09-15T22:07:45.628239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:54.268720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966793+0000","last_change":"2024-09-15T22:06:37.186917+0000","last_active":"2024-09-15T22:07:45.966793+0000","last_peered":"2024-09-15T22:07:45.966793+0000","last_clean":"2024-09-15T22:07:45.966793+0000","last_became_active":"2024-09-15T22:06:37.186571+0000","last_became_peered":"2024-09-15T22:06:37.186571+0000","last_unstale":"2024-09-15T22:07:45.966793+0000","last_undegraded":"2024-09-15T22:07:45.966793+0000","last_fullsized":"2024-09-15T22:07:45.966793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:02.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629838+0000","last_change":"2024-09-15T22:06:37.190960+0000","last_active":"2024-09-15T22:07:45.629838+0000","last_peered":"2024-09-15T22:07:45.629838+0000","last_clean":"2024-09-15T22:07:45.629838+0000","last_became_active":"2024-09-15T22:06:37.190650+0000","last_became_peered":"2024-09-15T22:06:37.190650+0000","last_unstale":"2024-09-15T22:07:45.629838+0000","last_undegraded":"2024-09-15T22:07:45.629838+0000","last_fullsized":"2024-09-15T22:07:45.629838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:34.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217037+0000","last_change":"2024-09-15T22:06:37.186718+0000","last_active":"2024-09-15T22:07:46.217037+0000","last_peered":"2024-09-15T22:07:46.217037+0000","last_clean":"2024-09-15T22:07:46.217037+0000","last_became_active":"2024-09-15T22:06:37.186424+0000","last_became_peered":"2024-09-15T22:06:37.186424+0000","last_unstale":"2024-09-15T22:07:46.217037+0000","last_undegraded":"2024-09-15T22:07:46.217037+0000","last_fullsized":"2024-09-15T22:07:46.217037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:53.266007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216870+0000","last_change":"2024-09-15T22:06:37.185522+0000","last_active":"2024-09-15T22:07:46.216870+0000","last_peered":"2024-09-15T22:07:46.216870+0000","last_clean":"2024-09-15T22:07:46.216870+0000","last_became_active":"2024-09-15T22:06:37.185192+0000","last_became_peered":"2024-09-15T22:06:37.185192+0000","last_unstale":"2024-09-15T22:07:46.216870+0000","last_undegraded":"2024-09-15T22:07:46.216870+0000","last_fullsized":"2024-09-15T22:07:46.216870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:12.868324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.218447+0000","last_change":"2024-09-15T22:06:37.189553+0000","last_active":"2024-09-15T22:07:46.218447+0000","last_peered":"2024-09-15T22:07:46.218447+0000","last_clean":"2024-09-15T22:07:46.218447+0000","last_became_active":"2024-09-15T22:06:37.189270+0000","last_became_peered":"2024-09-15T22:06:37.189270+0000","last_unstale":"2024-09-15T22:07:46.218447+0000","last_undegraded":"2024-09-15T22:07:46.218447+0000","last_fullsized":"2024-09-15T22:07:46.218447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:34.157077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66472,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":343,"num_read_kb":241,"num_write":174,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":199416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579936,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1011712,"data_stored":502844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808869,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32848,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712304,"statfs":{"total":95995035648,"available":95961399296,"internally_reserved":0,"allocated":1695744,"data_stored":1151999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005098,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1028096,"data_stored":506594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234030,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":501836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495668,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1613824,"data_stored":1098140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659193,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1003520,"data_stored":497674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888126,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1626112,"data_stored":1093917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018948,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1019904,"data_stored":503299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247880,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1015808,"data_stored":497868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444110,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":560821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1073152,"data_stored":560242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1028096,"data_stored":499062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:08:10.036 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:08:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:10.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:09 smithi099 ceph-mon[34892]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:10.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:09 smithi099 ceph-mon[34892]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:10.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:09 smithi129 ceph-mon[35314]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:10.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:09 smithi129 ceph-mon[35314]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:10.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:09 smithi005 ceph-mon[30944]: pgmap v331: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:10.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:09 smithi005 ceph-mon[30944]: from='client.15000 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.794+0000 7f70eea8d640 1 -- 172.21.15.5:0/820885877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e8106a80 msgr2=0x7f70e8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.794+0000 7f70eea8d640 1 --2- 172.21.15.5:0/820885877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e8106a80 0x7f70e8106e60 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f70d0009930 tx=0x7f70d002a420 comp rx=0 tx=0).stop 2024-09-15T22:08:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.794+0000 7f70eea8d640 1 -- 172.21.15.5:0/820885877 shutdown_connections 2024-09-15T22:08:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.794+0000 7f70eea8d640 1 --2- 172.21.15.5:0/820885877 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e8101420 0x7f70e810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.794+0000 7f70eea8d640 1 --2- 172.21.15.5:0/820885877 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e8100a80 0x7f70e8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.794+0000 7f70eea8d640 1 --2- 172.21.15.5:0/820885877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e8106a80 0x7f70e8106e60 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.794+0000 7f70eea8d640 1 -- 172.21.15.5:0/820885877 >> 172.21.15.5:0/820885877 conn(0x7f70e80fc7a0 msgr2=0x7f70e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.794+0000 7f70eea8d640 1 -- 172.21.15.5:0/820885877 shutdown_connections 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.794+0000 7f70eea8d640 1 -- 172.21.15.5:0/820885877 wait complete. 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70eea8d640 1 Processor -- start 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70eea8d640 1 -- start start 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70eea8d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e8100a80 0x7f70e8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70eea8d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e8101420 0x7f70e8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70eea8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e8106a80 0x7f70e81a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70eea8d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8110ed0 con 0x7f70e8106a80 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70eea8d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8110d50 con 0x7f70e8100a80 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70eea8d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f70e8111050 con 0x7f70e8101420 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70ed02a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e8106a80 0x7f70e81a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70ed02a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e8106a80 0x7f70e81a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57240/0 (socket says 172.21.15.5:57240) 2024-09-15T22:08:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70ed02a640 1 -- 172.21.15.5:0/2715018205 learned_addr learned my addr 172.21.15.5:0/2715018205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70ec829640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e8100a80 0x7f70e8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70e7fff640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e8101420 0x7f70e8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70ed02a640 1 -- 172.21.15.5:0/2715018205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e8101420 msgr2=0x7f70e8199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70ed02a640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e8101420 0x7f70e8199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70ed02a640 1 -- 172.21.15.5:0/2715018205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e8100a80 msgr2=0x7f70e8199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70ed02a640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e8100a80 0x7f70e8199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.795+0000 7f70ed02a640 1 -- 172.21.15.5:0/2715018205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e81a4800 con 0x7f70e8106a80 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.796+0000 7f70ed02a640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e8106a80 0x7f70e81a42c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f70e000cce0 tx=0x7f70e000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.796+0000 7f70e5ffb640 1 -- 172.21.15.5:0/2715018205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e000f450 con 0x7f70e8106a80 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.796+0000 7f70e5ffb640 1 -- 172.21.15.5:0/2715018205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70e000fad0 con 0x7f70e8106a80 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.796+0000 7f70eea8d640 1 -- 172.21.15.5:0/2715018205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e81a49d0 con 0x7f70e8106a80 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.796+0000 7f70e5ffb640 1 -- 172.21.15.5:0/2715018205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e0018c10 con 0x7f70e8106a80 2024-09-15T22:08:10.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.796+0000 7f70eea8d640 1 -- 172.21.15.5:0/2715018205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e81a4f10 con 0x7f70e8106a80 2024-09-15T22:08:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.797+0000 7f70eea8d640 1 -- 172.21.15.5:0/2715018205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e8100ee0 con 0x7f70e8106a80 2024-09-15T22:08:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.798+0000 7f70e5ffb640 1 -- 172.21.15.5:0/2715018205 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f70e000f5f0 con 0x7f70e8106a80 2024-09-15T22:08:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.799+0000 7f70e5ffb640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70d40a3350 0x7f70d40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.799+0000 7f70ec829640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70d40a3350 0x7f70d40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.799+0000 7f70e5ffb640 1 -- 172.21.15.5:0/2715018205 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f70e00d0360 con 0x7f70e8106a80 2024-09-15T22:08:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.799+0000 7f70ec829640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70d40a3350 0x7f70d40a5810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f70d0006010 tx=0x7f70d0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:10.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.803+0000 7f70e5ffb640 1 -- 172.21.15.5:0/2715018205 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70e0014030 con 0x7f70e8106a80 2024-09-15T22:08:10.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.976+0000 7f70eea8d640 1 -- 172.21.15.5:0/2715018205 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70e80fe5b0 con 0x7f70d40a3350 2024-09-15T22:08:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.984+0000 7f70e5ffb640 1 -- 172.21.15.5:0/2715018205 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340359 (secure 0 0 0) 0x7f70e80fe5b0 con 0x7f70d40a3350 2024-09-15T22:08:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-15T22:08:10.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.987+0000 7f70eea8d640 1 -- 172.21.15.5:0/2715018205 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70d40a3350 msgr2=0x7f70d40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:10.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.987+0000 7f70eea8d640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70d40a3350 0x7f70d40a5810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f70d0006010 tx=0x7f70d0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:08:10.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.987+0000 7f70eea8d640 1 -- 172.21.15.5:0/2715018205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e8106a80 msgr2=0x7f70e81a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:10.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.987+0000 7f70eea8d640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e8106a80 0x7f70e81a42c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f70e000cce0 tx=0x7f70e000a620 comp rx=0 tx=0).stop 2024-09-15T22:08:10.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.987+0000 7f70eea8d640 1 -- 172.21.15.5:0/2715018205 shutdown_connections 2024-09-15T22:08:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.987+0000 7f70eea8d640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f70d40a3350 0x7f70d40a5810 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.987+0000 7f70eea8d640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70e8106a80 0x7f70e81a42c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.987+0000 7f70eea8d640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f70e8101420 0x7f70e8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.987+0000 7f70eea8d640 1 --2- 172.21.15.5:0/2715018205 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f70e8100a80 0x7f70e8199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.987+0000 7f70eea8d640 1 -- 172.21.15.5:0/2715018205 >> 172.21.15.5:0/2715018205 conn(0x7f70e80fc7a0 msgr2=0x7f70e80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.988+0000 7f70eea8d640 1 -- 172.21.15.5:0/2715018205 shutdown_connections 2024-09-15T22:08:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:10.988+0000 7f70eea8d640 1 -- 172.21.15.5:0/2715018205 wait complete. 2024-09-15T22:08:11.783 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-15T22:08:10.809667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657074,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":494,"num_read_kb":370,"num_write":357,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":286,"ondisk_log_size":286,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388168,"kb_used_data":13892,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553656,"statfs":{"total":1151940427776,"available":1151542943744,"internally_reserved":0,"allocated":14225408,"data_stored":7974296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":382973243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002935"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":72,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.226073+0000","last_change":"2024-09-15T22:07:46.226073+0000","last_active":"2024-09-15T22:07:46.226073+0000","last_peered":"2024-09-15T22:07:46.226073+0000","last_clean":"2024-09-15T22:07:46.226073+0000","last_became_active":"2024-09-15T22:07:46.219250+0000","last_became_peered":"2024-09-15T22:07:46.219250+0000","last_unstale":"2024-09-15T22:07:46.226073+0000","last_undegraded":"2024-09-15T22:07:46.226073+0000","last_fullsized":"2024-09-15T22:07:46.226073+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:56.324959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,4],"acting":[9,11,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811629+0000","last_change":"2024-09-15T22:06:37.184096+0000","last_active":"2024-09-15T22:07:45.811629+0000","last_peered":"2024-09-15T22:07:45.811629+0000","last_clean":"2024-09-15T22:07:45.811629+0000","last_became_active":"2024-09-15T22:06:37.183758+0000","last_became_peered":"2024-09-15T22:06:37.183758+0000","last_unstale":"2024-09-15T22:07:45.811629+0000","last_undegraded":"2024-09-15T22:07:45.811629+0000","last_fullsized":"2024-09-15T22:07:45.811629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:04.210551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630641+0000","last_change":"2024-09-15T22:06:44.653042+0000","last_active":"2024-09-15T22:07:45.630641+0000","last_peered":"2024-09-15T22:07:45.630641+0000","last_clean":"2024-09-15T22:07:45.630641+0000","last_became_active":"2024-09-15T22:06:44.652658+0000","last_became_peered":"2024-09-15T22:06:44.652658+0000","last_unstale":"2024-09-15T22:07:45.630641+0000","last_undegraded":"2024-09-15T22:07:45.630641+0000","last_fullsized":"2024-09-15T22:07:45.630641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:59.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811815+0000","last_change":"2024-09-15T22:06:35.355392+0000","last_active":"2024-09-15T22:07:45.811815+0000","last_peered":"2024-09-15T22:07:45.811815+0000","last_clean":"2024-09-15T22:07:45.811815+0000","last_became_active":"2024-09-15T22:06:35.355042+0000","last_became_peered":"2024-09-15T22:06:35.355042+0000","last_unstale":"2024-09-15T22:07:45.811815+0000","last_undegraded":"2024-09-15T22:07:45.811815+0000","last_fullsized":"2024-09-15T22:07:45.811815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:05.186937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217282+0000","last_change":"2024-09-15T22:06:37.184421+0000","last_active":"2024-09-15T22:07:46.217282+0000","last_peered":"2024-09-15T22:07:46.217282+0000","last_clean":"2024-09-15T22:07:46.217282+0000","last_became_active":"2024-09-15T22:06:37.184092+0000","last_became_peered":"2024-09-15T22:06:37.184092+0000","last_unstale":"2024-09-15T22:07:46.217282+0000","last_undegraded":"2024-09-15T22:07:46.217282+0000","last_fullsized":"2024-09-15T22:07:46.217282+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:41.080981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.968640+0000","last_change":"2024-09-15T22:07:45.968640+0000","last_active":"2024-09-15T22:07:45.968640+0000","last_peered":"2024-09-15T22:07:45.968640+0000","last_clean":"2024-09-15T22:07:45.968640+0000","last_became_active":"2024-09-15T22:07:45.968320+0000","last_became_peered":"2024-09-15T22:07:45.968320+0000","last_unstale":"2024-09-15T22:07:45.968640+0000","last_undegraded":"2024-09-15T22:07:45.968640+0000","last_fullsized":"2024-09-15T22:07:45.968640+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:28.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.967279+0000","last_change":"2024-09-15T22:06:35.134606+0000","last_active":"2024-09-15T22:07:45.967279+0000","last_peered":"2024-09-15T22:07:45.967279+0000","last_clean":"2024-09-15T22:07:45.967279+0000","last_became_active":"2024-09-15T22:06:35.134263+0000","last_became_peered":"2024-09-15T22:06:35.134263+0000","last_unstale":"2024-09-15T22:07:45.967279+0000","last_undegraded":"2024-09-15T22:07:45.967279+0000","last_fullsized":"2024-09-15T22:07:45.967279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:45.008935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216998+0000","last_change":"2024-09-15T22:06:37.177556+0000","last_active":"2024-09-15T22:07:46.216998+0000","last_peered":"2024-09-15T22:07:46.216998+0000","last_clean":"2024-09-15T22:07:46.216998+0000","last_became_active":"2024-09-15T22:06:37.177161+0000","last_became_peered":"2024-09-15T22:06:37.177161+0000","last_unstale":"2024-09-15T22:07:46.216998+0000","last_undegraded":"2024-09-15T22:07:46.216998+0000","last_fullsized":"2024-09-15T22:07:46.216998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:30.192728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629350+0000","last_change":"2024-09-15T22:06:44.659097+0000","last_active":"2024-09-15T22:07:45.629350+0000","last_peered":"2024-09-15T22:07:45.629350+0000","last_clean":"2024-09-15T22:07:45.629350+0000","last_became_active":"2024-09-15T22:06:44.658626+0000","last_became_peered":"2024-09-15T22:06:44.658626+0000","last_unstale":"2024-09-15T22:07:45.629350+0000","last_undegraded":"2024-09-15T22:07:45.629350+0000","last_fullsized":"2024-09-15T22:07:45.629350+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:01.308004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.967234+0000","last_change":"2024-09-15T22:06:35.136326+0000","last_active":"2024-09-15T22:07:45.967234+0000","last_peered":"2024-09-15T22:07:45.967234+0000","last_clean":"2024-09-15T22:07:45.967234+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:07:45.967234+0000","last_undegraded":"2024-09-15T22:07:45.967234+0000","last_fullsized":"2024-09-15T22:07:45.967234+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:28.675832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632320+0000","last_change":"2024-09-15T22:06:37.192939+0000","last_active":"2024-09-15T22:07:45.632320+0000","last_peered":"2024-09-15T22:07:45.632320+0000","last_clean":"2024-09-15T22:07:45.632320+0000","last_became_active":"2024-09-15T22:06:37.192616+0000","last_became_peered":"2024-09-15T22:06:37.192616+0000","last_unstale":"2024-09-15T22:07:45.632320+0000","last_undegraded":"2024-09-15T22:07:45.632320+0000","last_fullsized":"2024-09-15T22:07:45.632320+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:03.615203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629078+0000","last_change":"2024-09-15T22:06:44.654741+0000","last_active":"2024-09-15T22:07:45.629078+0000","last_peered":"2024-09-15T22:07:45.629078+0000","last_clean":"2024-09-15T22:07:45.629078+0000","last_became_active":"2024-09-15T22:06:44.654362+0000","last_became_peered":"2024-09-15T22:06:44.654362+0000","last_unstale":"2024-09-15T22:07:45.629078+0000","last_undegraded":"2024-09-15T22:07:45.629078+0000","last_fullsized":"2024-09-15T22:07:45.629078+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:59.269948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631095+0000","last_change":"2024-09-15T22:06:37.186385+0000","last_active":"2024-09-15T22:07:45.631095+0000","last_peered":"2024-09-15T22:07:45.631095+0000","last_clean":"2024-09-15T22:07:45.631095+0000","last_became_active":"2024-09-15T22:06:37.185588+0000","last_became_peered":"2024-09-15T22:06:37.185588+0000","last_unstale":"2024-09-15T22:07:45.631095+0000","last_undegraded":"2024-09-15T22:07:45.631095+0000","last_fullsized":"2024-09-15T22:07:45.631095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:22.523083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966841+0000","last_change":"2024-09-15T22:06:35.137383+0000","last_active":"2024-09-15T22:07:45.966841+0000","last_peered":"2024-09-15T22:07:45.966841+0000","last_clean":"2024-09-15T22:07:45.966841+0000","last_became_active":"2024-09-15T22:06:35.137022+0000","last_became_peered":"2024-09-15T22:06:35.137022+0000","last_unstale":"2024-09-15T22:07:45.966841+0000","last_undegraded":"2024-09-15T22:07:45.966841+0000","last_fullsized":"2024-09-15T22:07:45.966841+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.863155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629442+0000","last_change":"2024-09-15T22:06:44.653807+0000","last_active":"2024-09-15T22:07:45.629442+0000","last_peered":"2024-09-15T22:07:45.629442+0000","last_clean":"2024-09-15T22:07:45.629442+0000","last_became_active":"2024-09-15T22:06:44.653406+0000","last_became_peered":"2024-09-15T22:06:44.653406+0000","last_unstale":"2024-09-15T22:07:45.629442+0000","last_undegraded":"2024-09-15T22:07:45.629442+0000","last_fullsized":"2024-09-15T22:07:45.629442+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:12.018208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811325+0000","last_change":"2024-09-15T22:06:37.186899+0000","last_active":"2024-09-15T22:07:45.811325+0000","last_peered":"2024-09-15T22:07:45.811325+0000","last_clean":"2024-09-15T22:07:45.811325+0000","last_became_active":"2024-09-15T22:06:37.183782+0000","last_became_peered":"2024-09-15T22:06:37.183782+0000","last_unstale":"2024-09-15T22:07:45.811325+0000","last_undegraded":"2024-09-15T22:07:45.811325+0000","last_fullsized":"2024-09-15T22:07:45.811325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:39.269186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217605+0000","last_change":"2024-09-15T22:06:44.652686+0000","last_active":"2024-09-15T22:07:46.217605+0000","last_peered":"2024-09-15T22:07:46.217605+0000","last_clean":"2024-09-15T22:07:46.217605+0000","last_became_active":"2024-09-15T22:06:44.652306+0000","last_became_peered":"2024-09-15T22:06:44.652306+0000","last_unstale":"2024-09-15T22:07:46.217605+0000","last_undegraded":"2024-09-15T22:07:46.217605+0000","last_fullsized":"2024-09-15T22:07:46.217605+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629036+0000","last_change":"2024-09-15T22:06:44.655424+0000","last_active":"2024-09-15T22:07:45.629036+0000","last_peered":"2024-09-15T22:07:45.629036+0000","last_clean":"2024-09-15T22:07:45.629036+0000","last_became_active":"2024-09-15T22:06:44.654959+0000","last_became_peered":"2024-09-15T22:06:44.654959+0000","last_unstale":"2024-09-15T22:07:45.629036+0000","last_undegraded":"2024-09-15T22:07:45.629036+0000","last_fullsized":"2024-09-15T22:07:45.629036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:07.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217592+0000","last_change":"2024-09-15T22:06:37.188424+0000","last_active":"2024-09-15T22:07:46.217592+0000","last_peered":"2024-09-15T22:07:46.217592+0000","last_clean":"2024-09-15T22:07:46.217592+0000","last_became_active":"2024-09-15T22:06:37.188082+0000","last_became_peered":"2024-09-15T22:06:37.188082+0000","last_unstale":"2024-09-15T22:07:46.217592+0000","last_undegraded":"2024-09-15T22:07:46.217592+0000","last_fullsized":"2024-09-15T22:07:46.217592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:07.159341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811407+0000","last_change":"2024-09-15T22:06:35.134237+0000","last_active":"2024-09-15T22:07:45.811407+0000","last_peered":"2024-09-15T22:07:45.811407+0000","last_clean":"2024-09-15T22:07:45.811407+0000","last_became_active":"2024-09-15T22:06:35.133885+0000","last_became_peered":"2024-09-15T22:06:35.133885+0000","last_unstale":"2024-09-15T22:07:45.811407+0000","last_undegraded":"2024-09-15T22:07:45.811407+0000","last_fullsized":"2024-09-15T22:07:45.811407+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:07:35.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627325+0000","last_change":"2024-09-15T22:06:44.664578+0000","last_active":"2024-09-15T22:07:45.627325+0000","last_peered":"2024-09-15T22:07:45.627325+0000","last_clean":"2024-09-15T22:07:45.627325+0000","last_became_active":"2024-09-15T22:06:44.664170+0000","last_became_peered":"2024-09-15T22:06:44.664170+0000","last_unstale":"2024-09-15T22:07:45.627325+0000","last_undegraded":"2024-09-15T22:07:45.627325+0000","last_fullsized":"2024-09-15T22:07:45.627325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:41.443816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966606+0000","last_change":"2024-09-15T22:06:37.186107+0000","last_active":"2024-09-15T22:07:45.966606+0000","last_peered":"2024-09-15T22:07:45.966606+0000","last_clean":"2024-09-15T22:07:45.966606+0000","last_became_active":"2024-09-15T22:06:37.185757+0000","last_became_peered":"2024-09-15T22:06:37.185757+0000","last_unstale":"2024-09-15T22:07:45.966606+0000","last_undegraded":"2024-09-15T22:07:45.966606+0000","last_fullsized":"2024-09-15T22:07:45.966606+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:14.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216800+0000","last_change":"2024-09-15T22:06:35.356281+0000","last_active":"2024-09-15T22:07:46.216800+0000","last_peered":"2024-09-15T22:07:46.216800+0000","last_clean":"2024-09-15T22:07:46.216800+0000","last_became_active":"2024-09-15T22:06:35.355922+0000","last_became_peered":"2024-09-15T22:06:35.355922+0000","last_unstale":"2024-09-15T22:07:46.216800+0000","last_undegraded":"2024-09-15T22:07:46.216800+0000","last_fullsized":"2024-09-15T22:07:46.216800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:06.107462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629586+0000","last_change":"2024-09-15T22:06:44.659230+0000","last_active":"2024-09-15T22:07:45.629586+0000","last_peered":"2024-09-15T22:07:45.629586+0000","last_clean":"2024-09-15T22:07:45.629586+0000","last_became_active":"2024-09-15T22:06:44.658649+0000","last_became_peered":"2024-09-15T22:06:44.658649+0000","last_unstale":"2024-09-15T22:07:45.629586+0000","last_undegraded":"2024-09-15T22:07:45.629586+0000","last_fullsized":"2024-09-15T22:07:45.629586+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:14:21.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216942+0000","last_change":"2024-09-15T22:06:44.649394+0000","last_active":"2024-09-15T22:07:46.216942+0000","last_peered":"2024-09-15T22:07:46.216942+0000","last_clean":"2024-09-15T22:07:46.216942+0000","last_became_active":"2024-09-15T22:06:44.649043+0000","last_became_peered":"2024-09-15T22:06:44.649043+0000","last_unstale":"2024-09-15T22:07:46.216942+0000","last_undegraded":"2024-09-15T22:07:46.216942+0000","last_fullsized":"2024-09-15T22:07:46.216942+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:56:28.335869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629894+0000","last_change":"2024-09-15T22:06:35.265970+0000","last_active":"2024-09-15T22:07:45.629894+0000","last_peered":"2024-09-15T22:07:45.629894+0000","last_clean":"2024-09-15T22:07:45.629894+0000","last_became_active":"2024-09-15T22:06:35.265628+0000","last_became_peered":"2024-09-15T22:06:35.265628+0000","last_unstale":"2024-09-15T22:07:45.629894+0000","last_undegraded":"2024-09-15T22:07:45.629894+0000","last_fullsized":"2024-09-15T22:07:45.629894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:41.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630784+0000","last_change":"2024-09-15T22:06:44.649652+0000","last_active":"2024-09-15T22:07:45.630784+0000","last_peered":"2024-09-15T22:07:45.630784+0000","last_clean":"2024-09-15T22:07:45.630784+0000","last_became_active":"2024-09-15T22:06:44.649112+0000","last_became_peered":"2024-09-15T22:06:44.649112+0000","last_unstale":"2024-09-15T22:07:45.630784+0000","last_undegraded":"2024-09-15T22:07:45.630784+0000","last_fullsized":"2024-09-15T22:07:45.630784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:53.789320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811736+0000","last_change":"2024-09-15T22:06:37.190701+0000","last_active":"2024-09-15T22:07:45.811736+0000","last_peered":"2024-09-15T22:07:45.811736+0000","last_clean":"2024-09-15T22:07:45.811736+0000","last_became_active":"2024-09-15T22:06:37.190430+0000","last_became_peered":"2024-09-15T22:06:37.190430+0000","last_unstale":"2024-09-15T22:07:45.811736+0000","last_undegraded":"2024-09-15T22:07:45.811736+0000","last_fullsized":"2024-09-15T22:07:45.811736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:14.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'14","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627467+0000","last_change":"2024-09-15T22:06:35.133899+0000","last_active":"2024-09-15T22:07:45.627467+0000","last_peered":"2024-09-15T22:07:45.627467+0000","last_clean":"2024-09-15T22:07:45.627467+0000","last_became_active":"2024-09-15T22:06:35.133556+0000","last_became_peered":"2024-09-15T22:06:35.133556+0000","last_unstale":"2024-09-15T22:07:45.627467+0000","last_undegraded":"2024-09-15T22:07:45.627467+0000","last_fullsized":"2024-09-15T22:07:45.627467+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:01.692416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811382+0000","last_change":"2024-09-15T22:06:44.655280+0000","last_active":"2024-09-15T22:07:45.811382+0000","last_peered":"2024-09-15T22:07:45.811382+0000","last_clean":"2024-09-15T22:07:45.811382+0000","last_became_active":"2024-09-15T22:06:44.654883+0000","last_became_peered":"2024-09-15T22:06:44.654883+0000","last_unstale":"2024-09-15T22:07:45.811382+0000","last_undegraded":"2024-09-15T22:07:45.811382+0000","last_fullsized":"2024-09-15T22:07:45.811382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.074035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786557+0000","last_change":"2024-09-15T22:06:37.182006+0000","last_active":"2024-09-15T22:07:45.786557+0000","last_peered":"2024-09-15T22:07:45.786557+0000","last_clean":"2024-09-15T22:07:45.786557+0000","last_became_active":"2024-09-15T22:06:37.181693+0000","last_became_peered":"2024-09-15T22:06:37.181693+0000","last_unstale":"2024-09-15T22:07:45.786557+0000","last_undegraded":"2024-09-15T22:07:45.786557+0000","last_fullsized":"2024-09-15T22:07:45.786557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:23.483930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.967289+0000","last_change":"2024-09-15T22:06:35.355854+0000","last_active":"2024-09-15T22:07:45.967289+0000","last_peered":"2024-09-15T22:07:45.967289+0000","last_clean":"2024-09-15T22:07:45.967289+0000","last_became_active":"2024-09-15T22:06:35.355486+0000","last_became_peered":"2024-09-15T22:06:35.355486+0000","last_unstale":"2024-09-15T22:07:45.967289+0000","last_undegraded":"2024-09-15T22:07:45.967289+0000","last_fullsized":"2024-09-15T22:07:45.967289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:36.708476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217530+0000","last_change":"2024-09-15T22:06:44.660523+0000","last_active":"2024-09-15T22:07:46.217530+0000","last_peered":"2024-09-15T22:07:46.217530+0000","last_clean":"2024-09-15T22:07:46.217530+0000","last_became_active":"2024-09-15T22:06:44.660113+0000","last_became_peered":"2024-09-15T22:06:44.660113+0000","last_unstale":"2024-09-15T22:07:46.217530+0000","last_undegraded":"2024-09-15T22:07:46.217530+0000","last_fullsized":"2024-09-15T22:07:46.217530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:42.164558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627691+0000","last_change":"2024-09-15T22:06:37.183649+0000","last_active":"2024-09-15T22:07:45.627691+0000","last_peered":"2024-09-15T22:07:45.627691+0000","last_clean":"2024-09-15T22:07:45.627691+0000","last_became_active":"2024-09-15T22:06:37.183296+0000","last_became_peered":"2024-09-15T22:06:37.183296+0000","last_unstale":"2024-09-15T22:07:45.627691+0000","last_undegraded":"2024-09-15T22:07:45.627691+0000","last_fullsized":"2024-09-15T22:07:45.627691+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:22.010288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627247+0000","last_change":"2024-09-15T22:06:35.355068+0000","last_active":"2024-09-15T22:07:45.627247+0000","last_peered":"2024-09-15T22:07:45.627247+0000","last_clean":"2024-09-15T22:07:45.627247+0000","last_became_active":"2024-09-15T22:06:35.354728+0000","last_became_peered":"2024-09-15T22:06:35.354728+0000","last_unstale":"2024-09-15T22:07:45.627247+0000","last_undegraded":"2024-09-15T22:07:45.627247+0000","last_fullsized":"2024-09-15T22:07:45.627247+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:35.479290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966634+0000","last_change":"2024-09-15T22:06:44.653849+0000","last_active":"2024-09-15T22:07:45.966634+0000","last_peered":"2024-09-15T22:07:45.966634+0000","last_clean":"2024-09-15T22:07:45.966634+0000","last_became_active":"2024-09-15T22:06:44.653419+0000","last_became_peered":"2024-09-15T22:06:44.653419+0000","last_unstale":"2024-09-15T22:07:45.966634+0000","last_undegraded":"2024-09-15T22:07:45.966634+0000","last_fullsized":"2024-09-15T22:07:45.966634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:04.131553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627824+0000","last_change":"2024-09-15T22:06:37.191438+0000","last_active":"2024-09-15T22:07:45.627824+0000","last_peered":"2024-09-15T22:07:45.627824+0000","last_clean":"2024-09-15T22:07:45.627824+0000","last_became_active":"2024-09-15T22:06:37.191088+0000","last_became_peered":"2024-09-15T22:06:37.191088+0000","last_unstale":"2024-09-15T22:07:45.627824+0000","last_undegraded":"2024-09-15T22:07:45.627824+0000","last_fullsized":"2024-09-15T22:07:45.627824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:46:36.078872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631155+0000","last_change":"2024-09-15T22:06:35.135866+0000","last_active":"2024-09-15T22:07:45.631155+0000","last_peered":"2024-09-15T22:07:45.631155+0000","last_clean":"2024-09-15T22:07:45.631155+0000","last_became_active":"2024-09-15T22:06:35.135524+0000","last_became_peered":"2024-09-15T22:06:35.135524+0000","last_unstale":"2024-09-15T22:07:45.631155+0000","last_undegraded":"2024-09-15T22:07:45.631155+0000","last_fullsized":"2024-09-15T22:07:45.631155+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786408+0000","last_change":"2024-09-15T22:06:44.649943+0000","last_active":"2024-09-15T22:07:45.786408+0000","last_peered":"2024-09-15T22:07:45.786408+0000","last_clean":"2024-09-15T22:07:45.786408+0000","last_became_active":"2024-09-15T22:06:44.649579+0000","last_became_peered":"2024-09-15T22:06:44.649579+0000","last_unstale":"2024-09-15T22:07:45.786408+0000","last_undegraded":"2024-09-15T22:07:45.786408+0000","last_fullsized":"2024-09-15T22:07:45.786408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:18.390608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631303+0000","last_change":"2024-09-15T22:06:37.189988+0000","last_active":"2024-09-15T22:07:45.631303+0000","last_peered":"2024-09-15T22:07:45.631303+0000","last_clean":"2024-09-15T22:07:45.631303+0000","last_became_active":"2024-09-15T22:06:37.189319+0000","last_became_peered":"2024-09-15T22:06:37.189319+0000","last_unstale":"2024-09-15T22:07:45.631303+0000","last_undegraded":"2024-09-15T22:07:45.631303+0000","last_fullsized":"2024-09-15T22:07:45.631303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:08.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":92,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:08:06.818441+0000","last_change":"2024-09-15T22:06:35.355371+0000","last_active":"2024-09-15T22:08:06.818441+0000","last_peered":"2024-09-15T22:08:06.818441+0000","last_clean":"2024-09-15T22:08:06.818441+0000","last_became_active":"2024-09-15T22:06:35.354687+0000","last_became_peered":"2024-09-15T22:06:35.354687+0000","last_unstale":"2024-09-15T22:08:06.818441+0000","last_undegraded":"2024-09-15T22:08:06.818441+0000","last_fullsized":"2024-09-15T22:08:06.818441+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:27.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786580+0000","last_change":"2024-09-15T22:06:44.657929+0000","last_active":"2024-09-15T22:07:45.786580+0000","last_peered":"2024-09-15T22:07:45.786580+0000","last_clean":"2024-09-15T22:07:45.786580+0000","last_became_active":"2024-09-15T22:06:44.657519+0000","last_became_peered":"2024-09-15T22:06:44.657519+0000","last_unstale":"2024-09-15T22:07:45.786580+0000","last_undegraded":"2024-09-15T22:07:45.786580+0000","last_fullsized":"2024-09-15T22:07:45.786580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:12.313877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.785906+0000","last_change":"2024-09-15T22:06:37.193288+0000","last_active":"2024-09-15T22:07:45.785906+0000","last_peered":"2024-09-15T22:07:45.785906+0000","last_clean":"2024-09-15T22:07:45.785906+0000","last_became_active":"2024-09-15T22:06:37.193007+0000","last_became_peered":"2024-09-15T22:06:37.193007+0000","last_unstale":"2024-09-15T22:07:45.785906+0000","last_undegraded":"2024-09-15T22:07:45.785906+0000","last_fullsized":"2024-09-15T22:07:45.785906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:45.956206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630622+0000","last_change":"2024-09-15T22:06:35.138726+0000","last_active":"2024-09-15T22:07:45.630622+0000","last_peered":"2024-09-15T22:07:45.630622+0000","last_clean":"2024-09-15T22:07:45.630622+0000","last_became_active":"2024-09-15T22:06:35.138390+0000","last_became_peered":"2024-09-15T22:06:35.138390+0000","last_unstale":"2024-09-15T22:07:45.630622+0000","last_undegraded":"2024-09-15T22:07:45.630622+0000","last_fullsized":"2024-09-15T22:07:45.630622+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:51.336171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.219720+0000","last_change":"2024-09-15T22:07:46.219720+0000","last_active":"2024-09-15T22:07:46.219720+0000","last_peered":"2024-09-15T22:07:46.219720+0000","last_clean":"2024-09-15T22:07:46.219720+0000","last_became_active":"2024-09-15T22:07:46.219303+0000","last_became_peered":"2024-09-15T22:07:46.219303+0000","last_unstale":"2024-09-15T22:07:46.219720+0000","last_undegraded":"2024-09-15T22:07:46.219720+0000","last_fullsized":"2024-09-15T22:07:46.219720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:03.596795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811307+0000","last_change":"2024-09-15T22:06:37.176478+0000","last_active":"2024-09-15T22:07:45.811307+0000","last_peered":"2024-09-15T22:07:45.811307+0000","last_clean":"2024-09-15T22:07:45.811307+0000","last_became_active":"2024-09-15T22:06:37.176171+0000","last_became_peered":"2024-09-15T22:06:37.176171+0000","last_unstale":"2024-09-15T22:07:45.811307+0000","last_undegraded":"2024-09-15T22:07:45.811307+0000","last_fullsized":"2024-09-15T22:07:45.811307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629800+0000","last_change":"2024-09-15T22:06:35.134651+0000","last_active":"2024-09-15T22:07:45.629800+0000","last_peered":"2024-09-15T22:07:45.629800+0000","last_clean":"2024-09-15T22:07:45.629800+0000","last_became_active":"2024-09-15T22:06:35.134185+0000","last_became_peered":"2024-09-15T22:06:35.134185+0000","last_unstale":"2024-09-15T22:07:45.629800+0000","last_undegraded":"2024-09-15T22:07:45.629800+0000","last_fullsized":"2024-09-15T22:07:45.629800+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:56.385046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":356,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630845+0000","last_change":"2024-09-15T22:05:23.048068+0000","last_active":"2024-09-15T22:07:45.630845+0000","last_peered":"2024-09-15T22:07:45.630845+0000","last_clean":"2024-09-15T22:07:45.630845+0000","last_became_active":"2024-09-15T22:05:23.047692+0000","last_became_peered":"2024-09-15T22:05:23.047692+0000","last_unstale":"2024-09-15T22:07:45.630845+0000","last_undegraded":"2024-09-15T22:07:45.630845+0000","last_fullsized":"2024-09-15T22:07:45.630845+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_clean_scrub_stamp":"2024-09-15T22:01:57.477629+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631277+0000","last_change":"2024-09-15T22:06:44.657213+0000","last_active":"2024-09-15T22:07:45.631277+0000","last_peered":"2024-09-15T22:07:45.631277+0000","last_clean":"2024-09-15T22:07:45.631277+0000","last_became_active":"2024-09-15T22:06:44.656815+0000","last_became_peered":"2024-09-15T22:06:44.656815+0000","last_unstale":"2024-09-15T22:07:45.631277+0000","last_undegraded":"2024-09-15T22:07:45.631277+0000","last_fullsized":"2024-09-15T22:07:45.631277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:26.525992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631397+0000","last_change":"2024-09-15T22:06:37.184467+0000","last_active":"2024-09-15T22:07:45.631397+0000","last_peered":"2024-09-15T22:07:45.631397+0000","last_clean":"2024-09-15T22:07:45.631397+0000","last_became_active":"2024-09-15T22:06:37.183864+0000","last_became_peered":"2024-09-15T22:06:37.183864+0000","last_unstale":"2024-09-15T22:07:45.631397+0000","last_undegraded":"2024-09-15T22:07:45.631397+0000","last_fullsized":"2024-09-15T22:07:45.631397+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:35.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216954+0000","last_change":"2024-09-15T22:06:44.651594+0000","last_active":"2024-09-15T22:07:46.216954+0000","last_peered":"2024-09-15T22:07:46.216954+0000","last_clean":"2024-09-15T22:07:46.216954+0000","last_became_active":"2024-09-15T22:06:44.650393+0000","last_became_peered":"2024-09-15T22:06:44.650393+0000","last_unstale":"2024-09-15T22:07:46.216954+0000","last_undegraded":"2024-09-15T22:07:46.216954+0000","last_fullsized":"2024-09-15T22:07:46.216954+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:13.599744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216738+0000","last_change":"2024-09-15T22:06:44.660789+0000","last_active":"2024-09-15T22:07:46.216738+0000","last_peered":"2024-09-15T22:07:46.216738+0000","last_clean":"2024-09-15T22:07:46.216738+0000","last_became_active":"2024-09-15T22:06:44.660331+0000","last_became_peered":"2024-09-15T22:06:44.660331+0000","last_unstale":"2024-09-15T22:07:46.216738+0000","last_undegraded":"2024-09-15T22:07:46.216738+0000","last_fullsized":"2024-09-15T22:07:46.216738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:04:45.562795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631098+0000","last_change":"2024-09-15T22:06:37.185697+0000","last_active":"2024-09-15T22:07:45.631098+0000","last_peered":"2024-09-15T22:07:45.631098+0000","last_clean":"2024-09-15T22:07:45.631098+0000","last_became_active":"2024-09-15T22:06:37.185207+0000","last_became_peered":"2024-09-15T22:06:37.185207+0000","last_unstale":"2024-09-15T22:07:45.631098+0000","last_undegraded":"2024-09-15T22:07:45.631098+0000","last_fullsized":"2024-09-15T22:07:45.631098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:39.005273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632692+0000","last_change":"2024-09-15T22:06:35.135176+0000","last_active":"2024-09-15T22:07:45.632692+0000","last_peered":"2024-09-15T22:07:45.632692+0000","last_clean":"2024-09-15T22:07:45.632692+0000","last_became_active":"2024-09-15T22:06:35.134649+0000","last_became_peered":"2024-09-15T22:06:35.134649+0000","last_unstale":"2024-09-15T22:07:45.632692+0000","last_undegraded":"2024-09-15T22:07:45.632692+0000","last_fullsized":"2024-09-15T22:07:45.632692+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:33.780162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216836+0000","last_change":"2024-09-15T22:06:44.656972+0000","last_active":"2024-09-15T22:07:46.216836+0000","last_peered":"2024-09-15T22:07:46.216836+0000","last_clean":"2024-09-15T22:07:46.216836+0000","last_became_active":"2024-09-15T22:06:44.656528+0000","last_became_peered":"2024-09-15T22:06:44.656528+0000","last_unstale":"2024-09-15T22:07:46.216836+0000","last_undegraded":"2024-09-15T22:07:46.216836+0000","last_fullsized":"2024-09-15T22:07:46.216836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:09.164596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630362+0000","last_change":"2024-09-15T22:06:37.186118+0000","last_active":"2024-09-15T22:07:45.630362+0000","last_peered":"2024-09-15T22:07:45.630362+0000","last_clean":"2024-09-15T22:07:45.630362+0000","last_became_active":"2024-09-15T22:06:37.185118+0000","last_became_peered":"2024-09-15T22:06:37.185118+0000","last_unstale":"2024-09-15T22:07:45.630362+0000","last_undegraded":"2024-09-15T22:07:45.630362+0000","last_fullsized":"2024-09-15T22:07:45.630362+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:53.357045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786815+0000","last_change":"2024-09-15T22:06:35.133341+0000","last_active":"2024-09-15T22:07:45.786815+0000","last_peered":"2024-09-15T22:07:45.786815+0000","last_clean":"2024-09-15T22:07:45.786815+0000","last_became_active":"2024-09-15T22:06:35.132961+0000","last_became_peered":"2024-09-15T22:06:35.132961+0000","last_unstale":"2024-09-15T22:07:45.786815+0000","last_undegraded":"2024-09-15T22:07:45.786815+0000","last_fullsized":"2024-09-15T22:07:45.786815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:39.579470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630891+0000","last_change":"2024-09-15T22:06:44.659148+0000","last_active":"2024-09-15T22:07:45.630891+0000","last_peered":"2024-09-15T22:07:45.630891+0000","last_clean":"2024-09-15T22:07:45.630891+0000","last_became_active":"2024-09-15T22:06:44.658750+0000","last_became_peered":"2024-09-15T22:06:44.658750+0000","last_unstale":"2024-09-15T22:07:45.630891+0000","last_undegraded":"2024-09-15T22:07:45.630891+0000","last_fullsized":"2024-09-15T22:07:45.630891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:48.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.218042+0000","last_change":"2024-09-15T22:06:37.194066+0000","last_active":"2024-09-15T22:07:46.218042+0000","last_peered":"2024-09-15T22:07:46.218042+0000","last_clean":"2024-09-15T22:07:46.218042+0000","last_became_active":"2024-09-15T22:06:37.193738+0000","last_became_peered":"2024-09-15T22:06:37.193738+0000","last_unstale":"2024-09-15T22:07:46.218042+0000","last_undegraded":"2024-09-15T22:07:46.218042+0000","last_fullsized":"2024-09-15T22:07:46.218042+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.107188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.967402+0000","last_change":"2024-09-15T22:06:35.136470+0000","last_active":"2024-09-15T22:07:45.967402+0000","last_peered":"2024-09-15T22:07:45.967402+0000","last_clean":"2024-09-15T22:07:45.967402+0000","last_became_active":"2024-09-15T22:06:35.135953+0000","last_became_peered":"2024-09-15T22:06:35.135953+0000","last_unstale":"2024-09-15T22:07:45.967402+0000","last_undegraded":"2024-09-15T22:07:45.967402+0000","last_fullsized":"2024-09-15T22:07:45.967402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:44.548566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631818+0000","last_change":"2024-09-15T22:06:44.653682+0000","last_active":"2024-09-15T22:07:45.631818+0000","last_peered":"2024-09-15T22:07:45.631818+0000","last_clean":"2024-09-15T22:07:45.631818+0000","last_became_active":"2024-09-15T22:06:44.653303+0000","last_became_peered":"2024-09-15T22:06:44.653303+0000","last_unstale":"2024-09-15T22:07:45.631818+0000","last_undegraded":"2024-09-15T22:07:45.631818+0000","last_fullsized":"2024-09-15T22:07:45.631818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:07.847971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631929+0000","last_change":"2024-09-15T22:06:44.658436+0000","last_active":"2024-09-15T22:07:45.631929+0000","last_peered":"2024-09-15T22:07:45.631929+0000","last_clean":"2024-09-15T22:07:45.631929+0000","last_became_active":"2024-09-15T22:06:44.658037+0000","last_became_peered":"2024-09-15T22:06:44.658037+0000","last_unstale":"2024-09-15T22:07:45.631929+0000","last_undegraded":"2024-09-15T22:07:45.631929+0000","last_fullsized":"2024-09-15T22:07:45.631929+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:34:20.369249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966897+0000","last_change":"2024-09-15T22:06:35.358018+0000","last_active":"2024-09-15T22:07:45.966897+0000","last_peered":"2024-09-15T22:07:45.966897+0000","last_clean":"2024-09-15T22:07:45.966897+0000","last_became_active":"2024-09-15T22:06:35.357666+0000","last_became_peered":"2024-09-15T22:06:35.357666+0000","last_unstale":"2024-09-15T22:07:45.966897+0000","last_undegraded":"2024-09-15T22:07:45.966897+0000","last_fullsized":"2024-09-15T22:07:45.966897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:34.089113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786697+0000","last_change":"2024-09-15T22:06:44.656807+0000","last_active":"2024-09-15T22:07:45.786697+0000","last_peered":"2024-09-15T22:07:45.786697+0000","last_clean":"2024-09-15T22:07:45.786697+0000","last_became_active":"2024-09-15T22:06:44.656409+0000","last_became_peered":"2024-09-15T22:06:44.656409+0000","last_unstale":"2024-09-15T22:07:45.786697+0000","last_undegraded":"2024-09-15T22:07:45.786697+0000","last_fullsized":"2024-09-15T22:07:45.786697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630373+0000","last_change":"2024-09-15T22:06:37.180615+0000","last_active":"2024-09-15T22:07:45.630373+0000","last_peered":"2024-09-15T22:07:45.630373+0000","last_clean":"2024-09-15T22:07:45.630373+0000","last_became_active":"2024-09-15T22:06:37.180306+0000","last_became_peered":"2024-09-15T22:06:37.180306+0000","last_unstale":"2024-09-15T22:07:45.630373+0000","last_undegraded":"2024-09-15T22:07:45.630373+0000","last_fullsized":"2024-09-15T22:07:45.630373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:31.145128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'4","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:08:06.819117+0000","last_change":"2024-09-15T22:06:35.136192+0000","last_active":"2024-09-15T22:08:06.819117+0000","last_peered":"2024-09-15T22:08:06.819117+0000","last_clean":"2024-09-15T22:08:06.819117+0000","last_became_active":"2024-09-15T22:06:35.135608+0000","last_became_peered":"2024-09-15T22:06:35.135608+0000","last_unstale":"2024-09-15T22:08:06.819117+0000","last_undegraded":"2024-09-15T22:08:06.819117+0000","last_fullsized":"2024-09-15T22:08:06.819117+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:48.472717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627405+0000","last_change":"2024-09-15T22:06:44.664763+0000","last_active":"2024-09-15T22:07:45.627405+0000","last_peered":"2024-09-15T22:07:45.627405+0000","last_clean":"2024-09-15T22:07:45.627405+0000","last_became_active":"2024-09-15T22:06:44.664410+0000","last_became_peered":"2024-09-15T22:06:44.664410+0000","last_unstale":"2024-09-15T22:07:45.627405+0000","last_undegraded":"2024-09-15T22:07:45.627405+0000","last_fullsized":"2024-09-15T22:07:45.627405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:14.364387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631277+0000","last_change":"2024-09-15T22:06:44.652747+0000","last_active":"2024-09-15T22:07:45.631277+0000","last_peered":"2024-09-15T22:07:45.631277+0000","last_clean":"2024-09-15T22:07:45.631277+0000","last_became_active":"2024-09-15T22:06:44.652368+0000","last_became_peered":"2024-09-15T22:06:44.652368+0000","last_unstale":"2024-09-15T22:07:45.631277+0000","last_undegraded":"2024-09-15T22:07:45.631277+0000","last_fullsized":"2024-09-15T22:07:45.631277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:51.047549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632221+0000","last_change":"2024-09-15T22:06:35.136311+0000","last_active":"2024-09-15T22:07:45.632221+0000","last_peered":"2024-09-15T22:07:45.632221+0000","last_clean":"2024-09-15T22:07:45.632221+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:07:45.632221+0000","last_undegraded":"2024-09-15T22:07:45.632221+0000","last_fullsized":"2024-09-15T22:07:45.632221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:36:07.138056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217545+0000","last_change":"2024-09-15T22:06:44.653747+0000","last_active":"2024-09-15T22:07:46.217545+0000","last_peered":"2024-09-15T22:07:46.217545+0000","last_clean":"2024-09-15T22:07:46.217545+0000","last_became_active":"2024-09-15T22:06:44.653357+0000","last_became_peered":"2024-09-15T22:06:44.653357+0000","last_unstale":"2024-09-15T22:07:46.217545+0000","last_undegraded":"2024-09-15T22:07:46.217545+0000","last_fullsized":"2024-09-15T22:07:46.217545+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.113924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786191+0000","last_change":"2024-09-15T22:06:37.178308+0000","last_active":"2024-09-15T22:07:45.786191+0000","last_peered":"2024-09-15T22:07:45.786191+0000","last_clean":"2024-09-15T22:07:45.786191+0000","last_became_active":"2024-09-15T22:06:37.177920+0000","last_became_peered":"2024-09-15T22:06:37.177920+0000","last_unstale":"2024-09-15T22:07:45.786191+0000","last_undegraded":"2024-09-15T22:07:45.786191+0000","last_fullsized":"2024-09-15T22:07:45.786191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631690+0000","last_change":"2024-09-15T22:06:35.136132+0000","last_active":"2024-09-15T22:07:45.631690+0000","last_peered":"2024-09-15T22:07:45.631690+0000","last_clean":"2024-09-15T22:07:45.631690+0000","last_became_active":"2024-09-15T22:06:35.135789+0000","last_became_peered":"2024-09-15T22:06:35.135789+0000","last_unstale":"2024-09-15T22:07:45.631690+0000","last_undegraded":"2024-09-15T22:07:45.631690+0000","last_fullsized":"2024-09-15T22:07:45.631690+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:01.692885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632214+0000","last_change":"2024-09-15T22:06:44.658920+0000","last_active":"2024-09-15T22:07:45.632214+0000","last_peered":"2024-09-15T22:07:45.632214+0000","last_clean":"2024-09-15T22:07:45.632214+0000","last_became_active":"2024-09-15T22:06:44.658434+0000","last_became_peered":"2024-09-15T22:06:44.658434+0000","last_unstale":"2024-09-15T22:07:45.632214+0000","last_undegraded":"2024-09-15T22:07:45.632214+0000","last_fullsized":"2024-09-15T22:07:45.632214+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:37.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.967121+0000","last_change":"2024-09-15T22:06:37.184958+0000","last_active":"2024-09-15T22:07:45.967121+0000","last_peered":"2024-09-15T22:07:45.967121+0000","last_clean":"2024-09-15T22:07:45.967121+0000","last_became_active":"2024-09-15T22:06:37.184648+0000","last_became_peered":"2024-09-15T22:06:37.184648+0000","last_unstale":"2024-09-15T22:07:45.967121+0000","last_undegraded":"2024-09-15T22:07:45.967121+0000","last_fullsized":"2024-09-15T22:07:45.967121+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:28.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"86'33","reported_seq":188,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:08:09.823322+0000","last_change":"2024-09-15T22:06:35.138533+0000","last_active":"2024-09-15T22:08:09.823322+0000","last_peered":"2024-09-15T22:08:09.823322+0000","last_clean":"2024-09-15T22:08:09.823322+0000","last_became_active":"2024-09-15T22:06:35.138142+0000","last_became_peered":"2024-09-15T22:06:35.138142+0000","last_unstale":"2024-09-15T22:08:09.823322+0000","last_undegraded":"2024-09-15T22:08:09.823322+0000","last_fullsized":"2024-09-15T22:08:09.823322+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:32.950036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":34,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.218222+0000","last_change":"2024-09-15T22:07:46.218222+0000","last_active":"2024-09-15T22:07:46.218222+0000","last_peered":"2024-09-15T22:07:46.218222+0000","last_clean":"2024-09-15T22:07:46.218222+0000","last_became_active":"2024-09-15T22:07:46.217903+0000","last_became_peered":"2024-09-15T22:07:46.217903+0000","last_unstale":"2024-09-15T22:07:46.218222+0000","last_undegraded":"2024-09-15T22:07:46.218222+0000","last_fullsized":"2024-09-15T22:07:46.218222+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:17.730288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217066+0000","last_change":"2024-09-15T22:06:37.194297+0000","last_active":"2024-09-15T22:07:46.217066+0000","last_peered":"2024-09-15T22:07:46.217066+0000","last_clean":"2024-09-15T22:07:46.217066+0000","last_became_active":"2024-09-15T22:06:37.193986+0000","last_became_peered":"2024-09-15T22:06:37.193986+0000","last_unstale":"2024-09-15T22:07:46.217066+0000","last_undegraded":"2024-09-15T22:07:46.217066+0000","last_fullsized":"2024-09-15T22:07:46.217066+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:27.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632496+0000","last_change":"2024-09-15T22:06:35.356537+0000","last_active":"2024-09-15T22:07:45.632496+0000","last_peered":"2024-09-15T22:07:45.632496+0000","last_clean":"2024-09-15T22:07:45.632496+0000","last_became_active":"2024-09-15T22:06:35.355997+0000","last_became_peered":"2024-09-15T22:06:35.355997+0000","last_unstale":"2024-09-15T22:07:45.632496+0000","last_undegraded":"2024-09-15T22:07:45.632496+0000","last_fullsized":"2024-09-15T22:07:45.632496+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:11.563083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629628+0000","last_change":"2024-09-15T22:06:44.654579+0000","last_active":"2024-09-15T22:07:45.629628+0000","last_peered":"2024-09-15T22:07:45.629628+0000","last_clean":"2024-09-15T22:07:45.629628+0000","last_became_active":"2024-09-15T22:06:44.654187+0000","last_became_peered":"2024-09-15T22:06:44.654187+0000","last_unstale":"2024-09-15T22:07:45.629628+0000","last_undegraded":"2024-09-15T22:07:45.629628+0000","last_fullsized":"2024-09-15T22:07:45.629628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:15.617528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217296+0000","last_change":"2024-09-15T22:06:44.661533+0000","last_active":"2024-09-15T22:07:46.217296+0000","last_peered":"2024-09-15T22:07:46.217296+0000","last_clean":"2024-09-15T22:07:46.217296+0000","last_became_active":"2024-09-15T22:06:44.661193+0000","last_became_peered":"2024-09-15T22:06:44.661193+0000","last_unstale":"2024-09-15T22:07:46.217296+0000","last_undegraded":"2024-09-15T22:07:46.217296+0000","last_fullsized":"2024-09-15T22:07:46.217296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:06.559720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629989+0000","last_change":"2024-09-15T22:06:35.140578+0000","last_active":"2024-09-15T22:07:45.629989+0000","last_peered":"2024-09-15T22:07:45.629989+0000","last_clean":"2024-09-15T22:07:45.629989+0000","last_became_active":"2024-09-15T22:06:35.139818+0000","last_became_peered":"2024-09-15T22:06:35.139818+0000","last_unstale":"2024-09-15T22:07:45.629989+0000","last_undegraded":"2024-09-15T22:07:45.629989+0000","last_fullsized":"2024-09-15T22:07:45.629989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:59.806964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.626672+0000","last_change":"2024-09-15T22:06:44.654755+0000","last_active":"2024-09-15T22:07:45.626672+0000","last_peered":"2024-09-15T22:07:45.626672+0000","last_clean":"2024-09-15T22:07:45.626672+0000","last_became_active":"2024-09-15T22:06:44.654390+0000","last_became_peered":"2024-09-15T22:06:44.654390+0000","last_unstale":"2024-09-15T22:07:45.626672+0000","last_undegraded":"2024-09-15T22:07:45.626672+0000","last_fullsized":"2024-09-15T22:07:45.626672+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:35.781878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217329+0000","last_change":"2024-09-15T22:06:37.182457+0000","last_active":"2024-09-15T22:07:46.217329+0000","last_peered":"2024-09-15T22:07:46.217329+0000","last_clean":"2024-09-15T22:07:46.217329+0000","last_became_active":"2024-09-15T22:06:37.182106+0000","last_became_peered":"2024-09-15T22:06:37.182106+0000","last_unstale":"2024-09-15T22:07:46.217329+0000","last_undegraded":"2024-09-15T22:07:46.217329+0000","last_fullsized":"2024-09-15T22:07:46.217329+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:57.560894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:08:08.821795+0000","last_change":"2024-09-15T22:06:35.140590+0000","last_active":"2024-09-15T22:08:08.821795+0000","last_peered":"2024-09-15T22:08:08.821795+0000","last_clean":"2024-09-15T22:08:08.821795+0000","last_became_active":"2024-09-15T22:06:35.140198+0000","last_became_peered":"2024-09-15T22:06:35.140198+0000","last_unstale":"2024-09-15T22:08:08.821795+0000","last_undegraded":"2024-09-15T22:08:08.821795+0000","last_fullsized":"2024-09-15T22:08:08.821795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:46.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811193+0000","last_change":"2024-09-15T22:06:44.660188+0000","last_active":"2024-09-15T22:07:45.811193+0000","last_peered":"2024-09-15T22:07:45.811193+0000","last_clean":"2024-09-15T22:07:45.811193+0000","last_became_active":"2024-09-15T22:06:44.659808+0000","last_became_peered":"2024-09-15T22:06:44.659808+0000","last_unstale":"2024-09-15T22:07:45.811193+0000","last_undegraded":"2024-09-15T22:07:45.811193+0000","last_fullsized":"2024-09-15T22:07:45.811193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:39.473765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216787+0000","last_change":"2024-09-15T22:06:44.647967+0000","last_active":"2024-09-15T22:07:46.216787+0000","last_peered":"2024-09-15T22:07:46.216787+0000","last_clean":"2024-09-15T22:07:46.216787+0000","last_became_active":"2024-09-15T22:06:44.647608+0000","last_became_peered":"2024-09-15T22:06:44.647608+0000","last_unstale":"2024-09-15T22:07:46.216787+0000","last_undegraded":"2024-09-15T22:07:46.216787+0000","last_fullsized":"2024-09-15T22:07:46.216787+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:06.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.626850+0000","last_change":"2024-09-15T22:06:37.187020+0000","last_active":"2024-09-15T22:07:45.626850+0000","last_peered":"2024-09-15T22:07:45.626850+0000","last_clean":"2024-09-15T22:07:45.626850+0000","last_became_active":"2024-09-15T22:06:37.186564+0000","last_became_peered":"2024-09-15T22:06:37.186564+0000","last_unstale":"2024-09-15T22:07:45.626850+0000","last_undegraded":"2024-09-15T22:07:45.626850+0000","last_fullsized":"2024-09-15T22:07:45.626850+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:20.586763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631462+0000","last_change":"2024-09-15T22:06:35.135126+0000","last_active":"2024-09-15T22:07:45.631462+0000","last_peered":"2024-09-15T22:07:45.631462+0000","last_clean":"2024-09-15T22:07:45.631462+0000","last_became_active":"2024-09-15T22:06:35.134783+0000","last_became_peered":"2024-09-15T22:06:35.134783+0000","last_unstale":"2024-09-15T22:07:45.631462+0000","last_undegraded":"2024-09-15T22:07:45.631462+0000","last_fullsized":"2024-09-15T22:07:45.631462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:31.910988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.218056+0000","last_change":"2024-09-15T22:06:44.656981+0000","last_active":"2024-09-15T22:07:46.218056+0000","last_peered":"2024-09-15T22:07:46.218056+0000","last_clean":"2024-09-15T22:07:46.218056+0000","last_became_active":"2024-09-15T22:06:44.656477+0000","last_became_peered":"2024-09-15T22:06:44.656477+0000","last_unstale":"2024-09-15T22:07:46.218056+0000","last_undegraded":"2024-09-15T22:07:46.218056+0000","last_fullsized":"2024-09-15T22:07:46.218056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:06.153490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811588+0000","last_change":"2024-09-15T22:06:37.183034+0000","last_active":"2024-09-15T22:07:45.811588+0000","last_peered":"2024-09-15T22:07:45.811588+0000","last_clean":"2024-09-15T22:07:45.811588+0000","last_became_active":"2024-09-15T22:06:37.182734+0000","last_became_peered":"2024-09-15T22:06:37.182734+0000","last_unstale":"2024-09-15T22:07:45.811588+0000","last_undegraded":"2024-09-15T22:07:45.811588+0000","last_fullsized":"2024-09-15T22:07:45.811588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:19.352000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.785911+0000","last_change":"2024-09-15T22:06:35.134954+0000","last_active":"2024-09-15T22:07:45.785911+0000","last_peered":"2024-09-15T22:07:45.785911+0000","last_clean":"2024-09-15T22:07:45.785911+0000","last_became_active":"2024-09-15T22:06:35.134583+0000","last_became_peered":"2024-09-15T22:06:35.134583+0000","last_unstale":"2024-09-15T22:07:45.785911+0000","last_undegraded":"2024-09-15T22:07:45.785911+0000","last_fullsized":"2024-09-15T22:07:45.785911+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:56.082123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.628342+0000","last_change":"2024-09-15T22:06:35.139229+0000","last_active":"2024-09-15T22:07:45.628342+0000","last_peered":"2024-09-15T22:07:45.628342+0000","last_clean":"2024-09-15T22:07:45.628342+0000","last_became_active":"2024-09-15T22:06:35.138834+0000","last_became_peered":"2024-09-15T22:06:35.138834+0000","last_unstale":"2024-09-15T22:07:45.628342+0000","last_undegraded":"2024-09-15T22:07:45.628342+0000","last_fullsized":"2024-09-15T22:07:45.628342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:05.877404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.632504+0000","last_change":"2024-09-15T22:06:37.192212+0000","last_active":"2024-09-15T22:07:45.632504+0000","last_peered":"2024-09-15T22:07:45.632504+0000","last_clean":"2024-09-15T22:07:45.632504+0000","last_became_active":"2024-09-15T22:06:37.191872+0000","last_became_peered":"2024-09-15T22:06:37.191872+0000","last_unstale":"2024-09-15T22:07:45.632504+0000","last_undegraded":"2024-09-15T22:07:45.632504+0000","last_fullsized":"2024-09-15T22:07:45.632504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:32.845794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.968398+0000","last_change":"2024-09-15T22:07:45.968398+0000","last_active":"2024-09-15T22:07:45.968398+0000","last_peered":"2024-09-15T22:07:45.968398+0000","last_clean":"2024-09-15T22:07:45.968398+0000","last_became_active":"2024-09-15T22:07:45.967912+0000","last_became_peered":"2024-09-15T22:07:45.967912+0000","last_unstale":"2024-09-15T22:07:45.968398+0000","last_undegraded":"2024-09-15T22:07:45.968398+0000","last_fullsized":"2024-09-15T22:07:45.968398+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:59.909290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:08:05.818159+0000","last_change":"2024-09-15T22:06:35.133846+0000","last_active":"2024-09-15T22:08:05.818159+0000","last_peered":"2024-09-15T22:08:05.818159+0000","last_clean":"2024-09-15T22:08:05.818159+0000","last_became_active":"2024-09-15T22:06:35.133516+0000","last_became_peered":"2024-09-15T22:06:35.133516+0000","last_unstale":"2024-09-15T22:08:05.818159+0000","last_undegraded":"2024-09-15T22:08:05.818159+0000","last_fullsized":"2024-09-15T22:08:05.818159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:29.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966752+0000","last_change":"2024-09-15T22:06:37.182904+0000","last_active":"2024-09-15T22:07:45.966752+0000","last_peered":"2024-09-15T22:07:45.966752+0000","last_clean":"2024-09-15T22:07:45.966752+0000","last_became_active":"2024-09-15T22:06:37.182563+0000","last_became_peered":"2024-09-15T22:06:37.182563+0000","last_unstale":"2024-09-15T22:07:45.966752+0000","last_undegraded":"2024-09-15T22:07:45.966752+0000","last_fullsized":"2024-09-15T22:07:45.966752+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631298+0000","last_change":"2024-09-15T22:06:44.660997+0000","last_active":"2024-09-15T22:07:45.631298+0000","last_peered":"2024-09-15T22:07:45.631298+0000","last_clean":"2024-09-15T22:07:45.631298+0000","last_became_active":"2024-09-15T22:06:44.660577+0000","last_became_peered":"2024-09-15T22:06:44.660577+0000","last_unstale":"2024-09-15T22:07:45.631298+0000","last_undegraded":"2024-09-15T22:07:45.631298+0000","last_fullsized":"2024-09-15T22:07:45.631298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:56.006891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217462+0000","last_change":"2024-09-15T22:06:37.182077+0000","last_active":"2024-09-15T22:07:46.217462+0000","last_peered":"2024-09-15T22:07:46.217462+0000","last_clean":"2024-09-15T22:07:46.217462+0000","last_became_active":"2024-09-15T22:06:37.181622+0000","last_became_peered":"2024-09-15T22:06:37.181622+0000","last_unstale":"2024-09-15T22:07:46.217462+0000","last_undegraded":"2024-09-15T22:07:46.217462+0000","last_fullsized":"2024-09-15T22:07:46.217462+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:07.586406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631109+0000","last_change":"2024-09-15T22:06:44.653845+0000","last_active":"2024-09-15T22:07:45.631109+0000","last_peered":"2024-09-15T22:07:45.631109+0000","last_clean":"2024-09-15T22:07:45.631109+0000","last_became_active":"2024-09-15T22:06:44.653483+0000","last_became_peered":"2024-09-15T22:06:44.653483+0000","last_unstale":"2024-09-15T22:07:45.631109+0000","last_undegraded":"2024-09-15T22:07:45.631109+0000","last_fullsized":"2024-09-15T22:07:45.631109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:08:07.822279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630930+0000","last_change":"2024-09-15T22:06:37.187476+0000","last_active":"2024-09-15T22:07:45.630930+0000","last_peered":"2024-09-15T22:07:45.630930+0000","last_clean":"2024-09-15T22:07:45.630930+0000","last_became_active":"2024-09-15T22:06:37.187047+0000","last_became_peered":"2024-09-15T22:06:37.187047+0000","last_unstale":"2024-09-15T22:07:45.630930+0000","last_undegraded":"2024-09-15T22:07:45.630930+0000","last_fullsized":"2024-09-15T22:07:45.630930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.220834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216953+0000","last_change":"2024-09-15T22:06:37.190583+0000","last_active":"2024-09-15T22:07:46.216953+0000","last_peered":"2024-09-15T22:07:46.216953+0000","last_clean":"2024-09-15T22:07:46.216953+0000","last_became_active":"2024-09-15T22:06:37.190212+0000","last_became_peered":"2024-09-15T22:06:37.190212+0000","last_unstale":"2024-09-15T22:07:46.216953+0000","last_undegraded":"2024-09-15T22:07:46.216953+0000","last_fullsized":"2024-09-15T22:07:46.216953+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:48.574188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786533+0000","last_change":"2024-09-15T22:06:37.192508+0000","last_active":"2024-09-15T22:07:45.786533+0000","last_peered":"2024-09-15T22:07:45.786533+0000","last_clean":"2024-09-15T22:07:45.786533+0000","last_became_active":"2024-09-15T22:06:37.192117+0000","last_became_peered":"2024-09-15T22:06:37.192117+0000","last_unstale":"2024-09-15T22:07:45.786533+0000","last_undegraded":"2024-09-15T22:07:45.786533+0000","last_fullsized":"2024-09-15T22:07:45.786533+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:41.355966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217986+0000","last_change":"2024-09-15T22:06:37.183377+0000","last_active":"2024-09-15T22:07:46.217986+0000","last_peered":"2024-09-15T22:07:46.217986+0000","last_clean":"2024-09-15T22:07:46.217986+0000","last_became_active":"2024-09-15T22:06:37.183072+0000","last_became_peered":"2024-09-15T22:06:37.183072+0000","last_unstale":"2024-09-15T22:07:46.217986+0000","last_undegraded":"2024-09-15T22:07:46.217986+0000","last_fullsized":"2024-09-15T22:07:46.217986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.318702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631812+0000","last_change":"2024-09-15T22:06:37.185348+0000","last_active":"2024-09-15T22:07:45.631812+0000","last_peered":"2024-09-15T22:07:45.631812+0000","last_clean":"2024-09-15T22:07:45.631812+0000","last_became_active":"2024-09-15T22:06:37.184934+0000","last_became_peered":"2024-09-15T22:06:37.184934+0000","last_unstale":"2024-09-15T22:07:45.631812+0000","last_undegraded":"2024-09-15T22:07:45.631812+0000","last_fullsized":"2024-09-15T22:07:45.631812+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:57.564860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811690+0000","last_change":"2024-09-15T22:06:37.192152+0000","last_active":"2024-09-15T22:07:45.811690+0000","last_peered":"2024-09-15T22:07:45.811690+0000","last_clean":"2024-09-15T22:07:45.811690+0000","last_became_active":"2024-09-15T22:06:37.191836+0000","last_became_peered":"2024-09-15T22:06:37.191836+0000","last_unstale":"2024-09-15T22:07:45.811690+0000","last_undegraded":"2024-09-15T22:07:45.811690+0000","last_fullsized":"2024-09-15T22:07:45.811690+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:31.781110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631207+0000","last_change":"2024-09-15T22:06:37.179996+0000","last_active":"2024-09-15T22:07:45.631207+0000","last_peered":"2024-09-15T22:07:45.631207+0000","last_clean":"2024-09-15T22:07:45.631207+0000","last_became_active":"2024-09-15T22:06:37.179646+0000","last_became_peered":"2024-09-15T22:06:37.179646+0000","last_unstale":"2024-09-15T22:07:45.631207+0000","last_undegraded":"2024-09-15T22:07:45.631207+0000","last_fullsized":"2024-09-15T22:07:45.631207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:35.240468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811293+0000","last_change":"2024-09-15T22:06:37.182992+0000","last_active":"2024-09-15T22:07:45.811293+0000","last_peered":"2024-09-15T22:07:45.811293+0000","last_clean":"2024-09-15T22:07:45.811293+0000","last_became_active":"2024-09-15T22:06:37.182676+0000","last_became_peered":"2024-09-15T22:06:37.182676+0000","last_unstale":"2024-09-15T22:07:45.811293+0000","last_undegraded":"2024-09-15T22:07:45.811293+0000","last_fullsized":"2024-09-15T22:07:45.811293+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:41.524600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216932+0000","last_change":"2024-09-15T22:06:37.180107+0000","last_active":"2024-09-15T22:07:46.216932+0000","last_peered":"2024-09-15T22:07:46.216932+0000","last_clean":"2024-09-15T22:07:46.216932+0000","last_became_active":"2024-09-15T22:06:37.179785+0000","last_became_peered":"2024-09-15T22:06:37.179785+0000","last_unstale":"2024-09-15T22:07:46.216932+0000","last_undegraded":"2024-09-15T22:07:46.216932+0000","last_fullsized":"2024-09-15T22:07:46.216932+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:46.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786319+0000","last_change":"2024-09-15T22:06:37.177406+0000","last_active":"2024-09-15T22:07:45.786319+0000","last_peered":"2024-09-15T22:07:45.786319+0000","last_clean":"2024-09-15T22:07:45.786319+0000","last_became_active":"2024-09-15T22:06:37.177093+0000","last_became_peered":"2024-09-15T22:06:37.177093+0000","last_unstale":"2024-09-15T22:07:45.786319+0000","last_undegraded":"2024-09-15T22:07:45.786319+0000","last_fullsized":"2024-09-15T22:07:45.786319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:34.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217704+0000","last_change":"2024-09-15T22:06:37.192314+0000","last_active":"2024-09-15T22:07:46.217704+0000","last_peered":"2024-09-15T22:07:46.217704+0000","last_clean":"2024-09-15T22:07:46.217704+0000","last_became_active":"2024-09-15T22:06:37.191958+0000","last_became_peered":"2024-09-15T22:06:37.191958+0000","last_unstale":"2024-09-15T22:07:46.217704+0000","last_undegraded":"2024-09-15T22:07:46.217704+0000","last_fullsized":"2024-09-15T22:07:46.217704+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:47.763742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631408+0000","last_change":"2024-09-15T22:06:37.190212+0000","last_active":"2024-09-15T22:07:45.631408+0000","last_peered":"2024-09-15T22:07:45.631408+0000","last_clean":"2024-09-15T22:07:45.631408+0000","last_became_active":"2024-09-15T22:06:37.189881+0000","last_became_peered":"2024-09-15T22:06:37.189881+0000","last_unstale":"2024-09-15T22:07:45.631408+0000","last_undegraded":"2024-09-15T22:07:45.631408+0000","last_fullsized":"2024-09-15T22:07:45.631408+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:59.943574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627394+0000","last_change":"2024-09-15T22:06:37.187967+0000","last_active":"2024-09-15T22:07:45.627394+0000","last_peered":"2024-09-15T22:07:45.627394+0000","last_clean":"2024-09-15T22:07:45.627394+0000","last_became_active":"2024-09-15T22:06:37.187596+0000","last_became_peered":"2024-09-15T22:06:37.187596+0000","last_unstale":"2024-09-15T22:07:45.627394+0000","last_undegraded":"2024-09-15T22:07:45.627394+0000","last_fullsized":"2024-09-15T22:07:45.627394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:00.042784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.218545+0000","last_change":"2024-09-15T22:06:37.189434+0000","last_active":"2024-09-15T22:07:46.218545+0000","last_peered":"2024-09-15T22:07:46.218545+0000","last_clean":"2024-09-15T22:07:46.218545+0000","last_became_active":"2024-09-15T22:06:37.189053+0000","last_became_peered":"2024-09-15T22:06:37.189053+0000","last_unstale":"2024-09-15T22:07:46.218545+0000","last_undegraded":"2024-09-15T22:07:46.218545+0000","last_fullsized":"2024-09-15T22:07:46.218545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:51.640495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216636+0000","last_change":"2024-09-15T22:06:37.179229+0000","last_active":"2024-09-15T22:07:46.216636+0000","last_peered":"2024-09-15T22:07:46.216636+0000","last_clean":"2024-09-15T22:07:46.216636+0000","last_became_active":"2024-09-15T22:06:37.178828+0000","last_became_peered":"2024-09-15T22:06:37.178828+0000","last_unstale":"2024-09-15T22:07:46.216636+0000","last_undegraded":"2024-09-15T22:07:46.216636+0000","last_fullsized":"2024-09-15T22:07:46.216636+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:25.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216741+0000","last_change":"2024-09-15T22:06:37.193095+0000","last_active":"2024-09-15T22:07:46.216741+0000","last_peered":"2024-09-15T22:07:46.216741+0000","last_clean":"2024-09-15T22:07:46.216741+0000","last_became_active":"2024-09-15T22:06:37.192801+0000","last_became_peered":"2024-09-15T22:06:37.192801+0000","last_unstale":"2024-09-15T22:07:46.216741+0000","last_undegraded":"2024-09-15T22:07:46.216741+0000","last_fullsized":"2024-09-15T22:07:46.216741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:12.521510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216619+0000","last_change":"2024-09-15T22:06:37.182332+0000","last_active":"2024-09-15T22:07:46.216619+0000","last_peered":"2024-09-15T22:07:46.216619+0000","last_clean":"2024-09-15T22:07:46.216619+0000","last_became_active":"2024-09-15T22:06:37.182001+0000","last_became_peered":"2024-09-15T22:06:37.182001+0000","last_unstale":"2024-09-15T22:07:46.216619+0000","last_undegraded":"2024-09-15T22:07:46.216619+0000","last_fullsized":"2024-09-15T22:07:46.216619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:52.230452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.786312+0000","last_change":"2024-09-15T22:06:37.180584+0000","last_active":"2024-09-15T22:07:45.786312+0000","last_peered":"2024-09-15T22:07:45.786312+0000","last_clean":"2024-09-15T22:07:45.786312+0000","last_became_active":"2024-09-15T22:06:37.179341+0000","last_became_peered":"2024-09-15T22:06:37.179341+0000","last_unstale":"2024-09-15T22:07:45.786312+0000","last_undegraded":"2024-09-15T22:07:45.786312+0000","last_fullsized":"2024-09-15T22:07:45.786312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.610835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629971+0000","last_change":"2024-09-15T22:06:37.189500+0000","last_active":"2024-09-15T22:07:45.629971+0000","last_peered":"2024-09-15T22:07:45.629971+0000","last_clean":"2024-09-15T22:07:45.629971+0000","last_became_active":"2024-09-15T22:06:37.189137+0000","last_became_peered":"2024-09-15T22:06:37.189137+0000","last_unstale":"2024-09-15T22:07:45.629971+0000","last_undegraded":"2024-09-15T22:07:45.629971+0000","last_fullsized":"2024-09-15T22:07:45.629971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:27.199854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.631325+0000","last_change":"2024-09-15T22:06:37.182618+0000","last_active":"2024-09-15T22:07:45.631325+0000","last_peered":"2024-09-15T22:07:45.631325+0000","last_clean":"2024-09-15T22:07:45.631325+0000","last_became_active":"2024-09-15T22:06:37.182257+0000","last_became_peered":"2024-09-15T22:06:37.182257+0000","last_unstale":"2024-09-15T22:07:45.631325+0000","last_undegraded":"2024-09-15T22:07:45.631325+0000","last_fullsized":"2024-09-15T22:07:45.631325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:17.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.628425+0000","last_change":"2024-09-15T22:06:37.193036+0000","last_active":"2024-09-15T22:07:45.628425+0000","last_peered":"2024-09-15T22:07:45.628425+0000","last_clean":"2024-09-15T22:07:45.628425+0000","last_became_active":"2024-09-15T22:06:37.192668+0000","last_became_peered":"2024-09-15T22:06:37.192668+0000","last_unstale":"2024-09-15T22:07:45.628425+0000","last_undegraded":"2024-09-15T22:07:45.628425+0000","last_fullsized":"2024-09-15T22:07:45.628425+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:05.436045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.627388+0000","last_change":"2024-09-15T22:06:44.659503+0000","last_active":"2024-09-15T22:07:45.627388+0000","last_peered":"2024-09-15T22:07:45.627388+0000","last_clean":"2024-09-15T22:07:45.627388+0000","last_became_active":"2024-09-15T22:06:44.659155+0000","last_became_peered":"2024-09-15T22:06:44.659155+0000","last_unstale":"2024-09-15T22:07:45.627388+0000","last_undegraded":"2024-09-15T22:07:45.627388+0000","last_fullsized":"2024-09-15T22:07:45.627388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:41:04.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.811547+0000","last_change":"2024-09-15T22:06:37.178386+0000","last_active":"2024-09-15T22:07:45.811547+0000","last_peered":"2024-09-15T22:07:45.811547+0000","last_clean":"2024-09-15T22:07:45.811547+0000","last_became_active":"2024-09-15T22:06:37.178083+0000","last_became_peered":"2024-09-15T22:06:37.178083+0000","last_unstale":"2024-09-15T22:07:45.811547+0000","last_undegraded":"2024-09-15T22:07:45.811547+0000","last_fullsized":"2024-09-15T22:07:45.811547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:35.350698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.630748+0000","last_change":"2024-09-15T22:06:37.186741+0000","last_active":"2024-09-15T22:07:45.630748+0000","last_peered":"2024-09-15T22:07:45.630748+0000","last_clean":"2024-09-15T22:07:45.630748+0000","last_became_active":"2024-09-15T22:06:37.185740+0000","last_became_peered":"2024-09-15T22:06:37.185740+0000","last_unstale":"2024-09-15T22:07:45.630748+0000","last_undegraded":"2024-09-15T22:07:45.630748+0000","last_fullsized":"2024-09-15T22:07:45.630748+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.762407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.628239+0000","last_change":"2024-09-15T22:06:37.189552+0000","last_active":"2024-09-15T22:07:45.628239+0000","last_peered":"2024-09-15T22:07:45.628239+0000","last_clean":"2024-09-15T22:07:45.628239+0000","last_became_active":"2024-09-15T22:06:37.189197+0000","last_became_peered":"2024-09-15T22:06:37.189197+0000","last_unstale":"2024-09-15T22:07:45.628239+0000","last_undegraded":"2024-09-15T22:07:45.628239+0000","last_fullsized":"2024-09-15T22:07:45.628239+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:54.268720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.966793+0000","last_change":"2024-09-15T22:06:37.186917+0000","last_active":"2024-09-15T22:07:45.966793+0000","last_peered":"2024-09-15T22:07:45.966793+0000","last_clean":"2024-09-15T22:07:45.966793+0000","last_became_active":"2024-09-15T22:06:37.186571+0000","last_became_peered":"2024-09-15T22:06:37.186571+0000","last_unstale":"2024-09-15T22:07:45.966793+0000","last_undegraded":"2024-09-15T22:07:45.966793+0000","last_fullsized":"2024-09-15T22:07:45.966793+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:02.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:45.629838+0000","last_change":"2024-09-15T22:06:37.190960+0000","last_active":"2024-09-15T22:07:45.629838+0000","last_peered":"2024-09-15T22:07:45.629838+0000","last_clean":"2024-09-15T22:07:45.629838+0000","last_became_active":"2024-09-15T22:06:37.190650+0000","last_became_peered":"2024-09-15T22:06:37.190650+0000","last_unstale":"2024-09-15T22:07:45.629838+0000","last_undegraded":"2024-09-15T22:07:45.629838+0000","last_fullsized":"2024-09-15T22:07:45.629838+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:34.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.217037+0000","last_change":"2024-09-15T22:06:37.186718+0000","last_active":"2024-09-15T22:07:46.217037+0000","last_peered":"2024-09-15T22:07:46.217037+0000","last_clean":"2024-09-15T22:07:46.217037+0000","last_became_active":"2024-09-15T22:06:37.186424+0000","last_became_peered":"2024-09-15T22:06:37.186424+0000","last_unstale":"2024-09-15T22:07:46.217037+0000","last_undegraded":"2024-09-15T22:07:46.217037+0000","last_fullsized":"2024-09-15T22:07:46.217037+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:53.266007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.216870+0000","last_change":"2024-09-15T22:06:37.185522+0000","last_active":"2024-09-15T22:07:46.216870+0000","last_peered":"2024-09-15T22:07:46.216870+0000","last_clean":"2024-09-15T22:07:46.216870+0000","last_became_active":"2024-09-15T22:06:37.185192+0000","last_became_peered":"2024-09-15T22:06:37.185192+0000","last_unstale":"2024-09-15T22:07:46.216870+0000","last_undegraded":"2024-09-15T22:07:46.216870+0000","last_fullsized":"2024-09-15T22:07:46.216870+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:12.868324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-15T22:07:46.218447+0000","last_change":"2024-09-15T22:06:37.189553+0000","last_active":"2024-09-15T22:07:46.218447+0000","last_peered":"2024-09-15T22:07:46.218447+0000","last_clean":"2024-09-15T22:07:46.218447+0000","last_became_active":"2024-09-15T22:06:37.189270+0000","last_became_peered":"2024-09-15T22:06:37.189270+0000","last_unstale":"2024-09-15T22:07:46.218447+0000","last_undegraded":"2024-09-15T22:07:46.218447+0000","last_fullsized":"2024-09-15T22:07:46.218447+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:34.157077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66472,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":358,"num_read_kb":251,"num_write":174,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":199416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147,"ondisk_log_size":147,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579937,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1011712,"data_stored":502844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808870,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32848,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712304,"statfs":{"total":95995035648,"available":95961399296,"internally_reserved":0,"allocated":1695744,"data_stored":1151999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005099,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1028096,"data_stored":506594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234030,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1019904,"data_stored":501836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495669,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1613824,"data_stored":1098140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659193,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1003520,"data_stored":497674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888126,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32772,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712380,"statfs":{"total":95995035648,"available":95961477120,"internally_reserved":0,"allocated":1626112,"data_stored":1093917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018948,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1019904,"data_stored":503299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247881,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1015808,"data_stored":497868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444110,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":560821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36400,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708752,"statfs":{"total":95995035648,"available":95957762048,"internally_reserved":0,"allocated":1073152,"data_stored":560242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1028096,"data_stored":499062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":64047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:08:11.788 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T22:08:11.788 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T22:08:11.788 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-15T22:08:11.789 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-15T22:08:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:12.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:11 smithi005 ceph-mon[30944]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:12.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:11 smithi005 ceph-mon[30944]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:12.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:11 smithi099 ceph-mon[34892]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:12.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:11 smithi099 ceph-mon[34892]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:12.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:11 smithi129 ceph-mon[35314]: pgmap v332: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:12.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:11 smithi129 ceph-mon[35314]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.548+0000 7f5a77b22640 1 -- 172.21.15.5:0/3216117518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a70101280 msgr2=0x7f5a7010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.548+0000 7f5a77b22640 1 --2- 172.21.15.5:0/3216117518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a70101280 0x7f5a7010df40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009900 tx=0x7f5a6002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.548+0000 7f5a77b22640 1 -- 172.21.15.5:0/3216117518 shutdown_connections 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.548+0000 7f5a77b22640 1 --2- 172.21.15.5:0/3216117518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a7010e480 0x7f5a70110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.548+0000 7f5a77b22640 1 --2- 172.21.15.5:0/3216117518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a70101280 0x7f5a7010df40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.548+0000 7f5a77b22640 1 --2- 172.21.15.5:0/3216117518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a701008d0 0x7f5a70100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.548+0000 7f5a77b22640 1 -- 172.21.15.5:0/3216117518 >> 172.21.15.5:0/3216117518 conn(0x7f5a700fc7a0 msgr2=0x7f5a700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.548+0000 7f5a77b22640 1 -- 172.21.15.5:0/3216117518 shutdown_connections 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.548+0000 7f5a77b22640 1 -- 172.21.15.5:0/3216117518 wait complete. 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.548+0000 7f5a77b22640 1 Processor -- start 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a77b22640 1 -- start start 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a77b22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a701008d0 0x7f5a7019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a77b22640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a70101280 0x7f5a7019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a77b22640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a7010e480 0x7f5a7019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a77b22640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70114190 con 0x7f5a701008d0 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a77b22640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70114010 con 0x7f5a7010e480 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a77b22640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70114310 con 0x7f5a70101280 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a758be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a701008d0 0x7f5a7019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a758be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a701008d0 0x7f5a7019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57256/0 (socket says 172.21.15.5:57256) 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a758be640 1 -- 172.21.15.5:0/3000684938 learned_addr learned my addr 172.21.15.5:0/3000684938 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a760bf640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a7010e480 0x7f5a7019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a750bd640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a70101280 0x7f5a7019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a758be640 1 -- 172.21.15.5:0/3000684938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a70101280 msgr2=0x7f5a7019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a758be640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a70101280 0x7f5a7019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a758be640 1 -- 172.21.15.5:0/3000684938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a7010e480 msgr2=0x7f5a7019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a758be640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a7010e480 0x7f5a7019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a758be640 1 -- 172.21.15.5:0/3000684938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a7019f5e0 con 0x7f5a701008d0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.549+0000 7f5a758be640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a701008d0 0x7f5a7019a630 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00c890 tx=0x7f5a6c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.550+0000 7f5a66ffd640 1 -- 172.21.15.5:0/3000684938 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c004220 con 0x7f5a701008d0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.550+0000 7f5a66ffd640 1 -- 172.21.15.5:0/3000684938 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a6c0043c0 con 0x7f5a701008d0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.550+0000 7f5a77b22640 1 -- 172.21.15.5:0/3000684938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a7019f8d0 con 0x7f5a701008d0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.550+0000 7f5a66ffd640 1 -- 172.21.15.5:0/3000684938 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0049e0 con 0x7f5a701008d0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.550+0000 7f5a77b22640 1 -- 172.21.15.5:0/3000684938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a701a71b0 con 0x7f5a701008d0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.551+0000 7f5a77b22640 1 -- 172.21.15.5:0/3000684938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a70100cb0 con 0x7f5a701008d0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.552+0000 7f5a66ffd640 1 -- 172.21.15.5:0/3000684938 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f5a6c0079b0 con 0x7f5a701008d0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.552+0000 7f5a66ffd640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5a5c0a32d0 0x7f5a5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.553+0000 7f5a66ffd640 1 -- 172.21.15.5:0/3000684938 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f5a6c0cfe10 con 0x7f5a701008d0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.553+0000 7f5a750bd640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5a5c0a32d0 0x7f5a5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.553+0000 7f5a750bd640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5a5c0a32d0 0x7f5a5c0a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5a60000c00 tx=0x7f5a600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:12.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.557+0000 7f5a66ffd640 1 -- 172.21.15.5:0/3000684938 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a6c014030 con 0x7f5a701008d0 2024-09-15T22:08:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.776+0000 7f5a77b22640 1 -- 172.21.15.5:0/3000684938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5a70109610 con 0x7f5a701008d0 2024-09-15T22:08:12.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.776+0000 7f5a66ffd640 1 -- 172.21.15.5:0/3000684938 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5a6c09bfc0 con 0x7f5a701008d0 2024-09-15T22:08:12.778 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T22:08:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.778+0000 7f5a77b22640 1 -- 172.21.15.5:0/3000684938 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5a5c0a32d0 msgr2=0x7f5a5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.778+0000 7f5a77b22640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5a5c0a32d0 0x7f5a5c0a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5a60000c00 tx=0x7f5a600058d0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.778+0000 7f5a77b22640 1 -- 172.21.15.5:0/3000684938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a701008d0 msgr2=0x7f5a7019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.778+0000 7f5a77b22640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a701008d0 0x7f5a7019a630 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00c890 tx=0x7f5a6c00cd50 comp rx=0 tx=0).stop 2024-09-15T22:08:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.779+0000 7f5a77b22640 1 -- 172.21.15.5:0/3000684938 shutdown_connections 2024-09-15T22:08:12.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.779+0000 7f5a77b22640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a7010e480 0x7f5a7019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.779+0000 7f5a77b22640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a70101280 0x7f5a7019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.779+0000 7f5a77b22640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f5a5c0a32d0 0x7f5a5c0a5790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.779+0000 7f5a77b22640 1 --2- 172.21.15.5:0/3000684938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a701008d0 0x7f5a7019a630 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.779+0000 7f5a77b22640 1 -- 172.21.15.5:0/3000684938 >> 172.21.15.5:0/3000684938 conn(0x7f5a700fc7a0 msgr2=0x7f5a700feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.779+0000 7f5a77b22640 1 -- 172.21.15.5:0/3000684938 shutdown_connections 2024-09-15T22:08:12.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:12.779+0000 7f5a77b22640 1 -- 172.21.15.5:0/3000684938 wait complete. 2024-09-15T22:08:13.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:12 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3000684938' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:08:13.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:12 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3000684938' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:08:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:12 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3000684938' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-15T22:08:13.577 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-15T22:08:13.578 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:08:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:14.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:13 smithi005 ceph-mon[30944]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.265+0000 7fe3d0c14640 1 -- 172.21.15.5:0/3623575951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3cc102570 msgr2=0x7fe3cc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.265+0000 7fe3d0c14640 1 --2- 172.21.15.5:0/3623575951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3cc102570 0x7fe3cc1029b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009900 tx=0x7fe3c002a480 comp rx=0 tx=0).stop 2024-09-15T22:08:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.266+0000 7fe3d0c14640 1 -- 172.21.15.5:0/3623575951 shutdown_connections 2024-09-15T22:08:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.266+0000 7fe3d0c14640 1 --2- 172.21.15.5:0/3623575951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3cc102ef0 0x7fe3cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:14.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.266+0000 7fe3d0c14640 1 --2- 172.21.15.5:0/3623575951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3cc102570 0x7fe3cc1029b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.266+0000 7fe3d0c14640 1 --2- 172.21.15.5:0/3623575951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3cc100e20 0x7fe3cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.266+0000 7fe3d0c14640 1 -- 172.21.15.5:0/3623575951 >> 172.21.15.5:0/3623575951 conn(0x7fe3cc0fc7d0 msgr2=0x7fe3cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.266+0000 7fe3d0c14640 1 -- 172.21.15.5:0/3623575951 shutdown_connections 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.266+0000 7fe3d0c14640 1 -- 172.21.15.5:0/3623575951 wait complete. 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.266+0000 7fe3d0c14640 1 Processor -- start 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.266+0000 7fe3d0c14640 1 -- start start 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3d0c14640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3cc100e20 0x7fe3cc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3d0c14640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3cc102570 0x7fe3cc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3d0c14640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3cc102ef0 0x7fe3cc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3d0c14640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc110f80 con 0x7fe3cc102570 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3d0c14640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc110e00 con 0x7fe3cc102ef0 2024-09-15T22:08:14.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3d0c14640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc111100 con 0x7fe3cc100e20 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3cc102570 0x7fe3cc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3c9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3cc102570 0x7fe3cc0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38164/0 (socket says 172.21.15.5:38164) 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3c9d9b640 1 -- 172.21.15.5:0/901721948 learned_addr learned my addr 172.21.15.5:0/901721948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3ca59c640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3cc100e20 0x7fe3cc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3cad9d640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3cc102ef0 0x7fe3cc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3c9d9b640 1 -- 172.21.15.5:0/901721948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3cc100e20 msgr2=0x7fe3cc0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3c9d9b640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3cc100e20 0x7fe3cc0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3c9d9b640 1 -- 172.21.15.5:0/901721948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3cc102ef0 msgr2=0x7fe3cc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3c9d9b640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3cc102ef0 0x7fe3cc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3c9d9b640 1 -- 172.21.15.5:0/901721948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3cc1a69b0 con 0x7fe3cc102570 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3cad9d640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3cc102ef0 0x7fe3cc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3c9d9b640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3cc102570 0x7fe3cc0fd1c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0006010 tx=0x7fe3c0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:14.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3bb7fe640 1 -- 172.21.15.5:0/901721948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c003f070 con 0x7fe3cc102570 2024-09-15T22:08:14.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3d0c14640 1 -- 172.21.15.5:0/901721948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3cc1a6c40 con 0x7fe3cc102570 2024-09-15T22:08:14.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3bb7fe640 1 -- 172.21.15.5:0/901721948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3c0031c30 con 0x7fe3cc102570 2024-09-15T22:08:14.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.267+0000 7fe3d0c14640 1 -- 172.21.15.5:0/901721948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3cc1a7170 con 0x7fe3cc102570 2024-09-15T22:08:14.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.268+0000 7fe3bb7fe640 1 -- 172.21.15.5:0/901721948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0049060 con 0x7fe3cc102570 2024-09-15T22:08:14.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.269+0000 7fe3d0c14640 1 -- 172.21.15.5:0/901721948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3cc101200 con 0x7fe3cc102570 2024-09-15T22:08:14.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.270+0000 7fe3bb7fe640 1 -- 172.21.15.5:0/901721948 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fe3c0042070 con 0x7fe3cc102570 2024-09-15T22:08:14.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.270+0000 7fe3bb7fe640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3b40a32d0 0x7fe3b40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:14.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.270+0000 7fe3bb7fe640 1 -- 172.21.15.5:0/901721948 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fe3c00f0a50 con 0x7fe3cc102570 2024-09-15T22:08:14.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.270+0000 7fe3ca59c640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3b40a32d0 0x7fe3b40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:14.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.271+0000 7fe3ca59c640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3b40a32d0 0x7fe3b40a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00bbe0 tx=0x7fe3bc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:14.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.275+0000 7fe3bb7fe640 1 -- 172.21.15.5:0/901721948 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3c0031de0 con 0x7fe3cc102570 2024-09-15T22:08:14.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:13 smithi099 ceph-mon[34892]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:14.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:13 smithi129 ceph-mon[35314]: pgmap v333: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:14.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.486+0000 7fe3d0c14640 1 -- 172.21.15.5:0/901721948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3cc10a0a0 con 0x7fe3cc102570 2024-09-15T22:08:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.486+0000 7fe3bb7fe640 1 -- 172.21.15.5:0/901721948 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11897 (secure 0 0 0) 0x7fe3c00bd250 con 0x7fe3cc102570 2024-09-15T22:08:14.488 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:14.488 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"btime":"2024-09-15T22:07:16:736368+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:07:16.364125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24466,24467,24491]},"id":1}]} 2024-09-15T22:08:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 24 2024-09-15T22:08:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 -- 172.21.15.5:0/901721948 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3b40a32d0 msgr2=0x7fe3b40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3b40a32d0 0x7fe3b40a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc00bbe0 tx=0x7fe3bc00b390 comp rx=0 tx=0).stop 2024-09-15T22:08:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 -- 172.21.15.5:0/901721948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3cc102570 msgr2=0x7fe3cc0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3cc102570 0x7fe3cc0fd1c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0006010 tx=0x7fe3c0003c10 comp rx=0 tx=0).stop 2024-09-15T22:08:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 -- 172.21.15.5:0/901721948 shutdown_connections 2024-09-15T22:08:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe3cc102ef0 0x7fe3cc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fe3b40a32d0 0x7fe3b40a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3cc102570 0x7fe3cc0fd1c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 --2- 172.21.15.5:0/901721948 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe3cc100e20 0x7fe3cc0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 -- 172.21.15.5:0/901721948 >> 172.21.15.5:0/901721948 conn(0x7fe3cc0fc7d0 msgr2=0x7fe3cc06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 -- 172.21.15.5:0/901721948 shutdown_connections 2024-09-15T22:08:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:14.489+0000 7fe3d0c14640 1 -- 172.21.15.5:0/901721948 wait complete. 2024-09-15T22:08:15.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:14 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/901721948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:15.150 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:08:15.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:14 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/901721948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:15.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:14 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/901721948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:15.408 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.873+0000 7f24a636c640 1 -- 172.21.15.5:0/355300060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a0106f10 msgr2=0x7f24a010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.873+0000 7f24a636c640 1 --2- 172.21.15.5:0/355300060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a0106f10 0x7f24a010ea90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2498009ca0 tx=0x7f249802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:08:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.873+0000 7f24a636c640 1 -- 172.21.15.5:0/355300060 shutdown_connections 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.873+0000 7f24a636c640 1 --2- 172.21.15.5:0/355300060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a0106f10 0x7f24a010ea90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.873+0000 7f24a636c640 1 --2- 172.21.15.5:0/355300060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a0102790 0x7f24a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.873+0000 7f24a636c640 1 --2- 172.21.15.5:0/355300060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a0101de0 0x7f24a01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.873+0000 7f24a636c640 1 -- 172.21.15.5:0/355300060 >> 172.21.15.5:0/355300060 conn(0x7f24a00fd8c0 msgr2=0x7f24a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.873+0000 7f24a636c640 1 -- 172.21.15.5:0/355300060 shutdown_connections 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.873+0000 7f24a636c640 1 -- 172.21.15.5:0/355300060 wait complete. 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.874+0000 7f24a636c640 1 Processor -- start 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.874+0000 7f24a636c640 1 -- start start 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.874+0000 7f24a636c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a0101de0 0x7f24a0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.874+0000 7f24a636c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a0102790 0x7f24a01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.874+0000 7f24a636c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a0106f10 0x7f24a019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.874+0000 7f24a636c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0111f90 con 0x7f24a0106f10 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.874+0000 7f24a636c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0111e10 con 0x7f24a0101de0 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.874+0000 7f24a636c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0112110 con 0x7f24a0102790 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f24a4909640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a0106f10 0x7f24a019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f24a4909640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a0106f10 0x7f24a019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38188/0 (socket says 172.21.15.5:38188) 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f24a4909640 1 -- 172.21.15.5:0/1099010709 learned_addr learned my addr 172.21.15.5:0/1099010709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f249ffff640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a0101de0 0x7f24a0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f24a4909640 1 -- 172.21.15.5:0/1099010709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a0102790 msgr2=0x7f24a01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f24a4909640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a0102790 0x7f24a01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f24a4909640 1 -- 172.21.15.5:0/1099010709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a0101de0 msgr2=0x7f24a0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f24a4909640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a0101de0 0x7f24a0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f24a4909640 1 -- 172.21.15.5:0/1099010709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a019d420 con 0x7f24a0106f10 2024-09-15T22:08:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f24a4909640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a0106f10 0x7f24a019cd40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2498031a60 tx=0x7f2498031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f249d7fa640 1 -- 172.21.15.5:0/1099010709 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2498003c30 con 0x7f24a0106f10 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f249d7fa640 1 -- 172.21.15.5:0/1099010709 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2498003dd0 con 0x7f24a0106f10 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f24a636c640 1 -- 172.21.15.5:0/1099010709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a019d6b0 con 0x7f24a0106f10 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f249d7fa640 1 -- 172.21.15.5:0/1099010709 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24980047e0 con 0x7f24a0106f10 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.875+0000 7f24a636c640 1 -- 172.21.15.5:0/1099010709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a01a4ff0 con 0x7f24a0106f10 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.876+0000 7f24a636c640 1 -- 172.21.15.5:0/1099010709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a01021c0 con 0x7f24a0106f10 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.878+0000 7f249d7fa640 1 -- 172.21.15.5:0/1099010709 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f249802f790 con 0x7f24a0106f10 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.878+0000 7f249d7fa640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24880a32d0 0x7f24880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.878+0000 7f249d7fa640 1 -- 172.21.15.5:0/1099010709 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f24980f1230 con 0x7f24a0106f10 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.878+0000 7f249ffff640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24880a32d0 0x7f24880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.879+0000 7f249ffff640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24880a32d0 0x7f24880a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f24900058c0 tx=0x7f249000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:15.882+0000 7f249d7fa640 1 -- 172.21.15.5:0/1099010709 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24980bdb40 con 0x7f24a0106f10 2024-09-15T22:08:16.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.092+0000 7f24a636c640 1 -- 172.21.15.5:0/1099010709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24a010a0f0 con 0x7f24a0106f10 2024-09-15T22:08:16.094 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:15 smithi005 ceph-mon[30944]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:16.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.093+0000 7f249d7fa640 1 -- 172.21.15.5:0/1099010709 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11897 (secure 0 0 0) 0x7f24980bd440 con 0x7f24a0106f10 2024-09-15T22:08:16.095 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:16.095 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"btime":"2024-09-15T22:07:16:736368+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:07:16.364125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24466,24467,24491]},"id":1}]} 2024-09-15T22:08:16.096 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 24 2024-09-15T22:08:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 -- 172.21.15.5:0/1099010709 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24880a32d0 msgr2=0x7f24880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24880a32d0 0x7f24880a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f24900058c0 tx=0x7f249000a4c0 comp rx=0 tx=0).stop 2024-09-15T22:08:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 -- 172.21.15.5:0/1099010709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a0106f10 msgr2=0x7f24a019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a0106f10 0x7f24a019cd40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2498031a60 tx=0x7f2498031a90 comp rx=0 tx=0).stop 2024-09-15T22:08:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 -- 172.21.15.5:0/1099010709 shutdown_connections 2024-09-15T22:08:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f24880a32d0 0x7f24880a5790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:16.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24a0106f10 0x7f24a019cd40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f24a0102790 0x7f24a01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 --2- 172.21.15.5:0/1099010709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f24a0101de0 0x7f24a0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 -- 172.21.15.5:0/1099010709 >> 172.21.15.5:0/1099010709 conn(0x7f24a00fd8c0 msgr2=0x7f24a00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 -- 172.21.15.5:0/1099010709 shutdown_connections 2024-09-15T22:08:16.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:16.096+0000 7f24a636c640 1 -- 172.21.15.5:0/1099010709 wait complete. 2024-09-15T22:08:16.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:15 smithi099 ceph-mon[34892]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:16.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:15 smithi129 ceph-mon[35314]: pgmap v334: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:16.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:08:17.027 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:17.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:16 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1099010709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:17.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:16 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1099010709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:16 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1099010709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.490+0000 7f200cdef640 1 -- 172.21.15.5:0/2241114114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20081036e0 msgr2=0x7f2008107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.490+0000 7f200cdef640 1 --2- 172.21.15.5:0/2241114114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20081036e0 0x7f2008107b90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009930 tx=0x7f1ff002a420 comp rx=0 tx=0).stop 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.490+0000 7f200cdef640 1 -- 172.21.15.5:0/2241114114 shutdown_connections 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.490+0000 7f200cdef640 1 --2- 172.21.15.5:0/2241114114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20081082a0 0x7f200810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.490+0000 7f200cdef640 1 --2- 172.21.15.5:0/2241114114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20081036e0 0x7f2008107b90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.490+0000 7f200cdef640 1 --2- 172.21.15.5:0/2241114114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2008102d30 0x7f2008103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.490+0000 7f200cdef640 1 -- 172.21.15.5:0/2241114114 >> 172.21.15.5:0/2241114114 conn(0x7f20080fc7d0 msgr2=0x7f20080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.490+0000 7f200cdef640 1 -- 172.21.15.5:0/2241114114 shutdown_connections 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.490+0000 7f200cdef640 1 -- 172.21.15.5:0/2241114114 wait complete. 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.490+0000 7f200cdef640 1 Processor -- start 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f200cdef640 1 -- start start 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f200cdef640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2008102d30 0x7f2008198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f200cdef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20081036e0 0x7f2008198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f200cdef640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20081082a0 0x7f200819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f200cdef640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20081120c0 con 0x7f20081036e0 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f200cdef640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2008111f40 con 0x7f20081082a0 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f200cdef640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2008112240 con 0x7f2008102d30 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f2005d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20081036e0 0x7f2008198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f2005d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20081036e0 0x7f2008198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38208/0 (socket says 172.21.15.5:38208) 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f2005d9b640 1 -- 172.21.15.5:0/3594019576 learned_addr learned my addr 172.21.15.5:0/3594019576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f2006d9d640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20081082a0 0x7f200819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f2005d9b640 1 -- 172.21.15.5:0/3594019576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2008102d30 msgr2=0x7f2008198640 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f2005d9b640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2008102d30 0x7f2008198640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f2005d9b640 1 -- 172.21.15.5:0/3594019576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20081082a0 msgr2=0x7f200819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f2005d9b640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20081082a0 0x7f200819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.491+0000 7f2005d9b640 1 -- 172.21.15.5:0/3594019576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200819d5f0 con 0x7f20081036e0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.492+0000 7f2005d9b640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20081036e0 0x7f2008198b80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0005940 tx=0x7f1ff00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.492+0000 7f1fff7fe640 1 -- 172.21.15.5:0/3594019576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff003f030 con 0x7f20081036e0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.492+0000 7f1fff7fe640 1 -- 172.21.15.5:0/3594019576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ff0003ce0 con 0x7f20081036e0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.492+0000 7f200cdef640 1 -- 172.21.15.5:0/3594019576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200819d880 con 0x7f20081036e0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.492+0000 7f1fff7fe640 1 -- 172.21.15.5:0/3594019576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0035070 con 0x7f20081036e0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.492+0000 7f200cdef640 1 -- 172.21.15.5:0/3594019576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20081a51c0 con 0x7f20081036e0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.493+0000 7f200cdef640 1 -- 172.21.15.5:0/3594019576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2008103110 con 0x7f20081036e0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.494+0000 7f1fff7fe640 1 -- 172.21.15.5:0/3594019576 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f1ff0042030 con 0x7f20081036e0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.495+0000 7f1fff7fe640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff40a3350 0x7f1ff40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.495+0000 7f200659c640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff40a3350 0x7f1ff40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.495+0000 7f1fff7fe640 1 -- 172.21.15.5:0/3594019576 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f1ff00f0330 con 0x7f20081036e0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.495+0000 7f200659c640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff40a3350 0x7f1ff40a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80096b0 tx=0x7f1ff8006c20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.499+0000 7f1fff7fe640 1 -- 172.21.15.5:0/3594019576 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ff00bcb30 con 0x7f20081036e0 2024-09-15T22:08:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.676+0000 7f200cdef640 1 -- 172.21.15.5:0/3594019576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f200810a2b0 con 0x7f20081036e0 2024-09-15T22:08:17.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.678+0000 7f1fff7fe640 1 -- 172.21.15.5:0/3594019576 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24757 (secure 0 0 0) 0x7f1ff00bc430 con 0x7f20081036e0 2024-09-15T22:08:17.679 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:17.680 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:07:45.620119+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:08:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 -- 172.21.15.5:0/3594019576 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff40a3350 msgr2=0x7f1ff40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff40a3350 0x7f1ff40a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80096b0 tx=0x7f1ff8006c20 comp rx=0 tx=0).stop 2024-09-15T22:08:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 -- 172.21.15.5:0/3594019576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20081036e0 msgr2=0x7f2008198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:17.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20081036e0 0x7f2008198b80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0005940 tx=0x7f1ff00042f0 comp rx=0 tx=0).stop 2024-09-15T22:08:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 -- 172.21.15.5:0/3594019576 shutdown_connections 2024-09-15T22:08:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f20081082a0 0x7f200819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f1ff40a3350 0x7f1ff40a5810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20081036e0 0x7f2008198b80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 --2- 172.21.15.5:0/3594019576 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2008102d30 0x7f2008198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 -- 172.21.15.5:0/3594019576 >> 172.21.15.5:0/3594019576 conn(0x7f20080fc7d0 msgr2=0x7f2008105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 -- 172.21.15.5:0/3594019576 shutdown_connections 2024-09-15T22:08:17.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:17.680+0000 7f200cdef640 1 -- 172.21.15.5:0/3594019576 wait complete. 2024-09-15T22:08:18.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:17 smithi005 ceph-mon[30944]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:18.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:17 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3594019576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:08:18.330 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:08:18.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:17 smithi099 ceph-mon[34892]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:18.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:17 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3594019576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:08:18.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:17 smithi129 ceph-mon[35314]: pgmap v335: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:18.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:17 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3594019576' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:08:18.589 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.092+0000 7f3635f74640 1 -- 172.21.15.5:0/412001752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3630103660 msgr2=0x7f363010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.092+0000 7f3635f74640 1 --2- 172.21.15.5:0/412001752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3630103660 0x7f363010bb50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3624009900 tx=0x7f362402a480 comp rx=0 tx=0).stop 2024-09-15T22:08:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.092+0000 7f3635f74640 1 -- 172.21.15.5:0/412001752 shutdown_connections 2024-09-15T22:08:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.092+0000 7f3635f74640 1 --2- 172.21.15.5:0/412001752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f363010c170 0x7f363010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.092+0000 7f3635f74640 1 --2- 172.21.15.5:0/412001752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3630103660 0x7f363010bb50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.092+0000 7f3635f74640 1 --2- 172.21.15.5:0/412001752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3630102d40 0x7f3630103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.092+0000 7f3635f74640 1 -- 172.21.15.5:0/412001752 >> 172.21.15.5:0/412001752 conn(0x7f36300fc7a0 msgr2=0x7f36300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.092+0000 7f3635f74640 1 -- 172.21.15.5:0/412001752 shutdown_connections 2024-09-15T22:08:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.092+0000 7f3635f74640 1 -- 172.21.15.5:0/412001752 wait complete. 2024-09-15T22:08:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.092+0000 7f3635f74640 1 Processor -- start 2024-09-15T22:08:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f3635f74640 1 -- start start 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f3635f74640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3630102d40 0x7f3630198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f3635f74640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3630103660 0x7f3630198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f3635f74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f363010c170 0x7f363019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f3635f74640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3630112030 con 0x7f363010c170 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f3635f74640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3630111eb0 con 0x7f3630103660 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f3635f74640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36301121b0 con 0x7f3630102d40 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f362ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f363010c170 0x7f363019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f362ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f363010c170 0x7f363019ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38222/0 (socket says 172.21.15.5:38222) 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f362ffff640 1 -- 172.21.15.5:0/2783223813 learned_addr learned my addr 172.21.15.5:0/2783223813 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f362f7fe640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3630102d40 0x7f3630198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f362effd640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3630103660 0x7f3630198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f362ffff640 1 -- 172.21.15.5:0/2783223813 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3630102d40 msgr2=0x7f3630198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f362ffff640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3630102d40 0x7f3630198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f362ffff640 1 -- 172.21.15.5:0/2783223813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3630103660 msgr2=0x7f3630198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f362ffff640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3630103660 0x7f3630198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f362ffff640 1 -- 172.21.15.5:0/2783223813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363019d530 con 0x7f363010c170 2024-09-15T22:08:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.093+0000 7f362effd640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3630103660 0x7f3630198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.094+0000 7f362ffff640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f363010c170 0x7f363019ce50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3628009fd0 tx=0x7f362800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.094+0000 7f362cff9640 1 -- 172.21.15.5:0/2783223813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3628011830 con 0x7f363010c170 2024-09-15T22:08:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.094+0000 7f362cff9640 1 -- 172.21.15.5:0/2783223813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3628016070 con 0x7f363010c170 2024-09-15T22:08:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.094+0000 7f3635f74640 1 -- 172.21.15.5:0/2783223813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363019d820 con 0x7f363010c170 2024-09-15T22:08:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.094+0000 7f362cff9640 1 -- 172.21.15.5:0/2783223813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362800f2e0 con 0x7f363010c170 2024-09-15T22:08:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.094+0000 7f3635f74640 1 -- 172.21.15.5:0/2783223813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36301a5100 con 0x7f363010c170 2024-09-15T22:08:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.095+0000 7f3635f74640 1 -- 172.21.15.5:0/2783223813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36301031a0 con 0x7f363010c170 2024-09-15T22:08:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.096+0000 7f362cff9640 1 -- 172.21.15.5:0/2783223813 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f36280119d0 con 0x7f363010c170 2024-09-15T22:08:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.097+0000 7f362cff9640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36180a32d0 0x7f36180a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.097+0000 7f362cff9640 1 -- 172.21.15.5:0/2783223813 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f36280cf370 con 0x7f363010c170 2024-09-15T22:08:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.097+0000 7f362f7fe640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36180a32d0 0x7f36180a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.097+0000 7f362f7fe640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36180a32d0 0x7f36180a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f362000a400 tx=0x7f362000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.101+0000 7f362cff9640 1 -- 172.21.15.5:0/2783223813 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f362809bc80 con 0x7f363010c170 2024-09-15T22:08:19.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.318+0000 7f3635f74640 1 -- 172.21.15.5:0/2783223813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f363002d7c0 con 0x7f363010c170 2024-09-15T22:08:19.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.318+0000 7f362cff9640 1 -- 172.21.15.5:0/2783223813 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11897 (secure 0 0 0) 0x7f362809b580 con 0x7f363010c170 2024-09-15T22:08:19.320 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:19.320 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"btime":"2024-09-15T22:07:16:736368+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:07:16.364125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24466,24467,24491]},"id":1}]} 2024-09-15T22:08:19.320 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 24 2024-09-15T22:08:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 -- 172.21.15.5:0/2783223813 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36180a32d0 msgr2=0x7f36180a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36180a32d0 0x7f36180a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f362000a400 tx=0x7f362000d040 comp rx=0 tx=0).stop 2024-09-15T22:08:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 -- 172.21.15.5:0/2783223813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f363010c170 msgr2=0x7f363019ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f363010c170 0x7f363019ce50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3628009fd0 tx=0x7f362800b8e0 comp rx=0 tx=0).stop 2024-09-15T22:08:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 -- 172.21.15.5:0/2783223813 shutdown_connections 2024-09-15T22:08:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36180a32d0 0x7f36180a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f363010c170 0x7f363019ce50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3630103660 0x7f3630198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 --2- 172.21.15.5:0/2783223813 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3630102d40 0x7f3630198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 -- 172.21.15.5:0/2783223813 >> 172.21.15.5:0/2783223813 conn(0x7f36300fc7a0 msgr2=0x7f36300fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 -- 172.21.15.5:0/2783223813 shutdown_connections 2024-09-15T22:08:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:19.321+0000 7f3635f74640 1 -- 172.21.15.5:0/2783223813 wait complete. 2024-09-15T22:08:20.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T22:06:37.854436+0000', 'modified': '2024-09-15T22:07:16.364125+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24491, 'mds_1': 24467, 'mds_2': 24466}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/157351762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 157351762}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 157351762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6835/3511675647', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 3511675647}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 3511675647}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6837/3535375231', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6836', 'nonce': 3535375231}, {'type': 'v1', 'addr': '172.21.15.129:6837', 'nonce': 3535375231}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.129:6839/2032213198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 2032213198}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 2032213198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.99:6837/3952304847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 3952304847}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 3952304847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.99:6839/3763383986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 3763383986}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 3763383986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24491, 'qdb_cluster': [24466, 24467, 24491]} 2024-09-15T22:08:20.115 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-15T22:08:20.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T22:08:20.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:19 smithi099 ceph-mon[34892]: pgmap v336: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:20.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:19 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2783223813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:19 smithi129 ceph-mon[35314]: pgmap v336: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:19 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2783223813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:20.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:19 smithi005 ceph-mon[30944]: pgmap v336: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:20.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:19 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2783223813' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:20.378 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.844+0000 7f89af280640 1 -- 172.21.15.5:0/2886305447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a8102ef0 msgr2=0x7f89a810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:20.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.844+0000 7f89af280640 1 --2- 172.21.15.5:0/2886305447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a8102ef0 0x7f89a810da40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f89a0009900 tx=0x7f89a002a480 comp rx=0 tx=0).stop 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.844+0000 7f89af280640 1 -- 172.21.15.5:0/2886305447 shutdown_connections 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.844+0000 7f89af280640 1 --2- 172.21.15.5:0/2886305447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a8102ef0 0x7f89a810da40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.844+0000 7f89af280640 1 --2- 172.21.15.5:0/2886305447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89a8102570 0x7f89a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.844+0000 7f89af280640 1 --2- 172.21.15.5:0/2886305447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89a8100e20 0x7f89a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.844+0000 7f89af280640 1 -- 172.21.15.5:0/2886305447 >> 172.21.15.5:0/2886305447 conn(0x7f89a80fc7d0 msgr2=0x7f89a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.844+0000 7f89af280640 1 -- 172.21.15.5:0/2886305447 shutdown_connections 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.844+0000 7f89af280640 1 -- 172.21.15.5:0/2886305447 wait complete. 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89af280640 1 Processor -- start 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89af280640 1 -- start start 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89af280640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89a8100e20 0x7f89a8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89af280640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a8102570 0x7f89a81998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89af280640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89a8102ef0 0x7f89a81a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89af280640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8110f80 con 0x7f89a8102570 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89af280640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8110e00 con 0x7f89a8102ef0 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89af280640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f89a8111100 con 0x7f89a8100e20 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89ac81b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a8102570 0x7f89a81998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89ac81b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a8102570 0x7f89a81998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38230/0 (socket says 172.21.15.5:38230) 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89ac81b640 1 -- 172.21.15.5:0/2706584738 learned_addr learned my addr 172.21.15.5:0/2706584738 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.845+0000 7f89ad01c640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89a8100e20 0x7f89a8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f89ad81d640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89a8102ef0 0x7f89a81a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f89ac81b640 1 -- 172.21.15.5:0/2706584738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89a8100e20 msgr2=0x7f89a8199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f89ac81b640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89a8100e20 0x7f89a8199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f89ac81b640 1 -- 172.21.15.5:0/2706584738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89a8102ef0 msgr2=0x7f89a81a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f89ac81b640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89a8102ef0 0x7f89a81a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f89ac81b640 1 -- 172.21.15.5:0/2706584738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a81a4930 con 0x7f89a8102570 2024-09-15T22:08:20.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f89ac81b640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a8102570 0x7f89a81998b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f899800d950 tx=0x7f899800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f899e7fc640 1 -- 172.21.15.5:0/2706584738 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998017070 con 0x7f89a8102570 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f899e7fc640 1 -- 172.21.15.5:0/2706584738 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8998009c30 con 0x7f89a8102570 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f899e7fc640 1 -- 172.21.15.5:0/2706584738 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899800f6b0 con 0x7f89a8102570 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f89af280640 1 -- 172.21.15.5:0/2706584738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a81a4b00 con 0x7f89a8102570 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.846+0000 7f89af280640 1 -- 172.21.15.5:0/2706584738 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a81a5030 con 0x7f89a8102570 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.848+0000 7f899e7fc640 1 -- 172.21.15.5:0/2706584738 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f89980109d0 con 0x7f89a8102570 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89b1659740 1 Processor -- start 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89b1659740 1 -- start start 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89b1659740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 0x55a5e3945db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89b1659740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39462f0 0x55a5e3950670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89b1659740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e3950bb0 0x55a5e3952fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89b1659740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a5e38ac090 con 0x55a5e39462f0 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89b1659740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3853a70 con 0x55a5e39459d0 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89b1659740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a5e38a8050 con 0x55a5e3950bb0 2024-09-15T22:08:20.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f899e7fc640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f89940a3350 0x7f89940a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89ad01c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39462f0 0x55a5e3950670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89ad01c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39462f0 0x55a5e3950670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38242/0 (socket says 172.21.15.5:38242) 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89ad01c640 1 -- 172.21.15.5:0/1446827582 learned_addr learned my addr 172.21.15.5:0/1446827582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f899e7fc640 1 -- 172.21.15.5:0/2706584738 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f89980cf910 con 0x7f89a8102570 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.849+0000 7f89ad81d640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f89940a3350 0x7f89940a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.850+0000 7f89ad81d640 1 --2- 172.21.15.5:0/1446827582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 0x55a5e3945db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.850+0000 7f89ad01c640 1 -- 172.21.15.5:0/1446827582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e3950bb0 msgr2=0x55a5e3952fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.850+0000 7f89ad01c640 1 --2- 172.21.15.5:0/1446827582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e3950bb0 0x55a5e3952fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.850+0000 7f89ad01c640 1 -- 172.21.15.5:0/1446827582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 msgr2=0x55a5e3945db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.850+0000 7f89ad01c640 1 --2- 172.21.15.5:0/1446827582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 0x55a5e3945db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.850+0000 7f89ad01c640 1 -- 172.21.15.5:0/1446827582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e3860130 con 0x55a5e39462f0 2024-09-15T22:08:20.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.850+0000 7f89ad01c640 1 --2- 172.21.15.5:0/1446827582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39462f0 0x55a5e3950670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89a40114f0 tx=0x7f89a40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693f0157a9e7ddfe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.850+0000 7f89ad81d640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f89940a3350 0x7f89940a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89a0005940 tx=0x7f89a00058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.850+0000 7f89937fe640 1 -- 172.21.15.5:0/1446827582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a401e070 con 0x55a5e39462f0 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.850+0000 7f89937fe640 1 -- 172.21.15.5:0/1446827582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a400fb00 con 0x55a5e39462f0 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.850+0000 7f89937fe640 1 -- 172.21.15.5:0/1446827582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a4019ac0 con 0x55a5e39462f0 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.851+0000 7f89b1659740 1 -- 172.21.15.5:0/1446827582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39462f0 msgr2=0x55a5e3950670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.851+0000 7f89b1659740 1 --2- 172.21.15.5:0/1446827582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39462f0 0x55a5e3950670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f89a40114f0 tx=0x7f89a40119b0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.851+0000 7f89b1659740 1 -- 172.21.15.5:0/1446827582 shutdown_connections 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.851+0000 7f89b1659740 1 --2- 172.21.15.5:0/1446827582 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e3950bb0 0x55a5e3952fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.851+0000 7f89b1659740 1 --2- 172.21.15.5:0/1446827582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39462f0 0x55a5e3950670 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.851+0000 7f89b1659740 1 --2- 172.21.15.5:0/1446827582 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 0x55a5e3945db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.851+0000 7f89b1659740 1 -- 172.21.15.5:0/1446827582 >> 172.21.15.5:0/1446827582 conn(0x55a5e3941f60 msgr2=0x55a5e3942340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:20.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.851+0000 7f89b1659740 1 -- 172.21.15.5:0/1446827582 shutdown_connections 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.851+0000 7f89b1659740 1 -- 172.21.15.5:0/1446827582 wait complete. 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.851+0000 7f89b1659740 1 -- start start 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.851+0000 7f89b1659740 1 Processor -- start 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89b1659740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 0x55a5e3950460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89b1659740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 0x55a5e394d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89b1659740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e3950bb0 0x55a5e394dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89b1659740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3953780 con 0x55a5e3950bb0 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89b1659740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3866f20 con 0x55a5e39459d0 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89b1659740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3953900 con 0x55a5e39462f0 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89b1659740 10 client.0 _ll_register_callbacks cb 0x55a5e3830030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ad81d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e3950bb0 0x55a5e394dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ad01c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 0x55a5e3950460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ad01c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 0x55a5e3950460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:35024/0 (socket says 172.21.15.5:35024) 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ad81d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e3950bb0 0x55a5e394dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38244/0 (socket says 172.21.15.5:38244) 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ad01c640 1 -- 172.21.15.5:0/3654410799 learned_addr learned my addr 172.21.15.5:0/3654410799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ac81b640 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 0x55a5e394d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ad01c640 1 -- 172.21.15.5:0/3654410799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 msgr2=0x55a5e394d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ad01c640 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 0x55a5e394d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ad01c640 1 -- 172.21.15.5:0/3654410799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e3950bb0 msgr2=0x55a5e394dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ad01c640 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e3950bb0 0x55a5e394dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ad01c640 1 -- 172.21.15.5:0/3654410799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e361d2b0 con 0x55a5e39459d0 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ad81d640 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e3950bb0 0x55a5e394dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.852+0000 7f89ac81b640 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 0x55a5e394d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.853+0000 7f89ad01c640 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 0x55a5e3950460 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f89a40114c0 tx=0x7f89a4019600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97871d6ef120b60f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.853+0000 7f8990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.853+0000 7f8990ff9640 1 -- 172.21.15.5:0/3654410799 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a401e040 con 0x55a5e39459d0 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.853+0000 7f89b1659740 1 -- 172.21.15.5:0/3654410799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89a400fb00 con 0x55a5e39459d0 2024-09-15T22:08:20.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.853+0000 7f8990ff9640 1 -- 172.21.15.5:0/3654410799 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f89a400f7a0 con 0x55a5e39459d0 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.853+0000 7f8990ff9640 1 -- 172.21.15.5:0/3654410799 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f89a4032050 con 0x55a5e39459d0 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.853+0000 7f89b1659740 10 client.24773 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.853+0000 7f89b1659740 1 -- 172.21.15.5:0/3654410799 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a5e395f0d0 con 0x55a5e39459d0 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.853+0000 7f8990ff9640 1 -- 172.21.15.5:0/3654410799 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a400fb00 con 0x55a5e39459d0 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.854+0000 7f8990ff9640 1 -- 172.21.15.5:0/3654410799 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f89a4005040 con 0x55a5e39459d0 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.854+0000 7f89b1659740 10 client.24773 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.854+0000 7f89b1659740 10 client.24773 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.854+0000 7f89b1659740 20 client.24773 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.854+0000 7f89b1659740 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55a5e39614c0 0x55a5e3962930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.854+0000 7f89b1659740 4 client.24773 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.854+0000 7f89b1659740 1 -- 172.21.15.5:0/3654410799 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5e38b9600 con 0x55a5e39614c0 2024-09-15T22:08:20.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.854+0000 7f89ac81b640 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55a5e39614c0 0x55a5e3962930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.854+0000 7f8990ff9640 10 client.24773 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:08:20.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.854+0000 7f89ac81b640 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55a5e39614c0 0x55a5e3962930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=116a863f8b7da05f server_cookie=a06fff02afe0affb in_seq=0 out_seq=0 2024-09-15T22:08:20.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.855+0000 7f8990ff9640 1 -- 172.21.15.5:0/3654410799 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5e38b9600 con 0x55a5e39614c0 2024-09-15T22:08:20.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.855+0000 7f8990ff9640 10 client.24773 handle_command_reply: tid=0 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.855+0000 7f89b1659740 1 client.24773 shutdown 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.855+0000 7f89b1659740 1 -- 172.21.15.5:0/3654410799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 msgr2=0x55a5e3950460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.856+0000 7f89b1659740 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 0x55a5e3950460 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f89a40114c0 tx=0x7f89a4019600 comp rx=0 tx=0).stop 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.856+0000 7f89b1659740 1 -- 172.21.15.5:0/3654410799 shutdown_connections 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.856+0000 7f89b1659740 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55a5e39614c0 0x55a5e3962930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.856+0000 7f89b1659740 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e3950bb0 0x55a5e394dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.856+0000 7f89b1659740 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 0x55a5e394d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.856+0000 7f89b1659740 1 --2- 172.21.15.5:0/3654410799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39459d0 0x55a5e3950460 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.856+0000 7f8990ff9640 0 client.24773 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.856+0000 7f89b1659740 1 -- 172.21.15.5:0/3654410799 >> 172.21.15.5:0/3654410799 conn(0x55a5e3941f60 msgr2=0x55a5e3947c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.856+0000 7f89b1659740 1 -- 172.21.15.5:0/3654410799 shutdown_connections 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.856+0000 7f89b1659740 1 -- 172.21.15.5:0/3654410799 wait complete. 2024-09-15T22:08:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.856+0000 7f89b1659740 20 client.24773 trim_cache size 0 max 16384 2024-09-15T22:08:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89b1659740 1 Processor -- start 2024-09-15T22:08:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89b1659740 1 -- start start 2024-09-15T22:08:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89b1659740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3962930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89b1659740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39462f0 0x55a5e39407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89b1659740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e3950bb0 0x55a5e3940d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89b1659740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3960c70 con 0x55a5e39459d0 2024-09-15T22:08:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89b1659740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3961340 con 0x55a5e39462f0 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89b1659740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3961a10 con 0x55a5e3950bb0 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89ad01c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3962930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89ad01c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3962930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38246/0 (socket says 172.21.15.5:38246) 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89ad01c640 1 -- 172.21.15.5:0/49515296 learned_addr learned my addr 172.21.15.5:0/49515296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89ac81b640 1 --2- 172.21.15.5:0/49515296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39462f0 0x55a5e39407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89ad81d640 1 --2- 172.21.15.5:0/49515296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e3950bb0 0x55a5e3940d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89ad01c640 1 -- 172.21.15.5:0/49515296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e3950bb0 msgr2=0x55a5e3940d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89ad01c640 1 --2- 172.21.15.5:0/49515296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e3950bb0 0x55a5e3940d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89ad01c640 1 -- 172.21.15.5:0/49515296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39462f0 msgr2=0x55a5e39407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89ad01c640 1 --2- 172.21.15.5:0/49515296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39462f0 0x55a5e39407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89ad01c640 1 -- 172.21.15.5:0/49515296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e394e2f0 con 0x55a5e39459d0 2024-09-15T22:08:20.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.874+0000 7f89ac81b640 1 --2- 172.21.15.5:0/49515296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39462f0 0x55a5e39407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f89ad01c640 1 --2- 172.21.15.5:0/49515296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3962930 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f89a4005410 tx=0x7f89a4012ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887f2e35096ad1e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f8992ffd640 1 -- 172.21.15.5:0/49515296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a401e070 con 0x55a5e39459d0 2024-09-15T22:08:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f8992ffd640 1 -- 172.21.15.5:0/49515296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a4016070 con 0x55a5e39459d0 2024-09-15T22:08:20.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f8992ffd640 1 -- 172.21.15.5:0/49515296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a400f880 con 0x55a5e39459d0 2024-09-15T22:08:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f89b1659740 1 -- 172.21.15.5:0/49515296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 msgr2=0x55a5e3962930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f89b1659740 1 --2- 172.21.15.5:0/49515296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3962930 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f89a4005410 tx=0x7f89a4012ec0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f89b1659740 1 -- 172.21.15.5:0/49515296 shutdown_connections 2024-09-15T22:08:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f89b1659740 1 --2- 172.21.15.5:0/49515296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e3950bb0 0x55a5e3940d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f89b1659740 1 --2- 172.21.15.5:0/49515296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e39462f0 0x55a5e39407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f89b1659740 1 --2- 172.21.15.5:0/49515296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3962930 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f89b1659740 1 -- 172.21.15.5:0/49515296 >> 172.21.15.5:0/49515296 conn(0x55a5e3941f60 msgr2=0x55a5e3960540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.875+0000 7f89b1659740 1 -- 172.21.15.5:0/49515296 shutdown_connections 2024-09-15T22:08:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89b1659740 1 -- 172.21.15.5:0/49515296 wait complete. 2024-09-15T22:08:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89b1659740 1 -- start start 2024-09-15T22:08:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89b1659740 1 Processor -- start 2024-09-15T22:08:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89b1659740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3963c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89b1659740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 0x55a5e39641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89b1659740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e3950bb0 0x55a5e3964750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89b1659740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a5e39627b0 con 0x55a5e39459d0 2024-09-15T22:08:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89b1659740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a5e39620e0 con 0x55a5e3950bb0 2024-09-15T22:08:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89b1659740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a5e3962e80 con 0x55a5e39462f0 2024-09-15T22:08:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89b1659740 10 client.0 _ll_register_callbacks cb 0x7f89a401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89ad01c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3963c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89ad01c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3963c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38256/0 (socket says 172.21.15.5:38256) 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.876+0000 7f89ad01c640 1 -- 172.21.15.5:0/2430722708 learned_addr learned my addr 172.21.15.5:0/2430722708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f89ac81b640 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 0x55a5e39641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f89ad81d640 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e3950bb0 0x55a5e3964750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f89ad01c640 1 -- 172.21.15.5:0/2430722708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 msgr2=0x55a5e39641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f89ad01c640 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 0x55a5e39641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f89ad01c640 1 -- 172.21.15.5:0/2430722708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e3950bb0 msgr2=0x55a5e3964750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f89ad01c640 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e3950bb0 0x55a5e3964750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f89ad01c640 1 -- 172.21.15.5:0/2430722708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e39c9eb0 con 0x55a5e39459d0 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f89ac81b640 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 0x55a5e39641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:20.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f89ad81d640 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e3950bb0 0x55a5e3964750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f89ad01c640 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3963c90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89a40099b0 tx=0x7f89a40196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ee5a8027380a3b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f8990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f8990ff9640 1 -- 172.21.15.5:0/2430722708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a401e040 con 0x55a5e39459d0 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f8990ff9640 1 -- 172.21.15.5:0/2430722708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f89a4003b70 con 0x55a5e39459d0 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.877+0000 7f89b1659740 1 -- 172.21.15.5:0/2430722708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89a4016070 con 0x55a5e39459d0 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f8990ff9640 1 -- 172.21.15.5:0/2430722708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f89a4049050 con 0x55a5e39459d0 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f89b1659740 10 client.15054 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f89b1659740 1 -- 172.21.15.5:0/2430722708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a5e39cb1b0 con 0x55a5e39459d0 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f8990ff9640 1 -- 172.21.15.5:0/2430722708 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89a4016070 con 0x55a5e39459d0 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f8990ff9640 1 -- 172.21.15.5:0/2430722708 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f89a4019b00 con 0x55a5e39459d0 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f89b1659740 10 client.15054 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:20.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f89b1659740 10 client.15054 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f89b1659740 20 client.15054 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f89b1659740 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55a5e39cf9d0 0x55a5e39ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f89b1659740 4 client.15054 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f89b1659740 1 -- 172.21.15.5:0/2430722708 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "status"}) -- 0x55a5e38b3b10 con 0x55a5e39cf9d0 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.878+0000 7f89ac81b640 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55a5e39cf9d0 0x55a5e39ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89ac81b640 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55a5e39cf9d0 0x55a5e39ce6e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1a67aebfb065203d server_cookie=75274289f4679a4e in_seq=0 out_seq=0 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f8990ff9640 10 client.15054 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f8990ff9640 1 -- 172.21.15.5:0/2430722708 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a5e38b3b10 con 0x55a5e39cf9d0 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f8990ff9640 10 client.15054 handle_command_reply: tid=0 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 1 client.15054 shutdown 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 1 -- 172.21.15.5:0/2430722708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 msgr2=0x55a5e3963c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3963c90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89a40099b0 tx=0x7f89a40196a0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 1 -- 172.21.15.5:0/2430722708 shutdown_connections 2024-09-15T22:08:20.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55a5e39cf9d0 0x55a5e39ce6e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a5e3950bb0 0x55a5e3964750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a5e39462f0 0x55a5e39641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f8990ff9640 0 client.15054 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:08:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 1 --2- 172.21.15.5:0/2430722708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e39459d0 0x55a5e3963c90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 1 -- 172.21.15.5:0/2430722708 >> 172.21.15.5:0/2430722708 conn(0x55a5e3941f60 msgr2=0x55a5e39407e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 1 -- 172.21.15.5:0/2430722708 shutdown_connections 2024-09-15T22:08:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 1 -- 172.21.15.5:0/2430722708 wait complete. 2024-09-15T22:08:20.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.879+0000 7f89b1659740 20 client.15054 trim_cache size 0 max 16384 2024-09-15T22:08:20.887 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24466, 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 67.152484827999999, 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 24, 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 123.774051774 2024-09-15T22:08:20.888 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:08:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 -- 172.21.15.5:0/2706584738 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f89940a3350 msgr2=0x7f89940a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f89940a3350 0x7f89940a5810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f89a0005940 tx=0x7f89a00058b0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 -- 172.21.15.5:0/2706584738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a8102570 msgr2=0x7f89a81998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a8102570 0x7f89a81998b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f899800d950 tx=0x7f899800de10 comp rx=0 tx=0).stop 2024-09-15T22:08:20.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 -- 172.21.15.5:0/2706584738 shutdown_connections 2024-09-15T22:08:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89a8102ef0 0x7f89a81a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f89940a3350 0x7f89940a5810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89a8102570 0x7f89a81998b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 --2- 172.21.15.5:0/2706584738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f89a8100e20 0x7f89a8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 -- 172.21.15.5:0/2706584738 >> 172.21.15.5:0/2706584738 conn(0x7f89a80fc7d0 msgr2=0x7f89a81068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 -- 172.21.15.5:0/2706584738 shutdown_connections 2024-09-15T22:08:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:20.880+0000 7f89af280640 1 -- 172.21.15.5:0/2706584738 wait complete. 2024-09-15T22:08:21.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T22:08:21.953 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:22.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:21 smithi005 ceph-mon[30944]: pgmap v337: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:22.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:21 smithi099 ceph-mon[34892]: pgmap v337: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:22.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:21 smithi129 ceph-mon[35314]: pgmap v337: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.468+0000 7f46d799a640 1 -- 172.21.15.5:0/3402083209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d0102790 msgr2=0x7f46d01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.468+0000 7f46d799a640 1 --2- 172.21.15.5:0/3402083209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d0102790 0x7f46d01067e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009900 tx=0x7f46c002a410 comp rx=0 tx=0).stop 2024-09-15T22:08:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 -- 172.21.15.5:0/3402083209 shutdown_connections 2024-09-15T22:08:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 --2- 172.21.15.5:0/3402083209 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46d0106f10 0x7f46d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 --2- 172.21.15.5:0/3402083209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d0102790 0x7f46d01067e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 --2- 172.21.15.5:0/3402083209 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d0101de0 0x7f46d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 -- 172.21.15.5:0/3402083209 >> 172.21.15.5:0/3402083209 conn(0x7f46d00fd8c0 msgr2=0x7f46d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 -- 172.21.15.5:0/3402083209 shutdown_connections 2024-09-15T22:08:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 -- 172.21.15.5:0/3402083209 wait complete. 2024-09-15T22:08:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 Processor -- start 2024-09-15T22:08:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 -- start start 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d0101de0 0x7f46d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d0102790 0x7f46d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46d0106f10 0x7f46d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.469+0000 7f46d799a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0111f70 con 0x7f46d0101de0 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d799a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0111df0 con 0x7f46d0102790 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d799a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f46d01120f0 con 0x7f46d0106f10 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d5736640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d0101de0 0x7f46d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d5736640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d0101de0 0x7f46d00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38272/0 (socket says 172.21.15.5:38272) 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d5736640 1 -- 172.21.15.5:0/3669993572 learned_addr learned my addr 172.21.15.5:0/3669993572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d5f37640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46d0106f10 0x7f46d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d4f35640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d0102790 0x7f46d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d5736640 1 -- 172.21.15.5:0/3669993572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46d0106f10 msgr2=0x7f46d0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d5736640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46d0106f10 0x7f46d0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d5736640 1 -- 172.21.15.5:0/3669993572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d0102790 msgr2=0x7f46d0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d5736640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d0102790 0x7f46d0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d5736640 1 -- 172.21.15.5:0/3669993572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d01a6930 con 0x7f46d0101de0 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d5736640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d0101de0 0x7f46d00744f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00d950 tx=0x7f46cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46c67fc640 1 -- 172.21.15.5:0/3669993572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc017070 con 0x7f46d0101de0 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d799a640 1 -- 172.21.15.5:0/3669993572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d01a6c20 con 0x7f46d0101de0 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46c67fc640 1 -- 172.21.15.5:0/3669993572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46cc009c30 con 0x7f46d0101de0 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46c67fc640 1 -- 172.21.15.5:0/3669993572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46cc00f6b0 con 0x7f46d0101de0 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.470+0000 7f46d799a640 1 -- 172.21.15.5:0/3669993572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d01a71b0 con 0x7f46d0101de0 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46c67fc640 1 -- 172.21.15.5:0/3669993572 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f46cc0109d0 con 0x7f46d0101de0 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46d9d73740 1 Processor -- start 2024-09-15T22:08:22.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46d9d73740 1 -- start start 2024-09-15T22:08:22.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555139d0 0x555a55513db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555142f0 0x555a5551e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55520fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46d9d73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555a5547a090 con 0x555a555139d0 2024-09-15T22:08:22.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46d9d73740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555a55421a70 con 0x555a5551ebb0 2024-09-15T22:08:22.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46d9d73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555a55476050 con 0x555a555142f0 2024-09-15T22:08:22.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46d5f37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555139d0 0x555a55513db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46d5f37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555139d0 0x555a55513db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38274/0 (socket says 172.21.15.5:38274) 2024-09-15T22:08:22.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.473+0000 7f46d5f37640 1 -- 172.21.15.5:0/359929058 learned_addr learned my addr 172.21.15.5:0/359929058 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:22.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46c67fc640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46bc0a32d0 0x7f46bc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46d5736640 1 --2- 172.21.15.5:0/359929058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555142f0 0x555a5551e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46d4f35640 1 --2- 172.21.15.5:0/359929058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55520fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46d5f37640 1 -- 172.21.15.5:0/359929058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555142f0 msgr2=0x555a5551e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46d5f37640 1 --2- 172.21.15.5:0/359929058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555142f0 0x555a5551e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46d5f37640 1 -- 172.21.15.5:0/359929058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 msgr2=0x555a55520fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46d5f37640 1 --2- 172.21.15.5:0/359929058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55520fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46d5f37640 1 -- 172.21.15.5:0/359929058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a5542e130 con 0x555a555139d0 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46d4f35640 1 --2- 172.21.15.5:0/359929058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55520fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46c67fc640 1 -- 172.21.15.5:0/3669993572 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f46cc0cfb60 con 0x7f46d0101de0 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46d5f37640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46bc0a32d0 0x7f46bc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46d5f37640 1 --2- 172.21.15.5:0/359929058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555139d0 0x555a55513db0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009fb0 tx=0x7f46c800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb682fd1fc991620 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:22.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46bb7fe640 1 -- 172.21.15.5:0/359929058 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8012e40 con 0x555a555139d0 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46bb7fe640 1 -- 172.21.15.5:0/359929058 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c801c8e0 con 0x555a555139d0 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.474+0000 7f46bb7fe640 1 -- 172.21.15.5:0/359929058 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8010910 con 0x555a555139d0 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.475+0000 7f46d9d73740 1 -- 172.21.15.5:0/359929058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555139d0 msgr2=0x555a55513db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.475+0000 7f46d5f37640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46bc0a32d0 0x7f46bc0a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46c8020000 tx=0x7f46c801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.475+0000 7f46d9d73740 1 --2- 172.21.15.5:0/359929058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555139d0 0x555a55513db0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009fb0 tx=0x7f46c800f7f0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.475+0000 7f46d9d73740 1 -- 172.21.15.5:0/359929058 shutdown_connections 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.475+0000 7f46d9d73740 1 --2- 172.21.15.5:0/359929058 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55520fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.475+0000 7f46d9d73740 1 --2- 172.21.15.5:0/359929058 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555142f0 0x555a5551e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.475+0000 7f46d9d73740 1 --2- 172.21.15.5:0/359929058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555139d0 0x555a55513db0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.475+0000 7f46d9d73740 1 -- 172.21.15.5:0/359929058 >> 172.21.15.5:0/359929058 conn(0x555a5550ff60 msgr2=0x555a55510340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.475+0000 7f46d9d73740 1 -- 172.21.15.5:0/359929058 shutdown_connections 2024-09-15T22:08:22.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.475+0000 7f46d9d73740 1 -- 172.21.15.5:0/359929058 wait complete. 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.475+0000 7f46d9d73740 1 -- start start 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d9d73740 1 Processor -- start 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a555139d0 0x555a5551e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555142f0 0x555a5551b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a5551ebb0 0x555a5551bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d9d73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555a55521780 con 0x555a5551ebb0 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d9d73740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555a55434f20 con 0x555a555139d0 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d9d73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555a55521900 con 0x555a555142f0 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d9d73740 10 client.0 _ll_register_callbacks cb 0x555a553fe030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d5f37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a5551ebb0 0x555a5551bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d5f37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a5551ebb0 0x555a5551bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38282/0 (socket says 172.21.15.5:38282) 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d5f37640 1 -- 172.21.15.5:0/700582528 learned_addr learned my addr 172.21.15.5:0/700582528 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:22.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d4f35640 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555142f0 0x555a5551b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.476+0000 7f46d5736640 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a555139d0 0x555a5551e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46d5f37640 1 -- 172.21.15.5:0/700582528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555142f0 msgr2=0x555a5551b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46d5f37640 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555142f0 0x555a5551b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46d5f37640 1 -- 172.21.15.5:0/700582528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a555139d0 msgr2=0x555a5551e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46d5f37640 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a555139d0 0x555a5551e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46d5f37640 1 -- 172.21.15.5:0/700582528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a551eb2b0 con 0x555a5551ebb0 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46d5736640 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a555139d0 0x555a5551e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46d4f35640 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555142f0 0x555a5551b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46d5f37640 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a5551ebb0 0x555a5551bcc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f46c8005fd0 tx=0x7f46c8010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b553794e246a57 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46b8ff9640 1 -- 172.21.15.5:0/700582528 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c80108a0 con 0x555a5551ebb0 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46b8ff9640 1 -- 172.21.15.5:0/700582528 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f46c801ca90 con 0x555a5551ebb0 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46d9d73740 1 -- 172.21.15.5:0/700582528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c801c8e0 con 0x555a5551ebb0 2024-09-15T22:08:22.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46b8ff9640 1 -- 172.21.15.5:0/700582528 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f46c801cc90 con 0x555a5551ebb0 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46d9d73740 10 client.15066 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46d9d73740 1 -- 172.21.15.5:0/700582528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555a5552d030 con 0x555a5551ebb0 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46b8ff9640 1 -- 172.21.15.5:0/700582528 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c801c8e0 con 0x555a5551ebb0 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.477+0000 7f46b8ff9640 1 -- 172.21.15.5:0/700582528 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f46c802d070 con 0x555a5551ebb0 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.478+0000 7f46d9d73740 10 client.15066 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.478+0000 7f46d9d73740 10 client.15066 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.478+0000 7f46d9d73740 20 client.15066 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.478+0000 7f46d9d73740 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555a5552f3d0 0x555a55530840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.478+0000 7f46d9d73740 4 client.15066 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.478+0000 7f46d9d73740 1 -- 172.21.15.5:0/700582528 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a55487600 con 0x555a5552f3d0 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.478+0000 7f46d5736640 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555a5552f3d0 0x555a55530840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.478+0000 7f46b8ff9640 10 client.15066 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:08:22.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.478+0000 7f46d5736640 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555a5552f3d0 0x555a55530840 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29279064a453d810 server_cookie=90ba9345cf4dc43c in_seq=0 out_seq=0 2024-09-15T22:08:22.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.479+0000 7f46b8ff9640 1 -- 172.21.15.5:0/700582528 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555a55487600 con 0x555a5552f3d0 2024-09-15T22:08:22.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.479+0000 7f46b8ff9640 10 client.15066 handle_command_reply: tid=0 2024-09-15T22:08:22.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.479+0000 7f46d9d73740 1 client.15066 shutdown 2024-09-15T22:08:22.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.479+0000 7f46d9d73740 1 -- 172.21.15.5:0/700582528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a5551ebb0 msgr2=0x555a5551bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:22.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.479+0000 7f46d9d73740 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a5551ebb0 0x555a5551bcc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f46c8005fd0 tx=0x7f46c8010470 comp rx=0 tx=0).stop 2024-09-15T22:08:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.479+0000 7f46d9d73740 1 -- 172.21.15.5:0/700582528 shutdown_connections 2024-09-15T22:08:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.479+0000 7f46d9d73740 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555a5552f3d0 0x555a55530840 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.480+0000 7f46d9d73740 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a5551ebb0 0x555a5551bcc0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.480+0000 7f46d9d73740 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555142f0 0x555a5551b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.480+0000 7f46b8ff9640 0 client.15066 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:08:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.480+0000 7f46d9d73740 1 --2- 172.21.15.5:0/700582528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a555139d0 0x555a5551e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.480+0000 7f46d9d73740 1 -- 172.21.15.5:0/700582528 >> 172.21.15.5:0/700582528 conn(0x555a5550ff60 msgr2=0x555a55515c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.480+0000 7f46d9d73740 1 -- 172.21.15.5:0/700582528 shutdown_connections 2024-09-15T22:08:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.480+0000 7f46d9d73740 1 -- 172.21.15.5:0/700582528 wait complete. 2024-09-15T22:08:22.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.480+0000 7f46d9d73740 20 client.15066 trim_cache size 0 max 16384 2024-09-15T22:08:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d9d73740 1 Processor -- start 2024-09-15T22:08:22.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d9d73740 1 -- start start 2024-09-15T22:08:22.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555139d0 0x555a5552faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55538b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55539060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d9d73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555a5552eb80 con 0x555a555142f0 2024-09-15T22:08:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d9d73740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555a5552f250 con 0x555a5551ebb0 2024-09-15T22:08:22.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d9d73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555a5552f920 con 0x555a555139d0 2024-09-15T22:08:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d4f35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55538b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d4f35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55538b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38284/0 (socket says 172.21.15.5:38284) 2024-09-15T22:08:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d4f35640 1 -- 172.21.15.5:0/3638097831 learned_addr learned my addr 172.21.15.5:0/3638097831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d5736640 1 --2- 172.21.15.5:0/3638097831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555139d0 0x555a5552faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d5f37640 1 --2- 172.21.15.5:0/3638097831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55539060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d4f35640 1 -- 172.21.15.5:0/3638097831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555139d0 msgr2=0x555a5552faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d4f35640 1 --2- 172.21.15.5:0/3638097831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555139d0 0x555a5552faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d4f35640 1 -- 172.21.15.5:0/3638097831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 msgr2=0x555a55539060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.498+0000 7f46d4f35640 1 --2- 172.21.15.5:0/3638097831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55539060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.499+0000 7f46d4f35640 1 -- 172.21.15.5:0/3638097831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a5551c540 con 0x555a555142f0 2024-09-15T22:08:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.499+0000 7f46d4f35640 1 --2- 172.21.15.5:0/3638097831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55538b20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f46c0005940 tx=0x7f46c0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8003b7f2eafb5f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.499+0000 7f46baffd640 1 -- 172.21.15.5:0/3638097831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c003f070 con 0x555a555142f0 2024-09-15T22:08:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.499+0000 7f46baffd640 1 -- 172.21.15.5:0/3638097831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c0004030 con 0x555a555142f0 2024-09-15T22:08:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.499+0000 7f46baffd640 1 -- 172.21.15.5:0/3638097831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0035070 con 0x555a555142f0 2024-09-15T22:08:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.500+0000 7f46d9d73740 1 -- 172.21.15.5:0/3638097831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 msgr2=0x555a55538b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.500+0000 7f46d9d73740 1 --2- 172.21.15.5:0/3638097831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55538b20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f46c0005940 tx=0x7f46c0005dd0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.500+0000 7f46d9d73740 1 -- 172.21.15.5:0/3638097831 shutdown_connections 2024-09-15T22:08:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.500+0000 7f46d9d73740 1 --2- 172.21.15.5:0/3638097831 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55539060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.500+0000 7f46d9d73740 1 --2- 172.21.15.5:0/3638097831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55538b20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.500+0000 7f46d9d73740 1 --2- 172.21.15.5:0/3638097831 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555139d0 0x555a5552faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.500+0000 7f46d9d73740 1 -- 172.21.15.5:0/3638097831 >> 172.21.15.5:0/3638097831 conn(0x555a5550ff60 msgr2=0x555a5550ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.500+0000 7f46d9d73740 1 -- 172.21.15.5:0/3638097831 shutdown_connections 2024-09-15T22:08:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.500+0000 7f46d9d73740 1 -- 172.21.15.5:0/3638097831 wait complete. 2024-09-15T22:08:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.500+0000 7f46d9d73740 1 -- start start 2024-09-15T22:08:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.500+0000 7f46d9d73740 1 Processor -- start 2024-09-15T22:08:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555139d0 0x555a55533c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55534160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d9d73740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55597ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d9d73740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555a555306c0 con 0x555a555142f0 2024-09-15T22:08:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d9d73740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555a5552fff0 con 0x555a5551ebb0 2024-09-15T22:08:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d9d73740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555a55530d90 con 0x555a555139d0 2024-09-15T22:08:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d9d73740 10 client.0 _ll_register_callbacks cb 0x7f46c801f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d4f35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55534160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d4f35640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55534160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38286/0 (socket says 172.21.15.5:38286) 2024-09-15T22:08:22.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d4f35640 1 -- 172.21.15.5:0/1277617483 learned_addr learned my addr 172.21.15.5:0/1277617483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:22.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d5736640 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555139d0 0x555a55533c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d5f37640 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55597ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d4f35640 1 -- 172.21.15.5:0/1277617483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555139d0 msgr2=0x555a55533c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:22.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d4f35640 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555139d0 0x555a55533c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d4f35640 1 -- 172.21.15.5:0/1277617483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 msgr2=0x555a55597ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d4f35640 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55597ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d4f35640 1 -- 172.21.15.5:0/1277617483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a55598010 con 0x555a555142f0 2024-09-15T22:08:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46d4f35640 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55534160 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f46c002a930 tx=0x7f46c002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c791b0493dc997 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.501+0000 7f46b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:08:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46b8ff9640 1 -- 172.21.15.5:0/1277617483 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c0035070 con 0x555a555142f0 2024-09-15T22:08:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46b8ff9640 1 -- 172.21.15.5:0/1277617483 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f46c00315b0 con 0x555a555142f0 2024-09-15T22:08:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46d9d73740 1 -- 172.21.15.5:0/1277617483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c0004030 con 0x555a555142f0 2024-09-15T22:08:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46b8ff9640 1 -- 172.21.15.5:0/1277617483 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f46c0055050 con 0x555a555142f0 2024-09-15T22:08:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46d9d73740 10 client.15072 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46d9d73740 1 -- 172.21.15.5:0/1277617483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555a55599190 con 0x555a555142f0 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46b8ff9640 1 -- 172.21.15.5:0/1277617483 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46c0004030 con 0x555a555142f0 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46b8ff9640 1 -- 172.21.15.5:0/1277617483 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f46c0004750 con 0x555a555142f0 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46d9d73740 10 client.15072 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46d9d73740 10 client.15072 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46d9d73740 20 client.15072 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46d9d73740 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555a5559d910 0x555a5559c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46d9d73740 4 client.15072 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.502+0000 7f46d9d73740 1 -- 172.21.15.5:0/1277617483 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "status"}) -- 0x555a55481b10 con 0x555a5559d910 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.503+0000 7f46d5736640 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555a5559d910 0x555a5559c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.503+0000 7f46b8ff9640 10 client.15072 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.503+0000 7f46d5736640 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555a5559d910 0x555a5559c620 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e2ab26071a0482c server_cookie=500257f153af58fa in_seq=0 out_seq=0 2024-09-15T22:08:22.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.503+0000 7f46b8ff9640 1 -- 172.21.15.5:0/1277617483 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555a55481b10 con 0x555a5559d910 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.503+0000 7f46b8ff9640 10 client.15072 handle_command_reply: tid=0 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.503+0000 7f46d9d73740 1 client.15072 shutdown 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.503+0000 7f46d9d73740 1 -- 172.21.15.5:0/1277617483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 msgr2=0x555a55534160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.503+0000 7f46d9d73740 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55534160 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f46c002a930 tx=0x7f46c002a410 comp rx=0 tx=0).stop 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.504+0000 7f46d9d73740 1 -- 172.21.15.5:0/1277617483 shutdown_connections 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.504+0000 7f46d9d73740 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555a5559d910 0x555a5559c620 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.504+0000 7f46d9d73740 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555a5551ebb0 0x555a55597ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.504+0000 7f46d9d73740 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555a555142f0 0x555a55534160 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.504+0000 7f46b8ff9640 0 client.15072 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.504+0000 7f46d9d73740 1 --2- 172.21.15.5:0/1277617483 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555a555139d0 0x555a55533c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.504+0000 7f46d9d73740 1 -- 172.21.15.5:0/1277617483 >> 172.21.15.5:0/1277617483 conn(0x555a5550ff60 msgr2=0x555a5551ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.504+0000 7f46d9d73740 1 -- 172.21.15.5:0/1277617483 shutdown_connections 2024-09-15T22:08:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.504+0000 7f46d9d73740 1 -- 172.21.15.5:0/1277617483 wait complete. 2024-09-15T22:08:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.504+0000 7f46d9d73740 20 client.15072 trim_cache size 0 max 16384 2024-09-15T22:08:22.517 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:08:22.517 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T22:08:22.517 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-15T22:08:22.517 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24467, 2024-09-15T22:08:22.517 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-15T22:08:22.517 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-15T22:08:22.517 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-15T22:08:22.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 69.785889878000006, 2024-09-15T22:08:22.518 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 24, 2024-09-15T22:08:22.518 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-09-15T22:08:22.518 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:08:22.518 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 127.948700469 2024-09-15T22:08:22.518 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:08:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 -- 172.21.15.5:0/3669993572 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46bc0a32d0 msgr2=0x7f46bc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46bc0a32d0 0x7f46bc0a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f46c8020000 tx=0x7f46c801d000 comp rx=0 tx=0).stop 2024-09-15T22:08:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 -- 172.21.15.5:0/3669993572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d0101de0 msgr2=0x7f46d00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d0101de0 0x7f46d00744f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f46cc00d950 tx=0x7f46cc00de10 comp rx=0 tx=0).stop 2024-09-15T22:08:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 -- 172.21.15.5:0/3669993572 shutdown_connections 2024-09-15T22:08:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f46d0106f10 0x7f46d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f46d0102790 0x7f46d0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f46bc0a32d0 0x7f46bc0a5790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 --2- 172.21.15.5:0/3669993572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46d0101de0 0x7f46d00744f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 -- 172.21.15.5:0/3669993572 >> 172.21.15.5:0/3669993572 conn(0x7f46d00fd8c0 msgr2=0x7f46d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 -- 172.21.15.5:0/3669993572 shutdown_connections 2024-09-15T22:08:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:22.505+0000 7f46d799a640 1 -- 172.21.15.5:0/3669993572 wait complete. 2024-09-15T22:08:23.351 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:08:23.614 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:24.014 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:23 smithi005 ceph-mon[30944]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.095+0000 7f36acfad640 1 -- 172.21.15.5:0/785197659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a8102790 msgr2=0x7f36a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.095+0000 7f36acfad640 1 --2- 172.21.15.5:0/785197659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a8102790 0x7f36a81067e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f369c009900 tx=0x7f369c02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.095+0000 7f36acfad640 1 -- 172.21.15.5:0/785197659 shutdown_connections 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.095+0000 7f36acfad640 1 --2- 172.21.15.5:0/785197659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36a8106f10 0x7f36a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.095+0000 7f36acfad640 1 --2- 172.21.15.5:0/785197659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a8102790 0x7f36a81067e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.095+0000 7f36acfad640 1 --2- 172.21.15.5:0/785197659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36a8101de0 0x7f36a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.095+0000 7f36acfad640 1 -- 172.21.15.5:0/785197659 >> 172.21.15.5:0/785197659 conn(0x7f36a80fd8c0 msgr2=0x7f36a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.095+0000 7f36acfad640 1 -- 172.21.15.5:0/785197659 shutdown_connections 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.095+0000 7f36acfad640 1 -- 172.21.15.5:0/785197659 wait complete. 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.095+0000 7f36acfad640 1 Processor -- start 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36acfad640 1 -- start start 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36acfad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a8101de0 0x7f36a8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36acfad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36a8102790 0x7f36a81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36acfad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36a8106f10 0x7f36a819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36acfad640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36a8111fe0 con 0x7f36a8101de0 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36acfad640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f36a8111e60 con 0x7f36a8102790 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36acfad640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36a8112160 con 0x7f36a8106f10 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a8101de0 0x7f36a8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a8101de0 0x7f36a8198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35368/0 (socket says 172.21.15.5:35368) 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a659c640 1 -- 172.21.15.5:0/3300695943 learned_addr learned my addr 172.21.15.5:0/3300695943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a5d9b640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36a8102790 0x7f36a81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a659c640 1 -- 172.21.15.5:0/3300695943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36a8106f10 msgr2=0x7f36a819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a6d9d640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36a8106f10 0x7f36a819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a659c640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36a8106f10 0x7f36a819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a659c640 1 -- 172.21.15.5:0/3300695943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36a8102790 msgr2=0x7f36a81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a659c640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36a8102790 0x7f36a81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a659c640 1 -- 172.21.15.5:0/3300695943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a819d410 con 0x7f36a8101de0 2024-09-15T22:08:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a5d9b640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36a8102790 0x7f36a81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a6d9d640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36a8106f10 0x7f36a819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.096+0000 7f36a659c640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a8101de0 0x7f36a8198460 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f369800c920 tx=0x7f369800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.097+0000 7f36977fe640 1 -- 172.21.15.5:0/3300695943 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698007c70 con 0x7f36a8101de0 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.097+0000 7f36acfad640 1 -- 172.21.15.5:0/3300695943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a819d700 con 0x7f36a8101de0 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.097+0000 7f36977fe640 1 -- 172.21.15.5:0/3300695943 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3698007e10 con 0x7f36a8101de0 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.097+0000 7f36977fe640 1 -- 172.21.15.5:0/3300695943 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698018890 con 0x7f36a8101de0 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.097+0000 7f36acfad640 1 -- 172.21.15.5:0/3300695943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a81a4fe0 con 0x7f36a8101de0 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.099+0000 7f36977fe640 1 -- 172.21.15.5:0/3300695943 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f3698018a70 con 0x7f36a8101de0 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.099+0000 7f36af386740 1 Processor -- start 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.099+0000 7f36af386740 1 -- start start 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36af386740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841e8d0 0x55987841ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36af386740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841f1f0 0x55987842da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36af386740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x5598784303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36af386740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559878385090 con 0x55987842dfc0 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36af386740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55987832ca70 con 0x55987841e8d0 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36af386740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559878381050 con 0x55987841f1f0 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36a5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x5598784303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36a5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x5598784303b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35378/0 (socket says 172.21.15.5:35378) 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36a5d9b640 1 -- 172.21.15.5:0/1654635270 learned_addr learned my addr 172.21.15.5:0/1654635270 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36a659c640 1 --2- 172.21.15.5:0/1654635270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841f1f0 0x55987842da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36a6d9d640 1 --2- 172.21.15.5:0/1654635270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841e8d0 0x55987841ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36977fe640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36900a32d0 0x7f36900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36a5d9b640 1 -- 172.21.15.5:0/1654635270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841f1f0 msgr2=0x55987842da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36a5d9b640 1 --2- 172.21.15.5:0/1654635270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841f1f0 0x55987842da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36a6d9d640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36900a32d0 0x7f36900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36a5d9b640 1 -- 172.21.15.5:0/1654635270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841e8d0 msgr2=0x55987841ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36a5d9b640 1 --2- 172.21.15.5:0/1654635270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841e8d0 0x55987841ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36a5d9b640 1 -- 172.21.15.5:0/1654635270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559878339130 con 0x55987842dfc0 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.100+0000 7f36977fe640 1 -- 172.21.15.5:0/3300695943 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f36980d00c0 con 0x7f36a8101de0 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f36a5d9b640 1 --2- 172.21.15.5:0/1654635270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x5598784303b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f369c006010 tx=0x7f369c004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6c09b2a032dc8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f36a6d9d640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36900a32d0 0x7f36900a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f36a0013870 tx=0x7f36a0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f3694ff9640 1 -- 172.21.15.5:0/1654635270 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c03f070 con 0x55987842dfc0 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f3694ff9640 1 -- 172.21.15.5:0/1654635270 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f369c005520 con 0x55987842dfc0 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f3694ff9640 1 -- 172.21.15.5:0/1654635270 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c035030 con 0x55987842dfc0 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f36af386740 1 -- 172.21.15.5:0/1654635270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 msgr2=0x5598784303b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f36af386740 1 --2- 172.21.15.5:0/1654635270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x5598784303b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f369c006010 tx=0x7f369c004200 comp rx=0 tx=0).stop 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f36af386740 1 -- 172.21.15.5:0/1654635270 shutdown_connections 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f36af386740 1 --2- 172.21.15.5:0/1654635270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x5598784303b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f36af386740 1 --2- 172.21.15.5:0/1654635270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841f1f0 0x55987842da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f36af386740 1 --2- 172.21.15.5:0/1654635270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841e8d0 0x55987841ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.101+0000 7f36af386740 1 -- 172.21.15.5:0/1654635270 >> 172.21.15.5:0/1654635270 conn(0x55987841af60 msgr2=0x55987841b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36af386740 1 -- 172.21.15.5:0/1654635270 shutdown_connections 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36af386740 1 -- 172.21.15.5:0/1654635270 wait complete. 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36af386740 1 -- start start 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36af386740 1 Processor -- start 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36af386740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x55987842d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36af386740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841f1f0 0x55987842abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36af386740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x55987842b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36af386740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559878430b90 con 0x55987842dfc0 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36af386740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55987833ff20 con 0x55987841f1f0 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36af386740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559878430d10 con 0x55987841e8d0 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36af386740 10 client.0 _ll_register_callbacks cb 0x559878309030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:24.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36a6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x55987842b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36a6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x55987842b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35382/0 (socket says 172.21.15.5:35382) 2024-09-15T22:08:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.102+0000 7f36a6d9d640 1 -- 172.21.15.5:0/3801011409 learned_addr learned my addr 172.21.15.5:0/3801011409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36a5d9b640 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841f1f0 0x55987842abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36a659c640 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x55987842d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36a6d9d640 1 -- 172.21.15.5:0/3801011409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 msgr2=0x55987842d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36a6d9d640 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x55987842d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36a6d9d640 1 -- 172.21.15.5:0/3801011409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841f1f0 msgr2=0x55987842abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36a6d9d640 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841f1f0 0x55987842abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36a6d9d640 1 -- 172.21.15.5:0/3801011409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598780f62b0 con 0x55987842dfc0 2024-09-15T22:08:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36a5d9b640 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841f1f0 0x55987842abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36a659c640 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x55987842d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36a6d9d640 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x55987842b100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f36a0010f70 tx=0x7f36a0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d82609abf1f9dba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f368dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f368dffb640 1 -- 172.21.15.5:0/3801011409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0022070 con 0x55987842dfc0 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f368dffb640 1 -- 172.21.15.5:0/3801011409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f36a001e800 con 0x55987842dfc0 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36af386740 1 -- 172.21.15.5:0/3801011409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f369c005520 con 0x55987842dfc0 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f368dffb640 1 -- 172.21.15.5:0/3801011409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f36a002e050 con 0x55987842dfc0 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36af386740 10 client.15084 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f36af386740 1 -- 172.21.15.5:0/3801011409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55987843c550 con 0x55987842dfc0 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.103+0000 7f368dffb640 1 -- 172.21.15.5:0/3801011409 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f369c005520 con 0x55987842dfc0 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.104+0000 7f368dffb640 1 -- 172.21.15.5:0/3801011409 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f36a001d070 con 0x55987842dfc0 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.104+0000 7f36af386740 10 client.15084 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.104+0000 7f36af386740 10 client.15084 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.104+0000 7f36af386740 20 client.15084 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.104+0000 7f36af386740 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55987843ebb0 0x559878440020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.104+0000 7f36af386740 4 client.15084 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.104+0000 7f36af386740 1 -- 172.21.15.5:0/3801011409 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559878392600 con 0x55987843ebb0 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.104+0000 7f36a659c640 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55987843ebb0 0x559878440020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.104+0000 7f368dffb640 10 client.15084 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.104+0000 7f36a659c640 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55987843ebb0 0x559878440020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1cac8fd89b53e7 server_cookie=383fcc96e82325a5 in_seq=0 out_seq=0 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f368dffb640 1 -- 172.21.15.5:0/3801011409 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559878392600 con 0x55987843ebb0 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f368dffb640 10 client.15084 handle_command_reply: tid=0 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f36af386740 1 client.15084 shutdown 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f36af386740 1 -- 172.21.15.5:0/3801011409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 msgr2=0x55987842b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f36af386740 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x55987842b100 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f36a0010f70 tx=0x7f36a0004b50 comp rx=0 tx=0).stop 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f36af386740 1 -- 172.21.15.5:0/3801011409 shutdown_connections 2024-09-15T22:08:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f36af386740 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x55987842b100 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f36af386740 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55987843ebb0 0x559878440020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f36af386740 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841f1f0 0x55987842abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f36af386740 1 --2- 172.21.15.5:0/3801011409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x55987842d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f368dffb640 0 client.15084 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:08:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f36af386740 1 -- 172.21.15.5:0/3801011409 >> 172.21.15.5:0/3801011409 conn(0x55987841af60 msgr2=0x559878420cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f36af386740 1 -- 172.21.15.5:0/3801011409 shutdown_connections 2024-09-15T22:08:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.105+0000 7f36af386740 1 -- 172.21.15.5:0/3801011409 wait complete. 2024-09-15T22:08:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.106+0000 7f36af386740 20 client.15084 trim_cache size 0 max 16384 2024-09-15T22:08:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.123+0000 7f36af386740 1 Processor -- start 2024-09-15T22:08:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.123+0000 7f36af386740 1 -- start start 2024-09-15T22:08:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36af386740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x559878429490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36af386740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987841f1f0 0x5598784299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36af386740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987842dfc0 0x5598784482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36af386740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55987843e360 con 0x55987841f1f0 2024-09-15T22:08:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36af386740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55987843ea30 con 0x55987842dfc0 2024-09-15T22:08:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36af386740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55987843f100 con 0x55987841e8d0 2024-09-15T22:08:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36a5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987841f1f0 0x5598784299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36a5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987841f1f0 0x5598784299d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35384/0 (socket says 172.21.15.5:35384) 2024-09-15T22:08:24.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36a5d9b640 1 -- 172.21.15.5:0/161501433 learned_addr learned my addr 172.21.15.5:0/161501433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36a5d9b640 1 -- 172.21.15.5:0/161501433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 msgr2=0x559878429490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36a659c640 1 --2- 172.21.15.5:0/161501433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x559878429490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36a5d9b640 1 --2- 172.21.15.5:0/161501433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x559878429490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36a5d9b640 1 -- 172.21.15.5:0/161501433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987842dfc0 msgr2=0x5598784482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36a5d9b640 1 --2- 172.21.15.5:0/161501433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987842dfc0 0x5598784482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36a5d9b640 1 -- 172.21.15.5:0/161501433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55987842dbe0 con 0x55987841f1f0 2024-09-15T22:08:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f36a5d9b640 1 --2- 172.21.15.5:0/161501433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987841f1f0 0x5598784299d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f369c004270 tx=0x7f369c02aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60c96d6f0d656aa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f368ffff640 1 -- 172.21.15.5:0/161501433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c03f050 con 0x55987841f1f0 2024-09-15T22:08:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.124+0000 7f368ffff640 1 -- 172.21.15.5:0/161501433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f369c04d070 con 0x55987841f1f0 2024-09-15T22:08:24.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.125+0000 7f368ffff640 1 -- 172.21.15.5:0/161501433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c035040 con 0x55987841f1f0 2024-09-15T22:08:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.125+0000 7f36af386740 1 -- 172.21.15.5:0/161501433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987841f1f0 msgr2=0x5598784299d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.125+0000 7f36af386740 1 --2- 172.21.15.5:0/161501433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987841f1f0 0x5598784299d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f369c004270 tx=0x7f369c02aab0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.125+0000 7f36af386740 1 -- 172.21.15.5:0/161501433 shutdown_connections 2024-09-15T22:08:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.125+0000 7f36af386740 1 --2- 172.21.15.5:0/161501433 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987842dfc0 0x5598784482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.125+0000 7f36af386740 1 --2- 172.21.15.5:0/161501433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987841f1f0 0x5598784299d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.125+0000 7f36af386740 1 --2- 172.21.15.5:0/161501433 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x559878429490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.125+0000 7f36af386740 1 -- 172.21.15.5:0/161501433 >> 172.21.15.5:0/161501433 conn(0x55987841af60 msgr2=0x55987842ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.125+0000 7f36af386740 1 -- 172.21.15.5:0/161501433 shutdown_connections 2024-09-15T22:08:24.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.125+0000 7f36af386740 1 -- 172.21.15.5:0/161501433 wait complete. 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36af386740 1 -- start start 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36af386740 1 Processor -- start 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36af386740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x559878441470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36af386740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841f1f0 0x5598784419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36af386740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x559878441ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36af386740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55987843fea0 con 0x55987842dfc0 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36af386740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55987843f7d0 con 0x55987841f1f0 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36af386740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559878440570 con 0x55987841e8d0 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36af386740 10 client.0 _ll_register_callbacks cb 0x7f36a0013450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36a6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x559878441ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36a6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x559878441ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35386/0 (socket says 172.21.15.5:35386) 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36a6d9d640 1 -- 172.21.15.5:0/3877741138 learned_addr learned my addr 172.21.15.5:0/3877741138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:24.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36a659c640 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x559878441470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36a5d9b640 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841f1f0 0x5598784419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36a6d9d640 1 -- 172.21.15.5:0/3877741138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 msgr2=0x559878441470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36a6d9d640 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x559878441470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36a6d9d640 1 -- 172.21.15.5:0/3877741138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841f1f0 msgr2=0x5598784419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36a6d9d640 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841f1f0 0x5598784419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.126+0000 7f36a6d9d640 1 -- 172.21.15.5:0/3877741138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598784a7550 con 0x55987842dfc0 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.127+0000 7f36a6d9d640 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x559878441ef0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f36a0011010 tx=0x7f36a0007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203afe29811d58db server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.127+0000 7f368dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.127+0000 7f368dffb640 1 -- 172.21.15.5:0/3877741138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a0037060 con 0x55987842dfc0 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.127+0000 7f368dffb640 1 -- 172.21.15.5:0/3877741138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f36a00267c0 con 0x55987842dfc0 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.127+0000 7f36af386740 1 -- 172.21.15.5:0/3877741138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f369c04d070 con 0x55987842dfc0 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.127+0000 7f368dffb640 1 -- 172.21.15.5:0/3877741138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f36a003e050 con 0x55987842dfc0 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.127+0000 7f36af386740 10 client.15090 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:24.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.127+0000 7f36af386740 1 -- 172.21.15.5:0/3877741138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5598784a8830 con 0x55987842dfc0 2024-09-15T22:08:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.127+0000 7f368dffb640 1 -- 172.21.15.5:0/3877741138 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f369c04d070 con 0x55987842dfc0 2024-09-15T22:08:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.127+0000 7f368dffb640 1 -- 172.21.15.5:0/3877741138 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f36a001d050 con 0x55987842dfc0 2024-09-15T22:08:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.127+0000 7f36af386740 10 client.15090 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.128+0000 7f36af386740 10 client.15090 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:08:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.128+0000 7f36af386740 20 client.15090 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.128+0000 7f36af386740 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5598784ad420 0x5598784ac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.128+0000 7f36af386740 4 client.15090 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:08:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.128+0000 7f36af386740 1 -- 172.21.15.5:0/3877741138 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "status"}) -- 0x55987838cb10 con 0x5598784ad420 2024-09-15T22:08:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.128+0000 7f36a659c640 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5598784ad420 0x5598784ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:24.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.128+0000 7f36a659c640 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5598784ad420 0x5598784ac130 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa21f3068d2f4f15 server_cookie=d2b64aeb310b518e in_seq=0 out_seq=0 2024-09-15T22:08:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.128+0000 7f368dffb640 10 client.15090 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:08:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f368dffb640 1 -- 172.21.15.5:0/3877741138 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55987838cb10 con 0x5598784ad420 2024-09-15T22:08:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f368dffb640 10 client.15090 handle_command_reply: tid=0 2024-09-15T22:08:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 1 client.15090 shutdown 2024-09-15T22:08:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 1 -- 172.21.15.5:0/3877741138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 msgr2=0x559878441ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x559878441ef0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f36a0011010 tx=0x7f36a0007db0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 1 -- 172.21.15.5:0/3877741138 shutdown_connections 2024-09-15T22:08:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55987842dfc0 0x559878441ef0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5598784ad420 0x5598784ac130 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55987841f1f0 0x5598784419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 1 --2- 172.21.15.5:0/3877741138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55987841e8d0 0x559878441470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f368dffb640 0 client.15090 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:08:24.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 1 -- 172.21.15.5:0/3877741138 >> 172.21.15.5:0/3877741138 conn(0x55987841af60 msgr2=0x559878429450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:24.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 1 -- 172.21.15.5:0/3877741138 shutdown_connections 2024-09-15T22:08:24.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 1 -- 172.21.15.5:0/3877741138 wait complete. 2024-09-15T22:08:24.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.129+0000 7f36af386740 20 client.15090 trim_cache size 0 max 16384 2024-09-15T22:08:24.136 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:08:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T22:08:24.136 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-15T22:08:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24491, 2024-09-15T22:08:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-15T22:08:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-15T22:08:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-15T22:08:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 105.939088919, 2024-09-15T22:08:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 24, 2024-09-15T22:08:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-09-15T22:08:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:08:24.137 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 117.074808723 2024-09-15T22:08:24.137 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:08:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 -- 172.21.15.5:0/3300695943 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36900a32d0 msgr2=0x7f36900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36900a32d0 0x7f36900a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f36a0013870 tx=0x7f36a0004a40 comp rx=0 tx=0).stop 2024-09-15T22:08:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 -- 172.21.15.5:0/3300695943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a8101de0 msgr2=0x7f36a8198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a8101de0 0x7f36a8198460 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f369800c920 tx=0x7f369800cde0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 -- 172.21.15.5:0/3300695943 shutdown_connections 2024-09-15T22:08:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36a8106f10 0x7f36a819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36a8102790 0x7f36a81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f36900a32d0 0x7f36900a5790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 --2- 172.21.15.5:0/3300695943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36a8101de0 0x7f36a8198460 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:24.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 -- 172.21.15.5:0/3300695943 >> 172.21.15.5:0/3300695943 conn(0x7f36a80fd8c0 msgr2=0x7f36a80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:24.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 -- 172.21.15.5:0/3300695943 shutdown_connections 2024-09-15T22:08:24.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:24.130+0000 7f36acfad640 1 -- 172.21.15.5:0/3300695943 wait complete. 2024-09-15T22:08:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:23 smithi099 ceph-mon[34892]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:23 smithi129 ceph-mon[35314]: pgmap v338: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:25.003 INFO:tasks.cephadm:Setup complete, yielding 2024-09-15T22:08:25.003 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-15T22:08:25.013 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-15T22:08:25.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-15T22:08:25.275 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:25.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.765+0000 7f4ee5363640 1 -- 172.21.15.5:0/3185500704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee0101c00 msgr2=0x7f4ee0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:25.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.765+0000 7f4ee5363640 1 --2- 172.21.15.5:0/3185500704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee0101c00 0x7f4ee0102000 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009900 tx=0x7f4ed002a410 comp rx=0 tx=0).stop 2024-09-15T22:08:25.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.765+0000 7f4ee5363640 1 -- 172.21.15.5:0/3185500704 shutdown_connections 2024-09-15T22:08:25.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.765+0000 7f4ee5363640 1 --2- 172.21.15.5:0/3185500704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee01037c0 0x7f4ee010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:25.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.765+0000 7f4ee5363640 1 --2- 172.21.15.5:0/3185500704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee0102e00 0x7f4ee0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:25.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.765+0000 7f4ee5363640 1 --2- 172.21.15.5:0/3185500704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee0101c00 0x7f4ee0102000 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:25.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.765+0000 7f4ee5363640 1 -- 172.21.15.5:0/3185500704 >> 172.21.15.5:0/3185500704 conn(0x7f4ee00fd390 msgr2=0x7f4ee00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:25.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.765+0000 7f4ee5363640 1 -- 172.21.15.5:0/3185500704 shutdown_connections 2024-09-15T22:08:25.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.765+0000 7f4ee5363640 1 -- 172.21.15.5:0/3185500704 wait complete. 2024-09-15T22:08:25.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.766+0000 7f4ee5363640 1 Processor -- start 2024-09-15T22:08:25.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.766+0000 7f4ee5363640 1 -- start start 2024-09-15T22:08:25.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.766+0000 7f4ee5363640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee0101c00 0x7f4ee0197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:25.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.766+0000 7f4ee5363640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee0102e00 0x7f4ee0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:25.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.766+0000 7f4ee5363640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee01037c0 0x7f4ee019f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:25.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.766+0000 7f4ee5363640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee010d890 con 0x7f4ee0102e00 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.766+0000 7f4ee5363640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee010d710 con 0x7f4ee0101c00 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.766+0000 7f4ee5363640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee010da10 con 0x7f4ee01037c0 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.766+0000 7f4ed67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee0102e00 0x7f4ee0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.766+0000 7f4ed67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee0102e00 0x7f4ee0198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35402/0 (socket says 172.21.15.5:35402) 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.766+0000 7f4ed67fc640 1 -- 172.21.15.5:0/1666455655 learned_addr learned my addr 172.21.15.5:0/1666455655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4edf7fe640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee01037c0 0x7f4ee019f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4edeffd640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee0101c00 0x7f4ee0197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4ed67fc640 1 -- 172.21.15.5:0/1666455655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee01037c0 msgr2=0x7f4ee019f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4ed67fc640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee01037c0 0x7f4ee019f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4ed67fc640 1 -- 172.21.15.5:0/1666455655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee0101c00 msgr2=0x7f4ee0197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4ed67fc640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee0101c00 0x7f4ee0197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:25.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4ed67fc640 1 -- 172.21.15.5:0/1666455655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee019fa80 con 0x7f4ee0102e00 2024-09-15T22:08:25.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4ed67fc640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee0102e00 0x7f4ee0198390 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800c920 tx=0x7f4ec800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:25.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4edcff9640 1 -- 172.21.15.5:0/1666455655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8007c70 con 0x7f4ee0102e00 2024-09-15T22:08:25.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4edcff9640 1 -- 172.21.15.5:0/1666455655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec8007e10 con 0x7f4ee0102e00 2024-09-15T22:08:25.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4ee5363640 1 -- 172.21.15.5:0/1666455655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee019fd70 con 0x7f4ee0102e00 2024-09-15T22:08:25.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4edcff9640 1 -- 172.21.15.5:0/1666455655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8018890 con 0x7f4ee0102e00 2024-09-15T22:08:25.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.767+0000 7f4ee5363640 1 -- 172.21.15.5:0/1666455655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee01a0270 con 0x7f4ee0102e00 2024-09-15T22:08:25.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.768+0000 7f4ee5363640 1 -- 172.21.15.5:0/1666455655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee0108fb0 con 0x7f4ee0102e00 2024-09-15T22:08:25.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.769+0000 7f4edcff9640 1 -- 172.21.15.5:0/1666455655 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f4ec8018a30 con 0x7f4ee0102e00 2024-09-15T22:08:25.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.770+0000 7f4edcff9640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4ecc0a3380 0x7f4ecc0a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:25.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.770+0000 7f4edeffd640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4ecc0a3380 0x7f4ecc0a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:25.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.770+0000 7f4edcff9640 1 -- 172.21.15.5:0/1666455655 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f4ec80cfe30 con 0x7f4ee0102e00 2024-09-15T22:08:25.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.771+0000 7f4edeffd640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4ecc0a3380 0x7f4ecc0a5840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0005ab0 tx=0x7f4ed0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:25.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.774+0000 7f4edcff9640 1 -- 172.21.15.5:0/1666455655 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ec8014030 con 0x7f4ee0102e00 2024-09-15T22:08:25.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.968+0000 7f4ee5363640 1 -- 172.21.15.5:0/1666455655 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f4ee0062630 con 0x7f4ecc0a3380 2024-09-15T22:08:25.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.969+0000 7f4edcff9640 1 -- 172.21.15.5:0/1666455655 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f4ee0062630 con 0x7f4ecc0a3380 2024-09-15T22:08:25.971 INFO:teuthology.orchestra.run.smithi005.stdout:Backend: cephadm 2024-09-15T22:08:25.971 INFO:teuthology.orchestra.run.smithi005.stdout:Available: Yes 2024-09-15T22:08:25.971 INFO:teuthology.orchestra.run.smithi005.stdout:Paused: No 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.971+0000 7f4ee5363640 1 -- 172.21.15.5:0/1666455655 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4ecc0a3380 msgr2=0x7f4ecc0a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.971+0000 7f4ee5363640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4ecc0a3380 0x7f4ecc0a5840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0005ab0 tx=0x7f4ed0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.972+0000 7f4ee5363640 1 -- 172.21.15.5:0/1666455655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee0102e00 msgr2=0x7f4ee0198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.972+0000 7f4ee5363640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee0102e00 0x7f4ee0198390 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800c920 tx=0x7f4ec800cde0 comp rx=0 tx=0).stop 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.972+0000 7f4ee5363640 1 -- 172.21.15.5:0/1666455655 shutdown_connections 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.972+0000 7f4ee5363640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ee01037c0 0x7f4ee019f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.972+0000 7f4ee5363640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4ecc0a3380 0x7f4ecc0a5840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.972+0000 7f4ee5363640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ee0102e00 0x7f4ee0198390 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.972+0000 7f4ee5363640 1 --2- 172.21.15.5:0/1666455655 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ee0101c00 0x7f4ee0197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.972+0000 7f4ee5363640 1 -- 172.21.15.5:0/1666455655 >> 172.21.15.5:0/1666455655 conn(0x7f4ee00fd390 msgr2=0x7f4ee0106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.972+0000 7f4ee5363640 1 -- 172.21.15.5:0/1666455655 shutdown_connections 2024-09-15T22:08:25.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:25.972+0000 7f4ee5363640 1 -- 172.21.15.5:0/1666455655 wait complete. 2024-09-15T22:08:26.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:25 smithi099 ceph-mon[34892]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:26.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:25 smithi129 ceph-mon[35314]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:26.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:25 smithi005 ceph-mon[30944]: pgmap v339: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:26.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-15T22:08:27.068 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:27.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:26 smithi005 ceph-mon[30944]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:27.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:26 smithi099 ceph-mon[34892]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:27.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:26 smithi129 ceph-mon[35314]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:27.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.557+0000 7fd30fee0640 1 -- 172.21.15.5:0/3038659253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd308102690 msgr2=0x7fd308108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:27.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.557+0000 7fd30fee0640 1 --2- 172.21.15.5:0/3038659253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd308102690 0x7fd308108f20 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd300009ca0 tx=0x7fd30002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:08:27.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.557+0000 7fd30fee0640 1 -- 172.21.15.5:0/3038659253 shutdown_connections 2024-09-15T22:08:27.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.557+0000 7fd30fee0640 1 --2- 172.21.15.5:0/3038659253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd308102690 0x7fd308108f20 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.557+0000 7fd30fee0640 1 --2- 172.21.15.5:0/3038659253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd308101cd0 0x7fd308102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.557+0000 7fd30fee0640 1 --2- 172.21.15.5:0/3038659253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd308100ad0 0x7fd308100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.557+0000 7fd30fee0640 1 -- 172.21.15.5:0/3038659253 >> 172.21.15.5:0/3038659253 conn(0x7fd3080fc280 msgr2=0x7fd3080fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.557+0000 7fd30fee0640 1 -- 172.21.15.5:0/3038659253 shutdown_connections 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.557+0000 7fd30fee0640 1 -- 172.21.15.5:0/3038659253 wait complete. 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.558+0000 7fd30fee0640 1 Processor -- start 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.558+0000 7fd30fee0640 1 -- start start 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.558+0000 7fd30fee0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd308100ad0 0x7fd308104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.558+0000 7fd30fee0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd308101cd0 0x7fd3081030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.558+0000 7fd30fee0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd308102690 0x7fd3081035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.558+0000 7fd30fee0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd30810c730 con 0x7fd308102690 2024-09-15T22:08:27.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.558+0000 7fd30fee0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd30810c5b0 con 0x7fd308101cd0 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.558+0000 7fd30fee0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd30810c8b0 con 0x7fd308100ad0 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.558+0000 7fd30e47d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd308102690 0x7fd3081035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.558+0000 7fd30e47d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd308102690 0x7fd3081035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35428/0 (socket says 172.21.15.5:35428) 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.558+0000 7fd30e47d640 1 -- 172.21.15.5:0/2942808631 learned_addr learned my addr 172.21.15.5:0/2942808631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd30d47b640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd308101cd0 0x7fd3081030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd30dc7c640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd308100ad0 0x7fd308104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd30e47d640 1 -- 172.21.15.5:0/2942808631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd308100ad0 msgr2=0x7fd308104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd30e47d640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd308100ad0 0x7fd308104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd30e47d640 1 -- 172.21.15.5:0/2942808631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd308101cd0 msgr2=0x7fd3081030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd30e47d640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd308101cd0 0x7fd3081030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd30e47d640 1 -- 172.21.15.5:0/2942808631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd308103eb0 con 0x7fd308102690 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd30e47d640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd308102690 0x7fd3081035f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd300031ae0 tx=0x7fd300031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:27.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd2feffd640 1 -- 172.21.15.5:0/2942808631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd300003c30 con 0x7fd308102690 2024-09-15T22:08:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd2feffd640 1 -- 172.21.15.5:0/2942808631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd300003dd0 con 0x7fd308102690 2024-09-15T22:08:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd30fee0640 1 -- 172.21.15.5:0/2942808631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3081a1f40 con 0x7fd308102690 2024-09-15T22:08:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd2feffd640 1 -- 172.21.15.5:0/2942808631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30002fcb0 con 0x7fd308102690 2024-09-15T22:08:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.559+0000 7fd30fee0640 1 -- 172.21.15.5:0/2942808631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3081a23f0 con 0x7fd308102690 2024-09-15T22:08:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.560+0000 7fd30fee0640 1 -- 172.21.15.5:0/2942808631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3081a2620 con 0x7fd308102690 2024-09-15T22:08:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.561+0000 7fd2feffd640 1 -- 172.21.15.5:0/2942808631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fd300004050 con 0x7fd308102690 2024-09-15T22:08:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.562+0000 7fd2feffd640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd2f40a32d0 0x7fd2f40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.563+0000 7fd30dc7c640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd2f40a32d0 0x7fd2f40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.563+0000 7fd2feffd640 1 -- 172.21.15.5:0/2942808631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fd3000f1310 con 0x7fd308102690 2024-09-15T22:08:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.563+0000 7fd30dc7c640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd2f40a32d0 0x7fd2f40a5790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd308101b30 tx=0x7fd304005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:27.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.566+0000 7fd2feffd640 1 -- 172.21.15.5:0/2942808631 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3000bdbc0 con 0x7fd308102690 2024-09-15T22:08:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.749+0000 7fd30fee0640 1 -- 172.21.15.5:0/2942808631 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd3081a27f0 con 0x7fd2f40a32d0 2024-09-15T22:08:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.761+0000 7fd2feffd640 1 -- 172.21.15.5:0/2942808631 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7fd3081a27f0 con 0x7fd2f40a32d0 2024-09-15T22:08:27.763 INFO:teuthology.orchestra.run.smithi005.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-15T22:08:27.763 INFO:teuthology.orchestra.run.smithi005.stdout:mds.a smithi005 running (2m) 53s ago 2m 13.5M - 19.1.1-326-g8921c8ef f1455bd0219c 18bcf8c0d601 2024-09-15T22:08:27.763 INFO:teuthology.orchestra.run.smithi005.stdout:mds.b smithi099 running (2m) 54s ago 2m 25.9M - 19.1.1-326-g8921c8ef f1455bd0219c 51a3dc6feead 2024-09-15T22:08:27.763 INFO:teuthology.orchestra.run.smithi005.stdout:mds.c smithi129 running (2m) 54s ago 2m 16.8M - 19.1.1-326-g8921c8ef f1455bd0219c d13e9bfd1eef 2024-09-15T22:08:27.763 INFO:teuthology.orchestra.run.smithi005.stdout:mds.d smithi005 running (2m) 53s ago 2m 14.3M - 19.1.1-326-g8921c8ef f1455bd0219c ab9a334f36f0 2024-09-15T22:08:27.763 INFO:teuthology.orchestra.run.smithi005.stdout:mds.e smithi099 running (2m) 54s ago 2m 15.0M - 19.1.1-326-g8921c8ef f1455bd0219c 3c51429a29f3 2024-09-15T22:08:27.763 INFO:teuthology.orchestra.run.smithi005.stdout:mds.f smithi129 running (2m) 54s ago 2m 18.0M - 19.1.1-326-g8921c8ef f1455bd0219c 44bb71243fa8 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mds.g smithi005 running (2m) 53s ago 2m 14.3M - 19.1.1-326-g8921c8ef f1455bd0219c 99ebb54bd236 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mds.h smithi099 running (2m) 54s ago 2m 17.5M - 19.1.1-326-g8921c8ef f1455bd0219c 6f5ba94cdc81 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mds.i smithi129 running (2m) 54s ago 2m 17.5M - 19.1.1-326-g8921c8ef f1455bd0219c 54daa7354c84 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mds.j smithi005 running (2m) 53s ago 2m 16.0M - 19.1.1-326-g8921c8ef f1455bd0219c f40634a25b83 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mds.k smithi099 running (2m) 54s ago 2m 14.5M - 19.1.1-326-g8921c8ef f1455bd0219c 7f7f5a520440 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mds.l smithi129 running (2m) 54s ago 2m 13.6M - 19.1.1-326-g8921c8ef f1455bd0219c e98fd1bf7eed 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.x smithi005 *:9283,8765 running (9m) 53s ago 9m 510M - 19.1.1-326-g8921c8ef f1455bd0219c 4b0c5ec44a62 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.y smithi099 *:8443,8765 running (7m) 54s ago 7m 444M - 19.1.1-326-g8921c8ef f1455bd0219c d366fade87e0 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.z smithi129 *:8443,8765 running (7m) 54s ago 7m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 1ca9519e701b 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a smithi005 running (9m) 53s ago 9m 50.6M 2048M 19.1.1-326-g8921c8ef f1455bd0219c e647ceea0e91 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b smithi099 running (8m) 54s ago 8m 40.4M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 4f087ea6937a 2024-09-15T22:08:27.764 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c smithi129 running (8m) 54s ago 8m 41.7M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 43f1b4bee469 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 smithi005 running (7m) 53s ago 7m 53.6M 4273M 19.1.1-326-g8921c8ef f1455bd0219c dbbba2aea544 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 smithi099 running (7m) 54s ago 7m 55.3M 4272M 19.1.1-326-g8921c8ef f1455bd0219c f8feb1ca0875 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 smithi129 running (6m) 54s ago 6m 46.1M 4272M 19.1.1-326-g8921c8ef f1455bd0219c fc68eca74413 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 smithi005 running (6m) 53s ago 6m 53.3M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 67b7b99eb6a0 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 smithi099 running (5m) 54s ago 5m 51.2M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 5ba73cbb2aea 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 smithi129 running (5m) 54s ago 5m 55.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 13a4ccd5f724 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 smithi005 running (4m) 53s ago 4m 42.1M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 217772f3cd40 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 smithi099 running (4m) 54s ago 4m 44.8M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 573ca3113b92 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 smithi129 running (4m) 54s ago 4m 43.8M 4272M 19.1.1-326-g8921c8ef f1455bd0219c de161a3aed44 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 smithi005 running (3m) 53s ago 3m 50.2M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 9ef01b984b60 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 smithi099 running (3m) 54s ago 3m 52.0M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 9011664cffc8 2024-09-15T22:08:27.765 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 smithi129 running (2m) 54s ago 2m 48.8M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 1281fae7a845 2024-09-15T22:08:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.763+0000 7fd30fee0640 1 -- 172.21.15.5:0/2942808631 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd2f40a32d0 msgr2=0x7fd2f40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.763+0000 7fd30fee0640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd2f40a32d0 0x7fd2f40a5790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd308101b30 tx=0x7fd304005f90 comp rx=0 tx=0).stop 2024-09-15T22:08:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.763+0000 7fd30fee0640 1 -- 172.21.15.5:0/2942808631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd308102690 msgr2=0x7fd3081035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.763+0000 7fd30fee0640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd308102690 0x7fd3081035f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd300031ae0 tx=0x7fd300031ee0 comp rx=0 tx=0).stop 2024-09-15T22:08:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.763+0000 7fd30fee0640 1 -- 172.21.15.5:0/2942808631 shutdown_connections 2024-09-15T22:08:27.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.763+0000 7fd30fee0640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd2f40a32d0 0x7fd2f40a5790 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.763+0000 7fd30fee0640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd308102690 0x7fd3081035f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.763+0000 7fd30fee0640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd308101cd0 0x7fd3081030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.763+0000 7fd30fee0640 1 --2- 172.21.15.5:0/2942808631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd308100ad0 0x7fd308104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.763+0000 7fd30fee0640 1 -- 172.21.15.5:0/2942808631 >> 172.21.15.5:0/2942808631 conn(0x7fd3080fc280 msgr2=0x7fd3080fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.764+0000 7fd30fee0640 1 -- 172.21.15.5:0/2942808631 shutdown_connections 2024-09-15T22:08:27.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:27.764+0000 7fd30fee0640 1 -- 172.21.15.5:0/2942808631 wait complete. 2024-09-15T22:08:28.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:27 smithi005 ceph-mon[30944]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:28.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:27 smithi099 ceph-mon[34892]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:28.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:27 smithi129 ceph-mon[35314]: pgmap v340: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:28.657 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-15T22:08:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:29.210 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:28 smithi005 ceph-mon[30944]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:29.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:28 smithi099 ceph-mon[34892]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:29.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:28 smithi129 ceph-mon[35314]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:29.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.415+0000 7fa2b8f19640 1 -- 172.21.15.5:0/2081582184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b4102e00 msgr2=0x7fa2b4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.415+0000 7fa2b8f19640 1 --2- 172.21.15.5:0/2081582184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b4102e00 0x7fa2b4103280 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009900 tx=0x7fa29c02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.415+0000 7fa2b8f19640 1 -- 172.21.15.5:0/2081582184 shutdown_connections 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.415+0000 7fa2b8f19640 1 --2- 172.21.15.5:0/2081582184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2b41037c0 0x7fa2b410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.415+0000 7fa2b8f19640 1 --2- 172.21.15.5:0/2081582184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b4102e00 0x7fa2b4103280 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.415+0000 7fa2b8f19640 1 --2- 172.21.15.5:0/2081582184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b4101c00 0x7fa2b4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.415+0000 7fa2b8f19640 1 -- 172.21.15.5:0/2081582184 >> 172.21.15.5:0/2081582184 conn(0x7fa2b40fd390 msgr2=0x7fa2b40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.415+0000 7fa2b8f19640 1 -- 172.21.15.5:0/2081582184 shutdown_connections 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.415+0000 7fa2b8f19640 1 -- 172.21.15.5:0/2081582184 wait complete. 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.416+0000 7fa2b8f19640 1 Processor -- start 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.416+0000 7fa2b8f19640 1 -- start start 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.416+0000 7fa2b8f19640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2b4101c00 0x7fa2b4197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:29.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b8f19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b4102e00 0x7fa2b4198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b8f19640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b41037c0 0x7fa2b419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b8f19640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b410d880 con 0x7fa2b41037c0 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b8f19640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b410d700 con 0x7fa2b4102e00 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b8f19640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa2b410da00 con 0x7fa2b4101c00 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b41037c0 0x7fa2b419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b41037c0 0x7fa2b419f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35448/0 (socket says 172.21.15.5:35448) 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b2d9d640 1 -- 172.21.15.5:0/1084699146 learned_addr learned my addr 172.21.15.5:0/1084699146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b1d9b640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b4102e00 0x7fa2b4198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b259c640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2b4101c00 0x7fa2b4197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b2d9d640 1 -- 172.21.15.5:0/1084699146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2b4101c00 msgr2=0x7fa2b4197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b2d9d640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2b4101c00 0x7fa2b4197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b2d9d640 1 -- 172.21.15.5:0/1084699146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b4102e00 msgr2=0x7fa2b4198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:29.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b2d9d640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b4102e00 0x7fa2b4198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b2d9d640 1 -- 172.21.15.5:0/1084699146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2b419fa70 con 0x7fa2b41037c0 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b1d9b640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b4102e00 0x7fa2b4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b2d9d640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b41037c0 0x7fa2b419f400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00d750 tx=0x7fa2ac00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2ab7fe640 1 -- 172.21.15.5:0/1084699146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac017070 con 0x7fa2b41037c0 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2ab7fe640 1 -- 172.21.15.5:0/1084699146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2ac005500 con 0x7fa2b41037c0 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b8f19640 1 -- 172.21.15.5:0/1084699146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2b419fd60 con 0x7fa2b41037c0 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2ab7fe640 1 -- 172.21.15.5:0/1084699146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac004d90 con 0x7fa2b41037c0 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.417+0000 7fa2b8f19640 1 -- 172.21.15.5:0/1084699146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2b41a0210 con 0x7fa2b41037c0 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.419+0000 7fa2b8f19640 1 -- 172.21.15.5:0/1084699146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2b4108fb0 con 0x7fa2b41037c0 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.420+0000 7fa2ab7fe640 1 -- 172.21.15.5:0/1084699146 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fa2ac004f30 con 0x7fa2b41037c0 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.421+0000 7fa2ab7fe640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa2a00a3300 0x7fa2a00a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.421+0000 7fa2b259c640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa2a00a3300 0x7fa2a00a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.421+0000 7fa2ab7fe640 1 -- 172.21.15.5:0/1084699146 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fa2ac0cf910 con 0x7fa2b41037c0 2024-09-15T22:08:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.422+0000 7fa2b259c640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa2a00a3300 0x7fa2a00a57c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4005c40 tx=0x7fa2a4005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.426+0000 7fa2ab7fe640 1 -- 172.21.15.5:0/1084699146 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2ac09c0b0 con 0x7fa2b41037c0 2024-09-15T22:08:29.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.609+0000 7fa2b8f19640 1 -- 172.21.15.5:0/1084699146 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fa2b4062630 con 0x7fa2a00a3300 2024-09-15T22:08:29.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.615+0000 7fa2ab7fe640 1 -- 172.21.15.5:0/1084699146 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fa2b4062630 con 0x7fa2a00a3300 2024-09-15T22:08:29.616 INFO:teuthology.orchestra.run.smithi005.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-15T22:08:29.616 INFO:teuthology.orchestra.run.smithi005.stdout:mds.all 12/12 56s ago 2m smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi099=b;smithi099=e;smithi099=h;smithi099=k;smithi129=c;smithi129=f;smithi129=i;smithi129=l;count:12 2024-09-15T22:08:29.616 INFO:teuthology.orchestra.run.smithi005.stdout:mgr 3/3 56s ago 7m smithi005=x;smithi099=y;smithi129=z;count:3 2024-09-15T22:08:29.616 INFO:teuthology.orchestra.run.smithi005.stdout:mon 3/3 56s ago 8m smithi005:172.21.15.5=a;smithi099:172.21.15.99=b;smithi129:172.21.15.129=c;count:3 2024-09-15T22:08:29.616 INFO:teuthology.orchestra.run.smithi005.stdout:osd 12 56s ago - 2024-09-15T22:08:29.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 -- 172.21.15.5:0/1084699146 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa2a00a3300 msgr2=0x7fa2a00a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:29.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa2a00a3300 0x7fa2a00a57c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4005c40 tx=0x7fa2a4005bd0 comp rx=0 tx=0).stop 2024-09-15T22:08:29.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 -- 172.21.15.5:0/1084699146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b41037c0 msgr2=0x7fa2b419f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:29.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b41037c0 0x7fa2b419f400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00d750 tx=0x7fa2ac00dc10 comp rx=0 tx=0).stop 2024-09-15T22:08:29.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 -- 172.21.15.5:0/1084699146 shutdown_connections 2024-09-15T22:08:29.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fa2a00a3300 0x7fa2a00a57c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:29.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2b41037c0 0x7fa2b419f400 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:29.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa2b4102e00 0x7fa2b4198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:29.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 --2- 172.21.15.5:0/1084699146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa2b4101c00 0x7fa2b4197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:29.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 -- 172.21.15.5:0/1084699146 >> 172.21.15.5:0/1084699146 conn(0x7fa2b40fd390 msgr2=0x7fa2b4106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:29.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 -- 172.21.15.5:0/1084699146 shutdown_connections 2024-09-15T22:08:29.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:29.617+0000 7fa2b8f19640 1 -- 172.21.15.5:0/1084699146 wait complete. 2024-09-15T22:08:30.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:29 smithi099 ceph-mon[34892]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:30.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:29 smithi099 ceph-mon[34892]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:30.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:29 smithi129 ceph-mon[35314]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:30.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:29 smithi129 ceph-mon[35314]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:30.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:29 smithi005 ceph-mon[30944]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:30.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:29 smithi005 ceph-mon[30944]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:30.439 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-15T22:08:30.700 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:31.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.317+0000 7f28845fe640 1 -- 172.21.15.5:0/274904707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c102e00 msgr2=0x7f287c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:31.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.317+0000 7f28845fe640 1 --2- 172.21.15.5:0/274904707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c102e00 0x7f287c103280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f286c009900 tx=0x7f286c02a410 comp rx=0 tx=0).stop 2024-09-15T22:08:31.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.317+0000 7f28845fe640 1 -- 172.21.15.5:0/274904707 shutdown_connections 2024-09-15T22:08:31.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.317+0000 7f28845fe640 1 --2- 172.21.15.5:0/274904707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f287c1037c0 0x7f287c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:31.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.317+0000 7f28845fe640 1 --2- 172.21.15.5:0/274904707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c102e00 0x7f287c103280 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:31.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.317+0000 7f28845fe640 1 --2- 172.21.15.5:0/274904707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f287c101c00 0x7f287c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:31.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.317+0000 7f28845fe640 1 -- 172.21.15.5:0/274904707 >> 172.21.15.5:0/274904707 conn(0x7f287c0fd390 msgr2=0x7f287c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.317+0000 7f28845fe640 1 -- 172.21.15.5:0/274904707 shutdown_connections 2024-09-15T22:08:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.317+0000 7f28845fe640 1 -- 172.21.15.5:0/274904707 wait complete. 2024-09-15T22:08:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f28845fe640 1 Processor -- start 2024-09-15T22:08:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f28845fe640 1 -- start start 2024-09-15T22:08:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f28845fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f287c101c00 0x7f287c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f28845fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c102e00 0x7f287c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f28845fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f287c1037c0 0x7f287c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f28845fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f287c10d900 con 0x7f287c102e00 2024-09-15T22:08:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f28845fe640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f287c10d780 con 0x7f287c101c00 2024-09-15T22:08:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f28845fe640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f287c10da80 con 0x7f287c1037c0 2024-09-15T22:08:31.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f2881b99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c102e00 0x7f287c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f2881b99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c102e00 0x7f287c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35474/0 (socket says 172.21.15.5:35474) 2024-09-15T22:08:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f2881b99640 1 -- 172.21.15.5:0/2565073788 learned_addr learned my addr 172.21.15.5:0/2565073788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f288239a640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f287c101c00 0x7f287c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f2882b9b640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f287c1037c0 0x7f287c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f2881b99640 1 -- 172.21.15.5:0/2565073788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f287c1037c0 msgr2=0x7f287c19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f2881b99640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f287c1037c0 0x7f287c19f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f2881b99640 1 -- 172.21.15.5:0/2565073788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f287c101c00 msgr2=0x7f287c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f2881b99640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f287c101c00 0x7f287c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.318+0000 7f2881b99640 1 -- 172.21.15.5:0/2565073788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c19fb60 con 0x7f287c102e00 2024-09-15T22:08:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.319+0000 7f2881b99640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c102e00 0x7f287c198470 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f286c009230 tx=0x7f286c004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.319+0000 7f28737fe640 1 -- 172.21.15.5:0/2565073788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c03f030 con 0x7f287c102e00 2024-09-15T22:08:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.319+0000 7f28845fe640 1 -- 172.21.15.5:0/2565073788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287c19fdf0 con 0x7f287c102e00 2024-09-15T22:08:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.319+0000 7f28737fe640 1 -- 172.21.15.5:0/2565073788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f286c003ce0 con 0x7f287c102e00 2024-09-15T22:08:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.319+0000 7f28737fe640 1 -- 172.21.15.5:0/2565073788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c035070 con 0x7f287c102e00 2024-09-15T22:08:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.319+0000 7f28845fe640 1 -- 172.21.15.5:0/2565073788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287c1a02c0 con 0x7f287c102e00 2024-09-15T22:08:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.320+0000 7f28845fe640 1 -- 172.21.15.5:0/2565073788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287c108fb0 con 0x7f287c102e00 2024-09-15T22:08:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.321+0000 7f28737fe640 1 -- 172.21.15.5:0/2565073788 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f286c042030 con 0x7f287c102e00 2024-09-15T22:08:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.322+0000 7f28737fe640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f28680a3300 0x7f28680a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.322+0000 7f288239a640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f28680a3300 0x7f28680a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.322+0000 7f28737fe640 1 -- 172.21.15.5:0/2565073788 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f286c0f0550 con 0x7f287c102e00 2024-09-15T22:08:31.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.322+0000 7f288239a640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f28680a3300 0x7f28680a57c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f287c102c60 tx=0x7f2878009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:31.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.326+0000 7f28737fe640 1 -- 172.21.15.5:0/2565073788 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f286c0bcd50 con 0x7f287c102e00 2024-09-15T22:08:31.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.507+0000 7f28845fe640 1 -- 172.21.15.5:0/2565073788 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f287c062630 con 0x7f28680a3300 2024-09-15T22:08:31.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.508+0000 7f28737fe640 1 -- 172.21.15.5:0/2565073788 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f287c062630 con 0x7f28680a3300 2024-09-15T22:08:31.510 INFO:teuthology.orchestra.run.smithi005.stdout:HOST ADDR LABELS STATUS 2024-09-15T22:08:31.510 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 172.21.15.5 2024-09-15T22:08:31.510 INFO:teuthology.orchestra.run.smithi005.stdout:smithi099 172.21.15.99 2024-09-15T22:08:31.510 INFO:teuthology.orchestra.run.smithi005.stdout:smithi129 172.21.15.129 2024-09-15T22:08:31.510 INFO:teuthology.orchestra.run.smithi005.stdout:3 hosts in cluster 2024-09-15T22:08:31.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 -- 172.21.15.5:0/2565073788 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f28680a3300 msgr2=0x7f28680a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:31.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f28680a3300 0x7f28680a57c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f287c102c60 tx=0x7f2878009110 comp rx=0 tx=0).stop 2024-09-15T22:08:31.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 -- 172.21.15.5:0/2565073788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c102e00 msgr2=0x7f287c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:31.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c102e00 0x7f287c198470 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f286c009230 tx=0x7f286c004430 comp rx=0 tx=0).stop 2024-09-15T22:08:31.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 -- 172.21.15.5:0/2565073788 shutdown_connections 2024-09-15T22:08:31.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f287c1037c0 0x7f287c19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:31.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f28680a3300 0x7f28680a57c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:31.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f287c102e00 0x7f287c198470 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:31.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 --2- 172.21.15.5:0/2565073788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f287c101c00 0x7f287c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:31.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 -- 172.21.15.5:0/2565073788 >> 172.21.15.5:0/2565073788 conn(0x7f287c0fd390 msgr2=0x7f287c106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:31.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 -- 172.21.15.5:0/2565073788 shutdown_connections 2024-09-15T22:08:31.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:31.510+0000 7f28845fe640 1 -- 172.21.15.5:0/2565073788 wait complete. 2024-09-15T22:08:32.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:31 smithi005 ceph-mon[30944]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:32.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:31 smithi005 ceph-mon[30944]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:32.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-15T22:08:32.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:31 smithi099 ceph-mon[34892]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:32.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:31 smithi099 ceph-mon[34892]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:32.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:31 smithi129 ceph-mon[35314]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:32.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:31 smithi129 ceph-mon[35314]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:32.446 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:32.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.918+0000 7f559e3c9640 1 -- 172.21.15.5:0/1613968179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5598105050 msgr2=0x7f5598107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.918+0000 7f559e3c9640 1 --2- 172.21.15.5:0/1613968179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5598105050 0x7f5598107440 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5580009900 tx=0x7f558002a410 comp rx=0 tx=0).stop 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.918+0000 7f559e3c9640 1 -- 172.21.15.5:0/1613968179 shutdown_connections 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.918+0000 7f559e3c9640 1 --2- 172.21.15.5:0/1613968179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5598107980 0x7f5598109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.918+0000 7f559e3c9640 1 --2- 172.21.15.5:0/1613968179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5598105050 0x7f5598107440 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.918+0000 7f559e3c9640 1 --2- 172.21.15.5:0/1613968179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5598102720 0x7f5598104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.918+0000 7f559e3c9640 1 -- 172.21.15.5:0/1613968179 >> 172.21.15.5:0/1613968179 conn(0x7f55980fc280 msgr2=0x7f55980fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.918+0000 7f559e3c9640 1 -- 172.21.15.5:0/1613968179 shutdown_connections 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.918+0000 7f559e3c9640 1 -- 172.21.15.5:0/1613968179 wait complete. 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.918+0000 7f559e3c9640 1 Processor -- start 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.918+0000 7f559e3c9640 1 -- start start 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f559e3c9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5598102720 0x7f5598197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:32.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f559e3c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5598105050 0x7f55981984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f559e3c9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5598107980 0x7f559819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f559e3c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f559810d8c0 con 0x7f5598105050 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f559e3c9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f559810d740 con 0x7f5598102720 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f559e3c9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f559810da40 con 0x7f5598107980 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f558ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5598105050 0x7f55981984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f558ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5598105050 0x7f55981984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35490/0 (socket says 172.21.15.5:35490) 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f558ffff640 1 -- 172.21.15.5:0/2470080234 learned_addr learned my addr 172.21.15.5:0/2470080234 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f559c966640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5598107980 0x7f559819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f5597fff640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5598102720 0x7f5598197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f558ffff640 1 -- 172.21.15.5:0/2470080234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5598107980 msgr2=0x7f559819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f558ffff640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5598107980 0x7f559819f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:32.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f558ffff640 1 -- 172.21.15.5:0/2470080234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5598102720 msgr2=0x7f5598197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f558ffff640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5598102720 0x7f5598197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f558ffff640 1 -- 172.21.15.5:0/2470080234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559819fbd0 con 0x7f5598105050 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f5597fff640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5598102720 0x7f5598197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f558ffff640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5598105050 0x7f55981984e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5580031610 tx=0x7f55800043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.919+0000 7f5595ffb640 1 -- 172.21.15.5:0/2470080234 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558003f030 con 0x7f5598105050 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.920+0000 7f5595ffb640 1 -- 172.21.15.5:0/2470080234 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5580003ce0 con 0x7f5598105050 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.920+0000 7f559e3c9640 1 -- 172.21.15.5:0/2470080234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559819fe60 con 0x7f5598105050 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.920+0000 7f5595ffb640 1 -- 172.21.15.5:0/2470080234 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580035070 con 0x7f5598105050 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.920+0000 7f559e3c9640 1 -- 172.21.15.5:0/2470080234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55981a02c0 con 0x7f5598105050 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.921+0000 7f559e3c9640 1 -- 172.21.15.5:0/2470080234 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55980fdd70 con 0x7f5598105050 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.922+0000 7f5595ffb640 1 -- 172.21.15.5:0/2470080234 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f5580042030 con 0x7f5598105050 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.922+0000 7f5595ffb640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55840a3380 0x7f55840a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.923+0000 7f5597fff640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55840a3380 0x7f55840a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:32.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.923+0000 7f5595ffb640 1 -- 172.21.15.5:0/2470080234 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f55800f03b0 con 0x7f5598105050 2024-09-15T22:08:32.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.923+0000 7f5597fff640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55840a3380 0x7f55840a5840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5588005d00 tx=0x7f5588005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:32.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:32.927+0000 7f5595ffb640 1 -- 172.21.15.5:0/2470080234 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55800bcc50 con 0x7f5598105050 2024-09-15T22:08:33.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.109+0000 7f559e3c9640 1 -- 172.21.15.5:0/2470080234 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f5598062630 con 0x7f55840a3380 2024-09-15T22:08:33.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.111+0000 7f5595ffb640 1 -- 172.21.15.5:0/2470080234 <== mgr.14150 v2:172.21.15.5:6800/801026306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f5598062630 con 0x7f55840a3380 2024-09-15T22:08:33.112 INFO:teuthology.orchestra.run.smithi005.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-15T22:08:33.112 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EW480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-15T22:08:33.112 INFO:teuthology.orchestra.run.smithi005.stdout:smithi099 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040101400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-15T22:08:33.113 INFO:teuthology.orchestra.run.smithi005.stdout:smithi129 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300HY400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-15T22:08:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 -- 172.21.15.5:0/2470080234 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55840a3380 msgr2=0x7f55840a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55840a3380 0x7f55840a5840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5588005d00 tx=0x7f5588005c70 comp rx=0 tx=0).stop 2024-09-15T22:08:33.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 -- 172.21.15.5:0/2470080234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5598105050 msgr2=0x7f55981984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5598105050 0x7f55981984e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5580031610 tx=0x7f55800043b0 comp rx=0 tx=0).stop 2024-09-15T22:08:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 -- 172.21.15.5:0/2470080234 shutdown_connections 2024-09-15T22:08:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5598107980 0x7f559819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f55840a3380 0x7f55840a5840 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5598105050 0x7f55981984e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 --2- 172.21.15.5:0/2470080234 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5598102720 0x7f5598197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 -- 172.21.15.5:0/2470080234 >> 172.21.15.5:0/2470080234 conn(0x7f55980fc280 msgr2=0x7f55981030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 -- 172.21.15.5:0/2470080234 shutdown_connections 2024-09-15T22:08:33.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:33.113+0000 7f559e3c9640 1 -- 172.21.15.5:0/2470080234 wait complete. 2024-09-15T22:08:33.887 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-15T22:08:33.896 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-15T22:08:33.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-15T22:08:34.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:33 smithi005 ceph-mon[30944]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:34.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:33 smithi005 ceph-mon[30944]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:34.156 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:34.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:33 smithi099 ceph-mon[34892]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:34.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:33 smithi099 ceph-mon[34892]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:34.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:33 smithi129 ceph-mon[35314]: pgmap v343: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:08:34.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:33 smithi129 ceph-mon[35314]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.624+0000 7f71efa97640 1 -- 172.21.15.5:0/3297143904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e81026d0 msgr2=0x7f71e8108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.624+0000 7f71efa97640 1 --2- 172.21.15.5:0/3297143904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e81026d0 0x7f71e8108f60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009ca0 tx=0x7f71e002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:08:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.625+0000 7f71efa97640 1 -- 172.21.15.5:0/3297143904 shutdown_connections 2024-09-15T22:08:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.625+0000 7f71efa97640 1 --2- 172.21.15.5:0/3297143904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e81026d0 0x7f71e8108f60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.625+0000 7f71efa97640 1 --2- 172.21.15.5:0/3297143904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e8101d10 0x7f71e8102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.625+0000 7f71efa97640 1 --2- 172.21.15.5:0/3297143904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8100b10 0x7f71e8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.625+0000 7f71efa97640 1 -- 172.21.15.5:0/3297143904 >> 172.21.15.5:0/3297143904 conn(0x7f71e80fc2a0 msgr2=0x7f71e80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.625+0000 7f71efa97640 1 -- 172.21.15.5:0/3297143904 shutdown_connections 2024-09-15T22:08:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.625+0000 7f71efa97640 1 -- 172.21.15.5:0/3297143904 wait complete. 2024-09-15T22:08:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.625+0000 7f71efa97640 1 Processor -- start 2024-09-15T22:08:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.625+0000 7f71efa97640 1 -- start start 2024-09-15T22:08:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71efa97640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8100b10 0x7f71e8197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71efa97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d10 0x7f71e8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71efa97640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81026d0 0x7f71e819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71efa97640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71e810c790 con 0x7f71e8101d10 2024-09-15T22:08:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71efa97640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71e810c610 con 0x7f71e81026d0 2024-09-15T22:08:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71efa97640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71e810c910 con 0x7f71e8100b10 2024-09-15T22:08:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ed032640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d10 0x7f71e8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ed032640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d10 0x7f71e8198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53270/0 (socket says 172.21.15.5:53270) 2024-09-15T22:08:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ed833640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8100b10 0x7f71e8197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ed032640 1 -- 172.21.15.5:0/3518638510 learned_addr learned my addr 172.21.15.5:0/3518638510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ee034640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81026d0 0x7f71e819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ed032640 1 -- 172.21.15.5:0/3518638510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8100b10 msgr2=0x7f71e8197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ed032640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8100b10 0x7f71e8197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ed032640 1 -- 172.21.15.5:0/3518638510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81026d0 msgr2=0x7f71e819f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ed032640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81026d0 0x7f71e819f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ed032640 1 -- 172.21.15.5:0/3518638510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e819fb00 con 0x7f71e8101d10 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ed833640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8100b10 0x7f71e8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71ed032640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d10 0x7f71e8198410 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f71e400c890 tx=0x7f71e400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.626+0000 7f71deffd640 1 -- 172.21.15.5:0/3518638510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e4004220 con 0x7f71e8101d10 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.627+0000 7f71deffd640 1 -- 172.21.15.5:0/3518638510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e40043c0 con 0x7f71e8101d10 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.627+0000 7f71deffd640 1 -- 172.21.15.5:0/3518638510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e40049e0 con 0x7f71e8101d10 2024-09-15T22:08:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.627+0000 7f71efa97640 1 -- 172.21.15.5:0/3518638510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e819fdf0 con 0x7f71e8101d10 2024-09-15T22:08:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.627+0000 7f71efa97640 1 -- 172.21.15.5:0/3518638510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e81a0250 con 0x7f71e8101d10 2024-09-15T22:08:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.628+0000 7f71efa97640 1 -- 172.21.15.5:0/3518638510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e81a04e0 con 0x7f71e8101d10 2024-09-15T22:08:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.629+0000 7f71deffd640 1 -- 172.21.15.5:0/3518638510 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f71e40079b0 con 0x7f71e8101d10 2024-09-15T22:08:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.629+0000 7f71deffd640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f71d40a3300 0x7f71d40a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.630+0000 7f71deffd640 1 -- 172.21.15.5:0/3518638510 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f71e40cfe00 con 0x7f71e8101d10 2024-09-15T22:08:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.630+0000 7f71ed833640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f71d40a3300 0x7f71d40a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.630+0000 7f71ed833640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f71d40a3300 0x7f71d40a57c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fd0 tx=0x7f71d800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.634+0000 7f71deffd640 1 -- 172.21.15.5:0/3518638510 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71e4014030 con 0x7f71e8101d10 2024-09-15T22:08:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.864+0000 7f71efa97640 1 -- 172.21.15.5:0/3518638510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f71e81a0840 con 0x7f71e8101d10 2024-09-15T22:08:34.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.864+0000 7f71deffd640 1 -- 172.21.15.5:0/3518638510 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3078 (secure 0 0 0) 0x7f71e409bfb0 con 0x7f71e8101d10 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout:e24 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout:btime 2024-09-15T22:07:16:736368+0000 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout:legacy client fscid: 1 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem 'cephfs' (1) 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout:fs_name cephfs 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 24 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-15T22:06:37.854436+0000 2024-09-15T22:08:34.866 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-09-15T22:07:16.364125+0000 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:tableserver 0 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:root 0 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:session_timeout 300 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:session_autoclose 300 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:max_file_size 1099511627776 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:max_xattr_size 65536 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:required_client_features {} 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure 0 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure_osd_epoch 0 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:max_mds 3 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:in 0,1,2 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:up {0=24491,1=24467,2=24466} 2024-09-15T22:08:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:failed 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:damaged 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:stopped 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:data_pools [3,4] 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:metadata_pool 2 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:inline_data disabled 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:balancer 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:bal_rank_mask -1 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:standby_count_wanted 0 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:qdb_cluster leader: 24491 members: 24466,24467,24491 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b{0:24491} state up:active seq 4 addr [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c{0:24451} state up:standby-replay seq 1 addr [v2:172.21.15.129:6836/3535375231,v1:172.21.15.129:6837/3535375231] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.h{1:24467} state up:active seq 16 addr [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.f{1:24442} state up:standby-replay seq 1 addr [v2:172.21.15.129:6834/3511675647,v1:172.21.15.129:6835/3511675647] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.i{2:24466} state up:active seq 16 addr [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.869 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.e{2:24440} state up:standby-replay seq 1 addr [v2:172.21.15.99:6834/157351762,v1:172.21.15.99:6835/157351762] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.869 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:34.869 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:34.869 INFO:teuthology.orchestra.run.smithi005.stdout:Standby daemons: 2024-09-15T22:08:34.869 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:34.869 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.j{-1:14526} state up:standby seq 1 addr [v2:172.21.15.5:6834/93159772,v1:172.21.15.5:6835/93159772] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.869 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d{-1:14538} state up:standby seq 1 addr [v2:172.21.15.5:6836/2795083273,v1:172.21.15.5:6837/2795083273] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.869 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.g{-1:14583} state up:standby seq 1 addr [v2:172.21.15.5:6838/2796220965,v1:172.21.15.5:6839/2796220965] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.869 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a{-1:14595} state up:standby seq 1 addr [v2:172.21.15.5:6840/3376413003,v1:172.21.15.5:6841/3376413003] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.869 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.l{-1:24418} state up:standby seq 1 addr [v2:172.21.15.129:6832/3381054167,v1:172.21.15.129:6833/3381054167] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.869 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.k{-1:24419} state up:standby seq 1 addr [v2:172.21.15.99:6832/1006057528,v1:172.21.15.99:6833/1006057528] compat {c=[1],r=[1],i=[1fff]}] 2024-09-15T22:08:34.870 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 24 2024-09-15T22:08:34.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 -- 172.21.15.5:0/3518638510 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f71d40a3300 msgr2=0x7f71d40a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:34.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f71d40a3300 0x7f71d40a57c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71d8005fd0 tx=0x7f71d800b390 comp rx=0 tx=0).stop 2024-09-15T22:08:34.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 -- 172.21.15.5:0/3518638510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d10 msgr2=0x7f71e8198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:34.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d10 0x7f71e8198410 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f71e400c890 tx=0x7f71e400cd50 comp rx=0 tx=0).stop 2024-09-15T22:08:34.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 -- 172.21.15.5:0/3518638510 shutdown_connections 2024-09-15T22:08:34.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f71e81026d0 0x7f71e819f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:34.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f71d40a3300 0x7f71d40a57c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:34.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d10 0x7f71e8198410 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:34.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 --2- 172.21.15.5:0/3518638510 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f71e8100b10 0x7f71e8197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:34.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 -- 172.21.15.5:0/3518638510 >> 172.21.15.5:0/3518638510 conn(0x7f71e80fc2a0 msgr2=0x7f71e80fc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:34.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 -- 172.21.15.5:0/3518638510 shutdown_connections 2024-09-15T22:08:34.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:34.866+0000 7f71efa97640 1 -- 172.21.15.5:0/3518638510 wait complete. 2024-09-15T22:08:35.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:34 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:35.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:34 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3518638510' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:08:35.327 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:34 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:35.327 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:34 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3518638510' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:08:35.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:34 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:35.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:34 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3518638510' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-15T22:08:35.655 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-15T22:08:36.318 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:35 smithi005 ceph-mon[30944]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:36.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:35 smithi099 ceph-mon[34892]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:36.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:35 smithi129 ceph-mon[35314]: pgmap v344: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-15T22:08:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.078+0000 7f41bbdd1640 1 -- 172.21.15.5:0/1624515067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b41037d0 msgr2=0x7f41b410a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.078+0000 7f41bbdd1640 1 --2- 172.21.15.5:0/1624515067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b41037d0 0x7f41b410a060 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009ca0 tx=0x7f41ac02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.078+0000 7f41bbdd1640 1 -- 172.21.15.5:0/1624515067 shutdown_connections 2024-09-15T22:08:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.078+0000 7f41bbdd1640 1 --2- 172.21.15.5:0/1624515067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b41037d0 0x7f41b410a060 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.078+0000 7f41bbdd1640 1 --2- 172.21.15.5:0/1624515067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41b4102e10 0x7f41b4103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 --2- 172.21.15.5:0/1624515067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b4101c10 0x7f41b4102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 -- 172.21.15.5:0/1624515067 >> 172.21.15.5:0/1624515067 conn(0x7f41b40fd3c0 msgr2=0x7f41b40ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 -- 172.21.15.5:0/1624515067 shutdown_connections 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 -- 172.21.15.5:0/1624515067 wait complete. 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 Processor -- start 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 -- start start 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b4101c10 0x7f41b4198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102e10 0x7f41b4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41b41037d0 0x7f41b419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41b410d9e0 con 0x7f41b4102e10 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f41b410d860 con 0x7f41b4101c10 2024-09-15T22:08:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41bbdd1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f41b410db60 con 0x7f41b41037d0 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.079+0000 7f41b936c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102e10 0x7f41b4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41b936c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102e10 0x7f41b4198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53298/0 (socket says 172.21.15.5:53298) 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41b936c640 1 -- 172.21.15.5:0/643305461 learned_addr learned my addr 172.21.15.5:0/643305461 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41ba36e640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41b41037d0 0x7f41b419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41b9b6d640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b4101c10 0x7f41b4198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41b936c640 1 -- 172.21.15.5:0/643305461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41b41037d0 msgr2=0x7f41b419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41b936c640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41b41037d0 0x7f41b419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41b936c640 1 -- 172.21.15.5:0/643305461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b4101c10 msgr2=0x7f41b4198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41b936c640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b4101c10 0x7f41b4198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41b936c640 1 -- 172.21.15.5:0/643305461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b419fc70 con 0x7f41b4102e10 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41b9b6d640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b4101c10 0x7f41b4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41ba36e640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41b41037d0 0x7f41b419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41b936c640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102e10 0x7f41b4198580 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f41a400b4a0 tx=0x7f41a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41aaffd640 1 -- 172.21.15.5:0/643305461 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4014070 con 0x7f41b4102e10 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41aaffd640 1 -- 172.21.15.5:0/643305461 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a4010af0 con 0x7f41b4102e10 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41bbdd1640 1 -- 172.21.15.5:0/643305461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b419ff60 con 0x7f41b4102e10 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41aaffd640 1 -- 172.21.15.5:0/643305461 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4017d10 con 0x7f41b4102e10 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.080+0000 7f41bbdd1640 1 -- 172.21.15.5:0/643305461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b41a0440 con 0x7f41b4102e10 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.081+0000 7f41bbdd1640 1 -- 172.21.15.5:0/643305461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b41090b0 con 0x7f41b4102e10 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.082+0000 7f41aaffd640 1 -- 172.21.15.5:0/643305461 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7f41a4017070 con 0x7f41b4102e10 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.083+0000 7f41aaffd640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f41a00a3300 0x7f41a00a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.084+0000 7f41b9b6d640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f41a00a3300 0x7f41a00a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.084+0000 7f41aaffd640 1 -- 172.21.15.5:0/643305461 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f41a40cf550 con 0x7f41b4102e10 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.084+0000 7f41b9b6d640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f41a00a3300 0x7f41a00a57c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f41b0006fd0 tx=0x7f41b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.087+0000 7f41aaffd640 1 -- 172.21.15.5:0/643305461 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41a409be00 con 0x7f41b4102e10 2024-09-15T22:08:37.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.288+0000 7f41bbdd1640 1 -- 172.21.15.5:0/643305461 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f41b402d9c0 con 0x7f41b4102e10 2024-09-15T22:08:37.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:37.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:37.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:37.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:37 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:37.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.290+0000 7f41aaffd640 1 -- 172.21.15.5:0/643305461 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5645 (secure 0 0 0) 0x7f41a409b700 con 0x7f41b4102e10 2024-09-15T22:08:37.291 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 88 2024-09-15T22:08:37.291 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:08:37.291 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-15T21:58:48.668955+0000 2024-09-15T22:08:37.291 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-09-15T22:07:45.620119+0000 2024-09-15T22:08:37.291 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 27 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client luminous 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release squid 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-15T22:08:37.292 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 12 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 8 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6802/3136788865,v1:172.21.15.5:6803/3136788865] [v2:172.21.15.5:6804/3136788865,v1:172.21.15.5:6805/3136788865] exists,up 32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/2888502880,v1:172.21.15.99:6801/2888502880] [v2:172.21.15.99:6802/2888502880,v1:172.21.15.99:6803/2888502880] exists,up 24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 17 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6800/2902807483,v1:172.21.15.129:6801/2902807483] [v2:172.21.15.129:6802/2902807483,v1:172.21.15.129:6803/2902807483] exists,up 0cf9aa4e-8865-471c-a841-4be5386a3fcd 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 23 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6810/1557519616,v1:172.21.15.5:6811/1557519616] [v2:172.21.15.5:6812/1557519616,v1:172.21.15.5:6813/1557519616] exists,up 0fcbf74d-7a3d-471e-9447-688e10d0c350 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 30 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6808/79942025,v1:172.21.15.99:6809/79942025] [v2:172.21.15.99:6810/79942025,v1:172.21.15.99:6811/79942025] exists,up 511bc0a8-e4f3-48e2-824b-7afd2ead90e4 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 34 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6808/3964498949,v1:172.21.15.129:6809/3964498949] [v2:172.21.15.129:6810/3964498949,v1:172.21.15.129:6811/3964498949] exists,up 5524a904-d0c6-4276-9e11-2dcf0e6ec037 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 41 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6818/1626090223,v1:172.21.15.5:6819/1626090223] [v2:172.21.15.5:6820/1626090223,v1:172.21.15.5:6821/1626090223] exists,up a5b89d06-6596-4cd8-9083-3e023d156603 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 46 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6816/521420883,v1:172.21.15.99:6817/521420883] [v2:172.21.15.99:6818/521420883,v1:172.21.15.99:6819/521420883] exists,up 4e8598f5-b1c0-4196-8668-780ad8f79e76 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 up in weight 1 up_from 54 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6816/2292146431,v1:172.21.15.129:6817/2292146431] [v2:172.21.15.129:6818/2292146431,v1:172.21.15.129:6819/2292146431] exists,up ae5048f4-2e10-4e65-a7e9-130180439ad9 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 up in weight 1 up_from 61 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6826/3480349355,v1:172.21.15.5:6827/3480349355] [v2:172.21.15.5:6828/3480349355,v1:172.21.15.5:6829/3480349355] exists,up 93835974-5099-475b-86ce-057cf6614dff 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 up in weight 1 up_from 67 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6824/154036498,v1:172.21.15.99:6825/154036498] [v2:172.21.15.99:6826/154036498,v1:172.21.15.99:6827/154036498] exists,up d019941f-3a11-449b-8e5d-dac790b1530f 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 up in weight 1 up_from 74 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.129:6824/4002757789,v1:172.21.15.129:6825/4002757789] [v2:172.21.15.129:6826/4002757789,v1:172.21.15.129:6827/4002757789] exists,up 2cff3d48-81e6-4748-96bb-c20ba054f287 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.a [2,11,8,6] 2024-09-15T22:08:37.293 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.c [2,1] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1f [8,4,7,11] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.6 [3,6] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.11 [4,7] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.13 [4,9] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.17 [4,8] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.21 [5,9] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.37 [1,9] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.6 [9,11] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.12 [9,10] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.18 [9,8] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.1c [5,8] 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3146025957 expires 2024-09-16T21:59:42.685255+0000 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3880791045 expires 2024-09-16T21:59:42.685255+0000 2024-09-15T22:08:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3326533830 expires 2024-09-16T21:59:42.685255+0000 2024-09-15T22:08:37.295 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/3785442724 expires 2024-09-16T21:59:42.685255+0000 2024-09-15T22:08:37.295 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1445520321 expires 2024-09-16T21:59:18.773927+0000 2024-09-15T22:08:37.295 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2658926924 expires 2024-09-16T21:59:18.773927+0000 2024-09-15T22:08:37.295 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/4013564474 expires 2024-09-16T21:59:18.773927+0000 2024-09-15T22:08:37.295 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/3785442724 expires 2024-09-16T21:59:42.685255+0000 2024-09-15T22:08:37.295 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/1812503826 expires 2024-09-16T21:59:18.773927+0000 2024-09-15T22:08:37.295 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/1812503826 expires 2024-09-16T21:59:18.773927+0000 2024-09-15T22:08:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.293+0000 7f41bbdd1640 1 -- 172.21.15.5:0/643305461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f41a00a3300 msgr2=0x7f41a00a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.293+0000 7f41bbdd1640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f41a00a3300 0x7f41a00a57c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f41b0006fd0 tx=0x7f41b0008040 comp rx=0 tx=0).stop 2024-09-15T22:08:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.293+0000 7f41bbdd1640 1 -- 172.21.15.5:0/643305461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102e10 msgr2=0x7f41b4198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.293+0000 7f41bbdd1640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102e10 0x7f41b4198580 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f41a400b4a0 tx=0x7f41a400b960 comp rx=0 tx=0).stop 2024-09-15T22:08:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.293+0000 7f41bbdd1640 1 -- 172.21.15.5:0/643305461 shutdown_connections 2024-09-15T22:08:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.293+0000 7f41bbdd1640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f41b41037d0 0x7f41b419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.293+0000 7f41bbdd1640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f41a00a3300 0x7f41a00a57c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.293+0000 7f41bbdd1640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b4102e10 0x7f41b4198580 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.293+0000 7f41bbdd1640 1 --2- 172.21.15.5:0/643305461 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f41b4101c10 0x7f41b4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.293+0000 7f41bbdd1640 1 -- 172.21.15.5:0/643305461 >> 172.21.15.5:0/643305461 conn(0x7f41b40fd3c0 msgr2=0x7f41b4106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:37.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.294+0000 7f41bbdd1640 1 -- 172.21.15.5:0/643305461 shutdown_connections 2024-09-15T22:08:37.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:37.294+0000 7f41bbdd1640 1 -- 172.21.15.5:0/643305461 wait complete. 2024-09-15T22:08:37.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:37.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:37.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:37.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:37 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:37 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:38.038 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-15T22:08:38.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:08:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:38.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:38 smithi005 ceph-mon[30944]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:38.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:38 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/643305461' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:08:38.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:38.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:38 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:38.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:38 smithi099 ceph-mon[34892]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:38.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:38 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/643305461' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:08:38.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:38.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:38 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:38.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:38 smithi129 ceph-mon[35314]: pgmap v345: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:38.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:38 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/643305461' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-15T22:08:38.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:38.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:38 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.823+0000 7fab69415640 1 -- 172.21.15.5:0/3914654128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab641036c0 msgr2=0x7fab6410bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.823+0000 7fab69415640 1 --2- 172.21.15.5:0/3914654128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab641036c0 0x7fab6410bbb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009900 tx=0x7fab4c02a480 comp rx=0 tx=0).stop 2024-09-15T22:08:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.823+0000 7fab69415640 1 -- 172.21.15.5:0/3914654128 shutdown_connections 2024-09-15T22:08:38.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.823+0000 7fab69415640 1 --2- 172.21.15.5:0/3914654128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab6410c180 0x7fab6410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.823+0000 7fab69415640 1 --2- 172.21.15.5:0/3914654128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab641036c0 0x7fab6410bbb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.823+0000 7fab69415640 1 --2- 172.21.15.5:0/3914654128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab64102da0 0x7fab64103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.823+0000 7fab69415640 1 -- 172.21.15.5:0/3914654128 >> 172.21.15.5:0/3914654128 conn(0x7fab640fc7a0 msgr2=0x7fab640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.823+0000 7fab69415640 1 -- 172.21.15.5:0/3914654128 shutdown_connections 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.823+0000 7fab69415640 1 -- 172.21.15.5:0/3914654128 wait complete. 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.824+0000 7fab69415640 1 Processor -- start 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.824+0000 7fab69415640 1 -- start start 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.824+0000 7fab69415640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab64102da0 0x7fab641983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.824+0000 7fab69415640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab641036c0 0x7fab64198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.824+0000 7fab69415640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab6410c180 0x7fab6419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.824+0000 7fab69415640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab64111f60 con 0x7fab6410c180 2024-09-15T22:08:38.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.824+0000 7fab69415640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fab64111de0 con 0x7fab64102da0 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.824+0000 7fab69415640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fab641120e0 con 0x7fab641036c0 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab6410c180 0x7fab6419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab6410c180 0x7fab6419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53326/0 (socket says 172.21.15.5:53326) 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab637fe640 1 -- 172.21.15.5:0/444949425 learned_addr learned my addr 172.21.15.5:0/444949425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab627fc640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab641036c0 0x7fab64198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab62ffd640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab64102da0 0x7fab641983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab637fe640 1 -- 172.21.15.5:0/444949425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab641036c0 msgr2=0x7fab64198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab637fe640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab641036c0 0x7fab64198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab637fe640 1 -- 172.21.15.5:0/444949425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab64102da0 msgr2=0x7fab641983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab637fe640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab64102da0 0x7fab641983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab637fe640 1 -- 172.21.15.5:0/444949425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6419d390 con 0x7fab6410c180 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab637fe640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab6410c180 0x7fab6419ccb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c970 tx=0x7fab5c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:38.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab5bfff640 1 -- 172.21.15.5:0/444949425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c007bb0 con 0x7fab6410c180 2024-09-15T22:08:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab5bfff640 1 -- 172.21.15.5:0/444949425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab5c007d50 con 0x7fab6410c180 2024-09-15T22:08:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab69415640 1 -- 172.21.15.5:0/444949425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6419d680 con 0x7fab6410c180 2024-09-15T22:08:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab5bfff640 1 -- 172.21.15.5:0/444949425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c018800 con 0x7fab6410c180 2024-09-15T22:08:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.825+0000 7fab69415640 1 -- 172.21.15.5:0/444949425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab641a4f60 con 0x7fab6410c180 2024-09-15T22:08:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.826+0000 7fab69415640 1 -- 172.21.15.5:0/444949425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab64103180 con 0x7fab6410c180 2024-09-15T22:08:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.828+0000 7fab5bfff640 1 -- 172.21.15.5:0/444949425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fab5c00fa50 con 0x7fab6410c180 2024-09-15T22:08:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.828+0000 7fab5bfff640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab500a32d0 0x7fab500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.829+0000 7fab62ffd640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab500a32d0 0x7fab500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.829+0000 7fab5bfff640 1 -- 172.21.15.5:0/444949425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fab5c0cfe40 con 0x7fab6410c180 2024-09-15T22:08:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.829+0000 7fab62ffd640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab500a32d0 0x7fab500a5790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fab54005fd0 tx=0x7fab54009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:38.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:38.833+0000 7fab5bfff640 1 -- 172.21.15.5:0/444949425 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab5c014030 con 0x7fab6410c180 2024-09-15T22:08:39.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.048+0000 7fab69415640 1 -- 172.21.15.5:0/444949425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab64107220 con 0x7fab6410c180 2024-09-15T22:08:39.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.049+0000 7fab5bfff640 1 -- 172.21.15.5:0/444949425 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11897 (secure 0 0 0) 0x7fab5c09c640 con 0x7fab6410c180 2024-09-15T22:08:39.051 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:39.051 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"btime":"2024-09-15T22:07:16:736368+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:07:16.364125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24466,24467,24491]},"id":1}]} 2024-09-15T22:08:39.051 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 24 2024-09-15T22:08:39.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 -- 172.21.15.5:0/444949425 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab500a32d0 msgr2=0x7fab500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:39.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab500a32d0 0x7fab500a5790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fab54005fd0 tx=0x7fab54009500 comp rx=0 tx=0).stop 2024-09-15T22:08:39.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 -- 172.21.15.5:0/444949425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab6410c180 msgr2=0x7fab6419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:39.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab6410c180 0x7fab6419ccb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fab5c00c970 tx=0x7fab5c00ce30 comp rx=0 tx=0).stop 2024-09-15T22:08:39.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 -- 172.21.15.5:0/444949425 shutdown_connections 2024-09-15T22:08:39.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fab500a32d0 0x7fab500a5790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:39.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab6410c180 0x7fab6419ccb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:39.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fab641036c0 0x7fab64198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:39.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 --2- 172.21.15.5:0/444949425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fab64102da0 0x7fab641983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:39.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 -- 172.21.15.5:0/444949425 >> 172.21.15.5:0/444949425 conn(0x7fab640fc7a0 msgr2=0x7fab640fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:39.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 -- 172.21.15.5:0/444949425 shutdown_connections 2024-09-15T22:08:39.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:39.052+0000 7fab69415640 1 -- 172.21.15.5:0/444949425 wait complete. 2024-09-15T22:08:39.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:39 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:39.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:39 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:39.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:39 smithi005 ceph-mon[30944]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:39.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:39 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/444949425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:39.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:39 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:39 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:39 smithi099 ceph-mon[34892]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:39 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/444949425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:39 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:39 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:39 smithi129 ceph-mon[35314]: from='mgr.14150 172.21.15.5:0/70321102' entity='mgr.x' 2024-09-15T22:08:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:39 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/444949425' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:39.716 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:08:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:40.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:40 smithi005 ceph-mon[30944]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.479+0000 7fccbf32c640 1 -- 172.21.15.5:0/179787983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8103740 msgr2=0x7fccb8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.479+0000 7fccbf32c640 1 --2- 172.21.15.5:0/179787983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8103740 0x7fccb8107b80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fccac009930 tx=0x7fccac02a420 comp rx=0 tx=0).stop 2024-09-15T22:08:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.479+0000 7fccbf32c640 1 -- 172.21.15.5:0/179787983 shutdown_connections 2024-09-15T22:08:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.479+0000 7fccbf32c640 1 --2- 172.21.15.5:0/179787983 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fccb8108290 0x7fccb810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.479+0000 7fccbf32c640 1 --2- 172.21.15.5:0/179787983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8103740 0x7fccb8107b80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.479+0000 7fccbf32c640 1 --2- 172.21.15.5:0/179787983 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb8102d90 0x7fccb8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:40.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.479+0000 7fccbf32c640 1 -- 172.21.15.5:0/179787983 >> 172.21.15.5:0/179787983 conn(0x7fccb80fc7d0 msgr2=0x7fccb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.479+0000 7fccbf32c640 1 -- 172.21.15.5:0/179787983 shutdown_connections 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.479+0000 7fccbf32c640 1 -- 172.21.15.5:0/179787983 wait complete. 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.479+0000 7fccbf32c640 1 Processor -- start 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.479+0000 7fccbf32c640 1 -- start start 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbf32c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb8102d90 0x7fccb8198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbf32c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8103740 0x7fccb8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbf32c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fccb8108290 0x7fccb819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbf32c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8112090 con 0x7fccb8103740 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbf32c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8111f10 con 0x7fccb8102d90 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbf32c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8112210 con 0x7fccb8108290 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbc8c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8103740 0x7fccb8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbc8c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8103740 0x7fccb8198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53352/0 (socket says 172.21.15.5:53352) 2024-09-15T22:08:40.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbc8c7640 1 -- 172.21.15.5:0/618213783 learned_addr learned my addr 172.21.15.5:0/618213783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbd8c9640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fccb8108290 0x7fccb819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbd0c8640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb8102d90 0x7fccb8198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbc8c7640 1 -- 172.21.15.5:0/618213783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fccb8108290 msgr2=0x7fccb819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbc8c7640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fccb8108290 0x7fccb819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbc8c7640 1 -- 172.21.15.5:0/618213783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb8102d90 msgr2=0x7fccb8198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbc8c7640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb8102d90 0x7fccb8198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbc8c7640 1 -- 172.21.15.5:0/618213783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb819d4d0 con 0x7fccb8103740 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbd0c8640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb8102d90 0x7fccb8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.480+0000 7fccbc8c7640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8103740 0x7fccb8198a60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fccac005940 tx=0x7fccac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.481+0000 7fccb67fc640 1 -- 172.21.15.5:0/618213783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac03f030 con 0x7fccb8103740 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.481+0000 7fccb67fc640 1 -- 172.21.15.5:0/618213783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac003ce0 con 0x7fccb8103740 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.481+0000 7fccbf32c640 1 -- 172.21.15.5:0/618213783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb819d760 con 0x7fccb8103740 2024-09-15T22:08:40.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.481+0000 7fccb67fc640 1 -- 172.21.15.5:0/618213783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac035070 con 0x7fccb8103740 2024-09-15T22:08:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.481+0000 7fccbf32c640 1 -- 172.21.15.5:0/618213783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb81a50a0 con 0x7fccb8103740 2024-09-15T22:08:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.482+0000 7fccbf32c640 1 -- 172.21.15.5:0/618213783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb8103170 con 0x7fccb8103740 2024-09-15T22:08:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.483+0000 7fccb67fc640 1 -- 172.21.15.5:0/618213783 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7fccac042030 con 0x7fccb8103740 2024-09-15T22:08:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.483+0000 7fccb67fc640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcca00a3350 0x7fcca00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.484+0000 7fccbd0c8640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcca00a3350 0x7fcca00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.484+0000 7fccb67fc640 1 -- 172.21.15.5:0/618213783 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fccac0f05e0 con 0x7fccb8103740 2024-09-15T22:08:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.484+0000 7fccbd0c8640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcca00a3350 0x7fcca00a5810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcca8007e70 tx=0x7fcca8007850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.487+0000 7fccb67fc640 1 -- 172.21.15.5:0/618213783 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fccac0bcd80 con 0x7fccb8103740 2024-09-15T22:08:40.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:40 smithi099 ceph-mon[34892]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:40.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:40 smithi129 ceph-mon[35314]: pgmap v346: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-15T22:08:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.698+0000 7fccbf32c640 1 -- 172.21.15.5:0/618213783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccb8199880 con 0x7fccb8103740 2024-09-15T22:08:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.699+0000 7fccb67fc640 1 -- 172.21.15.5:0/618213783 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11897 (secure 0 0 0) 0x7fccb8199880 con 0x7fccb8103740 2024-09-15T22:08:40.701 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:40.701 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":24,"btime":"2024-09-15T22:07:16:736368+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:07:16.364125+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24466,24467,24491]},"id":1}]} 2024-09-15T22:08:40.701 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 24 2024-09-15T22:08:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 -- 172.21.15.5:0/618213783 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcca00a3350 msgr2=0x7fcca00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcca00a3350 0x7fcca00a5810 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcca8007e70 tx=0x7fcca8007850 comp rx=0 tx=0).stop 2024-09-15T22:08:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 -- 172.21.15.5:0/618213783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8103740 msgr2=0x7fccb8198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8103740 0x7fccb8198a60 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fccac005940 tx=0x7fccac0042f0 comp rx=0 tx=0).stop 2024-09-15T22:08:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 -- 172.21.15.5:0/618213783 shutdown_connections 2024-09-15T22:08:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fccb8108290 0x7fccb819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fcca00a3350 0x7fcca00a5810 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8103740 0x7fccb8198a60 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 --2- 172.21.15.5:0/618213783 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fccb8102d90 0x7fccb8198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 -- 172.21.15.5:0/618213783 >> 172.21.15.5:0/618213783 conn(0x7fccb80fc7d0 msgr2=0x7fccb8106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 -- 172.21.15.5:0/618213783 shutdown_connections 2024-09-15T22:08:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:40.702+0000 7fccbf32c640 1 -- 172.21.15.5:0/618213783 wait complete. 2024-09-15T22:08:41.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:41 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/618213783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:41.341 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:08:41.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:41 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/618213783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:41.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:41 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/618213783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.081+0000 7facab39c640 1 -- 172.21.15.5:0/849975305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faca4102570 msgr2=0x7faca41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.081+0000 7facab39c640 1 --2- 172.21.15.5:0/849975305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faca4102570 0x7faca41029b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faca0009900 tx=0x7faca002a480 comp rx=0 tx=0).stop 2024-09-15T22:08:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.081+0000 7facab39c640 1 -- 172.21.15.5:0/849975305 shutdown_connections 2024-09-15T22:08:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.081+0000 7facab39c640 1 --2- 172.21.15.5:0/849975305 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faca4102ef0 0x7faca410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.081+0000 7facab39c640 1 --2- 172.21.15.5:0/849975305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faca4102570 0x7faca41029b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.081+0000 7facab39c640 1 --2- 172.21.15.5:0/849975305 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca4100e20 0x7faca4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.081+0000 7facab39c640 1 -- 172.21.15.5:0/849975305 >> 172.21.15.5:0/849975305 conn(0x7faca40fc7d0 msgr2=0x7faca40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.081+0000 7facab39c640 1 -- 172.21.15.5:0/849975305 shutdown_connections 2024-09-15T22:08:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.081+0000 7facab39c640 1 -- 172.21.15.5:0/849975305 wait complete. 2024-09-15T22:08:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7facab39c640 1 Processor -- start 2024-09-15T22:08:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7facab39c640 1 -- start start 2024-09-15T22:08:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7facab39c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faca4100e20 0x7faca40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7facab39c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faca4102570 0x7faca40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7facab39c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca4102ef0 0x7faca40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7facab39c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faca4111020 con 0x7faca4102570 2024-09-15T22:08:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7facab39c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faca4110ea0 con 0x7faca4102ef0 2024-09-15T22:08:42.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7facab39c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faca41111a0 con 0x7faca4100e20 2024-09-15T22:08:42.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7faca8937640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faca4102570 0x7faca40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:42.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7faca8937640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faca4102570 0x7faca40fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53380/0 (socket says 172.21.15.5:53380) 2024-09-15T22:08:42.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7faca8937640 1 -- 172.21.15.5:0/1366429713 learned_addr learned my addr 172.21.15.5:0/1366429713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:42.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.082+0000 7faca9138640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faca4100e20 0x7faca40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:42.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7faca8937640 1 -- 172.21.15.5:0/1366429713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faca4100e20 msgr2=0x7faca40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:42.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7faca8937640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faca4100e20 0x7faca40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:42.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7faca9939640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca4102ef0 0x7faca40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:42.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7faca8937640 1 -- 172.21.15.5:0/1366429713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca4102ef0 msgr2=0x7faca40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:42.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7faca8937640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca4102ef0 0x7faca40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:42.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7faca8937640 1 -- 172.21.15.5:0/1366429713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca41a6b30 con 0x7faca4102570 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7faca9939640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca4102ef0 0x7faca40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7faca8937640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faca4102570 0x7faca40fd1c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faca0006010 tx=0x7faca0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7fac967fc640 1 -- 172.21.15.5:0/1366429713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca003f070 con 0x7faca4102570 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7facab39c640 1 -- 172.21.15.5:0/1366429713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca41a6dc0 con 0x7faca4102570 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7fac967fc640 1 -- 172.21.15.5:0/1366429713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faca0031e30 con 0x7faca4102570 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7fac967fc640 1 -- 172.21.15.5:0/1366429713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faca0035030 con 0x7faca4102570 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.083+0000 7facab39c640 1 -- 172.21.15.5:0/1366429713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca41a7260 con 0x7faca4102570 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.084+0000 7facab39c640 1 -- 172.21.15.5:0/1366429713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca4101280 con 0x7faca4102570 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.085+0000 7fac967fc640 1 -- 172.21.15.5:0/1366429713 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 138405+0+0 (secure 0 0 0) 0x7faca0042070 con 0x7faca4102570 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.086+0000 7fac967fc640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fac900a3350 0x7fac900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.086+0000 7faca9138640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fac900a3350 0x7fac900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.086+0000 7fac967fc640 1 -- 172.21.15.5:0/1366429713 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7faca00f0d90 con 0x7faca4102570 2024-09-15T22:08:42.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.087+0000 7faca9138640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fac900a3350 0x7fac900a5810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fac98009890 tx=0x7fac98008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:42.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.090+0000 7fac967fc640 1 -- 172.21.15.5:0/1366429713 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faca00bd5c0 con 0x7faca4102570 2024-09-15T22:08:42.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.268+0000 7facab39c640 1 -- 172.21.15.5:0/1366429713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faca410a0a0 con 0x7faca4102570 2024-09-15T22:08:42.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.269+0000 7fac967fc640 1 -- 172.21.15.5:0/1366429713 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24757 (secure 0 0 0) 0x7faca00bcec0 con 0x7faca4102570 2024-09-15T22:08:42.271 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:42.271 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":88,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:07:45.620119+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:08:42.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 -- 172.21.15.5:0/1366429713 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fac900a3350 msgr2=0x7fac900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:42.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fac900a3350 0x7fac900a5810 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fac98009890 tx=0x7fac98008040 comp rx=0 tx=0).stop 2024-09-15T22:08:42.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 -- 172.21.15.5:0/1366429713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faca4102570 msgr2=0x7faca40fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:42.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faca4102570 0x7faca40fd1c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7faca0006010 tx=0x7faca0003c80 comp rx=0 tx=0).stop 2024-09-15T22:08:42.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 -- 172.21.15.5:0/1366429713 shutdown_connections 2024-09-15T22:08:42.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faca4102ef0 0x7faca40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:42.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fac900a3350 0x7fac900a5810 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:42.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faca4102570 0x7faca40fd1c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:42.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 --2- 172.21.15.5:0/1366429713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faca4100e20 0x7faca40fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:42.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 -- 172.21.15.5:0/1366429713 >> 172.21.15.5:0/1366429713 conn(0x7faca40fc7d0 msgr2=0x7faca406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:42.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 -- 172.21.15.5:0/1366429713 shutdown_connections 2024-09-15T22:08:42.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:42.272+0000 7facab39c640 1 -- 172.21.15.5:0/1366429713 wait complete. 2024-09-15T22:08:42.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:42 smithi099 ceph-mon[34892]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:42.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:42 smithi129 ceph-mon[35314]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:42.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:42 smithi005 ceph-mon[30944]: pgmap v347: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-15T22:08:42.996 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T22:06:37.854436+0000', 'modified': '2024-09-15T22:07:16.364125+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24491, 'mds_1': 24467, 'mds_2': 24466}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/157351762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 157351762}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 157351762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6835/3511675647', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 3511675647}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 3511675647}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6837/3535375231', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6836', 'nonce': 3535375231}, {'type': 'v1', 'addr': '172.21.15.129:6837', 'nonce': 3535375231}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.129:6839/2032213198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 2032213198}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 2032213198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.99:6837/3952304847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 3952304847}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 3952304847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.99:6839/3763383986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 3763383986}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 3763383986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24491, 'qdb_cluster': [24466, 24467, 24491]} 2024-09-15T22:08:42.997 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-15T22:08:42.997 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T22:08:43.257 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:43.514 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:43 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1366429713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:08:43.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:43 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1366429713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:08:43.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:43 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1366429713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:08:43.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.730+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3009926196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ac8102ef0 msgr2=0x7f3ac810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:43.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.730+0000 7f3aceeb9640 1 --2- 172.21.15.5:0/3009926196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ac8102ef0 0x7f3ac810da40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009900 tx=0x7f3ac002a480 comp rx=0 tx=0).stop 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.730+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3009926196 shutdown_connections 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.730+0000 7f3aceeb9640 1 --2- 172.21.15.5:0/3009926196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ac8102ef0 0x7f3ac810da40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.730+0000 7f3aceeb9640 1 --2- 172.21.15.5:0/3009926196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ac8102570 0x7f3ac81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.730+0000 7f3aceeb9640 1 --2- 172.21.15.5:0/3009926196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac8100e20 0x7f3ac8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.730+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3009926196 >> 172.21.15.5:0/3009926196 conn(0x7f3ac80fc7d0 msgr2=0x7f3ac80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.730+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3009926196 shutdown_connections 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.730+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3009926196 wait complete. 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3aceeb9640 1 Processor -- start 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3aceeb9640 1 -- start start 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3aceeb9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ac8100e20 0x7f3ac8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3aceeb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ac8102570 0x7f3ac81997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3aceeb9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac8102ef0 0x7f3ac81a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3aceeb9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac8110f80 con 0x7f3ac8102570 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3aceeb9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac8110e00 con 0x7f3ac8102ef0 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3aceeb9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac8111100 con 0x7f3ac8100e20 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3ac7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ac8102570 0x7f3ac81997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3ac7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ac8102570 0x7f3ac81997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51974/0 (socket says 172.21.15.5:51974) 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3ac7fff640 1 -- 172.21.15.5:0/3562971039 learned_addr learned my addr 172.21.15.5:0/3562971039 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3accc55640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ac8100e20 0x7f3ac8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3acd456640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac8102ef0 0x7f3ac81a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3ac7fff640 1 -- 172.21.15.5:0/3562971039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ac8100e20 msgr2=0x7f3ac8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ac8100e20 0x7f3ac8199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3ac7fff640 1 -- 172.21.15.5:0/3562971039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac8102ef0 msgr2=0x7f3ac81a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac8102ef0 0x7f3ac81a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.731+0000 7f3ac7fff640 1 -- 172.21.15.5:0/3562971039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac81a4830 con 0x7f3ac8102570 2024-09-15T22:08:43.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.732+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ac8102570 0x7f3ac81997b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800cce0 tx=0x7f3ab800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:43.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.732+0000 7f3ac5ffb640 1 -- 172.21.15.5:0/3562971039 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab800f650 con 0x7f3ac8102570 2024-09-15T22:08:43.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.732+0000 7f3ac5ffb640 1 -- 172.21.15.5:0/3562971039 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab800fcd0 con 0x7f3ac8102570 2024-09-15T22:08:43.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.732+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3562971039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac81a4a00 con 0x7f3ac8102570 2024-09-15T22:08:43.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.732+0000 7f3ac5ffb640 1 -- 172.21.15.5:0/3562971039 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8018c70 con 0x7f3ac8102570 2024-09-15T22:08:43.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.733+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3562971039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac81a4f40 con 0x7f3ac8102570 2024-09-15T22:08:43.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.734+0000 7f3ac5ffb640 1 -- 172.21.15.5:0/3562971039 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f3ab800f7f0 con 0x7f3ac8102570 2024-09-15T22:08:43.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.735+0000 7f3ac5ffb640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ab40a32c0 0x7f3ab40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.735+0000 7f3accc55640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ab40a32c0 0x7f3ab40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.735+0000 7f3ac5ffb640 1 -- 172.21.15.5:0/3562971039 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f3ab80d00a0 con 0x7f3ac8102570 2024-09-15T22:08:43.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.735+0000 7f3ad1292740 1 Processor -- start 2024-09-15T22:08:43.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.735+0000 7f3ad1292740 1 -- start start 2024-09-15T22:08:43.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.735+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1e58b0 0x56320a1e5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.735+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.735+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1f4fc0 0x56320a1f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.735+0000 7f3ad1292740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56320a14c090 con 0x56320a1e61d0 2024-09-15T22:08:43.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.735+0000 7f3ad1292740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56320a0f3a70 con 0x56320a1f4fc0 2024-09-15T22:08:43.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.735+0000 7f3ad1292740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56320a148050 con 0x56320a1e58b0 2024-09-15T22:08:43.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3accc55640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ab40a32c0 0x7f3ab40a5780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00058c0 tx=0x7f3ab000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:43.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3ac7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3ac7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51978/0 (socket says 172.21.15.5:51978) 2024-09-15T22:08:43.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3ac7fff640 1 -- 172.21.15.5:0/668572644 learned_addr learned my addr 172.21.15.5:0/668572644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:43.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3accc55640 1 --2- 172.21.15.5:0/668572644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1e58b0 0x56320a1e5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3acd456640 1 --2- 172.21.15.5:0/668572644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1f4fc0 0x56320a1f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3ac7fff640 1 -- 172.21.15.5:0/668572644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1e58b0 msgr2=0x56320a1e5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:43.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/668572644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1e58b0 0x56320a1e5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3ac7fff640 1 -- 172.21.15.5:0/668572644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1f4fc0 msgr2=0x56320a1f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:43.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/668572644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1f4fc0 0x56320a1f73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3ac7fff640 1 -- 172.21.15.5:0/668572644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320a100130 con 0x56320a1e61d0 2024-09-15T22:08:43.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3acd456640 1 --2- 172.21.15.5:0/668572644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1f4fc0 0x56320a1f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:43.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/668572644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f4a80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80a4a40 tx=0x7f3ab80986d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181ee22ac6a9bbd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:43.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3abeffd640 1 -- 172.21.15.5:0/668572644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab809ca00 con 0x56320a1e61d0 2024-09-15T22:08:43.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3abeffd640 1 -- 172.21.15.5:0/668572644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab809c300 con 0x56320a1e61d0 2024-09-15T22:08:43.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.736+0000 7f3abeffd640 1 -- 172.21.15.5:0/668572644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8098970 con 0x56320a1e61d0 2024-09-15T22:08:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.737+0000 7f3ad1292740 1 -- 172.21.15.5:0/668572644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 msgr2=0x56320a1f4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.737+0000 7f3ad1292740 1 --2- 172.21.15.5:0/668572644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f4a80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80a4a40 tx=0x7f3ab80986d0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.737+0000 7f3ad1292740 1 -- 172.21.15.5:0/668572644 shutdown_connections 2024-09-15T22:08:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.737+0000 7f3ad1292740 1 --2- 172.21.15.5:0/668572644 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1f4fc0 0x56320a1f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.737+0000 7f3ad1292740 1 --2- 172.21.15.5:0/668572644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f4a80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.737+0000 7f3ad1292740 1 --2- 172.21.15.5:0/668572644 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1e58b0 0x56320a1e5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.737+0000 7f3ad1292740 1 -- 172.21.15.5:0/668572644 >> 172.21.15.5:0/668572644 conn(0x56320a1e1f60 msgr2=0x56320a1e2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.737+0000 7f3ad1292740 1 -- 172.21.15.5:0/668572644 shutdown_connections 2024-09-15T22:08:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.737+0000 7f3ad1292740 1 -- 172.21.15.5:0/668572644 wait complete. 2024-09-15T22:08:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.737+0000 7f3ad1292740 1 -- start start 2024-09-15T22:08:43.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3ad1292740 1 Processor -- start 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e58b0 0x56320a1f4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1f4fc0 0x56320a1f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3ad1292740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56320a1f7b90 con 0x56320a1e61d0 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3ad1292740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56320a106f20 con 0x56320a1e58b0 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3ad1292740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56320a1f7d10 con 0x56320a1f4fc0 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3ad1292740 10 client.0 _ll_register_callbacks cb 0x56320a0d0030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3ac7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3ac7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51992/0 (socket says 172.21.15.5:51992) 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3ac7fff640 1 -- 172.21.15.5:0/649353186 learned_addr learned my addr 172.21.15.5:0/649353186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3acd456640 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1f4fc0 0x56320a1f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.738+0000 7f3accc55640 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e58b0 0x56320a1f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3ac7fff640 1 -- 172.21.15.5:0/649353186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1f4fc0 msgr2=0x56320a1f21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1f4fc0 0x56320a1f21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3ac7fff640 1 -- 172.21.15.5:0/649353186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e58b0 msgr2=0x56320a1f4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e58b0 0x56320a1f4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3ac7fff640 1 -- 172.21.15.5:0/649353186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563209ebd2b0 con 0x56320a1e61d0 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3accc55640 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e58b0 0x56320a1f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f1c60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80cdb50 tx=0x7f3ab80a4f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fee5b7213b42422 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3aa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3aa7fff640 1 -- 172.21.15.5:0/649353186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8018e10 con 0x56320a1e61d0 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3aa7fff640 1 -- 172.21.15.5:0/649353186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f3ab80a4cf0 con 0x56320a1e61d0 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3ad1292740 1 -- 172.21.15.5:0/649353186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ab809c300 con 0x56320a1e61d0 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3aa7fff640 1 -- 172.21.15.5:0/649353186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3ab809a410 con 0x56320a1e61d0 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3ad1292740 10 client.15162 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3ad1292740 1 -- 172.21.15.5:0/649353186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56320a2035d0 con 0x56320a1e61d0 2024-09-15T22:08:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3aa7fff640 1 -- 172.21.15.5:0/649353186 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ab809c300 con 0x56320a1e61d0 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.739+0000 7f3aa7fff640 1 -- 172.21.15.5:0/649353186 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f3ab80bc110 con 0x56320a1e61d0 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.740+0000 7f3ad1292740 10 client.15162 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.740+0000 7f3ad1292740 10 client.15162 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.740+0000 7f3ad1292740 20 client.15162 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.740+0000 7f3ad1292740 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x56320a204920 0x56320a206ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.740+0000 7f3ad1292740 4 client.15162 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.740+0000 7f3ad1292740 1 -- 172.21.15.5:0/649353186 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56320a159600 con 0x56320a204920 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.740+0000 7f3accc55640 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x56320a204920 0x56320a206ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.740+0000 7f3accc55640 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x56320a204920 0x56320a206ad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f5f6149605ae83ca server_cookie=4c46f6ccdccf9319 in_seq=0 out_seq=0 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.740+0000 7f3aa7fff640 10 client.15162 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3aa7fff640 1 -- 172.21.15.5:0/649353186 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56320a159600 con 0x56320a204920 2024-09-15T22:08:43.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3aa7fff640 10 client.15162 handle_command_reply: tid=0 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3ad1292740 1 client.15162 shutdown 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3ad1292740 1 -- 172.21.15.5:0/649353186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 msgr2=0x56320a1f1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3ad1292740 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f1c60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80cdb50 tx=0x7f3ab80a4f70 comp rx=0 tx=0).stop 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3ad1292740 1 -- 172.21.15.5:0/649353186 shutdown_connections 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3ad1292740 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x56320a204920 0x56320a206ad0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3ad1292740 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1f4fc0 0x56320a1f21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3ad1292740 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a1f1c60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3aa7fff640 0 client.15162 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3ad1292740 1 --2- 172.21.15.5:0/649353186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e58b0 0x56320a1f4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3ad1292740 1 -- 172.21.15.5:0/649353186 >> 172.21.15.5:0/649353186 conn(0x56320a1e1f60 msgr2=0x56320a1e7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3ad1292740 1 -- 172.21.15.5:0/649353186 shutdown_connections 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.741+0000 7f3ad1292740 1 -- 172.21.15.5:0/649353186 wait complete. 2024-09-15T22:08:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.742+0000 7f3ad1292740 20 client.15162 trim_cache size 0 max 16384 2024-09-15T22:08:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.759+0000 7f3ad1292740 1 Processor -- start 2024-09-15T22:08:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.759+0000 7f3ad1292740 1 -- start start 2024-09-15T22:08:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.759+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1e58b0 0x56320a206ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.759+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e61d0 0x56320a205d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.759+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1f4fc0 0x56320a204f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.759+0000 7f3ad1292740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56320a204e10 con 0x56320a1f4fc0 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.759+0000 7f3ad1292740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56320a2054e0 con 0x56320a1e61d0 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.759+0000 7f3ad1292740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56320a205bb0 con 0x56320a1e58b0 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3acd456640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1f4fc0 0x56320a204f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3acd456640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1f4fc0 0x56320a204f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51996/0 (socket says 172.21.15.5:51996) 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3acd456640 1 -- 172.21.15.5:0/78735531 learned_addr learned my addr 172.21.15.5:0/78735531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/78735531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e61d0 0x56320a205d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3accc55640 1 --2- 172.21.15.5:0/78735531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1e58b0 0x56320a206ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3acd456640 1 -- 172.21.15.5:0/78735531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1e58b0 msgr2=0x56320a206ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3acd456640 1 --2- 172.21.15.5:0/78735531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1e58b0 0x56320a206ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3acd456640 1 -- 172.21.15.5:0/78735531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e61d0 msgr2=0x56320a205d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3acd456640 1 --2- 172.21.15.5:0/78735531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e61d0 0x56320a205d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3acd456640 1 -- 172.21.15.5:0/78735531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56320a1f4c80 con 0x56320a1f4fc0 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3acd456640 1 --2- 172.21.15.5:0/78735531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1f4fc0 0x56320a204f90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009900 tx=0x7f3ac0004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3f92beb07bae1c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3abe7fc640 1 -- 172.21.15.5:0/78735531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0035070 con 0x56320a1f4fc0 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3abe7fc640 1 -- 172.21.15.5:0/78735531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac0003ad0 con 0x56320a1f4fc0 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.760+0000 7f3abe7fc640 1 -- 172.21.15.5:0/78735531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0031b70 con 0x56320a1f4fc0 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.761+0000 7f3ad1292740 1 -- 172.21.15.5:0/78735531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1f4fc0 msgr2=0x56320a204f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.761+0000 7f3ad1292740 1 --2- 172.21.15.5:0/78735531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1f4fc0 0x56320a204f90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009900 tx=0x7f3ac0004370 comp rx=0 tx=0).stop 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.761+0000 7f3ad1292740 1 -- 172.21.15.5:0/78735531 shutdown_connections 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.761+0000 7f3ad1292740 1 --2- 172.21.15.5:0/78735531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1f4fc0 0x56320a204f90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.761+0000 7f3ad1292740 1 --2- 172.21.15.5:0/78735531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e61d0 0x56320a205d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.761+0000 7f3ad1292740 1 --2- 172.21.15.5:0/78735531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1e58b0 0x56320a206ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.761+0000 7f3ad1292740 1 -- 172.21.15.5:0/78735531 >> 172.21.15.5:0/78735531 conn(0x56320a1e1f60 msgr2=0x56320a1dcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.761+0000 7f3ad1292740 1 -- 172.21.15.5:0/78735531 shutdown_connections 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.761+0000 7f3ad1292740 1 -- 172.21.15.5:0/78735531 wait complete. 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.761+0000 7f3ad1292740 1 -- start start 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ad1292740 1 Processor -- start 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e58b0 0x56320a20eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a20f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ad1292740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1f4fc0 0x56320a208240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ad1292740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56320a206950 con 0x56320a1e61d0 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ad1292740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56320a206280 con 0x56320a1e58b0 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ad1292740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56320a207020 con 0x56320a1f4fc0 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ad1292740 10 client.0 _ll_register_callbacks cb 0x7f3ab000a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:43.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ac7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a20f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ac7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a20f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52008/0 (socket says 172.21.15.5:52008) 2024-09-15T22:08:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ac7fff640 1 -- 172.21.15.5:0/3679495917 learned_addr learned my addr 172.21.15.5:0/3679495917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3accc55640 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e58b0 0x56320a20eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3acd456640 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1f4fc0 0x56320a208240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.762+0000 7f3ac7fff640 1 -- 172.21.15.5:0/3679495917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1f4fc0 msgr2=0x56320a208240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:43.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1f4fc0 0x56320a208240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3ac7fff640 1 -- 172.21.15.5:0/3679495917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e58b0 msgr2=0x56320a20eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e58b0 0x56320a20eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3ac7fff640 1 -- 172.21.15.5:0/3679495917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56320a1e6000 con 0x56320a1e61d0 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3ac7fff640 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a20f400 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3ab809c000 tx=0x7f3ab809bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406d5f989789f444 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3aa7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3aa7fff640 1 -- 172.21.15.5:0/3679495917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab80985a0 con 0x56320a1e61d0 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3aa7fff640 1 -- 172.21.15.5:0/3679495917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f3ab809b960 con 0x56320a1e61d0 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3ad1292740 1 -- 172.21.15.5:0/3679495917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ac0003ad0 con 0x56320a1e61d0 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3aa7fff640 1 -- 172.21.15.5:0/3679495917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3ab809ca00 con 0x56320a1e61d0 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3ad1292740 10 client.15168 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3ad1292740 1 -- 172.21.15.5:0/3679495917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56320a20d1b0 con 0x56320a1e61d0 2024-09-15T22:08:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.763+0000 7f3aa7fff640 1 -- 172.21.15.5:0/3679495917 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ac0003ad0 con 0x56320a1e61d0 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.764+0000 7f3aa7fff640 1 -- 172.21.15.5:0/3679495917 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f3ab809b1c0 con 0x56320a1e61d0 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.764+0000 7f3ad1292740 10 client.15168 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.764+0000 7f3ad1292740 10 client.15168 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.764+0000 7f3ad1292740 20 client.15168 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.764+0000 7f3ad1292740 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x56320a273f30 0x56320a272c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.764+0000 7f3ad1292740 4 client.15168 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.764+0000 7f3ad1292740 1 -- 172.21.15.5:0/3679495917 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "status"}) -- 0x56320a153b10 con 0x56320a273f30 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.764+0000 7f3accc55640 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x56320a273f30 0x56320a272c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.764+0000 7f3aa7fff640 10 client.15168 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.764+0000 7f3accc55640 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x56320a273f30 0x56320a272c40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=694c84eb9824a2d4 server_cookie=dd95f7014ac29eb7 in_seq=0 out_seq=0 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3aa7fff640 1 -- 172.21.15.5:0/3679495917 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56320a153b10 con 0x56320a273f30 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3aa7fff640 10 client.15168 handle_command_reply: tid=0 2024-09-15T22:08:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 1 client.15168 shutdown 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 1 -- 172.21.15.5:0/3679495917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 msgr2=0x56320a20f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a20f400 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3ab809c000 tx=0x7f3ab809bec0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 1 -- 172.21.15.5:0/3679495917 shutdown_connections 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x56320a273f30 0x56320a272c40 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56320a1f4fc0 0x56320a208240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56320a1e61d0 0x56320a20f400 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 1 --2- 172.21.15.5:0/3679495917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56320a1e58b0 0x56320a20eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3aa7fff640 0 client.15168 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 1 -- 172.21.15.5:0/3679495917 >> 172.21.15.5:0/3679495917 conn(0x56320a1e1f60 msgr2=0x56320a1ed240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 1 -- 172.21.15.5:0/3679495917 shutdown_connections 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 1 -- 172.21.15.5:0/3679495917 wait complete. 2024-09-15T22:08:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.765+0000 7f3ad1292740 20 client.15168 trim_cache size 0 max 16384 2024-09-15T22:08:43.776 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:08:43.776 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T22:08:43.776 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-15T22:08:43.776 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24466, 2024-09-15T22:08:43.776 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-15T22:08:43.776 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-15T22:08:43.776 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-15T22:08:43.777 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 90.039072121000004, 2024-09-15T22:08:43.777 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 24, 2024-09-15T22:08:43.777 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-09-15T22:08:43.777 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:08:43.777 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 146.66064281600001 2024-09-15T22:08:43.777 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:08:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3562971039 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ab40a32c0 msgr2=0x7f3ab40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ab40a32c0 0x7f3ab40a5780 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3ab00058c0 tx=0x7f3ab000a2d0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3562971039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ac8102570 msgr2=0x7f3ac81997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ac8102570 0x7f3ac81997b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800cce0 tx=0x7f3ab800a620 comp rx=0 tx=0).stop 2024-09-15T22:08:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3562971039 shutdown_connections 2024-09-15T22:08:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3ac8102ef0 0x7f3ac81a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f3ab40a32c0 0x7f3ab40a5780 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ac8102570 0x7f3ac81997b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 --2- 172.21.15.5:0/3562971039 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3ac8100e20 0x7f3ac8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3562971039 >> 172.21.15.5:0/3562971039 conn(0x7f3ac80fc7d0 msgr2=0x7f3ac81067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3562971039 shutdown_connections 2024-09-15T22:08:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:43.766+0000 7f3aceeb9640 1 -- 172.21.15.5:0/3562971039 wait complete. 2024-09-15T22:08:44.426 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:44 smithi005 ceph-mon[30944]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:08:44.427 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:44 smithi005 ceph-mon[30944]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:08:44.458 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T22:08:44.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:44 smithi099 ceph-mon[34892]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:08:44.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:44 smithi099 ceph-mon[34892]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:08:44.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:44 smithi129 ceph-mon[35314]: pgmap v348: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-15T22:08:44.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:44 smithi129 ceph-mon[35314]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-15T22:08:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:45.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.211+0000 7fdc03ca8640 1 -- 172.21.15.5:0/1383879638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbfc102790 msgr2=0x7fdbfc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:45.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.211+0000 7fdc03ca8640 1 --2- 172.21.15.5:0/1383879638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbfc102790 0x7fdbfc1067e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009900 tx=0x7fdbec02a410 comp rx=0 tx=0).stop 2024-09-15T22:08:45.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.211+0000 7fdc03ca8640 1 -- 172.21.15.5:0/1383879638 shutdown_connections 2024-09-15T22:08:45.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.211+0000 7fdc03ca8640 1 --2- 172.21.15.5:0/1383879638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbfc106f10 0x7fdbfc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.211+0000 7fdc03ca8640 1 --2- 172.21.15.5:0/1383879638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbfc102790 0x7fdbfc1067e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.211+0000 7fdc03ca8640 1 --2- 172.21.15.5:0/1383879638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbfc101de0 0x7fdbfc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.211+0000 7fdc03ca8640 1 -- 172.21.15.5:0/1383879638 >> 172.21.15.5:0/1383879638 conn(0x7fdbfc0fd8c0 msgr2=0x7fdbfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:45.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.212+0000 7fdc03ca8640 1 -- 172.21.15.5:0/1383879638 shutdown_connections 2024-09-15T22:08:45.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.212+0000 7fdc03ca8640 1 -- 172.21.15.5:0/1383879638 wait complete. 2024-09-15T22:08:45.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.212+0000 7fdc03ca8640 1 Processor -- start 2024-09-15T22:08:45.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.212+0000 7fdc03ca8640 1 -- start start 2024-09-15T22:08:45.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.212+0000 7fdc03ca8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbfc101de0 0x7fdbfc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.212+0000 7fdc03ca8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbfc102790 0x7fdbfc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.212+0000 7fdc03ca8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbfc106f10 0x7fdbfc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.212+0000 7fdc03ca8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc111f70 con 0x7fdbfc102790 2024-09-15T22:08:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.212+0000 7fdc03ca8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc111df0 con 0x7fdbfc106f10 2024-09-15T22:08:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.212+0000 7fdc03ca8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbfc1120f0 con 0x7fdbfc101de0 2024-09-15T22:08:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc01243640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbfc102790 0x7fdbfc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc01243640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbfc102790 0x7fdbfc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52022/0 (socket says 172.21.15.5:52022) 2024-09-15T22:08:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc01243640 1 -- 172.21.15.5:0/2135738262 learned_addr learned my addr 172.21.15.5:0/2135738262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc02245640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbfc106f10 0x7fdbfc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc01a44640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbfc101de0 0x7fdbfc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc01243640 1 -- 172.21.15.5:0/2135738262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbfc101de0 msgr2=0x7fdbfc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc01243640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbfc101de0 0x7fdbfc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc01243640 1 -- 172.21.15.5:0/2135738262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbfc106f10 msgr2=0x7fdbfc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc01243640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbfc106f10 0x7fdbfc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc01243640 1 -- 172.21.15.5:0/2135738262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbfc1a6970 con 0x7fdbfc102790 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc01243640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbfc102790 0x7fdbfc074a30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdbec006010 tx=0x7fdbec0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdbf2ffd640 1 -- 172.21.15.5:0/2135738262 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec03f030 con 0x7fdbfc102790 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdbf2ffd640 1 -- 172.21.15.5:0/2135738262 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbec003ce0 con 0x7fdbfc102790 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc03ca8640 1 -- 172.21.15.5:0/2135738262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbfc1a6c00 con 0x7fdbfc102790 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdbf2ffd640 1 -- 172.21.15.5:0/2135738262 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec035070 con 0x7fdbfc102790 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.213+0000 7fdc03ca8640 1 -- 172.21.15.5:0/2135738262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbfc1a7130 con 0x7fdbfc102790 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.216+0000 7fdbf2ffd640 1 -- 172.21.15.5:0/2135738262 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fdbec042030 con 0x7fdbfc102790 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.216+0000 7fdc06081740 1 Processor -- start 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc06081740 1 -- start start 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc06081740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b036db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc06081740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b041670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc06081740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b041bb0 0x55b99b043fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc06081740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b99af9d090 con 0x55b99b0372f0 2024-09-15T22:08:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc06081740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b99af44a70 con 0x55b99b041bb0 2024-09-15T22:08:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc06081740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b99af99050 con 0x55b99b0369d0 2024-09-15T22:08:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc01a44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b041670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc01a44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b041670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52038/0 (socket says 172.21.15.5:52038) 2024-09-15T22:08:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc01a44640 1 -- 172.21.15.5:0/2244262902 learned_addr learned my addr 172.21.15.5:0/2244262902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc02245640 1 --2- 172.21.15.5:0/2244262902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b036db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc01243640 1 --2- 172.21.15.5:0/2244262902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b041bb0 0x55b99b043fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdbf2ffd640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdbe80a32c0 0x7fdbe80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc01a44640 1 -- 172.21.15.5:0/2244262902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 msgr2=0x55b99b036db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc01a44640 1 --2- 172.21.15.5:0/2244262902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b036db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc01a44640 1 -- 172.21.15.5:0/2244262902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b041bb0 msgr2=0x55b99b043fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc01a44640 1 --2- 172.21.15.5:0/2244262902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b041bb0 0x55b99b043fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc01a44640 1 -- 172.21.15.5:0/2244262902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b99af51130 con 0x55b99b0372f0 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc01243640 1 --2- 172.21.15.5:0/2244262902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b041bb0 0x55b99b043fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.217+0000 7fdc02245640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdbe80a32c0 0x7fdbe80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.218+0000 7fdbf2ffd640 1 -- 172.21.15.5:0/2135738262 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fdbec0f0230 con 0x7fdbfc102790 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.218+0000 7fdc01a44640 1 --2- 172.21.15.5:0/2244262902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b041670 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800ac10 tx=0x7fdbf800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588a2446c88032f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.218+0000 7fdbe7fff640 1 -- 172.21.15.5:0/2244262902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf801d070 con 0x55b99b0372f0 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.218+0000 7fdbe7fff640 1 -- 172.21.15.5:0/2244262902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf8018b20 con 0x55b99b0372f0 2024-09-15T22:08:45.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.218+0000 7fdbe7fff640 1 -- 172.21.15.5:0/2244262902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8010ba0 con 0x55b99b0372f0 2024-09-15T22:08:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc06081740 1 -- 172.21.15.5:0/2244262902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 msgr2=0x55b99b041670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc06081740 1 --2- 172.21.15.5:0/2244262902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b041670 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800ac10 tx=0x7fdbf800f790 comp rx=0 tx=0).stop 2024-09-15T22:08:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc02245640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdbe80a32c0 0x7fdbe80a5780 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4005fd0 tx=0x7fdbf4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc06081740 1 -- 172.21.15.5:0/2244262902 shutdown_connections 2024-09-15T22:08:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc06081740 1 --2- 172.21.15.5:0/2244262902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b041bb0 0x55b99b043fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc06081740 1 --2- 172.21.15.5:0/2244262902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b041670 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc06081740 1 --2- 172.21.15.5:0/2244262902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b036db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc06081740 1 -- 172.21.15.5:0/2244262902 >> 172.21.15.5:0/2244262902 conn(0x55b99b032f60 msgr2=0x55b99b033340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc06081740 1 -- 172.21.15.5:0/2244262902 shutdown_connections 2024-09-15T22:08:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc06081740 1 -- 172.21.15.5:0/2244262902 wait complete. 2024-09-15T22:08:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc06081740 1 -- start start 2024-09-15T22:08:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.219+0000 7fdc06081740 1 Processor -- start 2024-09-15T22:08:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc06081740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b041470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc06081740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 0x55b99b03e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc06081740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc06081740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b99b044780 con 0x55b99b041bb0 2024-09-15T22:08:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc06081740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b99af57f20 con 0x55b99b0372f0 2024-09-15T22:08:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc06081740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b99b044900 con 0x55b99b0369d0 2024-09-15T22:08:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc06081740 10 client.0 _ll_register_callbacks cb 0x55b99af21030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc02245640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc02245640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52042/0 (socket says 172.21.15.5:52042) 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc01a44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b041470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc02245640 1 -- 172.21.15.5:0/1813360401 learned_addr learned my addr 172.21.15.5:0/1813360401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc01243640 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 0x55b99b03e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc02245640 1 -- 172.21.15.5:0/1813360401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 msgr2=0x55b99b041470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc02245640 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b041470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc02245640 1 -- 172.21.15.5:0/1813360401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 msgr2=0x55b99b03e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc02245640 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 0x55b99b03e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc02245640 1 -- 172.21.15.5:0/1813360401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b99ad0e2b0 con 0x55b99b041bb0 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc01a44640 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b041470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc01243640 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 0x55b99b03e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdc02245640 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03ed00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40126f0 tx=0x7fdbf4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=618db7fdb81f9416 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdbe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:08:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.220+0000 7fdbe57fa640 1 -- 172.21.15.5:0/1813360401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf40107d0 con 0x55b99b041bb0 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdbe57fa640 1 -- 172.21.15.5:0/1813360401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fdbf4010970 con 0x55b99b041bb0 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdc06081740 1 -- 172.21.15.5:0/1813360401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf8018b20 con 0x55b99b041bb0 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdbe57fa640 1 -- 172.21.15.5:0/1813360401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdbf4010b70 con 0x55b99b041bb0 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdc06081740 10 client.15180 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdc06081740 1 -- 172.21.15.5:0/1813360401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b99b050160 con 0x55b99b041bb0 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdbe57fa640 1 -- 172.21.15.5:0/1813360401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf8018b20 con 0x55b99b041bb0 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdbe57fa640 1 -- 172.21.15.5:0/1813360401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fdbf400f040 con 0x55b99b041bb0 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdc06081740 10 client.15180 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdc06081740 10 client.15180 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdc06081740 20 client.15180 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdc06081740 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b99b052810 0x55b99b053c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdc06081740 4 client.15180 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:08:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.221+0000 7fdc06081740 1 -- 172.21.15.5:0/1813360401 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b99afaa600 con 0x55b99b052810 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.222+0000 7fdc01a44640 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b99b052810 0x55b99b053c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.222+0000 7fdbe57fa640 10 client.15180 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.222+0000 7fdc01a44640 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b99b052810 0x55b99b053c80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bbecfdcc9869b68 server_cookie=f31642b628963617 in_seq=0 out_seq=0 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdbe57fa640 1 -- 172.21.15.5:0/1813360401 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b99afaa600 con 0x55b99b052810 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdbe57fa640 10 client.15180 handle_command_reply: tid=0 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 1 client.15180 shutdown 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 1 -- 172.21.15.5:0/1813360401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 msgr2=0x55b99b03ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03ed00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40126f0 tx=0x7fdbf4004ab0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 1 -- 172.21.15.5:0/1813360401 shutdown_connections 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b99b052810 0x55b99b053c80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03ed00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 0x55b99b03e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdbe57fa640 0 client.15180 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:08:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 1 --2- 172.21.15.5:0/1813360401 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b041470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 1 -- 172.21.15.5:0/1813360401 >> 172.21.15.5:0/1813360401 conn(0x55b99b032f60 msgr2=0x55b99b038c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 1 -- 172.21.15.5:0/1813360401 shutdown_connections 2024-09-15T22:08:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 1 -- 172.21.15.5:0/1813360401 wait complete. 2024-09-15T22:08:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.223+0000 7fdc06081740 20 client.15180 trim_cache size 0 max 16384 2024-09-15T22:08:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.241+0000 7fdc06081740 1 Processor -- start 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.241+0000 7fdc06081740 1 -- start start 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.241+0000 7fdc06081740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b053c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.241+0000 7fdc06081740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 0x55b99b03d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.241+0000 7fdc06081740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.241+0000 7fdc06081740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b99b051fc0 con 0x55b99b041bb0 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.241+0000 7fdc06081740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b99b052690 con 0x55b99b0372f0 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.241+0000 7fdc06081740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b99b052d60 con 0x55b99b0369d0 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc02245640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc02245640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52050/0 (socket says 172.21.15.5:52050) 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc02245640 1 -- 172.21.15.5:0/3263504538 learned_addr learned my addr 172.21.15.5:0/3263504538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc01a44640 1 --2- 172.21.15.5:0/3263504538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b053c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc01243640 1 --2- 172.21.15.5:0/3263504538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 0x55b99b03d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc02245640 1 -- 172.21.15.5:0/3263504538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 msgr2=0x55b99b053c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc02245640 1 --2- 172.21.15.5:0/3263504538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b053c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc02245640 1 -- 172.21.15.5:0/3263504538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 msgr2=0x55b99b03d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc02245640 1 --2- 172.21.15.5:0/3263504538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 0x55b99b03d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc02245640 1 -- 172.21.15.5:0/3263504538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b99b041090 con 0x55b99b041bb0 2024-09-15T22:08:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc01243640 1 --2- 172.21.15.5:0/3263504538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 0x55b99b03d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdc02245640 1 --2- 172.21.15.5:0/3263504538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03d5d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40044b0 tx=0x7fdbf4009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3f9a1e5645b86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdbe77fe640 1 -- 172.21.15.5:0/3263504538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4039070 con 0x55b99b041bb0 2024-09-15T22:08:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdbe77fe640 1 -- 172.21.15.5:0/3263504538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf4025020 con 0x55b99b041bb0 2024-09-15T22:08:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.242+0000 7fdbe77fe640 1 -- 172.21.15.5:0/3263504538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4024020 con 0x55b99b041bb0 2024-09-15T22:08:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.243+0000 7fdc06081740 1 -- 172.21.15.5:0/3263504538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 msgr2=0x55b99b03d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.243+0000 7fdc06081740 1 --2- 172.21.15.5:0/3263504538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03d5d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdbf40044b0 tx=0x7fdbf4009600 comp rx=0 tx=0).stop 2024-09-15T22:08:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.243+0000 7fdc06081740 1 -- 172.21.15.5:0/3263504538 shutdown_connections 2024-09-15T22:08:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.243+0000 7fdc06081740 1 --2- 172.21.15.5:0/3263504538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b041bb0 0x55b99b03d5d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.243+0000 7fdc06081740 1 --2- 172.21.15.5:0/3263504538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0372f0 0x55b99b03d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.243+0000 7fdc06081740 1 --2- 172.21.15.5:0/3263504538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b0369d0 0x55b99b053c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.243+0000 7fdc06081740 1 -- 172.21.15.5:0/3263504538 >> 172.21.15.5:0/3263504538 conn(0x55b99b032f60 msgr2=0x55b99b03e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.243+0000 7fdc06081740 1 -- 172.21.15.5:0/3263504538 shutdown_connections 2024-09-15T22:08:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.243+0000 7fdc06081740 1 -- 172.21.15.5:0/3263504538 wait complete. 2024-09-15T22:08:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.243+0000 7fdc06081740 1 -- start start 2024-09-15T22:08:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.243+0000 7fdc06081740 1 Processor -- start 2024-09-15T22:08:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc06081740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0369d0 0x55b99b057660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc06081740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b057ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc06081740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b041bb0 0x55b99b0b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc06081740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b99b053b00 con 0x55b99b0372f0 2024-09-15T22:08:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc06081740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b99b053430 con 0x55b99b0369d0 2024-09-15T22:08:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc06081740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b99b0541d0 con 0x55b99b041bb0 2024-09-15T22:08:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc06081740 10 client.0 _ll_register_callbacks cb 0x7fdbf4004860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc01243640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b057ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc01243640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b057ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52060/0 (socket says 172.21.15.5:52060) 2024-09-15T22:08:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc01243640 1 -- 172.21.15.5:0/2430732482 learned_addr learned my addr 172.21.15.5:0/2430732482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc02245640 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b041bb0 0x55b99b0b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc01a44640 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0369d0 0x55b99b057660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc01243640 1 -- 172.21.15.5:0/2430732482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b041bb0 msgr2=0x55b99b0b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc01243640 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b041bb0 0x55b99b0b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc01243640 1 -- 172.21.15.5:0/2430732482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0369d0 msgr2=0x55b99b057660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc01243640 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0369d0 0x55b99b057660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc01243640 1 -- 172.21.15.5:0/2430732482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b99b0b5580 con 0x55b99b0372f0 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.244+0000 7fdc02245640 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b041bb0 0x55b99b0b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdc01243640 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b057ba0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdbec000c00 tx=0x7fdbec0bc370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6f677570256642 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdbe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdbe57fa640 1 -- 172.21.15.5:0/2430732482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec049070 con 0x55b99b0372f0 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdbe57fa640 1 -- 172.21.15.5:0/2430732482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fdbec0bc800 con 0x55b99b0372f0 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdc06081740 1 -- 172.21.15.5:0/2430732482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbf4025020 con 0x55b99b0372f0 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdbe57fa640 1 -- 172.21.15.5:0/2430732482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdbec03bcb0 con 0x55b99b0372f0 2024-09-15T22:08:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdc06081740 10 client.15186 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdc06081740 1 -- 172.21.15.5:0/2430732482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b99b0bc8a0 con 0x55b99b0372f0 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdbe57fa640 1 -- 172.21.15.5:0/2430732482 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbf4025020 con 0x55b99b0372f0 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdbe57fa640 1 -- 172.21.15.5:0/2430732482 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fdbec0c4cd0 con 0x55b99b0372f0 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdc06081740 10 client.15186 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdc06081740 10 client.15186 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdc06081740 20 client.15186 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdc06081740 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b99b0c14f0 0x55b99b0c0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdc06081740 4 client.15186 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.245+0000 7fdc06081740 1 -- 172.21.15.5:0/2430732482 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "status"}) -- 0x55b99afa4b10 con 0x55b99b0c14f0 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.246+0000 7fdc01a44640 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b99b0c14f0 0x55b99b0c0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.246+0000 7fdbe57fa640 10 client.15186 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.246+0000 7fdc01a44640 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b99b0c14f0 0x55b99b0c0200 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10cb6686f0430ae7 server_cookie=3e6a8f11120412e2 in_seq=0 out_seq=0 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.246+0000 7fdbe57fa640 1 -- 172.21.15.5:0/2430732482 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b99afa4b10 con 0x55b99b0c14f0 2024-09-15T22:08:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.246+0000 7fdbe57fa640 10 client.15186 handle_command_reply: tid=0 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.246+0000 7fdc06081740 1 client.15186 shutdown 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.246+0000 7fdc06081740 1 -- 172.21.15.5:0/2430732482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 msgr2=0x55b99b057ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.246+0000 7fdc06081740 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b057ba0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdbec000c00 tx=0x7fdbec0bc370 comp rx=0 tx=0).stop 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc06081740 1 -- 172.21.15.5:0/2430732482 shutdown_connections 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc06081740 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b99b0c14f0 0x55b99b0c0200 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc06081740 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b99b041bb0 0x55b99b0b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc06081740 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b99b0372f0 0x55b99b057ba0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc06081740 1 --2- 172.21.15.5:0/2430732482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b99b0369d0 0x55b99b057660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdbe57fa640 0 client.15186 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc06081740 1 -- 172.21.15.5:0/2430732482 >> 172.21.15.5:0/2430732482 conn(0x55b99b032f60 msgr2=0x55b99b0317e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc06081740 1 -- 172.21.15.5:0/2430732482 shutdown_connections 2024-09-15T22:08:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc06081740 1 -- 172.21.15.5:0/2430732482 wait complete. 2024-09-15T22:08:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc06081740 20 client.15186 trim_cache size 0 max 16384 2024-09-15T22:08:45.256 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:08:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T22:08:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-15T22:08:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24467, 2024-09-15T22:08:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-15T22:08:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-15T22:08:45.256 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-15T22:08:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 92.529817136000005, 2024-09-15T22:08:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 24, 2024-09-15T22:08:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-09-15T22:08:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:08:45.257 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 150.692627695 2024-09-15T22:08:45.257 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:08:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 -- 172.21.15.5:0/2135738262 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdbe80a32c0 msgr2=0x7fdbe80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdbe80a32c0 0x7fdbe80a5780 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4005fd0 tx=0x7fdbf4004a40 comp rx=0 tx=0).stop 2024-09-15T22:08:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 -- 172.21.15.5:0/2135738262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbfc102790 msgr2=0x7fdbfc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbfc102790 0x7fdbfc074a30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdbec006010 tx=0x7fdbec0042f0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 -- 172.21.15.5:0/2135738262 shutdown_connections 2024-09-15T22:08:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbfc106f10 0x7fdbfc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fdbe80a32c0 0x7fdbe80a5780 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbfc102790 0x7fdbfc074a30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 --2- 172.21.15.5:0/2135738262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbfc101de0 0x7fdbfc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:45.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 -- 172.21.15.5:0/2135738262 >> 172.21.15.5:0/2135738262 conn(0x7fdbfc0fd8c0 msgr2=0x7fdbfc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 -- 172.21.15.5:0/2135738262 shutdown_connections 2024-09-15T22:08:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:45.247+0000 7fdc03ca8640 1 -- 172.21.15.5:0/2135738262 wait complete. 2024-09-15T22:08:45.988 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:08:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:46.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:46 smithi005 ceph-mon[30944]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:08:46.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:46 smithi099 ceph-mon[34892]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:08:46.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:46 smithi129 ceph-mon[35314]: pgmap v349: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-15T22:08:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.727+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/2962629191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb0103660 msgr2=0x7f8cb010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.727+0000 7f8cb4a9e640 1 --2- 172.21.15.5:0/2962629191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb0103660 0x7f8cb010bb50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009900 tx=0x7f8c9802a480 comp rx=0 tx=0).stop 2024-09-15T22:08:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.727+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/2962629191 shutdown_connections 2024-09-15T22:08:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.727+0000 7f8cb4a9e640 1 --2- 172.21.15.5:0/2962629191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cb010c170 0x7f8cb010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.727+0000 7f8cb4a9e640 1 --2- 172.21.15.5:0/2962629191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb0103660 0x7f8cb010bb50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.727+0000 7f8cb4a9e640 1 --2- 172.21.15.5:0/2962629191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb0102d40 0x7f8cb0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.727+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/2962629191 >> 172.21.15.5:0/2962629191 conn(0x7f8cb00fc7a0 msgr2=0x7f8cb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.727+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/2962629191 shutdown_connections 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.727+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/2962629191 wait complete. 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cb4a9e640 1 Processor -- start 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cb4a9e640 1 -- start start 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cb4a9e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb0102d40 0x7f8cb0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cb4a9e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cb0103660 0x7f8cb01989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cb4a9e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb010c170 0x7f8cb019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cb4a9e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb0111fb0 con 0x7f8cb0102d40 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cb4a9e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb0111e30 con 0x7f8cb0103660 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cb4a9e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb0112130 con 0x7f8cb010c170 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb0102d40 0x7f8cb0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb0102d40 0x7f8cb0198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52080/0 (socket says 172.21.15.5:52080) 2024-09-15T22:08:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cae59c640 1 -- 172.21.15.5:0/211112882 learned_addr learned my addr 172.21.15.5:0/211112882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8caed9d640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb010c170 0x7f8cb019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8ca7fff640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cb0103660 0x7f8cb01989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cae59c640 1 -- 172.21.15.5:0/211112882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb010c170 msgr2=0x7f8cb019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cae59c640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb010c170 0x7f8cb019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cae59c640 1 -- 172.21.15.5:0/211112882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cb0103660 msgr2=0x7f8cb01989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cae59c640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cb0103660 0x7f8cb01989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cae59c640 1 -- 172.21.15.5:0/211112882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb019d440 con 0x7f8cb0102d40 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8ca7fff640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cb0103660 0x7f8cb01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.728+0000 7f8cae59c640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb0102d40 0x7f8cb0198490 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b4a0 tx=0x7f8ca000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.729+0000 7f8ca77fe640 1 -- 172.21.15.5:0/211112882 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0014070 con 0x7f8cb0102d40 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.729+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/211112882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb019d730 con 0x7f8cb0102d40 2024-09-15T22:08:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.729+0000 7f8ca77fe640 1 -- 172.21.15.5:0/211112882 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca0010af0 con 0x7f8cb0102d40 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.729+0000 7f8ca77fe640 1 -- 172.21.15.5:0/211112882 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0017d10 con 0x7f8cb0102d40 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.729+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/211112882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb01a5010 con 0x7f8cb0102d40 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.731+0000 7f8ca77fe640 1 -- 172.21.15.5:0/211112882 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7f8ca0017070 con 0x7f8cb0102d40 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.731+0000 7f8cb6e77740 1 Processor -- start 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cb6e77740 1 -- start start 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da68b0 0x565474da6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da71d0 0x565474db5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474db83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565474d0d090 con 0x565474da71d0 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565474cb4a70 con 0x565474db5fc0 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565474d09050 con 0x565474da68b0 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8ca77fe640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8c9c0a3240 0x7f8c9c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da71d0 0x565474db5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da71d0 0x565474db5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52094/0 (socket says 172.21.15.5:52094) 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cae59c640 1 -- 172.21.15.5:0/1253580931 learned_addr learned my addr 172.21.15.5:0/1253580931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8caed9d640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8c9c0a3240 0x7f8c9c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8caed9d640 1 --2- 172.21.15.5:0/1253580931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da68b0 0x565474da6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8ca7fff640 1 --2- 172.21.15.5:0/1253580931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474db83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8ca77fe640 1 -- 172.21.15.5:0/211112882 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f8ca00cf0a0 con 0x7f8cb0102d40 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cae59c640 1 -- 172.21.15.5:0/1253580931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da68b0 msgr2=0x565474da6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cae59c640 1 --2- 172.21.15.5:0/1253580931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da68b0 0x565474da6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cae59c640 1 -- 172.21.15.5:0/1253580931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 msgr2=0x565474db83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cae59c640 1 --2- 172.21.15.5:0/1253580931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474db83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.732+0000 7f8cae59c640 1 -- 172.21.15.5:0/1253580931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565474cc1130 con 0x565474da71d0 2024-09-15T22:08:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.733+0000 7f8caed9d640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8c9c0a3240 0x7f8c9c0a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80098b0 tx=0x7f8ca8009e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.733+0000 7f8cae59c640 1 --2- 172.21.15.5:0/1253580931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da71d0 0x565474db5a80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00cc7e0 tx=0x7f8ca00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceee48f5c03a1c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.733+0000 7f8ca4ff9640 1 -- 172.21.15.5:0/1253580931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca001e070 con 0x565474da71d0 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.733+0000 7f8ca4ff9640 1 -- 172.21.15.5:0/1253580931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca009ba00 con 0x565474da71d0 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.733+0000 7f8ca4ff9640 1 -- 172.21.15.5:0/1253580931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0017430 con 0x565474da71d0 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1253580931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da71d0 msgr2=0x565474db5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/1253580931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da71d0 0x565474db5a80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8ca00cc7e0 tx=0x7f8ca00047d0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1253580931 shutdown_connections 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/1253580931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474db83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/1253580931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da71d0 0x565474db5a80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/1253580931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da68b0 0x565474da6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1253580931 >> 172.21.15.5:0/1253580931 conn(0x565474da2f60 msgr2=0x565474da3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1253580931 shutdown_connections 2024-09-15T22:08:46.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1253580931 wait complete. 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 -- start start 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 Processor -- start 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474db58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da71d0 0x565474db2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474db3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565474db8b90 con 0x565474da68b0 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565474cc7f20 con 0x565474db5fc0 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565474db8d10 con 0x565474da71d0 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cb6e77740 10 client.0 _ll_register_callbacks cb 0x565474c91030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.734+0000 7f8cae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474db58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8cae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474db58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52104/0 (socket says 172.21.15.5:52104) 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8cae59c640 1 -- 172.21.15.5:0/1524179733 learned_addr learned my addr 172.21.15.5:0/1524179733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8caed9d640 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474db3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8cae59c640 1 -- 172.21.15.5:0/1524179733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da71d0 msgr2=0x565474db2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8cae59c640 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da71d0 0x565474db2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8cae59c640 1 -- 172.21.15.5:0/1524179733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 msgr2=0x565474db3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8cae59c640 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474db3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8cae59c640 1 -- 172.21.15.5:0/1524179733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565474a7e2b0 con 0x565474da68b0 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8cae59c640 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474db58f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b470 tx=0x7f8ca0009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7c8982c06035f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8c95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8c95ffb640 1 -- 172.21.15.5:0/1524179733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00b4060 con 0x565474da68b0 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8c95ffb640 1 -- 172.21.15.5:0/1524179733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f8ca00b4200 con 0x565474da68b0 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1524179733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca009ba00 con 0x565474da68b0 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8c95ffb640 1 -- 172.21.15.5:0/1524179733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8ca001e040 con 0x565474da68b0 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8cb6e77740 10 client.15198 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1524179733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x565474dc4580 con 0x565474da68b0 2024-09-15T22:08:46.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.735+0000 7f8c95ffb640 1 -- 172.21.15.5:0/1524179733 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca009ba00 con 0x565474da68b0 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.736+0000 7f8c95ffb640 1 -- 172.21.15.5:0/1524179733 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f8ca00bb0d0 con 0x565474da68b0 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.736+0000 7f8cb6e77740 10 client.15198 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.736+0000 7f8cb6e77740 10 client.15198 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.736+0000 7f8cb6e77740 20 client.15198 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.736+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x565474dc58d0 0x565474dc7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.736+0000 7f8cb6e77740 4 client.15198 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.736+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1524179733 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565474d1a600 con 0x565474dc58d0 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.736+0000 7f8ca7fff640 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x565474dc58d0 0x565474dc7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.736+0000 7f8ca7fff640 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x565474dc58d0 0x565474dc7a40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e91d7fd27af36b9a server_cookie=78c83765de3af99f in_seq=0 out_seq=0 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.736+0000 7f8c95ffb640 10 client.15198 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8c95ffb640 1 -- 172.21.15.5:0/1524179733 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565474d1a600 con 0x565474dc58d0 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8c95ffb640 10 client.15198 handle_command_reply: tid=0 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8cb6e77740 1 client.15198 shutdown 2024-09-15T22:08:46.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1524179733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 msgr2=0x565474db58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474db58f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b470 tx=0x7f8ca0009390 comp rx=0 tx=0).stop 2024-09-15T22:08:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1524179733 shutdown_connections 2024-09-15T22:08:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x565474dc58d0 0x565474dc7a40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474db3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da71d0 0x565474db2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/1524179733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474db58f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1524179733 >> 172.21.15.5:0/1524179733 conn(0x565474da2f60 msgr2=0x565474da8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8c95ffb640 0 client.15198 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:08:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1524179733 shutdown_connections 2024-09-15T22:08:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.737+0000 7f8cb6e77740 1 -- 172.21.15.5:0/1524179733 wait complete. 2024-09-15T22:08:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.738+0000 7f8cb6e77740 20 client.15198 trim_cache size 0 max 16384 2024-09-15T22:08:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.755+0000 7f8cb6e77740 1 Processor -- start 2024-09-15T22:08:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.755+0000 7f8cb6e77740 1 -- start start 2024-09-15T22:08:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.755+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da68b0 0x565474dc6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.755+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474da71d0 0x565474dc7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.755+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474db5fc0 0x565474dc5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.755+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565474dc5d80 con 0x565474db5fc0 2024-09-15T22:08:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.755+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565474dc6450 con 0x565474da71d0 2024-09-15T22:08:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.755+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565474dc6b20 con 0x565474da68b0 2024-09-15T22:08:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8caed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474db5fc0 0x565474dc5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8caed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474db5fc0 0x565474dc5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52118/0 (socket says 172.21.15.5:52118) 2024-09-15T22:08:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8caed9d640 1 -- 172.21.15.5:0/3132272648 learned_addr learned my addr 172.21.15.5:0/3132272648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8ca7fff640 1 --2- 172.21.15.5:0/3132272648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474da71d0 0x565474dc7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8cae59c640 1 --2- 172.21.15.5:0/3132272648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da68b0 0x565474dc6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8caed9d640 1 -- 172.21.15.5:0/3132272648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da68b0 msgr2=0x565474dc6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8caed9d640 1 --2- 172.21.15.5:0/3132272648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da68b0 0x565474dc6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8caed9d640 1 -- 172.21.15.5:0/3132272648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474da71d0 msgr2=0x565474dc7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8caed9d640 1 --2- 172.21.15.5:0/3132272648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474da71d0 0x565474dc7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8caed9d640 1 -- 172.21.15.5:0/3132272648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565474db5c60 con 0x565474db5fc0 2024-09-15T22:08:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8cae59c640 1 --2- 172.21.15.5:0/3132272648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da68b0 0x565474dc6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:08:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8ca7fff640 1 --2- 172.21.15.5:0/3132272648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474da71d0 0x565474dc7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8caed9d640 1 --2- 172.21.15.5:0/3132272648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474db5fc0 0x565474dc5f00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005fd0 tx=0x7f8ca801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183483d45ea43368 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8c97fff640 1 -- 172.21.15.5:0/3132272648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8009920 con 0x565474db5fc0 2024-09-15T22:08:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8c97fff640 1 -- 172.21.15.5:0/3132272648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca8020400 con 0x565474db5fc0 2024-09-15T22:08:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.756+0000 7f8c97fff640 1 -- 172.21.15.5:0/3132272648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca802a650 con 0x565474db5fc0 2024-09-15T22:08:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.757+0000 7f8cb6e77740 1 -- 172.21.15.5:0/3132272648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474db5fc0 msgr2=0x565474dc5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.757+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/3132272648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474db5fc0 0x565474dc5f00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8005fd0 tx=0x7f8ca801f000 comp rx=0 tx=0).stop 2024-09-15T22:08:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.757+0000 7f8cb6e77740 1 -- 172.21.15.5:0/3132272648 shutdown_connections 2024-09-15T22:08:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.757+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/3132272648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474db5fc0 0x565474dc5f00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.757+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/3132272648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474da71d0 0x565474dc7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.757+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/3132272648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da68b0 0x565474dc6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.757+0000 7f8cb6e77740 1 -- 172.21.15.5:0/3132272648 >> 172.21.15.5:0/3132272648 conn(0x565474da2f60 msgr2=0x565474d9dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.757+0000 7f8cb6e77740 1 -- 172.21.15.5:0/3132272648 shutdown_connections 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.757+0000 7f8cb6e77740 1 -- 172.21.15.5:0/3132272648 wait complete. 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.757+0000 7f8cb6e77740 1 -- start start 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.757+0000 7f8cb6e77740 1 Processor -- start 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474dc8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da71d0 0x565474dc9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cb6e77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474e2ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565474dc78c0 con 0x565474da68b0 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x565474dc71f0 con 0x565474db5fc0 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cb6e77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x565474dc7f90 con 0x565474da71d0 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cb6e77740 10 client.0 _ll_register_callbacks cb 0x7f8cb006b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474dc8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cae59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474dc8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52130/0 (socket says 172.21.15.5:52130) 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cae59c640 1 -- 172.21.15.5:0/2277728682 learned_addr learned my addr 172.21.15.5:0/2277728682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8ca7fff640 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da71d0 0x565474dc9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8caed9d640 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474e2ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cae59c640 1 -- 172.21.15.5:0/2277728682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da71d0 msgr2=0x565474dc9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cae59c640 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da71d0 0x565474dc9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cae59c640 1 -- 172.21.15.5:0/2277728682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 msgr2=0x565474e2ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cae59c640 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474e2ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cae59c640 1 -- 172.21.15.5:0/2277728682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565474dc9ad0 con 0x565474da68b0 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8caed9d640 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474e2ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cae59c640 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474dc8d00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8ca009add0 tx=0x7f8ca0017360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80aa72906a10669 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8c95ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8c95ffb640 1 -- 172.21.15.5:0/2277728682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00b9060 con 0x565474da68b0 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8c95ffb640 1 -- 172.21.15.5:0/2277728682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f8ca0010ca0 con 0x565474da68b0 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.758+0000 7f8cb6e77740 1 -- 172.21.15.5:0/2277728682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ca8020400 con 0x565474da68b0 2024-09-15T22:08:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8c95ffb640 1 -- 172.21.15.5:0/2277728682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f8ca00b4040 con 0x565474da68b0 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8cb6e77740 10 client.15204 fetch_fsmap learned FSMap version 24 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8cb6e77740 1 -- 172.21.15.5:0/2277728682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x565474e30250 con 0x565474da68b0 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8c95ffb640 1 -- 172.21.15.5:0/2277728682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ca8020400 con 0x565474da68b0 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8c95ffb640 1 -- 172.21.15.5:0/2277728682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f8ca00bb0b0 con 0x565474da68b0 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8cb6e77740 10 client.15204 fetch_fsmap finished waiting for FSMap version 24 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8cb6e77740 10 client.15204 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8cb6e77740 20 client.15204 populate_metadata read hostname 'smithi005' 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x565474e34b50 0x565474e33860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8cb6e77740 4 client.15204 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8cb6e77740 1 -- 172.21.15.5:0/2277728682 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "status"}) -- 0x565474d14b10 con 0x565474e34b50 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.759+0000 7f8ca7fff640 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x565474e34b50 0x565474e33860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8ca7fff640 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x565474e34b50 0x565474e33860 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1004c08b8fe75ce8 server_cookie=12a19dce96789df4 in_seq=0 out_seq=0 2024-09-15T22:08:46.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8c95ffb640 10 client.15204 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8c95ffb640 1 -- 172.21.15.5:0/2277728682 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x565474d14b10 con 0x565474e34b50 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8c95ffb640 10 client.15204 handle_command_reply: tid=0 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 1 client.15204 shutdown 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 1 -- 172.21.15.5:0/2277728682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 msgr2=0x565474dc8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474dc8d00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8ca009add0 tx=0x7f8ca0017360 comp rx=0 tx=0).stop 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 1 -- 172.21.15.5:0/2277728682 shutdown_connections 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x565474e34b50 0x565474e33860 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x565474db5fc0 0x565474e2ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x565474da71d0 0x565474dc9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 1 --2- 172.21.15.5:0/2277728682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565474da68b0 0x565474dc8d00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 1 -- 172.21.15.5:0/2277728682 >> 172.21.15.5:0/2277728682 conn(0x565474da2f60 msgr2=0x565474dae240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8c95ffb640 0 client.15204 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:08:46.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 1 -- 172.21.15.5:0/2277728682 shutdown_connections 2024-09-15T22:08:46.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 1 -- 172.21.15.5:0/2277728682 wait complete. 2024-09-15T22:08:46.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.760+0000 7f8cb6e77740 20 client.15204 trim_cache size 0 max 16384 2024-09-15T22:08:46.770 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:08:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T22:08:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-15T22:08:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24491, 2024-09-15T22:08:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-15T22:08:46.770 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-15T22:08:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-15T22:08:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 128.57156210599999, 2024-09-15T22:08:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 24, 2024-09-15T22:08:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 88, 2024-09-15T22:08:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 86, 2024-09-15T22:08:46.771 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 139.70728227699999 2024-09-15T22:08:46.771 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/211112882 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8c9c0a3240 msgr2=0x7f8c9c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8c9c0a3240 0x7f8c9c0a5700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80098b0 tx=0x7f8ca8009e00 comp rx=0 tx=0).stop 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/211112882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb0102d40 msgr2=0x7f8cb0198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb0102d40 0x7f8cb0198490 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f8ca000b4a0 tx=0x7f8ca000b960 comp rx=0 tx=0).stop 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/211112882 shutdown_connections 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8cb010c170 0x7f8cb019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8cb0103660 0x7f8cb01989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f8c9c0a3240 0x7f8c9c0a5700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 --2- 172.21.15.5:0/211112882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cb0102d40 0x7f8cb0198490 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/211112882 >> 172.21.15.5:0/211112882 conn(0x7f8cb00fc7a0 msgr2=0x7f8cb00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/211112882 shutdown_connections 2024-09-15T22:08:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:46.761+0000 7f8cb4a9e640 1 -- 172.21.15.5:0/211112882 wait complete. 2024-09-15T22:08:47.525 INFO:teuthology.run_tasks:Running task sequential... 2024-09-15T22:08:47.534 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-15T22:08:47.534 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-15T22:08:47.535 INFO:teuthology.task.print:Enabling mgr modules 2024-09-15T22:08:47.535 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-15T22:08:47.536 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:08:47.536 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-15T22:08:47.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-15T22:08:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.642+0000 7fd8c968d640 1 -- 172.21.15.5:0/3909140968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c41025c0 msgr2=0x7fd8c41049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.642+0000 7fd8c968d640 1 --2- 172.21.15.5:0/3909140968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c41025c0 0x7fd8c41049b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009ca0 tx=0x7fd8b402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:08:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.642+0000 7fd8c968d640 1 -- 172.21.15.5:0/3909140968 shutdown_connections 2024-09-15T22:08:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.642+0000 7fd8c968d640 1 --2- 172.21.15.5:0/3909140968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c4107820 0x7fd8c4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.642+0000 7fd8c968d640 1 --2- 172.21.15.5:0/3909140968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c4104ef0 0x7fd8c41072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.642+0000 7fd8c968d640 1 --2- 172.21.15.5:0/3909140968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c41025c0 0x7fd8c41049b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.642+0000 7fd8c968d640 1 -- 172.21.15.5:0/3909140968 >> 172.21.15.5:0/3909140968 conn(0x7fd8c40fc1c0 msgr2=0x7fd8c40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.642+0000 7fd8c968d640 1 -- 172.21.15.5:0/3909140968 shutdown_connections 2024-09-15T22:08:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.642+0000 7fd8c968d640 1 -- 172.21.15.5:0/3909140968 wait complete. 2024-09-15T22:08:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.642+0000 7fd8c968d640 1 Processor -- start 2024-09-15T22:08:47.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c968d640 1 -- start start 2024-09-15T22:08:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c968d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c41025c0 0x7fd8c419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c968d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c4104ef0 0x7fd8c419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c968d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c4107820 0x7fd8c41a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c968d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c410d830 con 0x7fd8c4104ef0 2024-09-15T22:08:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c968d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c410d6b0 con 0x7fd8c41025c0 2024-09-15T22:08:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c968d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c410d9b0 con 0x7fd8c4107820 2024-09-15T22:08:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c4104ef0 0x7fd8c419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c4104ef0 0x7fd8c419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52144/0 (socket says 172.21.15.5:52144) 2024-09-15T22:08:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c27fc640 1 -- 172.21.15.5:0/3407200775 learned_addr learned my addr 172.21.15.5:0/3407200775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c2ffd640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c41025c0 0x7fd8c419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c37fe640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c4107820 0x7fd8c41a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c27fc640 1 -- 172.21.15.5:0/3407200775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c4107820 msgr2=0x7fd8c41a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c27fc640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c4107820 0x7fd8c41a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c27fc640 1 -- 172.21.15.5:0/3407200775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c41025c0 msgr2=0x7fd8c419a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c27fc640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c41025c0 0x7fd8c419a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.643+0000 7fd8c27fc640 1 -- 172.21.15.5:0/3407200775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c41a1d10 con 0x7fd8c4104ef0 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.644+0000 7fd8c27fc640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c4104ef0 0x7fd8c419a630 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800c970 tx=0x7fd8b800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.644+0000 7fd8abfff640 1 -- 172.21.15.5:0/3407200775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8007bb0 con 0x7fd8c4104ef0 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.644+0000 7fd8c968d640 1 -- 172.21.15.5:0/3407200775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c41a1fa0 con 0x7fd8c4104ef0 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.644+0000 7fd8abfff640 1 -- 172.21.15.5:0/3407200775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8b8007d50 con 0x7fd8c4104ef0 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.644+0000 7fd8abfff640 1 -- 172.21.15.5:0/3407200775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b800f450 con 0x7fd8c4104ef0 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.644+0000 7fd8c968d640 1 -- 172.21.15.5:0/3407200775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c41a24c0 con 0x7fd8c4104ef0 2024-09-15T22:08:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.645+0000 7fd8c968d640 1 -- 172.21.15.5:0/3407200775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c4109390 con 0x7fd8c4104ef0 2024-09-15T22:08:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.646+0000 7fd8abfff640 1 -- 172.21.15.5:0/3407200775 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 138354+0+0 (secure 0 0 0) 0x7fd8b8004f70 con 0x7fd8c4104ef0 2024-09-15T22:08:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.647+0000 7fd8abfff640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd8ac0a3240 0x7fd8ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.647+0000 7fd8abfff640 1 -- 172.21.15.5:0/3407200775 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7fd8b80cff00 con 0x7fd8c4104ef0 2024-09-15T22:08:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.647+0000 7fd8c2ffd640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd8ac0a3240 0x7fd8ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.647+0000 7fd8c2ffd640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd8ac0a3240 0x7fd8ac0a5700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4000f80 tx=0x7fd8b4009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.650+0000 7fd8abfff640 1 -- 172.21.15.5:0/3407200775 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd8b8014030 con 0x7fd8c4104ef0 2024-09-15T22:08:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:47.880+0000 7fd8c968d640 1 -- 172.21.15.5:0/3407200775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fd8c402da10 con 0x7fd8c4104ef0 2024-09-15T22:08:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.267+0000 7fd8abfff640 1 -- 172.21.15.5:0/3407200775 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fd8b809c800 con 0x7fd8c4104ef0 2024-09-15T22:08:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 -- 172.21.15.5:0/3407200775 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd8ac0a3240 msgr2=0x7fd8ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd8ac0a3240 0x7fd8ac0a5700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4000f80 tx=0x7fd8b4009360 comp rx=0 tx=0).stop 2024-09-15T22:08:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 -- 172.21.15.5:0/3407200775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c4104ef0 msgr2=0x7fd8c419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c4104ef0 0x7fd8c419a630 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800c970 tx=0x7fd8b800ce30 comp rx=0 tx=0).stop 2024-09-15T22:08:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 -- 172.21.15.5:0/3407200775 shutdown_connections 2024-09-15T22:08:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd8c4107820 0x7fd8c41a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7fd8ac0a3240 0x7fd8ac0a5700 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8c4104ef0 0x7fd8c419a630 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 --2- 172.21.15.5:0/3407200775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd8c41025c0 0x7fd8c419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 -- 172.21.15.5:0/3407200775 >> 172.21.15.5:0/3407200775 conn(0x7fd8c40fc1c0 msgr2=0x7fd8c4101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 -- 172.21.15.5:0/3407200775 shutdown_connections 2024-09-15T22:08:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:48.270+0000 7fd8c968d640 1 -- 172.21.15.5:0/3407200775 wait complete. 2024-09-15T22:08:48.291 INFO:teuthology.run_tasks:Running task kclient... 2024-09-15T22:08:48.305 INFO:tasks.kclient:Mounting kernel clients... 2024-09-15T22:08:48.305 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-15T22:08:48.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:08:48.306 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-15T22:08:48.325 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-15T22:08:48.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:08:48.326 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-15T22:08:48.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:08:48.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-09-15T22:08:48.497 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync', 'context=system_u:object_r:postgresql_db_t:s0'], 'syntax': 'v1', 'mount_subvol_num': 1, 'mountpoint': '/tmp/cephfs'} 2024-09-15T22:08:48.497 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T22:08:48.497 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2024-09-15T22:08:48.497 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2024-09-15T22:08:48.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-15T22:08:48.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:48 smithi099 ceph-mon[34892]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-15T22:08:48.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:48 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3407200775' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-15T22:08:48.590 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: ignoring --setuser ceph since I am not root 2024-09-15T22:08:48.590 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: ignoring --setgroup ceph since I am not root 2024-09-15T22:08:48.590 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:48 smithi099 ceph-mgr[36265]: -- 172.21.15.99:0/4061803268 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649db7170e0 con 0x5649db6f4800 2024-09-15T22:08:48.590 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:48.420+0000 7fb5445b0040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:08:48.590 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:48.477+0000 7fb5445b0040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:08:48.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:48 smithi129 ceph-mon[35314]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-15T22:08:48.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:48 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3407200775' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-15T22:08:48.595 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: ignoring --setuser ceph since I am not root 2024-09-15T22:08:48.595 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: ignoring --setgroup ceph since I am not root 2024-09-15T22:08:48.595 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:48 smithi129 ceph-mgr[37098]: -- 172.21.15.129:0/127673655 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e87a8d70e0 con 0x55e87a8b4800 2024-09-15T22:08:48.595 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:48.424+0000 7f6b1d7ef040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:08:48.595 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:48.481+0000 7f6b1d7ef040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:08:48.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:48 smithi005 ceph-mon[30944]: pgmap v350: 129 pgs: 129 active+clean; 642 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-15T22:08:48.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:48 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3407200775' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-15T22:08:48.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: ignoring --setuser ceph since I am not root 2024-09-15T22:08:48.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: ignoring --setgroup ceph since I am not root 2024-09-15T22:08:48.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:48.434+0000 7f77a4057040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T22:08:48.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:48.495+0000 7f77a4057040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T22:08:48.771 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:48.898 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:48.657+0000 7f77a4057040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:08:48.898 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:48.732+0000 7f77a4057040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:08:48.976 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:48.623+0000 7fb5445b0040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:08:48.976 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:48.688+0000 7fb5445b0040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:08:48.976 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:48.833+0000 7fb5445b0040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:08:48.985 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:48.628+0000 7f6b1d7ef040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T22:08:48.985 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:48.694+0000 7f6b1d7ef040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T22:08:48.985 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:48.841+0000 7f6b1d7ef040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:08:48.985 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:48.982+0000 7f6b1d7ef040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:08:49.190 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:48.904+0000 7f77a4057040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:08:49.191 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:49 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:49.055+0000 7f77a4057040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:08:49.284 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:48.974+0000 7fb5445b0040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T22:08:49.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.272+0000 7f4e8b118640 1 -- 172.21.15.5:0/1652405488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84102ef0 msgr2=0x7f4e8410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:49.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.272+0000 7f4e8b118640 1 --2- 172.21.15.5:0/1652405488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84102ef0 0x7f4e8410da40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009ca0 tx=0x7f4e7c02a950 comp rx=0 tx=0).stop 2024-09-15T22:08:49.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.272+0000 7f4e8b118640 1 -- 172.21.15.5:0/1652405488 shutdown_connections 2024-09-15T22:08:49.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.272+0000 7f4e8b118640 1 --2- 172.21.15.5:0/1652405488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84102ef0 0x7f4e8410da40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.272+0000 7f4e8b118640 1 --2- 172.21.15.5:0/1652405488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e84102570 0x7f4e841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.272+0000 7f4e8b118640 1 --2- 172.21.15.5:0/1652405488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84100e20 0x7f4e84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.272+0000 7f4e8b118640 1 -- 172.21.15.5:0/1652405488 >> 172.21.15.5:0/1652405488 conn(0x7f4e840fc7d0 msgr2=0x7f4e840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.272+0000 7f4e8b118640 1 -- 172.21.15.5:0/1652405488 shutdown_connections 2024-09-15T22:08:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.272+0000 7f4e8b118640 1 -- 172.21.15.5:0/1652405488 wait complete. 2024-09-15T22:08:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.272+0000 7f4e8b118640 1 Processor -- start 2024-09-15T22:08:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e8b118640 1 -- start start 2024-09-15T22:08:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e8b118640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84100e20 0x7f4e840fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e8b118640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84102570 0x7f4e840fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e8b118640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e84102ef0 0x7f4e840fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:49.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e8b118640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84110f80 con 0x7f4e84100e20 2024-09-15T22:08:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e8b118640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84110e00 con 0x7f4e84102570 2024-09-15T22:08:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e8b118640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84111100 con 0x7f4e84102ef0 2024-09-15T22:08:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e88eb4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84100e20 0x7f4e840fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e88eb4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84100e20 0x7f4e840fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52184/0 (socket says 172.21.15.5:52184) 2024-09-15T22:08:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e88eb4640 1 -- 172.21.15.5:0/1242548219 learned_addr learned my addr 172.21.15.5:0/1242548219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e83fff640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84102570 0x7f4e840fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e896b5640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e84102ef0 0x7f4e840fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e88eb4640 1 -- 172.21.15.5:0/1242548219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e84102ef0 msgr2=0x7f4e840fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e88eb4640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e84102ef0 0x7f4e840fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e88eb4640 1 -- 172.21.15.5:0/1242548219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84102570 msgr2=0x7f4e840fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:49.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e88eb4640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84102570 0x7f4e840fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e88eb4640 1 -- 172.21.15.5:0/1242548219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e841a6970 con 0x7f4e84100e20 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e83fff640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84102570 0x7f4e840fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.273+0000 7f4e88eb4640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84100e20 0x7f4e840fcc80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800e9f0 tx=0x7f4e7800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.274+0000 7f4e81ffb640 1 -- 172.21.15.5:0/1242548219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7800cd80 con 0x7f4e84100e20 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.274+0000 7f4e81ffb640 1 -- 172.21.15.5:0/1242548219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e78010910 con 0x7f4e84100e20 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.274+0000 7f4e81ffb640 1 -- 172.21.15.5:0/1242548219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78018990 con 0x7f4e84100e20 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.274+0000 7f4e8b118640 1 -- 172.21.15.5:0/1242548219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e841a6c60 con 0x7f4e84100e20 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.274+0000 7f4e8b118640 1 -- 172.21.15.5:0/1242548219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e841a71a0 con 0x7f4e84100e20 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.275+0000 7f4e8b118640 1 -- 172.21.15.5:0/1242548219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e84101200 con 0x7f4e84100e20 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.277+0000 7f4e81ffb640 1 -- 172.21.15.5:0/1242548219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 138371+0+0 (secure 0 0 0) 0x7f4e78010ac0 con 0x7f4e84100e20 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.278+0000 7f4e81ffb640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4e740a3290 0x7f4e740a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.278+0000 7f4e83fff640 1 -- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4e740a3290 msgr2=0x7f4e740a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/801026306 2024-09-15T22:08:49.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.278+0000 7f4e81ffb640 1 -- 172.21.15.5:0/1242548219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f4e780d0080 con 0x7f4e84100e20 2024-09-15T22:08:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.278+0000 7f4e83fff640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4e740a3290 0x7f4e740a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:08:49.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.282+0000 7f4e81ffb640 1 -- 172.21.15.5:0/1242548219 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e7809c330 con 0x7f4e84100e20 2024-09-15T22:08:49.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:49 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3407200775' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-15T22:08:49.461 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:49 smithi005 ceph-mon[30944]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:08:49.461 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:49 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:49.394+0000 7f77a4057040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:08:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.479+0000 7f4e83fff640 1 -- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4e740a3290 msgr2=0x7f4e740a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/801026306 2024-09-15T22:08:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.479+0000 7f4e83fff640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4e740a3290 0x7f4e740a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:08:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.499+0000 7f4e8b118640 1 -- 172.21.15.5:0/1242548219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4e8402d7c0 con 0x7f4e84100e20 2024-09-15T22:08:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.500+0000 7f4e81ffb640 1 -- 172.21.15.5:0/1242548219 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f4e780a0630 con 0x7f4e84100e20 2024-09-15T22:08:49.502 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-15T22:08:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.502+0000 7f4e8b118640 1 -- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4e740a3290 msgr2=0x7f4e740a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:08:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.502+0000 7f4e8b118640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4e740a3290 0x7f4e740a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.502+0000 7f4e8b118640 1 -- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84100e20 msgr2=0x7f4e840fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.502+0000 7f4e8b118640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84100e20 0x7f4e840fcc80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4e7800e9f0 tx=0x7f4e7800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:08:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.502+0000 7f4e8b118640 1 -- 172.21.15.5:0/1242548219 shutdown_connections 2024-09-15T22:08:49.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.502+0000 7f4e8b118640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4e84102ef0 0x7f4e840fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:49.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.502+0000 7f4e8b118640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4e84102570 0x7f4e840fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:49.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.502+0000 7f4e8b118640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f4e740a3290 0x7f4e740a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:49.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.503+0000 7f4e8b118640 1 --2- 172.21.15.5:0/1242548219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84100e20 0x7f4e840fcc80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:49.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.503+0000 7f4e8b118640 1 -- 172.21.15.5:0/1242548219 >> 172.21.15.5:0/1242548219 conn(0x7f4e840fc7d0 msgr2=0x7f4e8406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:49.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.503+0000 7f4e8b118640 1 -- 172.21.15.5:0/1242548219 shutdown_connections 2024-09-15T22:08:49.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:49.503+0000 7f4e8b118640 1 -- 172.21.15.5:0/1242548219 wait complete. 2024-09-15T22:08:49.543 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:49 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3407200775' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-15T22:08:49.543 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:49 smithi129 ceph-mon[35314]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:08:49.543 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:49 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:49.295+0000 7f6b1d7ef040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:08:49.543 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:49 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:49.356+0000 7f6b1d7ef040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:08:49.543 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:49 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:49.540+0000 7f6b1d7ef040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:08:49.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:49 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3407200775' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-15T22:08:49.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:49 smithi099 ceph-mon[34892]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-15T22:08:49.590 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:49 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:49.282+0000 7fb5445b0040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T22:08:49.590 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:49 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:49.342+0000 7fb5445b0040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:08:49.590 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:49 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:49.524+0000 7fb5445b0040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:08:49.856 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:49 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:49.458+0000 7f77a4057040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T22:08:49.856 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:49 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:49.657+0000 7f77a4057040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T22:08:50.209 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-15T22:08:50.209 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T22:08:50.209 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-09-15T22:08:50.209 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-15T22:08:50.209 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2024-09-15T22:08:50.209 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-15T22:08:50.209 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T22:08:50.210 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T22:08:50.210 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2024-09-15T22:08:50.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:08:50.210 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-09-15T22:08:50.228 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T22:08:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T22:08:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T22:08:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:08:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-09-15T22:08:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42190sec preferred_lft 42190sec 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-15T22:08:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-15T22:08:50.230 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T22:08:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-09-15T22:08:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-09-15T22:08:50.230 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-15T22:08:50.230 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:08:50.230 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-15T22:08:50.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-09-15T22:08:50.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-09-15T22:08:50.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-09-15T22:08:50.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-15T22:08:50.230 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-15T22:08:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:08:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:08:50.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:08:50.374 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T22:08:50.399 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-09-15T22:08:50.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:08:50.401 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-09-15T22:08:50.458 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-09-15T22:08:50.458 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T22:08:50.459 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-15T22:08:50.459 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-15T22:08:50.459 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T22:08:50.459 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:08:50.459 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-15T22:08:50.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-15T22:08:50.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-15T22:08:50.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-15T22:08:50.459 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-15T22:08:50.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:08:50.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:50 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1242548219' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:08:50.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:08:50.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:50 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1242548219' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:08:50.589 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:50 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:50.403+0000 7fb5445b0040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:08:50.589 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:50 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:50.470+0000 7fb5445b0040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:08:50.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:08:50.592 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-15T22:08:50.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:50 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1242548219' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-15T22:08:50.595 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:50 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:50.422+0000 7f6b1d7ef040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:08:50.595 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:50 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:50.490+0000 7f6b1d7ef040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:08:50.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:08:50.608 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-09-15T22:08:50.665 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:08:50.665 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-15T22:08:50.665 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--tmp-cephfs 2024-09-15T22:08:50.665 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--tmp-cephfs 0 2024-09-15T22:08:50.665 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-15T22:08:50.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:08:50.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:08:50.786 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2024-09-15T22:08:50.787 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:08:50.787 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-15T22:08:50.787 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2024-09-15T22:08:50.787 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-15T22:08:50.787 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2024-09-15T22:08:50.787 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2024-09-15T22:08:50.787 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2024-09-15T22:08:50.787 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-15T22:08:50.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:08:50.845 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:50 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:50.612+0000 7f77a4057040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T22:08:50.845 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:50 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:50.684+0000 7f77a4057040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T22:08:50.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:08:50.907 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T22:08:50.907 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-15T22:08:50.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-09-15T22:08:50.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-09-15T22:08:50.908 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-15T22:08:50.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:08:50.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:08:50.990 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:08:50.990 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /tmp/cephfs 2024-09-15T22:08:51.017 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/tmp/cephfs' 2024-09-15T22:08:51.017 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T22:08:51.018 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /tmp/cephfs 2024-09-15T22:08:51.075 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-15T22:08:51.075 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-15T22:08:51.076 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/b72084f4-a7c4-4312-ae50-6ba93e80f47a 2024-09-15T22:08:51.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:08:51.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/b72084f4-a7c4-4312-ae50-6ba93e80f47a /tmp/cephfs -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context=system_u:object_r:postgresql_db_t:s0 2024-09-15T22:08:51.162 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-15T22:08:51.162 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-15T22:08:51.162 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-09-15T22:08:51.312 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0" 2024-09-15T22:08:51.312 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0"". 2024-09-15T22:08:51.312 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-09-15T22:08:51.312 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.99:3300,172.21.15.129:3300" 2024-09-15T22:08:51.312 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:3300,172.21.15.99:3300,172.21.15.129:3300:/volumes/qa/sv_1/b72084f4-a7c4-4312-ae50-6ba93e80f47a 2024-09-15T22:08:51.313 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,context="system_u:object_r:postgresql_db_t:s0",key=0,fsid=962891f2-73ad-11ef-bceb-c7b262605968" will pass to kernel 2024-09-15T22:08:51.315 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-15T22:08:51.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:08:51.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /tmp/cephfs 2024-09-15T22:08:51.345 DEBUG:teuthology.orchestra.run.smithi005:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:08:51.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:08:51.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:08:51.442 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:51 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:51.179+0000 7fb5445b0040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:08:51.442 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:51 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:51.295+0000 7fb5445b0040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:08:51.444 INFO:teuthology.orchestra.run.smithi005.stdout:client24871 2024-09-15T22:08:51.444 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T22:08:51.445 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:08:51.462 DEBUG:teuthology.orchestra.run.smithi005:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:08:51.466 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:51.201+0000 7f6b1d7ef040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:08:51.466 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:51.319+0000 7f6b1d7ef040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:08:51.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:08:51.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:08:51.561 INFO:teuthology.orchestra.run.smithi005.stdout:client24871 2024-09-15T22:08:51.562 DEBUG:teuthology.orchestra.run.smithi005:> (cd /tmp/cephfs && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T22:08:51.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T22:08:51.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:08:51.619 INFO:teuthology.orchestra.run.smithi005.stdout:client24871 2024-09-15T22:08:51.620 DEBUG:teuthology.orchestra.run.smithi005:> (cd /tmp/cephfs && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/962891f2-73ad-11ef-bceb-c7b262605968.client24871/status') 2024-09-15T22:08:51.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T22:08:51.680 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.24871 (3)172.21.15.5:0/50450237 2024-09-15T22:08:51.680 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-09-15T22:08:51.680 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-09-15T22:08:51.681 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-09-15T22:08:51.681 INFO:teuthology.orchestra.run.smithi005.stderr:68 bytes copied, 2.5141e-05 s, 2.7 MB/s 2024-09-15T22:08:51.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T22:08:51.688 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T22:08:51.697 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:08:51.697 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-15T22:08:51.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-15T22:08:51.720 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:51 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:51.441+0000 7fb5445b0040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:08:51.720 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:51 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:51.502+0000 7fb5445b0040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:08:51.721 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:51 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:51.651+0000 7fb5445b0040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:08:51.721 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:51 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:51.718+0000 7fb5445b0040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:08:51.721 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:51.467+0000 7f77a4057040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T22:08:51.721 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:51.593+0000 7f77a4057040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T22:08:51.746 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:51.464+0000 7f6b1d7ef040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:08:51.746 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:51.525+0000 7f6b1d7ef040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:08:51.746 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:51.674+0000 7f6b1d7ef040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:08:51.746 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:51.743+0000 7f6b1d7ef040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:08:51.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.802+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/1347045767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf8102630 msgr2=0x7f6cf8104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.802+0000 7f6cfd6d0640 1 --2- 172.21.15.5:0/1347045767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf8102630 0x7f6cf8104a20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009900 tx=0x7f6ce802a410 comp rx=0 tx=0).stop 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.802+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/1347045767 shutdown_connections 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.802+0000 7f6cfd6d0640 1 --2- 172.21.15.5:0/1347045767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf8107890 0x7f6cf8109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.802+0000 7f6cfd6d0640 1 --2- 172.21.15.5:0/1347045767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6cf8104f60 0x7f6cf8107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.802+0000 7f6cfd6d0640 1 --2- 172.21.15.5:0/1347045767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf8102630 0x7f6cf8104a20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.802+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/1347045767 >> 172.21.15.5:0/1347045767 conn(0x7f6cf80fc1b0 msgr2=0x7f6cf80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.802+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/1347045767 shutdown_connections 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.802+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/1347045767 wait complete. 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cfd6d0640 1 Processor -- start 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cfd6d0640 1 -- start start 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cfd6d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf8102630 0x7f6cf8101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:51.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cfd6d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6cf8104f60 0x7f6cf81005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cfd6d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf8107890 0x7f6cf8100b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cfd6d0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf810d820 con 0x7f6cf8107890 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cfd6d0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf810d6a0 con 0x7f6cf8102630 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cfd6d0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf810d9a0 con 0x7f6cf8104f60 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cf6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf8102630 0x7f6cf8101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cf77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf8107890 0x7f6cf8100b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cf6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf8102630 0x7f6cf8101fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:50412/0 (socket says 172.21.15.5:50412) 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cf77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf8107890 0x7f6cf8100b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52202/0 (socket says 172.21.15.5:52202) 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cf6ffd640 1 -- 172.21.15.5:0/2688346149 learned_addr learned my addr 172.21.15.5:0/2688346149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6ceffff640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6cf8104f60 0x7f6cf81005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cf6ffd640 1 -- 172.21.15.5:0/2688346149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6cf8104f60 msgr2=0x7f6cf81005e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cf6ffd640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6cf8104f60 0x7f6cf81005e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:51.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cf6ffd640 1 -- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf8107890 msgr2=0x7f6cf8100b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cf6ffd640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf8107890 0x7f6cf8100b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cf6ffd640 1 -- 172.21.15.5:0/2688346149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf81013b0 con 0x7f6cf8102630 2024-09-15T22:08:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.803+0000 7f6cf77fe640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf8107890 0x7f6cf8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:08:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.804+0000 7f6cf6ffd640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf8102630 0x7f6cf8101fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8005940 tx=0x7f6ce80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.804+0000 7f6cf4ff9640 1 -- 172.21.15.5:0/2688346149 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce803f030 con 0x7f6cf8102630 2024-09-15T22:08:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.804+0000 7f6cf4ff9640 1 -- 172.21.15.5:0/2688346149 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ce8003ce0 con 0x7f6cf8102630 2024-09-15T22:08:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.804+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/2688346149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf81a1fd0 con 0x7f6cf8102630 2024-09-15T22:08:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.804+0000 7f6cf4ff9640 1 -- 172.21.15.5:0/2688346149 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8035070 con 0x7f6cf8102630 2024-09-15T22:08:51.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.804+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/2688346149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf81a23d0 con 0x7f6cf8102630 2024-09-15T22:08:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.805+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/2688346149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf80fdca0 con 0x7f6cf8102630 2024-09-15T22:08:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.806+0000 7f6cf4ff9640 1 -- 172.21.15.5:0/2688346149 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 19) ==== 138371+0+0 (secure 0 0 0) 0x7f6ce8042030 con 0x7f6cf8102630 2024-09-15T22:08:51.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.807+0000 7f6cf4ff9640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ce40a3290 0x7f6ce40a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.807+0000 7f6ceffff640 1 -- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ce40a3290 msgr2=0x7f6ce40a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/801026306 2024-09-15T22:08:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.807+0000 7f6cf4ff9640 1 -- 172.21.15.5:0/2688346149 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f6ce80f03a0 con 0x7f6cf8102630 2024-09-15T22:08:51.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.807+0000 7f6ceffff640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ce40a3290 0x7f6ce40a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:08:51.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:51.810+0000 7f6cf4ff9640 1 -- 172.21.15.5:0/2688346149 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6ce80bc650 con 0x7f6cf8102630 2024-09-15T22:08:51.979 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:51.750+0000 7f77a4057040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T22:08:51.979 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:51.817+0000 7f77a4057040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T22:08:51.979 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:51.976+0000 7f77a4057040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T22:08:52.003 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:51 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:51.779+0000 7fb5445b0040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:08:52.003 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:51 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:51.937+0000 7fb5445b0040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:08:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.008+0000 7f6ceffff640 1 -- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ce40a3290 msgr2=0x7f6ce40a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/801026306 2024-09-15T22:08:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.008+0000 7f6ceffff640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ce40a3290 0x7f6ce40a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:08:52.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.026+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/2688346149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f6cf802d7c0 con 0x7f6cf8102630 2024-09-15T22:08:52.030 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:51.803+0000 7f6b1d7ef040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:08:52.031 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:51.962+0000 7f6b1d7ef040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:08:52.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.081+0000 7f6cf4ff9640 1 -- 172.21.15.5:0/2688346149 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f6ce80c0950 con 0x7f6cf8102630 2024-09-15T22:08:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.083+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ce40a3290 msgr2=0x7f6ce40a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:08:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.083+0000 7f6cfd6d0640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ce40a3290 0x7f6ce40a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.083+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/2688346149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf8102630 msgr2=0x7f6cf8101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.083+0000 7f6cfd6d0640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf8102630 0x7f6cf8101fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8005940 tx=0x7f6ce80042f0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.083+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/2688346149 shutdown_connections 2024-09-15T22:08:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.083+0000 7f6cfd6d0640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f6ce40a3290 0x7f6ce40a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.083+0000 7f6cfd6d0640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cf8107890 0x7f6cf8100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.083+0000 7f6cfd6d0640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6cf8104f60 0x7f6cf81005e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.083+0000 7f6cfd6d0640 1 --2- 172.21.15.5:0/2688346149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6cf8102630 0x7f6cf8101fa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.083+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/2688346149 >> 172.21.15.5:0/2688346149 conn(0x7f6cf80fc1b0 msgr2=0x7f6cf8103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.083+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/2688346149 shutdown_connections 2024-09-15T22:08:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.084+0000 7f6cfd6d0640 1 -- 172.21.15.5:0/2688346149 wait complete. 2024-09-15T22:08:52.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-15T22:08:52.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.243+0000 7f331262e640 1 -- 172.21.15.5:0/173337830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330c1025f0 msgr2=0x7f330c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:52.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.243+0000 7f331262e640 1 --2- 172.21.15.5:0/173337830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330c1025f0 0x7f330c108e80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3304009ca0 tx=0x7f330402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.243+0000 7f331262e640 1 -- 172.21.15.5:0/173337830 shutdown_connections 2024-09-15T22:08:52.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.243+0000 7f331262e640 1 --2- 172.21.15.5:0/173337830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330c1025f0 0x7f330c108e80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.243+0000 7f331262e640 1 --2- 172.21.15.5:0/173337830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f330c101c30 0x7f330c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.243+0000 7f331262e640 1 --2- 172.21.15.5:0/173337830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f330c100a40 0x7f330c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.243+0000 7f331262e640 1 -- 172.21.15.5:0/173337830 >> 172.21.15.5:0/173337830 conn(0x7f330c0fc1c0 msgr2=0x7f330c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.243+0000 7f331262e640 1 -- 172.21.15.5:0/173337830 shutdown_connections 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.243+0000 7f331262e640 1 -- 172.21.15.5:0/173337830 wait complete. 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.243+0000 7f331262e640 1 Processor -- start 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f331262e640 1 -- start start 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f331262e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f330c100a40 0x7f330c1049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f331262e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330c101c30 0x7f330c103010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f331262e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f330c1025f0 0x7f330c103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f331262e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f330c10c6f0 con 0x7f330c101c30 2024-09-15T22:08:52.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f331262e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f330c10c570 con 0x7f330c1025f0 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f331262e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f330c10c870 con 0x7f330c100a40 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f3303fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330c101c30 0x7f330c103010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f3303fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330c101c30 0x7f330c103010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52216/0 (socket says 172.21.15.5:52216) 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f3303fff640 1 -- 172.21.15.5:0/756141804 learned_addr learned my addr 172.21.15.5:0/756141804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f3310bcb640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f330c1025f0 0x7f330c103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f330bfff640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f330c100a40 0x7f330c1049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f3303fff640 1 -- 172.21.15.5:0/756141804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f330c100a40 msgr2=0x7f330c1049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f3303fff640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f330c100a40 0x7f330c1049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f3303fff640 1 -- 172.21.15.5:0/756141804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f330c1025f0 msgr2=0x7f330c103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f3303fff640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f330c1025f0 0x7f330c103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f3303fff640 1 -- 172.21.15.5:0/756141804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f330c103de0 con 0x7f330c101c30 2024-09-15T22:08:52.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.244+0000 7f3303fff640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330c101c30 0x7f330c103010 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f32f400ea60 tx=0x7f32f400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.245+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f400cde0 con 0x7f330c101c30 2024-09-15T22:08:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.245+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32f40035f0 con 0x7f330c101c30 2024-09-15T22:08:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.245+0000 7f331262e640 1 -- 172.21.15.5:0/756141804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f330c1a1f60 con 0x7f330c101c30 2024-09-15T22:08:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.245+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f4010670 con 0x7f330c101c30 2024-09-15T22:08:52.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.245+0000 7f331262e640 1 -- 172.21.15.5:0/756141804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f330c1a23d0 con 0x7f330c101c30 2024-09-15T22:08:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.246+0000 7f331262e640 1 -- 172.21.15.5:0/756141804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f330c1a26f0 con 0x7f330c101c30 2024-09-15T22:08:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.247+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 138371+0+0 (secure 0 0 0) 0x7f32f40037a0 con 0x7f330c101c30 2024-09-15T22:08:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.248+0000 7f3309ffb640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f32f80a3310 0x7f32f80a57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.248+0000 7f330bfff640 1 -- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f32f80a3310 msgr2=0x7f32f80a57d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/801026306 2024-09-15T22:08:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.248+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9195+0+0 (secure 0 0 0) 0x7f32f40d0030 con 0x7f330c101c30 2024-09-15T22:08:52.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.248+0000 7f330bfff640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f32f80a3310 0x7f32f80a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T22:08:52.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.252+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32f409ca20 con 0x7f330c101c30 2024-09-15T22:08:52.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:52 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2688346149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:08:52.272 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:52 smithi099 ceph-mon[34892]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:08:52.272 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:52.002+0000 7fb5445b0040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:08:52.272 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:52.068+0000 7fb5445b0040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:08:52.272 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:52.152+0000 7fb5445b0040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:08:52.272 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:52.210+0000 7fb5445b0040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:08:52.272 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:52.270+0000 7fb5445b0040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:08:52.289 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:52 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2688346149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:08:52.289 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:52 smithi005 ceph-mon[30944]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:08:52.289 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:52 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:52.049+0000 7f77a4057040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T22:08:52.289 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:52 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:52.114+0000 7f77a4057040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T22:08:52.289 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:52 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:52.287+0000 7f77a4057040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T22:08:52.301 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:52 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2688346149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:08:52.301 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:52 smithi129 ceph-mon[35314]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-15T22:08:52.301 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:52 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:52.028+0000 7f6b1d7ef040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:08:52.301 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:52 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:52.097+0000 7f6b1d7ef040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:08:52.301 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:52 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:52.179+0000 7f6b1d7ef040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:08:52.301 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:52 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:52.238+0000 7f6b1d7ef040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:08:52.301 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:52 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:52.298+0000 7f6b1d7ef040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:08:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.449+0000 7f330bfff640 1 -- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f32f80a3310 msgr2=0x7f32f80a57d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/801026306 2024-09-15T22:08:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.449+0000 7f330bfff640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f32f80a3310 0x7f32f80a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T22:08:52.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.456+0000 7f331262e640 1 -- 172.21.15.5:0/756141804 --> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f330c0fe3b0 con 0x7f32f80a3310 2024-09-15T22:08:52.579 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:52 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:52.356+0000 7f77a4057040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T22:08:52.579 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:52 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:52.427+0000 7f77a4057040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T22:08:52.579 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:52 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:52.514+0000 7f77a4057040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T22:08:52.579 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:52 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:52.577+0000 7f77a4057040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T22:08:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.850+0000 7f330bfff640 1 -- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f32f80a3310 msgr2=0x7f32f80a57d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/801026306 2024-09-15T22:08:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:52.850+0000 7f330bfff640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f32f80a3310 0x7f32f80a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T22:08:52.856 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:52 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:52.639+0000 7f77a4057040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T22:08:53.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:53.095+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 20) ==== 138371+0+0 (secure 0 0 0) 0x7f32f4098990 con 0x7f330c101c30 2024-09-15T22:08:53.339 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:08:53 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y[36261]: 2024-09-15T22:08:53.061+0000 7fb5445b0040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:08:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:53 smithi099 ceph-mon[34892]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-15T22:08:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:53 smithi099 ceph-mon[34892]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:08:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:53 smithi099 ceph-mon[34892]: Standby manager daemon y restarted 2024-09-15T22:08:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:53 smithi099 ceph-mon[34892]: Standby manager daemon y started 2024-09-15T22:08:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:53 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:08:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:53 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:08:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:53 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:08:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:53 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:08:53.345 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:08:53 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z[37094]: 2024-09-15T22:08:53.087+0000 7f6b1d7ef040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:08:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:53 smithi129 ceph-mon[35314]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-15T22:08:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:53 smithi129 ceph-mon[35314]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:08:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:53 smithi129 ceph-mon[35314]: Standby manager daemon y restarted 2024-09-15T22:08:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:53 smithi129 ceph-mon[35314]: Standby manager daemon y started 2024-09-15T22:08:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:53 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:08:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:53 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:08:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:53 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:08:53.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:53 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:08:53.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-mon[30944]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-15T22:08:53.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-mon[30944]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-09-15T22:08:53.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-mon[30944]: Standby manager daemon y restarted 2024-09-15T22:08:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-mon[30944]: Standby manager daemon y started 2024-09-15T22:08:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-15T22:08:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:08:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-15T22:08:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.99:0/3598965845' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:08:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:53.494+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7f32f409c580 con 0x7f330c101c30 2024-09-15T22:08:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:53.494+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f32f80a3310 msgr2=0x7f32f80a57d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:08:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:53.494+0000 7f3309ffb640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/801026306,v1:172.21.15.5:6801/801026306] conn(0x7f32f80a3310 0x7f32f80a57d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:53.745 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:53.481+0000 7f77a4057040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T22:08:53.999 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:53.742+0000 7f770b669640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:53.999 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:53.742+0000 7f770b669640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:53.999 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:53.742+0000 7f770b669640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.000 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:53.742+0000 7f770b669640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.000 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:53.742+0000 7f770b669640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.000 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:53.743+0000 7f770de6e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.000 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:53.744+0000 7f770de6e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.000 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:53.744+0000 7f770de6e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.000 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:53.744+0000 7f770de6e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.000 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:53 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:53.744+0000 7f770de6e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.247 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:08:54.247 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-15T22:08:54.247 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:08:54.247 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: Standby manager daemon z restarted 2024-09-15T22:08:54.247 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: Standby manager daemon z started 2024-09-15T22:08:54.247 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: Active manager daemon x restarted 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: Activating manager daemon x 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: mgrmap e21: x(active, starting, since 0.011758s), standbys: y, z 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:08:54.248 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:08:54.249 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: Manager daemon x is now available 2024-09-15T22:08:54.250 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:54.251 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:08:54.251 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:54 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:08:54.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:08:54.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-15T22:08:54.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:08:54.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: Standby manager daemon z restarted 2024-09-15T22:08:54.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: Standby manager daemon z started 2024-09-15T22:08:54.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: Active manager daemon x restarted 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: Activating manager daemon x 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: mgrmap e21: x(active, starting, since 0.011758s), standbys: y, z 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:08:54.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:08:54.342 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: Manager daemon x is now available 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:08:54.343 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:54 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:08:54.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-15T22:08:54.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-09-15T22:08:54.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-15T22:08:54.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: Standby manager daemon z restarted 2024-09-15T22:08:54.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: Standby manager daemon z started 2024-09-15T22:08:54.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-15T22:08:54.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.? 172.21.15.129:0/1526607193' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-15T22:08:54.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: Active manager daemon x restarted 2024-09-15T22:08:54.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: Activating manager daemon x 2024-09-15T22:08:54.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: osdmap e89: 12 total, 12 up, 12 in 2024-09-15T22:08:54.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: mgrmap e21: x(active, starting, since 0.011758s), standbys: y, z 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:08:54.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:08:54.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:08:54.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:08:54.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-15T22:08:54.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-15T22:08:54.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-15T22:08:54.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-15T22:08:54.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-15T22:08:54.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-15T22:08:54.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: Manager daemon x is now available 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:54.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-15T22:08:54.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-15T22:08:54.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.498+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 <== mon.0 v2:172.21.15.5:3300/0 9 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f32f40cd7c0 con 0x7f330c101c30 2024-09-15T22:08:54.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.499+0000 7f3309ffb640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f32f8032160 0x7f32f8032540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:54.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.499+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f32f40cd300 con 0x7f32f8032160 2024-09-15T22:08:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.501+0000 7f330bfff640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f32f8032160 0x7f32f8032540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:54.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.505+0000 7f330bfff640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f32f8032160 0x7f32f8032540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f330c101a90 tx=0x7f32fc005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.524+0000 7f3309ffb640 1 -- 172.21.15.5:0/756141804 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f32f40cd300 con 0x7f32f8032160 2024-09-15T22:08:54.526 INFO:teuthology.orchestra.run.smithi005.stdout:{} 2024-09-15T22:08:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.526+0000 7f331262e640 1 -- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f32f8032160 msgr2=0x7f32f8032540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.526+0000 7f331262e640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f32f8032160 0x7f32f8032540 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f330c101a90 tx=0x7f32fc005f90 comp rx=0 tx=0).stop 2024-09-15T22:08:54.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.526+0000 7f331262e640 1 -- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330c101c30 msgr2=0x7f330c103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.526+0000 7f331262e640 1 --2- 172.21.15.5:0/756141804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f330c101c30 0x7f330c103010 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f32f400ea60 tx=0x7f32f400ef20 comp rx=0 tx=0).stop 2024-09-15T22:08:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.526+0000 7f3310bcb640 1 -- 172.21.15.5:0/756141804 reap_dead start 2024-09-15T22:08:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.526+0000 7f331262e640 1 -- 172.21.15.5:0/756141804 shutdown_connections 2024-09-15T22:08:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.526+0000 7f331262e640 1 -- 172.21.15.5:0/756141804 >> 172.21.15.5:0/756141804 conn(0x7f330c0fc1c0 msgr2=0x7f330c0fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.526+0000 7f331262e640 1 -- 172.21.15.5:0/756141804 shutdown_connections 2024-09-15T22:08:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.526+0000 7f331262e640 1 -- 172.21.15.5:0/756141804 wait complete. 2024-09-15T22:08:54.539 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-15T22:08:54.550 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-15T22:08:54.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T22:08:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.690+0000 7f44121ed640 1 -- 172.21.15.5:0/3899044952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c107890 msgr2=0x7f440c109ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.690+0000 7f44121ed640 1 --2- 172.21.15.5:0/3899044952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c107890 0x7f440c109ca0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4404009ca0 tx=0x7f440402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:08:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.690+0000 7f44121ed640 1 -- 172.21.15.5:0/3899044952 shutdown_connections 2024-09-15T22:08:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.690+0000 7f44121ed640 1 --2- 172.21.15.5:0/3899044952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c107890 0x7f440c109ca0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.690+0000 7f44121ed640 1 --2- 172.21.15.5:0/3899044952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f440c104f60 0x7f440c107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.690+0000 7f44121ed640 1 --2- 172.21.15.5:0/3899044952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c102630 0x7f440c104a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:54.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.691+0000 7f44121ed640 1 -- 172.21.15.5:0/3899044952 >> 172.21.15.5:0/3899044952 conn(0x7f440c0fc1f0 msgr2=0x7f440c0fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.691+0000 7f44121ed640 1 -- 172.21.15.5:0/3899044952 shutdown_connections 2024-09-15T22:08:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.691+0000 7f44121ed640 1 -- 172.21.15.5:0/3899044952 wait complete. 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.691+0000 7f44121ed640 1 Processor -- start 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f44121ed640 1 -- start start 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f44121ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f440c102630 0x7f440c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f44121ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c104f60 0x7f440c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f44121ed640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c107890 0x7f440c19f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f44121ed640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f440c10d810 con 0x7f440c104f60 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f44121ed640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f440c10d690 con 0x7f440c107890 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f44121ed640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f440c10d990 con 0x7f440c102630 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f440affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c104f60 0x7f440c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f440b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f440c102630 0x7f440c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f440affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c104f60 0x7f440c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55912/0 (socket says 172.21.15.5:55912) 2024-09-15T22:08:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f440bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c107890 0x7f440c19f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f440affd640 1 -- 172.21.15.5:0/2403448047 learned_addr learned my addr 172.21.15.5:0/2403448047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f440b7fe640 1 -- 172.21.15.5:0/2403448047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c107890 msgr2=0x7f440c19f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f440b7fe640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c107890 0x7f440c19f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f440b7fe640 1 -- 172.21.15.5:0/2403448047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c104f60 msgr2=0x7f440c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f440b7fe640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c104f60 0x7f440c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f440b7fe640 1 -- 172.21.15.5:0/2403448047 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440c19fac0 con 0x7f440c102630 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.692+0000 7f440affd640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c104f60 0x7f440c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.693+0000 7f440b7fe640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f440c102630 0x7f440c197ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00e9f0 tx=0x7f43fc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.693+0000 7f4408ff9640 1 -- 172.21.15.5:0/2403448047 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc00ccc0 con 0x7f440c102630 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.693+0000 7f44121ed640 1 -- 172.21.15.5:0/2403448047 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440c19fdb0 con 0x7f440c102630 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.693+0000 7f4408ff9640 1 -- 172.21.15.5:0/2403448047 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43fc010910 con 0x7f440c102630 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.693+0000 7f4408ff9640 1 -- 172.21.15.5:0/2403448047 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc018960 con 0x7f440c102630 2024-09-15T22:08:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.693+0000 7f44121ed640 1 -- 172.21.15.5:0/2403448047 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440c1a0220 con 0x7f440c102630 2024-09-15T22:08:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.694+0000 7f44121ed640 1 -- 172.21.15.5:0/2403448047 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f440c0fdce0 con 0x7f440c102630 2024-09-15T22:08:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.695+0000 7f4408ff9640 1 -- 172.21.15.5:0/2403448047 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f43fc010ac0 con 0x7f440c102630 2024-09-15T22:08:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.696+0000 7f4408ff9640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43f40a3050 0x7f43f40a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.696+0000 7f440affd640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43f40a3050 0x7f43f40a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.696+0000 7f4408ff9640 1 -- 172.21.15.5:0/2403448047 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f43fc0cffd0 con 0x7f440c102630 2024-09-15T22:08:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.697+0000 7f440affd640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43f40a3050 0x7f43f40a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f440c1993b0 tx=0x7f4400005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.699+0000 7f4408ff9640 1 -- 172.21.15.5:0/2403448047 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f43fc09c490 con 0x7f440c102630 2024-09-15T22:08:54.820 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:54.512+0000 7f7708663640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.820 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:54.512+0000 7f7708663640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.820 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:54.512+0000 7f7708663640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.820 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:54.512+0000 7f7708663640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.820 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:54 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:54.512+0000 7f7708663640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:08:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.904+0000 7f44121ed640 1 -- 172.21.15.5:0/2403448047 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f440c02da30 con 0x7f440c102630 2024-09-15T22:08:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.906+0000 7f4408ff9640 1 -- 172.21.15.5:0/2403448047 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+25126 (secure 0 0 0) 0x7f43fc0a0790 con 0x7f440c102630 2024-09-15T22:08:54.909 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:54.909 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:08:53.484177+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2453332511":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/878276657":"2024-09-16T22:08:53.484141+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1408571851":"2024-09-16T22:08:53.484141+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/801026306":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6800/801026306":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2083481614":"2024-09-16T22:08:53.484141+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:08:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 -- 172.21.15.5:0/2403448047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43f40a3050 msgr2=0x7f43f40a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43f40a3050 0x7f43f40a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f440c1993b0 tx=0x7f4400005f90 comp rx=0 tx=0).stop 2024-09-15T22:08:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 -- 172.21.15.5:0/2403448047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f440c102630 msgr2=0x7f440c197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f440c102630 0x7f440c197ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43fc00e9f0 tx=0x7f43fc00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:08:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 -- 172.21.15.5:0/2403448047 shutdown_connections 2024-09-15T22:08:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43f40a3050 0x7f43f40a5510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f440c107890 0x7f440c19f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f440c104f60 0x7f440c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 --2- 172.21.15.5:0/2403448047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f440c102630 0x7f440c197ea0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 -- 172.21.15.5:0/2403448047 >> 172.21.15.5:0/2403448047 conn(0x7f440c0fc1f0 msgr2=0x7f440c102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 -- 172.21.15.5:0/2403448047 shutdown_connections 2024-09-15T22:08:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:54.908+0000 7f44121ed640 1 -- 172.21.15.5:0/2403448047 wait complete. 2024-09-15T22:08:54.923 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T22:01:57.184697+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-15T22:06:33.616843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-15T22:06:36.161538+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-15T22:06:43.068085+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-15T22:08:54.923 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T22:08:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.028+0000 7fa6b62af640 1 -- 172.21.15.5:0/2973059575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b0107810 msgr2=0x7fa6b0109c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.028+0000 7fa6b62af640 1 --2- 172.21.15.5:0/2973059575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b0107810 0x7fa6b0109c80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009ca0 tx=0x7fa6a802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.028+0000 7fa6b62af640 1 -- 172.21.15.5:0/2973059575 shutdown_connections 2024-09-15T22:08:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.028+0000 7fa6b62af640 1 --2- 172.21.15.5:0/2973059575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b0107810 0x7fa6b0109c80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.028+0000 7fa6b62af640 1 --2- 172.21.15.5:0/2973059575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6b0104ee0 0x7fa6b01072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.028+0000 7fa6b62af640 1 --2- 172.21.15.5:0/2973059575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b01025b0 0x7fa6b01049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.028+0000 7fa6b62af640 1 -- 172.21.15.5:0/2973059575 >> 172.21.15.5:0/2973059575 conn(0x7fa6b00fc190 msgr2=0x7fa6b00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.028+0000 7fa6b62af640 1 -- 172.21.15.5:0/2973059575 shutdown_connections 2024-09-15T22:08:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.028+0000 7fa6b62af640 1 -- 172.21.15.5:0/2973059575 wait complete. 2024-09-15T22:08:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.028+0000 7fa6b62af640 1 Processor -- start 2024-09-15T22:08:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.028+0000 7fa6b62af640 1 -- start start 2024-09-15T22:08:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b62af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b01025b0 0x7fa6b0197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b62af640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6b0104ee0 0x7fa6b0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b62af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b0107810 0x7fa6b019f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b62af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b010d7f0 con 0x7fa6b0107810 2024-09-15T22:08:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b62af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b010d670 con 0x7fa6b01025b0 2024-09-15T22:08:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b62af640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b010d970 con 0x7fa6b0104ee0 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b484c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b0107810 0x7fa6b019f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b484c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b0107810 0x7fa6b019f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55926/0 (socket says 172.21.15.5:55926) 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b484c640 1 -- 172.21.15.5:0/3209261933 learned_addr learned my addr 172.21.15.5:0/3209261933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6affff640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b01025b0 0x7fa6b0197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6af7fe640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6b0104ee0 0x7fa6b0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b484c640 1 -- 172.21.15.5:0/3209261933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6b0104ee0 msgr2=0x7fa6b0198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b484c640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6b0104ee0 0x7fa6b0198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b484c640 1 -- 172.21.15.5:0/3209261933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b01025b0 msgr2=0x7fa6b0197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b484c640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b01025b0 0x7fa6b0197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b484c640 1 -- 172.21.15.5:0/3209261933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b019fa70 con 0x7fa6b0107810 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6affff640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b01025b0 0x7fa6b0197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:08:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.029+0000 7fa6b484c640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b0107810 0x7fa6b019f400 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8031a40 tx=0x7fa6a8031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.030+0000 7fa6ad7fa640 1 -- 172.21.15.5:0/3209261933 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8003b70 con 0x7fa6b0107810 2024-09-15T22:08:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.030+0000 7fa6b62af640 1 -- 172.21.15.5:0/3209261933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b019fd00 con 0x7fa6b0107810 2024-09-15T22:08:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.030+0000 7fa6ad7fa640 1 -- 172.21.15.5:0/3209261933 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6a8003d10 con 0x7fa6b0107810 2024-09-15T22:08:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.030+0000 7fa6ad7fa640 1 -- 172.21.15.5:0/3209261933 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a802fca0 con 0x7fa6b0107810 2024-09-15T22:08:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.030+0000 7fa6b62af640 1 -- 172.21.15.5:0/3209261933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b01a01f0 con 0x7fa6b0107810 2024-09-15T22:08:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.031+0000 7fa6b62af640 1 -- 172.21.15.5:0/3209261933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b00fdc80 con 0x7fa6b0107810 2024-09-15T22:08:55.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.032+0000 7fa6ad7fa640 1 -- 172.21.15.5:0/3209261933 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fa6a8004050 con 0x7fa6b0107810 2024-09-15T22:08:55.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.032+0000 7fa6ad7fa640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6980a3050 0x7fa6980a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.033+0000 7fa6affff640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6980a3050 0x7fa6980a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.033+0000 7fa6ad7fa640 1 -- 172.21.15.5:0/3209261933 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa6a80f0ba0 con 0x7fa6b0107810 2024-09-15T22:08:55.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.033+0000 7fa6affff640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6980a3050 0x7fa6980a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000afb0 tx=0x7fa6a000a2d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.037+0000 7fa6ad7fa640 1 -- 172.21.15.5:0/3209261933 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6a80bd810 con 0x7fa6b0107810 2024-09-15T22:08:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.264+0000 7fa6b62af640 1 -- 172.21.15.5:0/3209261933 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa6b0062e90 con 0x7fa6b0107810 2024-09-15T22:08:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.265+0000 7fa6ad7fa640 1 -- 172.21.15.5:0/3209261933 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fa6a80bd110 con 0x7fa6b0107810 2024-09-15T22:08:55.267 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-15T22:08:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 -- 172.21.15.5:0/3209261933 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6980a3050 msgr2=0x7fa6980a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6980a3050 0x7fa6980a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa6a000afb0 tx=0x7fa6a000a2d0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 -- 172.21.15.5:0/3209261933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b0107810 msgr2=0x7fa6b019f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b0107810 0x7fa6b019f400 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8031a40 tx=0x7fa6a8031ee0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 -- 172.21.15.5:0/3209261933 shutdown_connections 2024-09-15T22:08:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6980a3050 0x7fa6980a5510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b0107810 0x7fa6b019f400 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa6b0104ee0 0x7fa6b0198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 --2- 172.21.15.5:0/3209261933 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa6b01025b0 0x7fa6b0197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 -- 172.21.15.5:0/3209261933 >> 172.21.15.5:0/3209261933 conn(0x7fa6b00fc190 msgr2=0x7fa6b0102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 -- 172.21.15.5:0/3209261933 shutdown_connections 2024-09-15T22:08:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.267+0000 7fa6b62af640 1 -- 172.21.15.5:0/3209261933 wait complete. 2024-09-15T22:08:55.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-15T22:08:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.396+0000 7f8f5b265640 1 -- 172.21.15.5:0/1881522197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f54101c20 msgr2=0x7f8f541020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.396+0000 7f8f5b265640 1 --2- 172.21.15.5:0/1881522197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f54101c20 0x7f8f541020a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009900 tx=0x7f8f4402a410 comp rx=0 tx=0).stop 2024-09-15T22:08:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.396+0000 7f8f5b265640 1 -- 172.21.15.5:0/1881522197 shutdown_connections 2024-09-15T22:08:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.396+0000 7f8f5b265640 1 --2- 172.21.15.5:0/1881522197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f541025e0 0x7f8f54108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.396+0000 7f8f5b265640 1 --2- 172.21.15.5:0/1881522197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f54101c20 0x7f8f541020a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.396+0000 7f8f5b265640 1 --2- 172.21.15.5:0/1881522197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f54100a30 0x7f8f54100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.396+0000 7f8f5b265640 1 -- 172.21.15.5:0/1881522197 >> 172.21.15.5:0/1881522197 conn(0x7f8f540fc190 msgr2=0x7f8f540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.396+0000 7f8f5b265640 1 -- 172.21.15.5:0/1881522197 shutdown_connections 2024-09-15T22:08:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.396+0000 7f8f5b265640 1 -- 172.21.15.5:0/1881522197 wait complete. 2024-09-15T22:08:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.397+0000 7f8f5b265640 1 Processor -- start 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.397+0000 7f8f5b265640 1 -- start start 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f5b265640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f54100a30 0x7f8f5406a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f5b265640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f54101c20 0x7f8f5419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f5b265640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f541025e0 0x7f8f5419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f5b265640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5410c700 con 0x7f8f54101c20 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f5b265640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5410c580 con 0x7f8f541025e0 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f5b265640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f5410c880 con 0x7f8f54100a30 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f58800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f54101c20 0x7f8f5419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f58800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f54101c20 0x7f8f5419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55944/0 (socket says 172.21.15.5:55944) 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f58800640 1 -- 172.21.15.5:0/2894077498 learned_addr learned my addr 172.21.15.5:0/2894077498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f59802640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f541025e0 0x7f8f5419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f59001640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f54100a30 0x7f8f5406a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f58800640 1 -- 172.21.15.5:0/2894077498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f54100a30 msgr2=0x7f8f5406a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f58800640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f54100a30 0x7f8f5406a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f58800640 1 -- 172.21.15.5:0/2894077498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f541025e0 msgr2=0x7f8f5419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f58800640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f541025e0 0x7f8f5419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.398+0000 7f8f58800640 1 -- 172.21.15.5:0/2894077498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f5419fb30 con 0x7f8f54101c20 2024-09-15T22:08:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.399+0000 7f8f58800640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f54101c20 0x7f8f5419cc00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f44031440 tx=0x7f8f44032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.399+0000 7f8f4a7fc640 1 -- 172.21.15.5:0/2894077498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f4403f030 con 0x7f8f54101c20 2024-09-15T22:08:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.399+0000 7f8f4a7fc640 1 -- 172.21.15.5:0/2894077498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f44003ce0 con 0x7f8f54101c20 2024-09-15T22:08:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.399+0000 7f8f5b265640 1 -- 172.21.15.5:0/2894077498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f5419fd60 con 0x7f8f54101c20 2024-09-15T22:08:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.399+0000 7f8f4a7fc640 1 -- 172.21.15.5:0/2894077498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f44035070 con 0x7f8f54101c20 2024-09-15T22:08:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.399+0000 7f8f5b265640 1 -- 172.21.15.5:0/2894077498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f541a01f0 con 0x7f8f54101c20 2024-09-15T22:08:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.401+0000 7f8f5b265640 1 -- 172.21.15.5:0/2894077498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f54107e60 con 0x7f8f54101c20 2024-09-15T22:08:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.402+0000 7f8f4a7fc640 1 -- 172.21.15.5:0/2894077498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f8f44042030 con 0x7f8f54101c20 2024-09-15T22:08:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.403+0000 7f8f4a7fc640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f400a3050 0x7f8f400a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.403+0000 7f8f4a7fc640 1 -- 172.21.15.5:0/2894077498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f8f440efba0 con 0x7f8f54101c20 2024-09-15T22:08:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.403+0000 7f8f59001640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f400a3050 0x7f8f400a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.403+0000 7f8f59001640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f400a3050 0x7f8f400a5510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8f54101a80 tx=0x7f8f50005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:55.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.407+0000 7f8f4a7fc640 1 -- 172.21.15.5:0/2894077498 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f440bc6f0 con 0x7f8f54101c20 2024-09-15T22:08:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.513+0000 7f8f4a7fc640 1 -- 172.21.15.5:0/2894077498 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 23) ==== 138525+0+0 (secure 0 0 0) 0x7f8f4403bcd0 con 0x7f8f54101c20 2024-09-15T22:08:55.539 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:55 smithi129 ceph-mon[35314]: mgrmap e22: x(active, since 1.0173s), standbys: y, z 2024-09-15T22:08:55.539 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:55 smithi129 ceph-mon[35314]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:55.540 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:55 smithi129 ceph-mon[35314]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:08:55.540 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:55 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2403448047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:08:55.540 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:55 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3209261933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:08:55.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:55 smithi005 ceph-mon[30944]: mgrmap e22: x(active, since 1.0173s), standbys: y, z 2024-09-15T22:08:55.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:55 smithi005 ceph-mon[30944]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:55.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:55 smithi005 ceph-mon[30944]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:08:55.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:55 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2403448047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:08:55.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:55 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3209261933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:08:55.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.623+0000 7f8f5b265640 1 -- 172.21.15.5:0/2894077498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8f54062df0 con 0x7f8f54101c20 2024-09-15T22:08:55.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.624+0000 7f8f4a7fc640 1 -- 172.21.15.5:0/2894077498 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f8f44077fa0 con 0x7f8f54101c20 2024-09-15T22:08:55.627 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-15T22:08:55.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 -- 172.21.15.5:0/2894077498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f400a3050 msgr2=0x7f8f400a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f400a3050 0x7f8f400a5510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8f54101a80 tx=0x7f8f50005f90 comp rx=0 tx=0).stop 2024-09-15T22:08:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 -- 172.21.15.5:0/2894077498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f54101c20 msgr2=0x7f8f5419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f54101c20 0x7f8f5419cc00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f44031440 tx=0x7f8f44032ce0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 -- 172.21.15.5:0/2894077498 shutdown_connections 2024-09-15T22:08:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f400a3050 0x7f8f400a5510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f541025e0 0x7f8f5419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f54101c20 0x7f8f5419cc00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 --2- 172.21.15.5:0/2894077498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f54100a30 0x7f8f5406a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 -- 172.21.15.5:0/2894077498 >> 172.21.15.5:0/2894077498 conn(0x7f8f540fc190 msgr2=0x7f8f54105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 -- 172.21.15.5:0/2894077498 shutdown_connections 2024-09-15T22:08:55.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.626+0000 7f8f5b265640 1 -- 172.21.15.5:0/2894077498 wait complete. 2024-09-15T22:08:55.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-15T22:08:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.752+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3732379622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d68101c20 msgr2=0x7f6d681020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.752+0000 7f6d6e7a6640 1 --2- 172.21.15.5:0/3732379622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d68101c20 0x7f6d681020a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009900 tx=0x7f6d5c02a410 comp rx=0 tx=0).stop 2024-09-15T22:08:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.752+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3732379622 shutdown_connections 2024-09-15T22:08:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.752+0000 7f6d6e7a6640 1 --2- 172.21.15.5:0/3732379622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d681025e0 0x7f6d68108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.752+0000 7f6d6e7a6640 1 --2- 172.21.15.5:0/3732379622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d68101c20 0x7f6d681020a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.752+0000 7f6d6e7a6640 1 --2- 172.21.15.5:0/3732379622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d68100a30 0x7f6d68100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.752+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3732379622 >> 172.21.15.5:0/3732379622 conn(0x7f6d680fc190 msgr2=0x7f6d680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.752+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3732379622 shutdown_connections 2024-09-15T22:08:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.752+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3732379622 wait complete. 2024-09-15T22:08:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6e7a6640 1 Processor -- start 2024-09-15T22:08:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6e7a6640 1 -- start start 2024-09-15T22:08:55.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6e7a6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d68100a30 0x7f6d68197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6e7a6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d68101c20 0x7f6d681983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6e7a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d681025e0 0x7f6d6819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6e7a6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6810d800 con 0x7f6d681025e0 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6e7a6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6810d680 con 0x7f6d68100a30 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6e7a6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6810d980 con 0x7f6d68101c20 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6cd43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d681025e0 0x7f6d6819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6cd43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d681025e0 0x7f6d6819f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55998/0 (socket says 172.21.15.5:55998) 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6cd43640 1 -- 172.21.15.5:0/3569419794 learned_addr learned my addr 172.21.15.5:0/3569419794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d677fe640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d68101c20 0x7f6d681983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d67fff640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d68100a30 0x7f6d68197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6cd43640 1 -- 172.21.15.5:0/3569419794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d68101c20 msgr2=0x7f6d681983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6cd43640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d68101c20 0x7f6d681983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6cd43640 1 -- 172.21.15.5:0/3569419794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d68100a30 msgr2=0x7f6d68197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6cd43640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d68100a30 0x7f6d68197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.753+0000 7f6d6cd43640 1 -- 172.21.15.5:0/3569419794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6819fac0 con 0x7f6d681025e0 2024-09-15T22:08:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.754+0000 7f6d6cd43640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d681025e0 0x7f6d6819f450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000eb10 tx=0x7f6d6000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.754+0000 7f6d657fa640 1 -- 172.21.15.5:0/3569419794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60004b20 con 0x7f6d681025e0 2024-09-15T22:08:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.754+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3569419794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6819fdb0 con 0x7f6d681025e0 2024-09-15T22:08:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.754+0000 7f6d657fa640 1 -- 172.21.15.5:0/3569419794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d60004cc0 con 0x7f6d681025e0 2024-09-15T22:08:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.754+0000 7f6d657fa640 1 -- 172.21.15.5:0/3569419794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d60010760 con 0x7f6d681025e0 2024-09-15T22:08:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.754+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3569419794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d681a0220 con 0x7f6d681025e0 2024-09-15T22:08:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.755+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3569419794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d68107e70 con 0x7f6d681025e0 2024-09-15T22:08:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.756+0000 7f6d657fa640 1 -- 172.21.15.5:0/3569419794 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138525+0+0 (secure 0 0 0) 0x7f6d60010900 con 0x7f6d681025e0 2024-09-15T22:08:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.757+0000 7f6d657fa640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6d500a3450 0x7f6d500a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.757+0000 7f6d67fff640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6d500a3450 0x7f6d500a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.757+0000 7f6d657fa640 1 -- 172.21.15.5:0/3569419794 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f6d600d09f0 con 0x7f6d681025e0 2024-09-15T22:08:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.758+0000 7f6d67fff640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6d500a3450 0x7f6d500a5910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6d68101a80 tx=0x7f6d58005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.760+0000 7f6d657fa640 1 -- 172.21.15.5:0/3569419794 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6d600541b0 con 0x7f6d681025e0 2024-09-15T22:08:55.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:55 smithi099 ceph-mon[34892]: mgrmap e22: x(active, since 1.0173s), standbys: y, z 2024-09-15T22:08:55.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:55 smithi099 ceph-mon[34892]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:08:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:55 smithi099 ceph-mon[34892]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:08:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:55 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2403448047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:08:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:55 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3209261933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:08:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.965+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3569419794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6d6802d950 con 0x7f6d681025e0 2024-09-15T22:08:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.966+0000 7f6d657fa640 1 -- 172.21.15.5:0/3569419794 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f6d6009cea0 con 0x7f6d681025e0 2024-09-15T22:08:55.968 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-09-15T22:08:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3569419794 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6d500a3450 msgr2=0x7f6d500a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6d500a3450 0x7f6d500a5910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6d68101a80 tx=0x7f6d58005f90 comp rx=0 tx=0).stop 2024-09-15T22:08:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3569419794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d681025e0 msgr2=0x7f6d6819f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d681025e0 0x7f6d6819f450 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000eb10 tx=0x7f6d6000c2d0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3569419794 shutdown_connections 2024-09-15T22:08:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6d500a3450 0x7f6d500a5910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d681025e0 0x7f6d6819f450 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6d68101c20 0x7f6d681983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 --2- 172.21.15.5:0/3569419794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6d68100a30 0x7f6d68197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3569419794 >> 172.21.15.5:0/3569419794 conn(0x7f6d680fc190 msgr2=0x7f6d68105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3569419794 shutdown_connections 2024-09-15T22:08:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:55.968+0000 7f6d6e7a6640 1 -- 172.21.15.5:0/3569419794 wait complete. 2024-09-15T22:08:55.982 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-15T22:08:56.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.089+0000 7fe05bf91640 1 -- 172.21.15.5:0/471397506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0541025b0 msgr2=0x7fe0541049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:56.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.089+0000 7fe05bf91640 1 --2- 172.21.15.5:0/471397506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0541025b0 0x7fe0541049a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe050009900 tx=0x7fe05002a410 comp rx=0 tx=0).stop 2024-09-15T22:08:56.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.089+0000 7fe05bf91640 1 -- 172.21.15.5:0/471397506 shutdown_connections 2024-09-15T22:08:56.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.089+0000 7fe05bf91640 1 --2- 172.21.15.5:0/471397506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054107810 0x7fe054109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:56.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.089+0000 7fe05bf91640 1 --2- 172.21.15.5:0/471397506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe054104ee0 0x7fe0541072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:56.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.089+0000 7fe05bf91640 1 --2- 172.21.15.5:0/471397506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0541025b0 0x7fe0541049a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:56.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.089+0000 7fe05bf91640 1 -- 172.21.15.5:0/471397506 >> 172.21.15.5:0/471397506 conn(0x7fe0540fc190 msgr2=0x7fe0540fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:56.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.089+0000 7fe05bf91640 1 -- 172.21.15.5:0/471397506 shutdown_connections 2024-09-15T22:08:56.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.089+0000 7fe05bf91640 1 -- 172.21.15.5:0/471397506 wait complete. 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe05bf91640 1 Processor -- start 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe05bf91640 1 -- start start 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe05bf91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0541025b0 0x7fe054197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe05bf91640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054104ee0 0x7fe054198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe05bf91640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe054107810 0x7fe05419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe05bf91640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe05410d870 con 0x7fe0541025b0 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe05bf91640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe05410d6f0 con 0x7fe054107810 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe05bf91640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe05410d9f0 con 0x7fe054104ee0 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe059d2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0541025b0 0x7fe054197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe059d2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0541025b0 0x7fe054197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56012/0 (socket says 172.21.15.5:56012) 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe059d2d640 1 -- 172.21.15.5:0/1342259963 learned_addr learned my addr 172.21.15.5:0/1342259963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe05952c640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054104ee0 0x7fe054198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.090+0000 7fe05a52e640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe054107810 0x7fe05419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe059d2d640 1 -- 172.21.15.5:0/1342259963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054104ee0 msgr2=0x7fe054198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe059d2d640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054104ee0 0x7fe054198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe059d2d640 1 -- 172.21.15.5:0/1342259963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe054107810 msgr2=0x7fe05419f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe059d2d640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe054107810 0x7fe05419f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe059d2d640 1 -- 172.21.15.5:0/1342259963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05419fb10 con 0x7fe0541025b0 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe05952c640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054104ee0 0x7fe054198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe05a52e640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe054107810 0x7fe05419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe059d2d640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0541025b0 0x7fe054197ef0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe050031440 tx=0x7fe050032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe04affd640 1 -- 172.21.15.5:0/1342259963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05003f030 con 0x7fe0541025b0 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe05bf91640 1 -- 172.21.15.5:0/1342259963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05419fda0 con 0x7fe0541025b0 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe04affd640 1 -- 172.21.15.5:0/1342259963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe050003ce0 con 0x7fe0541025b0 2024-09-15T22:08:56.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe04affd640 1 -- 172.21.15.5:0/1342259963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050035070 con 0x7fe0541025b0 2024-09-15T22:08:56.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.091+0000 7fe05bf91640 1 -- 172.21.15.5:0/1342259963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0541a01d0 con 0x7fe0541025b0 2024-09-15T22:08:56.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.092+0000 7fe05bf91640 1 -- 172.21.15.5:0/1342259963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0540fdc80 con 0x7fe0541025b0 2024-09-15T22:08:56.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.094+0000 7fe04affd640 1 -- 172.21.15.5:0/1342259963 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138525+0+0 (secure 0 0 0) 0x7fe050042030 con 0x7fe0541025b0 2024-09-15T22:08:56.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.095+0000 7fe04affd640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0400a3450 0x7fe0400a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:56.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.095+0000 7fe05952c640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0400a3450 0x7fe0400a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:56.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.095+0000 7fe04affd640 1 -- 172.21.15.5:0/1342259963 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe0500f0b70 con 0x7fe0541025b0 2024-09-15T22:08:56.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.096+0000 7fe05952c640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0400a3450 0x7fe0400a5910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe0440095f0 tx=0x7fe044005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:56.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.098+0000 7fe04affd640 1 -- 172.21.15.5:0/1342259963 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0500bc990 con 0x7fe0541025b0 2024-09-15T22:08:56.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.285+0000 7fe05bf91640 1 -- 172.21.15.5:0/1342259963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe054062e90 con 0x7fe0541025b0 2024-09-15T22:08:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.285+0000 7fe04affd640 1 -- 172.21.15.5:0/1342259963 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7fe0500c0c90 con 0x7fe0541025b0 2024-09-15T22:08:56.288 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-15T22:08:56.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 -- 172.21.15.5:0/1342259963 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0400a3450 msgr2=0x7fe0400a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:56.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0400a3450 0x7fe0400a5910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe0440095f0 tx=0x7fe044005f90 comp rx=0 tx=0).stop 2024-09-15T22:08:56.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 -- 172.21.15.5:0/1342259963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0541025b0 msgr2=0x7fe054197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:56.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0541025b0 0x7fe054197ef0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe050031440 tx=0x7fe050032ce0 comp rx=0 tx=0).stop 2024-09-15T22:08:56.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 -- 172.21.15.5:0/1342259963 shutdown_connections 2024-09-15T22:08:56.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0400a3450 0x7fe0400a5910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:56.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe054107810 0x7fe05419f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:56.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe054104ee0 0x7fe054198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:56.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 --2- 172.21.15.5:0/1342259963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0541025b0 0x7fe054197ef0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:56.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 -- 172.21.15.5:0/1342259963 >> 172.21.15.5:0/1342259963 conn(0x7fe0540fc190 msgr2=0x7fe054103010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:56.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 -- 172.21.15.5:0/1342259963 shutdown_connections 2024-09-15T22:08:56.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:56.288+0000 7fe05bf91640 1 -- 172.21.15.5:0/1342259963 wait complete. 2024-09-15T22:08:56.302 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-15T22:08:56.302 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:08:56.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:56.239+0000 7f76eeff1640 -1 log_channel(cephadm) log [ERR] : [15/Sep/2024:22:08:56] ENGINE Error in HTTPServer.serve 2024-09-15T22:08:56.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: Traceback (most recent call last): 2024-09-15T22:08:56.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:08:56.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self._connections.run(self.expiration_interval) 2024-09-15T22:08:56.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:08:56.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self._run(expiration_interval) 2024-09-15T22:08:56.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:08:56.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: s = self.context.wrap_socket( 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: return self.sslsocket_class._create( 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self.do_handshake() 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: self._sslobj.do_handshake() 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:08:56.508 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:08:56.597 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:56.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-15T22:08:56.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2894077498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3569419794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: [15/Sep/2024:22:08:56] ENGINE Bus STARTING 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-15T22:08:56.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: Unable to set osd_memory_target on smithi129 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: [15/Sep/2024:22:08:56] ENGINE Serving on https://172.21.15.5:7150 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: [15/Sep/2024:22:08:56] ENGINE Error in HTTPServer.serve 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:08:56.790 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1342259963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:56.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:56 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-15T22:08:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2894077498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:08:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3569419794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: [15/Sep/2024:22:08:56] ENGINE Bus STARTING 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-15T22:08:56.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: Unable to set osd_memory_target on smithi129 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: [15/Sep/2024:22:08:56] ENGINE Serving on https://172.21.15.5:7150 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: [15/Sep/2024:22:08:56] ENGINE Error in HTTPServer.serve 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout: Traceback (most recent call last): 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout: self._run(expiration_interval) 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:08:56.842 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout: s = self.context.wrap_socket( 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout: return self.sslsocket_class._create( 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout: self.do_handshake() 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout: self._sslobj.do_handshake() 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1342259963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.843 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.844 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.844 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.844 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.844 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.844 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:56.844 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:56 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-09-15T22:08:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2894077498' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:08:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3569419794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:08:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: [15/Sep/2024:22:08:56] ENGINE Bus STARTING 2024-09-15T22:08:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-15T22:08:56.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: Unable to set osd_memory_target on smithi129 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: [15/Sep/2024:22:08:56] ENGINE Serving on https://172.21.15.5:7150 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: [15/Sep/2024:22:08:56] ENGINE Error in HTTPServer.serve 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: Traceback (most recent call last): 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: self._connections.run(self.expiration_interval) 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: self._run(expiration_interval) 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: s = self.context.wrap_socket( 2024-09-15T22:08:56.848 INFO:journalctl@ceph.mon.c.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout: return self.sslsocket_class._create( 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout: self.do_handshake() 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout: self._sslobj.do_handshake() 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1342259963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.849 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:08:56.850 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:56.850 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:56 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.169+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/1959057579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d01082a0 msgr2=0x7fa7d010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.169+0000 7fa7d7e2a640 1 --2- 172.21.15.5:0/1959057579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d01082a0 0x7fa7d010ec40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009cd0 tx=0x7fa7c802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.170+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/1959057579 shutdown_connections 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.170+0000 7fa7d7e2a640 1 --2- 172.21.15.5:0/1959057579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d01082a0 0x7fa7d010ec40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.170+0000 7fa7d7e2a640 1 --2- 172.21.15.5:0/1959057579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7d01036e0 0x7fa7d0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.170+0000 7fa7d7e2a640 1 --2- 172.21.15.5:0/1959057579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d0102d30 0x7fa7d0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.170+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/1959057579 >> 172.21.15.5:0/1959057579 conn(0x7fa7d00fc7d0 msgr2=0x7fa7d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.170+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/1959057579 shutdown_connections 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.170+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/1959057579 wait complete. 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.170+0000 7fa7d7e2a640 1 Processor -- start 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.170+0000 7fa7d7e2a640 1 -- start start 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d7e2a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7d0102d30 0x7fa7d0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d7e2a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d01036e0 0x7fa7d0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d7e2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d01082a0 0x7fa7d0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d7e2a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d01120f0 con 0x7fa7d01082a0 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d7e2a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d0111f70 con 0x7fa7d01036e0 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d7e2a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d0112270 con 0x7fa7d0102d30 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d63c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d01082a0 0x7fa7d0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d5bc6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7d0102d30 0x7fa7d0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d63c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d01082a0 0x7fa7d0101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56038/0 (socket says 172.21.15.5:56038) 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d63c7640 1 -- 172.21.15.5:0/3581371617 learned_addr learned my addr 172.21.15.5:0/3581371617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d5bc6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7d0102d30 0x7fa7d0100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57174/0 (socket says 172.21.15.5:57174) 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d53c5640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d01036e0 0x7fa7d0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d63c7640 1 -- 172.21.15.5:0/3581371617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7d0102d30 msgr2=0x7fa7d0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d63c7640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7d0102d30 0x7fa7d0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d63c7640 1 -- 172.21.15.5:0/3581371617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d01036e0 msgr2=0x7fa7d0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d63c7640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d01036e0 0x7fa7d0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d63c7640 1 -- 172.21.15.5:0/3581371617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d01a6af0 con 0x7fa7d01082a0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d5bc6640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7d0102d30 0x7fa7d0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.171+0000 7fa7d63c7640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d01082a0 0x7fa7d0101410 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8033040 tx=0x7fa7c8031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.172+0000 7fa7c6ffd640 1 -- 172.21.15.5:0/3581371617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8004040 con 0x7fa7d01082a0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.172+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/3581371617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d01a6d80 con 0x7fa7d01082a0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.172+0000 7fa7c6ffd640 1 -- 172.21.15.5:0/3581371617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7c80046c0 con 0x7fa7d01082a0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.172+0000 7fa7c6ffd640 1 -- 172.21.15.5:0/3581371617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8046480 con 0x7fa7d01082a0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.172+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/3581371617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d01a7290 con 0x7fa7d01082a0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.173+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/3581371617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7d0103110 con 0x7fa7d01082a0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.174+0000 7fa7c6ffd640 1 -- 172.21.15.5:0/3581371617 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138525+0+0 (secure 0 0 0) 0x7fa7c80305a0 con 0x7fa7d01082a0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.175+0000 7fa7c6ffd640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7bc0a3440 0x7fa7bc0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.175+0000 7fa7d5bc6640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7bc0a3440 0x7fa7bc0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.175+0000 7fa7c6ffd640 1 -- 172.21.15.5:0/3581371617 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa7c802e030 con 0x7fa7d01082a0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.175+0000 7fa7d5bc6640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7bc0a3440 0x7fa7bc0a5900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc005d00 tx=0x7fa7cc00b410 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.179+0000 7fa7c6ffd640 1 -- 172.21.15.5:0/3581371617 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7c804b060 con 0x7fa7d01082a0 2024-09-15T22:08:57.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.392+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/3581371617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7d010a2b0 con 0x7fa7d01082a0 2024-09-15T22:08:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.393+0000 7fa7c6ffd640 1 -- 172.21.15.5:0/3581371617 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fa7c80bdd60 con 0x7fa7d01082a0 2024-09-15T22:08:57.396 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:57.396 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:08:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:08:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.395+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/3581371617 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7bc0a3440 msgr2=0x7fa7bc0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.395+0000 7fa7d7e2a640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7bc0a3440 0x7fa7bc0a5900 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc005d00 tx=0x7fa7cc00b410 comp rx=0 tx=0).stop 2024-09-15T22:08:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.395+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/3581371617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d01082a0 msgr2=0x7fa7d0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.395+0000 7fa7d7e2a640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d01082a0 0x7fa7d0101410 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8033040 tx=0x7fa7c8031e60 comp rx=0 tx=0).stop 2024-09-15T22:08:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.395+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/3581371617 shutdown_connections 2024-09-15T22:08:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.396+0000 7fa7d7e2a640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7bc0a3440 0x7fa7bc0a5900 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.396+0000 7fa7d7e2a640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7d01082a0 0x7fa7d0101410 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.396+0000 7fa7d7e2a640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7d01036e0 0x7fa7d0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.396+0000 7fa7d7e2a640 1 --2- 172.21.15.5:0/3581371617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7d0102d30 0x7fa7d0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:57.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.396+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/3581371617 >> 172.21.15.5:0/3581371617 conn(0x7fa7d00fc7d0 msgr2=0x7fa7d00fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:57.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.396+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/3581371617 shutdown_connections 2024-09-15T22:08:57.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:57.396+0000 7fa7d7e2a640 1 -- 172.21.15.5:0/3581371617 wait complete. 2024-09-15T22:08:57.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: [15/Sep/2024:22:08:56] ENGINE Serving on http://172.21.15.5:8765 2024-09-15T22:08:57.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: [15/Sep/2024:22:08:56] ENGINE Bus STARTED 2024-09-15T22:08:57.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-15T22:08:57.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Unable to set osd_memory_target on smithi005 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-15T22:08:57.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: Updating smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3581371617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:57.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:57 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: [15/Sep/2024:22:08:56] ENGINE Serving on http://172.21.15.5:8765 2024-09-15T22:08:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: [15/Sep/2024:22:08:56] ENGINE Bus STARTED 2024-09-15T22:08:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-15T22:08:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Unable to set osd_memory_target on smithi005 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-15T22:08:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-15T22:08:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-15T22:08:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-15T22:08:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:08:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:08:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: Updating smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3581371617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.842 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:57 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: [15/Sep/2024:22:08:56] ENGINE Serving on http://172.21.15.5:8765 2024-09-15T22:08:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: [15/Sep/2024:22:08:56] ENGINE Bus STARTED 2024-09-15T22:08:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-15T22:08:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Unable to set osd_memory_target on smithi005 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-15T22:08:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-15T22:08:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi099:/etc/ceph/ceph.conf 2024-09-15T22:08:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi129:/etc/ceph/ceph.conf 2024-09-15T22:08:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:08:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi099:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi129:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: Updating smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3581371617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:57.847 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:57 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:58.129 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-15T22:08:58.129 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-15T22:08:58.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T22:08:58.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.236+0000 7f408149b640 1 -- 172.21.15.5:0/696911425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f407c1009c0 msgr2=0x7f407c100e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.236+0000 7f408149b640 1 --2- 172.21.15.5:0/696911425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f407c1009c0 0x7f407c100e20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4070009930 tx=0x7f407002a420 comp rx=0 tx=0).stop 2024-09-15T22:08:58.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.236+0000 7f408149b640 1 -- 172.21.15.5:0/696911425 shutdown_connections 2024-09-15T22:08:58.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.236+0000 7f408149b640 1 --2- 172.21.15.5:0/696911425 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f407c101360 0x7f407c10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.236+0000 7f408149b640 1 --2- 172.21.15.5:0/696911425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f407c1009c0 0x7f407c100e20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.236+0000 7f408149b640 1 --2- 172.21.15.5:0/696911425 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c1069d0 0x7f407c106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.236+0000 7f408149b640 1 -- 172.21.15.5:0/696911425 >> 172.21.15.5:0/696911425 conn(0x7f407c077a10 msgr2=0x7f407c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:58.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.236+0000 7f408149b640 1 -- 172.21.15.5:0/696911425 shutdown_connections 2024-09-15T22:08:58.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.236+0000 7f408149b640 1 -- 172.21.15.5:0/696911425 wait complete. 2024-09-15T22:08:58.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.236+0000 7f408149b640 1 Processor -- start 2024-09-15T22:08:58.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.236+0000 7f408149b640 1 -- start start 2024-09-15T22:08:58.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f408149b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f407c1009c0 0x7f407c0fce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f408149b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f407c101360 0x7f407c0fd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f408149b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c1069d0 0x7f407c0fd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f408149b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f407c110e60 con 0x7f407c1009c0 2024-09-15T22:08:58.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f408149b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f407c110ce0 con 0x7f407c101360 2024-09-15T22:08:58.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f408149b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f407c110fe0 con 0x7f407c1069d0 2024-09-15T22:08:58.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f407affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f407c1009c0 0x7f407c0fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f407affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f407c1009c0 0x7f407c0fce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56062/0 (socket says 172.21.15.5:56062) 2024-09-15T22:08:58.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f407affd640 1 -- 172.21.15.5:0/1158111047 learned_addr learned my addr 172.21.15.5:0/1158111047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:58.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f407a7fc640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f407c101360 0x7f407c0fd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f407b7fe640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c1069d0 0x7f407c0fd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f407affd640 1 -- 172.21.15.5:0/1158111047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c1069d0 msgr2=0x7f407c0fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f407affd640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c1069d0 0x7f407c0fd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f407affd640 1 -- 172.21.15.5:0/1158111047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f407c101360 msgr2=0x7f407c0fd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f407affd640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f407c101360 0x7f407c0fd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f407affd640 1 -- 172.21.15.5:0/1158111047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407c1a6880 con 0x7f407c1009c0 2024-09-15T22:08:58.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f407affd640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f407c1009c0 0x7f407c0fce50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f406c00eab0 tx=0x7f406c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:58.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f4063fff640 1 -- 172.21.15.5:0/1158111047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c00ce40 con 0x7f407c1009c0 2024-09-15T22:08:58.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f4063fff640 1 -- 172.21.15.5:0/1158111047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f406c011950 con 0x7f407c1009c0 2024-09-15T22:08:58.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f408149b640 1 -- 172.21.15.5:0/1158111047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407c1a6b70 con 0x7f407c1009c0 2024-09-15T22:08:58.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f4063fff640 1 -- 172.21.15.5:0/1158111047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c010ae0 con 0x7f407c1009c0 2024-09-15T22:08:58.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.237+0000 7f408149b640 1 -- 172.21.15.5:0/1158111047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f407c1a7090 con 0x7f407c1009c0 2024-09-15T22:08:58.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.238+0000 7f408149b640 1 -- 172.21.15.5:0/1158111047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f407c100e20 con 0x7f407c1009c0 2024-09-15T22:08:58.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.240+0000 7f4063fff640 1 -- 172.21.15.5:0/1158111047 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) ==== 138525+0+0 (secure 0 0 0) 0x7f406c011b00 con 0x7f407c1009c0 2024-09-15T22:08:58.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.240+0000 7f4063fff640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f40640a3450 0x7f40640a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.240+0000 7f4063fff640 1 -- 172.21.15.5:0/1158111047 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f406c0d07a0 con 0x7f407c1009c0 2024-09-15T22:08:58.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.240+0000 7f407a7fc640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f40640a3450 0x7f40640a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.241+0000 7f407a7fc640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f40640a3450 0x7f40640a5910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4070006010 tx=0x7f4070000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:58.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.243+0000 7f4063fff640 1 -- 172.21.15.5:0/1158111047 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f406c016030 con 0x7f407c1009c0 2024-09-15T22:08:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.416+0000 7f408149b640 1 -- 172.21.15.5:0/1158111047 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f407c069d70 con 0x7f40640a3450 2024-09-15T22:08:58.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.425+0000 7f4063fff640 1 -- 172.21.15.5:0/1158111047 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339383 (secure 0 0 0) 0x7f407c069d70 con 0x7f40640a3450 2024-09-15T22:08:58.431 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:58.433 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-15T22:08:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.428+0000 7f408149b640 1 -- 172.21.15.5:0/1158111047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f40640a3450 msgr2=0x7f40640a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.428+0000 7f408149b640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f40640a3450 0x7f40640a5910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4070006010 tx=0x7f4070000bc0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.428+0000 7f408149b640 1 -- 172.21.15.5:0/1158111047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f407c1009c0 msgr2=0x7f407c0fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.428+0000 7f408149b640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f407c1009c0 0x7f407c0fce50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f406c00eab0 tx=0x7f406c00ef70 comp rx=0 tx=0).stop 2024-09-15T22:08:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.428+0000 7f408149b640 1 -- 172.21.15.5:0/1158111047 shutdown_connections 2024-09-15T22:08:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.428+0000 7f408149b640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f40640a3450 0x7f40640a5910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.428+0000 7f408149b640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f407c1069d0 0x7f407c0fd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.428+0000 7f408149b640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f407c101360 0x7f407c0fd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.428+0000 7f408149b640 1 --2- 172.21.15.5:0/1158111047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f407c1009c0 0x7f407c0fce50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.428+0000 7f408149b640 1 -- 172.21.15.5:0/1158111047 >> 172.21.15.5:0/1158111047 conn(0x7f407c077a10 msgr2=0x7f407c069660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:58.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.428+0000 7f408149b640 1 -- 172.21.15.5:0/1158111047 shutdown_connections 2024-09-15T22:08:58.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.429+0000 7f408149b640 1 -- 172.21.15.5:0/1158111047 wait complete. 2024-09-15T22:08:58.441 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-15T22:08:57.501792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189982,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":866,"num_read_kb":597,"num_write":485,"num_write_kb":4385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":393524,"kb_used_data":15124,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124548300,"statfs":{"total":1151940427776,"available":1151537459200,"internally_reserved":0,"allocated":15486976,"data_stored":9157225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":387233083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.989795"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489253+0000","last_change":"2024-09-15T22:07:46.226073+0000","last_active":"2024-09-15T22:08:53.489253+0000","last_peered":"2024-09-15T22:08:53.489253+0000","last_clean":"2024-09-15T22:08:53.489253+0000","last_became_active":"2024-09-15T22:07:46.219250+0000","last_became_peered":"2024-09-15T22:07:46.219250+0000","last_unstale":"2024-09-15T22:08:53.489253+0000","last_undegraded":"2024-09-15T22:08:53.489253+0000","last_fullsized":"2024-09-15T22:08:53.489253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:56.324959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,4],"acting":[9,11,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582824+0000","last_change":"2024-09-15T22:06:37.184096+0000","last_active":"2024-09-15T22:08:53.582824+0000","last_peered":"2024-09-15T22:08:53.582824+0000","last_clean":"2024-09-15T22:08:53.582824+0000","last_became_active":"2024-09-15T22:06:37.183758+0000","last_became_peered":"2024-09-15T22:06:37.183758+0000","last_unstale":"2024-09-15T22:08:53.582824+0000","last_undegraded":"2024-09-15T22:08:53.582824+0000","last_fullsized":"2024-09-15T22:08:53.582824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:04.210551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750577+0000","last_change":"2024-09-15T22:06:44.653042+0000","last_active":"2024-09-15T22:08:53.750577+0000","last_peered":"2024-09-15T22:08:53.750577+0000","last_clean":"2024-09-15T22:08:53.750577+0000","last_became_active":"2024-09-15T22:06:44.652658+0000","last_became_peered":"2024-09-15T22:06:44.652658+0000","last_unstale":"2024-09-15T22:08:53.750577+0000","last_undegraded":"2024-09-15T22:08:53.750577+0000","last_fullsized":"2024-09-15T22:08:53.750577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:59.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583148+0000","last_change":"2024-09-15T22:06:35.355392+0000","last_active":"2024-09-15T22:08:53.583148+0000","last_peered":"2024-09-15T22:08:53.583148+0000","last_clean":"2024-09-15T22:08:53.583148+0000","last_became_active":"2024-09-15T22:06:35.355042+0000","last_became_peered":"2024-09-15T22:06:35.355042+0000","last_unstale":"2024-09-15T22:08:53.583148+0000","last_undegraded":"2024-09-15T22:08:53.583148+0000","last_fullsized":"2024-09-15T22:08:53.583148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:05.186937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929360+0000","last_change":"2024-09-15T22:06:37.184421+0000","last_active":"2024-09-15T22:08:53.929360+0000","last_peered":"2024-09-15T22:08:53.929360+0000","last_clean":"2024-09-15T22:08:53.929360+0000","last_became_active":"2024-09-15T22:06:37.184092+0000","last_became_peered":"2024-09-15T22:06:37.184092+0000","last_unstale":"2024-09-15T22:08:53.929360+0000","last_undegraded":"2024-09-15T22:08:53.929360+0000","last_fullsized":"2024-09-15T22:08:53.929360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:41.080981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752738+0000","last_change":"2024-09-15T22:07:45.968640+0000","last_active":"2024-09-15T22:08:53.752738+0000","last_peered":"2024-09-15T22:08:53.752738+0000","last_clean":"2024-09-15T22:08:53.752738+0000","last_became_active":"2024-09-15T22:07:45.968320+0000","last_became_peered":"2024-09-15T22:07:45.968320+0000","last_unstale":"2024-09-15T22:08:53.752738+0000","last_undegraded":"2024-09-15T22:08:53.752738+0000","last_fullsized":"2024-09-15T22:08:53.752738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:28.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580416+0000","last_change":"2024-09-15T22:06:35.134606+0000","last_active":"2024-09-15T22:08:53.580416+0000","last_peered":"2024-09-15T22:08:53.580416+0000","last_clean":"2024-09-15T22:08:53.580416+0000","last_became_active":"2024-09-15T22:06:35.134263+0000","last_became_peered":"2024-09-15T22:06:35.134263+0000","last_unstale":"2024-09-15T22:08:53.580416+0000","last_undegraded":"2024-09-15T22:08:53.580416+0000","last_fullsized":"2024-09-15T22:08:53.580416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:45.008935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750806+0000","last_change":"2024-09-15T22:06:37.177556+0000","last_active":"2024-09-15T22:08:53.750806+0000","last_peered":"2024-09-15T22:08:53.750806+0000","last_clean":"2024-09-15T22:08:53.750806+0000","last_became_active":"2024-09-15T22:06:37.177161+0000","last_became_peered":"2024-09-15T22:06:37.177161+0000","last_unstale":"2024-09-15T22:08:53.750806+0000","last_undegraded":"2024-09-15T22:08:53.750806+0000","last_fullsized":"2024-09-15T22:08:53.750806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:30.192728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753347+0000","last_change":"2024-09-15T22:06:44.659097+0000","last_active":"2024-09-15T22:08:53.753347+0000","last_peered":"2024-09-15T22:08:53.753347+0000","last_clean":"2024-09-15T22:08:53.753347+0000","last_became_active":"2024-09-15T22:06:44.658626+0000","last_became_peered":"2024-09-15T22:06:44.658626+0000","last_unstale":"2024-09-15T22:08:53.753347+0000","last_undegraded":"2024-09-15T22:08:53.753347+0000","last_fullsized":"2024-09-15T22:08:53.753347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:01.308004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580430+0000","last_change":"2024-09-15T22:06:35.136326+0000","last_active":"2024-09-15T22:08:53.580430+0000","last_peered":"2024-09-15T22:08:53.580430+0000","last_clean":"2024-09-15T22:08:53.580430+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.580430+0000","last_undegraded":"2024-09-15T22:08:53.580430+0000","last_fullsized":"2024-09-15T22:08:53.580430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:28.675832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490507+0000","last_change":"2024-09-15T22:06:37.192939+0000","last_active":"2024-09-15T22:08:53.490507+0000","last_peered":"2024-09-15T22:08:53.490507+0000","last_clean":"2024-09-15T22:08:53.490507+0000","last_became_active":"2024-09-15T22:06:37.192616+0000","last_became_peered":"2024-09-15T22:06:37.192616+0000","last_unstale":"2024-09-15T22:08:53.490507+0000","last_undegraded":"2024-09-15T22:08:53.490507+0000","last_fullsized":"2024-09-15T22:08:53.490507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:03.615203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752853+0000","last_change":"2024-09-15T22:06:44.654741+0000","last_active":"2024-09-15T22:08:53.752853+0000","last_peered":"2024-09-15T22:08:53.752853+0000","last_clean":"2024-09-15T22:08:53.752853+0000","last_became_active":"2024-09-15T22:06:44.654362+0000","last_became_peered":"2024-09-15T22:06:44.654362+0000","last_unstale":"2024-09-15T22:08:53.752853+0000","last_undegraded":"2024-09-15T22:08:53.752853+0000","last_fullsized":"2024-09-15T22:08:53.752853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:59.269948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490328+0000","last_change":"2024-09-15T22:06:37.186385+0000","last_active":"2024-09-15T22:08:53.490328+0000","last_peered":"2024-09-15T22:08:53.490328+0000","last_clean":"2024-09-15T22:08:53.490328+0000","last_became_active":"2024-09-15T22:06:37.185588+0000","last_became_peered":"2024-09-15T22:06:37.185588+0000","last_unstale":"2024-09-15T22:08:53.490328+0000","last_undegraded":"2024-09-15T22:08:53.490328+0000","last_fullsized":"2024-09-15T22:08:53.490328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:22.523083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580272+0000","last_change":"2024-09-15T22:06:35.137383+0000","last_active":"2024-09-15T22:08:53.580272+0000","last_peered":"2024-09-15T22:08:53.580272+0000","last_clean":"2024-09-15T22:08:53.580272+0000","last_became_active":"2024-09-15T22:06:35.137022+0000","last_became_peered":"2024-09-15T22:06:35.137022+0000","last_unstale":"2024-09-15T22:08:53.580272+0000","last_undegraded":"2024-09-15T22:08:53.580272+0000","last_fullsized":"2024-09-15T22:08:53.580272+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.863155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753002+0000","last_change":"2024-09-15T22:06:44.653807+0000","last_active":"2024-09-15T22:08:53.753002+0000","last_peered":"2024-09-15T22:08:53.753002+0000","last_clean":"2024-09-15T22:08:53.753002+0000","last_became_active":"2024-09-15T22:06:44.653406+0000","last_became_peered":"2024-09-15T22:06:44.653406+0000","last_unstale":"2024-09-15T22:08:53.753002+0000","last_undegraded":"2024-09-15T22:08:53.753002+0000","last_fullsized":"2024-09-15T22:08:53.753002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:12.018208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582122+0000","last_change":"2024-09-15T22:06:37.186899+0000","last_active":"2024-09-15T22:08:53.582122+0000","last_peered":"2024-09-15T22:08:53.582122+0000","last_clean":"2024-09-15T22:08:53.582122+0000","last_became_active":"2024-09-15T22:06:37.183782+0000","last_became_peered":"2024-09-15T22:06:37.183782+0000","last_unstale":"2024-09-15T22:08:53.582122+0000","last_undegraded":"2024-09-15T22:08:53.582122+0000","last_fullsized":"2024-09-15T22:08:53.582122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:39.269186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928379+0000","last_change":"2024-09-15T22:06:44.652686+0000","last_active":"2024-09-15T22:08:53.928379+0000","last_peered":"2024-09-15T22:08:53.928379+0000","last_clean":"2024-09-15T22:08:53.928379+0000","last_became_active":"2024-09-15T22:06:44.652306+0000","last_became_peered":"2024-09-15T22:06:44.652306+0000","last_unstale":"2024-09-15T22:08:53.928379+0000","last_undegraded":"2024-09-15T22:08:53.928379+0000","last_fullsized":"2024-09-15T22:08:53.928379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753037+0000","last_change":"2024-09-15T22:06:44.655424+0000","last_active":"2024-09-15T22:08:53.753037+0000","last_peered":"2024-09-15T22:08:53.753037+0000","last_clean":"2024-09-15T22:08:53.753037+0000","last_became_active":"2024-09-15T22:06:44.654959+0000","last_became_peered":"2024-09-15T22:06:44.654959+0000","last_unstale":"2024-09-15T22:08:53.753037+0000","last_undegraded":"2024-09-15T22:08:53.753037+0000","last_fullsized":"2024-09-15T22:08:53.753037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:07.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928379+0000","last_change":"2024-09-15T22:06:37.188424+0000","last_active":"2024-09-15T22:08:53.928379+0000","last_peered":"2024-09-15T22:08:53.928379+0000","last_clean":"2024-09-15T22:08:53.928379+0000","last_became_active":"2024-09-15T22:06:37.188082+0000","last_became_peered":"2024-09-15T22:06:37.188082+0000","last_unstale":"2024-09-15T22:08:53.928379+0000","last_undegraded":"2024-09-15T22:08:53.928379+0000","last_fullsized":"2024-09-15T22:08:53.928379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:07.159341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581279+0000","last_change":"2024-09-15T22:06:35.134237+0000","last_active":"2024-09-15T22:08:53.581279+0000","last_peered":"2024-09-15T22:08:53.581279+0000","last_clean":"2024-09-15T22:08:53.581279+0000","last_became_active":"2024-09-15T22:06:35.133885+0000","last_became_peered":"2024-09-15T22:06:35.133885+0000","last_unstale":"2024-09-15T22:08:53.581279+0000","last_undegraded":"2024-09-15T22:08:53.581279+0000","last_fullsized":"2024-09-15T22:08:53.581279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:07:35.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489075+0000","last_change":"2024-09-15T22:06:44.664578+0000","last_active":"2024-09-15T22:08:53.489075+0000","last_peered":"2024-09-15T22:08:53.489075+0000","last_clean":"2024-09-15T22:08:53.489075+0000","last_became_active":"2024-09-15T22:06:44.664170+0000","last_became_peered":"2024-09-15T22:06:44.664170+0000","last_unstale":"2024-09-15T22:08:53.489075+0000","last_undegraded":"2024-09-15T22:08:53.489075+0000","last_fullsized":"2024-09-15T22:08:53.489075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:41.443816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580024+0000","last_change":"2024-09-15T22:06:37.186107+0000","last_active":"2024-09-15T22:08:53.580024+0000","last_peered":"2024-09-15T22:08:53.580024+0000","last_clean":"2024-09-15T22:08:53.580024+0000","last_became_active":"2024-09-15T22:06:37.185757+0000","last_became_peered":"2024-09-15T22:06:37.185757+0000","last_unstale":"2024-09-15T22:08:53.580024+0000","last_undegraded":"2024-09-15T22:08:53.580024+0000","last_fullsized":"2024-09-15T22:08:53.580024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:14.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750623+0000","last_change":"2024-09-15T22:06:35.356281+0000","last_active":"2024-09-15T22:08:53.750623+0000","last_peered":"2024-09-15T22:08:53.750623+0000","last_clean":"2024-09-15T22:08:53.750623+0000","last_became_active":"2024-09-15T22:06:35.355922+0000","last_became_peered":"2024-09-15T22:06:35.355922+0000","last_unstale":"2024-09-15T22:08:53.750623+0000","last_undegraded":"2024-09-15T22:08:53.750623+0000","last_fullsized":"2024-09-15T22:08:53.750623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:06.107462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753010+0000","last_change":"2024-09-15T22:06:44.659230+0000","last_active":"2024-09-15T22:08:53.753010+0000","last_peered":"2024-09-15T22:08:53.753010+0000","last_clean":"2024-09-15T22:08:53.753010+0000","last_became_active":"2024-09-15T22:06:44.658649+0000","last_became_peered":"2024-09-15T22:06:44.658649+0000","last_unstale":"2024-09-15T22:08:53.753010+0000","last_undegraded":"2024-09-15T22:08:53.753010+0000","last_fullsized":"2024-09-15T22:08:53.753010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:14:21.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929164+0000","last_change":"2024-09-15T22:06:44.649394+0000","last_active":"2024-09-15T22:08:53.929164+0000","last_peered":"2024-09-15T22:08:53.929164+0000","last_clean":"2024-09-15T22:08:53.929164+0000","last_became_active":"2024-09-15T22:06:44.649043+0000","last_became_peered":"2024-09-15T22:06:44.649043+0000","last_unstale":"2024-09-15T22:08:53.929164+0000","last_undegraded":"2024-09-15T22:08:53.929164+0000","last_fullsized":"2024-09-15T22:08:53.929164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:56:28.335869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753504+0000","last_change":"2024-09-15T22:06:35.265970+0000","last_active":"2024-09-15T22:08:53.753504+0000","last_peered":"2024-09-15T22:08:53.753504+0000","last_clean":"2024-09-15T22:08:53.753504+0000","last_became_active":"2024-09-15T22:06:35.265628+0000","last_became_peered":"2024-09-15T22:06:35.265628+0000","last_unstale":"2024-09-15T22:08:53.753504+0000","last_undegraded":"2024-09-15T22:08:53.753504+0000","last_fullsized":"2024-09-15T22:08:53.753504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:41.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750664+0000","last_change":"2024-09-15T22:06:44.649652+0000","last_active":"2024-09-15T22:08:53.750664+0000","last_peered":"2024-09-15T22:08:53.750664+0000","last_clean":"2024-09-15T22:08:53.750664+0000","last_became_active":"2024-09-15T22:06:44.649112+0000","last_became_peered":"2024-09-15T22:06:44.649112+0000","last_unstale":"2024-09-15T22:08:53.750664+0000","last_undegraded":"2024-09-15T22:08:53.750664+0000","last_fullsized":"2024-09-15T22:08:53.750664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:53.789320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583103+0000","last_change":"2024-09-15T22:06:37.190701+0000","last_active":"2024-09-15T22:08:53.583103+0000","last_peered":"2024-09-15T22:08:53.583103+0000","last_clean":"2024-09-15T22:08:53.583103+0000","last_became_active":"2024-09-15T22:06:37.190430+0000","last_became_peered":"2024-09-15T22:06:37.190430+0000","last_unstale":"2024-09-15T22:08:53.583103+0000","last_undegraded":"2024-09-15T22:08:53.583103+0000","last_fullsized":"2024-09-15T22:08:53.583103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:14.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'51","reported_seq":111,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.701060+0000","last_change":"2024-09-15T22:06:35.133899+0000","last_active":"2024-09-15T22:08:53.701060+0000","last_peered":"2024-09-15T22:08:53.701060+0000","last_clean":"2024-09-15T22:08:53.701060+0000","last_became_active":"2024-09-15T22:06:35.133556+0000","last_became_peered":"2024-09-15T22:06:35.133556+0000","last_unstale":"2024-09-15T22:08:53.701060+0000","last_undegraded":"2024-09-15T22:08:53.701060+0000","last_fullsized":"2024-09-15T22:08:53.701060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:01.692416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580427+0000","last_change":"2024-09-15T22:06:44.655280+0000","last_active":"2024-09-15T22:08:53.580427+0000","last_peered":"2024-09-15T22:08:53.580427+0000","last_clean":"2024-09-15T22:08:53.580427+0000","last_became_active":"2024-09-15T22:06:44.654883+0000","last_became_peered":"2024-09-15T22:06:44.654883+0000","last_unstale":"2024-09-15T22:08:53.580427+0000","last_undegraded":"2024-09-15T22:08:53.580427+0000","last_fullsized":"2024-09-15T22:08:53.580427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.074035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889279+0000","last_change":"2024-09-15T22:06:37.182006+0000","last_active":"2024-09-15T22:08:53.889279+0000","last_peered":"2024-09-15T22:08:53.889279+0000","last_clean":"2024-09-15T22:08:53.889279+0000","last_became_active":"2024-09-15T22:06:37.181693+0000","last_became_peered":"2024-09-15T22:06:37.181693+0000","last_unstale":"2024-09-15T22:08:53.889279+0000","last_undegraded":"2024-09-15T22:08:53.889279+0000","last_fullsized":"2024-09-15T22:08:53.889279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:23.483930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580471+0000","last_change":"2024-09-15T22:06:35.355854+0000","last_active":"2024-09-15T22:08:53.580471+0000","last_peered":"2024-09-15T22:08:53.580471+0000","last_clean":"2024-09-15T22:08:53.580471+0000","last_became_active":"2024-09-15T22:06:35.355486+0000","last_became_peered":"2024-09-15T22:06:35.355486+0000","last_unstale":"2024-09-15T22:08:53.580471+0000","last_undegraded":"2024-09-15T22:08:53.580471+0000","last_fullsized":"2024-09-15T22:08:53.580471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:36.708476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928367+0000","last_change":"2024-09-15T22:06:44.660523+0000","last_active":"2024-09-15T22:08:53.928367+0000","last_peered":"2024-09-15T22:08:53.928367+0000","last_clean":"2024-09-15T22:08:53.928367+0000","last_became_active":"2024-09-15T22:06:44.660113+0000","last_became_peered":"2024-09-15T22:06:44.660113+0000","last_unstale":"2024-09-15T22:08:53.928367+0000","last_undegraded":"2024-09-15T22:08:53.928367+0000","last_fullsized":"2024-09-15T22:08:53.928367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:42.164558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589575+0000","last_change":"2024-09-15T22:06:37.183649+0000","last_active":"2024-09-15T22:08:53.589575+0000","last_peered":"2024-09-15T22:08:53.589575+0000","last_clean":"2024-09-15T22:08:53.589575+0000","last_became_active":"2024-09-15T22:06:37.183296+0000","last_became_peered":"2024-09-15T22:06:37.183296+0000","last_unstale":"2024-09-15T22:08:53.589575+0000","last_undegraded":"2024-09-15T22:08:53.589575+0000","last_fullsized":"2024-09-15T22:08:53.589575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:22.010288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488860+0000","last_change":"2024-09-15T22:06:35.355068+0000","last_active":"2024-09-15T22:08:53.488860+0000","last_peered":"2024-09-15T22:08:53.488860+0000","last_clean":"2024-09-15T22:08:53.488860+0000","last_became_active":"2024-09-15T22:06:35.354728+0000","last_became_peered":"2024-09-15T22:06:35.354728+0000","last_unstale":"2024-09-15T22:08:53.488860+0000","last_undegraded":"2024-09-15T22:08:53.488860+0000","last_fullsized":"2024-09-15T22:08:53.488860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:35.479290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580098+0000","last_change":"2024-09-15T22:06:44.653849+0000","last_active":"2024-09-15T22:08:53.580098+0000","last_peered":"2024-09-15T22:08:53.580098+0000","last_clean":"2024-09-15T22:08:53.580098+0000","last_became_active":"2024-09-15T22:06:44.653419+0000","last_became_peered":"2024-09-15T22:06:44.653419+0000","last_unstale":"2024-09-15T22:08:53.580098+0000","last_undegraded":"2024-09-15T22:08:53.580098+0000","last_fullsized":"2024-09-15T22:08:53.580098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:04.131553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589446+0000","last_change":"2024-09-15T22:06:37.191438+0000","last_active":"2024-09-15T22:08:53.589446+0000","last_peered":"2024-09-15T22:08:53.589446+0000","last_clean":"2024-09-15T22:08:53.589446+0000","last_became_active":"2024-09-15T22:06:37.191088+0000","last_became_peered":"2024-09-15T22:06:37.191088+0000","last_unstale":"2024-09-15T22:08:53.589446+0000","last_undegraded":"2024-09-15T22:08:53.589446+0000","last_fullsized":"2024-09-15T22:08:53.589446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:46:36.078872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490414+0000","last_change":"2024-09-15T22:06:35.135866+0000","last_active":"2024-09-15T22:08:53.490414+0000","last_peered":"2024-09-15T22:08:53.490414+0000","last_clean":"2024-09-15T22:08:53.490414+0000","last_became_active":"2024-09-15T22:06:35.135524+0000","last_became_peered":"2024-09-15T22:06:35.135524+0000","last_unstale":"2024-09-15T22:08:53.490414+0000","last_undegraded":"2024-09-15T22:08:53.490414+0000","last_fullsized":"2024-09-15T22:08:53.490414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888980+0000","last_change":"2024-09-15T22:06:44.649943+0000","last_active":"2024-09-15T22:08:53.888980+0000","last_peered":"2024-09-15T22:08:53.888980+0000","last_clean":"2024-09-15T22:08:53.888980+0000","last_became_active":"2024-09-15T22:06:44.649579+0000","last_became_peered":"2024-09-15T22:06:44.649579+0000","last_unstale":"2024-09-15T22:08:53.888980+0000","last_undegraded":"2024-09-15T22:08:53.888980+0000","last_fullsized":"2024-09-15T22:08:53.888980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:18.390608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489996+0000","last_change":"2024-09-15T22:06:37.189988+0000","last_active":"2024-09-15T22:08:53.489996+0000","last_peered":"2024-09-15T22:08:53.489996+0000","last_clean":"2024-09-15T22:08:53.489996+0000","last_became_active":"2024-09-15T22:06:37.189319+0000","last_became_peered":"2024-09-15T22:06:37.189319+0000","last_unstale":"2024-09-15T22:08:53.489996+0000","last_undegraded":"2024-09-15T22:08:53.489996+0000","last_fullsized":"2024-09-15T22:08:53.489996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:08.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889084+0000","last_change":"2024-09-15T22:06:35.355371+0000","last_active":"2024-09-15T22:08:53.889084+0000","last_peered":"2024-09-15T22:08:53.889084+0000","last_clean":"2024-09-15T22:08:53.889084+0000","last_became_active":"2024-09-15T22:06:35.354687+0000","last_became_peered":"2024-09-15T22:06:35.354687+0000","last_unstale":"2024-09-15T22:08:53.889084+0000","last_undegraded":"2024-09-15T22:08:53.889084+0000","last_fullsized":"2024-09-15T22:08:53.889084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:27.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889532+0000","last_change":"2024-09-15T22:06:44.657929+0000","last_active":"2024-09-15T22:08:53.889532+0000","last_peered":"2024-09-15T22:08:53.889532+0000","last_clean":"2024-09-15T22:08:53.889532+0000","last_became_active":"2024-09-15T22:06:44.657519+0000","last_became_peered":"2024-09-15T22:06:44.657519+0000","last_unstale":"2024-09-15T22:08:53.889532+0000","last_undegraded":"2024-09-15T22:08:53.889532+0000","last_fullsized":"2024-09-15T22:08:53.889532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:12.313877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888747+0000","last_change":"2024-09-15T22:06:37.193288+0000","last_active":"2024-09-15T22:08:53.888747+0000","last_peered":"2024-09-15T22:08:53.888747+0000","last_clean":"2024-09-15T22:08:53.888747+0000","last_became_active":"2024-09-15T22:06:37.193007+0000","last_became_peered":"2024-09-15T22:06:37.193007+0000","last_unstale":"2024-09-15T22:08:53.888747+0000","last_undegraded":"2024-09-15T22:08:53.888747+0000","last_fullsized":"2024-09-15T22:08:53.888747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:45.956206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490201+0000","last_change":"2024-09-15T22:06:35.138726+0000","last_active":"2024-09-15T22:08:53.490201+0000","last_peered":"2024-09-15T22:08:53.490201+0000","last_clean":"2024-09-15T22:08:53.490201+0000","last_became_active":"2024-09-15T22:06:35.138390+0000","last_became_peered":"2024-09-15T22:06:35.138390+0000","last_unstale":"2024-09-15T22:08:53.490201+0000","last_undegraded":"2024-09-15T22:08:53.490201+0000","last_fullsized":"2024-09-15T22:08:53.490201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:51.336171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490388+0000","last_change":"2024-09-15T22:07:46.219720+0000","last_active":"2024-09-15T22:08:53.490388+0000","last_peered":"2024-09-15T22:08:53.490388+0000","last_clean":"2024-09-15T22:08:53.490388+0000","last_became_active":"2024-09-15T22:07:46.219303+0000","last_became_peered":"2024-09-15T22:07:46.219303+0000","last_unstale":"2024-09-15T22:08:53.490388+0000","last_undegraded":"2024-09-15T22:08:53.490388+0000","last_fullsized":"2024-09-15T22:08:53.490388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:03.596795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581603+0000","last_change":"2024-09-15T22:06:37.176478+0000","last_active":"2024-09-15T22:08:53.581603+0000","last_peered":"2024-09-15T22:08:53.581603+0000","last_clean":"2024-09-15T22:08:53.581603+0000","last_became_active":"2024-09-15T22:06:37.176171+0000","last_became_peered":"2024-09-15T22:06:37.176171+0000","last_unstale":"2024-09-15T22:08:53.581603+0000","last_undegraded":"2024-09-15T22:08:53.581603+0000","last_fullsized":"2024-09-15T22:08:53.581603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753268+0000","last_change":"2024-09-15T22:06:35.134651+0000","last_active":"2024-09-15T22:08:53.753268+0000","last_peered":"2024-09-15T22:08:53.753268+0000","last_clean":"2024-09-15T22:08:53.753268+0000","last_became_active":"2024-09-15T22:06:35.134185+0000","last_became_peered":"2024-09-15T22:06:35.134185+0000","last_unstale":"2024-09-15T22:08:53.753268+0000","last_undegraded":"2024-09-15T22:08:53.753268+0000","last_fullsized":"2024-09-15T22:08:53.753268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:56.385046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":358,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490393+0000","last_change":"2024-09-15T22:05:23.048068+0000","last_active":"2024-09-15T22:08:53.490393+0000","last_peered":"2024-09-15T22:08:53.490393+0000","last_clean":"2024-09-15T22:08:53.490393+0000","last_became_active":"2024-09-15T22:05:23.047692+0000","last_became_peered":"2024-09-15T22:05:23.047692+0000","last_unstale":"2024-09-15T22:08:53.490393+0000","last_undegraded":"2024-09-15T22:08:53.490393+0000","last_fullsized":"2024-09-15T22:08:53.490393+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_clean_scrub_stamp":"2024-09-15T22:01:57.477629+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490625+0000","last_change":"2024-09-15T22:06:44.657213+0000","last_active":"2024-09-15T22:08:53.490625+0000","last_peered":"2024-09-15T22:08:53.490625+0000","last_clean":"2024-09-15T22:08:53.490625+0000","last_became_active":"2024-09-15T22:06:44.656815+0000","last_became_peered":"2024-09-15T22:06:44.656815+0000","last_unstale":"2024-09-15T22:08:53.490625+0000","last_undegraded":"2024-09-15T22:08:53.490625+0000","last_fullsized":"2024-09-15T22:08:53.490625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:26.525992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490755+0000","last_change":"2024-09-15T22:06:37.184467+0000","last_active":"2024-09-15T22:08:53.490755+0000","last_peered":"2024-09-15T22:08:53.490755+0000","last_clean":"2024-09-15T22:08:53.490755+0000","last_became_active":"2024-09-15T22:06:37.183864+0000","last_became_peered":"2024-09-15T22:06:37.183864+0000","last_unstale":"2024-09-15T22:08:53.490755+0000","last_undegraded":"2024-09-15T22:08:53.490755+0000","last_fullsized":"2024-09-15T22:08:53.490755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:35.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750757+0000","last_change":"2024-09-15T22:06:44.651594+0000","last_active":"2024-09-15T22:08:53.750757+0000","last_peered":"2024-09-15T22:08:53.750757+0000","last_clean":"2024-09-15T22:08:53.750757+0000","last_became_active":"2024-09-15T22:06:44.650393+0000","last_became_peered":"2024-09-15T22:06:44.650393+0000","last_unstale":"2024-09-15T22:08:53.750757+0000","last_undegraded":"2024-09-15T22:08:53.750757+0000","last_fullsized":"2024-09-15T22:08:53.750757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:13.599744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750561+0000","last_change":"2024-09-15T22:06:44.660789+0000","last_active":"2024-09-15T22:08:53.750561+0000","last_peered":"2024-09-15T22:08:53.750561+0000","last_clean":"2024-09-15T22:08:53.750561+0000","last_became_active":"2024-09-15T22:06:44.660331+0000","last_became_peered":"2024-09-15T22:06:44.660331+0000","last_unstale":"2024-09-15T22:08:53.750561+0000","last_undegraded":"2024-09-15T22:08:53.750561+0000","last_fullsized":"2024-09-15T22:08:53.750561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:04:45.562795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490532+0000","last_change":"2024-09-15T22:06:37.185697+0000","last_active":"2024-09-15T22:08:53.490532+0000","last_peered":"2024-09-15T22:08:53.490532+0000","last_clean":"2024-09-15T22:08:53.490532+0000","last_became_active":"2024-09-15T22:06:37.185207+0000","last_became_peered":"2024-09-15T22:06:37.185207+0000","last_unstale":"2024-09-15T22:08:53.490532+0000","last_undegraded":"2024-09-15T22:08:53.490532+0000","last_fullsized":"2024-09-15T22:08:53.490532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:39.005273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":84,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.699454+0000","last_change":"2024-09-15T22:06:35.135176+0000","last_active":"2024-09-15T22:08:53.699454+0000","last_peered":"2024-09-15T22:08:53.699454+0000","last_clean":"2024-09-15T22:08:53.699454+0000","last_became_active":"2024-09-15T22:06:35.134649+0000","last_became_peered":"2024-09-15T22:06:35.134649+0000","last_unstale":"2024-09-15T22:08:53.699454+0000","last_undegraded":"2024-09-15T22:08:53.699454+0000","last_fullsized":"2024-09-15T22:08:53.699454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:33.780162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928727+0000","last_change":"2024-09-15T22:06:44.656972+0000","last_active":"2024-09-15T22:08:53.928727+0000","last_peered":"2024-09-15T22:08:53.928727+0000","last_clean":"2024-09-15T22:08:53.928727+0000","last_became_active":"2024-09-15T22:06:44.656528+0000","last_became_peered":"2024-09-15T22:06:44.656528+0000","last_unstale":"2024-09-15T22:08:53.928727+0000","last_undegraded":"2024-09-15T22:08:53.928727+0000","last_fullsized":"2024-09-15T22:08:53.928727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:09.164596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490782+0000","last_change":"2024-09-15T22:06:37.186118+0000","last_active":"2024-09-15T22:08:53.490782+0000","last_peered":"2024-09-15T22:08:53.490782+0000","last_clean":"2024-09-15T22:08:53.490782+0000","last_became_active":"2024-09-15T22:06:37.185118+0000","last_became_peered":"2024-09-15T22:06:37.185118+0000","last_unstale":"2024-09-15T22:08:53.490782+0000","last_undegraded":"2024-09-15T22:08:53.490782+0000","last_fullsized":"2024-09-15T22:08:53.490782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:53.357045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889478+0000","last_change":"2024-09-15T22:06:35.133341+0000","last_active":"2024-09-15T22:08:53.889478+0000","last_peered":"2024-09-15T22:08:53.889478+0000","last_clean":"2024-09-15T22:08:53.889478+0000","last_became_active":"2024-09-15T22:06:35.132961+0000","last_became_peered":"2024-09-15T22:06:35.132961+0000","last_unstale":"2024-09-15T22:08:53.889478+0000","last_undegraded":"2024-09-15T22:08:53.889478+0000","last_fullsized":"2024-09-15T22:08:53.889478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:39.579470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489819+0000","last_change":"2024-09-15T22:06:44.659148+0000","last_active":"2024-09-15T22:08:53.489819+0000","last_peered":"2024-09-15T22:08:53.489819+0000","last_clean":"2024-09-15T22:08:53.489819+0000","last_became_active":"2024-09-15T22:06:44.658750+0000","last_became_peered":"2024-09-15T22:06:44.658750+0000","last_unstale":"2024-09-15T22:08:53.489819+0000","last_undegraded":"2024-09-15T22:08:53.489819+0000","last_fullsized":"2024-09-15T22:08:53.489819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:48.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928598+0000","last_change":"2024-09-15T22:06:37.194066+0000","last_active":"2024-09-15T22:08:53.928598+0000","last_peered":"2024-09-15T22:08:53.928598+0000","last_clean":"2024-09-15T22:08:53.928598+0000","last_became_active":"2024-09-15T22:06:37.193738+0000","last_became_peered":"2024-09-15T22:06:37.193738+0000","last_unstale":"2024-09-15T22:08:53.928598+0000","last_undegraded":"2024-09-15T22:08:53.928598+0000","last_fullsized":"2024-09-15T22:08:53.928598+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.107188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580476+0000","last_change":"2024-09-15T22:06:35.136470+0000","last_active":"2024-09-15T22:08:53.580476+0000","last_peered":"2024-09-15T22:08:53.580476+0000","last_clean":"2024-09-15T22:08:53.580476+0000","last_became_active":"2024-09-15T22:06:35.135953+0000","last_became_peered":"2024-09-15T22:06:35.135953+0000","last_unstale":"2024-09-15T22:08:53.580476+0000","last_undegraded":"2024-09-15T22:08:53.580476+0000","last_fullsized":"2024-09-15T22:08:53.580476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:44.548566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489963+0000","last_change":"2024-09-15T22:06:44.653682+0000","last_active":"2024-09-15T22:08:53.489963+0000","last_peered":"2024-09-15T22:08:53.489963+0000","last_clean":"2024-09-15T22:08:53.489963+0000","last_became_active":"2024-09-15T22:06:44.653303+0000","last_became_peered":"2024-09-15T22:06:44.653303+0000","last_unstale":"2024-09-15T22:08:53.489963+0000","last_undegraded":"2024-09-15T22:08:53.489963+0000","last_fullsized":"2024-09-15T22:08:53.489963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:07.847971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489970+0000","last_change":"2024-09-15T22:06:44.658436+0000","last_active":"2024-09-15T22:08:53.489970+0000","last_peered":"2024-09-15T22:08:53.489970+0000","last_clean":"2024-09-15T22:08:53.489970+0000","last_became_active":"2024-09-15T22:06:44.658037+0000","last_became_peered":"2024-09-15T22:06:44.658037+0000","last_unstale":"2024-09-15T22:08:53.489970+0000","last_undegraded":"2024-09-15T22:08:53.489970+0000","last_fullsized":"2024-09-15T22:08:53.489970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:34:20.369249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580102+0000","last_change":"2024-09-15T22:06:35.358018+0000","last_active":"2024-09-15T22:08:53.580102+0000","last_peered":"2024-09-15T22:08:53.580102+0000","last_clean":"2024-09-15T22:08:53.580102+0000","last_became_active":"2024-09-15T22:06:35.357666+0000","last_became_peered":"2024-09-15T22:06:35.357666+0000","last_unstale":"2024-09-15T22:08:53.580102+0000","last_undegraded":"2024-09-15T22:08:53.580102+0000","last_fullsized":"2024-09-15T22:08:53.580102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:34.089113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889340+0000","last_change":"2024-09-15T22:06:44.656807+0000","last_active":"2024-09-15T22:08:53.889340+0000","last_peered":"2024-09-15T22:08:53.889340+0000","last_clean":"2024-09-15T22:08:53.889340+0000","last_became_active":"2024-09-15T22:06:44.656409+0000","last_became_peered":"2024-09-15T22:06:44.656409+0000","last_unstale":"2024-09-15T22:08:53.889340+0000","last_undegraded":"2024-09-15T22:08:53.889340+0000","last_fullsized":"2024-09-15T22:08:53.889340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490123+0000","last_change":"2024-09-15T22:06:37.180615+0000","last_active":"2024-09-15T22:08:53.490123+0000","last_peered":"2024-09-15T22:08:53.490123+0000","last_clean":"2024-09-15T22:08:53.490123+0000","last_became_active":"2024-09-15T22:06:37.180306+0000","last_became_peered":"2024-09-15T22:06:37.180306+0000","last_unstale":"2024-09-15T22:08:53.490123+0000","last_undegraded":"2024-09-15T22:08:53.490123+0000","last_fullsized":"2024-09-15T22:08:53.490123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:31.145128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889942+0000","last_change":"2024-09-15T22:06:35.136192+0000","last_active":"2024-09-15T22:08:53.889942+0000","last_peered":"2024-09-15T22:08:53.889942+0000","last_clean":"2024-09-15T22:08:53.889942+0000","last_became_active":"2024-09-15T22:06:35.135608+0000","last_became_peered":"2024-09-15T22:06:35.135608+0000","last_unstale":"2024-09-15T22:08:53.889942+0000","last_undegraded":"2024-09-15T22:08:53.889942+0000","last_fullsized":"2024-09-15T22:08:53.889942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:48.472717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489151+0000","last_change":"2024-09-15T22:06:44.664763+0000","last_active":"2024-09-15T22:08:53.489151+0000","last_peered":"2024-09-15T22:08:53.489151+0000","last_clean":"2024-09-15T22:08:53.489151+0000","last_became_active":"2024-09-15T22:06:44.664410+0000","last_became_peered":"2024-09-15T22:06:44.664410+0000","last_unstale":"2024-09-15T22:08:53.489151+0000","last_undegraded":"2024-09-15T22:08:53.489151+0000","last_fullsized":"2024-09-15T22:08:53.489151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:14.364387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490299+0000","last_change":"2024-09-15T22:06:44.652747+0000","last_active":"2024-09-15T22:08:53.490299+0000","last_peered":"2024-09-15T22:08:53.490299+0000","last_clean":"2024-09-15T22:08:53.490299+0000","last_became_active":"2024-09-15T22:06:44.652368+0000","last_became_peered":"2024-09-15T22:06:44.652368+0000","last_unstale":"2024-09-15T22:08:53.490299+0000","last_undegraded":"2024-09-15T22:08:53.490299+0000","last_fullsized":"2024-09-15T22:08:53.490299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:51.047549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490109+0000","last_change":"2024-09-15T22:06:35.136311+0000","last_active":"2024-09-15T22:08:53.490109+0000","last_peered":"2024-09-15T22:08:53.490109+0000","last_clean":"2024-09-15T22:08:53.490109+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.490109+0000","last_undegraded":"2024-09-15T22:08:53.490109+0000","last_fullsized":"2024-09-15T22:08:53.490109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:36:07.138056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928366+0000","last_change":"2024-09-15T22:06:44.653747+0000","last_active":"2024-09-15T22:08:53.928366+0000","last_peered":"2024-09-15T22:08:53.928366+0000","last_clean":"2024-09-15T22:08:53.928366+0000","last_became_active":"2024-09-15T22:06:44.653357+0000","last_became_peered":"2024-09-15T22:06:44.653357+0000","last_unstale":"2024-09-15T22:08:53.928366+0000","last_undegraded":"2024-09-15T22:08:53.928366+0000","last_fullsized":"2024-09-15T22:08:53.928366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.113924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888246+0000","last_change":"2024-09-15T22:06:37.178308+0000","last_active":"2024-09-15T22:08:53.888246+0000","last_peered":"2024-09-15T22:08:53.888246+0000","last_clean":"2024-09-15T22:08:53.888246+0000","last_became_active":"2024-09-15T22:06:37.177920+0000","last_became_peered":"2024-09-15T22:06:37.177920+0000","last_unstale":"2024-09-15T22:08:53.888246+0000","last_undegraded":"2024-09-15T22:08:53.888246+0000","last_fullsized":"2024-09-15T22:08:53.888246+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750846+0000","last_change":"2024-09-15T22:06:35.136132+0000","last_active":"2024-09-15T22:08:53.750846+0000","last_peered":"2024-09-15T22:08:53.750846+0000","last_clean":"2024-09-15T22:08:53.750846+0000","last_became_active":"2024-09-15T22:06:35.135789+0000","last_became_peered":"2024-09-15T22:06:35.135789+0000","last_unstale":"2024-09-15T22:08:53.750846+0000","last_undegraded":"2024-09-15T22:08:53.750846+0000","last_fullsized":"2024-09-15T22:08:53.750846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:01.692885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490410+0000","last_change":"2024-09-15T22:06:44.658920+0000","last_active":"2024-09-15T22:08:53.490410+0000","last_peered":"2024-09-15T22:08:53.490410+0000","last_clean":"2024-09-15T22:08:53.490410+0000","last_became_active":"2024-09-15T22:06:44.658434+0000","last_became_peered":"2024-09-15T22:06:44.658434+0000","last_unstale":"2024-09-15T22:08:53.490410+0000","last_undegraded":"2024-09-15T22:08:53.490410+0000","last_fullsized":"2024-09-15T22:08:53.490410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:37.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580330+0000","last_change":"2024-09-15T22:06:37.184958+0000","last_active":"2024-09-15T22:08:53.580330+0000","last_peered":"2024-09-15T22:08:53.580330+0000","last_clean":"2024-09-15T22:08:53.580330+0000","last_became_active":"2024-09-15T22:06:37.184648+0000","last_became_peered":"2024-09-15T22:06:37.184648+0000","last_unstale":"2024-09-15T22:08:53.580330+0000","last_undegraded":"2024-09-15T22:08:53.580330+0000","last_fullsized":"2024-09-15T22:08:53.580330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:28.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":289,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.890809+0000","last_change":"2024-09-15T22:06:35.138533+0000","last_active":"2024-09-15T22:08:53.890809+0000","last_peered":"2024-09-15T22:08:53.890809+0000","last_clean":"2024-09-15T22:08:53.890809+0000","last_became_active":"2024-09-15T22:06:35.138142+0000","last_became_peered":"2024-09-15T22:06:35.138142+0000","last_unstale":"2024-09-15T22:08:53.890809+0000","last_undegraded":"2024-09-15T22:08:53.890809+0000","last_fullsized":"2024-09-15T22:08:53.890809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:32.950036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68792,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":213,"num_read_kb":174,"num_write":41,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589290+0000","last_change":"2024-09-15T22:07:46.218222+0000","last_active":"2024-09-15T22:08:53.589290+0000","last_peered":"2024-09-15T22:08:53.589290+0000","last_clean":"2024-09-15T22:08:53.589290+0000","last_became_active":"2024-09-15T22:07:46.217903+0000","last_became_peered":"2024-09-15T22:07:46.217903+0000","last_unstale":"2024-09-15T22:08:53.589290+0000","last_undegraded":"2024-09-15T22:08:53.589290+0000","last_fullsized":"2024-09-15T22:08:53.589290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:17.730288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750879+0000","last_change":"2024-09-15T22:06:37.194297+0000","last_active":"2024-09-15T22:08:53.750879+0000","last_peered":"2024-09-15T22:08:53.750879+0000","last_clean":"2024-09-15T22:08:53.750879+0000","last_became_active":"2024-09-15T22:06:37.193986+0000","last_became_peered":"2024-09-15T22:06:37.193986+0000","last_unstale":"2024-09-15T22:08:53.750879+0000","last_undegraded":"2024-09-15T22:08:53.750879+0000","last_fullsized":"2024-09-15T22:08:53.750879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:27.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490573+0000","last_change":"2024-09-15T22:06:35.356537+0000","last_active":"2024-09-15T22:08:53.490573+0000","last_peered":"2024-09-15T22:08:53.490573+0000","last_clean":"2024-09-15T22:08:53.490573+0000","last_became_active":"2024-09-15T22:06:35.355997+0000","last_became_peered":"2024-09-15T22:06:35.355997+0000","last_unstale":"2024-09-15T22:08:53.490573+0000","last_undegraded":"2024-09-15T22:08:53.490573+0000","last_fullsized":"2024-09-15T22:08:53.490573+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:11.563083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753107+0000","last_change":"2024-09-15T22:06:44.654579+0000","last_active":"2024-09-15T22:08:53.753107+0000","last_peered":"2024-09-15T22:08:53.753107+0000","last_clean":"2024-09-15T22:08:53.753107+0000","last_became_active":"2024-09-15T22:06:44.654187+0000","last_became_peered":"2024-09-15T22:06:44.654187+0000","last_unstale":"2024-09-15T22:08:53.753107+0000","last_undegraded":"2024-09-15T22:08:53.753107+0000","last_fullsized":"2024-09-15T22:08:53.753107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:15.617528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929412+0000","last_change":"2024-09-15T22:06:44.661533+0000","last_active":"2024-09-15T22:08:53.929412+0000","last_peered":"2024-09-15T22:08:53.929412+0000","last_clean":"2024-09-15T22:08:53.929412+0000","last_became_active":"2024-09-15T22:06:44.661193+0000","last_became_peered":"2024-09-15T22:06:44.661193+0000","last_unstale":"2024-09-15T22:08:53.929412+0000","last_undegraded":"2024-09-15T22:08:53.929412+0000","last_fullsized":"2024-09-15T22:08:53.929412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:06.559720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490695+0000","last_change":"2024-09-15T22:06:35.140578+0000","last_active":"2024-09-15T22:08:53.490695+0000","last_peered":"2024-09-15T22:08:53.490695+0000","last_clean":"2024-09-15T22:08:53.490695+0000","last_became_active":"2024-09-15T22:06:35.139818+0000","last_became_peered":"2024-09-15T22:06:35.139818+0000","last_unstale":"2024-09-15T22:08:53.490695+0000","last_undegraded":"2024-09-15T22:08:53.490695+0000","last_fullsized":"2024-09-15T22:08:53.490695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:59.806964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488473+0000","last_change":"2024-09-15T22:06:44.654755+0000","last_active":"2024-09-15T22:08:53.488473+0000","last_peered":"2024-09-15T22:08:53.488473+0000","last_clean":"2024-09-15T22:08:53.488473+0000","last_became_active":"2024-09-15T22:06:44.654390+0000","last_became_peered":"2024-09-15T22:06:44.654390+0000","last_unstale":"2024-09-15T22:08:53.488473+0000","last_undegraded":"2024-09-15T22:08:53.488473+0000","last_fullsized":"2024-09-15T22:08:53.488473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:35.781878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751028+0000","last_change":"2024-09-15T22:06:37.182457+0000","last_active":"2024-09-15T22:08:53.751028+0000","last_peered":"2024-09-15T22:08:53.751028+0000","last_clean":"2024-09-15T22:08:53.751028+0000","last_became_active":"2024-09-15T22:06:37.182106+0000","last_became_peered":"2024-09-15T22:06:37.182106+0000","last_unstale":"2024-09-15T22:08:53.751028+0000","last_undegraded":"2024-09-15T22:08:53.751028+0000","last_fullsized":"2024-09-15T22:08:53.751028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:57.560894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":140,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589780+0000","last_change":"2024-09-15T22:06:35.140590+0000","last_active":"2024-09-15T22:08:53.589780+0000","last_peered":"2024-09-15T22:08:53.589780+0000","last_clean":"2024-09-15T22:08:53.589780+0000","last_became_active":"2024-09-15T22:06:35.140198+0000","last_became_peered":"2024-09-15T22:06:35.140198+0000","last_unstale":"2024-09-15T22:08:53.589780+0000","last_undegraded":"2024-09-15T22:08:53.589780+0000","last_fullsized":"2024-09-15T22:08:53.589780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:46.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580877+0000","last_change":"2024-09-15T22:06:44.660188+0000","last_active":"2024-09-15T22:08:53.580877+0000","last_peered":"2024-09-15T22:08:53.580877+0000","last_clean":"2024-09-15T22:08:53.580877+0000","last_became_active":"2024-09-15T22:06:44.659808+0000","last_became_peered":"2024-09-15T22:06:44.659808+0000","last_unstale":"2024-09-15T22:08:53.580877+0000","last_undegraded":"2024-09-15T22:08:53.580877+0000","last_fullsized":"2024-09-15T22:08:53.580877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:39.473765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929017+0000","last_change":"2024-09-15T22:06:44.647967+0000","last_active":"2024-09-15T22:08:53.929017+0000","last_peered":"2024-09-15T22:08:53.929017+0000","last_clean":"2024-09-15T22:08:53.929017+0000","last_became_active":"2024-09-15T22:06:44.647608+0000","last_became_peered":"2024-09-15T22:06:44.647608+0000","last_unstale":"2024-09-15T22:08:53.929017+0000","last_undegraded":"2024-09-15T22:08:53.929017+0000","last_fullsized":"2024-09-15T22:08:53.929017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:06.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488584+0000","last_change":"2024-09-15T22:06:37.187020+0000","last_active":"2024-09-15T22:08:53.488584+0000","last_peered":"2024-09-15T22:08:53.488584+0000","last_clean":"2024-09-15T22:08:53.488584+0000","last_became_active":"2024-09-15T22:06:37.186564+0000","last_became_peered":"2024-09-15T22:06:37.186564+0000","last_unstale":"2024-09-15T22:08:53.488584+0000","last_undegraded":"2024-09-15T22:08:53.488584+0000","last_fullsized":"2024-09-15T22:08:53.488584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:20.586763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490131+0000","last_change":"2024-09-15T22:06:35.135126+0000","last_active":"2024-09-15T22:08:53.490131+0000","last_peered":"2024-09-15T22:08:53.490131+0000","last_clean":"2024-09-15T22:08:53.490131+0000","last_became_active":"2024-09-15T22:06:35.134783+0000","last_became_peered":"2024-09-15T22:06:35.134783+0000","last_unstale":"2024-09-15T22:08:53.490131+0000","last_undegraded":"2024-09-15T22:08:53.490131+0000","last_fullsized":"2024-09-15T22:08:53.490131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:31.910988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928656+0000","last_change":"2024-09-15T22:06:44.656981+0000","last_active":"2024-09-15T22:08:53.928656+0000","last_peered":"2024-09-15T22:08:53.928656+0000","last_clean":"2024-09-15T22:08:53.928656+0000","last_became_active":"2024-09-15T22:06:44.656477+0000","last_became_peered":"2024-09-15T22:06:44.656477+0000","last_unstale":"2024-09-15T22:08:53.928656+0000","last_undegraded":"2024-09-15T22:08:53.928656+0000","last_fullsized":"2024-09-15T22:08:53.928656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:06.153490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580796+0000","last_change":"2024-09-15T22:06:37.183034+0000","last_active":"2024-09-15T22:08:53.580796+0000","last_peered":"2024-09-15T22:08:53.580796+0000","last_clean":"2024-09-15T22:08:53.580796+0000","last_became_active":"2024-09-15T22:06:37.182734+0000","last_became_peered":"2024-09-15T22:06:37.182734+0000","last_unstale":"2024-09-15T22:08:53.580796+0000","last_undegraded":"2024-09-15T22:08:53.580796+0000","last_fullsized":"2024-09-15T22:08:53.580796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:19.352000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888909+0000","last_change":"2024-09-15T22:06:35.134954+0000","last_active":"2024-09-15T22:08:53.888909+0000","last_peered":"2024-09-15T22:08:53.888909+0000","last_clean":"2024-09-15T22:08:53.888909+0000","last_became_active":"2024-09-15T22:06:35.134583+0000","last_became_peered":"2024-09-15T22:06:35.134583+0000","last_unstale":"2024-09-15T22:08:53.888909+0000","last_undegraded":"2024-09-15T22:08:53.888909+0000","last_fullsized":"2024-09-15T22:08:53.888909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:56.082123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589453+0000","last_change":"2024-09-15T22:06:35.139229+0000","last_active":"2024-09-15T22:08:53.589453+0000","last_peered":"2024-09-15T22:08:53.589453+0000","last_clean":"2024-09-15T22:08:53.589453+0000","last_became_active":"2024-09-15T22:06:35.138834+0000","last_became_peered":"2024-09-15T22:06:35.138834+0000","last_unstale":"2024-09-15T22:08:53.589453+0000","last_undegraded":"2024-09-15T22:08:53.589453+0000","last_fullsized":"2024-09-15T22:08:53.589453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:05.877404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490250+0000","last_change":"2024-09-15T22:06:37.192212+0000","last_active":"2024-09-15T22:08:53.490250+0000","last_peered":"2024-09-15T22:08:53.490250+0000","last_clean":"2024-09-15T22:08:53.490250+0000","last_became_active":"2024-09-15T22:06:37.191872+0000","last_became_peered":"2024-09-15T22:06:37.191872+0000","last_unstale":"2024-09-15T22:08:53.490250+0000","last_undegraded":"2024-09-15T22:08:53.490250+0000","last_fullsized":"2024-09-15T22:08:53.490250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:32.845794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750771+0000","last_change":"2024-09-15T22:07:45.968398+0000","last_active":"2024-09-15T22:08:53.750771+0000","last_peered":"2024-09-15T22:08:53.750771+0000","last_clean":"2024-09-15T22:08:53.750771+0000","last_became_active":"2024-09-15T22:07:45.967912+0000","last_became_peered":"2024-09-15T22:07:45.967912+0000","last_unstale":"2024-09-15T22:08:53.750771+0000","last_undegraded":"2024-09-15T22:08:53.750771+0000","last_fullsized":"2024-09-15T22:08:53.750771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:59.909290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'9","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.890548+0000","last_change":"2024-09-15T22:06:35.133846+0000","last_active":"2024-09-15T22:08:53.890548+0000","last_peered":"2024-09-15T22:08:53.890548+0000","last_clean":"2024-09-15T22:08:53.890548+0000","last_became_active":"2024-09-15T22:06:35.133516+0000","last_became_peered":"2024-09-15T22:06:35.133516+0000","last_unstale":"2024-09-15T22:08:53.890548+0000","last_undegraded":"2024-09-15T22:08:53.890548+0000","last_fullsized":"2024-09-15T22:08:53.890548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:29.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580118+0000","last_change":"2024-09-15T22:06:37.182904+0000","last_active":"2024-09-15T22:08:53.580118+0000","last_peered":"2024-09-15T22:08:53.580118+0000","last_clean":"2024-09-15T22:08:53.580118+0000","last_became_active":"2024-09-15T22:06:37.182563+0000","last_became_peered":"2024-09-15T22:06:37.182563+0000","last_unstale":"2024-09-15T22:08:53.580118+0000","last_undegraded":"2024-09-15T22:08:53.580118+0000","last_fullsized":"2024-09-15T22:08:53.580118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750976+0000","last_change":"2024-09-15T22:06:44.660997+0000","last_active":"2024-09-15T22:08:53.750976+0000","last_peered":"2024-09-15T22:08:53.750976+0000","last_clean":"2024-09-15T22:08:53.750976+0000","last_became_active":"2024-09-15T22:06:44.660577+0000","last_became_peered":"2024-09-15T22:06:44.660577+0000","last_unstale":"2024-09-15T22:08:53.750976+0000","last_undegraded":"2024-09-15T22:08:53.750976+0000","last_fullsized":"2024-09-15T22:08:53.750976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:56.006891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928243+0000","last_change":"2024-09-15T22:06:37.182077+0000","last_active":"2024-09-15T22:08:53.928243+0000","last_peered":"2024-09-15T22:08:53.928243+0000","last_clean":"2024-09-15T22:08:53.928243+0000","last_became_active":"2024-09-15T22:06:37.181622+0000","last_became_peered":"2024-09-15T22:06:37.181622+0000","last_unstale":"2024-09-15T22:08:53.928243+0000","last_undegraded":"2024-09-15T22:08:53.928243+0000","last_fullsized":"2024-09-15T22:08:53.928243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:07.586406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750818+0000","last_change":"2024-09-15T22:06:44.653845+0000","last_active":"2024-09-15T22:08:53.750818+0000","last_peered":"2024-09-15T22:08:53.750818+0000","last_clean":"2024-09-15T22:08:53.750818+0000","last_became_active":"2024-09-15T22:06:44.653483+0000","last_became_peered":"2024-09-15T22:06:44.653483+0000","last_unstale":"2024-09-15T22:08:53.750818+0000","last_undegraded":"2024-09-15T22:08:53.750818+0000","last_fullsized":"2024-09-15T22:08:53.750818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:08:07.822279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750649+0000","last_change":"2024-09-15T22:06:37.187476+0000","last_active":"2024-09-15T22:08:53.750649+0000","last_peered":"2024-09-15T22:08:53.750649+0000","last_clean":"2024-09-15T22:08:53.750649+0000","last_became_active":"2024-09-15T22:06:37.187047+0000","last_became_peered":"2024-09-15T22:06:37.187047+0000","last_unstale":"2024-09-15T22:08:53.750649+0000","last_undegraded":"2024-09-15T22:08:53.750649+0000","last_fullsized":"2024-09-15T22:08:53.750649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.220834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928765+0000","last_change":"2024-09-15T22:06:37.190583+0000","last_active":"2024-09-15T22:08:53.928765+0000","last_peered":"2024-09-15T22:08:53.928765+0000","last_clean":"2024-09-15T22:08:53.928765+0000","last_became_active":"2024-09-15T22:06:37.190212+0000","last_became_peered":"2024-09-15T22:06:37.190212+0000","last_unstale":"2024-09-15T22:08:53.928765+0000","last_undegraded":"2024-09-15T22:08:53.928765+0000","last_fullsized":"2024-09-15T22:08:53.928765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:48.574188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889153+0000","last_change":"2024-09-15T22:06:37.192508+0000","last_active":"2024-09-15T22:08:53.889153+0000","last_peered":"2024-09-15T22:08:53.889153+0000","last_clean":"2024-09-15T22:08:53.889153+0000","last_became_active":"2024-09-15T22:06:37.192117+0000","last_became_peered":"2024-09-15T22:06:37.192117+0000","last_unstale":"2024-09-15T22:08:53.889153+0000","last_undegraded":"2024-09-15T22:08:53.889153+0000","last_fullsized":"2024-09-15T22:08:53.889153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:41.355966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928601+0000","last_change":"2024-09-15T22:06:37.183377+0000","last_active":"2024-09-15T22:08:53.928601+0000","last_peered":"2024-09-15T22:08:53.928601+0000","last_clean":"2024-09-15T22:08:53.928601+0000","last_became_active":"2024-09-15T22:06:37.183072+0000","last_became_peered":"2024-09-15T22:06:37.183072+0000","last_unstale":"2024-09-15T22:08:53.928601+0000","last_undegraded":"2024-09-15T22:08:53.928601+0000","last_fullsized":"2024-09-15T22:08:53.928601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.318702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751128+0000","last_change":"2024-09-15T22:06:37.185348+0000","last_active":"2024-09-15T22:08:53.751128+0000","last_peered":"2024-09-15T22:08:53.751128+0000","last_clean":"2024-09-15T22:08:53.751128+0000","last_became_active":"2024-09-15T22:06:37.184934+0000","last_became_peered":"2024-09-15T22:06:37.184934+0000","last_unstale":"2024-09-15T22:08:53.751128+0000","last_undegraded":"2024-09-15T22:08:53.751128+0000","last_fullsized":"2024-09-15T22:08:53.751128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:57.564860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582987+0000","last_change":"2024-09-15T22:06:37.192152+0000","last_active":"2024-09-15T22:08:53.582987+0000","last_peered":"2024-09-15T22:08:53.582987+0000","last_clean":"2024-09-15T22:08:53.582987+0000","last_became_active":"2024-09-15T22:06:37.191836+0000","last_became_peered":"2024-09-15T22:06:37.191836+0000","last_unstale":"2024-09-15T22:08:53.582987+0000","last_undegraded":"2024-09-15T22:08:53.582987+0000","last_fullsized":"2024-09-15T22:08:53.582987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:31.781110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489966+0000","last_change":"2024-09-15T22:06:37.179996+0000","last_active":"2024-09-15T22:08:53.489966+0000","last_peered":"2024-09-15T22:08:53.489966+0000","last_clean":"2024-09-15T22:08:53.489966+0000","last_became_active":"2024-09-15T22:06:37.179646+0000","last_became_peered":"2024-09-15T22:06:37.179646+0000","last_unstale":"2024-09-15T22:08:53.489966+0000","last_undegraded":"2024-09-15T22:08:53.489966+0000","last_fullsized":"2024-09-15T22:08:53.489966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:35.240468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581098+0000","last_change":"2024-09-15T22:06:37.182992+0000","last_active":"2024-09-15T22:08:53.581098+0000","last_peered":"2024-09-15T22:08:53.581098+0000","last_clean":"2024-09-15T22:08:53.581098+0000","last_became_active":"2024-09-15T22:06:37.182676+0000","last_became_peered":"2024-09-15T22:06:37.182676+0000","last_unstale":"2024-09-15T22:08:53.581098+0000","last_undegraded":"2024-09-15T22:08:53.581098+0000","last_fullsized":"2024-09-15T22:08:53.581098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:41.524600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750731+0000","last_change":"2024-09-15T22:06:37.180107+0000","last_active":"2024-09-15T22:08:53.750731+0000","last_peered":"2024-09-15T22:08:53.750731+0000","last_clean":"2024-09-15T22:08:53.750731+0000","last_became_active":"2024-09-15T22:06:37.179785+0000","last_became_peered":"2024-09-15T22:06:37.179785+0000","last_unstale":"2024-09-15T22:08:53.750731+0000","last_undegraded":"2024-09-15T22:08:53.750731+0000","last_fullsized":"2024-09-15T22:08:53.750731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:46.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888926+0000","last_change":"2024-09-15T22:06:37.177406+0000","last_active":"2024-09-15T22:08:53.888926+0000","last_peered":"2024-09-15T22:08:53.888926+0000","last_clean":"2024-09-15T22:08:53.888926+0000","last_became_active":"2024-09-15T22:06:37.177093+0000","last_became_peered":"2024-09-15T22:06:37.177093+0000","last_unstale":"2024-09-15T22:08:53.888926+0000","last_undegraded":"2024-09-15T22:08:53.888926+0000","last_fullsized":"2024-09-15T22:08:53.888926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:34.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928419+0000","last_change":"2024-09-15T22:06:37.192314+0000","last_active":"2024-09-15T22:08:53.928419+0000","last_peered":"2024-09-15T22:08:53.928419+0000","last_clean":"2024-09-15T22:08:53.928419+0000","last_became_active":"2024-09-15T22:06:37.191958+0000","last_became_peered":"2024-09-15T22:06:37.191958+0000","last_unstale":"2024-09-15T22:08:53.928419+0000","last_undegraded":"2024-09-15T22:08:53.928419+0000","last_fullsized":"2024-09-15T22:08:53.928419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:47.763742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751178+0000","last_change":"2024-09-15T22:06:37.190212+0000","last_active":"2024-09-15T22:08:53.751178+0000","last_peered":"2024-09-15T22:08:53.751178+0000","last_clean":"2024-09-15T22:08:53.751178+0000","last_became_active":"2024-09-15T22:06:37.189881+0000","last_became_peered":"2024-09-15T22:06:37.189881+0000","last_unstale":"2024-09-15T22:08:53.751178+0000","last_undegraded":"2024-09-15T22:08:53.751178+0000","last_fullsized":"2024-09-15T22:08:53.751178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:59.943574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488963+0000","last_change":"2024-09-15T22:06:37.187967+0000","last_active":"2024-09-15T22:08:53.488963+0000","last_peered":"2024-09-15T22:08:53.488963+0000","last_clean":"2024-09-15T22:08:53.488963+0000","last_became_active":"2024-09-15T22:06:37.187596+0000","last_became_peered":"2024-09-15T22:06:37.187596+0000","last_unstale":"2024-09-15T22:08:53.488963+0000","last_undegraded":"2024-09-15T22:08:53.488963+0000","last_fullsized":"2024-09-15T22:08:53.488963+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:00.042784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928757+0000","last_change":"2024-09-15T22:06:37.189434+0000","last_active":"2024-09-15T22:08:53.928757+0000","last_peered":"2024-09-15T22:08:53.928757+0000","last_clean":"2024-09-15T22:08:53.928757+0000","last_became_active":"2024-09-15T22:06:37.189053+0000","last_became_peered":"2024-09-15T22:06:37.189053+0000","last_unstale":"2024-09-15T22:08:53.928757+0000","last_undegraded":"2024-09-15T22:08:53.928757+0000","last_fullsized":"2024-09-15T22:08:53.928757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:51.640495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750486+0000","last_change":"2024-09-15T22:06:37.179229+0000","last_active":"2024-09-15T22:08:53.750486+0000","last_peered":"2024-09-15T22:08:53.750486+0000","last_clean":"2024-09-15T22:08:53.750486+0000","last_became_active":"2024-09-15T22:06:37.178828+0000","last_became_peered":"2024-09-15T22:06:37.178828+0000","last_unstale":"2024-09-15T22:08:53.750486+0000","last_undegraded":"2024-09-15T22:08:53.750486+0000","last_fullsized":"2024-09-15T22:08:53.750486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:25.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928489+0000","last_change":"2024-09-15T22:06:37.193095+0000","last_active":"2024-09-15T22:08:53.928489+0000","last_peered":"2024-09-15T22:08:53.928489+0000","last_clean":"2024-09-15T22:08:53.928489+0000","last_became_active":"2024-09-15T22:06:37.192801+0000","last_became_peered":"2024-09-15T22:06:37.192801+0000","last_unstale":"2024-09-15T22:08:53.928489+0000","last_undegraded":"2024-09-15T22:08:53.928489+0000","last_fullsized":"2024-09-15T22:08:53.928489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:12.521510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928614+0000","last_change":"2024-09-15T22:06:37.182332+0000","last_active":"2024-09-15T22:08:53.928614+0000","last_peered":"2024-09-15T22:08:53.928614+0000","last_clean":"2024-09-15T22:08:53.928614+0000","last_became_active":"2024-09-15T22:06:37.182001+0000","last_became_peered":"2024-09-15T22:06:37.182001+0000","last_unstale":"2024-09-15T22:08:53.928614+0000","last_undegraded":"2024-09-15T22:08:53.928614+0000","last_fullsized":"2024-09-15T22:08:53.928614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:52.230452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888644+0000","last_change":"2024-09-15T22:06:37.180584+0000","last_active":"2024-09-15T22:08:53.888644+0000","last_peered":"2024-09-15T22:08:53.888644+0000","last_clean":"2024-09-15T22:08:53.888644+0000","last_became_active":"2024-09-15T22:06:37.179341+0000","last_became_peered":"2024-09-15T22:06:37.179341+0000","last_unstale":"2024-09-15T22:08:53.888644+0000","last_undegraded":"2024-09-15T22:08:53.888644+0000","last_fullsized":"2024-09-15T22:08:53.888644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.610835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753439+0000","last_change":"2024-09-15T22:06:37.189500+0000","last_active":"2024-09-15T22:08:53.753439+0000","last_peered":"2024-09-15T22:08:53.753439+0000","last_clean":"2024-09-15T22:08:53.753439+0000","last_became_active":"2024-09-15T22:06:37.189137+0000","last_became_peered":"2024-09-15T22:06:37.189137+0000","last_unstale":"2024-09-15T22:08:53.753439+0000","last_undegraded":"2024-09-15T22:08:53.753439+0000","last_fullsized":"2024-09-15T22:08:53.753439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:27.199854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490697+0000","last_change":"2024-09-15T22:06:37.182618+0000","last_active":"2024-09-15T22:08:53.490697+0000","last_peered":"2024-09-15T22:08:53.490697+0000","last_clean":"2024-09-15T22:08:53.490697+0000","last_became_active":"2024-09-15T22:06:37.182257+0000","last_became_peered":"2024-09-15T22:06:37.182257+0000","last_unstale":"2024-09-15T22:08:53.490697+0000","last_undegraded":"2024-09-15T22:08:53.490697+0000","last_fullsized":"2024-09-15T22:08:53.490697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:17.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589823+0000","last_change":"2024-09-15T22:06:37.193036+0000","last_active":"2024-09-15T22:08:53.589823+0000","last_peered":"2024-09-15T22:08:53.589823+0000","last_clean":"2024-09-15T22:08:53.589823+0000","last_became_active":"2024-09-15T22:06:37.192668+0000","last_became_peered":"2024-09-15T22:06:37.192668+0000","last_unstale":"2024-09-15T22:08:53.589823+0000","last_undegraded":"2024-09-15T22:08:53.589823+0000","last_fullsized":"2024-09-15T22:08:53.589823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:05.436045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489086+0000","last_change":"2024-09-15T22:06:44.659503+0000","last_active":"2024-09-15T22:08:53.489086+0000","last_peered":"2024-09-15T22:08:53.489086+0000","last_clean":"2024-09-15T22:08:53.489086+0000","last_became_active":"2024-09-15T22:06:44.659155+0000","last_became_peered":"2024-09-15T22:06:44.659155+0000","last_unstale":"2024-09-15T22:08:53.489086+0000","last_undegraded":"2024-09-15T22:08:53.489086+0000","last_fullsized":"2024-09-15T22:08:53.489086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:41:04.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580648+0000","last_change":"2024-09-15T22:06:37.178386+0000","last_active":"2024-09-15T22:08:53.580648+0000","last_peered":"2024-09-15T22:08:53.580648+0000","last_clean":"2024-09-15T22:08:53.580648+0000","last_became_active":"2024-09-15T22:06:37.178083+0000","last_became_peered":"2024-09-15T22:06:37.178083+0000","last_unstale":"2024-09-15T22:08:53.580648+0000","last_undegraded":"2024-09-15T22:08:53.580648+0000","last_fullsized":"2024-09-15T22:08:53.580648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:35.350698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490190+0000","last_change":"2024-09-15T22:06:37.186741+0000","last_active":"2024-09-15T22:08:53.490190+0000","last_peered":"2024-09-15T22:08:53.490190+0000","last_clean":"2024-09-15T22:08:53.490190+0000","last_became_active":"2024-09-15T22:06:37.185740+0000","last_became_peered":"2024-09-15T22:06:37.185740+0000","last_unstale":"2024-09-15T22:08:53.490190+0000","last_undegraded":"2024-09-15T22:08:53.490190+0000","last_fullsized":"2024-09-15T22:08:53.490190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.762407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589411+0000","last_change":"2024-09-15T22:06:37.189552+0000","last_active":"2024-09-15T22:08:53.589411+0000","last_peered":"2024-09-15T22:08:53.589411+0000","last_clean":"2024-09-15T22:08:53.589411+0000","last_became_active":"2024-09-15T22:06:37.189197+0000","last_became_peered":"2024-09-15T22:06:37.189197+0000","last_unstale":"2024-09-15T22:08:53.589411+0000","last_undegraded":"2024-09-15T22:08:53.589411+0000","last_fullsized":"2024-09-15T22:08:53.589411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:54.268720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580092+0000","last_change":"2024-09-15T22:06:37.186917+0000","last_active":"2024-09-15T22:08:53.580092+0000","last_peered":"2024-09-15T22:08:53.580092+0000","last_clean":"2024-09-15T22:08:53.580092+0000","last_became_active":"2024-09-15T22:06:37.186571+0000","last_became_peered":"2024-09-15T22:06:37.186571+0000","last_unstale":"2024-09-15T22:08:53.580092+0000","last_undegraded":"2024-09-15T22:08:53.580092+0000","last_fullsized":"2024-09-15T22:08:53.580092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:02.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753272+0000","last_change":"2024-09-15T22:06:37.190960+0000","last_active":"2024-09-15T22:08:53.753272+0000","last_peered":"2024-09-15T22:08:53.753272+0000","last_clean":"2024-09-15T22:08:53.753272+0000","last_became_active":"2024-09-15T22:06:37.190650+0000","last_became_peered":"2024-09-15T22:06:37.190650+0000","last_unstale":"2024-09-15T22:08:53.753272+0000","last_undegraded":"2024-09-15T22:08:53.753272+0000","last_fullsized":"2024-09-15T22:08:53.753272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:34.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928796+0000","last_change":"2024-09-15T22:06:37.186718+0000","last_active":"2024-09-15T22:08:53.928796+0000","last_peered":"2024-09-15T22:08:53.928796+0000","last_clean":"2024-09-15T22:08:53.928796+0000","last_became_active":"2024-09-15T22:06:37.186424+0000","last_became_peered":"2024-09-15T22:06:37.186424+0000","last_unstale":"2024-09-15T22:08:53.928796+0000","last_undegraded":"2024-09-15T22:08:53.928796+0000","last_fullsized":"2024-09-15T22:08:53.928796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:53.266007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929020+0000","last_change":"2024-09-15T22:06:37.185522+0000","last_active":"2024-09-15T22:08:53.929020+0000","last_peered":"2024-09-15T22:08:53.929020+0000","last_clean":"2024-09-15T22:08:53.929020+0000","last_became_active":"2024-09-15T22:06:37.185192+0000","last_became_peered":"2024-09-15T22:06:37.185192+0000","last_unstale":"2024-09-15T22:08:53.929020+0000","last_undegraded":"2024-09-15T22:08:53.929020+0000","last_fullsized":"2024-09-15T22:08:53.929020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:12.868324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928738+0000","last_change":"2024-09-15T22:06:37.189553+0000","last_active":"2024-09-15T22:08:53.928738+0000","last_peered":"2024-09-15T22:08:53.928738+0000","last_clean":"2024-09-15T22:08:53.928738+0000","last_became_active":"2024-09-15T22:06:37.189270+0000","last_became_peered":"2024-09-15T22:06:37.189270+0000","last_unstale":"2024-09-15T22:08:53.928738+0000","last_undegraded":"2024-09-15T22:08:53.928738+0000","last_fullsized":"2024-09-15T22:08:53.928738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:34.157077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599380,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":730,"num_read_kb":478,"num_write":302,"num_write_kb":1445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1798140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579945,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":502844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808878,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32996,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712156,"statfs":{"total":95995035648,"available":95961247744,"internally_reserved":0,"allocated":1855488,"data_stored":1300760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005107,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1437696,"data_stored":909721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1429504,"data_stored":904963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495677,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1630208,"data_stored":1109427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659201,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":507585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888135,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37080,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708072,"statfs":{"total":95995035648,"available":95957065728,"internally_reserved":0,"allocated":1777664,"data_stored":1236270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018956,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":513744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247889,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1032192,"data_stored":507779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444119,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1114112,"data_stored":578290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1097728,"data_stored":576869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1044480,"data_stored":508973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":202897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:08:58.447 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T22:08:58.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.583+0000 7f03ffb13640 1 -- 172.21.15.5:0/246062319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03f8101260 msgr2=0x7f03f810de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.583+0000 7f03ffb13640 1 --2- 172.21.15.5:0/246062319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03f8101260 0x7f03f810de90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009900 tx=0x7f03e802a410 comp rx=0 tx=0).stop 2024-09-15T22:08:58.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.584+0000 7f03ffb13640 1 -- 172.21.15.5:0/246062319 shutdown_connections 2024-09-15T22:08:58.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.584+0000 7f03ffb13640 1 --2- 172.21.15.5:0/246062319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03f810e460 0x7f03f8110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.584+0000 7f03ffb13640 1 --2- 172.21.15.5:0/246062319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03f8101260 0x7f03f810de90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.584+0000 7f03ffb13640 1 --2- 172.21.15.5:0/246062319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f81008b0 0x7f03f8100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.584+0000 7f03ffb13640 1 -- 172.21.15.5:0/246062319 >> 172.21.15.5:0/246062319 conn(0x7f03f80fc6b0 msgr2=0x7f03f80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:58.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.584+0000 7f03ffb13640 1 -- 172.21.15.5:0/246062319 shutdown_connections 2024-09-15T22:08:58.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.584+0000 7f03ffb13640 1 -- 172.21.15.5:0/246062319 wait complete. 2024-09-15T22:08:58.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.584+0000 7f03ffb13640 1 Processor -- start 2024-09-15T22:08:58.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.584+0000 7f03ffb13640 1 -- start start 2024-09-15T22:08:58.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03ffb13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03f81008b0 0x7f03f819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03ffb13640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03f8101260 0x7f03f819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03ffb13640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f810e460 0x7f03f819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03ffb13640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03f81140e0 con 0x7f03f81008b0 2024-09-15T22:08:58.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03ffb13640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8113f60 con 0x7f03f810e460 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03ffb13640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f03f8114260 con 0x7f03f8101260 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fd8af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03f81008b0 0x7f03f819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fd8af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03f81008b0 0x7f03f819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56080/0 (socket says 172.21.15.5:56080) 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fd8af640 1 -- 172.21.15.5:0/2683447331 learned_addr learned my addr 172.21.15.5:0/2683447331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fd0ae640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03f8101260 0x7f03f819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fe0b0640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f810e460 0x7f03f819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fd8af640 1 -- 172.21.15.5:0/2683447331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03f8101260 msgr2=0x7f03f819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fd8af640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03f8101260 0x7f03f819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fd8af640 1 -- 172.21.15.5:0/2683447331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f810e460 msgr2=0x7f03f819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fd8af640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f810e460 0x7f03f819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fd8af640 1 -- 172.21.15.5:0/2683447331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f819f620 con 0x7f03f81008b0 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fe0b0640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f810e460 0x7f03f819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03fd8af640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03f81008b0 0x7f03f819a630 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f03f400d950 tx=0x7f03f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:58.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03eeffd640 1 -- 172.21.15.5:0/2683447331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4017030 con 0x7f03f81008b0 2024-09-15T22:08:58.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03eeffd640 1 -- 172.21.15.5:0/2683447331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03f4004020 con 0x7f03f81008b0 2024-09-15T22:08:58.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03ffb13640 1 -- 172.21.15.5:0/2683447331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f819f910 con 0x7f03f81008b0 2024-09-15T22:08:58.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03eeffd640 1 -- 172.21.15.5:0/2683447331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f40047a0 con 0x7f03f81008b0 2024-09-15T22:08:58.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.585+0000 7f03ffb13640 1 -- 172.21.15.5:0/2683447331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f81a71a0 con 0x7f03f81008b0 2024-09-15T22:08:58.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.586+0000 7f03ffb13640 1 -- 172.21.15.5:0/2683447331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f8100c90 con 0x7f03f81008b0 2024-09-15T22:08:58.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.588+0000 7f03eeffd640 1 -- 172.21.15.5:0/2683447331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f03f40041d0 con 0x7f03f81008b0 2024-09-15T22:08:58.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.588+0000 7f03eeffd640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f03e40a3560 0x7f03e40a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.588+0000 7f03fd0ae640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f03e40a3560 0x7f03e40a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.588+0000 7f03eeffd640 1 -- 172.21.15.5:0/2683447331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f03f40cfcd0 con 0x7f03f81008b0 2024-09-15T22:08:58.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.589+0000 7f03fd0ae640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f03e40a3560 0x7f03e40a5a20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009230 tx=0x7f03e80058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:58.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.592+0000 7f03eeffd640 1 -- 172.21.15.5:0/2683447331 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03f409c0c0 con 0x7f03f81008b0 2024-09-15T22:08:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.766+0000 7f03ffb13640 1 -- 172.21.15.5:0/2683447331 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03f8062630 con 0x7f03e40a3560 2024-09-15T22:08:58.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.775+0000 7f03eeffd640 1 -- 172.21.15.5:0/2683447331 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339383 (secure 0 0 0) 0x7f03f8062630 con 0x7f03e40a3560 2024-09-15T22:08:58.780 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:58.782 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-15T22:08:58.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 -- 172.21.15.5:0/2683447331 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f03e40a3560 msgr2=0x7f03e40a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f03e40a3560 0x7f03e40a5a20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009230 tx=0x7f03e80058d0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 -- 172.21.15.5:0/2683447331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03f81008b0 msgr2=0x7f03f819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03f81008b0 0x7f03f819a630 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f03f400d950 tx=0x7f03f400de10 comp rx=0 tx=0).stop 2024-09-15T22:08:58.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 -- 172.21.15.5:0/2683447331 shutdown_connections 2024-09-15T22:08:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f03e40a3560 0x7f03e40a5a20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f03f810e460 0x7f03f819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f03f8101260 0x7f03f819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 --2- 172.21.15.5:0/2683447331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03f81008b0 0x7f03f819a630 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 -- 172.21.15.5:0/2683447331 >> 172.21.15.5:0/2683447331 conn(0x7f03f80fc6b0 msgr2=0x7f03f80feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 -- 172.21.15.5:0/2683447331 shutdown_connections 2024-09-15T22:08:58.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.779+0000 7f03ffb13640 1 -- 172.21.15.5:0/2683447331 wait complete. 2024-09-15T22:08:58.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:58 smithi005 ceph-mon[30944]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:08:58.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:58 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:58.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:58 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:58.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:58 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:58.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:58 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:58.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:58 smithi005 ceph-mon[30944]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-15T22:08:58.791 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-15T22:08:57.501792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189982,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":866,"num_read_kb":597,"num_write":485,"num_write_kb":4385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":393524,"kb_used_data":15124,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124548300,"statfs":{"total":1151940427776,"available":1151537459200,"internally_reserved":0,"allocated":15486976,"data_stored":9157225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":387233083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.989795"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489253+0000","last_change":"2024-09-15T22:07:46.226073+0000","last_active":"2024-09-15T22:08:53.489253+0000","last_peered":"2024-09-15T22:08:53.489253+0000","last_clean":"2024-09-15T22:08:53.489253+0000","last_became_active":"2024-09-15T22:07:46.219250+0000","last_became_peered":"2024-09-15T22:07:46.219250+0000","last_unstale":"2024-09-15T22:08:53.489253+0000","last_undegraded":"2024-09-15T22:08:53.489253+0000","last_fullsized":"2024-09-15T22:08:53.489253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:56.324959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,4],"acting":[9,11,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582824+0000","last_change":"2024-09-15T22:06:37.184096+0000","last_active":"2024-09-15T22:08:53.582824+0000","last_peered":"2024-09-15T22:08:53.582824+0000","last_clean":"2024-09-15T22:08:53.582824+0000","last_became_active":"2024-09-15T22:06:37.183758+0000","last_became_peered":"2024-09-15T22:06:37.183758+0000","last_unstale":"2024-09-15T22:08:53.582824+0000","last_undegraded":"2024-09-15T22:08:53.582824+0000","last_fullsized":"2024-09-15T22:08:53.582824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:04.210551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750577+0000","last_change":"2024-09-15T22:06:44.653042+0000","last_active":"2024-09-15T22:08:53.750577+0000","last_peered":"2024-09-15T22:08:53.750577+0000","last_clean":"2024-09-15T22:08:53.750577+0000","last_became_active":"2024-09-15T22:06:44.652658+0000","last_became_peered":"2024-09-15T22:06:44.652658+0000","last_unstale":"2024-09-15T22:08:53.750577+0000","last_undegraded":"2024-09-15T22:08:53.750577+0000","last_fullsized":"2024-09-15T22:08:53.750577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:59.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583148+0000","last_change":"2024-09-15T22:06:35.355392+0000","last_active":"2024-09-15T22:08:53.583148+0000","last_peered":"2024-09-15T22:08:53.583148+0000","last_clean":"2024-09-15T22:08:53.583148+0000","last_became_active":"2024-09-15T22:06:35.355042+0000","last_became_peered":"2024-09-15T22:06:35.355042+0000","last_unstale":"2024-09-15T22:08:53.583148+0000","last_undegraded":"2024-09-15T22:08:53.583148+0000","last_fullsized":"2024-09-15T22:08:53.583148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:05.186937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929360+0000","last_change":"2024-09-15T22:06:37.184421+0000","last_active":"2024-09-15T22:08:53.929360+0000","last_peered":"2024-09-15T22:08:53.929360+0000","last_clean":"2024-09-15T22:08:53.929360+0000","last_became_active":"2024-09-15T22:06:37.184092+0000","last_became_peered":"2024-09-15T22:06:37.184092+0000","last_unstale":"2024-09-15T22:08:53.929360+0000","last_undegraded":"2024-09-15T22:08:53.929360+0000","last_fullsized":"2024-09-15T22:08:53.929360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:41.080981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752738+0000","last_change":"2024-09-15T22:07:45.968640+0000","last_active":"2024-09-15T22:08:53.752738+0000","last_peered":"2024-09-15T22:08:53.752738+0000","last_clean":"2024-09-15T22:08:53.752738+0000","last_became_active":"2024-09-15T22:07:45.968320+0000","last_became_peered":"2024-09-15T22:07:45.968320+0000","last_unstale":"2024-09-15T22:08:53.752738+0000","last_undegraded":"2024-09-15T22:08:53.752738+0000","last_fullsized":"2024-09-15T22:08:53.752738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:28.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580416+0000","last_change":"2024-09-15T22:06:35.134606+0000","last_active":"2024-09-15T22:08:53.580416+0000","last_peered":"2024-09-15T22:08:53.580416+0000","last_clean":"2024-09-15T22:08:53.580416+0000","last_became_active":"2024-09-15T22:06:35.134263+0000","last_became_peered":"2024-09-15T22:06:35.134263+0000","last_unstale":"2024-09-15T22:08:53.580416+0000","last_undegraded":"2024-09-15T22:08:53.580416+0000","last_fullsized":"2024-09-15T22:08:53.580416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:45.008935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750806+0000","last_change":"2024-09-15T22:06:37.177556+0000","last_active":"2024-09-15T22:08:53.750806+0000","last_peered":"2024-09-15T22:08:53.750806+0000","last_clean":"2024-09-15T22:08:53.750806+0000","last_became_active":"2024-09-15T22:06:37.177161+0000","last_became_peered":"2024-09-15T22:06:37.177161+0000","last_unstale":"2024-09-15T22:08:53.750806+0000","last_undegraded":"2024-09-15T22:08:53.750806+0000","last_fullsized":"2024-09-15T22:08:53.750806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:30.192728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753347+0000","last_change":"2024-09-15T22:06:44.659097+0000","last_active":"2024-09-15T22:08:53.753347+0000","last_peered":"2024-09-15T22:08:53.753347+0000","last_clean":"2024-09-15T22:08:53.753347+0000","last_became_active":"2024-09-15T22:06:44.658626+0000","last_became_peered":"2024-09-15T22:06:44.658626+0000","last_unstale":"2024-09-15T22:08:53.753347+0000","last_undegraded":"2024-09-15T22:08:53.753347+0000","last_fullsized":"2024-09-15T22:08:53.753347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:01.308004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580430+0000","last_change":"2024-09-15T22:06:35.136326+0000","last_active":"2024-09-15T22:08:53.580430+0000","last_peered":"2024-09-15T22:08:53.580430+0000","last_clean":"2024-09-15T22:08:53.580430+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.580430+0000","last_undegraded":"2024-09-15T22:08:53.580430+0000","last_fullsized":"2024-09-15T22:08:53.580430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:28.675832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490507+0000","last_change":"2024-09-15T22:06:37.192939+0000","last_active":"2024-09-15T22:08:53.490507+0000","last_peered":"2024-09-15T22:08:53.490507+0000","last_clean":"2024-09-15T22:08:53.490507+0000","last_became_active":"2024-09-15T22:06:37.192616+0000","last_became_peered":"2024-09-15T22:06:37.192616+0000","last_unstale":"2024-09-15T22:08:53.490507+0000","last_undegraded":"2024-09-15T22:08:53.490507+0000","last_fullsized":"2024-09-15T22:08:53.490507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:03.615203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752853+0000","last_change":"2024-09-15T22:06:44.654741+0000","last_active":"2024-09-15T22:08:53.752853+0000","last_peered":"2024-09-15T22:08:53.752853+0000","last_clean":"2024-09-15T22:08:53.752853+0000","last_became_active":"2024-09-15T22:06:44.654362+0000","last_became_peered":"2024-09-15T22:06:44.654362+0000","last_unstale":"2024-09-15T22:08:53.752853+0000","last_undegraded":"2024-09-15T22:08:53.752853+0000","last_fullsized":"2024-09-15T22:08:53.752853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:59.269948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490328+0000","last_change":"2024-09-15T22:06:37.186385+0000","last_active":"2024-09-15T22:08:53.490328+0000","last_peered":"2024-09-15T22:08:53.490328+0000","last_clean":"2024-09-15T22:08:53.490328+0000","last_became_active":"2024-09-15T22:06:37.185588+0000","last_became_peered":"2024-09-15T22:06:37.185588+0000","last_unstale":"2024-09-15T22:08:53.490328+0000","last_undegraded":"2024-09-15T22:08:53.490328+0000","last_fullsized":"2024-09-15T22:08:53.490328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:22.523083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580272+0000","last_change":"2024-09-15T22:06:35.137383+0000","last_active":"2024-09-15T22:08:53.580272+0000","last_peered":"2024-09-15T22:08:53.580272+0000","last_clean":"2024-09-15T22:08:53.580272+0000","last_became_active":"2024-09-15T22:06:35.137022+0000","last_became_peered":"2024-09-15T22:06:35.137022+0000","last_unstale":"2024-09-15T22:08:53.580272+0000","last_undegraded":"2024-09-15T22:08:53.580272+0000","last_fullsized":"2024-09-15T22:08:53.580272+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.863155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753002+0000","last_change":"2024-09-15T22:06:44.653807+0000","last_active":"2024-09-15T22:08:53.753002+0000","last_peered":"2024-09-15T22:08:53.753002+0000","last_clean":"2024-09-15T22:08:53.753002+0000","last_became_active":"2024-09-15T22:06:44.653406+0000","last_became_peered":"2024-09-15T22:06:44.653406+0000","last_unstale":"2024-09-15T22:08:53.753002+0000","last_undegraded":"2024-09-15T22:08:53.753002+0000","last_fullsized":"2024-09-15T22:08:53.753002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:12.018208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582122+0000","last_change":"2024-09-15T22:06:37.186899+0000","last_active":"2024-09-15T22:08:53.582122+0000","last_peered":"2024-09-15T22:08:53.582122+0000","last_clean":"2024-09-15T22:08:53.582122+0000","last_became_active":"2024-09-15T22:06:37.183782+0000","last_became_peered":"2024-09-15T22:06:37.183782+0000","last_unstale":"2024-09-15T22:08:53.582122+0000","last_undegraded":"2024-09-15T22:08:53.582122+0000","last_fullsized":"2024-09-15T22:08:53.582122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:39.269186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928379+0000","last_change":"2024-09-15T22:06:44.652686+0000","last_active":"2024-09-15T22:08:53.928379+0000","last_peered":"2024-09-15T22:08:53.928379+0000","last_clean":"2024-09-15T22:08:53.928379+0000","last_became_active":"2024-09-15T22:06:44.652306+0000","last_became_peered":"2024-09-15T22:06:44.652306+0000","last_unstale":"2024-09-15T22:08:53.928379+0000","last_undegraded":"2024-09-15T22:08:53.928379+0000","last_fullsized":"2024-09-15T22:08:53.928379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753037+0000","last_change":"2024-09-15T22:06:44.655424+0000","last_active":"2024-09-15T22:08:53.753037+0000","last_peered":"2024-09-15T22:08:53.753037+0000","last_clean":"2024-09-15T22:08:53.753037+0000","last_became_active":"2024-09-15T22:06:44.654959+0000","last_became_peered":"2024-09-15T22:06:44.654959+0000","last_unstale":"2024-09-15T22:08:53.753037+0000","last_undegraded":"2024-09-15T22:08:53.753037+0000","last_fullsized":"2024-09-15T22:08:53.753037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:07.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928379+0000","last_change":"2024-09-15T22:06:37.188424+0000","last_active":"2024-09-15T22:08:53.928379+0000","last_peered":"2024-09-15T22:08:53.928379+0000","last_clean":"2024-09-15T22:08:53.928379+0000","last_became_active":"2024-09-15T22:06:37.188082+0000","last_became_peered":"2024-09-15T22:06:37.188082+0000","last_unstale":"2024-09-15T22:08:53.928379+0000","last_undegraded":"2024-09-15T22:08:53.928379+0000","last_fullsized":"2024-09-15T22:08:53.928379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:07.159341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581279+0000","last_change":"2024-09-15T22:06:35.134237+0000","last_active":"2024-09-15T22:08:53.581279+0000","last_peered":"2024-09-15T22:08:53.581279+0000","last_clean":"2024-09-15T22:08:53.581279+0000","last_became_active":"2024-09-15T22:06:35.133885+0000","last_became_peered":"2024-09-15T22:06:35.133885+0000","last_unstale":"2024-09-15T22:08:53.581279+0000","last_undegraded":"2024-09-15T22:08:53.581279+0000","last_fullsized":"2024-09-15T22:08:53.581279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:07:35.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489075+0000","last_change":"2024-09-15T22:06:44.664578+0000","last_active":"2024-09-15T22:08:53.489075+0000","last_peered":"2024-09-15T22:08:53.489075+0000","last_clean":"2024-09-15T22:08:53.489075+0000","last_became_active":"2024-09-15T22:06:44.664170+0000","last_became_peered":"2024-09-15T22:06:44.664170+0000","last_unstale":"2024-09-15T22:08:53.489075+0000","last_undegraded":"2024-09-15T22:08:53.489075+0000","last_fullsized":"2024-09-15T22:08:53.489075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:41.443816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580024+0000","last_change":"2024-09-15T22:06:37.186107+0000","last_active":"2024-09-15T22:08:53.580024+0000","last_peered":"2024-09-15T22:08:53.580024+0000","last_clean":"2024-09-15T22:08:53.580024+0000","last_became_active":"2024-09-15T22:06:37.185757+0000","last_became_peered":"2024-09-15T22:06:37.185757+0000","last_unstale":"2024-09-15T22:08:53.580024+0000","last_undegraded":"2024-09-15T22:08:53.580024+0000","last_fullsized":"2024-09-15T22:08:53.580024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:14.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750623+0000","last_change":"2024-09-15T22:06:35.356281+0000","last_active":"2024-09-15T22:08:53.750623+0000","last_peered":"2024-09-15T22:08:53.750623+0000","last_clean":"2024-09-15T22:08:53.750623+0000","last_became_active":"2024-09-15T22:06:35.355922+0000","last_became_peered":"2024-09-15T22:06:35.355922+0000","last_unstale":"2024-09-15T22:08:53.750623+0000","last_undegraded":"2024-09-15T22:08:53.750623+0000","last_fullsized":"2024-09-15T22:08:53.750623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:06.107462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753010+0000","last_change":"2024-09-15T22:06:44.659230+0000","last_active":"2024-09-15T22:08:53.753010+0000","last_peered":"2024-09-15T22:08:53.753010+0000","last_clean":"2024-09-15T22:08:53.753010+0000","last_became_active":"2024-09-15T22:06:44.658649+0000","last_became_peered":"2024-09-15T22:06:44.658649+0000","last_unstale":"2024-09-15T22:08:53.753010+0000","last_undegraded":"2024-09-15T22:08:53.753010+0000","last_fullsized":"2024-09-15T22:08:53.753010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:14:21.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929164+0000","last_change":"2024-09-15T22:06:44.649394+0000","last_active":"2024-09-15T22:08:53.929164+0000","last_peered":"2024-09-15T22:08:53.929164+0000","last_clean":"2024-09-15T22:08:53.929164+0000","last_became_active":"2024-09-15T22:06:44.649043+0000","last_became_peered":"2024-09-15T22:06:44.649043+0000","last_unstale":"2024-09-15T22:08:53.929164+0000","last_undegraded":"2024-09-15T22:08:53.929164+0000","last_fullsized":"2024-09-15T22:08:53.929164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:56:28.335869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753504+0000","last_change":"2024-09-15T22:06:35.265970+0000","last_active":"2024-09-15T22:08:53.753504+0000","last_peered":"2024-09-15T22:08:53.753504+0000","last_clean":"2024-09-15T22:08:53.753504+0000","last_became_active":"2024-09-15T22:06:35.265628+0000","last_became_peered":"2024-09-15T22:06:35.265628+0000","last_unstale":"2024-09-15T22:08:53.753504+0000","last_undegraded":"2024-09-15T22:08:53.753504+0000","last_fullsized":"2024-09-15T22:08:53.753504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:41.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750664+0000","last_change":"2024-09-15T22:06:44.649652+0000","last_active":"2024-09-15T22:08:53.750664+0000","last_peered":"2024-09-15T22:08:53.750664+0000","last_clean":"2024-09-15T22:08:53.750664+0000","last_became_active":"2024-09-15T22:06:44.649112+0000","last_became_peered":"2024-09-15T22:06:44.649112+0000","last_unstale":"2024-09-15T22:08:53.750664+0000","last_undegraded":"2024-09-15T22:08:53.750664+0000","last_fullsized":"2024-09-15T22:08:53.750664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:53.789320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583103+0000","last_change":"2024-09-15T22:06:37.190701+0000","last_active":"2024-09-15T22:08:53.583103+0000","last_peered":"2024-09-15T22:08:53.583103+0000","last_clean":"2024-09-15T22:08:53.583103+0000","last_became_active":"2024-09-15T22:06:37.190430+0000","last_became_peered":"2024-09-15T22:06:37.190430+0000","last_unstale":"2024-09-15T22:08:53.583103+0000","last_undegraded":"2024-09-15T22:08:53.583103+0000","last_fullsized":"2024-09-15T22:08:53.583103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:14.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'51","reported_seq":111,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.701060+0000","last_change":"2024-09-15T22:06:35.133899+0000","last_active":"2024-09-15T22:08:53.701060+0000","last_peered":"2024-09-15T22:08:53.701060+0000","last_clean":"2024-09-15T22:08:53.701060+0000","last_became_active":"2024-09-15T22:06:35.133556+0000","last_became_peered":"2024-09-15T22:06:35.133556+0000","last_unstale":"2024-09-15T22:08:53.701060+0000","last_undegraded":"2024-09-15T22:08:53.701060+0000","last_fullsized":"2024-09-15T22:08:53.701060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:01.692416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580427+0000","last_change":"2024-09-15T22:06:44.655280+0000","last_active":"2024-09-15T22:08:53.580427+0000","last_peered":"2024-09-15T22:08:53.580427+0000","last_clean":"2024-09-15T22:08:53.580427+0000","last_became_active":"2024-09-15T22:06:44.654883+0000","last_became_peered":"2024-09-15T22:06:44.654883+0000","last_unstale":"2024-09-15T22:08:53.580427+0000","last_undegraded":"2024-09-15T22:08:53.580427+0000","last_fullsized":"2024-09-15T22:08:53.580427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.074035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889279+0000","last_change":"2024-09-15T22:06:37.182006+0000","last_active":"2024-09-15T22:08:53.889279+0000","last_peered":"2024-09-15T22:08:53.889279+0000","last_clean":"2024-09-15T22:08:53.889279+0000","last_became_active":"2024-09-15T22:06:37.181693+0000","last_became_peered":"2024-09-15T22:06:37.181693+0000","last_unstale":"2024-09-15T22:08:53.889279+0000","last_undegraded":"2024-09-15T22:08:53.889279+0000","last_fullsized":"2024-09-15T22:08:53.889279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:23.483930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580471+0000","last_change":"2024-09-15T22:06:35.355854+0000","last_active":"2024-09-15T22:08:53.580471+0000","last_peered":"2024-09-15T22:08:53.580471+0000","last_clean":"2024-09-15T22:08:53.580471+0000","last_became_active":"2024-09-15T22:06:35.355486+0000","last_became_peered":"2024-09-15T22:06:35.355486+0000","last_unstale":"2024-09-15T22:08:53.580471+0000","last_undegraded":"2024-09-15T22:08:53.580471+0000","last_fullsized":"2024-09-15T22:08:53.580471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:36.708476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928367+0000","last_change":"2024-09-15T22:06:44.660523+0000","last_active":"2024-09-15T22:08:53.928367+0000","last_peered":"2024-09-15T22:08:53.928367+0000","last_clean":"2024-09-15T22:08:53.928367+0000","last_became_active":"2024-09-15T22:06:44.660113+0000","last_became_peered":"2024-09-15T22:06:44.660113+0000","last_unstale":"2024-09-15T22:08:53.928367+0000","last_undegraded":"2024-09-15T22:08:53.928367+0000","last_fullsized":"2024-09-15T22:08:53.928367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:42.164558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589575+0000","last_change":"2024-09-15T22:06:37.183649+0000","last_active":"2024-09-15T22:08:53.589575+0000","last_peered":"2024-09-15T22:08:53.589575+0000","last_clean":"2024-09-15T22:08:53.589575+0000","last_became_active":"2024-09-15T22:06:37.183296+0000","last_became_peered":"2024-09-15T22:06:37.183296+0000","last_unstale":"2024-09-15T22:08:53.589575+0000","last_undegraded":"2024-09-15T22:08:53.589575+0000","last_fullsized":"2024-09-15T22:08:53.589575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:22.010288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488860+0000","last_change":"2024-09-15T22:06:35.355068+0000","last_active":"2024-09-15T22:08:53.488860+0000","last_peered":"2024-09-15T22:08:53.488860+0000","last_clean":"2024-09-15T22:08:53.488860+0000","last_became_active":"2024-09-15T22:06:35.354728+0000","last_became_peered":"2024-09-15T22:06:35.354728+0000","last_unstale":"2024-09-15T22:08:53.488860+0000","last_undegraded":"2024-09-15T22:08:53.488860+0000","last_fullsized":"2024-09-15T22:08:53.488860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:35.479290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580098+0000","last_change":"2024-09-15T22:06:44.653849+0000","last_active":"2024-09-15T22:08:53.580098+0000","last_peered":"2024-09-15T22:08:53.580098+0000","last_clean":"2024-09-15T22:08:53.580098+0000","last_became_active":"2024-09-15T22:06:44.653419+0000","last_became_peered":"2024-09-15T22:06:44.653419+0000","last_unstale":"2024-09-15T22:08:53.580098+0000","last_undegraded":"2024-09-15T22:08:53.580098+0000","last_fullsized":"2024-09-15T22:08:53.580098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:04.131553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589446+0000","last_change":"2024-09-15T22:06:37.191438+0000","last_active":"2024-09-15T22:08:53.589446+0000","last_peered":"2024-09-15T22:08:53.589446+0000","last_clean":"2024-09-15T22:08:53.589446+0000","last_became_active":"2024-09-15T22:06:37.191088+0000","last_became_peered":"2024-09-15T22:06:37.191088+0000","last_unstale":"2024-09-15T22:08:53.589446+0000","last_undegraded":"2024-09-15T22:08:53.589446+0000","last_fullsized":"2024-09-15T22:08:53.589446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:46:36.078872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490414+0000","last_change":"2024-09-15T22:06:35.135866+0000","last_active":"2024-09-15T22:08:53.490414+0000","last_peered":"2024-09-15T22:08:53.490414+0000","last_clean":"2024-09-15T22:08:53.490414+0000","last_became_active":"2024-09-15T22:06:35.135524+0000","last_became_peered":"2024-09-15T22:06:35.135524+0000","last_unstale":"2024-09-15T22:08:53.490414+0000","last_undegraded":"2024-09-15T22:08:53.490414+0000","last_fullsized":"2024-09-15T22:08:53.490414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888980+0000","last_change":"2024-09-15T22:06:44.649943+0000","last_active":"2024-09-15T22:08:53.888980+0000","last_peered":"2024-09-15T22:08:53.888980+0000","last_clean":"2024-09-15T22:08:53.888980+0000","last_became_active":"2024-09-15T22:06:44.649579+0000","last_became_peered":"2024-09-15T22:06:44.649579+0000","last_unstale":"2024-09-15T22:08:53.888980+0000","last_undegraded":"2024-09-15T22:08:53.888980+0000","last_fullsized":"2024-09-15T22:08:53.888980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:18.390608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489996+0000","last_change":"2024-09-15T22:06:37.189988+0000","last_active":"2024-09-15T22:08:53.489996+0000","last_peered":"2024-09-15T22:08:53.489996+0000","last_clean":"2024-09-15T22:08:53.489996+0000","last_became_active":"2024-09-15T22:06:37.189319+0000","last_became_peered":"2024-09-15T22:06:37.189319+0000","last_unstale":"2024-09-15T22:08:53.489996+0000","last_undegraded":"2024-09-15T22:08:53.489996+0000","last_fullsized":"2024-09-15T22:08:53.489996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:08.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889084+0000","last_change":"2024-09-15T22:06:35.355371+0000","last_active":"2024-09-15T22:08:53.889084+0000","last_peered":"2024-09-15T22:08:53.889084+0000","last_clean":"2024-09-15T22:08:53.889084+0000","last_became_active":"2024-09-15T22:06:35.354687+0000","last_became_peered":"2024-09-15T22:06:35.354687+0000","last_unstale":"2024-09-15T22:08:53.889084+0000","last_undegraded":"2024-09-15T22:08:53.889084+0000","last_fullsized":"2024-09-15T22:08:53.889084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:27.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889532+0000","last_change":"2024-09-15T22:06:44.657929+0000","last_active":"2024-09-15T22:08:53.889532+0000","last_peered":"2024-09-15T22:08:53.889532+0000","last_clean":"2024-09-15T22:08:53.889532+0000","last_became_active":"2024-09-15T22:06:44.657519+0000","last_became_peered":"2024-09-15T22:06:44.657519+0000","last_unstale":"2024-09-15T22:08:53.889532+0000","last_undegraded":"2024-09-15T22:08:53.889532+0000","last_fullsized":"2024-09-15T22:08:53.889532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:12.313877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888747+0000","last_change":"2024-09-15T22:06:37.193288+0000","last_active":"2024-09-15T22:08:53.888747+0000","last_peered":"2024-09-15T22:08:53.888747+0000","last_clean":"2024-09-15T22:08:53.888747+0000","last_became_active":"2024-09-15T22:06:37.193007+0000","last_became_peered":"2024-09-15T22:06:37.193007+0000","last_unstale":"2024-09-15T22:08:53.888747+0000","last_undegraded":"2024-09-15T22:08:53.888747+0000","last_fullsized":"2024-09-15T22:08:53.888747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:45.956206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490201+0000","last_change":"2024-09-15T22:06:35.138726+0000","last_active":"2024-09-15T22:08:53.490201+0000","last_peered":"2024-09-15T22:08:53.490201+0000","last_clean":"2024-09-15T22:08:53.490201+0000","last_became_active":"2024-09-15T22:06:35.138390+0000","last_became_peered":"2024-09-15T22:06:35.138390+0000","last_unstale":"2024-09-15T22:08:53.490201+0000","last_undegraded":"2024-09-15T22:08:53.490201+0000","last_fullsized":"2024-09-15T22:08:53.490201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:51.336171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490388+0000","last_change":"2024-09-15T22:07:46.219720+0000","last_active":"2024-09-15T22:08:53.490388+0000","last_peered":"2024-09-15T22:08:53.490388+0000","last_clean":"2024-09-15T22:08:53.490388+0000","last_became_active":"2024-09-15T22:07:46.219303+0000","last_became_peered":"2024-09-15T22:07:46.219303+0000","last_unstale":"2024-09-15T22:08:53.490388+0000","last_undegraded":"2024-09-15T22:08:53.490388+0000","last_fullsized":"2024-09-15T22:08:53.490388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:03.596795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581603+0000","last_change":"2024-09-15T22:06:37.176478+0000","last_active":"2024-09-15T22:08:53.581603+0000","last_peered":"2024-09-15T22:08:53.581603+0000","last_clean":"2024-09-15T22:08:53.581603+0000","last_became_active":"2024-09-15T22:06:37.176171+0000","last_became_peered":"2024-09-15T22:06:37.176171+0000","last_unstale":"2024-09-15T22:08:53.581603+0000","last_undegraded":"2024-09-15T22:08:53.581603+0000","last_fullsized":"2024-09-15T22:08:53.581603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753268+0000","last_change":"2024-09-15T22:06:35.134651+0000","last_active":"2024-09-15T22:08:53.753268+0000","last_peered":"2024-09-15T22:08:53.753268+0000","last_clean":"2024-09-15T22:08:53.753268+0000","last_became_active":"2024-09-15T22:06:35.134185+0000","last_became_peered":"2024-09-15T22:06:35.134185+0000","last_unstale":"2024-09-15T22:08:53.753268+0000","last_undegraded":"2024-09-15T22:08:53.753268+0000","last_fullsized":"2024-09-15T22:08:53.753268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:56.385046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":358,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490393+0000","last_change":"2024-09-15T22:05:23.048068+0000","last_active":"2024-09-15T22:08:53.490393+0000","last_peered":"2024-09-15T22:08:53.490393+0000","last_clean":"2024-09-15T22:08:53.490393+0000","last_became_active":"2024-09-15T22:05:23.047692+0000","last_became_peered":"2024-09-15T22:05:23.047692+0000","last_unstale":"2024-09-15T22:08:53.490393+0000","last_undegraded":"2024-09-15T22:08:53.490393+0000","last_fullsized":"2024-09-15T22:08:53.490393+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_clean_scrub_stamp":"2024-09-15T22:01:57.477629+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490625+0000","last_change":"2024-09-15T22:06:44.657213+0000","last_active":"2024-09-15T22:08:53.490625+0000","last_peered":"2024-09-15T22:08:53.490625+0000","last_clean":"2024-09-15T22:08:53.490625+0000","last_became_active":"2024-09-15T22:06:44.656815+0000","last_became_peered":"2024-09-15T22:06:44.656815+0000","last_unstale":"2024-09-15T22:08:53.490625+0000","last_undegraded":"2024-09-15T22:08:53.490625+0000","last_fullsized":"2024-09-15T22:08:53.490625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:26.525992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490755+0000","last_change":"2024-09-15T22:06:37.184467+0000","last_active":"2024-09-15T22:08:53.490755+0000","last_peered":"2024-09-15T22:08:53.490755+0000","last_clean":"2024-09-15T22:08:53.490755+0000","last_became_active":"2024-09-15T22:06:37.183864+0000","last_became_peered":"2024-09-15T22:06:37.183864+0000","last_unstale":"2024-09-15T22:08:53.490755+0000","last_undegraded":"2024-09-15T22:08:53.490755+0000","last_fullsized":"2024-09-15T22:08:53.490755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:35.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750757+0000","last_change":"2024-09-15T22:06:44.651594+0000","last_active":"2024-09-15T22:08:53.750757+0000","last_peered":"2024-09-15T22:08:53.750757+0000","last_clean":"2024-09-15T22:08:53.750757+0000","last_became_active":"2024-09-15T22:06:44.650393+0000","last_became_peered":"2024-09-15T22:06:44.650393+0000","last_unstale":"2024-09-15T22:08:53.750757+0000","last_undegraded":"2024-09-15T22:08:53.750757+0000","last_fullsized":"2024-09-15T22:08:53.750757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:13.599744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750561+0000","last_change":"2024-09-15T22:06:44.660789+0000","last_active":"2024-09-15T22:08:53.750561+0000","last_peered":"2024-09-15T22:08:53.750561+0000","last_clean":"2024-09-15T22:08:53.750561+0000","last_became_active":"2024-09-15T22:06:44.660331+0000","last_became_peered":"2024-09-15T22:06:44.660331+0000","last_unstale":"2024-09-15T22:08:53.750561+0000","last_undegraded":"2024-09-15T22:08:53.750561+0000","last_fullsized":"2024-09-15T22:08:53.750561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:04:45.562795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490532+0000","last_change":"2024-09-15T22:06:37.185697+0000","last_active":"2024-09-15T22:08:53.490532+0000","last_peered":"2024-09-15T22:08:53.490532+0000","last_clean":"2024-09-15T22:08:53.490532+0000","last_became_active":"2024-09-15T22:06:37.185207+0000","last_became_peered":"2024-09-15T22:06:37.185207+0000","last_unstale":"2024-09-15T22:08:53.490532+0000","last_undegraded":"2024-09-15T22:08:53.490532+0000","last_fullsized":"2024-09-15T22:08:53.490532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:39.005273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":84,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.699454+0000","last_change":"2024-09-15T22:06:35.135176+0000","last_active":"2024-09-15T22:08:53.699454+0000","last_peered":"2024-09-15T22:08:53.699454+0000","last_clean":"2024-09-15T22:08:53.699454+0000","last_became_active":"2024-09-15T22:06:35.134649+0000","last_became_peered":"2024-09-15T22:06:35.134649+0000","last_unstale":"2024-09-15T22:08:53.699454+0000","last_undegraded":"2024-09-15T22:08:53.699454+0000","last_fullsized":"2024-09-15T22:08:53.699454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:33.780162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928727+0000","last_change":"2024-09-15T22:06:44.656972+0000","last_active":"2024-09-15T22:08:53.928727+0000","last_peered":"2024-09-15T22:08:53.928727+0000","last_clean":"2024-09-15T22:08:53.928727+0000","last_became_active":"2024-09-15T22:06:44.656528+0000","last_became_peered":"2024-09-15T22:06:44.656528+0000","last_unstale":"2024-09-15T22:08:53.928727+0000","last_undegraded":"2024-09-15T22:08:53.928727+0000","last_fullsized":"2024-09-15T22:08:53.928727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:09.164596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490782+0000","last_change":"2024-09-15T22:06:37.186118+0000","last_active":"2024-09-15T22:08:53.490782+0000","last_peered":"2024-09-15T22:08:53.490782+0000","last_clean":"2024-09-15T22:08:53.490782+0000","last_became_active":"2024-09-15T22:06:37.185118+0000","last_became_peered":"2024-09-15T22:06:37.185118+0000","last_unstale":"2024-09-15T22:08:53.490782+0000","last_undegraded":"2024-09-15T22:08:53.490782+0000","last_fullsized":"2024-09-15T22:08:53.490782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:53.357045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889478+0000","last_change":"2024-09-15T22:06:35.133341+0000","last_active":"2024-09-15T22:08:53.889478+0000","last_peered":"2024-09-15T22:08:53.889478+0000","last_clean":"2024-09-15T22:08:53.889478+0000","last_became_active":"2024-09-15T22:06:35.132961+0000","last_became_peered":"2024-09-15T22:06:35.132961+0000","last_unstale":"2024-09-15T22:08:53.889478+0000","last_undegraded":"2024-09-15T22:08:53.889478+0000","last_fullsized":"2024-09-15T22:08:53.889478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:39.579470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489819+0000","last_change":"2024-09-15T22:06:44.659148+0000","last_active":"2024-09-15T22:08:53.489819+0000","last_peered":"2024-09-15T22:08:53.489819+0000","last_clean":"2024-09-15T22:08:53.489819+0000","last_became_active":"2024-09-15T22:06:44.658750+0000","last_became_peered":"2024-09-15T22:06:44.658750+0000","last_unstale":"2024-09-15T22:08:53.489819+0000","last_undegraded":"2024-09-15T22:08:53.489819+0000","last_fullsized":"2024-09-15T22:08:53.489819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:48.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928598+0000","last_change":"2024-09-15T22:06:37.194066+0000","last_active":"2024-09-15T22:08:53.928598+0000","last_peered":"2024-09-15T22:08:53.928598+0000","last_clean":"2024-09-15T22:08:53.928598+0000","last_became_active":"2024-09-15T22:06:37.193738+0000","last_became_peered":"2024-09-15T22:06:37.193738+0000","last_unstale":"2024-09-15T22:08:53.928598+0000","last_undegraded":"2024-09-15T22:08:53.928598+0000","last_fullsized":"2024-09-15T22:08:53.928598+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.107188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580476+0000","last_change":"2024-09-15T22:06:35.136470+0000","last_active":"2024-09-15T22:08:53.580476+0000","last_peered":"2024-09-15T22:08:53.580476+0000","last_clean":"2024-09-15T22:08:53.580476+0000","last_became_active":"2024-09-15T22:06:35.135953+0000","last_became_peered":"2024-09-15T22:06:35.135953+0000","last_unstale":"2024-09-15T22:08:53.580476+0000","last_undegraded":"2024-09-15T22:08:53.580476+0000","last_fullsized":"2024-09-15T22:08:53.580476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:44.548566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489963+0000","last_change":"2024-09-15T22:06:44.653682+0000","last_active":"2024-09-15T22:08:53.489963+0000","last_peered":"2024-09-15T22:08:53.489963+0000","last_clean":"2024-09-15T22:08:53.489963+0000","last_became_active":"2024-09-15T22:06:44.653303+0000","last_became_peered":"2024-09-15T22:06:44.653303+0000","last_unstale":"2024-09-15T22:08:53.489963+0000","last_undegraded":"2024-09-15T22:08:53.489963+0000","last_fullsized":"2024-09-15T22:08:53.489963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:07.847971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489970+0000","last_change":"2024-09-15T22:06:44.658436+0000","last_active":"2024-09-15T22:08:53.489970+0000","last_peered":"2024-09-15T22:08:53.489970+0000","last_clean":"2024-09-15T22:08:53.489970+0000","last_became_active":"2024-09-15T22:06:44.658037+0000","last_became_peered":"2024-09-15T22:06:44.658037+0000","last_unstale":"2024-09-15T22:08:53.489970+0000","last_undegraded":"2024-09-15T22:08:53.489970+0000","last_fullsized":"2024-09-15T22:08:53.489970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:34:20.369249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580102+0000","last_change":"2024-09-15T22:06:35.358018+0000","last_active":"2024-09-15T22:08:53.580102+0000","last_peered":"2024-09-15T22:08:53.580102+0000","last_clean":"2024-09-15T22:08:53.580102+0000","last_became_active":"2024-09-15T22:06:35.357666+0000","last_became_peered":"2024-09-15T22:06:35.357666+0000","last_unstale":"2024-09-15T22:08:53.580102+0000","last_undegraded":"2024-09-15T22:08:53.580102+0000","last_fullsized":"2024-09-15T22:08:53.580102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:34.089113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889340+0000","last_change":"2024-09-15T22:06:44.656807+0000","last_active":"2024-09-15T22:08:53.889340+0000","last_peered":"2024-09-15T22:08:53.889340+0000","last_clean":"2024-09-15T22:08:53.889340+0000","last_became_active":"2024-09-15T22:06:44.656409+0000","last_became_peered":"2024-09-15T22:06:44.656409+0000","last_unstale":"2024-09-15T22:08:53.889340+0000","last_undegraded":"2024-09-15T22:08:53.889340+0000","last_fullsized":"2024-09-15T22:08:53.889340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490123+0000","last_change":"2024-09-15T22:06:37.180615+0000","last_active":"2024-09-15T22:08:53.490123+0000","last_peered":"2024-09-15T22:08:53.490123+0000","last_clean":"2024-09-15T22:08:53.490123+0000","last_became_active":"2024-09-15T22:06:37.180306+0000","last_became_peered":"2024-09-15T22:06:37.180306+0000","last_unstale":"2024-09-15T22:08:53.490123+0000","last_undegraded":"2024-09-15T22:08:53.490123+0000","last_fullsized":"2024-09-15T22:08:53.490123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:31.145128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889942+0000","last_change":"2024-09-15T22:06:35.136192+0000","last_active":"2024-09-15T22:08:53.889942+0000","last_peered":"2024-09-15T22:08:53.889942+0000","last_clean":"2024-09-15T22:08:53.889942+0000","last_became_active":"2024-09-15T22:06:35.135608+0000","last_became_peered":"2024-09-15T22:06:35.135608+0000","last_unstale":"2024-09-15T22:08:53.889942+0000","last_undegraded":"2024-09-15T22:08:53.889942+0000","last_fullsized":"2024-09-15T22:08:53.889942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:48.472717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489151+0000","last_change":"2024-09-15T22:06:44.664763+0000","last_active":"2024-09-15T22:08:53.489151+0000","last_peered":"2024-09-15T22:08:53.489151+0000","last_clean":"2024-09-15T22:08:53.489151+0000","last_became_active":"2024-09-15T22:06:44.664410+0000","last_became_peered":"2024-09-15T22:06:44.664410+0000","last_unstale":"2024-09-15T22:08:53.489151+0000","last_undegraded":"2024-09-15T22:08:53.489151+0000","last_fullsized":"2024-09-15T22:08:53.489151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:14.364387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490299+0000","last_change":"2024-09-15T22:06:44.652747+0000","last_active":"2024-09-15T22:08:53.490299+0000","last_peered":"2024-09-15T22:08:53.490299+0000","last_clean":"2024-09-15T22:08:53.490299+0000","last_became_active":"2024-09-15T22:06:44.652368+0000","last_became_peered":"2024-09-15T22:06:44.652368+0000","last_unstale":"2024-09-15T22:08:53.490299+0000","last_undegraded":"2024-09-15T22:08:53.490299+0000","last_fullsized":"2024-09-15T22:08:53.490299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:51.047549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490109+0000","last_change":"2024-09-15T22:06:35.136311+0000","last_active":"2024-09-15T22:08:53.490109+0000","last_peered":"2024-09-15T22:08:53.490109+0000","last_clean":"2024-09-15T22:08:53.490109+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.490109+0000","last_undegraded":"2024-09-15T22:08:53.490109+0000","last_fullsized":"2024-09-15T22:08:53.490109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:36:07.138056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928366+0000","last_change":"2024-09-15T22:06:44.653747+0000","last_active":"2024-09-15T22:08:53.928366+0000","last_peered":"2024-09-15T22:08:53.928366+0000","last_clean":"2024-09-15T22:08:53.928366+0000","last_became_active":"2024-09-15T22:06:44.653357+0000","last_became_peered":"2024-09-15T22:06:44.653357+0000","last_unstale":"2024-09-15T22:08:53.928366+0000","last_undegraded":"2024-09-15T22:08:53.928366+0000","last_fullsized":"2024-09-15T22:08:53.928366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.113924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888246+0000","last_change":"2024-09-15T22:06:37.178308+0000","last_active":"2024-09-15T22:08:53.888246+0000","last_peered":"2024-09-15T22:08:53.888246+0000","last_clean":"2024-09-15T22:08:53.888246+0000","last_became_active":"2024-09-15T22:06:37.177920+0000","last_became_peered":"2024-09-15T22:06:37.177920+0000","last_unstale":"2024-09-15T22:08:53.888246+0000","last_undegraded":"2024-09-15T22:08:53.888246+0000","last_fullsized":"2024-09-15T22:08:53.888246+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750846+0000","last_change":"2024-09-15T22:06:35.136132+0000","last_active":"2024-09-15T22:08:53.750846+0000","last_peered":"2024-09-15T22:08:53.750846+0000","last_clean":"2024-09-15T22:08:53.750846+0000","last_became_active":"2024-09-15T22:06:35.135789+0000","last_became_peered":"2024-09-15T22:06:35.135789+0000","last_unstale":"2024-09-15T22:08:53.750846+0000","last_undegraded":"2024-09-15T22:08:53.750846+0000","last_fullsized":"2024-09-15T22:08:53.750846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:01.692885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490410+0000","last_change":"2024-09-15T22:06:44.658920+0000","last_active":"2024-09-15T22:08:53.490410+0000","last_peered":"2024-09-15T22:08:53.490410+0000","last_clean":"2024-09-15T22:08:53.490410+0000","last_became_active":"2024-09-15T22:06:44.658434+0000","last_became_peered":"2024-09-15T22:06:44.658434+0000","last_unstale":"2024-09-15T22:08:53.490410+0000","last_undegraded":"2024-09-15T22:08:53.490410+0000","last_fullsized":"2024-09-15T22:08:53.490410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:37.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580330+0000","last_change":"2024-09-15T22:06:37.184958+0000","last_active":"2024-09-15T22:08:53.580330+0000","last_peered":"2024-09-15T22:08:53.580330+0000","last_clean":"2024-09-15T22:08:53.580330+0000","last_became_active":"2024-09-15T22:06:37.184648+0000","last_became_peered":"2024-09-15T22:06:37.184648+0000","last_unstale":"2024-09-15T22:08:53.580330+0000","last_undegraded":"2024-09-15T22:08:53.580330+0000","last_fullsized":"2024-09-15T22:08:53.580330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:28.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":289,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.890809+0000","last_change":"2024-09-15T22:06:35.138533+0000","last_active":"2024-09-15T22:08:53.890809+0000","last_peered":"2024-09-15T22:08:53.890809+0000","last_clean":"2024-09-15T22:08:53.890809+0000","last_became_active":"2024-09-15T22:06:35.138142+0000","last_became_peered":"2024-09-15T22:06:35.138142+0000","last_unstale":"2024-09-15T22:08:53.890809+0000","last_undegraded":"2024-09-15T22:08:53.890809+0000","last_fullsized":"2024-09-15T22:08:53.890809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:32.950036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68792,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":213,"num_read_kb":174,"num_write":41,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589290+0000","last_change":"2024-09-15T22:07:46.218222+0000","last_active":"2024-09-15T22:08:53.589290+0000","last_peered":"2024-09-15T22:08:53.589290+0000","last_clean":"2024-09-15T22:08:53.589290+0000","last_became_active":"2024-09-15T22:07:46.217903+0000","last_became_peered":"2024-09-15T22:07:46.217903+0000","last_unstale":"2024-09-15T22:08:53.589290+0000","last_undegraded":"2024-09-15T22:08:53.589290+0000","last_fullsized":"2024-09-15T22:08:53.589290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:17.730288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750879+0000","last_change":"2024-09-15T22:06:37.194297+0000","last_active":"2024-09-15T22:08:53.750879+0000","last_peered":"2024-09-15T22:08:53.750879+0000","last_clean":"2024-09-15T22:08:53.750879+0000","last_became_active":"2024-09-15T22:06:37.193986+0000","last_became_peered":"2024-09-15T22:06:37.193986+0000","last_unstale":"2024-09-15T22:08:53.750879+0000","last_undegraded":"2024-09-15T22:08:53.750879+0000","last_fullsized":"2024-09-15T22:08:53.750879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:27.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490573+0000","last_change":"2024-09-15T22:06:35.356537+0000","last_active":"2024-09-15T22:08:53.490573+0000","last_peered":"2024-09-15T22:08:53.490573+0000","last_clean":"2024-09-15T22:08:53.490573+0000","last_became_active":"2024-09-15T22:06:35.355997+0000","last_became_peered":"2024-09-15T22:06:35.355997+0000","last_unstale":"2024-09-15T22:08:53.490573+0000","last_undegraded":"2024-09-15T22:08:53.490573+0000","last_fullsized":"2024-09-15T22:08:53.490573+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:11.563083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753107+0000","last_change":"2024-09-15T22:06:44.654579+0000","last_active":"2024-09-15T22:08:53.753107+0000","last_peered":"2024-09-15T22:08:53.753107+0000","last_clean":"2024-09-15T22:08:53.753107+0000","last_became_active":"2024-09-15T22:06:44.654187+0000","last_became_peered":"2024-09-15T22:06:44.654187+0000","last_unstale":"2024-09-15T22:08:53.753107+0000","last_undegraded":"2024-09-15T22:08:53.753107+0000","last_fullsized":"2024-09-15T22:08:53.753107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:15.617528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929412+0000","last_change":"2024-09-15T22:06:44.661533+0000","last_active":"2024-09-15T22:08:53.929412+0000","last_peered":"2024-09-15T22:08:53.929412+0000","last_clean":"2024-09-15T22:08:53.929412+0000","last_became_active":"2024-09-15T22:06:44.661193+0000","last_became_peered":"2024-09-15T22:06:44.661193+0000","last_unstale":"2024-09-15T22:08:53.929412+0000","last_undegraded":"2024-09-15T22:08:53.929412+0000","last_fullsized":"2024-09-15T22:08:53.929412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:06.559720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490695+0000","last_change":"2024-09-15T22:06:35.140578+0000","last_active":"2024-09-15T22:08:53.490695+0000","last_peered":"2024-09-15T22:08:53.490695+0000","last_clean":"2024-09-15T22:08:53.490695+0000","last_became_active":"2024-09-15T22:06:35.139818+0000","last_became_peered":"2024-09-15T22:06:35.139818+0000","last_unstale":"2024-09-15T22:08:53.490695+0000","last_undegraded":"2024-09-15T22:08:53.490695+0000","last_fullsized":"2024-09-15T22:08:53.490695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:59.806964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488473+0000","last_change":"2024-09-15T22:06:44.654755+0000","last_active":"2024-09-15T22:08:53.488473+0000","last_peered":"2024-09-15T22:08:53.488473+0000","last_clean":"2024-09-15T22:08:53.488473+0000","last_became_active":"2024-09-15T22:06:44.654390+0000","last_became_peered":"2024-09-15T22:06:44.654390+0000","last_unstale":"2024-09-15T22:08:53.488473+0000","last_undegraded":"2024-09-15T22:08:53.488473+0000","last_fullsized":"2024-09-15T22:08:53.488473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:35.781878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751028+0000","last_change":"2024-09-15T22:06:37.182457+0000","last_active":"2024-09-15T22:08:53.751028+0000","last_peered":"2024-09-15T22:08:53.751028+0000","last_clean":"2024-09-15T22:08:53.751028+0000","last_became_active":"2024-09-15T22:06:37.182106+0000","last_became_peered":"2024-09-15T22:06:37.182106+0000","last_unstale":"2024-09-15T22:08:53.751028+0000","last_undegraded":"2024-09-15T22:08:53.751028+0000","last_fullsized":"2024-09-15T22:08:53.751028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:57.560894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":140,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589780+0000","last_change":"2024-09-15T22:06:35.140590+0000","last_active":"2024-09-15T22:08:53.589780+0000","last_peered":"2024-09-15T22:08:53.589780+0000","last_clean":"2024-09-15T22:08:53.589780+0000","last_became_active":"2024-09-15T22:06:35.140198+0000","last_became_peered":"2024-09-15T22:06:35.140198+0000","last_unstale":"2024-09-15T22:08:53.589780+0000","last_undegraded":"2024-09-15T22:08:53.589780+0000","last_fullsized":"2024-09-15T22:08:53.589780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:46.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580877+0000","last_change":"2024-09-15T22:06:44.660188+0000","last_active":"2024-09-15T22:08:53.580877+0000","last_peered":"2024-09-15T22:08:53.580877+0000","last_clean":"2024-09-15T22:08:53.580877+0000","last_became_active":"2024-09-15T22:06:44.659808+0000","last_became_peered":"2024-09-15T22:06:44.659808+0000","last_unstale":"2024-09-15T22:08:53.580877+0000","last_undegraded":"2024-09-15T22:08:53.580877+0000","last_fullsized":"2024-09-15T22:08:53.580877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:39.473765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929017+0000","last_change":"2024-09-15T22:06:44.647967+0000","last_active":"2024-09-15T22:08:53.929017+0000","last_peered":"2024-09-15T22:08:53.929017+0000","last_clean":"2024-09-15T22:08:53.929017+0000","last_became_active":"2024-09-15T22:06:44.647608+0000","last_became_peered":"2024-09-15T22:06:44.647608+0000","last_unstale":"2024-09-15T22:08:53.929017+0000","last_undegraded":"2024-09-15T22:08:53.929017+0000","last_fullsized":"2024-09-15T22:08:53.929017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:06.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488584+0000","last_change":"2024-09-15T22:06:37.187020+0000","last_active":"2024-09-15T22:08:53.488584+0000","last_peered":"2024-09-15T22:08:53.488584+0000","last_clean":"2024-09-15T22:08:53.488584+0000","last_became_active":"2024-09-15T22:06:37.186564+0000","last_became_peered":"2024-09-15T22:06:37.186564+0000","last_unstale":"2024-09-15T22:08:53.488584+0000","last_undegraded":"2024-09-15T22:08:53.488584+0000","last_fullsized":"2024-09-15T22:08:53.488584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:20.586763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490131+0000","last_change":"2024-09-15T22:06:35.135126+0000","last_active":"2024-09-15T22:08:53.490131+0000","last_peered":"2024-09-15T22:08:53.490131+0000","last_clean":"2024-09-15T22:08:53.490131+0000","last_became_active":"2024-09-15T22:06:35.134783+0000","last_became_peered":"2024-09-15T22:06:35.134783+0000","last_unstale":"2024-09-15T22:08:53.490131+0000","last_undegraded":"2024-09-15T22:08:53.490131+0000","last_fullsized":"2024-09-15T22:08:53.490131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:31.910988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928656+0000","last_change":"2024-09-15T22:06:44.656981+0000","last_active":"2024-09-15T22:08:53.928656+0000","last_peered":"2024-09-15T22:08:53.928656+0000","last_clean":"2024-09-15T22:08:53.928656+0000","last_became_active":"2024-09-15T22:06:44.656477+0000","last_became_peered":"2024-09-15T22:06:44.656477+0000","last_unstale":"2024-09-15T22:08:53.928656+0000","last_undegraded":"2024-09-15T22:08:53.928656+0000","last_fullsized":"2024-09-15T22:08:53.928656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:06.153490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580796+0000","last_change":"2024-09-15T22:06:37.183034+0000","last_active":"2024-09-15T22:08:53.580796+0000","last_peered":"2024-09-15T22:08:53.580796+0000","last_clean":"2024-09-15T22:08:53.580796+0000","last_became_active":"2024-09-15T22:06:37.182734+0000","last_became_peered":"2024-09-15T22:06:37.182734+0000","last_unstale":"2024-09-15T22:08:53.580796+0000","last_undegraded":"2024-09-15T22:08:53.580796+0000","last_fullsized":"2024-09-15T22:08:53.580796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:19.352000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888909+0000","last_change":"2024-09-15T22:06:35.134954+0000","last_active":"2024-09-15T22:08:53.888909+0000","last_peered":"2024-09-15T22:08:53.888909+0000","last_clean":"2024-09-15T22:08:53.888909+0000","last_became_active":"2024-09-15T22:06:35.134583+0000","last_became_peered":"2024-09-15T22:06:35.134583+0000","last_unstale":"2024-09-15T22:08:53.888909+0000","last_undegraded":"2024-09-15T22:08:53.888909+0000","last_fullsized":"2024-09-15T22:08:53.888909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:56.082123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589453+0000","last_change":"2024-09-15T22:06:35.139229+0000","last_active":"2024-09-15T22:08:53.589453+0000","last_peered":"2024-09-15T22:08:53.589453+0000","last_clean":"2024-09-15T22:08:53.589453+0000","last_became_active":"2024-09-15T22:06:35.138834+0000","last_became_peered":"2024-09-15T22:06:35.138834+0000","last_unstale":"2024-09-15T22:08:53.589453+0000","last_undegraded":"2024-09-15T22:08:53.589453+0000","last_fullsized":"2024-09-15T22:08:53.589453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:05.877404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490250+0000","last_change":"2024-09-15T22:06:37.192212+0000","last_active":"2024-09-15T22:08:53.490250+0000","last_peered":"2024-09-15T22:08:53.490250+0000","last_clean":"2024-09-15T22:08:53.490250+0000","last_became_active":"2024-09-15T22:06:37.191872+0000","last_became_peered":"2024-09-15T22:06:37.191872+0000","last_unstale":"2024-09-15T22:08:53.490250+0000","last_undegraded":"2024-09-15T22:08:53.490250+0000","last_fullsized":"2024-09-15T22:08:53.490250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:32.845794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750771+0000","last_change":"2024-09-15T22:07:45.968398+0000","last_active":"2024-09-15T22:08:53.750771+0000","last_peered":"2024-09-15T22:08:53.750771+0000","last_clean":"2024-09-15T22:08:53.750771+0000","last_became_active":"2024-09-15T22:07:45.967912+0000","last_became_peered":"2024-09-15T22:07:45.967912+0000","last_unstale":"2024-09-15T22:08:53.750771+0000","last_undegraded":"2024-09-15T22:08:53.750771+0000","last_fullsized":"2024-09-15T22:08:53.750771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:59.909290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'9","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.890548+0000","last_change":"2024-09-15T22:06:35.133846+0000","last_active":"2024-09-15T22:08:53.890548+0000","last_peered":"2024-09-15T22:08:53.890548+0000","last_clean":"2024-09-15T22:08:53.890548+0000","last_became_active":"2024-09-15T22:06:35.133516+0000","last_became_peered":"2024-09-15T22:06:35.133516+0000","last_unstale":"2024-09-15T22:08:53.890548+0000","last_undegraded":"2024-09-15T22:08:53.890548+0000","last_fullsized":"2024-09-15T22:08:53.890548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:29.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580118+0000","last_change":"2024-09-15T22:06:37.182904+0000","last_active":"2024-09-15T22:08:53.580118+0000","last_peered":"2024-09-15T22:08:53.580118+0000","last_clean":"2024-09-15T22:08:53.580118+0000","last_became_active":"2024-09-15T22:06:37.182563+0000","last_became_peered":"2024-09-15T22:06:37.182563+0000","last_unstale":"2024-09-15T22:08:53.580118+0000","last_undegraded":"2024-09-15T22:08:53.580118+0000","last_fullsized":"2024-09-15T22:08:53.580118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750976+0000","last_change":"2024-09-15T22:06:44.660997+0000","last_active":"2024-09-15T22:08:53.750976+0000","last_peered":"2024-09-15T22:08:53.750976+0000","last_clean":"2024-09-15T22:08:53.750976+0000","last_became_active":"2024-09-15T22:06:44.660577+0000","last_became_peered":"2024-09-15T22:06:44.660577+0000","last_unstale":"2024-09-15T22:08:53.750976+0000","last_undegraded":"2024-09-15T22:08:53.750976+0000","last_fullsized":"2024-09-15T22:08:53.750976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:56.006891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928243+0000","last_change":"2024-09-15T22:06:37.182077+0000","last_active":"2024-09-15T22:08:53.928243+0000","last_peered":"2024-09-15T22:08:53.928243+0000","last_clean":"2024-09-15T22:08:53.928243+0000","last_became_active":"2024-09-15T22:06:37.181622+0000","last_became_peered":"2024-09-15T22:06:37.181622+0000","last_unstale":"2024-09-15T22:08:53.928243+0000","last_undegraded":"2024-09-15T22:08:53.928243+0000","last_fullsized":"2024-09-15T22:08:53.928243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:07.586406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750818+0000","last_change":"2024-09-15T22:06:44.653845+0000","last_active":"2024-09-15T22:08:53.750818+0000","last_peered":"2024-09-15T22:08:53.750818+0000","last_clean":"2024-09-15T22:08:53.750818+0000","last_became_active":"2024-09-15T22:06:44.653483+0000","last_became_peered":"2024-09-15T22:06:44.653483+0000","last_unstale":"2024-09-15T22:08:53.750818+0000","last_undegraded":"2024-09-15T22:08:53.750818+0000","last_fullsized":"2024-09-15T22:08:53.750818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:08:07.822279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750649+0000","last_change":"2024-09-15T22:06:37.187476+0000","last_active":"2024-09-15T22:08:53.750649+0000","last_peered":"2024-09-15T22:08:53.750649+0000","last_clean":"2024-09-15T22:08:53.750649+0000","last_became_active":"2024-09-15T22:06:37.187047+0000","last_became_peered":"2024-09-15T22:06:37.187047+0000","last_unstale":"2024-09-15T22:08:53.750649+0000","last_undegraded":"2024-09-15T22:08:53.750649+0000","last_fullsized":"2024-09-15T22:08:53.750649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.220834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928765+0000","last_change":"2024-09-15T22:06:37.190583+0000","last_active":"2024-09-15T22:08:53.928765+0000","last_peered":"2024-09-15T22:08:53.928765+0000","last_clean":"2024-09-15T22:08:53.928765+0000","last_became_active":"2024-09-15T22:06:37.190212+0000","last_became_peered":"2024-09-15T22:06:37.190212+0000","last_unstale":"2024-09-15T22:08:53.928765+0000","last_undegraded":"2024-09-15T22:08:53.928765+0000","last_fullsized":"2024-09-15T22:08:53.928765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:48.574188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889153+0000","last_change":"2024-09-15T22:06:37.192508+0000","last_active":"2024-09-15T22:08:53.889153+0000","last_peered":"2024-09-15T22:08:53.889153+0000","last_clean":"2024-09-15T22:08:53.889153+0000","last_became_active":"2024-09-15T22:06:37.192117+0000","last_became_peered":"2024-09-15T22:06:37.192117+0000","last_unstale":"2024-09-15T22:08:53.889153+0000","last_undegraded":"2024-09-15T22:08:53.889153+0000","last_fullsized":"2024-09-15T22:08:53.889153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:41.355966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928601+0000","last_change":"2024-09-15T22:06:37.183377+0000","last_active":"2024-09-15T22:08:53.928601+0000","last_peered":"2024-09-15T22:08:53.928601+0000","last_clean":"2024-09-15T22:08:53.928601+0000","last_became_active":"2024-09-15T22:06:37.183072+0000","last_became_peered":"2024-09-15T22:06:37.183072+0000","last_unstale":"2024-09-15T22:08:53.928601+0000","last_undegraded":"2024-09-15T22:08:53.928601+0000","last_fullsized":"2024-09-15T22:08:53.928601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.318702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751128+0000","last_change":"2024-09-15T22:06:37.185348+0000","last_active":"2024-09-15T22:08:53.751128+0000","last_peered":"2024-09-15T22:08:53.751128+0000","last_clean":"2024-09-15T22:08:53.751128+0000","last_became_active":"2024-09-15T22:06:37.184934+0000","last_became_peered":"2024-09-15T22:06:37.184934+0000","last_unstale":"2024-09-15T22:08:53.751128+0000","last_undegraded":"2024-09-15T22:08:53.751128+0000","last_fullsized":"2024-09-15T22:08:53.751128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:57.564860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582987+0000","last_change":"2024-09-15T22:06:37.192152+0000","last_active":"2024-09-15T22:08:53.582987+0000","last_peered":"2024-09-15T22:08:53.582987+0000","last_clean":"2024-09-15T22:08:53.582987+0000","last_became_active":"2024-09-15T22:06:37.191836+0000","last_became_peered":"2024-09-15T22:06:37.191836+0000","last_unstale":"2024-09-15T22:08:53.582987+0000","last_undegraded":"2024-09-15T22:08:53.582987+0000","last_fullsized":"2024-09-15T22:08:53.582987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:31.781110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489966+0000","last_change":"2024-09-15T22:06:37.179996+0000","last_active":"2024-09-15T22:08:53.489966+0000","last_peered":"2024-09-15T22:08:53.489966+0000","last_clean":"2024-09-15T22:08:53.489966+0000","last_became_active":"2024-09-15T22:06:37.179646+0000","last_became_peered":"2024-09-15T22:06:37.179646+0000","last_unstale":"2024-09-15T22:08:53.489966+0000","last_undegraded":"2024-09-15T22:08:53.489966+0000","last_fullsized":"2024-09-15T22:08:53.489966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:35.240468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581098+0000","last_change":"2024-09-15T22:06:37.182992+0000","last_active":"2024-09-15T22:08:53.581098+0000","last_peered":"2024-09-15T22:08:53.581098+0000","last_clean":"2024-09-15T22:08:53.581098+0000","last_became_active":"2024-09-15T22:06:37.182676+0000","last_became_peered":"2024-09-15T22:06:37.182676+0000","last_unstale":"2024-09-15T22:08:53.581098+0000","last_undegraded":"2024-09-15T22:08:53.581098+0000","last_fullsized":"2024-09-15T22:08:53.581098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:41.524600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750731+0000","last_change":"2024-09-15T22:06:37.180107+0000","last_active":"2024-09-15T22:08:53.750731+0000","last_peered":"2024-09-15T22:08:53.750731+0000","last_clean":"2024-09-15T22:08:53.750731+0000","last_became_active":"2024-09-15T22:06:37.179785+0000","last_became_peered":"2024-09-15T22:06:37.179785+0000","last_unstale":"2024-09-15T22:08:53.750731+0000","last_undegraded":"2024-09-15T22:08:53.750731+0000","last_fullsized":"2024-09-15T22:08:53.750731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:46.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888926+0000","last_change":"2024-09-15T22:06:37.177406+0000","last_active":"2024-09-15T22:08:53.888926+0000","last_peered":"2024-09-15T22:08:53.888926+0000","last_clean":"2024-09-15T22:08:53.888926+0000","last_became_active":"2024-09-15T22:06:37.177093+0000","last_became_peered":"2024-09-15T22:06:37.177093+0000","last_unstale":"2024-09-15T22:08:53.888926+0000","last_undegraded":"2024-09-15T22:08:53.888926+0000","last_fullsized":"2024-09-15T22:08:53.888926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:34.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928419+0000","last_change":"2024-09-15T22:06:37.192314+0000","last_active":"2024-09-15T22:08:53.928419+0000","last_peered":"2024-09-15T22:08:53.928419+0000","last_clean":"2024-09-15T22:08:53.928419+0000","last_became_active":"2024-09-15T22:06:37.191958+0000","last_became_peered":"2024-09-15T22:06:37.191958+0000","last_unstale":"2024-09-15T22:08:53.928419+0000","last_undegraded":"2024-09-15T22:08:53.928419+0000","last_fullsized":"2024-09-15T22:08:53.928419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:47.763742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751178+0000","last_change":"2024-09-15T22:06:37.190212+0000","last_active":"2024-09-15T22:08:53.751178+0000","last_peered":"2024-09-15T22:08:53.751178+0000","last_clean":"2024-09-15T22:08:53.751178+0000","last_became_active":"2024-09-15T22:06:37.189881+0000","last_became_peered":"2024-09-15T22:06:37.189881+0000","last_unstale":"2024-09-15T22:08:53.751178+0000","last_undegraded":"2024-09-15T22:08:53.751178+0000","last_fullsized":"2024-09-15T22:08:53.751178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:59.943574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488963+0000","last_change":"2024-09-15T22:06:37.187967+0000","last_active":"2024-09-15T22:08:53.488963+0000","last_peered":"2024-09-15T22:08:53.488963+0000","last_clean":"2024-09-15T22:08:53.488963+0000","last_became_active":"2024-09-15T22:06:37.187596+0000","last_became_peered":"2024-09-15T22:06:37.187596+0000","last_unstale":"2024-09-15T22:08:53.488963+0000","last_undegraded":"2024-09-15T22:08:53.488963+0000","last_fullsized":"2024-09-15T22:08:53.488963+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:00.042784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928757+0000","last_change":"2024-09-15T22:06:37.189434+0000","last_active":"2024-09-15T22:08:53.928757+0000","last_peered":"2024-09-15T22:08:53.928757+0000","last_clean":"2024-09-15T22:08:53.928757+0000","last_became_active":"2024-09-15T22:06:37.189053+0000","last_became_peered":"2024-09-15T22:06:37.189053+0000","last_unstale":"2024-09-15T22:08:53.928757+0000","last_undegraded":"2024-09-15T22:08:53.928757+0000","last_fullsized":"2024-09-15T22:08:53.928757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:51.640495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750486+0000","last_change":"2024-09-15T22:06:37.179229+0000","last_active":"2024-09-15T22:08:53.750486+0000","last_peered":"2024-09-15T22:08:53.750486+0000","last_clean":"2024-09-15T22:08:53.750486+0000","last_became_active":"2024-09-15T22:06:37.178828+0000","last_became_peered":"2024-09-15T22:06:37.178828+0000","last_unstale":"2024-09-15T22:08:53.750486+0000","last_undegraded":"2024-09-15T22:08:53.750486+0000","last_fullsized":"2024-09-15T22:08:53.750486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:25.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928489+0000","last_change":"2024-09-15T22:06:37.193095+0000","last_active":"2024-09-15T22:08:53.928489+0000","last_peered":"2024-09-15T22:08:53.928489+0000","last_clean":"2024-09-15T22:08:53.928489+0000","last_became_active":"2024-09-15T22:06:37.192801+0000","last_became_peered":"2024-09-15T22:06:37.192801+0000","last_unstale":"2024-09-15T22:08:53.928489+0000","last_undegraded":"2024-09-15T22:08:53.928489+0000","last_fullsized":"2024-09-15T22:08:53.928489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:12.521510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928614+0000","last_change":"2024-09-15T22:06:37.182332+0000","last_active":"2024-09-15T22:08:53.928614+0000","last_peered":"2024-09-15T22:08:53.928614+0000","last_clean":"2024-09-15T22:08:53.928614+0000","last_became_active":"2024-09-15T22:06:37.182001+0000","last_became_peered":"2024-09-15T22:06:37.182001+0000","last_unstale":"2024-09-15T22:08:53.928614+0000","last_undegraded":"2024-09-15T22:08:53.928614+0000","last_fullsized":"2024-09-15T22:08:53.928614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:52.230452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888644+0000","last_change":"2024-09-15T22:06:37.180584+0000","last_active":"2024-09-15T22:08:53.888644+0000","last_peered":"2024-09-15T22:08:53.888644+0000","last_clean":"2024-09-15T22:08:53.888644+0000","last_became_active":"2024-09-15T22:06:37.179341+0000","last_became_peered":"2024-09-15T22:06:37.179341+0000","last_unstale":"2024-09-15T22:08:53.888644+0000","last_undegraded":"2024-09-15T22:08:53.888644+0000","last_fullsized":"2024-09-15T22:08:53.888644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.610835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753439+0000","last_change":"2024-09-15T22:06:37.189500+0000","last_active":"2024-09-15T22:08:53.753439+0000","last_peered":"2024-09-15T22:08:53.753439+0000","last_clean":"2024-09-15T22:08:53.753439+0000","last_became_active":"2024-09-15T22:06:37.189137+0000","last_became_peered":"2024-09-15T22:06:37.189137+0000","last_unstale":"2024-09-15T22:08:53.753439+0000","last_undegraded":"2024-09-15T22:08:53.753439+0000","last_fullsized":"2024-09-15T22:08:53.753439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:27.199854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490697+0000","last_change":"2024-09-15T22:06:37.182618+0000","last_active":"2024-09-15T22:08:53.490697+0000","last_peered":"2024-09-15T22:08:53.490697+0000","last_clean":"2024-09-15T22:08:53.490697+0000","last_became_active":"2024-09-15T22:06:37.182257+0000","last_became_peered":"2024-09-15T22:06:37.182257+0000","last_unstale":"2024-09-15T22:08:53.490697+0000","last_undegraded":"2024-09-15T22:08:53.490697+0000","last_fullsized":"2024-09-15T22:08:53.490697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:17.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589823+0000","last_change":"2024-09-15T22:06:37.193036+0000","last_active":"2024-09-15T22:08:53.589823+0000","last_peered":"2024-09-15T22:08:53.589823+0000","last_clean":"2024-09-15T22:08:53.589823+0000","last_became_active":"2024-09-15T22:06:37.192668+0000","last_became_peered":"2024-09-15T22:06:37.192668+0000","last_unstale":"2024-09-15T22:08:53.589823+0000","last_undegraded":"2024-09-15T22:08:53.589823+0000","last_fullsized":"2024-09-15T22:08:53.589823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:05.436045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489086+0000","last_change":"2024-09-15T22:06:44.659503+0000","last_active":"2024-09-15T22:08:53.489086+0000","last_peered":"2024-09-15T22:08:53.489086+0000","last_clean":"2024-09-15T22:08:53.489086+0000","last_became_active":"2024-09-15T22:06:44.659155+0000","last_became_peered":"2024-09-15T22:06:44.659155+0000","last_unstale":"2024-09-15T22:08:53.489086+0000","last_undegraded":"2024-09-15T22:08:53.489086+0000","last_fullsized":"2024-09-15T22:08:53.489086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:41:04.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580648+0000","last_change":"2024-09-15T22:06:37.178386+0000","last_active":"2024-09-15T22:08:53.580648+0000","last_peered":"2024-09-15T22:08:53.580648+0000","last_clean":"2024-09-15T22:08:53.580648+0000","last_became_active":"2024-09-15T22:06:37.178083+0000","last_became_peered":"2024-09-15T22:06:37.178083+0000","last_unstale":"2024-09-15T22:08:53.580648+0000","last_undegraded":"2024-09-15T22:08:53.580648+0000","last_fullsized":"2024-09-15T22:08:53.580648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:35.350698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490190+0000","last_change":"2024-09-15T22:06:37.186741+0000","last_active":"2024-09-15T22:08:53.490190+0000","last_peered":"2024-09-15T22:08:53.490190+0000","last_clean":"2024-09-15T22:08:53.490190+0000","last_became_active":"2024-09-15T22:06:37.185740+0000","last_became_peered":"2024-09-15T22:06:37.185740+0000","last_unstale":"2024-09-15T22:08:53.490190+0000","last_undegraded":"2024-09-15T22:08:53.490190+0000","last_fullsized":"2024-09-15T22:08:53.490190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.762407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589411+0000","last_change":"2024-09-15T22:06:37.189552+0000","last_active":"2024-09-15T22:08:53.589411+0000","last_peered":"2024-09-15T22:08:53.589411+0000","last_clean":"2024-09-15T22:08:53.589411+0000","last_became_active":"2024-09-15T22:06:37.189197+0000","last_became_peered":"2024-09-15T22:06:37.189197+0000","last_unstale":"2024-09-15T22:08:53.589411+0000","last_undegraded":"2024-09-15T22:08:53.589411+0000","last_fullsized":"2024-09-15T22:08:53.589411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:54.268720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580092+0000","last_change":"2024-09-15T22:06:37.186917+0000","last_active":"2024-09-15T22:08:53.580092+0000","last_peered":"2024-09-15T22:08:53.580092+0000","last_clean":"2024-09-15T22:08:53.580092+0000","last_became_active":"2024-09-15T22:06:37.186571+0000","last_became_peered":"2024-09-15T22:06:37.186571+0000","last_unstale":"2024-09-15T22:08:53.580092+0000","last_undegraded":"2024-09-15T22:08:53.580092+0000","last_fullsized":"2024-09-15T22:08:53.580092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:02.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753272+0000","last_change":"2024-09-15T22:06:37.190960+0000","last_active":"2024-09-15T22:08:53.753272+0000","last_peered":"2024-09-15T22:08:53.753272+0000","last_clean":"2024-09-15T22:08:53.753272+0000","last_became_active":"2024-09-15T22:06:37.190650+0000","last_became_peered":"2024-09-15T22:06:37.190650+0000","last_unstale":"2024-09-15T22:08:53.753272+0000","last_undegraded":"2024-09-15T22:08:53.753272+0000","last_fullsized":"2024-09-15T22:08:53.753272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:34.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928796+0000","last_change":"2024-09-15T22:06:37.186718+0000","last_active":"2024-09-15T22:08:53.928796+0000","last_peered":"2024-09-15T22:08:53.928796+0000","last_clean":"2024-09-15T22:08:53.928796+0000","last_became_active":"2024-09-15T22:06:37.186424+0000","last_became_peered":"2024-09-15T22:06:37.186424+0000","last_unstale":"2024-09-15T22:08:53.928796+0000","last_undegraded":"2024-09-15T22:08:53.928796+0000","last_fullsized":"2024-09-15T22:08:53.928796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:53.266007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929020+0000","last_change":"2024-09-15T22:06:37.185522+0000","last_active":"2024-09-15T22:08:53.929020+0000","last_peered":"2024-09-15T22:08:53.929020+0000","last_clean":"2024-09-15T22:08:53.929020+0000","last_became_active":"2024-09-15T22:06:37.185192+0000","last_became_peered":"2024-09-15T22:06:37.185192+0000","last_unstale":"2024-09-15T22:08:53.929020+0000","last_undegraded":"2024-09-15T22:08:53.929020+0000","last_fullsized":"2024-09-15T22:08:53.929020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:12.868324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928738+0000","last_change":"2024-09-15T22:06:37.189553+0000","last_active":"2024-09-15T22:08:53.928738+0000","last_peered":"2024-09-15T22:08:53.928738+0000","last_clean":"2024-09-15T22:08:53.928738+0000","last_became_active":"2024-09-15T22:06:37.189270+0000","last_became_peered":"2024-09-15T22:06:37.189270+0000","last_unstale":"2024-09-15T22:08:53.928738+0000","last_undegraded":"2024-09-15T22:08:53.928738+0000","last_fullsized":"2024-09-15T22:08:53.928738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:34.157077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599380,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":730,"num_read_kb":478,"num_write":302,"num_write_kb":1445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1798140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579945,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":502844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808878,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32996,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712156,"statfs":{"total":95995035648,"available":95961247744,"internally_reserved":0,"allocated":1855488,"data_stored":1300760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005107,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1437696,"data_stored":909721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1429504,"data_stored":904963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495677,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1630208,"data_stored":1109427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659201,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":507585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888135,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37080,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708072,"statfs":{"total":95995035648,"available":95957065728,"internally_reserved":0,"allocated":1777664,"data_stored":1236270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018956,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":513744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247889,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1032192,"data_stored":507779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444119,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1114112,"data_stored":578290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1097728,"data_stored":576869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1044480,"data_stored":508973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":202897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:08:58.796 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-15T22:08:58.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T22:08:58.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:58 smithi099 ceph-mon[34892]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:08:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:58 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:58 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:58 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:58 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:58 smithi099 ceph-mon[34892]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-15T22:08:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:58 smithi129 ceph-mon[35314]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:08:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:58 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:08:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:58 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:08:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:58 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:08:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:58 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:08:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:58 smithi129 ceph-mon[35314]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-09-15T22:08:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.933+0000 7f8b3868d640 1 -- 172.21.15.5:0/3148192374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30102cf0 msgr2=0x7f8b301030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.933+0000 7f8b3868d640 1 --2- 172.21.15.5:0/3148192374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30102cf0 0x7f8b301030d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009900 tx=0x7f8b2802a480 comp rx=0 tx=0).stop 2024-09-15T22:08:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.933+0000 7f8b3868d640 1 -- 172.21.15.5:0/3148192374 shutdown_connections 2024-09-15T22:08:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.933+0000 7f8b3868d640 1 --2- 172.21.15.5:0/3148192374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b3010c0c0 0x7f8b3010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.933+0000 7f8b3868d640 1 --2- 172.21.15.5:0/3148192374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b30103610 0x7f8b3010baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.933+0000 7f8b3868d640 1 --2- 172.21.15.5:0/3148192374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30102cf0 0x7f8b301030d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.933+0000 7f8b3868d640 1 -- 172.21.15.5:0/3148192374 >> 172.21.15.5:0/3148192374 conn(0x7f8b30077a10 msgr2=0x7f8b300fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.933+0000 7f8b3868d640 1 -- 172.21.15.5:0/3148192374 shutdown_connections 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.933+0000 7f8b3868d640 1 -- 172.21.15.5:0/3148192374 wait complete. 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.933+0000 7f8b3868d640 1 Processor -- start 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.933+0000 7f8b3868d640 1 -- start start 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b3868d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b30102cf0 0x7f8b301983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b3868d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30103610 0x7f8b301988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b3868d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b3010c0c0 0x7f8b3019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b3868d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30111ec0 con 0x7f8b30103610 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b3868d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30111d40 con 0x7f8b3010c0c0 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b3868d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8b30112040 con 0x7f8b30102cf0 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b35c28640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30103610 0x7f8b301988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b35c28640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30103610 0x7f8b301988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56100/0 (socket says 172.21.15.5:56100) 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b35c28640 1 -- 172.21.15.5:0/4101935879 learned_addr learned my addr 172.21.15.5:0/4101935879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b36429640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b30102cf0 0x7f8b301983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b36c2a640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b3010c0c0 0x7f8b3019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b35c28640 1 -- 172.21.15.5:0/4101935879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b30102cf0 msgr2=0x7f8b301983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b35c28640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b30102cf0 0x7f8b301983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b35c28640 1 -- 172.21.15.5:0/4101935879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b3010c0c0 msgr2=0x7f8b3019cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b35c28640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b3010c0c0 0x7f8b3019cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b35c28640 1 -- 172.21.15.5:0/4101935879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3019d3a0 con 0x7f8b30103610 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b35c28640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30103610 0x7f8b301988f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400d950 tx=0x7f8b2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b2f7fe640 1 -- 172.21.15.5:0/4101935879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b24017070 con 0x7f8b30103610 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b2f7fe640 1 -- 172.21.15.5:0/4101935879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b24009c30 con 0x7f8b30103610 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b2f7fe640 1 -- 172.21.15.5:0/4101935879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2400f6b0 con 0x7f8b30103610 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b3868d640 1 -- 172.21.15.5:0/4101935879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3019d690 con 0x7f8b30103610 2024-09-15T22:08:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.934+0000 7f8b3868d640 1 -- 172.21.15.5:0/4101935879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b301a4f20 con 0x7f8b30103610 2024-09-15T22:08:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.935+0000 7f8b3868d640 1 -- 172.21.15.5:0/4101935879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b301030d0 con 0x7f8b30103610 2024-09-15T22:08:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.936+0000 7f8b2f7fe640 1 -- 172.21.15.5:0/4101935879 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f8b240109d0 con 0x7f8b30103610 2024-09-15T22:08:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.937+0000 7f8b2f7fe640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b180a34e0 0x7f8b180a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.937+0000 7f8b36429640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b180a34e0 0x7f8b180a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.937+0000 7f8b2f7fe640 1 -- 172.21.15.5:0/4101935879 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f8b240cff10 con 0x7f8b30103610 2024-09-15T22:08:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.938+0000 7f8b36429640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b180a34e0 0x7f8b180a59a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009230 tx=0x7f8b280058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:58.941+0000 7f8b2f7fe640 1 -- 172.21.15.5:0/4101935879 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b24009de0 con 0x7f8b30103610 2024-09-15T22:08:59.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.116+0000 7f8b3868d640 1 -- 172.21.15.5:0/4101935879 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8b30062630 con 0x7f8b180a34e0 2024-09-15T22:08:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.124+0000 7f8b2f7fe640 1 -- 172.21.15.5:0/4101935879 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339383 (secure 0 0 0) 0x7f8b30062630 con 0x7f8b180a34e0 2024-09-15T22:08:59.128 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:08:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-15T22:08:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 -- 172.21.15.5:0/4101935879 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b180a34e0 msgr2=0x7f8b180a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b180a34e0 0x7f8b180a59a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009230 tx=0x7f8b280058d0 comp rx=0 tx=0).stop 2024-09-15T22:08:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 -- 172.21.15.5:0/4101935879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30103610 msgr2=0x7f8b301988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30103610 0x7f8b301988f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400d950 tx=0x7f8b2400de10 comp rx=0 tx=0).stop 2024-09-15T22:08:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 -- 172.21.15.5:0/4101935879 shutdown_connections 2024-09-15T22:08:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b180a34e0 0x7f8b180a59a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:59.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b3010c0c0 0x7f8b3019cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:59.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30103610 0x7f8b301988f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:59.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 --2- 172.21.15.5:0/4101935879 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b30102cf0 0x7f8b301983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:59.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 -- 172.21.15.5:0/4101935879 >> 172.21.15.5:0/4101935879 conn(0x7f8b30077a10 msgr2=0x7f8b300fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:59.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 -- 172.21.15.5:0/4101935879 shutdown_connections 2024-09-15T22:08:59.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.128+0000 7f8b3868d640 1 -- 172.21.15.5:0/4101935879 wait complete. 2024-09-15T22:08:59.140 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-15T22:08:57.501792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1189982,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":866,"num_read_kb":597,"num_write":485,"num_write_kb":4385,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":875,"ondisk_log_size":875,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":393524,"kb_used_data":15124,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124548300,"statfs":{"total":1151940427776,"available":1151537459200,"internally_reserved":0,"allocated":15486976,"data_stored":9157225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":387233083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.989795"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":75,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489253+0000","last_change":"2024-09-15T22:07:46.226073+0000","last_active":"2024-09-15T22:08:53.489253+0000","last_peered":"2024-09-15T22:08:53.489253+0000","last_clean":"2024-09-15T22:08:53.489253+0000","last_became_active":"2024-09-15T22:07:46.219250+0000","last_became_peered":"2024-09-15T22:07:46.219250+0000","last_unstale":"2024-09-15T22:08:53.489253+0000","last_undegraded":"2024-09-15T22:08:53.489253+0000","last_fullsized":"2024-09-15T22:08:53.489253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:56.324959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,4],"acting":[9,11,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582824+0000","last_change":"2024-09-15T22:06:37.184096+0000","last_active":"2024-09-15T22:08:53.582824+0000","last_peered":"2024-09-15T22:08:53.582824+0000","last_clean":"2024-09-15T22:08:53.582824+0000","last_became_active":"2024-09-15T22:06:37.183758+0000","last_became_peered":"2024-09-15T22:06:37.183758+0000","last_unstale":"2024-09-15T22:08:53.582824+0000","last_undegraded":"2024-09-15T22:08:53.582824+0000","last_fullsized":"2024-09-15T22:08:53.582824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:04.210551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750577+0000","last_change":"2024-09-15T22:06:44.653042+0000","last_active":"2024-09-15T22:08:53.750577+0000","last_peered":"2024-09-15T22:08:53.750577+0000","last_clean":"2024-09-15T22:08:53.750577+0000","last_became_active":"2024-09-15T22:06:44.652658+0000","last_became_peered":"2024-09-15T22:06:44.652658+0000","last_unstale":"2024-09-15T22:08:53.750577+0000","last_undegraded":"2024-09-15T22:08:53.750577+0000","last_fullsized":"2024-09-15T22:08:53.750577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:59.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583148+0000","last_change":"2024-09-15T22:06:35.355392+0000","last_active":"2024-09-15T22:08:53.583148+0000","last_peered":"2024-09-15T22:08:53.583148+0000","last_clean":"2024-09-15T22:08:53.583148+0000","last_became_active":"2024-09-15T22:06:35.355042+0000","last_became_peered":"2024-09-15T22:06:35.355042+0000","last_unstale":"2024-09-15T22:08:53.583148+0000","last_undegraded":"2024-09-15T22:08:53.583148+0000","last_fullsized":"2024-09-15T22:08:53.583148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:05.186937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929360+0000","last_change":"2024-09-15T22:06:37.184421+0000","last_active":"2024-09-15T22:08:53.929360+0000","last_peered":"2024-09-15T22:08:53.929360+0000","last_clean":"2024-09-15T22:08:53.929360+0000","last_became_active":"2024-09-15T22:06:37.184092+0000","last_became_peered":"2024-09-15T22:06:37.184092+0000","last_unstale":"2024-09-15T22:08:53.929360+0000","last_undegraded":"2024-09-15T22:08:53.929360+0000","last_fullsized":"2024-09-15T22:08:53.929360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:41.080981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752738+0000","last_change":"2024-09-15T22:07:45.968640+0000","last_active":"2024-09-15T22:08:53.752738+0000","last_peered":"2024-09-15T22:08:53.752738+0000","last_clean":"2024-09-15T22:08:53.752738+0000","last_became_active":"2024-09-15T22:07:45.968320+0000","last_became_peered":"2024-09-15T22:07:45.968320+0000","last_unstale":"2024-09-15T22:08:53.752738+0000","last_undegraded":"2024-09-15T22:08:53.752738+0000","last_fullsized":"2024-09-15T22:08:53.752738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:28.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580416+0000","last_change":"2024-09-15T22:06:35.134606+0000","last_active":"2024-09-15T22:08:53.580416+0000","last_peered":"2024-09-15T22:08:53.580416+0000","last_clean":"2024-09-15T22:08:53.580416+0000","last_became_active":"2024-09-15T22:06:35.134263+0000","last_became_peered":"2024-09-15T22:06:35.134263+0000","last_unstale":"2024-09-15T22:08:53.580416+0000","last_undegraded":"2024-09-15T22:08:53.580416+0000","last_fullsized":"2024-09-15T22:08:53.580416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:45.008935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750806+0000","last_change":"2024-09-15T22:06:37.177556+0000","last_active":"2024-09-15T22:08:53.750806+0000","last_peered":"2024-09-15T22:08:53.750806+0000","last_clean":"2024-09-15T22:08:53.750806+0000","last_became_active":"2024-09-15T22:06:37.177161+0000","last_became_peered":"2024-09-15T22:06:37.177161+0000","last_unstale":"2024-09-15T22:08:53.750806+0000","last_undegraded":"2024-09-15T22:08:53.750806+0000","last_fullsized":"2024-09-15T22:08:53.750806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:30.192728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753347+0000","last_change":"2024-09-15T22:06:44.659097+0000","last_active":"2024-09-15T22:08:53.753347+0000","last_peered":"2024-09-15T22:08:53.753347+0000","last_clean":"2024-09-15T22:08:53.753347+0000","last_became_active":"2024-09-15T22:06:44.658626+0000","last_became_peered":"2024-09-15T22:06:44.658626+0000","last_unstale":"2024-09-15T22:08:53.753347+0000","last_undegraded":"2024-09-15T22:08:53.753347+0000","last_fullsized":"2024-09-15T22:08:53.753347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:01.308004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580430+0000","last_change":"2024-09-15T22:06:35.136326+0000","last_active":"2024-09-15T22:08:53.580430+0000","last_peered":"2024-09-15T22:08:53.580430+0000","last_clean":"2024-09-15T22:08:53.580430+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.580430+0000","last_undegraded":"2024-09-15T22:08:53.580430+0000","last_fullsized":"2024-09-15T22:08:53.580430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:28.675832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490507+0000","last_change":"2024-09-15T22:06:37.192939+0000","last_active":"2024-09-15T22:08:53.490507+0000","last_peered":"2024-09-15T22:08:53.490507+0000","last_clean":"2024-09-15T22:08:53.490507+0000","last_became_active":"2024-09-15T22:06:37.192616+0000","last_became_peered":"2024-09-15T22:06:37.192616+0000","last_unstale":"2024-09-15T22:08:53.490507+0000","last_undegraded":"2024-09-15T22:08:53.490507+0000","last_fullsized":"2024-09-15T22:08:53.490507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:03.615203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752853+0000","last_change":"2024-09-15T22:06:44.654741+0000","last_active":"2024-09-15T22:08:53.752853+0000","last_peered":"2024-09-15T22:08:53.752853+0000","last_clean":"2024-09-15T22:08:53.752853+0000","last_became_active":"2024-09-15T22:06:44.654362+0000","last_became_peered":"2024-09-15T22:06:44.654362+0000","last_unstale":"2024-09-15T22:08:53.752853+0000","last_undegraded":"2024-09-15T22:08:53.752853+0000","last_fullsized":"2024-09-15T22:08:53.752853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:59.269948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490328+0000","last_change":"2024-09-15T22:06:37.186385+0000","last_active":"2024-09-15T22:08:53.490328+0000","last_peered":"2024-09-15T22:08:53.490328+0000","last_clean":"2024-09-15T22:08:53.490328+0000","last_became_active":"2024-09-15T22:06:37.185588+0000","last_became_peered":"2024-09-15T22:06:37.185588+0000","last_unstale":"2024-09-15T22:08:53.490328+0000","last_undegraded":"2024-09-15T22:08:53.490328+0000","last_fullsized":"2024-09-15T22:08:53.490328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:22.523083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580272+0000","last_change":"2024-09-15T22:06:35.137383+0000","last_active":"2024-09-15T22:08:53.580272+0000","last_peered":"2024-09-15T22:08:53.580272+0000","last_clean":"2024-09-15T22:08:53.580272+0000","last_became_active":"2024-09-15T22:06:35.137022+0000","last_became_peered":"2024-09-15T22:06:35.137022+0000","last_unstale":"2024-09-15T22:08:53.580272+0000","last_undegraded":"2024-09-15T22:08:53.580272+0000","last_fullsized":"2024-09-15T22:08:53.580272+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.863155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753002+0000","last_change":"2024-09-15T22:06:44.653807+0000","last_active":"2024-09-15T22:08:53.753002+0000","last_peered":"2024-09-15T22:08:53.753002+0000","last_clean":"2024-09-15T22:08:53.753002+0000","last_became_active":"2024-09-15T22:06:44.653406+0000","last_became_peered":"2024-09-15T22:06:44.653406+0000","last_unstale":"2024-09-15T22:08:53.753002+0000","last_undegraded":"2024-09-15T22:08:53.753002+0000","last_fullsized":"2024-09-15T22:08:53.753002+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:12.018208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582122+0000","last_change":"2024-09-15T22:06:37.186899+0000","last_active":"2024-09-15T22:08:53.582122+0000","last_peered":"2024-09-15T22:08:53.582122+0000","last_clean":"2024-09-15T22:08:53.582122+0000","last_became_active":"2024-09-15T22:06:37.183782+0000","last_became_peered":"2024-09-15T22:06:37.183782+0000","last_unstale":"2024-09-15T22:08:53.582122+0000","last_undegraded":"2024-09-15T22:08:53.582122+0000","last_fullsized":"2024-09-15T22:08:53.582122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:39.269186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928379+0000","last_change":"2024-09-15T22:06:44.652686+0000","last_active":"2024-09-15T22:08:53.928379+0000","last_peered":"2024-09-15T22:08:53.928379+0000","last_clean":"2024-09-15T22:08:53.928379+0000","last_became_active":"2024-09-15T22:06:44.652306+0000","last_became_peered":"2024-09-15T22:06:44.652306+0000","last_unstale":"2024-09-15T22:08:53.928379+0000","last_undegraded":"2024-09-15T22:08:53.928379+0000","last_fullsized":"2024-09-15T22:08:53.928379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753037+0000","last_change":"2024-09-15T22:06:44.655424+0000","last_active":"2024-09-15T22:08:53.753037+0000","last_peered":"2024-09-15T22:08:53.753037+0000","last_clean":"2024-09-15T22:08:53.753037+0000","last_became_active":"2024-09-15T22:06:44.654959+0000","last_became_peered":"2024-09-15T22:06:44.654959+0000","last_unstale":"2024-09-15T22:08:53.753037+0000","last_undegraded":"2024-09-15T22:08:53.753037+0000","last_fullsized":"2024-09-15T22:08:53.753037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:07.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928379+0000","last_change":"2024-09-15T22:06:37.188424+0000","last_active":"2024-09-15T22:08:53.928379+0000","last_peered":"2024-09-15T22:08:53.928379+0000","last_clean":"2024-09-15T22:08:53.928379+0000","last_became_active":"2024-09-15T22:06:37.188082+0000","last_became_peered":"2024-09-15T22:06:37.188082+0000","last_unstale":"2024-09-15T22:08:53.928379+0000","last_undegraded":"2024-09-15T22:08:53.928379+0000","last_fullsized":"2024-09-15T22:08:53.928379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:07.159341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581279+0000","last_change":"2024-09-15T22:06:35.134237+0000","last_active":"2024-09-15T22:08:53.581279+0000","last_peered":"2024-09-15T22:08:53.581279+0000","last_clean":"2024-09-15T22:08:53.581279+0000","last_became_active":"2024-09-15T22:06:35.133885+0000","last_became_peered":"2024-09-15T22:06:35.133885+0000","last_unstale":"2024-09-15T22:08:53.581279+0000","last_undegraded":"2024-09-15T22:08:53.581279+0000","last_fullsized":"2024-09-15T22:08:53.581279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:07:35.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489075+0000","last_change":"2024-09-15T22:06:44.664578+0000","last_active":"2024-09-15T22:08:53.489075+0000","last_peered":"2024-09-15T22:08:53.489075+0000","last_clean":"2024-09-15T22:08:53.489075+0000","last_became_active":"2024-09-15T22:06:44.664170+0000","last_became_peered":"2024-09-15T22:06:44.664170+0000","last_unstale":"2024-09-15T22:08:53.489075+0000","last_undegraded":"2024-09-15T22:08:53.489075+0000","last_fullsized":"2024-09-15T22:08:53.489075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:41.443816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580024+0000","last_change":"2024-09-15T22:06:37.186107+0000","last_active":"2024-09-15T22:08:53.580024+0000","last_peered":"2024-09-15T22:08:53.580024+0000","last_clean":"2024-09-15T22:08:53.580024+0000","last_became_active":"2024-09-15T22:06:37.185757+0000","last_became_peered":"2024-09-15T22:06:37.185757+0000","last_unstale":"2024-09-15T22:08:53.580024+0000","last_undegraded":"2024-09-15T22:08:53.580024+0000","last_fullsized":"2024-09-15T22:08:53.580024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:14.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750623+0000","last_change":"2024-09-15T22:06:35.356281+0000","last_active":"2024-09-15T22:08:53.750623+0000","last_peered":"2024-09-15T22:08:53.750623+0000","last_clean":"2024-09-15T22:08:53.750623+0000","last_became_active":"2024-09-15T22:06:35.355922+0000","last_became_peered":"2024-09-15T22:06:35.355922+0000","last_unstale":"2024-09-15T22:08:53.750623+0000","last_undegraded":"2024-09-15T22:08:53.750623+0000","last_fullsized":"2024-09-15T22:08:53.750623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:06.107462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753010+0000","last_change":"2024-09-15T22:06:44.659230+0000","last_active":"2024-09-15T22:08:53.753010+0000","last_peered":"2024-09-15T22:08:53.753010+0000","last_clean":"2024-09-15T22:08:53.753010+0000","last_became_active":"2024-09-15T22:06:44.658649+0000","last_became_peered":"2024-09-15T22:06:44.658649+0000","last_unstale":"2024-09-15T22:08:53.753010+0000","last_undegraded":"2024-09-15T22:08:53.753010+0000","last_fullsized":"2024-09-15T22:08:53.753010+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:14:21.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929164+0000","last_change":"2024-09-15T22:06:44.649394+0000","last_active":"2024-09-15T22:08:53.929164+0000","last_peered":"2024-09-15T22:08:53.929164+0000","last_clean":"2024-09-15T22:08:53.929164+0000","last_became_active":"2024-09-15T22:06:44.649043+0000","last_became_peered":"2024-09-15T22:06:44.649043+0000","last_unstale":"2024-09-15T22:08:53.929164+0000","last_undegraded":"2024-09-15T22:08:53.929164+0000","last_fullsized":"2024-09-15T22:08:53.929164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:56:28.335869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753504+0000","last_change":"2024-09-15T22:06:35.265970+0000","last_active":"2024-09-15T22:08:53.753504+0000","last_peered":"2024-09-15T22:08:53.753504+0000","last_clean":"2024-09-15T22:08:53.753504+0000","last_became_active":"2024-09-15T22:06:35.265628+0000","last_became_peered":"2024-09-15T22:06:35.265628+0000","last_unstale":"2024-09-15T22:08:53.753504+0000","last_undegraded":"2024-09-15T22:08:53.753504+0000","last_fullsized":"2024-09-15T22:08:53.753504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:41.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750664+0000","last_change":"2024-09-15T22:06:44.649652+0000","last_active":"2024-09-15T22:08:53.750664+0000","last_peered":"2024-09-15T22:08:53.750664+0000","last_clean":"2024-09-15T22:08:53.750664+0000","last_became_active":"2024-09-15T22:06:44.649112+0000","last_became_peered":"2024-09-15T22:06:44.649112+0000","last_unstale":"2024-09-15T22:08:53.750664+0000","last_undegraded":"2024-09-15T22:08:53.750664+0000","last_fullsized":"2024-09-15T22:08:53.750664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:53.789320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583103+0000","last_change":"2024-09-15T22:06:37.190701+0000","last_active":"2024-09-15T22:08:53.583103+0000","last_peered":"2024-09-15T22:08:53.583103+0000","last_clean":"2024-09-15T22:08:53.583103+0000","last_became_active":"2024-09-15T22:06:37.190430+0000","last_became_peered":"2024-09-15T22:06:37.190430+0000","last_unstale":"2024-09-15T22:08:53.583103+0000","last_undegraded":"2024-09-15T22:08:53.583103+0000","last_fullsized":"2024-09-15T22:08:53.583103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:14.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'51","reported_seq":111,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.701060+0000","last_change":"2024-09-15T22:06:35.133899+0000","last_active":"2024-09-15T22:08:53.701060+0000","last_peered":"2024-09-15T22:08:53.701060+0000","last_clean":"2024-09-15T22:08:53.701060+0000","last_became_active":"2024-09-15T22:06:35.133556+0000","last_became_peered":"2024-09-15T22:06:35.133556+0000","last_unstale":"2024-09-15T22:08:53.701060+0000","last_undegraded":"2024-09-15T22:08:53.701060+0000","last_fullsized":"2024-09-15T22:08:53.701060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:01.692416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580427+0000","last_change":"2024-09-15T22:06:44.655280+0000","last_active":"2024-09-15T22:08:53.580427+0000","last_peered":"2024-09-15T22:08:53.580427+0000","last_clean":"2024-09-15T22:08:53.580427+0000","last_became_active":"2024-09-15T22:06:44.654883+0000","last_became_peered":"2024-09-15T22:06:44.654883+0000","last_unstale":"2024-09-15T22:08:53.580427+0000","last_undegraded":"2024-09-15T22:08:53.580427+0000","last_fullsized":"2024-09-15T22:08:53.580427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.074035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889279+0000","last_change":"2024-09-15T22:06:37.182006+0000","last_active":"2024-09-15T22:08:53.889279+0000","last_peered":"2024-09-15T22:08:53.889279+0000","last_clean":"2024-09-15T22:08:53.889279+0000","last_became_active":"2024-09-15T22:06:37.181693+0000","last_became_peered":"2024-09-15T22:06:37.181693+0000","last_unstale":"2024-09-15T22:08:53.889279+0000","last_undegraded":"2024-09-15T22:08:53.889279+0000","last_fullsized":"2024-09-15T22:08:53.889279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:23.483930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580471+0000","last_change":"2024-09-15T22:06:35.355854+0000","last_active":"2024-09-15T22:08:53.580471+0000","last_peered":"2024-09-15T22:08:53.580471+0000","last_clean":"2024-09-15T22:08:53.580471+0000","last_became_active":"2024-09-15T22:06:35.355486+0000","last_became_peered":"2024-09-15T22:06:35.355486+0000","last_unstale":"2024-09-15T22:08:53.580471+0000","last_undegraded":"2024-09-15T22:08:53.580471+0000","last_fullsized":"2024-09-15T22:08:53.580471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:36.708476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928367+0000","last_change":"2024-09-15T22:06:44.660523+0000","last_active":"2024-09-15T22:08:53.928367+0000","last_peered":"2024-09-15T22:08:53.928367+0000","last_clean":"2024-09-15T22:08:53.928367+0000","last_became_active":"2024-09-15T22:06:44.660113+0000","last_became_peered":"2024-09-15T22:06:44.660113+0000","last_unstale":"2024-09-15T22:08:53.928367+0000","last_undegraded":"2024-09-15T22:08:53.928367+0000","last_fullsized":"2024-09-15T22:08:53.928367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:42.164558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589575+0000","last_change":"2024-09-15T22:06:37.183649+0000","last_active":"2024-09-15T22:08:53.589575+0000","last_peered":"2024-09-15T22:08:53.589575+0000","last_clean":"2024-09-15T22:08:53.589575+0000","last_became_active":"2024-09-15T22:06:37.183296+0000","last_became_peered":"2024-09-15T22:06:37.183296+0000","last_unstale":"2024-09-15T22:08:53.589575+0000","last_undegraded":"2024-09-15T22:08:53.589575+0000","last_fullsized":"2024-09-15T22:08:53.589575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:22.010288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488860+0000","last_change":"2024-09-15T22:06:35.355068+0000","last_active":"2024-09-15T22:08:53.488860+0000","last_peered":"2024-09-15T22:08:53.488860+0000","last_clean":"2024-09-15T22:08:53.488860+0000","last_became_active":"2024-09-15T22:06:35.354728+0000","last_became_peered":"2024-09-15T22:06:35.354728+0000","last_unstale":"2024-09-15T22:08:53.488860+0000","last_undegraded":"2024-09-15T22:08:53.488860+0000","last_fullsized":"2024-09-15T22:08:53.488860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:35.479290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580098+0000","last_change":"2024-09-15T22:06:44.653849+0000","last_active":"2024-09-15T22:08:53.580098+0000","last_peered":"2024-09-15T22:08:53.580098+0000","last_clean":"2024-09-15T22:08:53.580098+0000","last_became_active":"2024-09-15T22:06:44.653419+0000","last_became_peered":"2024-09-15T22:06:44.653419+0000","last_unstale":"2024-09-15T22:08:53.580098+0000","last_undegraded":"2024-09-15T22:08:53.580098+0000","last_fullsized":"2024-09-15T22:08:53.580098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:04.131553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589446+0000","last_change":"2024-09-15T22:06:37.191438+0000","last_active":"2024-09-15T22:08:53.589446+0000","last_peered":"2024-09-15T22:08:53.589446+0000","last_clean":"2024-09-15T22:08:53.589446+0000","last_became_active":"2024-09-15T22:06:37.191088+0000","last_became_peered":"2024-09-15T22:06:37.191088+0000","last_unstale":"2024-09-15T22:08:53.589446+0000","last_undegraded":"2024-09-15T22:08:53.589446+0000","last_fullsized":"2024-09-15T22:08:53.589446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:46:36.078872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490414+0000","last_change":"2024-09-15T22:06:35.135866+0000","last_active":"2024-09-15T22:08:53.490414+0000","last_peered":"2024-09-15T22:08:53.490414+0000","last_clean":"2024-09-15T22:08:53.490414+0000","last_became_active":"2024-09-15T22:06:35.135524+0000","last_became_peered":"2024-09-15T22:06:35.135524+0000","last_unstale":"2024-09-15T22:08:53.490414+0000","last_undegraded":"2024-09-15T22:08:53.490414+0000","last_fullsized":"2024-09-15T22:08:53.490414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888980+0000","last_change":"2024-09-15T22:06:44.649943+0000","last_active":"2024-09-15T22:08:53.888980+0000","last_peered":"2024-09-15T22:08:53.888980+0000","last_clean":"2024-09-15T22:08:53.888980+0000","last_became_active":"2024-09-15T22:06:44.649579+0000","last_became_peered":"2024-09-15T22:06:44.649579+0000","last_unstale":"2024-09-15T22:08:53.888980+0000","last_undegraded":"2024-09-15T22:08:53.888980+0000","last_fullsized":"2024-09-15T22:08:53.888980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:18.390608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489996+0000","last_change":"2024-09-15T22:06:37.189988+0000","last_active":"2024-09-15T22:08:53.489996+0000","last_peered":"2024-09-15T22:08:53.489996+0000","last_clean":"2024-09-15T22:08:53.489996+0000","last_became_active":"2024-09-15T22:06:37.189319+0000","last_became_peered":"2024-09-15T22:06:37.189319+0000","last_unstale":"2024-09-15T22:08:53.489996+0000","last_undegraded":"2024-09-15T22:08:53.489996+0000","last_fullsized":"2024-09-15T22:08:53.489996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:08.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":142,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889084+0000","last_change":"2024-09-15T22:06:35.355371+0000","last_active":"2024-09-15T22:08:53.889084+0000","last_peered":"2024-09-15T22:08:53.889084+0000","last_clean":"2024-09-15T22:08:53.889084+0000","last_became_active":"2024-09-15T22:06:35.354687+0000","last_became_peered":"2024-09-15T22:06:35.354687+0000","last_unstale":"2024-09-15T22:08:53.889084+0000","last_undegraded":"2024-09-15T22:08:53.889084+0000","last_fullsized":"2024-09-15T22:08:53.889084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:27.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889532+0000","last_change":"2024-09-15T22:06:44.657929+0000","last_active":"2024-09-15T22:08:53.889532+0000","last_peered":"2024-09-15T22:08:53.889532+0000","last_clean":"2024-09-15T22:08:53.889532+0000","last_became_active":"2024-09-15T22:06:44.657519+0000","last_became_peered":"2024-09-15T22:06:44.657519+0000","last_unstale":"2024-09-15T22:08:53.889532+0000","last_undegraded":"2024-09-15T22:08:53.889532+0000","last_fullsized":"2024-09-15T22:08:53.889532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:12.313877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888747+0000","last_change":"2024-09-15T22:06:37.193288+0000","last_active":"2024-09-15T22:08:53.888747+0000","last_peered":"2024-09-15T22:08:53.888747+0000","last_clean":"2024-09-15T22:08:53.888747+0000","last_became_active":"2024-09-15T22:06:37.193007+0000","last_became_peered":"2024-09-15T22:06:37.193007+0000","last_unstale":"2024-09-15T22:08:53.888747+0000","last_undegraded":"2024-09-15T22:08:53.888747+0000","last_fullsized":"2024-09-15T22:08:53.888747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:45.956206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490201+0000","last_change":"2024-09-15T22:06:35.138726+0000","last_active":"2024-09-15T22:08:53.490201+0000","last_peered":"2024-09-15T22:08:53.490201+0000","last_clean":"2024-09-15T22:08:53.490201+0000","last_became_active":"2024-09-15T22:06:35.138390+0000","last_became_peered":"2024-09-15T22:06:35.138390+0000","last_unstale":"2024-09-15T22:08:53.490201+0000","last_undegraded":"2024-09-15T22:08:53.490201+0000","last_fullsized":"2024-09-15T22:08:53.490201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:51.336171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490388+0000","last_change":"2024-09-15T22:07:46.219720+0000","last_active":"2024-09-15T22:08:53.490388+0000","last_peered":"2024-09-15T22:08:53.490388+0000","last_clean":"2024-09-15T22:08:53.490388+0000","last_became_active":"2024-09-15T22:07:46.219303+0000","last_became_peered":"2024-09-15T22:07:46.219303+0000","last_unstale":"2024-09-15T22:08:53.490388+0000","last_undegraded":"2024-09-15T22:08:53.490388+0000","last_fullsized":"2024-09-15T22:08:53.490388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:03.596795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581603+0000","last_change":"2024-09-15T22:06:37.176478+0000","last_active":"2024-09-15T22:08:53.581603+0000","last_peered":"2024-09-15T22:08:53.581603+0000","last_clean":"2024-09-15T22:08:53.581603+0000","last_became_active":"2024-09-15T22:06:37.176171+0000","last_became_peered":"2024-09-15T22:06:37.176171+0000","last_unstale":"2024-09-15T22:08:53.581603+0000","last_undegraded":"2024-09-15T22:08:53.581603+0000","last_fullsized":"2024-09-15T22:08:53.581603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753268+0000","last_change":"2024-09-15T22:06:35.134651+0000","last_active":"2024-09-15T22:08:53.753268+0000","last_peered":"2024-09-15T22:08:53.753268+0000","last_clean":"2024-09-15T22:08:53.753268+0000","last_became_active":"2024-09-15T22:06:35.134185+0000","last_became_peered":"2024-09-15T22:06:35.134185+0000","last_unstale":"2024-09-15T22:08:53.753268+0000","last_undegraded":"2024-09-15T22:08:53.753268+0000","last_fullsized":"2024-09-15T22:08:53.753268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:56.385046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":358,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490393+0000","last_change":"2024-09-15T22:05:23.048068+0000","last_active":"2024-09-15T22:08:53.490393+0000","last_peered":"2024-09-15T22:08:53.490393+0000","last_clean":"2024-09-15T22:08:53.490393+0000","last_became_active":"2024-09-15T22:05:23.047692+0000","last_became_peered":"2024-09-15T22:05:23.047692+0000","last_unstale":"2024-09-15T22:08:53.490393+0000","last_undegraded":"2024-09-15T22:08:53.490393+0000","last_fullsized":"2024-09-15T22:08:53.490393+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_clean_scrub_stamp":"2024-09-15T22:01:57.477629+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490625+0000","last_change":"2024-09-15T22:06:44.657213+0000","last_active":"2024-09-15T22:08:53.490625+0000","last_peered":"2024-09-15T22:08:53.490625+0000","last_clean":"2024-09-15T22:08:53.490625+0000","last_became_active":"2024-09-15T22:06:44.656815+0000","last_became_peered":"2024-09-15T22:06:44.656815+0000","last_unstale":"2024-09-15T22:08:53.490625+0000","last_undegraded":"2024-09-15T22:08:53.490625+0000","last_fullsized":"2024-09-15T22:08:53.490625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:26.525992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490755+0000","last_change":"2024-09-15T22:06:37.184467+0000","last_active":"2024-09-15T22:08:53.490755+0000","last_peered":"2024-09-15T22:08:53.490755+0000","last_clean":"2024-09-15T22:08:53.490755+0000","last_became_active":"2024-09-15T22:06:37.183864+0000","last_became_peered":"2024-09-15T22:06:37.183864+0000","last_unstale":"2024-09-15T22:08:53.490755+0000","last_undegraded":"2024-09-15T22:08:53.490755+0000","last_fullsized":"2024-09-15T22:08:53.490755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:35.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750757+0000","last_change":"2024-09-15T22:06:44.651594+0000","last_active":"2024-09-15T22:08:53.750757+0000","last_peered":"2024-09-15T22:08:53.750757+0000","last_clean":"2024-09-15T22:08:53.750757+0000","last_became_active":"2024-09-15T22:06:44.650393+0000","last_became_peered":"2024-09-15T22:06:44.650393+0000","last_unstale":"2024-09-15T22:08:53.750757+0000","last_undegraded":"2024-09-15T22:08:53.750757+0000","last_fullsized":"2024-09-15T22:08:53.750757+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:13.599744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750561+0000","last_change":"2024-09-15T22:06:44.660789+0000","last_active":"2024-09-15T22:08:53.750561+0000","last_peered":"2024-09-15T22:08:53.750561+0000","last_clean":"2024-09-15T22:08:53.750561+0000","last_became_active":"2024-09-15T22:06:44.660331+0000","last_became_peered":"2024-09-15T22:06:44.660331+0000","last_unstale":"2024-09-15T22:08:53.750561+0000","last_undegraded":"2024-09-15T22:08:53.750561+0000","last_fullsized":"2024-09-15T22:08:53.750561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:04:45.562795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490532+0000","last_change":"2024-09-15T22:06:37.185697+0000","last_active":"2024-09-15T22:08:53.490532+0000","last_peered":"2024-09-15T22:08:53.490532+0000","last_clean":"2024-09-15T22:08:53.490532+0000","last_became_active":"2024-09-15T22:06:37.185207+0000","last_became_peered":"2024-09-15T22:06:37.185207+0000","last_unstale":"2024-09-15T22:08:53.490532+0000","last_undegraded":"2024-09-15T22:08:53.490532+0000","last_fullsized":"2024-09-15T22:08:53.490532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:39.005273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'39","reported_seq":84,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.699454+0000","last_change":"2024-09-15T22:06:35.135176+0000","last_active":"2024-09-15T22:08:53.699454+0000","last_peered":"2024-09-15T22:08:53.699454+0000","last_clean":"2024-09-15T22:08:53.699454+0000","last_became_active":"2024-09-15T22:06:35.134649+0000","last_became_peered":"2024-09-15T22:06:35.134649+0000","last_unstale":"2024-09-15T22:08:53.699454+0000","last_undegraded":"2024-09-15T22:08:53.699454+0000","last_fullsized":"2024-09-15T22:08:53.699454+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:33.780162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928727+0000","last_change":"2024-09-15T22:06:44.656972+0000","last_active":"2024-09-15T22:08:53.928727+0000","last_peered":"2024-09-15T22:08:53.928727+0000","last_clean":"2024-09-15T22:08:53.928727+0000","last_became_active":"2024-09-15T22:06:44.656528+0000","last_became_peered":"2024-09-15T22:06:44.656528+0000","last_unstale":"2024-09-15T22:08:53.928727+0000","last_undegraded":"2024-09-15T22:08:53.928727+0000","last_fullsized":"2024-09-15T22:08:53.928727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:09.164596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490782+0000","last_change":"2024-09-15T22:06:37.186118+0000","last_active":"2024-09-15T22:08:53.490782+0000","last_peered":"2024-09-15T22:08:53.490782+0000","last_clean":"2024-09-15T22:08:53.490782+0000","last_became_active":"2024-09-15T22:06:37.185118+0000","last_became_peered":"2024-09-15T22:06:37.185118+0000","last_unstale":"2024-09-15T22:08:53.490782+0000","last_undegraded":"2024-09-15T22:08:53.490782+0000","last_fullsized":"2024-09-15T22:08:53.490782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:53.357045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889478+0000","last_change":"2024-09-15T22:06:35.133341+0000","last_active":"2024-09-15T22:08:53.889478+0000","last_peered":"2024-09-15T22:08:53.889478+0000","last_clean":"2024-09-15T22:08:53.889478+0000","last_became_active":"2024-09-15T22:06:35.132961+0000","last_became_peered":"2024-09-15T22:06:35.132961+0000","last_unstale":"2024-09-15T22:08:53.889478+0000","last_undegraded":"2024-09-15T22:08:53.889478+0000","last_fullsized":"2024-09-15T22:08:53.889478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:39.579470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489819+0000","last_change":"2024-09-15T22:06:44.659148+0000","last_active":"2024-09-15T22:08:53.489819+0000","last_peered":"2024-09-15T22:08:53.489819+0000","last_clean":"2024-09-15T22:08:53.489819+0000","last_became_active":"2024-09-15T22:06:44.658750+0000","last_became_peered":"2024-09-15T22:06:44.658750+0000","last_unstale":"2024-09-15T22:08:53.489819+0000","last_undegraded":"2024-09-15T22:08:53.489819+0000","last_fullsized":"2024-09-15T22:08:53.489819+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:48.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928598+0000","last_change":"2024-09-15T22:06:37.194066+0000","last_active":"2024-09-15T22:08:53.928598+0000","last_peered":"2024-09-15T22:08:53.928598+0000","last_clean":"2024-09-15T22:08:53.928598+0000","last_became_active":"2024-09-15T22:06:37.193738+0000","last_became_peered":"2024-09-15T22:06:37.193738+0000","last_unstale":"2024-09-15T22:08:53.928598+0000","last_undegraded":"2024-09-15T22:08:53.928598+0000","last_fullsized":"2024-09-15T22:08:53.928598+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.107188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580476+0000","last_change":"2024-09-15T22:06:35.136470+0000","last_active":"2024-09-15T22:08:53.580476+0000","last_peered":"2024-09-15T22:08:53.580476+0000","last_clean":"2024-09-15T22:08:53.580476+0000","last_became_active":"2024-09-15T22:06:35.135953+0000","last_became_peered":"2024-09-15T22:06:35.135953+0000","last_unstale":"2024-09-15T22:08:53.580476+0000","last_undegraded":"2024-09-15T22:08:53.580476+0000","last_fullsized":"2024-09-15T22:08:53.580476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:44.548566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489963+0000","last_change":"2024-09-15T22:06:44.653682+0000","last_active":"2024-09-15T22:08:53.489963+0000","last_peered":"2024-09-15T22:08:53.489963+0000","last_clean":"2024-09-15T22:08:53.489963+0000","last_became_active":"2024-09-15T22:06:44.653303+0000","last_became_peered":"2024-09-15T22:06:44.653303+0000","last_unstale":"2024-09-15T22:08:53.489963+0000","last_undegraded":"2024-09-15T22:08:53.489963+0000","last_fullsized":"2024-09-15T22:08:53.489963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:07.847971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489970+0000","last_change":"2024-09-15T22:06:44.658436+0000","last_active":"2024-09-15T22:08:53.489970+0000","last_peered":"2024-09-15T22:08:53.489970+0000","last_clean":"2024-09-15T22:08:53.489970+0000","last_became_active":"2024-09-15T22:06:44.658037+0000","last_became_peered":"2024-09-15T22:06:44.658037+0000","last_unstale":"2024-09-15T22:08:53.489970+0000","last_undegraded":"2024-09-15T22:08:53.489970+0000","last_fullsized":"2024-09-15T22:08:53.489970+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:34:20.369249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580102+0000","last_change":"2024-09-15T22:06:35.358018+0000","last_active":"2024-09-15T22:08:53.580102+0000","last_peered":"2024-09-15T22:08:53.580102+0000","last_clean":"2024-09-15T22:08:53.580102+0000","last_became_active":"2024-09-15T22:06:35.357666+0000","last_became_peered":"2024-09-15T22:06:35.357666+0000","last_unstale":"2024-09-15T22:08:53.580102+0000","last_undegraded":"2024-09-15T22:08:53.580102+0000","last_fullsized":"2024-09-15T22:08:53.580102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:34.089113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889340+0000","last_change":"2024-09-15T22:06:44.656807+0000","last_active":"2024-09-15T22:08:53.889340+0000","last_peered":"2024-09-15T22:08:53.889340+0000","last_clean":"2024-09-15T22:08:53.889340+0000","last_became_active":"2024-09-15T22:06:44.656409+0000","last_became_peered":"2024-09-15T22:06:44.656409+0000","last_unstale":"2024-09-15T22:08:53.889340+0000","last_undegraded":"2024-09-15T22:08:53.889340+0000","last_fullsized":"2024-09-15T22:08:53.889340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490123+0000","last_change":"2024-09-15T22:06:37.180615+0000","last_active":"2024-09-15T22:08:53.490123+0000","last_peered":"2024-09-15T22:08:53.490123+0000","last_clean":"2024-09-15T22:08:53.490123+0000","last_became_active":"2024-09-15T22:06:37.180306+0000","last_became_peered":"2024-09-15T22:06:37.180306+0000","last_unstale":"2024-09-15T22:08:53.490123+0000","last_undegraded":"2024-09-15T22:08:53.490123+0000","last_fullsized":"2024-09-15T22:08:53.490123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:31.145128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":147,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889942+0000","last_change":"2024-09-15T22:06:35.136192+0000","last_active":"2024-09-15T22:08:53.889942+0000","last_peered":"2024-09-15T22:08:53.889942+0000","last_clean":"2024-09-15T22:08:53.889942+0000","last_became_active":"2024-09-15T22:06:35.135608+0000","last_became_peered":"2024-09-15T22:06:35.135608+0000","last_unstale":"2024-09-15T22:08:53.889942+0000","last_undegraded":"2024-09-15T22:08:53.889942+0000","last_fullsized":"2024-09-15T22:08:53.889942+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:48.472717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489151+0000","last_change":"2024-09-15T22:06:44.664763+0000","last_active":"2024-09-15T22:08:53.489151+0000","last_peered":"2024-09-15T22:08:53.489151+0000","last_clean":"2024-09-15T22:08:53.489151+0000","last_became_active":"2024-09-15T22:06:44.664410+0000","last_became_peered":"2024-09-15T22:06:44.664410+0000","last_unstale":"2024-09-15T22:08:53.489151+0000","last_undegraded":"2024-09-15T22:08:53.489151+0000","last_fullsized":"2024-09-15T22:08:53.489151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:14.364387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":52,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490299+0000","last_change":"2024-09-15T22:06:44.652747+0000","last_active":"2024-09-15T22:08:53.490299+0000","last_peered":"2024-09-15T22:08:53.490299+0000","last_clean":"2024-09-15T22:08:53.490299+0000","last_became_active":"2024-09-15T22:06:44.652368+0000","last_became_peered":"2024-09-15T22:06:44.652368+0000","last_unstale":"2024-09-15T22:08:53.490299+0000","last_undegraded":"2024-09-15T22:08:53.490299+0000","last_fullsized":"2024-09-15T22:08:53.490299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:51.047549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490109+0000","last_change":"2024-09-15T22:06:35.136311+0000","last_active":"2024-09-15T22:08:53.490109+0000","last_peered":"2024-09-15T22:08:53.490109+0000","last_clean":"2024-09-15T22:08:53.490109+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.490109+0000","last_undegraded":"2024-09-15T22:08:53.490109+0000","last_fullsized":"2024-09-15T22:08:53.490109+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:36:07.138056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928366+0000","last_change":"2024-09-15T22:06:44.653747+0000","last_active":"2024-09-15T22:08:53.928366+0000","last_peered":"2024-09-15T22:08:53.928366+0000","last_clean":"2024-09-15T22:08:53.928366+0000","last_became_active":"2024-09-15T22:06:44.653357+0000","last_became_peered":"2024-09-15T22:06:44.653357+0000","last_unstale":"2024-09-15T22:08:53.928366+0000","last_undegraded":"2024-09-15T22:08:53.928366+0000","last_fullsized":"2024-09-15T22:08:53.928366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.113924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888246+0000","last_change":"2024-09-15T22:06:37.178308+0000","last_active":"2024-09-15T22:08:53.888246+0000","last_peered":"2024-09-15T22:08:53.888246+0000","last_clean":"2024-09-15T22:08:53.888246+0000","last_became_active":"2024-09-15T22:06:37.177920+0000","last_became_peered":"2024-09-15T22:06:37.177920+0000","last_unstale":"2024-09-15T22:08:53.888246+0000","last_undegraded":"2024-09-15T22:08:53.888246+0000","last_fullsized":"2024-09-15T22:08:53.888246+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750846+0000","last_change":"2024-09-15T22:06:35.136132+0000","last_active":"2024-09-15T22:08:53.750846+0000","last_peered":"2024-09-15T22:08:53.750846+0000","last_clean":"2024-09-15T22:08:53.750846+0000","last_became_active":"2024-09-15T22:06:35.135789+0000","last_became_peered":"2024-09-15T22:06:35.135789+0000","last_unstale":"2024-09-15T22:08:53.750846+0000","last_undegraded":"2024-09-15T22:08:53.750846+0000","last_fullsized":"2024-09-15T22:08:53.750846+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:01.692885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490410+0000","last_change":"2024-09-15T22:06:44.658920+0000","last_active":"2024-09-15T22:08:53.490410+0000","last_peered":"2024-09-15T22:08:53.490410+0000","last_clean":"2024-09-15T22:08:53.490410+0000","last_became_active":"2024-09-15T22:06:44.658434+0000","last_became_peered":"2024-09-15T22:06:44.658434+0000","last_unstale":"2024-09-15T22:08:53.490410+0000","last_undegraded":"2024-09-15T22:08:53.490410+0000","last_fullsized":"2024-09-15T22:08:53.490410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:37.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580330+0000","last_change":"2024-09-15T22:06:37.184958+0000","last_active":"2024-09-15T22:08:53.580330+0000","last_peered":"2024-09-15T22:08:53.580330+0000","last_clean":"2024-09-15T22:08:53.580330+0000","last_became_active":"2024-09-15T22:06:37.184648+0000","last_became_peered":"2024-09-15T22:06:37.184648+0000","last_unstale":"2024-09-15T22:08:53.580330+0000","last_undegraded":"2024-09-15T22:08:53.580330+0000","last_fullsized":"2024-09-15T22:08:53.580330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:28.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'40","reported_seq":289,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.890809+0000","last_change":"2024-09-15T22:06:35.138533+0000","last_active":"2024-09-15T22:08:53.890809+0000","last_peered":"2024-09-15T22:08:53.890809+0000","last_clean":"2024-09-15T22:08:53.890809+0000","last_became_active":"2024-09-15T22:06:35.138142+0000","last_became_peered":"2024-09-15T22:06:35.138142+0000","last_unstale":"2024-09-15T22:08:53.890809+0000","last_undegraded":"2024-09-15T22:08:53.890809+0000","last_fullsized":"2024-09-15T22:08:53.890809+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:32.950036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68792,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":213,"num_read_kb":174,"num_write":41,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589290+0000","last_change":"2024-09-15T22:07:46.218222+0000","last_active":"2024-09-15T22:08:53.589290+0000","last_peered":"2024-09-15T22:08:53.589290+0000","last_clean":"2024-09-15T22:08:53.589290+0000","last_became_active":"2024-09-15T22:07:46.217903+0000","last_became_peered":"2024-09-15T22:07:46.217903+0000","last_unstale":"2024-09-15T22:08:53.589290+0000","last_undegraded":"2024-09-15T22:08:53.589290+0000","last_fullsized":"2024-09-15T22:08:53.589290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:17.730288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750879+0000","last_change":"2024-09-15T22:06:37.194297+0000","last_active":"2024-09-15T22:08:53.750879+0000","last_peered":"2024-09-15T22:08:53.750879+0000","last_clean":"2024-09-15T22:08:53.750879+0000","last_became_active":"2024-09-15T22:06:37.193986+0000","last_became_peered":"2024-09-15T22:06:37.193986+0000","last_unstale":"2024-09-15T22:08:53.750879+0000","last_undegraded":"2024-09-15T22:08:53.750879+0000","last_fullsized":"2024-09-15T22:08:53.750879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:27.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490573+0000","last_change":"2024-09-15T22:06:35.356537+0000","last_active":"2024-09-15T22:08:53.490573+0000","last_peered":"2024-09-15T22:08:53.490573+0000","last_clean":"2024-09-15T22:08:53.490573+0000","last_became_active":"2024-09-15T22:06:35.355997+0000","last_became_peered":"2024-09-15T22:06:35.355997+0000","last_unstale":"2024-09-15T22:08:53.490573+0000","last_undegraded":"2024-09-15T22:08:53.490573+0000","last_fullsized":"2024-09-15T22:08:53.490573+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:11.563083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753107+0000","last_change":"2024-09-15T22:06:44.654579+0000","last_active":"2024-09-15T22:08:53.753107+0000","last_peered":"2024-09-15T22:08:53.753107+0000","last_clean":"2024-09-15T22:08:53.753107+0000","last_became_active":"2024-09-15T22:06:44.654187+0000","last_became_peered":"2024-09-15T22:06:44.654187+0000","last_unstale":"2024-09-15T22:08:53.753107+0000","last_undegraded":"2024-09-15T22:08:53.753107+0000","last_fullsized":"2024-09-15T22:08:53.753107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:15.617528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929412+0000","last_change":"2024-09-15T22:06:44.661533+0000","last_active":"2024-09-15T22:08:53.929412+0000","last_peered":"2024-09-15T22:08:53.929412+0000","last_clean":"2024-09-15T22:08:53.929412+0000","last_became_active":"2024-09-15T22:06:44.661193+0000","last_became_peered":"2024-09-15T22:06:44.661193+0000","last_unstale":"2024-09-15T22:08:53.929412+0000","last_undegraded":"2024-09-15T22:08:53.929412+0000","last_fullsized":"2024-09-15T22:08:53.929412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:06.559720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490695+0000","last_change":"2024-09-15T22:06:35.140578+0000","last_active":"2024-09-15T22:08:53.490695+0000","last_peered":"2024-09-15T22:08:53.490695+0000","last_clean":"2024-09-15T22:08:53.490695+0000","last_became_active":"2024-09-15T22:06:35.139818+0000","last_became_peered":"2024-09-15T22:06:35.139818+0000","last_unstale":"2024-09-15T22:08:53.490695+0000","last_undegraded":"2024-09-15T22:08:53.490695+0000","last_fullsized":"2024-09-15T22:08:53.490695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:59.806964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488473+0000","last_change":"2024-09-15T22:06:44.654755+0000","last_active":"2024-09-15T22:08:53.488473+0000","last_peered":"2024-09-15T22:08:53.488473+0000","last_clean":"2024-09-15T22:08:53.488473+0000","last_became_active":"2024-09-15T22:06:44.654390+0000","last_became_peered":"2024-09-15T22:06:44.654390+0000","last_unstale":"2024-09-15T22:08:53.488473+0000","last_undegraded":"2024-09-15T22:08:53.488473+0000","last_fullsized":"2024-09-15T22:08:53.488473+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:35.781878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751028+0000","last_change":"2024-09-15T22:06:37.182457+0000","last_active":"2024-09-15T22:08:53.751028+0000","last_peered":"2024-09-15T22:08:53.751028+0000","last_clean":"2024-09-15T22:08:53.751028+0000","last_became_active":"2024-09-15T22:06:37.182106+0000","last_became_peered":"2024-09-15T22:06:37.182106+0000","last_unstale":"2024-09-15T22:08:53.751028+0000","last_undegraded":"2024-09-15T22:08:53.751028+0000","last_fullsized":"2024-09-15T22:08:53.751028+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:57.560894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":140,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589780+0000","last_change":"2024-09-15T22:06:35.140590+0000","last_active":"2024-09-15T22:08:53.589780+0000","last_peered":"2024-09-15T22:08:53.589780+0000","last_clean":"2024-09-15T22:08:53.589780+0000","last_became_active":"2024-09-15T22:06:35.140198+0000","last_became_peered":"2024-09-15T22:06:35.140198+0000","last_unstale":"2024-09-15T22:08:53.589780+0000","last_undegraded":"2024-09-15T22:08:53.589780+0000","last_fullsized":"2024-09-15T22:08:53.589780+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:46.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580877+0000","last_change":"2024-09-15T22:06:44.660188+0000","last_active":"2024-09-15T22:08:53.580877+0000","last_peered":"2024-09-15T22:08:53.580877+0000","last_clean":"2024-09-15T22:08:53.580877+0000","last_became_active":"2024-09-15T22:06:44.659808+0000","last_became_peered":"2024-09-15T22:06:44.659808+0000","last_unstale":"2024-09-15T22:08:53.580877+0000","last_undegraded":"2024-09-15T22:08:53.580877+0000","last_fullsized":"2024-09-15T22:08:53.580877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:39.473765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929017+0000","last_change":"2024-09-15T22:06:44.647967+0000","last_active":"2024-09-15T22:08:53.929017+0000","last_peered":"2024-09-15T22:08:53.929017+0000","last_clean":"2024-09-15T22:08:53.929017+0000","last_became_active":"2024-09-15T22:06:44.647608+0000","last_became_peered":"2024-09-15T22:06:44.647608+0000","last_unstale":"2024-09-15T22:08:53.929017+0000","last_undegraded":"2024-09-15T22:08:53.929017+0000","last_fullsized":"2024-09-15T22:08:53.929017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:06.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488584+0000","last_change":"2024-09-15T22:06:37.187020+0000","last_active":"2024-09-15T22:08:53.488584+0000","last_peered":"2024-09-15T22:08:53.488584+0000","last_clean":"2024-09-15T22:08:53.488584+0000","last_became_active":"2024-09-15T22:06:37.186564+0000","last_became_peered":"2024-09-15T22:06:37.186564+0000","last_unstale":"2024-09-15T22:08:53.488584+0000","last_undegraded":"2024-09-15T22:08:53.488584+0000","last_fullsized":"2024-09-15T22:08:53.488584+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:20.586763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490131+0000","last_change":"2024-09-15T22:06:35.135126+0000","last_active":"2024-09-15T22:08:53.490131+0000","last_peered":"2024-09-15T22:08:53.490131+0000","last_clean":"2024-09-15T22:08:53.490131+0000","last_became_active":"2024-09-15T22:06:35.134783+0000","last_became_peered":"2024-09-15T22:06:35.134783+0000","last_unstale":"2024-09-15T22:08:53.490131+0000","last_undegraded":"2024-09-15T22:08:53.490131+0000","last_fullsized":"2024-09-15T22:08:53.490131+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:31.910988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928656+0000","last_change":"2024-09-15T22:06:44.656981+0000","last_active":"2024-09-15T22:08:53.928656+0000","last_peered":"2024-09-15T22:08:53.928656+0000","last_clean":"2024-09-15T22:08:53.928656+0000","last_became_active":"2024-09-15T22:06:44.656477+0000","last_became_peered":"2024-09-15T22:06:44.656477+0000","last_unstale":"2024-09-15T22:08:53.928656+0000","last_undegraded":"2024-09-15T22:08:53.928656+0000","last_fullsized":"2024-09-15T22:08:53.928656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:06.153490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580796+0000","last_change":"2024-09-15T22:06:37.183034+0000","last_active":"2024-09-15T22:08:53.580796+0000","last_peered":"2024-09-15T22:08:53.580796+0000","last_clean":"2024-09-15T22:08:53.580796+0000","last_became_active":"2024-09-15T22:06:37.182734+0000","last_became_peered":"2024-09-15T22:06:37.182734+0000","last_unstale":"2024-09-15T22:08:53.580796+0000","last_undegraded":"2024-09-15T22:08:53.580796+0000","last_fullsized":"2024-09-15T22:08:53.580796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:19.352000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888909+0000","last_change":"2024-09-15T22:06:35.134954+0000","last_active":"2024-09-15T22:08:53.888909+0000","last_peered":"2024-09-15T22:08:53.888909+0000","last_clean":"2024-09-15T22:08:53.888909+0000","last_became_active":"2024-09-15T22:06:35.134583+0000","last_became_peered":"2024-09-15T22:06:35.134583+0000","last_unstale":"2024-09-15T22:08:53.888909+0000","last_undegraded":"2024-09-15T22:08:53.888909+0000","last_fullsized":"2024-09-15T22:08:53.888909+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:56.082123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589453+0000","last_change":"2024-09-15T22:06:35.139229+0000","last_active":"2024-09-15T22:08:53.589453+0000","last_peered":"2024-09-15T22:08:53.589453+0000","last_clean":"2024-09-15T22:08:53.589453+0000","last_became_active":"2024-09-15T22:06:35.138834+0000","last_became_peered":"2024-09-15T22:06:35.138834+0000","last_unstale":"2024-09-15T22:08:53.589453+0000","last_undegraded":"2024-09-15T22:08:53.589453+0000","last_fullsized":"2024-09-15T22:08:53.589453+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:05.877404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490250+0000","last_change":"2024-09-15T22:06:37.192212+0000","last_active":"2024-09-15T22:08:53.490250+0000","last_peered":"2024-09-15T22:08:53.490250+0000","last_clean":"2024-09-15T22:08:53.490250+0000","last_became_active":"2024-09-15T22:06:37.191872+0000","last_became_peered":"2024-09-15T22:06:37.191872+0000","last_unstale":"2024-09-15T22:08:53.490250+0000","last_undegraded":"2024-09-15T22:08:53.490250+0000","last_fullsized":"2024-09-15T22:08:53.490250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:32.845794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750771+0000","last_change":"2024-09-15T22:07:45.968398+0000","last_active":"2024-09-15T22:08:53.750771+0000","last_peered":"2024-09-15T22:08:53.750771+0000","last_clean":"2024-09-15T22:08:53.750771+0000","last_became_active":"2024-09-15T22:07:45.967912+0000","last_became_peered":"2024-09-15T22:07:45.967912+0000","last_unstale":"2024-09-15T22:08:53.750771+0000","last_undegraded":"2024-09-15T22:08:53.750771+0000","last_fullsized":"2024-09-15T22:08:53.750771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:59.909290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'9","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.890548+0000","last_change":"2024-09-15T22:06:35.133846+0000","last_active":"2024-09-15T22:08:53.890548+0000","last_peered":"2024-09-15T22:08:53.890548+0000","last_clean":"2024-09-15T22:08:53.890548+0000","last_became_active":"2024-09-15T22:06:35.133516+0000","last_became_peered":"2024-09-15T22:06:35.133516+0000","last_unstale":"2024-09-15T22:08:53.890548+0000","last_undegraded":"2024-09-15T22:08:53.890548+0000","last_fullsized":"2024-09-15T22:08:53.890548+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:29.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1857,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580118+0000","last_change":"2024-09-15T22:06:37.182904+0000","last_active":"2024-09-15T22:08:53.580118+0000","last_peered":"2024-09-15T22:08:53.580118+0000","last_clean":"2024-09-15T22:08:53.580118+0000","last_became_active":"2024-09-15T22:06:37.182563+0000","last_became_peered":"2024-09-15T22:06:37.182563+0000","last_unstale":"2024-09-15T22:08:53.580118+0000","last_undegraded":"2024-09-15T22:08:53.580118+0000","last_fullsized":"2024-09-15T22:08:53.580118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750976+0000","last_change":"2024-09-15T22:06:44.660997+0000","last_active":"2024-09-15T22:08:53.750976+0000","last_peered":"2024-09-15T22:08:53.750976+0000","last_clean":"2024-09-15T22:08:53.750976+0000","last_became_active":"2024-09-15T22:06:44.660577+0000","last_became_peered":"2024-09-15T22:06:44.660577+0000","last_unstale":"2024-09-15T22:08:53.750976+0000","last_undegraded":"2024-09-15T22:08:53.750976+0000","last_fullsized":"2024-09-15T22:08:53.750976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:56.006891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928243+0000","last_change":"2024-09-15T22:06:37.182077+0000","last_active":"2024-09-15T22:08:53.928243+0000","last_peered":"2024-09-15T22:08:53.928243+0000","last_clean":"2024-09-15T22:08:53.928243+0000","last_became_active":"2024-09-15T22:06:37.181622+0000","last_became_peered":"2024-09-15T22:06:37.181622+0000","last_unstale":"2024-09-15T22:08:53.928243+0000","last_undegraded":"2024-09-15T22:08:53.928243+0000","last_fullsized":"2024-09-15T22:08:53.928243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:07.586406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750818+0000","last_change":"2024-09-15T22:06:44.653845+0000","last_active":"2024-09-15T22:08:53.750818+0000","last_peered":"2024-09-15T22:08:53.750818+0000","last_clean":"2024-09-15T22:08:53.750818+0000","last_became_active":"2024-09-15T22:06:44.653483+0000","last_became_peered":"2024-09-15T22:06:44.653483+0000","last_unstale":"2024-09-15T22:08:53.750818+0000","last_undegraded":"2024-09-15T22:08:53.750818+0000","last_fullsized":"2024-09-15T22:08:53.750818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:08:07.822279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750649+0000","last_change":"2024-09-15T22:06:37.187476+0000","last_active":"2024-09-15T22:08:53.750649+0000","last_peered":"2024-09-15T22:08:53.750649+0000","last_clean":"2024-09-15T22:08:53.750649+0000","last_became_active":"2024-09-15T22:06:37.187047+0000","last_became_peered":"2024-09-15T22:06:37.187047+0000","last_unstale":"2024-09-15T22:08:53.750649+0000","last_undegraded":"2024-09-15T22:08:53.750649+0000","last_fullsized":"2024-09-15T22:08:53.750649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.220834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928765+0000","last_change":"2024-09-15T22:06:37.190583+0000","last_active":"2024-09-15T22:08:53.928765+0000","last_peered":"2024-09-15T22:08:53.928765+0000","last_clean":"2024-09-15T22:08:53.928765+0000","last_became_active":"2024-09-15T22:06:37.190212+0000","last_became_peered":"2024-09-15T22:06:37.190212+0000","last_unstale":"2024-09-15T22:08:53.928765+0000","last_undegraded":"2024-09-15T22:08:53.928765+0000","last_fullsized":"2024-09-15T22:08:53.928765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:48.574188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889153+0000","last_change":"2024-09-15T22:06:37.192508+0000","last_active":"2024-09-15T22:08:53.889153+0000","last_peered":"2024-09-15T22:08:53.889153+0000","last_clean":"2024-09-15T22:08:53.889153+0000","last_became_active":"2024-09-15T22:06:37.192117+0000","last_became_peered":"2024-09-15T22:06:37.192117+0000","last_unstale":"2024-09-15T22:08:53.889153+0000","last_undegraded":"2024-09-15T22:08:53.889153+0000","last_fullsized":"2024-09-15T22:08:53.889153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:41.355966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928601+0000","last_change":"2024-09-15T22:06:37.183377+0000","last_active":"2024-09-15T22:08:53.928601+0000","last_peered":"2024-09-15T22:08:53.928601+0000","last_clean":"2024-09-15T22:08:53.928601+0000","last_became_active":"2024-09-15T22:06:37.183072+0000","last_became_peered":"2024-09-15T22:06:37.183072+0000","last_unstale":"2024-09-15T22:08:53.928601+0000","last_undegraded":"2024-09-15T22:08:53.928601+0000","last_fullsized":"2024-09-15T22:08:53.928601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.318702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751128+0000","last_change":"2024-09-15T22:06:37.185348+0000","last_active":"2024-09-15T22:08:53.751128+0000","last_peered":"2024-09-15T22:08:53.751128+0000","last_clean":"2024-09-15T22:08:53.751128+0000","last_became_active":"2024-09-15T22:06:37.184934+0000","last_became_peered":"2024-09-15T22:06:37.184934+0000","last_unstale":"2024-09-15T22:08:53.751128+0000","last_undegraded":"2024-09-15T22:08:53.751128+0000","last_fullsized":"2024-09-15T22:08:53.751128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:57.564860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582987+0000","last_change":"2024-09-15T22:06:37.192152+0000","last_active":"2024-09-15T22:08:53.582987+0000","last_peered":"2024-09-15T22:08:53.582987+0000","last_clean":"2024-09-15T22:08:53.582987+0000","last_became_active":"2024-09-15T22:06:37.191836+0000","last_became_peered":"2024-09-15T22:06:37.191836+0000","last_unstale":"2024-09-15T22:08:53.582987+0000","last_undegraded":"2024-09-15T22:08:53.582987+0000","last_fullsized":"2024-09-15T22:08:53.582987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:31.781110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489966+0000","last_change":"2024-09-15T22:06:37.179996+0000","last_active":"2024-09-15T22:08:53.489966+0000","last_peered":"2024-09-15T22:08:53.489966+0000","last_clean":"2024-09-15T22:08:53.489966+0000","last_became_active":"2024-09-15T22:06:37.179646+0000","last_became_peered":"2024-09-15T22:06:37.179646+0000","last_unstale":"2024-09-15T22:08:53.489966+0000","last_undegraded":"2024-09-15T22:08:53.489966+0000","last_fullsized":"2024-09-15T22:08:53.489966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:35.240468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581098+0000","last_change":"2024-09-15T22:06:37.182992+0000","last_active":"2024-09-15T22:08:53.581098+0000","last_peered":"2024-09-15T22:08:53.581098+0000","last_clean":"2024-09-15T22:08:53.581098+0000","last_became_active":"2024-09-15T22:06:37.182676+0000","last_became_peered":"2024-09-15T22:06:37.182676+0000","last_unstale":"2024-09-15T22:08:53.581098+0000","last_undegraded":"2024-09-15T22:08:53.581098+0000","last_fullsized":"2024-09-15T22:08:53.581098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:41.524600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750731+0000","last_change":"2024-09-15T22:06:37.180107+0000","last_active":"2024-09-15T22:08:53.750731+0000","last_peered":"2024-09-15T22:08:53.750731+0000","last_clean":"2024-09-15T22:08:53.750731+0000","last_became_active":"2024-09-15T22:06:37.179785+0000","last_became_peered":"2024-09-15T22:06:37.179785+0000","last_unstale":"2024-09-15T22:08:53.750731+0000","last_undegraded":"2024-09-15T22:08:53.750731+0000","last_fullsized":"2024-09-15T22:08:53.750731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:46.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888926+0000","last_change":"2024-09-15T22:06:37.177406+0000","last_active":"2024-09-15T22:08:53.888926+0000","last_peered":"2024-09-15T22:08:53.888926+0000","last_clean":"2024-09-15T22:08:53.888926+0000","last_became_active":"2024-09-15T22:06:37.177093+0000","last_became_peered":"2024-09-15T22:06:37.177093+0000","last_unstale":"2024-09-15T22:08:53.888926+0000","last_undegraded":"2024-09-15T22:08:53.888926+0000","last_fullsized":"2024-09-15T22:08:53.888926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:34.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928419+0000","last_change":"2024-09-15T22:06:37.192314+0000","last_active":"2024-09-15T22:08:53.928419+0000","last_peered":"2024-09-15T22:08:53.928419+0000","last_clean":"2024-09-15T22:08:53.928419+0000","last_became_active":"2024-09-15T22:06:37.191958+0000","last_became_peered":"2024-09-15T22:06:37.191958+0000","last_unstale":"2024-09-15T22:08:53.928419+0000","last_undegraded":"2024-09-15T22:08:53.928419+0000","last_fullsized":"2024-09-15T22:08:53.928419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:47.763742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751178+0000","last_change":"2024-09-15T22:06:37.190212+0000","last_active":"2024-09-15T22:08:53.751178+0000","last_peered":"2024-09-15T22:08:53.751178+0000","last_clean":"2024-09-15T22:08:53.751178+0000","last_became_active":"2024-09-15T22:06:37.189881+0000","last_became_peered":"2024-09-15T22:06:37.189881+0000","last_unstale":"2024-09-15T22:08:53.751178+0000","last_undegraded":"2024-09-15T22:08:53.751178+0000","last_fullsized":"2024-09-15T22:08:53.751178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:59.943574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488963+0000","last_change":"2024-09-15T22:06:37.187967+0000","last_active":"2024-09-15T22:08:53.488963+0000","last_peered":"2024-09-15T22:08:53.488963+0000","last_clean":"2024-09-15T22:08:53.488963+0000","last_became_active":"2024-09-15T22:06:37.187596+0000","last_became_peered":"2024-09-15T22:06:37.187596+0000","last_unstale":"2024-09-15T22:08:53.488963+0000","last_undegraded":"2024-09-15T22:08:53.488963+0000","last_fullsized":"2024-09-15T22:08:53.488963+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:00.042784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928757+0000","last_change":"2024-09-15T22:06:37.189434+0000","last_active":"2024-09-15T22:08:53.928757+0000","last_peered":"2024-09-15T22:08:53.928757+0000","last_clean":"2024-09-15T22:08:53.928757+0000","last_became_active":"2024-09-15T22:06:37.189053+0000","last_became_peered":"2024-09-15T22:06:37.189053+0000","last_unstale":"2024-09-15T22:08:53.928757+0000","last_undegraded":"2024-09-15T22:08:53.928757+0000","last_fullsized":"2024-09-15T22:08:53.928757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:51.640495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750486+0000","last_change":"2024-09-15T22:06:37.179229+0000","last_active":"2024-09-15T22:08:53.750486+0000","last_peered":"2024-09-15T22:08:53.750486+0000","last_clean":"2024-09-15T22:08:53.750486+0000","last_became_active":"2024-09-15T22:06:37.178828+0000","last_became_peered":"2024-09-15T22:06:37.178828+0000","last_unstale":"2024-09-15T22:08:53.750486+0000","last_undegraded":"2024-09-15T22:08:53.750486+0000","last_fullsized":"2024-09-15T22:08:53.750486+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:25.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928489+0000","last_change":"2024-09-15T22:06:37.193095+0000","last_active":"2024-09-15T22:08:53.928489+0000","last_peered":"2024-09-15T22:08:53.928489+0000","last_clean":"2024-09-15T22:08:53.928489+0000","last_became_active":"2024-09-15T22:06:37.192801+0000","last_became_peered":"2024-09-15T22:06:37.192801+0000","last_unstale":"2024-09-15T22:08:53.928489+0000","last_undegraded":"2024-09-15T22:08:53.928489+0000","last_fullsized":"2024-09-15T22:08:53.928489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:12.521510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928614+0000","last_change":"2024-09-15T22:06:37.182332+0000","last_active":"2024-09-15T22:08:53.928614+0000","last_peered":"2024-09-15T22:08:53.928614+0000","last_clean":"2024-09-15T22:08:53.928614+0000","last_became_active":"2024-09-15T22:06:37.182001+0000","last_became_peered":"2024-09-15T22:06:37.182001+0000","last_unstale":"2024-09-15T22:08:53.928614+0000","last_undegraded":"2024-09-15T22:08:53.928614+0000","last_fullsized":"2024-09-15T22:08:53.928614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:52.230452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888644+0000","last_change":"2024-09-15T22:06:37.180584+0000","last_active":"2024-09-15T22:08:53.888644+0000","last_peered":"2024-09-15T22:08:53.888644+0000","last_clean":"2024-09-15T22:08:53.888644+0000","last_became_active":"2024-09-15T22:06:37.179341+0000","last_became_peered":"2024-09-15T22:06:37.179341+0000","last_unstale":"2024-09-15T22:08:53.888644+0000","last_undegraded":"2024-09-15T22:08:53.888644+0000","last_fullsized":"2024-09-15T22:08:53.888644+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.610835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753439+0000","last_change":"2024-09-15T22:06:37.189500+0000","last_active":"2024-09-15T22:08:53.753439+0000","last_peered":"2024-09-15T22:08:53.753439+0000","last_clean":"2024-09-15T22:08:53.753439+0000","last_became_active":"2024-09-15T22:06:37.189137+0000","last_became_peered":"2024-09-15T22:06:37.189137+0000","last_unstale":"2024-09-15T22:08:53.753439+0000","last_undegraded":"2024-09-15T22:08:53.753439+0000","last_fullsized":"2024-09-15T22:08:53.753439+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:27.199854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490697+0000","last_change":"2024-09-15T22:06:37.182618+0000","last_active":"2024-09-15T22:08:53.490697+0000","last_peered":"2024-09-15T22:08:53.490697+0000","last_clean":"2024-09-15T22:08:53.490697+0000","last_became_active":"2024-09-15T22:06:37.182257+0000","last_became_peered":"2024-09-15T22:06:37.182257+0000","last_unstale":"2024-09-15T22:08:53.490697+0000","last_undegraded":"2024-09-15T22:08:53.490697+0000","last_fullsized":"2024-09-15T22:08:53.490697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:17.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589823+0000","last_change":"2024-09-15T22:06:37.193036+0000","last_active":"2024-09-15T22:08:53.589823+0000","last_peered":"2024-09-15T22:08:53.589823+0000","last_clean":"2024-09-15T22:08:53.589823+0000","last_became_active":"2024-09-15T22:06:37.192668+0000","last_became_peered":"2024-09-15T22:06:37.192668+0000","last_unstale":"2024-09-15T22:08:53.589823+0000","last_undegraded":"2024-09-15T22:08:53.589823+0000","last_fullsized":"2024-09-15T22:08:53.589823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:05.436045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489086+0000","last_change":"2024-09-15T22:06:44.659503+0000","last_active":"2024-09-15T22:08:53.489086+0000","last_peered":"2024-09-15T22:08:53.489086+0000","last_clean":"2024-09-15T22:08:53.489086+0000","last_became_active":"2024-09-15T22:06:44.659155+0000","last_became_peered":"2024-09-15T22:06:44.659155+0000","last_unstale":"2024-09-15T22:08:53.489086+0000","last_undegraded":"2024-09-15T22:08:53.489086+0000","last_fullsized":"2024-09-15T22:08:53.489086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:41:04.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580648+0000","last_change":"2024-09-15T22:06:37.178386+0000","last_active":"2024-09-15T22:08:53.580648+0000","last_peered":"2024-09-15T22:08:53.580648+0000","last_clean":"2024-09-15T22:08:53.580648+0000","last_became_active":"2024-09-15T22:06:37.178083+0000","last_became_peered":"2024-09-15T22:06:37.178083+0000","last_unstale":"2024-09-15T22:08:53.580648+0000","last_undegraded":"2024-09-15T22:08:53.580648+0000","last_fullsized":"2024-09-15T22:08:53.580648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:35.350698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490190+0000","last_change":"2024-09-15T22:06:37.186741+0000","last_active":"2024-09-15T22:08:53.490190+0000","last_peered":"2024-09-15T22:08:53.490190+0000","last_clean":"2024-09-15T22:08:53.490190+0000","last_became_active":"2024-09-15T22:06:37.185740+0000","last_became_peered":"2024-09-15T22:06:37.185740+0000","last_unstale":"2024-09-15T22:08:53.490190+0000","last_undegraded":"2024-09-15T22:08:53.490190+0000","last_fullsized":"2024-09-15T22:08:53.490190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.762407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589411+0000","last_change":"2024-09-15T22:06:37.189552+0000","last_active":"2024-09-15T22:08:53.589411+0000","last_peered":"2024-09-15T22:08:53.589411+0000","last_clean":"2024-09-15T22:08:53.589411+0000","last_became_active":"2024-09-15T22:06:37.189197+0000","last_became_peered":"2024-09-15T22:06:37.189197+0000","last_unstale":"2024-09-15T22:08:53.589411+0000","last_undegraded":"2024-09-15T22:08:53.589411+0000","last_fullsized":"2024-09-15T22:08:53.589411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:54.268720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580092+0000","last_change":"2024-09-15T22:06:37.186917+0000","last_active":"2024-09-15T22:08:53.580092+0000","last_peered":"2024-09-15T22:08:53.580092+0000","last_clean":"2024-09-15T22:08:53.580092+0000","last_became_active":"2024-09-15T22:06:37.186571+0000","last_became_peered":"2024-09-15T22:06:37.186571+0000","last_unstale":"2024-09-15T22:08:53.580092+0000","last_undegraded":"2024-09-15T22:08:53.580092+0000","last_fullsized":"2024-09-15T22:08:53.580092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:02.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753272+0000","last_change":"2024-09-15T22:06:37.190960+0000","last_active":"2024-09-15T22:08:53.753272+0000","last_peered":"2024-09-15T22:08:53.753272+0000","last_clean":"2024-09-15T22:08:53.753272+0000","last_became_active":"2024-09-15T22:06:37.190650+0000","last_became_peered":"2024-09-15T22:06:37.190650+0000","last_unstale":"2024-09-15T22:08:53.753272+0000","last_undegraded":"2024-09-15T22:08:53.753272+0000","last_fullsized":"2024-09-15T22:08:53.753272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:34.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928796+0000","last_change":"2024-09-15T22:06:37.186718+0000","last_active":"2024-09-15T22:08:53.928796+0000","last_peered":"2024-09-15T22:08:53.928796+0000","last_clean":"2024-09-15T22:08:53.928796+0000","last_became_active":"2024-09-15T22:06:37.186424+0000","last_became_peered":"2024-09-15T22:06:37.186424+0000","last_unstale":"2024-09-15T22:08:53.928796+0000","last_undegraded":"2024-09-15T22:08:53.928796+0000","last_fullsized":"2024-09-15T22:08:53.928796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:53.266007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929020+0000","last_change":"2024-09-15T22:06:37.185522+0000","last_active":"2024-09-15T22:08:53.929020+0000","last_peered":"2024-09-15T22:08:53.929020+0000","last_clean":"2024-09-15T22:08:53.929020+0000","last_became_active":"2024-09-15T22:06:37.185192+0000","last_became_peered":"2024-09-15T22:06:37.185192+0000","last_unstale":"2024-09-15T22:08:53.929020+0000","last_undegraded":"2024-09-15T22:08:53.929020+0000","last_fullsized":"2024-09-15T22:08:53.929020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:12.868324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928738+0000","last_change":"2024-09-15T22:06:37.189553+0000","last_active":"2024-09-15T22:08:53.928738+0000","last_peered":"2024-09-15T22:08:53.928738+0000","last_clean":"2024-09-15T22:08:53.928738+0000","last_became_active":"2024-09-15T22:06:37.189270+0000","last_became_peered":"2024-09-15T22:06:37.189270+0000","last_unstale":"2024-09-15T22:08:53.928738+0000","last_undegraded":"2024-09-15T22:08:53.928738+0000","last_fullsized":"2024-09-15T22:08:53.928738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:34.157077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":599380,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":730,"num_read_kb":478,"num_write":302,"num_write_kb":1445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1798140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":236,"ondisk_log_size":236,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579945,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":502844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808878,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32996,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712156,"statfs":{"total":95995035648,"available":95961247744,"internally_reserved":0,"allocated":1855488,"data_stored":1300760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005107,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1437696,"data_stored":909721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1429504,"data_stored":904963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495677,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1630208,"data_stored":1109427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659201,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":507585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888135,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37080,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708072,"statfs":{"total":95995035648,"available":95957065728,"internally_reserved":0,"allocated":1777664,"data_stored":1236270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018956,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":513744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247889,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1032192,"data_stored":507779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444119,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1114112,"data_stored":578290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1097728,"data_stored":576869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1044480,"data_stored":508973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":202897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:08:59.145 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:08:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:08:59.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:59 smithi099 ceph-mon[34892]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:59.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:59 smithi099 ceph-mon[34892]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:59.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:08:59 smithi099 ceph-mon[34892]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:59 smithi129 ceph-mon[35314]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:59 smithi129 ceph-mon[35314]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:08:59 smithi129 ceph-mon[35314]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:59.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:59 smithi005 ceph-mon[30944]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:59.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:59 smithi005 ceph-mon[30944]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:59.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:08:59 smithi005 ceph-mon[30944]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:08:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.949+0000 7f8c46f53640 1 -- 172.21.15.5:0/2606763216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c40103660 msgr2=0x7f8c4010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.949+0000 7f8c46f53640 1 --2- 172.21.15.5:0/2606763216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c40103660 0x7f8c4010bb50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8c34009900 tx=0x7f8c3402a480 comp rx=0 tx=0).stop 2024-09-15T22:08:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 -- 172.21.15.5:0/2606763216 shutdown_connections 2024-09-15T22:08:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 --2- 172.21.15.5:0/2606763216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c4010c170 0x7f8c4010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 --2- 172.21.15.5:0/2606763216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c40103660 0x7f8c4010bb50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 --2- 172.21.15.5:0/2606763216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c40102d40 0x7f8c40103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 -- 172.21.15.5:0/2606763216 >> 172.21.15.5:0/2606763216 conn(0x7f8c400fc7a0 msgr2=0x7f8c400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 -- 172.21.15.5:0/2606763216 shutdown_connections 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 -- 172.21.15.5:0/2606763216 wait complete. 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 Processor -- start 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 -- start start 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c40102d40 0x7f8c401009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c40103660 0x7f8c40100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c4010c170 0x7f8c40101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40111f70 con 0x7f8c40103660 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40111df0 con 0x7f8c40102d40 2024-09-15T22:08:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.950+0000 7f8c46f53640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8c401120f0 con 0x7f8c4010c170 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c40103660 0x7f8c40100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c40103660 0x7f8c40100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56110/0 (socket says 172.21.15.5:56110) 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3ffff640 1 -- 172.21.15.5:0/4250682604 learned_addr learned my addr 172.21.15.5:0/4250682604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c44cef640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c40102d40 0x7f8c401009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c454f0640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c4010c170 0x7f8c40101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3ffff640 1 -- 172.21.15.5:0/4250682604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c4010c170 msgr2=0x7f8c40101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3ffff640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c4010c170 0x7f8c40101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3ffff640 1 -- 172.21.15.5:0/4250682604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c40102d40 msgr2=0x7f8c401009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3ffff640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c40102d40 0x7f8c401009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3ffff640 1 -- 172.21.15.5:0/4250682604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c401a68f0 con 0x7f8c40103660 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3ffff640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c40103660 0x7f8c40100ee0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8c34006010 tx=0x7f8c340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3dffb640 1 -- 172.21.15.5:0/4250682604 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3403f030 con 0x7f8c40103660 2024-09-15T22:08:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c46f53640 1 -- 172.21.15.5:0/4250682604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c401a6b80 con 0x7f8c40103660 2024-09-15T22:08:59.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3dffb640 1 -- 172.21.15.5:0/4250682604 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c34003ce0 con 0x7f8c40103660 2024-09-15T22:08:59.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.951+0000 7f8c3dffb640 1 -- 172.21.15.5:0/4250682604 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c34035070 con 0x7f8c40103660 2024-09-15T22:08:59.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.952+0000 7f8c46f53640 1 -- 172.21.15.5:0/4250682604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c401a70b0 con 0x7f8c40103660 2024-09-15T22:08:59.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.953+0000 7f8c46f53640 1 -- 172.21.15.5:0/4250682604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c40103120 con 0x7f8c40103660 2024-09-15T22:08:59.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.954+0000 7f8c3dffb640 1 -- 172.21.15.5:0/4250682604 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f8c34042030 con 0x7f8c40103660 2024-09-15T22:08:59.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.955+0000 7f8c3dffb640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8c280a34d0 0x7f8c280a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:08:59.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.955+0000 7f8c44cef640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8c280a34d0 0x7f8c280a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:08:59.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.955+0000 7f8c3dffb640 1 -- 172.21.15.5:0/4250682604 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f8c340f08e0 con 0x7f8c40103660 2024-09-15T22:08:59.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.955+0000 7f8c44cef640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8c280a34d0 0x7f8c280a5990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8c30005d00 tx=0x7f8c3000b410 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:08:59.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:08:59.959+0000 7f8c3dffb640 1 -- 172.21.15.5:0/4250682604 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8c340bce00 con 0x7f8c40103660 2024-09-15T22:09:00.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.169+0000 7f8c46f53640 1 -- 172.21.15.5:0/4250682604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c4002d7c0 con 0x7f8c40103660 2024-09-15T22:09:00.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.170+0000 7f8c3dffb640 1 -- 172.21.15.5:0/4250682604 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f8c340bc700 con 0x7f8c40103660 2024-09-15T22:09:00.172 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:00.172 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:09:00.172 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.172+0000 7f8c46f53640 1 -- 172.21.15.5:0/4250682604 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8c280a34d0 msgr2=0x7f8c280a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.172+0000 7f8c46f53640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8c280a34d0 0x7f8c280a5990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8c30005d00 tx=0x7f8c3000b410 comp rx=0 tx=0).stop 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.172+0000 7f8c46f53640 1 -- 172.21.15.5:0/4250682604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c40103660 msgr2=0x7f8c40100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.172+0000 7f8c46f53640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c40103660 0x7f8c40100ee0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8c34006010 tx=0x7f8c340042f0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.172+0000 7f8c46f53640 1 -- 172.21.15.5:0/4250682604 shutdown_connections 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.172+0000 7f8c46f53640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8c280a34d0 0x7f8c280a5990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.172+0000 7f8c46f53640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8c4010c170 0x7f8c40101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.172+0000 7f8c46f53640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c40103660 0x7f8c40100ee0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.172+0000 7f8c46f53640 1 --2- 172.21.15.5:0/4250682604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8c40102d40 0x7f8c401009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.172+0000 7f8c46f53640 1 -- 172.21.15.5:0/4250682604 >> 172.21.15.5:0/4250682604 conn(0x7f8c400fc7a0 msgr2=0x7f8c400fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.172+0000 7f8c46f53640 1 -- 172.21.15.5:0/4250682604 shutdown_connections 2024-09-15T22:09:00.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:00.173+0000 7f8c46f53640 1 -- 172.21.15.5:0/4250682604 wait complete. 2024-09-15T22:09:00.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:00 smithi099 ceph-mon[34892]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:00.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:00 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4250682604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:00.840 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:00 smithi005 ceph-mon[30944]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:00.840 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:00 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4250682604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:00.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:00 smithi129 ceph-mon[35314]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:09:00.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:00 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4250682604' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:00.868 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:09:01.125 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.627+0000 7fe7beeac640 1 -- 172.21.15.5:0/1157018817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b8101420 msgr2=0x7fe7b810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.627+0000 7fe7beeac640 1 --2- 172.21.15.5:0/1157018817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b8101420 0x7fe7b810d950 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009930 tx=0x7fe7b002a420 comp rx=0 tx=0).stop 2024-09-15T22:09:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.627+0000 7fe7beeac640 1 -- 172.21.15.5:0/1157018817 shutdown_connections 2024-09-15T22:09:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.627+0000 7fe7beeac640 1 --2- 172.21.15.5:0/1157018817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b8101420 0x7fe7b810d950 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.627+0000 7fe7beeac640 1 --2- 172.21.15.5:0/1157018817 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8100a80 0x7fe7b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.627+0000 7fe7beeac640 1 --2- 172.21.15.5:0/1157018817 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b8106a80 0x7fe7b8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.627+0000 7fe7beeac640 1 -- 172.21.15.5:0/1157018817 >> 172.21.15.5:0/1157018817 conn(0x7fe7b80fc7a0 msgr2=0x7fe7b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.627+0000 7fe7beeac640 1 -- 172.21.15.5:0/1157018817 shutdown_connections 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.627+0000 7fe7beeac640 1 -- 172.21.15.5:0/1157018817 wait complete. 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.628+0000 7fe7beeac640 1 Processor -- start 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.628+0000 7fe7beeac640 1 -- start start 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.628+0000 7fe7beeac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b8100a80 0x7fe7b8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.628+0000 7fe7beeac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8101420 0x7fe7b81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.628+0000 7fe7beeac640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b8106a80 0x7fe7b819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.628+0000 7fe7beeac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8111fa0 con 0x7fe7b8100a80 2024-09-15T22:09:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.628+0000 7fe7beeac640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8111e20 con 0x7fe7b8106a80 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.628+0000 7fe7beeac640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8112120 con 0x7fe7b8101420 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.628+0000 7fe7bcc48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b8100a80 0x7fe7b8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.628+0000 7fe7bcc48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b8100a80 0x7fe7b8198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56126/0 (socket says 172.21.15.5:56126) 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.628+0000 7fe7bcc48640 1 -- 172.21.15.5:0/3111592065 learned_addr learned my addr 172.21.15.5:0/3111592065 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7b7fff640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8101420 0x7fe7b81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7bd449640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b8106a80 0x7fe7b819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7bcc48640 1 -- 172.21.15.5:0/3111592065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8101420 msgr2=0x7fe7b81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7bcc48640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8101420 0x7fe7b81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7bcc48640 1 -- 172.21.15.5:0/3111592065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b8106a80 msgr2=0x7fe7b819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7bcc48640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b8106a80 0x7fe7b819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7bcc48640 1 -- 172.21.15.5:0/3111592065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b819d330 con 0x7fe7b8100a80 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7bd449640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b8106a80 0x7fe7b819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7bcc48640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b8100a80 0x7fe7b8198380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000d950 tx=0x7fe7a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7b5ffb640 1 -- 172.21.15.5:0/3111592065 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0017070 con 0x7fe7b8100a80 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7b5ffb640 1 -- 172.21.15.5:0/3111592065 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7a0009c30 con 0x7fe7b8100a80 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7beeac640 1 -- 172.21.15.5:0/3111592065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b819d620 con 0x7fe7b8100a80 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7b5ffb640 1 -- 172.21.15.5:0/3111592065 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a000f6b0 con 0x7fe7b8100a80 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.629+0000 7fe7beeac640 1 -- 172.21.15.5:0/3111592065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b81a4f00 con 0x7fe7b8100a80 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.630+0000 7fe7beeac640 1 -- 172.21.15.5:0/3111592065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b8100ee0 con 0x7fe7b8100a80 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.632+0000 7fe7b5ffb640 1 -- 172.21.15.5:0/3111592065 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7fe7a00109d0 con 0x7fe7b8100a80 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.632+0000 7fe7b5ffb640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe7a40a34d0 0x7fe7a40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.632+0000 7fe7b7fff640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe7a40a34d0 0x7fe7a40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.632+0000 7fe7b5ffb640 1 -- 172.21.15.5:0/3111592065 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe7a00cff10 con 0x7fe7b8100a80 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.633+0000 7fe7b7fff640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe7a40a34d0 0x7fe7a40a5990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8106020 tx=0x7fe7a80075b0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.637+0000 7fe7b5ffb640 1 -- 172.21.15.5:0/3111592065 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe7a0009de0 con 0x7fe7b8100a80 2024-09-15T22:09:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.812+0000 7fe7beeac640 1 -- 172.21.15.5:0/3111592065 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe7b81a5270 con 0x7fe7b8100a80 2024-09-15T22:09:01.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.815+0000 7fe7b5ffb640 1 -- 172.21.15.5:0/3111592065 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+25126 (secure 0 0 0) 0x7fe7a009c300 con 0x7fe7b8100a80 2024-09-15T22:09:01.816 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:01.816 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:08:53.484177+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2453332511":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/878276657":"2024-09-16T22:08:53.484141+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1408571851":"2024-09-16T22:08:53.484141+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/801026306":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6800/801026306":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2083481614":"2024-09-16T22:08:53.484141+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 -- 172.21.15.5:0/3111592065 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe7a40a34d0 msgr2=0x7fe7a40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe7a40a34d0 0x7fe7a40a5990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8106020 tx=0x7fe7a80075b0 comp rx=0 tx=0).stop 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 -- 172.21.15.5:0/3111592065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b8100a80 msgr2=0x7fe7b8198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b8100a80 0x7fe7b8198380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe7a000d950 tx=0x7fe7a000de10 comp rx=0 tx=0).stop 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 -- 172.21.15.5:0/3111592065 shutdown_connections 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe7a40a34d0 0x7fe7a40a5990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe7b8106a80 0x7fe7b819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe7b8101420 0x7fe7b81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 --2- 172.21.15.5:0/3111592065 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7b8100a80 0x7fe7b8198380 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 -- 172.21.15.5:0/3111592065 >> 172.21.15.5:0/3111592065 conn(0x7fe7b80fc7a0 msgr2=0x7fe7b80fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 -- 172.21.15.5:0/3111592065 shutdown_connections 2024-09-15T22:09:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:01.817+0000 7fe7beeac640 1 -- 172.21.15.5:0/3111592065 wait complete. 2024-09-15T22:09:02.532 DEBUG:tasks.fwd_scrub:Yielding 2024-09-15T22:09:02.532 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T22:09:02.541 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:09:02.541 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-15T22:09:02.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-15T22:09:02.542 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-15T22:09:02.542 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:09:02.542 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:09:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.652+0000 7f5ea7b86640 1 -- 172.21.15.5:0/2149878421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ea01009a0 msgr2=0x7f5ea0100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.652+0000 7f5ea7b86640 1 --2- 172.21.15.5:0/2149878421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ea01009a0 0x7f5ea0100e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009930 tx=0x7f5e9c02a420 comp rx=0 tx=0).stop 2024-09-15T22:09:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.652+0000 7f5ea7b86640 1 -- 172.21.15.5:0/2149878421 shutdown_connections 2024-09-15T22:09:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.652+0000 7f5ea7b86640 1 --2- 172.21.15.5:0/2149878421 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea0101340 0x7f5ea010d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.652+0000 7f5ea7b86640 1 --2- 172.21.15.5:0/2149878421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ea01009a0 0x7f5ea0100e00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.652+0000 7f5ea7b86640 1 --2- 172.21.15.5:0/2149878421 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea01069b0 0x7f5ea0106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.653+0000 7f5ea7b86640 1 -- 172.21.15.5:0/2149878421 >> 172.21.15.5:0/2149878421 conn(0x7f5ea00779a0 msgr2=0x7f5ea00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.653+0000 7f5ea7b86640 1 -- 172.21.15.5:0/2149878421 shutdown_connections 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.653+0000 7f5ea7b86640 1 -- 172.21.15.5:0/2149878421 wait complete. 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.653+0000 7f5ea7b86640 1 Processor -- start 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.653+0000 7f5ea7b86640 1 -- start start 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.653+0000 7f5ea7b86640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea01009a0 0x7f5ea0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea7b86640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea0101340 0x7f5ea01997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea7b86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ea01069b0 0x7f5ea01a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea7b86640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea0110e40 con 0x7f5ea01069b0 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea7b86640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea0110cc0 con 0x7f5ea0101340 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea7b86640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea0110fc0 con 0x7f5ea01009a0 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea6123640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ea01069b0 0x7f5ea01a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea6123640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ea01069b0 0x7f5ea01a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56146/0 (socket says 172.21.15.5:56146) 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea6123640 1 -- 172.21.15.5:0/910105866 learned_addr learned my addr 172.21.15.5:0/910105866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea5922640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea01009a0 0x7f5ea0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea5121640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea0101340 0x7f5ea01997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea6123640 1 -- 172.21.15.5:0/910105866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea01009a0 msgr2=0x7f5ea0199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea6123640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea01009a0 0x7f5ea0199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea6123640 1 -- 172.21.15.5:0/910105866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea0101340 msgr2=0x7f5ea01997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea6123640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea0101340 0x7f5ea01997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea6123640 1 -- 172.21.15.5:0/910105866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea01a4850 con 0x7f5ea01069b0 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea6123640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ea01069b0 0x7f5ea01a4310 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800eaa0 tx=0x7f5e9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5e92ffd640 1 -- 172.21.15.5:0/910105866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9800cdb0 con 0x7f5ea01069b0 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5ea7b86640 1 -- 172.21.15.5:0/910105866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea01a4a20 con 0x7f5ea01069b0 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5e92ffd640 1 -- 172.21.15.5:0/910105866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e98005020 con 0x7f5ea01069b0 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.654+0000 7f5e92ffd640 1 -- 172.21.15.5:0/910105866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980116b0 con 0x7f5ea01069b0 2024-09-15T22:09:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.655+0000 7f5ea7b86640 1 -- 172.21.15.5:0/910105866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea01a4ef0 con 0x7f5ea01069b0 2024-09-15T22:09:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.655+0000 7f5ea7b86640 1 -- 172.21.15.5:0/910105866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea0100e00 con 0x7f5ea01069b0 2024-09-15T22:09:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.657+0000 7f5e92ffd640 1 -- 172.21.15.5:0/910105866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f5e98005560 con 0x7f5ea01069b0 2024-09-15T22:09:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.657+0000 7f5e92ffd640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5e8c0a34d0 0x7f5e8c0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:02.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.657+0000 7f5e92ffd640 1 -- 172.21.15.5:0/910105866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f5e980d0490 con 0x7f5ea01069b0 2024-09-15T22:09:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.657+0000 7f5ea5922640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5e8c0a34d0 0x7f5e8c0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:02.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.658+0000 7f5ea5922640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5e8c0a34d0 0x7f5e8c0a5990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400bb60 tx=0x7f5e9400b390 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:02.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.661+0000 7f5e92ffd640 1 -- 172.21.15.5:0/910105866 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e98016030 con 0x7f5ea01069b0 2024-09-15T22:09:02.817 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:02.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:02 smithi099 ceph-mon[34892]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.3 KiB/s wr, 23 op/s 2024-09-15T22:09:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:02 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3111592065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:09:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:02 smithi129 ceph-mon[35314]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.3 KiB/s wr, 23 op/s 2024-09-15T22:09:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:02 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3111592065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:09:02.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:02 smithi005 ceph-mon[30944]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.3 KiB/s wr, 23 op/s 2024-09-15T22:09:02.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:02 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3111592065' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:09:02.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.856+0000 7f5ea7b86640 1 -- 172.21.15.5:0/910105866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f5ea0108980 con 0x7f5ea01069b0 2024-09-15T22:09:02.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.859+0000 7f5e92ffd640 1 -- 172.21.15.5:0/910105866 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v31)=0 v31) ==== 129+0+0 (secure 0 0 0) 0x7f5e9809c2d0 con 0x7f5ea01069b0 2024-09-15T22:09:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 -- 172.21.15.5:0/910105866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5e8c0a34d0 msgr2=0x7f5e8c0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5e8c0a34d0 0x7f5e8c0a5990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400bb60 tx=0x7f5e9400b390 comp rx=0 tx=0).stop 2024-09-15T22:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 -- 172.21.15.5:0/910105866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ea01069b0 msgr2=0x7f5ea01a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ea01069b0 0x7f5ea01a4310 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800eaa0 tx=0x7f5e9800ef60 comp rx=0 tx=0).stop 2024-09-15T22:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 -- 172.21.15.5:0/910105866 shutdown_connections 2024-09-15T22:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5e8c0a34d0 0x7f5e8c0a5990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ea01069b0 0x7f5ea01a4310 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5ea0101340 0x7f5ea01997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 --2- 172.21.15.5:0/910105866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5ea01009a0 0x7f5ea0199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 -- 172.21.15.5:0/910105866 >> 172.21.15.5:0/910105866 conn(0x7f5ea00779a0 msgr2=0x7f5ea0103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 -- 172.21.15.5:0/910105866 shutdown_connections 2024-09-15T22:09:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.862+0000 7f5ea7b86640 1 -- 172.21.15.5:0/910105866 wait complete. 2024-09-15T22:09:02.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-15T22:09:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.983+0000 7f31c58ba640 1 -- 172.21.15.5:0/2832347175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31c01080d0 msgr2=0x7f31c010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.983+0000 7f31c58ba640 1 --2- 172.21.15.5:0/2832347175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31c01080d0 0x7f31c010ea20 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f31b8009cd0 tx=0x7f31b802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.983+0000 7f31c58ba640 1 -- 172.21.15.5:0/2832347175 shutdown_connections 2024-09-15T22:09:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.983+0000 7f31c58ba640 1 --2- 172.21.15.5:0/2832347175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31c01080d0 0x7f31c010ea20 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.983+0000 7f31c58ba640 1 --2- 172.21.15.5:0/2832347175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31c01011c0 0x7f31c0107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.983+0000 7f31c58ba640 1 --2- 172.21.15.5:0/2832347175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c0100810 0x7f31c0100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.983+0000 7f31c58ba640 1 -- 172.21.15.5:0/2832347175 >> 172.21.15.5:0/2832347175 conn(0x7f31c0075eb0 msgr2=0x7f31c00762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.983+0000 7f31c58ba640 1 -- 172.21.15.5:0/2832347175 shutdown_connections 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.983+0000 7f31c58ba640 1 -- 172.21.15.5:0/2832347175 wait complete. 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31c58ba640 1 Processor -- start 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31c58ba640 1 -- start start 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31c58ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31c0100810 0x7f31c0101780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31c58ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c01080d0 0x7f31c0101cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31c58ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31c0102200 0x7f31c01a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31c58ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31c0111e90 con 0x7f31c0100810 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31c58ba640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f31c0111d10 con 0x7f31c0102200 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31c58ba640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31c0112010 con 0x7f31c01080d0 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31c0100810 0x7f31c0101780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31c0100810 0x7f31c0101780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56166/0 (socket says 172.21.15.5:56166) 2024-09-15T22:09:02.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31beffd640 1 -- 172.21.15.5:0/3102748595 learned_addr learned my addr 172.21.15.5:0/3102748595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31b67fc640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c01080d0 0x7f31c0101cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31bf7fe640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31c0102200 0x7f31c01a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31beffd640 1 -- 172.21.15.5:0/3102748595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c01080d0 msgr2=0x7f31c0101cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31beffd640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c01080d0 0x7f31c0101cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31beffd640 1 -- 172.21.15.5:0/3102748595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31c0102200 msgr2=0x7f31c01a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.984+0000 7f31beffd640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31c0102200 0x7f31c01a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.985+0000 7f31beffd640 1 -- 172.21.15.5:0/3102748595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c01a6c90 con 0x7f31c0100810 2024-09-15T22:09:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.985+0000 7f31bf7fe640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31c0102200 0x7f31c01a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.985+0000 7f31beffd640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31c0100810 0x7f31c0101780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f31b000d950 tx=0x7f31b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:02.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.985+0000 7f31c48b8640 1 -- 172.21.15.5:0/3102748595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b0017030 con 0x7f31c0100810 2024-09-15T22:09:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.985+0000 7f31c48b8640 1 -- 172.21.15.5:0/3102748595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31b0004020 con 0x7f31c0100810 2024-09-15T22:09:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.985+0000 7f31c48b8640 1 -- 172.21.15.5:0/3102748595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00047a0 con 0x7f31c0100810 2024-09-15T22:09:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.985+0000 7f31c58ba640 1 -- 172.21.15.5:0/3102748595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c01a6f80 con 0x7f31c0100810 2024-09-15T22:09:02.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.985+0000 7f31c58ba640 1 -- 172.21.15.5:0/3102748595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c01a73f0 con 0x7f31c0100810 2024-09-15T22:09:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.986+0000 7f31c58ba640 1 -- 172.21.15.5:0/3102748595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c0069390 con 0x7f31c0100810 2024-09-15T22:09:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.987+0000 7f31c48b8640 1 -- 172.21.15.5:0/3102748595 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f31b00041d0 con 0x7f31c0100810 2024-09-15T22:09:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.988+0000 7f31c48b8640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31ac0a3500 0x7f31ac0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.988+0000 7f31c48b8640 1 -- 172.21.15.5:0/3102748595 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f31b00d03c0 con 0x7f31c0100810 2024-09-15T22:09:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.988+0000 7f31b67fc640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31ac0a3500 0x7f31ac0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:02.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.989+0000 7f31b67fc640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31ac0a3500 0x7f31ac0a59c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f31c0103210 tx=0x7f31a800b390 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:02.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:02.991+0000 7f31c48b8640 1 -- 172.21.15.5:0/3102748595 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f31b009c750 con 0x7f31c0100810 2024-09-15T22:09:03.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.188+0000 7f31c58ba640 1 -- 172.21.15.5:0/3102748595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f31c0100bf0 con 0x7f31c0100810 2024-09-15T22:09:03.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.192+0000 7f31c48b8640 1 -- 172.21.15.5:0/3102748595 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7f31c0100bf0 con 0x7f31c0100810 2024-09-15T22:09:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 -- 172.21.15.5:0/3102748595 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31ac0a3500 msgr2=0x7f31ac0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31ac0a3500 0x7f31ac0a59c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f31c0103210 tx=0x7f31a800b390 comp rx=0 tx=0).stop 2024-09-15T22:09:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 -- 172.21.15.5:0/3102748595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31c0100810 msgr2=0x7f31c0101780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31c0100810 0x7f31c0101780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f31b000d950 tx=0x7f31b000de10 comp rx=0 tx=0).stop 2024-09-15T22:09:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 -- 172.21.15.5:0/3102748595 shutdown_connections 2024-09-15T22:09:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31ac0a3500 0x7f31ac0a59c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31c0102200 0x7f31c01a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31c01080d0 0x7f31c0101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 --2- 172.21.15.5:0/3102748595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31c0100810 0x7f31c0101780 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 -- 172.21.15.5:0/3102748595 >> 172.21.15.5:0/3102748595 conn(0x7f31c0075eb0 msgr2=0x7f31c0105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 -- 172.21.15.5:0/3102748595 shutdown_connections 2024-09-15T22:09:03.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.194+0000 7f31c58ba640 1 -- 172.21.15.5:0/3102748595 wait complete. 2024-09-15T22:09:03.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-15T22:09:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.314+0000 7f68fb5af640 1 -- 172.21.15.5:0/3097482109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68f4105ea0 msgr2=0x7f68f4106330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.314+0000 7f68fb5af640 1 --2- 172.21.15.5:0/3097482109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68f4105ea0 0x7f68f4106330 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009930 tx=0x7f68ec02a4d0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.314+0000 7f68fb5af640 1 -- 172.21.15.5:0/3097482109 shutdown_connections 2024-09-15T22:09:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.314+0000 7f68fb5af640 1 --2- 172.21.15.5:0/3097482109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68f4105ea0 0x7f68f4106330 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.314+0000 7f68fb5af640 1 --2- 172.21.15.5:0/3097482109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f4101390 0x7f68f4105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.314+0000 7f68fb5af640 1 --2- 172.21.15.5:0/3097482109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4100a70 0x7f68f4100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.314+0000 7f68fb5af640 1 -- 172.21.15.5:0/3097482109 >> 172.21.15.5:0/3097482109 conn(0x7f68f40779a0 msgr2=0x7f68f40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.314+0000 7f68fb5af640 1 -- 172.21.15.5:0/3097482109 shutdown_connections 2024-09-15T22:09:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.314+0000 7f68fb5af640 1 -- 172.21.15.5:0/3097482109 wait complete. 2024-09-15T22:09:03.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.314+0000 7f68fb5af640 1 Processor -- start 2024-09-15T22:09:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68fb5af640 1 -- start start 2024-09-15T22:09:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68fb5af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68f4100a70 0x7f68f41a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68fb5af640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f4101390 0x7f68f41a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68fb5af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4105ea0 0x7f68f419e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68fb5af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68f41184a0 con 0x7f68f4100a70 2024-09-15T22:09:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68fb5af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f68f4118320 con 0x7f68f4105ea0 2024-09-15T22:09:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68fb5af640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f68f4118620 con 0x7f68f4101390 2024-09-15T22:09:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68f934b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68f4100a70 0x7f68f41a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68f934b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68f4100a70 0x7f68f41a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56194/0 (socket says 172.21.15.5:56194) 2024-09-15T22:09:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68f934b640 1 -- 172.21.15.5:0/3622393237 learned_addr learned my addr 172.21.15.5:0/3622393237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:03.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68f9b4c640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4105ea0 0x7f68f419e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68f8b4a640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f4101390 0x7f68f41a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68f934b640 1 -- 172.21.15.5:0/3622393237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f4101390 msgr2=0x7f68f41a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68f934b640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f4101390 0x7f68f41a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68f934b640 1 -- 172.21.15.5:0/3622393237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4105ea0 msgr2=0x7f68f419e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68f934b640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4105ea0 0x7f68f419e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68f934b640 1 -- 172.21.15.5:0/3622393237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f419efe0 con 0x7f68f4100a70 2024-09-15T22:09:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.315+0000 7f68f934b640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68f4100a70 0x7f68f41a46a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f68f000cce0 tx=0x7f68f000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.316+0000 7f68ea7fc640 1 -- 172.21.15.5:0/3622393237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f0004b20 con 0x7f68f4100a70 2024-09-15T22:09:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.316+0000 7f68fb5af640 1 -- 172.21.15.5:0/3622393237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f419f2d0 con 0x7f68f4100a70 2024-09-15T22:09:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.316+0000 7f68ea7fc640 1 -- 172.21.15.5:0/3622393237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68f0004cc0 con 0x7f68f4100a70 2024-09-15T22:09:03.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.316+0000 7f68ea7fc640 1 -- 172.21.15.5:0/3622393237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f000f7d0 con 0x7f68f4100a70 2024-09-15T22:09:03.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.316+0000 7f68fb5af640 1 -- 172.21.15.5:0/3622393237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f41ab4f0 con 0x7f68f4100a70 2024-09-15T22:09:03.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.317+0000 7f68fb5af640 1 -- 172.21.15.5:0/3622393237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68f406a2e0 con 0x7f68f4100a70 2024-09-15T22:09:03.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.318+0000 7f68ea7fc640 1 -- 172.21.15.5:0/3622393237 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f68f000f970 con 0x7f68f4100a70 2024-09-15T22:09:03.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.319+0000 7f68ea7fc640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68e00a34d0 0x7f68e00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.319+0000 7f68ea7fc640 1 -- 172.21.15.5:0/3622393237 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f68f00d0750 con 0x7f68f4100a70 2024-09-15T22:09:03.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.319+0000 7f68f8b4a640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68e00a34d0 0x7f68e00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.319+0000 7f68f8b4a640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68e00a34d0 0x7f68e00a5990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f68f41065c0 tx=0x7f68e4005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.322+0000 7f68ea7fc640 1 -- 172.21.15.5:0/3622393237 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68f0014030 con 0x7f68f4100a70 2024-09-15T22:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.332+0000 7ff6f6569640 1 -- 172.21.15.5:0/43725523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6f01008e0 msgr2=0x7ff6f0100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.332+0000 7ff6f6569640 1 --2- 172.21.15.5:0/43725523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6f01008e0 0x7ff6f0100cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009930 tx=0x7ff6e002a420 comp rx=0 tx=0).stop 2024-09-15T22:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.332+0000 7ff6f6569640 1 -- 172.21.15.5:0/43725523 shutdown_connections 2024-09-15T22:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.332+0000 7ff6f6569640 1 --2- 172.21.15.5:0/43725523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6f0108160 0x7ff6f010eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.332+0000 7ff6f6569640 1 --2- 172.21.15.5:0/43725523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f0101290 0x7ff6f0107a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.332+0000 7ff6f6569640 1 --2- 172.21.15.5:0/43725523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6f01008e0 0x7ff6f0100cc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.332+0000 7ff6f6569640 1 -- 172.21.15.5:0/43725523 >> 172.21.15.5:0/43725523 conn(0x7ff6f0077210 msgr2=0x7ff6f0077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.332+0000 7ff6f6569640 1 -- 172.21.15.5:0/43725523 shutdown_connections 2024-09-15T22:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.332+0000 7ff6f6569640 1 -- 172.21.15.5:0/43725523 wait complete. 2024-09-15T22:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.333+0000 7ff6f6569640 1 Processor -- start 2024-09-15T22:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.333+0000 7ff6f6569640 1 -- start start 2024-09-15T22:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.333+0000 7ff6f6569640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6f01008e0 0x7ff6f01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.333+0000 7ff6f6569640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f0101290 0x7ff6f0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.333+0000 7ff6f6569640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6f0108160 0x7ff6f019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.333+0000 7ff6f6569640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f0111fd0 con 0x7ff6f01008e0 2024-09-15T22:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.333+0000 7ff6f6569640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f0111e50 con 0x7ff6f0101290 2024-09-15T22:09:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.333+0000 7ff6f6569640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f0112150 con 0x7ff6f0108160 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.333+0000 7ff6effff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6f01008e0 0x7ff6f01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.333+0000 7ff6effff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6f01008e0 0x7ff6f01984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56220/0 (socket says 172.21.15.5:56220) 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.333+0000 7ff6effff640 1 -- 172.21.15.5:0/2874424301 learned_addr learned my addr 172.21.15.5:0/2874424301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f0101290 0x7ff6f0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6f0108160 0x7ff6f019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6effff640 1 -- 172.21.15.5:0/2874424301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6f0108160 msgr2=0x7ff6f019cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6effff640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6f0108160 0x7ff6f019cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6effff640 1 -- 172.21.15.5:0/2874424301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f0101290 msgr2=0x7ff6f0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6effff640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f0101290 0x7ff6f0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6effff640 1 -- 172.21.15.5:0/2874424301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f019d470 con 0x7ff6f01008e0 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6effff640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6f01008e0 0x7ff6f01984c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009230 tx=0x7ff6e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6ed7fa640 1 -- 172.21.15.5:0/2874424301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e003f030 con 0x7ff6f01008e0 2024-09-15T22:09:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6ed7fa640 1 -- 172.21.15.5:0/2874424301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e0003ad0 con 0x7ff6f01008e0 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6f6569640 1 -- 172.21.15.5:0/2874424301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f019d700 con 0x7ff6f01008e0 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6ed7fa640 1 -- 172.21.15.5:0/2874424301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0035070 con 0x7ff6f01008e0 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.334+0000 7ff6f6569640 1 -- 172.21.15.5:0/2874424301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f01a5040 con 0x7ff6f01008e0 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.337+0000 7ff6f8942740 1 Processor -- start 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.337+0000 7ff6ed7fa640 1 -- 172.21.15.5:0/2874424301 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7ff6e0042030 con 0x7ff6f01008e0 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.337+0000 7ff6f8942740 1 -- start start 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e1ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1b360 0x55e7e4e29c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e2a150 0x55e7e4e2c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f8942740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e7e4d80280 con 0x55e7e4e1aa40 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f8942740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e7e4d27b10 con 0x55e7e4e1b360 2024-09-15T22:09:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f8942740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7e4d7c240 con 0x55e7e4e2a150 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f4b06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e1ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f4b06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e1ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56228/0 (socket says 172.21.15.5:56228) 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f4b06640 1 -- 172.21.15.5:0/2457194359 learned_addr learned my addr 172.21.15.5:0/2457194359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6effff640 1 --2- 172.21.15.5:0/2457194359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1b360 0x55e7e4e29c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/2457194359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e2a150 0x55e7e4e2c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f4b06640 1 -- 172.21.15.5:0/2457194359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e2a150 msgr2=0x55e7e4e2c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/2457194359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e2a150 0x55e7e4e2c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f4b06640 1 -- 172.21.15.5:0/2457194359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1b360 msgr2=0x55e7e4e29c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/2457194359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1b360 0x55e7e4e29c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6f4b06640 1 -- 172.21.15.5:0/2457194359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e4d341f0 con 0x55e7e4e1aa40 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.338+0000 7ff6ed7fa640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff6d80a34d0 0x7ff6d80a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.339+0000 7ff6effff640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff6d80a34d0 0x7ff6d80a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.339+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/2457194359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e1ae20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009f70 tx=0x7ff6e8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce1d193a787de2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.339+0000 7ff6ed7fa640 1 -- 172.21.15.5:0/2874424301 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7ff6e00f0850 con 0x7ff6f01008e0 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.339+0000 7ff6d67fc640 1 -- 172.21.15.5:0/2457194359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8016070 con 0x55e7e4e1aa40 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.339+0000 7ff6d67fc640 1 -- 172.21.15.5:0/2457194359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e8004d20 con 0x55e7e4e1aa40 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.339+0000 7ff6d67fc640 1 -- 172.21.15.5:0/2457194359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8012910 con 0x55e7e4e1aa40 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 -- 172.21.15.5:0/2457194359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1aa40 msgr2=0x55e7e4e1ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6effff640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff6d80a34d0 0x7ff6d80a5990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00edae0 tx=0x7ff6e00f6510 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 --2- 172.21.15.5:0/2457194359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e1ae20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009f70 tx=0x7ff6e8010800 comp rx=0 tx=0).stop 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 -- 172.21.15.5:0/2457194359 shutdown_connections 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 --2- 172.21.15.5:0/2457194359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e2a150 0x55e7e4e2c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 --2- 172.21.15.5:0/2457194359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1b360 0x55e7e4e29c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 --2- 172.21.15.5:0/2457194359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e1ae20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 -- 172.21.15.5:0/2457194359 >> 172.21.15.5:0/2457194359 conn(0x55e7e4e170f0 msgr2=0x55e7e4e174d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 -- 172.21.15.5:0/2457194359 shutdown_connections 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 -- 172.21.15.5:0/2457194359 wait complete. 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 -- start start 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 Processor -- start 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.340+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e29910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1b360 0x55e7e4e22cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e231f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f8942740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e7e4e2cd70 con 0x55e7e4e2a150 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f8942740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e7e4d3af10 con 0x55e7e4e1aa40 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f8942740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e7e4e2cef0 con 0x55e7e4e1b360 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f8942740 10 client.0 _ll_register_callbacks cb 0x55e7e4d03a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f4b06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6ef7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1b360 0x55e7e4e22cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f4b06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e231f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56238/0 (socket says 172.21.15.5:56238) 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f4b06640 1 -- 172.21.15.5:0/1754070023 learned_addr learned my addr 172.21.15.5:0/1754070023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e29910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f4b06640 1 -- 172.21.15.5:0/1754070023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1b360 msgr2=0x55e7e4e22cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1b360 0x55e7e4e22cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f4b06640 1 -- 172.21.15.5:0/1754070023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1aa40 msgr2=0x55e7e4e29910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e29910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f4b06640 1 -- 172.21.15.5:0/1754070023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e4af16a0 con 0x55e7e4e2a150 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1b360 0x55e7e4e22cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e231f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8005f40 tx=0x7ff6e80124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba2b7b3410b58fb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.341+0000 7ff6cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/1754070023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8016070 con 0x55e7e4e2a150 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/1754070023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7ff6e8012a50 con 0x55e7e4e2a150 2024-09-15T22:09:03.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6f8942740 1 -- 172.21.15.5:0/1754070023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e8004d20 con 0x55e7e4e2a150 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/1754070023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6e8004ed0 con 0x55e7e4e2a150 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6f8942740 10 client.15387 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6f8942740 1 -- 172.21.15.5:0/1754070023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e7e4e282a0 con 0x55e7e4e2a150 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/1754070023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e8004d20 con 0x55e7e4e2a150 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/1754070023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff6e800f040 con 0x55e7e4e2a150 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6f8942740 10 client.15387 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6f8942740 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6f8942740 20 client.15387 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6f8942740 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e7e4e3e1d0 0x55e7e4e405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6f8942740 4 client.15387 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.342+0000 7ff6f8942740 1 -- 172.21.15.5:0/1754070023 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7e4d8d7f0 con 0x55e7e4e3e1d0 2024-09-15T22:09:03.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.343+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e7e4e3e1d0 0x55e7e4e405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.343+0000 7ff6cf7fe640 10 client.15387 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.343+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e7e4e3e1d0 0x55e7e4e405c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea0d810590f0de4e server_cookie=3b97c34ce2cf9baf in_seq=0 out_seq=0 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/1754070023 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7e4d8d7f0 con 0x55e7e4e3e1d0 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6cf7fe640 10 client.15387 handle_command_reply: tid=0 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6f8942740 1 client.15387 shutdown 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6f8942740 1 -- 172.21.15.5:0/1754070023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 msgr2=0x55e7e4e231f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6f8942740 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e231f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8005f40 tx=0x7ff6e80124f0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6f8942740 1 -- 172.21.15.5:0/1754070023 shutdown_connections 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6f8942740 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e231f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6f8942740 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1b360 0x55e7e4e22cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6f8942740 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e7e4e3e1d0 0x55e7e4e405c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6f8942740 1 --2- 172.21.15.5:0/1754070023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e29910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6cf7fe640 0 client.15387 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6f8942740 1 -- 172.21.15.5:0/1754070023 >> 172.21.15.5:0/1754070023 conn(0x55e7e4e170f0 msgr2=0x55e7e4e1ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.344+0000 7ff6f8942740 1 -- 172.21.15.5:0/1754070023 shutdown_connections 2024-09-15T22:09:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.345+0000 7ff6f8942740 1 -- 172.21.15.5:0/1754070023 wait complete. 2024-09-15T22:09:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.345+0000 7ff6f8942740 20 client.15387 trim_cache size 0 max 16384 2024-09-15T22:09:03.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6f8942740 1 Processor -- start 2024-09-15T22:09:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6f8942740 1 -- start start 2024-09-15T22:09:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e27190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1b360 0x55e7e4e16160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e2a150 0x55e7e4e45d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6f8942740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01af7d0 con 0x55e7e4e1b360 2024-09-15T22:09:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6f8942740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01ad0a0 con 0x55e7e4e2a150 2024-09-15T22:09:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6f8942740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01aaef0 con 0x55e7e4e1aa40 2024-09-15T22:09:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6ef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1b360 0x55e7e4e16160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6ef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1b360 0x55e7e4e16160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56250/0 (socket says 172.21.15.5:56250) 2024-09-15T22:09:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6ef7fe640 1 -- 172.21.15.5:0/126046899 learned_addr learned my addr 172.21.15.5:0/126046899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/126046899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e27190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.362+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/126046899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e2a150 0x55e7e4e45d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.363+0000 7ff6ef7fe640 1 -- 172.21.15.5:0/126046899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1aa40 msgr2=0x55e7e4e27190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:03.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.363+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/126046899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e27190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.363+0000 7ff6ef7fe640 1 -- 172.21.15.5:0/126046899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e2a150 msgr2=0x55e7e4e45d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:03.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.363+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/126046899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e2a150 0x55e7e4e45d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.363+0000 7ff6ef7fe640 1 -- 172.21.15.5:0/126046899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7e4e21c60 con 0x55e7e4e1b360 2024-09-15T22:09:03.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.363+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/126046899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1b360 0x55e7e4e16160 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0078330 tx=0x7ff6e4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f1ab7cf69ee01f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.363+0000 7ff6d5ffb640 1 -- 172.21.15.5:0/126046899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40122d0 con 0x55e7e4e1b360 2024-09-15T22:09:03.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.363+0000 7ff6d5ffb640 1 -- 172.21.15.5:0/126046899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e4005020 con 0x55e7e4e1b360 2024-09-15T22:09:03.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.363+0000 7ff6d5ffb640 1 -- 172.21.15.5:0/126046899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e400fbd0 con 0x55e7e4e1b360 2024-09-15T22:09:03.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.364+0000 7ff6f8942740 1 -- 172.21.15.5:0/126046899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1b360 msgr2=0x55e7e4e16160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.364+0000 7ff6f8942740 1 --2- 172.21.15.5:0/126046899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1b360 0x55e7e4e16160 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0078330 tx=0x7ff6e4005f90 comp rx=0 tx=0).stop 2024-09-15T22:09:03.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.364+0000 7ff6f8942740 1 -- 172.21.15.5:0/126046899 shutdown_connections 2024-09-15T22:09:03.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.364+0000 7ff6f8942740 1 --2- 172.21.15.5:0/126046899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e2a150 0x55e7e4e45d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.364+0000 7ff6f8942740 1 --2- 172.21.15.5:0/126046899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e1b360 0x55e7e4e16160 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.364+0000 7ff6f8942740 1 --2- 172.21.15.5:0/126046899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e27190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.364+0000 7ff6f8942740 1 -- 172.21.15.5:0/126046899 >> 172.21.15.5:0/126046899 conn(0x55e7e4e170f0 msgr2=0x55e7e4e15a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.364+0000 7ff6f8942740 1 -- 172.21.15.5:0/126046899 shutdown_connections 2024-09-15T22:09:03.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.364+0000 7ff6f8942740 1 -- 172.21.15.5:0/126046899 wait complete. 2024-09-15T22:09:03.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.364+0000 7ff6f8942740 1 -- start start 2024-09-15T22:09:03.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.364+0000 7ff6f8942740 1 Processor -- start 2024-09-15T22:09:03.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e3aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1b360 0x55e7e4e373d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f8942740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e37910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f8942740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01ab560 con 0x55e7e4e2a150 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f8942740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01ac9d0 con 0x55e7e4e1b360 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f8942740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f01abc30 con 0x55e7e4e1aa40 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f4b06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e37910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f4b06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e37910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56258/0 (socket says 172.21.15.5:56258) 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f4b06640 1 -- 172.21.15.5:0/2474127296 learned_addr learned my addr 172.21.15.5:0/2474127296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f8942740 10 client.0 _ll_register_callbacks cb 0x7ff6e00f63e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1b360 0x55e7e4e373d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e3aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f4b06640 1 -- 172.21.15.5:0/2474127296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1aa40 msgr2=0x55e7e4e3aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e3aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f4b06640 1 -- 172.21.15.5:0/2474127296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1b360 msgr2=0x55e7e4e373d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1b360 0x55e7e4e373d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f4b06640 1 -- 172.21.15.5:0/2474127296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7e4e1b190 con 0x55e7e4e2a150 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1b360 0x55e7e4e373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6f4b06640 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e37910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8002230 tx=0x7ff6e80124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23d661450ff77fb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.365+0000 7ff6cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/2474127296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e8016040 con 0x55e7e4e2a150 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/2474127296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7ff6e8023070 con 0x55e7e4e2a150 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6f8942740 1 -- 172.21.15.5:0/2474127296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6e4005020 con 0x55e7e4e2a150 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/2474127296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff6e8012a50 con 0x55e7e4e2a150 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6f8942740 10 client.15393 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:03.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6f8942740 1 -- 172.21.15.5:0/2474127296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e7e4ea9870 con 0x55e7e4e2a150 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/2474127296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6e4005020 con 0x55e7e4e2a150 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/2474127296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff6e8029800 con 0x55e7e4e2a150 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6f8942740 10 client.15393 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6f8942740 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6f8942740 20 client.15393 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.366+0000 7ff6f8942740 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e7e4ea9e70 0x55e7e4eac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.367+0000 7ff6f8942740 4 client.15393 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.367+0000 7ff6f8942740 1 -- 172.21.15.5:0/2474127296 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e7e4d87d00 con 0x55e7e4ea9e70 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.367+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e7e4ea9e70 0x55e7e4eac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.367+0000 7ff6cf7fe640 10 client.15393 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.367+0000 7ff6ef7fe640 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e7e4ea9e70 0x55e7e4eac260 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31a08e97016132da server_cookie=c4a64b0470785701 in_seq=0 out_seq=0 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.368+0000 7ff6cf7fe640 1 -- 172.21.15.5:0/2474127296 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e7e4d87d00 con 0x55e7e4ea9e70 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.368+0000 7ff6cf7fe640 10 client.15393 handle_command_reply: tid=0 2024-09-15T22:09:03.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.368+0000 7ff6f8942740 1 client.15393 shutdown 2024-09-15T22:09:03.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6f8942740 1 -- 172.21.15.5:0/2474127296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 msgr2=0x55e7e4e37910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6f8942740 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e37910 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8002230 tx=0x7ff6e80124f0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6f8942740 1 -- 172.21.15.5:0/2474127296 shutdown_connections 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6f8942740 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7e4e2a150 0x55e7e4e37910 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6f8942740 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e7e4ea9e70 0x55e7e4eac260 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6f8942740 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e7e4e1b360 0x55e7e4e373d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6f8942740 1 --2- 172.21.15.5:0/2474127296 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e7e4e1aa40 0x55e7e4e3aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6f8942740 1 -- 172.21.15.5:0/2474127296 >> 172.21.15.5:0/2474127296 conn(0x55e7e4e170f0 msgr2=0x55e7e4e28d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6cf7fe640 0 client.15393 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6f8942740 1 -- 172.21.15.5:0/2474127296 shutdown_connections 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6f8942740 1 -- 172.21.15.5:0/2474127296 wait complete. 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.369+0000 7ff6f8942740 20 client.15393 trim_cache size 0 max 16384 2024-09-15T22:09:03.375 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:09:03.376 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:09:03.376 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "267dcd72-9f77-4024-bf52-4f92af68be48", 2024-09-15T22:09:03.376 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:09:03.376 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:09:03.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 -- 172.21.15.5:0/2874424301 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff6d80a34d0 msgr2=0x7ff6d80a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff6d80a34d0 0x7ff6d80a5990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff6e00edae0 tx=0x7ff6e00f6510 comp rx=0 tx=0).stop 2024-09-15T22:09:03.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 -- 172.21.15.5:0/2874424301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6f01008e0 msgr2=0x7ff6f01984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6f01008e0 0x7ff6f01984c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff6e0009230 tx=0x7ff6e0004270 comp rx=0 tx=0).stop 2024-09-15T22:09:03.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 -- 172.21.15.5:0/2874424301 shutdown_connections 2024-09-15T22:09:03.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff6d80a34d0 0x7ff6d80a5990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff6f0108160 0x7ff6f019cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff6f0101290 0x7ff6f0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 --2- 172.21.15.5:0/2874424301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6f01008e0 0x7ff6f01984c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 -- 172.21.15.5:0/2874424301 >> 172.21.15.5:0/2874424301 conn(0x7ff6f0077210 msgr2=0x7ff6f00ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 -- 172.21.15.5:0/2874424301 shutdown_connections 2024-09-15T22:09:03.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.370+0000 7ff6f6569640 1 -- 172.21.15.5:0/2874424301 wait complete. 2024-09-15T22:09:03.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.522+0000 7f68fb5af640 1 -- 172.21.15.5:0/3622393237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f68f4105850 con 0x7f68f4100a70 2024-09-15T22:09:03.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.526+0000 7f68ea7fc640 1 -- 172.21.15.5:0/3622393237 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7f68f009c4f0 con 0x7f68f4100a70 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 -- 172.21.15.5:0/3622393237 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68e00a34d0 msgr2=0x7f68e00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68e00a34d0 0x7f68e00a5990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f68f41065c0 tx=0x7f68e4005f90 comp rx=0 tx=0).stop 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 -- 172.21.15.5:0/3622393237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68f4100a70 msgr2=0x7f68f41a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68f4100a70 0x7f68f41a46a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f68f000cce0 tx=0x7f68f000a620 comp rx=0 tx=0).stop 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 -- 172.21.15.5:0/3622393237 shutdown_connections 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68e00a34d0 0x7f68e00a5990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f68f4105ea0 0x7f68f419e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f68f4101390 0x7f68f41a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 --2- 172.21.15.5:0/3622393237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68f4100a70 0x7f68f41a46a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 -- 172.21.15.5:0/3622393237 >> 172.21.15.5:0/3622393237 conn(0x7f68f40779a0 msgr2=0x7f68f4103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 -- 172.21.15.5:0/3622393237 shutdown_connections 2024-09-15T22:09:03.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.528+0000 7f68fb5af640 1 -- 172.21.15.5:0/3622393237 wait complete. 2024-09-15T22:09:03.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-15T22:09:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.658+0000 7fae226f1640 1 -- 172.21.15.5:0/1692462447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae1c101ad0 msgr2=0x7fae1c101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.658+0000 7fae226f1640 1 --2- 172.21.15.5:0/1692462447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae1c101ad0 0x7fae1c101f30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fae04009930 tx=0x7fae0402a420 comp rx=0 tx=0).stop 2024-09-15T22:09:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.658+0000 7fae226f1640 1 -- 172.21.15.5:0/1692462447 shutdown_connections 2024-09-15T22:09:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.658+0000 7fae226f1640 1 --2- 172.21.15.5:0/1692462447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c102470 0x7fae1c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.658+0000 7fae226f1640 1 --2- 172.21.15.5:0/1692462447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae1c101ad0 0x7fae1c101f30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.658+0000 7fae226f1640 1 --2- 172.21.15.5:0/1692462447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae1c107b70 0x7fae1c107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.658+0000 7fae226f1640 1 -- 172.21.15.5:0/1692462447 >> 172.21.15.5:0/1692462447 conn(0x7fae1c0fd7d0 msgr2=0x7fae1c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.658+0000 7fae226f1640 1 -- 172.21.15.5:0/1692462447 shutdown_connections 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.658+0000 7fae226f1640 1 -- 172.21.15.5:0/1692462447 wait complete. 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.659+0000 7fae226f1640 1 Processor -- start 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.659+0000 7fae226f1640 1 -- start start 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.659+0000 7fae226f1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae1c101ad0 0x7fae1c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.659+0000 7fae226f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c102470 0x7fae1c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.659+0000 7fae226f1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae1c107b70 0x7fae1c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.659+0000 7fae226f1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c111ee0 con 0x7fae1c102470 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.659+0000 7fae226f1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c111d60 con 0x7fae1c107b70 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.659+0000 7fae226f1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c112060 con 0x7fae1c101ad0 2024-09-15T22:09:03.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.659+0000 7fae13fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c102470 0x7fae1c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.659+0000 7fae13fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c102470 0x7fae1c198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46462/0 (socket says 172.21.15.5:46462) 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.659+0000 7fae13fff640 1 -- 172.21.15.5:0/503261750 learned_addr learned my addr 172.21.15.5:0/503261750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae20c8e640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae1c107b70 0x7fae1c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae1bfff640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae1c101ad0 0x7fae1c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae13fff640 1 -- 172.21.15.5:0/503261750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae1c101ad0 msgr2=0x7fae1c1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae13fff640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae1c101ad0 0x7fae1c1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae13fff640 1 -- 172.21.15.5:0/503261750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae1c107b70 msgr2=0x7fae1c19ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae13fff640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae1c107b70 0x7fae1c19ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae13fff640 1 -- 172.21.15.5:0/503261750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1c19d3e0 con 0x7fae1c102470 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae13fff640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c102470 0x7fae1c198930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fae04009230 tx=0x7fae04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae19ffb640 1 -- 172.21.15.5:0/503261750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0403f030 con 0x7fae1c102470 2024-09-15T22:09:03.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae226f1640 1 -- 172.21.15.5:0/503261750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1c19d670 con 0x7fae1c102470 2024-09-15T22:09:03.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae19ffb640 1 -- 172.21.15.5:0/503261750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae04003ad0 con 0x7fae1c102470 2024-09-15T22:09:03.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae19ffb640 1 -- 172.21.15.5:0/503261750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae04035070 con 0x7fae1c102470 2024-09-15T22:09:03.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.660+0000 7fae226f1640 1 -- 172.21.15.5:0/503261750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae1c1a4f60 con 0x7fae1c102470 2024-09-15T22:09:03.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.661+0000 7fae226f1640 1 -- 172.21.15.5:0/503261750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae1c101f30 con 0x7fae1c102470 2024-09-15T22:09:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.663+0000 7fae19ffb640 1 -- 172.21.15.5:0/503261750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7fae04042030 con 0x7fae1c102470 2024-09-15T22:09:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.663+0000 7fae19ffb640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fae080a34d0 0x7fae080a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.663+0000 7fae19ffb640 1 -- 172.21.15.5:0/503261750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fae040f0660 con 0x7fae1c102470 2024-09-15T22:09:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.663+0000 7fae1bfff640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fae080a34d0 0x7fae080a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:03.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.664+0000 7fae1bfff640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fae080a34d0 0x7fae080a5990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fae0c005fd0 tx=0x7fae0c005850 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:03.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.668+0000 7fae19ffb640 1 -- 172.21.15.5:0/503261750 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fae040bcb20 con 0x7fae1c102470 2024-09-15T22:09:03.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.866+0000 7fae226f1640 1 -- 172.21.15.5:0/503261750 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fae1c10a070 con 0x7fae080a34d0 2024-09-15T22:09:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.931+0000 7fae19ffb640 1 -- 172.21.15.5:0/503261750 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fae1c10a070 con 0x7fae080a34d0 2024-09-15T22:09:03.932 INFO:teuthology.orchestra.run.smithi005.stdout:Schedule set for path / 2024-09-15T22:09:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 -- 172.21.15.5:0/503261750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fae080a34d0 msgr2=0x7fae080a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fae080a34d0 0x7fae080a5990 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fae0c005fd0 tx=0x7fae0c005850 comp rx=0 tx=0).stop 2024-09-15T22:09:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 -- 172.21.15.5:0/503261750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c102470 msgr2=0x7fae1c198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c102470 0x7fae1c198930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fae04009230 tx=0x7fae04004270 comp rx=0 tx=0).stop 2024-09-15T22:09:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 -- 172.21.15.5:0/503261750 shutdown_connections 2024-09-15T22:09:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fae080a34d0 0x7fae080a5990 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fae1c107b70 0x7fae1c19ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae1c102470 0x7fae1c198930 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 --2- 172.21.15.5:0/503261750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fae1c101ad0 0x7fae1c1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 -- 172.21.15.5:0/503261750 >> 172.21.15.5:0/503261750 conn(0x7fae1c0fd7d0 msgr2=0x7fae1c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 -- 172.21.15.5:0/503261750 shutdown_connections 2024-09-15T22:09:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:03.933+0000 7fae226f1640 1 -- 172.21.15.5:0/503261750 wait complete. 2024-09-15T22:09:03.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-15T22:09:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.050+0000 7f5cdc78d640 1 -- 172.21.15.5:0/4285346821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cd4101340 msgr2=0x7f5cd410d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.050+0000 7f5cdc78d640 1 --2- 172.21.15.5:0/4285346821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cd4101340 0x7f5cd410d870 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009cd0 tx=0x7f5ccc02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.050+0000 7f5cdc78d640 1 -- 172.21.15.5:0/4285346821 shutdown_connections 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.050+0000 7f5cdc78d640 1 --2- 172.21.15.5:0/4285346821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cd4101340 0x7f5cd410d870 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.050+0000 7f5cdc78d640 1 --2- 172.21.15.5:0/4285346821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd41009a0 0x7f5cd4100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.050+0000 7f5cdc78d640 1 --2- 172.21.15.5:0/4285346821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd41069b0 0x7f5cd4106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.050+0000 7f5cdc78d640 1 -- 172.21.15.5:0/4285346821 >> 172.21.15.5:0/4285346821 conn(0x7f5cd40779a0 msgr2=0x7f5cd40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.050+0000 7f5cdc78d640 1 -- 172.21.15.5:0/4285346821 shutdown_connections 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.050+0000 7f5cdc78d640 1 -- 172.21.15.5:0/4285346821 wait complete. 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.051+0000 7f5cdc78d640 1 Processor -- start 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.051+0000 7f5cdc78d640 1 -- start start 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.051+0000 7f5cdc78d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cd41009a0 0x7f5cd410e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.051+0000 7f5cdc78d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd4101340 0x7f5cd410e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.051+0000 7f5cdc78d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd41069b0 0x7f5cd410ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.051+0000 7f5cdc78d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4111ee0 con 0x7f5cd41009a0 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.051+0000 7f5cdc78d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4111d60 con 0x7f5cd4101340 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.051+0000 7f5cdc78d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd4112060 con 0x7f5cd41069b0 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.051+0000 7f5cda529640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cd41009a0 0x7f5cd410e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.051+0000 7f5cda529640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cd41009a0 0x7f5cd410e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46478/0 (socket says 172.21.15.5:46478) 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.051+0000 7f5cda529640 1 -- 172.21.15.5:0/2956537588 learned_addr learned my addr 172.21.15.5:0/2956537588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5cd9d28640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd4101340 0x7f5cd410e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5cdad2a640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd41069b0 0x7f5cd410ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5cda529640 1 -- 172.21.15.5:0/2956537588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd41069b0 msgr2=0x7f5cd410ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5cda529640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd41069b0 0x7f5cd410ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5cda529640 1 -- 172.21.15.5:0/2956537588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd4101340 msgr2=0x7f5cd410e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5cda529640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd4101340 0x7f5cd410e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5cda529640 1 -- 172.21.15.5:0/2956537588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd41a6910 con 0x7f5cd41009a0 2024-09-15T22:09:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5cda529640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cd41009a0 0x7f5cd410e180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400d770 tx=0x7f5cc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5ccb7fe640 1 -- 172.21.15.5:0/2956537588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4017070 con 0x7f5cd41009a0 2024-09-15T22:09:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5cdc78d640 1 -- 172.21.15.5:0/2956537588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd41a6c00 con 0x7f5cd41009a0 2024-09-15T22:09:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5ccb7fe640 1 -- 172.21.15.5:0/2956537588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc4005500 con 0x7f5cd41009a0 2024-09-15T22:09:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5ccb7fe640 1 -- 172.21.15.5:0/2956537588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4004e00 con 0x7f5cd41009a0 2024-09-15T22:09:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.052+0000 7f5cdc78d640 1 -- 172.21.15.5:0/2956537588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd41a7150 con 0x7f5cd41009a0 2024-09-15T22:09:04.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.054+0000 7f5cdc78d640 1 -- 172.21.15.5:0/2956537588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd41020e0 con 0x7f5cd41009a0 2024-09-15T22:09:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.055+0000 7f5ccb7fe640 1 -- 172.21.15.5:0/2956537588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f5cc4004fa0 con 0x7f5cd41009a0 2024-09-15T22:09:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.056+0000 7f5ccb7fe640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5cc00a34d0 0x7f5cc00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.056+0000 7f5cd9d28640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5cc00a34d0 0x7f5cc00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.056+0000 7f5ccb7fe640 1 -- 172.21.15.5:0/2956537588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f5cc40cfe10 con 0x7f5cd41009a0 2024-09-15T22:09:04.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.056+0000 7f5cd9d28640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5cc00a34d0 0x7f5cc00a5990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005fd0 tx=0x7f5cd0005850 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.061+0000 7f5ccb7fe640 1 -- 172.21.15.5:0/2956537588 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cc409c200 con 0x7f5cd41009a0 2024-09-15T22:09:04.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:09:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.261+0000 7f5cdc78d640 1 -- 172.21.15.5:0/2956537588 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f5cd4062630 con 0x7f5cc00a34d0 2024-09-15T22:09:04.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.327+0000 7f5ccb7fe640 1 -- 172.21.15.5:0/2956537588 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f5cd4062630 con 0x7f5cc00a34d0 2024-09-15T22:09:04.328 INFO:teuthology.orchestra.run.smithi005.stdout:Retention added to path / 2024-09-15T22:09:04.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 -- 172.21.15.5:0/2956537588 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5cc00a34d0 msgr2=0x7f5cc00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5cc00a34d0 0x7f5cc00a5990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0005fd0 tx=0x7f5cd0005850 comp rx=0 tx=0).stop 2024-09-15T22:09:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 -- 172.21.15.5:0/2956537588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cd41009a0 msgr2=0x7f5cd410e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cd41009a0 0x7f5cd410e180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400d770 tx=0x7f5cc400dc30 comp rx=0 tx=0).stop 2024-09-15T22:09:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 -- 172.21.15.5:0/2956537588 shutdown_connections 2024-09-15T22:09:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5cc00a34d0 0x7f5cc00a5990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5cd41069b0 0x7f5cd410ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5cd4101340 0x7f5cd410e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 --2- 172.21.15.5:0/2956537588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cd41009a0 0x7f5cd410e180 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 -- 172.21.15.5:0/2956537588 >> 172.21.15.5:0/2956537588 conn(0x7f5cd40779a0 msgr2=0x7f5cd4103900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 -- 172.21.15.5:0/2956537588 shutdown_connections 2024-09-15T22:09:04.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.329+0000 7f5cdc78d640 1 -- 172.21.15.5:0/2956537588 wait complete. 2024-09-15T22:09:04.338 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:04.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/910105866' entity='client.admin' 2024-09-15T22:09:04.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3102748595' entity='client.admin' 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3622393237' entity='client.admin' 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:09:04.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/910105866' entity='client.admin' 2024-09-15T22:09:04.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:04.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:04.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:04.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:09:04.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3102748595' entity='client.admin' 2024-09-15T22:09:04.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:04.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:04.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:04.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:09:04.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3622393237' entity='client.admin' 2024-09-15T22:09:04.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:04.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:04.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:04.342 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:09:04.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-15T22:09:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/910105866' entity='client.admin' 2024-09-15T22:09:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3102748595' entity='client.admin' 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3622393237' entity='client.admin' 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:09:04.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:09:04.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.448+0000 7fa9357e8640 1 -- 172.21.15.5:0/2496608707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa930102ca0 msgr2=0x7fa930103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.448+0000 7fa9357e8640 1 --2- 172.21.15.5:0/2496608707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa930102ca0 0x7fa930103080 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa920009900 tx=0x7fa92002a480 comp rx=0 tx=0).stop 2024-09-15T22:09:04.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.448+0000 7fa9357e8640 1 -- 172.21.15.5:0/2496608707 shutdown_connections 2024-09-15T22:09:04.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.448+0000 7fa9357e8640 1 --2- 172.21.15.5:0/2496608707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa93010c0f0 0x7fa93010e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.448+0000 7fa9357e8640 1 --2- 172.21.15.5:0/2496608707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa9301035c0 0x7fa93010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.448+0000 7fa9357e8640 1 --2- 172.21.15.5:0/2496608707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa930102ca0 0x7fa930103080 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.448+0000 7fa9357e8640 1 -- 172.21.15.5:0/2496608707 >> 172.21.15.5:0/2496608707 conn(0x7fa9300fc6b0 msgr2=0x7fa9300fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:04.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.448+0000 7fa9357e8640 1 -- 172.21.15.5:0/2496608707 shutdown_connections 2024-09-15T22:09:04.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.448+0000 7fa9357e8640 1 -- 172.21.15.5:0/2496608707 wait complete. 2024-09-15T22:09:04.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.448+0000 7fa9357e8640 1 Processor -- start 2024-09-15T22:09:04.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.448+0000 7fa9357e8640 1 -- start start 2024-09-15T22:09:04.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa9357e8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa930102ca0 0x7fa9301984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa9357e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9301035c0 0x7fa9301989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa9357e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa93010c0f0 0x7fa93019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa9357e8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa930111f60 con 0x7fa93010c0f0 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa9357e8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa930111de0 con 0x7fa9301035c0 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa9357e8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa9301120e0 con 0x7fa930102ca0 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa93010c0f0 0x7fa93019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa93010c0f0 0x7fa93019cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46502/0 (socket says 172.21.15.5:46502) 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92f7fe640 1 -- 172.21.15.5:0/4291074266 learned_addr learned my addr 172.21.15.5:0/4291074266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92e7fc640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9301035c0 0x7fa9301989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92effd640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa930102ca0 0x7fa9301984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92f7fe640 1 -- 172.21.15.5:0/4291074266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa930102ca0 msgr2=0x7fa9301984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92f7fe640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa930102ca0 0x7fa9301984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92f7fe640 1 -- 172.21.15.5:0/4291074266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9301035c0 msgr2=0x7fa9301989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92f7fe640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9301035c0 0x7fa9301989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92f7fe640 1 -- 172.21.15.5:0/4291074266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa93019d4a0 con 0x7fa93010c0f0 2024-09-15T22:09:04.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92effd640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa930102ca0 0x7fa9301984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:04.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.449+0000 7fa92f7fe640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa93010c0f0 0x7fa93019cd70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa92800b970 tx=0x7fa92800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.450+0000 7fa917fff640 1 -- 172.21.15.5:0/4291074266 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928019070 con 0x7fa93010c0f0 2024-09-15T22:09:04.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.450+0000 7fa917fff640 1 -- 172.21.15.5:0/4291074266 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa92800cb40 con 0x7fa93010c0f0 2024-09-15T22:09:04.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.450+0000 7fa9357e8640 1 -- 172.21.15.5:0/4291074266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa93019d790 con 0x7fa93010c0f0 2024-09-15T22:09:04.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.450+0000 7fa917fff640 1 -- 172.21.15.5:0/4291074266 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa928007650 con 0x7fa93010c0f0 2024-09-15T22:09:04.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.450+0000 7fa9357e8640 1 -- 172.21.15.5:0/4291074266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9301a5020 con 0x7fa93010c0f0 2024-09-15T22:09:04.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.451+0000 7fa9357e8640 1 -- 172.21.15.5:0/4291074266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa930103080 con 0x7fa93010c0f0 2024-09-15T22:09:04.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.452+0000 7fa917fff640 1 -- 172.21.15.5:0/4291074266 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7fa92801c070 con 0x7fa93010c0f0 2024-09-15T22:09:04.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.452+0000 7fa917fff640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9180a34d0 0x7fa9180a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.453+0000 7fa92effd640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9180a34d0 0x7fa9180a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.453+0000 7fa917fff640 1 -- 172.21.15.5:0/4291074266 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa9280cf860 con 0x7fa93010c0f0 2024-09-15T22:09:04.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.453+0000 7fa92effd640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9180a34d0 0x7fa9180a5990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa920000c00 tx=0x7fa920005a20 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.457+0000 7fa917fff640 1 -- 172.21.15.5:0/4291074266 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa92809b6b0 con 0x7fa93010c0f0 2024-09-15T22:09:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.673+0000 7fa9357e8640 1 -- 172.21.15.5:0/4291074266 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa930062630 con 0x7fa9180a34d0 2024-09-15T22:09:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.681+0000 7fa917fff640 1 -- 172.21.15.5:0/4291074266 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fa930062630 con 0x7fa9180a34d0 2024-09-15T22:09:04.683 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-15T00:00:00", "created": "2024-09-15T22:09:03", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-15T22:09:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 -- 172.21.15.5:0/4291074266 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9180a34d0 msgr2=0x7fa9180a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9180a34d0 0x7fa9180a5990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa920000c00 tx=0x7fa920005a20 comp rx=0 tx=0).stop 2024-09-15T22:09:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 -- 172.21.15.5:0/4291074266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa93010c0f0 msgr2=0x7fa93019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa93010c0f0 0x7fa93019cd70 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa92800b970 tx=0x7fa92800be30 comp rx=0 tx=0).stop 2024-09-15T22:09:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 -- 172.21.15.5:0/4291074266 shutdown_connections 2024-09-15T22:09:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9180a34d0 0x7fa9180a5990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa93010c0f0 0x7fa93019cd70 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa9301035c0 0x7fa9301989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 --2- 172.21.15.5:0/4291074266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa930102ca0 0x7fa9301984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 -- 172.21.15.5:0/4291074266 >> 172.21.15.5:0/4291074266 conn(0x7fa9300fc6b0 msgr2=0x7fa9300fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 -- 172.21.15.5:0/4291074266 shutdown_connections 2024-09-15T22:09:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.683+0000 7fa9357e8640 1 -- 172.21.15.5:0/4291074266 wait complete. 2024-09-15T22:09:04.695 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-15T22:09:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.800+0000 7f87ee5e8640 1 -- 172.21.15.5:0/2694503631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87e8102c80 msgr2=0x7f87e8103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.800+0000 7f87ee5e8640 1 --2- 172.21.15.5:0/2694503631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87e8102c80 0x7f87e8103060 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009900 tx=0x7f87d802a480 comp rx=0 tx=0).stop 2024-09-15T22:09:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.800+0000 7f87ee5e8640 1 -- 172.21.15.5:0/2694503631 shutdown_connections 2024-09-15T22:09:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.800+0000 7f87ee5e8640 1 --2- 172.21.15.5:0/2694503631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e810c0a0 0x7f87e810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.800+0000 7f87ee5e8640 1 --2- 172.21.15.5:0/2694503631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e81035a0 0x7f87e810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.800+0000 7f87ee5e8640 1 --2- 172.21.15.5:0/2694503631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87e8102c80 0x7f87e8103060 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.800+0000 7f87ee5e8640 1 -- 172.21.15.5:0/2694503631 >> 172.21.15.5:0/2694503631 conn(0x7f87e80779a0 msgr2=0x7f87e80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.801+0000 7f87ee5e8640 1 -- 172.21.15.5:0/2694503631 shutdown_connections 2024-09-15T22:09:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.801+0000 7f87ee5e8640 1 -- 172.21.15.5:0/2694503631 wait complete. 2024-09-15T22:09:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.801+0000 7f87ee5e8640 1 Processor -- start 2024-09-15T22:09:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.801+0000 7f87ee5e8640 1 -- start start 2024-09-15T22:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87ee5e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87e8102c80 0x7f87e819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87ee5e8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e81035a0 0x7f87e819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87ee5e8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e810c0a0 0x7f87e819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87ee5e8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8111ea0 con 0x7f87e8102c80 2024-09-15T22:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87ee5e8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8111d20 con 0x7f87e810c0a0 2024-09-15T22:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87ee5e8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8112020 con 0x7f87e81035a0 2024-09-15T22:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87e8102c80 0x7f87e819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87e8102c80 0x7f87e819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46516/0 (socket says 172.21.15.5:46516) 2024-09-15T22:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87e7fff640 1 -- 172.21.15.5:0/408404269 learned_addr learned my addr 172.21.15.5:0/408404269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87e77fe640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e81035a0 0x7f87e819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87ecb85640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e810c0a0 0x7f87e819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87e7fff640 1 -- 172.21.15.5:0/408404269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e81035a0 msgr2=0x7f87e819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87e7fff640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e81035a0 0x7f87e819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87e7fff640 1 -- 172.21.15.5:0/408404269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e810c0a0 msgr2=0x7f87e819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87e7fff640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e810c0a0 0x7f87e819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87e7fff640 1 -- 172.21.15.5:0/408404269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e819f620 con 0x7f87e8102c80 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87e77fe640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e81035a0 0x7f87e819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87ecb85640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e810c0a0 0x7f87e819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.802+0000 7f87e7fff640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87e8102c80 0x7f87e819a630 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f87d8006010 tx=0x7f87d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.803+0000 7f87e57fa640 1 -- 172.21.15.5:0/408404269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d803f030 con 0x7f87e8102c80 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.803+0000 7f87ee5e8640 1 -- 172.21.15.5:0/408404269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e819f8b0 con 0x7f87e8102c80 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.803+0000 7f87e57fa640 1 -- 172.21.15.5:0/408404269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d8003ad0 con 0x7f87e8102c80 2024-09-15T22:09:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.803+0000 7f87e57fa640 1 -- 172.21.15.5:0/408404269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8035070 con 0x7f87e8102c80 2024-09-15T22:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.803+0000 7f87ee5e8640 1 -- 172.21.15.5:0/408404269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e81a71a0 con 0x7f87e8102c80 2024-09-15T22:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.804+0000 7f87ee5e8640 1 -- 172.21.15.5:0/408404269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e806a2e0 con 0x7f87e8102c80 2024-09-15T22:09:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.806+0000 7f87e57fa640 1 -- 172.21.15.5:0/408404269 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f87d8042030 con 0x7f87e8102c80 2024-09-15T22:09:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.807+0000 7f87e57fa640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f87d00a34d0 0x7f87d00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.807+0000 7f87e77fe640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f87d00a34d0 0x7f87d00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.807+0000 7f87e57fa640 1 -- 172.21.15.5:0/408404269 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f87d80f0af0 con 0x7f87e8102c80 2024-09-15T22:09:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.807+0000 7f87e77fe640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f87d00a34d0 0x7f87d00a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f87e810e380 tx=0x7f87dc005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.809+0000 7f87e57fa640 1 -- 172.21.15.5:0/408404269 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87d80bc960 con 0x7f87e8102c80 2024-09-15T22:09:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.837+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2690129771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c102ef0 msgr2=0x7f4d1c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.837+0000 7f4d20fa6640 1 --2- 172.21.15.5:0/2690129771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c102ef0 0x7f4d1c10da40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009ca0 tx=0x7f4d1402a950 comp rx=0 tx=0).stop 2024-09-15T22:09:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.838+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2690129771 shutdown_connections 2024-09-15T22:09:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.838+0000 7f4d20fa6640 1 --2- 172.21.15.5:0/2690129771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c102ef0 0x7f4d1c10da40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.838+0000 7f4d20fa6640 1 --2- 172.21.15.5:0/2690129771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d1c102570 0x7f4d1c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.838+0000 7f4d20fa6640 1 --2- 172.21.15.5:0/2690129771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d1c100e20 0x7f4d1c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.838+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2690129771 >> 172.21.15.5:0/2690129771 conn(0x7f4d1c0fc7d0 msgr2=0x7f4d1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.838+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2690129771 shutdown_connections 2024-09-15T22:09:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.838+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2690129771 wait complete. 2024-09-15T22:09:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.838+0000 7f4d20fa6640 1 Processor -- start 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.838+0000 7f4d20fa6640 1 -- start start 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d20fa6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d1c100e20 0x7f4d1c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d20fa6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c102570 0x7f4d1c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d20fa6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d1c102ef0 0x7f4d1c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d20fa6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c110f80 con 0x7f4d1c102570 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d20fa6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c110e00 con 0x7f4d1c102ef0 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d20fa6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c111100 con 0x7f4d1c100e20 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d19d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c102570 0x7f4d1c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d19d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c102570 0x7f4d1c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46542/0 (socket says 172.21.15.5:46542) 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d19d9b640 1 -- 172.21.15.5:0/2305685977 learned_addr learned my addr 172.21.15.5:0/2305685977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d1ad9d640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d1c102ef0 0x7f4d1c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d19d9b640 1 -- 172.21.15.5:0/2305685977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d1c100e20 msgr2=0x7f4d1c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d19d9b640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d1c100e20 0x7f4d1c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d19d9b640 1 -- 172.21.15.5:0/2305685977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d1c102ef0 msgr2=0x7f4d1c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d19d9b640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d1c102ef0 0x7f4d1c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d19d9b640 1 -- 172.21.15.5:0/2305685977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c1a69c0 con 0x7f4d1c102570 2024-09-15T22:09:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.839+0000 7f4d1ad9d640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d1c102ef0 0x7f4d1c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.840+0000 7f4d19d9b640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c102570 0x7f4d1c0fd1c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000e9f0 tx=0x7f4d1000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.840+0000 7f4d0b7fe640 1 -- 172.21.15.5:0/2305685977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1000cd80 con 0x7f4d1c102570 2024-09-15T22:09:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.840+0000 7f4d0b7fe640 1 -- 172.21.15.5:0/2305685977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d10005500 con 0x7f4d1c102570 2024-09-15T22:09:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.840+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2305685977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c1a6cb0 con 0x7f4d1c102570 2024-09-15T22:09:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.840+0000 7f4d0b7fe640 1 -- 172.21.15.5:0/2305685977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d10010690 con 0x7f4d1c102570 2024-09-15T22:09:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.840+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2305685977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c1a71a0 con 0x7f4d1c102570 2024-09-15T22:09:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d2337f740 1 Processor -- start 2024-09-15T22:09:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d2337f740 1 -- start start 2024-09-15T22:09:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 0x55e86076c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076cdc0 0x55e860777140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 0x55e860779a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d2337f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e8606d22e0 con 0x55e86076cdc0 2024-09-15T22:09:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d2337f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e860679ce0 con 0x55e860777680 2024-09-15T22:09:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d2337f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e8606ce2a0 con 0x55e86076c4a0 2024-09-15T22:09:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d1a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076cdc0 0x55e860777140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d1ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 0x55e86076c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d1a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076cdc0 0x55e860777140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46552/0 (socket says 172.21.15.5:46552) 2024-09-15T22:09:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.843+0000 7f4d1a59c640 1 -- 172.21.15.5:0/1979727249 learned_addr learned my addr 172.21.15.5:0/1979727249 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.844+0000 7f4d1a59c640 1 -- 172.21.15.5:0/1979727249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 msgr2=0x55e86076c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.844+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/1979727249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 0x55e86076c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.844+0000 7f4d1a59c640 1 -- 172.21.15.5:0/1979727249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 msgr2=0x55e860779a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:09:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.844+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/1979727249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 0x55e860779a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.844+0000 7f4d1a59c640 1 -- 172.21.15.5:0/1979727249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e86065e440 con 0x55e86076cdc0 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.844+0000 7f4d0b7fe640 1 -- 172.21.15.5:0/2305685977 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f4d100046c0 con 0x7f4d1c102570 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.844+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/1979727249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076cdc0 0x55e860777140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00bb10 tx=0x7f4d0c00d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0721bda25400151 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.844+0000 7f4d08ff9640 1 -- 172.21.15.5:0/1979727249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c016070 con 0x55e86076cdc0 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.844+0000 7f4d08ff9640 1 -- 172.21.15.5:0/1979727249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0c005500 con 0x55e86076cdc0 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.844+0000 7f4d08ff9640 1 -- 172.21.15.5:0/1979727249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c010460 con 0x55e86076cdc0 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.845+0000 7f4d2337f740 1 -- 172.21.15.5:0/1979727249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076cdc0 msgr2=0x55e860777140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.845+0000 7f4d2337f740 1 --2- 172.21.15.5:0/1979727249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076cdc0 0x55e860777140 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00bb10 tx=0x7f4d0c00d2e0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.845+0000 7f4d2337f740 1 -- 172.21.15.5:0/1979727249 shutdown_connections 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.845+0000 7f4d2337f740 1 --2- 172.21.15.5:0/1979727249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 0x55e860779a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.845+0000 7f4d2337f740 1 --2- 172.21.15.5:0/1979727249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076cdc0 0x55e860777140 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.845+0000 7f4d2337f740 1 --2- 172.21.15.5:0/1979727249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 0x55e86076c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.845+0000 7f4d2337f740 1 -- 172.21.15.5:0/1979727249 >> 172.21.15.5:0/1979727249 conn(0x55e860768a30 msgr2=0x55e860768e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.845+0000 7f4d0b7fe640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d040a3550 0x7f4d040a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.846+0000 7f4d0b7fe640 1 -- 172.21.15.5:0/2305685977 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4d100d07c0 con 0x7f4d1c102570 2024-09-15T22:09:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.846+0000 7f4d2337f740 1 -- 172.21.15.5:0/1979727249 shutdown_connections 2024-09-15T22:09:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.846+0000 7f4d2337f740 1 -- 172.21.15.5:0/1979727249 wait complete. 2024-09-15T22:09:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.846+0000 7f4d2337f740 1 -- start start 2024-09-15T22:09:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.846+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d040a3550 0x7f4d040a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.846+0000 7f4d2337f740 1 Processor -- start 2024-09-15T22:09:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.847+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d040a3550 0x7f4d040a5a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c000c00 tx=0x7f4d0c005c90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.847+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 0x55e8607744a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.847+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860777680 0x55e8607749e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.847+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860774f20 0x55e8607836f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.847+0000 7f4d2337f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e86077a250 con 0x55e860777680 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.847+0000 7f4d2337f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e86068d050 con 0x55e860774f20 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.847+0000 7f4d2337f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e86077a3d0 con 0x55e86076c4a0 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.847+0000 7f4d2337f740 10 client.0 _ll_register_callbacks cb 0x55e8606cd160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.847+0000 7f4d19d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860777680 0x55e8607749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.847+0000 7f4d19d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860777680 0x55e8607749e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46562/0 (socket says 172.21.15.5:46562) 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.847+0000 7f4d19d9b640 1 -- 172.21.15.5:0/666242416 learned_addr learned my addr 172.21.15.5:0/666242416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 0x55e8607744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d1ad9d640 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860774f20 0x55e8607836f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d19d9b640 1 -- 172.21.15.5:0/666242416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 msgr2=0x55e8607744a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d19d9b640 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 0x55e8607744a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d19d9b640 1 -- 172.21.15.5:0/666242416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860774f20 msgr2=0x55e8607836f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d19d9b640 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860774f20 0x55e8607836f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d19d9b640 1 -- 172.21.15.5:0/666242416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8604433c0 con 0x55e860777680 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d1ad9d640 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860774f20 0x55e8607836f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d19d9b640 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860777680 0x55e8607749e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4d100ce170 tx=0x7f4d1009be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331daf69775868ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d01ffb640 1 -- 172.21.15.5:0/666242416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d100d05d0 con 0x55e860777680 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d01ffb640 1 -- 172.21.15.5:0/666242416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4d100a4c90 con 0x55e860777680 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d2337f740 1 -- 172.21.15.5:0/666242416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d0c005500 con 0x55e860777680 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d01ffb640 1 -- 172.21.15.5:0/666242416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d100a0860 con 0x55e860777680 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d2337f740 10 client.15429 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.848+0000 7f4d2337f740 1 -- 172.21.15.5:0/666242416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8607861f0 con 0x55e860777680 2024-09-15T22:09:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.849+0000 7f4d01ffb640 1 -- 172.21.15.5:0/666242416 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0c005500 con 0x55e860777680 2024-09-15T22:09:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.849+0000 7f4d01ffb640 1 -- 172.21.15.5:0/666242416 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4d100cdcd0 con 0x55e860777680 2024-09-15T22:09:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.849+0000 7f4d2337f740 10 client.15429 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.849+0000 7f4d2337f740 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.849+0000 7f4d2337f740 20 client.15429 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.849+0000 7f4d2337f740 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e860787700 0x55e860789af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.849+0000 7f4d2337f740 4 client.15429 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.849+0000 7f4d2337f740 1 -- 172.21.15.5:0/666242416 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8606df850 con 0x55e860787700 2024-09-15T22:09:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.849+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e860787700 0x55e860789af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.850+0000 7f4d01ffb640 10 client.15429 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.850+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e860787700 0x55e860789af0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55d2587b7248a9a server_cookie=87ac1ec2cb80c26e in_seq=0 out_seq=0 2024-09-15T22:09:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d01ffb640 1 -- 172.21.15.5:0/666242416 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8606df850 con 0x55e860787700 2024-09-15T22:09:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d01ffb640 10 client.15429 handle_command_reply: tid=0 2024-09-15T22:09:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 1 client.15429 shutdown 2024-09-15T22:09:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 1 -- 172.21.15.5:0/666242416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860777680 msgr2=0x55e8607749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860777680 0x55e8607749e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4d100ce170 tx=0x7f4d1009be10 comp rx=0 tx=0).stop 2024-09-15T22:09:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 1 -- 172.21.15.5:0/666242416 shutdown_connections 2024-09-15T22:09:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e860787700 0x55e860789af0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860774f20 0x55e8607836f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d01ffb640 0 client.15429 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860777680 0x55e8607749e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 1 --2- 172.21.15.5:0/666242416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 0x55e8607744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 1 -- 172.21.15.5:0/666242416 >> 172.21.15.5:0/666242416 conn(0x55e860768a30 msgr2=0x55e86076e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 1 -- 172.21.15.5:0/666242416 shutdown_connections 2024-09-15T22:09:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 1 -- 172.21.15.5:0/666242416 wait complete. 2024-09-15T22:09:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.851+0000 7f4d2337f740 20 client.15429 trim_cache size 0 max 16384 2024-09-15T22:09:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d2337f740 1 Processor -- start 2024-09-15T22:09:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d2337f740 1 -- start start 2024-09-15T22:09:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076c4a0 0x55e86076fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e860774f20 0x55e8607834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 0x55e860783a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d2337f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1b3aa0 con 0x55e86076c4a0 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d2337f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1b1370 con 0x55e860777680 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d2337f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1af1c0 con 0x55e860774f20 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d1a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076c4a0 0x55e86076fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d1a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076c4a0 0x55e86076fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46570/0 (socket says 172.21.15.5:46570) 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d1a59c640 1 -- 172.21.15.5:0/3534912788 learned_addr learned my addr 172.21.15.5:0/3534912788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d19d9b640 1 --2- 172.21.15.5:0/3534912788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e860774f20 0x55e8607834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d1ad9d640 1 --2- 172.21.15.5:0/3534912788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 0x55e860783a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d1a59c640 1 -- 172.21.15.5:0/3534912788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e860774f20 msgr2=0x55e8607834e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/3534912788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e860774f20 0x55e8607834e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d1a59c640 1 -- 172.21.15.5:0/3534912788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 msgr2=0x55e860783a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/3534912788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 0x55e860783a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.869+0000 7f4d1a59c640 1 -- 172.21.15.5:0/3534912788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e860776b60 con 0x55e86076c4a0 2024-09-15T22:09:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.870+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/3534912788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076c4a0 0x55e86076fbb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c004b20 tx=0x7f4d0c00b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28676f2e9bcb9baf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.870+0000 7f4d03fff640 1 -- 172.21.15.5:0/3534912788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c016040 con 0x55e86076c4a0 2024-09-15T22:09:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.870+0000 7f4d03fff640 1 -- 172.21.15.5:0/3534912788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0c0048c0 con 0x55e86076c4a0 2024-09-15T22:09:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.870+0000 7f4d03fff640 1 -- 172.21.15.5:0/3534912788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c032ce0 con 0x55e86076c4a0 2024-09-15T22:09:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.871+0000 7f4d2337f740 1 -- 172.21.15.5:0/3534912788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076c4a0 msgr2=0x55e86076fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.871+0000 7f4d2337f740 1 --2- 172.21.15.5:0/3534912788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076c4a0 0x55e86076fbb0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c004b20 tx=0x7f4d0c00b400 comp rx=0 tx=0).stop 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.871+0000 7f4d2337f740 1 -- 172.21.15.5:0/3534912788 shutdown_connections 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.871+0000 7f4d2337f740 1 --2- 172.21.15.5:0/3534912788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 0x55e860783a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.871+0000 7f4d2337f740 1 --2- 172.21.15.5:0/3534912788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e860774f20 0x55e8607834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.871+0000 7f4d2337f740 1 --2- 172.21.15.5:0/3534912788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e86076c4a0 0x55e86076fbb0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.871+0000 7f4d2337f740 1 -- 172.21.15.5:0/3534912788 >> 172.21.15.5:0/3534912788 conn(0x55e860768a30 msgr2=0x55e86076e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.871+0000 7f4d2337f740 1 -- 172.21.15.5:0/3534912788 shutdown_connections 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.871+0000 7f4d2337f740 1 -- 172.21.15.5:0/3534912788 wait complete. 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.871+0000 7f4d2337f740 1 -- start start 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.871+0000 7f4d2337f740 1 Processor -- start 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 0x55e86077b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860774f20 0x55e86077bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d2337f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 0x55e86077bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d2337f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1af830 con 0x55e860774f20 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d2337f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1b0ca0 con 0x55e860777680 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d2337f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c1aff00 con 0x55e86076c4a0 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d2337f740 10 client.0 _ll_register_callbacks cb 0x7f4d1000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d19d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860774f20 0x55e86077bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d19d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860774f20 0x55e86077bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46576/0 (socket says 172.21.15.5:46576) 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d19d9b640 1 -- 172.21.15.5:0/911907579 learned_addr learned my addr 172.21.15.5:0/911907579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d1ad9d640 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 0x55e86077bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d19d9b640 1 -- 172.21.15.5:0/911907579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 msgr2=0x55e86077b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d19d9b640 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 0x55e86077b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d19d9b640 1 -- 172.21.15.5:0/911907579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 msgr2=0x55e86077bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d19d9b640 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 0x55e86077bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d19d9b640 1 -- 172.21.15.5:0/911907579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8607f1530 con 0x55e860774f20 2024-09-15T22:09:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d19d9b640 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860774f20 0x55e86077bab0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000cfd0 tx=0x7f4d1009bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fca05725130f5ec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d01ffb640 1 -- 172.21.15.5:0/911907579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d100a4e90 con 0x55e860774f20 2024-09-15T22:09:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d01ffb640 1 -- 172.21.15.5:0/911907579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4d10098200 con 0x55e860774f20 2024-09-15T22:09:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.872+0000 7f4d2337f740 1 -- 172.21.15.5:0/911907579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d0c0048c0 con 0x55e860774f20 2024-09-15T22:09:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d01ffb640 1 -- 172.21.15.5:0/911907579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d1009c280 con 0x55e860774f20 2024-09-15T22:09:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d2337f740 10 client.15435 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d2337f740 1 -- 172.21.15.5:0/911907579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8607f27e0 con 0x55e860774f20 2024-09-15T22:09:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d01ffb640 1 -- 172.21.15.5:0/911907579 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d0c0048c0 con 0x55e860774f20 2024-09-15T22:09:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d01ffb640 1 -- 172.21.15.5:0/911907579 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4d10098400 con 0x55e860774f20 2024-09-15T22:09:04.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d2337f740 10 client.15435 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:04.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d2337f740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:04.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d2337f740 20 client.15435 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:04.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d2337f740 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e860787700 0x55e8607f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:04.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d2337f740 4 client.15435 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:09:04.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d2337f740 1 -- 172.21.15.5:0/911907579 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e8606d9d60 con 0x55e860787700 2024-09-15T22:09:04.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.873+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e860787700 0x55e8607f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:04.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d01ffb640 10 client.15435 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:04.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d1a59c640 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e860787700 0x55e8607f4fa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7dd3710b9231c3d server_cookie=8ba0e4303148e0a4 in_seq=0 out_seq=0 2024-09-15T22:09:04.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d01ffb640 1 -- 172.21.15.5:0/911907579 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e8606d9d60 con 0x55e860787700 2024-09-15T22:09:04.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d01ffb640 10 client.15435 handle_command_reply: tid=0 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d2337f740 1 client.15435 shutdown 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d2337f740 1 -- 172.21.15.5:0/911907579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860774f20 msgr2=0x55e86077bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d2337f740 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860774f20 0x55e86077bab0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000cfd0 tx=0x7f4d1009bf40 comp rx=0 tx=0).stop 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d2337f740 1 -- 172.21.15.5:0/911907579 shutdown_connections 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d2337f740 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e860787700 0x55e8607f4fa0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d2337f740 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e860777680 0x55e86077bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d2337f740 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e860774f20 0x55e86077bab0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d2337f740 1 --2- 172.21.15.5:0/911907579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e86076c4a0 0x55e86077b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d2337f740 1 -- 172.21.15.5:0/911907579 >> 172.21.15.5:0/911907579 conn(0x55e860768a30 msgr2=0x55e86076dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.874+0000 7f4d01ffb640 0 client.15435 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d2337f740 1 -- 172.21.15.5:0/911907579 shutdown_connections 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d2337f740 1 -- 172.21.15.5:0/911907579 wait complete. 2024-09-15T22:09:04.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d2337f740 20 client.15435 trim_cache size 0 max 16384 2024-09-15T22:09:04.882 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:09:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:09:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:09:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: "267dcd72-9f77-4024-bf52-4f92af68be48": { 2024-09-15T22:09:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:09:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "267dcd72-9f77-4024-bf52-4f92af68be48", 2024-09-15T22:09:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:09:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:09:04.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:09:04.882 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:09:04.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2305685977 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d040a3550 msgr2=0x7f4d040a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d040a3550 0x7f4d040a5a10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c000c00 tx=0x7f4d0c005c90 comp rx=0 tx=0).stop 2024-09-15T22:09:04.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2305685977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c102570 msgr2=0x7f4d1c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:04.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c102570 0x7f4d1c0fd1c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000e9f0 tx=0x7f4d1000edf0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2305685977 shutdown_connections 2024-09-15T22:09:04.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d040a3550 0x7f4d040a5a10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d1c102ef0 0x7f4d1c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d1c102570 0x7f4d1c0fd1c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 --2- 172.21.15.5:0/2305685977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d1c100e20 0x7f4d1c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2305685977 >> 172.21.15.5:0/2305685977 conn(0x7f4d1c0fc7d0 msgr2=0x7f4d1c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2305685977 shutdown_connections 2024-09-15T22:09:04.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:04.875+0000 7f4d20fa6640 1 -- 172.21.15.5:0/2305685977 wait complete. 2024-09-15T22:09:05.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.017+0000 7f87ee5e8640 1 -- 172.21.15.5:0/408404269 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f87e8100f70 con 0x7f87d00a34d0 2024-09-15T22:09:05.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.024+0000 7f87e57fa640 1 -- 172.21.15.5:0/408404269 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f87e8100f70 con 0x7f87d00a34d0 2024-09-15T22:09:05.025 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-09-15T22:09:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 -- 172.21.15.5:0/408404269 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f87d00a34d0 msgr2=0x7f87d00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:05.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f87d00a34d0 0x7f87d00a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f87e810e380 tx=0x7f87dc005f90 comp rx=0 tx=0).stop 2024-09-15T22:09:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 -- 172.21.15.5:0/408404269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87e8102c80 msgr2=0x7f87e819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87e8102c80 0x7f87e819a630 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f87d8006010 tx=0x7f87d8004270 comp rx=0 tx=0).stop 2024-09-15T22:09:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 -- 172.21.15.5:0/408404269 shutdown_connections 2024-09-15T22:09:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f87d00a34d0 0x7f87d00a5990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f87e810c0a0 0x7f87e819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f87e81035a0 0x7f87e819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 --2- 172.21.15.5:0/408404269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87e8102c80 0x7f87e819a630 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 -- 172.21.15.5:0/408404269 >> 172.21.15.5:0/408404269 conn(0x7f87e80779a0 msgr2=0x7f87e8100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 -- 172.21.15.5:0/408404269 shutdown_connections 2024-09-15T22:09:05.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:05.026+0000 7f87ee5e8640 1 -- 172.21.15.5:0/408404269 wait complete. 2024-09-15T22:09:05.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-15T22:09:05.065 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-15T22:09:05.076 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-15T22:09:05.088 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-15T22:09:05.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:09:05.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:04 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:09:05.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:04 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:05.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:04 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:09:05.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:04 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:05.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:04 smithi005 ceph-mon[30944]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1 KiB/s wr, 18 op/s 2024-09-15T22:09:05.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:04 smithi005 ceph-mon[30944]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:05.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:04 smithi005 ceph-mon[30944]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:04 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:09:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:04 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:04 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:09:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:04 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:04 smithi099 ceph-mon[34892]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1 KiB/s wr, 18 op/s 2024-09-15T22:09:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:04 smithi099 ceph-mon[34892]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:04 smithi099 ceph-mon[34892]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:04 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:09:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:04 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:04 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:09:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:04 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:04 smithi129 ceph-mon[35314]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1 KiB/s wr, 18 op/s 2024-09-15T22:09:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:04 smithi129 ceph-mon[35314]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:04 smithi129 ceph-mon[35314]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:05.592 INFO:tasks.cephfs.filesystem:scrub status for tag:267dcd72-9f77-4024-bf52-4f92af68be48 - {'path': '/', 'tag': '267dcd72-9f77-4024-bf52-4f92af68be48', 'options': 'recursive,force'} 2024-09-15T22:09:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:06.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:05 smithi005 ceph-mon[30944]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:06.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:05 smithi005 ceph-mon[30944]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:06.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.189+0000 7f2772e5d640 1 -- 172.21.15.5:0/632575108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c106f20 msgr2=0x7f276c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.189+0000 7f2772e5d640 1 --2- 172.21.15.5:0/632575108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c106f20 0x7f276c10eaa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2764009ca0 tx=0x7f276402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:09:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.189+0000 7f2772e5d640 1 -- 172.21.15.5:0/632575108 shutdown_connections 2024-09-15T22:09:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.189+0000 7f2772e5d640 1 --2- 172.21.15.5:0/632575108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c106f20 0x7f276c10eaa0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.189+0000 7f2772e5d640 1 --2- 172.21.15.5:0/632575108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276c1027a0 0x7f276c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.189+0000 7f2772e5d640 1 --2- 172.21.15.5:0/632575108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f276c101df0 0x7f276c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.189+0000 7f2772e5d640 1 -- 172.21.15.5:0/632575108 >> 172.21.15.5:0/632575108 conn(0x7f276c0fd8b0 msgr2=0x7f276c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.190+0000 7f2772e5d640 1 -- 172.21.15.5:0/632575108 shutdown_connections 2024-09-15T22:09:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.190+0000 7f2772e5d640 1 -- 172.21.15.5:0/632575108 wait complete. 2024-09-15T22:09:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.190+0000 7f2772e5d640 1 Processor -- start 2024-09-15T22:09:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.190+0000 7f2772e5d640 1 -- start start 2024-09-15T22:09:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.190+0000 7f2772e5d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f276c101df0 0x7f276c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.190+0000 7f2772e5d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276c1027a0 0x7f276c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.190+0000 7f2772e5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c106f20 0x7f276c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.190+0000 7f2772e5d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f276c111fd0 con 0x7f276c106f20 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.190+0000 7f2772e5d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f276c111e50 con 0x7f276c1027a0 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.190+0000 7f2772e5d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f276c112150 con 0x7f276c101df0 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f27713fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c106f20 0x7f276c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f27713fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c106f20 0x7f276c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46600/0 (socket says 172.21.15.5:46600) 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f27713fa640 1 -- 172.21.15.5:0/2107195560 learned_addr learned my addr 172.21.15.5:0/2107195560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f2770bf9640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f276c101df0 0x7f276c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f276bfff640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276c1027a0 0x7f276c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f27713fa640 1 -- 172.21.15.5:0/2107195560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f276c101df0 msgr2=0x7f276c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f27713fa640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f276c101df0 0x7f276c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f27713fa640 1 -- 172.21.15.5:0/2107195560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276c1027a0 msgr2=0x7f276c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f27713fa640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276c1027a0 0x7f276c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f27713fa640 1 -- 172.21.15.5:0/2107195560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f276c1a6830 con 0x7f276c106f20 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f276bfff640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276c1027a0 0x7f276c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f27713fa640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c106f20 0x7f276c074f70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2764003f10 tx=0x7f2764003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f2769ffb640 1 -- 172.21.15.5:0/2107195560 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2764031710 con 0x7f276c106f20 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f2769ffb640 1 -- 172.21.15.5:0/2107195560 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27640318b0 con 0x7f276c106f20 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f2772e5d640 1 -- 172.21.15.5:0/2107195560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f276c1a6ac0 con 0x7f276c106f20 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.191+0000 7f2769ffb640 1 -- 172.21.15.5:0/2107195560 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f276402fb50 con 0x7f276c106f20 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.192+0000 7f2772e5d640 1 -- 172.21.15.5:0/2107195560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f276c1a7050 con 0x7f276c106f20 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.193+0000 7f2772e5d640 1 -- 172.21.15.5:0/2107195560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f276c1021d0 con 0x7f276c106f20 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.194+0000 7f2769ffb640 1 -- 172.21.15.5:0/2107195560 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f276402fd30 con 0x7f276c106f20 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.194+0000 7f2769ffb640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f275c0a3550 0x7f275c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:06.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.195+0000 7f2770bf9640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f275c0a3550 0x7f275c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:06.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.195+0000 7f2769ffb640 1 -- 172.21.15.5:0/2107195560 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f27640f1770 con 0x7f276c106f20 2024-09-15T22:09:06.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.195+0000 7f2770bf9640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f275c0a3550 0x7f275c0a5a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2760005fd0 tx=0x7f276000b390 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:06.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.198+0000 7f2769ffb640 1 -- 172.21.15.5:0/2107195560 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27640bdc70 con 0x7f276c106f20 2024-09-15T22:09:06.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:05 smithi099 ceph-mon[34892]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:06.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:05 smithi099 ceph-mon[34892]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:06.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:05 smithi129 ceph-mon[35314]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:06.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:05 smithi129 ceph-mon[35314]: from='client.15417 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:09:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.372+0000 7f2772e5d640 1 -- 172.21.15.5:0/2107195560 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f276c0fea20 con 0x7f275c0a3550 2024-09-15T22:09:06.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.380+0000 7f2769ffb640 1 -- 172.21.15.5:0/2107195560 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339415 (secure 0 0 0) 0x7f276c0fea20 con 0x7f275c0a3550 2024-09-15T22:09:06.386 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 -- 172.21.15.5:0/2107195560 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f275c0a3550 msgr2=0x7f275c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f275c0a3550 0x7f275c0a5a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2760005fd0 tx=0x7f276000b390 comp rx=0 tx=0).stop 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 -- 172.21.15.5:0/2107195560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c106f20 msgr2=0x7f276c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c106f20 0x7f276c074f70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2764003f10 tx=0x7f2764003f40 comp rx=0 tx=0).stop 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 -- 172.21.15.5:0/2107195560 shutdown_connections 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f275c0a3550 0x7f275c0a5a10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c106f20 0x7f276c074f70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f276c1027a0 0x7f276c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 --2- 172.21.15.5:0/2107195560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f276c101df0 0x7f276c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 -- 172.21.15.5:0/2107195560 >> 172.21.15.5:0/2107195560 conn(0x7f276c0fd8b0 msgr2=0x7f276c0fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 -- 172.21.15.5:0/2107195560 shutdown_connections 2024-09-15T22:09:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:06.384+0000 7f2772e5d640 1 -- 172.21.15.5:0/2107195560 wait complete. 2024-09-15T22:09:07.080 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:06 smithi005 ceph-mon[30944]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-09-15T22:09:07.111 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-15T22:09:05.503620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1390313,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1264,"num_read_kb":1056,"num_write":633,"num_write_kb":5308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":968,"ondisk_log_size":968,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":398600,"kb_used_data":16052,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124543224,"statfs":{"total":1151940427776,"available":1151532261376,"internally_reserved":0,"allocated":16437248,"data_stored":10096313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":391492923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200331,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":398,"num_read_kb":459,"num_write":148,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.991623"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489282+0000","last_change":"2024-09-15T22:07:46.226073+0000","last_active":"2024-09-15T22:08:53.489282+0000","last_peered":"2024-09-15T22:08:53.489282+0000","last_clean":"2024-09-15T22:08:53.489282+0000","last_became_active":"2024-09-15T22:07:46.219250+0000","last_became_peered":"2024-09-15T22:07:46.219250+0000","last_unstale":"2024-09-15T22:08:53.489282+0000","last_undegraded":"2024-09-15T22:08:53.489282+0000","last_fullsized":"2024-09-15T22:08:53.489282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:56.324959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,4],"acting":[9,11,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582824+0000","last_change":"2024-09-15T22:06:37.184096+0000","last_active":"2024-09-15T22:08:53.582824+0000","last_peered":"2024-09-15T22:08:53.582824+0000","last_clean":"2024-09-15T22:08:53.582824+0000","last_became_active":"2024-09-15T22:06:37.183758+0000","last_became_peered":"2024-09-15T22:06:37.183758+0000","last_unstale":"2024-09-15T22:08:53.582824+0000","last_undegraded":"2024-09-15T22:08:53.582824+0000","last_fullsized":"2024-09-15T22:08:53.582824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:04.210551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750577+0000","last_change":"2024-09-15T22:06:44.653042+0000","last_active":"2024-09-15T22:08:53.750577+0000","last_peered":"2024-09-15T22:08:53.750577+0000","last_clean":"2024-09-15T22:08:53.750577+0000","last_became_active":"2024-09-15T22:06:44.652658+0000","last_became_peered":"2024-09-15T22:06:44.652658+0000","last_unstale":"2024-09-15T22:08:53.750577+0000","last_undegraded":"2024-09-15T22:08:53.750577+0000","last_fullsized":"2024-09-15T22:08:53.750577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:59.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583148+0000","last_change":"2024-09-15T22:06:35.355392+0000","last_active":"2024-09-15T22:08:53.583148+0000","last_peered":"2024-09-15T22:08:53.583148+0000","last_clean":"2024-09-15T22:08:53.583148+0000","last_became_active":"2024-09-15T22:06:35.355042+0000","last_became_peered":"2024-09-15T22:06:35.355042+0000","last_unstale":"2024-09-15T22:08:53.583148+0000","last_undegraded":"2024-09-15T22:08:53.583148+0000","last_fullsized":"2024-09-15T22:08:53.583148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:05.186937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929360+0000","last_change":"2024-09-15T22:06:37.184421+0000","last_active":"2024-09-15T22:08:53.929360+0000","last_peered":"2024-09-15T22:08:53.929360+0000","last_clean":"2024-09-15T22:08:53.929360+0000","last_became_active":"2024-09-15T22:06:37.184092+0000","last_became_peered":"2024-09-15T22:06:37.184092+0000","last_unstale":"2024-09-15T22:08:53.929360+0000","last_undegraded":"2024-09-15T22:08:53.929360+0000","last_fullsized":"2024-09-15T22:08:53.929360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:41.080981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752738+0000","last_change":"2024-09-15T22:07:45.968640+0000","last_active":"2024-09-15T22:08:53.752738+0000","last_peered":"2024-09-15T22:08:53.752738+0000","last_clean":"2024-09-15T22:08:53.752738+0000","last_became_active":"2024-09-15T22:07:45.968320+0000","last_became_peered":"2024-09-15T22:07:45.968320+0000","last_unstale":"2024-09-15T22:08:53.752738+0000","last_undegraded":"2024-09-15T22:08:53.752738+0000","last_fullsized":"2024-09-15T22:08:53.752738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:28.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580416+0000","last_change":"2024-09-15T22:06:35.134606+0000","last_active":"2024-09-15T22:08:53.580416+0000","last_peered":"2024-09-15T22:08:53.580416+0000","last_clean":"2024-09-15T22:08:53.580416+0000","last_became_active":"2024-09-15T22:06:35.134263+0000","last_became_peered":"2024-09-15T22:06:35.134263+0000","last_unstale":"2024-09-15T22:08:53.580416+0000","last_undegraded":"2024-09-15T22:08:53.580416+0000","last_fullsized":"2024-09-15T22:08:53.580416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:45.008935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750806+0000","last_change":"2024-09-15T22:06:37.177556+0000","last_active":"2024-09-15T22:08:53.750806+0000","last_peered":"2024-09-15T22:08:53.750806+0000","last_clean":"2024-09-15T22:08:53.750806+0000","last_became_active":"2024-09-15T22:06:37.177161+0000","last_became_peered":"2024-09-15T22:06:37.177161+0000","last_unstale":"2024-09-15T22:08:53.750806+0000","last_undegraded":"2024-09-15T22:08:53.750806+0000","last_fullsized":"2024-09-15T22:08:53.750806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:30.192728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753347+0000","last_change":"2024-09-15T22:06:44.659097+0000","last_active":"2024-09-15T22:08:53.753347+0000","last_peered":"2024-09-15T22:08:53.753347+0000","last_clean":"2024-09-15T22:08:53.753347+0000","last_became_active":"2024-09-15T22:06:44.658626+0000","last_became_peered":"2024-09-15T22:06:44.658626+0000","last_unstale":"2024-09-15T22:08:53.753347+0000","last_undegraded":"2024-09-15T22:08:53.753347+0000","last_fullsized":"2024-09-15T22:08:53.753347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:01.308004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580430+0000","last_change":"2024-09-15T22:06:35.136326+0000","last_active":"2024-09-15T22:08:53.580430+0000","last_peered":"2024-09-15T22:08:53.580430+0000","last_clean":"2024-09-15T22:08:53.580430+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.580430+0000","last_undegraded":"2024-09-15T22:08:53.580430+0000","last_fullsized":"2024-09-15T22:08:53.580430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:28.675832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490507+0000","last_change":"2024-09-15T22:06:37.192939+0000","last_active":"2024-09-15T22:08:53.490507+0000","last_peered":"2024-09-15T22:08:53.490507+0000","last_clean":"2024-09-15T22:08:53.490507+0000","last_became_active":"2024-09-15T22:06:37.192616+0000","last_became_peered":"2024-09-15T22:06:37.192616+0000","last_unstale":"2024-09-15T22:08:53.490507+0000","last_undegraded":"2024-09-15T22:08:53.490507+0000","last_fullsized":"2024-09-15T22:08:53.490507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:03.615203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752853+0000","last_change":"2024-09-15T22:06:44.654741+0000","last_active":"2024-09-15T22:08:53.752853+0000","last_peered":"2024-09-15T22:08:53.752853+0000","last_clean":"2024-09-15T22:08:53.752853+0000","last_became_active":"2024-09-15T22:06:44.654362+0000","last_became_peered":"2024-09-15T22:06:44.654362+0000","last_unstale":"2024-09-15T22:08:53.752853+0000","last_undegraded":"2024-09-15T22:08:53.752853+0000","last_fullsized":"2024-09-15T22:08:53.752853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:59.269948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490328+0000","last_change":"2024-09-15T22:06:37.186385+0000","last_active":"2024-09-15T22:08:53.490328+0000","last_peered":"2024-09-15T22:08:53.490328+0000","last_clean":"2024-09-15T22:08:53.490328+0000","last_became_active":"2024-09-15T22:06:37.185588+0000","last_became_peered":"2024-09-15T22:06:37.185588+0000","last_unstale":"2024-09-15T22:08:53.490328+0000","last_undegraded":"2024-09-15T22:08:53.490328+0000","last_fullsized":"2024-09-15T22:08:53.490328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:22.523083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580272+0000","last_change":"2024-09-15T22:06:35.137383+0000","last_active":"2024-09-15T22:08:53.580272+0000","last_peered":"2024-09-15T22:08:53.580272+0000","last_clean":"2024-09-15T22:08:53.580272+0000","last_became_active":"2024-09-15T22:06:35.137022+0000","last_became_peered":"2024-09-15T22:06:35.137022+0000","last_unstale":"2024-09-15T22:08:53.580272+0000","last_undegraded":"2024-09-15T22:08:53.580272+0000","last_fullsized":"2024-09-15T22:08:53.580272+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.863155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753052+0000","last_change":"2024-09-15T22:06:44.653807+0000","last_active":"2024-09-15T22:08:53.753052+0000","last_peered":"2024-09-15T22:08:53.753052+0000","last_clean":"2024-09-15T22:08:53.753052+0000","last_became_active":"2024-09-15T22:06:44.653406+0000","last_became_peered":"2024-09-15T22:06:44.653406+0000","last_unstale":"2024-09-15T22:08:53.753052+0000","last_undegraded":"2024-09-15T22:08:53.753052+0000","last_fullsized":"2024-09-15T22:08:53.753052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:12.018208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582122+0000","last_change":"2024-09-15T22:06:37.186899+0000","last_active":"2024-09-15T22:08:53.582122+0000","last_peered":"2024-09-15T22:08:53.582122+0000","last_clean":"2024-09-15T22:08:53.582122+0000","last_became_active":"2024-09-15T22:06:37.183782+0000","last_became_peered":"2024-09-15T22:06:37.183782+0000","last_unstale":"2024-09-15T22:08:53.582122+0000","last_undegraded":"2024-09-15T22:08:53.582122+0000","last_fullsized":"2024-09-15T22:08:53.582122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:39.269186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:54.735821+0000","last_change":"2024-09-15T22:06:44.652686+0000","last_active":"2024-09-15T22:08:54.735821+0000","last_peered":"2024-09-15T22:08:54.735821+0000","last_clean":"2024-09-15T22:08:54.735821+0000","last_became_active":"2024-09-15T22:06:44.652306+0000","last_became_peered":"2024-09-15T22:06:44.652306+0000","last_unstale":"2024-09-15T22:08:54.735821+0000","last_undegraded":"2024-09-15T22:08:54.735821+0000","last_fullsized":"2024-09-15T22:08:54.735821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753085+0000","last_change":"2024-09-15T22:06:44.655424+0000","last_active":"2024-09-15T22:08:53.753085+0000","last_peered":"2024-09-15T22:08:53.753085+0000","last_clean":"2024-09-15T22:08:53.753085+0000","last_became_active":"2024-09-15T22:06:44.654959+0000","last_became_peered":"2024-09-15T22:06:44.654959+0000","last_unstale":"2024-09-15T22:08:53.753085+0000","last_undegraded":"2024-09-15T22:08:53.753085+0000","last_fullsized":"2024-09-15T22:08:53.753085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:07.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928379+0000","last_change":"2024-09-15T22:06:37.188424+0000","last_active":"2024-09-15T22:08:53.928379+0000","last_peered":"2024-09-15T22:08:53.928379+0000","last_clean":"2024-09-15T22:08:53.928379+0000","last_became_active":"2024-09-15T22:06:37.188082+0000","last_became_peered":"2024-09-15T22:06:37.188082+0000","last_unstale":"2024-09-15T22:08:53.928379+0000","last_undegraded":"2024-09-15T22:08:53.928379+0000","last_fullsized":"2024-09-15T22:08:53.928379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:07.159341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581279+0000","last_change":"2024-09-15T22:06:35.134237+0000","last_active":"2024-09-15T22:08:53.581279+0000","last_peered":"2024-09-15T22:08:53.581279+0000","last_clean":"2024-09-15T22:08:53.581279+0000","last_became_active":"2024-09-15T22:06:35.133885+0000","last_became_peered":"2024-09-15T22:06:35.133885+0000","last_unstale":"2024-09-15T22:08:53.581279+0000","last_undegraded":"2024-09-15T22:08:53.581279+0000","last_fullsized":"2024-09-15T22:08:53.581279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:07:35.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489113+0000","last_change":"2024-09-15T22:06:44.664578+0000","last_active":"2024-09-15T22:08:53.489113+0000","last_peered":"2024-09-15T22:08:53.489113+0000","last_clean":"2024-09-15T22:08:53.489113+0000","last_became_active":"2024-09-15T22:06:44.664170+0000","last_became_peered":"2024-09-15T22:06:44.664170+0000","last_unstale":"2024-09-15T22:08:53.489113+0000","last_undegraded":"2024-09-15T22:08:53.489113+0000","last_fullsized":"2024-09-15T22:08:53.489113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:41.443816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580024+0000","last_change":"2024-09-15T22:06:37.186107+0000","last_active":"2024-09-15T22:08:53.580024+0000","last_peered":"2024-09-15T22:08:53.580024+0000","last_clean":"2024-09-15T22:08:53.580024+0000","last_became_active":"2024-09-15T22:06:37.185757+0000","last_became_peered":"2024-09-15T22:06:37.185757+0000","last_unstale":"2024-09-15T22:08:53.580024+0000","last_undegraded":"2024-09-15T22:08:53.580024+0000","last_fullsized":"2024-09-15T22:08:53.580024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:14.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750623+0000","last_change":"2024-09-15T22:06:35.356281+0000","last_active":"2024-09-15T22:08:53.750623+0000","last_peered":"2024-09-15T22:08:53.750623+0000","last_clean":"2024-09-15T22:08:53.750623+0000","last_became_active":"2024-09-15T22:06:35.355922+0000","last_became_peered":"2024-09-15T22:06:35.355922+0000","last_unstale":"2024-09-15T22:08:53.750623+0000","last_undegraded":"2024-09-15T22:08:53.750623+0000","last_fullsized":"2024-09-15T22:08:53.750623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:06.107462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753055+0000","last_change":"2024-09-15T22:06:44.659230+0000","last_active":"2024-09-15T22:08:53.753055+0000","last_peered":"2024-09-15T22:08:53.753055+0000","last_clean":"2024-09-15T22:08:53.753055+0000","last_became_active":"2024-09-15T22:06:44.658649+0000","last_became_peered":"2024-09-15T22:06:44.658649+0000","last_unstale":"2024-09-15T22:08:53.753055+0000","last_undegraded":"2024-09-15T22:08:53.753055+0000","last_fullsized":"2024-09-15T22:08:53.753055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:14:21.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929164+0000","last_change":"2024-09-15T22:06:44.649394+0000","last_active":"2024-09-15T22:08:53.929164+0000","last_peered":"2024-09-15T22:08:53.929164+0000","last_clean":"2024-09-15T22:08:53.929164+0000","last_became_active":"2024-09-15T22:06:44.649043+0000","last_became_peered":"2024-09-15T22:06:44.649043+0000","last_unstale":"2024-09-15T22:08:53.929164+0000","last_undegraded":"2024-09-15T22:08:53.929164+0000","last_fullsized":"2024-09-15T22:08:53.929164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:56:28.335869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753504+0000","last_change":"2024-09-15T22:06:35.265970+0000","last_active":"2024-09-15T22:08:53.753504+0000","last_peered":"2024-09-15T22:08:53.753504+0000","last_clean":"2024-09-15T22:08:53.753504+0000","last_became_active":"2024-09-15T22:06:35.265628+0000","last_became_peered":"2024-09-15T22:06:35.265628+0000","last_unstale":"2024-09-15T22:08:53.753504+0000","last_undegraded":"2024-09-15T22:08:53.753504+0000","last_fullsized":"2024-09-15T22:08:53.753504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:41.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750664+0000","last_change":"2024-09-15T22:06:44.649652+0000","last_active":"2024-09-15T22:08:53.750664+0000","last_peered":"2024-09-15T22:08:53.750664+0000","last_clean":"2024-09-15T22:08:53.750664+0000","last_became_active":"2024-09-15T22:06:44.649112+0000","last_became_peered":"2024-09-15T22:06:44.649112+0000","last_unstale":"2024-09-15T22:08:53.750664+0000","last_undegraded":"2024-09-15T22:08:53.750664+0000","last_fullsized":"2024-09-15T22:08:53.750664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:53.789320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583103+0000","last_change":"2024-09-15T22:06:37.190701+0000","last_active":"2024-09-15T22:08:53.583103+0000","last_peered":"2024-09-15T22:08:53.583103+0000","last_clean":"2024-09-15T22:08:53.583103+0000","last_became_active":"2024-09-15T22:06:37.190430+0000","last_became_peered":"2024-09-15T22:06:37.190430+0000","last_unstale":"2024-09-15T22:08:53.583103+0000","last_undegraded":"2024-09-15T22:08:53.583103+0000","last_fullsized":"2024-09-15T22:08:53.583103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:14.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'101","reported_seq":224,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:04.325283+0000","last_change":"2024-09-15T22:06:35.133899+0000","last_active":"2024-09-15T22:09:04.325283+0000","last_peered":"2024-09-15T22:09:04.325283+0000","last_clean":"2024-09-15T22:09:04.325283+0000","last_became_active":"2024-09-15T22:06:35.133556+0000","last_became_peered":"2024-09-15T22:06:35.133556+0000","last_unstale":"2024-09-15T22:09:04.325283+0000","last_undegraded":"2024-09-15T22:09:04.325283+0000","last_fullsized":"2024-09-15T22:09:04.325283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:01.692416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":185,"num_write_kb":1289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580427+0000","last_change":"2024-09-15T22:06:44.655280+0000","last_active":"2024-09-15T22:08:53.580427+0000","last_peered":"2024-09-15T22:08:53.580427+0000","last_clean":"2024-09-15T22:08:53.580427+0000","last_became_active":"2024-09-15T22:06:44.654883+0000","last_became_peered":"2024-09-15T22:06:44.654883+0000","last_unstale":"2024-09-15T22:08:53.580427+0000","last_undegraded":"2024-09-15T22:08:53.580427+0000","last_fullsized":"2024-09-15T22:08:53.580427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.074035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889279+0000","last_change":"2024-09-15T22:06:37.182006+0000","last_active":"2024-09-15T22:08:53.889279+0000","last_peered":"2024-09-15T22:08:53.889279+0000","last_clean":"2024-09-15T22:08:53.889279+0000","last_became_active":"2024-09-15T22:06:37.181693+0000","last_became_peered":"2024-09-15T22:06:37.181693+0000","last_unstale":"2024-09-15T22:08:53.889279+0000","last_undegraded":"2024-09-15T22:08:53.889279+0000","last_fullsized":"2024-09-15T22:08:53.889279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:23.483930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580471+0000","last_change":"2024-09-15T22:06:35.355854+0000","last_active":"2024-09-15T22:08:53.580471+0000","last_peered":"2024-09-15T22:08:53.580471+0000","last_clean":"2024-09-15T22:08:53.580471+0000","last_became_active":"2024-09-15T22:06:35.355486+0000","last_became_peered":"2024-09-15T22:06:35.355486+0000","last_unstale":"2024-09-15T22:08:53.580471+0000","last_undegraded":"2024-09-15T22:08:53.580471+0000","last_fullsized":"2024-09-15T22:08:53.580471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:36.708476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928367+0000","last_change":"2024-09-15T22:06:44.660523+0000","last_active":"2024-09-15T22:08:53.928367+0000","last_peered":"2024-09-15T22:08:53.928367+0000","last_clean":"2024-09-15T22:08:53.928367+0000","last_became_active":"2024-09-15T22:06:44.660113+0000","last_became_peered":"2024-09-15T22:06:44.660113+0000","last_unstale":"2024-09-15T22:08:53.928367+0000","last_undegraded":"2024-09-15T22:08:53.928367+0000","last_fullsized":"2024-09-15T22:08:53.928367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:42.164558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589575+0000","last_change":"2024-09-15T22:06:37.183649+0000","last_active":"2024-09-15T22:08:53.589575+0000","last_peered":"2024-09-15T22:08:53.589575+0000","last_clean":"2024-09-15T22:08:53.589575+0000","last_became_active":"2024-09-15T22:06:37.183296+0000","last_became_peered":"2024-09-15T22:06:37.183296+0000","last_unstale":"2024-09-15T22:08:53.589575+0000","last_undegraded":"2024-09-15T22:08:53.589575+0000","last_fullsized":"2024-09-15T22:08:53.589575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:22.010288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488860+0000","last_change":"2024-09-15T22:06:35.355068+0000","last_active":"2024-09-15T22:08:53.488860+0000","last_peered":"2024-09-15T22:08:53.488860+0000","last_clean":"2024-09-15T22:08:53.488860+0000","last_became_active":"2024-09-15T22:06:35.354728+0000","last_became_peered":"2024-09-15T22:06:35.354728+0000","last_unstale":"2024-09-15T22:08:53.488860+0000","last_undegraded":"2024-09-15T22:08:53.488860+0000","last_fullsized":"2024-09-15T22:08:53.488860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:35.479290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580098+0000","last_change":"2024-09-15T22:06:44.653849+0000","last_active":"2024-09-15T22:08:53.580098+0000","last_peered":"2024-09-15T22:08:53.580098+0000","last_clean":"2024-09-15T22:08:53.580098+0000","last_became_active":"2024-09-15T22:06:44.653419+0000","last_became_peered":"2024-09-15T22:06:44.653419+0000","last_unstale":"2024-09-15T22:08:53.580098+0000","last_undegraded":"2024-09-15T22:08:53.580098+0000","last_fullsized":"2024-09-15T22:08:53.580098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:04.131553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589446+0000","last_change":"2024-09-15T22:06:37.191438+0000","last_active":"2024-09-15T22:08:53.589446+0000","last_peered":"2024-09-15T22:08:53.589446+0000","last_clean":"2024-09-15T22:08:53.589446+0000","last_became_active":"2024-09-15T22:06:37.191088+0000","last_became_peered":"2024-09-15T22:06:37.191088+0000","last_unstale":"2024-09-15T22:08:53.589446+0000","last_undegraded":"2024-09-15T22:08:53.589446+0000","last_fullsized":"2024-09-15T22:08:53.589446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:46:36.078872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490414+0000","last_change":"2024-09-15T22:06:35.135866+0000","last_active":"2024-09-15T22:08:53.490414+0000","last_peered":"2024-09-15T22:08:53.490414+0000","last_clean":"2024-09-15T22:08:53.490414+0000","last_became_active":"2024-09-15T22:06:35.135524+0000","last_became_peered":"2024-09-15T22:06:35.135524+0000","last_unstale":"2024-09-15T22:08:53.490414+0000","last_undegraded":"2024-09-15T22:08:53.490414+0000","last_fullsized":"2024-09-15T22:08:53.490414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888980+0000","last_change":"2024-09-15T22:06:44.649943+0000","last_active":"2024-09-15T22:08:53.888980+0000","last_peered":"2024-09-15T22:08:53.888980+0000","last_clean":"2024-09-15T22:08:53.888980+0000","last_became_active":"2024-09-15T22:06:44.649579+0000","last_became_peered":"2024-09-15T22:06:44.649579+0000","last_unstale":"2024-09-15T22:08:53.888980+0000","last_undegraded":"2024-09-15T22:08:53.888980+0000","last_fullsized":"2024-09-15T22:08:53.888980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:18.390608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489996+0000","last_change":"2024-09-15T22:06:37.189988+0000","last_active":"2024-09-15T22:08:53.489996+0000","last_peered":"2024-09-15T22:08:53.489996+0000","last_clean":"2024-09-15T22:08:53.489996+0000","last_became_active":"2024-09-15T22:06:37.189319+0000","last_became_peered":"2024-09-15T22:06:37.189319+0000","last_unstale":"2024-09-15T22:08:53.489996+0000","last_undegraded":"2024-09-15T22:08:53.489996+0000","last_fullsized":"2024-09-15T22:08:53.489996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:08.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.902547+0000","last_change":"2024-09-15T22:06:35.355371+0000","last_active":"2024-09-15T22:09:02.902547+0000","last_peered":"2024-09-15T22:09:02.902547+0000","last_clean":"2024-09-15T22:09:02.902547+0000","last_became_active":"2024-09-15T22:06:35.354687+0000","last_became_peered":"2024-09-15T22:06:35.354687+0000","last_unstale":"2024-09-15T22:09:02.902547+0000","last_undegraded":"2024-09-15T22:09:02.902547+0000","last_fullsized":"2024-09-15T22:09:02.902547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:27.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889597+0000","last_change":"2024-09-15T22:06:44.657929+0000","last_active":"2024-09-15T22:08:53.889597+0000","last_peered":"2024-09-15T22:08:53.889597+0000","last_clean":"2024-09-15T22:08:53.889597+0000","last_became_active":"2024-09-15T22:06:44.657519+0000","last_became_peered":"2024-09-15T22:06:44.657519+0000","last_unstale":"2024-09-15T22:08:53.889597+0000","last_undegraded":"2024-09-15T22:08:53.889597+0000","last_fullsized":"2024-09-15T22:08:53.889597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:12.313877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888747+0000","last_change":"2024-09-15T22:06:37.193288+0000","last_active":"2024-09-15T22:08:53.888747+0000","last_peered":"2024-09-15T22:08:53.888747+0000","last_clean":"2024-09-15T22:08:53.888747+0000","last_became_active":"2024-09-15T22:06:37.193007+0000","last_became_peered":"2024-09-15T22:06:37.193007+0000","last_unstale":"2024-09-15T22:08:53.888747+0000","last_undegraded":"2024-09-15T22:08:53.888747+0000","last_fullsized":"2024-09-15T22:08:53.888747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:45.956206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490201+0000","last_change":"2024-09-15T22:06:35.138726+0000","last_active":"2024-09-15T22:08:53.490201+0000","last_peered":"2024-09-15T22:08:53.490201+0000","last_clean":"2024-09-15T22:08:53.490201+0000","last_became_active":"2024-09-15T22:06:35.138390+0000","last_became_peered":"2024-09-15T22:06:35.138390+0000","last_unstale":"2024-09-15T22:08:53.490201+0000","last_undegraded":"2024-09-15T22:08:53.490201+0000","last_fullsized":"2024-09-15T22:08:53.490201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:51.336171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490388+0000","last_change":"2024-09-15T22:07:46.219720+0000","last_active":"2024-09-15T22:08:53.490388+0000","last_peered":"2024-09-15T22:08:53.490388+0000","last_clean":"2024-09-15T22:08:53.490388+0000","last_became_active":"2024-09-15T22:07:46.219303+0000","last_became_peered":"2024-09-15T22:07:46.219303+0000","last_unstale":"2024-09-15T22:08:53.490388+0000","last_undegraded":"2024-09-15T22:08:53.490388+0000","last_fullsized":"2024-09-15T22:08:53.490388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:03.596795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581603+0000","last_change":"2024-09-15T22:06:37.176478+0000","last_active":"2024-09-15T22:08:53.581603+0000","last_peered":"2024-09-15T22:08:53.581603+0000","last_clean":"2024-09-15T22:08:53.581603+0000","last_became_active":"2024-09-15T22:06:37.176171+0000","last_became_peered":"2024-09-15T22:06:37.176171+0000","last_unstale":"2024-09-15T22:08:53.581603+0000","last_undegraded":"2024-09-15T22:08:53.581603+0000","last_fullsized":"2024-09-15T22:08:53.581603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753268+0000","last_change":"2024-09-15T22:06:35.134651+0000","last_active":"2024-09-15T22:08:53.753268+0000","last_peered":"2024-09-15T22:08:53.753268+0000","last_clean":"2024-09-15T22:08:53.753268+0000","last_became_active":"2024-09-15T22:06:35.134185+0000","last_became_peered":"2024-09-15T22:06:35.134185+0000","last_unstale":"2024-09-15T22:08:53.753268+0000","last_undegraded":"2024-09-15T22:08:53.753268+0000","last_fullsized":"2024-09-15T22:08:53.753268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:56.385046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"89'148","reported_seq":402,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:55.596394+0000","last_change":"2024-09-15T22:05:23.048068+0000","last_active":"2024-09-15T22:08:55.596394+0000","last_peered":"2024-09-15T22:08:55.596394+0000","last_clean":"2024-09-15T22:08:55.596394+0000","last_became_active":"2024-09-15T22:05:23.047692+0000","last_became_peered":"2024-09-15T22:05:23.047692+0000","last_unstale":"2024-09-15T22:08:55.596394+0000","last_undegraded":"2024-09-15T22:08:55.596394+0000","last_fullsized":"2024-09-15T22:08:55.596394+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_clean_scrub_stamp":"2024-09-15T22:01:57.477629+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490625+0000","last_change":"2024-09-15T22:06:44.657213+0000","last_active":"2024-09-15T22:08:53.490625+0000","last_peered":"2024-09-15T22:08:53.490625+0000","last_clean":"2024-09-15T22:08:53.490625+0000","last_became_active":"2024-09-15T22:06:44.656815+0000","last_became_peered":"2024-09-15T22:06:44.656815+0000","last_unstale":"2024-09-15T22:08:53.490625+0000","last_undegraded":"2024-09-15T22:08:53.490625+0000","last_fullsized":"2024-09-15T22:08:53.490625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:26.525992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490755+0000","last_change":"2024-09-15T22:06:37.184467+0000","last_active":"2024-09-15T22:08:53.490755+0000","last_peered":"2024-09-15T22:08:53.490755+0000","last_clean":"2024-09-15T22:08:53.490755+0000","last_became_active":"2024-09-15T22:06:37.183864+0000","last_became_peered":"2024-09-15T22:06:37.183864+0000","last_unstale":"2024-09-15T22:08:53.490755+0000","last_undegraded":"2024-09-15T22:08:53.490755+0000","last_fullsized":"2024-09-15T22:08:53.490755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:35.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750805+0000","last_change":"2024-09-15T22:06:44.651594+0000","last_active":"2024-09-15T22:08:53.750805+0000","last_peered":"2024-09-15T22:08:53.750805+0000","last_clean":"2024-09-15T22:08:53.750805+0000","last_became_active":"2024-09-15T22:06:44.650393+0000","last_became_peered":"2024-09-15T22:06:44.650393+0000","last_unstale":"2024-09-15T22:08:53.750805+0000","last_undegraded":"2024-09-15T22:08:53.750805+0000","last_fullsized":"2024-09-15T22:08:53.750805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:13.599744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750608+0000","last_change":"2024-09-15T22:06:44.660789+0000","last_active":"2024-09-15T22:08:53.750608+0000","last_peered":"2024-09-15T22:08:53.750608+0000","last_clean":"2024-09-15T22:08:53.750608+0000","last_became_active":"2024-09-15T22:06:44.660331+0000","last_became_peered":"2024-09-15T22:06:44.660331+0000","last_unstale":"2024-09-15T22:08:53.750608+0000","last_undegraded":"2024-09-15T22:08:53.750608+0000","last_fullsized":"2024-09-15T22:08:53.750608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:04:45.562795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490532+0000","last_change":"2024-09-15T22:06:37.185697+0000","last_active":"2024-09-15T22:08:53.490532+0000","last_peered":"2024-09-15T22:08:53.490532+0000","last_clean":"2024-09-15T22:08:53.490532+0000","last_became_active":"2024-09-15T22:06:37.185207+0000","last_became_peered":"2024-09-15T22:06:37.185207+0000","last_unstale":"2024-09-15T22:08:53.490532+0000","last_undegraded":"2024-09-15T22:08:53.490532+0000","last_fullsized":"2024-09-15T22:08:53.490532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:39.005273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'65","reported_seq":131,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:04.319716+0000","last_change":"2024-09-15T22:06:35.135176+0000","last_active":"2024-09-15T22:09:04.319716+0000","last_peered":"2024-09-15T22:09:04.319716+0000","last_clean":"2024-09-15T22:09:04.319716+0000","last_became_active":"2024-09-15T22:06:35.134649+0000","last_became_peered":"2024-09-15T22:06:35.134649+0000","last_unstale":"2024-09-15T22:09:04.319716+0000","last_undegraded":"2024-09-15T22:09:04.319716+0000","last_fullsized":"2024-09-15T22:09:04.319716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:33.780162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":31,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928755+0000","last_change":"2024-09-15T22:06:44.656972+0000","last_active":"2024-09-15T22:08:53.928755+0000","last_peered":"2024-09-15T22:08:53.928755+0000","last_clean":"2024-09-15T22:08:53.928755+0000","last_became_active":"2024-09-15T22:06:44.656528+0000","last_became_peered":"2024-09-15T22:06:44.656528+0000","last_unstale":"2024-09-15T22:08:53.928755+0000","last_undegraded":"2024-09-15T22:08:53.928755+0000","last_fullsized":"2024-09-15T22:08:53.928755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:09.164596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490782+0000","last_change":"2024-09-15T22:06:37.186118+0000","last_active":"2024-09-15T22:08:53.490782+0000","last_peered":"2024-09-15T22:08:53.490782+0000","last_clean":"2024-09-15T22:08:53.490782+0000","last_became_active":"2024-09-15T22:06:37.185118+0000","last_became_peered":"2024-09-15T22:06:37.185118+0000","last_unstale":"2024-09-15T22:08:53.490782+0000","last_undegraded":"2024-09-15T22:08:53.490782+0000","last_fullsized":"2024-09-15T22:08:53.490782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:53.357045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889478+0000","last_change":"2024-09-15T22:06:35.133341+0000","last_active":"2024-09-15T22:08:53.889478+0000","last_peered":"2024-09-15T22:08:53.889478+0000","last_clean":"2024-09-15T22:08:53.889478+0000","last_became_active":"2024-09-15T22:06:35.132961+0000","last_became_peered":"2024-09-15T22:06:35.132961+0000","last_unstale":"2024-09-15T22:08:53.889478+0000","last_undegraded":"2024-09-15T22:08:53.889478+0000","last_fullsized":"2024-09-15T22:08:53.889478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:39.579470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489873+0000","last_change":"2024-09-15T22:06:44.659148+0000","last_active":"2024-09-15T22:08:53.489873+0000","last_peered":"2024-09-15T22:08:53.489873+0000","last_clean":"2024-09-15T22:08:53.489873+0000","last_became_active":"2024-09-15T22:06:44.658750+0000","last_became_peered":"2024-09-15T22:06:44.658750+0000","last_unstale":"2024-09-15T22:08:53.489873+0000","last_undegraded":"2024-09-15T22:08:53.489873+0000","last_fullsized":"2024-09-15T22:08:53.489873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:48.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928647+0000","last_change":"2024-09-15T22:06:37.194066+0000","last_active":"2024-09-15T22:08:53.928647+0000","last_peered":"2024-09-15T22:08:53.928647+0000","last_clean":"2024-09-15T22:08:53.928647+0000","last_became_active":"2024-09-15T22:06:37.193738+0000","last_became_peered":"2024-09-15T22:06:37.193738+0000","last_unstale":"2024-09-15T22:08:53.928647+0000","last_undegraded":"2024-09-15T22:08:53.928647+0000","last_fullsized":"2024-09-15T22:08:53.928647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.107188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:54.504334+0000","last_change":"2024-09-15T22:06:35.136470+0000","last_active":"2024-09-15T22:08:54.504334+0000","last_peered":"2024-09-15T22:08:54.504334+0000","last_clean":"2024-09-15T22:08:54.504334+0000","last_became_active":"2024-09-15T22:06:35.135953+0000","last_became_peered":"2024-09-15T22:06:35.135953+0000","last_unstale":"2024-09-15T22:08:54.504334+0000","last_undegraded":"2024-09-15T22:08:54.504334+0000","last_fullsized":"2024-09-15T22:08:54.504334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:44.548566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490037+0000","last_change":"2024-09-15T22:06:44.653682+0000","last_active":"2024-09-15T22:08:53.490037+0000","last_peered":"2024-09-15T22:08:53.490037+0000","last_clean":"2024-09-15T22:08:53.490037+0000","last_became_active":"2024-09-15T22:06:44.653303+0000","last_became_peered":"2024-09-15T22:06:44.653303+0000","last_unstale":"2024-09-15T22:08:53.490037+0000","last_undegraded":"2024-09-15T22:08:53.490037+0000","last_fullsized":"2024-09-15T22:08:53.490037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:07.847971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490017+0000","last_change":"2024-09-15T22:06:44.658436+0000","last_active":"2024-09-15T22:08:53.490017+0000","last_peered":"2024-09-15T22:08:53.490017+0000","last_clean":"2024-09-15T22:08:53.490017+0000","last_became_active":"2024-09-15T22:06:44.658037+0000","last_became_peered":"2024-09-15T22:06:44.658037+0000","last_unstale":"2024-09-15T22:08:53.490017+0000","last_undegraded":"2024-09-15T22:08:53.490017+0000","last_fullsized":"2024-09-15T22:08:53.490017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:34:20.369249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580102+0000","last_change":"2024-09-15T22:06:35.358018+0000","last_active":"2024-09-15T22:08:53.580102+0000","last_peered":"2024-09-15T22:08:53.580102+0000","last_clean":"2024-09-15T22:08:53.580102+0000","last_became_active":"2024-09-15T22:06:35.357666+0000","last_became_peered":"2024-09-15T22:06:35.357666+0000","last_unstale":"2024-09-15T22:08:53.580102+0000","last_undegraded":"2024-09-15T22:08:53.580102+0000","last_fullsized":"2024-09-15T22:08:53.580102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:34.089113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889340+0000","last_change":"2024-09-15T22:06:44.656807+0000","last_active":"2024-09-15T22:08:53.889340+0000","last_peered":"2024-09-15T22:08:53.889340+0000","last_clean":"2024-09-15T22:08:53.889340+0000","last_became_active":"2024-09-15T22:06:44.656409+0000","last_became_peered":"2024-09-15T22:06:44.656409+0000","last_unstale":"2024-09-15T22:08:53.889340+0000","last_undegraded":"2024-09-15T22:08:53.889340+0000","last_fullsized":"2024-09-15T22:08:53.889340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490317+0000","last_change":"2024-09-15T22:06:37.180615+0000","last_active":"2024-09-15T22:08:53.490317+0000","last_peered":"2024-09-15T22:08:53.490317+0000","last_clean":"2024-09-15T22:08:53.490317+0000","last_became_active":"2024-09-15T22:06:37.180306+0000","last_became_peered":"2024-09-15T22:06:37.180306+0000","last_unstale":"2024-09-15T22:08:53.490317+0000","last_undegraded":"2024-09-15T22:08:53.490317+0000","last_fullsized":"2024-09-15T22:08:53.490317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:31.145128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.903158+0000","last_change":"2024-09-15T22:06:35.136192+0000","last_active":"2024-09-15T22:09:02.903158+0000","last_peered":"2024-09-15T22:09:02.903158+0000","last_clean":"2024-09-15T22:09:02.903158+0000","last_became_active":"2024-09-15T22:06:35.135608+0000","last_became_peered":"2024-09-15T22:06:35.135608+0000","last_unstale":"2024-09-15T22:09:02.903158+0000","last_undegraded":"2024-09-15T22:09:02.903158+0000","last_fullsized":"2024-09-15T22:09:02.903158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:48.472717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489151+0000","last_change":"2024-09-15T22:06:44.664763+0000","last_active":"2024-09-15T22:08:53.489151+0000","last_peered":"2024-09-15T22:08:53.489151+0000","last_clean":"2024-09-15T22:08:53.489151+0000","last_became_active":"2024-09-15T22:06:44.664410+0000","last_became_peered":"2024-09-15T22:06:44.664410+0000","last_unstale":"2024-09-15T22:08:53.489151+0000","last_undegraded":"2024-09-15T22:08:53.489151+0000","last_fullsized":"2024-09-15T22:08:53.489151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:14.364387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490348+0000","last_change":"2024-09-15T22:06:44.652747+0000","last_active":"2024-09-15T22:08:53.490348+0000","last_peered":"2024-09-15T22:08:53.490348+0000","last_clean":"2024-09-15T22:08:53.490348+0000","last_became_active":"2024-09-15T22:06:44.652368+0000","last_became_peered":"2024-09-15T22:06:44.652368+0000","last_unstale":"2024-09-15T22:08:53.490348+0000","last_undegraded":"2024-09-15T22:08:53.490348+0000","last_fullsized":"2024-09-15T22:08:53.490348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:51.047549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490159+0000","last_change":"2024-09-15T22:06:35.136311+0000","last_active":"2024-09-15T22:08:53.490159+0000","last_peered":"2024-09-15T22:08:53.490159+0000","last_clean":"2024-09-15T22:08:53.490159+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.490159+0000","last_undegraded":"2024-09-15T22:08:53.490159+0000","last_fullsized":"2024-09-15T22:08:53.490159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:36:07.138056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928366+0000","last_change":"2024-09-15T22:06:44.653747+0000","last_active":"2024-09-15T22:08:53.928366+0000","last_peered":"2024-09-15T22:08:53.928366+0000","last_clean":"2024-09-15T22:08:53.928366+0000","last_became_active":"2024-09-15T22:06:44.653357+0000","last_became_peered":"2024-09-15T22:06:44.653357+0000","last_unstale":"2024-09-15T22:08:53.928366+0000","last_undegraded":"2024-09-15T22:08:53.928366+0000","last_fullsized":"2024-09-15T22:08:53.928366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.113924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888302+0000","last_change":"2024-09-15T22:06:37.178308+0000","last_active":"2024-09-15T22:08:53.888302+0000","last_peered":"2024-09-15T22:08:53.888302+0000","last_clean":"2024-09-15T22:08:53.888302+0000","last_became_active":"2024-09-15T22:06:37.177920+0000","last_became_peered":"2024-09-15T22:06:37.177920+0000","last_unstale":"2024-09-15T22:08:53.888302+0000","last_undegraded":"2024-09-15T22:08:53.888302+0000","last_fullsized":"2024-09-15T22:08:53.888302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750890+0000","last_change":"2024-09-15T22:06:35.136132+0000","last_active":"2024-09-15T22:08:53.750890+0000","last_peered":"2024-09-15T22:08:53.750890+0000","last_clean":"2024-09-15T22:08:53.750890+0000","last_became_active":"2024-09-15T22:06:35.135789+0000","last_became_peered":"2024-09-15T22:06:35.135789+0000","last_unstale":"2024-09-15T22:08:53.750890+0000","last_undegraded":"2024-09-15T22:08:53.750890+0000","last_fullsized":"2024-09-15T22:08:53.750890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:01.692885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490410+0000","last_change":"2024-09-15T22:06:44.658920+0000","last_active":"2024-09-15T22:08:53.490410+0000","last_peered":"2024-09-15T22:08:53.490410+0000","last_clean":"2024-09-15T22:08:53.490410+0000","last_became_active":"2024-09-15T22:06:44.658434+0000","last_became_peered":"2024-09-15T22:06:44.658434+0000","last_unstale":"2024-09-15T22:08:53.490410+0000","last_undegraded":"2024-09-15T22:08:53.490410+0000","last_fullsized":"2024-09-15T22:08:53.490410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:37.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580380+0000","last_change":"2024-09-15T22:06:37.184958+0000","last_active":"2024-09-15T22:08:53.580380+0000","last_peered":"2024-09-15T22:08:53.580380+0000","last_clean":"2024-09-15T22:08:53.580380+0000","last_became_active":"2024-09-15T22:06:37.184648+0000","last_became_peered":"2024-09-15T22:06:37.184648+0000","last_unstale":"2024-09-15T22:08:53.580380+0000","last_undegraded":"2024-09-15T22:08:53.580380+0000","last_fullsized":"2024-09-15T22:08:53.580380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:28.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'42","reported_seq":315,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:03.905592+0000","last_change":"2024-09-15T22:06:35.138533+0000","last_active":"2024-09-15T22:09:03.905592+0000","last_peered":"2024-09-15T22:09:03.905592+0000","last_clean":"2024-09-15T22:09:03.905592+0000","last_became_active":"2024-09-15T22:06:35.138142+0000","last_became_peered":"2024-09-15T22:06:35.138142+0000","last_unstale":"2024-09-15T22:09:03.905592+0000","last_undegraded":"2024-09-15T22:09:03.905592+0000","last_fullsized":"2024-09-15T22:09:03.905592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:32.950036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71705,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":236,"num_read_kb":189,"num_write":43,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589290+0000","last_change":"2024-09-15T22:07:46.218222+0000","last_active":"2024-09-15T22:08:53.589290+0000","last_peered":"2024-09-15T22:08:53.589290+0000","last_clean":"2024-09-15T22:08:53.589290+0000","last_became_active":"2024-09-15T22:07:46.217903+0000","last_became_peered":"2024-09-15T22:07:46.217903+0000","last_unstale":"2024-09-15T22:08:53.589290+0000","last_undegraded":"2024-09-15T22:08:53.589290+0000","last_fullsized":"2024-09-15T22:08:53.589290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:17.730288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750930+0000","last_change":"2024-09-15T22:06:37.194297+0000","last_active":"2024-09-15T22:08:53.750930+0000","last_peered":"2024-09-15T22:08:53.750930+0000","last_clean":"2024-09-15T22:08:53.750930+0000","last_became_active":"2024-09-15T22:06:37.193986+0000","last_became_peered":"2024-09-15T22:06:37.193986+0000","last_unstale":"2024-09-15T22:08:53.750930+0000","last_undegraded":"2024-09-15T22:08:53.750930+0000","last_fullsized":"2024-09-15T22:08:53.750930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:27.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490625+0000","last_change":"2024-09-15T22:06:35.356537+0000","last_active":"2024-09-15T22:08:53.490625+0000","last_peered":"2024-09-15T22:08:53.490625+0000","last_clean":"2024-09-15T22:08:53.490625+0000","last_became_active":"2024-09-15T22:06:35.355997+0000","last_became_peered":"2024-09-15T22:06:35.355997+0000","last_unstale":"2024-09-15T22:08:53.490625+0000","last_undegraded":"2024-09-15T22:08:53.490625+0000","last_fullsized":"2024-09-15T22:08:53.490625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:11.563083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753107+0000","last_change":"2024-09-15T22:06:44.654579+0000","last_active":"2024-09-15T22:08:53.753107+0000","last_peered":"2024-09-15T22:08:53.753107+0000","last_clean":"2024-09-15T22:08:53.753107+0000","last_became_active":"2024-09-15T22:06:44.654187+0000","last_became_peered":"2024-09-15T22:06:44.654187+0000","last_unstale":"2024-09-15T22:08:53.753107+0000","last_undegraded":"2024-09-15T22:08:53.753107+0000","last_fullsized":"2024-09-15T22:08:53.753107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:15.617528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929471+0000","last_change":"2024-09-15T22:06:44.661533+0000","last_active":"2024-09-15T22:08:53.929471+0000","last_peered":"2024-09-15T22:08:53.929471+0000","last_clean":"2024-09-15T22:08:53.929471+0000","last_became_active":"2024-09-15T22:06:44.661193+0000","last_became_peered":"2024-09-15T22:06:44.661193+0000","last_unstale":"2024-09-15T22:08:53.929471+0000","last_undegraded":"2024-09-15T22:08:53.929471+0000","last_fullsized":"2024-09-15T22:08:53.929471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:06.559720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490757+0000","last_change":"2024-09-15T22:06:35.140578+0000","last_active":"2024-09-15T22:08:53.490757+0000","last_peered":"2024-09-15T22:08:53.490757+0000","last_clean":"2024-09-15T22:08:53.490757+0000","last_became_active":"2024-09-15T22:06:35.139818+0000","last_became_peered":"2024-09-15T22:06:35.139818+0000","last_unstale":"2024-09-15T22:08:53.490757+0000","last_undegraded":"2024-09-15T22:08:53.490757+0000","last_fullsized":"2024-09-15T22:08:53.490757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:59.806964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488530+0000","last_change":"2024-09-15T22:06:44.654755+0000","last_active":"2024-09-15T22:08:53.488530+0000","last_peered":"2024-09-15T22:08:53.488530+0000","last_clean":"2024-09-15T22:08:53.488530+0000","last_became_active":"2024-09-15T22:06:44.654390+0000","last_became_peered":"2024-09-15T22:06:44.654390+0000","last_unstale":"2024-09-15T22:08:53.488530+0000","last_undegraded":"2024-09-15T22:08:53.488530+0000","last_fullsized":"2024-09-15T22:08:53.488530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:35.781878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751079+0000","last_change":"2024-09-15T22:06:37.182457+0000","last_active":"2024-09-15T22:08:53.751079+0000","last_peered":"2024-09-15T22:08:53.751079+0000","last_clean":"2024-09-15T22:08:53.751079+0000","last_became_active":"2024-09-15T22:06:37.182106+0000","last_became_peered":"2024-09-15T22:06:37.182106+0000","last_unstale":"2024-09-15T22:08:53.751079+0000","last_undegraded":"2024-09-15T22:08:53.751079+0000","last_fullsized":"2024-09-15T22:08:53.751079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:57.560894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.903955+0000","last_change":"2024-09-15T22:06:35.140590+0000","last_active":"2024-09-15T22:09:02.903955+0000","last_peered":"2024-09-15T22:09:02.903955+0000","last_clean":"2024-09-15T22:09:02.903955+0000","last_became_active":"2024-09-15T22:06:35.140198+0000","last_became_peered":"2024-09-15T22:06:35.140198+0000","last_unstale":"2024-09-15T22:09:02.903955+0000","last_undegraded":"2024-09-15T22:09:02.903955+0000","last_fullsized":"2024-09-15T22:09:02.903955+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:46.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580920+0000","last_change":"2024-09-15T22:06:44.660188+0000","last_active":"2024-09-15T22:08:53.580920+0000","last_peered":"2024-09-15T22:08:53.580920+0000","last_clean":"2024-09-15T22:08:53.580920+0000","last_became_active":"2024-09-15T22:06:44.659808+0000","last_became_peered":"2024-09-15T22:06:44.659808+0000","last_unstale":"2024-09-15T22:08:53.580920+0000","last_undegraded":"2024-09-15T22:08:53.580920+0000","last_fullsized":"2024-09-15T22:08:53.580920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:39.473765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929017+0000","last_change":"2024-09-15T22:06:44.647967+0000","last_active":"2024-09-15T22:08:53.929017+0000","last_peered":"2024-09-15T22:08:53.929017+0000","last_clean":"2024-09-15T22:08:53.929017+0000","last_became_active":"2024-09-15T22:06:44.647608+0000","last_became_peered":"2024-09-15T22:06:44.647608+0000","last_unstale":"2024-09-15T22:08:53.929017+0000","last_undegraded":"2024-09-15T22:08:53.929017+0000","last_fullsized":"2024-09-15T22:08:53.929017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:06.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488639+0000","last_change":"2024-09-15T22:06:37.187020+0000","last_active":"2024-09-15T22:08:53.488639+0000","last_peered":"2024-09-15T22:08:53.488639+0000","last_clean":"2024-09-15T22:08:53.488639+0000","last_became_active":"2024-09-15T22:06:37.186564+0000","last_became_peered":"2024-09-15T22:06:37.186564+0000","last_unstale":"2024-09-15T22:08:53.488639+0000","last_undegraded":"2024-09-15T22:08:53.488639+0000","last_fullsized":"2024-09-15T22:08:53.488639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:20.586763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490159+0000","last_change":"2024-09-15T22:06:35.135126+0000","last_active":"2024-09-15T22:08:53.490159+0000","last_peered":"2024-09-15T22:08:53.490159+0000","last_clean":"2024-09-15T22:08:53.490159+0000","last_became_active":"2024-09-15T22:06:35.134783+0000","last_became_peered":"2024-09-15T22:06:35.134783+0000","last_unstale":"2024-09-15T22:08:53.490159+0000","last_undegraded":"2024-09-15T22:08:53.490159+0000","last_fullsized":"2024-09-15T22:08:53.490159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:31.910988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928656+0000","last_change":"2024-09-15T22:06:44.656981+0000","last_active":"2024-09-15T22:08:53.928656+0000","last_peered":"2024-09-15T22:08:53.928656+0000","last_clean":"2024-09-15T22:08:53.928656+0000","last_became_active":"2024-09-15T22:06:44.656477+0000","last_became_peered":"2024-09-15T22:06:44.656477+0000","last_unstale":"2024-09-15T22:08:53.928656+0000","last_undegraded":"2024-09-15T22:08:53.928656+0000","last_fullsized":"2024-09-15T22:08:53.928656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:06.153490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580796+0000","last_change":"2024-09-15T22:06:37.183034+0000","last_active":"2024-09-15T22:08:53.580796+0000","last_peered":"2024-09-15T22:08:53.580796+0000","last_clean":"2024-09-15T22:08:53.580796+0000","last_became_active":"2024-09-15T22:06:37.182734+0000","last_became_peered":"2024-09-15T22:06:37.182734+0000","last_unstale":"2024-09-15T22:08:53.580796+0000","last_undegraded":"2024-09-15T22:08:53.580796+0000","last_fullsized":"2024-09-15T22:08:53.580796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:19.352000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888964+0000","last_change":"2024-09-15T22:06:35.134954+0000","last_active":"2024-09-15T22:08:53.888964+0000","last_peered":"2024-09-15T22:08:53.888964+0000","last_clean":"2024-09-15T22:08:53.888964+0000","last_became_active":"2024-09-15T22:06:35.134583+0000","last_became_peered":"2024-09-15T22:06:35.134583+0000","last_unstale":"2024-09-15T22:08:53.888964+0000","last_undegraded":"2024-09-15T22:08:53.888964+0000","last_fullsized":"2024-09-15T22:08:53.888964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:56.082123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589513+0000","last_change":"2024-09-15T22:06:35.139229+0000","last_active":"2024-09-15T22:08:53.589513+0000","last_peered":"2024-09-15T22:08:53.589513+0000","last_clean":"2024-09-15T22:08:53.589513+0000","last_became_active":"2024-09-15T22:06:35.138834+0000","last_became_peered":"2024-09-15T22:06:35.138834+0000","last_unstale":"2024-09-15T22:08:53.589513+0000","last_undegraded":"2024-09-15T22:08:53.589513+0000","last_fullsized":"2024-09-15T22:08:53.589513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:05.877404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490250+0000","last_change":"2024-09-15T22:06:37.192212+0000","last_active":"2024-09-15T22:08:53.490250+0000","last_peered":"2024-09-15T22:08:53.490250+0000","last_clean":"2024-09-15T22:08:53.490250+0000","last_became_active":"2024-09-15T22:06:37.191872+0000","last_became_peered":"2024-09-15T22:06:37.191872+0000","last_unstale":"2024-09-15T22:08:53.490250+0000","last_undegraded":"2024-09-15T22:08:53.490250+0000","last_fullsized":"2024-09-15T22:08:53.490250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:32.845794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750771+0000","last_change":"2024-09-15T22:07:45.968398+0000","last_active":"2024-09-15T22:08:53.750771+0000","last_peered":"2024-09-15T22:08:53.750771+0000","last_clean":"2024-09-15T22:08:53.750771+0000","last_became_active":"2024-09-15T22:07:45.967912+0000","last_became_peered":"2024-09-15T22:07:45.967912+0000","last_unstale":"2024-09-15T22:08:53.750771+0000","last_undegraded":"2024-09-15T22:08:53.750771+0000","last_fullsized":"2024-09-15T22:08:53.750771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:59.909290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'10","reported_seq":163,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.904724+0000","last_change":"2024-09-15T22:06:35.133846+0000","last_active":"2024-09-15T22:09:02.904724+0000","last_peered":"2024-09-15T22:09:02.904724+0000","last_clean":"2024-09-15T22:09:02.904724+0000","last_became_active":"2024-09-15T22:06:35.133516+0000","last_became_peered":"2024-09-15T22:06:35.133516+0000","last_unstale":"2024-09-15T22:09:02.904724+0000","last_undegraded":"2024-09-15T22:09:02.904724+0000","last_fullsized":"2024-09-15T22:09:02.904724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:29.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2250,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":7,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580169+0000","last_change":"2024-09-15T22:06:37.182904+0000","last_active":"2024-09-15T22:08:53.580169+0000","last_peered":"2024-09-15T22:08:53.580169+0000","last_clean":"2024-09-15T22:08:53.580169+0000","last_became_active":"2024-09-15T22:06:37.182563+0000","last_became_peered":"2024-09-15T22:06:37.182563+0000","last_unstale":"2024-09-15T22:08:53.580169+0000","last_undegraded":"2024-09-15T22:08:53.580169+0000","last_fullsized":"2024-09-15T22:08:53.580169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750976+0000","last_change":"2024-09-15T22:06:44.660997+0000","last_active":"2024-09-15T22:08:53.750976+0000","last_peered":"2024-09-15T22:08:53.750976+0000","last_clean":"2024-09-15T22:08:53.750976+0000","last_became_active":"2024-09-15T22:06:44.660577+0000","last_became_peered":"2024-09-15T22:06:44.660577+0000","last_unstale":"2024-09-15T22:08:53.750976+0000","last_undegraded":"2024-09-15T22:08:53.750976+0000","last_fullsized":"2024-09-15T22:08:53.750976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:56.006891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928243+0000","last_change":"2024-09-15T22:06:37.182077+0000","last_active":"2024-09-15T22:08:53.928243+0000","last_peered":"2024-09-15T22:08:53.928243+0000","last_clean":"2024-09-15T22:08:53.928243+0000","last_became_active":"2024-09-15T22:06:37.181622+0000","last_became_peered":"2024-09-15T22:06:37.181622+0000","last_unstale":"2024-09-15T22:08:53.928243+0000","last_undegraded":"2024-09-15T22:08:53.928243+0000","last_fullsized":"2024-09-15T22:08:53.928243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:07.586406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750818+0000","last_change":"2024-09-15T22:06:44.653845+0000","last_active":"2024-09-15T22:08:53.750818+0000","last_peered":"2024-09-15T22:08:53.750818+0000","last_clean":"2024-09-15T22:08:53.750818+0000","last_became_active":"2024-09-15T22:06:44.653483+0000","last_became_peered":"2024-09-15T22:06:44.653483+0000","last_unstale":"2024-09-15T22:08:53.750818+0000","last_undegraded":"2024-09-15T22:08:53.750818+0000","last_fullsized":"2024-09-15T22:08:53.750818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:08:07.822279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750649+0000","last_change":"2024-09-15T22:06:37.187476+0000","last_active":"2024-09-15T22:08:53.750649+0000","last_peered":"2024-09-15T22:08:53.750649+0000","last_clean":"2024-09-15T22:08:53.750649+0000","last_became_active":"2024-09-15T22:06:37.187047+0000","last_became_peered":"2024-09-15T22:06:37.187047+0000","last_unstale":"2024-09-15T22:08:53.750649+0000","last_undegraded":"2024-09-15T22:08:53.750649+0000","last_fullsized":"2024-09-15T22:08:53.750649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.220834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928765+0000","last_change":"2024-09-15T22:06:37.190583+0000","last_active":"2024-09-15T22:08:53.928765+0000","last_peered":"2024-09-15T22:08:53.928765+0000","last_clean":"2024-09-15T22:08:53.928765+0000","last_became_active":"2024-09-15T22:06:37.190212+0000","last_became_peered":"2024-09-15T22:06:37.190212+0000","last_unstale":"2024-09-15T22:08:53.928765+0000","last_undegraded":"2024-09-15T22:08:53.928765+0000","last_fullsized":"2024-09-15T22:08:53.928765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:48.574188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889153+0000","last_change":"2024-09-15T22:06:37.192508+0000","last_active":"2024-09-15T22:08:53.889153+0000","last_peered":"2024-09-15T22:08:53.889153+0000","last_clean":"2024-09-15T22:08:53.889153+0000","last_became_active":"2024-09-15T22:06:37.192117+0000","last_became_peered":"2024-09-15T22:06:37.192117+0000","last_unstale":"2024-09-15T22:08:53.889153+0000","last_undegraded":"2024-09-15T22:08:53.889153+0000","last_fullsized":"2024-09-15T22:08:53.889153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:41.355966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928601+0000","last_change":"2024-09-15T22:06:37.183377+0000","last_active":"2024-09-15T22:08:53.928601+0000","last_peered":"2024-09-15T22:08:53.928601+0000","last_clean":"2024-09-15T22:08:53.928601+0000","last_became_active":"2024-09-15T22:06:37.183072+0000","last_became_peered":"2024-09-15T22:06:37.183072+0000","last_unstale":"2024-09-15T22:08:53.928601+0000","last_undegraded":"2024-09-15T22:08:53.928601+0000","last_fullsized":"2024-09-15T22:08:53.928601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.318702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751128+0000","last_change":"2024-09-15T22:06:37.185348+0000","last_active":"2024-09-15T22:08:53.751128+0000","last_peered":"2024-09-15T22:08:53.751128+0000","last_clean":"2024-09-15T22:08:53.751128+0000","last_became_active":"2024-09-15T22:06:37.184934+0000","last_became_peered":"2024-09-15T22:06:37.184934+0000","last_unstale":"2024-09-15T22:08:53.751128+0000","last_undegraded":"2024-09-15T22:08:53.751128+0000","last_fullsized":"2024-09-15T22:08:53.751128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:57.564860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582987+0000","last_change":"2024-09-15T22:06:37.192152+0000","last_active":"2024-09-15T22:08:53.582987+0000","last_peered":"2024-09-15T22:08:53.582987+0000","last_clean":"2024-09-15T22:08:53.582987+0000","last_became_active":"2024-09-15T22:06:37.191836+0000","last_became_peered":"2024-09-15T22:06:37.191836+0000","last_unstale":"2024-09-15T22:08:53.582987+0000","last_undegraded":"2024-09-15T22:08:53.582987+0000","last_fullsized":"2024-09-15T22:08:53.582987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:31.781110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489966+0000","last_change":"2024-09-15T22:06:37.179996+0000","last_active":"2024-09-15T22:08:53.489966+0000","last_peered":"2024-09-15T22:08:53.489966+0000","last_clean":"2024-09-15T22:08:53.489966+0000","last_became_active":"2024-09-15T22:06:37.179646+0000","last_became_peered":"2024-09-15T22:06:37.179646+0000","last_unstale":"2024-09-15T22:08:53.489966+0000","last_undegraded":"2024-09-15T22:08:53.489966+0000","last_fullsized":"2024-09-15T22:08:53.489966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:35.240468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581098+0000","last_change":"2024-09-15T22:06:37.182992+0000","last_active":"2024-09-15T22:08:53.581098+0000","last_peered":"2024-09-15T22:08:53.581098+0000","last_clean":"2024-09-15T22:08:53.581098+0000","last_became_active":"2024-09-15T22:06:37.182676+0000","last_became_peered":"2024-09-15T22:06:37.182676+0000","last_unstale":"2024-09-15T22:08:53.581098+0000","last_undegraded":"2024-09-15T22:08:53.581098+0000","last_fullsized":"2024-09-15T22:08:53.581098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:41.524600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750731+0000","last_change":"2024-09-15T22:06:37.180107+0000","last_active":"2024-09-15T22:08:53.750731+0000","last_peered":"2024-09-15T22:08:53.750731+0000","last_clean":"2024-09-15T22:08:53.750731+0000","last_became_active":"2024-09-15T22:06:37.179785+0000","last_became_peered":"2024-09-15T22:06:37.179785+0000","last_unstale":"2024-09-15T22:08:53.750731+0000","last_undegraded":"2024-09-15T22:08:53.750731+0000","last_fullsized":"2024-09-15T22:08:53.750731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:46.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888926+0000","last_change":"2024-09-15T22:06:37.177406+0000","last_active":"2024-09-15T22:08:53.888926+0000","last_peered":"2024-09-15T22:08:53.888926+0000","last_clean":"2024-09-15T22:08:53.888926+0000","last_became_active":"2024-09-15T22:06:37.177093+0000","last_became_peered":"2024-09-15T22:06:37.177093+0000","last_unstale":"2024-09-15T22:08:53.888926+0000","last_undegraded":"2024-09-15T22:08:53.888926+0000","last_fullsized":"2024-09-15T22:08:53.888926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:34.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928467+0000","last_change":"2024-09-15T22:06:37.192314+0000","last_active":"2024-09-15T22:08:53.928467+0000","last_peered":"2024-09-15T22:08:53.928467+0000","last_clean":"2024-09-15T22:08:53.928467+0000","last_became_active":"2024-09-15T22:06:37.191958+0000","last_became_peered":"2024-09-15T22:06:37.191958+0000","last_unstale":"2024-09-15T22:08:53.928467+0000","last_undegraded":"2024-09-15T22:08:53.928467+0000","last_fullsized":"2024-09-15T22:08:53.928467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:47.763742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751178+0000","last_change":"2024-09-15T22:06:37.190212+0000","last_active":"2024-09-15T22:08:53.751178+0000","last_peered":"2024-09-15T22:08:53.751178+0000","last_clean":"2024-09-15T22:08:53.751178+0000","last_became_active":"2024-09-15T22:06:37.189881+0000","last_became_peered":"2024-09-15T22:06:37.189881+0000","last_unstale":"2024-09-15T22:08:53.751178+0000","last_undegraded":"2024-09-15T22:08:53.751178+0000","last_fullsized":"2024-09-15T22:08:53.751178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:59.943574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488994+0000","last_change":"2024-09-15T22:06:37.187967+0000","last_active":"2024-09-15T22:08:53.488994+0000","last_peered":"2024-09-15T22:08:53.488994+0000","last_clean":"2024-09-15T22:08:53.488994+0000","last_became_active":"2024-09-15T22:06:37.187596+0000","last_became_peered":"2024-09-15T22:06:37.187596+0000","last_unstale":"2024-09-15T22:08:53.488994+0000","last_undegraded":"2024-09-15T22:08:53.488994+0000","last_fullsized":"2024-09-15T22:08:53.488994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:00.042784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928782+0000","last_change":"2024-09-15T22:06:37.189434+0000","last_active":"2024-09-15T22:08:53.928782+0000","last_peered":"2024-09-15T22:08:53.928782+0000","last_clean":"2024-09-15T22:08:53.928782+0000","last_became_active":"2024-09-15T22:06:37.189053+0000","last_became_peered":"2024-09-15T22:06:37.189053+0000","last_unstale":"2024-09-15T22:08:53.928782+0000","last_undegraded":"2024-09-15T22:08:53.928782+0000","last_fullsized":"2024-09-15T22:08:53.928782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:51.640495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750544+0000","last_change":"2024-09-15T22:06:37.179229+0000","last_active":"2024-09-15T22:08:53.750544+0000","last_peered":"2024-09-15T22:08:53.750544+0000","last_clean":"2024-09-15T22:08:53.750544+0000","last_became_active":"2024-09-15T22:06:37.178828+0000","last_became_peered":"2024-09-15T22:06:37.178828+0000","last_unstale":"2024-09-15T22:08:53.750544+0000","last_undegraded":"2024-09-15T22:08:53.750544+0000","last_fullsized":"2024-09-15T22:08:53.750544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:25.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928548+0000","last_change":"2024-09-15T22:06:37.193095+0000","last_active":"2024-09-15T22:08:53.928548+0000","last_peered":"2024-09-15T22:08:53.928548+0000","last_clean":"2024-09-15T22:08:53.928548+0000","last_became_active":"2024-09-15T22:06:37.192801+0000","last_became_peered":"2024-09-15T22:06:37.192801+0000","last_unstale":"2024-09-15T22:08:53.928548+0000","last_undegraded":"2024-09-15T22:08:53.928548+0000","last_fullsized":"2024-09-15T22:08:53.928548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:12.521510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928651+0000","last_change":"2024-09-15T22:06:37.182332+0000","last_active":"2024-09-15T22:08:53.928651+0000","last_peered":"2024-09-15T22:08:53.928651+0000","last_clean":"2024-09-15T22:08:53.928651+0000","last_became_active":"2024-09-15T22:06:37.182001+0000","last_became_peered":"2024-09-15T22:06:37.182001+0000","last_unstale":"2024-09-15T22:08:53.928651+0000","last_undegraded":"2024-09-15T22:08:53.928651+0000","last_fullsized":"2024-09-15T22:08:53.928651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:52.230452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888722+0000","last_change":"2024-09-15T22:06:37.180584+0000","last_active":"2024-09-15T22:08:53.888722+0000","last_peered":"2024-09-15T22:08:53.888722+0000","last_clean":"2024-09-15T22:08:53.888722+0000","last_became_active":"2024-09-15T22:06:37.179341+0000","last_became_peered":"2024-09-15T22:06:37.179341+0000","last_unstale":"2024-09-15T22:08:53.888722+0000","last_undegraded":"2024-09-15T22:08:53.888722+0000","last_fullsized":"2024-09-15T22:08:53.888722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.610835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753487+0000","last_change":"2024-09-15T22:06:37.189500+0000","last_active":"2024-09-15T22:08:53.753487+0000","last_peered":"2024-09-15T22:08:53.753487+0000","last_clean":"2024-09-15T22:08:53.753487+0000","last_became_active":"2024-09-15T22:06:37.189137+0000","last_became_peered":"2024-09-15T22:06:37.189137+0000","last_unstale":"2024-09-15T22:08:53.753487+0000","last_undegraded":"2024-09-15T22:08:53.753487+0000","last_fullsized":"2024-09-15T22:08:53.753487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:27.199854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490749+0000","last_change":"2024-09-15T22:06:37.182618+0000","last_active":"2024-09-15T22:08:53.490749+0000","last_peered":"2024-09-15T22:08:53.490749+0000","last_clean":"2024-09-15T22:08:53.490749+0000","last_became_active":"2024-09-15T22:06:37.182257+0000","last_became_peered":"2024-09-15T22:06:37.182257+0000","last_unstale":"2024-09-15T22:08:53.490749+0000","last_undegraded":"2024-09-15T22:08:53.490749+0000","last_fullsized":"2024-09-15T22:08:53.490749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:17.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589848+0000","last_change":"2024-09-15T22:06:37.193036+0000","last_active":"2024-09-15T22:08:53.589848+0000","last_peered":"2024-09-15T22:08:53.589848+0000","last_clean":"2024-09-15T22:08:53.589848+0000","last_became_active":"2024-09-15T22:06:37.192668+0000","last_became_peered":"2024-09-15T22:06:37.192668+0000","last_unstale":"2024-09-15T22:08:53.589848+0000","last_undegraded":"2024-09-15T22:08:53.589848+0000","last_fullsized":"2024-09-15T22:08:53.589848+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:05.436045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489086+0000","last_change":"2024-09-15T22:06:44.659503+0000","last_active":"2024-09-15T22:08:53.489086+0000","last_peered":"2024-09-15T22:08:53.489086+0000","last_clean":"2024-09-15T22:08:53.489086+0000","last_became_active":"2024-09-15T22:06:44.659155+0000","last_became_peered":"2024-09-15T22:06:44.659155+0000","last_unstale":"2024-09-15T22:08:53.489086+0000","last_undegraded":"2024-09-15T22:08:53.489086+0000","last_fullsized":"2024-09-15T22:08:53.489086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:41:04.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580648+0000","last_change":"2024-09-15T22:06:37.178386+0000","last_active":"2024-09-15T22:08:53.580648+0000","last_peered":"2024-09-15T22:08:53.580648+0000","last_clean":"2024-09-15T22:08:53.580648+0000","last_became_active":"2024-09-15T22:06:37.178083+0000","last_became_peered":"2024-09-15T22:06:37.178083+0000","last_unstale":"2024-09-15T22:08:53.580648+0000","last_undegraded":"2024-09-15T22:08:53.580648+0000","last_fullsized":"2024-09-15T22:08:53.580648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:35.350698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490190+0000","last_change":"2024-09-15T22:06:37.186741+0000","last_active":"2024-09-15T22:08:53.490190+0000","last_peered":"2024-09-15T22:08:53.490190+0000","last_clean":"2024-09-15T22:08:53.490190+0000","last_became_active":"2024-09-15T22:06:37.185740+0000","last_became_peered":"2024-09-15T22:06:37.185740+0000","last_unstale":"2024-09-15T22:08:53.490190+0000","last_undegraded":"2024-09-15T22:08:53.490190+0000","last_fullsized":"2024-09-15T22:08:53.490190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.762407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589411+0000","last_change":"2024-09-15T22:06:37.189552+0000","last_active":"2024-09-15T22:08:53.589411+0000","last_peered":"2024-09-15T22:08:53.589411+0000","last_clean":"2024-09-15T22:08:53.589411+0000","last_became_active":"2024-09-15T22:06:37.189197+0000","last_became_peered":"2024-09-15T22:06:37.189197+0000","last_unstale":"2024-09-15T22:08:53.589411+0000","last_undegraded":"2024-09-15T22:08:53.589411+0000","last_fullsized":"2024-09-15T22:08:53.589411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:54.268720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580092+0000","last_change":"2024-09-15T22:06:37.186917+0000","last_active":"2024-09-15T22:08:53.580092+0000","last_peered":"2024-09-15T22:08:53.580092+0000","last_clean":"2024-09-15T22:08:53.580092+0000","last_became_active":"2024-09-15T22:06:37.186571+0000","last_became_peered":"2024-09-15T22:06:37.186571+0000","last_unstale":"2024-09-15T22:08:53.580092+0000","last_undegraded":"2024-09-15T22:08:53.580092+0000","last_fullsized":"2024-09-15T22:08:53.580092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:02.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753272+0000","last_change":"2024-09-15T22:06:37.190960+0000","last_active":"2024-09-15T22:08:53.753272+0000","last_peered":"2024-09-15T22:08:53.753272+0000","last_clean":"2024-09-15T22:08:53.753272+0000","last_became_active":"2024-09-15T22:06:37.190650+0000","last_became_peered":"2024-09-15T22:06:37.190650+0000","last_unstale":"2024-09-15T22:08:53.753272+0000","last_undegraded":"2024-09-15T22:08:53.753272+0000","last_fullsized":"2024-09-15T22:08:53.753272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:34.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928796+0000","last_change":"2024-09-15T22:06:37.186718+0000","last_active":"2024-09-15T22:08:53.928796+0000","last_peered":"2024-09-15T22:08:53.928796+0000","last_clean":"2024-09-15T22:08:53.928796+0000","last_became_active":"2024-09-15T22:06:37.186424+0000","last_became_peered":"2024-09-15T22:06:37.186424+0000","last_unstale":"2024-09-15T22:08:53.928796+0000","last_undegraded":"2024-09-15T22:08:53.928796+0000","last_fullsized":"2024-09-15T22:08:53.928796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:53.266007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929020+0000","last_change":"2024-09-15T22:06:37.185522+0000","last_active":"2024-09-15T22:08:53.929020+0000","last_peered":"2024-09-15T22:08:53.929020+0000","last_clean":"2024-09-15T22:08:53.929020+0000","last_became_active":"2024-09-15T22:06:37.185192+0000","last_became_peered":"2024-09-15T22:06:37.185192+0000","last_unstale":"2024-09-15T22:08:53.929020+0000","last_undegraded":"2024-09-15T22:08:53.929020+0000","last_fullsized":"2024-09-15T22:08:53.929020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:12.868324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928738+0000","last_change":"2024-09-15T22:06:37.189553+0000","last_active":"2024-09-15T22:08:53.928738+0000","last_peered":"2024-09-15T22:08:53.928738+0000","last_clean":"2024-09-15T22:08:53.928738+0000","last_became_active":"2024-09-15T22:06:37.189270+0000","last_became_peered":"2024-09-15T22:06:37.189270+0000","last_unstale":"2024-09-15T22:08:53.928738+0000","last_undegraded":"2024-09-15T22:08:53.928738+0000","last_fullsized":"2024-09-15T22:08:53.928738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:34.157077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799711,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1027,"num_read_kb":726,"num_write":430,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2399133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579947,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1028096,"data_stored":512755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808880,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37352,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707800,"statfs":{"total":95995035648,"available":95956787200,"internally_reserved":0,"allocated":2056192,"data_stored":1500698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005109,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1437696,"data_stored":909721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234041,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1429504,"data_stored":904963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495679,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1110213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659203,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":507585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888137,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37272,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707880,"statfs":{"total":95995035648,"available":95956869120,"internally_reserved":0,"allocated":1974272,"data_stored":1433295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":514137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247891,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36868,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708284,"statfs":{"total":95995035648,"available":95957282816,"internally_reserved":0,"allocated":1560576,"data_stored":1032595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444121,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":581596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":579782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1044480,"data_stored":508973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":72287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":74101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":402835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:09:07.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:09:07.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:06 smithi099 ceph-mon[34892]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-09-15T22:09:07.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:06 smithi129 ceph-mon[35314]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-09-15T22:09:07.374 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:07.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.859+0000 7f3680b81640 1 -- 172.21.15.5:0/2552172703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f367c100a60 msgr2=0x7f367c100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:07.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.859+0000 7f3680b81640 1 --2- 172.21.15.5:0/2552172703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f367c100a60 0x7f367c100ec0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3664009930 tx=0x7f366402a420 comp rx=0 tx=0).stop 2024-09-15T22:09:07.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.859+0000 7f3680b81640 1 -- 172.21.15.5:0/2552172703 shutdown_connections 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.859+0000 7f3680b81640 1 --2- 172.21.15.5:0/2552172703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f367c101400 0x7f367c10d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.859+0000 7f3680b81640 1 --2- 172.21.15.5:0/2552172703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f367c100a60 0x7f367c100ec0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.859+0000 7f3680b81640 1 --2- 172.21.15.5:0/2552172703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f367c106a60 0x7f367c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.859+0000 7f3680b81640 1 -- 172.21.15.5:0/2552172703 >> 172.21.15.5:0/2552172703 conn(0x7f367c0fc780 msgr2=0x7f367c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.859+0000 7f3680b81640 1 -- 172.21.15.5:0/2552172703 shutdown_connections 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.859+0000 7f3680b81640 1 -- 172.21.15.5:0/2552172703 wait complete. 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f3680b81640 1 Processor -- start 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f3680b81640 1 -- start start 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f3680b81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f367c100a60 0x7f367c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f3680b81640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f367c101400 0x7f367c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f3680b81640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f367c106a60 0x7f367c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f3680b81640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f367c110e60 con 0x7f367c100a60 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f3680b81640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f367c110ce0 con 0x7f367c101400 2024-09-15T22:09:07.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f3680b81640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f367c110fe0 con 0x7f367c106a60 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f367a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f367c100a60 0x7f367c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f367a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f367c100a60 0x7f367c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46624/0 (socket says 172.21.15.5:46624) 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f367a59c640 1 -- 172.21.15.5:0/2225809262 learned_addr learned my addr 172.21.15.5:0/2225809262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f367ad9d640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f367c106a60 0x7f367c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.860+0000 7f3671d9b640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f367c101400 0x7f367c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.861+0000 7f367a59c640 1 -- 172.21.15.5:0/2225809262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f367c106a60 msgr2=0x7f367c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.861+0000 7f367a59c640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f367c106a60 0x7f367c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.861+0000 7f367a59c640 1 -- 172.21.15.5:0/2225809262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f367c101400 msgr2=0x7f367c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.861+0000 7f367a59c640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f367c101400 0x7f367c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.861+0000 7f367a59c640 1 -- 172.21.15.5:0/2225809262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c19d420 con 0x7f367c100a60 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.861+0000 7f367a59c640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f367c100a60 0x7f367c198470 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f366c00d950 tx=0x7f366c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:07.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.861+0000 7f3673fff640 1 -- 172.21.15.5:0/2225809262 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c017030 con 0x7f367c100a60 2024-09-15T22:09:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.861+0000 7f3673fff640 1 -- 172.21.15.5:0/2225809262 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f366c004020 con 0x7f367c100a60 2024-09-15T22:09:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.861+0000 7f3680b81640 1 -- 172.21.15.5:0/2225809262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367c19d710 con 0x7f367c100a60 2024-09-15T22:09:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.861+0000 7f3673fff640 1 -- 172.21.15.5:0/2225809262 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c0047a0 con 0x7f367c100a60 2024-09-15T22:09:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.861+0000 7f3680b81640 1 -- 172.21.15.5:0/2225809262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367c1a4ff0 con 0x7f367c100a60 2024-09-15T22:09:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.862+0000 7f3680b81640 1 -- 172.21.15.5:0/2225809262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f367c100ec0 con 0x7f367c100a60 2024-09-15T22:09:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.863+0000 7f3673fff640 1 -- 172.21.15.5:0/2225809262 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f366c0041d0 con 0x7f367c100a60 2024-09-15T22:09:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.864+0000 7f3673fff640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36680a34d0 0x7f36680a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.865+0000 7f3671d9b640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36680a34d0 0x7f36680a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.865+0000 7f3673fff640 1 -- 172.21.15.5:0/2225809262 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f366c0cfc50 con 0x7f367c100a60 2024-09-15T22:09:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.865+0000 7f3671d9b640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36680a34d0 0x7f36680a5990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3664009900 tx=0x7f36640058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:07.869+0000 7f3673fff640 1 -- 172.21.15.5:0/2225809262 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f366c0d5020 con 0x7f367c100a60 2024-09-15T22:09:08.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.041+0000 7f3680b81640 1 -- 172.21.15.5:0/2225809262 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f367c062630 con 0x7f36680a34d0 2024-09-15T22:09:08.043 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:07 smithi005 ceph-mon[30944]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:09:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.049+0000 7f3673fff640 1 -- 172.21.15.5:0/2225809262 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339416 (secure 0 0 0) 0x7f367c062630 con 0x7f36680a34d0 2024-09-15T22:09:08.054 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:08.055 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-15T22:09:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.053+0000 7f3680b81640 1 -- 172.21.15.5:0/2225809262 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36680a34d0 msgr2=0x7f36680a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.053+0000 7f3680b81640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36680a34d0 0x7f36680a5990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3664009900 tx=0x7f36640058d0 comp rx=0 tx=0).stop 2024-09-15T22:09:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.053+0000 7f3680b81640 1 -- 172.21.15.5:0/2225809262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f367c100a60 msgr2=0x7f367c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.053+0000 7f3680b81640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f367c100a60 0x7f367c198470 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f366c00d950 tx=0x7f366c00de10 comp rx=0 tx=0).stop 2024-09-15T22:09:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.054+0000 7f3680b81640 1 -- 172.21.15.5:0/2225809262 shutdown_connections 2024-09-15T22:09:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.054+0000 7f3680b81640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36680a34d0 0x7f36680a5990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.054+0000 7f3680b81640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f367c106a60 0x7f367c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.054+0000 7f3680b81640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f367c101400 0x7f367c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.054+0000 7f3680b81640 1 --2- 172.21.15.5:0/2225809262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f367c100a60 0x7f367c198470 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.054+0000 7f3680b81640 1 -- 172.21.15.5:0/2225809262 >> 172.21.15.5:0/2225809262 conn(0x7f367c0fc780 msgr2=0x7f367c06aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.054+0000 7f3680b81640 1 -- 172.21.15.5:0/2225809262 shutdown_connections 2024-09-15T22:09:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:08.054+0000 7f3680b81640 1 -- 172.21.15.5:0/2225809262 wait complete. 2024-09-15T22:09:08.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:07 smithi099 ceph-mon[34892]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:09:08.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:07 smithi129 ceph-mon[35314]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:09:08.795 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-15T22:09:07.504425+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1390313,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1264,"num_read_kb":1056,"num_write":633,"num_write_kb":5308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":968,"ondisk_log_size":968,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":398600,"kb_used_data":16052,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124543224,"statfs":{"total":1151940427776,"available":1151532261376,"internally_reserved":0,"allocated":16437248,"data_stored":10096313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":391492923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200331,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":398,"num_read_kb":459,"num_write":148,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003060"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489282+0000","last_change":"2024-09-15T22:07:46.226073+0000","last_active":"2024-09-15T22:08:53.489282+0000","last_peered":"2024-09-15T22:08:53.489282+0000","last_clean":"2024-09-15T22:08:53.489282+0000","last_became_active":"2024-09-15T22:07:46.219250+0000","last_became_peered":"2024-09-15T22:07:46.219250+0000","last_unstale":"2024-09-15T22:08:53.489282+0000","last_undegraded":"2024-09-15T22:08:53.489282+0000","last_fullsized":"2024-09-15T22:08:53.489282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:56.324959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,4],"acting":[9,11,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582824+0000","last_change":"2024-09-15T22:06:37.184096+0000","last_active":"2024-09-15T22:08:53.582824+0000","last_peered":"2024-09-15T22:08:53.582824+0000","last_clean":"2024-09-15T22:08:53.582824+0000","last_became_active":"2024-09-15T22:06:37.183758+0000","last_became_peered":"2024-09-15T22:06:37.183758+0000","last_unstale":"2024-09-15T22:08:53.582824+0000","last_undegraded":"2024-09-15T22:08:53.582824+0000","last_fullsized":"2024-09-15T22:08:53.582824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:04.210551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750577+0000","last_change":"2024-09-15T22:06:44.653042+0000","last_active":"2024-09-15T22:08:53.750577+0000","last_peered":"2024-09-15T22:08:53.750577+0000","last_clean":"2024-09-15T22:08:53.750577+0000","last_became_active":"2024-09-15T22:06:44.652658+0000","last_became_peered":"2024-09-15T22:06:44.652658+0000","last_unstale":"2024-09-15T22:08:53.750577+0000","last_undegraded":"2024-09-15T22:08:53.750577+0000","last_fullsized":"2024-09-15T22:08:53.750577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:59.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583148+0000","last_change":"2024-09-15T22:06:35.355392+0000","last_active":"2024-09-15T22:08:53.583148+0000","last_peered":"2024-09-15T22:08:53.583148+0000","last_clean":"2024-09-15T22:08:53.583148+0000","last_became_active":"2024-09-15T22:06:35.355042+0000","last_became_peered":"2024-09-15T22:06:35.355042+0000","last_unstale":"2024-09-15T22:08:53.583148+0000","last_undegraded":"2024-09-15T22:08:53.583148+0000","last_fullsized":"2024-09-15T22:08:53.583148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:05.186937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929360+0000","last_change":"2024-09-15T22:06:37.184421+0000","last_active":"2024-09-15T22:08:53.929360+0000","last_peered":"2024-09-15T22:08:53.929360+0000","last_clean":"2024-09-15T22:08:53.929360+0000","last_became_active":"2024-09-15T22:06:37.184092+0000","last_became_peered":"2024-09-15T22:06:37.184092+0000","last_unstale":"2024-09-15T22:08:53.929360+0000","last_undegraded":"2024-09-15T22:08:53.929360+0000","last_fullsized":"2024-09-15T22:08:53.929360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:41.080981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752738+0000","last_change":"2024-09-15T22:07:45.968640+0000","last_active":"2024-09-15T22:08:53.752738+0000","last_peered":"2024-09-15T22:08:53.752738+0000","last_clean":"2024-09-15T22:08:53.752738+0000","last_became_active":"2024-09-15T22:07:45.968320+0000","last_became_peered":"2024-09-15T22:07:45.968320+0000","last_unstale":"2024-09-15T22:08:53.752738+0000","last_undegraded":"2024-09-15T22:08:53.752738+0000","last_fullsized":"2024-09-15T22:08:53.752738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:28.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580416+0000","last_change":"2024-09-15T22:06:35.134606+0000","last_active":"2024-09-15T22:08:53.580416+0000","last_peered":"2024-09-15T22:08:53.580416+0000","last_clean":"2024-09-15T22:08:53.580416+0000","last_became_active":"2024-09-15T22:06:35.134263+0000","last_became_peered":"2024-09-15T22:06:35.134263+0000","last_unstale":"2024-09-15T22:08:53.580416+0000","last_undegraded":"2024-09-15T22:08:53.580416+0000","last_fullsized":"2024-09-15T22:08:53.580416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:45.008935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750806+0000","last_change":"2024-09-15T22:06:37.177556+0000","last_active":"2024-09-15T22:08:53.750806+0000","last_peered":"2024-09-15T22:08:53.750806+0000","last_clean":"2024-09-15T22:08:53.750806+0000","last_became_active":"2024-09-15T22:06:37.177161+0000","last_became_peered":"2024-09-15T22:06:37.177161+0000","last_unstale":"2024-09-15T22:08:53.750806+0000","last_undegraded":"2024-09-15T22:08:53.750806+0000","last_fullsized":"2024-09-15T22:08:53.750806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:30.192728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753347+0000","last_change":"2024-09-15T22:06:44.659097+0000","last_active":"2024-09-15T22:08:53.753347+0000","last_peered":"2024-09-15T22:08:53.753347+0000","last_clean":"2024-09-15T22:08:53.753347+0000","last_became_active":"2024-09-15T22:06:44.658626+0000","last_became_peered":"2024-09-15T22:06:44.658626+0000","last_unstale":"2024-09-15T22:08:53.753347+0000","last_undegraded":"2024-09-15T22:08:53.753347+0000","last_fullsized":"2024-09-15T22:08:53.753347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:01.308004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580430+0000","last_change":"2024-09-15T22:06:35.136326+0000","last_active":"2024-09-15T22:08:53.580430+0000","last_peered":"2024-09-15T22:08:53.580430+0000","last_clean":"2024-09-15T22:08:53.580430+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.580430+0000","last_undegraded":"2024-09-15T22:08:53.580430+0000","last_fullsized":"2024-09-15T22:08:53.580430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:28.675832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490507+0000","last_change":"2024-09-15T22:06:37.192939+0000","last_active":"2024-09-15T22:08:53.490507+0000","last_peered":"2024-09-15T22:08:53.490507+0000","last_clean":"2024-09-15T22:08:53.490507+0000","last_became_active":"2024-09-15T22:06:37.192616+0000","last_became_peered":"2024-09-15T22:06:37.192616+0000","last_unstale":"2024-09-15T22:08:53.490507+0000","last_undegraded":"2024-09-15T22:08:53.490507+0000","last_fullsized":"2024-09-15T22:08:53.490507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:03.615203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752853+0000","last_change":"2024-09-15T22:06:44.654741+0000","last_active":"2024-09-15T22:08:53.752853+0000","last_peered":"2024-09-15T22:08:53.752853+0000","last_clean":"2024-09-15T22:08:53.752853+0000","last_became_active":"2024-09-15T22:06:44.654362+0000","last_became_peered":"2024-09-15T22:06:44.654362+0000","last_unstale":"2024-09-15T22:08:53.752853+0000","last_undegraded":"2024-09-15T22:08:53.752853+0000","last_fullsized":"2024-09-15T22:08:53.752853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:59.269948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490328+0000","last_change":"2024-09-15T22:06:37.186385+0000","last_active":"2024-09-15T22:08:53.490328+0000","last_peered":"2024-09-15T22:08:53.490328+0000","last_clean":"2024-09-15T22:08:53.490328+0000","last_became_active":"2024-09-15T22:06:37.185588+0000","last_became_peered":"2024-09-15T22:06:37.185588+0000","last_unstale":"2024-09-15T22:08:53.490328+0000","last_undegraded":"2024-09-15T22:08:53.490328+0000","last_fullsized":"2024-09-15T22:08:53.490328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:22.523083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580272+0000","last_change":"2024-09-15T22:06:35.137383+0000","last_active":"2024-09-15T22:08:53.580272+0000","last_peered":"2024-09-15T22:08:53.580272+0000","last_clean":"2024-09-15T22:08:53.580272+0000","last_became_active":"2024-09-15T22:06:35.137022+0000","last_became_peered":"2024-09-15T22:06:35.137022+0000","last_unstale":"2024-09-15T22:08:53.580272+0000","last_undegraded":"2024-09-15T22:08:53.580272+0000","last_fullsized":"2024-09-15T22:08:53.580272+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.863155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753052+0000","last_change":"2024-09-15T22:06:44.653807+0000","last_active":"2024-09-15T22:08:53.753052+0000","last_peered":"2024-09-15T22:08:53.753052+0000","last_clean":"2024-09-15T22:08:53.753052+0000","last_became_active":"2024-09-15T22:06:44.653406+0000","last_became_peered":"2024-09-15T22:06:44.653406+0000","last_unstale":"2024-09-15T22:08:53.753052+0000","last_undegraded":"2024-09-15T22:08:53.753052+0000","last_fullsized":"2024-09-15T22:08:53.753052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:12.018208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582122+0000","last_change":"2024-09-15T22:06:37.186899+0000","last_active":"2024-09-15T22:08:53.582122+0000","last_peered":"2024-09-15T22:08:53.582122+0000","last_clean":"2024-09-15T22:08:53.582122+0000","last_became_active":"2024-09-15T22:06:37.183782+0000","last_became_peered":"2024-09-15T22:06:37.183782+0000","last_unstale":"2024-09-15T22:08:53.582122+0000","last_undegraded":"2024-09-15T22:08:53.582122+0000","last_fullsized":"2024-09-15T22:08:53.582122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:39.269186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:54.735821+0000","last_change":"2024-09-15T22:06:44.652686+0000","last_active":"2024-09-15T22:08:54.735821+0000","last_peered":"2024-09-15T22:08:54.735821+0000","last_clean":"2024-09-15T22:08:54.735821+0000","last_became_active":"2024-09-15T22:06:44.652306+0000","last_became_peered":"2024-09-15T22:06:44.652306+0000","last_unstale":"2024-09-15T22:08:54.735821+0000","last_undegraded":"2024-09-15T22:08:54.735821+0000","last_fullsized":"2024-09-15T22:08:54.735821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753085+0000","last_change":"2024-09-15T22:06:44.655424+0000","last_active":"2024-09-15T22:08:53.753085+0000","last_peered":"2024-09-15T22:08:53.753085+0000","last_clean":"2024-09-15T22:08:53.753085+0000","last_became_active":"2024-09-15T22:06:44.654959+0000","last_became_peered":"2024-09-15T22:06:44.654959+0000","last_unstale":"2024-09-15T22:08:53.753085+0000","last_undegraded":"2024-09-15T22:08:53.753085+0000","last_fullsized":"2024-09-15T22:08:53.753085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:07.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928379+0000","last_change":"2024-09-15T22:06:37.188424+0000","last_active":"2024-09-15T22:08:53.928379+0000","last_peered":"2024-09-15T22:08:53.928379+0000","last_clean":"2024-09-15T22:08:53.928379+0000","last_became_active":"2024-09-15T22:06:37.188082+0000","last_became_peered":"2024-09-15T22:06:37.188082+0000","last_unstale":"2024-09-15T22:08:53.928379+0000","last_undegraded":"2024-09-15T22:08:53.928379+0000","last_fullsized":"2024-09-15T22:08:53.928379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:07.159341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581279+0000","last_change":"2024-09-15T22:06:35.134237+0000","last_active":"2024-09-15T22:08:53.581279+0000","last_peered":"2024-09-15T22:08:53.581279+0000","last_clean":"2024-09-15T22:08:53.581279+0000","last_became_active":"2024-09-15T22:06:35.133885+0000","last_became_peered":"2024-09-15T22:06:35.133885+0000","last_unstale":"2024-09-15T22:08:53.581279+0000","last_undegraded":"2024-09-15T22:08:53.581279+0000","last_fullsized":"2024-09-15T22:08:53.581279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:07:35.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489113+0000","last_change":"2024-09-15T22:06:44.664578+0000","last_active":"2024-09-15T22:08:53.489113+0000","last_peered":"2024-09-15T22:08:53.489113+0000","last_clean":"2024-09-15T22:08:53.489113+0000","last_became_active":"2024-09-15T22:06:44.664170+0000","last_became_peered":"2024-09-15T22:06:44.664170+0000","last_unstale":"2024-09-15T22:08:53.489113+0000","last_undegraded":"2024-09-15T22:08:53.489113+0000","last_fullsized":"2024-09-15T22:08:53.489113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:41.443816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580024+0000","last_change":"2024-09-15T22:06:37.186107+0000","last_active":"2024-09-15T22:08:53.580024+0000","last_peered":"2024-09-15T22:08:53.580024+0000","last_clean":"2024-09-15T22:08:53.580024+0000","last_became_active":"2024-09-15T22:06:37.185757+0000","last_became_peered":"2024-09-15T22:06:37.185757+0000","last_unstale":"2024-09-15T22:08:53.580024+0000","last_undegraded":"2024-09-15T22:08:53.580024+0000","last_fullsized":"2024-09-15T22:08:53.580024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:14.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750623+0000","last_change":"2024-09-15T22:06:35.356281+0000","last_active":"2024-09-15T22:08:53.750623+0000","last_peered":"2024-09-15T22:08:53.750623+0000","last_clean":"2024-09-15T22:08:53.750623+0000","last_became_active":"2024-09-15T22:06:35.355922+0000","last_became_peered":"2024-09-15T22:06:35.355922+0000","last_unstale":"2024-09-15T22:08:53.750623+0000","last_undegraded":"2024-09-15T22:08:53.750623+0000","last_fullsized":"2024-09-15T22:08:53.750623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:06.107462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753055+0000","last_change":"2024-09-15T22:06:44.659230+0000","last_active":"2024-09-15T22:08:53.753055+0000","last_peered":"2024-09-15T22:08:53.753055+0000","last_clean":"2024-09-15T22:08:53.753055+0000","last_became_active":"2024-09-15T22:06:44.658649+0000","last_became_peered":"2024-09-15T22:06:44.658649+0000","last_unstale":"2024-09-15T22:08:53.753055+0000","last_undegraded":"2024-09-15T22:08:53.753055+0000","last_fullsized":"2024-09-15T22:08:53.753055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:14:21.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929164+0000","last_change":"2024-09-15T22:06:44.649394+0000","last_active":"2024-09-15T22:08:53.929164+0000","last_peered":"2024-09-15T22:08:53.929164+0000","last_clean":"2024-09-15T22:08:53.929164+0000","last_became_active":"2024-09-15T22:06:44.649043+0000","last_became_peered":"2024-09-15T22:06:44.649043+0000","last_unstale":"2024-09-15T22:08:53.929164+0000","last_undegraded":"2024-09-15T22:08:53.929164+0000","last_fullsized":"2024-09-15T22:08:53.929164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:56:28.335869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753504+0000","last_change":"2024-09-15T22:06:35.265970+0000","last_active":"2024-09-15T22:08:53.753504+0000","last_peered":"2024-09-15T22:08:53.753504+0000","last_clean":"2024-09-15T22:08:53.753504+0000","last_became_active":"2024-09-15T22:06:35.265628+0000","last_became_peered":"2024-09-15T22:06:35.265628+0000","last_unstale":"2024-09-15T22:08:53.753504+0000","last_undegraded":"2024-09-15T22:08:53.753504+0000","last_fullsized":"2024-09-15T22:08:53.753504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:41.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750664+0000","last_change":"2024-09-15T22:06:44.649652+0000","last_active":"2024-09-15T22:08:53.750664+0000","last_peered":"2024-09-15T22:08:53.750664+0000","last_clean":"2024-09-15T22:08:53.750664+0000","last_became_active":"2024-09-15T22:06:44.649112+0000","last_became_peered":"2024-09-15T22:06:44.649112+0000","last_unstale":"2024-09-15T22:08:53.750664+0000","last_undegraded":"2024-09-15T22:08:53.750664+0000","last_fullsized":"2024-09-15T22:08:53.750664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:53.789320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583103+0000","last_change":"2024-09-15T22:06:37.190701+0000","last_active":"2024-09-15T22:08:53.583103+0000","last_peered":"2024-09-15T22:08:53.583103+0000","last_clean":"2024-09-15T22:08:53.583103+0000","last_became_active":"2024-09-15T22:06:37.190430+0000","last_became_peered":"2024-09-15T22:06:37.190430+0000","last_unstale":"2024-09-15T22:08:53.583103+0000","last_undegraded":"2024-09-15T22:08:53.583103+0000","last_fullsized":"2024-09-15T22:08:53.583103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:14.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'101","reported_seq":224,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:04.325283+0000","last_change":"2024-09-15T22:06:35.133899+0000","last_active":"2024-09-15T22:09:04.325283+0000","last_peered":"2024-09-15T22:09:04.325283+0000","last_clean":"2024-09-15T22:09:04.325283+0000","last_became_active":"2024-09-15T22:06:35.133556+0000","last_became_peered":"2024-09-15T22:06:35.133556+0000","last_unstale":"2024-09-15T22:09:04.325283+0000","last_undegraded":"2024-09-15T22:09:04.325283+0000","last_fullsized":"2024-09-15T22:09:04.325283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:01.692416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":185,"num_write_kb":1289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580427+0000","last_change":"2024-09-15T22:06:44.655280+0000","last_active":"2024-09-15T22:08:53.580427+0000","last_peered":"2024-09-15T22:08:53.580427+0000","last_clean":"2024-09-15T22:08:53.580427+0000","last_became_active":"2024-09-15T22:06:44.654883+0000","last_became_peered":"2024-09-15T22:06:44.654883+0000","last_unstale":"2024-09-15T22:08:53.580427+0000","last_undegraded":"2024-09-15T22:08:53.580427+0000","last_fullsized":"2024-09-15T22:08:53.580427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.074035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889279+0000","last_change":"2024-09-15T22:06:37.182006+0000","last_active":"2024-09-15T22:08:53.889279+0000","last_peered":"2024-09-15T22:08:53.889279+0000","last_clean":"2024-09-15T22:08:53.889279+0000","last_became_active":"2024-09-15T22:06:37.181693+0000","last_became_peered":"2024-09-15T22:06:37.181693+0000","last_unstale":"2024-09-15T22:08:53.889279+0000","last_undegraded":"2024-09-15T22:08:53.889279+0000","last_fullsized":"2024-09-15T22:08:53.889279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:23.483930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580471+0000","last_change":"2024-09-15T22:06:35.355854+0000","last_active":"2024-09-15T22:08:53.580471+0000","last_peered":"2024-09-15T22:08:53.580471+0000","last_clean":"2024-09-15T22:08:53.580471+0000","last_became_active":"2024-09-15T22:06:35.355486+0000","last_became_peered":"2024-09-15T22:06:35.355486+0000","last_unstale":"2024-09-15T22:08:53.580471+0000","last_undegraded":"2024-09-15T22:08:53.580471+0000","last_fullsized":"2024-09-15T22:08:53.580471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:36.708476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928367+0000","last_change":"2024-09-15T22:06:44.660523+0000","last_active":"2024-09-15T22:08:53.928367+0000","last_peered":"2024-09-15T22:08:53.928367+0000","last_clean":"2024-09-15T22:08:53.928367+0000","last_became_active":"2024-09-15T22:06:44.660113+0000","last_became_peered":"2024-09-15T22:06:44.660113+0000","last_unstale":"2024-09-15T22:08:53.928367+0000","last_undegraded":"2024-09-15T22:08:53.928367+0000","last_fullsized":"2024-09-15T22:08:53.928367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:42.164558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589575+0000","last_change":"2024-09-15T22:06:37.183649+0000","last_active":"2024-09-15T22:08:53.589575+0000","last_peered":"2024-09-15T22:08:53.589575+0000","last_clean":"2024-09-15T22:08:53.589575+0000","last_became_active":"2024-09-15T22:06:37.183296+0000","last_became_peered":"2024-09-15T22:06:37.183296+0000","last_unstale":"2024-09-15T22:08:53.589575+0000","last_undegraded":"2024-09-15T22:08:53.589575+0000","last_fullsized":"2024-09-15T22:08:53.589575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:22.010288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488860+0000","last_change":"2024-09-15T22:06:35.355068+0000","last_active":"2024-09-15T22:08:53.488860+0000","last_peered":"2024-09-15T22:08:53.488860+0000","last_clean":"2024-09-15T22:08:53.488860+0000","last_became_active":"2024-09-15T22:06:35.354728+0000","last_became_peered":"2024-09-15T22:06:35.354728+0000","last_unstale":"2024-09-15T22:08:53.488860+0000","last_undegraded":"2024-09-15T22:08:53.488860+0000","last_fullsized":"2024-09-15T22:08:53.488860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:35.479290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580098+0000","last_change":"2024-09-15T22:06:44.653849+0000","last_active":"2024-09-15T22:08:53.580098+0000","last_peered":"2024-09-15T22:08:53.580098+0000","last_clean":"2024-09-15T22:08:53.580098+0000","last_became_active":"2024-09-15T22:06:44.653419+0000","last_became_peered":"2024-09-15T22:06:44.653419+0000","last_unstale":"2024-09-15T22:08:53.580098+0000","last_undegraded":"2024-09-15T22:08:53.580098+0000","last_fullsized":"2024-09-15T22:08:53.580098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:04.131553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589446+0000","last_change":"2024-09-15T22:06:37.191438+0000","last_active":"2024-09-15T22:08:53.589446+0000","last_peered":"2024-09-15T22:08:53.589446+0000","last_clean":"2024-09-15T22:08:53.589446+0000","last_became_active":"2024-09-15T22:06:37.191088+0000","last_became_peered":"2024-09-15T22:06:37.191088+0000","last_unstale":"2024-09-15T22:08:53.589446+0000","last_undegraded":"2024-09-15T22:08:53.589446+0000","last_fullsized":"2024-09-15T22:08:53.589446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:46:36.078872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490414+0000","last_change":"2024-09-15T22:06:35.135866+0000","last_active":"2024-09-15T22:08:53.490414+0000","last_peered":"2024-09-15T22:08:53.490414+0000","last_clean":"2024-09-15T22:08:53.490414+0000","last_became_active":"2024-09-15T22:06:35.135524+0000","last_became_peered":"2024-09-15T22:06:35.135524+0000","last_unstale":"2024-09-15T22:08:53.490414+0000","last_undegraded":"2024-09-15T22:08:53.490414+0000","last_fullsized":"2024-09-15T22:08:53.490414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888980+0000","last_change":"2024-09-15T22:06:44.649943+0000","last_active":"2024-09-15T22:08:53.888980+0000","last_peered":"2024-09-15T22:08:53.888980+0000","last_clean":"2024-09-15T22:08:53.888980+0000","last_became_active":"2024-09-15T22:06:44.649579+0000","last_became_peered":"2024-09-15T22:06:44.649579+0000","last_unstale":"2024-09-15T22:08:53.888980+0000","last_undegraded":"2024-09-15T22:08:53.888980+0000","last_fullsized":"2024-09-15T22:08:53.888980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:18.390608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489996+0000","last_change":"2024-09-15T22:06:37.189988+0000","last_active":"2024-09-15T22:08:53.489996+0000","last_peered":"2024-09-15T22:08:53.489996+0000","last_clean":"2024-09-15T22:08:53.489996+0000","last_became_active":"2024-09-15T22:06:37.189319+0000","last_became_peered":"2024-09-15T22:06:37.189319+0000","last_unstale":"2024-09-15T22:08:53.489996+0000","last_undegraded":"2024-09-15T22:08:53.489996+0000","last_fullsized":"2024-09-15T22:08:53.489996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:08.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.902547+0000","last_change":"2024-09-15T22:06:35.355371+0000","last_active":"2024-09-15T22:09:02.902547+0000","last_peered":"2024-09-15T22:09:02.902547+0000","last_clean":"2024-09-15T22:09:02.902547+0000","last_became_active":"2024-09-15T22:06:35.354687+0000","last_became_peered":"2024-09-15T22:06:35.354687+0000","last_unstale":"2024-09-15T22:09:02.902547+0000","last_undegraded":"2024-09-15T22:09:02.902547+0000","last_fullsized":"2024-09-15T22:09:02.902547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:27.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889597+0000","last_change":"2024-09-15T22:06:44.657929+0000","last_active":"2024-09-15T22:08:53.889597+0000","last_peered":"2024-09-15T22:08:53.889597+0000","last_clean":"2024-09-15T22:08:53.889597+0000","last_became_active":"2024-09-15T22:06:44.657519+0000","last_became_peered":"2024-09-15T22:06:44.657519+0000","last_unstale":"2024-09-15T22:08:53.889597+0000","last_undegraded":"2024-09-15T22:08:53.889597+0000","last_fullsized":"2024-09-15T22:08:53.889597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:12.313877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888747+0000","last_change":"2024-09-15T22:06:37.193288+0000","last_active":"2024-09-15T22:08:53.888747+0000","last_peered":"2024-09-15T22:08:53.888747+0000","last_clean":"2024-09-15T22:08:53.888747+0000","last_became_active":"2024-09-15T22:06:37.193007+0000","last_became_peered":"2024-09-15T22:06:37.193007+0000","last_unstale":"2024-09-15T22:08:53.888747+0000","last_undegraded":"2024-09-15T22:08:53.888747+0000","last_fullsized":"2024-09-15T22:08:53.888747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:45.956206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490201+0000","last_change":"2024-09-15T22:06:35.138726+0000","last_active":"2024-09-15T22:08:53.490201+0000","last_peered":"2024-09-15T22:08:53.490201+0000","last_clean":"2024-09-15T22:08:53.490201+0000","last_became_active":"2024-09-15T22:06:35.138390+0000","last_became_peered":"2024-09-15T22:06:35.138390+0000","last_unstale":"2024-09-15T22:08:53.490201+0000","last_undegraded":"2024-09-15T22:08:53.490201+0000","last_fullsized":"2024-09-15T22:08:53.490201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:51.336171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490388+0000","last_change":"2024-09-15T22:07:46.219720+0000","last_active":"2024-09-15T22:08:53.490388+0000","last_peered":"2024-09-15T22:08:53.490388+0000","last_clean":"2024-09-15T22:08:53.490388+0000","last_became_active":"2024-09-15T22:07:46.219303+0000","last_became_peered":"2024-09-15T22:07:46.219303+0000","last_unstale":"2024-09-15T22:08:53.490388+0000","last_undegraded":"2024-09-15T22:08:53.490388+0000","last_fullsized":"2024-09-15T22:08:53.490388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:03.596795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581603+0000","last_change":"2024-09-15T22:06:37.176478+0000","last_active":"2024-09-15T22:08:53.581603+0000","last_peered":"2024-09-15T22:08:53.581603+0000","last_clean":"2024-09-15T22:08:53.581603+0000","last_became_active":"2024-09-15T22:06:37.176171+0000","last_became_peered":"2024-09-15T22:06:37.176171+0000","last_unstale":"2024-09-15T22:08:53.581603+0000","last_undegraded":"2024-09-15T22:08:53.581603+0000","last_fullsized":"2024-09-15T22:08:53.581603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753268+0000","last_change":"2024-09-15T22:06:35.134651+0000","last_active":"2024-09-15T22:08:53.753268+0000","last_peered":"2024-09-15T22:08:53.753268+0000","last_clean":"2024-09-15T22:08:53.753268+0000","last_became_active":"2024-09-15T22:06:35.134185+0000","last_became_peered":"2024-09-15T22:06:35.134185+0000","last_unstale":"2024-09-15T22:08:53.753268+0000","last_undegraded":"2024-09-15T22:08:53.753268+0000","last_fullsized":"2024-09-15T22:08:53.753268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:56.385046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"89'148","reported_seq":402,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:55.596394+0000","last_change":"2024-09-15T22:05:23.048068+0000","last_active":"2024-09-15T22:08:55.596394+0000","last_peered":"2024-09-15T22:08:55.596394+0000","last_clean":"2024-09-15T22:08:55.596394+0000","last_became_active":"2024-09-15T22:05:23.047692+0000","last_became_peered":"2024-09-15T22:05:23.047692+0000","last_unstale":"2024-09-15T22:08:55.596394+0000","last_undegraded":"2024-09-15T22:08:55.596394+0000","last_fullsized":"2024-09-15T22:08:55.596394+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_clean_scrub_stamp":"2024-09-15T22:01:57.477629+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490625+0000","last_change":"2024-09-15T22:06:44.657213+0000","last_active":"2024-09-15T22:08:53.490625+0000","last_peered":"2024-09-15T22:08:53.490625+0000","last_clean":"2024-09-15T22:08:53.490625+0000","last_became_active":"2024-09-15T22:06:44.656815+0000","last_became_peered":"2024-09-15T22:06:44.656815+0000","last_unstale":"2024-09-15T22:08:53.490625+0000","last_undegraded":"2024-09-15T22:08:53.490625+0000","last_fullsized":"2024-09-15T22:08:53.490625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:26.525992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490755+0000","last_change":"2024-09-15T22:06:37.184467+0000","last_active":"2024-09-15T22:08:53.490755+0000","last_peered":"2024-09-15T22:08:53.490755+0000","last_clean":"2024-09-15T22:08:53.490755+0000","last_became_active":"2024-09-15T22:06:37.183864+0000","last_became_peered":"2024-09-15T22:06:37.183864+0000","last_unstale":"2024-09-15T22:08:53.490755+0000","last_undegraded":"2024-09-15T22:08:53.490755+0000","last_fullsized":"2024-09-15T22:08:53.490755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:35.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750805+0000","last_change":"2024-09-15T22:06:44.651594+0000","last_active":"2024-09-15T22:08:53.750805+0000","last_peered":"2024-09-15T22:08:53.750805+0000","last_clean":"2024-09-15T22:08:53.750805+0000","last_became_active":"2024-09-15T22:06:44.650393+0000","last_became_peered":"2024-09-15T22:06:44.650393+0000","last_unstale":"2024-09-15T22:08:53.750805+0000","last_undegraded":"2024-09-15T22:08:53.750805+0000","last_fullsized":"2024-09-15T22:08:53.750805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:13.599744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750608+0000","last_change":"2024-09-15T22:06:44.660789+0000","last_active":"2024-09-15T22:08:53.750608+0000","last_peered":"2024-09-15T22:08:53.750608+0000","last_clean":"2024-09-15T22:08:53.750608+0000","last_became_active":"2024-09-15T22:06:44.660331+0000","last_became_peered":"2024-09-15T22:06:44.660331+0000","last_unstale":"2024-09-15T22:08:53.750608+0000","last_undegraded":"2024-09-15T22:08:53.750608+0000","last_fullsized":"2024-09-15T22:08:53.750608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:04:45.562795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490532+0000","last_change":"2024-09-15T22:06:37.185697+0000","last_active":"2024-09-15T22:08:53.490532+0000","last_peered":"2024-09-15T22:08:53.490532+0000","last_clean":"2024-09-15T22:08:53.490532+0000","last_became_active":"2024-09-15T22:06:37.185207+0000","last_became_peered":"2024-09-15T22:06:37.185207+0000","last_unstale":"2024-09-15T22:08:53.490532+0000","last_undegraded":"2024-09-15T22:08:53.490532+0000","last_fullsized":"2024-09-15T22:08:53.490532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:39.005273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'65","reported_seq":131,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:04.319716+0000","last_change":"2024-09-15T22:06:35.135176+0000","last_active":"2024-09-15T22:09:04.319716+0000","last_peered":"2024-09-15T22:09:04.319716+0000","last_clean":"2024-09-15T22:09:04.319716+0000","last_became_active":"2024-09-15T22:06:35.134649+0000","last_became_peered":"2024-09-15T22:06:35.134649+0000","last_unstale":"2024-09-15T22:09:04.319716+0000","last_undegraded":"2024-09-15T22:09:04.319716+0000","last_fullsized":"2024-09-15T22:09:04.319716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:33.780162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":31,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928755+0000","last_change":"2024-09-15T22:06:44.656972+0000","last_active":"2024-09-15T22:08:53.928755+0000","last_peered":"2024-09-15T22:08:53.928755+0000","last_clean":"2024-09-15T22:08:53.928755+0000","last_became_active":"2024-09-15T22:06:44.656528+0000","last_became_peered":"2024-09-15T22:06:44.656528+0000","last_unstale":"2024-09-15T22:08:53.928755+0000","last_undegraded":"2024-09-15T22:08:53.928755+0000","last_fullsized":"2024-09-15T22:08:53.928755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:09.164596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490782+0000","last_change":"2024-09-15T22:06:37.186118+0000","last_active":"2024-09-15T22:08:53.490782+0000","last_peered":"2024-09-15T22:08:53.490782+0000","last_clean":"2024-09-15T22:08:53.490782+0000","last_became_active":"2024-09-15T22:06:37.185118+0000","last_became_peered":"2024-09-15T22:06:37.185118+0000","last_unstale":"2024-09-15T22:08:53.490782+0000","last_undegraded":"2024-09-15T22:08:53.490782+0000","last_fullsized":"2024-09-15T22:08:53.490782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:53.357045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889478+0000","last_change":"2024-09-15T22:06:35.133341+0000","last_active":"2024-09-15T22:08:53.889478+0000","last_peered":"2024-09-15T22:08:53.889478+0000","last_clean":"2024-09-15T22:08:53.889478+0000","last_became_active":"2024-09-15T22:06:35.132961+0000","last_became_peered":"2024-09-15T22:06:35.132961+0000","last_unstale":"2024-09-15T22:08:53.889478+0000","last_undegraded":"2024-09-15T22:08:53.889478+0000","last_fullsized":"2024-09-15T22:08:53.889478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:39.579470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489873+0000","last_change":"2024-09-15T22:06:44.659148+0000","last_active":"2024-09-15T22:08:53.489873+0000","last_peered":"2024-09-15T22:08:53.489873+0000","last_clean":"2024-09-15T22:08:53.489873+0000","last_became_active":"2024-09-15T22:06:44.658750+0000","last_became_peered":"2024-09-15T22:06:44.658750+0000","last_unstale":"2024-09-15T22:08:53.489873+0000","last_undegraded":"2024-09-15T22:08:53.489873+0000","last_fullsized":"2024-09-15T22:08:53.489873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:48.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928647+0000","last_change":"2024-09-15T22:06:37.194066+0000","last_active":"2024-09-15T22:08:53.928647+0000","last_peered":"2024-09-15T22:08:53.928647+0000","last_clean":"2024-09-15T22:08:53.928647+0000","last_became_active":"2024-09-15T22:06:37.193738+0000","last_became_peered":"2024-09-15T22:06:37.193738+0000","last_unstale":"2024-09-15T22:08:53.928647+0000","last_undegraded":"2024-09-15T22:08:53.928647+0000","last_fullsized":"2024-09-15T22:08:53.928647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.107188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:54.504334+0000","last_change":"2024-09-15T22:06:35.136470+0000","last_active":"2024-09-15T22:08:54.504334+0000","last_peered":"2024-09-15T22:08:54.504334+0000","last_clean":"2024-09-15T22:08:54.504334+0000","last_became_active":"2024-09-15T22:06:35.135953+0000","last_became_peered":"2024-09-15T22:06:35.135953+0000","last_unstale":"2024-09-15T22:08:54.504334+0000","last_undegraded":"2024-09-15T22:08:54.504334+0000","last_fullsized":"2024-09-15T22:08:54.504334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:44.548566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490037+0000","last_change":"2024-09-15T22:06:44.653682+0000","last_active":"2024-09-15T22:08:53.490037+0000","last_peered":"2024-09-15T22:08:53.490037+0000","last_clean":"2024-09-15T22:08:53.490037+0000","last_became_active":"2024-09-15T22:06:44.653303+0000","last_became_peered":"2024-09-15T22:06:44.653303+0000","last_unstale":"2024-09-15T22:08:53.490037+0000","last_undegraded":"2024-09-15T22:08:53.490037+0000","last_fullsized":"2024-09-15T22:08:53.490037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:07.847971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490017+0000","last_change":"2024-09-15T22:06:44.658436+0000","last_active":"2024-09-15T22:08:53.490017+0000","last_peered":"2024-09-15T22:08:53.490017+0000","last_clean":"2024-09-15T22:08:53.490017+0000","last_became_active":"2024-09-15T22:06:44.658037+0000","last_became_peered":"2024-09-15T22:06:44.658037+0000","last_unstale":"2024-09-15T22:08:53.490017+0000","last_undegraded":"2024-09-15T22:08:53.490017+0000","last_fullsized":"2024-09-15T22:08:53.490017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:34:20.369249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580102+0000","last_change":"2024-09-15T22:06:35.358018+0000","last_active":"2024-09-15T22:08:53.580102+0000","last_peered":"2024-09-15T22:08:53.580102+0000","last_clean":"2024-09-15T22:08:53.580102+0000","last_became_active":"2024-09-15T22:06:35.357666+0000","last_became_peered":"2024-09-15T22:06:35.357666+0000","last_unstale":"2024-09-15T22:08:53.580102+0000","last_undegraded":"2024-09-15T22:08:53.580102+0000","last_fullsized":"2024-09-15T22:08:53.580102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:34.089113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889340+0000","last_change":"2024-09-15T22:06:44.656807+0000","last_active":"2024-09-15T22:08:53.889340+0000","last_peered":"2024-09-15T22:08:53.889340+0000","last_clean":"2024-09-15T22:08:53.889340+0000","last_became_active":"2024-09-15T22:06:44.656409+0000","last_became_peered":"2024-09-15T22:06:44.656409+0000","last_unstale":"2024-09-15T22:08:53.889340+0000","last_undegraded":"2024-09-15T22:08:53.889340+0000","last_fullsized":"2024-09-15T22:08:53.889340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490317+0000","last_change":"2024-09-15T22:06:37.180615+0000","last_active":"2024-09-15T22:08:53.490317+0000","last_peered":"2024-09-15T22:08:53.490317+0000","last_clean":"2024-09-15T22:08:53.490317+0000","last_became_active":"2024-09-15T22:06:37.180306+0000","last_became_peered":"2024-09-15T22:06:37.180306+0000","last_unstale":"2024-09-15T22:08:53.490317+0000","last_undegraded":"2024-09-15T22:08:53.490317+0000","last_fullsized":"2024-09-15T22:08:53.490317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:31.145128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.903158+0000","last_change":"2024-09-15T22:06:35.136192+0000","last_active":"2024-09-15T22:09:02.903158+0000","last_peered":"2024-09-15T22:09:02.903158+0000","last_clean":"2024-09-15T22:09:02.903158+0000","last_became_active":"2024-09-15T22:06:35.135608+0000","last_became_peered":"2024-09-15T22:06:35.135608+0000","last_unstale":"2024-09-15T22:09:02.903158+0000","last_undegraded":"2024-09-15T22:09:02.903158+0000","last_fullsized":"2024-09-15T22:09:02.903158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:48.472717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489151+0000","last_change":"2024-09-15T22:06:44.664763+0000","last_active":"2024-09-15T22:08:53.489151+0000","last_peered":"2024-09-15T22:08:53.489151+0000","last_clean":"2024-09-15T22:08:53.489151+0000","last_became_active":"2024-09-15T22:06:44.664410+0000","last_became_peered":"2024-09-15T22:06:44.664410+0000","last_unstale":"2024-09-15T22:08:53.489151+0000","last_undegraded":"2024-09-15T22:08:53.489151+0000","last_fullsized":"2024-09-15T22:08:53.489151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:14.364387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490348+0000","last_change":"2024-09-15T22:06:44.652747+0000","last_active":"2024-09-15T22:08:53.490348+0000","last_peered":"2024-09-15T22:08:53.490348+0000","last_clean":"2024-09-15T22:08:53.490348+0000","last_became_active":"2024-09-15T22:06:44.652368+0000","last_became_peered":"2024-09-15T22:06:44.652368+0000","last_unstale":"2024-09-15T22:08:53.490348+0000","last_undegraded":"2024-09-15T22:08:53.490348+0000","last_fullsized":"2024-09-15T22:08:53.490348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:51.047549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490159+0000","last_change":"2024-09-15T22:06:35.136311+0000","last_active":"2024-09-15T22:08:53.490159+0000","last_peered":"2024-09-15T22:08:53.490159+0000","last_clean":"2024-09-15T22:08:53.490159+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.490159+0000","last_undegraded":"2024-09-15T22:08:53.490159+0000","last_fullsized":"2024-09-15T22:08:53.490159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:36:07.138056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928366+0000","last_change":"2024-09-15T22:06:44.653747+0000","last_active":"2024-09-15T22:08:53.928366+0000","last_peered":"2024-09-15T22:08:53.928366+0000","last_clean":"2024-09-15T22:08:53.928366+0000","last_became_active":"2024-09-15T22:06:44.653357+0000","last_became_peered":"2024-09-15T22:06:44.653357+0000","last_unstale":"2024-09-15T22:08:53.928366+0000","last_undegraded":"2024-09-15T22:08:53.928366+0000","last_fullsized":"2024-09-15T22:08:53.928366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.113924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888302+0000","last_change":"2024-09-15T22:06:37.178308+0000","last_active":"2024-09-15T22:08:53.888302+0000","last_peered":"2024-09-15T22:08:53.888302+0000","last_clean":"2024-09-15T22:08:53.888302+0000","last_became_active":"2024-09-15T22:06:37.177920+0000","last_became_peered":"2024-09-15T22:06:37.177920+0000","last_unstale":"2024-09-15T22:08:53.888302+0000","last_undegraded":"2024-09-15T22:08:53.888302+0000","last_fullsized":"2024-09-15T22:08:53.888302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750890+0000","last_change":"2024-09-15T22:06:35.136132+0000","last_active":"2024-09-15T22:08:53.750890+0000","last_peered":"2024-09-15T22:08:53.750890+0000","last_clean":"2024-09-15T22:08:53.750890+0000","last_became_active":"2024-09-15T22:06:35.135789+0000","last_became_peered":"2024-09-15T22:06:35.135789+0000","last_unstale":"2024-09-15T22:08:53.750890+0000","last_undegraded":"2024-09-15T22:08:53.750890+0000","last_fullsized":"2024-09-15T22:08:53.750890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:01.692885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490410+0000","last_change":"2024-09-15T22:06:44.658920+0000","last_active":"2024-09-15T22:08:53.490410+0000","last_peered":"2024-09-15T22:08:53.490410+0000","last_clean":"2024-09-15T22:08:53.490410+0000","last_became_active":"2024-09-15T22:06:44.658434+0000","last_became_peered":"2024-09-15T22:06:44.658434+0000","last_unstale":"2024-09-15T22:08:53.490410+0000","last_undegraded":"2024-09-15T22:08:53.490410+0000","last_fullsized":"2024-09-15T22:08:53.490410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:37.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580380+0000","last_change":"2024-09-15T22:06:37.184958+0000","last_active":"2024-09-15T22:08:53.580380+0000","last_peered":"2024-09-15T22:08:53.580380+0000","last_clean":"2024-09-15T22:08:53.580380+0000","last_became_active":"2024-09-15T22:06:37.184648+0000","last_became_peered":"2024-09-15T22:06:37.184648+0000","last_unstale":"2024-09-15T22:08:53.580380+0000","last_undegraded":"2024-09-15T22:08:53.580380+0000","last_fullsized":"2024-09-15T22:08:53.580380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:28.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'42","reported_seq":315,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:03.905592+0000","last_change":"2024-09-15T22:06:35.138533+0000","last_active":"2024-09-15T22:09:03.905592+0000","last_peered":"2024-09-15T22:09:03.905592+0000","last_clean":"2024-09-15T22:09:03.905592+0000","last_became_active":"2024-09-15T22:06:35.138142+0000","last_became_peered":"2024-09-15T22:06:35.138142+0000","last_unstale":"2024-09-15T22:09:03.905592+0000","last_undegraded":"2024-09-15T22:09:03.905592+0000","last_fullsized":"2024-09-15T22:09:03.905592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:32.950036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71705,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":236,"num_read_kb":189,"num_write":43,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589290+0000","last_change":"2024-09-15T22:07:46.218222+0000","last_active":"2024-09-15T22:08:53.589290+0000","last_peered":"2024-09-15T22:08:53.589290+0000","last_clean":"2024-09-15T22:08:53.589290+0000","last_became_active":"2024-09-15T22:07:46.217903+0000","last_became_peered":"2024-09-15T22:07:46.217903+0000","last_unstale":"2024-09-15T22:08:53.589290+0000","last_undegraded":"2024-09-15T22:08:53.589290+0000","last_fullsized":"2024-09-15T22:08:53.589290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:17.730288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750930+0000","last_change":"2024-09-15T22:06:37.194297+0000","last_active":"2024-09-15T22:08:53.750930+0000","last_peered":"2024-09-15T22:08:53.750930+0000","last_clean":"2024-09-15T22:08:53.750930+0000","last_became_active":"2024-09-15T22:06:37.193986+0000","last_became_peered":"2024-09-15T22:06:37.193986+0000","last_unstale":"2024-09-15T22:08:53.750930+0000","last_undegraded":"2024-09-15T22:08:53.750930+0000","last_fullsized":"2024-09-15T22:08:53.750930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:27.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490625+0000","last_change":"2024-09-15T22:06:35.356537+0000","last_active":"2024-09-15T22:08:53.490625+0000","last_peered":"2024-09-15T22:08:53.490625+0000","last_clean":"2024-09-15T22:08:53.490625+0000","last_became_active":"2024-09-15T22:06:35.355997+0000","last_became_peered":"2024-09-15T22:06:35.355997+0000","last_unstale":"2024-09-15T22:08:53.490625+0000","last_undegraded":"2024-09-15T22:08:53.490625+0000","last_fullsized":"2024-09-15T22:08:53.490625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:11.563083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753107+0000","last_change":"2024-09-15T22:06:44.654579+0000","last_active":"2024-09-15T22:08:53.753107+0000","last_peered":"2024-09-15T22:08:53.753107+0000","last_clean":"2024-09-15T22:08:53.753107+0000","last_became_active":"2024-09-15T22:06:44.654187+0000","last_became_peered":"2024-09-15T22:06:44.654187+0000","last_unstale":"2024-09-15T22:08:53.753107+0000","last_undegraded":"2024-09-15T22:08:53.753107+0000","last_fullsized":"2024-09-15T22:08:53.753107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:15.617528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929471+0000","last_change":"2024-09-15T22:06:44.661533+0000","last_active":"2024-09-15T22:08:53.929471+0000","last_peered":"2024-09-15T22:08:53.929471+0000","last_clean":"2024-09-15T22:08:53.929471+0000","last_became_active":"2024-09-15T22:06:44.661193+0000","last_became_peered":"2024-09-15T22:06:44.661193+0000","last_unstale":"2024-09-15T22:08:53.929471+0000","last_undegraded":"2024-09-15T22:08:53.929471+0000","last_fullsized":"2024-09-15T22:08:53.929471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:06.559720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490757+0000","last_change":"2024-09-15T22:06:35.140578+0000","last_active":"2024-09-15T22:08:53.490757+0000","last_peered":"2024-09-15T22:08:53.490757+0000","last_clean":"2024-09-15T22:08:53.490757+0000","last_became_active":"2024-09-15T22:06:35.139818+0000","last_became_peered":"2024-09-15T22:06:35.139818+0000","last_unstale":"2024-09-15T22:08:53.490757+0000","last_undegraded":"2024-09-15T22:08:53.490757+0000","last_fullsized":"2024-09-15T22:08:53.490757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:59.806964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488530+0000","last_change":"2024-09-15T22:06:44.654755+0000","last_active":"2024-09-15T22:08:53.488530+0000","last_peered":"2024-09-15T22:08:53.488530+0000","last_clean":"2024-09-15T22:08:53.488530+0000","last_became_active":"2024-09-15T22:06:44.654390+0000","last_became_peered":"2024-09-15T22:06:44.654390+0000","last_unstale":"2024-09-15T22:08:53.488530+0000","last_undegraded":"2024-09-15T22:08:53.488530+0000","last_fullsized":"2024-09-15T22:08:53.488530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:35.781878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751079+0000","last_change":"2024-09-15T22:06:37.182457+0000","last_active":"2024-09-15T22:08:53.751079+0000","last_peered":"2024-09-15T22:08:53.751079+0000","last_clean":"2024-09-15T22:08:53.751079+0000","last_became_active":"2024-09-15T22:06:37.182106+0000","last_became_peered":"2024-09-15T22:06:37.182106+0000","last_unstale":"2024-09-15T22:08:53.751079+0000","last_undegraded":"2024-09-15T22:08:53.751079+0000","last_fullsized":"2024-09-15T22:08:53.751079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:57.560894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.903955+0000","last_change":"2024-09-15T22:06:35.140590+0000","last_active":"2024-09-15T22:09:02.903955+0000","last_peered":"2024-09-15T22:09:02.903955+0000","last_clean":"2024-09-15T22:09:02.903955+0000","last_became_active":"2024-09-15T22:06:35.140198+0000","last_became_peered":"2024-09-15T22:06:35.140198+0000","last_unstale":"2024-09-15T22:09:02.903955+0000","last_undegraded":"2024-09-15T22:09:02.903955+0000","last_fullsized":"2024-09-15T22:09:02.903955+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:46.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580920+0000","last_change":"2024-09-15T22:06:44.660188+0000","last_active":"2024-09-15T22:08:53.580920+0000","last_peered":"2024-09-15T22:08:53.580920+0000","last_clean":"2024-09-15T22:08:53.580920+0000","last_became_active":"2024-09-15T22:06:44.659808+0000","last_became_peered":"2024-09-15T22:06:44.659808+0000","last_unstale":"2024-09-15T22:08:53.580920+0000","last_undegraded":"2024-09-15T22:08:53.580920+0000","last_fullsized":"2024-09-15T22:08:53.580920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:39.473765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929017+0000","last_change":"2024-09-15T22:06:44.647967+0000","last_active":"2024-09-15T22:08:53.929017+0000","last_peered":"2024-09-15T22:08:53.929017+0000","last_clean":"2024-09-15T22:08:53.929017+0000","last_became_active":"2024-09-15T22:06:44.647608+0000","last_became_peered":"2024-09-15T22:06:44.647608+0000","last_unstale":"2024-09-15T22:08:53.929017+0000","last_undegraded":"2024-09-15T22:08:53.929017+0000","last_fullsized":"2024-09-15T22:08:53.929017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:06.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488639+0000","last_change":"2024-09-15T22:06:37.187020+0000","last_active":"2024-09-15T22:08:53.488639+0000","last_peered":"2024-09-15T22:08:53.488639+0000","last_clean":"2024-09-15T22:08:53.488639+0000","last_became_active":"2024-09-15T22:06:37.186564+0000","last_became_peered":"2024-09-15T22:06:37.186564+0000","last_unstale":"2024-09-15T22:08:53.488639+0000","last_undegraded":"2024-09-15T22:08:53.488639+0000","last_fullsized":"2024-09-15T22:08:53.488639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:20.586763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490159+0000","last_change":"2024-09-15T22:06:35.135126+0000","last_active":"2024-09-15T22:08:53.490159+0000","last_peered":"2024-09-15T22:08:53.490159+0000","last_clean":"2024-09-15T22:08:53.490159+0000","last_became_active":"2024-09-15T22:06:35.134783+0000","last_became_peered":"2024-09-15T22:06:35.134783+0000","last_unstale":"2024-09-15T22:08:53.490159+0000","last_undegraded":"2024-09-15T22:08:53.490159+0000","last_fullsized":"2024-09-15T22:08:53.490159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:31.910988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928656+0000","last_change":"2024-09-15T22:06:44.656981+0000","last_active":"2024-09-15T22:08:53.928656+0000","last_peered":"2024-09-15T22:08:53.928656+0000","last_clean":"2024-09-15T22:08:53.928656+0000","last_became_active":"2024-09-15T22:06:44.656477+0000","last_became_peered":"2024-09-15T22:06:44.656477+0000","last_unstale":"2024-09-15T22:08:53.928656+0000","last_undegraded":"2024-09-15T22:08:53.928656+0000","last_fullsized":"2024-09-15T22:08:53.928656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:06.153490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580796+0000","last_change":"2024-09-15T22:06:37.183034+0000","last_active":"2024-09-15T22:08:53.580796+0000","last_peered":"2024-09-15T22:08:53.580796+0000","last_clean":"2024-09-15T22:08:53.580796+0000","last_became_active":"2024-09-15T22:06:37.182734+0000","last_became_peered":"2024-09-15T22:06:37.182734+0000","last_unstale":"2024-09-15T22:08:53.580796+0000","last_undegraded":"2024-09-15T22:08:53.580796+0000","last_fullsized":"2024-09-15T22:08:53.580796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:19.352000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888964+0000","last_change":"2024-09-15T22:06:35.134954+0000","last_active":"2024-09-15T22:08:53.888964+0000","last_peered":"2024-09-15T22:08:53.888964+0000","last_clean":"2024-09-15T22:08:53.888964+0000","last_became_active":"2024-09-15T22:06:35.134583+0000","last_became_peered":"2024-09-15T22:06:35.134583+0000","last_unstale":"2024-09-15T22:08:53.888964+0000","last_undegraded":"2024-09-15T22:08:53.888964+0000","last_fullsized":"2024-09-15T22:08:53.888964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:56.082123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589513+0000","last_change":"2024-09-15T22:06:35.139229+0000","last_active":"2024-09-15T22:08:53.589513+0000","last_peered":"2024-09-15T22:08:53.589513+0000","last_clean":"2024-09-15T22:08:53.589513+0000","last_became_active":"2024-09-15T22:06:35.138834+0000","last_became_peered":"2024-09-15T22:06:35.138834+0000","last_unstale":"2024-09-15T22:08:53.589513+0000","last_undegraded":"2024-09-15T22:08:53.589513+0000","last_fullsized":"2024-09-15T22:08:53.589513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:05.877404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490250+0000","last_change":"2024-09-15T22:06:37.192212+0000","last_active":"2024-09-15T22:08:53.490250+0000","last_peered":"2024-09-15T22:08:53.490250+0000","last_clean":"2024-09-15T22:08:53.490250+0000","last_became_active":"2024-09-15T22:06:37.191872+0000","last_became_peered":"2024-09-15T22:06:37.191872+0000","last_unstale":"2024-09-15T22:08:53.490250+0000","last_undegraded":"2024-09-15T22:08:53.490250+0000","last_fullsized":"2024-09-15T22:08:53.490250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:32.845794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750771+0000","last_change":"2024-09-15T22:07:45.968398+0000","last_active":"2024-09-15T22:08:53.750771+0000","last_peered":"2024-09-15T22:08:53.750771+0000","last_clean":"2024-09-15T22:08:53.750771+0000","last_became_active":"2024-09-15T22:07:45.967912+0000","last_became_peered":"2024-09-15T22:07:45.967912+0000","last_unstale":"2024-09-15T22:08:53.750771+0000","last_undegraded":"2024-09-15T22:08:53.750771+0000","last_fullsized":"2024-09-15T22:08:53.750771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:59.909290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'10","reported_seq":163,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.904724+0000","last_change":"2024-09-15T22:06:35.133846+0000","last_active":"2024-09-15T22:09:02.904724+0000","last_peered":"2024-09-15T22:09:02.904724+0000","last_clean":"2024-09-15T22:09:02.904724+0000","last_became_active":"2024-09-15T22:06:35.133516+0000","last_became_peered":"2024-09-15T22:06:35.133516+0000","last_unstale":"2024-09-15T22:09:02.904724+0000","last_undegraded":"2024-09-15T22:09:02.904724+0000","last_fullsized":"2024-09-15T22:09:02.904724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:29.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2250,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":7,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580169+0000","last_change":"2024-09-15T22:06:37.182904+0000","last_active":"2024-09-15T22:08:53.580169+0000","last_peered":"2024-09-15T22:08:53.580169+0000","last_clean":"2024-09-15T22:08:53.580169+0000","last_became_active":"2024-09-15T22:06:37.182563+0000","last_became_peered":"2024-09-15T22:06:37.182563+0000","last_unstale":"2024-09-15T22:08:53.580169+0000","last_undegraded":"2024-09-15T22:08:53.580169+0000","last_fullsized":"2024-09-15T22:08:53.580169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750976+0000","last_change":"2024-09-15T22:06:44.660997+0000","last_active":"2024-09-15T22:08:53.750976+0000","last_peered":"2024-09-15T22:08:53.750976+0000","last_clean":"2024-09-15T22:08:53.750976+0000","last_became_active":"2024-09-15T22:06:44.660577+0000","last_became_peered":"2024-09-15T22:06:44.660577+0000","last_unstale":"2024-09-15T22:08:53.750976+0000","last_undegraded":"2024-09-15T22:08:53.750976+0000","last_fullsized":"2024-09-15T22:08:53.750976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:56.006891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928243+0000","last_change":"2024-09-15T22:06:37.182077+0000","last_active":"2024-09-15T22:08:53.928243+0000","last_peered":"2024-09-15T22:08:53.928243+0000","last_clean":"2024-09-15T22:08:53.928243+0000","last_became_active":"2024-09-15T22:06:37.181622+0000","last_became_peered":"2024-09-15T22:06:37.181622+0000","last_unstale":"2024-09-15T22:08:53.928243+0000","last_undegraded":"2024-09-15T22:08:53.928243+0000","last_fullsized":"2024-09-15T22:08:53.928243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:07.586406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750818+0000","last_change":"2024-09-15T22:06:44.653845+0000","last_active":"2024-09-15T22:08:53.750818+0000","last_peered":"2024-09-15T22:08:53.750818+0000","last_clean":"2024-09-15T22:08:53.750818+0000","last_became_active":"2024-09-15T22:06:44.653483+0000","last_became_peered":"2024-09-15T22:06:44.653483+0000","last_unstale":"2024-09-15T22:08:53.750818+0000","last_undegraded":"2024-09-15T22:08:53.750818+0000","last_fullsized":"2024-09-15T22:08:53.750818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:08:07.822279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750649+0000","last_change":"2024-09-15T22:06:37.187476+0000","last_active":"2024-09-15T22:08:53.750649+0000","last_peered":"2024-09-15T22:08:53.750649+0000","last_clean":"2024-09-15T22:08:53.750649+0000","last_became_active":"2024-09-15T22:06:37.187047+0000","last_became_peered":"2024-09-15T22:06:37.187047+0000","last_unstale":"2024-09-15T22:08:53.750649+0000","last_undegraded":"2024-09-15T22:08:53.750649+0000","last_fullsized":"2024-09-15T22:08:53.750649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.220834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928765+0000","last_change":"2024-09-15T22:06:37.190583+0000","last_active":"2024-09-15T22:08:53.928765+0000","last_peered":"2024-09-15T22:08:53.928765+0000","last_clean":"2024-09-15T22:08:53.928765+0000","last_became_active":"2024-09-15T22:06:37.190212+0000","last_became_peered":"2024-09-15T22:06:37.190212+0000","last_unstale":"2024-09-15T22:08:53.928765+0000","last_undegraded":"2024-09-15T22:08:53.928765+0000","last_fullsized":"2024-09-15T22:08:53.928765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:48.574188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889153+0000","last_change":"2024-09-15T22:06:37.192508+0000","last_active":"2024-09-15T22:08:53.889153+0000","last_peered":"2024-09-15T22:08:53.889153+0000","last_clean":"2024-09-15T22:08:53.889153+0000","last_became_active":"2024-09-15T22:06:37.192117+0000","last_became_peered":"2024-09-15T22:06:37.192117+0000","last_unstale":"2024-09-15T22:08:53.889153+0000","last_undegraded":"2024-09-15T22:08:53.889153+0000","last_fullsized":"2024-09-15T22:08:53.889153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:41.355966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928601+0000","last_change":"2024-09-15T22:06:37.183377+0000","last_active":"2024-09-15T22:08:53.928601+0000","last_peered":"2024-09-15T22:08:53.928601+0000","last_clean":"2024-09-15T22:08:53.928601+0000","last_became_active":"2024-09-15T22:06:37.183072+0000","last_became_peered":"2024-09-15T22:06:37.183072+0000","last_unstale":"2024-09-15T22:08:53.928601+0000","last_undegraded":"2024-09-15T22:08:53.928601+0000","last_fullsized":"2024-09-15T22:08:53.928601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.318702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751128+0000","last_change":"2024-09-15T22:06:37.185348+0000","last_active":"2024-09-15T22:08:53.751128+0000","last_peered":"2024-09-15T22:08:53.751128+0000","last_clean":"2024-09-15T22:08:53.751128+0000","last_became_active":"2024-09-15T22:06:37.184934+0000","last_became_peered":"2024-09-15T22:06:37.184934+0000","last_unstale":"2024-09-15T22:08:53.751128+0000","last_undegraded":"2024-09-15T22:08:53.751128+0000","last_fullsized":"2024-09-15T22:08:53.751128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:57.564860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582987+0000","last_change":"2024-09-15T22:06:37.192152+0000","last_active":"2024-09-15T22:08:53.582987+0000","last_peered":"2024-09-15T22:08:53.582987+0000","last_clean":"2024-09-15T22:08:53.582987+0000","last_became_active":"2024-09-15T22:06:37.191836+0000","last_became_peered":"2024-09-15T22:06:37.191836+0000","last_unstale":"2024-09-15T22:08:53.582987+0000","last_undegraded":"2024-09-15T22:08:53.582987+0000","last_fullsized":"2024-09-15T22:08:53.582987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:31.781110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489966+0000","last_change":"2024-09-15T22:06:37.179996+0000","last_active":"2024-09-15T22:08:53.489966+0000","last_peered":"2024-09-15T22:08:53.489966+0000","last_clean":"2024-09-15T22:08:53.489966+0000","last_became_active":"2024-09-15T22:06:37.179646+0000","last_became_peered":"2024-09-15T22:06:37.179646+0000","last_unstale":"2024-09-15T22:08:53.489966+0000","last_undegraded":"2024-09-15T22:08:53.489966+0000","last_fullsized":"2024-09-15T22:08:53.489966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:35.240468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581098+0000","last_change":"2024-09-15T22:06:37.182992+0000","last_active":"2024-09-15T22:08:53.581098+0000","last_peered":"2024-09-15T22:08:53.581098+0000","last_clean":"2024-09-15T22:08:53.581098+0000","last_became_active":"2024-09-15T22:06:37.182676+0000","last_became_peered":"2024-09-15T22:06:37.182676+0000","last_unstale":"2024-09-15T22:08:53.581098+0000","last_undegraded":"2024-09-15T22:08:53.581098+0000","last_fullsized":"2024-09-15T22:08:53.581098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:41.524600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750731+0000","last_change":"2024-09-15T22:06:37.180107+0000","last_active":"2024-09-15T22:08:53.750731+0000","last_peered":"2024-09-15T22:08:53.750731+0000","last_clean":"2024-09-15T22:08:53.750731+0000","last_became_active":"2024-09-15T22:06:37.179785+0000","last_became_peered":"2024-09-15T22:06:37.179785+0000","last_unstale":"2024-09-15T22:08:53.750731+0000","last_undegraded":"2024-09-15T22:08:53.750731+0000","last_fullsized":"2024-09-15T22:08:53.750731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:46.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888926+0000","last_change":"2024-09-15T22:06:37.177406+0000","last_active":"2024-09-15T22:08:53.888926+0000","last_peered":"2024-09-15T22:08:53.888926+0000","last_clean":"2024-09-15T22:08:53.888926+0000","last_became_active":"2024-09-15T22:06:37.177093+0000","last_became_peered":"2024-09-15T22:06:37.177093+0000","last_unstale":"2024-09-15T22:08:53.888926+0000","last_undegraded":"2024-09-15T22:08:53.888926+0000","last_fullsized":"2024-09-15T22:08:53.888926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:34.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928467+0000","last_change":"2024-09-15T22:06:37.192314+0000","last_active":"2024-09-15T22:08:53.928467+0000","last_peered":"2024-09-15T22:08:53.928467+0000","last_clean":"2024-09-15T22:08:53.928467+0000","last_became_active":"2024-09-15T22:06:37.191958+0000","last_became_peered":"2024-09-15T22:06:37.191958+0000","last_unstale":"2024-09-15T22:08:53.928467+0000","last_undegraded":"2024-09-15T22:08:53.928467+0000","last_fullsized":"2024-09-15T22:08:53.928467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:47.763742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751178+0000","last_change":"2024-09-15T22:06:37.190212+0000","last_active":"2024-09-15T22:08:53.751178+0000","last_peered":"2024-09-15T22:08:53.751178+0000","last_clean":"2024-09-15T22:08:53.751178+0000","last_became_active":"2024-09-15T22:06:37.189881+0000","last_became_peered":"2024-09-15T22:06:37.189881+0000","last_unstale":"2024-09-15T22:08:53.751178+0000","last_undegraded":"2024-09-15T22:08:53.751178+0000","last_fullsized":"2024-09-15T22:08:53.751178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:59.943574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488994+0000","last_change":"2024-09-15T22:06:37.187967+0000","last_active":"2024-09-15T22:08:53.488994+0000","last_peered":"2024-09-15T22:08:53.488994+0000","last_clean":"2024-09-15T22:08:53.488994+0000","last_became_active":"2024-09-15T22:06:37.187596+0000","last_became_peered":"2024-09-15T22:06:37.187596+0000","last_unstale":"2024-09-15T22:08:53.488994+0000","last_undegraded":"2024-09-15T22:08:53.488994+0000","last_fullsized":"2024-09-15T22:08:53.488994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:00.042784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928782+0000","last_change":"2024-09-15T22:06:37.189434+0000","last_active":"2024-09-15T22:08:53.928782+0000","last_peered":"2024-09-15T22:08:53.928782+0000","last_clean":"2024-09-15T22:08:53.928782+0000","last_became_active":"2024-09-15T22:06:37.189053+0000","last_became_peered":"2024-09-15T22:06:37.189053+0000","last_unstale":"2024-09-15T22:08:53.928782+0000","last_undegraded":"2024-09-15T22:08:53.928782+0000","last_fullsized":"2024-09-15T22:08:53.928782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:51.640495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750544+0000","last_change":"2024-09-15T22:06:37.179229+0000","last_active":"2024-09-15T22:08:53.750544+0000","last_peered":"2024-09-15T22:08:53.750544+0000","last_clean":"2024-09-15T22:08:53.750544+0000","last_became_active":"2024-09-15T22:06:37.178828+0000","last_became_peered":"2024-09-15T22:06:37.178828+0000","last_unstale":"2024-09-15T22:08:53.750544+0000","last_undegraded":"2024-09-15T22:08:53.750544+0000","last_fullsized":"2024-09-15T22:08:53.750544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:25.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928548+0000","last_change":"2024-09-15T22:06:37.193095+0000","last_active":"2024-09-15T22:08:53.928548+0000","last_peered":"2024-09-15T22:08:53.928548+0000","last_clean":"2024-09-15T22:08:53.928548+0000","last_became_active":"2024-09-15T22:06:37.192801+0000","last_became_peered":"2024-09-15T22:06:37.192801+0000","last_unstale":"2024-09-15T22:08:53.928548+0000","last_undegraded":"2024-09-15T22:08:53.928548+0000","last_fullsized":"2024-09-15T22:08:53.928548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:12.521510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928651+0000","last_change":"2024-09-15T22:06:37.182332+0000","last_active":"2024-09-15T22:08:53.928651+0000","last_peered":"2024-09-15T22:08:53.928651+0000","last_clean":"2024-09-15T22:08:53.928651+0000","last_became_active":"2024-09-15T22:06:37.182001+0000","last_became_peered":"2024-09-15T22:06:37.182001+0000","last_unstale":"2024-09-15T22:08:53.928651+0000","last_undegraded":"2024-09-15T22:08:53.928651+0000","last_fullsized":"2024-09-15T22:08:53.928651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:52.230452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888722+0000","last_change":"2024-09-15T22:06:37.180584+0000","last_active":"2024-09-15T22:08:53.888722+0000","last_peered":"2024-09-15T22:08:53.888722+0000","last_clean":"2024-09-15T22:08:53.888722+0000","last_became_active":"2024-09-15T22:06:37.179341+0000","last_became_peered":"2024-09-15T22:06:37.179341+0000","last_unstale":"2024-09-15T22:08:53.888722+0000","last_undegraded":"2024-09-15T22:08:53.888722+0000","last_fullsized":"2024-09-15T22:08:53.888722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.610835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753487+0000","last_change":"2024-09-15T22:06:37.189500+0000","last_active":"2024-09-15T22:08:53.753487+0000","last_peered":"2024-09-15T22:08:53.753487+0000","last_clean":"2024-09-15T22:08:53.753487+0000","last_became_active":"2024-09-15T22:06:37.189137+0000","last_became_peered":"2024-09-15T22:06:37.189137+0000","last_unstale":"2024-09-15T22:08:53.753487+0000","last_undegraded":"2024-09-15T22:08:53.753487+0000","last_fullsized":"2024-09-15T22:08:53.753487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:27.199854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490749+0000","last_change":"2024-09-15T22:06:37.182618+0000","last_active":"2024-09-15T22:08:53.490749+0000","last_peered":"2024-09-15T22:08:53.490749+0000","last_clean":"2024-09-15T22:08:53.490749+0000","last_became_active":"2024-09-15T22:06:37.182257+0000","last_became_peered":"2024-09-15T22:06:37.182257+0000","last_unstale":"2024-09-15T22:08:53.490749+0000","last_undegraded":"2024-09-15T22:08:53.490749+0000","last_fullsized":"2024-09-15T22:08:53.490749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:17.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589848+0000","last_change":"2024-09-15T22:06:37.193036+0000","last_active":"2024-09-15T22:08:53.589848+0000","last_peered":"2024-09-15T22:08:53.589848+0000","last_clean":"2024-09-15T22:08:53.589848+0000","last_became_active":"2024-09-15T22:06:37.192668+0000","last_became_peered":"2024-09-15T22:06:37.192668+0000","last_unstale":"2024-09-15T22:08:53.589848+0000","last_undegraded":"2024-09-15T22:08:53.589848+0000","last_fullsized":"2024-09-15T22:08:53.589848+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:05.436045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489086+0000","last_change":"2024-09-15T22:06:44.659503+0000","last_active":"2024-09-15T22:08:53.489086+0000","last_peered":"2024-09-15T22:08:53.489086+0000","last_clean":"2024-09-15T22:08:53.489086+0000","last_became_active":"2024-09-15T22:06:44.659155+0000","last_became_peered":"2024-09-15T22:06:44.659155+0000","last_unstale":"2024-09-15T22:08:53.489086+0000","last_undegraded":"2024-09-15T22:08:53.489086+0000","last_fullsized":"2024-09-15T22:08:53.489086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:41:04.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580648+0000","last_change":"2024-09-15T22:06:37.178386+0000","last_active":"2024-09-15T22:08:53.580648+0000","last_peered":"2024-09-15T22:08:53.580648+0000","last_clean":"2024-09-15T22:08:53.580648+0000","last_became_active":"2024-09-15T22:06:37.178083+0000","last_became_peered":"2024-09-15T22:06:37.178083+0000","last_unstale":"2024-09-15T22:08:53.580648+0000","last_undegraded":"2024-09-15T22:08:53.580648+0000","last_fullsized":"2024-09-15T22:08:53.580648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:35.350698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490190+0000","last_change":"2024-09-15T22:06:37.186741+0000","last_active":"2024-09-15T22:08:53.490190+0000","last_peered":"2024-09-15T22:08:53.490190+0000","last_clean":"2024-09-15T22:08:53.490190+0000","last_became_active":"2024-09-15T22:06:37.185740+0000","last_became_peered":"2024-09-15T22:06:37.185740+0000","last_unstale":"2024-09-15T22:08:53.490190+0000","last_undegraded":"2024-09-15T22:08:53.490190+0000","last_fullsized":"2024-09-15T22:08:53.490190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.762407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589411+0000","last_change":"2024-09-15T22:06:37.189552+0000","last_active":"2024-09-15T22:08:53.589411+0000","last_peered":"2024-09-15T22:08:53.589411+0000","last_clean":"2024-09-15T22:08:53.589411+0000","last_became_active":"2024-09-15T22:06:37.189197+0000","last_became_peered":"2024-09-15T22:06:37.189197+0000","last_unstale":"2024-09-15T22:08:53.589411+0000","last_undegraded":"2024-09-15T22:08:53.589411+0000","last_fullsized":"2024-09-15T22:08:53.589411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:54.268720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580092+0000","last_change":"2024-09-15T22:06:37.186917+0000","last_active":"2024-09-15T22:08:53.580092+0000","last_peered":"2024-09-15T22:08:53.580092+0000","last_clean":"2024-09-15T22:08:53.580092+0000","last_became_active":"2024-09-15T22:06:37.186571+0000","last_became_peered":"2024-09-15T22:06:37.186571+0000","last_unstale":"2024-09-15T22:08:53.580092+0000","last_undegraded":"2024-09-15T22:08:53.580092+0000","last_fullsized":"2024-09-15T22:08:53.580092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:02.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753272+0000","last_change":"2024-09-15T22:06:37.190960+0000","last_active":"2024-09-15T22:08:53.753272+0000","last_peered":"2024-09-15T22:08:53.753272+0000","last_clean":"2024-09-15T22:08:53.753272+0000","last_became_active":"2024-09-15T22:06:37.190650+0000","last_became_peered":"2024-09-15T22:06:37.190650+0000","last_unstale":"2024-09-15T22:08:53.753272+0000","last_undegraded":"2024-09-15T22:08:53.753272+0000","last_fullsized":"2024-09-15T22:08:53.753272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:34.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928796+0000","last_change":"2024-09-15T22:06:37.186718+0000","last_active":"2024-09-15T22:08:53.928796+0000","last_peered":"2024-09-15T22:08:53.928796+0000","last_clean":"2024-09-15T22:08:53.928796+0000","last_became_active":"2024-09-15T22:06:37.186424+0000","last_became_peered":"2024-09-15T22:06:37.186424+0000","last_unstale":"2024-09-15T22:08:53.928796+0000","last_undegraded":"2024-09-15T22:08:53.928796+0000","last_fullsized":"2024-09-15T22:08:53.928796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:53.266007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929020+0000","last_change":"2024-09-15T22:06:37.185522+0000","last_active":"2024-09-15T22:08:53.929020+0000","last_peered":"2024-09-15T22:08:53.929020+0000","last_clean":"2024-09-15T22:08:53.929020+0000","last_became_active":"2024-09-15T22:06:37.185192+0000","last_became_peered":"2024-09-15T22:06:37.185192+0000","last_unstale":"2024-09-15T22:08:53.929020+0000","last_undegraded":"2024-09-15T22:08:53.929020+0000","last_fullsized":"2024-09-15T22:08:53.929020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:12.868324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928738+0000","last_change":"2024-09-15T22:06:37.189553+0000","last_active":"2024-09-15T22:08:53.928738+0000","last_peered":"2024-09-15T22:08:53.928738+0000","last_clean":"2024-09-15T22:08:53.928738+0000","last_became_active":"2024-09-15T22:06:37.189270+0000","last_became_peered":"2024-09-15T22:06:37.189270+0000","last_unstale":"2024-09-15T22:08:53.928738+0000","last_undegraded":"2024-09-15T22:08:53.928738+0000","last_fullsized":"2024-09-15T22:08:53.928738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:34.157077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799711,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1027,"num_read_kb":726,"num_write":430,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2399133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579947,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1028096,"data_stored":512755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808880,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37352,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707800,"statfs":{"total":95995035648,"available":95956787200,"internally_reserved":0,"allocated":2056192,"data_stored":1500698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005109,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1437696,"data_stored":909721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234041,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1429504,"data_stored":904963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495679,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1110213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659203,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":507585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888137,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37272,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707880,"statfs":{"total":95995035648,"available":95956869120,"internally_reserved":0,"allocated":1974272,"data_stored":1433295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":514137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247891,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36868,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708284,"statfs":{"total":95995035648,"available":95957282816,"internally_reserved":0,"allocated":1560576,"data_stored":1032595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444121,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":581596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":579782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1044480,"data_stored":508973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":72287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":74101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":402835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:09:08.800 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-15T22:09:08.801 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-15T22:09:09.059 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:09.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:08 smithi005 ceph-mon[30944]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-15T22:09:09.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:08 smithi005 ceph-mon[30944]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:09:09.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:08 smithi099 ceph-mon[34892]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-15T22:09:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:08 smithi099 ceph-mon[34892]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:09:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:08 smithi129 ceph-mon[35314]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-15T22:09:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:08 smithi129 ceph-mon[35314]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.559+0000 7f90c2cc8640 1 -- 172.21.15.5:0/4154912819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90bc10c180 msgr2=0x7f90bc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.559+0000 7f90c2cc8640 1 --2- 172.21.15.5:0/4154912819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90bc10c180 0x7f90bc10e610 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f90b4009ca0 tx=0x7f90b402a950 comp rx=0 tx=0).stop 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.559+0000 7f90c2cc8640 1 -- 172.21.15.5:0/4154912819 shutdown_connections 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.559+0000 7f90c2cc8640 1 --2- 172.21.15.5:0/4154912819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90bc10c180 0x7f90bc10e610 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.559+0000 7f90c2cc8640 1 --2- 172.21.15.5:0/4154912819 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90bc103700 0x7f90bc10bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.559+0000 7f90c2cc8640 1 --2- 172.21.15.5:0/4154912819 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc102de0 0x7f90bc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.559+0000 7f90c2cc8640 1 -- 172.21.15.5:0/4154912819 >> 172.21.15.5:0/4154912819 conn(0x7f90bc0fc7a0 msgr2=0x7f90bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.559+0000 7f90c2cc8640 1 -- 172.21.15.5:0/4154912819 shutdown_connections 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.559+0000 7f90c2cc8640 1 -- 172.21.15.5:0/4154912819 wait complete. 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c2cc8640 1 Processor -- start 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c2cc8640 1 -- start start 2024-09-15T22:09:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c2cc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90bc102de0 0x7f90bc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c2cc8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc103700 0x7f90bc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c2cc8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90bc10c180 0x7f90bc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c2cc8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc111f70 con 0x7f90bc102de0 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c2cc8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc111df0 con 0x7f90bc10c180 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c2cc8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc1120f0 con 0x7f90bc103700 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c0a64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90bc102de0 0x7f90bc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c0a64640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90bc102de0 0x7f90bc1983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46646/0 (socket says 172.21.15.5:46646) 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c0a64640 1 -- 172.21.15.5:0/2058768838 learned_addr learned my addr 172.21.15.5:0/2058768838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c1265640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90bc10c180 0x7f90bc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90b3fff640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc103700 0x7f90bc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c0a64640 1 -- 172.21.15.5:0/2058768838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc103700 msgr2=0x7f90bc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c0a64640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc103700 0x7f90bc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c0a64640 1 -- 172.21.15.5:0/2058768838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90bc10c180 msgr2=0x7f90bc19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c0a64640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90bc10c180 0x7f90bc19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.560+0000 7f90c0a64640 1 -- 172.21.15.5:0/2058768838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc19d3a0 con 0x7f90bc102de0 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.561+0000 7f90c0a64640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90bc102de0 0x7f90bc1983f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f90b800d950 tx=0x7f90b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.561+0000 7f90b1ffb640 1 -- 172.21.15.5:0/2058768838 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8017070 con 0x7f90bc102de0 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.561+0000 7f90b1ffb640 1 -- 172.21.15.5:0/2058768838 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90b8009c30 con 0x7f90bc102de0 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.561+0000 7f90c2cc8640 1 -- 172.21.15.5:0/2058768838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc19d690 con 0x7f90bc102de0 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.561+0000 7f90b1ffb640 1 -- 172.21.15.5:0/2058768838 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b800f6b0 con 0x7f90bc102de0 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.561+0000 7f90c2cc8640 1 -- 172.21.15.5:0/2058768838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc1a4f70 con 0x7f90bc102de0 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.562+0000 7f90c2cc8640 1 -- 172.21.15.5:0/2058768838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90bc1031c0 con 0x7f90bc102de0 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.563+0000 7f90b1ffb640 1 -- 172.21.15.5:0/2058768838 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f90b80109d0 con 0x7f90bc102de0 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.564+0000 7f90b1ffb640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90a40a3550 0x7f90a40a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.564+0000 7f90b1ffb640 1 -- 172.21.15.5:0/2058768838 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f90b80cff60 con 0x7f90bc102de0 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.564+0000 7f90b3fff640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90a40a3550 0x7f90a40a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.565+0000 7f90b3fff640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90a40a3550 0x7f90a40a5a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f90ac005d90 tx=0x7f90ac005d20 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.568+0000 7f90b1ffb640 1 -- 172.21.15.5:0/2058768838 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90b8009de0 con 0x7f90bc102de0 2024-09-15T22:09:09.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.740+0000 7f90c2cc8640 1 -- 172.21.15.5:0/2058768838 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90bc062630 con 0x7f90a40a3550 2024-09-15T22:09:09.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.751+0000 7f90b1ffb640 1 -- 172.21.15.5:0/2058768838 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339416 (secure 0 0 0) 0x7f90bc062630 con 0x7f90a40a3550 2024-09-15T22:09:09.756 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:09.757 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-15T22:09:09.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.754+0000 7f90c2cc8640 1 -- 172.21.15.5:0/2058768838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90a40a3550 msgr2=0x7f90a40a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:09.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.754+0000 7f90c2cc8640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90a40a3550 0x7f90a40a5a10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f90ac005d90 tx=0x7f90ac005d20 comp rx=0 tx=0).stop 2024-09-15T22:09:09.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.754+0000 7f90c2cc8640 1 -- 172.21.15.5:0/2058768838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90bc102de0 msgr2=0x7f90bc1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:09.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.754+0000 7f90c2cc8640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90bc102de0 0x7f90bc1983f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f90b800d950 tx=0x7f90b800de10 comp rx=0 tx=0).stop 2024-09-15T22:09:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.755+0000 7f90c2cc8640 1 -- 172.21.15.5:0/2058768838 shutdown_connections 2024-09-15T22:09:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.755+0000 7f90c2cc8640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90a40a3550 0x7f90a40a5a10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.755+0000 7f90c2cc8640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90bc10c180 0x7f90bc19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.755+0000 7f90c2cc8640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90bc103700 0x7f90bc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.755+0000 7f90c2cc8640 1 --2- 172.21.15.5:0/2058768838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90bc102de0 0x7f90bc1983f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.755+0000 7f90c2cc8640 1 -- 172.21.15.5:0/2058768838 >> 172.21.15.5:0/2058768838 conn(0x7f90bc0fc7a0 msgr2=0x7f90bc0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.755+0000 7f90c2cc8640 1 -- 172.21.15.5:0/2058768838 shutdown_connections 2024-09-15T22:09:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:09.755+0000 7f90c2cc8640 1 -- 172.21.15.5:0/2058768838 wait complete. 2024-09-15T22:09:10.497 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-15T22:09:09.505006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1390313,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1264,"num_read_kb":1056,"num_write":633,"num_write_kb":5308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181900,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":968,"ondisk_log_size":968,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":398600,"kb_used_data":16052,"kb_used_omap":18,"kb_used_meta":382317,"kb_avail":1124543224,"statfs":{"total":1151940427776,"available":1151532261376,"internally_reserved":0,"allocated":16437248,"data_stored":10096313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":391492923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200331,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":398,"num_read_kb":459,"num_write":148,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003213"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489282+0000","last_change":"2024-09-15T22:07:46.226073+0000","last_active":"2024-09-15T22:08:53.489282+0000","last_peered":"2024-09-15T22:08:53.489282+0000","last_clean":"2024-09-15T22:08:53.489282+0000","last_became_active":"2024-09-15T22:07:46.219250+0000","last_became_peered":"2024-09-15T22:07:46.219250+0000","last_unstale":"2024-09-15T22:08:53.489282+0000","last_undegraded":"2024-09-15T22:08:53.489282+0000","last_fullsized":"2024-09-15T22:08:53.489282+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:56.324959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,4],"acting":[9,11,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582824+0000","last_change":"2024-09-15T22:06:37.184096+0000","last_active":"2024-09-15T22:08:53.582824+0000","last_peered":"2024-09-15T22:08:53.582824+0000","last_clean":"2024-09-15T22:08:53.582824+0000","last_became_active":"2024-09-15T22:06:37.183758+0000","last_became_peered":"2024-09-15T22:06:37.183758+0000","last_unstale":"2024-09-15T22:08:53.582824+0000","last_undegraded":"2024-09-15T22:08:53.582824+0000","last_fullsized":"2024-09-15T22:08:53.582824+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:04.210551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750577+0000","last_change":"2024-09-15T22:06:44.653042+0000","last_active":"2024-09-15T22:08:53.750577+0000","last_peered":"2024-09-15T22:08:53.750577+0000","last_clean":"2024-09-15T22:08:53.750577+0000","last_became_active":"2024-09-15T22:06:44.652658+0000","last_became_peered":"2024-09-15T22:06:44.652658+0000","last_unstale":"2024-09-15T22:08:53.750577+0000","last_undegraded":"2024-09-15T22:08:53.750577+0000","last_fullsized":"2024-09-15T22:08:53.750577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:59.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583148+0000","last_change":"2024-09-15T22:06:35.355392+0000","last_active":"2024-09-15T22:08:53.583148+0000","last_peered":"2024-09-15T22:08:53.583148+0000","last_clean":"2024-09-15T22:08:53.583148+0000","last_became_active":"2024-09-15T22:06:35.355042+0000","last_became_peered":"2024-09-15T22:06:35.355042+0000","last_unstale":"2024-09-15T22:08:53.583148+0000","last_undegraded":"2024-09-15T22:08:53.583148+0000","last_fullsized":"2024-09-15T22:08:53.583148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:10:05.186937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929360+0000","last_change":"2024-09-15T22:06:37.184421+0000","last_active":"2024-09-15T22:08:53.929360+0000","last_peered":"2024-09-15T22:08:53.929360+0000","last_clean":"2024-09-15T22:08:53.929360+0000","last_became_active":"2024-09-15T22:06:37.184092+0000","last_became_peered":"2024-09-15T22:06:37.184092+0000","last_unstale":"2024-09-15T22:08:53.929360+0000","last_undegraded":"2024-09-15T22:08:53.929360+0000","last_fullsized":"2024-09-15T22:08:53.929360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:41.080981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752738+0000","last_change":"2024-09-15T22:07:45.968640+0000","last_active":"2024-09-15T22:08:53.752738+0000","last_peered":"2024-09-15T22:08:53.752738+0000","last_clean":"2024-09-15T22:08:53.752738+0000","last_became_active":"2024-09-15T22:07:45.968320+0000","last_became_peered":"2024-09-15T22:07:45.968320+0000","last_unstale":"2024-09-15T22:08:53.752738+0000","last_undegraded":"2024-09-15T22:08:53.752738+0000","last_fullsized":"2024-09-15T22:08:53.752738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:54:28.610671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580416+0000","last_change":"2024-09-15T22:06:35.134606+0000","last_active":"2024-09-15T22:08:53.580416+0000","last_peered":"2024-09-15T22:08:53.580416+0000","last_clean":"2024-09-15T22:08:53.580416+0000","last_became_active":"2024-09-15T22:06:35.134263+0000","last_became_peered":"2024-09-15T22:06:35.134263+0000","last_unstale":"2024-09-15T22:08:53.580416+0000","last_undegraded":"2024-09-15T22:08:53.580416+0000","last_fullsized":"2024-09-15T22:08:53.580416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:58:45.008935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750806+0000","last_change":"2024-09-15T22:06:37.177556+0000","last_active":"2024-09-15T22:08:53.750806+0000","last_peered":"2024-09-15T22:08:53.750806+0000","last_clean":"2024-09-15T22:08:53.750806+0000","last_became_active":"2024-09-15T22:06:37.177161+0000","last_became_peered":"2024-09-15T22:06:37.177161+0000","last_unstale":"2024-09-15T22:08:53.750806+0000","last_undegraded":"2024-09-15T22:08:53.750806+0000","last_fullsized":"2024-09-15T22:08:53.750806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:20:30.192728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753347+0000","last_change":"2024-09-15T22:06:44.659097+0000","last_active":"2024-09-15T22:08:53.753347+0000","last_peered":"2024-09-15T22:08:53.753347+0000","last_clean":"2024-09-15T22:08:53.753347+0000","last_became_active":"2024-09-15T22:06:44.658626+0000","last_became_peered":"2024-09-15T22:06:44.658626+0000","last_unstale":"2024-09-15T22:08:53.753347+0000","last_undegraded":"2024-09-15T22:08:53.753347+0000","last_fullsized":"2024-09-15T22:08:53.753347+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:25:01.308004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580430+0000","last_change":"2024-09-15T22:06:35.136326+0000","last_active":"2024-09-15T22:08:53.580430+0000","last_peered":"2024-09-15T22:08:53.580430+0000","last_clean":"2024-09-15T22:08:53.580430+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.580430+0000","last_undegraded":"2024-09-15T22:08:53.580430+0000","last_fullsized":"2024-09-15T22:08:53.580430+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:28.675832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":46,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490507+0000","last_change":"2024-09-15T22:06:37.192939+0000","last_active":"2024-09-15T22:08:53.490507+0000","last_peered":"2024-09-15T22:08:53.490507+0000","last_clean":"2024-09-15T22:08:53.490507+0000","last_became_active":"2024-09-15T22:06:37.192616+0000","last_became_peered":"2024-09-15T22:06:37.192616+0000","last_unstale":"2024-09-15T22:08:53.490507+0000","last_undegraded":"2024-09-15T22:08:53.490507+0000","last_fullsized":"2024-09-15T22:08:53.490507+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:27:03.615203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.752853+0000","last_change":"2024-09-15T22:06:44.654741+0000","last_active":"2024-09-15T22:08:53.752853+0000","last_peered":"2024-09-15T22:08:53.752853+0000","last_clean":"2024-09-15T22:08:53.752853+0000","last_became_active":"2024-09-15T22:06:44.654362+0000","last_became_peered":"2024-09-15T22:06:44.654362+0000","last_unstale":"2024-09-15T22:08:53.752853+0000","last_undegraded":"2024-09-15T22:08:53.752853+0000","last_fullsized":"2024-09-15T22:08:53.752853+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:59.269948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490328+0000","last_change":"2024-09-15T22:06:37.186385+0000","last_active":"2024-09-15T22:08:53.490328+0000","last_peered":"2024-09-15T22:08:53.490328+0000","last_clean":"2024-09-15T22:08:53.490328+0000","last_became_active":"2024-09-15T22:06:37.185588+0000","last_became_peered":"2024-09-15T22:06:37.185588+0000","last_unstale":"2024-09-15T22:08:53.490328+0000","last_undegraded":"2024-09-15T22:08:53.490328+0000","last_fullsized":"2024-09-15T22:08:53.490328+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:22.523083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580272+0000","last_change":"2024-09-15T22:06:35.137383+0000","last_active":"2024-09-15T22:08:53.580272+0000","last_peered":"2024-09-15T22:08:53.580272+0000","last_clean":"2024-09-15T22:08:53.580272+0000","last_became_active":"2024-09-15T22:06:35.137022+0000","last_became_peered":"2024-09-15T22:06:35.137022+0000","last_unstale":"2024-09-15T22:08:53.580272+0000","last_undegraded":"2024-09-15T22:08:53.580272+0000","last_fullsized":"2024-09-15T22:08:53.580272+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:21.863155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753052+0000","last_change":"2024-09-15T22:06:44.653807+0000","last_active":"2024-09-15T22:08:53.753052+0000","last_peered":"2024-09-15T22:08:53.753052+0000","last_clean":"2024-09-15T22:08:53.753052+0000","last_became_active":"2024-09-15T22:06:44.653406+0000","last_became_peered":"2024-09-15T22:06:44.653406+0000","last_unstale":"2024-09-15T22:08:53.753052+0000","last_undegraded":"2024-09-15T22:08:53.753052+0000","last_fullsized":"2024-09-15T22:08:53.753052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:12.018208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582122+0000","last_change":"2024-09-15T22:06:37.186899+0000","last_active":"2024-09-15T22:08:53.582122+0000","last_peered":"2024-09-15T22:08:53.582122+0000","last_clean":"2024-09-15T22:08:53.582122+0000","last_became_active":"2024-09-15T22:06:37.183782+0000","last_became_peered":"2024-09-15T22:06:37.183782+0000","last_unstale":"2024-09-15T22:08:53.582122+0000","last_undegraded":"2024-09-15T22:08:53.582122+0000","last_fullsized":"2024-09-15T22:08:53.582122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:39.269186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:54.735821+0000","last_change":"2024-09-15T22:06:44.652686+0000","last_active":"2024-09-15T22:08:54.735821+0000","last_peered":"2024-09-15T22:08:54.735821+0000","last_clean":"2024-09-15T22:08:54.735821+0000","last_became_active":"2024-09-15T22:06:44.652306+0000","last_became_peered":"2024-09-15T22:06:44.652306+0000","last_unstale":"2024-09-15T22:08:54.735821+0000","last_undegraded":"2024-09-15T22:08:54.735821+0000","last_fullsized":"2024-09-15T22:08:54.735821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:52.949861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,6],"acting":[11,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753085+0000","last_change":"2024-09-15T22:06:44.655424+0000","last_active":"2024-09-15T22:08:53.753085+0000","last_peered":"2024-09-15T22:08:53.753085+0000","last_clean":"2024-09-15T22:08:53.753085+0000","last_became_active":"2024-09-15T22:06:44.654959+0000","last_became_peered":"2024-09-15T22:06:44.654959+0000","last_unstale":"2024-09-15T22:08:53.753085+0000","last_undegraded":"2024-09-15T22:08:53.753085+0000","last_fullsized":"2024-09-15T22:08:53.753085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:07.249455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928379+0000","last_change":"2024-09-15T22:06:37.188424+0000","last_active":"2024-09-15T22:08:53.928379+0000","last_peered":"2024-09-15T22:08:53.928379+0000","last_clean":"2024-09-15T22:08:53.928379+0000","last_became_active":"2024-09-15T22:06:37.188082+0000","last_became_peered":"2024-09-15T22:06:37.188082+0000","last_unstale":"2024-09-15T22:08:53.928379+0000","last_undegraded":"2024-09-15T22:08:53.928379+0000","last_fullsized":"2024-09-15T22:08:53.928379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:07.159341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581279+0000","last_change":"2024-09-15T22:06:35.134237+0000","last_active":"2024-09-15T22:08:53.581279+0000","last_peered":"2024-09-15T22:08:53.581279+0000","last_clean":"2024-09-15T22:08:53.581279+0000","last_became_active":"2024-09-15T22:06:35.133885+0000","last_became_peered":"2024-09-15T22:06:35.133885+0000","last_unstale":"2024-09-15T22:08:53.581279+0000","last_undegraded":"2024-09-15T22:08:53.581279+0000","last_fullsized":"2024-09-15T22:08:53.581279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:07:35.112337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489113+0000","last_change":"2024-09-15T22:06:44.664578+0000","last_active":"2024-09-15T22:08:53.489113+0000","last_peered":"2024-09-15T22:08:53.489113+0000","last_clean":"2024-09-15T22:08:53.489113+0000","last_became_active":"2024-09-15T22:06:44.664170+0000","last_became_peered":"2024-09-15T22:06:44.664170+0000","last_unstale":"2024-09-15T22:08:53.489113+0000","last_undegraded":"2024-09-15T22:08:53.489113+0000","last_fullsized":"2024-09-15T22:08:53.489113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:10:41.443816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580024+0000","last_change":"2024-09-15T22:06:37.186107+0000","last_active":"2024-09-15T22:08:53.580024+0000","last_peered":"2024-09-15T22:08:53.580024+0000","last_clean":"2024-09-15T22:08:53.580024+0000","last_became_active":"2024-09-15T22:06:37.185757+0000","last_became_peered":"2024-09-15T22:06:37.185757+0000","last_unstale":"2024-09-15T22:08:53.580024+0000","last_undegraded":"2024-09-15T22:08:53.580024+0000","last_fullsized":"2024-09-15T22:08:53.580024+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:14.537307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750623+0000","last_change":"2024-09-15T22:06:35.356281+0000","last_active":"2024-09-15T22:08:53.750623+0000","last_peered":"2024-09-15T22:08:53.750623+0000","last_clean":"2024-09-15T22:08:53.750623+0000","last_became_active":"2024-09-15T22:06:35.355922+0000","last_became_peered":"2024-09-15T22:06:35.355922+0000","last_unstale":"2024-09-15T22:08:53.750623+0000","last_undegraded":"2024-09-15T22:08:53.750623+0000","last_fullsized":"2024-09-15T22:08:53.750623+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:06.107462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753055+0000","last_change":"2024-09-15T22:06:44.659230+0000","last_active":"2024-09-15T22:08:53.753055+0000","last_peered":"2024-09-15T22:08:53.753055+0000","last_clean":"2024-09-15T22:08:53.753055+0000","last_became_active":"2024-09-15T22:06:44.658649+0000","last_became_peered":"2024-09-15T22:06:44.658649+0000","last_unstale":"2024-09-15T22:08:53.753055+0000","last_undegraded":"2024-09-15T22:08:53.753055+0000","last_fullsized":"2024-09-15T22:08:53.753055+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:14:21.461643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":45,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929164+0000","last_change":"2024-09-15T22:06:44.649394+0000","last_active":"2024-09-15T22:08:53.929164+0000","last_peered":"2024-09-15T22:08:53.929164+0000","last_clean":"2024-09-15T22:08:53.929164+0000","last_became_active":"2024-09-15T22:06:44.649043+0000","last_became_peered":"2024-09-15T22:06:44.649043+0000","last_unstale":"2024-09-15T22:08:53.929164+0000","last_undegraded":"2024-09-15T22:08:53.929164+0000","last_fullsized":"2024-09-15T22:08:53.929164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:56:28.335869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753504+0000","last_change":"2024-09-15T22:06:35.265970+0000","last_active":"2024-09-15T22:08:53.753504+0000","last_peered":"2024-09-15T22:08:53.753504+0000","last_clean":"2024-09-15T22:08:53.753504+0000","last_became_active":"2024-09-15T22:06:35.265628+0000","last_became_peered":"2024-09-15T22:06:35.265628+0000","last_unstale":"2024-09-15T22:08:53.753504+0000","last_undegraded":"2024-09-15T22:08:53.753504+0000","last_fullsized":"2024-09-15T22:08:53.753504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:37:41.109624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750664+0000","last_change":"2024-09-15T22:06:44.649652+0000","last_active":"2024-09-15T22:08:53.750664+0000","last_peered":"2024-09-15T22:08:53.750664+0000","last_clean":"2024-09-15T22:08:53.750664+0000","last_became_active":"2024-09-15T22:06:44.649112+0000","last_became_peered":"2024-09-15T22:06:44.649112+0000","last_unstale":"2024-09-15T22:08:53.750664+0000","last_undegraded":"2024-09-15T22:08:53.750664+0000","last_fullsized":"2024-09-15T22:08:53.750664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:38:53.789320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.583103+0000","last_change":"2024-09-15T22:06:37.190701+0000","last_active":"2024-09-15T22:08:53.583103+0000","last_peered":"2024-09-15T22:08:53.583103+0000","last_clean":"2024-09-15T22:08:53.583103+0000","last_became_active":"2024-09-15T22:06:37.190430+0000","last_became_peered":"2024-09-15T22:06:37.190430+0000","last_unstale":"2024-09-15T22:08:53.583103+0000","last_undegraded":"2024-09-15T22:08:53.583103+0000","last_fullsized":"2024-09-15T22:08:53.583103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:41:14.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"89'101","reported_seq":224,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:04.325283+0000","last_change":"2024-09-15T22:06:35.133899+0000","last_active":"2024-09-15T22:09:04.325283+0000","last_peered":"2024-09-15T22:09:04.325283+0000","last_clean":"2024-09-15T22:09:04.325283+0000","last_became_active":"2024-09-15T22:06:35.133556+0000","last_became_peered":"2024-09-15T22:06:35.133556+0000","last_unstale":"2024-09-15T22:09:04.325283+0000","last_undegraded":"2024-09-15T22:09:04.325283+0000","last_fullsized":"2024-09-15T22:09:04.325283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:01.692416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":185,"num_write_kb":1289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580427+0000","last_change":"2024-09-15T22:06:44.655280+0000","last_active":"2024-09-15T22:08:53.580427+0000","last_peered":"2024-09-15T22:08:53.580427+0000","last_clean":"2024-09-15T22:08:53.580427+0000","last_became_active":"2024-09-15T22:06:44.654883+0000","last_became_peered":"2024-09-15T22:06:44.654883+0000","last_unstale":"2024-09-15T22:08:53.580427+0000","last_undegraded":"2024-09-15T22:08:53.580427+0000","last_fullsized":"2024-09-15T22:08:53.580427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:12.074035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889279+0000","last_change":"2024-09-15T22:06:37.182006+0000","last_active":"2024-09-15T22:08:53.889279+0000","last_peered":"2024-09-15T22:08:53.889279+0000","last_clean":"2024-09-15T22:08:53.889279+0000","last_became_active":"2024-09-15T22:06:37.181693+0000","last_became_peered":"2024-09-15T22:06:37.181693+0000","last_unstale":"2024-09-15T22:08:53.889279+0000","last_undegraded":"2024-09-15T22:08:53.889279+0000","last_fullsized":"2024-09-15T22:08:53.889279+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:08:23.483930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580471+0000","last_change":"2024-09-15T22:06:35.355854+0000","last_active":"2024-09-15T22:08:53.580471+0000","last_peered":"2024-09-15T22:08:53.580471+0000","last_clean":"2024-09-15T22:08:53.580471+0000","last_became_active":"2024-09-15T22:06:35.355486+0000","last_became_peered":"2024-09-15T22:06:35.355486+0000","last_unstale":"2024-09-15T22:08:53.580471+0000","last_undegraded":"2024-09-15T22:08:53.580471+0000","last_fullsized":"2024-09-15T22:08:53.580471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:36.708476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928367+0000","last_change":"2024-09-15T22:06:44.660523+0000","last_active":"2024-09-15T22:08:53.928367+0000","last_peered":"2024-09-15T22:08:53.928367+0000","last_clean":"2024-09-15T22:08:53.928367+0000","last_became_active":"2024-09-15T22:06:44.660113+0000","last_became_peered":"2024-09-15T22:06:44.660113+0000","last_unstale":"2024-09-15T22:08:53.928367+0000","last_undegraded":"2024-09-15T22:08:53.928367+0000","last_fullsized":"2024-09-15T22:08:53.928367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:42.164558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589575+0000","last_change":"2024-09-15T22:06:37.183649+0000","last_active":"2024-09-15T22:08:53.589575+0000","last_peered":"2024-09-15T22:08:53.589575+0000","last_clean":"2024-09-15T22:08:53.589575+0000","last_became_active":"2024-09-15T22:06:37.183296+0000","last_became_peered":"2024-09-15T22:06:37.183296+0000","last_unstale":"2024-09-15T22:08:53.589575+0000","last_undegraded":"2024-09-15T22:08:53.589575+0000","last_fullsized":"2024-09-15T22:08:53.589575+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:51:22.010288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488860+0000","last_change":"2024-09-15T22:06:35.355068+0000","last_active":"2024-09-15T22:08:53.488860+0000","last_peered":"2024-09-15T22:08:53.488860+0000","last_clean":"2024-09-15T22:08:53.488860+0000","last_became_active":"2024-09-15T22:06:35.354728+0000","last_became_peered":"2024-09-15T22:06:35.354728+0000","last_unstale":"2024-09-15T22:08:53.488860+0000","last_undegraded":"2024-09-15T22:08:53.488860+0000","last_fullsized":"2024-09-15T22:08:53.488860+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:24:35.479290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580098+0000","last_change":"2024-09-15T22:06:44.653849+0000","last_active":"2024-09-15T22:08:53.580098+0000","last_peered":"2024-09-15T22:08:53.580098+0000","last_clean":"2024-09-15T22:08:53.580098+0000","last_became_active":"2024-09-15T22:06:44.653419+0000","last_became_peered":"2024-09-15T22:06:44.653419+0000","last_unstale":"2024-09-15T22:08:53.580098+0000","last_undegraded":"2024-09-15T22:08:53.580098+0000","last_fullsized":"2024-09-15T22:08:53.580098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:04.131553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589446+0000","last_change":"2024-09-15T22:06:37.191438+0000","last_active":"2024-09-15T22:08:53.589446+0000","last_peered":"2024-09-15T22:08:53.589446+0000","last_clean":"2024-09-15T22:08:53.589446+0000","last_became_active":"2024-09-15T22:06:37.191088+0000","last_became_peered":"2024-09-15T22:06:37.191088+0000","last_unstale":"2024-09-15T22:08:53.589446+0000","last_undegraded":"2024-09-15T22:08:53.589446+0000","last_fullsized":"2024-09-15T22:08:53.589446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:46:36.078872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490414+0000","last_change":"2024-09-15T22:06:35.135866+0000","last_active":"2024-09-15T22:08:53.490414+0000","last_peered":"2024-09-15T22:08:53.490414+0000","last_clean":"2024-09-15T22:08:53.490414+0000","last_became_active":"2024-09-15T22:06:35.135524+0000","last_became_peered":"2024-09-15T22:06:35.135524+0000","last_unstale":"2024-09-15T22:08:53.490414+0000","last_undegraded":"2024-09-15T22:08:53.490414+0000","last_fullsized":"2024-09-15T22:08:53.490414+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.700010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888980+0000","last_change":"2024-09-15T22:06:44.649943+0000","last_active":"2024-09-15T22:08:53.888980+0000","last_peered":"2024-09-15T22:08:53.888980+0000","last_clean":"2024-09-15T22:08:53.888980+0000","last_became_active":"2024-09-15T22:06:44.649579+0000","last_became_peered":"2024-09-15T22:06:44.649579+0000","last_unstale":"2024-09-15T22:08:53.888980+0000","last_undegraded":"2024-09-15T22:08:53.888980+0000","last_fullsized":"2024-09-15T22:08:53.888980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:18.390608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489996+0000","last_change":"2024-09-15T22:06:37.189988+0000","last_active":"2024-09-15T22:08:53.489996+0000","last_peered":"2024-09-15T22:08:53.489996+0000","last_clean":"2024-09-15T22:08:53.489996+0000","last_became_active":"2024-09-15T22:06:37.189319+0000","last_became_peered":"2024-09-15T22:06:37.189319+0000","last_unstale":"2024-09-15T22:08:53.489996+0000","last_undegraded":"2024-09-15T22:08:53.489996+0000","last_fullsized":"2024-09-15T22:08:53.489996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:08.296873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":152,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.902547+0000","last_change":"2024-09-15T22:06:35.355371+0000","last_active":"2024-09-15T22:09:02.902547+0000","last_peered":"2024-09-15T22:09:02.902547+0000","last_clean":"2024-09-15T22:09:02.902547+0000","last_became_active":"2024-09-15T22:06:35.354687+0000","last_became_peered":"2024-09-15T22:06:35.354687+0000","last_unstale":"2024-09-15T22:09:02.902547+0000","last_undegraded":"2024-09-15T22:09:02.902547+0000","last_fullsized":"2024-09-15T22:09:02.902547+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:27.593758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889597+0000","last_change":"2024-09-15T22:06:44.657929+0000","last_active":"2024-09-15T22:08:53.889597+0000","last_peered":"2024-09-15T22:08:53.889597+0000","last_clean":"2024-09-15T22:08:53.889597+0000","last_became_active":"2024-09-15T22:06:44.657519+0000","last_became_peered":"2024-09-15T22:06:44.657519+0000","last_unstale":"2024-09-15T22:08:53.889597+0000","last_undegraded":"2024-09-15T22:08:53.889597+0000","last_fullsized":"2024-09-15T22:08:53.889597+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:12.313877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888747+0000","last_change":"2024-09-15T22:06:37.193288+0000","last_active":"2024-09-15T22:08:53.888747+0000","last_peered":"2024-09-15T22:08:53.888747+0000","last_clean":"2024-09-15T22:08:53.888747+0000","last_became_active":"2024-09-15T22:06:37.193007+0000","last_became_peered":"2024-09-15T22:06:37.193007+0000","last_unstale":"2024-09-15T22:08:53.888747+0000","last_undegraded":"2024-09-15T22:08:53.888747+0000","last_fullsized":"2024-09-15T22:08:53.888747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:45.956206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490201+0000","last_change":"2024-09-15T22:06:35.138726+0000","last_active":"2024-09-15T22:08:53.490201+0000","last_peered":"2024-09-15T22:08:53.490201+0000","last_clean":"2024-09-15T22:08:53.490201+0000","last_became_active":"2024-09-15T22:06:35.138390+0000","last_became_peered":"2024-09-15T22:06:35.138390+0000","last_unstale":"2024-09-15T22:08:53.490201+0000","last_undegraded":"2024-09-15T22:08:53.490201+0000","last_fullsized":"2024-09-15T22:08:53.490201+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:51.336171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490388+0000","last_change":"2024-09-15T22:07:46.219720+0000","last_active":"2024-09-15T22:08:53.490388+0000","last_peered":"2024-09-15T22:08:53.490388+0000","last_clean":"2024-09-15T22:08:53.490388+0000","last_became_active":"2024-09-15T22:07:46.219303+0000","last_became_peered":"2024-09-15T22:07:46.219303+0000","last_unstale":"2024-09-15T22:08:53.490388+0000","last_undegraded":"2024-09-15T22:08:53.490388+0000","last_fullsized":"2024-09-15T22:08:53.490388+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:42:03.596795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,11,6,10],"acting":[5,11,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581603+0000","last_change":"2024-09-15T22:06:37.176478+0000","last_active":"2024-09-15T22:08:53.581603+0000","last_peered":"2024-09-15T22:08:53.581603+0000","last_clean":"2024-09-15T22:08:53.581603+0000","last_became_active":"2024-09-15T22:06:37.176171+0000","last_became_peered":"2024-09-15T22:06:37.176171+0000","last_unstale":"2024-09-15T22:08:53.581603+0000","last_undegraded":"2024-09-15T22:08:53.581603+0000","last_fullsized":"2024-09-15T22:08:53.581603+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:58:19.426571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753268+0000","last_change":"2024-09-15T22:06:35.134651+0000","last_active":"2024-09-15T22:08:53.753268+0000","last_peered":"2024-09-15T22:08:53.753268+0000","last_clean":"2024-09-15T22:08:53.753268+0000","last_became_active":"2024-09-15T22:06:35.134185+0000","last_became_peered":"2024-09-15T22:06:35.134185+0000","last_unstale":"2024-09-15T22:08:53.753268+0000","last_undegraded":"2024-09-15T22:08:53.753268+0000","last_fullsized":"2024-09-15T22:08:53.753268+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:47:56.385046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"89'148","reported_seq":402,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:55.596394+0000","last_change":"2024-09-15T22:05:23.048068+0000","last_active":"2024-09-15T22:08:55.596394+0000","last_peered":"2024-09-15T22:08:55.596394+0000","last_clean":"2024-09-15T22:08:55.596394+0000","last_became_active":"2024-09-15T22:05:23.047692+0000","last_became_peered":"2024-09-15T22:05:23.047692+0000","last_unstale":"2024-09-15T22:08:55.596394+0000","last_undegraded":"2024-09-15T22:08:55.596394+0000","last_fullsized":"2024-09-15T22:08:55.596394+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:01:57.477629+0000","last_clean_scrub_stamp":"2024-09-15T22:01:57.477629+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:29:30.983177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490625+0000","last_change":"2024-09-15T22:06:44.657213+0000","last_active":"2024-09-15T22:08:53.490625+0000","last_peered":"2024-09-15T22:08:53.490625+0000","last_clean":"2024-09-15T22:08:53.490625+0000","last_became_active":"2024-09-15T22:06:44.656815+0000","last_became_peered":"2024-09-15T22:06:44.656815+0000","last_unstale":"2024-09-15T22:08:53.490625+0000","last_undegraded":"2024-09-15T22:08:53.490625+0000","last_fullsized":"2024-09-15T22:08:53.490625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:16:26.525992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490755+0000","last_change":"2024-09-15T22:06:37.184467+0000","last_active":"2024-09-15T22:08:53.490755+0000","last_peered":"2024-09-15T22:08:53.490755+0000","last_clean":"2024-09-15T22:08:53.490755+0000","last_became_active":"2024-09-15T22:06:37.183864+0000","last_became_peered":"2024-09-15T22:06:37.183864+0000","last_unstale":"2024-09-15T22:08:53.490755+0000","last_undegraded":"2024-09-15T22:08:53.490755+0000","last_fullsized":"2024-09-15T22:08:53.490755+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:35.747355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750805+0000","last_change":"2024-09-15T22:06:44.651594+0000","last_active":"2024-09-15T22:08:53.750805+0000","last_peered":"2024-09-15T22:08:53.750805+0000","last_clean":"2024-09-15T22:08:53.750805+0000","last_became_active":"2024-09-15T22:06:44.650393+0000","last_became_peered":"2024-09-15T22:06:44.650393+0000","last_unstale":"2024-09-15T22:08:53.750805+0000","last_undegraded":"2024-09-15T22:08:53.750805+0000","last_fullsized":"2024-09-15T22:08:53.750805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:13.599744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750608+0000","last_change":"2024-09-15T22:06:44.660789+0000","last_active":"2024-09-15T22:08:53.750608+0000","last_peered":"2024-09-15T22:08:53.750608+0000","last_clean":"2024-09-15T22:08:53.750608+0000","last_became_active":"2024-09-15T22:06:44.660331+0000","last_became_peered":"2024-09-15T22:06:44.660331+0000","last_unstale":"2024-09-15T22:08:53.750608+0000","last_undegraded":"2024-09-15T22:08:53.750608+0000","last_fullsized":"2024-09-15T22:08:53.750608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:04:45.562795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490532+0000","last_change":"2024-09-15T22:06:37.185697+0000","last_active":"2024-09-15T22:08:53.490532+0000","last_peered":"2024-09-15T22:08:53.490532+0000","last_clean":"2024-09-15T22:08:53.490532+0000","last_became_active":"2024-09-15T22:06:37.185207+0000","last_became_peered":"2024-09-15T22:06:37.185207+0000","last_unstale":"2024-09-15T22:08:53.490532+0000","last_undegraded":"2024-09-15T22:08:53.490532+0000","last_fullsized":"2024-09-15T22:08:53.490532+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:11:39.005273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'65","reported_seq":131,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:04.319716+0000","last_change":"2024-09-15T22:06:35.135176+0000","last_active":"2024-09-15T22:09:04.319716+0000","last_peered":"2024-09-15T22:09:04.319716+0000","last_clean":"2024-09-15T22:09:04.319716+0000","last_became_active":"2024-09-15T22:06:35.134649+0000","last_became_peered":"2024-09-15T22:06:35.134649+0000","last_unstale":"2024-09-15T22:09:04.319716+0000","last_undegraded":"2024-09-15T22:09:04.319716+0000","last_fullsized":"2024-09-15T22:09:04.319716+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:33.780162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":31,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928755+0000","last_change":"2024-09-15T22:06:44.656972+0000","last_active":"2024-09-15T22:08:53.928755+0000","last_peered":"2024-09-15T22:08:53.928755+0000","last_clean":"2024-09-15T22:08:53.928755+0000","last_became_active":"2024-09-15T22:06:44.656528+0000","last_became_peered":"2024-09-15T22:06:44.656528+0000","last_unstale":"2024-09-15T22:08:53.928755+0000","last_undegraded":"2024-09-15T22:08:53.928755+0000","last_fullsized":"2024-09-15T22:08:53.928755+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:09.164596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490782+0000","last_change":"2024-09-15T22:06:37.186118+0000","last_active":"2024-09-15T22:08:53.490782+0000","last_peered":"2024-09-15T22:08:53.490782+0000","last_clean":"2024-09-15T22:08:53.490782+0000","last_became_active":"2024-09-15T22:06:37.185118+0000","last_became_peered":"2024-09-15T22:06:37.185118+0000","last_unstale":"2024-09-15T22:08:53.490782+0000","last_undegraded":"2024-09-15T22:08:53.490782+0000","last_fullsized":"2024-09-15T22:08:53.490782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:51:53.357045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889478+0000","last_change":"2024-09-15T22:06:35.133341+0000","last_active":"2024-09-15T22:08:53.889478+0000","last_peered":"2024-09-15T22:08:53.889478+0000","last_clean":"2024-09-15T22:08:53.889478+0000","last_became_active":"2024-09-15T22:06:35.132961+0000","last_became_peered":"2024-09-15T22:06:35.132961+0000","last_unstale":"2024-09-15T22:08:53.889478+0000","last_undegraded":"2024-09-15T22:08:53.889478+0000","last_fullsized":"2024-09-15T22:08:53.889478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:50:39.579470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489873+0000","last_change":"2024-09-15T22:06:44.659148+0000","last_active":"2024-09-15T22:08:53.489873+0000","last_peered":"2024-09-15T22:08:53.489873+0000","last_clean":"2024-09-15T22:08:53.489873+0000","last_became_active":"2024-09-15T22:06:44.658750+0000","last_became_peered":"2024-09-15T22:06:44.658750+0000","last_unstale":"2024-09-15T22:08:53.489873+0000","last_undegraded":"2024-09-15T22:08:53.489873+0000","last_fullsized":"2024-09-15T22:08:53.489873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:59:48.985397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928647+0000","last_change":"2024-09-15T22:06:37.194066+0000","last_active":"2024-09-15T22:08:53.928647+0000","last_peered":"2024-09-15T22:08:53.928647+0000","last_clean":"2024-09-15T22:08:53.928647+0000","last_became_active":"2024-09-15T22:06:37.193738+0000","last_became_peered":"2024-09-15T22:06:37.193738+0000","last_unstale":"2024-09-15T22:08:53.928647+0000","last_undegraded":"2024-09-15T22:08:53.928647+0000","last_fullsized":"2024-09-15T22:08:53.928647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:59:56.107188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"89'3","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:54.504334+0000","last_change":"2024-09-15T22:06:35.136470+0000","last_active":"2024-09-15T22:08:54.504334+0000","last_peered":"2024-09-15T22:08:54.504334+0000","last_clean":"2024-09-15T22:08:54.504334+0000","last_became_active":"2024-09-15T22:06:35.135953+0000","last_became_peered":"2024-09-15T22:06:35.135953+0000","last_unstale":"2024-09-15T22:08:54.504334+0000","last_undegraded":"2024-09-15T22:08:54.504334+0000","last_fullsized":"2024-09-15T22:08:54.504334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:44.548566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490037+0000","last_change":"2024-09-15T22:06:44.653682+0000","last_active":"2024-09-15T22:08:53.490037+0000","last_peered":"2024-09-15T22:08:53.490037+0000","last_clean":"2024-09-15T22:08:53.490037+0000","last_became_active":"2024-09-15T22:06:44.653303+0000","last_became_peered":"2024-09-15T22:06:44.653303+0000","last_unstale":"2024-09-15T22:08:53.490037+0000","last_undegraded":"2024-09-15T22:08:53.490037+0000","last_fullsized":"2024-09-15T22:08:53.490037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:40:07.847971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490017+0000","last_change":"2024-09-15T22:06:44.658436+0000","last_active":"2024-09-15T22:08:53.490017+0000","last_peered":"2024-09-15T22:08:53.490017+0000","last_clean":"2024-09-15T22:08:53.490017+0000","last_became_active":"2024-09-15T22:06:44.658037+0000","last_became_peered":"2024-09-15T22:06:44.658037+0000","last_unstale":"2024-09-15T22:08:53.490017+0000","last_undegraded":"2024-09-15T22:08:53.490017+0000","last_fullsized":"2024-09-15T22:08:53.490017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:34:20.369249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580102+0000","last_change":"2024-09-15T22:06:35.358018+0000","last_active":"2024-09-15T22:08:53.580102+0000","last_peered":"2024-09-15T22:08:53.580102+0000","last_clean":"2024-09-15T22:08:53.580102+0000","last_became_active":"2024-09-15T22:06:35.357666+0000","last_became_peered":"2024-09-15T22:06:35.357666+0000","last_unstale":"2024-09-15T22:08:53.580102+0000","last_undegraded":"2024-09-15T22:08:53.580102+0000","last_fullsized":"2024-09-15T22:08:53.580102+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:34.089113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889340+0000","last_change":"2024-09-15T22:06:44.656807+0000","last_active":"2024-09-15T22:08:53.889340+0000","last_peered":"2024-09-15T22:08:53.889340+0000","last_clean":"2024-09-15T22:08:53.889340+0000","last_became_active":"2024-09-15T22:06:44.656409+0000","last_became_peered":"2024-09-15T22:06:44.656409+0000","last_unstale":"2024-09-15T22:08:53.889340+0000","last_undegraded":"2024-09-15T22:08:53.889340+0000","last_fullsized":"2024-09-15T22:08:53.889340+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:53.849264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490317+0000","last_change":"2024-09-15T22:06:37.180615+0000","last_active":"2024-09-15T22:08:53.490317+0000","last_peered":"2024-09-15T22:08:53.490317+0000","last_clean":"2024-09-15T22:08:53.490317+0000","last_became_active":"2024-09-15T22:06:37.180306+0000","last_became_peered":"2024-09-15T22:06:37.180306+0000","last_unstale":"2024-09-15T22:08:53.490317+0000","last_undegraded":"2024-09-15T22:08:53.490317+0000","last_fullsized":"2024-09-15T22:08:53.490317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:31.145128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'7","reported_seq":160,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.903158+0000","last_change":"2024-09-15T22:06:35.136192+0000","last_active":"2024-09-15T22:09:02.903158+0000","last_peered":"2024-09-15T22:09:02.903158+0000","last_clean":"2024-09-15T22:09:02.903158+0000","last_became_active":"2024-09-15T22:06:35.135608+0000","last_became_peered":"2024-09-15T22:06:35.135608+0000","last_unstale":"2024-09-15T22:09:02.903158+0000","last_undegraded":"2024-09-15T22:09:02.903158+0000","last_fullsized":"2024-09-15T22:09:02.903158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:57:48.472717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489151+0000","last_change":"2024-09-15T22:06:44.664763+0000","last_active":"2024-09-15T22:08:53.489151+0000","last_peered":"2024-09-15T22:08:53.489151+0000","last_clean":"2024-09-15T22:08:53.489151+0000","last_became_active":"2024-09-15T22:06:44.664410+0000","last_became_peered":"2024-09-15T22:06:44.664410+0000","last_unstale":"2024-09-15T22:08:53.489151+0000","last_undegraded":"2024-09-15T22:08:53.489151+0000","last_fullsized":"2024-09-15T22:08:53.489151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:14.364387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":53,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490348+0000","last_change":"2024-09-15T22:06:44.652747+0000","last_active":"2024-09-15T22:08:53.490348+0000","last_peered":"2024-09-15T22:08:53.490348+0000","last_clean":"2024-09-15T22:08:53.490348+0000","last_became_active":"2024-09-15T22:06:44.652368+0000","last_became_peered":"2024-09-15T22:06:44.652368+0000","last_unstale":"2024-09-15T22:08:53.490348+0000","last_undegraded":"2024-09-15T22:08:53.490348+0000","last_fullsized":"2024-09-15T22:08:53.490348+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:01:51.047549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490159+0000","last_change":"2024-09-15T22:06:35.136311+0000","last_active":"2024-09-15T22:08:53.490159+0000","last_peered":"2024-09-15T22:08:53.490159+0000","last_clean":"2024-09-15T22:08:53.490159+0000","last_became_active":"2024-09-15T22:06:35.135996+0000","last_became_peered":"2024-09-15T22:06:35.135996+0000","last_unstale":"2024-09-15T22:08:53.490159+0000","last_undegraded":"2024-09-15T22:08:53.490159+0000","last_fullsized":"2024-09-15T22:08:53.490159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:36:07.138056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928366+0000","last_change":"2024-09-15T22:06:44.653747+0000","last_active":"2024-09-15T22:08:53.928366+0000","last_peered":"2024-09-15T22:08:53.928366+0000","last_clean":"2024-09-15T22:08:53.928366+0000","last_became_active":"2024-09-15T22:06:44.653357+0000","last_became_peered":"2024-09-15T22:06:44.653357+0000","last_unstale":"2024-09-15T22:08:53.928366+0000","last_undegraded":"2024-09-15T22:08:53.928366+0000","last_fullsized":"2024-09-15T22:08:53.928366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:34.113924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888302+0000","last_change":"2024-09-15T22:06:37.178308+0000","last_active":"2024-09-15T22:08:53.888302+0000","last_peered":"2024-09-15T22:08:53.888302+0000","last_clean":"2024-09-15T22:08:53.888302+0000","last_became_active":"2024-09-15T22:06:37.177920+0000","last_became_peered":"2024-09-15T22:06:37.177920+0000","last_unstale":"2024-09-15T22:08:53.888302+0000","last_undegraded":"2024-09-15T22:08:53.888302+0000","last_fullsized":"2024-09-15T22:08:53.888302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:17:40.673743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":44,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750890+0000","last_change":"2024-09-15T22:06:35.136132+0000","last_active":"2024-09-15T22:08:53.750890+0000","last_peered":"2024-09-15T22:08:53.750890+0000","last_clean":"2024-09-15T22:08:53.750890+0000","last_became_active":"2024-09-15T22:06:35.135789+0000","last_became_peered":"2024-09-15T22:06:35.135789+0000","last_unstale":"2024-09-15T22:08:53.750890+0000","last_undegraded":"2024-09-15T22:08:53.750890+0000","last_fullsized":"2024-09-15T22:08:53.750890+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:01.692885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490410+0000","last_change":"2024-09-15T22:06:44.658920+0000","last_active":"2024-09-15T22:08:53.490410+0000","last_peered":"2024-09-15T22:08:53.490410+0000","last_clean":"2024-09-15T22:08:53.490410+0000","last_became_active":"2024-09-15T22:06:44.658434+0000","last_became_peered":"2024-09-15T22:06:44.658434+0000","last_unstale":"2024-09-15T22:08:53.490410+0000","last_undegraded":"2024-09-15T22:08:53.490410+0000","last_fullsized":"2024-09-15T22:08:53.490410+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:16:37.549807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580380+0000","last_change":"2024-09-15T22:06:37.184958+0000","last_active":"2024-09-15T22:08:53.580380+0000","last_peered":"2024-09-15T22:08:53.580380+0000","last_clean":"2024-09-15T22:08:53.580380+0000","last_became_active":"2024-09-15T22:06:37.184648+0000","last_became_peered":"2024-09-15T22:06:37.184648+0000","last_unstale":"2024-09-15T22:08:53.580380+0000","last_undegraded":"2024-09-15T22:08:53.580380+0000","last_fullsized":"2024-09-15T22:08:53.580380+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:28.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"89'42","reported_seq":315,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:03.905592+0000","last_change":"2024-09-15T22:06:35.138533+0000","last_active":"2024-09-15T22:09:03.905592+0000","last_peered":"2024-09-15T22:09:03.905592+0000","last_clean":"2024-09-15T22:09:03.905592+0000","last_became_active":"2024-09-15T22:06:35.138142+0000","last_became_peered":"2024-09-15T22:06:35.138142+0000","last_unstale":"2024-09-15T22:09:03.905592+0000","last_undegraded":"2024-09-15T22:09:03.905592+0000","last_fullsized":"2024-09-15T22:09:03.905592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:01:32.950036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71705,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":236,"num_read_kb":189,"num_write":43,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589290+0000","last_change":"2024-09-15T22:07:46.218222+0000","last_active":"2024-09-15T22:08:53.589290+0000","last_peered":"2024-09-15T22:08:53.589290+0000","last_clean":"2024-09-15T22:08:53.589290+0000","last_became_active":"2024-09-15T22:07:46.217903+0000","last_became_peered":"2024-09-15T22:07:46.217903+0000","last_unstale":"2024-09-15T22:08:53.589290+0000","last_undegraded":"2024-09-15T22:08:53.589290+0000","last_fullsized":"2024-09-15T22:08:53.589290+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:17.730288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750930+0000","last_change":"2024-09-15T22:06:37.194297+0000","last_active":"2024-09-15T22:08:53.750930+0000","last_peered":"2024-09-15T22:08:53.750930+0000","last_clean":"2024-09-15T22:08:53.750930+0000","last_became_active":"2024-09-15T22:06:37.193986+0000","last_became_peered":"2024-09-15T22:06:37.193986+0000","last_unstale":"2024-09-15T22:08:53.750930+0000","last_undegraded":"2024-09-15T22:08:53.750930+0000","last_fullsized":"2024-09-15T22:08:53.750930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:27.053685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490625+0000","last_change":"2024-09-15T22:06:35.356537+0000","last_active":"2024-09-15T22:08:53.490625+0000","last_peered":"2024-09-15T22:08:53.490625+0000","last_clean":"2024-09-15T22:08:53.490625+0000","last_became_active":"2024-09-15T22:06:35.355997+0000","last_became_peered":"2024-09-15T22:06:35.355997+0000","last_unstale":"2024-09-15T22:08:53.490625+0000","last_undegraded":"2024-09-15T22:08:53.490625+0000","last_fullsized":"2024-09-15T22:08:53.490625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:05:11.563083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753107+0000","last_change":"2024-09-15T22:06:44.654579+0000","last_active":"2024-09-15T22:08:53.753107+0000","last_peered":"2024-09-15T22:08:53.753107+0000","last_clean":"2024-09-15T22:08:53.753107+0000","last_became_active":"2024-09-15T22:06:44.654187+0000","last_became_peered":"2024-09-15T22:06:44.654187+0000","last_unstale":"2024-09-15T22:08:53.753107+0000","last_undegraded":"2024-09-15T22:08:53.753107+0000","last_fullsized":"2024-09-15T22:08:53.753107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:35:15.617528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929471+0000","last_change":"2024-09-15T22:06:44.661533+0000","last_active":"2024-09-15T22:08:53.929471+0000","last_peered":"2024-09-15T22:08:53.929471+0000","last_clean":"2024-09-15T22:08:53.929471+0000","last_became_active":"2024-09-15T22:06:44.661193+0000","last_became_peered":"2024-09-15T22:06:44.661193+0000","last_unstale":"2024-09-15T22:08:53.929471+0000","last_undegraded":"2024-09-15T22:08:53.929471+0000","last_fullsized":"2024-09-15T22:08:53.929471+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:06.559720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490757+0000","last_change":"2024-09-15T22:06:35.140578+0000","last_active":"2024-09-15T22:08:53.490757+0000","last_peered":"2024-09-15T22:08:53.490757+0000","last_clean":"2024-09-15T22:08:53.490757+0000","last_became_active":"2024-09-15T22:06:35.139818+0000","last_became_peered":"2024-09-15T22:06:35.139818+0000","last_unstale":"2024-09-15T22:08:53.490757+0000","last_undegraded":"2024-09-15T22:08:53.490757+0000","last_fullsized":"2024-09-15T22:08:53.490757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:52:59.806964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488530+0000","last_change":"2024-09-15T22:06:44.654755+0000","last_active":"2024-09-15T22:08:53.488530+0000","last_peered":"2024-09-15T22:08:53.488530+0000","last_clean":"2024-09-15T22:08:53.488530+0000","last_became_active":"2024-09-15T22:06:44.654390+0000","last_became_peered":"2024-09-15T22:06:44.654390+0000","last_unstale":"2024-09-15T22:08:53.488530+0000","last_undegraded":"2024-09-15T22:08:53.488530+0000","last_fullsized":"2024-09-15T22:08:53.488530+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:14:35.781878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751079+0000","last_change":"2024-09-15T22:06:37.182457+0000","last_active":"2024-09-15T22:08:53.751079+0000","last_peered":"2024-09-15T22:08:53.751079+0000","last_clean":"2024-09-15T22:08:53.751079+0000","last_became_active":"2024-09-15T22:06:37.182106+0000","last_became_peered":"2024-09-15T22:06:37.182106+0000","last_unstale":"2024-09-15T22:08:53.751079+0000","last_undegraded":"2024-09-15T22:08:53.751079+0000","last_fullsized":"2024-09-15T22:08:53.751079+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:57.560894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":151,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.903955+0000","last_change":"2024-09-15T22:06:35.140590+0000","last_active":"2024-09-15T22:09:02.903955+0000","last_peered":"2024-09-15T22:09:02.903955+0000","last_clean":"2024-09-15T22:09:02.903955+0000","last_became_active":"2024-09-15T22:06:35.140198+0000","last_became_peered":"2024-09-15T22:06:35.140198+0000","last_unstale":"2024-09-15T22:09:02.903955+0000","last_undegraded":"2024-09-15T22:09:02.903955+0000","last_fullsized":"2024-09-15T22:09:02.903955+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:16:46.525005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580920+0000","last_change":"2024-09-15T22:06:44.660188+0000","last_active":"2024-09-15T22:08:53.580920+0000","last_peered":"2024-09-15T22:08:53.580920+0000","last_clean":"2024-09-15T22:08:53.580920+0000","last_became_active":"2024-09-15T22:06:44.659808+0000","last_became_peered":"2024-09-15T22:06:44.659808+0000","last_unstale":"2024-09-15T22:08:53.580920+0000","last_undegraded":"2024-09-15T22:08:53.580920+0000","last_fullsized":"2024-09-15T22:08:53.580920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:19:39.473765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929017+0000","last_change":"2024-09-15T22:06:44.647967+0000","last_active":"2024-09-15T22:08:53.929017+0000","last_peered":"2024-09-15T22:08:53.929017+0000","last_clean":"2024-09-15T22:08:53.929017+0000","last_became_active":"2024-09-15T22:06:44.647608+0000","last_became_peered":"2024-09-15T22:06:44.647608+0000","last_unstale":"2024-09-15T22:08:53.929017+0000","last_undegraded":"2024-09-15T22:08:53.929017+0000","last_fullsized":"2024-09-15T22:08:53.929017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:41:06.591619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488639+0000","last_change":"2024-09-15T22:06:37.187020+0000","last_active":"2024-09-15T22:08:53.488639+0000","last_peered":"2024-09-15T22:08:53.488639+0000","last_clean":"2024-09-15T22:08:53.488639+0000","last_became_active":"2024-09-15T22:06:37.186564+0000","last_became_peered":"2024-09-15T22:06:37.186564+0000","last_unstale":"2024-09-15T22:08:53.488639+0000","last_undegraded":"2024-09-15T22:08:53.488639+0000","last_fullsized":"2024-09-15T22:08:53.488639+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:20.586763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490159+0000","last_change":"2024-09-15T22:06:35.135126+0000","last_active":"2024-09-15T22:08:53.490159+0000","last_peered":"2024-09-15T22:08:53.490159+0000","last_clean":"2024-09-15T22:08:53.490159+0000","last_became_active":"2024-09-15T22:06:35.134783+0000","last_became_peered":"2024-09-15T22:06:35.134783+0000","last_unstale":"2024-09-15T22:08:53.490159+0000","last_undegraded":"2024-09-15T22:08:53.490159+0000","last_fullsized":"2024-09-15T22:08:53.490159+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:14:31.910988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928656+0000","last_change":"2024-09-15T22:06:44.656981+0000","last_active":"2024-09-15T22:08:53.928656+0000","last_peered":"2024-09-15T22:08:53.928656+0000","last_clean":"2024-09-15T22:08:53.928656+0000","last_became_active":"2024-09-15T22:06:44.656477+0000","last_became_peered":"2024-09-15T22:06:44.656477+0000","last_unstale":"2024-09-15T22:08:53.928656+0000","last_undegraded":"2024-09-15T22:08:53.928656+0000","last_fullsized":"2024-09-15T22:08:53.928656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:06.153490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580796+0000","last_change":"2024-09-15T22:06:37.183034+0000","last_active":"2024-09-15T22:08:53.580796+0000","last_peered":"2024-09-15T22:08:53.580796+0000","last_clean":"2024-09-15T22:08:53.580796+0000","last_became_active":"2024-09-15T22:06:37.182734+0000","last_became_peered":"2024-09-15T22:06:37.182734+0000","last_unstale":"2024-09-15T22:08:53.580796+0000","last_undegraded":"2024-09-15T22:08:53.580796+0000","last_fullsized":"2024-09-15T22:08:53.580796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:19.352000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888964+0000","last_change":"2024-09-15T22:06:35.134954+0000","last_active":"2024-09-15T22:08:53.888964+0000","last_peered":"2024-09-15T22:08:53.888964+0000","last_clean":"2024-09-15T22:08:53.888964+0000","last_became_active":"2024-09-15T22:06:35.134583+0000","last_became_peered":"2024-09-15T22:06:35.134583+0000","last_unstale":"2024-09-15T22:08:53.888964+0000","last_undegraded":"2024-09-15T22:08:53.888964+0000","last_fullsized":"2024-09-15T22:08:53.888964+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:25:56.082123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589513+0000","last_change":"2024-09-15T22:06:35.139229+0000","last_active":"2024-09-15T22:08:53.589513+0000","last_peered":"2024-09-15T22:08:53.589513+0000","last_clean":"2024-09-15T22:08:53.589513+0000","last_became_active":"2024-09-15T22:06:35.138834+0000","last_became_peered":"2024-09-15T22:06:35.138834+0000","last_unstale":"2024-09-15T22:08:53.589513+0000","last_undegraded":"2024-09-15T22:08:53.589513+0000","last_fullsized":"2024-09-15T22:08:53.589513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:05.877404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490250+0000","last_change":"2024-09-15T22:06:37.192212+0000","last_active":"2024-09-15T22:08:53.490250+0000","last_peered":"2024-09-15T22:08:53.490250+0000","last_clean":"2024-09-15T22:08:53.490250+0000","last_became_active":"2024-09-15T22:06:37.191872+0000","last_became_peered":"2024-09-15T22:06:37.191872+0000","last_unstale":"2024-09-15T22:08:53.490250+0000","last_undegraded":"2024-09-15T22:08:53.490250+0000","last_fullsized":"2024-09-15T22:08:53.490250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:29:32.845794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750771+0000","last_change":"2024-09-15T22:07:45.968398+0000","last_active":"2024-09-15T22:08:53.750771+0000","last_peered":"2024-09-15T22:08:53.750771+0000","last_clean":"2024-09-15T22:08:53.750771+0000","last_became_active":"2024-09-15T22:07:45.967912+0000","last_became_peered":"2024-09-15T22:07:45.967912+0000","last_unstale":"2024-09-15T22:08:53.750771+0000","last_undegraded":"2024-09-15T22:08:53.750771+0000","last_fullsized":"2024-09-15T22:08:53.750771+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:59.909290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,8],"acting":[2,6,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'10","reported_seq":163,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:09:02.904724+0000","last_change":"2024-09-15T22:06:35.133846+0000","last_active":"2024-09-15T22:09:02.904724+0000","last_peered":"2024-09-15T22:09:02.904724+0000","last_clean":"2024-09-15T22:09:02.904724+0000","last_became_active":"2024-09-15T22:06:35.133516+0000","last_became_peered":"2024-09-15T22:06:35.133516+0000","last_unstale":"2024-09-15T22:09:02.904724+0000","last_undegraded":"2024-09-15T22:09:02.904724+0000","last_fullsized":"2024-09-15T22:09:02.904724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:34.110158+0000","last_clean_scrub_stamp":"2024-09-15T22:06:34.110158+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:29.719064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2250,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":7,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580169+0000","last_change":"2024-09-15T22:06:37.182904+0000","last_active":"2024-09-15T22:08:53.580169+0000","last_peered":"2024-09-15T22:08:53.580169+0000","last_clean":"2024-09-15T22:08:53.580169+0000","last_became_active":"2024-09-15T22:06:37.182563+0000","last_became_peered":"2024-09-15T22:06:37.182563+0000","last_unstale":"2024-09-15T22:08:53.580169+0000","last_undegraded":"2024-09-15T22:08:53.580169+0000","last_fullsized":"2024-09-15T22:08:53.580169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:18.338138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750976+0000","last_change":"2024-09-15T22:06:44.660997+0000","last_active":"2024-09-15T22:08:53.750976+0000","last_peered":"2024-09-15T22:08:53.750976+0000","last_clean":"2024-09-15T22:08:53.750976+0000","last_became_active":"2024-09-15T22:06:44.660577+0000","last_became_peered":"2024-09-15T22:06:44.660577+0000","last_unstale":"2024-09-15T22:08:53.750976+0000","last_undegraded":"2024-09-15T22:08:53.750976+0000","last_fullsized":"2024-09-15T22:08:53.750976+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:43.631630+0000","last_clean_scrub_stamp":"2024-09-15T22:06:43.631630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:56.006891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928243+0000","last_change":"2024-09-15T22:06:37.182077+0000","last_active":"2024-09-15T22:08:53.928243+0000","last_peered":"2024-09-15T22:08:53.928243+0000","last_clean":"2024-09-15T22:08:53.928243+0000","last_became_active":"2024-09-15T22:06:37.181622+0000","last_became_peered":"2024-09-15T22:06:37.181622+0000","last_unstale":"2024-09-15T22:08:53.928243+0000","last_undegraded":"2024-09-15T22:08:53.928243+0000","last_fullsized":"2024-09-15T22:08:53.928243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:15:07.586406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":50,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750818+0000","last_change":"2024-09-15T22:06:44.653845+0000","last_active":"2024-09-15T22:08:53.750818+0000","last_peered":"2024-09-15T22:08:53.750818+0000","last_clean":"2024-09-15T22:08:53.750818+0000","last_became_active":"2024-09-15T22:06:44.653483+0000","last_became_peered":"2024-09-15T22:06:44.653483+0000","last_unstale":"2024-09-15T22:08:53.750818+0000","last_undegraded":"2024-09-15T22:08:53.750818+0000","last_fullsized":"2024-09-15T22:08:53.750818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:08:07.822279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750649+0000","last_change":"2024-09-15T22:06:37.187476+0000","last_active":"2024-09-15T22:08:53.750649+0000","last_peered":"2024-09-15T22:08:53.750649+0000","last_clean":"2024-09-15T22:08:53.750649+0000","last_became_active":"2024-09-15T22:06:37.187047+0000","last_became_peered":"2024-09-15T22:06:37.187047+0000","last_unstale":"2024-09-15T22:08:53.750649+0000","last_undegraded":"2024-09-15T22:08:53.750649+0000","last_fullsized":"2024-09-15T22:08:53.750649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.220834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928765+0000","last_change":"2024-09-15T22:06:37.190583+0000","last_active":"2024-09-15T22:08:53.928765+0000","last_peered":"2024-09-15T22:08:53.928765+0000","last_clean":"2024-09-15T22:08:53.928765+0000","last_became_active":"2024-09-15T22:06:37.190212+0000","last_became_peered":"2024-09-15T22:06:37.190212+0000","last_unstale":"2024-09-15T22:08:53.928765+0000","last_undegraded":"2024-09-15T22:08:53.928765+0000","last_fullsized":"2024-09-15T22:08:53.928765+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:03:48.574188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.889153+0000","last_change":"2024-09-15T22:06:37.192508+0000","last_active":"2024-09-15T22:08:53.889153+0000","last_peered":"2024-09-15T22:08:53.889153+0000","last_clean":"2024-09-15T22:08:53.889153+0000","last_became_active":"2024-09-15T22:06:37.192117+0000","last_became_peered":"2024-09-15T22:06:37.192117+0000","last_unstale":"2024-09-15T22:08:53.889153+0000","last_undegraded":"2024-09-15T22:08:53.889153+0000","last_fullsized":"2024-09-15T22:08:53.889153+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:11:41.355966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928601+0000","last_change":"2024-09-15T22:06:37.183377+0000","last_active":"2024-09-15T22:08:53.928601+0000","last_peered":"2024-09-15T22:08:53.928601+0000","last_clean":"2024-09-15T22:08:53.928601+0000","last_became_active":"2024-09-15T22:06:37.183072+0000","last_became_peered":"2024-09-15T22:06:37.183072+0000","last_unstale":"2024-09-15T22:08:53.928601+0000","last_undegraded":"2024-09-15T22:08:53.928601+0000","last_fullsized":"2024-09-15T22:08:53.928601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:46.318702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":47,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751128+0000","last_change":"2024-09-15T22:06:37.185348+0000","last_active":"2024-09-15T22:08:53.751128+0000","last_peered":"2024-09-15T22:08:53.751128+0000","last_clean":"2024-09-15T22:08:53.751128+0000","last_became_active":"2024-09-15T22:06:37.184934+0000","last_became_peered":"2024-09-15T22:06:37.184934+0000","last_unstale":"2024-09-15T22:08:53.751128+0000","last_undegraded":"2024-09-15T22:08:53.751128+0000","last_fullsized":"2024-09-15T22:08:53.751128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:11:57.564860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.582987+0000","last_change":"2024-09-15T22:06:37.192152+0000","last_active":"2024-09-15T22:08:53.582987+0000","last_peered":"2024-09-15T22:08:53.582987+0000","last_clean":"2024-09-15T22:08:53.582987+0000","last_became_active":"2024-09-15T22:06:37.191836+0000","last_became_peered":"2024-09-15T22:06:37.191836+0000","last_unstale":"2024-09-15T22:08:53.582987+0000","last_undegraded":"2024-09-15T22:08:53.582987+0000","last_fullsized":"2024-09-15T22:08:53.582987+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:31.781110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489966+0000","last_change":"2024-09-15T22:06:37.179996+0000","last_active":"2024-09-15T22:08:53.489966+0000","last_peered":"2024-09-15T22:08:53.489966+0000","last_clean":"2024-09-15T22:08:53.489966+0000","last_became_active":"2024-09-15T22:06:37.179646+0000","last_became_peered":"2024-09-15T22:06:37.179646+0000","last_unstale":"2024-09-15T22:08:53.489966+0000","last_undegraded":"2024-09-15T22:08:53.489966+0000","last_fullsized":"2024-09-15T22:08:53.489966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:35.240468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.581098+0000","last_change":"2024-09-15T22:06:37.182992+0000","last_active":"2024-09-15T22:08:53.581098+0000","last_peered":"2024-09-15T22:08:53.581098+0000","last_clean":"2024-09-15T22:08:53.581098+0000","last_became_active":"2024-09-15T22:06:37.182676+0000","last_became_peered":"2024-09-15T22:06:37.182676+0000","last_unstale":"2024-09-15T22:08:53.581098+0000","last_undegraded":"2024-09-15T22:08:53.581098+0000","last_fullsized":"2024-09-15T22:08:53.581098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:43:41.524600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750731+0000","last_change":"2024-09-15T22:06:37.180107+0000","last_active":"2024-09-15T22:08:53.750731+0000","last_peered":"2024-09-15T22:08:53.750731+0000","last_clean":"2024-09-15T22:08:53.750731+0000","last_became_active":"2024-09-15T22:06:37.179785+0000","last_became_peered":"2024-09-15T22:06:37.179785+0000","last_unstale":"2024-09-15T22:08:53.750731+0000","last_undegraded":"2024-09-15T22:08:53.750731+0000","last_fullsized":"2024-09-15T22:08:53.750731+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:30:46.356748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":43,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888926+0000","last_change":"2024-09-15T22:06:37.177406+0000","last_active":"2024-09-15T22:08:53.888926+0000","last_peered":"2024-09-15T22:08:53.888926+0000","last_clean":"2024-09-15T22:08:53.888926+0000","last_became_active":"2024-09-15T22:06:37.177093+0000","last_became_peered":"2024-09-15T22:06:37.177093+0000","last_unstale":"2024-09-15T22:08:53.888926+0000","last_undegraded":"2024-09-15T22:08:53.888926+0000","last_fullsized":"2024-09-15T22:08:53.888926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:34.670882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928467+0000","last_change":"2024-09-15T22:06:37.192314+0000","last_active":"2024-09-15T22:08:53.928467+0000","last_peered":"2024-09-15T22:08:53.928467+0000","last_clean":"2024-09-15T22:08:53.928467+0000","last_became_active":"2024-09-15T22:06:37.191958+0000","last_became_peered":"2024-09-15T22:06:37.191958+0000","last_unstale":"2024-09-15T22:08:53.928467+0000","last_undegraded":"2024-09-15T22:08:53.928467+0000","last_fullsized":"2024-09-15T22:08:53.928467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:34:47.763742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.751178+0000","last_change":"2024-09-15T22:06:37.190212+0000","last_active":"2024-09-15T22:08:53.751178+0000","last_peered":"2024-09-15T22:08:53.751178+0000","last_clean":"2024-09-15T22:08:53.751178+0000","last_became_active":"2024-09-15T22:06:37.189881+0000","last_became_peered":"2024-09-15T22:06:37.189881+0000","last_unstale":"2024-09-15T22:08:53.751178+0000","last_undegraded":"2024-09-15T22:08:53.751178+0000","last_fullsized":"2024-09-15T22:08:53.751178+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:59.943574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.488994+0000","last_change":"2024-09-15T22:06:37.187967+0000","last_active":"2024-09-15T22:08:53.488994+0000","last_peered":"2024-09-15T22:08:53.488994+0000","last_clean":"2024-09-15T22:08:53.488994+0000","last_became_active":"2024-09-15T22:06:37.187596+0000","last_became_peered":"2024-09-15T22:06:37.187596+0000","last_unstale":"2024-09-15T22:08:53.488994+0000","last_undegraded":"2024-09-15T22:08:53.488994+0000","last_fullsized":"2024-09-15T22:08:53.488994+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:00.042784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928782+0000","last_change":"2024-09-15T22:06:37.189434+0000","last_active":"2024-09-15T22:08:53.928782+0000","last_peered":"2024-09-15T22:08:53.928782+0000","last_clean":"2024-09-15T22:08:53.928782+0000","last_became_active":"2024-09-15T22:06:37.189053+0000","last_became_peered":"2024-09-15T22:06:37.189053+0000","last_unstale":"2024-09-15T22:08:53.928782+0000","last_undegraded":"2024-09-15T22:08:53.928782+0000","last_fullsized":"2024-09-15T22:08:53.928782+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:50:51.640495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.750544+0000","last_change":"2024-09-15T22:06:37.179229+0000","last_active":"2024-09-15T22:08:53.750544+0000","last_peered":"2024-09-15T22:08:53.750544+0000","last_clean":"2024-09-15T22:08:53.750544+0000","last_became_active":"2024-09-15T22:06:37.178828+0000","last_became_peered":"2024-09-15T22:06:37.178828+0000","last_unstale":"2024-09-15T22:08:53.750544+0000","last_undegraded":"2024-09-15T22:08:53.750544+0000","last_fullsized":"2024-09-15T22:08:53.750544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:17:25.779122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928548+0000","last_change":"2024-09-15T22:06:37.193095+0000","last_active":"2024-09-15T22:08:53.928548+0000","last_peered":"2024-09-15T22:08:53.928548+0000","last_clean":"2024-09-15T22:08:53.928548+0000","last_became_active":"2024-09-15T22:06:37.192801+0000","last_became_peered":"2024-09-15T22:06:37.192801+0000","last_unstale":"2024-09-15T22:08:53.928548+0000","last_undegraded":"2024-09-15T22:08:53.928548+0000","last_fullsized":"2024-09-15T22:08:53.928548+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:30:12.521510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928651+0000","last_change":"2024-09-15T22:06:37.182332+0000","last_active":"2024-09-15T22:08:53.928651+0000","last_peered":"2024-09-15T22:08:53.928651+0000","last_clean":"2024-09-15T22:08:53.928651+0000","last_became_active":"2024-09-15T22:06:37.182001+0000","last_became_peered":"2024-09-15T22:06:37.182001+0000","last_unstale":"2024-09-15T22:08:53.928651+0000","last_undegraded":"2024-09-15T22:08:53.928651+0000","last_fullsized":"2024-09-15T22:08:53.928651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:52.230452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":42,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.888722+0000","last_change":"2024-09-15T22:06:37.180584+0000","last_active":"2024-09-15T22:08:53.888722+0000","last_peered":"2024-09-15T22:08:53.888722+0000","last_clean":"2024-09-15T22:08:53.888722+0000","last_became_active":"2024-09-15T22:06:37.179341+0000","last_became_peered":"2024-09-15T22:06:37.179341+0000","last_unstale":"2024-09-15T22:08:53.888722+0000","last_undegraded":"2024-09-15T22:08:53.888722+0000","last_fullsized":"2024-09-15T22:08:53.888722+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.610835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753487+0000","last_change":"2024-09-15T22:06:37.189500+0000","last_active":"2024-09-15T22:08:53.753487+0000","last_peered":"2024-09-15T22:08:53.753487+0000","last_clean":"2024-09-15T22:08:53.753487+0000","last_became_active":"2024-09-15T22:06:37.189137+0000","last_became_peered":"2024-09-15T22:06:37.189137+0000","last_unstale":"2024-09-15T22:08:53.753487+0000","last_undegraded":"2024-09-15T22:08:53.753487+0000","last_fullsized":"2024-09-15T22:08:53.753487+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:27.199854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490749+0000","last_change":"2024-09-15T22:06:37.182618+0000","last_active":"2024-09-15T22:08:53.490749+0000","last_peered":"2024-09-15T22:08:53.490749+0000","last_clean":"2024-09-15T22:08:53.490749+0000","last_became_active":"2024-09-15T22:06:37.182257+0000","last_became_peered":"2024-09-15T22:06:37.182257+0000","last_unstale":"2024-09-15T22:08:53.490749+0000","last_undegraded":"2024-09-15T22:08:53.490749+0000","last_fullsized":"2024-09-15T22:08:53.490749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:17.154601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":48,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589848+0000","last_change":"2024-09-15T22:06:37.193036+0000","last_active":"2024-09-15T22:08:53.589848+0000","last_peered":"2024-09-15T22:08:53.589848+0000","last_clean":"2024-09-15T22:08:53.589848+0000","last_became_active":"2024-09-15T22:06:37.192668+0000","last_became_peered":"2024-09-15T22:06:37.192668+0000","last_unstale":"2024-09-15T22:08:53.589848+0000","last_undegraded":"2024-09-15T22:08:53.589848+0000","last_fullsized":"2024-09-15T22:08:53.589848+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:05.436045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.489086+0000","last_change":"2024-09-15T22:06:44.659503+0000","last_active":"2024-09-15T22:08:53.489086+0000","last_peered":"2024-09-15T22:08:53.489086+0000","last_clean":"2024-09-15T22:08:53.489086+0000","last_became_active":"2024-09-15T22:06:44.659155+0000","last_became_peered":"2024-09-15T22:06:44.659155+0000","last_unstale":"2024-09-15T22:08:53.489086+0000","last_undegraded":"2024-09-15T22:08:53.489086+0000","last_fullsized":"2024-09-15T22:08:53.489086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:41:04.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,11],"acting":[9,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580648+0000","last_change":"2024-09-15T22:06:37.178386+0000","last_active":"2024-09-15T22:08:53.580648+0000","last_peered":"2024-09-15T22:08:53.580648+0000","last_clean":"2024-09-15T22:08:53.580648+0000","last_became_active":"2024-09-15T22:06:37.178083+0000","last_became_peered":"2024-09-15T22:06:37.178083+0000","last_unstale":"2024-09-15T22:08:53.580648+0000","last_undegraded":"2024-09-15T22:08:53.580648+0000","last_fullsized":"2024-09-15T22:08:53.580648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:35.350698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.490190+0000","last_change":"2024-09-15T22:06:37.186741+0000","last_active":"2024-09-15T22:08:53.490190+0000","last_peered":"2024-09-15T22:08:53.490190+0000","last_clean":"2024-09-15T22:08:53.490190+0000","last_became_active":"2024-09-15T22:06:37.185740+0000","last_became_peered":"2024-09-15T22:06:37.185740+0000","last_unstale":"2024-09-15T22:08:53.490190+0000","last_undegraded":"2024-09-15T22:08:53.490190+0000","last_fullsized":"2024-09-15T22:08:53.490190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:12.762407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":37,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.589411+0000","last_change":"2024-09-15T22:06:37.189552+0000","last_active":"2024-09-15T22:08:53.589411+0000","last_peered":"2024-09-15T22:08:53.589411+0000","last_clean":"2024-09-15T22:08:53.589411+0000","last_became_active":"2024-09-15T22:06:37.189197+0000","last_became_peered":"2024-09-15T22:06:37.189197+0000","last_unstale":"2024-09-15T22:08:53.589411+0000","last_undegraded":"2024-09-15T22:08:53.589411+0000","last_fullsized":"2024-09-15T22:08:53.589411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:29:54.268720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.580092+0000","last_change":"2024-09-15T22:06:37.186917+0000","last_active":"2024-09-15T22:08:53.580092+0000","last_peered":"2024-09-15T22:08:53.580092+0000","last_clean":"2024-09-15T22:08:53.580092+0000","last_became_active":"2024-09-15T22:06:37.186571+0000","last_became_peered":"2024-09-15T22:06:37.186571+0000","last_unstale":"2024-09-15T22:08:53.580092+0000","last_undegraded":"2024-09-15T22:08:53.580092+0000","last_fullsized":"2024-09-15T22:08:53.580092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:02.683596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.753272+0000","last_change":"2024-09-15T22:06:37.190960+0000","last_active":"2024-09-15T22:08:53.753272+0000","last_peered":"2024-09-15T22:08:53.753272+0000","last_clean":"2024-09-15T22:08:53.753272+0000","last_became_active":"2024-09-15T22:06:37.190650+0000","last_became_peered":"2024-09-15T22:06:37.190650+0000","last_unstale":"2024-09-15T22:08:53.753272+0000","last_undegraded":"2024-09-15T22:08:53.753272+0000","last_fullsized":"2024-09-15T22:08:53.753272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:26:34.047778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":39,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928796+0000","last_change":"2024-09-15T22:06:37.186718+0000","last_active":"2024-09-15T22:08:53.928796+0000","last_peered":"2024-09-15T22:08:53.928796+0000","last_clean":"2024-09-15T22:08:53.928796+0000","last_became_active":"2024-09-15T22:06:37.186424+0000","last_became_peered":"2024-09-15T22:06:37.186424+0000","last_unstale":"2024-09-15T22:08:53.928796+0000","last_undegraded":"2024-09-15T22:08:53.928796+0000","last_fullsized":"2024-09-15T22:08:53.928796+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:38:53.266007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":36,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.929020+0000","last_change":"2024-09-15T22:06:37.185522+0000","last_active":"2024-09-15T22:08:53.929020+0000","last_peered":"2024-09-15T22:08:53.929020+0000","last_clean":"2024-09-15T22:08:53.929020+0000","last_became_active":"2024-09-15T22:06:37.185192+0000","last_became_peered":"2024-09-15T22:06:37.185192+0000","last_unstale":"2024-09-15T22:08:53.929020+0000","last_undegraded":"2024-09-15T22:08:53.929020+0000","last_fullsized":"2024-09-15T22:08:53.929020+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:12.868324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-09-15T22:08:53.928738+0000","last_change":"2024-09-15T22:06:37.189553+0000","last_active":"2024-09-15T22:08:53.928738+0000","last_peered":"2024-09-15T22:08:53.928738+0000","last_clean":"2024-09-15T22:08:53.928738+0000","last_became_active":"2024-09-15T22:06:37.189270+0000","last_became_peered":"2024-09-15T22:06:37.189270+0000","last_unstale":"2024-09-15T22:08:53.928738+0000","last_undegraded":"2024-09-15T22:08:53.928738+0000","last_fullsized":"2024-09-15T22:08:53.928738+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T22:06:36.161693+0000","last_clean_scrub_stamp":"2024-09-15T22:06:36.161693+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:34.157077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":799711,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1027,"num_read_kb":726,"num_write":430,"num_write_kb":2368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2399133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":318,"ondisk_log_size":318,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579947,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1028096,"data_stored":512755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808880,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37352,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707800,"statfs":{"total":95995035648,"available":95956787200,"internally_reserved":0,"allocated":2056192,"data_stored":1500698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005109,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36748,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708404,"statfs":{"total":95995035648,"available":95957405696,"internally_reserved":0,"allocated":1437696,"data_stored":909721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234041,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28424,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716728,"statfs":{"total":95995035648,"available":95965929472,"internally_reserved":0,"allocated":1429504,"data_stored":904963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495679,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28616,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716536,"statfs":{"total":95995035648,"available":95965732864,"internally_reserved":0,"allocated":1630208,"data_stored":1110213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659203,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1019904,"data_stored":507585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888137,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37272,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707880,"statfs":{"total":95995035648,"available":95956869120,"internally_reserved":0,"allocated":1974272,"data_stored":1433295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018958,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1036288,"data_stored":514137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247891,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36868,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708284,"statfs":{"total":95995035648,"available":95957282816,"internally_reserved":0,"allocated":1560576,"data_stored":1032595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444121,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":581596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1101824,"data_stored":579782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1044480,"data_stored":508973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":72287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":74101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":4158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":402835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T22:09:10.558 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:11.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:10 smithi005 ceph-mon[30944]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-15T22:09:11.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:10 smithi005 ceph-mon[30944]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:09:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.267+0000 7fbeb778e640 1 -- 172.21.15.5:0/253826012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb0102570 msgr2=0x7fbeb01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.267+0000 7fbeb778e640 1 --2- 172.21.15.5:0/253826012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb0102570 0x7fbeb01029b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009900 tx=0x7fbea402a480 comp rx=0 tx=0).stop 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.267+0000 7fbeb778e640 1 -- 172.21.15.5:0/253826012 shutdown_connections 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.267+0000 7fbeb778e640 1 --2- 172.21.15.5:0/253826012 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeb0102ef0 0x7fbeb010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.267+0000 7fbeb778e640 1 --2- 172.21.15.5:0/253826012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb0102570 0x7fbeb01029b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.267+0000 7fbeb778e640 1 --2- 172.21.15.5:0/253826012 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb0100e20 0x7fbeb0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.267+0000 7fbeb778e640 1 -- 172.21.15.5:0/253826012 >> 172.21.15.5:0/253826012 conn(0x7fbeb00fc7d0 msgr2=0x7fbeb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.267+0000 7fbeb778e640 1 -- 172.21.15.5:0/253826012 shutdown_connections 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.268+0000 7fbeb778e640 1 -- 172.21.15.5:0/253826012 wait complete. 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.268+0000 7fbeb778e640 1 Processor -- start 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.268+0000 7fbeb778e640 1 -- start start 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.268+0000 7fbeb778e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb0100e20 0x7fbeb0197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.268+0000 7fbeb778e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeb0102570 0x7fbeb0197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.268+0000 7fbeb778e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb0102ef0 0x7fbeb019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.268+0000 7fbeb778e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0110f80 con 0x7fbeb0100e20 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.268+0000 7fbeb778e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0110e00 con 0x7fbeb0102570 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.268+0000 7fbeb778e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0111100 con 0x7fbeb0102ef0 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.268+0000 7fbeb552a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb0100e20 0x7fbeb0197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.268+0000 7fbeb552a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb0100e20 0x7fbeb0197400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46662/0 (socket says 172.21.15.5:46662) 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb552a640 1 -- 172.21.15.5:0/570931 learned_addr learned my addr 172.21.15.5:0/570931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb4d29640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeb0102570 0x7fbeb0197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb5d2b640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb0102ef0 0x7fbeb019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb552a640 1 -- 172.21.15.5:0/570931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb0102ef0 msgr2=0x7fbeb019bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb552a640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb0102ef0 0x7fbeb019bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb552a640 1 -- 172.21.15.5:0/570931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeb0102570 msgr2=0x7fbeb0197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb552a640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeb0102570 0x7fbeb0197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb552a640 1 -- 172.21.15.5:0/570931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb019c3b0 con 0x7fbeb0100e20 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb4d29640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeb0102570 0x7fbeb0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb552a640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb0100e20 0x7fbeb0197400 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00e9f0 tx=0x7fbeac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbea27fc640 1 -- 172.21.15.5:0/570931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac00cd80 con 0x7fbeb0100e20 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbea27fc640 1 -- 172.21.15.5:0/570931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbeac005500 con 0x7fbeb0100e20 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb778e640 1 -- 172.21.15.5:0/570931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb019c6a0 con 0x7fbeb0100e20 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbea27fc640 1 -- 172.21.15.5:0/570931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeac010690 con 0x7fbeb0100e20 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.269+0000 7fbeb778e640 1 -- 172.21.15.5:0/570931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb01a3f80 con 0x7fbeb0100e20 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.270+0000 7fbeb778e640 1 -- 172.21.15.5:0/570931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb0101200 con 0x7fbeb0100e20 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.272+0000 7fbea27fc640 1 -- 172.21.15.5:0/570931 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7fbeac0046c0 con 0x7fbeb0100e20 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.272+0000 7fbea27fc640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe9c0a3550 0x7fbe9c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.272+0000 7fbeb4d29640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe9c0a3550 0x7fbe9c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.272+0000 7fbea27fc640 1 -- 172.21.15.5:0/570931 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fbeac0d0840 con 0x7fbeb0100e20 2024-09-15T22:09:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.273+0000 7fbeb4d29640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe9c0a3550 0x7fbe9c0a5a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00ffeb0 tx=0x7fbea4005a20 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.276+0000 7fbea27fc640 1 -- 172.21.15.5:0/570931 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbeac09cc30 con 0x7fbeb0100e20 2024-09-15T22:09:11.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:10 smithi099 ceph-mon[34892]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-15T22:09:11.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:10 smithi099 ceph-mon[34892]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:09:11.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:10 smithi129 ceph-mon[35314]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-15T22:09:11.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:10 smithi129 ceph-mon[35314]: from='client.15453 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-15T22:09:11.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.487+0000 7fbeb778e640 1 -- 172.21.15.5:0/570931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbeb0109090 con 0x7fbeb0100e20 2024-09-15T22:09:11.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.488+0000 7fbea27fc640 1 -- 172.21.15.5:0/570931 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fbeac09c530 con 0x7fbeb0100e20 2024-09-15T22:09:11.490 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:11.490 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:09:11.490 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:09:11.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 -- 172.21.15.5:0/570931 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe9c0a3550 msgr2=0x7fbe9c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe9c0a3550 0x7fbe9c0a5a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbeb00ffeb0 tx=0x7fbea4005a20 comp rx=0 tx=0).stop 2024-09-15T22:09:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 -- 172.21.15.5:0/570931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb0100e20 msgr2=0x7fbeb0197400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb0100e20 0x7fbeb0197400 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbeac00e9f0 tx=0x7fbeac00edf0 comp rx=0 tx=0).stop 2024-09-15T22:09:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 -- 172.21.15.5:0/570931 shutdown_connections 2024-09-15T22:09:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe9c0a3550 0x7fbe9c0a5a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbeb0102ef0 0x7fbeb019bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbeb0102570 0x7fbeb0197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 --2- 172.21.15.5:0/570931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbeb0100e20 0x7fbeb0197400 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 -- 172.21.15.5:0/570931 >> 172.21.15.5:0/570931 conn(0x7fbeb00fc7d0 msgr2=0x7fbeb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 -- 172.21.15.5:0/570931 shutdown_connections 2024-09-15T22:09:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:11.491+0000 7fbeb778e640 1 -- 172.21.15.5:0/570931 wait complete. 2024-09-15T22:09:12.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:11 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/570931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:12.286 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:12.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:11 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/570931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:12.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:11 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/570931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:12.953 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:12 smithi005 ceph-mon[30944]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-15T22:09:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.035+0000 7fdd8a27f640 1 -- 172.21.15.5:0/996376842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd84102d40 msgr2=0x7fdd84103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.035+0000 7fdd8a27f640 1 --2- 172.21.15.5:0/996376842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd84102d40 0x7fdd84103120 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdd74009900 tx=0x7fdd7402a480 comp rx=0 tx=0).stop 2024-09-15T22:09:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.035+0000 7fdd8a27f640 1 -- 172.21.15.5:0/996376842 shutdown_connections 2024-09-15T22:09:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.035+0000 7fdd8a27f640 1 --2- 172.21.15.5:0/996376842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd8410c170 0x7fdd8410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.035+0000 7fdd8a27f640 1 --2- 172.21.15.5:0/996376842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd84103660 0x7fdd8410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.035+0000 7fdd8a27f640 1 --2- 172.21.15.5:0/996376842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd84102d40 0x7fdd84103120 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.035+0000 7fdd8a27f640 1 -- 172.21.15.5:0/996376842 >> 172.21.15.5:0/996376842 conn(0x7fdd840fc7a0 msgr2=0x7fdd840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.035+0000 7fdd8a27f640 1 -- 172.21.15.5:0/996376842 shutdown_connections 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.035+0000 7fdd8a27f640 1 -- 172.21.15.5:0/996376842 wait complete. 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.035+0000 7fdd8a27f640 1 Processor -- start 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.035+0000 7fdd8a27f640 1 -- start start 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd8a27f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd84102d40 0x7fdd841984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd8a27f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd84103660 0x7fdd84198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd8a27f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd8410c170 0x7fdd8419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd8a27f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd84111fd0 con 0x7fdd84102d40 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd8a27f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdd84111e50 con 0x7fdd84103660 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd8a27f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdd84112150 con 0x7fdd8410c170 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd84102d40 0x7fdd841984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd83fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd84102d40 0x7fdd841984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46688/0 (socket says 172.21.15.5:46688) 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd83fff640 1 -- 172.21.15.5:0/4233709022 learned_addr learned my addr 172.21.15.5:0/4233709022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd7bfff640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd84103660 0x7fdd84198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd8881c640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd8410c170 0x7fdd8419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd83fff640 1 -- 172.21.15.5:0/4233709022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd8410c170 msgr2=0x7fdd8419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd83fff640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd8410c170 0x7fdd8419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd83fff640 1 -- 172.21.15.5:0/4233709022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd84103660 msgr2=0x7fdd84198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd83fff640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd84103660 0x7fdd84198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd83fff640 1 -- 172.21.15.5:0/4233709022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8419d480 con 0x7fdd84102d40 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd8881c640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd8410c170 0x7fdd8419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.036+0000 7fdd83fff640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd84102d40 0x7fdd841984d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdd74006010 tx=0x7fdd740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.037+0000 7fdd81ffb640 1 -- 172.21.15.5:0/4233709022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7403f030 con 0x7fdd84102d40 2024-09-15T22:09:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.037+0000 7fdd8a27f640 1 -- 172.21.15.5:0/4233709022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8419d710 con 0x7fdd84102d40 2024-09-15T22:09:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.037+0000 7fdd81ffb640 1 -- 172.21.15.5:0/4233709022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd74003ce0 con 0x7fdd84102d40 2024-09-15T22:09:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.037+0000 7fdd81ffb640 1 -- 172.21.15.5:0/4233709022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74035070 con 0x7fdd84102d40 2024-09-15T22:09:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.037+0000 7fdd8a27f640 1 -- 172.21.15.5:0/4233709022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd841a5050 con 0x7fdd84102d40 2024-09-15T22:09:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.038+0000 7fdd8a27f640 1 -- 172.21.15.5:0/4233709022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd84103120 con 0x7fdd84102d40 2024-09-15T22:09:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.039+0000 7fdd81ffb640 1 -- 172.21.15.5:0/4233709022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7fdd74042030 con 0x7fdd84102d40 2024-09-15T22:09:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.039+0000 7fdd81ffb640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdd700a34d0 0x7fdd700a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.040+0000 7fdd7bfff640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdd700a34d0 0x7fdd700a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.040+0000 7fdd81ffb640 1 -- 172.21.15.5:0/4233709022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fdd740f0c60 con 0x7fdd84102d40 2024-09-15T22:09:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.040+0000 7fdd7bfff640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdd700a34d0 0x7fdd700a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd8410d5b0 tx=0x7fdd6c009060 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.043+0000 7fdd81ffb640 1 -- 172.21.15.5:0/4233709022 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd740bd180 con 0x7fdd84102d40 2024-09-15T22:09:13.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.254+0000 7fdd8a27f640 1 -- 172.21.15.5:0/4233709022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd84107210 con 0x7fdd84102d40 2024-09-15T22:09:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.255+0000 7fdd81ffb640 1 -- 172.21.15.5:0/4233709022 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fdd740bca80 con 0x7fdd84102d40 2024-09-15T22:09:13.256 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:13.256 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:09:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:09:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 -- 172.21.15.5:0/4233709022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdd700a34d0 msgr2=0x7fdd700a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdd700a34d0 0x7fdd700a5990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd8410d5b0 tx=0x7fdd6c009060 comp rx=0 tx=0).stop 2024-09-15T22:09:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 -- 172.21.15.5:0/4233709022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd84102d40 msgr2=0x7fdd841984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd84102d40 0x7fdd841984d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdd74006010 tx=0x7fdd740042f0 comp rx=0 tx=0).stop 2024-09-15T22:09:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 -- 172.21.15.5:0/4233709022 shutdown_connections 2024-09-15T22:09:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdd700a34d0 0x7fdd700a5990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdd8410c170 0x7fdd8419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdd84103660 0x7fdd84198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 --2- 172.21.15.5:0/4233709022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd84102d40 0x7fdd841984d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 -- 172.21.15.5:0/4233709022 >> 172.21.15.5:0/4233709022 conn(0x7fdd840fc7a0 msgr2=0x7fdd840fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 -- 172.21.15.5:0/4233709022 shutdown_connections 2024-09-15T22:09:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:13.257+0000 7fdd8a27f640 1 -- 172.21.15.5:0/4233709022 wait complete. 2024-09-15T22:09:13.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:12 smithi099 ceph-mon[34892]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-15T22:09:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:12 smithi129 ceph-mon[35314]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-09-15T22:09:13.970 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:09:14.228 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:14.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:13 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4233709022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:14.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:13 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4233709022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:14.350 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:13 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4233709022' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.725+0000 7f31073e7640 1 -- 172.21.15.5:0/587169770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100103ff0 msgr2=0x7f31001043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.725+0000 7f31073e7640 1 --2- 172.21.15.5:0/587169770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100103ff0 0x7f31001043d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009900 tx=0x7f30fc02a410 comp rx=0 tx=0).stop 2024-09-15T22:09:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 -- 172.21.15.5:0/587169770 shutdown_connections 2024-09-15T22:09:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 --2- 172.21.15.5:0/587169770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3100109120 0x7f3100110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 --2- 172.21.15.5:0/587169770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31001049a0 0x7f31001089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 --2- 172.21.15.5:0/587169770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100103ff0 0x7f31001043d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:14.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 -- 172.21.15.5:0/587169770 >> 172.21.15.5:0/587169770 conn(0x7f31000ffab0 msgr2=0x7f3100101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 -- 172.21.15.5:0/587169770 shutdown_connections 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 -- 172.21.15.5:0/587169770 wait complete. 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 Processor -- start 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 -- start start 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3100103ff0 0x7f310019a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31001049a0 0x7f310019aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.726+0000 7f31073e7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3100109120 0x7f310019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f31073e7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31001141a0 con 0x7f31001049a0 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f31073e7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3100114020 con 0x7f3100103ff0 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f31073e7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3100114320 con 0x7f3100109120 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f3104982640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31001049a0 0x7f310019aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f3104982640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31001049a0 0x7f310019aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55852/0 (socket says 172.21.15.5:55852) 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f3104982640 1 -- 172.21.15.5:0/1525575868 learned_addr learned my addr 172.21.15.5:0/1525575868 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:14.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f3105984640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3100109120 0x7f310019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f3105183640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3100103ff0 0x7f310019a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f3104982640 1 -- 172.21.15.5:0/1525575868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3100109120 msgr2=0x7f310019df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f3104982640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3100109120 0x7f310019df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f3104982640 1 -- 172.21.15.5:0/1525575868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3100103ff0 msgr2=0x7f310019a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f3104982640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3100103ff0 0x7f310019a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f3104982640 1 -- 172.21.15.5:0/1525575868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310019e6b0 con 0x7f31001049a0 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f3104982640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31001049a0 0x7f310019aca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f30f400c890 tx=0x7f30f400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f30f27fc640 1 -- 172.21.15.5:0/1525575868 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4004220 con 0x7f31001049a0 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f30f27fc640 1 -- 172.21.15.5:0/1525575868 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f40043c0 con 0x7f31001049a0 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f31073e7640 1 -- 172.21.15.5:0/1525575868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310019e9a0 con 0x7f31001049a0 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.727+0000 7f30f27fc640 1 -- 172.21.15.5:0/1525575868 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f40049e0 con 0x7f31001049a0 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.728+0000 7f31073e7640 1 -- 172.21.15.5:0/1525575868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31001a7310 con 0x7f31001049a0 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.729+0000 7f31073e7640 1 -- 172.21.15.5:0/1525575868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31001043d0 con 0x7f31001049a0 2024-09-15T22:09:14.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.730+0000 7f30f27fc640 1 -- 172.21.15.5:0/1525575868 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f30f40079b0 con 0x7f31001049a0 2024-09-15T22:09:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.730+0000 7f30f27fc640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f30ec0a34d0 0x7f30ec0a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.731+0000 7f3105183640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f30ec0a34d0 0x7f30ec0a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.731+0000 7f30f27fc640 1 -- 172.21.15.5:0/1525575868 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f30f40d0400 con 0x7f31001049a0 2024-09-15T22:09:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.731+0000 7f3105183640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f30ec0a34d0 0x7f30ec0a5990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0096c0 tx=0x7f30fc031490 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:14.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.734+0000 7f30f27fc640 1 -- 172.21.15.5:0/1525575868 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30f4014030 con 0x7f31001049a0 2024-09-15T22:09:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.910+0000 7f31073e7640 1 -- 172.21.15.5:0/1525575868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f310019bac0 con 0x7f31001049a0 2024-09-15T22:09:14.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:14 smithi005 ceph-mon[30944]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-15T22:09:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.912+0000 7f30f27fc640 1 -- 172.21.15.5:0/1525575868 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+25126 (secure 0 0 0) 0x7f310019bac0 con 0x7f31001049a0 2024-09-15T22:09:14.913 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:14.914 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":89,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:08:53.484177+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2453332511":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/878276657":"2024-09-16T22:08:53.484141+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1408571851":"2024-09-16T22:08:53.484141+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/801026306":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6800/801026306":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2083481614":"2024-09-16T22:08:53.484141+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:09:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 -- 172.21.15.5:0/1525575868 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f30ec0a34d0 msgr2=0x7f30ec0a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f30ec0a34d0 0x7f30ec0a5990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0096c0 tx=0x7f30fc031490 comp rx=0 tx=0).stop 2024-09-15T22:09:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 -- 172.21.15.5:0/1525575868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31001049a0 msgr2=0x7f310019aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31001049a0 0x7f310019aca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f30f400c890 tx=0x7f30f400cd50 comp rx=0 tx=0).stop 2024-09-15T22:09:14.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 -- 172.21.15.5:0/1525575868 shutdown_connections 2024-09-15T22:09:14.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f30ec0a34d0 0x7f30ec0a5990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:14.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3100109120 0x7f310019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:14.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31001049a0 0x7f310019aca0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:14.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 --2- 172.21.15.5:0/1525575868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3100103ff0 0x7f310019a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:14.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 -- 172.21.15.5:0/1525575868 >> 172.21.15.5:0/1525575868 conn(0x7f31000ffab0 msgr2=0x7f3100100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:14.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 -- 172.21.15.5:0/1525575868 shutdown_connections 2024-09-15T22:09:14.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:14.914+0000 7f31073e7640 1 -- 172.21.15.5:0/1525575868 wait complete. 2024-09-15T22:09:15.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:14 smithi099 ceph-mon[34892]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-15T22:09:15.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:14 smithi129 ceph-mon[35314]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 76 KiB/s wr, 36 op/s 2024-09-15T22:09:15.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-15T22:09:15.918 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:16.191 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:15 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1525575868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:09:16.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:15 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1525575868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:09:16.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:15 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1525575868' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:09:17.036 INFO:teuthology.orchestra.run.smithi005.stdout:{"run_dir":"/var/run/ceph"} 2024-09-15T22:09:17.036 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-15T22:09:17.036 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3393 2024-09-15T22:09:17.037 DEBUG:tasks.quiescer:Yielding 2024-09-15T22:09:17.037 INFO:teuthology.run_tasks:Running task exec... 2024-09-15T22:09:17.048 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:09:17.048 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi005.front.sepia.ceph.com 2024-09-15T22:09:17.048 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2024-09-15T22:09:17.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:17.086 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:17.098 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem Size Used Avail Use% Mounted on 2024-09-15T22:09:17.098 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:3300,172.21.15.99:3300,172.21.15.129:3300:/volumes/qa/sv_1/b72084f4-a7c4-4312-ae50-6ba93e80f47a 1.1T 392M 1.1T 1% /tmp/cephfs 2024-09-15T22:09:17.101 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2024-09-15T22:09:17.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:16 smithi005 ceph-mon[30944]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-15T22:09:17.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2024-09-15T22:09:17.282 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2024-09-15T22:09:17.322 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:17.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:16 smithi099 ceph-mon[34892]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-15T22:09:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:16 smithi129 ceph-mon[35314]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 76 KiB/s wr, 39 op/s 2024-09-15T22:09:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.830+0000 7f08b87e4640 1 -- 172.21.15.5:0/118675220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0108300 msgr2=0x7f08b010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.830+0000 7f08b87e4640 1 --2- 172.21.15.5:0/118675220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0108300 0x7f08b010ec70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009cd0 tx=0x7f08a802a8d0 comp rx=0 tx=0).stop 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.830+0000 7f08b87e4640 1 -- 172.21.15.5:0/118675220 shutdown_connections 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.830+0000 7f08b87e4640 1 --2- 172.21.15.5:0/118675220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0108300 0x7f08b010ec70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.830+0000 7f08b87e4640 1 --2- 172.21.15.5:0/118675220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0103780 0x7f08b0107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.830+0000 7f08b87e4640 1 --2- 172.21.15.5:0/118675220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08b0102dd0 0x7f08b01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.830+0000 7f08b87e4640 1 -- 172.21.15.5:0/118675220 >> 172.21.15.5:0/118675220 conn(0x7f08b00fc7d0 msgr2=0x7f08b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.831+0000 7f08b87e4640 1 -- 172.21.15.5:0/118675220 shutdown_connections 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.831+0000 7f08b87e4640 1 -- 172.21.15.5:0/118675220 wait complete. 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.831+0000 7f08b87e4640 1 Processor -- start 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.831+0000 7f08b87e4640 1 -- start start 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b87e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08b0102dd0 0x7f08b0100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b87e4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0103780 0x7f08b0100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b87e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0108300 0x7f08b01014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b87e4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08b01120d0 con 0x7f08b0108300 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b87e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0111f50 con 0x7f08b0102dd0 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b87e4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0112250 con 0x7f08b0103780 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b6d81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0108300 0x7f08b01014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b6d81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0108300 0x7f08b01014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55870/0 (socket says 172.21.15.5:55870) 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b6d81640 1 -- 172.21.15.5:0/2815425195 learned_addr learned my addr 172.21.15.5:0/2815425195 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b6580640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08b0102dd0 0x7f08b0100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b5d7f640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0103780 0x7f08b0100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b6d81640 1 -- 172.21.15.5:0/2815425195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0103780 msgr2=0x7f08b0100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b6d81640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0103780 0x7f08b0100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b6d81640 1 -- 172.21.15.5:0/2815425195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08b0102dd0 msgr2=0x7f08b0100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b6d81640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08b0102dd0 0x7f08b0100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b6d81640 1 -- 172.21.15.5:0/2815425195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b01a6a40 con 0x7f08b0108300 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.832+0000 7f08b6d81640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0108300 0x7f08b01014b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009ca0 tx=0x7f08a802f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.833+0000 7f08a77fe640 1 -- 172.21.15.5:0/2815425195 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a80317e0 con 0x7f08b0108300 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.833+0000 7f08a77fe640 1 -- 172.21.15.5:0/2815425195 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a8003e50 con 0x7f08b0108300 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.833+0000 7f08a77fe640 1 -- 172.21.15.5:0/2815425195 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a8030590 con 0x7f08b0108300 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.833+0000 7f08b87e4640 1 -- 172.21.15.5:0/2815425195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b01a6cd0 con 0x7f08b0108300 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.833+0000 7f08b87e4640 1 -- 172.21.15.5:0/2815425195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b01a71c0 con 0x7f08b0108300 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.834+0000 7f08b87e4640 1 -- 172.21.15.5:0/2815425195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b01031b0 con 0x7f08b0108300 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.835+0000 7f08a77fe640 1 -- 172.21.15.5:0/2815425195 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f08a8004040 con 0x7f08b0108300 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.836+0000 7f08a77fe640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f089c0a3550 0x7f089c0a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.836+0000 7f08b6580640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f089c0a3550 0x7f089c0a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.836+0000 7f08a77fe640 1 -- 172.21.15.5:0/2815425195 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f08a80f11d0 con 0x7f08b0108300 2024-09-15T22:09:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.837+0000 7f08b6580640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f089c0a3550 0x7f089c0a5a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009bd0 tx=0x7f08ac005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:17.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:17.843+0000 7f08a77fe640 1 -- 172.21.15.5:0/2815425195 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08a80bd670 con 0x7f08b0108300 2024-09-15T22:09:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.056+0000 7f08b87e4640 1 -- 172.21.15.5:0/2815425195 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08b010a290 con 0x7f08b0108300 2024-09-15T22:09:18.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:17 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:09:18.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.057+0000 7f08a77fe640 1 -- 172.21.15.5:0/2815425195 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f08a80bcf70 con 0x7f08b0108300 2024-09-15T22:09:18.059 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:18.059 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:09:18.059 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:09:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.059+0000 7f08b87e4640 1 -- 172.21.15.5:0/2815425195 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f089c0a3550 msgr2=0x7f089c0a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.059+0000 7f08b87e4640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f089c0a3550 0x7f089c0a5a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f08ac009bd0 tx=0x7f08ac005f90 comp rx=0 tx=0).stop 2024-09-15T22:09:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.059+0000 7f08b87e4640 1 -- 172.21.15.5:0/2815425195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0108300 msgr2=0x7f08b01014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.059+0000 7f08b87e4640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0108300 0x7f08b01014b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009ca0 tx=0x7f08a802f3b0 comp rx=0 tx=0).stop 2024-09-15T22:09:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.059+0000 7f08b87e4640 1 -- 172.21.15.5:0/2815425195 shutdown_connections 2024-09-15T22:09:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.059+0000 7f08b87e4640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f089c0a3550 0x7f089c0a5a10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:18.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.059+0000 7f08b87e4640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0108300 0x7f08b01014b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:18.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.059+0000 7f08b87e4640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f08b0103780 0x7f08b0100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:18.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.059+0000 7f08b87e4640 1 --2- 172.21.15.5:0/2815425195 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f08b0102dd0 0x7f08b0100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:18.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.059+0000 7f08b87e4640 1 -- 172.21.15.5:0/2815425195 >> 172.21.15.5:0/2815425195 conn(0x7f08b00fc7d0 msgr2=0x7f08b0105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:18.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.059+0000 7f08b87e4640 1 -- 172.21.15.5:0/2815425195 shutdown_connections 2024-09-15T22:09:18.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:18.060+0000 7f08b87e4640 1 -- 172.21.15.5:0/2815425195 wait complete. 2024-09-15T22:09:18.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:17 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:09:18.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:17 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:09:19.197 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:18 smithi005 ceph-mon[30944]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:09:19.197 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:18 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2815425195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:19.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:18 smithi099 ceph-mon[34892]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:09:19.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:18 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2815425195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:18 smithi129 ceph-mon[35314]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-15T22:09:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:18 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2815425195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:19.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2024-09-15T22:09:19.438 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2024-09-15T22:09:19.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2024-09-15T22:09:19.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-15T22:09:19.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-15T22:06:37.854436+0000', 'modified': '2024-09-15T22:08:52.028683+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24491, 'mds_1': 24467, 'mds_2': 24466}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.99:6835/157351762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6834', 'nonce': 157351762}, {'type': 'v1', 'addr': '172.21.15.99:6835', 'nonce': 157351762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6835/3511675647', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6834', 'nonce': 3511675647}, {'type': 'v1', 'addr': '172.21.15.129:6835', 'nonce': 3511675647}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.129:6837/3535375231', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6836', 'nonce': 3535375231}, {'type': 'v1', 'addr': '172.21.15.129:6837', 'nonce': 3535375231}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24466': {'gid': 24466, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.129:6839/2032213198', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.129:6838', 'nonce': 2032213198}, {'type': 'v1', 'addr': '172.21.15.129:6839', 'nonce': 2032213198}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.99:6837/3952304847', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6836', 'nonce': 3952304847}, {'type': 'v1', 'addr': '172.21.15.99:6837', 'nonce': 3952304847}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24491': {'gid': 24491, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.99:6839/3763383986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6838', 'nonce': 3763383986}, {'type': 'v1', 'addr': '172.21.15.99:6839', 'nonce': 3763383986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24491, 'qdb_cluster': [24491, 24467, 24466]} 2024-09-15T22:09:19.541 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-15T22:09:19.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.i status 2024-09-15T22:09:19.618 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/: 2024-09-15T22:09:19.619 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:19.619 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 . 2024-09-15T22:09:19.619 INFO:teuthology.orchestra.run.smithi005.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 .. 2024-09-15T22:09:19.619 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 data 2024-09-15T22:09:19.619 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:19.619 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data: 2024-09-15T22:09:19.619 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:19.619 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:19.619 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 .. 2024-09-15T22:09:19.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2024-09-15T22:09:19.706 INFO:teuthology.orchestra.run.smithi005.stdout:[Service] 2024-09-15T22:09:19.706 INFO:teuthology.orchestra.run.smithi005.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2024-09-15T22:09:19.706 INFO:teuthology.orchestra.run.smithi005.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2024-09-15T22:09:19.709 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2024-09-15T22:09:19.806 INFO:teuthology.orchestra.run.smithi005.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2024-09-15T22:09:19.847 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:20.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.370+0000 7faa0ec17640 1 -- 172.21.15.5:0/1417722755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa08108450 msgr2=0x7faa08108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.370+0000 7faa0ec17640 1 --2- 172.21.15.5:0/1417722755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa08108450 0x7faa08108830 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009900 tx=0x7fa9fc02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.370+0000 7faa0ec17640 1 -- 172.21.15.5:0/1417722755 shutdown_connections 2024-09-15T22:09:20.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.370+0000 7faa0ec17640 1 --2- 172.21.15.5:0/1417722755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa0810d530 0x7faa081150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.370+0000 7faa0ec17640 1 --2- 172.21.15.5:0/1417722755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa08108e00 0x7faa0810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.370+0000 7faa0ec17640 1 --2- 172.21.15.5:0/1417722755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa08108450 0x7faa08108830 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.370+0000 7faa0ec17640 1 -- 172.21.15.5:0/1417722755 >> 172.21.15.5:0/1417722755 conn(0x7faa08077ba0 msgr2=0x7faa08079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:20.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.370+0000 7faa0ec17640 1 -- 172.21.15.5:0/1417722755 shutdown_connections 2024-09-15T22:09:20.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.370+0000 7faa0ec17640 1 -- 172.21.15.5:0/1417722755 wait complete. 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.371+0000 7faa0ec17640 1 Processor -- start 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.371+0000 7faa0ec17640 1 -- start start 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.371+0000 7faa0ec17640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa08108450 0x7faa081a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.371+0000 7faa0ec17640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa08108e00 0x7faa081a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.371+0000 7faa0ec17640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa0810d530 0x7faa0819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.371+0000 7faa0ec17640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa08118530 con 0x7faa08108e00 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.371+0000 7faa0ec17640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa081183b0 con 0x7faa08108450 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.371+0000 7faa0ec17640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa081186b0 con 0x7faa0810d530 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.371+0000 7faa07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa08108e00 0x7faa081a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.371+0000 7faa07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa08108e00 0x7faa081a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55890/0 (socket says 172.21.15.5:55890) 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa07fff640 1 -- 172.21.15.5:0/1687423728 learned_addr learned my addr 172.21.15.5:0/1687423728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.371+0000 7faa0c9b3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa08108450 0x7faa081a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa0d1b4640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa0810d530 0x7faa0819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa07fff640 1 -- 172.21.15.5:0/1687423728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa0810d530 msgr2=0x7faa0819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa07fff640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa0810d530 0x7faa0819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa07fff640 1 -- 172.21.15.5:0/1687423728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa08108450 msgr2=0x7faa081a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa07fff640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa08108450 0x7faa081a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa07fff640 1 -- 172.21.15.5:0/1687423728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa0819ef40 con 0x7faa08108e00 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa08108450 0x7faa081a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa07fff640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa08108e00 0x7faa081a4b60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c920 tx=0x7fa9f400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa05ffb640 1 -- 172.21.15.5:0/1687423728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4007c70 con 0x7faa08108e00 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa05ffb640 1 -- 172.21.15.5:0/1687423728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f4007e10 con 0x7faa08108e00 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa0ec17640 1 -- 172.21.15.5:0/1687423728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa0819f230 con 0x7faa08108e00 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.372+0000 7faa05ffb640 1 -- 172.21.15.5:0/1687423728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4018890 con 0x7faa08108e00 2024-09-15T22:09:20.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.373+0000 7faa0ec17640 1 -- 172.21.15.5:0/1687423728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa081ab480 con 0x7faa08108e00 2024-09-15T22:09:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.375+0000 7faa05ffb640 1 -- 172.21.15.5:0/1687423728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7fa9f4018a70 con 0x7faa08108e00 2024-09-15T22:09:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.375+0000 7faa10ff0740 1 Processor -- start 2024-09-15T22:09:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.375+0000 7faa10ff0740 1 -- start start 2024-09-15T22:09:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a87371db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a8737c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a8737efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa05ffb640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9f80a34d0 0x7fa9f80a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa10ff0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a872d8090 con 0x562a8737cbb0 2024-09-15T22:09:20.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa10ff0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562a8727fa70 con 0x562a873719d0 2024-09-15T22:09:20.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa10ff0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562a872d4050 con 0x562a873722f0 2024-09-15T22:09:20.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa0c9b3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a8737c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa0c9b3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a8737c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:34794/0 (socket says 172.21.15.5:34794) 2024-09-15T22:09:20.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa0c9b3640 1 -- 172.21.15.5:0/2877014577 learned_addr learned my addr 172.21.15.5:0/2877014577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:20.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa07fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a8737efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa0d1b4640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9f80a34d0 0x7fa9f80a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa0d1b4640 1 --2- 172.21.15.5:0/2877014577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a87371db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa05ffb640 1 -- 172.21.15.5:0/1687423728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa9f40d04c0 con 0x7faa08108e00 2024-09-15T22:09:20.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa0c9b3640 1 -- 172.21.15.5:0/2877014577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 msgr2=0x562a87371db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:20.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/2877014577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a87371db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa0c9b3640 1 -- 172.21.15.5:0/2877014577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 msgr2=0x562a8737efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/2877014577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a8737efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa0c9b3640 1 -- 172.21.15.5:0/2877014577 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a8728c130 con 0x562a873722f0 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.376+0000 7faa07fff640 1 --2- 172.21.15.5:0/2877014577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a8737efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.377+0000 7faa0d1b4640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9f80a34d0 0x7fa9f80a5990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa00005fd0 tx=0x7faa00009dd0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.377+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/2877014577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a8737c670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc02a910 tx=0x7fa9fc031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725f5f3c5d4f3a8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.377+0000 7fa9f2ffd640 1 -- 172.21.15.5:0/2877014577 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc035040 con 0x562a873722f0 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.377+0000 7fa9f2ffd640 1 -- 172.21.15.5:0/2877014577 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9fc031a20 con 0x562a873722f0 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.377+0000 7fa9f2ffd640 1 -- 172.21.15.5:0/2877014577 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc004a00 con 0x562a873722f0 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.378+0000 7faa10ff0740 1 -- 172.21.15.5:0/2877014577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 msgr2=0x562a8737c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.378+0000 7faa10ff0740 1 --2- 172.21.15.5:0/2877014577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a8737c670 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc02a910 tx=0x7fa9fc031630 comp rx=0 tx=0).stop 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.378+0000 7faa10ff0740 1 -- 172.21.15.5:0/2877014577 shutdown_connections 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.378+0000 7faa10ff0740 1 --2- 172.21.15.5:0/2877014577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a8737efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.378+0000 7faa10ff0740 1 --2- 172.21.15.5:0/2877014577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a8737c670 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.378+0000 7faa10ff0740 1 --2- 172.21.15.5:0/2877014577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a87371db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.378+0000 7faa10ff0740 1 -- 172.21.15.5:0/2877014577 >> 172.21.15.5:0/2877014577 conn(0x562a8736df60 msgr2=0x562a8736e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.378+0000 7faa10ff0740 1 -- 172.21.15.5:0/2877014577 shutdown_connections 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.378+0000 7faa10ff0740 1 -- 172.21.15.5:0/2877014577 wait complete. 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.378+0000 7faa10ff0740 1 -- start start 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.378+0000 7faa10ff0740 1 Processor -- start 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a8737c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a87379670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa10ff0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a8737f780 con 0x562a8737cbb0 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa10ff0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562a87292f20 con 0x562a873719d0 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa10ff0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562a8737f900 con 0x562a873722f0 2024-09-15T22:09:20.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa10ff0740 10 client.0 _ll_register_callbacks cb 0x562a8725c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa0d1b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa0d1b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55902/0 (socket says 172.21.15.5:55902) 2024-09-15T22:09:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa0d1b4640 1 -- 172.21.15.5:0/2367028830 learned_addr learned my addr 172.21.15.5:0/2367028830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a8737c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa07fff640 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a87379670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa0d1b4640 1 -- 172.21.15.5:0/2367028830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 msgr2=0x562a87379670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:20.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa0d1b4640 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a87379670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa0d1b4640 1 -- 172.21.15.5:0/2367028830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 msgr2=0x562a8737c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:20.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa0d1b4640 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a8737c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa0d1b4640 1 -- 172.21.15.5:0/2367028830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a870492b0 con 0x562a8737cbb0 2024-09-15T22:09:20.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7faa0d1b4640 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379bb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faa00014500 tx=0x7faa00009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92fc56239d426f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.379+0000 7fa9e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7fa9e7fff640 1 -- 172.21.15.5:0/2367028830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00026070 con 0x562a8737cbb0 2024-09-15T22:09:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7fa9e7fff640 1 -- 172.21.15.5:0/2367028830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7faa000045b0 con 0x562a8737cbb0 2024-09-15T22:09:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7faa10ff0740 1 -- 172.21.15.5:0/2367028830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9fc031a20 con 0x562a8737cbb0 2024-09-15T22:09:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7fa9e7fff640 1 -- 172.21.15.5:0/2367028830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faa00032050 con 0x562a8737cbb0 2024-09-15T22:09:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7faa10ff0740 10 client.15489 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7faa10ff0740 1 -- 172.21.15.5:0/2367028830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a8738af60 con 0x562a8737cbb0 2024-09-15T22:09:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7fa9e7fff640 1 -- 172.21.15.5:0/2367028830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9fc031a20 con 0x562a8737cbb0 2024-09-15T22:09:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7fa9e7fff640 1 -- 172.21.15.5:0/2367028830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7faa00012450 con 0x562a8737cbb0 2024-09-15T22:09:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7faa10ff0740 10 client.15489 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7faa10ff0740 10 client.15489 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7faa10ff0740 20 client.15489 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7faa10ff0740 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a8738d5c0 0x562a8738ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7faa10ff0740 4 client.15489 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.380+0000 7faa10ff0740 1 -- 172.21.15.5:0/2367028830 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a872e5600 con 0x562a8738d5c0 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.381+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a8738d5c0 0x562a8738ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.381+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a8738d5c0 0x562a8738ea30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ff9730d23fbef53 server_cookie=c23031dccb7596ea in_seq=0 out_seq=0 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.381+0000 7fa9e7fff640 10 client.15489 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7fa9e7fff640 1 -- 172.21.15.5:0/2367028830 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a872e5600 con 0x562a8738d5c0 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7fa9e7fff640 10 client.15489 handle_command_reply: tid=0 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7faa10ff0740 1 client.15489 shutdown 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7faa10ff0740 1 -- 172.21.15.5:0/2367028830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 msgr2=0x562a87379bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7faa10ff0740 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379bb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7faa00014500 tx=0x7faa00009240 comp rx=0 tx=0).stop 2024-09-15T22:09:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7faa10ff0740 1 -- 172.21.15.5:0/2367028830 shutdown_connections 2024-09-15T22:09:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7faa10ff0740 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a8738d5c0 0x562a8738ea30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7fa9e7fff640 0 client.15489 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:09:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7faa10ff0740 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379bb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7faa10ff0740 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a87379670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7faa10ff0740 1 --2- 172.21.15.5:0/2367028830 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a8737c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7faa10ff0740 1 -- 172.21.15.5:0/2367028830 >> 172.21.15.5:0/2367028830 conn(0x562a8736df60 msgr2=0x562a87373c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7faa10ff0740 1 -- 172.21.15.5:0/2367028830 shutdown_connections 2024-09-15T22:09:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.382+0000 7faa10ff0740 1 -- 172.21.15.5:0/2367028830 wait complete. 2024-09-15T22:09:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.383+0000 7faa10ff0740 20 client.15489 trim_cache size 0 max 16384 2024-09-15T22:09:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.401+0000 7faa10ff0740 1 Processor -- start 2024-09-15T22:09:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.401+0000 7faa10ff0740 1 -- start start 2024-09-15T22:09:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.401+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a87368e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a87379630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa10ff0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a8738cd70 con 0x562a8737cbb0 2024-09-15T22:09:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa10ff0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562a8738d440 con 0x562a873719d0 2024-09-15T22:09:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa10ff0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562a8738db10 con 0x562a873722f0 2024-09-15T22:09:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa0d1b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa0d1b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55914/0 (socket says 172.21.15.5:55914) 2024-09-15T22:09:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa0d1b4640 1 -- 172.21.15.5:0/3808818940 learned_addr learned my addr 172.21.15.5:0/3808818940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/3808818940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a87368e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa07fff640 1 --2- 172.21.15.5:0/3808818940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a87379630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa0d1b4640 1 -- 172.21.15.5:0/3808818940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 msgr2=0x562a87379630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa0d1b4640 1 --2- 172.21.15.5:0/3808818940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a87379630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa0d1b4640 1 -- 172.21.15.5:0/3808818940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 msgr2=0x562a87368e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa0d1b4640 1 --2- 172.21.15.5:0/3808818940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a87368e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa0d1b4640 1 -- 172.21.15.5:0/3808818940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a8736dd90 con 0x562a8737cbb0 2024-09-15T22:09:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/3808818940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a87368e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7faa0d1b4640 1 --2- 172.21.15.5:0/3808818940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379b70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faa00003f90 tx=0x7faa000098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b6addd3fc705f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7fa9f27fc640 1 -- 172.21.15.5:0/3808818940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0002f070 con 0x562a8737cbb0 2024-09-15T22:09:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7fa9f27fc640 1 -- 172.21.15.5:0/3808818940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa00009b00 con 0x562a8737cbb0 2024-09-15T22:09:20.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.402+0000 7fa9f27fc640 1 -- 172.21.15.5:0/3808818940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa00026070 con 0x562a8737cbb0 2024-09-15T22:09:20.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.403+0000 7faa10ff0740 1 -- 172.21.15.5:0/3808818940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 msgr2=0x562a87379b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.403+0000 7faa10ff0740 1 --2- 172.21.15.5:0/3808818940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379b70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faa00003f90 tx=0x7faa000098f0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.403+0000 7faa10ff0740 1 -- 172.21.15.5:0/3808818940 shutdown_connections 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.403+0000 7faa10ff0740 1 --2- 172.21.15.5:0/3808818940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a8737cbb0 0x562a87379b70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.403+0000 7faa10ff0740 1 --2- 172.21.15.5:0/3808818940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a873722f0 0x562a87379630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.403+0000 7faa10ff0740 1 --2- 172.21.15.5:0/3808818940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873719d0 0x562a87368e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.403+0000 7faa10ff0740 1 -- 172.21.15.5:0/3808818940 >> 172.21.15.5:0/3808818940 conn(0x562a8736df60 msgr2=0x562a8736e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.403+0000 7faa10ff0740 1 -- 172.21.15.5:0/3808818940 shutdown_connections 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.403+0000 7faa10ff0740 1 -- 172.21.15.5:0/3808818940 wait complete. 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.403+0000 7faa10ff0740 1 -- start start 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa10ff0740 1 Processor -- start 2024-09-15T22:09:20.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a873719d0 0x562a87377380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873722f0 0x562a87375900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa10ff0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a8737cbb0 0x562a87376090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa10ff0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a8738e8b0 con 0x562a873719d0 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa10ff0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562a8738e1e0 con 0x562a873722f0 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa10ff0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562a8738ef80 con 0x562a8737cbb0 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa10ff0740 10 client.0 _ll_register_callbacks cb 0x7faa00014430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa0c9b3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a873719d0 0x562a87377380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa0c9b3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a873719d0 0x562a87377380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55924/0 (socket says 172.21.15.5:55924) 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa0c9b3640 1 -- 172.21.15.5:0/63536284 learned_addr learned my addr 172.21.15.5:0/63536284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa07fff640 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873722f0 0x562a87375900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.404+0000 7faa0d1b4640 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a8737cbb0 0x562a87376090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7faa0c9b3640 1 -- 172.21.15.5:0/63536284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a8737cbb0 msgr2=0x562a87376090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a8737cbb0 0x562a87376090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7faa0c9b3640 1 -- 172.21.15.5:0/63536284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873722f0 msgr2=0x562a87375900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873722f0 0x562a87375900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7faa0c9b3640 1 -- 172.21.15.5:0/63536284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a87395c80 con 0x562a873719d0 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7faa07fff640 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873722f0 0x562a87375900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7faa0c9b3640 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a873719d0 0x562a87377380 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009850 tx=0x7fa9fc02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab4b5c8b143cabe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7fa9e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7fa9e7fff640 1 -- 172.21.15.5:0/63536284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc035040 con 0x562a873719d0 2024-09-15T22:09:20.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7fa9e7fff640 1 -- 172.21.15.5:0/63536284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa9fc0043c0 con 0x562a873719d0 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7faa10ff0740 1 -- 172.21.15.5:0/63536284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a872dfb10 con 0x562a873719d0 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7fa9e7fff640 1 -- 172.21.15.5:0/63536284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9fc062050 con 0x562a873719d0 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7faa10ff0740 10 client.15495 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7faa10ff0740 1 -- 172.21.15.5:0/63536284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a87396f80 con 0x562a873719d0 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.405+0000 7fa9e7fff640 1 -- 172.21.15.5:0/63536284 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562a872dfb10 con 0x562a873719d0 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.406+0000 7fa9e7fff640 1 -- 172.21.15.5:0/63536284 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa9fc050050 con 0x562a873719d0 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.406+0000 7faa10ff0740 10 client.15495 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.406+0000 7faa10ff0740 10 client.15495 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.406+0000 7faa10ff0740 20 client.15495 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.406+0000 7faa10ff0740 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a8738d5c0 0x562a873fa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:20.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.406+0000 7faa10ff0740 4 client.15495 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.406+0000 7faa10ff0740 1 -- 172.21.15.5:0/63536284 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "status"}) -- 0x562a8725c0e0 con 0x562a8738d5c0 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.406+0000 7faa07fff640 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a8738d5c0 0x562a873fa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa07fff640 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a8738d5c0 0x562a873fa450 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3334238bfb43a24a server_cookie=874b16ac885129af in_seq=0 out_seq=0 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7fa9e7fff640 10 client.15495 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7fa9e7fff640 1 -- 172.21.15.5:0/63536284 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562a8725c0e0 con 0x562a8738d5c0 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7fa9e7fff640 10 client.15495 handle_command_reply: tid=0 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 1 client.15495 shutdown 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 1 -- 172.21.15.5:0/63536284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a873719d0 msgr2=0x562a87377380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a873719d0 0x562a87377380 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009850 tx=0x7fa9fc02a3f0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 1 -- 172.21.15.5:0/63536284 shutdown_connections 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a8738d5c0 0x562a873fa450 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a8737cbb0 0x562a87376090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a873722f0 0x562a87375900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 1 --2- 172.21.15.5:0/63536284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a873719d0 0x562a87377380 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7fa9e7fff640 0 client.15495 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:09:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 1 -- 172.21.15.5:0/63536284 >> 172.21.15.5:0/63536284 conn(0x562a8736df60 msgr2=0x562a87379630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 1 -- 172.21.15.5:0/63536284 shutdown_connections 2024-09-15T22:09:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 1 -- 172.21.15.5:0/63536284 wait complete. 2024-09-15T22:09:20.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.407+0000 7faa10ff0740 20 client.15495 trim_cache size 0 max 16384 2024-09-15T22:09:20.416 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24466, 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 126.680298196, 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 89, 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 183.30185536499999 2024-09-15T22:09:20.417 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:09:20.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 -- 172.21.15.5:0/1687423728 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9f80a34d0 msgr2=0x7fa9f80a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9f80a34d0 0x7fa9f80a5990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faa00005fd0 tx=0x7faa00009dd0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 -- 172.21.15.5:0/1687423728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa08108e00 msgr2=0x7faa081a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:20.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa08108e00 0x7faa081a4b60 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400c920 tx=0x7fa9f400cde0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 -- 172.21.15.5:0/1687423728 shutdown_connections 2024-09-15T22:09:20.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9f80a34d0 0x7fa9f80a5990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa0810d530 0x7faa0819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa08108e00 0x7faa081a4b60 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 --2- 172.21.15.5:0/1687423728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa08108450 0x7faa081a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:20.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 -- 172.21.15.5:0/1687423728 >> 172.21.15.5:0/1687423728 conn(0x7faa08077ba0 msgr2=0x7faa08078ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:20.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 -- 172.21.15.5:0/1687423728 shutdown_connections 2024-09-15T22:09:20.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:20.408+0000 7faa0ec17640 1 -- 172.21.15.5:0/1687423728 wait complete. 2024-09-15T22:09:21.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:20 smithi005 ceph-mon[30944]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:09:21.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-15T22:09:21.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:20 smithi099 ceph-mon[34892]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:09:21.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:20 smithi129 ceph-mon[35314]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-15T22:09:21.446 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.971+0000 7f484d4aa640 1 -- 172.21.15.5:0/673516566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4848106f10 msgr2=0x7f484810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.971+0000 7f484d4aa640 1 --2- 172.21.15.5:0/673516566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4848106f10 0x7f484810ea90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4840009900 tx=0x7f484002a410 comp rx=0 tx=0).stop 2024-09-15T22:09:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.971+0000 7f484d4aa640 1 -- 172.21.15.5:0/673516566 shutdown_connections 2024-09-15T22:09:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.971+0000 7f484d4aa640 1 --2- 172.21.15.5:0/673516566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4848106f10 0x7f484810ea90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.971+0000 7f484d4aa640 1 --2- 172.21.15.5:0/673516566 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4848102790 0x7f48481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.971+0000 7f484d4aa640 1 --2- 172.21.15.5:0/673516566 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4848101de0 0x7f48481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.971+0000 7f484d4aa640 1 -- 172.21.15.5:0/673516566 >> 172.21.15.5:0/673516566 conn(0x7f48480fd8c0 msgr2=0x7f48480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.971+0000 7f484d4aa640 1 -- 172.21.15.5:0/673516566 shutdown_connections 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.971+0000 7f484d4aa640 1 -- 172.21.15.5:0/673516566 wait complete. 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.972+0000 7f484d4aa640 1 Processor -- start 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.972+0000 7f484d4aa640 1 -- start start 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.972+0000 7f484d4aa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4848101de0 0x7f48481983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.972+0000 7f484d4aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4848102790 0x7f4848198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.972+0000 7f484d4aa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4848106f10 0x7f484819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.972+0000 7f484d4aa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4848111f70 con 0x7f4848102790 2024-09-15T22:09:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.972+0000 7f484d4aa640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4848111df0 con 0x7f4848106f10 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.972+0000 7f484d4aa640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48481120f0 con 0x7f4848101de0 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.972+0000 7f48467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4848102790 0x7f4848198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f48467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4848102790 0x7f4848198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55940/0 (socket says 172.21.15.5:55940) 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f4846ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4848101de0 0x7f48481983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f48467fc640 1 -- 172.21.15.5:0/119559997 learned_addr learned my addr 172.21.15.5:0/119559997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f48477fe640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4848106f10 0x7f484819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f48467fc640 1 -- 172.21.15.5:0/119559997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4848101de0 msgr2=0x7f48481983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f48467fc640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4848101de0 0x7f48481983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f48467fc640 1 -- 172.21.15.5:0/119559997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4848106f10 msgr2=0x7f484819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f48467fc640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4848106f10 0x7f484819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f48467fc640 1 -- 172.21.15.5:0/119559997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f484819d380 con 0x7f4848102790 2024-09-15T22:09:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f48467fc640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4848102790 0x7f4848198910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f483c00db80 tx=0x7f483c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f482ffff640 1 -- 172.21.15.5:0/119559997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c017070 con 0x7f4848102790 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f484d4aa640 1 -- 172.21.15.5:0/119559997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f484819d670 con 0x7f4848102790 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f482ffff640 1 -- 172.21.15.5:0/119559997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f483c003b70 con 0x7f4848102790 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f482ffff640 1 -- 172.21.15.5:0/119559997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c0106c0 con 0x7f4848102790 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.973+0000 7f484d4aa640 1 -- 172.21.15.5:0/119559997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48481a4f50 con 0x7f4848102790 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f482ffff640 1 -- 172.21.15.5:0/119559997 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f483c004020 con 0x7f4848102790 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f484f883740 1 Processor -- start 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f484f883740 1 -- start start 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f484f883740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7d9d0 0x560d98f7ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f484f883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f88670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f484f883740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f8afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f484f883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560d98ee4090 con 0x560d98f7e2f0 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f484f883740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560d98e8ba70 con 0x560d98f88bb0 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f484f883740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560d98ee0050 con 0x560d98f7d9d0 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f4846ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f88670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f4846ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f88670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55946/0 (socket says 172.21.15.5:55946) 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f4846ffd640 1 -- 172.21.15.5:0/2374657404 learned_addr learned my addr 172.21.15.5:0/2374657404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f482ffff640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48300a34d0 0x7f48300a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f48477fe640 1 --2- 172.21.15.5:0/2374657404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7d9d0 0x560d98f7ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f48467fc640 1 --2- 172.21.15.5:0/2374657404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f8afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.976+0000 7f4846ffd640 1 -- 172.21.15.5:0/2374657404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7d9d0 msgr2=0x560d98f7ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.977+0000 7f4846ffd640 1 --2- 172.21.15.5:0/2374657404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7d9d0 0x560d98f7ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.977+0000 7f4846ffd640 1 -- 172.21.15.5:0/2374657404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 msgr2=0x560d98f8afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.977+0000 7f4846ffd640 1 --2- 172.21.15.5:0/2374657404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f8afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.977+0000 7f4846ffd640 1 -- 172.21.15.5:0/2374657404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d98e98130 con 0x560d98f7e2f0 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.977+0000 7f48467fc640 1 --2- 172.21.15.5:0/2374657404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f8afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.977+0000 7f482ffff640 1 -- 172.21.15.5:0/119559997 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f483c0d0070 con 0x7f4848102790 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.977+0000 7f48477fe640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48300a34d0 0x7f48300a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.977+0000 7f4846ffd640 1 --2- 172.21.15.5:0/2374657404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f88670 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f483800bd10 tx=0x7f48380107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c55a7351d52b64fb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.977+0000 7f482d7fa640 1 -- 172.21.15.5:0/2374657404 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838004030 con 0x560d98f7e2f0 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.977+0000 7f482d7fa640 1 -- 172.21.15.5:0/2374657404 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48380041d0 con 0x560d98f7e2f0 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.977+0000 7f482d7fa640 1 -- 172.21.15.5:0/2374657404 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838011820 con 0x560d98f7e2f0 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 -- 172.21.15.5:0/2374657404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 msgr2=0x560d98f88670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f48477fe640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48300a34d0 0x7f48300a5990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f48400096c0 tx=0x7f4840009260 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 --2- 172.21.15.5:0/2374657404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f88670 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f483800bd10 tx=0x7f48380107a0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 -- 172.21.15.5:0/2374657404 shutdown_connections 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 --2- 172.21.15.5:0/2374657404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f8afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 --2- 172.21.15.5:0/2374657404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f88670 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 --2- 172.21.15.5:0/2374657404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7d9d0 0x560d98f7ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 -- 172.21.15.5:0/2374657404 >> 172.21.15.5:0/2374657404 conn(0x560d98f79f60 msgr2=0x560d98f7a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 -- 172.21.15.5:0/2374657404 shutdown_connections 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 -- 172.21.15.5:0/2374657404 wait complete. 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 -- start start 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 Processor -- start 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f88420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7e2f0 0x560d98f85770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.978+0000 7f484f883740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f85cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f484f883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560d98f8b780 con 0x560d98f7d9d0 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f484f883740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560d98e9ef20 con 0x560d98f88bb0 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f484f883740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560d98f8b900 con 0x560d98f7e2f0 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f484f883740 10 client.0 _ll_register_callbacks cb 0x560d98e68030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f4846ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f88420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f4846ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f88420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55956/0 (socket says 172.21.15.5:55956) 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f4846ffd640 1 -- 172.21.15.5:0/732357515 learned_addr learned my addr 172.21.15.5:0/732357515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f48467fc640 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7e2f0 0x560d98f85770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f48477fe640 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f85cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f4846ffd640 1 -- 172.21.15.5:0/732357515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7e2f0 msgr2=0x560d98f85770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f4846ffd640 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7e2f0 0x560d98f85770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f4846ffd640 1 -- 172.21.15.5:0/732357515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 msgr2=0x560d98f85cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f4846ffd640 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f85cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f4846ffd640 1 -- 172.21.15.5:0/732357515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d98c552b0 con 0x560d98f7d9d0 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f4846ffd640 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f88420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4838010ca0 tx=0x7f4838004140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f26e30aa76d0642 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f482aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.979+0000 7f482aff5640 1 -- 172.21.15.5:0/732357515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48380037f0 con 0x560d98f7d9d0 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f484f883740 1 -- 172.21.15.5:0/732357515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48380041d0 con 0x560d98f7d9d0 2024-09-15T22:09:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f482aff5640 1 -- 172.21.15.5:0/732357515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4838003990 con 0x560d98f7d9d0 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f482aff5640 1 -- 172.21.15.5:0/732357515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4838034050 con 0x560d98f7d9d0 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f484f883740 10 client.15507 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f484f883740 1 -- 172.21.15.5:0/732357515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d98f96fe0 con 0x560d98f7d9d0 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f482aff5640 1 -- 172.21.15.5:0/732357515 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48380041d0 con 0x560d98f7d9d0 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f482aff5640 1 -- 172.21.15.5:0/732357515 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4838004a80 con 0x560d98f7d9d0 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f484f883740 10 client.15507 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f484f883740 10 client.15507 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f484f883740 20 client.15507 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f484f883740 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560d98f99210 0x560d98f9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f484f883740 4 client.15507 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.980+0000 7f484f883740 1 -- 172.21.15.5:0/732357515 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d98ef1600 con 0x560d98f99210 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.981+0000 7f48467fc640 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560d98f99210 0x560d98f9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.981+0000 7f482aff5640 10 client.15507 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:09:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.981+0000 7f48467fc640 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560d98f99210 0x560d98f9a680 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9242d590fa9d507c server_cookie=2f2f2e6c4865a490 in_seq=0 out_seq=0 2024-09-15T22:09:21.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f482aff5640 1 -- 172.21.15.5:0/732357515 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d98ef1600 con 0x560d98f99210 2024-09-15T22:09:21.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f482aff5640 10 client.15507 handle_command_reply: tid=0 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 1 client.15507 shutdown 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 1 -- 172.21.15.5:0/732357515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 msgr2=0x560d98f88420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f88420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4838010ca0 tx=0x7f4838004140 comp rx=0 tx=0).stop 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 1 -- 172.21.15.5:0/732357515 shutdown_connections 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560d98f99210 0x560d98f9a680 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f85cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f482aff5640 0 client.15507 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7e2f0 0x560d98f85770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 1 --2- 172.21.15.5:0/732357515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f88420 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 1 -- 172.21.15.5:0/732357515 >> 172.21.15.5:0/732357515 conn(0x560d98f79f60 msgr2=0x560d98f7fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 1 -- 172.21.15.5:0/732357515 shutdown_connections 2024-09-15T22:09:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 1 -- 172.21.15.5:0/732357515 wait complete. 2024-09-15T22:09:21.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:21.982+0000 7f484f883740 20 client.15507 trim_cache size 0 max 16384 2024-09-15T22:09:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f484f883740 1 Processor -- start 2024-09-15T22:09:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f484f883740 1 -- start start 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f484f883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f484f883740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7e2f0 0x560d98f7fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f484f883740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98fa2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f484f883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560d98f989c0 con 0x560d98f7d9d0 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f484f883740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560d98f99090 con 0x560d98f88bb0 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f484f883740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560d98f99760 con 0x560d98f7e2f0 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f4846ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f4846ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f9a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55958/0 (socket says 172.21.15.5:55958) 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f4846ffd640 1 -- 172.21.15.5:0/1180894713 learned_addr learned my addr 172.21.15.5:0/1180894713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f48477fe640 1 --2- 172.21.15.5:0/1180894713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98fa2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f48467fc640 1 --2- 172.21.15.5:0/1180894713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7e2f0 0x560d98f7fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f4846ffd640 1 -- 172.21.15.5:0/1180894713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7e2f0 msgr2=0x560d98f7fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:22.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f4846ffd640 1 --2- 172.21.15.5:0/1180894713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7e2f0 0x560d98f7fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f4846ffd640 1 -- 172.21.15.5:0/1180894713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 msgr2=0x560d98fa2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f4846ffd640 1 --2- 172.21.15.5:0/1180894713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98fa2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f4846ffd640 1 -- 172.21.15.5:0/1180894713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d98f88040 con 0x560d98f7d9d0 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.000+0000 7f48467fc640 1 --2- 172.21.15.5:0/1180894713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7e2f0 0x560d98f7fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.001+0000 7f4846ffd640 1 --2- 172.21.15.5:0/1180894713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f9a680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f48380107e0 tx=0x7f4838003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d57cc91a93a258 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.001+0000 7f482cff9640 1 -- 172.21.15.5:0/1180894713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4838011c40 con 0x560d98f7d9d0 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.001+0000 7f482cff9640 1 -- 172.21.15.5:0/1180894713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4838026070 con 0x560d98f7d9d0 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.001+0000 7f482cff9640 1 -- 172.21.15.5:0/1180894713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48380035f0 con 0x560d98f7d9d0 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.001+0000 7f484f883740 1 -- 172.21.15.5:0/1180894713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 msgr2=0x560d98f9a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.001+0000 7f484f883740 1 --2- 172.21.15.5:0/1180894713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f9a680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f48380107e0 tx=0x7f4838003e80 comp rx=0 tx=0).stop 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.001+0000 7f484f883740 1 -- 172.21.15.5:0/1180894713 shutdown_connections 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.001+0000 7f484f883740 1 --2- 172.21.15.5:0/1180894713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98fa2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.001+0000 7f484f883740 1 --2- 172.21.15.5:0/1180894713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7e2f0 0x560d98f7fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.001+0000 7f484f883740 1 --2- 172.21.15.5:0/1180894713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7d9d0 0x560d98f9a680 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.001+0000 7f484f883740 1 -- 172.21.15.5:0/1180894713 >> 172.21.15.5:0/1180894713 conn(0x560d98f79f60 msgr2=0x560d98f7f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f484f883740 1 -- 172.21.15.5:0/1180894713 shutdown_connections 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f484f883740 1 -- 172.21.15.5:0/1180894713 wait complete. 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f484f883740 1 -- start start 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f484f883740 1 Processor -- start 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f484f883740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7d9d0 0x560d98fa2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f484f883740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f9c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f484f883740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f9c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f484f883740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560d98f9a500 con 0x560d98f7e2f0 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f484f883740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560d98f99e30 con 0x560d98f88bb0 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f484f883740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560d98f9abd0 con 0x560d98f7d9d0 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f484f883740 10 client.0 _ll_register_callbacks cb 0x7f483800bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f48467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f9c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f48467fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f9c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55962/0 (socket says 172.21.15.5:55962) 2024-09-15T22:09:22.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f48467fc640 1 -- 172.21.15.5:0/583916071 learned_addr learned my addr 172.21.15.5:0/583916071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f48477fe640 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f9c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.002+0000 7f4846ffd640 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7d9d0 0x560d98fa2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f48467fc640 1 -- 172.21.15.5:0/583916071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7d9d0 msgr2=0x560d98fa2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f48467fc640 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7d9d0 0x560d98fa2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f48467fc640 1 -- 172.21.15.5:0/583916071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 msgr2=0x560d98f9c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f48467fc640 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f9c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f48467fc640 1 -- 172.21.15.5:0/583916071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d98f9cb90 con 0x560d98f7e2f0 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f4846ffd640 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7d9d0 0x560d98fa2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f48467fc640 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f9c0e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f483c09bcf0 tx=0x7f483c00df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf3404dee3be23d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f482aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f482aff5640 1 -- 172.21.15.5:0/583916071 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483c09c460 con 0x560d98f7e2f0 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f482aff5640 1 -- 172.21.15.5:0/583916071 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f483c003d20 con 0x560d98f7e2f0 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f484f883740 1 -- 172.21.15.5:0/583916071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4838026070 con 0x560d98f7e2f0 2024-09-15T22:09:22.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f482aff5640 1 -- 172.21.15.5:0/583916071 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f483c09bd60 con 0x560d98f7e2f0 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f484f883740 10 client.15513 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f484f883740 1 -- 172.21.15.5:0/583916071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560d98fa1630 con 0x560d98f7e2f0 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.003+0000 7f482aff5640 1 -- 172.21.15.5:0/583916071 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4838026070 con 0x560d98f7e2f0 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f482aff5640 1 -- 172.21.15.5:0/583916071 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f483c0b2070 con 0x560d98f7e2f0 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f484f883740 10 client.15513 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f484f883740 10 client.15513 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f484f883740 20 client.15513 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f484f883740 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560d99008290 0x560d99006fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f484f883740 4 client.15513 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f484f883740 1 -- 172.21.15.5:0/583916071 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "status"}) -- 0x560d98eebb10 con 0x560d99008290 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f4846ffd640 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560d99008290 0x560d99006fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f4846ffd640 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560d99008290 0x560d99006fa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6226bbd1b1502bfb server_cookie=161d84ceff7ea92c in_seq=0 out_seq=0 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f482aff5640 10 client.15513 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:09:22.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f482aff5640 1 -- 172.21.15.5:0/583916071 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x560d98eebb10 con 0x560d99008290 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f482aff5640 10 client.15513 handle_command_reply: tid=0 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.004+0000 7f484f883740 1 client.15513 shutdown 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484f883740 1 -- 172.21.15.5:0/583916071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 msgr2=0x560d98f9c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484f883740 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f9c0e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f483c09bcf0 tx=0x7f483c00df10 comp rx=0 tx=0).stop 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484f883740 1 -- 172.21.15.5:0/583916071 shutdown_connections 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484f883740 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560d99008290 0x560d99006fa0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484f883740 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560d98f88bb0 0x560d98f9c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484f883740 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560d98f7e2f0 0x560d98f9c0e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f482aff5640 0 client.15513 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484f883740 1 --2- 172.21.15.5:0/583916071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560d98f7d9d0 0x560d98fa2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484f883740 1 -- 172.21.15.5:0/583916071 >> 172.21.15.5:0/583916071 conn(0x560d98f79f60 msgr2=0x560d98f7fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484f883740 1 -- 172.21.15.5:0/583916071 shutdown_connections 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484f883740 1 -- 172.21.15.5:0/583916071 wait complete. 2024-09-15T22:09:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484f883740 20 client.15513 trim_cache size 0 max 16384 2024-09-15T22:09:22.014 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:09:22.014 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T22:09:22.014 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-15T22:09:22.014 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24467, 2024-09-15T22:09:22.014 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-15T22:09:22.014 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-15T22:09:22.014 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-15T22:09:22.014 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 129.28704533999999, 2024-09-15T22:09:22.014 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-15T22:09:22.014 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 89, 2024-09-15T22:09:22.014 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:09:22.015 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 187.44985473 2024-09-15T22:09:22.015 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:09:22.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484d4aa640 1 -- 172.21.15.5:0/119559997 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48300a34d0 msgr2=0x7f48300a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:22.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484d4aa640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48300a34d0 0x7f48300a5990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f48400096c0 tx=0x7f4840009260 comp rx=0 tx=0).stop 2024-09-15T22:09:22.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484d4aa640 1 -- 172.21.15.5:0/119559997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4848102790 msgr2=0x7f4848198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:22.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484d4aa640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4848102790 0x7f4848198910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f483c00db80 tx=0x7f483c00b630 comp rx=0 tx=0).stop 2024-09-15T22:09:22.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484d4aa640 1 -- 172.21.15.5:0/119559997 shutdown_connections 2024-09-15T22:09:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484d4aa640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48300a34d0 0x7f48300a5990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484d4aa640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4848106f10 0x7f484819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484d4aa640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4848102790 0x7f4848198910 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484d4aa640 1 --2- 172.21.15.5:0/119559997 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4848101de0 0x7f48481983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.005+0000 7f484d4aa640 1 -- 172.21.15.5:0/119559997 >> 172.21.15.5:0/119559997 conn(0x7f48480fd8c0 msgr2=0x7f48480fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.006+0000 7f484d4aa640 1 -- 172.21.15.5:0/119559997 shutdown_connections 2024-09-15T22:09:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:22.006+0000 7f484d4aa640 1 -- 172.21.15.5:0/119559997 wait complete. 2024-09-15T22:09:22.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-15T22:09:23.042 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:23.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:22 smithi005 ceph-mon[30944]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-15T22:09:23.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:22 smithi099 ceph-mon[34892]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-15T22:09:23.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:22 smithi129 ceph-mon[35314]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-15T22:09:23.357 INFO:teuthology.orchestra.run.smithi005.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2024-09-15T22:09:23.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-15T22:09:23.403 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/: 2024-09-15T22:09:23.403 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.403 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 . 2024-09-15T22:09:23.403 INFO:teuthology.orchestra.run.smithi005.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 .. 2024-09-15T22:09:23.403 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 data 2024-09-15T22:09:23.403 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.403 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data: 2024-09-15T22:09:23.403 INFO:teuthology.orchestra.run.smithi005.stdout:total 35 2024-09-15T22:09:23.403 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 . 2024-09-15T22:09:23.403 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 .. 2024-09-15T22:09:23.403 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 base 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 15 22:09 global 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 log 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_commit_ts 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_dynshmem 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 15 22:09 pg_hba.conf 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 15 22:09 pg_ident.conf 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 pg_logical 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 15 22:09 pg_multixact 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_notify 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_replslot 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_serial 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_snapshots 2024-09-15T22:09:23.404 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_stat 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_stat_tmp 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 pg_subtrans 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_tblspc 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_twophase 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 PG_VERSION 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 15 22:09 pg_wal 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 pg_xact 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 15 22:09 postgresql.auto.conf 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 15 22:09 postgresql.conf 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base: 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.405 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 . 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 15 22:09 1 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 15 22:09 13493 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 15 22:09 13494 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:total 7777 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 15 22:09 . 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 .. 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 112 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 113 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 1247 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1247_fsm 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1247_vm 2024-09-15T22:09:23.406 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 15 22:09 1249 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1249_fsm 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1249_vm 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 15 22:09 1255 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1255_fsm 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1255_vm 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 15 22:09 1259 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1259_fsm 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1259_vm 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 15 22:09 13344 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13344_fsm 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13344_vm 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13346 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13348 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349 2024-09-15T22:09:23.407 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13349_fsm 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349_vm 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13351 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13353 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13354_fsm 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354_vm 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13356 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13358 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13359_fsm 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359_vm 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13361 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13363 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1417 2024-09-15T22:09:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1418 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 174 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 175 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2187 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2224 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2328 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2336 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2337 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2579 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2600 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2600_fsm 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2600_vm 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2601_fsm 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601_vm 2024-09-15T22:09:23.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2602 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2602_fsm 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2602_vm 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2603 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2603_fsm 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2603_vm 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2604 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2605 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2605_fsm 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2605_vm 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2606_fsm 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606_vm 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2607 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2607_fsm 2024-09-15T22:09:23.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2607_vm 2024-09-15T22:09:23.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 15 22:09 2608 2024-09-15T22:09:23.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2608_fsm 2024-09-15T22:09:23.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2608_vm 2024-09-15T22:09:23.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 15 22:09 2609 2024-09-15T22:09:23.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2609_fsm 2024-09-15T22:09:23.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2609_vm 2024-09-15T22:09:23.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2610 2024-09-15T22:09:23.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2610_fsm 2024-09-15T22:09:23.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2610_vm 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2611 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2612_fsm 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612_vm 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2613 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2615_fsm 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615_vm 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2616 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2616_fsm 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2616_vm 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 15 22:09 2617 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2617_fsm 2024-09-15T22:09:23.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2617_vm 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 15 22:09 2618 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2618_fsm 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2618_vm 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 15 22:09 2619 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2619_fsm 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2619_vm 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2620 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2650 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2651 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2652 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2653 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2654 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2655 2024-09-15T22:09:23.413 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2656 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2657 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 15 22:09 2658 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 2659 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2660 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2661 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2662 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2663 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2664 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2665 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2666 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2667 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2668 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2669 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2670 2024-09-15T22:09:23.414 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 15 22:09 2673 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 15 22:09 2674 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 15 22:09 2675 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2678 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2679 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2680 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2681 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2682 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2683 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2684 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2685 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2686 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2687 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2688 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2689 2024-09-15T22:09:23.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 15 22:09 2690 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 15 22:09 2691 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2692 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2693 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2696 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2699 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2701 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2702 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2703 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2704 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2753 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2753_fsm 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2753_vm 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2754 2024-09-15T22:09:23.416 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2755 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2756 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2757 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2830 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2831 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2832 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2833 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2834 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2835 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2836 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2837 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 15 22:09 2838 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2838_fsm 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2838_vm 2024-09-15T22:09:23.417 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2839 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840_fsm 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2840_vm 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2841 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2995 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2996 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3079_fsm 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079_vm 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3080 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3081 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3085 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3118 2024-09-15T22:09:23.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3119 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 15 22:09 3164 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3256 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3257 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3258 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3350 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3351 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3379 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3380 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3381 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3394 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3394_fsm 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3394_vm 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3395 2024-09-15T22:09:23.419 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3429 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3430 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3431 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3433 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3439 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3440 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3455 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 15 22:09 3456 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3456_fsm 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3456_vm 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3466 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3467 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3468 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3501 2024-09-15T22:09:23.420 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3502 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3503 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3534 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3541_fsm 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541_vm 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3542 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3574 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3575 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3576 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3596 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3597 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3598 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3599 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600 2024-09-15T22:09:23.421 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3600_fsm 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600_vm 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3601_fsm 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601_vm 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3602_fsm 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602_vm 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603_fsm 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3603_vm 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3604 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3605 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3606 2024-09-15T22:09:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3607 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3608 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 3609 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3712 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3764_fsm 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764_vm 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3766 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3767 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3997 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4143 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4144 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4145 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4146 2024-09-15T22:09:23.423 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4147 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4148 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4149 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4150 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4151 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4152 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4153 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4154 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4155 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4156 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4157 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4158 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4159 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4160 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4161 2024-09-15T22:09:23.424 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4162 2024-09-15T22:09:23.425 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4163 2024-09-15T22:09:23.425 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4164 2024-09-15T22:09:23.425 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4165 2024-09-15T22:09:23.427 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4166 2024-09-15T22:09:23.427 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4167 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4168 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4169 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4170 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4171 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4172 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4173 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4174 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 5002 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 548 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 549 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6102 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6104 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6106 2024-09-15T22:09:23.428 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6110 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6111 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6112 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6113 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6117 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 826 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 827 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 828 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 15 22:09 pg_filenode.map 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 PG_VERSION 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:total 7777 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 15 22:09 . 2024-09-15T22:09:23.429 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 .. 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 112 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 113 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 1247 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1247_fsm 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1247_vm 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 15 22:09 1249 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1249_fsm 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1249_vm 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 15 22:09 1255 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1255_fsm 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1255_vm 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 15 22:09 1259 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1259_fsm 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1259_vm 2024-09-15T22:09:23.430 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 15 22:09 13344 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13344_fsm 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13344_vm 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13346 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13348 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13349_fsm 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349_vm 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13351 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13353 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13354_fsm 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354_vm 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13356 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13358 2024-09-15T22:09:23.431 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13359_fsm 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359_vm 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13361 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13363 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1417 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1418 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 174 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 175 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2187 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2224 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2328 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2336 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2337 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2579 2024-09-15T22:09:23.432 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2600 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2600_fsm 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2600_vm 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2601_fsm 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601_vm 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2602 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2602_fsm 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2602_vm 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2603 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2603_fsm 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2603_vm 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2604 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2605 2024-09-15T22:09:23.433 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2605_fsm 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2605_vm 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2606_fsm 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606_vm 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2607 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2607_fsm 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2607_vm 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 15 22:09 2608 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2608_fsm 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2608_vm 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 15 22:09 2609 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2609_fsm 2024-09-15T22:09:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2609_vm 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2610 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2610_fsm 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2610_vm 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2611 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2612_fsm 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612_vm 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2613 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2615_fsm 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615_vm 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2616 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2616_fsm 2024-09-15T22:09:23.435 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2616_vm 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 15 22:09 2617 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2617_fsm 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2617_vm 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 15 22:09 2618 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2618_fsm 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2618_vm 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 15 22:09 2619 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2619_fsm 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2619_vm 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2620 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2650 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2651 2024-09-15T22:09:23.436 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2652 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2653 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2654 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2655 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2656 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2657 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 15 22:09 2658 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 2659 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2660 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2661 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2662 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2663 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2664 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2665 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2666 2024-09-15T22:09:23.437 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2667 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2668 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2669 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2670 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 15 22:09 2673 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 15 22:09 2674 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 15 22:09 2675 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2678 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2679 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2680 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2681 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2682 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2683 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2684 2024-09-15T22:09:23.438 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2685 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2686 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2687 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2688 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2689 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 15 22:09 2690 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 15 22:09 2691 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2692 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2693 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2696 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2699 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2701 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2702 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2703 2024-09-15T22:09:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2704 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2753 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2753_fsm 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2753_vm 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2754 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2755 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2756 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2757 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2830 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2831 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2832 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2833 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2834 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2835 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2836 2024-09-15T22:09:23.440 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2837 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 15 22:09 2838 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2838_fsm 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2838_vm 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2839 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840_fsm 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2840_vm 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2841 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2995 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2996 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3079_fsm 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079_vm 2024-09-15T22:09:23.441 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3080 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3081 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3085 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3118 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3119 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 15 22:09 3164 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3256 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3257 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3258 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3350 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3351 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3379 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3380 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3381 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3394 2024-09-15T22:09:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3394_fsm 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3394_vm 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3395 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3429 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3430 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3431 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3433 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3439 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3440 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3455 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 15 22:09 3456 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3456_fsm 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3456_vm 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3466 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3467 2024-09-15T22:09:23.443 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3468 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3501 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3502 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3503 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3534 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3541_fsm 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541_vm 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3542 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3574 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3575 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3576 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3596 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3597 2024-09-15T22:09:23.444 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3598 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3599 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3600_fsm 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600_vm 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3601_fsm 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601_vm 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3602_fsm 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602_vm 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603_fsm 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3603_vm 2024-09-15T22:09:23.445 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3604 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3605 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3606 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3607 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3608 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 3609 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3712 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3764_fsm 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764_vm 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3766 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3767 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3997 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4143 2024-09-15T22:09:23.446 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4144 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4145 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4146 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4147 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4148 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4149 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4150 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4151 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4152 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4153 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4154 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4155 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4156 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4157 2024-09-15T22:09:23.447 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4158 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4159 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4160 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4161 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4162 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4163 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4164 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4165 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4166 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4167 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4168 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4169 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4170 2024-09-15T22:09:23.448 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4171 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4172 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4173 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4174 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 5002 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 548 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 549 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6102 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6104 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6106 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6110 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6111 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6112 2024-09-15T22:09:23.449 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6113 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6117 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 826 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 827 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 828 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 15 22:09 pg_filenode.map 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 PG_VERSION 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:total 7777 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 15 22:09 . 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 .. 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 112 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 113 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 1247 2024-09-15T22:09:23.450 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1247_fsm 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1247_vm 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 15 22:09 1249 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1249_fsm 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1249_vm 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 15 22:09 1255 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1255_fsm 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1255_vm 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 15 22:09 1259 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1259_fsm 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1259_vm 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 15 22:09 13344 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13344_fsm 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13344_vm 2024-09-15T22:09:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13346 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13348 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13349_fsm 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349_vm 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13351 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13353 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13354_fsm 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354_vm 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13356 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13358 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13359_fsm 2024-09-15T22:09:23.452 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359_vm 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13361 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13363 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1417 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1418 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 174 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 175 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2187 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2224 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2328 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2336 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2337 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2579 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2600 2024-09-15T22:09:23.453 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2600_fsm 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2600_vm 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2601_fsm 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601_vm 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2602 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2602_fsm 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2602_vm 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2603 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2603_fsm 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2603_vm 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2604 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2605 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2605_fsm 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2605_vm 2024-09-15T22:09:23.454 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2606_fsm 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606_vm 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2607 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2607_fsm 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2607_vm 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 15 22:09 2608 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2608_fsm 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2608_vm 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 15 22:09 2609 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2609_fsm 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2609_vm 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2610 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2610_fsm 2024-09-15T22:09:23.455 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2610_vm 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2611 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2612_fsm 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612_vm 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2613 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2615_fsm 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615_vm 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2616 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2616_fsm 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2616_vm 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 15 22:09 2617 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2617_fsm 2024-09-15T22:09:23.456 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2617_vm 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 15 22:09 2618 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2618_fsm 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2618_vm 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 15 22:09 2619 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2619_fsm 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2619_vm 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2620 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2650 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2651 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2652 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2653 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2654 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2655 2024-09-15T22:09:23.457 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2656 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2657 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 15 22:09 2658 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 2659 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2660 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2661 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2662 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2663 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2664 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2665 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2666 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2667 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2668 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2669 2024-09-15T22:09:23.458 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2670 2024-09-15T22:09:23.459 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 15 22:09 2673 2024-09-15T22:09:23.459 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 15 22:09 2674 2024-09-15T22:09:23.459 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 15 22:09 2675 2024-09-15T22:09:23.459 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2678 2024-09-15T22:09:23.459 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2679 2024-09-15T22:09:23.459 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2680 2024-09-15T22:09:23.459 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2681 2024-09-15T22:09:23.459 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2682 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2683 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2684 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2685 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2686 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2687 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2688 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2689 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 15 22:09 2690 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 15 22:09 2691 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2692 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2693 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2696 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2699 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2701 2024-09-15T22:09:23.460 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2702 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2703 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2704 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2753 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2753_fsm 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2753_vm 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2754 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2755 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2756 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2757 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2830 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2831 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2832 2024-09-15T22:09:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2833 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2834 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2835 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2836 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2837 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 15 22:09 2838 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2838_fsm 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2838_vm 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2839 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840_fsm 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2840_vm 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2841 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2995 2024-09-15T22:09:23.462 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2996 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3079_fsm 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079_vm 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3080 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3081 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3085 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3118 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3119 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 15 22:09 3164 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3256 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3257 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3258 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3350 2024-09-15T22:09:23.463 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3351 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3379 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3380 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3381 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3394 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3394_fsm 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3394_vm 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3395 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3429 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3430 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3431 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3433 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3439 2024-09-15T22:09:23.464 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3440 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3455 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 15 22:09 3456 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3456_fsm 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3456_vm 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3466 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3467 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3468 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3501 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3502 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3503 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3534 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3541_fsm 2024-09-15T22:09:23.465 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541_vm 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3542 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3574 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3575 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3576 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3596 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3597 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3598 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3599 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3600_fsm 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600_vm 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601 2024-09-15T22:09:23.466 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3601_fsm 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601_vm 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3602_fsm 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602_vm 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603_fsm 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3603_vm 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3604 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3605 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3606 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3607 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3608 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 3609 2024-09-15T22:09:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3712 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3764_fsm 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764_vm 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3766 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3767 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3997 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4143 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4144 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4145 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4146 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4147 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4148 2024-09-15T22:09:23.468 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4149 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4150 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4151 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4152 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4153 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4154 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4155 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4156 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4157 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4158 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4159 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4160 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4161 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4162 2024-09-15T22:09:23.469 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4163 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4164 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4165 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4166 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4167 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4168 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4169 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4170 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4171 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4172 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4173 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4174 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 5002 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 548 2024-09-15T22:09:23.470 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 549 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6102 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6104 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6106 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6110 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6111 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6112 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6113 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6117 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 826 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 827 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 828 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 15 22:09 pg_filenode.map 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 PG_VERSION 2024-09-15T22:09:23.471 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/global: 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:total 537 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 57 Sep 15 22:09 . 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1213 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1213_fsm 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1213_vm 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1214 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1214_fsm 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1214_vm 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 1232 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 1233 2024-09-15T22:09:23.472 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1260 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1260_fsm 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1260_vm 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1261 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1261_fsm 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1261_vm 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1262 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1262_fsm 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1262_vm 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2396 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2396_fsm 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2396_vm 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2397 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2671 2024-09-15T22:09:23.473 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2672 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2676 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2677 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2694 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2695 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2697 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2698 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2846 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2847 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2964 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2965 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2966 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2967 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3592 2024-09-15T22:09:23.474 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3593 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4060 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4061 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4175 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4176 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4177 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4178 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4181 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4182 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4183 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4184 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4185 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4186 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6000 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6001 2024-09-15T22:09:23.475 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6002 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6100 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6114 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6115 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 pg_control 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 15 22:09 pg_filenode.map 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/log: 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-15T22:09:23.476 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 . 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 mappings 2024-09-15T22:09:23.477 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 15 22:09 replorigin_checkpoint 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 snapshots 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 .. 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 .. 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.478 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 15 22:09 . 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 members 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 offsets 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:total 8 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 . 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 15 22:09 .. 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 0000 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:total 8 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 . 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 15 22:09 .. 2024-09-15T22:09:23.479 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 0000 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.480 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-15T22:09:23.481 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:total 8 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 . 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 0000 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.482 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:total 16384 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 15 22:09 . 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:09 000000010000000000000001 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 archive_status 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:09:23.483 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:09:23.484 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 15 22:09 .. 2024-09-15T22:09:23.484 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:23.484 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-15T22:09:23.484 INFO:teuthology.orchestra.run.smithi005.stdout:total 8 2024-09-15T22:09:23.484 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 . 2024-09-15T22:09:23.484 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 23 Sep 15 22:09 .. 2024-09-15T22:09:23.484 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 0000 2024-09-15T22:09:23.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2024-09-15T22:09:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.541+0000 7f5a27b11640 1 -- 172.21.15.5:0/201467311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a20102ef0 msgr2=0x7f5a2010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.541+0000 7f5a27b11640 1 --2- 172.21.15.5:0/201467311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a20102ef0 0x7f5a2010da40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009900 tx=0x7f5a1802a480 comp rx=0 tx=0).stop 2024-09-15T22:09:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.541+0000 7f5a27b11640 1 -- 172.21.15.5:0/201467311 shutdown_connections 2024-09-15T22:09:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.541+0000 7f5a27b11640 1 --2- 172.21.15.5:0/201467311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a20102ef0 0x7f5a2010da40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.541+0000 7f5a27b11640 1 --2- 172.21.15.5:0/201467311 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a20102570 0x7f5a201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.541+0000 7f5a27b11640 1 --2- 172.21.15.5:0/201467311 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a20100e20 0x7f5a20101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.541+0000 7f5a27b11640 1 -- 172.21.15.5:0/201467311 >> 172.21.15.5:0/201467311 conn(0x7f5a200fc7d0 msgr2=0x7f5a200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.542+0000 7f5a27b11640 1 -- 172.21.15.5:0/201467311 shutdown_connections 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.542+0000 7f5a27b11640 1 -- 172.21.15.5:0/201467311 wait complete. 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.542+0000 7f5a27b11640 1 Processor -- start 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.542+0000 7f5a27b11640 1 -- start start 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a27b11640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a20100e20 0x7f5a20199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a27b11640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a20102570 0x7f5a201998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a27b11640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a20102ef0 0x7f5a201a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a27b11640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20110f80 con 0x7f5a20100e20 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a27b11640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20110e00 con 0x7f5a20102ef0 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a27b11640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5a20111100 con 0x7f5a20102570 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a258ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a20100e20 0x7f5a20199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a258ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a20100e20 0x7f5a20199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58326/0 (socket says 172.21.15.5:58326) 2024-09-15T22:09:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a258ad640 1 -- 172.21.15.5:0/1849944317 learned_addr learned my addr 172.21.15.5:0/1849944317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a250ac640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a20102570 0x7f5a201998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a260ae640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a20102ef0 0x7f5a201a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a258ad640 1 -- 172.21.15.5:0/1849944317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a20102570 msgr2=0x7f5a201998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a258ad640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a20102570 0x7f5a201998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a258ad640 1 -- 172.21.15.5:0/1849944317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a20102ef0 msgr2=0x7f5a201a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a258ad640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a20102ef0 0x7f5a201a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a258ad640 1 -- 172.21.15.5:0/1849944317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a201a4930 con 0x7f5a20100e20 2024-09-15T22:09:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a260ae640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a20102ef0 0x7f5a201a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.543+0000 7f5a258ad640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a20100e20 0x7f5a20199370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000eb90 tx=0x7f5a1000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.544+0000 7f5a1effd640 1 -- 172.21.15.5:0/1849944317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10004d20 con 0x7f5a20100e20 2024-09-15T22:09:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.544+0000 7f5a1effd640 1 -- 172.21.15.5:0/1849944317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a1000ee90 con 0x7f5a20100e20 2024-09-15T22:09:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.544+0000 7f5a27b11640 1 -- 172.21.15.5:0/1849944317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a201a4b00 con 0x7f5a20100e20 2024-09-15T22:09:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.544+0000 7f5a1effd640 1 -- 172.21.15.5:0/1849944317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a10010830 con 0x7f5a20100e20 2024-09-15T22:09:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.544+0000 7f5a27b11640 1 -- 172.21.15.5:0/1849944317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a201a4fc0 con 0x7f5a20100e20 2024-09-15T22:09:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.546+0000 7f5a1effd640 1 -- 172.21.15.5:0/1849944317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f5a10005020 con 0x7f5a20100e20 2024-09-15T22:09:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.547+0000 7f5a29eea740 1 Processor -- start 2024-09-15T22:09:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.547+0000 7f5a29eea740 1 -- start start 2024-09-15T22:09:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.547+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.547+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5daa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.547+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.547+0000 7f5a1effd640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a34d0 0x7f5a080a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.547+0000 7f5a29eea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c8a532090 con 0x557c8a5cc1f0 2024-09-15T22:09:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.547+0000 7f5a29eea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557c8a4d9a70 con 0x557c8a5cb8d0 2024-09-15T22:09:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.547+0000 7f5a29eea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c8a52e050 con 0x557c8a5dafc0 2024-09-15T22:09:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.547+0000 7f5a260ae640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a34d0 0x7f5a080a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a250ac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a258ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5daa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a260ae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a250ac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5dd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:32898/0 (socket says 172.21.15.5:32898) 2024-09-15T22:09:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a250ac640 1 -- 172.21.15.5:0/3642407709 learned_addr learned my addr 172.21.15.5:0/3642407709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a1effd640 1 -- 172.21.15.5:0/1849944317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f5a100d0ab0 con 0x7f5a20100e20 2024-09-15T22:09:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a260ae640 1 -- 172.21.15.5:0/3642407709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 msgr2=0x557c8a5dd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a260ae640 1 --2- 172.21.15.5:0/3642407709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5dd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a260ae640 1 -- 172.21.15.5:0/3642407709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 msgr2=0x557c8a5daa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a260ae640 1 --2- 172.21.15.5:0/3642407709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5daa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a260ae640 1 -- 172.21.15.5:0/3642407709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c8a4e6130 con 0x557c8a5cb8d0 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a258ad640 1 --2- 172.21.15.5:0/3642407709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a260ae640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a34d0 0x7f5a080a5990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5a180314e0 tx=0x7f5a180040c0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a260ae640 1 --2- 172.21.15.5:0/3642407709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5cbcb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a18006010 tx=0x7f5a18004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e41915f8c6df02 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a07fff640 1 -- 172.21.15.5:0/3642407709 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1804c070 con 0x557c8a5cb8d0 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a07fff640 1 -- 172.21.15.5:0/3642407709 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a18009520 con 0x557c8a5cb8d0 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.548+0000 7f5a07fff640 1 -- 172.21.15.5:0/3642407709 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18035040 con 0x557c8a5cb8d0 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.549+0000 7f5a29eea740 1 -- 172.21.15.5:0/3642407709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 msgr2=0x557c8a5cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.549+0000 7f5a29eea740 1 --2- 172.21.15.5:0/3642407709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5cbcb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5a18006010 tx=0x7f5a18004610 comp rx=0 tx=0).stop 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.549+0000 7f5a29eea740 1 -- 172.21.15.5:0/3642407709 shutdown_connections 2024-09-15T22:09:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.549+0000 7f5a29eea740 1 --2- 172.21.15.5:0/3642407709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5dd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.549+0000 7f5a29eea740 1 --2- 172.21.15.5:0/3642407709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.549+0000 7f5a29eea740 1 --2- 172.21.15.5:0/3642407709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5cbcb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.549+0000 7f5a29eea740 1 -- 172.21.15.5:0/3642407709 >> 172.21.15.5:0/3642407709 conn(0x557c8a5c7f60 msgr2=0x557c8a5c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.549+0000 7f5a29eea740 1 -- 172.21.15.5:0/3642407709 shutdown_connections 2024-09-15T22:09:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.549+0000 7f5a29eea740 1 -- 172.21.15.5:0/3642407709 wait complete. 2024-09-15T22:09:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.550+0000 7f5a29eea740 1 -- start start 2024-09-15T22:09:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.550+0000 7f5a29eea740 1 Processor -- start 2024-09-15T22:09:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.550+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5da850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.550+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.550+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5d80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.550+0000 7f5a29eea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c8a5ddb90 con 0x557c8a5cc1f0 2024-09-15T22:09:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.550+0000 7f5a29eea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557c8a4ecf20 con 0x557c8a5cb8d0 2024-09-15T22:09:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.550+0000 7f5a29eea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c8a5ddd10 con 0x557c8a5dafc0 2024-09-15T22:09:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a29eea740 10 client.0 _ll_register_callbacks cb 0x557c8a4b6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a250ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a250ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58356/0 (socket says 172.21.15.5:58356) 2024-09-15T22:09:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a250ac640 1 -- 172.21.15.5:0/4227460316 learned_addr learned my addr 172.21.15.5:0/4227460316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a258ad640 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5da850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a260ae640 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5d80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a250ac640 1 -- 172.21.15.5:0/4227460316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 msgr2=0x557c8a5d80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a250ac640 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5d80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a250ac640 1 -- 172.21.15.5:0/4227460316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 msgr2=0x557c8a5da850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a250ac640 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5da850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a250ac640 1 -- 172.21.15.5:0/4227460316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c8a2a32b0 con 0x557c8a5cc1f0 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a258ad640 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5da850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a250ac640 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d7ba0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5a140126e0 tx=0x7f5a14012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3f554c3ef799d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a057fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a057fa640 1 -- 172.21.15.5:0/4227460316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a140121d0 con 0x557c8a5cc1f0 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a057fa640 1 -- 172.21.15.5:0/4227460316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f5a14004030 con 0x557c8a5cc1f0 2024-09-15T22:09:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.551+0000 7f5a29eea740 1 -- 172.21.15.5:0/4227460316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a18009520 con 0x557c8a5cc1f0 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.552+0000 7f5a057fa640 1 -- 172.21.15.5:0/4227460316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a14025050 con 0x557c8a5cc1f0 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.552+0000 7f5a29eea740 10 client.15525 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.552+0000 7f5a29eea740 1 -- 172.21.15.5:0/4227460316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c8a5e95c0 con 0x557c8a5cc1f0 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.552+0000 7f5a057fa640 1 -- 172.21.15.5:0/4227460316 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a18009520 con 0x557c8a5cc1f0 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.552+0000 7f5a057fa640 1 -- 172.21.15.5:0/4227460316 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5a14003810 con 0x557c8a5cc1f0 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.552+0000 7f5a29eea740 10 client.15525 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.552+0000 7f5a29eea740 10 client.15525 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.552+0000 7f5a29eea740 20 client.15525 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.552+0000 7f5a29eea740 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c8a5ebc10 0x557c8a5ed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.552+0000 7f5a29eea740 4 client.15525 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.552+0000 7f5a29eea740 1 -- 172.21.15.5:0/4227460316 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c8a53f600 con 0x557c8a5ebc10 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.553+0000 7f5a258ad640 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c8a5ebc10 0x557c8a5ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.553+0000 7f5a057fa640 10 client.15525 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.553+0000 7f5a258ad640 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c8a5ebc10 0x557c8a5ed080 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71148c87b3dbb1e7 server_cookie=fee8373a03667c5f in_seq=0 out_seq=0 2024-09-15T22:09:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a057fa640 1 -- 172.21.15.5:0/4227460316 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c8a53f600 con 0x557c8a5ebc10 2024-09-15T22:09:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a057fa640 10 client.15525 handle_command_reply: tid=0 2024-09-15T22:09:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a29eea740 1 client.15525 shutdown 2024-09-15T22:09:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a29eea740 1 -- 172.21.15.5:0/4227460316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 msgr2=0x557c8a5d7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a29eea740 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d7ba0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5a140126e0 tx=0x7f5a14012ba0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a29eea740 1 -- 172.21.15.5:0/4227460316 shutdown_connections 2024-09-15T22:09:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a29eea740 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5d80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a29eea740 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d7ba0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a29eea740 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c8a5ebc10 0x557c8a5ed080 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a29eea740 1 --2- 172.21.15.5:0/4227460316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5da850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a29eea740 1 -- 172.21.15.5:0/4227460316 >> 172.21.15.5:0/4227460316 conn(0x557c8a5c7f60 msgr2=0x557c8a5cdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.554+0000 7f5a057fa640 0 client.15525 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.555+0000 7f5a29eea740 1 -- 172.21.15.5:0/4227460316 shutdown_connections 2024-09-15T22:09:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.555+0000 7f5a29eea740 1 -- 172.21.15.5:0/4227460316 wait complete. 2024-09-15T22:09:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.555+0000 7f5a29eea740 20 client.15525 trim_cache size 0 max 16384 2024-09-15T22:09:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a29eea740 1 Processor -- start 2024-09-15T22:09:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a29eea740 1 -- start start 2024-09-15T22:09:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5d5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a29eea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c8a5eb3c0 con 0x557c8a5cc1f0 2024-09-15T22:09:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a29eea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557c8a5eba90 con 0x557c8a5cb8d0 2024-09-15T22:09:23.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a29eea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c8a5ec160 con 0x557c8a5dafc0 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a250ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a258ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5ebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a250ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58362/0 (socket says 172.21.15.5:58362) 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a250ac640 1 -- 172.21.15.5:0/2251017759 learned_addr learned my addr 172.21.15.5:0/2251017759 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a260ae640 1 --2- 172.21.15.5:0/2251017759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5d5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a250ac640 1 -- 172.21.15.5:0/2251017759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 msgr2=0x557c8a5d5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a250ac640 1 --2- 172.21.15.5:0/2251017759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5d5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a250ac640 1 -- 172.21.15.5:0/2251017759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 msgr2=0x557c8a5ebc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.573+0000 7f5a250ac640 1 --2- 172.21.15.5:0/2251017759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5ebc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.574+0000 7f5a250ac640 1 -- 172.21.15.5:0/2251017759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c8a5c7d90 con 0x557c8a5cc1f0 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.574+0000 7f5a250ac640 1 --2- 172.21.15.5:0/2251017759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d4e20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5a14003c60 tx=0x7f5a140105c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f48cb85cb32d5aa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.574+0000 7f5a077fe640 1 -- 172.21.15.5:0/2251017759 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14000e60 con 0x557c8a5cc1f0 2024-09-15T22:09:23.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.574+0000 7f5a077fe640 1 -- 172.21.15.5:0/2251017759 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a14003e50 con 0x557c8a5cc1f0 2024-09-15T22:09:23.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.574+0000 7f5a077fe640 1 -- 172.21.15.5:0/2251017759 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a14022040 con 0x557c8a5cc1f0 2024-09-15T22:09:23.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.575+0000 7f5a29eea740 1 -- 172.21.15.5:0/2251017759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 msgr2=0x557c8a5d4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:23.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.575+0000 7f5a29eea740 1 --2- 172.21.15.5:0/2251017759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d4e20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5a14003c60 tx=0x7f5a140105c0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.575+0000 7f5a29eea740 1 -- 172.21.15.5:0/2251017759 shutdown_connections 2024-09-15T22:09:23.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.575+0000 7f5a29eea740 1 --2- 172.21.15.5:0/2251017759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5dafc0 0x557c8a5d5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.575+0000 7f5a29eea740 1 --2- 172.21.15.5:0/2251017759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d4e20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.575+0000 7f5a29eea740 1 --2- 172.21.15.5:0/2251017759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5ebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.575+0000 7f5a29eea740 1 -- 172.21.15.5:0/2251017759 >> 172.21.15.5:0/2251017759 conn(0x557c8a5c7f60 msgr2=0x557c8a5d6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.575+0000 7f5a29eea740 1 -- 172.21.15.5:0/2251017759 shutdown_connections 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.575+0000 7f5a29eea740 1 -- 172.21.15.5:0/2251017759 wait complete. 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.575+0000 7f5a29eea740 1 -- start start 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.575+0000 7f5a29eea740 1 Processor -- start 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5d57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a29eea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5dafc0 0x557c8a5d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a29eea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c8a5ecf00 con 0x557c8a5dafc0 2024-09-15T22:09:23.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a29eea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557c8a5ec830 con 0x557c8a5cb8d0 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a29eea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c8a5ed5d0 con 0x557c8a5cc1f0 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a29eea740 10 client.0 _ll_register_callbacks cb 0x7f5a14009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a258ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5d57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a260ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5dafc0 0x557c8a5d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a258ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5d57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:53880/0 (socket says 172.21.15.5:53880) 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a250ac640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a258ad640 1 -- 172.21.15.5:0/1356261977 learned_addr learned my addr 172.21.15.5:0/1356261977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a260ae640 1 -- 172.21.15.5:0/1356261977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5cc1f0 msgr2=0x557c8a5d3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a260ae640 1 --2- 172.21.15.5:0/1356261977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a260ae640 1 -- 172.21.15.5:0/1356261977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 msgr2=0x557c8a5d57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a260ae640 1 --2- 172.21.15.5:0/1356261977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5d57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a260ae640 1 -- 172.21.15.5:0/1356261977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c8a5d4c00 con 0x557c8a5dafc0 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a260ae640 1 --2- 172.21.15.5:0/1356261977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5dafc0 0x557c8a5d43d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5a1803d620 tx=0x7f5a1803d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ca164d03d48e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a057fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.576+0000 7f5a057fa640 1 -- 172.21.15.5:0/1356261977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1804c040 con 0x557c8a5dafc0 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a29eea740 1 -- 172.21.15.5:0/1356261977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a14003e50 con 0x557c8a5dafc0 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a057fa640 1 -- 172.21.15.5:0/1356261977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f5a1803db00 con 0x557c8a5dafc0 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a057fa640 1 -- 172.21.15.5:0/1356261977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5a18064050 con 0x557c8a5dafc0 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a29eea740 10 client.15531 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a29eea740 1 -- 172.21.15.5:0/1356261977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c8a5f5360 con 0x557c8a5dafc0 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a057fa640 1 -- 172.21.15.5:0/1356261977 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a14003e50 con 0x557c8a5dafc0 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a057fa640 1 -- 172.21.15.5:0/1356261977 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5a1803a040 con 0x557c8a5dafc0 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a29eea740 10 client.15531 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a29eea740 10 client.15531 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T22:09:23.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a29eea740 20 client.15531 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a29eea740 1 --2- 172.21.15.5:0/1356261977 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c8a659b80 0x557c8a658890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a29eea740 4 client.15531 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a29eea740 1 -- 172.21.15.5:0/1356261977 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "status"}) -- 0x557c8a539b10 con 0x557c8a659b80 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.577+0000 7f5a258ad640 1 --2- 172.21.15.5:0/1356261977 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c8a659b80 0x557c8a658890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.578+0000 7f5a057fa640 10 client.15531 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.578+0000 7f5a258ad640 1 --2- 172.21.15.5:0/1356261977 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c8a659b80 0x557c8a658890 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dfcc34da301cf5f server_cookie=a6f5f0769d26de34 in_seq=0 out_seq=0 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.578+0000 7f5a057fa640 1 -- 172.21.15.5:0/1356261977 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557c8a539b10 con 0x557c8a659b80 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.578+0000 7f5a057fa640 10 client.15531 handle_command_reply: tid=0 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.578+0000 7f5a29eea740 1 client.15531 shutdown 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a29eea740 1 -- 172.21.15.5:0/1356261977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5dafc0 msgr2=0x557c8a5d43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a29eea740 1 --2- 172.21.15.5:0/1356261977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5dafc0 0x557c8a5d43d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5a1803d620 tx=0x7f5a1803d880 comp rx=0 tx=0).stop 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a29eea740 1 -- 172.21.15.5:0/1356261977 shutdown_connections 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a29eea740 1 --2- 172.21.15.5:0/1356261977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c8a5dafc0 0x557c8a5d43d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a29eea740 1 --2- 172.21.15.5:0/1356261977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c8a5cc1f0 0x557c8a5d3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a29eea740 1 --2- 172.21.15.5:0/1356261977 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c8a659b80 0x557c8a658890 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a29eea740 1 --2- 172.21.15.5:0/1356261977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c8a5cb8d0 0x557c8a5d57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a057fa640 0 client.15531 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a29eea740 1 -- 172.21.15.5:0/1356261977 >> 172.21.15.5:0/1356261977 conn(0x557c8a5c7f60 msgr2=0x557c8a5d2a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a29eea740 1 -- 172.21.15.5:0/1356261977 shutdown_connections 2024-09-15T22:09:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a29eea740 1 -- 172.21.15.5:0/1356261977 wait complete. 2024-09-15T22:09:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.579+0000 7f5a29eea740 20 client.15531 trim_cache size 0 max 16384 2024-09-15T22:09:23.585 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:09:23.585 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "962891f2-73ad-11ef-bceb-c7b262605968", 2024-09-15T22:09:23.586 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-15T22:09:23.586 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24491, 2024-09-15T22:09:23.586 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-15T22:09:23.586 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-15T22:09:23.586 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-15T22:09:23.586 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 165.38904942600001, 2024-09-15T22:09:23.586 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-15T22:09:23.586 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 89, 2024-09-15T22:09:23.586 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 88, 2024-09-15T22:09:23.586 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 176.524773748 2024-09-15T22:09:23.586 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:09:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 -- 172.21.15.5:0/1849944317 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a34d0 msgr2=0x7f5a080a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a34d0 0x7f5a080a5990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5a180314e0 tx=0x7f5a180040c0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 -- 172.21.15.5:0/1849944317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a20100e20 msgr2=0x7f5a20199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a20100e20 0x7f5a20199370 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5a1000eb90 tx=0x7f5a1000c6a0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 -- 172.21.15.5:0/1849944317 shutdown_connections 2024-09-15T22:09:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a34d0 0x7f5a080a5990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a20102ef0 0x7f5a201a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a20102570 0x7f5a201998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 --2- 172.21.15.5:0/1849944317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a20100e20 0x7f5a20199370 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 -- 172.21.15.5:0/1849944317 >> 172.21.15.5:0/1849944317 conn(0x7f5a200fc7d0 msgr2=0x7f5a20106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 -- 172.21.15.5:0/1849944317 shutdown_connections 2024-09-15T22:09:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:23.580+0000 7f5a27b11640 1 -- 172.21.15.5:0/1849944317 wait complete. 2024-09-15T22:09:23.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 32 -i' 2024-09-15T22:09:23.678 INFO:teuthology.orchestra.run.smithi005.stderr:dropping old tables... 2024-09-15T22:09:23.678 INFO:teuthology.orchestra.run.smithi005.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2024-09-15T22:09:23.678 INFO:teuthology.orchestra.run.smithi005.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2024-09-15T22:09:23.678 INFO:teuthology.orchestra.run.smithi005.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2024-09-15T22:09:23.678 INFO:teuthology.orchestra.run.smithi005.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2024-09-15T22:09:23.678 INFO:teuthology.orchestra.run.smithi005.stderr:creating tables... 2024-09-15T22:09:23.690 INFO:teuthology.orchestra.run.smithi005.stderr:generating data (client-side)... 2024-09-15T22:09:23.767 INFO:teuthology.orchestra.run.smithi005.stderr:100000 of 3200000 tuples (3%) done (elapsed 0.06 s, remaining 1.86 s) 2024-09-15T22:09:23.921 INFO:teuthology.orchestra.run.smithi005.stderr:200000 of 3200000 tuples (6%) done (elapsed 0.21 s, remaining 3.21 s) 2024-09-15T22:09:24.019 INFO:teuthology.orchestra.run.smithi005.stderr:300000 of 3200000 tuples (9%) done (elapsed 0.31 s, remaining 3.02 s) 2024-09-15T22:09:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:400000 of 3200000 tuples (12%) done (elapsed 0.46 s, remaining 3.23 s) 2024-09-15T22:09:24.312 INFO:teuthology.orchestra.run.smithi005.stderr:500000 of 3200000 tuples (15%) done (elapsed 0.60 s, remaining 3.27 s) 2024-09-15T22:09:24.466 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-15T22:09:24.469 INFO:teuthology.orchestra.run.smithi005.stderr:600000 of 3200000 tuples (18%) done (elapsed 0.76 s, remaining 3.30 s) 2024-09-15T22:09:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:700000 of 3200000 tuples (21%) done (elapsed 0.82 s, remaining 2.94 s) 2024-09-15T22:09:24.685 INFO:teuthology.orchestra.run.smithi005.stderr:800000 of 3200000 tuples (25%) done (elapsed 0.98 s, remaining 2.94 s) 2024-09-15T22:09:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:900000 of 3200000 tuples (28%) done (elapsed 1.06 s, remaining 2.71 s) 2024-09-15T22:09:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:1000000 of 3200000 tuples (31%) done (elapsed 1.19 s, remaining 2.63 s) 2024-09-15T22:09:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:1100000 of 3200000 tuples (34%) done (elapsed 1.37 s, remaining 2.61 s) 2024-09-15T22:09:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:1200000 of 3200000 tuples (37%) done (elapsed 1.47 s, remaining 2.44 s) 2024-09-15T22:09:25.301 INFO:teuthology.orchestra.run.smithi005.stderr:1300000 of 3200000 tuples (40%) done (elapsed 1.59 s, remaining 2.33 s) 2024-09-15T22:09:25.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:24 smithi099 ceph-mon[34892]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-15T22:09:25.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:24 smithi129 ceph-mon[35314]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-15T22:09:25.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:24 smithi005 ceph-mon[30944]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-15T22:09:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:1400000 of 3200000 tuples (43%) done (elapsed 1.77 s, remaining 2.28 s) 2024-09-15T22:09:25.535 INFO:teuthology.orchestra.run.smithi005.stderr:1500000 of 3200000 tuples (46%) done (elapsed 1.83 s, remaining 2.07 s) 2024-09-15T22:09:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:1600000 of 3200000 tuples (50%) done (elapsed 2.01 s, remaining 2.01 s) 2024-09-15T22:09:25.867 INFO:teuthology.orchestra.run.smithi005.stderr:1700000 of 3200000 tuples (53%) done (elapsed 2.16 s, remaining 1.91 s) 2024-09-15T22:09:25.945 INFO:teuthology.orchestra.run.smithi005.stderr:1800000 of 3200000 tuples (56%) done (elapsed 2.24 s, remaining 1.74 s) 2024-09-15T22:09:26.091 INFO:teuthology.orchestra.run.smithi005.stderr:1900000 of 3200000 tuples (59%) done (elapsed 2.38 s, remaining 1.63 s) 2024-09-15T22:09:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2000000 of 3200000 tuples (62%) done (elapsed 2.47 s, remaining 1.48 s) 2024-09-15T22:09:26.314 INFO:teuthology.orchestra.run.smithi005.stderr:2100000 of 3200000 tuples (65%) done (elapsed 2.61 s, remaining 1.37 s) 2024-09-15T22:09:26.473 INFO:teuthology.orchestra.run.smithi005.stderr:2200000 of 3200000 tuples (68%) done (elapsed 2.77 s, remaining 1.26 s) 2024-09-15T22:09:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2300000 of 3200000 tuples (71%) done (elapsed 2.86 s, remaining 1.12 s) 2024-09-15T22:09:26.682 INFO:teuthology.orchestra.run.smithi005.stderr:2400000 of 3200000 tuples (75%) done (elapsed 2.97 s, remaining 0.99 s) 2024-09-15T22:09:26.847 INFO:teuthology.orchestra.run.smithi005.stderr:2500000 of 3200000 tuples (78%) done (elapsed 3.14 s, remaining 0.88 s) 2024-09-15T22:09:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2600000 of 3200000 tuples (81%) done (elapsed 3.23 s, remaining 0.75 s) 2024-09-15T22:09:27.080 INFO:teuthology.orchestra.run.smithi005.stderr:2700000 of 3200000 tuples (84%) done (elapsed 3.37 s, remaining 0.62 s) 2024-09-15T22:09:27.173 INFO:teuthology.orchestra.run.smithi005.stderr:2800000 of 3200000 tuples (87%) done (elapsed 3.47 s, remaining 0.50 s) 2024-09-15T22:09:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2900000 of 3200000 tuples (90%) done (elapsed 3.62 s, remaining 0.37 s) 2024-09-15T22:09:27.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:26 smithi099 ceph-mon[34892]: pgmap v19: 129 pgs: 129 active+clean; 109 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 14 MiB/s wr, 100 op/s 2024-09-15T22:09:27.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:26 smithi129 ceph-mon[35314]: pgmap v19: 129 pgs: 129 active+clean; 109 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 14 MiB/s wr, 100 op/s 2024-09-15T22:09:27.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:26 smithi005 ceph-mon[30944]: pgmap v19: 129 pgs: 129 active+clean; 109 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 14 MiB/s wr, 100 op/s 2024-09-15T22:09:27.465 INFO:teuthology.orchestra.run.smithi005.stderr:3000000 of 3200000 tuples (93%) done (elapsed 3.76 s, remaining 0.25 s) 2024-09-15T22:09:27.553 INFO:teuthology.orchestra.run.smithi005.stderr:3100000 of 3200000 tuples (96%) done (elapsed 3.85 s, remaining 0.12 s) 2024-09-15T22:09:27.723 INFO:teuthology.orchestra.run.smithi005.stderr:3200000 of 3200000 tuples (100%) done (elapsed 4.02 s, remaining 0.00 s) 2024-09-15T22:09:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:vacuuming... 2024-09-15T22:09:28.362 INFO:teuthology.orchestra.run.smithi005.stderr:creating primary keys... 2024-09-15T22:09:29.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:28 smithi099 ceph-mon[34892]: pgmap v20: 129 pgs: 129 active+clean; 109 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 14 MiB/s wr, 98 op/s 2024-09-15T22:09:29.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:28 smithi129 ceph-mon[35314]: pgmap v20: 129 pgs: 129 active+clean; 109 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 14 MiB/s wr, 98 op/s 2024-09-15T22:09:29.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:28 smithi005 ceph-mon[30944]: pgmap v20: 129 pgs: 129 active+clean; 109 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 14 MiB/s wr, 98 op/s 2024-09-15T22:09:29.561 INFO:teuthology.orchestra.run.smithi005.stderr:done in 5.88 s (drop tables 0.00 s, create tables 0.01 s, client-side generate 4.06 s, vacuum 0.61 s, primary keys 1.20 s). 2024-09-15T22:09:29.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2024-09-15T22:09:29.660 INFO:teuthology.orchestra.run.smithi005.stderr:starting vacuum...end. 2024-09-15T22:09:31.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:30 smithi099 ceph-mon[34892]: pgmap v21: 129 pgs: 129 active+clean; 345 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 50 MiB/s wr, 110 op/s 2024-09-15T22:09:31.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:30 smithi129 ceph-mon[35314]: pgmap v21: 129 pgs: 129 active+clean; 345 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 50 MiB/s wr, 110 op/s 2024-09-15T22:09:31.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:30 smithi005 ceph-mon[30944]: pgmap v21: 129 pgs: 129 active+clean; 345 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 50 MiB/s wr, 110 op/s 2024-09-15T22:09:33.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:32 smithi099 ceph-mon[34892]: pgmap v22: 129 pgs: 129 active+clean; 490 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 72 MiB/s wr, 133 op/s 2024-09-15T22:09:33.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:32 smithi129 ceph-mon[35314]: pgmap v22: 129 pgs: 129 active+clean; 490 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 72 MiB/s wr, 133 op/s 2024-09-15T22:09:33.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:32 smithi005 ceph-mon[30944]: pgmap v22: 129 pgs: 129 active+clean; 490 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 72 MiB/s wr, 133 op/s 2024-09-15T22:09:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 5.0 s, 4343.0 tps, lat 22.591 ms stddev 24.049 2024-09-15T22:09:35.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:34 smithi099 ceph-mon[34892]: pgmap v23: 129 pgs: 129 active+clean; 490 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 72 MiB/s wr, 130 op/s 2024-09-15T22:09:35.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:34 smithi129 ceph-mon[35314]: pgmap v23: 129 pgs: 129 active+clean; 490 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 72 MiB/s wr, 130 op/s 2024-09-15T22:09:35.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:34 smithi005 ceph-mon[30944]: pgmap v23: 129 pgs: 129 active+clean; 490 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 72 MiB/s wr, 130 op/s 2024-09-15T22:09:35.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:09:36.027 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.591+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1320103910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2d4102ef0 msgr2=0x7fc2d410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.591+0000 7fc2d9fb2640 1 --2- 172.21.15.5:0/1320103910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2d4102ef0 0x7fc2d410da40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40098f0 tx=0x7fc2c402c650 comp rx=0 tx=0).stop 2024-09-15T22:09:36.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.591+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1320103910 shutdown_connections 2024-09-15T22:09:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.591+0000 7fc2d9fb2640 1 --2- 172.21.15.5:0/1320103910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2d4102ef0 0x7fc2d410da40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.591+0000 7fc2d9fb2640 1 --2- 172.21.15.5:0/1320103910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4102570 0x7fc2d41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.591+0000 7fc2d9fb2640 1 --2- 172.21.15.5:0/1320103910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d4100e20 0x7fc2d4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.591+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1320103910 >> 172.21.15.5:0/1320103910 conn(0x7fc2d40fc7d0 msgr2=0x7fc2d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.592+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1320103910 shutdown_connections 2024-09-15T22:09:36.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.592+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1320103910 wait complete. 2024-09-15T22:09:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.592+0000 7fc2d9fb2640 1 Processor -- start 2024-09-15T22:09:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.593+0000 7fc2d9fb2640 1 -- start start 2024-09-15T22:09:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.593+0000 7fc2d9fb2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4100e20 0x7fc2d40738e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.593+0000 7fc2d9fb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d4102570 0x7fc2d4073e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.593+0000 7fc2d9fb2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2d4102ef0 0x7fc2d406ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.593+0000 7fc2d9fb2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d406d4a0 con 0x7fc2d4102570 2024-09-15T22:09:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.593+0000 7fc2d9fb2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d406d320 con 0x7fc2d4100e20 2024-09-15T22:09:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.593+0000 7fc2d9fb2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d406d620 con 0x7fc2d4102ef0 2024-09-15T22:09:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.594+0000 7fc2d3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2d4102ef0 0x7fc2d406ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.594+0000 7fc2d37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4100e20 0x7fc2d40738e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.594+0000 7fc2d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d4102570 0x7fc2d4073e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.594+0000 7fc2d2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d4102570 0x7fc2d4073e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44312/0 (socket says 172.21.15.5:44312) 2024-09-15T22:09:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.594+0000 7fc2d2ffd640 1 -- 172.21.15.5:0/1434516150 learned_addr learned my addr 172.21.15.5:0/1434516150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.594+0000 7fc2d2ffd640 1 -- 172.21.15.5:0/1434516150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2d4102ef0 msgr2=0x7fc2d406ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.594+0000 7fc2d2ffd640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2d4102ef0 0x7fc2d406ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.594+0000 7fc2d2ffd640 1 -- 172.21.15.5:0/1434516150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4100e20 msgr2=0x7fc2d40738e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.594+0000 7fc2d2ffd640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4100e20 0x7fc2d40738e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.594+0000 7fc2d2ffd640 1 -- 172.21.15.5:0/1434516150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d406f2c0 con 0x7fc2d4102570 2024-09-15T22:09:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.594+0000 7fc2d37fe640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4100e20 0x7fc2d40738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:36.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.595+0000 7fc2d2ffd640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d4102570 0x7fc2d4073e20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc2c801fa60 tx=0x7fc2c801ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.595+0000 7fc2d0ff9640 1 -- 172.21.15.5:0/1434516150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c801dc50 con 0x7fc2d4102570 2024-09-15T22:09:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.595+0000 7fc2d0ff9640 1 -- 172.21.15.5:0/1434516150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c801ddf0 con 0x7fc2d4102570 2024-09-15T22:09:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.595+0000 7fc2d0ff9640 1 -- 172.21.15.5:0/1434516150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80216a0 con 0x7fc2d4102570 2024-09-15T22:09:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.595+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1434516150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d406f5b0 con 0x7fc2d4102570 2024-09-15T22:09:36.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.595+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1434516150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d41a2d60 con 0x7fc2d4102570 2024-09-15T22:09:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.599+0000 7fc2dc38b740 1 Processor -- start 2024-09-15T22:09:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.599+0000 7fc2dc38b740 1 -- start start 2024-09-15T22:09:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2dc38b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fb9b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2dc38b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9bdc0 0x55c64fba6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2dc38b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fba6680 0x55c64fba8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2dc38b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c64fb012e0 con 0x55c64fba6680 2024-09-15T22:09:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2dc38b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c64faa8ce0 con 0x55c64fb9b4a0 2024-09-15T22:09:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2dc38b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c64fafd2a0 con 0x55c64fb9bdc0 2024-09-15T22:09:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2d3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fb9b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2d3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fb9b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:39646/0 (socket says 172.21.15.5:39646) 2024-09-15T22:09:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2d3fff640 1 -- 172.21.15.5:0/1960307634 learned_addr learned my addr 172.21.15.5:0/1960307634 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2d37fe640 1 --2- 172.21.15.5:0/1960307634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9bdc0 0x55c64fba6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2d3fff640 1 -- 172.21.15.5:0/1960307634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9bdc0 msgr2=0x55c64fba6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:36.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/1960307634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9bdc0 0x55c64fba6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2d3fff640 1 -- 172.21.15.5:0/1960307634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fba6680 msgr2=0x55c64fba8a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:09:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/1960307634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fba6680 0x55c64fba8a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.600+0000 7fc2d3fff640 1 -- 172.21.15.5:0/1960307634 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c64fa8d440 con 0x55c64fb9b4a0 2024-09-15T22:09:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.601+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/1960307634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fb9b880 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40095c0 tx=0x7fc2c4032af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcbc131fcc2abca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.601+0000 7fc2b9ffb640 1 -- 172.21.15.5:0/1960307634 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4032e60 con 0x55c64fb9b4a0 2024-09-15T22:09:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.602+0000 7fc2b9ffb640 1 -- 172.21.15.5:0/1960307634 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c4042070 con 0x55c64fb9b4a0 2024-09-15T22:09:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.602+0000 7fc2b9ffb640 1 -- 172.21.15.5:0/1960307634 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4004920 con 0x55c64fb9b4a0 2024-09-15T22:09:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.602+0000 7fc2d0ff9640 1 -- 172.21.15.5:0/1434516150 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7fc2c80046c0 con 0x7fc2d4102570 2024-09-15T22:09:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1960307634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 msgr2=0x55c64fb9b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:36.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/1960307634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fb9b880 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40095c0 tx=0x7fc2c4032af0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1960307634 shutdown_connections 2024-09-15T22:09:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/1960307634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fba6680 0x55c64fba8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/1960307634 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9bdc0 0x55c64fba6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/1960307634 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fb9b880 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1960307634 >> 172.21.15.5:0/1960307634 conn(0x55c64fb97a30 msgr2=0x55c64fb97e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1960307634 shutdown_connections 2024-09-15T22:09:36.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1960307634 wait complete. 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2d0ff9640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc2bc0a3620 0x7fc2bc0a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2d0ff9640 1 -- 172.21.15.5:0/1434516150 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fc2c80e1450 con 0x7fc2d4102570 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2d37fe640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc2bc0a3620 0x7fc2bc0a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.603+0000 7fc2dc38b740 1 -- start start 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.604+0000 7fc2dc38b740 1 Processor -- start 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.604+0000 7fc2d37fe640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc2bc0a3620 0x7fc2bc0a5ae0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55c64fb98890 tx=0x7fc2cc009680 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.604+0000 7fc2dc38b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9b4a0 0x55c64fba6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.604+0000 7fc2dc38b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9bdc0 0x55c64fba3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.604+0000 7fc2dc38b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fba6680 0x55c64fba38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.604+0000 7fc2dc38b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c64fba9250 con 0x55c64fb9bdc0 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.604+0000 7fc2dc38b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c64fabc050 con 0x55c64fba6680 2024-09-15T22:09:36.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.604+0000 7fc2dc38b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c64fba93d0 con 0x55c64fb9b4a0 2024-09-15T22:09:36.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.604+0000 7fc2dc38b740 10 client.0 _ll_register_callbacks cb 0x55c64fafc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:36.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.604+0000 7fc2d37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9b4a0 0x55c64fba6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.605+0000 7fc2d37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9b4a0 0x55c64fba6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43398/0 (socket says 172.21.15.5:43398) 2024-09-15T22:09:36.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.605+0000 7fc2d37fe640 1 -- 172.21.15.5:0/4034569082 learned_addr learned my addr 172.21.15.5:0/4034569082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:36.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.605+0000 7fc2d2ffd640 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9bdc0 0x55c64fba3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.605+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fba6680 0x55c64fba38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.605+0000 7fc2d2ffd640 1 -- 172.21.15.5:0/4034569082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9b4a0 msgr2=0x55c64fba6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.605+0000 7fc2d2ffd640 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9b4a0 0x55c64fba6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.605+0000 7fc2d2ffd640 1 -- 172.21.15.5:0/4034569082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fba6680 msgr2=0x55c64fba38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.605+0000 7fc2d2ffd640 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fba6680 0x55c64fba38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.605+0000 7fc2d2ffd640 1 -- 172.21.15.5:0/4034569082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c64f8723c0 con 0x55c64fb9bdc0 2024-09-15T22:09:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.605+0000 7fc2d37fe640 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9b4a0 0x55c64fba6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.606+0000 7fc2d2ffd640 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9bdc0 0x55c64fba3300 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80de680 tx=0x7fc2c80ad140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423bdaa5c6312ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.606+0000 7fc2b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.606+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/4034569082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c80c6060 con 0x55c64fb9bdc0 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.606+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/4034569082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fc2c80cd0d0 con 0x55c64fb9bdc0 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.606+0000 7fc2dc38b740 1 -- 172.21.15.5:0/4034569082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c4042070 con 0x55c64fb9bdc0 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.607+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/4034569082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2c80b1170 con 0x55c64fb9bdc0 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.607+0000 7fc2dc38b740 10 client.15540 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.607+0000 7fc2dc38b740 1 -- 172.21.15.5:0/4034569082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c64fbb4d10 con 0x55c64fb9bdc0 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.607+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/4034569082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c4042070 con 0x55c64fb9bdc0 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.607+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/4034569082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc2c80b59b0 con 0x55c64fb9bdc0 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.607+0000 7fc2dc38b740 10 client.15540 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.607+0000 7fc2dc38b740 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.607+0000 7fc2dc38b740 20 client.15540 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.607+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c64fbb61d0 0x55c64fbb85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.607+0000 7fc2dc38b740 4 client.15540 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:36.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.607+0000 7fc2dc38b740 1 -- 172.21.15.5:0/4034569082 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c64fb0e850 con 0x55c64fbb61d0 2024-09-15T22:09:36.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.608+0000 7fc2d37fe640 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c64fbb61d0 0x55c64fbb85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.609+0000 7fc2d37fe640 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c64fbb61d0 0x55c64fbb85c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8235ca342a076023 server_cookie=d9c4af40188bd85d in_seq=0 out_seq=0 2024-09-15T22:09:36.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.609+0000 7fc2b2ffd640 10 client.15540 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:36.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.609+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/4034569082 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c64fb0e850 con 0x55c64fbb61d0 2024-09-15T22:09:36.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.609+0000 7fc2b2ffd640 10 client.15540 handle_command_reply: tid=0 2024-09-15T22:09:36.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.609+0000 7fc2dc38b740 1 client.15540 shutdown 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.611+0000 7fc2dc38b740 1 -- 172.21.15.5:0/4034569082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9bdc0 msgr2=0x55c64fba3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.611+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9bdc0 0x55c64fba3300 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fc2c80de680 tx=0x7fc2c80ad140 comp rx=0 tx=0).stop 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.611+0000 7fc2dc38b740 1 -- 172.21.15.5:0/4034569082 shutdown_connections 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.611+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c64fbb61d0 0x55c64fbb85c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.611+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fba6680 0x55c64fba38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.611+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9bdc0 0x55c64fba3300 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.611+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/4034569082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9b4a0 0x55c64fba6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.611+0000 7fc2dc38b740 1 -- 172.21.15.5:0/4034569082 >> 172.21.15.5:0/4034569082 conn(0x55c64fb97a30 msgr2=0x55c64fb9d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.611+0000 7fc2b2ffd640 0 client.15540 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.611+0000 7fc2dc38b740 1 -- 172.21.15.5:0/4034569082 shutdown_connections 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.612+0000 7fc2dc38b740 1 -- 172.21.15.5:0/4034569082 wait complete. 2024-09-15T22:09:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.612+0000 7fc2dc38b740 20 client.15540 trim_cache size 0 max 16384 2024-09-15T22:09:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.644+0000 7fc2dc38b740 1 Processor -- start 2024-09-15T22:09:36.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.644+0000 7fc2dc38b740 1 -- start start 2024-09-15T22:09:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.644+0000 7fc2dc38b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fbb2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.645+0000 7fc2dc38b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9bdc0 0x55c64fb96a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.645+0000 7fc2d37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fbb2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.645+0000 7fc2d37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fbb2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:39666/0 (socket says 172.21.15.5:39666) 2024-09-15T22:09:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.645+0000 7fc2d37fe640 1 -- 172.21.15.5:0/119111409 learned_addr learned my addr 172.21.15.5:0/119111409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.645+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fba6680 0x55c64fbbde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.645+0000 7fc2d2ffd640 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9bdc0 0x55c64fb96a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.646+0000 7fc2dc38b740 1 -- 172.21.15.5:0/119111409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c0012880 con 0x55c64fba6680 2024-09-15T22:09:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.646+0000 7fc2dc38b740 1 -- 172.21.15.5:0/119111409 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c0010150 con 0x55c64fb9b4a0 2024-09-15T22:09:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.646+0000 7fc2dc38b740 1 -- 172.21.15.5:0/119111409 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c000dfa0 con 0x55c64fb9bdc0 2024-09-15T22:09:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.646+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fba6680 0x55c64fbbde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.647+0000 7fc2d3fff640 1 -- 172.21.15.5:0/119111409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9bdc0 msgr2=0x55c64fb96a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.647+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9bdc0 0x55c64fb96a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.647+0000 7fc2d3fff640 1 -- 172.21.15.5:0/119111409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 msgr2=0x55c64fbb2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.647+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fbb2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.647+0000 7fc2d3fff640 1 -- 172.21.15.5:0/119111409 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c64fba9130 con 0x55c64fba6680 2024-09-15T22:09:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.647+0000 7fc2d37fe640 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fbb2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.647+0000 7fc2d2ffd640 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9bdc0 0x55c64fb96a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.647+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fba6680 0x55c64fbbde60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40097e0 tx=0x7fc2c4005650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e03ec8baf2e64e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.648+0000 7fc2b97fa640 1 -- 172.21.15.5:0/119111409 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4004030 con 0x55c64fba6680 2024-09-15T22:09:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.648+0000 7fc2b97fa640 1 -- 172.21.15.5:0/119111409 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c4042040 con 0x55c64fba6680 2024-09-15T22:09:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.648+0000 7fc2b97fa640 1 -- 172.21.15.5:0/119111409 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4003870 con 0x55c64fba6680 2024-09-15T22:09:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.649+0000 7fc2dc38b740 1 -- 172.21.15.5:0/119111409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fba6680 msgr2=0x55c64fbbde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.649+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fba6680 0x55c64fbbde60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc2c40097e0 tx=0x7fc2c4005650 comp rx=0 tx=0).stop 2024-09-15T22:09:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.649+0000 7fc2dc38b740 1 -- 172.21.15.5:0/119111409 shutdown_connections 2024-09-15T22:09:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.649+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fba6680 0x55c64fbbde60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.650+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fb9bdc0 0x55c64fb96a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.650+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/119111409 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9b4a0 0x55c64fbb2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.650+0000 7fc2dc38b740 1 -- 172.21.15.5:0/119111409 >> 172.21.15.5:0/119111409 conn(0x55c64fb97a30 msgr2=0x55c64fb96350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:36.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.650+0000 7fc2dc38b740 1 -- 172.21.15.5:0/119111409 shutdown_connections 2024-09-15T22:09:36.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.650+0000 7fc2dc38b740 1 -- 172.21.15.5:0/119111409 wait complete. 2024-09-15T22:09:36.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.650+0000 7fc2dc38b740 1 -- start start 2024-09-15T22:09:36.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.651+0000 7fc2dc38b740 1 Processor -- start 2024-09-15T22:09:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.651+0000 7fc2dc38b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9b4a0 0x55c64fba9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.651+0000 7fc2dc38b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9bdc0 0x55c64fbaa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.651+0000 7fc2dc38b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fba6680 0x55c64fc1ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.651+0000 7fc2dc38b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c000e610 con 0x55c64fb9b4a0 2024-09-15T22:09:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.651+0000 7fc2dc38b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c000fa80 con 0x55c64fb9bdc0 2024-09-15T22:09:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.651+0000 7fc2dc38b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c000ece0 con 0x55c64fba6680 2024-09-15T22:09:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.651+0000 7fc2dc38b740 10 client.0 _ll_register_callbacks cb 0x7fc2c80a93e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.652+0000 7fc2d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9b4a0 0x55c64fba9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.652+0000 7fc2d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9b4a0 0x55c64fba9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44340/0 (socket says 172.21.15.5:44340) 2024-09-15T22:09:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.652+0000 7fc2d37fe640 1 -- 172.21.15.5:0/1614220926 learned_addr learned my addr 172.21.15.5:0/1614220926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.652+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fba6680 0x55c64fc1ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.652+0000 7fc2d3fff640 1 -- 172.21.15.5:0/1614220926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9bdc0 msgr2=0x55c64fbaa170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:09:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.652+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9bdc0 0x55c64fbaa170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.652+0000 7fc2d3fff640 1 -- 172.21.15.5:0/1614220926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9b4a0 msgr2=0x55c64fba9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.652+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9b4a0 0x55c64fba9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.652+0000 7fc2d3fff640 1 -- 172.21.15.5:0/1614220926 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c64fc1f3e0 con 0x55c64fba6680 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.653+0000 7fc2d3fff640 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fba6680 0x55c64fc1ee20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4005620 tx=0x7fc2c4032f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc447c2c1759cabc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.653+0000 7fc2b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.654+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/1614220926 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40325b0 con 0x55c64fba6680 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.654+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1614220926 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2c4042040 con 0x55c64fba6680 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.654+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/1614220926 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fc2c4032750 con 0x55c64fba6680 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.654+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/1614220926 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2c40605b0 con 0x55c64fba6680 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.654+0000 7fc2dc38b740 10 client.25150 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.654+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1614220926 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c64fc20640 con 0x55c64fba6680 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.655+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/1614220926 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2c4042040 con 0x55c64fba6680 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.655+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/1614220926 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc2c4047470 con 0x55c64fba6680 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.655+0000 7fc2dc38b740 10 client.25150 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.655+0000 7fc2dc38b740 10 client.25150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.655+0000 7fc2dc38b740 20 client.25150 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.655+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c64fc209d0 0x55c64fc22dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.655+0000 7fc2dc38b740 4 client.25150 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:09:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.655+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1614220926 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c64fb08d60 con 0x55c64fc209d0 2024-09-15T22:09:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.655+0000 7fc2d37fe640 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c64fc209d0 0x55c64fc22dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.656+0000 7fc2b2ffd640 10 client.25150 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.656+0000 7fc2d37fe640 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c64fc209d0 0x55c64fc22dc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3408fbaba3623abf server_cookie=c59ee3596e48a49 in_seq=0 out_seq=0 2024-09-15T22:09:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.656+0000 7fc2b2ffd640 1 -- 172.21.15.5:0/1614220926 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c64fb08d60 con 0x55c64fc209d0 2024-09-15T22:09:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.656+0000 7fc2b2ffd640 10 client.25150 handle_command_reply: tid=0 2024-09-15T22:09:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.656+0000 7fc2dc38b740 1 client.25150 shutdown 2024-09-15T22:09:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.657+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1614220926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fba6680 msgr2=0x55c64fc1ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.657+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fba6680 0x55c64fc1ee20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4005620 tx=0x7fc2c4032f50 comp rx=0 tx=0).stop 2024-09-15T22:09:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.658+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1614220926 shutdown_connections 2024-09-15T22:09:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.658+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c64fba6680 0x55c64fc1ee20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.658+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c64fc209d0 0x55c64fc22dc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.658+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c64fb9bdc0 0x55c64fbaa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.658+0000 7fc2dc38b740 1 --2- 172.21.15.5:0/1614220926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c64fb9b4a0 0x55c64fba9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.658+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1614220926 >> 172.21.15.5:0/1614220926 conn(0x55c64fb97a30 msgr2=0x55c64fba5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.658+0000 7fc2b2ffd640 0 client.25150 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.658+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1614220926 shutdown_connections 2024-09-15T22:09:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.658+0000 7fc2dc38b740 1 -- 172.21.15.5:0/1614220926 wait complete. 2024-09-15T22:09:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.658+0000 7fc2dc38b740 20 client.25150 trim_cache size 0 max 16384 2024-09-15T22:09:36.662 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:09:36.662 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:09:36.662 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:09:36.662 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:09:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1434516150 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc2bc0a3620 msgr2=0x7fc2bc0a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc2bc0a3620 0x7fc2bc0a5ae0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55c64fb98890 tx=0x7fc2cc009680 comp rx=0 tx=0).stop 2024-09-15T22:09:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1434516150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d4102570 msgr2=0x7fc2d4073e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d4102570 0x7fc2d4073e20 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc2c801fa60 tx=0x7fc2c801ff20 comp rx=0 tx=0).stop 2024-09-15T22:09:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1434516150 shutdown_connections 2024-09-15T22:09:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc2bc0a3620 0x7fc2bc0a5ae0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc2d4102ef0 0x7fc2d406ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2d4102570 0x7fc2d4073e20 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 --2- 172.21.15.5:0/1434516150 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc2d4100e20 0x7fc2d40738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1434516150 >> 172.21.15.5:0/1434516150 conn(0x7fc2d40fc7d0 msgr2=0x7fc2d40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1434516150 shutdown_connections 2024-09-15T22:09:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:36.659+0000 7fc2d9fb2640 1 -- 172.21.15.5:0/1434516150 wait complete. 2024-09-15T22:09:37.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:36 smithi099 ceph-mon[34892]: pgmap v24: 129 pgs: 129 active+clean; 876 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 78 MiB/s wr, 368 op/s 2024-09-15T22:09:37.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:36 smithi129 ceph-mon[35314]: pgmap v24: 129 pgs: 129 active+clean; 876 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 78 MiB/s wr, 368 op/s 2024-09-15T22:09:37.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:36 smithi005 ceph-mon[30944]: pgmap v24: 129 pgs: 129 active+clean; 876 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 78 MiB/s wr, 368 op/s 2024-09-15T22:09:37.480 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:09:37.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:37.930 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:38.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.529+0000 7f9a00abf640 1 -- 172.21.15.5:0/2149629185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99fc100e20 msgr2=0x7f99fc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:38.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.529+0000 7f9a00abf640 1 --2- 172.21.15.5:0/2149629185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99fc100e20 0x7f99fc101200 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009ca0 tx=0x7f99ec02a950 comp rx=0 tx=0).stop 2024-09-15T22:09:38.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.530+0000 7f9a00abf640 1 -- 172.21.15.5:0/2149629185 shutdown_connections 2024-09-15T22:09:38.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.530+0000 7f9a00abf640 1 --2- 172.21.15.5:0/2149629185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc102ef0 0x7f99fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:38.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.530+0000 7f9a00abf640 1 --2- 172.21.15.5:0/2149629185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99fc102570 0x7f99fc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:38.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.530+0000 7f9a00abf640 1 --2- 172.21.15.5:0/2149629185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99fc100e20 0x7f99fc101200 secure :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009ca0 tx=0x7f99ec02a950 comp rx=0 tx=0).stop 2024-09-15T22:09:38.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.530+0000 7f9a00abf640 1 -- 172.21.15.5:0/2149629185 >> 172.21.15.5:0/2149629185 conn(0x7f99fc0fc7d0 msgr2=0x7f99fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:38.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.531+0000 7f9a00abf640 1 -- 172.21.15.5:0/2149629185 shutdown_connections 2024-09-15T22:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.531+0000 7f9a00abf640 1 -- 172.21.15.5:0/2149629185 wait complete. 2024-09-15T22:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.532+0000 7f9a00abf640 1 Processor -- start 2024-09-15T22:09:38.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.532+0000 7f9a00abf640 1 -- start start 2024-09-15T22:09:38.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f9a00abf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99fc102570 0x7f99fc1a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f9a00abf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99fc102ef0 0x7f99fc1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f9a00abf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc19a760 0x7f99fc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f9a00abf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc10feb0 con 0x7f99fc102ef0 2024-09-15T22:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f9a00abf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc10fd30 con 0x7f99fc102570 2024-09-15T22:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f9a00abf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1145a0 con 0x7f99fc19a760 2024-09-15T22:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f99f9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99fc102ef0 0x7f99fc1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f99f9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99fc102ef0 0x7f99fc1a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44372/0 (socket says 172.21.15.5:44372) 2024-09-15T22:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f99f9d9b640 1 -- 172.21.15.5:0/1379832191 learned_addr learned my addr 172.21.15.5:0/1379832191 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f99fad9d640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc19a760 0x7f99fc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:38.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f99fa59c640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99fc102570 0x7f99fc1a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f99fad9d640 1 -- 172.21.15.5:0/1379832191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99fc102570 msgr2=0x7f99fc1a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f99fad9d640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99fc102570 0x7f99fc1a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f99fad9d640 1 -- 172.21.15.5:0/1379832191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99fc102ef0 msgr2=0x7f99fc1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f99fad9d640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99fc102ef0 0x7f99fc1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.533+0000 7f99fad9d640 1 -- 172.21.15.5:0/1379832191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc19b310 con 0x7f99fc19a760 2024-09-15T22:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.534+0000 7f99fad9d640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc19a760 0x7f99fc19abe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99f400e970 tx=0x7f99f400ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.534+0000 7f99eb7fe640 1 -- 172.21.15.5:0/1379832191 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f400ccc0 con 0x7f99fc19a760 2024-09-15T22:09:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.534+0000 7f9a00abf640 1 -- 172.21.15.5:0/1379832191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99fc110190 con 0x7f99fc19a760 2024-09-15T22:09:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.534+0000 7f9a00abf640 1 -- 172.21.15.5:0/1379832191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99fc110720 con 0x7f99fc19a760 2024-09-15T22:09:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.534+0000 7f99eb7fe640 1 -- 172.21.15.5:0/1379832191 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99f4005020 con 0x7f99fc19a760 2024-09-15T22:09:38.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.535+0000 7f99eb7fe640 1 -- 172.21.15.5:0/1379832191 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f400ccc0 con 0x7f99fc19a760 2024-09-15T22:09:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.537+0000 7f9a00abf640 1 -- 172.21.15.5:0/1379832191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99dc005290 con 0x7f99fc19a760 2024-09-15T22:09:38.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.537+0000 7f99eb7fe640 1 -- 172.21.15.5:0/1379832191 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f99f40047a0 con 0x7f99fc19a760 2024-09-15T22:09:38.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.538+0000 7f99eb7fe640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f99e40a38d0 0x7f99e40a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:38.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.538+0000 7f99fa59c640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f99e40a38d0 0x7f99e40a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:38.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.538+0000 7f99eb7fe640 1 -- 172.21.15.5:0/1379832191 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f99f400cac0 con 0x7f99fc19a760 2024-09-15T22:09:38.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.540+0000 7f99fa59c640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f99e40a38d0 0x7f99e40a5d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f99ec02af40 tx=0x7f99ec005d80 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:38.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.543+0000 7f99eb7fe640 1 -- 172.21.15.5:0/1379832191 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99f409c060 con 0x7f99fc19a760 2024-09-15T22:09:38.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.946+0000 7f9a00abf640 1 -- 172.21.15.5:0/1379832191 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99dc005850 con 0x7f99fc19a760 2024-09-15T22:09:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.947+0000 7f99eb7fe640 1 -- 172.21.15.5:0/1379832191 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f99f409c060 con 0x7f99fc19a760 2024-09-15T22:09:38.950 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:38.950 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:09:38.950 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:09:38.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.951+0000 7f9a00abf640 1 -- 172.21.15.5:0/1379832191 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f99e40a38d0 msgr2=0x7f99e40a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:38.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.951+0000 7f9a00abf640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f99e40a38d0 0x7f99e40a5d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f99ec02af40 tx=0x7f99ec005d80 comp rx=0 tx=0).stop 2024-09-15T22:09:38.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.951+0000 7f9a00abf640 1 -- 172.21.15.5:0/1379832191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc19a760 msgr2=0x7f99fc19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:38.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.951+0000 7f9a00abf640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc19a760 0x7f99fc19abe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99f400e970 tx=0x7f99f400ee30 comp rx=0 tx=0).stop 2024-09-15T22:09:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.953+0000 7f9a00abf640 1 -- 172.21.15.5:0/1379832191 shutdown_connections 2024-09-15T22:09:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.953+0000 7f9a00abf640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f99e40a38d0 0x7f99e40a5d90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.953+0000 7f9a00abf640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f99fc19a760 0x7f99fc19abe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:38.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.953+0000 7f9a00abf640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99fc102ef0 0x7f99fc1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.953+0000 7f9a00abf640 1 --2- 172.21.15.5:0/1379832191 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f99fc102570 0x7f99fc1a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.953+0000 7f9a00abf640 1 -- 172.21.15.5:0/1379832191 >> 172.21.15.5:0/1379832191 conn(0x7f99fc0fc7d0 msgr2=0x7f99fc1048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.953+0000 7f9a00abf640 1 -- 172.21.15.5:0/1379832191 shutdown_connections 2024-09-15T22:09:38.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:38.953+0000 7f9a00abf640 1 -- 172.21.15.5:0/1379832191 wait complete. 2024-09-15T22:09:39.298 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:38 smithi005 ceph-mon[30944]: pgmap v25: 129 pgs: 129 active+clean; 876 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 MiB/s wr, 273 op/s 2024-09-15T22:09:39.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:38 smithi099 ceph-mon[34892]: pgmap v25: 129 pgs: 129 active+clean; 876 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 MiB/s wr, 273 op/s 2024-09-15T22:09:39.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:38 smithi129 ceph-mon[35314]: pgmap v25: 129 pgs: 129 active+clean; 876 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 65 MiB/s wr, 273 op/s 2024-09-15T22:09:39.467 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.2 2024-09-15T22:09:39.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:09:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 10.0 s, 3252.5 tps, lat 30.787 ms stddev 34.623 2024-09-15T22:09:40.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:39 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1379832191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:40.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:39 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1379832191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:40.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:39 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1379832191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:41.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:40 smithi099 ceph-mon[34892]: pgmap v26: 129 pgs: 129 active+clean; 920 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 MiB/s wr, 517 op/s 2024-09-15T22:09:41.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:40 smithi129 ceph-mon[35314]: pgmap v26: 129 pgs: 129 active+clean; 920 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 MiB/s wr, 517 op/s 2024-09-15T22:09:41.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:40 smithi005 ceph-mon[30944]: pgmap v26: 129 pgs: 129 active+clean; 920 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 68 MiB/s wr, 517 op/s 2024-09-15T22:09:41.671 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:09:41.789 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.398+0000 7fa97ecc5640 1 -- 172.21.15.5:0/50318263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978102570 msgr2=0x7fa9781029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.399+0000 7fa97ecc5640 1 --2- 172.21.15.5:0/50318263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978102570 0x7fa9781029b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa964009890 tx=0x7fa96402a420 comp rx=0 tx=0).stop 2024-09-15T22:09:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.399+0000 7fa97ecc5640 1 -- 172.21.15.5:0/50318263 shutdown_connections 2024-09-15T22:09:42.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.399+0000 7fa97ecc5640 1 --2- 172.21.15.5:0/50318263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa978102ef0 0x7fa97810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.399+0000 7fa97ecc5640 1 --2- 172.21.15.5:0/50318263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978102570 0x7fa9781029b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.399+0000 7fa97ecc5640 1 --2- 172.21.15.5:0/50318263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa978100e20 0x7fa978101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.399+0000 7fa97ecc5640 1 -- 172.21.15.5:0/50318263 >> 172.21.15.5:0/50318263 conn(0x7fa9780fc7d0 msgr2=0x7fa9780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.399+0000 7fa97ecc5640 1 -- 172.21.15.5:0/50318263 shutdown_connections 2024-09-15T22:09:42.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.399+0000 7fa97ecc5640 1 -- 172.21.15.5:0/50318263 wait complete. 2024-09-15T22:09:42.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.401+0000 7fa97ecc5640 1 Processor -- start 2024-09-15T22:09:42.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.403+0000 7fa97ecc5640 1 -- start start 2024-09-15T22:09:42.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.404+0000 7fa97ecc5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978100e20 0x7fa978197420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:42.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.404+0000 7fa97ecc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa978102570 0x7fa978197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:42.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.404+0000 7fa97ca61640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978100e20 0x7fa978197420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:42.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.404+0000 7fa97ca61640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978100e20 0x7fa978197420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43444/0 (socket says 172.21.15.5:43444) 2024-09-15T22:09:42.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.404+0000 7fa97ca61640 1 -- 172.21.15.5:0/1177643128 learned_addr learned my addr 172.21.15.5:0/1177643128 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:42.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.404+0000 7fa97ecc5640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa978102ef0 0x7fa97819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:42.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.406+0000 7fa97d262640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa978102ef0 0x7fa97819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:42.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.406+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa978110fa0 con 0x7fa978102570 2024-09-15T22:09:42.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.406+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa978110e20 con 0x7fa978102ef0 2024-09-15T22:09:42.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.406+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa978111120 con 0x7fa978100e20 2024-09-15T22:09:42.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.406+0000 7fa977fff640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa978102570 0x7fa978197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:42.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.408+0000 7fa97ca61640 1 -- 172.21.15.5:0/1177643128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa978102ef0 msgr2=0x7fa97819bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:42.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.408+0000 7fa97ca61640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa978102ef0 0x7fa97819bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:42.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.408+0000 7fa97ca61640 1 -- 172.21.15.5:0/1177643128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa978102570 msgr2=0x7fa978197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:42.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.408+0000 7fa97ca61640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa978102570 0x7fa978197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:42.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.408+0000 7fa97ca61640 1 -- 172.21.15.5:0/1177643128 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97819c3d0 con 0x7fa978100e20 2024-09-15T22:09:42.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.409+0000 7fa97ca61640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978100e20 0x7fa978197420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00e9c0 tx=0x7fa96c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:42.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.410+0000 7fa975ffb640 1 -- 172.21.15.5:0/1177643128 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c00ccc0 con 0x7fa978100e20 2024-09-15T22:09:42.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.410+0000 7fa975ffb640 1 -- 172.21.15.5:0/1177643128 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa96c005500 con 0x7fa978100e20 2024-09-15T22:09:42.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.410+0000 7fa975ffb640 1 -- 172.21.15.5:0/1177643128 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96c00ccc0 con 0x7fa978100e20 2024-09-15T22:09:42.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.410+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa97819c6c0 con 0x7fa978100e20 2024-09-15T22:09:42.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.410+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa978069f70 con 0x7fa978100e20 2024-09-15T22:09:42.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.412+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa95c005290 con 0x7fa978100e20 2024-09-15T22:09:42.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.414+0000 7fa975ffb640 1 -- 172.21.15.5:0/1177643128 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7fa96c005020 con 0x7fa978100e20 2024-09-15T22:09:42.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.416+0000 7fa975ffb640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9680a38d0 0x7fa9680a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.416+0000 7fa975ffb640 1 -- 172.21.15.5:0/1177643128 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa96c00cac0 con 0x7fa978100e20 2024-09-15T22:09:42.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.416+0000 7fa977fff640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9680a38d0 0x7fa9680a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:42.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.418+0000 7fa977fff640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9680a38d0 0x7fa9680a5d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa9780fec10 tx=0x7fa964005d40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:42.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.422+0000 7fa975ffb640 1 -- 172.21.15.5:0/1177643128 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa96c004e40 con 0x7fa978100e20 2024-09-15T22:09:42.453 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:42.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.816+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa95c005850 con 0x7fa978100e20 2024-09-15T22:09:42.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.827+0000 7fa975ffb640 1 -- 172.21.15.5:0/1177643128 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fa96c004e40 con 0x7fa978100e20 2024-09-15T22:09:42.830 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:09:42.838 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:09:42.838 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:09:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.830+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9680a38d0 msgr2=0x7fa9680a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.830+0000 7fa97ecc5640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9680a38d0 0x7fa9680a5d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa9780fec10 tx=0x7fa964005d40 comp rx=0 tx=0).stop 2024-09-15T22:09:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.830+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978100e20 msgr2=0x7fa978197420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.830+0000 7fa97ecc5640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978100e20 0x7fa978197420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00e9c0 tx=0x7fa96c00ee80 comp rx=0 tx=0).stop 2024-09-15T22:09:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.830+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 shutdown_connections 2024-09-15T22:09:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.830+0000 7fa97ecc5640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9680a38d0 0x7fa9680a5d90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.830+0000 7fa97ecc5640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa978102ef0 0x7fa97819bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.830+0000 7fa97ecc5640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa978102570 0x7fa978197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.830+0000 7fa97ecc5640 1 --2- 172.21.15.5:0/1177643128 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa978100e20 0x7fa978197420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.830+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 >> 172.21.15.5:0/1177643128 conn(0x7fa9780fc7d0 msgr2=0x7fa9781048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.831+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 shutdown_connections 2024-09-15T22:09:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:42.831+0000 7fa97ecc5640 1 -- 172.21.15.5:0/1177643128 wait complete. 2024-09-15T22:09:43.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:42 smithi005 ceph-mon[30944]: pgmap v27: 129 pgs: 129 active+clean; 925 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 33 MiB/s wr, 565 op/s 2024-09-15T22:09:43.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:42 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1177643128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:43.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.064+0000 7feec0853640 1 -- 172.21.15.5:0/2013151268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feebc108da0 msgr2=0x7feebc10cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.064+0000 7feec0853640 1 --2- 172.21.15.5:0/2013151268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feebc108da0 0x7feebc10cdf0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009900 tx=0x7feeb002a410 comp rx=0 tx=0).stop 2024-09-15T22:09:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.065+0000 7feec0853640 1 -- 172.21.15.5:0/2013151268 shutdown_connections 2024-09-15T22:09:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.065+0000 7feec0853640 1 --2- 172.21.15.5:0/2013151268 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feebc10d520 0x7feebc1150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.065+0000 7feec0853640 1 --2- 172.21.15.5:0/2013151268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feebc108da0 0x7feebc10cdf0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.065+0000 7feec0853640 1 --2- 172.21.15.5:0/2013151268 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc1083f0 0x7feebc1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.065+0000 7feec0853640 1 -- 172.21.15.5:0/2013151268 >> 172.21.15.5:0/2013151268 conn(0x7feebc077af0 msgr2=0x7feebc079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.067+0000 7feec0853640 1 -- 172.21.15.5:0/2013151268 shutdown_connections 2024-09-15T22:09:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.067+0000 7feec0853640 1 -- 172.21.15.5:0/2013151268 wait complete. 2024-09-15T22:09:43.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.069+0000 7feec0853640 1 Processor -- start 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.069+0000 7feec0853640 1 -- start start 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.070+0000 7feec0853640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feebc1083f0 0x7feebc1a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.070+0000 7feec0853640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc108da0 0x7feebc1a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.070+0000 7feeba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feebc1083f0 0x7feebc1a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.070+0000 7feeba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feebc1083f0 0x7feebc1a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44406/0 (socket says 172.21.15.5:44406) 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.070+0000 7feec0853640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feebc10d520 0x7feebc19e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.070+0000 7feeba59c640 1 -- 172.21.15.5:0/2041929999 learned_addr learned my addr 172.21.15.5:0/2041929999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.071+0000 7feec0853640 1 -- 172.21.15.5:0/2041929999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feebc118540 con 0x7feebc1083f0 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.071+0000 7feec0853640 1 -- 172.21.15.5:0/2041929999 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1183c0 con 0x7feebc108da0 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.071+0000 7feebad9d640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feebc10d520 0x7feebc19e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.071+0000 7feec0853640 1 -- 172.21.15.5:0/2041929999 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1186c0 con 0x7feebc10d520 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.072+0000 7feeba59c640 1 -- 172.21.15.5:0/2041929999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feebc10d520 msgr2=0x7feebc19e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.072+0000 7feeba59c640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feebc10d520 0x7feebc19e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.072+0000 7feeba59c640 1 -- 172.21.15.5:0/2041929999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc108da0 msgr2=0x7feebc1a4b90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.072+0000 7feeba59c640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc108da0 0x7feebc1a4b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.072+0000 7feeba59c640 1 -- 172.21.15.5:0/2041929999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc19ef70 con 0x7feebc1083f0 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.072+0000 7feeba59c640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feebc1083f0 0x7feebc1a4650 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c920 tx=0x7feeac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.073+0000 7feeab7fe640 1 -- 172.21.15.5:0/2041929999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac007c70 con 0x7feebc1083f0 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.073+0000 7feeab7fe640 1 -- 172.21.15.5:0/2041929999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeac007e10 con 0x7feebc1083f0 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.073+0000 7feeab7fe640 1 -- 172.21.15.5:0/2041929999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac0188d0 con 0x7feebc1083f0 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.073+0000 7feec0853640 1 -- 172.21.15.5:0/2041929999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feebc19f260 con 0x7feebc1083f0 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.073+0000 7feec0853640 1 -- 172.21.15.5:0/2041929999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feebc1ab4b0 con 0x7feebc1083f0 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.077+0000 7feec2c2c740 1 Processor -- start 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.078+0000 7feeab7fe640 1 -- 172.21.15.5:0/2041929999 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7feeac018a70 con 0x7feebc1083f0 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.079+0000 7feec2c2c740 1 -- start start 2024-09-15T22:09:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.080+0000 7feeab7fe640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feea40a3620 0x7feea40a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.080+0000 7feec2c2c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eed9fb40 0x55e2eed9e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.080+0000 7feec2c2c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eed9e5d0 0x55e2eed9ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.080+0000 7feec2c2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 0x55e2eedab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.080+0000 7feec2c2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e2eed042f0 con 0x55e2eeda9370 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.080+0000 7feec2c2c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e2eecabcf0 con 0x55e2eed9fb40 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.080+0000 7feec2c2c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2eed002b0 con 0x55e2eed9e5d0 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.081+0000 7feebad9d640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feea40a3620 0x7feea40a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.081+0000 7feeb9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eed9e5d0 0x55e2eed9ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.081+0000 7feebad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 0x55e2eedab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.081+0000 7feebad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 0x55e2eedab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44416/0 (socket says 172.21.15.5:44416) 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.081+0000 7feebad9d640 1 -- 172.21.15.5:0/4041128269 learned_addr learned my addr 172.21.15.5:0/4041128269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.082+0000 7feebad9d640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feea40a3620 0x7feea40a5ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feeb4005fd0 tx=0x7feeb4009a70 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.082+0000 7feeba59c640 1 --2- 172.21.15.5:0/4041128269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eed9fb40 0x55e2eed9e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.081+0000 7feeab7fe640 1 -- 172.21.15.5:0/2041929999 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7feeac0d0530 con 0x7feebc1083f0 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.082+0000 7feeba59c640 1 -- 172.21.15.5:0/4041128269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eed9e5d0 msgr2=0x55e2eed9ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.082+0000 7feeba59c640 1 --2- 172.21.15.5:0/4041128269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eed9e5d0 0x55e2eed9ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.082+0000 7feeba59c640 1 -- 172.21.15.5:0/4041128269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 msgr2=0x55e2eedab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.082+0000 7feeba59c640 1 --2- 172.21.15.5:0/4041128269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 0x55e2eedab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.082+0000 7feeba59c640 1 -- 172.21.15.5:0/4041128269 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2eec903a0 con 0x55e2eed9fb40 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.083+0000 7feeb9d9b640 1 --2- 172.21.15.5:0/4041128269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eed9e5d0 0x55e2eed9ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.083+0000 7feeba59c640 1 --2- 172.21.15.5:0/4041128269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eed9fb40 0x55e2eed9e090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feeac0cd700 tx=0x7feeac09beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aa8dfab3f46de14 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.083+0000 7feea8ff9640 1 -- 172.21.15.5:0/4041128269 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac0a4eb0 con 0x55e2eed9fb40 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.083+0000 7feea8ff9640 1 -- 172.21.15.5:0/4041128269 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeac0b4030 con 0x55e2eed9fb40 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.083+0000 7feea8ff9640 1 -- 172.21.15.5:0/4041128269 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac0d0170 con 0x55e2eed9fb40 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.084+0000 7feec2c2c740 1 -- 172.21.15.5:0/4041128269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eed9fb40 msgr2=0x55e2eed9e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.084+0000 7feec2c2c740 1 --2- 172.21.15.5:0/4041128269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eed9fb40 0x55e2eed9e090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feeac0cd700 tx=0x7feeac09beb0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.084+0000 7feec2c2c740 1 -- 172.21.15.5:0/4041128269 shutdown_connections 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.084+0000 7feec2c2c740 1 --2- 172.21.15.5:0/4041128269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 0x55e2eedab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.084+0000 7feec2c2c740 1 --2- 172.21.15.5:0/4041128269 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eed9e5d0 0x55e2eed9ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.084+0000 7feec2c2c740 1 --2- 172.21.15.5:0/4041128269 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eed9fb40 0x55e2eed9e090 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7feeac0cd700 tx=0x7feeac09beb0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.084+0000 7feec2c2c740 1 -- 172.21.15.5:0/4041128269 >> 172.21.15.5:0/4041128269 conn(0x55e2eed67310 msgr2=0x55e2eed9ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.084+0000 7feec2c2c740 1 -- 172.21.15.5:0/4041128269 shutdown_connections 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.085+0000 7feec2c2c740 1 -- 172.21.15.5:0/4041128269 wait complete. 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.085+0000 7feec2c2c740 1 -- start start 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.085+0000 7feec2c2c740 1 Processor -- start 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feec2c2c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eed9e5d0 0x55e2eeda62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feec2c2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 0x55e2eeda67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feec2c2c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 0x55e2eedb5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feec2c2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e2eedac070 con 0x55e2eeda9370 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feec2c2c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e2eecbf060 con 0x55e2eeda6d30 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feec2c2c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2eedac1f0 con 0x55e2eed9e5d0 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feec2c2c740 10 client.0 _ll_register_callbacks cb 0x55e2eea128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feeb9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 0x55e2eeda67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feebad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 0x55e2eedb5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feeb9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 0x55e2eeda67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44424/0 (socket says 172.21.15.5:44424) 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feeb9d9b640 1 -- 172.21.15.5:0/2996001845 learned_addr learned my addr 172.21.15.5:0/2996001845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.086+0000 7feeba59c640 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eed9e5d0 0x55e2eeda62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.087+0000 7feebad9d640 1 -- 172.21.15.5:0/2996001845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eed9e5d0 msgr2=0x55e2eeda62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.087+0000 7feebad9d640 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eed9e5d0 0x55e2eeda62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.087+0000 7feebad9d640 1 -- 172.21.15.5:0/2996001845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 msgr2=0x55e2eeda67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.087+0000 7feebad9d640 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 0x55e2eeda67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.087+0000 7feebad9d640 1 -- 172.21.15.5:0/2996001845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2eea753c0 con 0x55e2eeda6d30 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.087+0000 7feebad9d640 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 0x55e2eedb5520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feeb4005c20 tx=0x7feeb4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f90455e6333999b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.087+0000 7feea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.087+0000 7feea1ffb640 1 -- 172.21.15.5:0/2996001845 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb402b060 con 0x55e2eeda6d30 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.087+0000 7feec2c2c740 1 -- 172.21.15.5:0/2996001845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeac0b4030 con 0x55e2eeda6d30 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.087+0000 7feea1ffb640 1 -- 172.21.15.5:0/2996001845 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7feeb4026410 con 0x55e2eeda6d30 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.088+0000 7feea1ffb640 1 -- 172.21.15.5:0/2996001845 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feeb4032050 con 0x55e2eeda6d30 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.088+0000 7feec2c2c740 10 client.25181 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.088+0000 7feec2c2c740 1 -- 172.21.15.5:0/2996001845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2eedb8230 con 0x55e2eeda6d30 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.088+0000 7feea1ffb640 1 -- 172.21.15.5:0/2996001845 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeac0b4030 con 0x55e2eeda6d30 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.089+0000 7feea1ffb640 1 -- 172.21.15.5:0/2996001845 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7feeb400f910 con 0x55e2eeda6d30 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.089+0000 7feec2c2c740 10 client.25181 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.089+0000 7feec2c2c740 10 client.25181 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:09:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.089+0000 7feec2c2c740 20 client.25181 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.089+0000 7feec2c2c740 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e2eedb9a20 0x55e2eedbbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.089+0000 7feec2c2c740 4 client.25181 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.089+0000 7feec2c2c740 1 -- 172.21.15.5:0/2996001845 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2eed11860 con 0x55e2eedb9a20 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.090+0000 7feeba59c640 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e2eedb9a20 0x55e2eedbbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.091+0000 7feeba59c640 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e2eedb9a20 0x55e2eedbbe70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31d425b1769d7038 server_cookie=56f120b68571f67 in_seq=0 out_seq=0 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.091+0000 7feea1ffb640 10 client.25181 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feea1ffb640 1 -- 172.21.15.5:0/2996001845 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2eed11860 con 0x55e2eedb9a20 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feea1ffb640 10 client.25181 handle_command_reply: tid=0 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 1 client.25181 shutdown 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 1 -- 172.21.15.5:0/2996001845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 msgr2=0x55e2eedb5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 0x55e2eedb5520 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feeb4005c20 tx=0x7feeb4004ab0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 1 -- 172.21.15.5:0/2996001845 shutdown_connections 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e2eedb9a20 0x55e2eedbbe70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 0x55e2eedb5520 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eeda9370 0x55e2eeda67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 1 --2- 172.21.15.5:0/2996001845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eed9e5d0 0x55e2eeda62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 1 -- 172.21.15.5:0/2996001845 >> 172.21.15.5:0/2996001845 conn(0x55e2eed67310 msgr2=0x55e2eeda09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feea1ffb640 0 client.25181 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 1 -- 172.21.15.5:0/2996001845 shutdown_connections 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 1 -- 172.21.15.5:0/2996001845 wait complete. 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.093+0000 7feec2c2c740 20 client.25181 trim_cache size 0 max 16384 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feec2c2c740 1 Processor -- start 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feec2c2c740 1 -- start start 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feec2c2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 0x55e2eed9ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feec2c2c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 0x55e2eedb5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feec2c2c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda9370 0x55e2eeda11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feec2c2c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b7ef0 con 0x55e2eed9e5d0 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feec2c2c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b57c0 con 0x55e2eeda6d30 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feec2c2c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b3610 con 0x55e2eeda9370 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feeba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 0x55e2eed9ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feebad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda9370 0x55e2eeda11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feebad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda9370 0x55e2eeda11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43498/0 (socket says 172.21.15.5:43498) 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feebad9d640 1 -- 172.21.15.5:0/737223456 learned_addr learned my addr 172.21.15.5:0/737223456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.117+0000 7feeb9d9b640 1 --2- 172.21.15.5:0/737223456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 0x55e2eedb5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.118+0000 7feebad9d640 1 -- 172.21.15.5:0/737223456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 msgr2=0x55e2eedb5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.118+0000 7feebad9d640 1 --2- 172.21.15.5:0/737223456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 0x55e2eedb5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.118+0000 7feebad9d640 1 -- 172.21.15.5:0/737223456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 msgr2=0x55e2eed9ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.118+0000 7feebad9d640 1 --2- 172.21.15.5:0/737223456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 0x55e2eed9ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.118+0000 7feebad9d640 1 -- 172.21.15.5:0/737223456 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2eed9b960 con 0x55e2eeda9370 2024-09-15T22:09:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.118+0000 7feeba59c640 1 --2- 172.21.15.5:0/737223456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 0x55e2eed9ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.118+0000 7feeb9d9b640 1 --2- 172.21.15.5:0/737223456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 0x55e2eedb5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.118+0000 7feebad9d640 1 --2- 172.21.15.5:0/737223456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda9370 0x55e2eeda11b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009df0 tx=0x7feeb400e520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67b0b15d63d7d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.118+0000 7feea3fff640 1 -- 172.21.15.5:0/737223456 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4009480 con 0x55e2eeda9370 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.118+0000 7feea3fff640 1 -- 172.21.15.5:0/737223456 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb400f8a0 con 0x55e2eeda9370 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.118+0000 7feea3fff640 1 -- 172.21.15.5:0/737223456 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4027d00 con 0x55e2eeda9370 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.119+0000 7feec2c2c740 1 -- 172.21.15.5:0/737223456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda9370 msgr2=0x55e2eeda11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.119+0000 7feec2c2c740 1 --2- 172.21.15.5:0/737223456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda9370 0x55e2eeda11b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009df0 tx=0x7feeb400e520 comp rx=0 tx=0).stop 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.119+0000 7feec2c2c740 1 -- 172.21.15.5:0/737223456 shutdown_connections 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.119+0000 7feec2c2c740 1 --2- 172.21.15.5:0/737223456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda9370 0x55e2eeda11b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.119+0000 7feec2c2c740 1 --2- 172.21.15.5:0/737223456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda6d30 0x55e2eedb5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.119+0000 7feec2c2c740 1 --2- 172.21.15.5:0/737223456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 0x55e2eed9ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.119+0000 7feec2c2c740 1 -- 172.21.15.5:0/737223456 >> 172.21.15.5:0/737223456 conn(0x55e2eed67310 msgr2=0x55e2eed9a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.119+0000 7feec2c2c740 1 -- 172.21.15.5:0/737223456 shutdown_connections 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.119+0000 7feec2c2c740 1 -- 172.21.15.5:0/737223456 wait complete. 2024-09-15T22:09:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.120+0000 7feec2c2c740 1 -- start start 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.120+0000 7feec2c2c740 1 Processor -- start 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.122+0000 7feec2c2c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 0x55e2eeda2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.122+0000 7feec2c2c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda6d30 0x55e2eeda31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feeba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 0x55e2eeda2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feeba59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 0x55e2eeda2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44448/0 (socket says 172.21.15.5:44448) 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feeba59c640 1 -- 172.21.15.5:0/1002840445 learned_addr learned my addr 172.21.15.5:0/1002840445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feeb9d9b640 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda6d30 0x55e2eeda31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feec2c2c740 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda9370 0x55e2eedb0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feec2c2c740 1 -- 172.21.15.5:0/1002840445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b3c80 con 0x55e2eed9e5d0 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feec2c2c740 1 -- 172.21.15.5:0/1002840445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b50f0 con 0x55e2eeda9370 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feec2c2c740 1 -- 172.21.15.5:0/1002840445 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feebc1b4350 con 0x55e2eeda6d30 2024-09-15T22:09:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feebad9d640 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda9370 0x55e2eedb0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feebad9d640 1 -- 172.21.15.5:0/1002840445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda6d30 msgr2=0x55e2eeda31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feebad9d640 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda6d30 0x55e2eeda31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feebad9d640 1 -- 172.21.15.5:0/1002840445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 msgr2=0x55e2eeda2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feebad9d640 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 0x55e2eeda2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.123+0000 7feebad9d640 1 -- 172.21.15.5:0/1002840445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2eedb09c0 con 0x55e2eeda9370 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.124+0000 7feebad9d640 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda9370 0x55e2eedb0480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feeb400ea00 tx=0x7feeb40276a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b79924386fe2607 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.124+0000 7feea1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.124+0000 7feec2c2c740 10 client.0 _ll_register_callbacks cb 0x7feeb400ea30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.124+0000 7feec2c2c740 1 -- 172.21.15.5:0/1002840445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb400f8a0 con 0x55e2eeda9370 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.124+0000 7feea1ffb640 1 -- 172.21.15.5:0/1002840445 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4009480 con 0x55e2eeda9370 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.124+0000 7feea1ffb640 1 -- 172.21.15.5:0/1002840445 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7feeb4023040 con 0x55e2eeda9370 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.125+0000 7feea1ffb640 1 -- 172.21.15.5:0/1002840445 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feeb400fa50 con 0x55e2eeda9370 2024-09-15T22:09:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.125+0000 7feec2c2c740 10 client.25187 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.125+0000 7feec2c2c740 1 -- 172.21.15.5:0/1002840445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55e2eedb18e0 con 0x55e2eeda9370 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.125+0000 7feea1ffb640 1 -- 172.21.15.5:0/1002840445 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7feeb4004a70 con 0x55e2eeda9370 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.126+0000 7feec2c2c740 10 client.25187 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.126+0000 7feec2c2c740 10 client.25187 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.126+0000 7feec2c2c740 20 client.25187 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.126+0000 7feec2c2c740 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e2eedb1e00 0x55e2eee26700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.126+0000 7feec2c2c740 4 client.25187 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.126+0000 7feec2c2c740 1 -- 172.21.15.5:0/1002840445 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7feeb400f8a0 con 0x55e2eedb1e00 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.126+0000 7feeba59c640 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e2eedb1e00 0x55e2eee26700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feeba59c640 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e2eedb1e00 0x55e2eee26700 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce67ff2561a3558d server_cookie=75999f671afb89a5 in_seq=0 out_seq=0 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feea1ffb640 10 client.25187 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feea1ffb640 1 -- 172.21.15.5:0/1002840445 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7feeb400f8a0 con 0x55e2eedb1e00 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feea1ffb640 10 client.25187 handle_command_reply: tid=0 2024-09-15T22:09:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feec2c2c740 1 client.25187 shutdown 2024-09-15T22:09:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feec2c2c740 1 -- 172.21.15.5:0/1002840445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda9370 msgr2=0x55e2eedb0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feec2c2c740 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda9370 0x55e2eedb0480 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feeb400ea00 tx=0x7feeb40276a0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.286 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:09:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feec2c2c740 1 -- 172.21.15.5:0/1002840445 shutdown_connections 2024-09-15T22:09:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feec2c2c740 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e2eedb1e00 0x55e2eee26700 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feec2c2c740 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e2eeda9370 0x55e2eedb0480 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feec2c2c740 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e2eeda6d30 0x55e2eeda31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feec2c2c740 1 --2- 172.21.15.5:0/1002840445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e2eed9e5d0 0x55e2eeda2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feea1ffb640 0 client.25187 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.127+0000 7feec2c2c740 1 -- 172.21.15.5:0/1002840445 >> 172.21.15.5:0/1002840445 conn(0x55e2eed67310 msgr2=0x55e2eeda8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.128+0000 7feec2c2c740 1 -- 172.21.15.5:0/1002840445 shutdown_connections 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.128+0000 7feec2c2c740 1 -- 172.21.15.5:0/1002840445 wait complete. 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.130+0000 7feec2c2c740 20 client.25187 trim_cache size 0 max 16384 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.135+0000 7feec0853640 1 -- 172.21.15.5:0/2041929999 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feea40a3620 msgr2=0x7feea40a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.135+0000 7feec0853640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feea40a3620 0x7feea40a5ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feeb4005fd0 tx=0x7feeb4009a70 comp rx=0 tx=0).stop 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.135+0000 7feec0853640 1 -- 172.21.15.5:0/2041929999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feebc1083f0 msgr2=0x7feebc1a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.135+0000 7feec0853640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feebc1083f0 0x7feebc1a4650 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7feeac00c920 tx=0x7feeac00cde0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.136+0000 7feec0853640 1 -- 172.21.15.5:0/2041929999 shutdown_connections 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.136+0000 7feec0853640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feea40a3620 0x7feea40a5ae0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.136+0000 7feec0853640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feebc10d520 0x7feebc19e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.136+0000 7feec0853640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feebc108da0 0x7feebc1a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.136+0000 7feec0853640 1 --2- 172.21.15.5:0/2041929999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feebc1083f0 0x7feebc1a4650 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:43.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.136+0000 7feec0853640 1 -- 172.21.15.5:0/2041929999 >> 172.21.15.5:0/2041929999 conn(0x7feebc077af0 msgr2=0x7feebc079f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:43.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.137+0000 7feec0853640 1 -- 172.21.15.5:0/2041929999 shutdown_connections 2024-09-15T22:09:43.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:43.137+0000 7feec0853640 1 -- 172.21.15.5:0/2041929999 wait complete. 2024-09-15T22:09:43.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:42 smithi099 ceph-mon[34892]: pgmap v27: 129 pgs: 129 active+clean; 925 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 33 MiB/s wr, 565 op/s 2024-09-15T22:09:43.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:42 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1177643128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:43.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:42 smithi129 ceph-mon[35314]: pgmap v27: 129 pgs: 129 active+clean; 925 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 33 MiB/s wr, 565 op/s 2024-09-15T22:09:43.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:42 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1177643128' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:09:44.093 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0' 2024-09-15T22:09:44.093 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0 2024-09-15T22:09:44.531 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:09:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 15.0 s, 3246.8 tps, lat 30.691 ms stddev 35.111 2024-09-15T22:09:45.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:44 smithi005 ceph-mon[30944]: pgmap v28: 129 pgs: 129 active+clean; 925 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 541 op/s 2024-09-15T22:09:45.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:44 smithi099 ceph-mon[34892]: pgmap v28: 129 pgs: 129 active+clean; 925 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 541 op/s 2024-09-15T22:09:45.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:44 smithi129 ceph-mon[35314]: pgmap v28: 129 pgs: 129 active+clean; 925 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 11 MiB/s wr, 541 op/s 2024-09-15T22:09:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.170+0000 7f2be4871640 1 -- 172.21.15.5:0/3145443007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be010c180 msgr2=0x7f2be010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.170+0000 7f2be4871640 1 --2- 172.21.15.5:0/3145443007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be010c180 0x7f2be010e600 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009ca0 tx=0x7f2bd802a950 comp rx=0 tx=0).stop 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.170+0000 7f2be4871640 1 -- 172.21.15.5:0/3145443007 shutdown_connections 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.170+0000 7f2be4871640 1 --2- 172.21.15.5:0/3145443007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be010c180 0x7f2be010e600 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.170+0000 7f2be4871640 1 --2- 172.21.15.5:0/3145443007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be01036c0 0x7f2be010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.170+0000 7f2be4871640 1 --2- 172.21.15.5:0/3145443007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be0102da0 0x7f2be0103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.170+0000 7f2be4871640 1 -- 172.21.15.5:0/3145443007 >> 172.21.15.5:0/3145443007 conn(0x7f2be00fc7a0 msgr2=0x7f2be00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.171+0000 7f2be4871640 1 -- 172.21.15.5:0/3145443007 shutdown_connections 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.171+0000 7f2be4871640 1 -- 172.21.15.5:0/3145443007 wait complete. 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.171+0000 7f2be4871640 1 Processor -- start 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.171+0000 7f2be4871640 1 -- start start 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.171+0000 7f2be4871640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be0102da0 0x7f2be019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2be4871640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be01036c0 0x7f2be019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2be4871640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be010c180 0x7f2be019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2be4871640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0111f40 con 0x7f2be0102da0 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2be4871640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2be0111dc0 con 0x7f2be010c180 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2be4871640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01120c0 con 0x7f2be01036c0 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bde59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be0102da0 0x7f2be019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bde59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be0102da0 0x7f2be019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38008/0 (socket says 172.21.15.5:38008) 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bde59c640 1 -- 172.21.15.5:0/2661029701 learned_addr learned my addr 172.21.15.5:0/2661029701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bddd9b640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be01036c0 0x7f2be019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bded9d640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be010c180 0x7f2be019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bde59c640 1 -- 172.21.15.5:0/2661029701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be01036c0 msgr2=0x7f2be019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bde59c640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be01036c0 0x7f2be019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bde59c640 1 -- 172.21.15.5:0/2661029701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be010c180 msgr2=0x7f2be019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bde59c640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be010c180 0x7f2be019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bde59c640 1 -- 172.21.15.5:0/2661029701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be019f5b0 con 0x7f2be0102da0 2024-09-15T22:09:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bde59c640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be0102da0 0x7f2be019a600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000e9f0 tx=0x7f2bd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bcf7fe640 1 -- 172.21.15.5:0/2661029701 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd000cd80 con 0x7f2be0102da0 2024-09-15T22:09:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2be4871640 1 -- 172.21.15.5:0/2661029701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be019f8a0 con 0x7f2be0102da0 2024-09-15T22:09:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bcf7fe640 1 -- 172.21.15.5:0/2661029701 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0010910 con 0x7f2be0102da0 2024-09-15T22:09:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2bcf7fe640 1 -- 172.21.15.5:0/2661029701 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd00188f0 con 0x7f2be0102da0 2024-09-15T22:09:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.172+0000 7f2be4871640 1 -- 172.21.15.5:0/2661029701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be01a7180 con 0x7f2be0102da0 2024-09-15T22:09:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.175+0000 7f2bcf7fe640 1 -- 172.21.15.5:0/2661029701 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f2bd0010ac0 con 0x7f2be0102da0 2024-09-15T22:09:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2be6c4a740 1 Processor -- start 2024-09-15T22:09:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2be6c4a740 1 -- start start 2024-09-15T22:09:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777241760 0x559777241b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777242080 0x55977724c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55977724c940 0x55977724ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5597771a72a0 con 0x559777241760 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55977714eb10 con 0x559777242080 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5597771a3260 con 0x55977724c940 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2bded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777241760 0x559777241b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2bded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777241760 0x559777241b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38016/0 (socket says 172.21.15.5:38016) 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2bded9d640 1 -- 172.21.15.5:0/587571625 learned_addr learned my addr 172.21.15.5:0/587571625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2bde59c640 1 --2- 172.21.15.5:0/587571625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777242080 0x55977724c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2bddd9b640 1 --2- 172.21.15.5:0/587571625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55977724c940 0x55977724ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2bded9d640 1 -- 172.21.15.5:0/587571625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55977724c940 msgr2=0x55977724ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2bded9d640 1 --2- 172.21.15.5:0/587571625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55977724c940 0x55977724ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2bded9d640 1 -- 172.21.15.5:0/587571625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777242080 msgr2=0x55977724c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2bded9d640 1 --2- 172.21.15.5:0/587571625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777242080 0x55977724c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.176+0000 7f2bded9d640 1 -- 172.21.15.5:0/587571625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559776f18b90 con 0x559777241760 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.177+0000 7f2bcf7fe640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2bc80a34d0 0x7f2bc80a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.177+0000 7f2bded9d640 1 --2- 172.21.15.5:0/587571625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777241760 0x559777241b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8030030 tx=0x7f2bd8009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43e6817a9517714 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.177+0000 7f2bde59c640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2bc80a34d0 0x7f2bc80a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.177+0000 7f2bcf7fe640 1 -- 172.21.15.5:0/2661029701 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2bd00d07b0 con 0x7f2be0102da0 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.177+0000 7f2bccff9640 1 -- 172.21.15.5:0/587571625 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8034040 con 0x559777241760 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.177+0000 7f2bccff9640 1 -- 172.21.15.5:0/587571625 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd802fb40 con 0x559777241760 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.177+0000 7f2bccff9640 1 -- 172.21.15.5:0/587571625 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8004440 con 0x559777241760 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2be6c4a740 1 -- 172.21.15.5:0/587571625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777241760 msgr2=0x559777241b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2bde59c640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2bc80a34d0 0x7f2bc80a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00cda40 tx=0x7f2bd00d6510 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/587571625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777241760 0x559777241b40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8030030 tx=0x7f2bd8009360 comp rx=0 tx=0).stop 2024-09-15T22:09:47.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2be6c4a740 1 -- 172.21.15.5:0/587571625 shutdown_connections 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/587571625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55977724c940 0x55977724ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/587571625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777242080 0x55977724c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/587571625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777241760 0x559777241b40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2be6c4a740 1 -- 172.21.15.5:0/587571625 >> 172.21.15.5:0/587571625 conn(0x55977723dcf0 msgr2=0x55977723e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2be6c4a740 1 -- 172.21.15.5:0/587571625 shutdown_connections 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2be6c4a740 1 -- 172.21.15.5:0/587571625 wait complete. 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2be6c4a740 1 -- start start 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.178+0000 7f2be6c4a740 1 Processor -- start 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 0x5597772497a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777242080 0x559777249ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55977724c940 0x55977724a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55977724f5b0 con 0x559777242080 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55977712f6e0 con 0x559777241760 2024-09-15T22:09:47.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55977724f730 con 0x55977724c940 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2be6c4a740 10 client.0 _ll_register_callbacks cb 0x559777153a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bddd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777242080 0x559777249ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bddd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777242080 0x559777249ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38022/0 (socket says 172.21.15.5:38022) 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bddd9b640 1 -- 172.21.15.5:0/473168693 learned_addr learned my addr 172.21.15.5:0/473168693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bded9d640 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 0x5597772497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bded9d640 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55977724c940 0x55977724a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bddd9b640 1 -- 172.21.15.5:0/473168693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55977724c940 msgr2=0x55977724a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bddd9b640 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55977724c940 0x55977724a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bddd9b640 1 -- 172.21.15.5:0/473168693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 msgr2=0x5597772497a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bddd9b640 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 0x5597772497a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bddd9b640 1 -- 172.21.15.5:0/473168693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559777258a20 con 0x559777242080 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bded9d640 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55977724c940 0x55977724a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bddd9b640 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777242080 0x559777249ce0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4012730 tx=0x7f2bd4012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a6c250ffa770b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:47.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/473168693 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4004050 con 0x559777242080 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/473168693 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2bd40041f0 con 0x559777242080 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.179+0000 7f2be6c4a740 1 -- 172.21.15.5:0/473168693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd802fb40 con 0x559777242080 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/473168693 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2bd40043f0 con 0x559777242080 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2be6c4a740 10 client.15585 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2be6c4a740 1 -- 172.21.15.5:0/473168693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55977725b410 con 0x559777242080 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/473168693 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd802fb40 con 0x559777242080 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/473168693 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2bd4003b30 con 0x559777242080 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2be6c4a740 10 client.15585 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2be6c4a740 10 client.15585 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2be6c4a740 20 client.15585 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55977725c8d0 0x55977725ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2be6c4a740 4 client.15585 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2be6c4a740 1 -- 172.21.15.5:0/473168693 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597771b49d0 con 0x55977725c8d0 2024-09-15T22:09:47.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.180+0000 7f2bded9d640 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55977725c8d0 0x55977725ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.181+0000 7f2bded9d640 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55977725c8d0 0x55977725ed70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37814e76b6884fa2 server_cookie=36365fad7eae2b8f in_seq=0 out_seq=0 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.181+0000 7f2bc5ffb640 10 client.15585 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.181+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/473168693 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597771b49d0 con 0x55977725c8d0 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.181+0000 7f2bc5ffb640 10 client.15585 handle_command_reply: tid=0 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.181+0000 7f2be6c4a740 1 client.15585 shutdown 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.181+0000 7f2be6c4a740 1 -- 172.21.15.5:0/473168693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777242080 msgr2=0x559777249ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.181+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777242080 0x559777249ce0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4012730 tx=0x7f2bd4012bf0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.181+0000 7f2be6c4a740 1 -- 172.21.15.5:0/473168693 shutdown_connections 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.181+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55977724c940 0x55977724a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.181+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559777242080 0x559777249ce0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.181+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55977725c8d0 0x55977725ed70 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.182+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/473168693 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 0x5597772497a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.182+0000 7f2be6c4a740 1 -- 172.21.15.5:0/473168693 >> 172.21.15.5:0/473168693 conn(0x55977723dcf0 msgr2=0x55977723ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.182+0000 7f2bc5ffb640 0 client.15585 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.182+0000 7f2be6c4a740 1 -- 172.21.15.5:0/473168693 shutdown_connections 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.182+0000 7f2be6c4a740 1 -- 172.21.15.5:0/473168693 wait complete. 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.182+0000 7f2be6c4a740 20 client.15585 trim_cache size 0 max 16384 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2be6c4a740 1 Processor -- start 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2be6c4a740 1 -- start start 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559777241760 0x5597772441c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777242080 0x55977723cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 0x559777264680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01b3b90 con 0x55977724c940 2024-09-15T22:09:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01b1460 con 0x559777242080 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01af2b0 con 0x559777241760 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2bded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 0x559777264680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2bded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 0x559777264680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38038/0 (socket says 172.21.15.5:38038) 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2bded9d640 1 -- 172.21.15.5:0/3727388712 learned_addr learned my addr 172.21.15.5:0/3727388712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2bddd9b640 1 --2- 172.21.15.5:0/3727388712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777242080 0x55977723cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2bded9d640 1 --2- 172.21.15.5:0/3727388712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559777241760 0x5597772441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.199+0000 7f2bded9d640 1 -- 172.21.15.5:0/3727388712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559777241760 msgr2=0x5597772441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2bded9d640 1 --2- 172.21.15.5:0/3727388712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559777241760 0x5597772441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2bded9d640 1 -- 172.21.15.5:0/3727388712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777242080 msgr2=0x55977723cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2bded9d640 1 --2- 172.21.15.5:0/3727388712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777242080 0x55977723cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2bded9d640 1 -- 172.21.15.5:0/3727388712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55977723eda0 con 0x55977724c940 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2bded9d640 1 --2- 172.21.15.5:0/3727388712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 0x559777264680 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80040b0 tx=0x7f2bd8046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccebfd35fef7c7f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2bc7fff640 1 -- 172.21.15.5:0/3727388712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd802fa40 con 0x55977724c940 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2bc7fff640 1 -- 172.21.15.5:0/3727388712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd802fbe0 con 0x55977724c940 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2bc7fff640 1 -- 172.21.15.5:0/3727388712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80044d0 con 0x55977724c940 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2be6c4a740 1 -- 172.21.15.5:0/3727388712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 msgr2=0x559777264680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/3727388712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 0x559777264680 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80040b0 tx=0x7f2bd8046040 comp rx=0 tx=0).stop 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2be6c4a740 1 -- 172.21.15.5:0/3727388712 shutdown_connections 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/3727388712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 0x559777264680 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/3727388712 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777242080 0x55977723cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/3727388712 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559777241760 0x5597772441c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2be6c4a740 1 -- 172.21.15.5:0/3727388712 >> 172.21.15.5:0/3727388712 conn(0x55977723dcf0 msgr2=0x55977723c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2be6c4a740 1 -- 172.21.15.5:0/3727388712 shutdown_connections 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.200+0000 7f2be6c4a740 1 -- 172.21.15.5:0/3727388712 wait complete. 2024-09-15T22:09:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.201+0000 7f2be6c4a740 1 -- start start 2024-09-15T22:09:47.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.201+0000 7f2be6c4a740 1 Processor -- start 2024-09-15T22:09:47.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.210+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 0x559777250b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.211+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559777242080 0x559777251050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.211+0000 7f2be6c4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 0x5597772c6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.211+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01af920 con 0x55977724c940 2024-09-15T22:09:47.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.211+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01b0d90 con 0x559777241760 2024-09-15T22:09:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.211+0000 7f2be6c4a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2be01afff0 con 0x559777242080 2024-09-15T22:09:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.211+0000 7f2bded9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 0x559777250b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.211+0000 7f2be6c4a740 10 client.0 _ll_register_callbacks cb 0x7f2bd4009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.211+0000 7f2bded9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 0x559777250b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:56122/0 (socket says 172.21.15.5:56122) 2024-09-15T22:09:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.211+0000 7f2bded9d640 1 -- 172.21.15.5:0/880627109 learned_addr learned my addr 172.21.15.5:0/880627109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.212+0000 7f2bddd9b640 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559777242080 0x559777251050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.214+0000 7f2bded9d640 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 0x5597772c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.215+0000 7f2bded9d640 1 -- 172.21.15.5:0/880627109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559777242080 msgr2=0x559777251050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.215+0000 7f2bded9d640 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559777242080 0x559777251050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.215+0000 7f2bded9d640 1 -- 172.21.15.5:0/880627109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 msgr2=0x5597772c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:47.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.215+0000 7f2bded9d640 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 0x5597772c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.215+0000 7f2bded9d640 1 -- 172.21.15.5:0/880627109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559777239260 con 0x559777241760 2024-09-15T22:09:47.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.216+0000 7f2bddd9b640 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559777242080 0x559777251050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:47.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.216+0000 7f2bded9d640 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 0x559777250b10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bd802ae70 tx=0x7f2bd8003e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b47b9baf6c6ae47 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:47.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.216+0000 7f2bc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:09:47.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.217+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/880627109 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8034070 con 0x559777241760 2024-09-15T22:09:47.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.217+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/880627109 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2bd8003970 con 0x559777241760 2024-09-15T22:09:47.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.218+0000 7f2be6c4a740 1 -- 172.21.15.5:0/880627109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd802fbe0 con 0x559777241760 2024-09-15T22:09:47.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.218+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/880627109 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2bd8003b70 con 0x559777241760 2024-09-15T22:09:47.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.219+0000 7f2be6c4a740 10 client.25205 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:47.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.219+0000 7f2be6c4a740 1 -- 172.21.15.5:0/880627109 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5597772c7b60 con 0x559777241760 2024-09-15T22:09:47.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.220+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/880627109 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd8004250 con 0x559777241760 2024-09-15T22:09:47.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.220+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/880627109 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2bd8056030 con 0x559777241760 2024-09-15T22:09:47.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.220+0000 7f2be6c4a740 10 client.25205 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:47.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.220+0000 7f2be6c4a740 10 client.25205 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:09:47.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.220+0000 7f2be6c4a740 20 client.25205 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:47.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.220+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5597772c81e0 0x5597772ca5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:47.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.220+0000 7f2be6c4a740 4 client.25205 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}] 2024-09-15T22:09:47.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.220+0000 7f2be6c4a740 1 -- 172.21.15.5:0/880627109 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}) -- 0x7f2bd802fbe0 con 0x5597772c81e0 2024-09-15T22:09:47.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.222+0000 7f2bddd9b640 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5597772c81e0 0x5597772ca5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:47.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.223+0000 7f2bddd9b640 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5597772c81e0 0x5597772ca5d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e50497338cd71c3 server_cookie=3b146b9e06f9b592 in_seq=0 out_seq=0 2024-09-15T22:09:47.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:46.225+0000 7f2bc5ffb640 10 client.25205 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:47.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:46 smithi099 ceph-mon[34892]: pgmap v29: 129 pgs: 129 active+clean; 935 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 843 op/s 2024-09-15T22:09:47.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:46 smithi129 ceph-mon[35314]: pgmap v29: 129 pgs: 129 active+clean; 935 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 843 op/s 2024-09-15T22:09:47.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:46 smithi005 ceph-mon[30944]: pgmap v29: 129 pgs: 129 active+clean; 935 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 14 MiB/s wr, 843 op/s 2024-09-15T22:09:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2bc5ffb640 1 -- 172.21.15.5:0/880627109 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f2bd802fbe0 con 0x5597772c81e0 2024-09-15T22:09:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2bc5ffb640 10 client.25205 handle_command_reply: tid=0 2024-09-15T22:09:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 1 client.25205 shutdown 2024-09-15T22:09:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 1 -- 172.21.15.5:0/880627109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 msgr2=0x559777250b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 0x559777250b10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2bd802ae70 tx=0x7f2bd8003e20 comp rx=0 tx=0).stop 2024-09-15T22:09:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 1 -- 172.21.15.5:0/880627109 shutdown_connections 2024-09-15T22:09:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55977724c940 0x5597772c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559777242080 0x559777251050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5597772c81e0 0x5597772ca5d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 1 --2- 172.21.15.5:0/880627109 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559777241760 0x559777250b10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2bc5ffb640 0 client.25205 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 1 -- 172.21.15.5:0/880627109 >> 172.21.15.5:0/880627109 conn(0x55977723dcf0 msgr2=0x55977723ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 1 -- 172.21.15.5:0/880627109 shutdown_connections 2024-09-15T22:09:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 1 -- 172.21.15.5:0/880627109 wait complete. 2024-09-15T22:09:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.393+0000 7f2be6c4a740 20 client.25205 trim_cache size 0 max 16384 2024-09-15T22:09:47.396 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:09:47.396 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:09:47.396 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:09:47.396 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 2, 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "a620f627": { 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 2, 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.2, 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:09:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:09:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:09:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:09:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:09:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:09:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:09:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:09:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:09:47.398 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:09:47.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 -- 172.21.15.5:0/2661029701 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2bc80a34d0 msgr2=0x7f2bc80a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2bc80a34d0 0x7f2bc80a5990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00cda40 tx=0x7f2bd00d6510 comp rx=0 tx=0).stop 2024-09-15T22:09:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 -- 172.21.15.5:0/2661029701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be0102da0 msgr2=0x7f2be019a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be0102da0 0x7f2be019a600 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2bd000e9f0 tx=0x7f2bd000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 -- 172.21.15.5:0/2661029701 shutdown_connections 2024-09-15T22:09:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2bc80a34d0 0x7f2bc80a5990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2be010c180 0x7f2be019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2be01036c0 0x7f2be019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 --2- 172.21.15.5:0/2661029701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2be0102da0 0x7f2be019a600 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 -- 172.21.15.5:0/2661029701 >> 172.21.15.5:0/2661029701 conn(0x7f2be00fc7a0 msgr2=0x7f2be0101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 -- 172.21.15.5:0/2661029701 shutdown_connections 2024-09-15T22:09:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:47.394+0000 7f2be4871640 1 -- 172.21.15.5:0/2661029701 wait complete. 2024-09-15T22:09:48.750 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a620f627', quiesce took 9.3 seconds. Will release after: 20.9 2024-09-15T22:09:49.188 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:48 smithi005 ceph-mon[30944]: pgmap v30: 129 pgs: 129 active+clean; 935 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.4 MiB/s wr, 605 op/s 2024-09-15T22:09:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.160+0000 7f16f0f1d640 1 -- 172.21.15.5:0/1551739722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec101420 msgr2=0x7f16ec10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.160+0000 7f16f0f1d640 1 --2- 172.21.15.5:0/1551739722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec101420 0x7f16ec10d950 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009930 tx=0x7f16e402a420 comp rx=0 tx=0).stop 2024-09-15T22:09:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.160+0000 7f16f0f1d640 1 -- 172.21.15.5:0/1551739722 shutdown_connections 2024-09-15T22:09:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.160+0000 7f16f0f1d640 1 --2- 172.21.15.5:0/1551739722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec101420 0x7f16ec10d950 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.160+0000 7f16f0f1d640 1 --2- 172.21.15.5:0/1551739722 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16ec100a80 0x7f16ec100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.160+0000 7f16f0f1d640 1 --2- 172.21.15.5:0/1551739722 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ec106a80 0x7f16ec106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.160+0000 7f16f0f1d640 1 -- 172.21.15.5:0/1551739722 >> 172.21.15.5:0/1551739722 conn(0x7f16ec0fc7a0 msgr2=0x7f16ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.160+0000 7f16f0f1d640 1 -- 172.21.15.5:0/1551739722 shutdown_connections 2024-09-15T22:09:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.160+0000 7f16f0f1d640 1 -- 172.21.15.5:0/1551739722 wait complete. 2024-09-15T22:09:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16f0f1d640 1 Processor -- start 2024-09-15T22:09:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16f0f1d640 1 -- start start 2024-09-15T22:09:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16f0f1d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ec100a80 0x7f16ec198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16f0f1d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16ec101420 0x7f16ec1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16f0f1d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec106a80 0x7f16ec19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16f0f1d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec111f90 con 0x7f16ec106a80 2024-09-15T22:09:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16f0f1d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec111e10 con 0x7f16ec100a80 2024-09-15T22:09:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16f0f1d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec112110 con 0x7f16ec101420 2024-09-15T22:09:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec106a80 0x7f16ec19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16ead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec106a80 0x7f16ec19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38062/0 (socket says 172.21.15.5:38062) 2024-09-15T22:09:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16ead9d640 1 -- 172.21.15.5:0/200640798 learned_addr learned my addr 172.21.15.5:0/200640798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16ec101420 0x7f16ec1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16ea59c640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ec100a80 0x7f16ec198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16ead9d640 1 -- 172.21.15.5:0/200640798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16ec101420 msgr2=0x7f16ec1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16ead9d640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16ec101420 0x7f16ec1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16ead9d640 1 -- 172.21.15.5:0/200640798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ec100a80 msgr2=0x7f16ec198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16ead9d640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ec100a80 0x7f16ec198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.161+0000 7f16ead9d640 1 -- 172.21.15.5:0/200640798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ec19d340 con 0x7f16ec106a80 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.162+0000 7f16ead9d640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec106a80 0x7f16ec19cc60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f16e4005ab0 tx=0x7f16e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.162+0000 7f16db7fe640 1 -- 172.21.15.5:0/200640798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e403f030 con 0x7f16ec106a80 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.162+0000 7f16db7fe640 1 -- 172.21.15.5:0/200640798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e4003ce0 con 0x7f16ec106a80 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.162+0000 7f16f0f1d640 1 -- 172.21.15.5:0/200640798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ec19d5d0 con 0x7f16ec106a80 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.162+0000 7f16db7fe640 1 -- 172.21.15.5:0/200640798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4035070 con 0x7f16ec106a80 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.162+0000 7f16f0f1d640 1 -- 172.21.15.5:0/200640798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ec1a4f10 con 0x7f16ec106a80 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.164+0000 7f16db7fe640 1 -- 172.21.15.5:0/200640798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f16e4042030 con 0x7f16ec106a80 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.164+0000 7f16f32f6740 1 Processor -- start 2024-09-15T22:09:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.164+0000 7f16f32f6740 1 -- start start 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cd5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9cd5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9ce0370 0x5606c9ce2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16f32f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5606c9c3b2f0 con 0x5606c9cd6b40 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16f32f6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5606c9be2cf0 con 0x5606c9cd55d0 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16f32f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5606c9c372b0 con 0x5606c9ce0370 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cd5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16db7fe640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f16d40a34d0 0x7f16d40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cd5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38066/0 (socket says 172.21.15.5:38066) 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16e9d9b640 1 -- 172.21.15.5:0/2355107219 learned_addr learned my addr 172.21.15.5:0/2355107219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16ea59c640 1 --2- 172.21.15.5:0/2355107219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9cd5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/2355107219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9ce0370 0x5606c9ce2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16ead9d640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f16d40a34d0 0x7f16d40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16e9d9b640 1 -- 172.21.15.5:0/2355107219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9ce0370 msgr2=0x5606c9ce2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/2355107219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9ce0370 0x5606c9ce2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16e9d9b640 1 -- 172.21.15.5:0/2355107219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 msgr2=0x5606c9cd5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/2355107219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9cd5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16e9d9b640 1 -- 172.21.15.5:0/2355107219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c9bc73a0 con 0x5606c9cd6b40 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16ea59c640 1 --2- 172.21.15.5:0/2355107219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9cd5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16db7fe640 1 -- 172.21.15.5:0/200640798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f16e40f06b0 con 0x7f16ec106a80 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.165+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/2355107219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cd5090 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f16e0004650 tx=0x7f16e0004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da5eaa2f9079c2e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.166+0000 7f16d8ff9640 1 -- 172.21.15.5:0/2355107219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e000be60 con 0x5606c9cd6b40 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.166+0000 7f16ead9d640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f16d40a34d0 0x7f16d40a5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f16e40ed8e0 tx=0x7f16e4009260 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.166+0000 7f16d8ff9640 1 -- 172.21.15.5:0/2355107219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0005500 con 0x5606c9cd6b40 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.166+0000 7f16d8ff9640 1 -- 172.21.15.5:0/2355107219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e00106c0 con 0x5606c9cd6b40 2024-09-15T22:09:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.166+0000 7f16f32f6740 1 -- 172.21.15.5:0/2355107219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 msgr2=0x5606c9cd5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.166+0000 7f16f32f6740 1 --2- 172.21.15.5:0/2355107219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cd5090 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f16e0004650 tx=0x7f16e0004250 comp rx=0 tx=0).stop 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.166+0000 7f16f32f6740 1 -- 172.21.15.5:0/2355107219 shutdown_connections 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.166+0000 7f16f32f6740 1 --2- 172.21.15.5:0/2355107219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9ce0370 0x5606c9ce2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.166+0000 7f16f32f6740 1 --2- 172.21.15.5:0/2355107219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9cd5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.166+0000 7f16f32f6740 1 --2- 172.21.15.5:0/2355107219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cd5090 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.166+0000 7f16f32f6740 1 -- 172.21.15.5:0/2355107219 >> 172.21.15.5:0/2355107219 conn(0x5606c9c9e310 msgr2=0x5606c9cd1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16f32f6740 1 -- 172.21.15.5:0/2355107219 shutdown_connections 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16f32f6740 1 -- 172.21.15.5:0/2355107219 wait complete. 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16f32f6740 1 -- start start 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16f32f6740 1 Processor -- start 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9cdfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cdcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9ce0370 0x5606c9cdd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16f32f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5606c9ce3010 con 0x5606c9cd6b40 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16f32f6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5606c9bf6060 con 0x5606c9cd55d0 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16f32f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5606c9ce3190 con 0x5606c9ce0370 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16f32f6740 10 client.0 _ll_register_callbacks cb 0x5606c99498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16ea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cdcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16ea59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cdcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38070/0 (socket says 172.21.15.5:38070) 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16ea59c640 1 -- 172.21.15.5:0/1599913253 learned_addr learned my addr 172.21.15.5:0/1599913253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.167+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9cdfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9ce0370 0x5606c9cdd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16ea59c640 1 -- 172.21.15.5:0/1599913253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9ce0370 msgr2=0x5606c9cdd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16ea59c640 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9ce0370 0x5606c9cdd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16ea59c640 1 -- 172.21.15.5:0/1599913253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 msgr2=0x5606c9cdfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16ea59c640 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9cdfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16ea59c640 1 -- 172.21.15.5:0/1599913253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c99ac3c0 con 0x5606c9cd6b40 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9cdfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16ea59c640 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cdcf70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f16dc011ae0 tx=0x7f16dc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b09ae4684fb33bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16d1ffb640 1 -- 172.21.15.5:0/1599913253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16dc01b3f0 con 0x5606c9cd6b40 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16d1ffb640 1 -- 172.21.15.5:0/1599913253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f16dc01b590 con 0x5606c9cd6b40 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16f32f6740 1 -- 172.21.15.5:0/1599913253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e0005500 con 0x5606c9cd6b40 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16d1ffb640 1 -- 172.21.15.5:0/1599913253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16dc026050 con 0x5606c9cd6b40 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16f32f6740 10 client.15603 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.168+0000 7f16f32f6740 1 -- 172.21.15.5:0/1599913253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5606c9ceeab0 con 0x5606c9cd6b40 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.169+0000 7f16d1ffb640 1 -- 172.21.15.5:0/1599913253 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0005500 con 0x5606c9cd6b40 2024-09-15T22:09:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.169+0000 7f16d1ffb640 1 -- 172.21.15.5:0/1599913253 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f16dc01a970 con 0x5606c9cd6b40 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.169+0000 7f16f32f6740 10 client.15603 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.169+0000 7f16f32f6740 10 client.15603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.169+0000 7f16f32f6740 20 client.15603 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.169+0000 7f16f32f6740 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5606c9cf0010 0x5606c9cf2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.169+0000 7f16f32f6740 4 client.15603 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.169+0000 7f16f32f6740 1 -- 172.21.15.5:0/1599913253 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606c9c48860 con 0x5606c9cf0010 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.169+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5606c9cf0010 0x5606c9cf2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.170+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5606c9cf0010 0x5606c9cf2430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc9139cb5290a5f0 server_cookie=254b5d8f58a9e1e3 in_seq=0 out_seq=0 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.170+0000 7f16d1ffb640 10 client.15603 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.170+0000 7f16d1ffb640 1 -- 172.21.15.5:0/1599913253 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606c9c48860 con 0x5606c9cf0010 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.170+0000 7f16d1ffb640 10 client.15603 handle_command_reply: tid=0 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.170+0000 7f16f32f6740 1 client.15603 shutdown 2024-09-15T22:09:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.170+0000 7f16f32f6740 1 -- 172.21.15.5:0/1599913253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 msgr2=0x5606c9cdcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.170+0000 7f16f32f6740 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cdcf70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f16dc011ae0 tx=0x7f16dc00f3c0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.171+0000 7f16f32f6740 1 -- 172.21.15.5:0/1599913253 shutdown_connections 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.171+0000 7f16f32f6740 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5606c9cf0010 0x5606c9cf2430 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.171+0000 7f16f32f6740 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9ce0370 0x5606c9cdd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.171+0000 7f16f32f6740 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9cd6b40 0x5606c9cdcf70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.171+0000 7f16d1ffb640 0 client.15603 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.171+0000 7f16f32f6740 1 --2- 172.21.15.5:0/1599913253 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9cdfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.171+0000 7f16f32f6740 1 -- 172.21.15.5:0/1599913253 >> 172.21.15.5:0/1599913253 conn(0x5606c9c9e310 msgr2=0x5606c9cd7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.171+0000 7f16f32f6740 1 -- 172.21.15.5:0/1599913253 shutdown_connections 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.171+0000 7f16f32f6740 1 -- 172.21.15.5:0/1599913253 wait complete. 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.171+0000 7f16f32f6740 20 client.15603 trim_cache size 0 max 16384 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.184+0000 7f16f32f6740 1 Processor -- start 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.184+0000 7f16f32f6740 1 -- start start 2024-09-15T22:09:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.184+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9cd55d0 0x5606c9cec2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.184+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd6b40 0x5606c9cf7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.184+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 0x5606c9cf8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.184+0000 7f16f32f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1b1850 con 0x5606c9ce0370 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.184+0000 7f16f32f6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1af120 con 0x5606c9cd6b40 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.184+0000 7f16f32f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1acf70 con 0x5606c9cd55d0 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 0x5606c9cf8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16ea59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd6b40 0x5606c9cf7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 0x5606c9cf8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38082/0 (socket says 172.21.15.5:38082) 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16e9d9b640 1 -- 172.21.15.5:0/84798444 learned_addr learned my addr 172.21.15.5:0/84798444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/84798444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9cd55d0 0x5606c9cec2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16e9d9b640 1 -- 172.21.15.5:0/84798444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9cd55d0 msgr2=0x5606c9cec2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/84798444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9cd55d0 0x5606c9cec2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16e9d9b640 1 -- 172.21.15.5:0/84798444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd6b40 msgr2=0x5606c9cf7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/84798444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd6b40 0x5606c9cf7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16e9d9b640 1 -- 172.21.15.5:0/84798444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606c9cd8120 con 0x5606c9ce0370 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16ea59c640 1 --2- 172.21.15.5:0/84798444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd6b40 0x5606c9cf7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/84798444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 0x5606c9cf8240 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f16e000bb70 tx=0x7f16e000b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ad8df5f1d25343 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16d3fff640 1 -- 172.21.15.5:0/84798444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e002b040 con 0x5606c9ce0370 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16d3fff640 1 -- 172.21.15.5:0/84798444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0027070 con 0x5606c9ce0370 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.185+0000 7f16d3fff640 1 -- 172.21.15.5:0/84798444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0010990 con 0x5606c9ce0370 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.186+0000 7f16f32f6740 1 -- 172.21.15.5:0/84798444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 msgr2=0x5606c9cf8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.186+0000 7f16f32f6740 1 --2- 172.21.15.5:0/84798444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 0x5606c9cf8240 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f16e000bb70 tx=0x7f16e000b9d0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.186+0000 7f16f32f6740 1 -- 172.21.15.5:0/84798444 shutdown_connections 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.186+0000 7f16f32f6740 1 --2- 172.21.15.5:0/84798444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 0x5606c9cf8240 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.186+0000 7f16f32f6740 1 --2- 172.21.15.5:0/84798444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd6b40 0x5606c9cf7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.186+0000 7f16f32f6740 1 --2- 172.21.15.5:0/84798444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9cd55d0 0x5606c9cec2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.186+0000 7f16f32f6740 1 -- 172.21.15.5:0/84798444 >> 172.21.15.5:0/84798444 conn(0x5606c9c9e310 msgr2=0x5606c9cdc060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.186+0000 7f16f32f6740 1 -- 172.21.15.5:0/84798444 shutdown_connections 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.187+0000 7f16f32f6740 1 -- 172.21.15.5:0/84798444 wait complete. 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.187+0000 7f16f32f6740 1 -- start start 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.187+0000 7f16f32f6740 1 Processor -- start 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9ce4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9cd6b40 0x5606c9ce4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16f32f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 0x5606c9ce4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16f32f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1ad5e0 con 0x5606c9ce0370 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16f32f6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1aea50 con 0x5606c9cd55d0 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16f32f6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec1adcb0 con 0x5606c9cd6b40 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16f32f6740 10 client.0 _ll_register_callbacks cb 0x7f16dc007910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16e9d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9ce4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16e9d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9ce4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:56162/0 (socket says 172.21.15.5:56162) 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16ea59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9cd6b40 0x5606c9ce4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16e9d9b640 1 -- 172.21.15.5:0/94927932 learned_addr learned my addr 172.21.15.5:0/94927932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 0x5606c9ce4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16e9d9b640 1 -- 172.21.15.5:0/94927932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9cd6b40 msgr2=0x5606c9ce4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9cd6b40 0x5606c9ce4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16e9d9b640 1 -- 172.21.15.5:0/94927932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 msgr2=0x5606c9ce4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 0x5606c9ce4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.188+0000 7f16e9d9b640 1 -- 172.21.15.5:0/94927932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606c9d59f20 con 0x5606c9cd55d0 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.189+0000 7f16e9d9b640 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9ce4420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f16e0024040 tx=0x7f16e002f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f08240982cc9ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.189+0000 7f16d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.189+0000 7f16d1ffb640 1 -- 172.21.15.5:0/94927932 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e0041060 con 0x5606c9cd55d0 2024-09-15T22:09:49.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.189+0000 7f16d1ffb640 1 -- 172.21.15.5:0/94927932 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f16e0004ed0 con 0x5606c9cd55d0 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.189+0000 7f16f32f6740 1 -- 172.21.15.5:0/94927932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16e0027070 con 0x5606c9cd55d0 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.189+0000 7f16d1ffb640 1 -- 172.21.15.5:0/94927932 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f16e00050d0 con 0x5606c9cd55d0 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.189+0000 7f16f32f6740 10 client.25220 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.189+0000 7f16f32f6740 1 -- 172.21.15.5:0/94927932 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5606c9d5f630 con 0x5606c9cd55d0 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.190+0000 7f16d1ffb640 1 -- 172.21.15.5:0/94927932 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e0027070 con 0x5606c9cd55d0 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.190+0000 7f16d1ffb640 1 -- 172.21.15.5:0/94927932 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f16e0039040 con 0x5606c9cd55d0 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.190+0000 7f16f32f6740 10 client.25220 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.190+0000 7f16f32f6740 10 client.25220 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.190+0000 7f16f32f6740 20 client.25220 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.190+0000 7f16f32f6740 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5606c9d5f960 0x5606c9d61d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.190+0000 7f16f32f6740 4 client.25220 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.190+0000 7f16f32f6740 1 -- 172.21.15.5:0/94927932 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606c9c42d70 con 0x5606c9d5f960 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.190+0000 7f16ea59c640 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5606c9d5f960 0x5606c9d61d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:49.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16d1ffb640 10 client.25220 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16ea59c640 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5606c9d5f960 0x5606c9d61d50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d830f3678ab98c4 server_cookie=fddd195105e19af3 in_seq=0 out_seq=0 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16d1ffb640 1 -- 172.21.15.5:0/94927932 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606c9c42d70 con 0x5606c9d5f960 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16d1ffb640 10 client.25220 handle_command_reply: tid=0 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 1 client.25220 shutdown 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 1 -- 172.21.15.5:0/94927932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 msgr2=0x5606c9ce4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9ce4420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f16e0024040 tx=0x7f16e002f040 comp rx=0 tx=0).stop 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 1 -- 172.21.15.5:0/94927932 shutdown_connections 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5606c9d5f960 0x5606c9d61d50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5606c9ce0370 0x5606c9ce4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16d1ffb640 0 client.25220 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5606c9cd6b40 0x5606c9ce4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 1 --2- 172.21.15.5:0/94927932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5606c9cd55d0 0x5606c9ce4420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 1 -- 172.21.15.5:0/94927932 >> 172.21.15.5:0/94927932 conn(0x5606c9c9e310 msgr2=0x5606c9cd04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:49.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 1 -- 172.21.15.5:0/94927932 shutdown_connections 2024-09-15T22:09:49.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 1 -- 172.21.15.5:0/94927932 wait complete. 2024-09-15T22:09:49.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.191+0000 7f16f32f6740 20 client.25220 trim_cache size 0 max 16384 2024-09-15T22:09:49.210 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:09:49.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 -- 172.21.15.5:0/200640798 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f16d40a34d0 msgr2=0x7f16d40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f16d40a34d0 0x7f16d40a5990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f16e40ed8e0 tx=0x7f16e4009260 comp rx=0 tx=0).stop 2024-09-15T22:09:49.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 -- 172.21.15.5:0/200640798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec106a80 msgr2=0x7f16ec19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:49.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec106a80 0x7f16ec19cc60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f16e4005ab0 tx=0x7f16e40042f0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 -- 172.21.15.5:0/200640798 shutdown_connections 2024-09-15T22:09:49.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f16d40a34d0 0x7f16d40a5990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16ec106a80 0x7f16ec19cc60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f16ec101420 0x7f16ec1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 --2- 172.21.15.5:0/200640798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f16ec100a80 0x7f16ec198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:49.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 -- 172.21.15.5:0/200640798 >> 172.21.15.5:0/200640798 conn(0x7f16ec0fc7a0 msgr2=0x7f16ec0fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 -- 172.21.15.5:0/200640798 shutdown_connections 2024-09-15T22:09:49.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:49.192+0000 7f16f0f1d640 1 -- 172.21.15.5:0/200640798 wait complete. 2024-09-15T22:09:49.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:48 smithi099 ceph-mon[34892]: pgmap v30: 129 pgs: 129 active+clean; 935 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.4 MiB/s wr, 605 op/s 2024-09-15T22:09:49.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:48 smithi129 ceph-mon[35314]: pgmap v30: 129 pgs: 129 active+clean; 935 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.4 MiB/s wr, 605 op/s 2024-09-15T22:09:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 20.0 s, 1549.6 tps, lat 19.605 ms stddev 21.431 2024-09-15T22:09:49.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:09:50.137 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.635+0000 7fa80149c640 1 -- 172.21.15.5:0/609090721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc102da0 msgr2=0x7fa7fc103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.635+0000 7fa80149c640 1 --2- 172.21.15.5:0/609090721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc102da0 0x7fa7fc103180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009900 tx=0x7fa7ec02a480 comp rx=0 tx=0).stop 2024-09-15T22:09:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.635+0000 7fa80149c640 1 -- 172.21.15.5:0/609090721 shutdown_connections 2024-09-15T22:09:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.635+0000 7fa80149c640 1 --2- 172.21.15.5:0/609090721 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7fc10c180 0x7fa7fc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.635+0000 7fa80149c640 1 --2- 172.21.15.5:0/609090721 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc1036c0 0x7fa7fc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.635+0000 7fa80149c640 1 --2- 172.21.15.5:0/609090721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc102da0 0x7fa7fc103180 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.635+0000 7fa80149c640 1 -- 172.21.15.5:0/609090721 >> 172.21.15.5:0/609090721 conn(0x7fa7fc0fc7a0 msgr2=0x7fa7fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.635+0000 7fa80149c640 1 -- 172.21.15.5:0/609090721 shutdown_connections 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.635+0000 7fa80149c640 1 -- 172.21.15.5:0/609090721 wait complete. 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa80149c640 1 Processor -- start 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa80149c640 1 -- start start 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa80149c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc102da0 0x7fa7fc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa80149c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7fc1036c0 0x7fa7fc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa80149c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc10c180 0x7fa7fc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa80149c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc111f60 con 0x7fa7fc10c180 2024-09-15T22:09:50.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa80149c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc111de0 con 0x7fa7fc1036c0 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa80149c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1120e0 con 0x7fa7fc102da0 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa7fb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc10c180 0x7fa7fc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa7fb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc10c180 0x7fa7fc19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38110/0 (socket says 172.21.15.5:38110) 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa7fb7fe640 1 -- 172.21.15.5:0/1426661229 learned_addr learned my addr 172.21.15.5:0/1426661229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7fc1036c0 0x7fa7fc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.636+0000 7fa7faffd640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc102da0 0x7fa7fc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa7fb7fe640 1 -- 172.21.15.5:0/1426661229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc102da0 msgr2=0x7fa7fc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa7fb7fe640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc102da0 0x7fa7fc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa7fb7fe640 1 -- 172.21.15.5:0/1426661229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7fc1036c0 msgr2=0x7fa7fc1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa7fb7fe640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7fc1036c0 0x7fa7fc1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa7fb7fe640 1 -- 172.21.15.5:0/1426661229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc19d350 con 0x7fa7fc10c180 2024-09-15T22:09:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7fc1036c0 0x7fa7fc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa7fb7fe640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc10c180 0x7fa7fc19cc70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400eaa0 tx=0x7fa7f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa7f8ff9640 1 -- 172.21.15.5:0/1426661229 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f400cdb0 con 0x7fa7fc10c180 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa7f8ff9640 1 -- 172.21.15.5:0/1426661229 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f4005020 con 0x7fa7fc10c180 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa7f8ff9640 1 -- 172.21.15.5:0/1426661229 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f40116b0 con 0x7fa7fc10c180 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa80149c640 1 -- 172.21.15.5:0/1426661229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc19d640 con 0x7fa7fc10c180 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.637+0000 7fa80149c640 1 -- 172.21.15.5:0/1426661229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc1a4f20 con 0x7fa7fc10c180 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.639+0000 7fa7f8ff9640 1 -- 172.21.15.5:0/1426661229 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7fa7f4005560 con 0x7fa7fc10c180 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa803875740 1 Processor -- start 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa803875740 1 -- start start 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa803875740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cdb40 0x5647008cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa803875740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008cc5d0 0x5647008cca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa803875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 0x5647008ddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa803875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647008322f0 con 0x5647008db780 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa803875740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5647007d9cf0 con 0x5647008cc5d0 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa803875740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56470082e2b0 con 0x5647008cdb40 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa7f8ff9640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7e80a34d0 0x7fa7e80a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa7fb7fe640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7e80a34d0 0x7fa7e80a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cdb40 0x5647008cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa7faffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008cc5d0 0x5647008cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.640+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cdb40 0x5647008cc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58750/0 (socket says 172.21.15.5:58750) 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7f3fff640 1 -- 172.21.15.5:0/3368637780 learned_addr learned my addr 172.21.15.5:0/3368637780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/3368637780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 0x5647008ddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7f8ff9640 1 -- 172.21.15.5:0/1426661229 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa7f40d0610 con 0x7fa7fc10c180 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7faffd640 1 -- 172.21.15.5:0/3368637780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cdb40 msgr2=0x5647008cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:50.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7faffd640 1 --2- 172.21.15.5:0/3368637780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cdb40 0x5647008cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7faffd640 1 -- 172.21.15.5:0/3368637780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 msgr2=0x5647008ddb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7faffd640 1 --2- 172.21.15.5:0/3368637780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 0x5647008ddb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7faffd640 1 -- 172.21.15.5:0/3368637780 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647007be3a0 con 0x5647008cc5d0 2024-09-15T22:09:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7fb7fe640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7e80a34d0 0x7fa7e80a5990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40cd8a0 tx=0x7fa7f408e040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7faffd640 1 --2- 172.21.15.5:0/3368637780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008cc5d0 0x5647008cca10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009230 tx=0x7fa7ec032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b4ec7f6951b4b31 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7f17fa640 1 -- 172.21.15.5:0/3368637780 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec03f070 con 0x5647008cc5d0 2024-09-15T22:09:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7f17fa640 1 -- 172.21.15.5:0/3368637780 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ec0317b0 con 0x5647008cc5d0 2024-09-15T22:09:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.641+0000 7fa7f17fa640 1 -- 172.21.15.5:0/3368637780 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec035070 con 0x5647008cc5d0 2024-09-15T22:09:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.642+0000 7fa803875740 1 -- 172.21.15.5:0/3368637780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008cc5d0 msgr2=0x5647008cca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:50.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.642+0000 7fa803875740 1 --2- 172.21.15.5:0/3368637780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008cc5d0 0x5647008cca10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009230 tx=0x7fa7ec032ce0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.642+0000 7fa803875740 1 -- 172.21.15.5:0/3368637780 shutdown_connections 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.642+0000 7fa803875740 1 --2- 172.21.15.5:0/3368637780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 0x5647008ddb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.642+0000 7fa803875740 1 --2- 172.21.15.5:0/3368637780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008cc5d0 0x5647008cca10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.642+0000 7fa803875740 1 --2- 172.21.15.5:0/3368637780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cdb40 0x5647008cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.642+0000 7fa803875740 1 -- 172.21.15.5:0/3368637780 >> 172.21.15.5:0/3368637780 conn(0x564700895310 msgr2=0x5647008c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.642+0000 7fa803875740 1 -- 172.21.15.5:0/3368637780 shutdown_connections 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.642+0000 7fa803875740 1 -- 172.21.15.5:0/3368637780 wait complete. 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa803875740 1 -- start start 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa803875740 1 Processor -- start 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa803875740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cc5d0 0x5647008db110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa803875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008d83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa803875740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008db780 0x5647008d8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa803875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5647008de420 con 0x5647008cdb40 2024-09-15T22:09:50.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa803875740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5647007ed060 con 0x5647008db780 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa803875740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5647008de5a0 con 0x5647008cc5d0 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa803875740 10 client.0 _ll_register_callbacks cb 0x5647005408c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa7faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008d83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa7faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008d83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38134/0 (socket says 172.21.15.5:38134) 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa7faffd640 1 -- 172.21.15.5:0/3348374194 learned_addr learned my addr 172.21.15.5:0/3348374194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008db780 0x5647008d8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa7faffd640 1 -- 172.21.15.5:0/3348374194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cc5d0 msgr2=0x5647008db110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa7faffd640 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cc5d0 0x5647008db110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa7faffd640 1 -- 172.21.15.5:0/3348374194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008db780 msgr2=0x5647008d8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa7faffd640 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008db780 0x5647008d8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa7faffd640 1 -- 172.21.15.5:0/3348374194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647005a33c0 con 0x5647008cdb40 2024-09-15T22:09:50.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008db780 0x5647008d8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.643+0000 7fa7faffd640 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008d83c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec031550 tx=0x7fa7ec0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1233d0db46db0646 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa7e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa7e27fc640 1 -- 172.21.15.5:0/3348374194 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec035070 con 0x5647008cdb40 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa7e27fc640 1 -- 172.21.15.5:0/3348374194 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa7ec0035f0 con 0x5647008cdb40 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa803875740 1 -- 172.21.15.5:0/3348374194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7ec0317b0 con 0x5647008cdb40 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa7e27fc640 1 -- 172.21.15.5:0/3348374194 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa7ec055050 con 0x5647008cdb40 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa803875740 10 client.15621 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa803875740 1 -- 172.21.15.5:0/3348374194 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5647008e9e50 con 0x5647008cdb40 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa7e27fc640 1 -- 172.21.15.5:0/3348374194 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ec0317b0 con 0x5647008cdb40 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa7e27fc640 1 -- 172.21.15.5:0/3348374194 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa7ec0049a0 con 0x5647008cdb40 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa803875740 10 client.15621 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa803875740 10 client.15621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:50.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa803875740 20 client.15621 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa803875740 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5647008eb1a0 0x5647008ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa803875740 4 client.15621 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.644+0000 7fa803875740 1 -- 172.21.15.5:0/3348374194 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56470083f860 con 0x5647008eb1a0 2024-09-15T22:09:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.645+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5647008eb1a0 0x5647008ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.645+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5647008eb1a0 0x5647008ed590 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707a262c04d94856 server_cookie=c7eb25719e2544dd in_seq=0 out_seq=0 2024-09-15T22:09:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.645+0000 7fa7e27fc640 10 client.15621 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa7e27fc640 1 -- 172.21.15.5:0/3348374194 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56470083f860 con 0x5647008eb1a0 2024-09-15T22:09:50.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa7e27fc640 10 client.15621 handle_command_reply: tid=0 2024-09-15T22:09:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 1 client.15621 shutdown 2024-09-15T22:09:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 1 -- 172.21.15.5:0/3348374194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 msgr2=0x5647008d83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008d83c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec031550 tx=0x7fa7ec0315b0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 1 -- 172.21.15.5:0/3348374194 shutdown_connections 2024-09-15T22:09:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5647008eb1a0 0x5647008ed590 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008db780 0x5647008d8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008d83c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa7e27fc640 0 client.15621 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 1 --2- 172.21.15.5:0/3348374194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cc5d0 0x5647008db110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 1 -- 172.21.15.5:0/3348374194 >> 172.21.15.5:0/3348374194 conn(0x564700895310 msgr2=0x5647008ce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 1 -- 172.21.15.5:0/3348374194 shutdown_connections 2024-09-15T22:09:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 1 -- 172.21.15.5:0/3348374194 wait complete. 2024-09-15T22:09:50.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.646+0000 7fa803875740 20 client.15621 trim_cache size 0 max 16384 2024-09-15T22:09:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.659+0000 7fa803875740 1 Processor -- start 2024-09-15T22:09:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.659+0000 7fa803875740 1 -- start start 2024-09-15T22:09:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa803875740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cc5d0 0x5647008c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa803875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa803875740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008db780 0x5647008d7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa803875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1af5b0 con 0x5647008cdb40 2024-09-15T22:09:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa803875740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1ace80 con 0x5647008db780 2024-09-15T22:09:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa803875740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1aacd0 con 0x5647008cc5d0 2024-09-15T22:09:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008e76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38136/0 (socket says 172.21.15.5:38136) 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7faffd640 1 -- 172.21.15.5:0/2476634406 learned_addr learned my addr 172.21.15.5:0/2476634406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/2476634406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cc5d0 0x5647008c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7faffd640 1 -- 172.21.15.5:0/2476634406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cc5d0 msgr2=0x5647008c8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7faffd640 1 --2- 172.21.15.5:0/2476634406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cc5d0 0x5647008c8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7faffd640 1 -- 172.21.15.5:0/2476634406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008db780 msgr2=0x5647008d7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7faffd640 1 --2- 172.21.15.5:0/2476634406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008db780 0x5647008d7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7faffd640 1 -- 172.21.15.5:0/2476634406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647008cf120 con 0x5647008cdb40 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/2476634406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cc5d0 0x5647008c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7faffd640 1 --2- 172.21.15.5:0/2476634406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008e76c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec004f30 tx=0x7fa7ec02ab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2c4b7d4f727d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.660+0000 7fa7f0ff9640 1 -- 172.21.15.5:0/2476634406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec035070 con 0x5647008cdb40 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.661+0000 7fa7f0ff9640 1 -- 172.21.15.5:0/2476634406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ec04c070 con 0x5647008cdb40 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.661+0000 7fa7f0ff9640 1 -- 172.21.15.5:0/2476634406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec03f040 con 0x5647008cdb40 2024-09-15T22:09:50.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.661+0000 7fa803875740 1 -- 172.21.15.5:0/2476634406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 msgr2=0x5647008e76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.661+0000 7fa803875740 1 --2- 172.21.15.5:0/2476634406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008e76c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec004f30 tx=0x7fa7ec02ab00 comp rx=0 tx=0).stop 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.661+0000 7fa803875740 1 -- 172.21.15.5:0/2476634406 shutdown_connections 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.661+0000 7fa803875740 1 --2- 172.21.15.5:0/2476634406 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008db780 0x5647008d7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.661+0000 7fa803875740 1 --2- 172.21.15.5:0/2476634406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008cdb40 0x5647008e76c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.661+0000 7fa803875740 1 --2- 172.21.15.5:0/2476634406 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cc5d0 0x5647008c8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.661+0000 7fa803875740 1 -- 172.21.15.5:0/2476634406 >> 172.21.15.5:0/2476634406 conn(0x564700895310 msgr2=0x5647008c8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.661+0000 7fa803875740 1 -- 172.21.15.5:0/2476634406 shutdown_connections 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.661+0000 7fa803875740 1 -- 172.21.15.5:0/2476634406 wait complete. 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa803875740 1 -- start start 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa803875740 1 Processor -- start 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa803875740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008cc5d0 0x5647008e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa803875740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cdb40 0x5647008e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa803875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 0x5647008d4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa803875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1ab340 con 0x5647008db780 2024-09-15T22:09:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa803875740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1ac7b0 con 0x5647008cc5d0 2024-09-15T22:09:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa803875740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc1aba10 con 0x5647008cdb40 2024-09-15T22:09:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa803875740 10 client.0 _ll_register_callbacks cb 0x7fa7e400bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 0x5647008d4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa7f3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 0x5647008d4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38138/0 (socket says 172.21.15.5:38138) 2024-09-15T22:09:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa7f3fff640 1 -- 172.21.15.5:0/227219601 learned_addr learned my addr 172.21.15.5:0/227219601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.662+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008cc5d0 0x5647008e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7faffd640 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cdb40 0x5647008e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7f3fff640 1 -- 172.21.15.5:0/227219601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cdb40 msgr2=0x5647008e3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cdb40 0x5647008e3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7f3fff640 1 -- 172.21.15.5:0/227219601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008cc5d0 msgr2=0x5647008e3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008cc5d0 0x5647008e3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7f3fff640 1 -- 172.21.15.5:0/227219601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647008d52b0 con 0x5647008db780 2024-09-15T22:09:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7f3fff640 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 0x5647008d4c10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40131f0 tx=0x7fa7e4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d797126ca38fb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:50.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7e27fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7e27fc640 1 -- 172.21.15.5:0/227219601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4004310 con 0x5647008db780 2024-09-15T22:09:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7e27fc640 1 -- 172.21.15.5:0/227219601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa7e4005020 con 0x5647008db780 2024-09-15T22:09:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa803875740 1 -- 172.21.15.5:0/227219601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7ec04c070 con 0x5647008db780 2024-09-15T22:09:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7e27fc640 1 -- 172.21.15.5:0/227219601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa7e402d050 con 0x5647008db780 2024-09-15T22:09:50.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa803875740 10 client.15627 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa803875740 1 -- 172.21.15.5:0/227219601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564700955450 con 0x5647008db780 2024-09-15T22:09:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.663+0000 7fa7e27fc640 1 -- 172.21.15.5:0/227219601 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7ec04c070 con 0x5647008db780 2024-09-15T22:09:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa7e27fc640 1 -- 172.21.15.5:0/227219601 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa7e40035f0 con 0x5647008db780 2024-09-15T22:09:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa803875740 10 client.15627 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa803875740 10 client.15627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:50.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa803875740 20 client.15627 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:50.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa803875740 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564700955bd0 0x564700957ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:50.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa803875740 4 client.15627 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:09:50.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa803875740 1 -- 172.21.15.5:0/227219601 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564700839d70 con 0x564700955bd0 2024-09-15T22:09:50.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa7faffd640 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564700955bd0 0x564700957ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:50.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa7e27fc640 10 client.15627 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:50.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa7faffd640 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564700955bd0 0x564700957ff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4ee58dec4639ad6 server_cookie=42431bb07e929d16 in_seq=0 out_seq=0 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa7e27fc640 1 -- 172.21.15.5:0/227219601 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564700839d70 con 0x564700955bd0 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa7e27fc640 10 client.15627 handle_command_reply: tid=0 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.664+0000 7fa803875740 1 client.15627 shutdown 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa803875740 1 -- 172.21.15.5:0/227219601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 msgr2=0x5647008d4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa803875740 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 0x5647008d4c10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa7e40131f0 tx=0x7fa7e4013040 comp rx=0 tx=0).stop 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa803875740 1 -- 172.21.15.5:0/227219601 shutdown_connections 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa803875740 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5647008db780 0x5647008d4c10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa803875740 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5647008cdb40 0x5647008e3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa803875740 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564700955bd0 0x564700957ff0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa803875740 1 --2- 172.21.15.5:0/227219601 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5647008cc5d0 0x5647008e3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa7e27fc640 0 client.15627 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa803875740 1 -- 172.21.15.5:0/227219601 >> 172.21.15.5:0/227219601 conn(0x564700895310 msgr2=0x5647008d68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa803875740 1 -- 172.21.15.5:0/227219601 shutdown_connections 2024-09-15T22:09:50.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa803875740 1 -- 172.21.15.5:0/227219601 wait complete. 2024-09-15T22:09:50.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa803875740 20 client.15627 trim_cache size 0 max 16384 2024-09-15T22:09:50.679 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa80149c640 1 -- 172.21.15.5:0/1426661229 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7e80a34d0 msgr2=0x7fa7e80a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa80149c640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7e80a34d0 0x7fa7e80a5990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40cd8a0 tx=0x7fa7f408e040 comp rx=0 tx=0).stop 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa80149c640 1 -- 172.21.15.5:0/1426661229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc10c180 msgr2=0x7fa7fc19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa80149c640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc10c180 0x7fa7fc19cc70 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400eaa0 tx=0x7fa7f400ef60 comp rx=0 tx=0).stop 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa80149c640 1 -- 172.21.15.5:0/1426661229 shutdown_connections 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa80149c640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7e80a34d0 0x7fa7e80a5990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa80149c640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7fc10c180 0x7fa7fc19cc70 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa80149c640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7fc1036c0 0x7fa7fc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa80149c640 1 --2- 172.21.15.5:0/1426661229 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7fc102da0 0x7fa7fc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.665+0000 7fa80149c640 1 -- 172.21.15.5:0/1426661229 >> 172.21.15.5:0/1426661229 conn(0x7fa7fc0fc7a0 msgr2=0x7fa7fc0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.666+0000 7fa80149c640 1 -- 172.21.15.5:0/1426661229 shutdown_connections 2024-09-15T22:09:50.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:50.666+0000 7fa80149c640 1 -- 172.21.15.5:0/1426661229 wait complete. 2024-09-15T22:09:51.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:50 smithi005 ceph-mon[30944]: pgmap v31: 129 pgs: 129 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 25 MiB/s wr, 1.04k op/s 2024-09-15T22:09:51.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:50 smithi099 ceph-mon[34892]: pgmap v31: 129 pgs: 129 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 25 MiB/s wr, 1.04k op/s 2024-09-15T22:09:51.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:50 smithi129 ceph-mon[35314]: pgmap v31: 129 pgs: 129 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 25 MiB/s wr, 1.04k op/s 2024-09-15T22:09:53.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:52 smithi099 ceph-mon[34892]: pgmap v32: 129 pgs: 129 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 22 MiB/s wr, 799 op/s 2024-09-15T22:09:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:52 smithi129 ceph-mon[35314]: pgmap v32: 129 pgs: 129 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 22 MiB/s wr, 799 op/s 2024-09-15T22:09:53.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:52 smithi005 ceph-mon[30944]: pgmap v32: 129 pgs: 129 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 22 MiB/s wr, 799 op/s 2024-09-15T22:09:54.163 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:09:54.163 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:09:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 25.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:09:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.923+0000 7fe69e4ff640 1 -- 172.21.15.5:0/1674947978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe698100e20 msgr2=0x7fe698101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.923+0000 7fe69e4ff640 1 --2- 172.21.15.5:0/1674947978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe698100e20 0x7fe698101200 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe688009900 tx=0x7fe68802a480 comp rx=0 tx=0).stop 2024-09-15T22:09:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.923+0000 7fe69e4ff640 1 -- 172.21.15.5:0/1674947978 shutdown_connections 2024-09-15T22:09:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.923+0000 7fe69e4ff640 1 --2- 172.21.15.5:0/1674947978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe698102ef0 0x7fe69810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.923+0000 7fe69e4ff640 1 --2- 172.21.15.5:0/1674947978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe698102570 0x7fe6981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.923+0000 7fe69e4ff640 1 --2- 172.21.15.5:0/1674947978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe698100e20 0x7fe698101200 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.923+0000 7fe69e4ff640 1 -- 172.21.15.5:0/1674947978 >> 172.21.15.5:0/1674947978 conn(0x7fe6980fc7d0 msgr2=0x7fe6980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.923+0000 7fe69e4ff640 1 -- 172.21.15.5:0/1674947978 shutdown_connections 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.923+0000 7fe69e4ff640 1 -- 172.21.15.5:0/1674947978 wait complete. 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.924+0000 7fe69e4ff640 1 Processor -- start 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.924+0000 7fe69e4ff640 1 -- start start 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.924+0000 7fe69e4ff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe698100e20 0x7fe698199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.924+0000 7fe69e4ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe698102570 0x7fe698199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.924+0000 7fe69e4ff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe698102ef0 0x7fe6981a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.924+0000 7fe69e4ff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe698110fa0 con 0x7fe698102570 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.924+0000 7fe69e4ff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe698110e20 con 0x7fe698102ef0 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.924+0000 7fe69e4ff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe698111120 con 0x7fe698100e20 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.924+0000 7fe6977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe698102570 0x7fe698199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.924+0000 7fe6977fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe698102570 0x7fe698199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52140/0 (socket says 172.21.15.5:52140) 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.924+0000 7fe6977fe640 1 -- 172.21.15.5:0/2110169647 learned_addr learned my addr 172.21.15.5:0/2110169647 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe697fff640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe698100e20 0x7fe698199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe69ca9c640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe698102ef0 0x7fe6981a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe6977fe640 1 -- 172.21.15.5:0/2110169647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe698100e20 msgr2=0x7fe698199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe6977fe640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe698100e20 0x7fe698199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe6977fe640 1 -- 172.21.15.5:0/2110169647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe698102ef0 msgr2=0x7fe6981a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe6977fe640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe698102ef0 0x7fe6981a4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe6977fe640 1 -- 172.21.15.5:0/2110169647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6981a49d0 con 0x7fe698102570 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe6977fe640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe698102570 0x7fe698199950 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00d950 tx=0x7fe68c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe6957fa640 1 -- 172.21.15.5:0/2110169647 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c017070 con 0x7fe698102570 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe69e4ff640 1 -- 172.21.15.5:0/2110169647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6981a4ba0 con 0x7fe698102570 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe6957fa640 1 -- 172.21.15.5:0/2110169647 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68c009c30 con 0x7fe698102570 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe6957fa640 1 -- 172.21.15.5:0/2110169647 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c00f6b0 con 0x7fe698102570 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.925+0000 7fe69e4ff640 1 -- 172.21.15.5:0/2110169647 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6981a50b0 con 0x7fe698102570 2024-09-15T22:09:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.928+0000 7fe6957fa640 1 -- 172.21.15.5:0/2110169647 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7fe68c0109d0 con 0x7fe698102570 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.928+0000 7fe6a08d8740 1 Processor -- start 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.928+0000 7fe6a08d8740 1 -- start start 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.928+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5cb60 0x5624aef5cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.928+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5d480 0x5624aef67800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.928+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef67d40 0x5624aef6a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.928+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5624aeec2280 con 0x5624aef5d480 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.928+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5624aee69b10 con 0x5624aef67d40 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.928+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5624aeebe240 con 0x5624aef5cb60 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe697fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5d480 0x5624aef67800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe697fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5d480 0x5624aef67800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52148/0 (socket says 172.21.15.5:52148) 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe697fff640 1 -- 172.21.15.5:0/2266544539 learned_addr learned my addr 172.21.15.5:0/2266544539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe6977fe640 1 --2- 172.21.15.5:0/2266544539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef67d40 0x5624aef6a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe69ca9c640 1 --2- 172.21.15.5:0/2266544539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5cb60 0x5624aef5cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe697fff640 1 -- 172.21.15.5:0/2266544539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5cb60 msgr2=0x5624aef5cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe697fff640 1 --2- 172.21.15.5:0/2266544539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5cb60 0x5624aef5cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe697fff640 1 -- 172.21.15.5:0/2266544539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef67d40 msgr2=0x5624aef6a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe697fff640 1 --2- 172.21.15.5:0/2266544539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef67d40 0x5624aef6a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe697fff640 1 -- 172.21.15.5:0/2266544539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624aee761f0 con 0x5624aef5d480 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe6977fe640 1 --2- 172.21.15.5:0/2266544539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef67d40 0x5624aef6a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe6957fa640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe6800a34d0 0x7fe6800a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe69ca9c640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe6800a34d0 0x7fe6800a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe697fff640 1 --2- 172.21.15.5:0/2266544539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5d480 0x5624aef67800 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe6880314e0 tx=0x7fe688004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4cd7972e88ec17c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe6957fa640 1 -- 172.21.15.5:0/2110169647 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe68c0d0030 con 0x7fe698102570 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.929+0000 7fe67e7fc640 1 -- 172.21.15.5:0/2266544539 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688035030 con 0x5624aef5d480 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe67e7fc640 1 -- 172.21.15.5:0/2266544539 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe688005520 con 0x5624aef5d480 2024-09-15T22:09:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe67e7fc640 1 -- 172.21.15.5:0/2266544539 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688003850 con 0x5624aef5d480 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe6a08d8740 1 -- 172.21.15.5:0/2266544539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5d480 msgr2=0x5624aef67800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe69ca9c640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe6800a34d0 0x7fe6800a5990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe690009fd0 tx=0x7fe69000e040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/2266544539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5d480 0x5624aef67800 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe6880314e0 tx=0x7fe688004170 comp rx=0 tx=0).stop 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe6a08d8740 1 -- 172.21.15.5:0/2266544539 shutdown_connections 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/2266544539 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef67d40 0x5624aef6a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/2266544539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5d480 0x5624aef67800 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/2266544539 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5cb60 0x5624aef5cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe6a08d8740 1 -- 172.21.15.5:0/2266544539 >> 172.21.15.5:0/2266544539 conn(0x5624aef590f0 msgr2=0x5624aef594d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe6a08d8740 1 -- 172.21.15.5:0/2266544539 shutdown_connections 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.930+0000 7fe6a08d8740 1 -- 172.21.15.5:0/2266544539 wait complete. 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe6a08d8740 1 -- start start 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe6a08d8740 1 Processor -- start 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef64b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5d480 0x5624aef650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef67d40 0x5624aef65600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5624aef6a910 con 0x5624aef5cb60 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5624aee7cf10 con 0x5624aef67d40 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5624aef6aa90 con 0x5624aef5d480 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe6a08d8740 10 client.0 _ll_register_callbacks cb 0x5624aee45a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe697fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef64b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe697fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef64b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52162/0 (socket says 172.21.15.5:52162) 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.931+0000 7fe697fff640 1 -- 172.21.15.5:0/224272108 learned_addr learned my addr 172.21.15.5:0/224272108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe6977fe640 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5d480 0x5624aef650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe69ca9c640 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef67d40 0x5624aef65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe697fff640 1 -- 172.21.15.5:0/224272108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5d480 msgr2=0x5624aef650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe697fff640 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5d480 0x5624aef650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe697fff640 1 -- 172.21.15.5:0/224272108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef67d40 msgr2=0x5624aef65600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe697fff640 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef67d40 0x5624aef65600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe697fff640 1 -- 172.21.15.5:0/224272108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624aec336a0 con 0x5624aef5cb60 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe697fff640 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef64b80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe688006010 tx=0x7fe6880048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271e5a63d707b6af server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe6777fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe6777fe640 1 -- 172.21.15.5:0/224272108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688035030 con 0x5624aef5cb60 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe6777fe640 1 -- 172.21.15.5:0/224272108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe6880050e0 con 0x5624aef5cb60 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe6a08d8740 1 -- 172.21.15.5:0/224272108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe688005520 con 0x5624aef5cb60 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe6777fe640 1 -- 172.21.15.5:0/224272108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe6880052e0 con 0x5624aef5cb60 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe6a08d8740 10 client.15639 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.932+0000 7fe6a08d8740 1 -- 172.21.15.5:0/224272108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624aef764b0 con 0x5624aef5cb60 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.933+0000 7fe6777fe640 1 -- 172.21.15.5:0/224272108 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe688005520 con 0x5624aef5cb60 2024-09-15T22:09:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.933+0000 7fe6777fe640 1 -- 172.21.15.5:0/224272108 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe6880317b0 con 0x5624aef5cb60 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.933+0000 7fe6a08d8740 10 client.15639 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.933+0000 7fe6a08d8740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.933+0000 7fe6a08d8740 20 client.15639 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.933+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5624aef77800 0x5624aef79bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.933+0000 7fe6a08d8740 4 client.15639 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.933+0000 7fe6a08d8740 1 -- 172.21.15.5:0/224272108 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624aeecf7f0 con 0x5624aef77800 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.933+0000 7fe6977fe640 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5624aef77800 0x5624aef79bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6777fe640 10 client.15639 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6977fe640 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5624aef77800 0x5624aef79bf0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b61986933341291e server_cookie=f02952799129cc1a in_seq=0 out_seq=0 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6777fe640 1 -- 172.21.15.5:0/224272108 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624aeecf7f0 con 0x5624aef77800 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6777fe640 10 client.15639 handle_command_reply: tid=0 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6a08d8740 1 client.15639 shutdown 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6a08d8740 1 -- 172.21.15.5:0/224272108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 msgr2=0x5624aef64b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:54.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef64b80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe688006010 tx=0x7fe6880048f0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6a08d8740 1 -- 172.21.15.5:0/224272108 shutdown_connections 2024-09-15T22:09:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5624aef77800 0x5624aef79bf0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef67d40 0x5624aef65600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5d480 0x5624aef650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/224272108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef64b80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6777fe640 0 client.15639 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.934+0000 7fe6a08d8740 1 -- 172.21.15.5:0/224272108 >> 172.21.15.5:0/224272108 conn(0x5624aef590f0 msgr2=0x5624aef5edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.935+0000 7fe6a08d8740 1 -- 172.21.15.5:0/224272108 shutdown_connections 2024-09-15T22:09:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.935+0000 7fe6a08d8740 1 -- 172.21.15.5:0/224272108 wait complete. 2024-09-15T22:09:54.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.935+0000 7fe6a08d8740 20 client.15639 trim_cache size 0 max 16384 2024-09-15T22:09:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.952+0000 7fe6a08d8740 1 Processor -- start 2024-09-15T22:09:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.952+0000 7fe6a08d8740 1 -- start start 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.952+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5cb60 0x5624aef5f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.952+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef5d480 0x5624aef7f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.952+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef67d40 0x5624aef7f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.952+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981b1870 con 0x5624aef67d40 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.952+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981af140 con 0x5624aef5d480 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.952+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981acf90 con 0x5624aef5cb60 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.952+0000 7fe69ca9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef67d40 0x5624aef7f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe69ca9c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef67d40 0x5624aef7f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52166/0 (socket says 172.21.15.5:52166) 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe69ca9c640 1 -- 172.21.15.5:0/3964737523 learned_addr learned my addr 172.21.15.5:0/3964737523 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe6977fe640 1 --2- 172.21.15.5:0/3964737523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef5d480 0x5624aef7f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe697fff640 1 --2- 172.21.15.5:0/3964737523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5cb60 0x5624aef5f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe69ca9c640 1 -- 172.21.15.5:0/3964737523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5cb60 msgr2=0x5624aef5f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:54.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe69ca9c640 1 --2- 172.21.15.5:0/3964737523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5cb60 0x5624aef5f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe69ca9c640 1 -- 172.21.15.5:0/3964737523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef5d480 msgr2=0x5624aef7f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe69ca9c640 1 --2- 172.21.15.5:0/3964737523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef5d480 0x5624aef7f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe69ca9c640 1 -- 172.21.15.5:0/3964737523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624aef6aa70 con 0x5624aef67d40 2024-09-15T22:09:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe697fff640 1 --2- 172.21.15.5:0/3964737523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5cb60 0x5624aef5f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe6977fe640 1 --2- 172.21.15.5:0/3964737523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef5d480 0x5624aef7f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe69ca9c640 1 --2- 172.21.15.5:0/3964737523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef67d40 0x5624aef7f730 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe690017820 tx=0x7fe69000c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6752b5ab310e1a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe67dffb640 1 -- 172.21.15.5:0/3964737523 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690027070 con 0x5624aef67d40 2024-09-15T22:09:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe67dffb640 1 -- 172.21.15.5:0/3964737523 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6900223f0 con 0x5624aef67d40 2024-09-15T22:09:54.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.953+0000 7fe67dffb640 1 -- 172.21.15.5:0/3964737523 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6900075d0 con 0x5624aef67d40 2024-09-15T22:09:54.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.954+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3964737523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef67d40 msgr2=0x5624aef7f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:54.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.954+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/3964737523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef67d40 0x5624aef7f730 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe690017820 tx=0x7fe69000c050 comp rx=0 tx=0).stop 2024-09-15T22:09:54.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.954+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3964737523 shutdown_connections 2024-09-15T22:09:54.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.954+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/3964737523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef67d40 0x5624aef7f730 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.954+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/3964737523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef5d480 0x5624aef7f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.954+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/3964737523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef5cb60 0x5624aef5f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.954+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3964737523 >> 172.21.15.5:0/3964737523 conn(0x5624aef590f0 msgr2=0x5624aef58270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.954+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3964737523 shutdown_connections 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.954+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3964737523 wait complete. 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.954+0000 7fe6a08d8740 1 -- start start 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe6a08d8740 1 Processor -- start 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef6fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef5d480 0x5624aef70260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe6a08d8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef67d40 0x5624aef6acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981ad600 con 0x5624aef5cb60 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981aea70 con 0x5624aef5d480 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe6a08d8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981adcd0 con 0x5624aef67d40 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe6a08d8740 10 client.0 _ll_register_callbacks cb 0x7fe688004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe697fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef6fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe697fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef6fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52176/0 (socket says 172.21.15.5:52176) 2024-09-15T22:09:54.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe697fff640 1 -- 172.21.15.5:0/3424753266 learned_addr learned my addr 172.21.15.5:0/3424753266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe6977fe640 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef5d480 0x5624aef70260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe69ca9c640 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef67d40 0x5624aef6acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe697fff640 1 -- 172.21.15.5:0/3424753266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef67d40 msgr2=0x5624aef6acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe697fff640 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef67d40 0x5624aef6acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe697fff640 1 -- 172.21.15.5:0/3424753266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef5d480 msgr2=0x5624aef70260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe697fff640 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef5d480 0x5624aef70260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe697fff640 1 -- 172.21.15.5:0/3424753266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624aef58f20 con 0x5624aef5cb60 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe697fff640 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef6fd20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe6880314e0 tx=0x7fe6880050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9bef8ebaab877d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.955+0000 7fe6777fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6777fe640 1 -- 172.21.15.5:0/3424753266 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68805f060 con 0x5624aef5cb60 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6777fe640 1 -- 172.21.15.5:0/3424753266 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe688031540 con 0x5624aef5cb60 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3424753266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6900223f0 con 0x5624aef5cb60 2024-09-15T22:09:54.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6777fe640 1 -- 172.21.15.5:0/3424753266 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe688035040 con 0x5624aef5cb60 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6a08d8740 10 client.15645 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3424753266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5624aefe28a0 con 0x5624aef5cb60 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6777fe640 1 -- 172.21.15.5:0/3424753266 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6900223f0 con 0x5624aef5cb60 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6777fe640 1 -- 172.21.15.5:0/3424753266 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe68803b550 con 0x5624aef5cb60 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6a08d8740 10 client.15645 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6a08d8740 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6a08d8740 20 client.15645 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5624aefe2e80 0x5624aefe52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6a08d8740 4 client.15645 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.956+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3424753266 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5624aeec9d00 con 0x5624aefe2e80 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.957+0000 7fe6977fe640 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5624aefe2e80 0x5624aefe52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.957+0000 7fe6977fe640 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5624aefe2e80 0x5624aefe52a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1fa7f3a818669c0 server_cookie=8536e96e9b5559a1 in_seq=0 out_seq=0 2024-09-15T22:09:54.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.957+0000 7fe6777fe640 10 client.15645 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:54.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6777fe640 1 -- 172.21.15.5:0/3424753266 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5624aeec9d00 con 0x5624aefe2e80 2024-09-15T22:09:54.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6777fe640 10 client.15645 handle_command_reply: tid=0 2024-09-15T22:09:54.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6a08d8740 1 client.15645 shutdown 2024-09-15T22:09:54.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3424753266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 msgr2=0x5624aef6fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:54.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef6fd20 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe6880314e0 tx=0x7fe6880050e0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3424753266 shutdown_connections 2024-09-15T22:09:54.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5624aef67d40 0x5624aef6acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5624aefe2e80 0x5624aefe52a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5624aef5d480 0x5624aef70260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6a08d8740 1 --2- 172.21.15.5:0/3424753266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5624aef5cb60 0x5624aef6fd20 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3424753266 >> 172.21.15.5:0/3424753266 conn(0x5624aef590f0 msgr2=0x5624aef72b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:54.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.958+0000 7fe6777fe640 0 client.15645 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:54.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3424753266 shutdown_connections 2024-09-15T22:09:54.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe6a08d8740 1 -- 172.21.15.5:0/3424753266 wait complete. 2024-09-15T22:09:54.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe6a08d8740 20 client.15645 trim_cache size 0 max 16384 2024-09-15T22:09:54.965 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:09:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:09:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "edf2c829-5b1e-4c41-b709-dc0983102964", 2024-09-15T22:09:54.965 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:09:54.965 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:09:54.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe69e4ff640 1 -- 172.21.15.5:0/2110169647 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe6800a34d0 msgr2=0x7fe6800a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:54.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe69e4ff640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe6800a34d0 0x7fe6800a5990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe690009fd0 tx=0x7fe69000e040 comp rx=0 tx=0).stop 2024-09-15T22:09:54.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe69e4ff640 1 -- 172.21.15.5:0/2110169647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe698102570 msgr2=0x7fe698199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:54.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe69e4ff640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe698102570 0x7fe698199950 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe68c00d950 tx=0x7fe68c00de10 comp rx=0 tx=0).stop 2024-09-15T22:09:54.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe69e4ff640 1 -- 172.21.15.5:0/2110169647 shutdown_connections 2024-09-15T22:09:54.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe69e4ff640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe6800a34d0 0x7fe6800a5990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe69e4ff640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe698102ef0 0x7fe6981a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe69e4ff640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe698102570 0x7fe698199950 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe69e4ff640 1 --2- 172.21.15.5:0/2110169647 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe698100e20 0x7fe698199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:54.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.959+0000 7fe69e4ff640 1 -- 172.21.15.5:0/2110169647 >> 172.21.15.5:0/2110169647 conn(0x7fe6980fc7d0 msgr2=0x7fe6980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:54.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.960+0000 7fe69e4ff640 1 -- 172.21.15.5:0/2110169647 shutdown_connections 2024-09-15T22:09:54.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:54.960+0000 7fe69e4ff640 1 -- 172.21.15.5:0/2110169647 wait complete. 2024-09-15T22:09:55.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:54 smithi005 ceph-mon[30944]: pgmap v33: 129 pgs: 129 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 21 MiB/s wr, 739 op/s 2024-09-15T22:09:55.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:54 smithi099 ceph-mon[34892]: pgmap v33: 129 pgs: 129 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 21 MiB/s wr, 739 op/s 2024-09-15T22:09:55.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:54 smithi129 ceph-mon[35314]: pgmap v33: 129 pgs: 129 active+clean; 1023 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 21 MiB/s wr, 739 op/s 2024-09-15T22:09:55.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:09:55.936 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:09:56.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:55 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:09:56.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:55 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:56.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:55 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:09:56.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:55 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:09:56.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:55 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:56.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:55 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:09:56.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:55 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:09:56.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:55 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:56.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:55 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:09:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.404+0000 7f0a7de81640 1 -- 172.21.15.5:0/812673254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a78102dd0 msgr2=0x7f0a781031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.404+0000 7f0a7de81640 1 --2- 172.21.15.5:0/812673254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a78102dd0 0x7f0a781031b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009930 tx=0x7f0a6802a420 comp rx=0 tx=0).stop 2024-09-15T22:09:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.404+0000 7f0a7de81640 1 -- 172.21.15.5:0/812673254 shutdown_connections 2024-09-15T22:09:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.404+0000 7f0a7de81640 1 --2- 172.21.15.5:0/812673254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a78108300 0x7f0a7810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.404+0000 7f0a7de81640 1 --2- 172.21.15.5:0/812673254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78103780 0x7f0a78107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.404+0000 7f0a7de81640 1 --2- 172.21.15.5:0/812673254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a78102dd0 0x7f0a781031b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.404+0000 7f0a7de81640 1 -- 172.21.15.5:0/812673254 >> 172.21.15.5:0/812673254 conn(0x7f0a780fc7d0 msgr2=0x7f0a780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.404+0000 7f0a7de81640 1 -- 172.21.15.5:0/812673254 shutdown_connections 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.404+0000 7f0a7de81640 1 -- 172.21.15.5:0/812673254 wait complete. 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.404+0000 7f0a7de81640 1 Processor -- start 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a7de81640 1 -- start start 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a7de81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a78102dd0 0x7f0a78198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a7de81640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78103780 0x7f0a78198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a7de81640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a78108300 0x7f0a7819cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a7de81640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781120b0 con 0x7f0a78102dd0 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a7de81640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78111f30 con 0x7f0a78103780 2024-09-15T22:09:56.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a7de81640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a78112230 con 0x7f0a78108300 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a78102dd0 0x7f0a78198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a78102dd0 0x7f0a78198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52194/0 (socket says 172.21.15.5:52194) 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a777fe640 1 -- 172.21.15.5:0/2897989606 learned_addr learned my addr 172.21.15.5:0/2897989606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78103780 0x7f0a78198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a77fff640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a78108300 0x7f0a7819cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a777fe640 1 -- 172.21.15.5:0/2897989606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a78108300 msgr2=0x7f0a7819cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a777fe640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a78108300 0x7f0a7819cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a777fe640 1 -- 172.21.15.5:0/2897989606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78103780 msgr2=0x7f0a78198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a777fe640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78103780 0x7f0a78198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a777fe640 1 -- 172.21.15.5:0/2897989606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7819d620 con 0x7f0a78102dd0 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.405+0000 7f0a777fe640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a78102dd0 0x7f0a78198670 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009230 tx=0x7f0a68004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.406+0000 7f0a74ff9640 1 -- 172.21.15.5:0/2897989606 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6803f030 con 0x7f0a78102dd0 2024-09-15T22:09:56.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.406+0000 7f0a74ff9640 1 -- 172.21.15.5:0/2897989606 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a68003ad0 con 0x7f0a78102dd0 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.406+0000 7f0a7de81640 1 -- 172.21.15.5:0/2897989606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7819d8b0 con 0x7f0a78102dd0 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.406+0000 7f0a74ff9640 1 -- 172.21.15.5:0/2897989606 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68035070 con 0x7f0a78102dd0 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.406+0000 7f0a7de81640 1 -- 172.21.15.5:0/2897989606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a781a51f0 con 0x7f0a78102dd0 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.408+0000 7f0a74ff9640 1 -- 172.21.15.5:0/2897989606 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) ==== 138581+0+0 (secure 0 0 0) 0x7f0a68042030 con 0x7f0a78102dd0 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.408+0000 7f0a8025a740 1 Processor -- start 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.408+0000 7f0a8025a740 1 -- start start 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf108b0 0x5618abf10c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf1b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf1ba90 0x5618abf1de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a8025a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618abe722e0 con 0x5618abf111d0 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a8025a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5618abe19ce0 con 0x5618abf1ba90 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a8025a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5618abe6e2a0 con 0x5618abf108b0 2024-09-15T22:09:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a74ff9640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a600a34d0 0x7f0a600a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf1b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf1b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52204/0 (socket says 172.21.15.5:52204) 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a777fe640 1 -- 172.21.15.5:0/3008705442 learned_addr learned my addr 172.21.15.5:0/3008705442 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a77fff640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a600a34d0 0x7f0a600a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a77fff640 1 --2- 172.21.15.5:0/3008705442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf108b0 0x5618abf10c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/3008705442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf1ba90 0x5618abf1de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a74ff9640 1 -- 172.21.15.5:0/2897989606 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f0a680f0790 con 0x7f0a78102dd0 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a777fe640 1 -- 172.21.15.5:0/3008705442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf108b0 msgr2=0x5618abf10c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a777fe640 1 --2- 172.21.15.5:0/3008705442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf108b0 0x5618abf10c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a777fe640 1 -- 172.21.15.5:0/3008705442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf1ba90 msgr2=0x5618abf1de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a777fe640 1 --2- 172.21.15.5:0/3008705442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf1ba90 0x5618abf1de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a777fe640 1 -- 172.21.15.5:0/3008705442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618abdfe440 con 0x5618abf111d0 2024-09-15T22:09:56.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a77fff640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a600a34d0 0x7f0a600a5990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009fd0 tx=0x7f0a70009ca0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.409+0000 7f0a777fe640 1 --2- 172.21.15.5:0/3008705442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf1b550 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0a680ed960 tx=0x7f0a680045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d059c4cc2bd51a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.410+0000 7f0a5dffb640 1 -- 172.21.15.5:0/3008705442 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68049070 con 0x5618abf111d0 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.410+0000 7f0a5dffb640 1 -- 172.21.15.5:0/3008705442 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a680bc550 con 0x5618abf111d0 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.410+0000 7f0a5dffb640 1 -- 172.21.15.5:0/3008705442 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a680bc7b0 con 0x5618abf111d0 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.410+0000 7f0a8025a740 1 -- 172.21.15.5:0/3008705442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 msgr2=0x5618abf1b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.410+0000 7f0a8025a740 1 --2- 172.21.15.5:0/3008705442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf1b550 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0a680ed960 tx=0x7f0a680045f0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.410+0000 7f0a8025a740 1 -- 172.21.15.5:0/3008705442 shutdown_connections 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.410+0000 7f0a8025a740 1 --2- 172.21.15.5:0/3008705442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf1ba90 0x5618abf1de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.410+0000 7f0a8025a740 1 --2- 172.21.15.5:0/3008705442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf1b550 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.410+0000 7f0a8025a740 1 --2- 172.21.15.5:0/3008705442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf108b0 0x5618abf10c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.410+0000 7f0a8025a740 1 -- 172.21.15.5:0/3008705442 >> 172.21.15.5:0/3008705442 conn(0x5618abf0ce40 msgr2=0x5618abf0d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:56.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.410+0000 7f0a8025a740 1 -- 172.21.15.5:0/3008705442 shutdown_connections 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.411+0000 7f0a8025a740 1 -- 172.21.15.5:0/3008705442 wait complete. 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.411+0000 7f0a8025a740 1 -- start start 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.411+0000 7f0a8025a740 1 Processor -- start 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.411+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf108b0 0x5618abf1b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.411+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf18650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf1ba90 0x5618abf18b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a8025a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618abf1e420 con 0x5618abf111d0 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a8025a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5618abe2d050 con 0x5618abf108b0 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a8025a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5618abf1e5a0 con 0x5618abf1ba90 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a8025a740 10 client.0 _ll_register_callbacks cb 0x5618abe6d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a76ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf18650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a76ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf18650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52218/0 (socket says 172.21.15.5:52218) 2024-09-15T22:09:56.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a76ffd640 1 -- 172.21.15.5:0/1198897226 learned_addr learned my addr 172.21.15.5:0/1198897226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a777fe640 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf108b0 0x5618abf1b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a77fff640 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf1ba90 0x5618abf18b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a76ffd640 1 -- 172.21.15.5:0/1198897226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf1ba90 msgr2=0x5618abf18b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf1ba90 0x5618abf18b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a76ffd640 1 -- 172.21.15.5:0/1198897226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf108b0 msgr2=0x5618abf1b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf108b0 0x5618abf1b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a76ffd640 1 -- 172.21.15.5:0/1198897226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618abbe33c0 con 0x5618abf111d0 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a777fe640 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf108b0 0x5618abf1b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf18650 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0a780697d0 tx=0x7f0a6c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea677a2ab570d25c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1198897226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c004020 con 0x5618abf111d0 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a8025a740 1 -- 172.21.15.5:0/1198897226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a680bc550 con 0x5618abf111d0 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.412+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1198897226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f0a6c0041c0 con 0x5618abf111d0 2024-09-15T22:09:56.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1198897226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0a6c024050 con 0x5618abf111d0 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a8025a740 10 client.15657 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a8025a740 1 -- 172.21.15.5:0/1198897226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5618abf29d60 con 0x5618abf111d0 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1198897226 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a680bc550 con 0x5618abf111d0 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1198897226 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0a6c004b40 con 0x5618abf111d0 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a8025a740 10 client.15657 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a8025a740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a8025a740 20 client.15657 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5618abf2b230 0x5618abf2d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a8025a740 4 client.15657 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a8025a740 1 -- 172.21.15.5:0/1198897226 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618abe7f850 con 0x5618abf2b230 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.413+0000 7f0a777fe640 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5618abf2b230 0x5618abf2d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.414+0000 7f0a56ffd640 10 client.15657 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:56.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.414+0000 7f0a777fe640 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5618abf2b230 0x5618abf2d6f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d02c2700c4c87e7 server_cookie=d58ad116c322a12e in_seq=0 out_seq=0 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.414+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1198897226 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618abe7f850 con 0x5618abf2b230 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.414+0000 7f0a56ffd640 10 client.15657 handle_command_reply: tid=0 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 1 client.15657 shutdown 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 1 -- 172.21.15.5:0/1198897226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 msgr2=0x5618abf18650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf18650 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0a780697d0 tx=0x7f0a6c0118d0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 1 -- 172.21.15.5:0/1198897226 shutdown_connections 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf1ba90 0x5618abf18b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf18650 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5618abf2b230 0x5618abf2d6f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1198897226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf108b0 0x5618abf1b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 1 -- 172.21.15.5:0/1198897226 >> 172.21.15.5:0/1198897226 conn(0x5618abf0ce40 msgr2=0x5618abf12b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a56ffd640 0 client.15657 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 1 -- 172.21.15.5:0/1198897226 shutdown_connections 2024-09-15T22:09:56.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 1 -- 172.21.15.5:0/1198897226 wait complete. 2024-09-15T22:09:56.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.415+0000 7f0a8025a740 20 client.15657 trim_cache size 0 max 16384 2024-09-15T22:09:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.432+0000 7f0a8025a740 1 Processor -- start 2024-09-15T22:09:56.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.432+0000 7f0a8025a740 1 -- start start 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.432+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf108b0 0x5618abf275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.432+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf32fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.432+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf1ba90 0x5618abf33500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.432+0000 7f0a8025a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781af770 con 0x5618abf111d0 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.432+0000 7f0a8025a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781ad040 con 0x5618abf1ba90 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.432+0000 7f0a8025a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781aae90 con 0x5618abf108b0 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a76ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf32fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a76ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf32fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52224/0 (socket says 172.21.15.5:52224) 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a76ffd640 1 -- 172.21.15.5:0/3426340966 learned_addr learned my addr 172.21.15.5:0/3426340966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a777fe640 1 --2- 172.21.15.5:0/3426340966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf108b0 0x5618abf275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a77fff640 1 --2- 172.21.15.5:0/3426340966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf1ba90 0x5618abf33500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a76ffd640 1 -- 172.21.15.5:0/3426340966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf108b0 msgr2=0x5618abf275b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:56.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/3426340966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf108b0 0x5618abf275b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a76ffd640 1 -- 172.21.15.5:0/3426340966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf1ba90 msgr2=0x5618abf33500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/3426340966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf1ba90 0x5618abf33500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a76ffd640 1 -- 172.21.15.5:0/3426340966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618abf0cc70 con 0x5618abf111d0 2024-09-15T22:09:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/3426340966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf32fc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c001eb0 tx=0x7f0a6c010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5785b6b8bb68d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a5d7fa640 1 -- 172.21.15.5:0/3426340966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c000e60 con 0x5618abf111d0 2024-09-15T22:09:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a5d7fa640 1 -- 172.21.15.5:0/3426340966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a6c00b480 con 0x5618abf111d0 2024-09-15T22:09:56.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.433+0000 7f0a5d7fa640 1 -- 172.21.15.5:0/3426340966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c0048e0 con 0x5618abf111d0 2024-09-15T22:09:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.434+0000 7f0a8025a740 1 -- 172.21.15.5:0/3426340966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 msgr2=0x5618abf32fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:56.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.434+0000 7f0a8025a740 1 --2- 172.21.15.5:0/3426340966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf32fc0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c001eb0 tx=0x7f0a6c010890 comp rx=0 tx=0).stop 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.434+0000 7f0a8025a740 1 -- 172.21.15.5:0/3426340966 shutdown_connections 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.434+0000 7f0a8025a740 1 --2- 172.21.15.5:0/3426340966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf1ba90 0x5618abf33500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.434+0000 7f0a8025a740 1 --2- 172.21.15.5:0/3426340966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf111d0 0x5618abf32fc0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.434+0000 7f0a8025a740 1 --2- 172.21.15.5:0/3426340966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf108b0 0x5618abf275b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.434+0000 7f0a8025a740 1 -- 172.21.15.5:0/3426340966 >> 172.21.15.5:0/3426340966 conn(0x5618abf0ce40 msgr2=0x5618abf13d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.434+0000 7f0a8025a740 1 -- 172.21.15.5:0/3426340966 shutdown_connections 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.434+0000 7f0a8025a740 1 -- 172.21.15.5:0/3426340966 wait complete. 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.434+0000 7f0a8025a740 1 -- start start 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a8025a740 1 Processor -- start 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf108b0 0x5618abf23c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf111d0 0x5618abf15170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a8025a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf1ba90 0x5618abf156b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a8025a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781ab500 con 0x5618abf108b0 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a8025a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781ac970 con 0x5618abf111d0 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a8025a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a781abbd0 con 0x5618abf1ba90 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a8025a740 10 client.0 _ll_register_callbacks cb 0x7f0a6c00bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf108b0 0x5618abf23c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf108b0 0x5618abf23c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52240/0 (socket says 172.21.15.5:52240) 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a777fe640 1 -- 172.21.15.5:0/1067745027 learned_addr learned my addr 172.21.15.5:0/1067745027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a77fff640 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf1ba90 0x5618abf156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf111d0 0x5618abf15170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a777fe640 1 -- 172.21.15.5:0/1067745027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf1ba90 msgr2=0x5618abf156b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a777fe640 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf1ba90 0x5618abf156b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a777fe640 1 -- 172.21.15.5:0/1067745027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf111d0 msgr2=0x5618abf15170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:09:56.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a777fe640 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf111d0 0x5618abf15170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a777fe640 1 -- 172.21.15.5:0/1067745027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618abf15ca0 con 0x5618abf108b0 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf111d0 0x5618abf15170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a777fe640 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf108b0 0x5618abf23c10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0a6803d040 tx=0x7f0a68005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b89747cec585317 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.435+0000 7f0a56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1067745027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a680d9070 con 0x5618abf108b0 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1067745027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f0a68003c80 con 0x5618abf108b0 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a8025a740 1 -- 172.21.15.5:0/1067745027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a6c00b480 con 0x5618abf108b0 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1067745027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0a680ce050 con 0x5618abf108b0 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a8025a740 10 client.15663 fetch_fsmap learned FSMap version 25 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a8025a740 1 -- 172.21.15.5:0/1067745027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5618abf95dc0 con 0x5618abf108b0 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1067745027 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a6c00b480 con 0x5618abf108b0 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1067745027 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0a680f03a0 con 0x5618abf108b0 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a8025a740 10 client.15663 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a8025a740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:09:56.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a8025a740 20 client.15663 populate_metadata read hostname 'smithi005' 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5618abf963a0 0x5618abf98790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a8025a740 4 client.15663 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.436+0000 7f0a8025a740 1 -- 172.21.15.5:0/1067745027 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5618abe79d60 con 0x5618abf963a0 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5618abf963a0 0x5618abf98790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a76ffd640 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5618abf963a0 0x5618abf98790 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=134ea7a754878260 server_cookie=a3889a60ac7eb654 in_seq=0 out_seq=0 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a56ffd640 10 client.15663 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a56ffd640 1 -- 172.21.15.5:0/1067745027 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5618abe79d60 con 0x5618abf963a0 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a56ffd640 10 client.15663 handle_command_reply: tid=0 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a8025a740 1 client.15663 shutdown 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a8025a740 1 -- 172.21.15.5:0/1067745027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf108b0 msgr2=0x5618abf23c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf108b0 0x5618abf23c10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0a6803d040 tx=0x7f0a68005e50 comp rx=0 tx=0).stop 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a8025a740 1 -- 172.21.15.5:0/1067745027 shutdown_connections 2024-09-15T22:09:56.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618abf1ba90 0x5618abf156b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5618abf963a0 0x5618abf98790 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618abf111d0 0x5618abf15170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a56ffd640 0 client.15663 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:09:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.437+0000 7f0a8025a740 1 --2- 172.21.15.5:0/1067745027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618abf108b0 0x5618abf23c10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a8025a740 1 -- 172.21.15.5:0/1067745027 >> 172.21.15.5:0/1067745027 conn(0x5618abf0ce40 msgr2=0x5618abf12300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a8025a740 1 -- 172.21.15.5:0/1067745027 shutdown_connections 2024-09-15T22:09:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a8025a740 1 -- 172.21.15.5:0/1067745027 wait complete. 2024-09-15T22:09:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a8025a740 20 client.15663 trim_cache size 0 max 16384 2024-09-15T22:09:56.444 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:09:56.445 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:09:56.445 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:09:56.445 INFO:teuthology.orchestra.run.smithi005.stdout: "edf2c829-5b1e-4c41-b709-dc0983102964": { 2024-09-15T22:09:56.445 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:09:56.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "edf2c829-5b1e-4c41-b709-dc0983102964", 2024-09-15T22:09:56.445 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:09:56.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:09:56.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:09:56.445 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:09:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a7de81640 1 -- 172.21.15.5:0/2897989606 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a600a34d0 msgr2=0x7f0a600a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a7de81640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a600a34d0 0x7f0a600a5990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009fd0 tx=0x7f0a70009ca0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a7de81640 1 -- 172.21.15.5:0/2897989606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a78102dd0 msgr2=0x7f0a78198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:09:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a7de81640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a78102dd0 0x7f0a78198670 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009230 tx=0x7f0a68004270 comp rx=0 tx=0).stop 2024-09-15T22:09:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a7de81640 1 -- 172.21.15.5:0/2897989606 shutdown_connections 2024-09-15T22:09:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a7de81640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a600a34d0 0x7f0a600a5990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a7de81640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a78108300 0x7f0a7819cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a7de81640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a78103780 0x7f0a78198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a7de81640 1 --2- 172.21.15.5:0/2897989606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a78102dd0 0x7f0a78198670 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:09:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a7de81640 1 -- 172.21.15.5:0/2897989606 >> 172.21.15.5:0/2897989606 conn(0x7f0a780fc7d0 msgr2=0x7f0a78105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:09:56.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.438+0000 7f0a7de81640 1 -- 172.21.15.5:0/2897989606 shutdown_connections 2024-09-15T22:09:56.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:09:56.439+0000 7f0a7de81640 1 -- 172.21.15.5:0/2897989606 wait complete. 2024-09-15T22:09:57.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:56 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:57.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:56 smithi005 ceph-mon[30944]: pgmap v34: 129 pgs: 129 active+clean; 965 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 36 MiB/s wr, 895 op/s 2024-09-15T22:09:57.147 INFO:tasks.cephfs.filesystem:scrub status for tag:edf2c829-5b1e-4c41-b709-dc0983102964 - {'path': '/', 'tag': 'edf2c829-5b1e-4c41-b709-dc0983102964', 'options': 'recursive,force'} 2024-09-15T22:09:57.241 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:56 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:57.241 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:56 smithi099 ceph-mon[34892]: pgmap v34: 129 pgs: 129 active+clean; 965 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 36 MiB/s wr, 895 op/s 2024-09-15T22:09:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:56 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:09:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:56 smithi129 ceph-mon[35314]: pgmap v34: 129 pgs: 129 active+clean; 965 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 36 MiB/s wr, 895 op/s 2024-09-15T22:09:58.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:57 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:09:58.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:57 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:09:58.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:57 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:09:59.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:09:58 smithi005 ceph-mon[30944]: pgmap v35: 129 pgs: 129 active+clean; 965 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 33 MiB/s wr, 593 op/s 2024-09-15T22:09:59.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:09:58 smithi099 ceph-mon[34892]: pgmap v35: 129 pgs: 129 active+clean; 965 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 33 MiB/s wr, 593 op/s 2024-09-15T22:09:59.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:09:58 smithi129 ceph-mon[35314]: pgmap v35: 129 pgs: 129 active+clean; 965 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 33 MiB/s wr, 593 op/s 2024-09-15T22:09:59.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 30.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:10:00.606 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:10:00 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:10:00.339+0000 7f76e371a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:10:00.606 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:10:00 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:10:00.339+0000 7f76e371a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:10:00.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:10:00 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:10:00.339+0000 7f76e371a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:10:00.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:10:00 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:10:00.339+0000 7f76e371a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:10:00.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:10:00 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:10:00.339+0000 7f76e371a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-15T22:10:01.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:00 smithi099 ceph-mon[34892]: pgmap v36: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 44 MiB/s wr, 715 op/s 2024-09-15T22:10:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:00 smithi099 ceph-mon[34892]: overall HEALTH_OK 2024-09-15T22:10:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:00 smithi129 ceph-mon[35314]: pgmap v36: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 44 MiB/s wr, 715 op/s 2024-09-15T22:10:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:00 smithi129 ceph-mon[35314]: overall HEALTH_OK 2024-09-15T22:10:01.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:00 smithi005 ceph-mon[30944]: pgmap v36: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 44 MiB/s wr, 715 op/s 2024-09-15T22:10:01.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:00 smithi005 ceph-mon[30944]: overall HEALTH_OK 2024-09-15T22:10:03.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:02 smithi099 ceph-mon[34892]: pgmap v37: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 277 op/s 2024-09-15T22:10:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:02 smithi099 ceph-mon[34892]: mgrmap e25: x(active, since 68s), standbys: y, z 2024-09-15T22:10:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:02 smithi129 ceph-mon[35314]: pgmap v37: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 277 op/s 2024-09-15T22:10:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:02 smithi129 ceph-mon[35314]: mgrmap e25: x(active, since 68s), standbys: y, z 2024-09-15T22:10:03.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:02 smithi005 ceph-mon[30944]: pgmap v37: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 277 op/s 2024-09-15T22:10:03.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:02 smithi005 ceph-mon[30944]: mgrmap e25: x(active, since 68s), standbys: y, z 2024-09-15T22:10:04.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:10:04.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:10:04.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:10:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 35.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:10:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:05 smithi099 ceph-mon[34892]: pgmap v38: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 277 op/s 2024-09-15T22:10:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:05 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:05 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:05 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:05 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:05 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:10:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:05 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:10:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:05 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:05 smithi129 ceph-mon[35314]: pgmap v38: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 277 op/s 2024-09-15T22:10:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:05 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:05 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:05 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:05 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:05 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:10:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:05 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:10:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:05 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:05 smithi005 ceph-mon[30944]: pgmap v38: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 26 MiB/s wr, 277 op/s 2024-09-15T22:10:05.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:05 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:05 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:05 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:05 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:05.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:05 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:10:05.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:05 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:10:05.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:05 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:10:07.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:07 smithi099 ceph-mon[34892]: pgmap v39: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 294 op/s 2024-09-15T22:10:07.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:07 smithi129 ceph-mon[35314]: pgmap v39: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 294 op/s 2024-09-15T22:10:07.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:07 smithi005 ceph-mon[30944]: pgmap v39: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 26 MiB/s wr, 294 op/s 2024-09-15T22:10:09.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:09 smithi099 ceph-mon[34892]: pgmap v40: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 139 op/s 2024-09-15T22:10:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:09 smithi129 ceph-mon[35314]: pgmap v40: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 139 op/s 2024-09-15T22:10:09.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:09 smithi005 ceph-mon[30944]: pgmap v40: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 MiB/s wr, 139 op/s 2024-09-15T22:10:09.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:10:09.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 40.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:10:09.922 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:10.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.506+0000 7f86088a4640 1 -- 172.21.15.5:0/84844354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8604102570 msgr2=0x7f86041029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:10.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.506+0000 7f86088a4640 1 --2- 172.21.15.5:0/84844354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8604102570 0x7f86041029b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009900 tx=0x7f85f802a4a0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.506+0000 7f86088a4640 1 -- 172.21.15.5:0/84844354 shutdown_connections 2024-09-15T22:10:10.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.506+0000 7f86088a4640 1 --2- 172.21.15.5:0/84844354 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8604102ef0 0x7f860410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.506+0000 7f86088a4640 1 --2- 172.21.15.5:0/84844354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8604102570 0x7f86041029b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.506+0000 7f86088a4640 1 --2- 172.21.15.5:0/84844354 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8604100e20 0x7f8604101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.506+0000 7f86088a4640 1 -- 172.21.15.5:0/84844354 >> 172.21.15.5:0/84844354 conn(0x7f86040fc7d0 msgr2=0x7f86040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.506+0000 7f86088a4640 1 -- 172.21.15.5:0/84844354 shutdown_connections 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.506+0000 7f86088a4640 1 -- 172.21.15.5:0/84844354 wait complete. 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.507+0000 7f86088a4640 1 Processor -- start 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f86088a4640 1 -- start start 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f86088a4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8604100e20 0x7f860410e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f86088a4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8604102570 0x7f860410ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f86088a4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8604102ef0 0x7f860410ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f86088a4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86041120c0 con 0x7f8604102570 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f86088a4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8604111f40 con 0x7f8604102ef0 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f86088a4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8604112240 con 0x7f8604100e20 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f8601d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8604102570 0x7f860410ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:10.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f8601d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8604102570 0x7f860410ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56232/0 (socket says 172.21.15.5:56232) 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f8601d9b640 1 -- 172.21.15.5:0/1947697927 learned_addr learned my addr 172.21.15.5:0/1947697927 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f8602d9d640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8604102ef0 0x7f860410ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f860259c640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8604100e20 0x7f860410e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f8601d9b640 1 -- 172.21.15.5:0/1947697927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8604100e20 msgr2=0x7f860410e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f8601d9b640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8604100e20 0x7f860410e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f8601d9b640 1 -- 172.21.15.5:0/1947697927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8604102ef0 msgr2=0x7f860410ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f8601d9b640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8604102ef0 0x7f860410ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.508+0000 7f8601d9b640 1 -- 172.21.15.5:0/1947697927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86041a6aa0 con 0x7f8604102570 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.509+0000 7f8601d9b640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8604102570 0x7f860410ea10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f85f8006010 tx=0x7f85f8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.509+0000 7f85f37fe640 1 -- 172.21.15.5:0/1947697927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f803f070 con 0x7f8604102570 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.509+0000 7f86088a4640 1 -- 172.21.15.5:0/1947697927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86041a6d30 con 0x7f8604102570 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.509+0000 7f86088a4640 1 -- 172.21.15.5:0/1947697927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86041a7200 con 0x7f8604102570 2024-09-15T22:10:10.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.509+0000 7f85f37fe640 1 -- 172.21.15.5:0/1947697927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85f8031c30 con 0x7f8604102570 2024-09-15T22:10:10.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.509+0000 7f85f37fe640 1 -- 172.21.15.5:0/1947697927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8035030 con 0x7f8604102570 2024-09-15T22:10:10.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.510+0000 7f86088a4640 1 -- 172.21.15.5:0/1947697927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e4005290 con 0x7f8604102570 2024-09-15T22:10:10.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.512+0000 7f85f37fe640 1 -- 172.21.15.5:0/1947697927 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138638+0+0 (secure 0 0 0) 0x7f85f8042070 con 0x7f8604102570 2024-09-15T22:10:10.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.513+0000 7f85f37fe640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85ec0a35e0 0x7f85ec0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:10.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.514+0000 7f860259c640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85ec0a35e0 0x7f85ec0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:10.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.514+0000 7f85f37fe640 1 -- 172.21.15.5:0/1947697927 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f85f80f1240 con 0x7f8604102570 2024-09-15T22:10:10.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.514+0000 7f860259c640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85ec0a35e0 0x7f85ec0a5aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006fd0 tx=0x7f85f4008040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:10.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.516+0000 7f85f37fe640 1 -- 172.21.15.5:0/1947697927 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85f8049030 con 0x7f8604102570 2024-09-15T22:10:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.728+0000 7f86088a4640 1 -- 172.21.15.5:0/1947697927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85e4005580 con 0x7f8604102570 2024-09-15T22:10:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.729+0000 7f85f37fe640 1 -- 172.21.15.5:0/1947697927 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f85f8031de0 con 0x7f8604102570 2024-09-15T22:10:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:10:10.731 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:10:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:10:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.731+0000 7f86088a4640 1 -- 172.21.15.5:0/1947697927 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85ec0a35e0 msgr2=0x7f85ec0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.731+0000 7f86088a4640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85ec0a35e0 0x7f85ec0a5aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006fd0 tx=0x7f85f4008040 comp rx=0 tx=0).stop 2024-09-15T22:10:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.731+0000 7f86088a4640 1 -- 172.21.15.5:0/1947697927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8604102570 msgr2=0x7f860410ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.731+0000 7f86088a4640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8604102570 0x7f860410ea10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f85f8006010 tx=0x7f85f8003c10 comp rx=0 tx=0).stop 2024-09-15T22:10:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.731+0000 7f86088a4640 1 -- 172.21.15.5:0/1947697927 shutdown_connections 2024-09-15T22:10:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.731+0000 7f86088a4640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85ec0a35e0 0x7f85ec0a5aa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.731+0000 7f86088a4640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8604102ef0 0x7f860410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.731+0000 7f86088a4640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8604102570 0x7f860410ea10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.731+0000 7f86088a4640 1 --2- 172.21.15.5:0/1947697927 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8604100e20 0x7f860410e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.731+0000 7f86088a4640 1 -- 172.21.15.5:0/1947697927 >> 172.21.15.5:0/1947697927 conn(0x7f86040fc7d0 msgr2=0x7f86040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.732+0000 7f86088a4640 1 -- 172.21.15.5:0/1947697927 shutdown_connections 2024-09-15T22:10:10.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:10.732+0000 7f86088a4640 1 -- 172.21.15.5:0/1947697927 wait complete. 2024-09-15T22:10:11.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:11 smithi005 ceph-mon[30944]: pgmap v41: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 142 op/s 2024-09-15T22:10:11.233 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:11 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1947697927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:11.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:11 smithi099 ceph-mon[34892]: pgmap v41: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 142 op/s 2024-09-15T22:10:11.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:11 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1947697927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:11.348 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:11 smithi129 ceph-mon[35314]: pgmap v41: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 11 MiB/s wr, 142 op/s 2024-09-15T22:10:11.348 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:11 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1947697927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:11.570 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id a620f627 --release --await' 2024-09-15T22:10:11.570 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id a620f627 --release --await 2024-09-15T22:10:11.869 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.327+0000 7f1a4e67d640 1 -- 172.21.15.5:0/3846209105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48100480 msgr2=0x7f1a48100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.327+0000 7f1a4e67d640 1 --2- 172.21.15.5:0/3846209105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48100480 0x7f1a48100860 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009900 tx=0x7f1a3802a410 comp rx=0 tx=0).stop 2024-09-15T22:10:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.327+0000 7f1a4e67d640 1 -- 172.21.15.5:0/3846209105 shutdown_connections 2024-09-15T22:10:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.327+0000 7f1a4e67d640 1 --2- 172.21.15.5:0/3846209105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a481055b0 0x7f1a4810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.327+0000 7f1a4e67d640 1 --2- 172.21.15.5:0/3846209105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48100e30 0x7f1a48104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.327+0000 7f1a4e67d640 1 --2- 172.21.15.5:0/3846209105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48100480 0x7f1a48100860 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.327+0000 7f1a4e67d640 1 -- 172.21.15.5:0/3846209105 >> 172.21.15.5:0/3846209105 conn(0x7f1a48077210 msgr2=0x7f1a480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.327+0000 7f1a4e67d640 1 -- 172.21.15.5:0/3846209105 shutdown_connections 2024-09-15T22:10:12.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.327+0000 7f1a4e67d640 1 -- 172.21.15.5:0/3846209105 wait complete. 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a4e67d640 1 Processor -- start 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a4e67d640 1 -- start start 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a4e67d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48100480 0x7f1a48197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a4e67d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48100e30 0x7f1a48197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a4e67d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a481055b0 0x7f1a48197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a4e67d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48110e50 con 0x7f1a48100e30 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a4e67d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48110cd0 con 0x7f1a48100480 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a4e67d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48110fd0 con 0x7f1a481055b0 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48100e30 0x7f1a48197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48100e30 0x7f1a48197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56246/0 (socket says 172.21.15.5:56246) 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a3ffff640 1 -- 172.21.15.5:0/4130277281 learned_addr learned my addr 172.21.15.5:0/4130277281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a47fff640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48100480 0x7f1a48197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a481055b0 0x7f1a48197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a3ffff640 1 -- 172.21.15.5:0/4130277281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a481055b0 msgr2=0x7f1a48197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a481055b0 0x7f1a48197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a3ffff640 1 -- 172.21.15.5:0/4130277281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48100480 msgr2=0x7f1a48197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48100480 0x7f1a48197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.328+0000 7f1a3ffff640 1 -- 172.21.15.5:0/4130277281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a481a69c0 con 0x7f1a48100e30 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.329+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48100e30 0x7f1a48197690 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000d950 tx=0x7f1a3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.329+0000 7f1a45ffb640 1 -- 172.21.15.5:0/4130277281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30017030 con 0x7f1a48100e30 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.329+0000 7f1a45ffb640 1 -- 172.21.15.5:0/4130277281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a30004020 con 0x7f1a48100e30 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.329+0000 7f1a4e67d640 1 -- 172.21.15.5:0/4130277281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a481a6cb0 con 0x7f1a48100e30 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.329+0000 7f1a45ffb640 1 -- 172.21.15.5:0/4130277281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a300047a0 con 0x7f1a48100e30 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.329+0000 7f1a4e67d640 1 -- 172.21.15.5:0/4130277281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a481a71a0 con 0x7f1a48100e30 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.331+0000 7f1a45ffb640 1 -- 172.21.15.5:0/4130277281 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) ==== 138638+0+0 (secure 0 0 0) 0x7f1a300041d0 con 0x7f1a48100e30 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a50a56740 1 Processor -- start 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a50a56740 1 -- start start 2024-09-15T22:10:12.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359060 0x561df3359440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df3363d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3366630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a50a56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561df32be620 con 0x561df3364240 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a50a56740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561df3265e50 con 0x561df3359060 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a50a56740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561df32ba5e0 con 0x561df3359980 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3366630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3366630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56254/0 (socket says 172.21.15.5:56254) 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a3ffff640 1 -- 172.21.15.5:0/52757931 learned_addr learned my addr 172.21.15.5:0/52757931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a45ffb640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1a340a3560 0x7f1a340a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.332+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/52757931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359060 0x561df3359440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a47fff640 1 --2- 172.21.15.5:0/52757931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df3363d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1a340a3560 0x7f1a340a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a45ffb640 1 -- 172.21.15.5:0/4130277281 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f1a300cfd70 con 0x7f1a48100e30 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a3ffff640 1 -- 172.21.15.5:0/52757931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 msgr2=0x561df3363d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/52757931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df3363d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a3ffff640 1 -- 172.21.15.5:0/52757931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359060 msgr2=0x561df3359440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/52757931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359060 0x561df3359440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a3ffff640 1 -- 172.21.15.5:0/52757931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df3256d40 con 0x561df3364240 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/52757931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359060 0x561df3359440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/52757931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3366630 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1a300cce80 tx=0x7f1a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20dd36e6b3670b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a3e7fc640 1 -- 172.21.15.5:0/52757931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a300d3430 con 0x561df3364240 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1a340a3560 0x7f1a340a5a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009fd0 tx=0x7f1a40004ab0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a3e7fc640 1 -- 172.21.15.5:0/52757931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a3009c170 con 0x561df3364240 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.333+0000 7f1a3e7fc640 1 -- 172.21.15.5:0/52757931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3009aec0 con 0x561df3364240 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.334+0000 7f1a50a56740 1 -- 172.21.15.5:0/52757931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 msgr2=0x561df3366630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:12.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.334+0000 7f1a50a56740 1 --2- 172.21.15.5:0/52757931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3366630 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1a300cce80 tx=0x7f1a300046c0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.334+0000 7f1a50a56740 1 -- 172.21.15.5:0/52757931 shutdown_connections 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.334+0000 7f1a50a56740 1 --2- 172.21.15.5:0/52757931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3366630 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.334+0000 7f1a50a56740 1 --2- 172.21.15.5:0/52757931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df3363d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.334+0000 7f1a50a56740 1 --2- 172.21.15.5:0/52757931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359060 0x561df3359440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.334+0000 7f1a50a56740 1 -- 172.21.15.5:0/52757931 >> 172.21.15.5:0/52757931 conn(0x561df33555f0 msgr2=0x561df33559d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.334+0000 7f1a50a56740 1 -- 172.21.15.5:0/52757931 shutdown_connections 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.334+0000 7f1a50a56740 1 -- 172.21.15.5:0/52757931 wait complete. 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.334+0000 7f1a50a56740 1 -- start start 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a50a56740 1 Processor -- start 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359060 0x561df3361200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359980 0x561df3361740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3361c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a50a56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561df3366e10 con 0x561df3364240 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a50a56740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561df3246f10 con 0x561df3359980 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a50a56740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561df3366f90 con 0x561df3359060 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a50a56740 10 client.0 _ll_register_callbacks cb 0x561df326adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a4cc1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3361c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a4cc1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3361c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56264/0 (socket says 172.21.15.5:56264) 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a4cc1a640 1 -- 172.21.15.5:0/1446484266 learned_addr learned my addr 172.21.15.5:0/1446484266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359980 0x561df3361740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a47fff640 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359060 0x561df3361200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a4cc1a640 1 -- 172.21.15.5:0/1446484266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359060 msgr2=0x561df3361200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359060 0x561df3361200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a4cc1a640 1 -- 172.21.15.5:0/1446484266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359980 msgr2=0x561df3361740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359980 0x561df3361740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a4cc1a640 1 -- 172.21.15.5:0/1446484266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df2fcd580 con 0x561df3364240 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.335+0000 7f1a47fff640 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359060 0x561df3361200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3361c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009b00 tx=0x7f1a40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844a55d8477f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a277fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a277fe640 1 -- 172.21.15.5:0/1446484266 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40023070 con 0x561df3364240 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a50a56740 1 -- 172.21.15.5:0/1446484266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a3009c170 con 0x561df3364240 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a277fe640 1 -- 172.21.15.5:0/1446484266 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f1a40015020 con 0x561df3364240 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a277fe640 1 -- 172.21.15.5:0/1446484266 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a4002f050 con 0x561df3364240 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a50a56740 10 client.15687 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a50a56740 1 -- 172.21.15.5:0/1446484266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561df3097960 con 0x561df3364240 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a277fe640 1 -- 172.21.15.5:0/1446484266 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a3009c170 con 0x561df3364240 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a277fe640 1 -- 172.21.15.5:0/1446484266 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1a4000f040 con 0x561df3364240 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a50a56740 10 client.15687 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a50a56740 10 client.15687 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a50a56740 20 client.15687 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a50a56740 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561df3373e70 0x561df3376280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a50a56740 4 client.15687 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.336+0000 7f1a50a56740 1 -- 172.21.15.5:0/1446484266 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561df32cbb90 con 0x561df3373e70 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.337+0000 7f1a47fff640 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561df3373e70 0x561df3376280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.337+0000 7f1a277fe640 10 client.15687 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.337+0000 7f1a47fff640 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561df3373e70 0x561df3376280 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5d3d2e6742ab86c server_cookie=a5bfbdc3878a9c38 in_seq=0 out_seq=0 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.338+0000 7f1a277fe640 1 -- 172.21.15.5:0/1446484266 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561df32cbb90 con 0x561df3373e70 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.338+0000 7f1a277fe640 10 client.15687 handle_command_reply: tid=0 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.338+0000 7f1a50a56740 1 client.15687 shutdown 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a50a56740 1 -- 172.21.15.5:0/1446484266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 msgr2=0x561df3361c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a50a56740 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3361c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009b00 tx=0x7f1a40004a40 comp rx=0 tx=0).stop 2024-09-15T22:10:12.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a50a56740 1 -- 172.21.15.5:0/1446484266 shutdown_connections 2024-09-15T22:10:12.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a50a56740 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df3361c80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a50a56740 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561df3373e70 0x561df3376280 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a50a56740 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359980 0x561df3361740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a50a56740 1 --2- 172.21.15.5:0/1446484266 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359060 0x561df3361200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a277fe640 0 client.15687 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:12.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a50a56740 1 -- 172.21.15.5:0/1446484266 >> 172.21.15.5:0/1446484266 conn(0x561df33555f0 msgr2=0x561df335b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:12.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a50a56740 1 -- 172.21.15.5:0/1446484266 shutdown_connections 2024-09-15T22:10:12.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a50a56740 1 -- 172.21.15.5:0/1446484266 wait complete. 2024-09-15T22:10:12.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.339+0000 7f1a50a56740 20 client.15687 trim_cache size 0 max 16384 2024-09-15T22:10:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.356+0000 7f1a50a56740 1 Processor -- start 2024-09-15T22:10:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.356+0000 7f1a50a56740 1 -- start start 2024-09-15T22:10:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.356+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359060 0x561df33559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.356+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df3363170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.356+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df337bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.356+0000 7f1a50a56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481b3c00 con 0x561df3364240 2024-09-15T22:10:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.356+0000 7f1a50a56740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481b14d0 con 0x561df3359060 2024-09-15T22:10:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.356+0000 7f1a50a56740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481af320 con 0x561df3359980 2024-09-15T22:10:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a4cc1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df337bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a4cc1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df337bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56280/0 (socket says 172.21.15.5:56280) 2024-09-15T22:10:12.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a4cc1a640 1 -- 172.21.15.5:0/1179650583 learned_addr learned my addr 172.21.15.5:0/1179650583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/1179650583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df3363170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a4cc1a640 1 -- 172.21.15.5:0/1179650583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 msgr2=0x561df3363170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/1179650583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df3363170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a4cc1a640 1 -- 172.21.15.5:0/1179650583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359060 msgr2=0x561df33559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a47fff640 1 --2- 172.21.15.5:0/1179650583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359060 0x561df33559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/1179650583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359060 0x561df33559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a4cc1a640 1 -- 172.21.15.5:0/1179650583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561df33618a0 con 0x561df3364240 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/1179650583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df3363170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/1179650583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df337bb20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1a40015ed0 tx=0x7f1a400125e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2695fc8be7dfa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a3dffb640 1 -- 172.21.15.5:0/1179650583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40023070 con 0x561df3364240 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a3dffb640 1 -- 172.21.15.5:0/1179650583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a40019070 con 0x561df3364240 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.357+0000 7f1a3dffb640 1 -- 172.21.15.5:0/1179650583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40004f40 con 0x561df3364240 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.358+0000 7f1a50a56740 1 -- 172.21.15.5:0/1179650583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 msgr2=0x561df337bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.358+0000 7f1a50a56740 1 --2- 172.21.15.5:0/1179650583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df337bb20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1a40015ed0 tx=0x7f1a400125e0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.358+0000 7f1a50a56740 1 -- 172.21.15.5:0/1179650583 shutdown_connections 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.358+0000 7f1a50a56740 1 --2- 172.21.15.5:0/1179650583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3364240 0x561df337bb20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.358+0000 7f1a50a56740 1 --2- 172.21.15.5:0/1179650583 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df3363170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.358+0000 7f1a50a56740 1 --2- 172.21.15.5:0/1179650583 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3359060 0x561df33559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.358+0000 7f1a50a56740 1 -- 172.21.15.5:0/1179650583 >> 172.21.15.5:0/1179650583 conn(0x561df33555f0 msgr2=0x561df3362a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.358+0000 7f1a50a56740 1 -- 172.21.15.5:0/1179650583 shutdown_connections 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.358+0000 7f1a50a56740 1 -- 172.21.15.5:0/1179650583 wait complete. 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a50a56740 1 -- start start 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a50a56740 1 Processor -- start 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3359060 0x561df336c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df336c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a50a56740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3364240 0x561df33691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a50a56740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481af990 con 0x561df3359060 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a50a56740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481b0e00 con 0x561df3364240 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a50a56740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1a481b0060 con 0x561df3359980 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a50a56740 10 client.0 _ll_register_callbacks cb 0x7f1a400126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3359060 0x561df336c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3359060 0x561df336c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56282/0 (socket says 172.21.15.5:56282) 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a47fff640 1 -- 172.21.15.5:0/3245264316 learned_addr learned my addr 172.21.15.5:0/3245264316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.359+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df336c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a47fff640 1 -- 172.21.15.5:0/3245264316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 msgr2=0x561df336c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a47fff640 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df336c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a47fff640 1 -- 172.21.15.5:0/3245264316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3364240 msgr2=0x561df33691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3364240 0x561df33691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a47fff640 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3364240 0x561df33691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a47fff640 1 -- 172.21.15.5:0/3245264316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561df3369820 con 0x561df3359060 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df336c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a4cc1a640 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3364240 0x561df33691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a47fff640 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3359060 0x561df336c3c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009900 tx=0x7f1a38031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6675409b9df5f0ca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a277fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:10:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a277fe640 1 -- 172.21.15.5:0/3245264316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38049030 con 0x561df3359060 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a277fe640 1 -- 172.21.15.5:0/3245264316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f1a38004250 con 0x561df3359060 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a50a56740 1 -- 172.21.15.5:0/3245264316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a40019070 con 0x561df3359060 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a277fe640 1 -- 172.21.15.5:0/3245264316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1a38055050 con 0x561df3359060 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a50a56740 10 client.15693 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a50a56740 1 -- 172.21.15.5:0/3245264316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561df33df410 con 0x561df3359060 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.360+0000 7f1a277fe640 1 -- 172.21.15.5:0/3245264316 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a40019070 con 0x561df3359060 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.361+0000 7f1a277fe640 1 -- 172.21.15.5:0/3245264316 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1a38045350 con 0x561df3359060 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.361+0000 7f1a50a56740 10 client.15693 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.361+0000 7f1a50a56740 10 client.15693 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.361+0000 7f1a50a56740 20 client.15693 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.361+0000 7f1a50a56740 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561df33dfb40 0x561df33e1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.361+0000 7f1a50a56740 4 client.15693 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a620f627", "release": true, "await": true}] 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.361+0000 7f1a50a56740 1 -- 172.21.15.5:0/3245264316 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a620f627", "release": true, "await": true}) -- 0x561df3362010 con 0x561df33dfb40 2024-09-15T22:10:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.361+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561df33dfb40 0x561df33e1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:12.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.361+0000 7f1a3ffff640 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561df33dfb40 0x561df33e1f60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73086fe2920cbf server_cookie=c9e40e5f8f4a65af in_seq=0 out_seq=0 2024-09-15T22:10:12.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.361+0000 7f1a277fe640 10 client.15693 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:12.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.420+0000 7f1a277fe640 1 -- 172.21.15.5:0/3245264316 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x561df3362010 con 0x561df33dfb40 2024-09-15T22:10:12.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.420+0000 7f1a277fe640 10 client.15693 handle_command_reply: tid=0 2024-09-15T22:10:12.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.427+0000 7f1a50a56740 1 client.15693 shutdown 2024-09-15T22:10:12.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a50a56740 1 -- 172.21.15.5:0/3245264316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3359060 msgr2=0x561df336c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:12.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a50a56740 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3359060 0x561df336c3c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009900 tx=0x7f1a38031a20 comp rx=0 tx=0).stop 2024-09-15T22:10:12.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a50a56740 1 -- 172.21.15.5:0/3245264316 shutdown_connections 2024-09-15T22:10:12.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a50a56740 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561df33dfb40 0x561df33e1f60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a50a56740 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561df3364240 0x561df33691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a50a56740 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561df3359980 0x561df336c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a50a56740 1 --2- 172.21.15.5:0/3245264316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df3359060 0x561df336c3c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a277fe640 0 client.15693 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a50a56740 1 -- 172.21.15.5:0/3245264316 >> 172.21.15.5:0/3245264316 conn(0x561df33555f0 msgr2=0x561df3354750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a50a56740 1 -- 172.21.15.5:0/3245264316 shutdown_connections 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a50a56740 1 -- 172.21.15.5:0/3245264316 wait complete. 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.430+0000 7f1a50a56740 20 client.15693 trim_cache size 0 max 16384 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:10:12.433 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 4, 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "a620f627": { 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 4, 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.2, 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:10:12.434 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:10:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:10:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:10:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:10:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:10:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:10:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:10:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:10:12.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:10:12.435 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.432+0000 7f1a4e67d640 1 -- 172.21.15.5:0/4130277281 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1a340a3560 msgr2=0x7f1a340a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.432+0000 7f1a4e67d640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1a340a3560 0x7f1a340a5a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009fd0 tx=0x7f1a40004ab0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.432+0000 7f1a4e67d640 1 -- 172.21.15.5:0/4130277281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48100e30 msgr2=0x7f1a48197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.432+0000 7f1a4e67d640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48100e30 0x7f1a48197690 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000d950 tx=0x7f1a3000de10 comp rx=0 tx=0).stop 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.433+0000 7f1a4e67d640 1 -- 172.21.15.5:0/4130277281 shutdown_connections 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.433+0000 7f1a4e67d640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1a340a3560 0x7f1a340a5a20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.433+0000 7f1a4e67d640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1a481055b0 0x7f1a48197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.433+0000 7f1a4e67d640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a48100e30 0x7f1a48197690 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.433+0000 7f1a4e67d640 1 --2- 172.21.15.5:0/4130277281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1a48100480 0x7f1a48197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.433+0000 7f1a4e67d640 1 -- 172.21.15.5:0/4130277281 >> 172.21.15.5:0/4130277281 conn(0x7f1a48077210 msgr2=0x7f1a48103810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.433+0000 7f1a4e67d640 1 -- 172.21.15.5:0/4130277281 shutdown_connections 2024-09-15T22:10:12.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:12.435+0000 7f1a4e67d640 1 -- 172.21.15.5:0/4130277281 wait complete. 2024-09-15T22:10:13.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:13 smithi005 ceph-mon[30944]: pgmap v42: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-15T22:10:13.270 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a620f627', total seconds elapsed: 33.8 2024-09-15T22:10:13.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:13 smithi099 ceph-mon[34892]: pgmap v42: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-15T22:10:13.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:13 smithi129 ceph-mon[35314]: pgmap v42: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-15T22:10:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 45.0 s, 1565.7 tps, lat 364.236 ms stddev 2956.806 2024-09-15T22:10:15.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:15 smithi099 ceph-mon[34892]: pgmap v43: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-15T22:10:15.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:15 smithi129 ceph-mon[35314]: pgmap v43: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-15T22:10:15.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:15 smithi005 ceph-mon[30944]: pgmap v43: 129 pgs: 129 active+clean; 970 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-09-15T22:10:17.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:17 smithi099 ceph-mon[34892]: pgmap v44: 129 pgs: 129 active+clean; 981 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-09-15T22:10:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:17 smithi129 ceph-mon[35314]: pgmap v44: 129 pgs: 129 active+clean; 981 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-09-15T22:10:17.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:17 smithi005 ceph-mon[30944]: pgmap v44: 129 pgs: 129 active+clean; 981 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-09-15T22:10:19.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:19 smithi099 ceph-mon[34892]: pgmap v45: 129 pgs: 129 active+clean; 981 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-15T22:10:19.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:19 smithi129 ceph-mon[35314]: pgmap v45: 129 pgs: 129 active+clean; 981 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-15T22:10:19.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:19 smithi005 ceph-mon[30944]: pgmap v45: 129 pgs: 129 active+clean; 981 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-09-15T22:10:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 50.0 s, 3736.0 tps, lat 26.724 ms stddev 29.655 2024-09-15T22:10:21.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:21 smithi005 ceph-mon[30944]: pgmap v46: 129 pgs: 129 active+clean; 1008 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-15T22:10:21.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:21 smithi099 ceph-mon[34892]: pgmap v46: 129 pgs: 129 active+clean; 1008 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-15T22:10:21.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:21 smithi129 ceph-mon[35314]: pgmap v46: 129 pgs: 129 active+clean; 1008 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 4.5 MiB/s wr, 453 op/s 2024-09-15T22:10:23.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:23 smithi099 ceph-mon[34892]: pgmap v47: 129 pgs: 129 active+clean; 1008 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 MiB/s wr, 450 op/s 2024-09-15T22:10:23.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:23 smithi129 ceph-mon[35314]: pgmap v47: 129 pgs: 129 active+clean; 1008 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 MiB/s wr, 450 op/s 2024-09-15T22:10:23.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:23 smithi005 ceph-mon[30944]: pgmap v47: 129 pgs: 129 active+clean; 1008 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 MiB/s wr, 450 op/s 2024-09-15T22:10:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 55.0 s, 3248.0 tps, lat 30.877 ms stddev 37.823 2024-09-15T22:10:25.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:25 smithi099 ceph-mon[34892]: pgmap v48: 129 pgs: 129 active+clean; 1008 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 MiB/s wr, 450 op/s 2024-09-15T22:10:25.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:25 smithi129 ceph-mon[35314]: pgmap v48: 129 pgs: 129 active+clean; 1008 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 MiB/s wr, 450 op/s 2024-09-15T22:10:25.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:25 smithi005 ceph-mon[30944]: pgmap v48: 129 pgs: 129 active+clean; 1008 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 4.5 MiB/s wr, 450 op/s 2024-09-15T22:10:27.149 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:10:27.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:27 smithi099 ceph-mon[34892]: pgmap v49: 129 pgs: 129 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.9 MiB/s wr, 756 op/s 2024-09-15T22:10:27.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:27 smithi099 ceph-mon[34892]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-15T22:10:27.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:27 smithi129 ceph-mon[35314]: pgmap v49: 129 pgs: 129 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.9 MiB/s wr, 756 op/s 2024-09-15T22:10:27.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:27 smithi129 ceph-mon[35314]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-15T22:10:27.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:27 smithi005 ceph-mon[30944]: pgmap v49: 129 pgs: 129 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.9 MiB/s wr, 756 op/s 2024-09-15T22:10:27.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:27 smithi005 ceph-mon[30944]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-09-15T22:10:27.592 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:28.784 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:10:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:10:28.785 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:10:28.785 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:10:28.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.665+0000 7f36c88fc640 1 -- 172.21.15.5:0/524736060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c4109830 msgr2=0x7f36c4109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.665+0000 7f36c88fc640 1 --2- 172.21.15.5:0/524736060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c4109830 0x7f36c4109c90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f36b401a910 tx=0x7f36b403b420 comp rx=0 tx=0).stop 2024-09-15T22:10:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.665+0000 7f36c88fc640 1 -- 172.21.15.5:0/524736060 shutdown_connections 2024-09-15T22:10:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.665+0000 7f36c88fc640 1 --2- 172.21.15.5:0/524736060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36c411b140 0x7f36c411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.665+0000 7f36c88fc640 1 --2- 172.21.15.5:0/524736060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c4109830 0x7f36c4109c90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.665+0000 7f36c88fc640 1 --2- 172.21.15.5:0/524736060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4109450 0x7f36c4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.665+0000 7f36c88fc640 1 -- 172.21.15.5:0/524736060 >> 172.21.15.5:0/524736060 conn(0x7f36c406e140 msgr2=0x7f36c406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.665+0000 7f36c88fc640 1 -- 172.21.15.5:0/524736060 shutdown_connections 2024-09-15T22:10:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.665+0000 7f36c88fc640 1 -- 172.21.15.5:0/524736060 wait complete. 2024-09-15T22:10:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.666+0000 7f36c88fc640 1 Processor -- start 2024-09-15T22:10:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.666+0000 7f36c88fc640 1 -- start start 2024-09-15T22:10:28.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.666+0000 7f36c88fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4109450 0x7f36c411a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.666+0000 7f36c88fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c4109830 0x7f36c4116830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.666+0000 7f36c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4109450 0x7f36c411a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.666+0000 7f36c259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4109450 0x7f36c411a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55042/0 (socket says 172.21.15.5:55042) 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.666+0000 7f36c1d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c4109830 0x7f36c4116830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c1d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c4109830 0x7f36c4116830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:60970/0 (socket says 172.21.15.5:60970) 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c88fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36c411b140 0x7f36c4116d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c88fc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41132b0 con 0x7f36c4109450 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c259c640 1 -- 172.21.15.5:0/225867557 learned_addr learned my addr 172.21.15.5:0/225867557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c88fc640 1 -- 172.21.15.5:0/225867557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4113130 con 0x7f36c411b140 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c88fc640 1 -- 172.21.15.5:0/225867557 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4113430 con 0x7f36c4109830 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36c411b140 0x7f36c4116d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c2d9d640 1 -- 172.21.15.5:0/225867557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c4109830 msgr2=0x7f36c4116830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c4109830 0x7f36c4116830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c2d9d640 1 -- 172.21.15.5:0/225867557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4109450 msgr2=0x7f36c411a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4109450 0x7f36c411a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.667+0000 7f36c2d9d640 1 -- 172.21.15.5:0/225867557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c4117360 con 0x7f36c411b140 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.668+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36c411b140 0x7f36c4116d70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00e9a0 tx=0x7f36ac00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.668+0000 7f36bb7fe640 1 -- 172.21.15.5:0/225867557 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac00ccc0 con 0x7f36c411b140 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.668+0000 7f36bb7fe640 1 -- 172.21.15.5:0/225867557 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ac00ce60 con 0x7f36c411b140 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.668+0000 7f36bb7fe640 1 -- 172.21.15.5:0/225867557 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac004a20 con 0x7f36c411b140 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.668+0000 7f36c88fc640 1 -- 172.21.15.5:0/225867557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c4117650 con 0x7f36c411b140 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.668+0000 7f36c88fc640 1 -- 172.21.15.5:0/225867557 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c41ab490 con 0x7f36c411b140 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.672+0000 7f36bb7fe640 1 -- 172.21.15.5:0/225867557 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f36ac004c00 con 0x7f36c411b140 2024-09-15T22:10:28.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.672+0000 7f36cacd5740 1 Processor -- start 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36cacd5740 1 -- start start 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36bb7fe640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36b00a3500 0x7f36b00a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2be140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 0x563c7f2c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36cacd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563c7f2192e0 con 0x563c7f2b3dc0 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36cacd5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c7f1c0ce0 con 0x563c7f2b34a0 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36cacd5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563c7f2152a0 con 0x563c7f2be680 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36c1d9b640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36b00a3500 0x7f36b00a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36bb7fe640 1 -- 172.21.15.5:0/225867557 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f36ac0cffc0 con 0x7f36c411b140 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36c1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2be140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36c259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36c259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:52080/0 (socket says 172.21.15.5:52080) 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36c259c640 1 -- 172.21.15.5:0/1836650684 learned_addr learned my addr 172.21.15.5:0/1836650684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36c1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2be140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55052/0 (socket says 172.21.15.5:55052) 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.673+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/1836650684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 0x563c7f2c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.674+0000 7f36c259c640 1 -- 172.21.15.5:0/1836650684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 msgr2=0x563c7f2c0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.674+0000 7f36c259c640 1 --2- 172.21.15.5:0/1836650684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 0x563c7f2c0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.674+0000 7f36c259c640 1 -- 172.21.15.5:0/1836650684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 msgr2=0x563c7f2be140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.674+0000 7f36c259c640 1 --2- 172.21.15.5:0/1836650684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2be140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.674+0000 7f36c259c640 1 -- 172.21.15.5:0/1836650684 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c7f1a5440 con 0x563c7f2b34a0 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.674+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/1836650684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 0x563c7f2c0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.674+0000 7f36c1d9b640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36b00a3500 0x7f36b00a59c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36c4118080 tx=0x7f36b4005e80 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.674+0000 7f36c259c640 1 --2- 172.21.15.5:0/1836650684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2b3880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36bc011490 tx=0x7f36bc011950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e956291a8a049c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:28.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.677+0000 7f36b8ff9640 1 -- 172.21.15.5:0/1836650684 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc012770 con 0x563c7f2b34a0 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.677+0000 7f36b8ff9640 1 -- 172.21.15.5:0/1836650684 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36bc016070 con 0x563c7f2b34a0 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.677+0000 7f36b8ff9640 1 -- 172.21.15.5:0/1836650684 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc01adb0 con 0x563c7f2b34a0 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.678+0000 7f36cacd5740 1 -- 172.21.15.5:0/1836650684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 msgr2=0x563c7f2b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.678+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1836650684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2b3880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f36bc011490 tx=0x7f36bc011950 comp rx=0 tx=0).stop 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.678+0000 7f36cacd5740 1 -- 172.21.15.5:0/1836650684 shutdown_connections 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.678+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1836650684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 0x563c7f2c0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.678+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1836650684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2be140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.678+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1836650684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2b3880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.678+0000 7f36cacd5740 1 -- 172.21.15.5:0/1836650684 >> 172.21.15.5:0/1836650684 conn(0x563c7f2afa30 msgr2=0x563c7f2afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.678+0000 7f36cacd5740 1 -- 172.21.15.5:0/1836650684 shutdown_connections 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.678+0000 7f36cacd5740 1 -- 172.21.15.5:0/1836650684 wait complete. 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.678+0000 7f36cacd5740 1 -- start start 2024-09-15T22:10:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.678+0000 7f36cacd5740 1 Processor -- start 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2bdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 0x563c7f2bb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36cacd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563c7f2c12a0 con 0x563c7f2b3dc0 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36cacd5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563c7f1d4050 con 0x563c7f2b34a0 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36cacd5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563c7f2c1420 con 0x563c7f2be680 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36cacd5740 10 client.0 _ll_register_callbacks cb 0x563c7f214160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36c1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36c1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2bb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55054/0 (socket says 172.21.15.5:55054) 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36c1d9b640 1 -- 172.21.15.5:0/1810724263 learned_addr learned my addr 172.21.15.5:0/1810724263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36c2d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 0x563c7f2bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36c259c640 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2bdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36c2d9d640 1 -- 172.21.15.5:0/1810724263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 msgr2=0x563c7f2bdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2bdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36c2d9d640 1 -- 172.21.15.5:0/1810724263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 msgr2=0x563c7f2bb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.679+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2bb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.680+0000 7f36c2d9d640 1 -- 172.21.15.5:0/1810724263 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7ef8a3c0 con 0x563c7f2be680 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.680+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 0x563c7f2bb780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0cd310 tx=0x7f36ac09c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f11b02608409dad8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.680+0000 7f36a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.680+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1810724263 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac01c040 con 0x563c7f2be680 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.680+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1810724263 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f36ac098510 con 0x563c7f2be680 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.680+0000 7f36cacd5740 1 -- 172.21.15.5:0/1810724263 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36bc016070 con 0x563c7f2be680 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.680+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1810724263 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36ac0ac050 con 0x563c7f2be680 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.680+0000 7f36cacd5740 10 client.25291 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:28.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.680+0000 7f36cacd5740 1 -- 172.21.15.5:0/1810724263 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c7f2ccc20 con 0x563c7f2be680 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.680+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1810724263 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36ac0a4b10 con 0x563c7f2be680 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.680+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1810724263 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f36ac09b2f0 con 0x563c7f2be680 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.681+0000 7f36cacd5740 10 client.25291 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.681+0000 7f36cacd5740 10 client.25291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.681+0000 7f36cacd5740 20 client.25291 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.681+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563c7f2ce130 0x563c7f2d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.681+0000 7f36cacd5740 4 client.25291 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.681+0000 7f36cacd5740 1 -- 172.21.15.5:0/1810724263 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f36bc016070 con 0x563c7f2ce130 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.681+0000 7f36c259c640 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563c7f2ce130 0x563c7f2d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.682+0000 7f36a9ffb640 10 client.25291 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.682+0000 7f36c259c640 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563c7f2ce130 0x563c7f2d0560 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f9797601463208 server_cookie=165bd734f5acace8 in_seq=0 out_seq=0 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1810724263 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f36bc016070 con 0x563c7f2ce130 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36a9ffb640 10 client.25291 handle_command_reply: tid=0 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36cacd5740 1 client.25291 shutdown 2024-09-15T22:10:28.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36cacd5740 1 -- 172.21.15.5:0/1810724263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 msgr2=0x563c7f2bb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 0x563c7f2bb780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f36ac0cd310 tx=0x7f36ac09c660 comp rx=0 tx=0).stop 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36cacd5740 1 -- 172.21.15.5:0/1810724263 shutdown_connections 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2be680 0x563c7f2bb780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2bb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563c7f2ce130 0x563c7f2d0560 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36a9ffb640 0 client.25291 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1810724263 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2bdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36cacd5740 1 -- 172.21.15.5:0/1810724263 >> 172.21.15.5:0/1810724263 conn(0x563c7f2afa30 msgr2=0x563c7f2aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36cacd5740 1 -- 172.21.15.5:0/1810724263 shutdown_connections 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.683+0000 7f36cacd5740 1 -- 172.21.15.5:0/1810724263 wait complete. 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.686+0000 7f36cacd5740 20 client.25291 trim_cache size 0 max 16384 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.717+0000 7f36cacd5740 1 Processor -- start 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.719+0000 7f36cacd5740 1 -- start start 2024-09-15T22:10:28.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.719+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2afa30 0x563c7f2bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.719+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2b34a0 0x563c7f2d5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.719+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b3dc0 0x563c7f2d6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.719+0000 7f36cacd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41b7e70 con 0x563c7f2afa30 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.719+0000 7f36cacd5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41b5740 con 0x563c7f2b3dc0 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.719+0000 7f36cacd5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41b3590 con 0x563c7f2b34a0 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c2d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b3dc0 0x563c7f2d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c2d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b3dc0 0x563c7f2d6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:52100/0 (socket says 172.21.15.5:52100) 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c1d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2b34a0 0x563c7f2d5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c2d9d640 1 -- 172.21.15.5:0/2680406773 learned_addr learned my addr 172.21.15.5:0/2680406773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c259c640 1 --2- 172.21.15.5:0/2680406773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2afa30 0x563c7f2bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c1d9b640 1 -- 172.21.15.5:0/2680406773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b3dc0 msgr2=0x563c7f2d6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c1d9b640 1 --2- 172.21.15.5:0/2680406773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b3dc0 0x563c7f2d6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c1d9b640 1 -- 172.21.15.5:0/2680406773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2afa30 msgr2=0x563c7f2bd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c1d9b640 1 --2- 172.21.15.5:0/2680406773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2afa30 0x563c7f2bd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c1d9b640 1 -- 172.21.15.5:0/2680406773 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c7f2b32d0 con 0x563c7f2b34a0 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c259c640 1 --2- 172.21.15.5:0/2680406773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2afa30 0x563c7f2bd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.720+0000 7f36c1d9b640 1 --2- 172.21.15.5:0/2680406773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2b34a0 0x563c7f2d5e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x563c7f2bc880 tx=0x7f36b403b420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d469b0a37bfa0a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.721+0000 7f36abfff640 1 -- 172.21.15.5:0/2680406773 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4046020 con 0x563c7f2b34a0 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.721+0000 7f36abfff640 1 -- 172.21.15.5:0/2680406773 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b4060070 con 0x563c7f2b34a0 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.721+0000 7f36abfff640 1 -- 172.21.15.5:0/2680406773 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4040c10 con 0x563c7f2b34a0 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.722+0000 7f36cacd5740 1 -- 172.21.15.5:0/2680406773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2b34a0 msgr2=0x563c7f2d5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.722+0000 7f36cacd5740 1 --2- 172.21.15.5:0/2680406773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2b34a0 0x563c7f2d5e30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x563c7f2bc880 tx=0x7f36b403b420 comp rx=0 tx=0).stop 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.722+0000 7f36cacd5740 1 -- 172.21.15.5:0/2680406773 shutdown_connections 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.722+0000 7f36cacd5740 1 --2- 172.21.15.5:0/2680406773 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b3dc0 0x563c7f2d6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.722+0000 7f36cacd5740 1 --2- 172.21.15.5:0/2680406773 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2b34a0 0x563c7f2d5e30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.722+0000 7f36cacd5740 1 --2- 172.21.15.5:0/2680406773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2afa30 0x563c7f2bd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.722+0000 7f36cacd5740 1 -- 172.21.15.5:0/2680406773 >> 172.21.15.5:0/2680406773 conn(0x563c7f2bb8e0 msgr2=0x563c7f2bcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.723+0000 7f36cacd5740 1 -- 172.21.15.5:0/2680406773 shutdown_connections 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.723+0000 7f36cacd5740 1 -- 172.21.15.5:0/2680406773 wait complete. 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.723+0000 7f36cacd5740 1 -- start start 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36cacd5740 1 Processor -- start 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2afa30 0x563c7f2c4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36cacd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36cacd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41b3c00 con 0x563c7f2b3dc0 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36cacd5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41b5070 con 0x563c7f2b34a0 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36cacd5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41b42d0 con 0x563c7f2afa30 2024-09-15T22:10:28.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36cacd5740 10 client.0 _ll_register_callbacks cb 0x7f36ac0bc050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36c1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36c259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2afa30 0x563c7f2c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36c1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:52112/0 (socket says 172.21.15.5:52112) 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36c1d9b640 1 -- 172.21.15.5:0/1654337796 learned_addr learned my addr 172.21.15.5:0/1654337796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36c259c640 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2afa30 0x563c7f2c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:32776/0 (socket says 172.21.15.5:32776) 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.724+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.725+0000 7f36c2d9d640 1 -- 172.21.15.5:0/1654337796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2afa30 msgr2=0x563c7f2c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.725+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2afa30 0x563c7f2c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.725+0000 7f36c2d9d640 1 -- 172.21.15.5:0/1654337796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 msgr2=0x563c7f2c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.725+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.725+0000 7f36c2d9d640 1 -- 172.21.15.5:0/1654337796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c7f337e10 con 0x563c7f2b3dc0 2024-09-15T22:10:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.728+0000 7f36c2d9d640 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2c4de0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00c6a0 tx=0x7f36ac09ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0e38e0793bb9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.728+0000 7f36a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.731+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1654337796 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36ac01c070 con 0x563c7f2b3dc0 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.731+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1654337796 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f36ac097f10 con 0x563c7f2b3dc0 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.732+0000 7f36cacd5740 1 -- 172.21.15.5:0/1654337796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b4060070 con 0x563c7f2b3dc0 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.732+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1654337796 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36ac0a4680 con 0x563c7f2b3dc0 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.732+0000 7f36cacd5740 10 client.15711 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.732+0000 7f36cacd5740 1 -- 172.21.15.5:0/1654337796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563c7f339080 con 0x563c7f2b3dc0 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.733+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1654337796 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b4060070 con 0x563c7f2b3dc0 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.734+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1654337796 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f36ac09c100 con 0x563c7f2b3dc0 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.734+0000 7f36cacd5740 10 client.15711 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.734+0000 7f36cacd5740 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.734+0000 7f36cacd5740 20 client.15711 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.734+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563c7f2be680 0x563c7f33b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.734+0000 7f36cacd5740 4 client.15711 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.734+0000 7f36cacd5740 1 -- 172.21.15.5:0/1654337796 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c7f226850 con 0x563c7f2be680 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.735+0000 7f36c259c640 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563c7f2be680 0x563c7f33b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.735+0000 7f36c259c640 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563c7f2be680 0x563c7f33b5b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7d9b62acd2a1754 server_cookie=a094d53b9425c79b in_seq=0 out_seq=0 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.736+0000 7f36a9ffb640 10 client.15711 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.737+0000 7f36a9ffb640 1 -- 172.21.15.5:0/1654337796 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563c7f226850 con 0x563c7f2be680 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.737+0000 7f36a9ffb640 10 client.15711 handle_command_reply: tid=0 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.737+0000 7f36cacd5740 1 client.15711 shutdown 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.742+0000 7f36cacd5740 1 -- 172.21.15.5:0/1654337796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 msgr2=0x563c7f2c4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.742+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2c4de0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00c6a0 tx=0x7f36ac09ba20 comp rx=0 tx=0).stop 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.745+0000 7f36cacd5740 1 -- 172.21.15.5:0/1654337796 shutdown_connections 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.745+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c7f2b3dc0 0x563c7f2c4de0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.745+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563c7f2be680 0x563c7f33b5b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.745+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563c7f2b34a0 0x563c7f2c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.745+0000 7f36cacd5740 1 --2- 172.21.15.5:0/1654337796 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563c7f2afa30 0x563c7f2c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.745+0000 7f36cacd5740 1 -- 172.21.15.5:0/1654337796 >> 172.21.15.5:0/1654337796 conn(0x563c7f2bb8e0 msgr2=0x563c7f2b9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.747+0000 7f36a9ffb640 0 client.15711 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.749+0000 7f36cacd5740 1 -- 172.21.15.5:0/1654337796 shutdown_connections 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.749+0000 7f36cacd5740 1 -- 172.21.15.5:0/1654337796 wait complete. 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.749+0000 7f36cacd5740 20 client.15711 trim_cache size 0 max 16384 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.752+0000 7f36c88fc640 1 -- 172.21.15.5:0/225867557 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36b00a3500 msgr2=0x7f36b00a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.752+0000 7f36c88fc640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36b00a3500 0x7f36b00a59c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36c4118080 tx=0x7f36b4005e80 comp rx=0 tx=0).stop 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.753+0000 7f36c88fc640 1 -- 172.21.15.5:0/225867557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36c411b140 msgr2=0x7f36c4116d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.753+0000 7f36c88fc640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36c411b140 0x7f36c4116d70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f36ac00e9a0 tx=0x7f36ac00ee60 comp rx=0 tx=0).stop 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.753+0000 7f36c88fc640 1 -- 172.21.15.5:0/225867557 shutdown_connections 2024-09-15T22:10:28.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.753+0000 7f36c88fc640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f36b00a3500 0x7f36b00a59c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.753+0000 7f36c88fc640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f36c411b140 0x7f36c4116d70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.753+0000 7f36c88fc640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f36c4109830 0x7f36c4116830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.753+0000 7f36c88fc640 1 --2- 172.21.15.5:0/225867557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f36c4109450 0x7f36c411a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.753+0000 7f36c88fc640 1 -- 172.21.15.5:0/225867557 >> 172.21.15.5:0/225867557 conn(0x7f36c406e140 msgr2=0x7f36c4070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.756+0000 7f36c88fc640 1 -- 172.21.15.5:0/225867557 shutdown_connections 2024-09-15T22:10:28.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:28.756+0000 7f36c88fc640 1 -- 172.21.15.5:0/225867557 wait complete. 2024-09-15T22:10:29.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:29 smithi099 ceph-mon[34892]: pgmap v50: 129 pgs: 129 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.6 MiB/s wr, 627 op/s 2024-09-15T22:10:29.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:29 smithi129 ceph-mon[35314]: pgmap v50: 129 pgs: 129 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.6 MiB/s wr, 627 op/s 2024-09-15T22:10:29.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:29 smithi005 ceph-mon[30944]: pgmap v50: 129 pgs: 129 active+clean; 1.0 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 7.6 MiB/s wr, 627 op/s 2024-09-15T22:10:29.586 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:10:29.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:10:29.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 60.0 s, 2985.0 tps, lat 33.085 ms stddev 43.425 2024-09-15T22:10:30.014 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:30.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.614+0000 7f071b713640 1 -- 172.21.15.5:0/605641711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0714104010 msgr2=0x7f07141043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:30.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.614+0000 7f071b713640 1 --2- 172.21.15.5:0/605641711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0714104010 0x7f07141043f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0704009900 tx=0x7f070402a410 comp rx=0 tx=0).stop 2024-09-15T22:10:30.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.614+0000 7f071b713640 1 -- 172.21.15.5:0/605641711 shutdown_connections 2024-09-15T22:10:30.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.614+0000 7f071b713640 1 --2- 172.21.15.5:0/605641711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0714109140 0x7f0714110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:30.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.614+0000 7f071b713640 1 --2- 172.21.15.5:0/605641711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07141049c0 0x7f0714108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:30.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.614+0000 7f071b713640 1 --2- 172.21.15.5:0/605641711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0714104010 0x7f07141043f0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0704009900 tx=0x7f070402a410 comp rx=0 tx=0).stop 2024-09-15T22:10:30.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.614+0000 7f071b713640 1 -- 172.21.15.5:0/605641711 >> 172.21.15.5:0/605641711 conn(0x7f07140ffa50 msgr2=0x7f0714101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:30.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.614+0000 7f071b713640 1 -- 172.21.15.5:0/605641711 shutdown_connections 2024-09-15T22:10:30.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.614+0000 7f071b713640 1 -- 172.21.15.5:0/605641711 wait complete. 2024-09-15T22:10:30.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.615+0000 7f071b713640 1 Processor -- start 2024-09-15T22:10:30.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.615+0000 7f071b713640 1 -- start start 2024-09-15T22:10:30.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f071b713640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07141049c0 0x7f0714076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:30.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f071b713640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0714109140 0x7f0714076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:30.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f071b713640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0714077180 0x7f07141aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:30.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f071b713640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0714114140 con 0x7f07141049c0 2024-09-15T22:10:30.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f071b713640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0714113fc0 con 0x7f0714109140 2024-09-15T22:10:30.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f071b713640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07141142c0 con 0x7f0714077180 2024-09-15T22:10:30.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f0718cae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0714109140 0x7f0714076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:30.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f0718cae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0714109140 0x7f0714076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:52142/0 (socket says 172.21.15.5:52142) 2024-09-15T22:10:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f0718cae640 1 -- 172.21.15.5:0/1853810226 learned_addr learned my addr 172.21.15.5:0/1853810226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f0719cb0640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0714077180 0x7f07141aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f0718cae640 1 -- 172.21.15.5:0/1853810226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0714077180 msgr2=0x7f07141aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f0718cae640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0714077180 0x7f07141aadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f0718cae640 1 -- 172.21.15.5:0/1853810226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07141049c0 msgr2=0x7f0714076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f0718cae640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07141049c0 0x7f0714076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.616+0000 7f0718cae640 1 -- 172.21.15.5:0/1853810226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f071419a590 con 0x7f0714109140 2024-09-15T22:10:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.617+0000 7f0719cb0640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0714077180 0x7f07141aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.617+0000 7f0718cae640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0714109140 0x7f0714076c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f070c00ebf0 tx=0x7f070c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.617+0000 7f070a7fc640 1 -- 172.21.15.5:0/1853810226 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c010630 con 0x7f0714109140 2024-09-15T22:10:30.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.617+0000 7f071b713640 1 -- 172.21.15.5:0/1853810226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f071419a880 con 0x7f0714109140 2024-09-15T22:10:30.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.617+0000 7f071b713640 1 -- 172.21.15.5:0/1853810226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f071419ae10 con 0x7f0714109140 2024-09-15T22:10:30.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.618+0000 7f070a7fc640 1 -- 172.21.15.5:0/1853810226 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f070c010cb0 con 0x7f0714109140 2024-09-15T22:10:30.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.619+0000 7f070a7fc640 1 -- 172.21.15.5:0/1853810226 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f070c010630 con 0x7f0714109140 2024-09-15T22:10:30.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.619+0000 7f071b713640 1 -- 172.21.15.5:0/1853810226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f8005290 con 0x7f0714109140 2024-09-15T22:10:30.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.622+0000 7f070a7fc640 1 -- 172.21.15.5:0/1853810226 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f070c0107d0 con 0x7f0714109140 2024-09-15T22:10:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.623+0000 7f070a7fc640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07000a37b0 0x7f07000a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.623+0000 7f070a7fc640 1 -- 172.21.15.5:0/1853810226 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f070c010430 con 0x7f0714109140 2024-09-15T22:10:30.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.623+0000 7f07194af640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07000a37b0 0x7f07000a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:30.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.624+0000 7f07194af640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07000a37b0 0x7f07000a5c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07040096c0 tx=0x7f0704000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:30.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:30.624+0000 7f070a7fc640 1 -- 172.21.15.5:0/1853810226 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f070c01f340 con 0x7f0714109140 2024-09-15T22:10:31.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.038+0000 7f071b713640 1 -- 172.21.15.5:0/1853810226 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06f8005580 con 0x7f0714109140 2024-09-15T22:10:31.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.039+0000 7f070a7fc640 1 -- 172.21.15.5:0/1853810226 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f070c09c700 con 0x7f0714109140 2024-09-15T22:10:31.042 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:10:31.042 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:10:31.042 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:10:31.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.042+0000 7f071b713640 1 -- 172.21.15.5:0/1853810226 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07000a37b0 msgr2=0x7f07000a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:31.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.043+0000 7f071b713640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07000a37b0 0x7f07000a5c70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f07040096c0 tx=0x7f0704000bc0 comp rx=0 tx=0).stop 2024-09-15T22:10:31.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.043+0000 7f071b713640 1 -- 172.21.15.5:0/1853810226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0714109140 msgr2=0x7f0714076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:31.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.043+0000 7f071b713640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0714109140 0x7f0714076c40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f070c00ebf0 tx=0x7f070c00c6a0 comp rx=0 tx=0).stop 2024-09-15T22:10:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.043+0000 7f071b713640 1 -- 172.21.15.5:0/1853810226 shutdown_connections 2024-09-15T22:10:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.043+0000 7f071b713640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07000a37b0 0x7f07000a5c70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.043+0000 7f071b713640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0714077180 0x7f07141aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.043+0000 7f071b713640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0714109140 0x7f0714076c40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.043+0000 7f071b713640 1 --2- 172.21.15.5:0/1853810226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07141049c0 0x7f0714076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.043+0000 7f071b713640 1 -- 172.21.15.5:0/1853810226 >> 172.21.15.5:0/1853810226 conn(0x7f07140ffa50 msgr2=0x7f0714100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.043+0000 7f071b713640 1 -- 172.21.15.5:0/1853810226 shutdown_connections 2024-09-15T22:10:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:31.043+0000 7f071b713640 1 -- 172.21.15.5:0/1853810226 wait complete. 2024-09-15T22:10:31.205 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:31 smithi005 ceph-mon[30944]: pgmap v51: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2024-09-15T22:10:31.205 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:31 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1853810226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:31.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:31 smithi099 ceph-mon[34892]: pgmap v51: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2024-09-15T22:10:31.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:31 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1853810226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:31.344 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:31 smithi129 ceph-mon[35314]: pgmap v51: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2024-09-15T22:10:31.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:31 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1853810226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:10:31.903 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:10:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:33.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.170+0000 7f9b3b185640 1 -- 172.21.15.5:0/2046339442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b341015c0 msgr2=0x7f9b34101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.170+0000 7f9b3b185640 1 --2- 172.21.15.5:0/2046339442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b341015c0 0x7f9b34101a40 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009930 tx=0x7f9b2002a420 comp rx=0 tx=0).stop 2024-09-15T22:10:33.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.171+0000 7f9b3b185640 1 -- 172.21.15.5:0/2046339442 shutdown_connections 2024-09-15T22:10:33.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.171+0000 7f9b3b185640 1 --2- 172.21.15.5:0/2046339442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b34101f80 0x7f9b3410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.171+0000 7f9b3b185640 1 --2- 172.21.15.5:0/2046339442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b341015c0 0x7f9b34101a40 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.171+0000 7f9b3b185640 1 --2- 172.21.15.5:0/2046339442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b34105a40 0x7f9b34105e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.171+0000 7f9b3b185640 1 -- 172.21.15.5:0/2046339442 >> 172.21.15.5:0/2046339442 conn(0x7f9b340fba90 msgr2=0x7f9b340fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:33.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.171+0000 7f9b3b185640 1 -- 172.21.15.5:0/2046339442 shutdown_connections 2024-09-15T22:10:33.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.171+0000 7f9b3b185640 1 -- 172.21.15.5:0/2046339442 wait complete. 2024-09-15T22:10:33.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.172+0000 7f9b3b185640 1 Processor -- start 2024-09-15T22:10:33.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.172+0000 7f9b3b185640 1 -- start start 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.172+0000 7f9b3b185640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b341015c0 0x7f9b3410fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.172+0000 7f9b3b185640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b34101f80 0x7f9b3410e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.172+0000 7f9b3b185640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b34105a40 0x7f9b3410e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.172+0000 7f9b3b185640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b34111f40 con 0x7f9b341015c0 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.172+0000 7f9b3b185640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b34111dc0 con 0x7f9b34105a40 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b39722640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b34105a40 0x7f9b3410e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b33fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b34101f80 0x7f9b3410e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b39722640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b34105a40 0x7f9b3410e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:52152/0 (socket says 172.21.15.5:52152) 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b33fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b34101f80 0x7f9b3410e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:32810/0 (socket says 172.21.15.5:32810) 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b33fff640 1 -- 172.21.15.5:0/2856993631 learned_addr learned my addr 172.21.15.5:0/2856993631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b3b185640 1 -- 172.21.15.5:0/2856993631 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b341120c0 con 0x7f9b34101f80 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b38f21640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b341015c0 0x7f9b3410fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b33fff640 1 -- 172.21.15.5:0/2856993631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b34105a40 msgr2=0x7f9b3410e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b33fff640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b34105a40 0x7f9b3410e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b33fff640 1 -- 172.21.15.5:0/2856993631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b341015c0 msgr2=0x7f9b3410fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b33fff640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b341015c0 0x7f9b3410fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.173+0000 7f9b33fff640 1 -- 172.21.15.5:0/2856993631 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3410f000 con 0x7f9b34101f80 2024-09-15T22:10:33.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.174+0000 7f9b33fff640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b34101f80 0x7f9b3410e3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009230 tx=0x7f9b200045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:33.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.174+0000 7f9b31ffb640 1 -- 172.21.15.5:0/2856993631 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20040070 con 0x7f9b34101f80 2024-09-15T22:10:33.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.174+0000 7f9b31ffb640 1 -- 172.21.15.5:0/2856993631 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b20031c80 con 0x7f9b34101f80 2024-09-15T22:10:33.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.174+0000 7f9b3b185640 1 -- 172.21.15.5:0/2856993631 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b341af400 con 0x7f9b34101f80 2024-09-15T22:10:33.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.174+0000 7f9b31ffb640 1 -- 172.21.15.5:0/2856993631 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b20035020 con 0x7f9b34101f80 2024-09-15T22:10:33.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.174+0000 7f9b3b185640 1 -- 172.21.15.5:0/2856993631 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b341af7a0 con 0x7f9b34101f80 2024-09-15T22:10:33.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.178+0000 7f9b3d55e740 1 Processor -- start 2024-09-15T22:10:33.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.178+0000 7f9b3d55e740 1 -- start start 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236e8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236e8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236fdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b3d55e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55592364e2f0 con 0x5559236fbb80 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b3d55e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5559235f5cf0 con 0x5559236e85d0 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b3d55e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55592364a2b0 con 0x5559236e9b40 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b39722640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b39722640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236e8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:32820/0 (socket says 172.21.15.5:32820) 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b39722640 1 -- 172.21.15.5:0/1636586444 learned_addr learned my addr 172.21.15.5:0/1636586444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b38f21640 1 --2- 172.21.15.5:0/1636586444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236e8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b31ffb640 1 -- 172.21.15.5:0/2856993631 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9b20043070 con 0x7f9b34101f80 2024-09-15T22:10:33.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b39722640 1 -- 172.21.15.5:0/1636586444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 msgr2=0x5559236e8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b39722640 1 --2- 172.21.15.5:0/1636586444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236e8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b39722640 1 -- 172.21.15.5:0/1636586444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 msgr2=0x5559236fdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b39722640 1 --2- 172.21.15.5:0/1636586444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236fdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b39722640 1 -- 172.21.15.5:0/1636586444 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559235da3a0 con 0x5559236e9b40 2024-09-15T22:10:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.179+0000 7f9b38f21640 1 --2- 172.21.15.5:0/1636586444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:33.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.180+0000 7f9b39722640 1 --2- 172.21.15.5:0/1636586444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236e8090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009f90 tx=0x7f9b2c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd08225bce075c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.180+0000 7f9b1effd640 1 -- 172.21.15.5:0/1636586444 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c004d20 con 0x5559236e9b40 2024-09-15T22:10:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.180+0000 7f9b1effd640 1 -- 172.21.15.5:0/1636586444 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2c004050 con 0x5559236e9b40 2024-09-15T22:10:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b1effd640 1 -- 172.21.15.5:0/1636586444 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c005260 con 0x5559236e9b40 2024-09-15T22:10:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b31ffb640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b240a3430 0x7f9b240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1636586444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 msgr2=0x5559236e8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1636586444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236e8090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009f90 tx=0x7f9b2c0118d0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1636586444 shutdown_connections 2024-09-15T22:10:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1636586444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236fdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1636586444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236e8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1636586444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236e8090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1636586444 >> 172.21.15.5:0/1636586444 conn(0x5559236b1310 msgr2=0x5559236e4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:33.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1636586444 shutdown_connections 2024-09-15T22:10:33.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b38f21640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b240a3430 0x7f9b240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b31ffb640 1 -- 172.21.15.5:0/2856993631 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f9b200f0ac0 con 0x7f9b34101f80 2024-09-15T22:10:33.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.181+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1636586444 wait complete. 2024-09-15T22:10:33.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.182+0000 7f9b3d55e740 1 -- start start 2024-09-15T22:10:33.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.182+0000 7f9b3d55e740 1 Processor -- start 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.182+0000 7f9b38f21640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b240a3430 0x7f9b240a58f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5559236e5ac0 tx=0x7f9b28005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236fb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236f8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236f9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b3d55e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559236fe8d0 con 0x5559236fbb80 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b3d55e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555923609060 con 0x5559236e85d0 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b3d55e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5559236fea50 con 0x5559236e9b40 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b3d55e740 10 client.0 _ll_register_callbacks cb 0x55592335c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b39722640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b39722640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236f9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55120/0 (socket says 172.21.15.5:55120) 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b39722640 1 -- 172.21.15.5:0/1589501252 learned_addr learned my addr 172.21.15.5:0/1589501252 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b33fff640 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236f8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b38f21640 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236fb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b33fff640 1 -- 172.21.15.5:0/1589501252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 msgr2=0x5559236fb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b33fff640 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236fb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b33fff640 1 -- 172.21.15.5:0/1589501252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 msgr2=0x5559236f9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b33fff640 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236f9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.183+0000 7f9b33fff640 1 -- 172.21.15.5:0/1589501252 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559233bf3c0 con 0x5559236e9b40 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.184+0000 7f9b33fff640 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236f8880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9b200ede10 tx=0x7f9b200bc8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=445d60158b2a1370 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.184+0000 7f9b13fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.184+0000 7f9b13fff640 1 -- 172.21.15.5:0/1589501252 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2004a040 con 0x5559236e9b40 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.184+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1589501252 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b2c004050 con 0x5559236e9b40 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.184+0000 7f9b13fff640 1 -- 172.21.15.5:0/1589501252 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f9b200ee090 con 0x5559236e9b40 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.184+0000 7f9b13fff640 1 -- 172.21.15.5:0/1589501252 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b200ee290 con 0x5559236e9b40 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.184+0000 7f9b3d55e740 10 client.25315 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:33.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.184+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1589501252 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5559236fe920 con 0x5559236e9b40 2024-09-15T22:10:33.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.184+0000 7f9b13fff640 1 -- 172.21.15.5:0/1589501252 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b2c004050 con 0x5559236e9b40 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.185+0000 7f9b13fff640 1 -- 172.21.15.5:0/1589501252 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9b2003a040 con 0x5559236e9b40 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.185+0000 7f9b3d55e740 10 client.25315 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.185+0000 7f9b3d55e740 10 client.25315 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.185+0000 7f9b3d55e740 20 client.25315 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.185+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55592370b720 0x55592370db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.185+0000 7f9b3d55e740 4 client.25315 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.185+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1589501252 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55592365b860 con 0x55592370b720 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.185+0000 7f9b38f21640 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55592370b720 0x55592370db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.186+0000 7f9b38f21640 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55592370b720 0x55592370db10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ccab17ffb6c010c8 server_cookie=bc497f628971084c in_seq=0 out_seq=0 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.186+0000 7f9b13fff640 10 client.25315 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.186+0000 7f9b13fff640 1 -- 172.21.15.5:0/1589501252 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55592365b860 con 0x55592370b720 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.186+0000 7f9b13fff640 10 client.25315 handle_command_reply: tid=0 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.186+0000 7f9b3d55e740 1 client.25315 shutdown 2024-09-15T22:10:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.186+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1589501252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 msgr2=0x5559236f8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.186+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236f8880 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9b200ede10 tx=0x7f9b200bc8e0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.187+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1589501252 shutdown_connections 2024-09-15T22:10:33.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.187+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55592370b720 0x55592370db10 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.187+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236f9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.187+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236f8880 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.187+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1589501252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236fb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.187+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1589501252 >> 172.21.15.5:0/1589501252 conn(0x5559236b1310 msgr2=0x5559236ea9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:33.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.187+0000 7f9b13fff640 0 client.25315 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:10:33.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.187+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1589501252 shutdown_connections 2024-09-15T22:10:33.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.187+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1589501252 wait complete. 2024-09-15T22:10:33.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.187+0000 7f9b3d55e740 20 client.25315 trim_cache size 0 max 16384 2024-09-15T22:10:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.210+0000 7f9b3d55e740 1 Processor -- start 2024-09-15T22:10:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.210+0000 7f9b3d55e740 1 -- start start 2024-09-15T22:10:33.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.210+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236e4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.210+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x555923713400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.211+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x555923713940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.211+0000 7f9b38f21640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236e4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.211+0000 7f9b38f21640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236e4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:52184/0 (socket says 172.21.15.5:52184) 2024-09-15T22:10:33.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.211+0000 7f9b33fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x555923713400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.211+0000 7f9b33fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x555923713400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:32840/0 (socket says 172.21.15.5:32840) 2024-09-15T22:10:33.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.211+0000 7f9b39722640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x555923713940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.211+0000 7f9b38f21640 1 -- 172.21.15.5:0/3076233597 learned_addr learned my addr 172.21.15.5:0/3076233597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.211+0000 7f9b3d55e740 1 -- 172.21.15.5:0/3076233597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b341b7e40 con 0x5559236fbb80 2024-09-15T22:10:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.211+0000 7f9b3d55e740 1 -- 172.21.15.5:0/3076233597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b341b5710 con 0x5559236e85d0 2024-09-15T22:10:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.211+0000 7f9b3d55e740 1 -- 172.21.15.5:0/3076233597 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b341b3560 con 0x5559236e9b40 2024-09-15T22:10:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.212+0000 7f9b38f21640 1 -- 172.21.15.5:0/3076233597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 msgr2=0x555923713400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.212+0000 7f9b38f21640 1 --2- 172.21.15.5:0/3076233597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x555923713400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.212+0000 7f9b38f21640 1 -- 172.21.15.5:0/3076233597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 msgr2=0x555923713940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.212+0000 7f9b38f21640 1 --2- 172.21.15.5:0/3076233597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x555923713940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.212+0000 7f9b38f21640 1 -- 172.21.15.5:0/3076233597 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559236f9380 con 0x5559236e85d0 2024-09-15T22:10:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.213+0000 7f9b38f21640 1 --2- 172.21.15.5:0/3076233597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236e4df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9b280138f0 tx=0x7f9b28016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75859dc3d85ca773 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.213+0000 7f9b1e7fc640 1 -- 172.21.15.5:0/3076233597 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2801c940 con 0x5559236e85d0 2024-09-15T22:10:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.213+0000 7f9b1e7fc640 1 -- 172.21.15.5:0/3076233597 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b28005020 con 0x5559236e85d0 2024-09-15T22:10:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.213+0000 7f9b1e7fc640 1 -- 172.21.15.5:0/3076233597 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28004e20 con 0x5559236e85d0 2024-09-15T22:10:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.214+0000 7f9b3d55e740 1 -- 172.21.15.5:0/3076233597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 msgr2=0x5559236e4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.214+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/3076233597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236e4df0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9b280138f0 tx=0x7f9b28016000 comp rx=0 tx=0).stop 2024-09-15T22:10:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.216+0000 7f9b3d55e740 1 -- 172.21.15.5:0/3076233597 shutdown_connections 2024-09-15T22:10:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.217+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/3076233597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x555923713940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.217+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/3076233597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x555923713400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.217+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/3076233597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236e4df0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.217+0000 7f9b3d55e740 1 -- 172.21.15.5:0/3076233597 >> 172.21.15.5:0/3076233597 conn(0x5559236b1310 msgr2=0x5559236e46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.217+0000 7f9b3d55e740 1 -- 172.21.15.5:0/3076233597 shutdown_connections 2024-09-15T22:10:33.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.217+0000 7f9b3d55e740 1 -- 172.21.15.5:0/3076233597 wait complete. 2024-09-15T22:10:33.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.218+0000 7f9b3d55e740 1 -- start start 2024-09-15T22:10:33.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.218+0000 7f9b3d55e740 1 Processor -- start 2024-09-15T22:10:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.219+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.219+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236f5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.219+0000 7f9b3d55e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236f6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.219+0000 7f9b3d55e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b341b3bd0 con 0x5559236fbb80 2024-09-15T22:10:33.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.219+0000 7f9b3d55e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b341b5040 con 0x5559236e85d0 2024-09-15T22:10:33.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.219+0000 7f9b3d55e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b341b42a0 con 0x5559236e9b40 2024-09-15T22:10:33.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.220+0000 7f9b3d55e740 10 client.0 _ll_register_callbacks cb 0x7f9b200095d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:33.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.220+0000 7f9b39722640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.220+0000 7f9b39722640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236f6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55132/0 (socket says 172.21.15.5:55132) 2024-09-15T22:10:33.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.220+0000 7f9b39722640 1 -- 172.21.15.5:0/1925675341 learned_addr learned my addr 172.21.15.5:0/1925675341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:33.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.220+0000 7f9b38f21640 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.221+0000 7f9b33fff640 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236f5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.221+0000 7f9b38f21640 1 -- 172.21.15.5:0/1925675341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 msgr2=0x5559236f5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.221+0000 7f9b38f21640 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236f5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.221+0000 7f9b38f21640 1 -- 172.21.15.5:0/1925675341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 msgr2=0x5559236f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.221+0000 7f9b38f21640 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.221+0000 7f9b38f21640 1 -- 172.21.15.5:0/1925675341 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559237754d0 con 0x5559236e85d0 2024-09-15T22:10:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.222+0000 7f9b38f21640 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236f59c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9b2801c810 tx=0x7f9b280047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b20d1b522cee9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.222+0000 7f9b13fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:10:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.222+0000 7f9b13fff640 1 -- 172.21.15.5:0/1925675341 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28037020 con 0x5559236e85d0 2024-09-15T22:10:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.223+0000 7f9b13fff640 1 -- 172.21.15.5:0/1925675341 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f9b28004a60 con 0x5559236e85d0 2024-09-15T22:10:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.224+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1925675341 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b28005020 con 0x5559236e85d0 2024-09-15T22:10:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.224+0000 7f9b13fff640 1 -- 172.21.15.5:0/1925675341 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b28021020 con 0x5559236e85d0 2024-09-15T22:10:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.224+0000 7f9b3d55e740 10 client.25328 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.224+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1925675341 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5559237767b0 con 0x5559236e85d0 2024-09-15T22:10:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.225+0000 7f9b13fff640 1 -- 172.21.15.5:0/1925675341 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b28005020 con 0x5559236e85d0 2024-09-15T22:10:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.225+0000 7f9b13fff640 1 -- 172.21.15.5:0/1925675341 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9b2801a040 con 0x5559236e85d0 2024-09-15T22:10:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.226+0000 7f9b3d55e740 10 client.25328 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.226+0000 7f9b3d55e740 10 client.25328 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:10:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.226+0000 7f9b3d55e740 20 client.25328 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.226+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555923776f80 0x5559237793a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.226+0000 7f9b3d55e740 4 client.25328 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:10:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.226+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1925675341 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555923655d70 con 0x555923776f80 2024-09-15T22:10:33.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.231+0000 7f9b33fff640 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555923776f80 0x5559237793a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.236+0000 7f9b13fff640 10 client.25328 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:10:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.236+0000 7f9b33fff640 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555923776f80 0x5559237793a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=642adfcb64eb40ab server_cookie=cc70aac03ff4fdb1 in_seq=0 out_seq=0 2024-09-15T22:10:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.237+0000 7f9b13fff640 1 -- 172.21.15.5:0/1925675341 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555923655d70 con 0x555923776f80 2024-09-15T22:10:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.237+0000 7f9b13fff640 10 client.25328 handle_command_reply: tid=0 2024-09-15T22:10:33.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.237+0000 7f9b3d55e740 1 client.25328 shutdown 2024-09-15T22:10:33.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.239+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1925675341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 msgr2=0x5559236f59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.239+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236f59c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9b2801c810 tx=0x7f9b280047b0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.239+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1925675341 shutdown_connections 2024-09-15T22:10:33.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.239+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555923776f80 0x5559237793a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.239+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559236fbb80 0x5559236f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.239+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5559236e9b40 0x5559236f5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.239+0000 7f9b3d55e740 1 --2- 172.21.15.5:0/1925675341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5559236e85d0 0x5559236f59c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.239+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1925675341 >> 172.21.15.5:0/1925675341 conn(0x5559236b1310 msgr2=0x5559236fa900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:33.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.239+0000 7f9b13fff640 0 client.25328 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:10:33.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.240+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1925675341 shutdown_connections 2024-09-15T22:10:33.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.240+0000 7f9b3d55e740 1 -- 172.21.15.5:0/1925675341 wait complete. 2024-09-15T22:10:33.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.240+0000 7f9b3d55e740 20 client.25328 trim_cache size 0 max 16384 2024-09-15T22:10:33.243 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:10:33.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 -- 172.21.15.5:0/2856993631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b240a3430 msgr2=0x7f9b240a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b240a3430 0x7f9b240a58f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5559236e5ac0 tx=0x7f9b28005f90 comp rx=0 tx=0).stop 2024-09-15T22:10:33.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 -- 172.21.15.5:0/2856993631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b34101f80 msgr2=0x7f9b3410e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:33.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b34101f80 0x7f9b3410e3c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009230 tx=0x7f9b200045e0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 -- 172.21.15.5:0/2856993631 shutdown_connections 2024-09-15T22:10:33.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b240a3430 0x7f9b240a58f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b34105a40 0x7f9b3410e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b34101f80 0x7f9b3410e3c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 --2- 172.21.15.5:0/2856993631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b341015c0 0x7f9b3410fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:33.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 -- 172.21.15.5:0/2856993631 >> 172.21.15.5:0/2856993631 conn(0x7f9b340fba90 msgr2=0x7f9b340fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:33.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 -- 172.21.15.5:0/2856993631 shutdown_connections 2024-09-15T22:10:33.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:33.242+0000 7f9b3b185640 1 -- 172.21.15.5:0/2856993631 wait complete. 2024-09-15T22:10:33.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:33 smithi099 ceph-mon[34892]: pgmap v52: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 935 op/s 2024-09-15T22:10:33.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:33 smithi129 ceph-mon[35314]: pgmap v52: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 935 op/s 2024-09-15T22:10:33.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:33 smithi005 ceph-mon[30944]: pgmap v52: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 935 op/s 2024-09-15T22:10:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 65.0 s, 2824.4 tps, lat 35.889 ms stddev 45.214 2024-09-15T22:10:34.803 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:10:35.250 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:35.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:35 smithi099 ceph-mon[34892]: pgmap v53: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 935 op/s 2024-09-15T22:10:35.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:35 smithi129 ceph-mon[35314]: pgmap v53: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 935 op/s 2024-09-15T22:10:35.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:35 smithi005 ceph-mon[30944]: pgmap v53: 129 pgs: 129 active+clean; 1.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 935 op/s 2024-09-15T22:10:35.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.878+0000 7efe3ce95640 1 -- 172.21.15.5:0/2910191434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe38101410 msgr2=0x7efe3810d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.878+0000 7efe3ce95640 1 --2- 172.21.15.5:0/2910191434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe38101410 0x7efe3810d940 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efe30009cd0 tx=0x7efe3002a8f0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.879+0000 7efe3ce95640 1 -- 172.21.15.5:0/2910191434 shutdown_connections 2024-09-15T22:10:35.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.879+0000 7efe3ce95640 1 --2- 172.21.15.5:0/2910191434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe38101410 0x7efe3810d940 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.879+0000 7efe3ce95640 1 --2- 172.21.15.5:0/2910191434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe38100a70 0x7efe38100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.879+0000 7efe3ce95640 1 --2- 172.21.15.5:0/2910191434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe38106a70 0x7efe38106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.879+0000 7efe3ce95640 1 -- 172.21.15.5:0/2910191434 >> 172.21.15.5:0/2910191434 conn(0x7efe380fc790 msgr2=0x7efe380febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:35.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.879+0000 7efe3ce95640 1 -- 172.21.15.5:0/2910191434 shutdown_connections 2024-09-15T22:10:35.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.879+0000 7efe3ce95640 1 -- 172.21.15.5:0/2910191434 wait complete. 2024-09-15T22:10:35.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.880+0000 7efe3ce95640 1 Processor -- start 2024-09-15T22:10:35.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.880+0000 7efe3ce95640 1 -- start start 2024-09-15T22:10:35.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.880+0000 7efe3ce95640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe38100a70 0x7efe381a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.880+0000 7efe3ce95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe38101410 0x7efe381a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.880+0000 7efe3ce95640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe38106a70 0x7efe3819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.880+0000 7efe3ce95640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe38118590 con 0x7efe38101410 2024-09-15T22:10:35.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.880+0000 7efe3ce95640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe38118410 con 0x7efe38106a70 2024-09-15T22:10:35.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.880+0000 7efe3ce95640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efe38118710 con 0x7efe38100a70 2024-09-15T22:10:35.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.881+0000 7efe3659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe38100a70 0x7efe381a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.881+0000 7efe3659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe38100a70 0x7efe381a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43144/0 (socket says 172.21.15.5:43144) 2024-09-15T22:10:35.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.881+0000 7efe3659c640 1 -- 172.21.15.5:0/3871303022 learned_addr learned my addr 172.21.15.5:0/3871303022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:35.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.881+0000 7efe36d9d640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe38106a70 0x7efe3819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.881+0000 7efe3659c640 1 -- 172.21.15.5:0/3871303022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe38106a70 msgr2=0x7efe3819e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.881+0000 7efe3659c640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe38106a70 0x7efe3819e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.881+0000 7efe3659c640 1 -- 172.21.15.5:0/3871303022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe38101410 msgr2=0x7efe381a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.881+0000 7efe3659c640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe38101410 0x7efe381a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.881+0000 7efe3659c640 1 -- 172.21.15.5:0/3871303022 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3819efb0 con 0x7efe38100a70 2024-09-15T22:10:35.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.882+0000 7efe3659c640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe38100a70 0x7efe381a46c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00b4a0 tx=0x7efe2c00b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:35.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.882+0000 7efe277fe640 1 -- 172.21.15.5:0/3871303022 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c014070 con 0x7efe38100a70 2024-09-15T22:10:35.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.882+0000 7efe277fe640 1 -- 172.21.15.5:0/3871303022 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe2c010af0 con 0x7efe38100a70 2024-09-15T22:10:35.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.882+0000 7efe277fe640 1 -- 172.21.15.5:0/3871303022 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c017c40 con 0x7efe38100a70 2024-09-15T22:10:35.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.882+0000 7efe3ce95640 1 -- 172.21.15.5:0/3871303022 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3819f2a0 con 0x7efe38100a70 2024-09-15T22:10:35.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.882+0000 7efe3ce95640 1 -- 172.21.15.5:0/3871303022 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe381ab590 con 0x7efe38100a70 2024-09-15T22:10:35.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.886+0000 7efe277fe640 1 -- 172.21.15.5:0/3871303022 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7efe2c017070 con 0x7efe38100a70 2024-09-15T22:10:35.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.886+0000 7efe3f26e740 1 Processor -- start 2024-09-15T22:10:35.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.887+0000 7efe3f26e740 1 -- start start 2024-09-15T22:10:35.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.887+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 0x5618dd127090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.887+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd1275d0 0x5618dd127a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.887+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd132370 0x5618dd134760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.887+0000 7efe3f26e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618dd08d2f0 con 0x5618dd1275d0 2024-09-15T22:10:35.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe3f26e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5618dd034cf0 con 0x5618dd132370 2024-09-15T22:10:35.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe3f26e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5618dd0892b0 con 0x5618dd128b40 2024-09-15T22:10:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe36d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 0x5618dd127090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe3659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd1275d0 0x5618dd127a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe35d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd132370 0x5618dd134760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe36d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 0x5618dd127090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43158/0 (socket says 172.21.15.5:43158) 2024-09-15T22:10:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe36d9d640 1 -- 172.21.15.5:0/1928500004 learned_addr learned my addr 172.21.15.5:0/1928500004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe35d9b640 1 -- 172.21.15.5:0/1928500004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 msgr2=0x5618dd127090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe35d9b640 1 --2- 172.21.15.5:0/1928500004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 0x5618dd127090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe35d9b640 1 -- 172.21.15.5:0/1928500004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd1275d0 msgr2=0x5618dd127a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe35d9b640 1 --2- 172.21.15.5:0/1928500004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd1275d0 0x5618dd127a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe35d9b640 1 -- 172.21.15.5:0/1928500004 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618dd0193a0 con 0x5618dd132370 2024-09-15T22:10:35.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe35d9b640 1 --2- 172.21.15.5:0/1928500004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd132370 0x5618dd134760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe381060d0 tx=0x7efe2800df40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aebe3cc8dc1b2d89 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:35.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe277fe640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efe200a3560 0x7efe200a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.888+0000 7efe24ff9640 1 -- 172.21.15.5:0/1928500004 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28009a80 con 0x5618dd132370 2024-09-15T22:10:35.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.889+0000 7efe24ff9640 1 -- 172.21.15.5:0/1928500004 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe28009c20 con 0x5618dd132370 2024-09-15T22:10:35.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.889+0000 7efe277fe640 1 -- 172.21.15.5:0/3871303022 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7efe2c0cf240 con 0x7efe38100a70 2024-09-15T22:10:35.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.889+0000 7efe36d9d640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efe200a3560 0x7efe200a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.889+0000 7efe24ff9640 1 -- 172.21.15.5:0/1928500004 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28016400 con 0x5618dd132370 2024-09-15T22:10:35.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.890+0000 7efe3f26e740 1 -- 172.21.15.5:0/1928500004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd132370 msgr2=0x5618dd134760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.890+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1928500004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd132370 0x5618dd134760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efe381060d0 tx=0x7efe2800df40 comp rx=0 tx=0).stop 2024-09-15T22:10:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.890+0000 7efe3f26e740 1 -- 172.21.15.5:0/1928500004 shutdown_connections 2024-09-15T22:10:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.890+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1928500004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd132370 0x5618dd134760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.890+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1928500004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd1275d0 0x5618dd127a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.890+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1928500004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 0x5618dd127090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.890+0000 7efe3f26e740 1 -- 172.21.15.5:0/1928500004 >> 172.21.15.5:0/1928500004 conn(0x5618dd0f0310 msgr2=0x5618dd123c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.890+0000 7efe36d9d640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efe200a3560 0x7efe200a5a20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe30009a60 tx=0x7efe30004a40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.890+0000 7efe3f26e740 1 -- 172.21.15.5:0/1928500004 shutdown_connections 2024-09-15T22:10:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.890+0000 7efe3f26e740 1 -- 172.21.15.5:0/1928500004 wait complete. 2024-09-15T22:10:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3f26e740 1 -- start start 2024-09-15T22:10:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3f26e740 1 Processor -- start 2024-09-15T22:10:35.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd1275d0 0x5618dd131c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd128b40 0x5618dd12eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 0x5618dd12f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3f26e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5618dd135110 con 0x5618dd132370 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3f26e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5618dd048060 con 0x5618dd128b40 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3f26e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5618dd135290 con 0x5618dd1275d0 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd1275d0 0x5618dd131c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3f26e740 10 client.0 _ll_register_callbacks cb 0x5618dcd9b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd1275d0 0x5618dd131c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43172/0 (socket says 172.21.15.5:43172) 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.891+0000 7efe3659c640 1 -- 172.21.15.5:0/1210932365 learned_addr learned my addr 172.21.15.5:0/1210932365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.892+0000 7efe35d9b640 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd128b40 0x5618dd12eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.892+0000 7efe36d9d640 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 0x5618dd12f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.892+0000 7efe35d9b640 1 -- 172.21.15.5:0/1210932365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd1275d0 msgr2=0x5618dd131c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.892+0000 7efe35d9b640 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd1275d0 0x5618dd131c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.892+0000 7efe35d9b640 1 -- 172.21.15.5:0/1210932365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 msgr2=0x5618dd12f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:35.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.892+0000 7efe35d9b640 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 0x5618dd12f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.892+0000 7efe35d9b640 1 -- 172.21.15.5:0/1210932365 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618dcdfe3c0 con 0x5618dd128b40 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.892+0000 7efe36d9d640 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 0x5618dd12f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.893+0000 7efe35d9b640 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd128b40 0x5618dd12eef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe28018040 tx=0x7efe28009aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3770b06eb10dc5d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.893+0000 7efe1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.893+0000 7efe1dffb640 1 -- 172.21.15.5:0/1210932365 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28024020 con 0x5618dd128b40 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.893+0000 7efe3f26e740 1 -- 172.21.15.5:0/1210932365 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe28009c20 con 0x5618dd128b40 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.893+0000 7efe1dffb640 1 -- 172.21.15.5:0/1210932365 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7efe28005670 con 0x5618dd128b40 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.893+0000 7efe1dffb640 1 -- 172.21.15.5:0/1210932365 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efe2802f050 con 0x5618dd128b40 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.893+0000 7efe3f26e740 10 client.25340 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.893+0000 7efe3f26e740 1 -- 172.21.15.5:0/1210932365 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5618dd140af0 con 0x5618dd128b40 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.894+0000 7efe1dffb640 1 -- 172.21.15.5:0/1210932365 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe28009c20 con 0x5618dd128b40 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.894+0000 7efe1dffb640 1 -- 172.21.15.5:0/1210932365 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efe28003e50 con 0x5618dd128b40 2024-09-15T22:10:35.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.894+0000 7efe3f26e740 10 client.25340 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.894+0000 7efe3f26e740 10 client.25340 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:10:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.894+0000 7efe3f26e740 20 client.25340 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.894+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5618dd141f00 0x5618dd1442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.894+0000 7efe3f26e740 4 client.25340 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:10:35.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.894+0000 7efe3f26e740 1 -- 172.21.15.5:0/1210932365 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618dd09a860 con 0x5618dd141f00 2024-09-15T22:10:35.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.894+0000 7efe3659c640 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5618dd141f00 0x5618dd1442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.895+0000 7efe3659c640 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5618dd141f00 0x5618dd1442f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f3e3dc47999a0f79 server_cookie=2038a2e34bb9e2f7 in_seq=0 out_seq=0 2024-09-15T22:10:35.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.895+0000 7efe1dffb640 10 client.25340 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:10:35.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.896+0000 7efe1dffb640 1 -- 172.21.15.5:0/1210932365 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618dd09a860 con 0x5618dd141f00 2024-09-15T22:10:35.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.896+0000 7efe1dffb640 10 client.25340 handle_command_reply: tid=0 2024-09-15T22:10:35.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.896+0000 7efe3f26e740 1 client.25340 shutdown 2024-09-15T22:10:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.898+0000 7efe3f26e740 1 -- 172.21.15.5:0/1210932365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd128b40 msgr2=0x5618dd12eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.898+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd128b40 0x5618dd12eef0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe28018040 tx=0x7efe28009aa0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.898+0000 7efe3f26e740 1 -- 172.21.15.5:0/1210932365 shutdown_connections 2024-09-15T22:10:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.898+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5618dd141f00 0x5618dd1442f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.898+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 0x5618dd12f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.898+0000 7efe1dffb640 0 client.25340 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:10:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.898+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd128b40 0x5618dd12eef0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.898+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1210932365 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd1275d0 0x5618dd131c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.898+0000 7efe3f26e740 1 -- 172.21.15.5:0/1210932365 >> 172.21.15.5:0/1210932365 conn(0x5618dd0f0310 msgr2=0x5618dd129960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.899+0000 7efe3f26e740 1 -- 172.21.15.5:0/1210932365 shutdown_connections 2024-09-15T22:10:35.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.899+0000 7efe3f26e740 1 -- 172.21.15.5:0/1210932365 wait complete. 2024-09-15T22:10:35.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.899+0000 7efe3f26e740 20 client.25340 trim_cache size 0 max 16384 2024-09-15T22:10:35.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.930+0000 7efe3f26e740 1 Processor -- start 2024-09-15T22:10:35.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.930+0000 7efe3f26e740 1 -- start start 2024-09-15T22:10:35.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.930+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd1275d0 0x5618dd13e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.930+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 0x5618dd149910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.930+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 0x5618dd149e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe3f26e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b7f60 con 0x5618dd132370 2024-09-15T22:10:35.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe3f26e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b5830 con 0x5618dd1275d0 2024-09-15T22:10:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe3f26e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b3680 con 0x5618dd128b40 2024-09-15T22:10:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe35d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 0x5618dd149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe35d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 0x5618dd149910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43176/0 (socket says 172.21.15.5:43176) 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe35d9b640 1 -- 172.21.15.5:0/1564442009 learned_addr learned my addr 172.21.15.5:0/1564442009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe3659c640 1 --2- 172.21.15.5:0/1564442009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd1275d0 0x5618dd13e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe36d9d640 1 --2- 172.21.15.5:0/1564442009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 0x5618dd149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe3659c640 1 -- 172.21.15.5:0/1564442009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 msgr2=0x5618dd149910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe3659c640 1 --2- 172.21.15.5:0/1564442009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 0x5618dd149910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe3659c640 1 -- 172.21.15.5:0/1564442009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 msgr2=0x5618dd149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe3659c640 1 --2- 172.21.15.5:0/1564442009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 0x5618dd149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe3659c640 1 -- 172.21.15.5:0/1564442009 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618dd12ab40 con 0x5618dd1275d0 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe36d9d640 1 --2- 172.21.15.5:0/1564442009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 0x5618dd149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.931+0000 7efe35d9b640 1 --2- 172.21.15.5:0/1564442009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 0x5618dd149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.932+0000 7efe3659c640 1 --2- 172.21.15.5:0/1564442009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd1275d0 0x5618dd13e400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efe2c09af90 tx=0x7efe2c0d1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed295dfab936135 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.932+0000 7efe1ffff640 1 -- 172.21.15.5:0/1564442009 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c01e040 con 0x5618dd1275d0 2024-09-15T22:10:35.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.932+0000 7efe1ffff640 1 -- 172.21.15.5:0/1564442009 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe2c017e40 con 0x5618dd1275d0 2024-09-15T22:10:35.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.932+0000 7efe1ffff640 1 -- 172.21.15.5:0/1564442009 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0a3f00 con 0x5618dd1275d0 2024-09-15T22:10:35.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.933+0000 7efe3f26e740 1 -- 172.21.15.5:0/1564442009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd1275d0 msgr2=0x5618dd13e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.933+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1564442009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd1275d0 0x5618dd13e400 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efe2c09af90 tx=0x7efe2c0d1040 comp rx=0 tx=0).stop 2024-09-15T22:10:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.933+0000 7efe3f26e740 1 -- 172.21.15.5:0/1564442009 shutdown_connections 2024-09-15T22:10:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.933+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1564442009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd132370 0x5618dd149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.933+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1564442009 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd128b40 0x5618dd149910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.934+0000 7efe3f26e740 1 --2- 172.21.15.5:0/1564442009 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd1275d0 0x5618dd13e400 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.934+0000 7efe3f26e740 1 -- 172.21.15.5:0/1564442009 >> 172.21.15.5:0/1564442009 conn(0x5618dd0f0310 msgr2=0x5618dd12eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:35.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.936+0000 7efe3f26e740 1 -- 172.21.15.5:0/1564442009 shutdown_connections 2024-09-15T22:10:35.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.936+0000 7efe3f26e740 1 -- 172.21.15.5:0/1564442009 wait complete. 2024-09-15T22:10:35.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.936+0000 7efe3f26e740 1 -- start start 2024-09-15T22:10:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.937+0000 7efe3f26e740 1 Processor -- start 2024-09-15T22:10:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.937+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd1275d0 0x5618dd136700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.937+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd128b40 0x5618dd136c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.937+0000 7efe3f26e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd132370 0x5618dd1ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.937+0000 7efe3f26e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b3cf0 con 0x5618dd1275d0 2024-09-15T22:10:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.937+0000 7efe3659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd1275d0 0x5618dd136700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.937+0000 7efe3659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd1275d0 0x5618dd136700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37750/0 (socket says 172.21.15.5:37750) 2024-09-15T22:10:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.937+0000 7efe3659c640 1 -- 172.21.15.5:0/345614949 learned_addr learned my addr 172.21.15.5:0/345614949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.937+0000 7efe3f26e740 1 -- 172.21.15.5:0/345614949 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b5160 con 0x5618dd128b40 2024-09-15T22:10:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.937+0000 7efe36d9d640 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd132370 0x5618dd1ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.937+0000 7efe3f26e740 1 -- 172.21.15.5:0/345614949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efe381b43c0 con 0x5618dd132370 2024-09-15T22:10:35.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.938+0000 7efe3f26e740 10 client.0 _ll_register_callbacks cb 0x7efe2c0d2c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.938+0000 7efe35d9b640 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd128b40 0x5618dd136c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.938+0000 7efe36d9d640 1 -- 172.21.15.5:0/345614949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd128b40 msgr2=0x5618dd136c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.938+0000 7efe36d9d640 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd128b40 0x5618dd136c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.938+0000 7efe36d9d640 1 -- 172.21.15.5:0/345614949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd1275d0 msgr2=0x5618dd136700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.938+0000 7efe36d9d640 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd1275d0 0x5618dd136700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.938+0000 7efe36d9d640 1 -- 172.21.15.5:0/345614949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618dd124960 con 0x5618dd132370 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.938+0000 7efe36d9d640 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd132370 0x5618dd1ab8e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe30006010 tx=0x7efe3002a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5c67993bb98fa7e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.938+0000 7efe1dffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe1dffb640 1 -- 172.21.15.5:0/345614949 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30030cc0 con 0x5618dd132370 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe1dffb640 1 -- 172.21.15.5:0/345614949 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7efe30042030 con 0x5618dd132370 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe3f26e740 1 -- 172.21.15.5:0/345614949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe2c017e40 con 0x5618dd132370 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe1dffb640 1 -- 172.21.15.5:0/345614949 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efe30030e60 con 0x5618dd132370 2024-09-15T22:10:35.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe3f26e740 10 client.25333 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe3f26e740 1 -- 172.21.15.5:0/345614949 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5618dd1acd00 con 0x5618dd132370 2024-09-15T22:10:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe1dffb640 1 -- 172.21.15.5:0/345614949 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe2c017e40 con 0x5618dd132370 2024-09-15T22:10:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe1dffb640 1 -- 172.21.15.5:0/345614949 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efe3005f820 con 0x5618dd132370 2024-09-15T22:10:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe3f26e740 10 client.25333 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe3f26e740 10 client.25333 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:10:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe3f26e740 20 client.25333 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe3f26e740 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5618dd141f00 0x5618dd1af460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe3f26e740 4 client.25333 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:10:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.939+0000 7efe3f26e740 1 -- 172.21.15.5:0/345614949 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5618dd094d70 con 0x5618dd141f00 2024-09-15T22:10:35.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.940+0000 7efe3659c640 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5618dd141f00 0x5618dd1af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.940+0000 7efe3659c640 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5618dd141f00 0x5618dd1af460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b04466d674683f80 server_cookie=66c7f9e78ab16517 in_seq=0 out_seq=0 2024-09-15T22:10:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.940+0000 7efe1dffb640 10 client.25333 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:10:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.941+0000 7efe1dffb640 1 -- 172.21.15.5:0/345614949 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5618dd094d70 con 0x5618dd141f00 2024-09-15T22:10:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.941+0000 7efe1dffb640 10 client.25333 handle_command_reply: tid=0 2024-09-15T22:10:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.941+0000 7efe3f26e740 1 client.25333 shutdown 2024-09-15T22:10:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.941+0000 7efe3f26e740 1 -- 172.21.15.5:0/345614949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd132370 msgr2=0x5618dd1ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.941+0000 7efe3f26e740 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd132370 0x5618dd1ab8e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe30006010 tx=0x7efe3002a8f0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.941+0000 7efe3f26e740 1 -- 172.21.15.5:0/345614949 shutdown_connections 2024-09-15T22:10:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.942+0000 7efe3f26e740 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5618dd141f00 0x5618dd1af460 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.942+0000 7efe3f26e740 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5618dd132370 0x5618dd1ab8e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.942+0000 7efe3f26e740 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5618dd128b40 0x5618dd136c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.942+0000 7efe3f26e740 1 --2- 172.21.15.5:0/345614949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5618dd1275d0 0x5618dd136700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.942+0000 7efe3f26e740 1 -- 172.21.15.5:0/345614949 >> 172.21.15.5:0/345614949 conn(0x5618dd0f0310 msgr2=0x5618dd129110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.942+0000 7efe1dffb640 0 client.25333 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:10:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.942+0000 7efe3f26e740 1 -- 172.21.15.5:0/345614949 shutdown_connections 2024-09-15T22:10:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.942+0000 7efe3f26e740 1 -- 172.21.15.5:0/345614949 wait complete. 2024-09-15T22:10:35.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.942+0000 7efe3f26e740 20 client.25333 trim_cache size 0 max 16384 2024-09-15T22:10:35.947 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:10:35.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 -- 172.21.15.5:0/3871303022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efe200a3560 msgr2=0x7efe200a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efe200a3560 0x7efe200a5a20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efe30009a60 tx=0x7efe30004a40 comp rx=0 tx=0).stop 2024-09-15T22:10:35.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 -- 172.21.15.5:0/3871303022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe38100a70 msgr2=0x7efe381a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:35.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe38100a70 0x7efe381a46c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00b4a0 tx=0x7efe2c00b960 comp rx=0 tx=0).stop 2024-09-15T22:10:35.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 -- 172.21.15.5:0/3871303022 shutdown_connections 2024-09-15T22:10:35.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efe200a3560 0x7efe200a5a20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efe38106a70 0x7efe3819e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe38101410 0x7efe381a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 --2- 172.21.15.5:0/3871303022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efe38100a70 0x7efe381a46c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:35.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 -- 172.21.15.5:0/3871303022 >> 172.21.15.5:0/3871303022 conn(0x7efe380fc790 msgr2=0x7efe380fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:35.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 -- 172.21.15.5:0/3871303022 shutdown_connections 2024-09-15T22:10:35.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:35.943+0000 7efe3ce95640 1 -- 172.21.15.5:0/3871303022 wait complete. 2024-09-15T22:10:36.655 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:10:37.078 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:37 smithi005 ceph-mon[30944]: pgmap v54: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-09-15T22:10:37.098 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:37.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:37 smithi099 ceph-mon[34892]: pgmap v54: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-09-15T22:10:37.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:37 smithi129 ceph-mon[35314]: pgmap v54: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 19 MiB/s wr, 1.23k op/s 2024-09-15T22:10:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.707+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/2887086942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b481091f0 msgr2=0x7f4b481095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.707+0000 7f4b4f2b6640 1 --2- 172.21.15.5:0/2887086942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b481091f0 0x7f4b481095d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009be0 tx=0x7f4b4002a750 comp rx=0 tx=0).stop 2024-09-15T22:10:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.707+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/2887086942 shutdown_connections 2024-09-15T22:10:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.707+0000 7f4b4f2b6640 1 --2- 172.21.15.5:0/2887086942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b48114ab0 0x7f4b48116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.707+0000 7f4b4f2b6640 1 --2- 172.21.15.5:0/2887086942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b48109b10 0x7f4b48114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.707+0000 7f4b4f2b6640 1 --2- 172.21.15.5:0/2887086942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b481091f0 0x7f4b481095d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.707+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/2887086942 >> 172.21.15.5:0/2887086942 conn(0x7f4b4806e140 msgr2=0x7f4b4806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.707+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/2887086942 shutdown_connections 2024-09-15T22:10:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.707+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/2887086942 wait complete. 2024-09-15T22:10:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.708+0000 7f4b4f2b6640 1 Processor -- start 2024-09-15T22:10:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.708+0000 7f4b4f2b6640 1 -- start start 2024-09-15T22:10:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4f2b6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b481091f0 0x7f4b481a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4f2b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b48109b10 0x7f4b481a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4f2b6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b48114ab0 0x7f4b481a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4f2b6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4811a7a0 con 0x7f4b48109b10 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4f2b6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4811a620 con 0x7f4b48114ab0 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4f2b6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4811a920 con 0x7f4b481091f0 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4d052640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b481091f0 0x7f4b481a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4d052640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b481091f0 0x7f4b481a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43212/0 (socket says 172.21.15.5:43212) 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4d052640 1 -- 172.21.15.5:0/3133028293 learned_addr learned my addr 172.21.15.5:0/3133028293 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4d052640 1 -- 172.21.15.5:0/3133028293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b48114ab0 msgr2=0x7f4b481a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4d052640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b48114ab0 0x7f4b481a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4d052640 1 -- 172.21.15.5:0/3133028293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b48109b10 msgr2=0x7f4b481a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4d052640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b48109b10 0x7f4b481a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.709+0000 7f4b4d052640 1 -- 172.21.15.5:0/3133028293 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b481a5cd0 con 0x7f4b481091f0 2024-09-15T22:10:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.710+0000 7f4b4d052640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b481091f0 0x7f4b481a0d20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009d00 tx=0x7f4b400305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.710+0000 7f4b3a7fc640 1 -- 172.21.15.5:0/3133028293 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40036040 con 0x7f4b481091f0 2024-09-15T22:10:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.710+0000 7f4b3a7fc640 1 -- 172.21.15.5:0/3133028293 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b40003e50 con 0x7f4b481091f0 2024-09-15T22:10:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.710+0000 7f4b3a7fc640 1 -- 172.21.15.5:0/3133028293 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4003ca90 con 0x7f4b481091f0 2024-09-15T22:10:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.710+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/3133028293 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b481a5f60 con 0x7f4b481091f0 2024-09-15T22:10:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.710+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/3133028293 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b481a63f0 con 0x7f4b481091f0 2024-09-15T22:10:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.714+0000 7f4b3a7fc640 1 -- 172.21.15.5:0/3133028293 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4b40031660 con 0x7f4b481091f0 2024-09-15T22:10:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.715+0000 7f4b5168f740 1 Processor -- start 2024-09-15T22:10:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.715+0000 7f4b5168f740 1 -- start start 2024-09-15T22:10:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b5168f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b3a7fc640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4b340a35c0 0x7f4b340a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b5168f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dba1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b5168f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dbae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b4d853640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b4d853640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37788/0 (socket says 172.21.15.5:37788) 2024-09-15T22:10:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b4d853640 1 -- 172.21.15.5:0/1748455123 learned_addr learned my addr 172.21.15.5:0/1748455123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b5168f740 1 -- 172.21.15.5:0/1748455123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561d5db072f0 con 0x561d5dba2b40 2024-09-15T22:10:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b5168f740 1 -- 172.21.15.5:0/1748455123 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561d5daaecf0 con 0x561d5dbac370 2024-09-15T22:10:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b5168f740 1 -- 172.21.15.5:0/1748455123 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561d5db032b0 con 0x561d5dba15d0 2024-09-15T22:10:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b4d853640 1 --2- 172.21.15.5:0/1748455123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dbae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b3a7fc640 1 -- 172.21.15.5:0/3133028293 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4b400f46c0 con 0x7f4b481091f0 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.716+0000 7f4b4c851640 1 --2- 172.21.15.5:0/1748455123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dba1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.717+0000 7f4b4c851640 1 -- 172.21.15.5:0/1748455123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 msgr2=0x561d5dbae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.717+0000 7f4b4c851640 1 --2- 172.21.15.5:0/1748455123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dbae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.717+0000 7f4b4d052640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4b340a35c0 0x7f4b340a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.717+0000 7f4b4c851640 1 -- 172.21.15.5:0/1748455123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 msgr2=0x561d5dba1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.717+0000 7f4b4c851640 1 --2- 172.21.15.5:0/1748455123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.717+0000 7f4b4c851640 1 -- 172.21.15.5:0/1748455123 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d5da933a0 con 0x561d5dba15d0 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.717+0000 7f4b4d853640 1 --2- 172.21.15.5:0/1748455123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dbae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.717+0000 7f4b4c851640 1 --2- 172.21.15.5:0/1748455123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dba1a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009b10 tx=0x7f4b3c00e350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c6226d4ce405277 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.717+0000 7f4b337fe640 1 -- 172.21.15.5:0/1748455123 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c00ed50 con 0x561d5dba15d0 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.718+0000 7f4b4d052640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4b340a35c0 0x7f4b340a5a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4b400c8df0 tx=0x7f4b4002ac50 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.718+0000 7f4b337fe640 1 -- 172.21.15.5:0/1748455123 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b3c005020 con 0x561d5dba15d0 2024-09-15T22:10:37.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.718+0000 7f4b337fe640 1 -- 172.21.15.5:0/1748455123 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c004b30 con 0x561d5dba15d0 2024-09-15T22:10:37.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.719+0000 7f4b5168f740 1 -- 172.21.15.5:0/1748455123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 msgr2=0x561d5dba1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.719+0000 7f4b5168f740 1 --2- 172.21.15.5:0/1748455123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dba1a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c009b10 tx=0x7f4b3c00e350 comp rx=0 tx=0).stop 2024-09-15T22:10:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.719+0000 7f4b5168f740 1 -- 172.21.15.5:0/1748455123 shutdown_connections 2024-09-15T22:10:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.719+0000 7f4b5168f740 1 --2- 172.21.15.5:0/1748455123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dbae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.719+0000 7f4b5168f740 1 --2- 172.21.15.5:0/1748455123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dba1a10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.719+0000 7f4b5168f740 1 --2- 172.21.15.5:0/1748455123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.719+0000 7f4b5168f740 1 -- 172.21.15.5:0/1748455123 >> 172.21.15.5:0/1748455123 conn(0x561d5db6a310 msgr2=0x561d5db9dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.719+0000 7f4b5168f740 1 -- 172.21.15.5:0/1748455123 shutdown_connections 2024-09-15T22:10:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.719+0000 7f4b5168f740 1 -- 172.21.15.5:0/1748455123 wait complete. 2024-09-15T22:10:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.720+0000 7f4b5168f740 1 -- start start 2024-09-15T22:10:37.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.720+0000 7f4b5168f740 1 Processor -- start 2024-09-15T22:10:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.720+0000 7f4b5168f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dbabdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.721+0000 7f4b5168f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.721+0000 7f4b4d853640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dbabdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.721+0000 7f4b4d853640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dbabdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43220/0 (socket says 172.21.15.5:43220) 2024-09-15T22:10:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.721+0000 7f4b4d853640 1 -- 172.21.15.5:0/2247901723 learned_addr learned my addr 172.21.15.5:0/2247901723 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.721+0000 7f4b5168f740 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dba9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.721+0000 7f4b5168f740 1 -- 172.21.15.5:0/2247901723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561d5dbaf070 con 0x561d5dba2b40 2024-09-15T22:10:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.721+0000 7f4b5168f740 1 -- 172.21.15.5:0/2247901723 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561d5dac2060 con 0x561d5dbac370 2024-09-15T22:10:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.721+0000 7f4b5168f740 1 -- 172.21.15.5:0/2247901723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561d5dbaf1f0 con 0x561d5dba15d0 2024-09-15T22:10:37.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.721+0000 7f4b5168f740 10 client.0 _ll_register_callbacks cb 0x561d5d8158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:37.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.722+0000 7f4b4d853640 1 -- 172.21.15.5:0/2247901723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 msgr2=0x561d5dba9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:37.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.722+0000 7f4b4d853640 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dba9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.722+0000 7f4b4d853640 1 -- 172.21.15.5:0/2247901723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 msgr2=0x561d5dba9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:37.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.722+0000 7f4b4d853640 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.722+0000 7f4b4d853640 1 -- 172.21.15.5:0/2247901723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5d8783c0 con 0x561d5dba15d0 2024-09-15T22:10:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.723+0000 7f4b4d853640 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dbabdc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009770 tx=0x7f4b44013cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a50a3b3771453b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.723+0000 7f4b30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:10:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.723+0000 7f4b30ff9640 1 -- 172.21.15.5:0/2247901723 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44011c70 con 0x561d5dba15d0 2024-09-15T22:10:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.723+0000 7f4b5168f740 1 -- 172.21.15.5:0/2247901723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b3c005020 con 0x561d5dba15d0 2024-09-15T22:10:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.723+0000 7f4b30ff9640 1 -- 172.21.15.5:0/2247901723 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4b44004030 con 0x561d5dba15d0 2024-09-15T22:10:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.724+0000 7f4b30ff9640 1 -- 172.21.15.5:0/2247901723 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4b44024050 con 0x561d5dba15d0 2024-09-15T22:10:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.724+0000 7f4b5168f740 10 client.25345 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.724+0000 7f4b5168f740 1 -- 172.21.15.5:0/2247901723 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d5dbc33f0 con 0x561d5dba15d0 2024-09-15T22:10:37.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.724+0000 7f4b30ff9640 1 -- 172.21.15.5:0/2247901723 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b3c005020 con 0x561d5dba15d0 2024-09-15T22:10:37.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.724+0000 7f4b30ff9640 1 -- 172.21.15.5:0/2247901723 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4b4400f040 con 0x561d5dba15d0 2024-09-15T22:10:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.724+0000 7f4b5168f740 10 client.25345 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.724+0000 7f4b5168f740 10 client.25345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:10:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.724+0000 7f4b5168f740 20 client.25345 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.725+0000 7f4b5168f740 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561d5dbc4cf0 0x561d5dbc71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.725+0000 7f4b5168f740 4 client.25345 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:10:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.725+0000 7f4b5168f740 1 -- 172.21.15.5:0/2247901723 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d5db14860 con 0x561d5dbc4cf0 2024-09-15T22:10:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.725+0000 7f4b4c851640 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561d5dbc4cf0 0x561d5dbc71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.726+0000 7f4b4c851640 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561d5dbc4cf0 0x561d5dbc71b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfc71c1912f2a1e7 server_cookie=34b99f393a232c81 in_seq=0 out_seq=0 2024-09-15T22:10:37.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.726+0000 7f4b30ff9640 10 client.25345 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b30ff9640 1 -- 172.21.15.5:0/2247901723 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d5db14860 con 0x561d5dbc4cf0 2024-09-15T22:10:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b30ff9640 10 client.25345 handle_command_reply: tid=0 2024-09-15T22:10:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b5168f740 1 client.25345 shutdown 2024-09-15T22:10:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b5168f740 1 -- 172.21.15.5:0/2247901723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 msgr2=0x561d5dbabdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b5168f740 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dbabdc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009770 tx=0x7f4b44013cb0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b5168f740 1 -- 172.21.15.5:0/2247901723 shutdown_connections 2024-09-15T22:10:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b5168f740 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561d5dbc4cf0 0x561d5dbc71b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b5168f740 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dba9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b5168f740 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b5168f740 1 --2- 172.21.15.5:0/2247901723 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dbabdc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b30ff9640 0 client.25345 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:37.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b5168f740 1 -- 172.21.15.5:0/2247901723 >> 172.21.15.5:0/2247901723 conn(0x561d5db6a310 msgr2=0x561d5dba3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:37.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b5168f740 1 -- 172.21.15.5:0/2247901723 shutdown_connections 2024-09-15T22:10:37.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.727+0000 7f4b5168f740 1 -- 172.21.15.5:0/2247901723 wait complete. 2024-09-15T22:10:37.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.728+0000 7f4b5168f740 20 client.25345 trim_cache size 0 max 16384 2024-09-15T22:10:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.752+0000 7f4b5168f740 1 Processor -- start 2024-09-15T22:10:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.752+0000 7f4b5168f740 1 -- start start 2024-09-15T22:10:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.752+0000 7f4b5168f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dba2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.752+0000 7f4b5168f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.752+0000 7f4b5168f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dba4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.752+0000 7f4b5168f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b8050 con 0x561d5dba2b40 2024-09-15T22:10:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.752+0000 7f4b5168f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b5920 con 0x561d5dbac370 2024-09-15T22:10:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.752+0000 7f4b5168f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b3770 con 0x561d5dba15d0 2024-09-15T22:10:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.753+0000 7f4b4d853640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dba2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.753+0000 7f4b4d853640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dba2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43222/0 (socket says 172.21.15.5:43222) 2024-09-15T22:10:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.753+0000 7f4b4d853640 1 -- 172.21.15.5:0/3596234863 learned_addr learned my addr 172.21.15.5:0/3596234863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:37.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.753+0000 7f4b4c851640 1 --2- 172.21.15.5:0/3596234863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.753+0000 7f4b4d853640 1 --2- 172.21.15.5:0/3596234863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dba4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.753+0000 7f4b4d853640 1 -- 172.21.15.5:0/3596234863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 msgr2=0x561d5dba4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.753+0000 7f4b4d853640 1 --2- 172.21.15.5:0/3596234863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dba4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.753+0000 7f4b4d853640 1 -- 172.21.15.5:0/3596234863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 msgr2=0x561d5dba4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.753+0000 7f4b4d853640 1 --2- 172.21.15.5:0/3596234863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.753+0000 7f4b4d853640 1 -- 172.21.15.5:0/3596234863 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d5dbac130 con 0x561d5dba15d0 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.753+0000 7f4b4c851640 1 --2- 172.21.15.5:0/3596234863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.754+0000 7f4b4d853640 1 --2- 172.21.15.5:0/3596234863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dba2300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4b44006010 tx=0x7f4b440117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aff704fdc4f82d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.754+0000 7f4b32ffd640 1 -- 172.21.15.5:0/3596234863 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44031070 con 0x561d5dba15d0 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.754+0000 7f4b32ffd640 1 -- 172.21.15.5:0/3596234863 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b44011a70 con 0x561d5dba15d0 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.754+0000 7f4b32ffd640 1 -- 172.21.15.5:0/3596234863 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44021040 con 0x561d5dba15d0 2024-09-15T22:10:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.755+0000 7f4b5168f740 1 -- 172.21.15.5:0/3596234863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 msgr2=0x561d5dba2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.755+0000 7f4b5168f740 1 --2- 172.21.15.5:0/3596234863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dba2300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4b44006010 tx=0x7f4b440117b0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.755+0000 7f4b5168f740 1 -- 172.21.15.5:0/3596234863 shutdown_connections 2024-09-15T22:10:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.755+0000 7f4b5168f740 1 --2- 172.21.15.5:0/3596234863 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dbac370 0x561d5dba4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.755+0000 7f4b5168f740 1 --2- 172.21.15.5:0/3596234863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dba4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.755+0000 7f4b5168f740 1 --2- 172.21.15.5:0/3596234863 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dba15d0 0x561d5dba2300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.755+0000 7f4b5168f740 1 -- 172.21.15.5:0/3596234863 >> 172.21.15.5:0/3596234863 conn(0x561d5db6a310 msgr2=0x561d5db98a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.755+0000 7f4b5168f740 1 -- 172.21.15.5:0/3596234863 shutdown_connections 2024-09-15T22:10:37.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.755+0000 7f4b5168f740 1 -- 172.21.15.5:0/3596234863 wait complete. 2024-09-15T22:10:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.756+0000 7f4b5168f740 1 -- start start 2024-09-15T22:10:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.756+0000 7f4b5168f740 1 Processor -- start 2024-09-15T22:10:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b5168f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dba15d0 0x561d5dbbd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b5168f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dbbd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b5168f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dbac370 0x561d5dba5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b5168f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b3de0 con 0x561d5dba2b40 2024-09-15T22:10:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b5168f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b5250 con 0x561d5dba15d0 2024-09-15T22:10:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b5168f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481b44b0 con 0x561d5dbac370 2024-09-15T22:10:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b5168f740 10 client.0 _ll_register_callbacks cb 0x7f4b44005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b4c851640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dbbd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b4c851640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dbbd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37804/0 (socket says 172.21.15.5:37804) 2024-09-15T22:10:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b4c851640 1 -- 172.21.15.5:0/647001613 learned_addr learned my addr 172.21.15.5:0/647001613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b4d853640 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dbac370 0x561d5dba5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.757+0000 7f4b4d853640 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dba15d0 0x561d5dbbd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.758+0000 7f4b4c851640 1 -- 172.21.15.5:0/647001613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dbac370 msgr2=0x561d5dba5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.758+0000 7f4b4c851640 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dbac370 0x561d5dba5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.758+0000 7f4b4c851640 1 -- 172.21.15.5:0/647001613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dba15d0 msgr2=0x561d5dbbd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.758+0000 7f4b4c851640 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dba15d0 0x561d5dbbd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.758+0000 7f4b4c851640 1 -- 172.21.15.5:0/647001613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d5dba64a0 con 0x561d5dba2b40 2024-09-15T22:10:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.759+0000 7f4b4c851640 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dbbd5e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0057c0 tx=0x7f4b3c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1367ac65a9677f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:37.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.760+0000 7f4b30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:10:37.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.760+0000 7f4b30ff9640 1 -- 172.21.15.5:0/647001613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c004230 con 0x561d5dba2b40 2024-09-15T22:10:37.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.760+0000 7f4b30ff9640 1 -- 172.21.15.5:0/647001613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4b3c0043d0 con 0x561d5dba2b40 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.760+0000 7f4b5168f740 1 -- 172.21.15.5:0/647001613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b44011a70 con 0x561d5dba2b40 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.761+0000 7f4b30ff9640 1 -- 172.21.15.5:0/647001613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4b3c034050 con 0x561d5dba2b40 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.761+0000 7f4b5168f740 10 client.15753 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.761+0000 7f4b5168f740 1 -- 172.21.15.5:0/647001613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d5dc2f750 con 0x561d5dba2b40 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.761+0000 7f4b30ff9640 1 -- 172.21.15.5:0/647001613 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b44011a70 con 0x561d5dba2b40 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.761+0000 7f4b30ff9640 1 -- 172.21.15.5:0/647001613 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4b3c023040 con 0x561d5dba2b40 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.761+0000 7f4b5168f740 10 client.15753 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.761+0000 7f4b5168f740 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.761+0000 7f4b5168f740 20 client.15753 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.761+0000 7f4b5168f740 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561d5dc2fbf0 0x561d5dc31fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.762+0000 7f4b5168f740 4 client.15753 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.762+0000 7f4b5168f740 1 -- 172.21.15.5:0/647001613 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d5db0ed70 con 0x561d5dc2fbf0 2024-09-15T22:10:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.762+0000 7f4b4d853640 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561d5dc2fbf0 0x561d5dc31fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.762+0000 7f4b30ff9640 10 client.15753 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.762+0000 7f4b4d853640 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561d5dc2fbf0 0x561d5dc31fe0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39245b3a1f320c53 server_cookie=175f4471819036ef in_seq=0 out_seq=0 2024-09-15T22:10:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b30ff9640 1 -- 172.21.15.5:0/647001613 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d5db0ed70 con 0x561d5dc2fbf0 2024-09-15T22:10:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b30ff9640 10 client.15753 handle_command_reply: tid=0 2024-09-15T22:10:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b5168f740 1 client.15753 shutdown 2024-09-15T22:10:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b5168f740 1 -- 172.21.15.5:0/647001613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 msgr2=0x561d5dbbd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b5168f740 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dbbd5e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c0057c0 tx=0x7f4b3c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b5168f740 1 -- 172.21.15.5:0/647001613 shutdown_connections 2024-09-15T22:10:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b5168f740 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561d5dbac370 0x561d5dba5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b5168f740 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d5dba2b40 0x561d5dbbd5e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b5168f740 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561d5dc2fbf0 0x561d5dc31fe0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b5168f740 1 --2- 172.21.15.5:0/647001613 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561d5dba15d0 0x561d5dbbd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b5168f740 1 -- 172.21.15.5:0/647001613 >> 172.21.15.5:0/647001613 conn(0x561d5db6a310 msgr2=0x561d5dbc03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b30ff9640 0 client.15753 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b5168f740 1 -- 172.21.15.5:0/647001613 shutdown_connections 2024-09-15T22:10:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.763+0000 7f4b5168f740 1 -- 172.21.15.5:0/647001613 wait complete. 2024-09-15T22:10:37.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.764+0000 7f4b5168f740 20 client.15753 trim_cache size 0 max 16384 2024-09-15T22:10:37.769 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:10:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.764+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/3133028293 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4b340a35c0 msgr2=0x7f4b340a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.764+0000 7f4b4f2b6640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4b340a35c0 0x7f4b340a5a80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4b400c8df0 tx=0x7f4b4002ac50 comp rx=0 tx=0).stop 2024-09-15T22:10:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.764+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/3133028293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b481091f0 msgr2=0x7f4b481a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.764+0000 7f4b4f2b6640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b481091f0 0x7f4b481a0d20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009d00 tx=0x7f4b400305d0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.764+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/3133028293 shutdown_connections 2024-09-15T22:10:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.764+0000 7f4b4f2b6640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4b340a35c0 0x7f4b340a5a80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.764+0000 7f4b4f2b6640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4b48114ab0 0x7f4b481a55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.764+0000 7f4b4f2b6640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b48109b10 0x7f4b481a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.764+0000 7f4b4f2b6640 1 --2- 172.21.15.5:0/3133028293 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4b481091f0 0x7f4b481a0d20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.764+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/3133028293 >> 172.21.15.5:0/3133028293 conn(0x7f4b4806e140 msgr2=0x7f4b480709a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.765+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/3133028293 shutdown_connections 2024-09-15T22:10:37.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:37.765+0000 7f4b4f2b6640 1 -- 172.21.15.5:0/3133028293 wait complete. 2024-09-15T22:10:39.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:39 smithi099 ceph-mon[34892]: pgmap v55: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-09-15T22:10:39.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:39 smithi129 ceph-mon[35314]: pgmap v55: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-09-15T22:10:39.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:39 smithi005 ceph-mon[30944]: pgmap v55: 129 pgs: 129 active+clean; 1.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 MiB/s wr, 924 op/s 2024-09-15T22:10:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 70.0 s, 2828.6 tps, lat 35.191 ms stddev 39.466 2024-09-15T22:10:41.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:41 smithi129 ceph-mon[35314]: pgmap v56: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-09-15T22:10:41.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:41 smithi005 ceph-mon[30944]: pgmap v56: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-09-15T22:10:41.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:41 smithi099 ceph-mon[34892]: pgmap v56: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 17 MiB/s wr, 1.22k op/s 2024-09-15T22:10:41.859 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:10:41.860 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:10:42.435 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:43.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:43 smithi005 ceph-mon[30944]: pgmap v57: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 586 op/s 2024-09-15T22:10:43.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:43 smithi099 ceph-mon[34892]: pgmap v57: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 586 op/s 2024-09-15T22:10:43.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:43 smithi129 ceph-mon[35314]: pgmap v57: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 586 op/s 2024-09-15T22:10:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 75.0 s, 1271.6 tps, lat 77.992 ms stddev 216.444 2024-09-15T22:10:45.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:45 smithi005 ceph-mon[30944]: pgmap v58: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 586 op/s 2024-09-15T22:10:45.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:45 smithi099 ceph-mon[34892]: pgmap v58: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 586 op/s 2024-09-15T22:10:45.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:45 smithi129 ceph-mon[35314]: pgmap v58: 129 pgs: 129 active+clean; 1.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 7.1 MiB/s wr, 586 op/s 2024-09-15T22:10:47.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:47 smithi005 ceph-mon[30944]: pgmap v59: 129 pgs: 129 active+clean; 1.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 1.90k op/s 2024-09-15T22:10:47.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:47 smithi099 ceph-mon[34892]: pgmap v59: 129 pgs: 129 active+clean; 1.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 1.90k op/s 2024-09-15T22:10:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:47 smithi129 ceph-mon[35314]: pgmap v59: 129 pgs: 129 active+clean; 1.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 33 MiB/s wr, 1.90k op/s 2024-09-15T22:10:48.019 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:10:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:10:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "75aab30d-859c-462d-81ca-49ecd7e3b858", 2024-09-15T22:10:48.019 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:10:48.019 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:10:48.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.042+0000 7f25d5a42640 1 -- 172.21.15.5:0/1385971078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25d0108290 msgr2=0x7f25d010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:48.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.042+0000 7f25d5a42640 1 --2- 172.21.15.5:0/1385971078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25d0108290 0x7f25d010ec30 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009cd0 tx=0x7f25c802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.042+0000 7f25d5a42640 1 -- 172.21.15.5:0/1385971078 shutdown_connections 2024-09-15T22:10:48.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.042+0000 7f25d5a42640 1 --2- 172.21.15.5:0/1385971078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25d0108290 0x7f25d010ec30 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.042+0000 7f25d5a42640 1 --2- 172.21.15.5:0/1385971078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d0103740 0x7f25d0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.043+0000 7f25d5a42640 1 --2- 172.21.15.5:0/1385971078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25d0102d90 0x7f25d0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.043+0000 7f25d5a42640 1 -- 172.21.15.5:0/1385971078 >> 172.21.15.5:0/1385971078 conn(0x7f25d00fc7d0 msgr2=0x7f25d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:48.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.043+0000 7f25d5a42640 1 -- 172.21.15.5:0/1385971078 shutdown_connections 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.043+0000 7f25d5a42640 1 -- 172.21.15.5:0/1385971078 wait complete. 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.043+0000 7f25d5a42640 1 Processor -- start 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25d5a42640 1 -- start start 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25d5a42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25d0102d90 0x7f25d01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25d5a42640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25d0103740 0x7f25d0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25d5a42640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d0108290 0x7f25d019ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25d5a42640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01120d0 con 0x7f25d0102d90 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25d5a42640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f25d0111f50 con 0x7f25d0103740 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25d5a42640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25d0112250 con 0x7f25d0108290 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25d0102d90 0x7f25d01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25d0102d90 0x7f25d01985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54994/0 (socket says 172.21.15.5:54994) 2024-09-15T22:10:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25ceffd640 1 -- 172.21.15.5:0/2934993866 learned_addr learned my addr 172.21.15.5:0/2934993866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25ce7fc640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25d0103740 0x7f25d0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25cf7fe640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d0108290 0x7f25d019ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25ceffd640 1 -- 172.21.15.5:0/2934993866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d0108290 msgr2=0x7f25d019ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25ceffd640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d0108290 0x7f25d019ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25ceffd640 1 -- 172.21.15.5:0/2934993866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25d0103740 msgr2=0x7f25d0198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25ceffd640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25d0103740 0x7f25d0198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.044+0000 7f25ceffd640 1 -- 172.21.15.5:0/2934993866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d019d550 con 0x7f25d0102d90 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.045+0000 7f25ceffd640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25d0102d90 0x7f25d01985a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f25c000e9f0 tx=0x7f25c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.045+0000 7f25d4a40640 1 -- 172.21.15.5:0/2934993866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c000ccc0 con 0x7f25d0102d90 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.045+0000 7f25d4a40640 1 -- 172.21.15.5:0/2934993866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c0010910 con 0x7f25d0102d90 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.045+0000 7f25d5a42640 1 -- 172.21.15.5:0/2934993866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d019d840 con 0x7f25d0102d90 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.045+0000 7f25d4a40640 1 -- 172.21.15.5:0/2934993866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00188a0 con 0x7f25d0102d90 2024-09-15T22:10:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.045+0000 7f25d5a42640 1 -- 172.21.15.5:0/2934993866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d01a5120 con 0x7f25d0102d90 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.047+0000 7f25d4a40640 1 -- 172.21.15.5:0/2934993866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f25c0010ac0 con 0x7f25d0102d90 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.047+0000 7f25d7e1b740 1 Processor -- start 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25d7e1b740 1 -- start start 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976eb60 0x55a8d976ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9779800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d9779d40 0x55a8d977c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a8d96d4280 con 0x55a8d976eb60 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a8d967bb10 con 0x55a8d976f480 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a8d96d0240 con 0x55a8d9779d40 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25d4a40640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25b80a33b0 0x7f25b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25cf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976eb60 0x55a8d976ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25cf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976eb60 0x55a8d976ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55004/0 (socket says 172.21.15.5:55004) 2024-09-15T22:10:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25cf7fe640 1 -- 172.21.15.5:0/1409579904 learned_addr learned my addr 172.21.15.5:0/1409579904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25cf7fe640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25b80a33b0 0x7f25b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25d4a40640 1 -- 172.21.15.5:0/2934993866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f25c00d0560 con 0x7f25d0102d90 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25ceffd640 1 --2- 172.21.15.5:0/1409579904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9779800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25ce7fc640 1 --2- 172.21.15.5:0/1409579904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d9779d40 0x55a8d977c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25ceffd640 1 -- 172.21.15.5:0/1409579904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d9779d40 msgr2=0x55a8d977c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25ceffd640 1 --2- 172.21.15.5:0/1409579904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d9779d40 0x55a8d977c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25ceffd640 1 -- 172.21.15.5:0/1409579904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976eb60 msgr2=0x55a8d976ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25ceffd640 1 --2- 172.21.15.5:0/1409579904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976eb60 0x55a8d976ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.048+0000 7f25ceffd640 1 -- 172.21.15.5:0/1409579904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d96881f0 con 0x55a8d976f480 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.049+0000 7f25ceffd640 1 --2- 172.21.15.5:0/1409579904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9779800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25c00cd910 tx=0x7f25c00097a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26b7af8a2fed06e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.049+0000 7f25cf7fe640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25b80a33b0 0x7f25b80a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25c8031860 tx=0x7f25c80095c0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.049+0000 7f25b5ffb640 1 -- 172.21.15.5:0/1409579904 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00a5110 con 0x55a8d976f480 2024-09-15T22:10:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.049+0000 7f25b5ffb640 1 -- 172.21.15.5:0/1409579904 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c00cdb90 con 0x55a8d976f480 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.049+0000 7f25b5ffb640 1 -- 172.21.15.5:0/1409579904 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c009b8f0 con 0x55a8d976f480 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.050+0000 7f25d7e1b740 1 -- 172.21.15.5:0/1409579904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 msgr2=0x55a8d9779800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.050+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/1409579904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9779800 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25c00cd910 tx=0x7f25c00097a0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.050+0000 7f25d7e1b740 1 -- 172.21.15.5:0/1409579904 shutdown_connections 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.050+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/1409579904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d9779d40 0x55a8d977c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.050+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/1409579904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9779800 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.050+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/1409579904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976eb60 0x55a8d976ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.050+0000 7f25d7e1b740 1 -- 172.21.15.5:0/1409579904 >> 172.21.15.5:0/1409579904 conn(0x55a8d976b0f0 msgr2=0x55a8d976b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.050+0000 7f25d7e1b740 1 -- 172.21.15.5:0/1409579904 shutdown_connections 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.050+0000 7f25d7e1b740 1 -- 172.21.15.5:0/1409579904 wait complete. 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.050+0000 7f25d7e1b740 1 -- start start 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.050+0000 7f25d7e1b740 1 Processor -- start 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d9776be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976f480 0x55a8d9777120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d9779d40 0x55a8d9777660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a8d977c910 con 0x55a8d976f480 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a8d968ef10 con 0x55a8d9779d40 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a8d977ca90 con 0x55a8d976eb60 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25d7e1b740 10 client.0 _ll_register_callbacks cb 0x55a8d9657a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25ce7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976f480 0x55a8d9777120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25ceffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d9776be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25ce7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976f480 0x55a8d9777120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55008/0 (socket says 172.21.15.5:55008) 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25ce7fc640 1 -- 172.21.15.5:0/2121567972 learned_addr learned my addr 172.21.15.5:0/2121567972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25cf7fe640 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d9779d40 0x55a8d9777660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25ceffd640 1 -- 172.21.15.5:0/2121567972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d9779d40 msgr2=0x55a8d9777660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25ceffd640 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d9779d40 0x55a8d9777660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25ceffd640 1 -- 172.21.15.5:0/2121567972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976f480 msgr2=0x55a8d9777120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25ceffd640 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976f480 0x55a8d9777120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25ceffd640 1 -- 172.21.15.5:0/2121567972 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d94456a0 con 0x55a8d976eb60 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25ce7fc640 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976f480 0x55a8d9777120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25ceffd640 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d9776be0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55a8d976bf50 tx=0x7f25c0009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44b9c3c5ece1f4ae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.051+0000 7f25aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25aeffd640 1 -- 172.21.15.5:0/2121567972 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00b9020 con 0x55a8d976eb60 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25aeffd640 1 -- 172.21.15.5:0/2121567972 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f25c00b5260 con 0x55a8d976eb60 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2121567972 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c00cdb90 con 0x55a8d976eb60 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25aeffd640 1 -- 172.21.15.5:0/2121567972 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25c007e050 con 0x55a8d976eb60 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25d7e1b740 10 client.25360 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2121567972 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a8d977c910 con 0x55a8d976eb60 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25aeffd640 1 -- 172.21.15.5:0/2121567972 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c00cdb90 con 0x55a8d976eb60 2024-09-15T22:10:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25aeffd640 1 -- 172.21.15.5:0/2121567972 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f25c0067120 con 0x55a8d976eb60 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25d7e1b740 10 client.25360 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25d7e1b740 10 client.25360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25d7e1b740 20 client.25360 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a8d9789b20 0x55a8d978bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25d7e1b740 4 client.25360 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.052+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2121567972 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a8d96e17f0 con 0x55a8d9789b20 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.053+0000 7f25ce7fc640 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a8d9789b20 0x55a8d978bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.053+0000 7f25aeffd640 10 client.25360 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.053+0000 7f25ce7fc640 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a8d9789b20 0x55a8d978bfa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d658f5b17b7b32b server_cookie=d0e499b45cbf3270 in_seq=0 out_seq=0 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25aeffd640 1 -- 172.21.15.5:0/2121567972 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a8d96e17f0 con 0x55a8d9789b20 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25aeffd640 10 client.25360 handle_command_reply: tid=0 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 1 client.25360 shutdown 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2121567972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 msgr2=0x55a8d9776be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d9776be0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55a8d976bf50 tx=0x7f25c0009700 comp rx=0 tx=0).stop 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2121567972 shutdown_connections 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a8d9789b20 0x55a8d978bfa0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d9779d40 0x55a8d9777660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d976f480 0x55a8d9777120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2121567972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d9776be0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25aeffd640 0 client.25360 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2121567972 >> 172.21.15.5:0/2121567972 conn(0x55a8d976b0f0 msgr2=0x55a8d9770e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2121567972 shutdown_connections 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2121567972 wait complete. 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.054+0000 7f25d7e1b740 20 client.25360 trim_cache size 0 max 16384 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25d7e1b740 1 Processor -- start 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25d7e1b740 1 -- start start 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d97715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9778a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d9779d40 0x55a8d97918b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01af760 con 0x55a8d9779d40 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01ad030 con 0x55a8d976f480 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01aae80 con 0x55a8d976eb60 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25cf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d9779d40 0x55a8d97918b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25ce7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9778a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25ceffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d97715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25ce7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9778a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:55972/0 (socket says 172.21.15.5:55972) 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25ceffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d97715c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:42234/0 (socket says 172.21.15.5:42234) 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25cf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d9779d40 0x55a8d97918b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55018/0 (socket says 172.21.15.5:55018) 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.082+0000 7f25ce7fc640 1 -- 172.21.15.5:0/771040932 learned_addr learned my addr 172.21.15.5:0/771040932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.083+0000 7f25ceffd640 1 -- 172.21.15.5:0/771040932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 msgr2=0x55a8d9778a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.083+0000 7f25ceffd640 1 --2- 172.21.15.5:0/771040932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9778a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.083+0000 7f25ceffd640 1 -- 172.21.15.5:0/771040932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d9779d40 msgr2=0x55a8d97918b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.083+0000 7f25ceffd640 1 --2- 172.21.15.5:0/771040932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d9779d40 0x55a8d97918b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.083+0000 7f25ceffd640 1 -- 172.21.15.5:0/771040932 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8d978c620 con 0x55a8d976eb60 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.083+0000 7f25ceffd640 1 --2- 172.21.15.5:0/771040932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d97715c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25c000e9c0 tx=0x7f25c00097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=599227d9aea5757e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.083+0000 7f25b57fa640 1 -- 172.21.15.5:0/771040932 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00b9070 con 0x55a8d976eb60 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.083+0000 7f25b57fa640 1 -- 172.21.15.5:0/771040932 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c00a5110 con 0x55a8d976eb60 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.083+0000 7f25b57fa640 1 -- 172.21.15.5:0/771040932 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00cdd20 con 0x55a8d976eb60 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.084+0000 7f25d7e1b740 1 -- 172.21.15.5:0/771040932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 msgr2=0x55a8d97715c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.084+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/771040932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d97715c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25c000e9c0 tx=0x7f25c00097a0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.086+0000 7f25d7e1b740 1 -- 172.21.15.5:0/771040932 shutdown_connections 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.086+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/771040932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d9779d40 0x55a8d97918b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.086+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/771040932 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9778a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.086+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/771040932 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d97715c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.086+0000 7f25d7e1b740 1 -- 172.21.15.5:0/771040932 >> 172.21.15.5:0/771040932 conn(0x55a8d976b0f0 msgr2=0x55a8d9778330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.086+0000 7f25d7e1b740 1 -- 172.21.15.5:0/771040932 shutdown_connections 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.086+0000 7f25d7e1b740 1 -- 172.21.15.5:0/771040932 wait complete. 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.086+0000 7f25d7e1b740 1 -- start start 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.086+0000 7f25d7e1b740 1 Processor -- start 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.087+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d9782370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.087+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9773880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.087+0000 7f25d7e1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d9779d40 0x55a8d9773dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.087+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01ab4f0 con 0x55a8d9779d40 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.087+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01ac960 con 0x55a8d976f480 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.087+0000 7f25d7e1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f25d01abbc0 con 0x55a8d976eb60 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.087+0000 7f25d7e1b740 10 client.0 _ll_register_callbacks cb 0x7f25c0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.087+0000 7f25ceffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d9782370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.087+0000 7f25ce7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9773880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25ce7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9773880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:55976/0 (socket says 172.21.15.5:55976) 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25ceffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d9782370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:42250/0 (socket says 172.21.15.5:42250) 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25ce7fc640 1 -- 172.21.15.5:0/2729331884 learned_addr learned my addr 172.21.15.5:0/2729331884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25cf7fe640 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d9779d40 0x55a8d9773dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25ceffd640 1 -- 172.21.15.5:0/2729331884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 msgr2=0x55a8d9773880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25ceffd640 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9773880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25ceffd640 1 -- 172.21.15.5:0/2729331884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d9779d40 msgr2=0x55a8d9773dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25ceffd640 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d9779d40 0x55a8d9773dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25ceffd640 1 -- 172.21.15.5:0/2729331884 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8d976af20 con 0x55a8d976eb60 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25ce7fc640 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9773880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25ceffd640 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d9782370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f25c009c430 tx=0x7f25c00985b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f04c186c6695dad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:10:48.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25aeffd640 1 -- 172.21.15.5:0/2729331884 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25c00b9070 con 0x55a8d976eb60 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25aeffd640 1 -- 172.21.15.5:0/2729331884 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f25c009b400 con 0x55a8d976eb60 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.088+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2729331884 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25c00a5110 con 0x55a8d976eb60 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.089+0000 7f25aeffd640 1 -- 172.21.15.5:0/2729331884 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25c0098740 con 0x55a8d976eb60 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.089+0000 7f25d7e1b740 10 client.25366 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.089+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2729331884 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a8d976af20 con 0x55a8d976eb60 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.089+0000 7f25aeffd640 1 -- 172.21.15.5:0/2729331884 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25c00a5110 con 0x55a8d976eb60 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.089+0000 7f25aeffd640 1 -- 172.21.15.5:0/2729331884 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f25c00851c0 con 0x55a8d976eb60 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.089+0000 7f25d7e1b740 10 client.25366 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.089+0000 7f25d7e1b740 10 client.25366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.089+0000 7f25d7e1b740 20 client.25366 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.089+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a8d97f4950 0x55a8d97f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.089+0000 7f25d7e1b740 4 client.25366 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.089+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2729331884 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a8d96dbd00 con 0x55a8d97f4950 2024-09-15T22:10:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.090+0000 7f25ce7fc640 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a8d97f4950 0x55a8d97f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.090+0000 7f25aeffd640 10 client.25366 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.091+0000 7f25ce7fc640 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a8d97f4950 0x55a8d97f6d40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a210e4f1fe0cb762 server_cookie=fa80a46b3d565caa in_seq=0 out_seq=0 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.092+0000 7f25aeffd640 1 -- 172.21.15.5:0/2729331884 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a8d96dbd00 con 0x55a8d97f4950 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.092+0000 7f25aeffd640 10 client.25366 handle_command_reply: tid=0 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.092+0000 7f25d7e1b740 1 client.25366 shutdown 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.092+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2729331884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 msgr2=0x55a8d9782370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.092+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d9782370 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f25c009c430 tx=0x7f25c00985b0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.094+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2729331884 shutdown_connections 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.094+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a8d9779d40 0x55a8d9773dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.094+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a8d97f4950 0x55a8d97f6d40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.094+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a8d976f480 0x55a8d9773880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.094+0000 7f25d7e1b740 1 --2- 172.21.15.5:0/2729331884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a8d976eb60 0x55a8d9782370 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.094+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2729331884 >> 172.21.15.5:0/2729331884 conn(0x55a8d976b0f0 msgr2=0x55a8d976a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.094+0000 7f25aeffd640 0 client.25366 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.094+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2729331884 shutdown_connections 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.094+0000 7f25d7e1b740 1 -- 172.21.15.5:0/2729331884 wait complete. 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.095+0000 7f25d7e1b740 20 client.25366 trim_cache size 0 max 16384 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.095+0000 7f25d5a42640 1 -- 172.21.15.5:0/2934993866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25b80a33b0 msgr2=0x7f25b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.095+0000 7f25d5a42640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25b80a33b0 0x7f25b80a5870 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25c8031860 tx=0x7f25c80095c0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.095+0000 7f25d5a42640 1 -- 172.21.15.5:0/2934993866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25d0102d90 msgr2=0x7f25d01985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.095+0000 7f25d5a42640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25d0102d90 0x7f25d01985a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f25c000e9f0 tx=0x7f25c000edf0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.095+0000 7f25d5a42640 1 -- 172.21.15.5:0/2934993866 shutdown_connections 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.095+0000 7f25d5a42640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25b80a33b0 0x7f25b80a5870 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.095+0000 7f25d5a42640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f25d0108290 0x7f25d019ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.095+0000 7f25d5a42640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f25d0103740 0x7f25d0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.095+0000 7f25d5a42640 1 --2- 172.21.15.5:0/2934993866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25d0102d90 0x7f25d01985a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.095+0000 7f25d5a42640 1 -- 172.21.15.5:0/2934993866 >> 172.21.15.5:0/2934993866 conn(0x7f25d00fc7d0 msgr2=0x7f25d0106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.096+0000 7f25d5a42640 1 -- 172.21.15.5:0/2934993866 shutdown_connections 2024-09-15T22:10:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:47.097+0000 7f25d5a42640 1 -- 172.21.15.5:0/2934993866 wait complete. 2024-09-15T22:10:48.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:48 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:10:48.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:48 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:10:48.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:48 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:10:48.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:48 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:10:48.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:48 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:10:48.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:48 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:10:48.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:48 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:10:48.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:48 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:10:48.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:48 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:10:48.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:48 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:10:48.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:48 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:10:48.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:48 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:10:49.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:49 smithi099 ceph-mon[34892]: pgmap v60: 129 pgs: 129 active+clean; 1.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 1.60k op/s 2024-09-15T22:10:49.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:49 smithi129 ceph-mon[35314]: pgmap v60: 129 pgs: 129 active+clean; 1.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 1.60k op/s 2024-09-15T22:10:49.609 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:49 smithi005 ceph-mon[30944]: pgmap v60: 129 pgs: 129 active+clean; 1.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 29 MiB/s wr, 1.60k op/s 2024-09-15T22:10:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 80.0 s, 2834.6 tps, lat 35.647 ms stddev 45.294 2024-09-15T22:10:50.748 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:10:51.209 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:10:51.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:51 smithi005 ceph-mon[30944]: pgmap v61: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 1.85k op/s 2024-09-15T22:10:51.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:51 smithi099 ceph-mon[34892]: pgmap v61: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 1.85k op/s 2024-09-15T22:10:51.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:51 smithi129 ceph-mon[35314]: pgmap v61: 129 pgs: 129 active+clean; 1.4 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 32 MiB/s wr, 1.85k op/s 2024-09-15T22:10:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.764+0000 7f86372e3640 1 -- 172.21.15.5:0/3784922590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8630077040 msgr2=0x7f86301124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.764+0000 7f86372e3640 1 --2- 172.21.15.5:0/3784922590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8630077040 0x7f86301124f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f862c009900 tx=0x7f862c02a410 comp rx=0 tx=0).stop 2024-09-15T22:10:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.764+0000 7f86372e3640 1 -- 172.21.15.5:0/3784922590 shutdown_connections 2024-09-15T22:10:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.764+0000 7f86372e3640 1 --2- 172.21.15.5:0/3784922590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8630112a30 0x7f8630114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.764+0000 7f86372e3640 1 --2- 172.21.15.5:0/3784922590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8630077040 0x7f86301124f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.764+0000 7f86372e3640 1 --2- 172.21.15.5:0/3784922590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8630076720 0x7f8630076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.764+0000 7f86372e3640 1 -- 172.21.15.5:0/3784922590 >> 172.21.15.5:0/3784922590 conn(0x7f86300ffa90 msgr2=0x7f8630101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.764+0000 7f86372e3640 1 -- 172.21.15.5:0/3784922590 shutdown_connections 2024-09-15T22:10:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.764+0000 7f86372e3640 1 -- 172.21.15.5:0/3784922590 wait complete. 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.765+0000 7f86372e3640 1 Processor -- start 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.765+0000 7f86372e3640 1 -- start start 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.765+0000 7f86372e3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8630076720 0x7f8630073d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.765+0000 7f86372e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8630077040 0x7f863006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.765+0000 7f86372e3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8630112a30 0x7f863006c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.766+0000 7f86372e3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8630118640 con 0x7f8630077040 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.766+0000 7f86372e3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86301184c0 con 0x7f8630112a30 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.766+0000 7f86372e3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86301187c0 con 0x7f8630076720 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.766+0000 7f8635880640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8630112a30 0x7f863006c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.766+0000 7f863507f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8630076720 0x7f8630073d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.766+0000 7f8635880640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8630112a30 0x7f863006c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:55992/0 (socket says 172.21.15.5:55992) 2024-09-15T22:10:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.766+0000 7f8635880640 1 -- 172.21.15.5:0/2839896858 learned_addr learned my addr 172.21.15.5:0/2839896858 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.766+0000 7f8635880640 1 -- 172.21.15.5:0/2839896858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8630076720 msgr2=0x7f8630073d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.766+0000 7f8635880640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8630076720 0x7f8630073d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.766+0000 7f8635880640 1 -- 172.21.15.5:0/2839896858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8630077040 msgr2=0x7f863006bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.766+0000 7f8635880640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8630077040 0x7f863006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.767+0000 7f8635880640 1 -- 172.21.15.5:0/2839896858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863006ca40 con 0x7f8630112a30 2024-09-15T22:10:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.767+0000 7f8635880640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8630112a30 0x7f863006c220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f862002d410 tx=0x7f862002bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.767+0000 7f86267fc640 1 -- 172.21.15.5:0/2839896858 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620037020 con 0x7f8630112a30 2024-09-15T22:10:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.767+0000 7f86372e3640 1 -- 172.21.15.5:0/2839896858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863019e8c0 con 0x7f8630112a30 2024-09-15T22:10:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.767+0000 7f86267fc640 1 -- 172.21.15.5:0/2839896858 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8620033d10 con 0x7f8630112a30 2024-09-15T22:10:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.767+0000 7f86267fc640 1 -- 172.21.15.5:0/2839896858 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86200312e0 con 0x7f8630112a30 2024-09-15T22:10:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.767+0000 7f86372e3640 1 -- 172.21.15.5:0/2839896858 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863019ee00 con 0x7f8630112a30 2024-09-15T22:10:51.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.771+0000 7f86396bc740 1 Processor -- start 2024-09-15T22:10:51.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.771+0000 7f86267fc640 1 -- 172.21.15.5:0/2839896858 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f862003a020 con 0x7f8630112a30 2024-09-15T22:10:51.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.771+0000 7f86396bc740 1 -- start start 2024-09-15T22:10:51.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.772+0000 7f86396bc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 0x555cd7e8f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.772+0000 7f86396bc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 0x555cd7e9a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.772+0000 7f86396bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 0x555cd7e9ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.772+0000 7f86396bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555cd7df52e0 con 0x555cd7e9a680 2024-09-15T22:10:51.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.772+0000 7f86396bc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555cd7d9cce0 con 0x555cd7e8f4a0 2024-09-15T22:10:51.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.772+0000 7f86396bc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555cd7df12a0 con 0x555cd7e8fdc0 2024-09-15T22:10:51.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863507f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 0x555cd7e9a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863487e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 0x555cd7e8f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863507f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 0x555cd7e9a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:42270/0 (socket says 172.21.15.5:42270) 2024-09-15T22:10:51.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863507f640 1 -- 172.21.15.5:0/2657203716 learned_addr learned my addr 172.21.15.5:0/2657203716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:51.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863487e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 0x555cd7e8f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:56008/0 (socket says 172.21.15.5:56008) 2024-09-15T22:10:51.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863507f640 1 -- 172.21.15.5:0/2657203716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 msgr2=0x555cd7e8f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:51.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863507f640 1 --2- 172.21.15.5:0/2657203716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 0x555cd7e8f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863487e640 1 --2- 172.21.15.5:0/2657203716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 0x555cd7e9ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863507f640 1 -- 172.21.15.5:0/2657203716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 msgr2=0x555cd7e9ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:51.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863507f640 1 --2- 172.21.15.5:0/2657203716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 0x555cd7e9ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863507f640 1 -- 172.21.15.5:0/2657203716 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd7d81440 con 0x555cd7e8fdc0 2024-09-15T22:10:51.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863487e640 1 --2- 172.21.15.5:0/2657203716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 0x555cd7e9ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:10:51.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.773+0000 7f863507f640 1 --2- 172.21.15.5:0/2657203716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 0x555cd7e9a140 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f86280114f0 tx=0x7f86280119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5687c25a7be7ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:51.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.774+0000 7f861b7fe640 1 -- 172.21.15.5:0/2657203716 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862801e070 con 0x555cd7e8fdc0 2024-09-15T22:10:51.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.774+0000 7f861b7fe640 1 -- 172.21.15.5:0/2657203716 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862800fb00 con 0x555cd7e8fdc0 2024-09-15T22:10:51.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.774+0000 7f861b7fe640 1 -- 172.21.15.5:0/2657203716 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628019bf0 con 0x555cd7e8fdc0 2024-09-15T22:10:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.774+0000 7f86267fc640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f861c0a3540 0x7f861c0a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.774+0000 7f86267fc640 1 -- 172.21.15.5:0/2839896858 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f86200f1720 con 0x7f8630112a30 2024-09-15T22:10:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.775+0000 7f86396bc740 1 -- 172.21.15.5:0/2657203716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 msgr2=0x555cd7e9a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.775+0000 7f86396bc740 1 --2- 172.21.15.5:0/2657203716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 0x555cd7e9a140 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f86280114f0 tx=0x7f86280119b0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.775+0000 7f863487e640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f861c0a3540 0x7f861c0a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.775+0000 7f86396bc740 1 -- 172.21.15.5:0/2657203716 shutdown_connections 2024-09-15T22:10:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.775+0000 7f86396bc740 1 --2- 172.21.15.5:0/2657203716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 0x555cd7e9ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.775+0000 7f86396bc740 1 --2- 172.21.15.5:0/2657203716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 0x555cd7e9a140 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.775+0000 7f86396bc740 1 --2- 172.21.15.5:0/2657203716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 0x555cd7e8f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.775+0000 7f86396bc740 1 -- 172.21.15.5:0/2657203716 >> 172.21.15.5:0/2657203716 conn(0x555cd7e8ba30 msgr2=0x555cd7e8be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:51.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.775+0000 7f86396bc740 1 -- 172.21.15.5:0/2657203716 shutdown_connections 2024-09-15T22:10:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.776+0000 7f86396bc740 1 -- 172.21.15.5:0/2657203716 wait complete. 2024-09-15T22:10:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.776+0000 7f863487e640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f861c0a3540 0x7f861c0a5a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f862c009850 tx=0x7f862c000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.776+0000 7f86396bc740 1 -- start start 2024-09-15T22:10:51.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.776+0000 7f86396bc740 1 Processor -- start 2024-09-15T22:10:51.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.777+0000 7f86396bc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 0x555cd7e99f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.777+0000 7f86396bc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 0x555cd7e971d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.777+0000 7f86396bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 0x555cd7e97710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.777+0000 7f86396bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555cd7e9d2f0 con 0x555cd7e9a680 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.777+0000 7f86396bc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555cd7db0050 con 0x555cd7e8f4a0 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.777+0000 7f86396bc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555cd7e9d470 con 0x555cd7e8fdc0 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.777+0000 7f86396bc740 10 client.0 _ll_register_callbacks cb 0x555cd7df0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.777+0000 7f863487e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 0x555cd7e971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.777+0000 7f863507f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 0x555cd7e99f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.778+0000 7f863507f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 0x555cd7e99f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:56018/0 (socket says 172.21.15.5:56018) 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.778+0000 7f863507f640 1 -- 172.21.15.5:0/3504563908 learned_addr learned my addr 172.21.15.5:0/3504563908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.778+0000 7f8635880640 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 0x555cd7e97710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.778+0000 7f863487e640 1 -- 172.21.15.5:0/3504563908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 msgr2=0x555cd7e99f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.778+0000 7f863487e640 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 0x555cd7e99f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.778+0000 7f863487e640 1 -- 172.21.15.5:0/3504563908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 msgr2=0x555cd7e97710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:51.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.778+0000 7f863487e640 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 0x555cd7e97710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.778+0000 7f863487e640 1 -- 172.21.15.5:0/3504563908 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd7b663c0 con 0x555cd7e8fdc0 2024-09-15T22:10:51.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.779+0000 7f8635880640 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 0x555cd7e97710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:10:51.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.779+0000 7f863507f640 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 0x555cd7e99f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:10:51.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.779+0000 7f863487e640 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 0x555cd7e971d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f862c02aa00 tx=0x7f862c005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab8bcd32206ca39b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:51.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.779+0000 7f8618ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:10:51.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.779+0000 7f8618ff9640 1 -- 172.21.15.5:0/3504563908 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c003c00 con 0x555cd7e8fdc0 2024-09-15T22:10:51.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.779+0000 7f8618ff9640 1 -- 172.21.15.5:0/3504563908 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f862c003da0 con 0x555cd7e8fdc0 2024-09-15T22:10:51.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.780+0000 7f86396bc740 1 -- 172.21.15.5:0/3504563908 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f862800fb00 con 0x555cd7e8fdc0 2024-09-15T22:10:51.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.780+0000 7f8618ff9640 1 -- 172.21.15.5:0/3504563908 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f862c0315b0 con 0x555cd7e8fdc0 2024-09-15T22:10:51.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.780+0000 7f86396bc740 10 client.25378 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:51.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.780+0000 7f86396bc740 1 -- 172.21.15.5:0/3504563908 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555cd7ea8b80 con 0x555cd7e8fdc0 2024-09-15T22:10:51.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.780+0000 7f8618ff9640 1 -- 172.21.15.5:0/3504563908 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862800fb00 con 0x555cd7e8fdc0 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.780+0000 7f8618ff9640 1 -- 172.21.15.5:0/3504563908 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f862c03a040 con 0x555cd7e8fdc0 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.780+0000 7f86396bc740 10 client.25378 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.780+0000 7f86396bc740 10 client.25378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.781+0000 7f86396bc740 20 client.25378 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.781+0000 7f86396bc740 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555cd7ea9f90 0x555cd7eac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.781+0000 7f86396bc740 4 client.25378 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.781+0000 7f86396bc740 1 -- 172.21.15.5:0/3504563908 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cd7e02850 con 0x555cd7ea9f90 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.781+0000 7f863507f640 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555cd7ea9f90 0x555cd7eac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.782+0000 7f8618ff9640 10 client.25378 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.782+0000 7f863507f640 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555cd7ea9f90 0x555cd7eac3c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71167b03b17415c1 server_cookie=af96414aa4dac055 in_seq=0 out_seq=0 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.782+0000 7f8618ff9640 1 -- 172.21.15.5:0/3504563908 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cd7e02850 con 0x555cd7ea9f90 2024-09-15T22:10:51.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.782+0000 7f8618ff9640 10 client.25378 handle_command_reply: tid=0 2024-09-15T22:10:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.782+0000 7f86396bc740 1 client.25378 shutdown 2024-09-15T22:10:51.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f86396bc740 1 -- 172.21.15.5:0/3504563908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 msgr2=0x555cd7e971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f86396bc740 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 0x555cd7e971d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f862c02aa00 tx=0x7f862c005a20 comp rx=0 tx=0).stop 2024-09-15T22:10:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f86396bc740 1 -- 172.21.15.5:0/3504563908 shutdown_connections 2024-09-15T22:10:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f86396bc740 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e9a680 0x555cd7e97710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f86396bc740 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8fdc0 0x555cd7e971d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f86396bc740 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555cd7ea9f90 0x555cd7eac3c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f86396bc740 1 --2- 172.21.15.5:0/3504563908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8f4a0 0x555cd7e99f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f86396bc740 1 -- 172.21.15.5:0/3504563908 >> 172.21.15.5:0/3504563908 conn(0x555cd7e8ba30 msgr2=0x555cd7e91710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f8618ff9640 0 client.25378 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f86396bc740 1 -- 172.21.15.5:0/3504563908 shutdown_connections 2024-09-15T22:10:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f86396bc740 1 -- 172.21.15.5:0/3504563908 wait complete. 2024-09-15T22:10:51.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.783+0000 7f86396bc740 20 client.25378 trim_cache size 0 max 16384 2024-09-15T22:10:51.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.820+0000 7f86396bc740 1 Processor -- start 2024-09-15T22:10:51.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.820+0000 7f86396bc740 1 -- start start 2024-09-15T22:10:51.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.821+0000 7f86396bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e8ba30 0x555cd7e99140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.821+0000 7f86396bc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8f4a0 0x555cd7eb1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.821+0000 7f86396bc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8fdc0 0x555cd7eb21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.821+0000 7f86396bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86301ab880 con 0x555cd7e8ba30 2024-09-15T22:10:51.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.821+0000 7f86396bc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86301a91d0 con 0x555cd7e8fdc0 2024-09-15T22:10:51.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.821+0000 7f86396bc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8630076b00 con 0x555cd7e8f4a0 2024-09-15T22:10:51.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.821+0000 7f8635880640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8fdc0 0x555cd7eb21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.821+0000 7f863487e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8f4a0 0x555cd7eb1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.821+0000 7f8635880640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8fdc0 0x555cd7eb21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:56026/0 (socket says 172.21.15.5:56026) 2024-09-15T22:10:51.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.821+0000 7f8635880640 1 -- 172.21.15.5:0/2746524809 learned_addr learned my addr 172.21.15.5:0/2746524809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:51.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.822+0000 7f863507f640 1 --2- 172.21.15.5:0/2746524809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e8ba30 0x555cd7e99140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.822+0000 7f863487e640 1 -- 172.21.15.5:0/2746524809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8fdc0 msgr2=0x555cd7eb21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:51.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.822+0000 7f863487e640 1 --2- 172.21.15.5:0/2746524809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8fdc0 0x555cd7eb21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.822+0000 7f863487e640 1 -- 172.21.15.5:0/2746524809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e8ba30 msgr2=0x555cd7e99140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:51.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.822+0000 7f863487e640 1 --2- 172.21.15.5:0/2746524809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e8ba30 0x555cd7e99140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.822+0000 7f863487e640 1 -- 172.21.15.5:0/2746524809 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cd7e8fbf0 con 0x555cd7e8f4a0 2024-09-15T22:10:51.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.822+0000 7f863487e640 1 --2- 172.21.15.5:0/2746524809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8f4a0 0x555cd7eb1c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x555cd7e98780 tx=0x7f862c009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ecc12d674a1e8ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.824+0000 7f861affd640 1 -- 172.21.15.5:0/2746524809 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c05f020 con 0x555cd7e8f4a0 2024-09-15T22:10:51.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.824+0000 7f861affd640 1 -- 172.21.15.5:0/2746524809 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f862c049650 con 0x555cd7e8f4a0 2024-09-15T22:10:51.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.825+0000 7f86396bc740 1 -- 172.21.15.5:0/2746524809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8f4a0 msgr2=0x555cd7eb1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:51.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.825+0000 7f86396bc740 1 --2- 172.21.15.5:0/2746524809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8f4a0 0x555cd7eb1c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x555cd7e98780 tx=0x7f862c009680 comp rx=0 tx=0).stop 2024-09-15T22:10:51.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.826+0000 7f86396bc740 1 -- 172.21.15.5:0/2746524809 shutdown_connections 2024-09-15T22:10:51.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.826+0000 7f86396bc740 1 --2- 172.21.15.5:0/2746524809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8fdc0 0x555cd7eb21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.826+0000 7f86396bc740 1 --2- 172.21.15.5:0/2746524809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8f4a0 0x555cd7eb1c90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.826+0000 7f86396bc740 1 --2- 172.21.15.5:0/2746524809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e8ba30 0x555cd7e99140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.826+0000 7f86396bc740 1 -- 172.21.15.5:0/2746524809 >> 172.21.15.5:0/2746524809 conn(0x555cd7e97870 msgr2=0x555cd7e98a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:51.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.829+0000 7f86396bc740 1 -- 172.21.15.5:0/2746524809 shutdown_connections 2024-09-15T22:10:51.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.830+0000 7f86396bc740 1 -- 172.21.15.5:0/2746524809 wait complete. 2024-09-15T22:10:51.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.830+0000 7f86396bc740 1 -- start start 2024-09-15T22:10:51.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.830+0000 7f86396bc740 1 Processor -- start 2024-09-15T22:10:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.831+0000 7f86396bc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8ba30 0x555cd7ea2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.831+0000 7f86396bc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8f4a0 0x555cd7ea2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.831+0000 7f86396bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e8fdc0 0x555cd7e9ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.831+0000 7f86396bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f86301a7690 con 0x555cd7e8fdc0 2024-09-15T22:10:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.831+0000 7f86396bc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f86301a8b00 con 0x555cd7e8ba30 2024-09-15T22:10:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.831+0000 7f86396bc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f86301a7d60 con 0x555cd7e8f4a0 2024-09-15T22:10:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.831+0000 7f86396bc740 10 client.0 _ll_register_callbacks cb 0x7f862c040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:10:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.832+0000 7f8635880640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e8fdc0 0x555cd7e9ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.832+0000 7f8635880640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e8fdc0 0x555cd7e9ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55064/0 (socket says 172.21.15.5:55064) 2024-09-15T22:10:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.832+0000 7f8635880640 1 -- 172.21.15.5:0/1428170844 learned_addr learned my addr 172.21.15.5:0/1428170844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:10:51.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.832+0000 7f863507f640 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8ba30 0x555cd7ea2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.832+0000 7f863487e640 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8f4a0 0x555cd7ea2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.833+0000 7f863507f640 1 -- 172.21.15.5:0/1428170844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8f4a0 msgr2=0x555cd7ea2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:51.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.833+0000 7f863507f640 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8f4a0 0x555cd7ea2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.833+0000 7f863507f640 1 -- 172.21.15.5:0/1428170844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e8fdc0 msgr2=0x555cd7e9ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:10:51.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.833+0000 7f863507f640 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e8fdc0 0x555cd7e9ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.833+0000 7f863507f640 1 -- 172.21.15.5:0/1428170844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cd7ea07e0 con 0x555cd7e8ba30 2024-09-15T22:10:51.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.835+0000 7f863507f640 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8ba30 0x555cd7ea2a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8628019850 tx=0x7f8628019750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90fc41a4fe84c8f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:10:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.837+0000 7f8618ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:10:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.837+0000 7f8618ff9640 1 -- 172.21.15.5:0/1428170844 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862801e040 con 0x555cd7e8ba30 2024-09-15T22:10:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.838+0000 7f8618ff9640 1 -- 172.21.15.5:0/1428170844 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f862800f8e0 con 0x555cd7e8ba30 2024-09-15T22:10:51.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.838+0000 7f86396bc740 1 -- 172.21.15.5:0/1428170844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555cd7dfcd60 con 0x555cd7e8ba30 2024-09-15T22:10:51.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.840+0000 7f8618ff9640 1 -- 172.21.15.5:0/1428170844 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f862800fae0 con 0x555cd7e8ba30 2024-09-15T22:10:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.842+0000 7f86396bc740 10 client.25394 fetch_fsmap learned FSMap version 25 2024-09-15T22:10:51.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.842+0000 7f86396bc740 1 -- 172.21.15.5:0/1428170844 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555cd7f14970 con 0x555cd7e8ba30 2024-09-15T22:10:51.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.843+0000 7f8618ff9640 1 -- 172.21.15.5:0/1428170844 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555cd7dfcd60 con 0x555cd7e8ba30 2024-09-15T22:10:51.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.843+0000 7f8618ff9640 1 -- 172.21.15.5:0/1428170844 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8628019a20 con 0x555cd7e8ba30 2024-09-15T22:10:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.844+0000 7f86396bc740 10 client.25394 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:10:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.844+0000 7f86396bc740 10 client.25394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:10:51.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.844+0000 7f86396bc740 20 client.25394 populate_metadata read hostname 'smithi005' 2024-09-15T22:10:51.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.844+0000 7f86396bc740 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555cd7e9a680 0x555cd7f16eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:10:51.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.844+0000 7f86396bc740 4 client.25394 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:10:51.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.844+0000 7f86396bc740 1 -- 172.21.15.5:0/1428170844 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555cd7d79210 con 0x555cd7e9a680 2024-09-15T22:10:51.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.844+0000 7f863487e640 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555cd7e9a680 0x555cd7f16eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:10:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.846+0000 7f8618ff9640 10 client.25394 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.846+0000 7f863487e640 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555cd7e9a680 0x555cd7f16eb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=330d77c0a62cd6cf server_cookie=ee08406c075da984 in_seq=0 out_seq=0 2024-09-15T22:10:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.847+0000 7f8618ff9640 1 -- 172.21.15.5:0/1428170844 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555cd7d79210 con 0x555cd7e9a680 2024-09-15T22:10:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.847+0000 7f8618ff9640 10 client.25394 handle_command_reply: tid=0 2024-09-15T22:10:51.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.847+0000 7f86396bc740 1 client.25394 shutdown 2024-09-15T22:10:51.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.850+0000 7f86396bc740 1 -- 172.21.15.5:0/1428170844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8ba30 msgr2=0x555cd7ea2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:51.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.850+0000 7f86396bc740 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8ba30 0x555cd7ea2a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8628019850 tx=0x7f8628019750 comp rx=0 tx=0).stop 2024-09-15T22:10:51.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.850+0000 7f86396bc740 1 -- 172.21.15.5:0/1428170844 shutdown_connections 2024-09-15T22:10:51.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.850+0000 7f86396bc740 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555cd7e8fdc0 0x555cd7e9ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.850+0000 7f86396bc740 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555cd7e8f4a0 0x555cd7ea2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.850+0000 7f86396bc740 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555cd7e9a680 0x555cd7f16eb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.850+0000 7f86396bc740 1 --2- 172.21.15.5:0/1428170844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555cd7e8ba30 0x555cd7ea2a60 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8628019850 tx=0x7f8628019750 comp rx=0 tx=0).stop 2024-09-15T22:10:51.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.850+0000 7f86396bc740 1 -- 172.21.15.5:0/1428170844 >> 172.21.15.5:0/1428170844 conn(0x555cd7e97870 msgr2=0x555cd7e8a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:51.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.850+0000 7f8618ff9640 0 client.25394 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:10:51.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.850+0000 7f86396bc740 1 -- 172.21.15.5:0/1428170844 shutdown_connections 2024-09-15T22:10:51.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.851+0000 7f86396bc740 1 -- 172.21.15.5:0/1428170844 wait complete. 2024-09-15T22:10:51.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.851+0000 7f86396bc740 20 client.25394 trim_cache size 0 max 16384 2024-09-15T22:10:51.853 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:10:51.853 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:10:51.853 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:10:51.853 INFO:teuthology.orchestra.run.smithi005.stdout: "75aab30d-859c-462d-81ca-49ecd7e3b858": { 2024-09-15T22:10:51.853 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:10:51.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "75aab30d-859c-462d-81ca-49ecd7e3b858", 2024-09-15T22:10:51.854 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:10:51.854 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:10:51.854 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:10:51.854 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.852+0000 7f86372e3640 1 -- 172.21.15.5:0/2839896858 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f861c0a3540 msgr2=0x7f861c0a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.852+0000 7f86372e3640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f861c0a3540 0x7f861c0a5a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f862c009850 tx=0x7f862c000bc0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.852+0000 7f86372e3640 1 -- 172.21.15.5:0/2839896858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8630112a30 msgr2=0x7f863006c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.852+0000 7f86372e3640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8630112a30 0x7f863006c220 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f862002d410 tx=0x7f862002bf90 comp rx=0 tx=0).stop 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.853+0000 7f86372e3640 1 -- 172.21.15.5:0/2839896858 shutdown_connections 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.853+0000 7f86372e3640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f861c0a3540 0x7f861c0a5a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.853+0000 7f86372e3640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8630112a30 0x7f863006c220 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.853+0000 7f86372e3640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8630077040 0x7f863006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.853+0000 7f86372e3640 1 --2- 172.21.15.5:0/2839896858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8630076720 0x7f8630073d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.853+0000 7f86372e3640 1 -- 172.21.15.5:0/2839896858 >> 172.21.15.5:0/2839896858 conn(0x7f86300ffa90 msgr2=0x7f8630101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.853+0000 7f86372e3640 1 -- 172.21.15.5:0/2839896858 shutdown_connections 2024-09-15T22:10:51.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:10:51.853+0000 7f86372e3640 1 -- 172.21.15.5:0/2839896858 wait complete. 2024-09-15T22:10:52.524 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:52 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:10:52.570 INFO:tasks.cephfs.filesystem:scrub status for tag:75aab30d-859c-462d-81ca-49ecd7e3b858 - {'path': '/', 'tag': '75aab30d-859c-462d-81ca-49ecd7e3b858', 'options': 'recursive,force'} 2024-09-15T22:10:52.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:52 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:10:52.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:52 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:10:53.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:53 smithi099 ceph-mon[34892]: pgmap v62: 129 pgs: 129 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.61k op/s 2024-09-15T22:10:53.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:53 smithi129 ceph-mon[35314]: pgmap v62: 129 pgs: 129 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.61k op/s 2024-09-15T22:10:53.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:53 smithi005 ceph-mon[30944]: pgmap v62: 129 pgs: 129 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.61k op/s 2024-09-15T22:10:54.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 85.0 s, 3384.0 tps, lat 29.564 ms stddev 34.789 2024-09-15T22:10:55.396 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:55 smithi129 ceph-mon[35314]: pgmap v63: 129 pgs: 129 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.61k op/s 2024-09-15T22:10:55.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:55 smithi099 ceph-mon[34892]: pgmap v63: 129 pgs: 129 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.61k op/s 2024-09-15T22:10:55.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:55 smithi005 ceph-mon[30944]: pgmap v63: 129 pgs: 129 active+clean; 1.4 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 30 MiB/s wr, 1.61k op/s 2024-09-15T22:10:57.390 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:57 smithi099 ceph-mon[34892]: pgmap v64: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 1.91k op/s 2024-09-15T22:10:57.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:57 smithi129 ceph-mon[35314]: pgmap v64: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 1.91k op/s 2024-09-15T22:10:57.618 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:57 smithi005 ceph-mon[30944]: pgmap v64: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 33 MiB/s wr, 1.91k op/s 2024-09-15T22:10:59.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:10:59 smithi099 ceph-mon[34892]: pgmap v65: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-09-15T22:10:59.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:10:59 smithi129 ceph-mon[35314]: pgmap v65: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-09-15T22:10:59.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:10:59 smithi005 ceph-mon[30944]: pgmap v65: 129 pgs: 129 active+clean; 1.4 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-09-15T22:10:59.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 90.0 s, 3909.0 tps, lat 25.590 ms stddev 29.413 2024-09-15T22:11:01.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:01 smithi099 ceph-mon[34892]: pgmap v66: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 958 op/s 2024-09-15T22:11:01.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:01 smithi129 ceph-mon[35314]: pgmap v66: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 958 op/s 2024-09-15T22:11:01.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:01 smithi005 ceph-mon[30944]: pgmap v66: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 MiB/s wr, 958 op/s 2024-09-15T22:11:03.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:03 smithi099 ceph-mon[34892]: pgmap v67: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.1 MiB/s wr, 744 op/s 2024-09-15T22:11:03.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:03 smithi129 ceph-mon[35314]: pgmap v67: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.1 MiB/s wr, 744 op/s 2024-09-15T22:11:03.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:03 smithi005 ceph-mon[30944]: pgmap v67: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 9.1 MiB/s wr, 744 op/s 2024-09-15T22:11:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 95.0 s, 732.4 tps, lat 135.335 ms stddev 306.119 2024-09-15T22:11:05.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:05 smithi099 ceph-mon[34892]: pgmap v68: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.6 MiB/s wr, 692 op/s 2024-09-15T22:11:05.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:05 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:11:05.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:05 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:11:05.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:05 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:11:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:05 smithi129 ceph-mon[35314]: pgmap v68: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.6 MiB/s wr, 692 op/s 2024-09-15T22:11:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:05 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:11:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:05 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:11:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:05 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:11:05.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:05 smithi005 ceph-mon[30944]: pgmap v68: 129 pgs: 129 active+clean; 1.4 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 8.6 MiB/s wr, 692 op/s 2024-09-15T22:11:05.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:05 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:11:05.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:05 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:11:05.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:05 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:11:06.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:06 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:11:06.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:06 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:11:06.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:06 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:11:06.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:06 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:11:06.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:06 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:11:06.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:06 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:11:06.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:06 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:11:06.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:06 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:11:06.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:06 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:11:07.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:07 smithi099 ceph-mon[34892]: pgmap v69: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 MiB/s wr, 2.03k op/s 2024-09-15T22:11:07.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:07 smithi129 ceph-mon[35314]: pgmap v69: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 MiB/s wr, 2.03k op/s 2024-09-15T22:11:07.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:07 smithi005 ceph-mon[30944]: pgmap v69: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 29 MiB/s wr, 2.03k op/s 2024-09-15T22:11:09.357 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:11:09.357 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-15T22:11:09.357 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:11:09.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:09 smithi099 ceph-mon[34892]: pgmap v70: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 1.72k op/s 2024-09-15T22:11:09.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:09 smithi129 ceph-mon[35314]: pgmap v70: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 1.72k op/s 2024-09-15T22:11:09.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:09 smithi005 ceph-mon[30944]: pgmap v70: 129 pgs: 129 active+clean; 1.5 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 1.72k op/s 2024-09-15T22:11:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 100.0 s, 2643.2 tps, lat 37.680 ms stddev 46.903 2024-09-15T22:11:09.739 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:10.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.676+0000 7f9e1fc40640 1 -- 172.21.15.5:0/3914459264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e18106b80 msgr2=0x7f9e18106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:10.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.676+0000 7f9e1c9da640 1 -- 172.21.15.5:0/3914459264 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e080307c0 con 0x7f9e18106b80 2024-09-15T22:11:10.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.677+0000 7f9e1fc40640 1 --2- 172.21.15.5:0/3914459264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e18106b80 0x7f9e18106f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009ca0 tx=0x7f9e0802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.683+0000 7f9e1fc40640 1 -- 172.21.15.5:0/3914459264 shutdown_connections 2024-09-15T22:11:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.683+0000 7f9e1fc40640 1 --2- 172.21.15.5:0/3914459264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e18100aa0 0x7f9e1810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.683+0000 7f9e1fc40640 1 --2- 172.21.15.5:0/3914459264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e18100100 0x7f9e18100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.683+0000 7f9e1fc40640 1 --2- 172.21.15.5:0/3914459264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e18106b80 0x7f9e18106f60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.683+0000 7f9e1fc40640 1 -- 172.21.15.5:0/3914459264 >> 172.21.15.5:0/3914459264 conn(0x7f9e18077230 msgr2=0x7f9e180fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:10.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.686+0000 7f9e1fc40640 1 -- 172.21.15.5:0/3914459264 shutdown_connections 2024-09-15T22:11:10.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.687+0000 7f9e1fc40640 1 -- 172.21.15.5:0/3914459264 wait complete. 2024-09-15T22:11:10.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.688+0000 7f9e1fc40640 1 Processor -- start 2024-09-15T22:11:10.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.688+0000 7f9e1fc40640 1 -- start start 2024-09-15T22:11:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.689+0000 7f9e1fc40640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e18100100 0x7f9e181984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.689+0000 7f9e1fc40640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e18100aa0 0x7f9e18198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.689+0000 7f9e1fc40640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e18106b80 0x7f9e1819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.689+0000 7f9e1fc40640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18112060 con 0x7f9e18100aa0 2024-09-15T22:11:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.689+0000 7f9e1fc40640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9e18111ee0 con 0x7f9e18100100 2024-09-15T22:11:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.689+0000 7f9e1fc40640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9e181121e0 con 0x7f9e18106b80 2024-09-15T22:11:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.691+0000 7f9e1e1dd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e18106b80 0x7f9e1819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.691+0000 7f9e1e1dd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e18106b80 0x7f9e1819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:40018/0 (socket says 172.21.15.5:40018) 2024-09-15T22:11:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.691+0000 7f9e1e1dd640 1 -- 172.21.15.5:0/2276274265 learned_addr learned my addr 172.21.15.5:0/2276274265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:10.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.693+0000 7f9e1e1dd640 1 -- 172.21.15.5:0/2276274265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e18100100 msgr2=0x7f9e181984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:10.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.693+0000 7f9e1e1dd640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e18100100 0x7f9e181984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.693+0000 7f9e1e1dd640 1 -- 172.21.15.5:0/2276274265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e18100aa0 msgr2=0x7f9e18198a00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:11:10.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.693+0000 7f9e1e1dd640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e18100aa0 0x7f9e18198a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.693+0000 7f9e1e1dd640 1 -- 172.21.15.5:0/2276274265 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1819d470 con 0x7f9e18106b80 2024-09-15T22:11:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.694+0000 7f9e1e1dd640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e18106b80 0x7f9e1819cd90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000be00 tx=0x7f9e1000dd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.698+0000 7f9e16ffd640 1 -- 172.21.15.5:0/2276274265 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10018070 con 0x7f9e18106b80 2024-09-15T22:11:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.698+0000 7f9e16ffd640 1 -- 172.21.15.5:0/2276274265 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e10012c40 con 0x7f9e18106b80 2024-09-15T22:11:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.698+0000 7f9e1fc40640 1 -- 172.21.15.5:0/2276274265 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1819d760 con 0x7f9e18106b80 2024-09-15T22:11:10.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.698+0000 7f9e16ffd640 1 -- 172.21.15.5:0/2276274265 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e1001c8c0 con 0x7f9e18106b80 2024-09-15T22:11:10.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.698+0000 7f9e1fc40640 1 -- 172.21.15.5:0/2276274265 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e18101210 con 0x7f9e18106b80 2024-09-15T22:11:10.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.702+0000 7f9e1fc40640 1 -- 172.21.15.5:0/2276274265 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e04005290 con 0x7f9e18106b80 2024-09-15T22:11:10.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.704+0000 7f9e16ffd640 1 -- 172.21.15.5:0/2276274265 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9e1001ca60 con 0x7f9e18106b80 2024-09-15T22:11:10.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.705+0000 7f9e16ffd640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9e000a35c0 0x7f9e000a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:10.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.706+0000 7f9e16ffd640 1 -- 172.21.15.5:0/2276274265 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f9e100d3a80 con 0x7f9e18106b80 2024-09-15T22:11:10.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.707+0000 7f9e1d9dc640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9e000a35c0 0x7f9e000a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:10.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.709+0000 7f9e16ffd640 1 -- 172.21.15.5:0/2276274265 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e100a0170 con 0x7f9e18106b80 2024-09-15T22:11:10.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:10.711+0000 7f9e1d9dc640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9e000a35c0 0x7f9e000a5a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e08006010 tx=0x7f9e08005d80 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.072+0000 7f9e1fc40640 1 -- 172.21.15.5:0/2276274265 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e04005850 con 0x7f9e18106b80 2024-09-15T22:11:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.073+0000 7f9e16ffd640 1 -- 172.21.15.5:0/2276274265 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f9e1009fa70 con 0x7f9e18106b80 2024-09-15T22:11:11.076 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:11:11.076 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:11:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:11:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.076+0000 7f9e1fc40640 1 -- 172.21.15.5:0/2276274265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9e000a35c0 msgr2=0x7f9e000a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.076+0000 7f9e1fc40640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9e000a35c0 0x7f9e000a5a80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e08006010 tx=0x7f9e08005d80 comp rx=0 tx=0).stop 2024-09-15T22:11:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.076+0000 7f9e1fc40640 1 -- 172.21.15.5:0/2276274265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e18106b80 msgr2=0x7f9e1819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.076+0000 7f9e1fc40640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e18106b80 0x7f9e1819cd90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000be00 tx=0x7f9e1000dd20 comp rx=0 tx=0).stop 2024-09-15T22:11:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.076+0000 7f9e1fc40640 1 -- 172.21.15.5:0/2276274265 shutdown_connections 2024-09-15T22:11:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.076+0000 7f9e1fc40640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9e000a35c0 0x7f9e000a5a80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.076+0000 7f9e1fc40640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9e18106b80 0x7f9e1819cd90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.076+0000 7f9e1fc40640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e18100aa0 0x7f9e18198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.076+0000 7f9e1fc40640 1 --2- 172.21.15.5:0/2276274265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9e18100100 0x7f9e181984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.076+0000 7f9e1fc40640 1 -- 172.21.15.5:0/2276274265 >> 172.21.15.5:0/2276274265 conn(0x7f9e18077230 msgr2=0x7f9e18104bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.077+0000 7f9e1fc40640 1 -- 172.21.15.5:0/2276274265 shutdown_connections 2024-09-15T22:11:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:11.077+0000 7f9e1fc40640 1 -- 172.21.15.5:0/2276274265 wait complete. 2024-09-15T22:11:11.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:11 smithi099 ceph-mon[34892]: pgmap v71: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 1.93k op/s 2024-09-15T22:11:11.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:11 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2276274265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:11.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:11 smithi129 ceph-mon[35314]: pgmap v71: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 1.93k op/s 2024-09-15T22:11:11.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:11 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2276274265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:11.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:11 smithi005 ceph-mon[30944]: pgmap v71: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 MiB/s wr, 1.93k op/s 2024-09-15T22:11:11.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:11 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2276274265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:13.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:13 smithi099 ceph-mon[34892]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 MiB/s wr, 1.67k op/s 2024-09-15T22:11:13.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:13 smithi129 ceph-mon[35314]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 MiB/s wr, 1.67k op/s 2024-09-15T22:11:13.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:13 smithi005 ceph-mon[30944]: pgmap v72: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 MiB/s wr, 1.67k op/s 2024-09-15T22:11:13.764 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-15T22:11:13.764 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-15T22:11:14.229 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 105.0 s, 2880.4 tps, lat 34.842 ms stddev 44.070 2024-09-15T22:11:14.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.847+0000 7facf6ffd640 1 -- 172.21.15.5:0/571296189 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80319b0 con 0x7facf8100920 2024-09-15T22:11:14.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.848+0000 7facfe7a0640 1 -- 172.21.15.5:0/571296189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facf8100920 msgr2=0x7facf8100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:14.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.848+0000 7facfe7a0640 1 --2- 172.21.15.5:0/571296189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facf8100920 0x7facf8100d00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7face8009900 tx=0x7face802a3f0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.849+0000 7facfe7a0640 1 -- 172.21.15.5:0/571296189 shutdown_connections 2024-09-15T22:11:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.849+0000 7facfe7a0640 1 --2- 172.21.15.5:0/571296189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf810e4d0 0x7facf81108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.849+0000 7facfe7a0640 1 --2- 172.21.15.5:0/571296189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf81012d0 0x7facf810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.849+0000 7facfe7a0640 1 --2- 172.21.15.5:0/571296189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facf8100920 0x7facf8100d00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.849+0000 7facfe7a0640 1 -- 172.21.15.5:0/571296189 >> 172.21.15.5:0/571296189 conn(0x7facf80fc790 msgr2=0x7facf80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.849+0000 7facfe7a0640 1 -- 172.21.15.5:0/571296189 shutdown_connections 2024-09-15T22:11:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.849+0000 7facfe7a0640 1 -- 172.21.15.5:0/571296189 wait complete. 2024-09-15T22:11:14.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.850+0000 7facfe7a0640 1 Processor -- start 2024-09-15T22:11:14.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.850+0000 7facfe7a0640 1 -- start start 2024-09-15T22:11:14.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facfe7a0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facf8100920 0x7facf810c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facfe7a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf81012d0 0x7facf810c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facfe7a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf810e4d0 0x7facf810cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facfe7a0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7facf8103930 con 0x7facf810e4d0 2024-09-15T22:11:14.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facfe7a0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7facf81037b0 con 0x7facf81012d0 2024-09-15T22:11:14.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facfe7a0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7facf8103ab0 con 0x7facf8100920 2024-09-15T22:11:14.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facf77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf81012d0 0x7facf810c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facf77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf81012d0 0x7facf810c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:42964/0 (socket says 172.21.15.5:42964) 2024-09-15T22:11:14.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facf77fe640 1 -- 172.21.15.5:0/815324567 learned_addr learned my addr 172.21.15.5:0/815324567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:14.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facfcd3d640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf810e4d0 0x7facf810cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facf7fff640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facf8100920 0x7facf810c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facf7fff640 1 -- 172.21.15.5:0/815324567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf81012d0 msgr2=0x7facf810c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:14.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facf7fff640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf81012d0 0x7facf810c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facf7fff640 1 -- 172.21.15.5:0/815324567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf810e4d0 msgr2=0x7facf810cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:14.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facf7fff640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf810e4d0 0x7facf810cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.851+0000 7facf7fff640 1 -- 172.21.15.5:0/815324567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf81aacc0 con 0x7facf8100920 2024-09-15T22:11:14.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.852+0000 7facf7fff640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facf8100920 0x7facf810c420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7face8006010 tx=0x7face80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:14.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.852+0000 7facf57fa640 1 -- 172.21.15.5:0/815324567 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face803f030 con 0x7facf8100920 2024-09-15T22:11:14.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.852+0000 7facf57fa640 1 -- 172.21.15.5:0/815324567 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7face8003ce0 con 0x7facf8100920 2024-09-15T22:11:14.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.852+0000 7facf57fa640 1 -- 172.21.15.5:0/815324567 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face8035070 con 0x7facf8100920 2024-09-15T22:11:14.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.853+0000 7facfe7a0640 1 -- 172.21.15.5:0/815324567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf81aaf50 con 0x7facf8100920 2024-09-15T22:11:14.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.853+0000 7facfe7a0640 1 -- 172.21.15.5:0/815324567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf81ab350 con 0x7facf8100920 2024-09-15T22:11:14.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.857+0000 7facf57fa640 1 -- 172.21.15.5:0/815324567 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7face8042030 con 0x7facf8100920 2024-09-15T22:11:14.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.858+0000 7fad00b79740 1 Processor -- start 2024-09-15T22:11:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.858+0000 7fad00b79740 1 -- start start 2024-09-15T22:11:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7fad00b79740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c4850 0x556d2f8c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7fad00b79740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7fad00b79740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8cfa30 0x556d2f8d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7fad00b79740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556d2f829160 con 0x556d2f8c5170 2024-09-15T22:11:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7facfcd3d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c4850 0x556d2f8c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7facfcd3d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c4850 0x556d2f8c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50896/0 (socket says 172.21.15.5:50896) 2024-09-15T22:11:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7facfcd3d640 1 -- 172.21.15.5:0/1593755794 learned_addr learned my addr 172.21.15.5:0/1593755794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:14.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7fad00b79740 1 -- 172.21.15.5:0/1593755794 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556d2f7d0720 con 0x556d2f8cfa30 2024-09-15T22:11:14.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7fad00b79740 1 -- 172.21.15.5:0/1593755794 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556d2f825120 con 0x556d2f8c4850 2024-09-15T22:11:14.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7facf77fe640 1 --2- 172.21.15.5:0/1593755794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8cfa30 0x556d2f8d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7facf7fff640 1 --2- 172.21.15.5:0/1593755794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.859+0000 7facf57fa640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7face00a33b0 0x7face00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.860+0000 7facf77fe640 1 -- 172.21.15.5:0/1593755794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c4850 msgr2=0x556d2f8c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:14.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.860+0000 7facf77fe640 1 --2- 172.21.15.5:0/1593755794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c4850 0x556d2f8c4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.860+0000 7facf77fe640 1 -- 172.21.15.5:0/1593755794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 msgr2=0x556d2f8cf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:14.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.860+0000 7facf57fa640 1 -- 172.21.15.5:0/815324567 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7face80f0540 con 0x7facf8100920 2024-09-15T22:11:14.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.860+0000 7facf77fe640 1 --2- 172.21.15.5:0/1593755794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8cf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.860+0000 7facf77fe640 1 -- 172.21.15.5:0/1593755794 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2f59aba0 con 0x556d2f8cfa30 2024-09-15T22:11:14.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.860+0000 7facfcd3d640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7face00a33b0 0x7face00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.861+0000 7facf77fe640 1 --2- 172.21.15.5:0/1593755794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8cfa30 0x556d2f8d1e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7facec0058c0 tx=0x7facec00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6890e0535cad047e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:14.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.861+0000 7facde7fc640 1 -- 172.21.15.5:0/1593755794 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec01d070 con 0x556d2f8cfa30 2024-09-15T22:11:14.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.861+0000 7facde7fc640 1 -- 172.21.15.5:0/1593755794 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facec018b20 con 0x556d2f8cfa30 2024-09-15T22:11:14.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.861+0000 7facde7fc640 1 -- 172.21.15.5:0/1593755794 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec010ae0 con 0x556d2f8cfa30 2024-09-15T22:11:14.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.862+0000 7fad00b79740 1 -- 172.21.15.5:0/1593755794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8cfa30 msgr2=0x556d2f8d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:14.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.862+0000 7fad00b79740 1 --2- 172.21.15.5:0/1593755794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8cfa30 0x556d2f8d1e20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7facec0058c0 tx=0x7facec00f790 comp rx=0 tx=0).stop 2024-09-15T22:11:14.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.863+0000 7fad00b79740 1 -- 172.21.15.5:0/1593755794 shutdown_connections 2024-09-15T22:11:14.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.863+0000 7fad00b79740 1 --2- 172.21.15.5:0/1593755794 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8cfa30 0x556d2f8d1e20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.863+0000 7fad00b79740 1 --2- 172.21.15.5:0/1593755794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8cf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.863+0000 7fad00b79740 1 --2- 172.21.15.5:0/1593755794 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c4850 0x556d2f8c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.863+0000 7facfcd3d640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7face00a33b0 0x7face00a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7facf0012510 tx=0x7facf0004a40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.863+0000 7fad00b79740 1 -- 172.21.15.5:0/1593755794 >> 172.21.15.5:0/1593755794 conn(0x556d2f8c0de0 msgr2=0x556d2f8c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.863+0000 7fad00b79740 1 -- 172.21.15.5:0/1593755794 shutdown_connections 2024-09-15T22:11:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.863+0000 7fad00b79740 1 -- 172.21.15.5:0/1593755794 wait complete. 2024-09-15T22:11:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.864+0000 7fad00b79740 1 -- start start 2024-09-15T22:11:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.864+0000 7fad00b79740 1 Processor -- start 2024-09-15T22:11:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.864+0000 7fad00b79740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.864+0000 7fad00b79740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.864+0000 7fad00b79740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 0x556d2f8cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.864+0000 7fad00b79740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556d2f8d2600 con 0x556d2f8c5170 2024-09-15T22:11:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.864+0000 7fad00b79740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556d2f7b1900 con 0x556d2f8c4850 2024-09-15T22:11:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.864+0000 7fad00b79740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556d2f8d2780 con 0x556d2f8cfa30 2024-09-15T22:11:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.864+0000 7fad00b79740 10 client.0 _ll_register_callbacks cb 0x556d2f7af780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facfcd3d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 0x556d2f8cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facfcd3d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 0x556d2f8cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50908/0 (socket says 172.21.15.5:50908) 2024-09-15T22:11:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facfcd3d640 1 -- 172.21.15.5:0/3681560853 learned_addr learned my addr 172.21.15.5:0/3681560853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facf7fff640 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facf77fe640 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facfcd3d640 1 -- 172.21.15.5:0/3681560853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 msgr2=0x556d2f8cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facfcd3d640 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facfcd3d640 1 -- 172.21.15.5:0/3681560853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 msgr2=0x556d2f8cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facfcd3d640 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facfcd3d640 1 -- 172.21.15.5:0/3681560853 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2f7f1c50 con 0x556d2f8cfa30 2024-09-15T22:11:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facf77fe640 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:14.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.865+0000 7facfcd3d640 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 0x556d2f8cd390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7facf0011f90 tx=0x7facf0007520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e221a6a296082d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.866+0000 7facd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:11:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.866+0000 7facd77fe640 1 -- 172.21.15.5:0/3681560853 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf001dd30 con 0x556d2f8cfa30 2024-09-15T22:11:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.866+0000 7facd77fe640 1 -- 172.21.15.5:0/3681560853 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7facf001b480 con 0x556d2f8cfa30 2024-09-15T22:11:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.866+0000 7fad00b79740 1 -- 172.21.15.5:0/3681560853 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facec018b20 con 0x556d2f8cfa30 2024-09-15T22:11:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.866+0000 7facd77fe640 1 -- 172.21.15.5:0/3681560853 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facf001b680 con 0x556d2f8cfa30 2024-09-15T22:11:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.866+0000 7fad00b79740 10 client.25402 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.866+0000 7fad00b79740 1 -- 172.21.15.5:0/3681560853 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556d2f8de310 con 0x556d2f8cfa30 2024-09-15T22:11:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.867+0000 7facd77fe640 1 -- 172.21.15.5:0/3681560853 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facec018b20 con 0x556d2f8cfa30 2024-09-15T22:11:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.867+0000 7facd77fe640 1 -- 172.21.15.5:0/3681560853 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7facf0024070 con 0x556d2f8cfa30 2024-09-15T22:11:14.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.867+0000 7fad00b79740 10 client.25402 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.867+0000 7fad00b79740 10 client.25402 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:11:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.867+0000 7fad00b79740 20 client.25402 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.867+0000 7fad00b79740 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556d2f8df870 0x556d2f8e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.867+0000 7fad00b79740 4 client.25402 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.867+0000 7fad00b79740 1 -- 172.21.15.5:0/3681560853 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d2f836890 con 0x556d2f8df870 2024-09-15T22:11:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.867+0000 7facf7fff640 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556d2f8df870 0x556d2f8e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.868+0000 7facd77fe640 10 client.25402 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.868+0000 7facf7fff640 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556d2f8df870 0x556d2f8e1d10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb1c60a61bc69c3 server_cookie=254923c5b4731122 in_seq=0 out_seq=0 2024-09-15T22:11:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.869+0000 7facd77fe640 1 -- 172.21.15.5:0/3681560853 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d2f836890 con 0x556d2f8df870 2024-09-15T22:11:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.869+0000 7facd77fe640 10 client.25402 handle_command_reply: tid=0 2024-09-15T22:11:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.869+0000 7fad00b79740 1 client.25402 shutdown 2024-09-15T22:11:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.870+0000 7fad00b79740 1 -- 172.21.15.5:0/3681560853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 msgr2=0x556d2f8cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.870+0000 7fad00b79740 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 0x556d2f8cd390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7facf0011f90 tx=0x7facf0007520 comp rx=0 tx=0).stop 2024-09-15T22:11:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.870+0000 7fad00b79740 1 -- 172.21.15.5:0/3681560853 shutdown_connections 2024-09-15T22:11:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.870+0000 7fad00b79740 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 0x556d2f8cd390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.870+0000 7fad00b79740 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.870+0000 7fad00b79740 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556d2f8df870 0x556d2f8e1d10 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.870+0000 7fad00b79740 1 --2- 172.21.15.5:0/3681560853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.870+0000 7fad00b79740 1 -- 172.21.15.5:0/3681560853 >> 172.21.15.5:0/3681560853 conn(0x556d2f8c0de0 msgr2=0x556d2f8c6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.870+0000 7facd77fe640 0 client.25402 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.870+0000 7fad00b79740 1 -- 172.21.15.5:0/3681560853 shutdown_connections 2024-09-15T22:11:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.871+0000 7fad00b79740 1 -- 172.21.15.5:0/3681560853 wait complete. 2024-09-15T22:11:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.871+0000 7fad00b79740 20 client.25402 trim_cache size 0 max 16384 2024-09-15T22:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.901+0000 7fad00b79740 1 Processor -- start 2024-09-15T22:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.901+0000 7fad00b79740 1 -- start start 2024-09-15T22:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.901+0000 7fad00b79740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8e75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7fad00b79740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8e7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7fad00b79740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 0x556d2f8c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7fad00b79740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7facf81b7e60 con 0x556d2f8c5170 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7fad00b79740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7facf81b5730 con 0x556d2f8c4850 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7fad00b79740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7facf81b3580 con 0x556d2f8cfa30 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7facf77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8e7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7facf77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8e7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35448/0 (socket says 172.21.15.5:35448) 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7facf77fe640 1 -- 172.21.15.5:0/3053724349 learned_addr learned my addr 172.21.15.5:0/3053724349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7facf7fff640 1 --2- 172.21.15.5:0/3053724349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8e75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7facfcd3d640 1 --2- 172.21.15.5:0/3053724349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 0x556d2f8c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7facf77fe640 1 -- 172.21.15.5:0/3053724349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 msgr2=0x556d2f8c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.902+0000 7facf77fe640 1 --2- 172.21.15.5:0/3053724349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 0x556d2f8c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.903+0000 7facf77fe640 1 -- 172.21.15.5:0/3053724349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 msgr2=0x556d2f8e75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.903+0000 7facf77fe640 1 --2- 172.21.15.5:0/3053724349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8e75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.903+0000 7facf77fe640 1 -- 172.21.15.5:0/3053724349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2f8bc3f0 con 0x556d2f8c5170 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.903+0000 7facf7fff640 1 --2- 172.21.15.5:0/3053724349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8e75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:14.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.903+0000 7facf77fe640 1 --2- 172.21.15.5:0/3053724349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8e7b20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7facec01b040 tx=0x7facec010620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e5c63700da277 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:14.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.904+0000 7facddffb640 1 -- 172.21.15.5:0/3053724349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec01d040 con 0x556d2f8c5170 2024-09-15T22:11:14.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.904+0000 7facddffb640 1 -- 172.21.15.5:0/3053724349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facec00a860 con 0x556d2f8c5170 2024-09-15T22:11:14.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.904+0000 7facddffb640 1 -- 172.21.15.5:0/3053724349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec004660 con 0x556d2f8c5170 2024-09-15T22:11:14.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.905+0000 7fad00b79740 1 -- 172.21.15.5:0/3053724349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 msgr2=0x556d2f8e7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:14.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.905+0000 7fad00b79740 1 --2- 172.21.15.5:0/3053724349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8e7b20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7facec01b040 tx=0x7facec010620 comp rx=0 tx=0).stop 2024-09-15T22:11:14.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.905+0000 7fad00b79740 1 -- 172.21.15.5:0/3053724349 shutdown_connections 2024-09-15T22:11:14.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.905+0000 7fad00b79740 1 --2- 172.21.15.5:0/3053724349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8cfa30 0x556d2f8c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.905+0000 7fad00b79740 1 --2- 172.21.15.5:0/3053724349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8c5170 0x556d2f8e7b20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.905+0000 7fad00b79740 1 --2- 172.21.15.5:0/3053724349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8e75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.905+0000 7fad00b79740 1 -- 172.21.15.5:0/3053724349 >> 172.21.15.5:0/3053724349 conn(0x556d2f8c0de0 msgr2=0x556d2f8ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:14.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.905+0000 7fad00b79740 1 -- 172.21.15.5:0/3053724349 shutdown_connections 2024-09-15T22:11:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.905+0000 7fad00b79740 1 -- 172.21.15.5:0/3053724349 wait complete. 2024-09-15T22:11:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.906+0000 7fad00b79740 1 -- start start 2024-09-15T22:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.906+0000 7fad00b79740 1 Processor -- start 2024-09-15T22:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.906+0000 7fad00b79740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8d37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.906+0000 7fad00b79740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c5170 0x556d2f8d3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.906+0000 7fad00b79740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8cfa30 0x556d2f8d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.906+0000 7fad00b79740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7facf81b3bf0 con 0x556d2f8cfa30 2024-09-15T22:11:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.906+0000 7fad00b79740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7facf81b5060 con 0x556d2f8c4850 2024-09-15T22:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.906+0000 7fad00b79740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7facf81b42c0 con 0x556d2f8c5170 2024-09-15T22:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.906+0000 7fad00b79740 10 client.0 _ll_register_callbacks cb 0x7facf0007cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.907+0000 7facf77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c5170 0x556d2f8d3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.907+0000 7facf77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c5170 0x556d2f8d3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50920/0 (socket says 172.21.15.5:50920) 2024-09-15T22:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.907+0000 7facf77fe640 1 -- 172.21.15.5:0/616077111 learned_addr learned my addr 172.21.15.5:0/616077111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.907+0000 7facf7fff640 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8d37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.907+0000 7facfcd3d640 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8cfa30 0x556d2f8d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.909+0000 7facf77fe640 1 -- 172.21.15.5:0/616077111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 msgr2=0x556d2f8d37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.909+0000 7facf77fe640 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8d37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.909+0000 7facf77fe640 1 -- 172.21.15.5:0/616077111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8cfa30 msgr2=0x556d2f8d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.909+0000 7facf77fe640 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8cfa30 0x556d2f8d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.909+0000 7facf77fe640 1 -- 172.21.15.5:0/616077111 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2f949630 con 0x556d2f8c5170 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.909+0000 7facfcd3d640 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8cfa30 0x556d2f8d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.909+0000 7facf77fe640 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c5170 0x556d2f8d3cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7facec00ab80 tx=0x7facec010960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d768b7abda45892a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.909+0000 7facd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.909+0000 7facd77fe640 1 -- 172.21.15.5:0/616077111 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec005450 con 0x556d2f8c5170 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.909+0000 7facd77fe640 1 -- 172.21.15.5:0/616077111 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7facec0055f0 con 0x556d2f8c5170 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.909+0000 7fad00b79740 1 -- 172.21.15.5:0/616077111 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d2f830be0 con 0x556d2f8c5170 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.910+0000 7facd77fe640 1 -- 172.21.15.5:0/616077111 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7facec010430 con 0x556d2f8c5170 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.910+0000 7fad00b79740 10 client.25408 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.910+0000 7fad00b79740 1 -- 172.21.15.5:0/616077111 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556d2f94a8f0 con 0x556d2f8c5170 2024-09-15T22:11:14.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.910+0000 7facd77fe640 1 -- 172.21.15.5:0/616077111 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556d2f830be0 con 0x556d2f8c5170 2024-09-15T22:11:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.910+0000 7facd77fe640 1 -- 172.21.15.5:0/616077111 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7facec0235f0 con 0x556d2f8c5170 2024-09-15T22:11:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.910+0000 7fad00b79740 10 client.25408 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.910+0000 7fad00b79740 10 client.25408 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:11:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.910+0000 7fad00b79740 20 client.25408 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.910+0000 7fad00b79740 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556d2f94ade0 0x556d2f94d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.910+0000 7fad00b79740 4 client.25408 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-15T22:11:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.910+0000 7fad00b79740 1 -- 172.21.15.5:0/616077111 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x556d2f7acf40 con 0x556d2f94ade0 2024-09-15T22:11:14.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.913+0000 7facf7fff640 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556d2f94ade0 0x556d2f94d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:14.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.918+0000 7facf7fff640 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556d2f94ade0 0x556d2f94d200 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41f8441a1c9e47c8 server_cookie=1e1a8d0149d4e777 in_seq=0 out_seq=0 2024-09-15T22:11:14.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:14.918+0000 7facd77fe640 10 client.25408 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:15.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facd77fe640 1 -- 172.21.15.5:0/616077111 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x556d2f7acf40 con 0x556d2f94ade0 2024-09-15T22:11:15.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facd77fe640 10 client.25408 handle_command_reply: tid=0 2024-09-15T22:11:15.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 1 client.25408 shutdown 2024-09-15T22:11:15.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 1 -- 172.21.15.5:0/616077111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c5170 msgr2=0x556d2f8d3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:15.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c5170 0x556d2f8d3cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7facec00ab80 tx=0x7facec010960 comp rx=0 tx=0).stop 2024-09-15T22:11:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 1 -- 172.21.15.5:0/616077111 shutdown_connections 2024-09-15T22:11:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d2f8cfa30 0x556d2f8d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556d2f8c5170 0x556d2f8d3cf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556d2f94ade0 0x556d2f94d200 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 1 --2- 172.21.15.5:0/616077111 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556d2f8c4850 0x556d2f8d37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facd77fe640 0 client.25408 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 1 -- 172.21.15.5:0/616077111 >> 172.21.15.5:0/616077111 conn(0x556d2f8c0de0 msgr2=0x556d2f8c62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 1 -- 172.21.15.5:0/616077111 shutdown_connections 2024-09-15T22:11:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 1 -- 172.21.15.5:0/616077111 wait complete. 2024-09-15T22:11:15.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7fad00b79740 20 client.25408 trim_cache size 0 max 16384 2024-09-15T22:11:15.227 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:11:15.228 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:11:15.228 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:11:15.228 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 6, 2024-09-15T22:11:15.228 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:11:15.228 INFO:teuthology.orchestra.run.smithi005.stdout: "dd7f13bb": { 2024-09-15T22:11:15.228 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 6, 2024-09-15T22:11:15.228 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:11:15.228 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:11:15.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:11:15.228 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:11:15.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.6, 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:15.229 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:11:15.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 -- 172.21.15.5:0/815324567 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7face00a33b0 msgr2=0x7face00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:15.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7face00a33b0 0x7face00a5870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7facf0012510 tx=0x7facf0004a40 comp rx=0 tx=0).stop 2024-09-15T22:11:15.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 -- 172.21.15.5:0/815324567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facf8100920 msgr2=0x7facf810c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:15.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facf8100920 0x7facf810c420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7face8006010 tx=0x7face80042f0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 -- 172.21.15.5:0/815324567 shutdown_connections 2024-09-15T22:11:15.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7face00a33b0 0x7face00a5870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7facf810e4d0 0x7facf810cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7facf81012d0 0x7facf810c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 --2- 172.21.15.5:0/815324567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7facf8100920 0x7facf810c420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:15.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 -- 172.21.15.5:0/815324567 >> 172.21.15.5:0/815324567 conn(0x7facf80fc790 msgr2=0x7facf80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:15.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 -- 172.21.15.5:0/815324567 shutdown_connections 2024-09-15T22:11:15.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:15.224+0000 7facfe7a0640 1 -- 172.21.15.5:0/815324567 wait complete. 2024-09-15T22:11:15.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:15 smithi099 ceph-mon[34892]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-09-15T22:11:15.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:15 smithi129 ceph-mon[35314]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-09-15T22:11:15.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:15 smithi005 ceph-mon[30944]: pgmap v73: 129 pgs: 129 active+clean; 1.5 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-09-15T22:11:17.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:17 smithi099 ceph-mon[34892]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.99k op/s 2024-09-15T22:11:17.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:17 smithi129 ceph-mon[35314]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.99k op/s 2024-09-15T22:11:17.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:17 smithi005 ceph-mon[30944]: pgmap v74: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 MiB/s wr, 1.99k op/s 2024-09-15T22:11:18.581 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dd7f13bb', quiesce took 9.2 seconds. Will release after: 10.350000000000001 2024-09-15T22:11:19.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:19 smithi099 ceph-mon[34892]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 659 op/s 2024-09-15T22:11:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:19 smithi129 ceph-mon[35314]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 659 op/s 2024-09-15T22:11:19.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:19 smithi005 ceph-mon[30944]: pgmap v75: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 659 op/s 2024-09-15T22:11:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 110.0 s, 169.8 tps, lat 36.984 ms stddev 44.606 2024-09-15T22:11:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:21 smithi005 ceph-mon[30944]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 676 op/s 2024-09-15T22:11:21.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:21 smithi099 ceph-mon[34892]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 676 op/s 2024-09-15T22:11:21.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:21 smithi129 ceph-mon[35314]: pgmap v76: 129 pgs: 129 active+clean; 1.5 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.0 MiB/s wr, 676 op/s 2024-09-15T22:11:22.571 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:11:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:23.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:23 smithi005 ceph-mon[30944]: pgmap v77: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 485 op/s 2024-09-15T22:11:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.297+0000 7f4d5846d640 1 -- 172.21.15.5:0/4030033278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d50106f10 msgr2=0x7f4d5010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.297+0000 7f4d5846d640 1 --2- 172.21.15.5:0/4030033278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d50106f10 0x7f4d5010ea90 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009ca0 tx=0x7f4d4802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.297+0000 7f4d5846d640 1 -- 172.21.15.5:0/4030033278 shutdown_connections 2024-09-15T22:11:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.297+0000 7f4d5846d640 1 --2- 172.21.15.5:0/4030033278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d50106f10 0x7f4d5010ea90 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.297+0000 7f4d5846d640 1 --2- 172.21.15.5:0/4030033278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d50102790 0x7f4d501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.297+0000 7f4d5846d640 1 --2- 172.21.15.5:0/4030033278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d50101de0 0x7f4d501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.297+0000 7f4d5846d640 1 -- 172.21.15.5:0/4030033278 >> 172.21.15.5:0/4030033278 conn(0x7f4d500fd8c0 msgr2=0x7f4d500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.297+0000 7f4d5846d640 1 -- 172.21.15.5:0/4030033278 shutdown_connections 2024-09-15T22:11:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.297+0000 7f4d5846d640 1 -- 172.21.15.5:0/4030033278 wait complete. 2024-09-15T22:11:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d5846d640 1 Processor -- start 2024-09-15T22:11:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d5846d640 1 -- start start 2024-09-15T22:11:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d5846d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d50101de0 0x7f4d50198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d5846d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d50102790 0x7f4d501989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d5846d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d50106f10 0x7f4d5019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d5846d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d50111fe0 con 0x7f4d50102790 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d5846d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d50111e60 con 0x7f4d50106f10 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d5846d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d50112160 con 0x7f4d50101de0 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d55a08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d50102790 0x7f4d501989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d55a08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d50102790 0x7f4d501989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35466/0 (socket says 172.21.15.5:35466) 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d55a08640 1 -- 172.21.15.5:0/3129902922 learned_addr learned my addr 172.21.15.5:0/3129902922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.298+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d50106f10 0x7f4d5019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d56209640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d50101de0 0x7f4d50198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d55a08640 1 -- 172.21.15.5:0/3129902922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d50101de0 msgr2=0x7f4d50198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d55a08640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d50101de0 0x7f4d50198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d55a08640 1 -- 172.21.15.5:0/3129902922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d50106f10 msgr2=0x7f4d5019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d55a08640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d50106f10 0x7f4d5019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d55a08640 1 -- 172.21.15.5:0/3129902922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5019d430 con 0x7f4d50102790 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d56209640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d50101de0 0x7f4d50198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d55a08640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d50102790 0x7f4d501989c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000b4a0 tx=0x7f4d4000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d477fe640 1 -- 172.21.15.5:0/3129902922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40014070 con 0x7f4d50102790 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d477fe640 1 -- 172.21.15.5:0/3129902922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d40010af0 con 0x7f4d50102790 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d477fe640 1 -- 172.21.15.5:0/3129902922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40017d10 con 0x7f4d50102790 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d5846d640 1 -- 172.21.15.5:0/3129902922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5019d720 con 0x7f4d50102790 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.299+0000 7f4d5846d640 1 -- 172.21.15.5:0/3129902922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d501a5000 con 0x7f4d50102790 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d477fe640 1 -- 172.21.15.5:0/3129902922 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4d40017070 con 0x7f4d50102790 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d5a846740 1 Processor -- start 2024-09-15T22:11:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d5a846740 1 -- start start 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4d3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4d3dc0 0x56218f4de140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4de680 0x56218f4e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d5a846740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56218f4392e0 con 0x56218f4d34a0 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d5a846740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56218f3e0ce0 con 0x56218f4d3dc0 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d5a846740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56218f4352a0 con 0x56218f4de680 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4d3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35474/0 (socket says 172.21.15.5:35474) 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56a0a640 1 -- 172.21.15.5:0/2249334625 learned_addr learned my addr 172.21.15.5:0/2249334625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56209640 1 --2- 172.21.15.5:0/2249334625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4d3dc0 0x56218f4de140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d477fe640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d3c0a3430 0x7f4d3c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d55a08640 1 --2- 172.21.15.5:0/2249334625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4de680 0x56218f4e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56a0a640 1 -- 172.21.15.5:0/2249334625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4de680 msgr2=0x56218f4e0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/2249334625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4de680 0x56218f4e0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56a0a640 1 -- 172.21.15.5:0/2249334625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4d3dc0 msgr2=0x56218f4de140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/2249334625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4d3dc0 0x56218f4de140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56a0a640 1 -- 172.21.15.5:0/2249334625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218f3c5440 con 0x56218f4d34a0 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d3c0a3430 0x7f4d3c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d477fe640 1 -- 172.21.15.5:0/3129902922 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4d400cf2a0 con 0x7f4d50102790 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/2249334625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4d3880 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009ca0 tx=0x7f4d48031bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4872e0d3ba42888c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d44dea640 1 -- 172.21.15.5:0/2249334625 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48042040 con 0x56218f4d34a0 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d3c0a3430 0x7f4d3c0a58f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d4804d000 tx=0x7f4d4804e000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d44dea640 1 -- 172.21.15.5:0/2249334625 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d48036070 con 0x56218f4d34a0 2024-09-15T22:11:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.303+0000 7f4d44dea640 1 -- 172.21.15.5:0/2249334625 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4802f810 con 0x56218f4d34a0 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.304+0000 7f4d5a846740 1 -- 172.21.15.5:0/2249334625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 msgr2=0x56218f4d3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.304+0000 7f4d5a846740 1 --2- 172.21.15.5:0/2249334625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4d3880 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009ca0 tx=0x7f4d48031bd0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.304+0000 7f4d5a846740 1 -- 172.21.15.5:0/2249334625 shutdown_connections 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.304+0000 7f4d5a846740 1 --2- 172.21.15.5:0/2249334625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4de680 0x56218f4e0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.304+0000 7f4d5a846740 1 --2- 172.21.15.5:0/2249334625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4d3dc0 0x56218f4de140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.304+0000 7f4d5a846740 1 --2- 172.21.15.5:0/2249334625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4d3880 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.304+0000 7f4d5a846740 1 -- 172.21.15.5:0/2249334625 >> 172.21.15.5:0/2249334625 conn(0x56218f4cfa30 msgr2=0x56218f4cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.304+0000 7f4d5a846740 1 -- 172.21.15.5:0/2249334625 shutdown_connections 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.304+0000 7f4d5a846740 1 -- 172.21.15.5:0/2249334625 wait complete. 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.304+0000 7f4d5a846740 1 -- start start 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d5a846740 1 Processor -- start 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4ddfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d3dc0 0x56218f4db260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4de680 0x56218f4db7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d5a846740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56218f4e1250 con 0x56218f4d34a0 2024-09-15T22:11:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d5a846740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56218f3f4050 con 0x56218f4de680 2024-09-15T22:11:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d5a846740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56218f4e13d0 con 0x56218f4d3dc0 2024-09-15T22:11:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d5a846740 10 client.0 _ll_register_callbacks cb 0x56218f434160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d56209640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4ddfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d56209640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4ddfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35484/0 (socket says 172.21.15.5:35484) 2024-09-15T22:11:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d56209640 1 -- 172.21.15.5:0/1580356189 learned_addr learned my addr 172.21.15.5:0/1580356189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d55a08640 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d3dc0 0x56218f4db260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4de680 0x56218f4db7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d56209640 1 -- 172.21.15.5:0/1580356189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d3dc0 msgr2=0x56218f4db260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d56209640 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d3dc0 0x56218f4db260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d56209640 1 -- 172.21.15.5:0/1580356189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4de680 msgr2=0x56218f4db7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d56209640 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4de680 0x56218f4db7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d56209640 1 -- 172.21.15.5:0/1580356189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218f1aa3c0 con 0x56218f4d34a0 2024-09-15T22:11:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4de680 0x56218f4db7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d56209640 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4ddfb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0107c0 tx=0x7f4d4c010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c311190a22d5a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.305+0000 7f4d39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d39ffb640 1 -- 172.21.15.5:0/1580356189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c00ed40 con 0x56218f4d34a0 2024-09-15T22:11:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d39ffb640 1 -- 172.21.15.5:0/1580356189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4d4c005020 con 0x56218f4d34a0 2024-09-15T22:11:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d5a846740 1 -- 172.21.15.5:0/1580356189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d48036070 con 0x56218f4d34a0 2024-09-15T22:11:23.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d39ffb640 1 -- 172.21.15.5:0/1580356189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d4c024050 con 0x56218f4d34a0 2024-09-15T22:11:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d5a846740 10 client.15807 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d5a846740 1 -- 172.21.15.5:0/1580356189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56218f4ecc60 con 0x56218f4d34a0 2024-09-15T22:11:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d39ffb640 1 -- 172.21.15.5:0/1580356189 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d48036070 con 0x56218f4d34a0 2024-09-15T22:11:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d39ffb640 1 -- 172.21.15.5:0/1580356189 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4d4c011440 con 0x56218f4d34a0 2024-09-15T22:11:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d5a846740 10 client.15807 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d5a846740 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d5a846740 20 client.15807 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d5a846740 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56218f4ee120 0x56218f4f0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d5a846740 4 client.15807 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.306+0000 7f4d5a846740 1 -- 172.21.15.5:0/1580356189 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56218f446850 con 0x56218f4ee120 2024-09-15T22:11:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.307+0000 7f4d55a08640 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56218f4ee120 0x56218f4f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.307+0000 7f4d39ffb640 10 client.15807 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.307+0000 7f4d55a08640 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56218f4ee120 0x56218f4f0590 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5aaf2b0fd9d81b3 server_cookie=be6bfb0a4cafc10d in_seq=0 out_seq=0 2024-09-15T22:11:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d39ffb640 1 -- 172.21.15.5:0/1580356189 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56218f446850 con 0x56218f4ee120 2024-09-15T22:11:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d39ffb640 10 client.15807 handle_command_reply: tid=0 2024-09-15T22:11:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 1 client.15807 shutdown 2024-09-15T22:11:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 1 -- 172.21.15.5:0/1580356189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 msgr2=0x56218f4ddfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4ddfb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0107c0 tx=0x7f4d4c010c80 comp rx=0 tx=0).stop 2024-09-15T22:11:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 1 -- 172.21.15.5:0/1580356189 shutdown_connections 2024-09-15T22:11:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56218f4ee120 0x56218f4f0590 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4de680 0x56218f4db7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d3dc0 0x56218f4db260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d39ffb640 0 client.15807 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 1 --2- 172.21.15.5:0/1580356189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d34a0 0x56218f4ddfb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 1 -- 172.21.15.5:0/1580356189 >> 172.21.15.5:0/1580356189 conn(0x56218f4cfa30 msgr2=0x56218f4ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 1 -- 172.21.15.5:0/1580356189 shutdown_connections 2024-09-15T22:11:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 1 -- 172.21.15.5:0/1580356189 wait complete. 2024-09-15T22:11:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.308+0000 7f4d5a846740 20 client.15807 trim_cache size 0 max 16384 2024-09-15T22:11:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.325+0000 7f4d5a846740 1 Processor -- start 2024-09-15T22:11:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.325+0000 7f4d5a846740 1 -- start start 2024-09-15T22:11:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4d34a0 0x56218f4ea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d3dc0 0x56218f4cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4de680 0x56218f4cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d5a846740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d501b1800 con 0x56218f4de680 2024-09-15T22:11:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d5a846740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d501af0d0 con 0x56218f4d34a0 2024-09-15T22:11:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d5a846740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d501acf20 con 0x56218f4d3dc0 2024-09-15T22:11:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d56a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4de680 0x56218f4cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d56a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4de680 0x56218f4cea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35486/0 (socket says 172.21.15.5:35486) 2024-09-15T22:11:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d56a0a640 1 -- 172.21.15.5:0/1849089051 learned_addr learned my addr 172.21.15.5:0/1849089051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d55a08640 1 --2- 172.21.15.5:0/1849089051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d3dc0 0x56218f4cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d56209640 1 --2- 172.21.15.5:0/1849089051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4d34a0 0x56218f4ea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d56a0a640 1 -- 172.21.15.5:0/1849089051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d3dc0 msgr2=0x56218f4cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/1849089051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d3dc0 0x56218f4cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d56a0a640 1 -- 172.21.15.5:0/1849089051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4d34a0 msgr2=0x56218f4ea4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/1849089051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4d34a0 0x56218f4ea4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d56a0a640 1 -- 172.21.15.5:0/1849089051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56218f4da210 con 0x56218f4de680 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/1849089051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4de680 0x56218f4cea80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009430 tx=0x7f4d4802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c13782f14227147 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d3bfff640 1 -- 172.21.15.5:0/1849089051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48042040 con 0x56218f4de680 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d3bfff640 1 -- 172.21.15.5:0/1849089051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d48059070 con 0x56218f4de680 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.326+0000 7f4d3bfff640 1 -- 172.21.15.5:0/1849089051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48004260 con 0x56218f4de680 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.327+0000 7f4d5a846740 1 -- 172.21.15.5:0/1849089051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4de680 msgr2=0x56218f4cea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.327+0000 7f4d5a846740 1 --2- 172.21.15.5:0/1849089051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4de680 0x56218f4cea80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009430 tx=0x7f4d4802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.327+0000 7f4d5a846740 1 -- 172.21.15.5:0/1849089051 shutdown_connections 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.327+0000 7f4d5a846740 1 --2- 172.21.15.5:0/1849089051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4de680 0x56218f4cea80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.327+0000 7f4d5a846740 1 --2- 172.21.15.5:0/1849089051 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d3dc0 0x56218f4cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.327+0000 7f4d5a846740 1 --2- 172.21.15.5:0/1849089051 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4d34a0 0x56218f4ea4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.327+0000 7f4d5a846740 1 -- 172.21.15.5:0/1849089051 >> 172.21.15.5:0/1849089051 conn(0x56218f4cfa30 msgr2=0x56218f4ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.327+0000 7f4d5a846740 1 -- 172.21.15.5:0/1849089051 shutdown_connections 2024-09-15T22:11:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.327+0000 7f4d5a846740 1 -- 172.21.15.5:0/1849089051 wait complete. 2024-09-15T22:11:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.327+0000 7f4d5a846740 1 -- start start 2024-09-15T22:11:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d5a846740 1 Processor -- start 2024-09-15T22:11:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d34a0 0x56218f4e2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d3dc0 0x56218f4e3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d5a846740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4de680 0x56218f557e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d5a846740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d501ad590 con 0x56218f4d3dc0 2024-09-15T22:11:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d5a846740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d501aea00 con 0x56218f4de680 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d5a846740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d501adc60 con 0x56218f4d34a0 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d5a846740 10 client.0 _ll_register_callbacks cb 0x7f4d4c00c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d55a08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d3dc0 0x56218f4e3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d55a08640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d3dc0 0x56218f4e3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35502/0 (socket says 172.21.15.5:35502) 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d55a08640 1 -- 172.21.15.5:0/3895714435 learned_addr learned my addr 172.21.15.5:0/3895714435 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d56a0a640 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4de680 0x56218f557e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d56209640 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d34a0 0x56218f4e2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d55a08640 1 -- 172.21.15.5:0/3895714435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d34a0 msgr2=0x56218f4e2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d55a08640 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d34a0 0x56218f4e2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d55a08640 1 -- 172.21.15.5:0/3895714435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4de680 msgr2=0x56218f557e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d55a08640 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4de680 0x56218f557e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d55a08640 1 -- 172.21.15.5:0/3895714435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56218f558370 con 0x56218f4d3dc0 2024-09-15T22:11:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d56209640 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d34a0 0x56218f4e2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d55a08640 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d3dc0 0x56218f4e3050 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4d400cc7c0 tx=0x7f4d40017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a440df012ce0aef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.328+0000 7f4d39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d39ffb640 1 -- 172.21.15.5:0/3895714435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40095e60 con 0x56218f4d3dc0 2024-09-15T22:11:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d39ffb640 1 -- 172.21.15.5:0/3895714435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4d40010ca0 con 0x56218f4d3dc0 2024-09-15T22:11:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 -- 172.21.15.5:0/3895714435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d48059070 con 0x56218f4d3dc0 2024-09-15T22:11:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d39ffb640 1 -- 172.21.15.5:0/3895714435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d4001e040 con 0x56218f4d3dc0 2024-09-15T22:11:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 10 client.15813 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 -- 172.21.15.5:0/3895714435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56218f559710 con 0x56218f4d3dc0 2024-09-15T22:11:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d39ffb640 1 -- 172.21.15.5:0/3895714435 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d48059070 con 0x56218f4d3dc0 2024-09-15T22:11:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d39ffb640 1 -- 172.21.15.5:0/3895714435 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4d40097960 con 0x56218f4d3dc0 2024-09-15T22:11:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 10 client.15813 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 10 client.15813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 20 client.15813 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56218f559e40 0x56218f55c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 4 client.15813 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:11:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 -- 172.21.15.5:0/3895714435 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56218f440d60 con 0x56218f559e40 2024-09-15T22:11:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d56209640 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56218f559e40 0x56218f55c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d56209640 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56218f559e40 0x56218f55c260 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=741ccf3bce615ab1 server_cookie=ad75491628502179 in_seq=0 out_seq=0 2024-09-15T22:11:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d39ffb640 10 client.15813 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d39ffb640 1 -- 172.21.15.5:0/3895714435 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56218f440d60 con 0x56218f559e40 2024-09-15T22:11:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d39ffb640 10 client.15813 handle_command_reply: tid=0 2024-09-15T22:11:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 client.15813 shutdown 2024-09-15T22:11:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 -- 172.21.15.5:0/3895714435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d3dc0 msgr2=0x56218f4e3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d3dc0 0x56218f4e3050 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4d400cc7c0 tx=0x7f4d40017730 comp rx=0 tx=0).stop 2024-09-15T22:11:23.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 -- 172.21.15.5:0/3895714435 shutdown_connections 2024-09-15T22:11:23.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56218f559e40 0x56218f55c260 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56218f4de680 0x56218f557e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56218f4d3dc0 0x56218f4e3050 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.330+0000 7f4d5a846740 1 --2- 172.21.15.5:0/3895714435 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56218f4d34a0 0x56218f4e2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5a846740 1 -- 172.21.15.5:0/3895714435 >> 172.21.15.5:0/3895714435 conn(0x56218f4cfa30 msgr2=0x56218f4d5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d39ffb640 0 client.15813 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5a846740 1 -- 172.21.15.5:0/3895714435 shutdown_connections 2024-09-15T22:11:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5a846740 1 -- 172.21.15.5:0/3895714435 wait complete. 2024-09-15T22:11:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5a846740 20 client.15813 trim_cache size 0 max 16384 2024-09-15T22:11:23.342 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:11:23.342 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:11:23.343 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:11:23.343 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:11:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5846d640 1 -- 172.21.15.5:0/3129902922 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d3c0a3430 msgr2=0x7f4d3c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5846d640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d3c0a3430 0x7f4d3c0a58f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d4804d000 tx=0x7f4d4804e000 comp rx=0 tx=0).stop 2024-09-15T22:11:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5846d640 1 -- 172.21.15.5:0/3129902922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d50102790 msgr2=0x7f4d501989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5846d640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d50102790 0x7f4d501989c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000b4a0 tx=0x7f4d4000b960 comp rx=0 tx=0).stop 2024-09-15T22:11:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5846d640 1 -- 172.21.15.5:0/3129902922 shutdown_connections 2024-09-15T22:11:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5846d640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d3c0a3430 0x7f4d3c0a58f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5846d640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d50106f10 0x7f4d5019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5846d640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d50102790 0x7f4d501989c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5846d640 1 --2- 172.21.15.5:0/3129902922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d50101de0 0x7f4d50198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.331+0000 7f4d5846d640 1 -- 172.21.15.5:0/3129902922 >> 172.21.15.5:0/3129902922 conn(0x7f4d500fd8c0 msgr2=0x7f4d500fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.332+0000 7f4d5846d640 1 -- 172.21.15.5:0/3129902922 shutdown_connections 2024-09-15T22:11:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:23.332+0000 7f4d5846d640 1 -- 172.21.15.5:0/3129902922 wait complete. 2024-09-15T22:11:23.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:23 smithi099 ceph-mon[34892]: pgmap v77: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 485 op/s 2024-09-15T22:11:23.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:23 smithi129 ceph-mon[35314]: pgmap v77: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.9 MiB/s wr, 485 op/s 2024-09-15T22:11:24.168 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:11:24.169 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:11:24.429 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 115.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:11:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.922+0000 7f1db1c8e640 1 -- 172.21.15.5:0/3253623868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dac102790 msgr2=0x7f1dac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.922+0000 7f1db1c8e640 1 --2- 172.21.15.5:0/3253623868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dac102790 0x7f1dac1067e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1da0009900 tx=0x7f1da002a410 comp rx=0 tx=0).stop 2024-09-15T22:11:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.922+0000 7f1db1c8e640 1 -- 172.21.15.5:0/3253623868 shutdown_connections 2024-09-15T22:11:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.922+0000 7f1db1c8e640 1 --2- 172.21.15.5:0/3253623868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dac106f10 0x7f1dac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.922+0000 7f1db1c8e640 1 --2- 172.21.15.5:0/3253623868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dac102790 0x7f1dac1067e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.922+0000 7f1db1c8e640 1 --2- 172.21.15.5:0/3253623868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dac101de0 0x7f1dac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.922+0000 7f1db1c8e640 1 -- 172.21.15.5:0/3253623868 >> 172.21.15.5:0/3253623868 conn(0x7f1dac0fd8c0 msgr2=0x7f1dac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.922+0000 7f1db1c8e640 1 -- 172.21.15.5:0/3253623868 shutdown_connections 2024-09-15T22:11:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.922+0000 7f1db1c8e640 1 -- 172.21.15.5:0/3253623868 wait complete. 2024-09-15T22:11:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.923+0000 7f1db1c8e640 1 Processor -- start 2024-09-15T22:11:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.923+0000 7f1db1c8e640 1 -- start start 2024-09-15T22:11:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.923+0000 7f1db1c8e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dac101de0 0x7f1dac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.923+0000 7f1db1c8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dac102790 0x7f1dac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.923+0000 7f1db1c8e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dac106f10 0x7f1dac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.923+0000 7f1db1c8e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac111f70 con 0x7f1dac102790 2024-09-15T22:11:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.923+0000 7f1db1c8e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac111df0 con 0x7f1dac106f10 2024-09-15T22:11:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.923+0000 7f1db1c8e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1120f0 con 0x7f1dac101de0 2024-09-15T22:11:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.923+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dac102790 0x7f1dac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1daaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dac102790 0x7f1dac074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56724/0 (socket says 172.21.15.5:56724) 2024-09-15T22:11:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1daaffd640 1 -- 172.21.15.5:0/30765708 learned_addr learned my addr 172.21.15.5:0/30765708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1dabfff640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dac106f10 0x7f1dac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1dab7fe640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dac101de0 0x7f1dac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1daaffd640 1 -- 172.21.15.5:0/30765708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dac101de0 msgr2=0x7f1dac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1daaffd640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dac101de0 0x7f1dac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1daaffd640 1 -- 172.21.15.5:0/30765708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dac106f10 msgr2=0x7f1dac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1daaffd640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dac106f10 0x7f1dac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1daaffd640 1 -- 172.21.15.5:0/30765708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac1a6910 con 0x7f1dac102790 2024-09-15T22:11:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1daaffd640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dac102790 0x7f1dac074a30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1da0006010 tx=0x7f1da00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1da8ff9640 1 -- 172.21.15.5:0/30765708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da003f030 con 0x7f1dac102790 2024-09-15T22:11:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1da8ff9640 1 -- 172.21.15.5:0/30765708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1da0003ce0 con 0x7f1dac102790 2024-09-15T22:11:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1db1c8e640 1 -- 172.21.15.5:0/30765708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac1a6ba0 con 0x7f1dac102790 2024-09-15T22:11:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1da8ff9640 1 -- 172.21.15.5:0/30765708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0035070 con 0x7f1dac102790 2024-09-15T22:11:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.924+0000 7f1db1c8e640 1 -- 172.21.15.5:0/30765708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac1a7080 con 0x7f1dac102790 2024-09-15T22:11:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.925+0000 7f1db1c8e640 1 -- 172.21.15.5:0/30765708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dac1021c0 con 0x7f1dac102790 2024-09-15T22:11:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.926+0000 7f1da8ff9640 1 -- 172.21.15.5:0/30765708 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1da0042030 con 0x7f1dac102790 2024-09-15T22:11:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.927+0000 7f1da8ff9640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1d940a33b0 0x7f1d940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.927+0000 7f1dab7fe640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1d940a33b0 0x7f1d940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.927+0000 7f1da8ff9640 1 -- 172.21.15.5:0/30765708 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f1da00f05d0 con 0x7f1dac102790 2024-09-15T22:11:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.928+0000 7f1dab7fe640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1d940a33b0 0x7f1d940a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0058c0 tx=0x7f1d9c00a4c0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:24.931+0000 7f1da8ff9640 1 -- 172.21.15.5:0/30765708 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1da00bccd0 con 0x7f1dac102790 2024-09-15T22:11:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.143+0000 7f1db1c8e640 1 -- 172.21.15.5:0/30765708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dac10a150 con 0x7f1dac102790 2024-09-15T22:11:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.144+0000 7f1da8ff9640 1 -- 172.21.15.5:0/30765708 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f1da00bc5d0 con 0x7f1dac102790 2024-09-15T22:11:25.145 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:11:25.146 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:11:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:11:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 -- 172.21.15.5:0/30765708 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1d940a33b0 msgr2=0x7f1d940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1d940a33b0 0x7f1d940a5870 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c0058c0 tx=0x7f1d9c00a4c0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 -- 172.21.15.5:0/30765708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dac102790 msgr2=0x7f1dac074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dac102790 0x7f1dac074a30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1da0006010 tx=0x7f1da00042f0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 -- 172.21.15.5:0/30765708 shutdown_connections 2024-09-15T22:11:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1d940a33b0 0x7f1d940a5870 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1dac106f10 0x7f1dac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dac102790 0x7f1dac074a30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 --2- 172.21.15.5:0/30765708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1dac101de0 0x7f1dac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 -- 172.21.15.5:0/30765708 >> 172.21.15.5:0/30765708 conn(0x7f1dac0fd8c0 msgr2=0x7f1dac0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 -- 172.21.15.5:0/30765708 shutdown_connections 2024-09-15T22:11:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:25.146+0000 7f1db1c8e640 1 -- 172.21.15.5:0/30765708 wait complete. 2024-09-15T22:11:25.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:25 smithi005 ceph-mon[30944]: pgmap v78: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 381 op/s 2024-09-15T22:11:25.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:25 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/30765708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:25.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:25 smithi099 ceph-mon[34892]: pgmap v78: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 381 op/s 2024-09-15T22:11:25.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:25 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/30765708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:25.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:25 smithi129 ceph-mon[35314]: pgmap v78: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 5.1 MiB/s wr, 381 op/s 2024-09-15T22:11:25.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:25 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/30765708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:25.972 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:11:26.234 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.396+0000 7f88d7e16640 1 -- 172.21.15.5:0/511622359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88d0100b30 msgr2=0x7f88d0100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.396+0000 7f88d7e16640 1 --2- 172.21.15.5:0/511622359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88d0100b30 0x7f88d0100f10 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009900 tx=0x7f88cc02a470 comp rx=0 tx=0).stop 2024-09-15T22:11:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.396+0000 7f88d7e16640 1 -- 172.21.15.5:0/511622359 shutdown_connections 2024-09-15T22:11:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.396+0000 7f88d7e16640 1 --2- 172.21.15.5:0/511622359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d0105f40 0x7f88d01063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.396+0000 7f88d7e16640 1 --2- 172.21.15.5:0/511622359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0101450 0x7f88d0105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.396+0000 7f88d7e16640 1 --2- 172.21.15.5:0/511622359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88d0100b30 0x7f88d0100f10 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.396+0000 7f88d7e16640 1 -- 172.21.15.5:0/511622359 >> 172.21.15.5:0/511622359 conn(0x7f88d00fc7a0 msgr2=0x7f88d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.396+0000 7f88d7e16640 1 -- 172.21.15.5:0/511622359 shutdown_connections 2024-09-15T22:11:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.396+0000 7f88d7e16640 1 -- 172.21.15.5:0/511622359 wait complete. 2024-09-15T22:11:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.397+0000 7f88d7e16640 1 Processor -- start 2024-09-15T22:11:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.397+0000 7f88d7e16640 1 -- start start 2024-09-15T22:11:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.397+0000 7f88d7e16640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0100b30 0x7f88d01a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.397+0000 7f88d7e16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88d0101450 0x7f88d01a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.397+0000 7f88d7e16640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d0105f40 0x7f88d019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.397+0000 7f88d7e16640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01185d0 con 0x7f88d0101450 2024-09-15T22:11:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.397+0000 7f88d7e16640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0118450 con 0x7f88d0100b30 2024-09-15T22:11:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.397+0000 7f88d7e16640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0118750 con 0x7f88d0105f40 2024-09-15T22:11:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.397+0000 7f88d53b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88d0101450 0x7f88d01a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.397+0000 7f88d53b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88d0101450 0x7f88d01a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56738/0 (socket says 172.21.15.5:56738) 2024-09-15T22:11:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d53b1640 1 -- 172.21.15.5:0/172438920 learned_addr learned my addr 172.21.15.5:0/172438920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0100b30 0x7f88d01a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d63b3640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d0105f40 0x7f88d019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d53b1640 1 -- 172.21.15.5:0/172438920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d0105f40 msgr2=0x7f88d019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d53b1640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d0105f40 0x7f88d019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d53b1640 1 -- 172.21.15.5:0/172438920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0100b30 msgr2=0x7f88d01a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d53b1640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0100b30 0x7f88d01a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d53b1640 1 -- 172.21.15.5:0/172438920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d019efb0 con 0x7f88d0101450 2024-09-15T22:11:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0100b30 0x7f88d01a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d53b1640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88d0101450 0x7f88d01a4c00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c970 tx=0x7f88c000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88c6ffd640 1 -- 172.21.15.5:0/172438920 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0007bb0 con 0x7f88d0101450 2024-09-15T22:11:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88c6ffd640 1 -- 172.21.15.5:0/172438920 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c0007d50 con 0x7f88d0101450 2024-09-15T22:11:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d7e16640 1 -- 172.21.15.5:0/172438920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d019f2a0 con 0x7f88d0101450 2024-09-15T22:11:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88c6ffd640 1 -- 172.21.15.5:0/172438920 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0018820 con 0x7f88d0101450 2024-09-15T22:11:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.398+0000 7f88d7e16640 1 -- 172.21.15.5:0/172438920 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d01ab590 con 0x7f88d0101450 2024-09-15T22:11:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.401+0000 7f88c6ffd640 1 -- 172.21.15.5:0/172438920 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f88c00189c0 con 0x7f88d0101450 2024-09-15T22:11:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.401+0000 7f88da1ef740 1 Processor -- start 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.401+0000 7f88da1ef740 1 -- start start 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ebf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ecc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88da1ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f3a8e252f0 con 0x55f3a8eca370 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88da1ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f3a8dcccf0 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88da1ef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f3a8e212b0 con 0x55f3a8ec0b40 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88c6ffd640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88bc0a3430 0x7f88bc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d53b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ecc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d5bb2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d53b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ecc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56752/0 (socket says 172.21.15.5:56752) 2024-09-15T22:11:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d63b3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ebf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d5bb2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:37810/0 (socket says 172.21.15.5:37810) 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d5bb2640 1 -- 172.21.15.5:0/3341564651 learned_addr learned my addr 172.21.15.5:0/3341564651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d63b3640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88bc0a3430 0x7f88bc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88c6ffd640 1 -- 172.21.15.5:0/172438920 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f88c00d00d0 con 0x7f88d0101450 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d63b3640 1 -- 172.21.15.5:0/3341564651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 msgr2=0x55f3a8ebfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d63b3640 1 --2- 172.21.15.5:0/3341564651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d63b3640 1 -- 172.21.15.5:0/3341564651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 msgr2=0x55f3a8ecc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d63b3640 1 --2- 172.21.15.5:0/3341564651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ecc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d63b3640 1 -- 172.21.15.5:0/3341564651 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a8db13a0 con 0x55f3a8ec0b40 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/3341564651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.402+0000 7f88d53b1640 1 --2- 172.21.15.5:0/3341564651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ecc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.403+0000 7f88d63b3640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88bc0a3430 0x7f88bc0a58f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009900 tx=0x7f88c801b000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.403+0000 7f88d63b3640 1 --2- 172.21.15.5:0/3341564651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ebf090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88c800bf80 tx=0x7f88c8008750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee51f204d28c4b0a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.403+0000 7f88bbfff640 1 -- 172.21.15.5:0/3341564651 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8027070 con 0x55f3a8ec0b40 2024-09-15T22:11:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.403+0000 7f88bbfff640 1 -- 172.21.15.5:0/3341564651 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c8009d90 con 0x55f3a8ec0b40 2024-09-15T22:11:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.403+0000 7f88bbfff640 1 -- 172.21.15.5:0/3341564651 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c801e070 con 0x55f3a8ec0b40 2024-09-15T22:11:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.404+0000 7f88da1ef740 1 -- 172.21.15.5:0/3341564651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 msgr2=0x55f3a8ebf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.404+0000 7f88da1ef740 1 --2- 172.21.15.5:0/3341564651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ebf090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f88c800bf80 tx=0x7f88c8008750 comp rx=0 tx=0).stop 2024-09-15T22:11:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.404+0000 7f88da1ef740 1 -- 172.21.15.5:0/3341564651 shutdown_connections 2024-09-15T22:11:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.404+0000 7f88da1ef740 1 --2- 172.21.15.5:0/3341564651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ecc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.404+0000 7f88da1ef740 1 --2- 172.21.15.5:0/3341564651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.404+0000 7f88da1ef740 1 --2- 172.21.15.5:0/3341564651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ebf090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.404+0000 7f88da1ef740 1 -- 172.21.15.5:0/3341564651 >> 172.21.15.5:0/3341564651 conn(0x55f3a8e88310 msgr2=0x55f3a8ebbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.404+0000 7f88da1ef740 1 -- 172.21.15.5:0/3341564651 shutdown_connections 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.404+0000 7f88da1ef740 1 -- 172.21.15.5:0/3341564651 wait complete. 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.404+0000 7f88da1ef740 1 -- start start 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.404+0000 7f88da1ef740 1 Processor -- start 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ec9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ec7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8eca370 0x55f3a8ec7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88da1ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f3a8ecd010 con 0x55f3a8ec0b40 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88da1ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f3a8de0060 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88da1ef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f3a8ecd190 con 0x55f3a8eca370 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88da1ef740 10 client.0 _ll_register_callbacks cb 0x55f3a8b338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88d53b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ec7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88d53b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ec7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56754/0 (socket says 172.21.15.5:56754) 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88d53b1640 1 -- 172.21.15.5:0/3194538254 learned_addr learned my addr 172.21.15.5:0/3194538254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ec9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88d63b3640 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8eca370 0x55f3a8ec7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88d53b1640 1 -- 172.21.15.5:0/3194538254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8eca370 msgr2=0x55f3a8ec7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88d53b1640 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8eca370 0x55f3a8ec7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88d53b1640 1 -- 172.21.15.5:0/3194538254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 msgr2=0x55f3a8ec9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88d53b1640 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ec9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.405+0000 7f88d53b1640 1 -- 172.21.15.5:0/3194538254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a8b963c0 con 0x55f3a8ec0b40 2024-09-15T22:11:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.406+0000 7f88d53b1640 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ec7020 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f88c0007590 tx=0x7f88c009bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a896991fcbe61bb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.406+0000 7f88b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.406+0000 7f88b97fa640 1 -- 172.21.15.5:0/3194538254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c00b4030 con 0x55f3a8ec0b40 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.406+0000 7f88b97fa640 1 -- 172.21.15.5:0/3194538254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f88c0098680 con 0x55f3a8ec0b40 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.406+0000 7f88da1ef740 1 -- 172.21.15.5:0/3194538254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88c8009d90 con 0x55f3a8ec0b40 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.406+0000 7f88b97fa640 1 -- 172.21.15.5:0/3194538254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88c007f050 con 0x55f3a8ec0b40 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.406+0000 7f88da1ef740 10 client.15831 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.406+0000 7f88da1ef740 1 -- 172.21.15.5:0/3194538254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f3a8ed8b10 con 0x55f3a8ec0b40 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.406+0000 7f88b97fa640 1 -- 172.21.15.5:0/3194538254 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c8009d90 con 0x55f3a8ec0b40 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.406+0000 7f88b97fa640 1 -- 172.21.15.5:0/3194538254 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f88c009c270 con 0x55f3a8ec0b40 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.407+0000 7f88da1ef740 10 client.15831 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.407+0000 7f88da1ef740 10 client.15831 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.407+0000 7f88da1ef740 20 client.15831 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.407+0000 7f88da1ef740 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55f3a8ed9fd0 0x55f3a8edc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.407+0000 7f88da1ef740 4 client.15831 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.407+0000 7f88da1ef740 1 -- 172.21.15.5:0/3194538254 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3a8e32860 con 0x55f3a8ed9fd0 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.407+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55f3a8ed9fd0 0x55f3a8edc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.407+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55f3a8ed9fd0 0x55f3a8edc3e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7b421bda7597915 server_cookie=cb89fa3170ed0814 in_seq=0 out_seq=0 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.407+0000 7f88b97fa640 10 client.15831 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88b97fa640 1 -- 172.21.15.5:0/3194538254 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3a8e32860 con 0x55f3a8ed9fd0 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88b97fa640 10 client.15831 handle_command_reply: tid=0 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88da1ef740 1 client.15831 shutdown 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88da1ef740 1 -- 172.21.15.5:0/3194538254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ec0b40 msgr2=0x55f3a8ec7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88da1ef740 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ec7020 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f88c0007590 tx=0x7f88c009bb50 comp rx=0 tx=0).stop 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88da1ef740 1 -- 172.21.15.5:0/3194538254 shutdown_connections 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88da1ef740 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55f3a8ed9fd0 0x55f3a8edc3e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88da1ef740 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8eca370 0x55f3a8ec7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88da1ef740 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ec7020 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88da1ef740 1 --2- 172.21.15.5:0/3194538254 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ec9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88da1ef740 1 -- 172.21.15.5:0/3194538254 >> 172.21.15.5:0/3194538254 conn(0x55f3a8e88310 msgr2=0x55f3a8ec1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88b97fa640 0 client.15831 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:11:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88da1ef740 1 -- 172.21.15.5:0/3194538254 shutdown_connections 2024-09-15T22:11:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.408+0000 7f88da1ef740 1 -- 172.21.15.5:0/3194538254 wait complete. 2024-09-15T22:11:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.409+0000 7f88da1ef740 20 client.15831 trim_cache size 0 max 16384 2024-09-15T22:11:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88da1ef740 1 Processor -- start 2024-09-15T22:11:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88da1ef740 1 -- start start 2024-09-15T22:11:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ed62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8eca370 0x55f3a8ec2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88da1ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01b7e70 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88da1ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01b5740 con 0x55f3a8eca370 2024-09-15T22:11:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88da1ef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01b3590 con 0x55f3a8ec0b40 2024-09-15T22:11:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88d5bb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88d5bb2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56762/0 (socket says 172.21.15.5:56762) 2024-09-15T22:11:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88d5bb2640 1 -- 172.21.15.5:0/834923686 learned_addr learned my addr 172.21.15.5:0/834923686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88d53b1640 1 --2- 172.21.15.5:0/834923686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ed62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.422+0000 7f88d63b3640 1 --2- 172.21.15.5:0/834923686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8eca370 0x55f3a8ec2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.423+0000 7f88d5bb2640 1 -- 172.21.15.5:0/834923686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 msgr2=0x55f3a8ed62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.423+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/834923686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ed62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.423+0000 7f88d5bb2640 1 -- 172.21.15.5:0/834923686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8eca370 msgr2=0x55f3a8ec2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.423+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/834923686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8eca370 0x55f3a8ec2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.423+0000 7f88d5bb2640 1 -- 172.21.15.5:0/834923686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3a8ed67f0 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.423+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/834923686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebbe10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f88cc031900 tx=0x7f88cc0092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca3fb7e911bd077 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.423+0000 7f88bb7fe640 1 -- 172.21.15.5:0/834923686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc035040 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.423+0000 7f88bb7fe640 1 -- 172.21.15.5:0/834923686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88cc005500 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.423+0000 7f88bb7fe640 1 -- 172.21.15.5:0/834923686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc004dd0 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.424+0000 7f88da1ef740 1 -- 172.21.15.5:0/834923686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ebf5d0 msgr2=0x55f3a8ebbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.424+0000 7f88da1ef740 1 --2- 172.21.15.5:0/834923686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebbe10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f88cc031900 tx=0x7f88cc0092e0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.424+0000 7f88da1ef740 1 -- 172.21.15.5:0/834923686 shutdown_connections 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.424+0000 7f88da1ef740 1 --2- 172.21.15.5:0/834923686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8eca370 0x55f3a8ec2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.424+0000 7f88da1ef740 1 --2- 172.21.15.5:0/834923686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ed62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.424+0000 7f88da1ef740 1 --2- 172.21.15.5:0/834923686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ebbe10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.424+0000 7f88da1ef740 1 -- 172.21.15.5:0/834923686 >> 172.21.15.5:0/834923686 conn(0x55f3a8e88310 msgr2=0x55f3a8ebb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.424+0000 7f88da1ef740 1 -- 172.21.15.5:0/834923686 shutdown_connections 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.424+0000 7f88da1ef740 1 -- 172.21.15.5:0/834923686 wait complete. 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.424+0000 7f88da1ef740 1 -- start start 2024-09-15T22:11:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.424+0000 7f88da1ef740 1 Processor -- start 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ed2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ecdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88da1ef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ece3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88da1ef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01b3c00 con 0x55f3a8eca370 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88da1ef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01b5070 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88da1ef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01b42d0 con 0x55f3a8ec0b40 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88da1ef740 10 client.0 _ll_register_callbacks cb 0x7f88c009bde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88d63b3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ece3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88d63b3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ece3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56766/0 (socket says 172.21.15.5:56766) 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88d5bb2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ed2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88d63b3640 1 -- 172.21.15.5:0/2699567570 learned_addr learned my addr 172.21.15.5:0/2699567570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88d53b1640 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ecdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88d5bb2640 1 -- 172.21.15.5:0/2699567570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 msgr2=0x55f3a8ecdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ecdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88d5bb2640 1 -- 172.21.15.5:0/2699567570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 msgr2=0x55f3a8ece3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ece3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88d5bb2640 1 -- 172.21.15.5:0/2699567570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3a8ecebc0 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.425+0000 7f88d63b3640 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ece3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.426+0000 7f88d5bb2640 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ed2b70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88cc031900 tx=0x7f88cc02a470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b82e4e326f693c36 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.426+0000 7f88b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.426+0000 7f88b97fa640 1 -- 172.21.15.5:0/2699567570 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88cc054070 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.426+0000 7f88da1ef740 1 -- 172.21.15.5:0/2699567570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88cc005500 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.426+0000 7f88b97fa640 1 -- 172.21.15.5:0/2699567570 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f88cc004ad0 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.426+0000 7f88b97fa640 1 -- 172.21.15.5:0/2699567570 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88cc062050 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.426+0000 7f88da1ef740 10 client.25448 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.426+0000 7f88da1ef740 1 -- 172.21.15.5:0/2699567570 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f3a8f44c90 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.426+0000 7f88b97fa640 1 -- 172.21.15.5:0/2699567570 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88cc005500 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.426+0000 7f88b97fa640 1 -- 172.21.15.5:0/2699567570 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f88cc003a30 con 0x55f3a8ebf5d0 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.427+0000 7f88da1ef740 10 client.25448 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.427+0000 7f88da1ef740 10 client.25448 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.427+0000 7f88da1ef740 20 client.25448 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.427+0000 7f88da1ef740 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55f3a8f45360 0x55f3a8f47750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.427+0000 7f88da1ef740 4 client.25448 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.427+0000 7f88da1ef740 1 -- 172.21.15.5:0/2699567570 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3a8e2cd70 con 0x55f3a8f45360 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.427+0000 7f88d53b1640 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55f3a8f45360 0x55f3a8f47750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.427+0000 7f88b97fa640 10 client.25448 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.427+0000 7f88d53b1640 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55f3a8f45360 0x55f3a8f47750 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1bf93cf0a06fe10 server_cookie=22d2c3f9eb35f9a3 in_seq=0 out_seq=0 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.427+0000 7f88b97fa640 1 -- 172.21.15.5:0/2699567570 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3a8e2cd70 con 0x55f3a8f45360 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.427+0000 7f88b97fa640 10 client.25448 handle_command_reply: tid=0 2024-09-15T22:11:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 1 client.25448 shutdown 2024-09-15T22:11:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 1 -- 172.21.15.5:0/2699567570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 msgr2=0x55f3a8ed2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ed2b70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88cc031900 tx=0x7f88cc02a470 comp rx=0 tx=0).stop 2024-09-15T22:11:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 1 -- 172.21.15.5:0/2699567570 shutdown_connections 2024-09-15T22:11:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55f3a8f45360 0x55f3a8f47750 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3a8eca370 0x55f3a8ece3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f3a8ec0b40 0x55f3a8ecdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 1 --2- 172.21.15.5:0/2699567570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f3a8ebf5d0 0x55f3a8ed2b70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88b97fa640 0 client.25448 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:11:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 1 -- 172.21.15.5:0/2699567570 >> 172.21.15.5:0/2699567570 conn(0x55f3a8e88310 msgr2=0x55f3a8ec90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 1 -- 172.21.15.5:0/2699567570 shutdown_connections 2024-09-15T22:11:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 1 -- 172.21.15.5:0/2699567570 wait complete. 2024-09-15T22:11:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88da1ef740 20 client.25448 trim_cache size 0 max 16384 2024-09-15T22:11:27.435 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:11:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88d7e16640 1 -- 172.21.15.5:0/172438920 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88bc0a3430 msgr2=0x7f88bc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88d7e16640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88bc0a3430 0x7f88bc0a58f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f88c8009900 tx=0x7f88c801b000 comp rx=0 tx=0).stop 2024-09-15T22:11:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88d7e16640 1 -- 172.21.15.5:0/172438920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88d0101450 msgr2=0x7f88d01a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.428+0000 7f88d7e16640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88d0101450 0x7f88d01a4c00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c970 tx=0x7f88c000ce30 comp rx=0 tx=0).stop 2024-09-15T22:11:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.429+0000 7f88d7e16640 1 -- 172.21.15.5:0/172438920 shutdown_connections 2024-09-15T22:11:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.429+0000 7f88d7e16640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88bc0a3430 0x7f88bc0a58f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.429+0000 7f88d7e16640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88d0105f40 0x7f88d019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.429+0000 7f88d7e16640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88d0101450 0x7f88d01a4c00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.429+0000 7f88d7e16640 1 --2- 172.21.15.5:0/172438920 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88d0100b30 0x7f88d01a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.429+0000 7f88d7e16640 1 -- 172.21.15.5:0/172438920 >> 172.21.15.5:0/172438920 conn(0x7f88d00fc7a0 msgr2=0x7f88d01038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.429+0000 7f88d7e16640 1 -- 172.21.15.5:0/172438920 shutdown_connections 2024-09-15T22:11:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:27.429+0000 7f88d7e16640 1 -- 172.21.15.5:0/172438920 wait complete. 2024-09-15T22:11:27.440 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:27 smithi005 ceph-mon[30944]: pgmap v79: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.1 MiB/s wr, 384 op/s 2024-09-15T22:11:27.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:27 smithi099 ceph-mon[34892]: pgmap v79: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.1 MiB/s wr, 384 op/s 2024-09-15T22:11:27.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:27 smithi129 ceph-mon[35314]: pgmap v79: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 5.1 MiB/s wr, 384 op/s 2024-09-15T22:11:28.235 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:11:28.500 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:28.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:11:28.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.961+0000 7fdbab0b0640 1 -- 172.21.15.5:0/4089964316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdba4103650 msgr2=0x7fdba410bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:28.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.961+0000 7fdbab0b0640 1 --2- 172.21.15.5:0/4089964316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdba4103650 0x7fdba410bb40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009900 tx=0x7fdb9802a480 comp rx=0 tx=0).stop 2024-09-15T22:11:28.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.961+0000 7fdbab0b0640 1 -- 172.21.15.5:0/4089964316 shutdown_connections 2024-09-15T22:11:28.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.961+0000 7fdbab0b0640 1 --2- 172.21.15.5:0/4089964316 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdba410c160 0x7fdba410e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.961+0000 7fdbab0b0640 1 --2- 172.21.15.5:0/4089964316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdba4103650 0x7fdba410bb40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.961+0000 7fdbab0b0640 1 --2- 172.21.15.5:0/4089964316 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4102d30 0x7fdba4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.961+0000 7fdbab0b0640 1 -- 172.21.15.5:0/4089964316 >> 172.21.15.5:0/4089964316 conn(0x7fdba40fc790 msgr2=0x7fdba40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.961+0000 7fdbab0b0640 1 -- 172.21.15.5:0/4089964316 shutdown_connections 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.961+0000 7fdbab0b0640 1 -- 172.21.15.5:0/4089964316 wait complete. 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdbab0b0640 1 Processor -- start 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdbab0b0640 1 -- start start 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdbab0b0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4102d30 0x7fdba4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdbab0b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdba4103650 0x7fdba4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdbab0b0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdba410c160 0x7fdba4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdbab0b0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdba4111f60 con 0x7fdba4103650 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdbab0b0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdba4111de0 con 0x7fdba410c160 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdbab0b0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41120e0 con 0x7fdba4102d30 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdba3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdba4103650 0x7fdba4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdba3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdba4103650 0x7fdba4100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56782/0 (socket says 172.21.15.5:56782) 2024-09-15T22:11:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdba3fff640 1 -- 172.21.15.5:0/1481988833 learned_addr learned my addr 172.21.15.5:0/1481988833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.962+0000 7fdba8e4c640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4102d30 0x7fdba4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdba964d640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdba410c160 0x7fdba4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdba3fff640 1 -- 172.21.15.5:0/1481988833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4102d30 msgr2=0x7fdba4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdba3fff640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4102d30 0x7fdba4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdba3fff640 1 -- 172.21.15.5:0/1481988833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdba410c160 msgr2=0x7fdba4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdba3fff640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdba410c160 0x7fdba4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdba3fff640 1 -- 172.21.15.5:0/1481988833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba41a68e0 con 0x7fdba4103650 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdba964d640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdba410c160 0x7fdba4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdba3fff640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdba4103650 0x7fdba4100ed0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdb98006010 tx=0x7fdb980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdba1ffb640 1 -- 172.21.15.5:0/1481988833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9803f030 con 0x7fdba4103650 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdba1ffb640 1 -- 172.21.15.5:0/1481988833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb98003ce0 con 0x7fdba4103650 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdba1ffb640 1 -- 172.21.15.5:0/1481988833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98035070 con 0x7fdba4103650 2024-09-15T22:11:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.963+0000 7fdbab0b0640 1 -- 172.21.15.5:0/1481988833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba41a6b70 con 0x7fdba4103650 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.964+0000 7fdbab0b0640 1 -- 172.21.15.5:0/1481988833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba41a7000 con 0x7fdba4103650 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.967+0000 7fdba1ffb640 1 -- 172.21.15.5:0/1481988833 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fdb98042030 con 0x7fdba4103650 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.967+0000 7fdbad489740 1 Processor -- start 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.967+0000 7fdbad489740 1 -- start start 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.967+0000 7fdbad489740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4e96b40 0x55faa4e95090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.967+0000 7fdbad489740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4e95a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.967+0000 7fdbad489740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4ea0370 0x55faa4ea2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.967+0000 7fdbad489740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55faa4dfb2f0 con 0x55faa4e955d0 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.967+0000 7fdbad489740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55faa4da2cf0 con 0x55faa4ea0370 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.967+0000 7fdbad489740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55faa4df72b0 con 0x55faa4e96b40 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.967+0000 7fdba1ffb640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdb8c0a33b0 0x7fdb8c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba1ffb640 1 -- 172.21.15.5:0/1481988833 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fdb980f0700 con 0x7fdba4103650 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba8e4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4e95a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4ea0370 0x55faa4ea2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba964d640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdb8c0a33b0 0x7fdb8c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba8e4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4e95a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56794/0 (socket says 172.21.15.5:56794) 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4ea0370 0x55faa4ea2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:37862/0 (socket says 172.21.15.5:37862) 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba8e4c640 1 -- 172.21.15.5:0/448533072 learned_addr learned my addr 172.21.15.5:0/448533072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba964d640 1 --2- 172.21.15.5:0/448533072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4e96b40 0x55faa4e95090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba3fff640 1 -- 172.21.15.5:0/448533072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4e96b40 msgr2=0x55faa4e95090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba3fff640 1 --2- 172.21.15.5:0/448533072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4e96b40 0x55faa4e95090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba3fff640 1 -- 172.21.15.5:0/448533072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 msgr2=0x55faa4e95a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba3fff640 1 --2- 172.21.15.5:0/448533072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4e95a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba3fff640 1 -- 172.21.15.5:0/448533072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa4d873a0 con 0x55faa4ea0370 2024-09-15T22:11:28.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba964d640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdb8c0a33b0 0x7fdb8c0a5870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c007310 tx=0x7fdb9c007d10 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdba3fff640 1 --2- 172.21.15.5:0/448533072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4ea0370 0x55faa4ea2760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb980eda50 tx=0x7fdb9802a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44cc751a030774af server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.968+0000 7fdb8affd640 1 -- 172.21.15.5:0/448533072 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98049050 con 0x55faa4ea0370 2024-09-15T22:11:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.969+0000 7fdb8affd640 1 -- 172.21.15.5:0/448533072 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb980bcda0 con 0x55faa4ea0370 2024-09-15T22:11:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.969+0000 7fdb8affd640 1 -- 172.21.15.5:0/448533072 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb980c5310 con 0x55faa4ea0370 2024-09-15T22:11:28.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.969+0000 7fdbad489740 1 -- 172.21.15.5:0/448533072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4ea0370 msgr2=0x55faa4ea2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:28.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.969+0000 7fdbad489740 1 --2- 172.21.15.5:0/448533072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4ea0370 0x55faa4ea2760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb980eda50 tx=0x7fdb9802a480 comp rx=0 tx=0).stop 2024-09-15T22:11:28.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.969+0000 7fdbad489740 1 -- 172.21.15.5:0/448533072 shutdown_connections 2024-09-15T22:11:28.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.969+0000 7fdbad489740 1 --2- 172.21.15.5:0/448533072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4ea0370 0x55faa4ea2760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.969+0000 7fdbad489740 1 --2- 172.21.15.5:0/448533072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4e95a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.969+0000 7fdbad489740 1 --2- 172.21.15.5:0/448533072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4e96b40 0x55faa4e95090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.969+0000 7fdbad489740 1 -- 172.21.15.5:0/448533072 >> 172.21.15.5:0/448533072 conn(0x55faa4e5e310 msgr2=0x55faa4e91c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.969+0000 7fdbad489740 1 -- 172.21.15.5:0/448533072 shutdown_connections 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.969+0000 7fdbad489740 1 -- 172.21.15.5:0/448533072 wait complete. 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.970+0000 7fdbad489740 1 -- start start 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.970+0000 7fdbad489740 1 Processor -- start 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.970+0000 7fdbad489740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e955d0 0x55faa4e9d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.970+0000 7fdbad489740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4e96b40 0x55faa4e9d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.970+0000 7fdbad489740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4ea0370 0x55faa4e9db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.970+0000 7fdbad489740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55faa4ea2e30 con 0x55faa4ea0370 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.970+0000 7fdbad489740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55faa4db6060 con 0x55faa4e955d0 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.970+0000 7fdbad489740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55faa4ea2fb0 con 0x55faa4e96b40 2024-09-15T22:11:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdbad489740 10 client.0 _ll_register_callbacks cb 0x55faa4b098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba964d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4ea0370 0x55faa4e9db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba8e4c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e955d0 0x55faa4e9d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba964d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4ea0370 0x55faa4e9db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56796/0 (socket says 172.21.15.5:56796) 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba964d640 1 -- 172.21.15.5:0/1271093685 learned_addr learned my addr 172.21.15.5:0/1271093685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba3fff640 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4e96b40 0x55faa4e9d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba8e4c640 1 -- 172.21.15.5:0/1271093685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4e96b40 msgr2=0x55faa4e9d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba8e4c640 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4e96b40 0x55faa4e9d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba8e4c640 1 -- 172.21.15.5:0/1271093685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4ea0370 msgr2=0x55faa4e9db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba8e4c640 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4ea0370 0x55faa4e9db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba8e4c640 1 -- 172.21.15.5:0/1271093685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa4b6c3c0 con 0x55faa4e955d0 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba964d640 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4ea0370 0x55faa4e9db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdba8e4c640 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e955d0 0x55faa4e9d0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400bac0 tx=0x7fdb9400bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60d14e7497a7f6bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.971+0000 7fdb83fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.972+0000 7fdb83fff640 1 -- 172.21.15.5:0/1271093685 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94003d60 con 0x55faa4e955d0 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.972+0000 7fdbad489740 1 -- 172.21.15.5:0/1271093685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb980bcda0 con 0x55faa4e955d0 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.972+0000 7fdb83fff640 1 -- 172.21.15.5:0/1271093685 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fdb94011440 con 0x55faa4e955d0 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.972+0000 7fdb83fff640 1 -- 172.21.15.5:0/1271093685 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb94011640 con 0x55faa4e955d0 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.972+0000 7fdbad489740 10 client.25457 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.972+0000 7fdbad489740 1 -- 172.21.15.5:0/1271093685 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55faa4eae960 con 0x55faa4e955d0 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.972+0000 7fdb83fff640 1 -- 172.21.15.5:0/1271093685 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb980bcda0 con 0x55faa4e955d0 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.972+0000 7fdb83fff640 1 -- 172.21.15.5:0/1271093685 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdb9401b9f0 con 0x55faa4e955d0 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.973+0000 7fdbad489740 10 client.25457 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.973+0000 7fdbad489740 10 client.25457 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.973+0000 7fdbad489740 20 client.25457 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.973+0000 7fdbad489740 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55faa4eafd70 0x55faa4eb21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.973+0000 7fdbad489740 4 client.25457 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.973+0000 7fdbad489740 1 -- 172.21.15.5:0/1271093685 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa4e08860 con 0x55faa4eafd70 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.973+0000 7fdba3fff640 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55faa4eafd70 0x55faa4eb21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.973+0000 7fdba3fff640 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55faa4eafd70 0x55faa4eb21f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=792dbea7efcad91b server_cookie=ef5b877506c9ce7a in_seq=0 out_seq=0 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.973+0000 7fdb83fff640 10 client.25457 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdb83fff640 1 -- 172.21.15.5:0/1271093685 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55faa4e08860 con 0x55faa4eafd70 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdb83fff640 10 client.25457 handle_command_reply: tid=0 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdbad489740 1 client.25457 shutdown 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdbad489740 1 -- 172.21.15.5:0/1271093685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e955d0 msgr2=0x55faa4e9d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdbad489740 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e955d0 0x55faa4e9d0f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb9400bac0 tx=0x7fdb9400bec0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdbad489740 1 -- 172.21.15.5:0/1271093685 shutdown_connections 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdbad489740 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55faa4eafd70 0x55faa4eb21f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdbad489740 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4ea0370 0x55faa4e9db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdbad489740 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4e96b40 0x55faa4e9d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdbad489740 1 --2- 172.21.15.5:0/1271093685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e955d0 0x55faa4e9d0f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdbad489740 1 -- 172.21.15.5:0/1271093685 >> 172.21.15.5:0/1271093685 conn(0x55faa4e5e310 msgr2=0x55faa4e979c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdb83fff640 0 client.25457 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:11:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdbad489740 1 -- 172.21.15.5:0/1271093685 shutdown_connections 2024-09-15T22:11:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.974+0000 7fdbad489740 1 -- 172.21.15.5:0/1271093685 wait complete. 2024-09-15T22:11:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.975+0000 7fdbad489740 20 client.25457 trim_cache size 0 max 16384 2024-09-15T22:11:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.987+0000 7fdbad489740 1 Processor -- start 2024-09-15T22:11:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdbad489740 1 -- start start 2024-09-15T22:11:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdbad489740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4e9b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdbad489740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 0x55faa4e98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdbad489740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4ea0370 0x55faa4e98660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdbad489740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41b3aa0 con 0x55faa4e955d0 2024-09-15T22:11:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdbad489740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41b1370 con 0x55faa4e96b40 2024-09-15T22:11:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdbad489740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41af1c0 con 0x55faa4ea0370 2024-09-15T22:11:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdba8e4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4e9b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdba8e4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4e9b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56808/0 (socket says 172.21.15.5:56808) 2024-09-15T22:11:28.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdba8e4c640 1 -- 172.21.15.5:0/3375045197 learned_addr learned my addr 172.21.15.5:0/3375045197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdba3fff640 1 --2- 172.21.15.5:0/3375045197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 0x55faa4e98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdba964d640 1 --2- 172.21.15.5:0/3375045197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4ea0370 0x55faa4e98660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdba8e4c640 1 -- 172.21.15.5:0/3375045197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4ea0370 msgr2=0x55faa4e98660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdba8e4c640 1 --2- 172.21.15.5:0/3375045197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4ea0370 0x55faa4e98660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdba8e4c640 1 -- 172.21.15.5:0/3375045197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 msgr2=0x55faa4e98120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdba8e4c640 1 --2- 172.21.15.5:0/3375045197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 0x55faa4e98120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdba8e4c640 1 -- 172.21.15.5:0/3375045197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa4ea2e80 con 0x55faa4e955d0 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.988+0000 7fdba3fff640 1 --2- 172.21.15.5:0/3375045197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 0x55faa4e98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.989+0000 7fdba8e4c640 1 --2- 172.21.15.5:0/3375045197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4e9b9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdb9401be60 tx=0x7fdb940115d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90539b6900b406f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.989+0000 7fdb8a7fc640 1 -- 172.21.15.5:0/3375045197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94003e20 con 0x55faa4e955d0 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.989+0000 7fdb8a7fc640 1 -- 172.21.15.5:0/3375045197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9402e070 con 0x55faa4e955d0 2024-09-15T22:11:28.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.989+0000 7fdb8a7fc640 1 -- 172.21.15.5:0/3375045197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb9401b420 con 0x55faa4e955d0 2024-09-15T22:11:28.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.989+0000 7fdbad489740 1 -- 172.21.15.5:0/3375045197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 msgr2=0x55faa4e9b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:28.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.990+0000 7fdbad489740 1 --2- 172.21.15.5:0/3375045197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4e9b9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdb9401be60 tx=0x7fdb940115d0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.990+0000 7fdbad489740 1 -- 172.21.15.5:0/3375045197 shutdown_connections 2024-09-15T22:11:28.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.990+0000 7fdbad489740 1 --2- 172.21.15.5:0/3375045197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4ea0370 0x55faa4e98660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.990+0000 7fdbad489740 1 --2- 172.21.15.5:0/3375045197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 0x55faa4e98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.990+0000 7fdbad489740 1 --2- 172.21.15.5:0/3375045197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4e9b9c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.990+0000 7fdbad489740 1 -- 172.21.15.5:0/3375045197 >> 172.21.15.5:0/3375045197 conn(0x55faa4e5e310 msgr2=0x55faa4e90d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.990+0000 7fdbad489740 1 -- 172.21.15.5:0/3375045197 shutdown_connections 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.990+0000 7fdbad489740 1 -- 172.21.15.5:0/3375045197 wait complete. 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.990+0000 7fdbad489740 1 -- start start 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.990+0000 7fdbad489740 1 Processor -- start 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdbad489740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4ea8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdbad489740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 0x55faa4ea3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdbad489740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4ea0370 0x55faa4ea3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdbad489740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41af830 con 0x55faa4e955d0 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdbad489740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41b0ca0 con 0x55faa4e96b40 2024-09-15T22:11:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdbad489740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdba41aff00 con 0x55faa4ea0370 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdbad489740 10 client.0 _ll_register_callbacks cb 0x7fdb94004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdba8e4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4ea8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdba8e4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4ea8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56822/0 (socket says 172.21.15.5:56822) 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdba8e4c640 1 -- 172.21.15.5:0/360572298 learned_addr learned my addr 172.21.15.5:0/360572298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdba3fff640 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 0x55faa4ea3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdba964d640 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4ea0370 0x55faa4ea3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdba3fff640 1 -- 172.21.15.5:0/360572298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4ea0370 msgr2=0x55faa4ea3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdba3fff640 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4ea0370 0x55faa4ea3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdba3fff640 1 -- 172.21.15.5:0/360572298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 msgr2=0x55faa4ea8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdba3fff640 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4ea8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.991+0000 7fdba3fff640 1 -- 172.21.15.5:0/360572298 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa4ea4470 con 0x55faa4e96b40 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdba3fff640 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 0x55faa4ea3970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdb98005d00 tx=0x7fdb98005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe588bc6d58e87 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdb83fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:11:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdb83fff640 1 -- 172.21.15.5:0/360572298 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98075070 con 0x55faa4e96b40 2024-09-15T22:11:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdb83fff640 1 -- 172.21.15.5:0/360572298 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fdb980b8d10 con 0x55faa4e96b40 2024-09-15T22:11:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdbad489740 1 -- 172.21.15.5:0/360572298 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9402e070 con 0x55faa4e96b40 2024-09-15T22:11:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdb83fff640 1 -- 172.21.15.5:0/360572298 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb98080050 con 0x55faa4e96b40 2024-09-15T22:11:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdbad489740 10 client.25463 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdbad489740 1 -- 172.21.15.5:0/360572298 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55faa4f1a850 con 0x55faa4e96b40 2024-09-15T22:11:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdb83fff640 1 -- 172.21.15.5:0/360572298 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb9402e070 con 0x55faa4e96b40 2024-09-15T22:11:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdb83fff640 1 -- 172.21.15.5:0/360572298 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdb980f02a0 con 0x55faa4e96b40 2024-09-15T22:11:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdbad489740 10 client.25463 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdbad489740 10 client.25463 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:11:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdbad489740 20 client.25463 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdbad489740 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55faa4f1aca0 0x55faa4f1d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdbad489740 4 client.25463 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.992+0000 7fdbad489740 1 -- 172.21.15.5:0/360572298 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55faa4e02d70 con 0x55faa4f1aca0 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdba8e4c640 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55faa4f1aca0 0x55faa4f1d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdb83fff640 10 client.25463 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdba8e4c640 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55faa4f1aca0 0x55faa4f1d090 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5d0d6582c10aae5 server_cookie=a109c2830b5df53e in_seq=0 out_seq=0 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdb83fff640 1 -- 172.21.15.5:0/360572298 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55faa4e02d70 con 0x55faa4f1aca0 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdb83fff640 10 client.25463 handle_command_reply: tid=0 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdbad489740 1 client.25463 shutdown 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdbad489740 1 -- 172.21.15.5:0/360572298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 msgr2=0x55faa4ea3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdbad489740 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 0x55faa4ea3970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdb98005d00 tx=0x7fdb98005c90 comp rx=0 tx=0).stop 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdbad489740 1 -- 172.21.15.5:0/360572298 shutdown_connections 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdbad489740 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55faa4f1aca0 0x55faa4f1d090 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:29.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdbad489740 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55faa4ea0370 0x55faa4ea3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdb83fff640 0 client.25463 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:11:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdbad489740 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55faa4e96b40 0x55faa4ea3970 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdbad489740 1 --2- 172.21.15.5:0/360572298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55faa4e955d0 0x55faa4ea8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.993+0000 7fdbad489740 1 -- 172.21.15.5:0/360572298 >> 172.21.15.5:0/360572298 conn(0x55faa4e5e310 msgr2=0x55faa4eaaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbad489740 1 -- 172.21.15.5:0/360572298 shutdown_connections 2024-09-15T22:11:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbad489740 1 -- 172.21.15.5:0/360572298 wait complete. 2024-09-15T22:11:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbad489740 20 client.25463 trim_cache size 0 max 16384 2024-09-15T22:11:29.002 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:11:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbab0b0640 1 -- 172.21.15.5:0/1481988833 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdb8c0a33b0 msgr2=0x7fdb8c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:29.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbab0b0640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdb8c0a33b0 0x7fdb8c0a5870 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c007310 tx=0x7fdb9c007d10 comp rx=0 tx=0).stop 2024-09-15T22:11:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbab0b0640 1 -- 172.21.15.5:0/1481988833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdba4103650 msgr2=0x7fdba4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbab0b0640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdba4103650 0x7fdba4100ed0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdb98006010 tx=0x7fdb980042f0 comp rx=0 tx=0).stop 2024-09-15T22:11:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbab0b0640 1 -- 172.21.15.5:0/1481988833 shutdown_connections 2024-09-15T22:11:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbab0b0640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdb8c0a33b0 0x7fdb8c0a5870 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbab0b0640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdba410c160 0x7fdba4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbab0b0640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdba4103650 0x7fdba4100ed0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbab0b0640 1 --2- 172.21.15.5:0/1481988833 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdba4102d30 0x7fdba4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.994+0000 7fdbab0b0640 1 -- 172.21.15.5:0/1481988833 >> 172.21.15.5:0/1481988833 conn(0x7fdba40fc790 msgr2=0x7fdba40fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.995+0000 7fdbab0b0640 1 -- 172.21.15.5:0/1481988833 shutdown_connections 2024-09-15T22:11:29.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:28.995+0000 7fdbab0b0640 1 -- 172.21.15.5:0/1481988833 wait complete. 2024-09-15T22:11:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:29 smithi099 ceph-mon[34892]: pgmap v80: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 33 op/s 2024-09-15T22:11:29.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:29 smithi129 ceph-mon[35314]: pgmap v80: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 33 op/s 2024-09-15T22:11:29.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:29 smithi005 ceph-mon[30944]: pgmap v80: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 33 op/s 2024-09-15T22:11:29.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 120.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:11:29.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:11:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.522+0000 7f2494456640 1 -- 172.21.15.5:0/4089540018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f248c106f10 msgr2=0x7f248c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.522+0000 7f2494456640 1 --2- 172.21.15.5:0/4089540018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f248c106f10 0x7f248c10ea90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2484009ca0 tx=0x7f248402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.522+0000 7f2494456640 1 -- 172.21.15.5:0/4089540018 shutdown_connections 2024-09-15T22:11:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.522+0000 7f2494456640 1 --2- 172.21.15.5:0/4089540018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f248c106f10 0x7f248c10ea90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.522+0000 7f2494456640 1 --2- 172.21.15.5:0/4089540018 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f248c102790 0x7f248c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.522+0000 7f2494456640 1 --2- 172.21.15.5:0/4089540018 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f248c101de0 0x7f248c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.522+0000 7f2494456640 1 -- 172.21.15.5:0/4089540018 >> 172.21.15.5:0/4089540018 conn(0x7f248c0fd8c0 msgr2=0x7f248c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:30.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.522+0000 7f2494456640 1 -- 172.21.15.5:0/4089540018 shutdown_connections 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.522+0000 7f2494456640 1 -- 172.21.15.5:0/4089540018 wait complete. 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f2494456640 1 Processor -- start 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f2494456640 1 -- start start 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f2494456640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f248c101de0 0x7f248c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f2494456640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f248c102790 0x7f248c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f2494456640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f248c106f10 0x7f248c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f2494456640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f248c111f70 con 0x7f248c106f10 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f2494456640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f248c111df0 con 0x7f248c102790 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f2494456640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1120f0 con 0x7f248c101de0 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f24929f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f248c106f10 0x7f248c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f24929f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f248c106f10 0x7f248c19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56838/0 (socket says 172.21.15.5:56838) 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f24929f3640 1 -- 172.21.15.5:0/1805361221 learned_addr learned my addr 172.21.15.5:0/1805361221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:30.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f24921f2640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f248c101de0 0x7f248c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f24919f1640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f248c102790 0x7f248c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f24929f3640 1 -- 172.21.15.5:0/1805361221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f248c101de0 msgr2=0x7f248c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f24929f3640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f248c101de0 0x7f248c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f24929f3640 1 -- 172.21.15.5:0/1805361221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f248c102790 msgr2=0x7f248c19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f24929f3640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f248c102790 0x7f248c19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.523+0000 7f24929f3640 1 -- 172.21.15.5:0/1805361221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c19f580 con 0x7f248c106f10 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.524+0000 7f24929f3640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f248c106f10 0x7f248c19eea0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2484031a20 tx=0x7f2484031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.524+0000 7f24837fe640 1 -- 172.21.15.5:0/1805361221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484003c30 con 0x7f248c106f10 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.524+0000 7f24837fe640 1 -- 172.21.15.5:0/1805361221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2484003dd0 con 0x7f248c106f10 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.524+0000 7f2494456640 1 -- 172.21.15.5:0/1805361221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c19f810 con 0x7f248c106f10 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.524+0000 7f24837fe640 1 -- 172.21.15.5:0/1805361221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248402fcf0 con 0x7f248c106f10 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.524+0000 7f2494456640 1 -- 172.21.15.5:0/1805361221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c1a7150 con 0x7f248c106f10 2024-09-15T22:11:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.525+0000 7f2494456640 1 -- 172.21.15.5:0/1805361221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f248c1021c0 con 0x7f248c106f10 2024-09-15T22:11:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.526+0000 7f24837fe640 1 -- 172.21.15.5:0/1805361221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2484004050 con 0x7f248c106f10 2024-09-15T22:11:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.527+0000 7f24837fe640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24780a33b0 0x7f24780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.527+0000 7f24921f2640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24780a33b0 0x7f24780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.527+0000 7f24837fe640 1 -- 172.21.15.5:0/1805361221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f24840f1540 con 0x7f248c106f10 2024-09-15T22:11:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.527+0000 7f24921f2640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24780a33b0 0x7f24780a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2488005d00 tx=0x7f248800b410 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:30.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.530+0000 7f24837fe640 1 -- 172.21.15.5:0/1805361221 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24840bdcc0 con 0x7f248c106f10 2024-09-15T22:11:30.599 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.746+0000 7f2494456640 1 -- 172.21.15.5:0/1805361221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f248c1a73b0 con 0x7f248c106f10 2024-09-15T22:11:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.746+0000 7f24837fe640 1 -- 172.21.15.5:0/1805361221 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f248402fe90 con 0x7f248c106f10 2024-09-15T22:11:30.748 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:11:30.749 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:11:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:11:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.748+0000 7f2494456640 1 -- 172.21.15.5:0/1805361221 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24780a33b0 msgr2=0x7f24780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.748+0000 7f2494456640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24780a33b0 0x7f24780a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2488005d00 tx=0x7f248800b410 comp rx=0 tx=0).stop 2024-09-15T22:11:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.749+0000 7f2494456640 1 -- 172.21.15.5:0/1805361221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f248c106f10 msgr2=0x7f248c19eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.749+0000 7f2494456640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f248c106f10 0x7f248c19eea0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2484031a20 tx=0x7f2484031a50 comp rx=0 tx=0).stop 2024-09-15T22:11:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.749+0000 7f2494456640 1 -- 172.21.15.5:0/1805361221 shutdown_connections 2024-09-15T22:11:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.749+0000 7f2494456640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24780a33b0 0x7f24780a5870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.749+0000 7f2494456640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f248c106f10 0x7f248c19eea0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.749+0000 7f2494456640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f248c102790 0x7f248c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.749+0000 7f2494456640 1 --2- 172.21.15.5:0/1805361221 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f248c101de0 0x7f248c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.749+0000 7f2494456640 1 -- 172.21.15.5:0/1805361221 >> 172.21.15.5:0/1805361221 conn(0x7f248c0fd8c0 msgr2=0x7f248c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.749+0000 7f2494456640 1 -- 172.21.15.5:0/1805361221 shutdown_connections 2024-09-15T22:11:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:30.749+0000 7f2494456640 1 -- 172.21.15.5:0/1805361221 wait complete. 2024-09-15T22:11:31.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:31 smithi099 ceph-mon[34892]: pgmap v81: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.6 MiB/s wr, 34 op/s 2024-09-15T22:11:31.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:31 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1805361221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:31.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:31 smithi129 ceph-mon[35314]: pgmap v81: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.6 MiB/s wr, 34 op/s 2024-09-15T22:11:31.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:31 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1805361221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:31.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:31 smithi005 ceph-mon[30944]: pgmap v81: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 1.6 MiB/s wr, 34 op/s 2024-09-15T22:11:31.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:31 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1805361221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:31.646 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id dd7f13bb --release --await' 2024-09-15T22:11:31.647 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id dd7f13bb --release --await 2024-09-15T22:11:31.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.797+0000 7fe11b8c2640 1 -- 172.21.15.5:0/1091433610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe114100e20 msgr2=0x7fe114101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.797+0000 7fe11b8c2640 1 --2- 172.21.15.5:0/1091433610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe114100e20 0x7fe114101200 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe110009900 tx=0x7fe11002a480 comp rx=0 tx=0).stop 2024-09-15T22:11:31.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.797+0000 7fe11b8c2640 1 -- 172.21.15.5:0/1091433610 shutdown_connections 2024-09-15T22:11:31.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.797+0000 7fe11b8c2640 1 --2- 172.21.15.5:0/1091433610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe114102ef0 0x7fe11410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.797+0000 7fe11b8c2640 1 --2- 172.21.15.5:0/1091433610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe114102570 0x7fe1141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.797+0000 7fe11b8c2640 1 --2- 172.21.15.5:0/1091433610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe114100e20 0x7fe114101200 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.797+0000 7fe11b8c2640 1 -- 172.21.15.5:0/1091433610 >> 172.21.15.5:0/1091433610 conn(0x7fe1140fc7d0 msgr2=0x7fe1140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:31.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.797+0000 7fe11b8c2640 1 -- 172.21.15.5:0/1091433610 shutdown_connections 2024-09-15T22:11:31.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.797+0000 7fe11b8c2640 1 -- 172.21.15.5:0/1091433610 wait complete. 2024-09-15T22:11:31.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe11b8c2640 1 Processor -- start 2024-09-15T22:11:31.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe11b8c2640 1 -- start start 2024-09-15T22:11:31.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe11b8c2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe114100e20 0x7fe114198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe11b8c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe114102570 0x7fe114198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe11b8c2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe114102ef0 0x7fe11419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe11b8c2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe114110f80 con 0x7fe114102570 2024-09-15T22:11:31.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe11b8c2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe114110e00 con 0x7fe114100e20 2024-09-15T22:11:31.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe11b8c2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe114111100 con 0x7fe114102ef0 2024-09-15T22:11:31.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe118e5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe114102570 0x7fe114198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe118e5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe114102570 0x7fe114198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56858/0 (socket says 172.21.15.5:56858) 2024-09-15T22:11:31.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe118e5d640 1 -- 172.21.15.5:0/439071093 learned_addr learned my addr 172.21.15.5:0/439071093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:31.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.798+0000 7fe11965e640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe114100e20 0x7fe114198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe119e5f640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe114102ef0 0x7fe11419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe118e5d640 1 -- 172.21.15.5:0/439071093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe114102ef0 msgr2=0x7fe11419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe118e5d640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe114102ef0 0x7fe11419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe118e5d640 1 -- 172.21.15.5:0/439071093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe114100e20 msgr2=0x7fe114198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe118e5d640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe114100e20 0x7fe114198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe118e5d640 1 -- 172.21.15.5:0/439071093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe11419d4b0 con 0x7fe114102570 2024-09-15T22:11:31.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe119e5f640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe114102ef0 0x7fe11419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:31.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe118e5d640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe114102570 0x7fe114198a40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe10400d950 tx=0x7fe10400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:31.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe10a7fc640 1 -- 172.21.15.5:0/439071093 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104017030 con 0x7fe114102570 2024-09-15T22:11:31.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe10a7fc640 1 -- 172.21.15.5:0/439071093 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe104004020 con 0x7fe114102570 2024-09-15T22:11:31.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe11b8c2640 1 -- 172.21.15.5:0/439071093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe11419d7a0 con 0x7fe114102570 2024-09-15T22:11:31.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe10a7fc640 1 -- 172.21.15.5:0/439071093 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104003d60 con 0x7fe114102570 2024-09-15T22:11:31.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.799+0000 7fe11b8c2640 1 -- 172.21.15.5:0/439071093 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1141a5080 con 0x7fe114102570 2024-09-15T22:11:31.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.802+0000 7fe10a7fc640 1 -- 172.21.15.5:0/439071093 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe10401a070 con 0x7fe114102570 2024-09-15T22:11:31.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.802+0000 7fe11dc9b740 1 Processor -- start 2024-09-15T22:11:31.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.802+0000 7fe11dc9b740 1 -- start start 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.802+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c1b40 0x55a1b06c0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.802+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c05d0 0x55a1b06c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.802+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06d1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.802+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a1b06262f0 con 0x55a1b06cf780 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a1b05cdcf0 con 0x55a1b06c1b40 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a1b06222b0 con 0x55a1b06c05d0 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe118e5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06d1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe118e5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06d1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56862/0 (socket says 172.21.15.5:56862) 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe118e5d640 1 -- 172.21.15.5:0/864815155 learned_addr learned my addr 172.21.15.5:0/864815155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe119e5f640 1 --2- 172.21.15.5:0/864815155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c1b40 0x55a1b06c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe11965e640 1 --2- 172.21.15.5:0/864815155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c05d0 0x55a1b06c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe10a7fc640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe1000a3430 0x7fe1000a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe119e5f640 1 -- 172.21.15.5:0/864815155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c05d0 msgr2=0x55a1b06c0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe119e5f640 1 --2- 172.21.15.5:0/864815155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c05d0 0x55a1b06c0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe119e5f640 1 -- 172.21.15.5:0/864815155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 msgr2=0x55a1b06d1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe119e5f640 1 --2- 172.21.15.5:0/864815155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06d1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe119e5f640 1 -- 172.21.15.5:0/864815155 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b05b23a0 con 0x55a1b06c1b40 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe10a7fc640 1 -- 172.21.15.5:0/439071093 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe1040cfc40 con 0x7fe114102570 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe119e5f640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe1000a3430 0x7fe1000a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.803+0000 7fe119e5f640 1 --2- 172.21.15.5:0/864815155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c1b40 0x55a1b06c0090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe10c005bf0 tx=0x7fe10c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e53da60e4fcee753 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.804+0000 7fe0ff7fe640 1 -- 172.21.15.5:0/864815155 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c013e50 con 0x55a1b06c1b40 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.804+0000 7fe119e5f640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe1000a3430 0x7fe1000a58f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe10c021000 tx=0x7fe10c01e000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.804+0000 7fe0ff7fe640 1 -- 172.21.15.5:0/864815155 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c01ddf0 con 0x55a1b06c1b40 2024-09-15T22:11:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.804+0000 7fe0ff7fe640 1 -- 172.21.15.5:0/864815155 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c01ce50 con 0x55a1b06c1b40 2024-09-15T22:11:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.805+0000 7fe11dc9b740 1 -- 172.21.15.5:0/864815155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c1b40 msgr2=0x55a1b06c0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.805+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/864815155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c1b40 0x55a1b06c0090 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe10c005bf0 tx=0x7fe10c010800 comp rx=0 tx=0).stop 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.805+0000 7fe11dc9b740 1 -- 172.21.15.5:0/864815155 shutdown_connections 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.805+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/864815155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06d1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.805+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/864815155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c05d0 0x55a1b06c0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.805+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/864815155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c1b40 0x55a1b06c0090 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.805+0000 7fe11dc9b740 1 -- 172.21.15.5:0/864815155 >> 172.21.15.5:0/864815155 conn(0x55a1b0689310 msgr2=0x55a1b06bcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.805+0000 7fe11dc9b740 1 -- 172.21.15.5:0/864815155 shutdown_connections 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.805+0000 7fe11dc9b740 1 -- 172.21.15.5:0/864815155 wait complete. 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.805+0000 7fe11dc9b740 1 -- start start 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.805+0000 7fe11dc9b740 1 Processor -- start 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c1b40 0x55a1b06cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a1b06d2480 con 0x55a1b06cf780 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a1b05e1060 con 0x55a1b06c05d0 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a1b06d2600 con 0x55a1b06c1b40 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11dc9b740 10 client.0 _ll_register_callbacks cb 0x55a1b03348c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11965e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe119e5f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11965e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06cf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:37938/0 (socket says 172.21.15.5:37938) 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11965e640 1 -- 172.21.15.5:0/1215704754 learned_addr learned my addr 172.21.15.5:0/1215704754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe118e5d640 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c1b40 0x55a1b06cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11965e640 1 -- 172.21.15.5:0/1215704754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c1b40 msgr2=0x55a1b06cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11965e640 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c1b40 0x55a1b06cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11965e640 1 -- 172.21.15.5:0/1215704754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 msgr2=0x55a1b06ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11965e640 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe11965e640 1 -- 172.21.15.5:0/1215704754 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b03973c0 con 0x55a1b06c05d0 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.806+0000 7fe119e5f640 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe11965e640 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06cf1b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d75d93612397631 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe0fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe0fcff9640 1 -- 172.21.15.5:0/1215704754 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110035040 con 0x55a1b06c05d0 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe11dc9b740 1 -- 172.21.15.5:0/1215704754 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10c01ddf0 con 0x55a1b06c05d0 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe0fcff9640 1 -- 172.21.15.5:0/1215704754 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe110003a00 con 0x55a1b06c05d0 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe0fcff9640 1 -- 172.21.15.5:0/1215704754 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe11004e050 con 0x55a1b06c05d0 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe11dc9b740 10 client.25481 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe11dc9b740 1 -- 172.21.15.5:0/1215704754 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1b06ddfb0 con 0x55a1b06c05d0 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe0fcff9640 1 -- 172.21.15.5:0/1215704754 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10c01ddf0 con 0x55a1b06c05d0 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe0fcff9640 1 -- 172.21.15.5:0/1215704754 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe11003c440 con 0x55a1b06c05d0 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe11dc9b740 10 client.25481 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.807+0000 7fe11dc9b740 10 client.25481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.808+0000 7fe11dc9b740 20 client.25481 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.808+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a1b06df3c0 0x55a1b06e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.808+0000 7fe11dc9b740 4 client.25481 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.808+0000 7fe11dc9b740 1 -- 172.21.15.5:0/1215704754 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1b0633860 con 0x55a1b06df3c0 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.808+0000 7fe118e5d640 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a1b06df3c0 0x55a1b06e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.808+0000 7fe0fcff9640 10 client.25481 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.808+0000 7fe118e5d640 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a1b06df3c0 0x55a1b06e17f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d143f074c1b2c7f7 server_cookie=da2f4dc1c67a305f in_seq=0 out_seq=0 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe0fcff9640 1 -- 172.21.15.5:0/1215704754 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1b0633860 con 0x55a1b06df3c0 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe0fcff9640 10 client.25481 handle_command_reply: tid=0 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 1 client.25481 shutdown 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 1 -- 172.21.15.5:0/1215704754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 msgr2=0x55a1b06cf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06cf1b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe110004440 comp rx=0 tx=0).stop 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 1 -- 172.21.15.5:0/1215704754 shutdown_connections 2024-09-15T22:11:31.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c1b40 0x55a1b06cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a1b06df3c0 0x55a1b06e17f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/1215704754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06cf1b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe0fcff9640 0 client.25481 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 1 -- 172.21.15.5:0/1215704754 >> 172.21.15.5:0/1215704754 conn(0x55a1b0689310 msgr2=0x55a1b06c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 1 -- 172.21.15.5:0/1215704754 shutdown_connections 2024-09-15T22:11:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 1 -- 172.21.15.5:0/1215704754 wait complete. 2024-09-15T22:11:31.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.809+0000 7fe11dc9b740 20 client.25481 trim_cache size 0 max 16384 2024-09-15T22:11:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.822+0000 7fe11dc9b740 1 Processor -- start 2024-09-15T22:11:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.822+0000 7fe11dc9b740 1 -- start start 2024-09-15T22:11:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06e7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06c1b40 0x55a1b06e7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06cf780 0x55a1b06c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141af7a0 con 0x55a1b06c1b40 2024-09-15T22:11:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141ad070 con 0x55a1b06c05d0 2024-09-15T22:11:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141aaec0 con 0x55a1b06cf780 2024-09-15T22:11:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe118e5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06c1b40 0x55a1b06e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe118e5d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06c1b40 0x55a1b06e7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56880/0 (socket says 172.21.15.5:56880) 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe118e5d640 1 -- 172.21.15.5:0/13150724 learned_addr learned my addr 172.21.15.5:0/13150724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe119e5f640 1 --2- 172.21.15.5:0/13150724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06cf780 0x55a1b06c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe11965e640 1 --2- 172.21.15.5:0/13150724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe118e5d640 1 -- 172.21.15.5:0/13150724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06cf780 msgr2=0x55a1b06c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe118e5d640 1 --2- 172.21.15.5:0/13150724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06cf780 0x55a1b06c96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe118e5d640 1 -- 172.21.15.5:0/13150724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 msgr2=0x55a1b06e7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe118e5d640 1 --2- 172.21.15.5:0/13150724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06e7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe118e5d640 1 -- 172.21.15.5:0/13150724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1b06bd960 con 0x55a1b06c1b40 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe11965e640 1 --2- 172.21.15.5:0/13150724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.823+0000 7fe118e5d640 1 --2- 172.21.15.5:0/13150724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06c1b40 0x55a1b06e7650 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe10409bb10 tx=0x7fe1040d3790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0efc7de78b2d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.824+0000 7fe0feffd640 1 -- 172.21.15.5:0/13150724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10409b4e0 con 0x55a1b06c1b40 2024-09-15T22:11:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.824+0000 7fe0feffd640 1 -- 172.21.15.5:0/13150724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10409c280 con 0x55a1b06c1b40 2024-09-15T22:11:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.824+0000 7fe0feffd640 1 -- 172.21.15.5:0/13150724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1040981f0 con 0x55a1b06c1b40 2024-09-15T22:11:31.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.825+0000 7fe11dc9b740 1 -- 172.21.15.5:0/13150724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06c1b40 msgr2=0x55a1b06e7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.825+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/13150724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06c1b40 0x55a1b06e7650 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe10409bb10 tx=0x7fe1040d3790 comp rx=0 tx=0).stop 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.825+0000 7fe11dc9b740 1 -- 172.21.15.5:0/13150724 shutdown_connections 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.825+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/13150724 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06cf780 0x55a1b06c96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.825+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/13150724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06c1b40 0x55a1b06e7650 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.825+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/13150724 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06e7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.825+0000 7fe11dc9b740 1 -- 172.21.15.5:0/13150724 >> 172.21.15.5:0/13150724 conn(0x55a1b0689310 msgr2=0x55a1b06c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.825+0000 7fe11dc9b740 1 -- 172.21.15.5:0/13150724 shutdown_connections 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.825+0000 7fe11dc9b740 1 -- 172.21.15.5:0/13150724 wait complete. 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.825+0000 7fe11dc9b740 1 -- start start 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.825+0000 7fe11dc9b740 1 Processor -- start 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c1b40 0x55a1b06d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe11dc9b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06d4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141ab530 con 0x55a1b06cf780 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141ac9a0 con 0x55a1b06c05d0 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe11dc9b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe1141abc00 con 0x55a1b06c1b40 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe11dc9b740 10 client.0 _ll_register_callbacks cb 0x7fe110009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe119e5f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06d4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe119e5f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06d4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56890/0 (socket says 172.21.15.5:56890) 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe119e5f640 1 -- 172.21.15.5:0/3149592825 learned_addr learned my addr 172.21.15.5:0/3149592825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe11965e640 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.826+0000 7fe118e5d640 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c1b40 0x55a1b06d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe119e5f640 1 -- 172.21.15.5:0/3149592825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c1b40 msgr2=0x55a1b06d3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe119e5f640 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c1b40 0x55a1b06d3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe119e5f640 1 -- 172.21.15.5:0/3149592825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 msgr2=0x55a1b06d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe119e5f640 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe119e5f640 1 -- 172.21.15.5:0/3149592825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1b07491e0 con 0x55a1b06cf780 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe11965e640 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe119e5f640 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06d4160 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe10c01c6c0 tx=0x7fe10c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddb7edbf35ea677 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe0fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe0fcff9640 1 -- 172.21.15.5:0/3149592825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe10c039060 con 0x55a1b06cf780 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe0fcff9640 1 -- 172.21.15.5:0/3149592825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe10c009810 con 0x55a1b06cf780 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe11dc9b740 1 -- 172.21.15.5:0/3149592825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe10409c280 con 0x55a1b06cf780 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe0fcff9640 1 -- 172.21.15.5:0/3149592825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe10c040050 con 0x55a1b06cf780 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe11dc9b740 10 client.15879 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.827+0000 7fe11dc9b740 1 -- 172.21.15.5:0/3149592825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1b074a430 con 0x55a1b06cf780 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.828+0000 7fe0fcff9640 1 -- 172.21.15.5:0/3149592825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe10409c280 con 0x55a1b06cf780 2024-09-15T22:11:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.828+0000 7fe0fcff9640 1 -- 172.21.15.5:0/3149592825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe10c00f040 con 0x55a1b06cf780 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.828+0000 7fe11dc9b740 10 client.15879 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.828+0000 7fe11dc9b740 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.828+0000 7fe11dc9b740 20 client.15879 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.828+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a1b074aa10 0x55a1b074ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.828+0000 7fe11dc9b740 4 client.15879 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.828+0000 7fe11dc9b740 1 -- 172.21.15.5:0/3149592825 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1b062dd70 con 0x55a1b074aa10 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.828+0000 7fe11965e640 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a1b074aa10 0x55a1b074ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.828+0000 7fe11965e640 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a1b074aa10 0x55a1b074ce30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cddabdfcf076c5c7 server_cookie=d8783241947979f7 in_seq=0 out_seq=0 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.828+0000 7fe0fcff9640 10 client.15879 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe0fcff9640 1 -- 172.21.15.5:0/3149592825 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1b062dd70 con 0x55a1b074aa10 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe0fcff9640 10 client.15879 handle_command_reply: tid=0 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 1 client.15879 shutdown 2024-09-15T22:11:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 1 -- 172.21.15.5:0/3149592825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 msgr2=0x55a1b06d4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06d4160 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe10c01c6c0 tx=0x7fe10c01c9b0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 1 -- 172.21.15.5:0/3149592825 shutdown_connections 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a1b06cf780 0x55a1b06d4160 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a1b06c1b40 0x55a1b06d3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a1b074aa10 0x55a1b074ce30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 1 --2- 172.21.15.5:0/3149592825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a1b06c05d0 0x55a1b06d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe0fcff9640 0 client.15879 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 1 -- 172.21.15.5:0/3149592825 >> 172.21.15.5:0/3149592825 conn(0x55a1b0689310 msgr2=0x55a1b06c2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 1 -- 172.21.15.5:0/3149592825 shutdown_connections 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 1 -- 172.21.15.5:0/3149592825 wait complete. 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.829+0000 7fe11dc9b740 20 client.15879 trim_cache size 0 max 16384 2024-09-15T22:11:31.835 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:11:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 -- 172.21.15.5:0/439071093 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe1000a3430 msgr2=0x7fe1000a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe1000a3430 0x7fe1000a58f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe10c021000 tx=0x7fe10c01e000 comp rx=0 tx=0).stop 2024-09-15T22:11:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 -- 172.21.15.5:0/439071093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe114102570 msgr2=0x7fe114198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe114102570 0x7fe114198a40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe10400d950 tx=0x7fe10400de10 comp rx=0 tx=0).stop 2024-09-15T22:11:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 -- 172.21.15.5:0/439071093 shutdown_connections 2024-09-15T22:11:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe1000a3430 0x7fe1000a58f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe114102ef0 0x7fe11419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe114102570 0x7fe114198a40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 --2- 172.21.15.5:0/439071093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe114100e20 0x7fe114198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 -- 172.21.15.5:0/439071093 >> 172.21.15.5:0/439071093 conn(0x7fe1140fc7d0 msgr2=0x7fe11406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 -- 172.21.15.5:0/439071093 shutdown_connections 2024-09-15T22:11:31.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:31.830+0000 7fe11b8c2640 1 -- 172.21.15.5:0/439071093 wait complete. 2024-09-15T22:11:32.687 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.184+0000 7faad489c640 1 -- 172.21.15.5:0/3154444487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad0102ef0 msgr2=0x7faad010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.184+0000 7faad489c640 1 --2- 172.21.15.5:0/3154444487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad0102ef0 0x7faad010da40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7faac8009ca0 tx=0x7faac802a950 comp rx=0 tx=0).stop 2024-09-15T22:11:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.184+0000 7faad489c640 1 -- 172.21.15.5:0/3154444487 shutdown_connections 2024-09-15T22:11:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.184+0000 7faad489c640 1 --2- 172.21.15.5:0/3154444487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad0102ef0 0x7faad010da40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.184+0000 7faad489c640 1 --2- 172.21.15.5:0/3154444487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faad0102570 0x7faad01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.184+0000 7faad489c640 1 --2- 172.21.15.5:0/3154444487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faad0100e20 0x7faad0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.184+0000 7faad489c640 1 -- 172.21.15.5:0/3154444487 >> 172.21.15.5:0/3154444487 conn(0x7faad00fc7d0 msgr2=0x7faad00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.184+0000 7faad489c640 1 -- 172.21.15.5:0/3154444487 shutdown_connections 2024-09-15T22:11:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.184+0000 7faad489c640 1 -- 172.21.15.5:0/3154444487 wait complete. 2024-09-15T22:11:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.184+0000 7faad489c640 1 Processor -- start 2024-09-15T22:11:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faad489c640 1 -- start start 2024-09-15T22:11:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faad489c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faad0100e20 0x7faad0198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faad489c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faad0102570 0x7faad0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faad489c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad0102ef0 0x7faad019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faad489c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faad01120a0 con 0x7faad0102ef0 2024-09-15T22:11:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faad489c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faad0111f20 con 0x7faad0102570 2024-09-15T22:11:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faad489c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faad0112220 con 0x7faad0100e20 2024-09-15T22:11:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faaced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad0102ef0 0x7faad019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faaced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad0102ef0 0x7faad019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56900/0 (socket says 172.21.15.5:56900) 2024-09-15T22:11:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faaced9d640 1 -- 172.21.15.5:0/2251846498 learned_addr learned my addr 172.21.15.5:0/2251846498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faac7fff640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faad0102570 0x7faad0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faace59c640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faad0100e20 0x7faad0198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faaced9d640 1 -- 172.21.15.5:0/2251846498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faad0100e20 msgr2=0x7faad0198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faaced9d640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faad0100e20 0x7faad0198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faaced9d640 1 -- 172.21.15.5:0/2251846498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faad0102570 msgr2=0x7faad0198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faaced9d640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faad0102570 0x7faad0198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faaced9d640 1 -- 172.21.15.5:0/2251846498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad019d3f0 con 0x7faad0102ef0 2024-09-15T22:11:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.185+0000 7faace59c640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faad0100e20 0x7faad0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.186+0000 7faaced9d640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad0102ef0 0x7faad019cd10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7faac802ae70 tx=0x7faac802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.186+0000 7faac77fe640 1 -- 172.21.15.5:0/2251846498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8034040 con 0x7faad0102ef0 2024-09-15T22:11:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.186+0000 7faac77fe640 1 -- 172.21.15.5:0/2251846498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac8030920 con 0x7faad0102ef0 2024-09-15T22:11:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.186+0000 7faac77fe640 1 -- 172.21.15.5:0/2251846498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8045480 con 0x7faad0102ef0 2024-09-15T22:11:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.186+0000 7faad489c640 1 -- 172.21.15.5:0/2251846498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad019d680 con 0x7faad0102ef0 2024-09-15T22:11:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.186+0000 7faad489c640 1 -- 172.21.15.5:0/2251846498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad01a4fc0 con 0x7faad0102ef0 2024-09-15T22:11:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.188+0000 7faac77fe640 1 -- 172.21.15.5:0/2251846498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7faac8037020 con 0x7faad0102ef0 2024-09-15T22:11:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.189+0000 7faad6c75740 1 Processor -- start 2024-09-15T22:11:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.189+0000 7faad6c75740 1 -- start start 2024-09-15T22:11:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.189+0000 7faad6c75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f177f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.189+0000 7faad6c75740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f178880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.189+0000 7faad6c75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f18b8e0 0x560b2f18dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.189+0000 7faad6c75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560b2f0dd620 con 0x560b2f1795e0 2024-09-15T22:11:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.189+0000 7faad6c75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560b2f084e50 con 0x560b2f18b8e0 2024-09-15T22:11:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.189+0000 7faad6c75740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560b2f0d95e0 con 0x560b2f178440 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faac77fe640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faabc0a33b0 0x7faabc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faac7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f177f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faac7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f177f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56908/0 (socket says 172.21.15.5:56908) 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faac7fff640 1 -- 172.21.15.5:0/2128996856 learned_addr learned my addr 172.21.15.5:0/2128996856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faac7fff640 1 --2- 172.21.15.5:0/2128996856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f18b8e0 0x560b2f18dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faace59c640 1 --2- 172.21.15.5:0/2128996856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f178880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faaced9d640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faabc0a33b0 0x7faabc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faac7fff640 1 -- 172.21.15.5:0/2128996856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 msgr2=0x560b2f178880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faac7fff640 1 --2- 172.21.15.5:0/2128996856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f178880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faac7fff640 1 -- 172.21.15.5:0/2128996856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f18b8e0 msgr2=0x560b2f18dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faac7fff640 1 --2- 172.21.15.5:0/2128996856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f18b8e0 0x560b2f18dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faac7fff640 1 -- 172.21.15.5:0/2128996856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2f075d40 con 0x560b2f1795e0 2024-09-15T22:11:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.190+0000 7faace59c640 1 --2- 172.21.15.5:0/2128996856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f178880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.191+0000 7faac7fff640 1 --2- 172.21.15.5:0/2128996856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f177f00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7faab800b6c0 tx=0x7faab800b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580cddec89498cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.191+0000 7faac77fe640 1 -- 172.21.15.5:0/2251846498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7faac80f0e90 con 0x7faad0102ef0 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.191+0000 7faaced9d640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faabc0a33b0 0x7faabc0a5870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faac80bcbe0 tx=0x7faac80b2370 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.194+0000 7faac4ff9640 1 -- 172.21.15.5:0/2128996856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8012da0 con 0x560b2f1795e0 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.194+0000 7faac4ff9640 1 -- 172.21.15.5:0/2128996856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab8010770 con 0x560b2f1795e0 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.194+0000 7faac4ff9640 1 -- 172.21.15.5:0/2128996856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8003810 con 0x560b2f1795e0 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.194+0000 7faad6c75740 1 -- 172.21.15.5:0/2128996856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 msgr2=0x560b2f177f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.194+0000 7faad6c75740 1 --2- 172.21.15.5:0/2128996856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f177f00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7faab800b6c0 tx=0x7faab800b650 comp rx=0 tx=0).stop 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.194+0000 7faad6c75740 1 -- 172.21.15.5:0/2128996856 shutdown_connections 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.194+0000 7faad6c75740 1 --2- 172.21.15.5:0/2128996856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f18b8e0 0x560b2f18dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.194+0000 7faad6c75740 1 --2- 172.21.15.5:0/2128996856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f178880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.194+0000 7faad6c75740 1 --2- 172.21.15.5:0/2128996856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f177f00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.194+0000 7faad6c75740 1 -- 172.21.15.5:0/2128996856 >> 172.21.15.5:0/2128996856 conn(0x560b2f1745f0 msgr2=0x560b2f1749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.195+0000 7faad6c75740 1 -- 172.21.15.5:0/2128996856 shutdown_connections 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.195+0000 7faad6c75740 1 -- 172.21.15.5:0/2128996856 wait complete. 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.195+0000 7faad6c75740 1 -- start start 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.195+0000 7faad6c75740 1 Processor -- start 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.195+0000 7faad6c75740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f17bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.195+0000 7faad6c75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f1795e0 0x560b2f17c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.195+0000 7faad6c75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f18b8e0 0x560b2f17c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.195+0000 7faad6c75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560b2f18e5b0 con 0x560b2f18b8e0 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.195+0000 7faad6c75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560b2f065f10 con 0x560b2f1795e0 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.195+0000 7faad6c75740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560b2f18e730 con 0x560b2f178440 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faad6c75740 10 client.0 _ll_register_callbacks cb 0x560b2f089db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faac7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f17bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faace59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f1795e0 0x560b2f17c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faac7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f17bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:35220/0 (socket says 172.21.15.5:35220) 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faac7fff640 1 -- 172.21.15.5:0/39587190 learned_addr learned my addr 172.21.15.5:0/39587190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faac7fff640 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f18b8e0 0x560b2f17c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faac7fff640 1 -- 172.21.15.5:0/39587190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f1795e0 msgr2=0x560b2f17c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faac7fff640 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f1795e0 0x560b2f17c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faac7fff640 1 -- 172.21.15.5:0/39587190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f18b8e0 msgr2=0x560b2f17c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faac7fff640 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f18b8e0 0x560b2f17c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faac7fff640 1 -- 172.21.15.5:0/39587190 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2edec580 con 0x560b2f178440 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faac7fff640 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f17bcf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faab8010940 tx=0x7faab801b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec504cf0e6995533 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.196+0000 7faab5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.197+0000 7faab5ffb640 1 -- 172.21.15.5:0/39587190 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8004020 con 0x560b2f178440 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.197+0000 7faab5ffb640 1 -- 172.21.15.5:0/39587190 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7faab80041c0 con 0x560b2f178440 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.197+0000 7faad6c75740 1 -- 172.21.15.5:0/39587190 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab8010770 con 0x560b2f178440 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.197+0000 7faab5ffb640 1 -- 172.21.15.5:0/39587190 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faab80043c0 con 0x560b2f178440 2024-09-15T22:11:33.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.197+0000 7faad6c75740 10 client.25486 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:33.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.197+0000 7faad6c75740 1 -- 172.21.15.5:0/39587190 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b2eeb6960 con 0x560b2f178440 2024-09-15T22:11:33.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.197+0000 7faab5ffb640 1 -- 172.21.15.5:0/39587190 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab8010770 con 0x560b2f178440 2024-09-15T22:11:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.197+0000 7faab5ffb640 1 -- 172.21.15.5:0/39587190 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7faab8004860 con 0x560b2f178440 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.198+0000 7faad6c75740 10 client.25486 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.198+0000 7faad6c75740 10 client.25486 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.198+0000 7faad6c75740 20 client.25486 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.198+0000 7faad6c75740 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x560b2f19b4d0 0x560b2f19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.198+0000 7faad6c75740 4 client.25486 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.198+0000 7faad6c75740 1 -- 172.21.15.5:0/39587190 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b2f0eab90 con 0x560b2f19b4d0 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.198+0000 7faace59c640 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x560b2f19b4d0 0x560b2f19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.198+0000 7faace59c640 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x560b2f19b4d0 0x560b2f19d8c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d079705167979264 server_cookie=43a0419e8de85f48 in_seq=0 out_seq=0 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.198+0000 7faab5ffb640 10 client.25486 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.199+0000 7faab5ffb640 1 -- 172.21.15.5:0/39587190 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b2f0eab90 con 0x560b2f19b4d0 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.199+0000 7faab5ffb640 10 client.25486 handle_command_reply: tid=0 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.199+0000 7faad6c75740 1 client.25486 shutdown 2024-09-15T22:11:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.199+0000 7faad6c75740 1 -- 172.21.15.5:0/39587190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 msgr2=0x560b2f17bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.200+0000 7faad6c75740 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f17bcf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faab8010940 tx=0x7faab801b040 comp rx=0 tx=0).stop 2024-09-15T22:11:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.200+0000 7faad6c75740 1 -- 172.21.15.5:0/39587190 shutdown_connections 2024-09-15T22:11:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.200+0000 7faad6c75740 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f18b8e0 0x560b2f17c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.200+0000 7faad6c75740 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x560b2f19b4d0 0x560b2f19d8c0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.200+0000 7faad6c75740 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f1795e0 0x560b2f17c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.200+0000 7faad6c75740 1 --2- 172.21.15.5:0/39587190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f17bcf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.200+0000 7faad6c75740 1 -- 172.21.15.5:0/39587190 >> 172.21.15.5:0/39587190 conn(0x560b2f1745f0 msgr2=0x560b2f17a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.200+0000 7faab5ffb640 0 client.25486 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.200+0000 7faad6c75740 1 -- 172.21.15.5:0/39587190 shutdown_connections 2024-09-15T22:11:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.200+0000 7faad6c75740 1 -- 172.21.15.5:0/39587190 wait complete. 2024-09-15T22:11:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.200+0000 7faad6c75740 20 client.25486 trim_cache size 0 max 16384 2024-09-15T22:11:33.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.218+0000 7faad6c75740 1 Processor -- start 2024-09-15T22:11:33.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.218+0000 7faad6c75740 1 -- start start 2024-09-15T22:11:33.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.218+0000 7faad6c75740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f173640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.218+0000 7faad6c75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f1a3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.218+0000 7faad6c75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f18b8e0 0x560b2f1a35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.218+0000 7faad6c75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faad01af5f0 con 0x560b2f1795e0 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.218+0000 7faad6c75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faad01acec0 con 0x560b2f18b8e0 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.218+0000 7faad6c75740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faad01aad10 con 0x560b2f178440 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.218+0000 7faace59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f1a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.218+0000 7faace59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f1a3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56922/0 (socket says 172.21.15.5:56922) 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.218+0000 7faace59c640 1 -- 172.21.15.5:0/1881778691 learned_addr learned my addr 172.21.15.5:0/1881778691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faac7fff640 1 --2- 172.21.15.5:0/1881778691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f18b8e0 0x560b2f1a35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faac7fff640 1 --2- 172.21.15.5:0/1881778691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f173640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faace59c640 1 -- 172.21.15.5:0/1881778691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 msgr2=0x560b2f173640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faace59c640 1 --2- 172.21.15.5:0/1881778691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f173640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faace59c640 1 -- 172.21.15.5:0/1881778691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f18b8e0 msgr2=0x560b2f1a35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faace59c640 1 --2- 172.21.15.5:0/1881778691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f18b8e0 0x560b2f1a35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faace59c640 1 -- 172.21.15.5:0/1881778691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b2f178e50 con 0x560b2f1795e0 2024-09-15T22:11:33.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faac7fff640 1 --2- 172.21.15.5:0/1881778691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f173640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:33.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faace59c640 1 --2- 172.21.15.5:0/1881778691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f1a3080 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faac0005fd0 tx=0x7faac00094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e36272a51b3689c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:33.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faab7fff640 1 -- 172.21.15.5:0/1881778691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac001f070 con 0x560b2f1795e0 2024-09-15T22:11:33.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faab7fff640 1 -- 172.21.15.5:0/1881778691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac0004220 con 0x560b2f1795e0 2024-09-15T22:11:33.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.219+0000 7faab7fff640 1 -- 172.21.15.5:0/1881778691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac001b6e0 con 0x560b2f1795e0 2024-09-15T22:11:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.220+0000 7faad6c75740 1 -- 172.21.15.5:0/1881778691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 msgr2=0x560b2f1a3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.220+0000 7faad6c75740 1 --2- 172.21.15.5:0/1881778691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f1a3080 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faac0005fd0 tx=0x7faac00094e0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.220+0000 7faad6c75740 1 -- 172.21.15.5:0/1881778691 shutdown_connections 2024-09-15T22:11:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.220+0000 7faad6c75740 1 --2- 172.21.15.5:0/1881778691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f18b8e0 0x560b2f1a35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.220+0000 7faad6c75740 1 --2- 172.21.15.5:0/1881778691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f1795e0 0x560b2f1a3080 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.220+0000 7faad6c75740 1 --2- 172.21.15.5:0/1881778691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f173640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.220+0000 7faad6c75740 1 -- 172.21.15.5:0/1881778691 >> 172.21.15.5:0/1881778691 conn(0x560b2f1745f0 msgr2=0x560b2f172f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.220+0000 7faad6c75740 1 -- 172.21.15.5:0/1881778691 shutdown_connections 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.220+0000 7faad6c75740 1 -- 172.21.15.5:0/1881778691 wait complete. 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.220+0000 7faad6c75740 1 -- start start 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.220+0000 7faad6c75740 1 Processor -- start 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faad6c75740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f193bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faad6c75740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f1795e0 0x560b2f194110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faad6c75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f18b8e0 0x560b2f1909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faad6c75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faad01ab380 con 0x560b2f18b8e0 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faad6c75740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faad01ac7f0 con 0x560b2f1795e0 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faad6c75740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faad01aba50 con 0x560b2f178440 2024-09-15T22:11:33.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faad6c75740 10 client.0 _ll_register_callbacks cb 0x7faab8012e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faace59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f1795e0 0x560b2f194110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faac7fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f193bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faace59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f1795e0 0x560b2f194110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:37988/0 (socket says 172.21.15.5:37988) 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faac7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f18b8e0 0x560b2f1909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faace59c640 1 -- 172.21.15.5:0/1816743049 learned_addr learned my addr 172.21.15.5:0/1816743049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faac7fff640 1 -- 172.21.15.5:0/1816743049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 msgr2=0x560b2f193bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faac7fff640 1 --2- 172.21.15.5:0/1816743049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f193bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faac7fff640 1 -- 172.21.15.5:0/1816743049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f1795e0 msgr2=0x560b2f194110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faac7fff640 1 --2- 172.21.15.5:0/1816743049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f1795e0 0x560b2f194110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faac7fff640 1 -- 172.21.15.5:0/1816743049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2f191000 con 0x560b2f18b8e0 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faac7fff640 1 --2- 172.21.15.5:0/1816743049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f18b8e0 0x560b2f1909c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faab80124d0 tx=0x7faab8004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d818b0f91c1d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.221+0000 7faab5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:33.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faab5ffb640 1 -- 172.21.15.5:0/1816743049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8010e80 con 0x560b2f18b8e0 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faab5ffb640 1 -- 172.21.15.5:0/1816743049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7faab8004660 con 0x560b2f18b8e0 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faad6c75740 1 -- 172.21.15.5:0/1816743049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faac0004220 con 0x560b2f18b8e0 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faab5ffb640 1 -- 172.21.15.5:0/1816743049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faab8004860 con 0x560b2f18b8e0 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faad6c75740 10 client.15897 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faad6c75740 1 -- 172.21.15.5:0/1816743049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560b2f206950 con 0x560b2f18b8e0 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faab5ffb640 1 -- 172.21.15.5:0/1816743049 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac0004220 con 0x560b2f18b8e0 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faab5ffb640 1 -- 172.21.15.5:0/1816743049 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7faab8035040 con 0x560b2f18b8e0 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faad6c75740 10 client.15897 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faad6c75740 10 client.15897 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faad6c75740 20 client.15897 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faad6c75740 1 --2- 172.21.15.5:0/1816743049 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x560b2f2070a0 0x560b2f2094c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faad6c75740 4 client.15897 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dd7f13bb", "release": true, "await": true}] 2024-09-15T22:11:33.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.222+0000 7faad6c75740 1 -- 172.21.15.5:0/1816743049 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dd7f13bb", "release": true, "await": true}) -- 0x560b2f0e50a0 con 0x560b2f2070a0 2024-09-15T22:11:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.223+0000 7faace59c640 1 --2- 172.21.15.5:0/1816743049 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x560b2f2070a0 0x560b2f2094c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.223+0000 7faace59c640 1 --2- 172.21.15.5:0/1816743049 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x560b2f2070a0 0x560b2f2094c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40243f0b16ebc80 server_cookie=8bc432b0cca9536d in_seq=0 out_seq=0 2024-09-15T22:11:33.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.223+0000 7faab5ffb640 10 client.15897 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:33.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.273+0000 7faab5ffb640 1 -- 172.21.15.5:0/1816743049 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x560b2f0e50a0 con 0x560b2f2070a0 2024-09-15T22:11:33.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.273+0000 7faab5ffb640 10 client.15897 handle_command_reply: tid=0 2024-09-15T22:11:33.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.273+0000 7faad6c75740 1 client.15897 shutdown 2024-09-15T22:11:33.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.278+0000 7faad6c75740 1 -- 172.21.15.5:0/1816743049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f18b8e0 msgr2=0x560b2f1909c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.278+0000 7faad6c75740 1 --2- 172.21.15.5:0/1816743049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f18b8e0 0x560b2f1909c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7faab80124d0 tx=0x7faab8004560 comp rx=0 tx=0).stop 2024-09-15T22:11:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.278+0000 7faad6c75740 1 -- 172.21.15.5:0/1816743049 shutdown_connections 2024-09-15T22:11:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.278+0000 7faad6c75740 1 --2- 172.21.15.5:0/1816743049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b2f18b8e0 0x560b2f1909c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.278+0000 7faad6c75740 1 --2- 172.21.15.5:0/1816743049 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x560b2f2070a0 0x560b2f2094c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.278+0000 7faad6c75740 1 --2- 172.21.15.5:0/1816743049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560b2f1795e0 0x560b2f194110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.278+0000 7faab5ffb640 0 client.15897 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.278+0000 7faad6c75740 1 --2- 172.21.15.5:0/1816743049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560b2f178440 0x560b2f193bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.278+0000 7faad6c75740 1 -- 172.21.15.5:0/1816743049 >> 172.21.15.5:0/1816743049 conn(0x560b2f1745f0 msgr2=0x560b2f17a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.278+0000 7faad6c75740 1 -- 172.21.15.5:0/1816743049 shutdown_connections 2024-09-15T22:11:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.278+0000 7faad6c75740 1 -- 172.21.15.5:0/1816743049 wait complete. 2024-09-15T22:11:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.279+0000 7faad6c75740 20 client.15897 trim_cache size 0 max 16384 2024-09-15T22:11:33.281 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:11:33.281 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:11:33.281 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:11:33.281 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 8, 2024-09-15T22:11:33.281 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:11:33.281 INFO:teuthology.orchestra.run.smithi005.stdout: "dd7f13bb": { 2024-09-15T22:11:33.281 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 8, 2024-09-15T22:11:33.281 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:11:33.281 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:11:33.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.6, 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:33.282 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:33.283 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:11:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.281+0000 7faad489c640 1 -- 172.21.15.5:0/2251846498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faabc0a33b0 msgr2=0x7faabc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.281+0000 7faad489c640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faabc0a33b0 0x7faabc0a5870 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faac80bcbe0 tx=0x7faac80b2370 comp rx=0 tx=0).stop 2024-09-15T22:11:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.281+0000 7faad489c640 1 -- 172.21.15.5:0/2251846498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad0102ef0 msgr2=0x7faad019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.281+0000 7faad489c640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad0102ef0 0x7faad019cd10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7faac802ae70 tx=0x7faac802f5d0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.285+0000 7faad489c640 1 -- 172.21.15.5:0/2251846498 shutdown_connections 2024-09-15T22:11:33.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.285+0000 7faad489c640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faabc0a33b0 0x7faabc0a5870 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.285+0000 7faad489c640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faad0102ef0 0x7faad019cd10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.285+0000 7faad489c640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faad0102570 0x7faad0198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.285+0000 7faad489c640 1 --2- 172.21.15.5:0/2251846498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faad0100e20 0x7faad0198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:33.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.285+0000 7faad489c640 1 -- 172.21.15.5:0/2251846498 >> 172.21.15.5:0/2251846498 conn(0x7faad00fc7d0 msgr2=0x7faad00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.287+0000 7faad489c640 1 -- 172.21.15.5:0/2251846498 shutdown_connections 2024-09-15T22:11:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:33.287+0000 7faad489c640 1 -- 172.21.15.5:0/2251846498 wait complete. 2024-09-15T22:11:33.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:33 smithi005 ceph-mon[30944]: pgmap v82: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-09-15T22:11:33.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:33 smithi099 ceph-mon[34892]: pgmap v82: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-09-15T22:11:33.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:33 smithi129 ceph-mon[35314]: pgmap v82: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.9 MiB/s wr, 172 op/s 2024-09-15T22:11:33.598 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:11:33.598 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:11:34.479 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dd7f13bb', total seconds elapsed: 25.1 2024-09-15T22:11:34.479 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-15T22:11:34.479 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:11:34.523 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 125.0 s, 803.0 tps, lat 491.929 ms stddev 2869.910 2024-09-15T22:11:35.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.129+0000 7f9691a96640 1 -- 172.21.15.5:0/3790267792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c106a70 msgr2=0x7f968c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.129+0000 7f9691a96640 1 --2- 172.21.15.5:0/3790267792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c106a70 0x7f968c106e50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f967c009930 tx=0x7f967c02a420 comp rx=0 tx=0).stop 2024-09-15T22:11:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.129+0000 7f9691a96640 1 -- 172.21.15.5:0/3790267792 shutdown_connections 2024-09-15T22:11:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.129+0000 7f9691a96640 1 --2- 172.21.15.5:0/3790267792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f968c101410 0x7f968c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.129+0000 7f9691a96640 1 --2- 172.21.15.5:0/3790267792 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c100a70 0x7f968c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.129+0000 7f9691a96640 1 --2- 172.21.15.5:0/3790267792 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c106a70 0x7f968c106e50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.129+0000 7f9691a96640 1 -- 172.21.15.5:0/3790267792 >> 172.21.15.5:0/3790267792 conn(0x7f968c0fc7d0 msgr2=0x7f968c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.130+0000 7f9691a96640 1 -- 172.21.15.5:0/3790267792 shutdown_connections 2024-09-15T22:11:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.130+0000 7f9691a96640 1 -- 172.21.15.5:0/3790267792 wait complete. 2024-09-15T22:11:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.131+0000 7f9691a96640 1 Processor -- start 2024-09-15T22:11:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.131+0000 7f9691a96640 1 -- start start 2024-09-15T22:11:35.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.131+0000 7f9691a96640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c100a70 0x7f968c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f9691a96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f968c101410 0x7f968c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f9691a96640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c106a70 0x7f968c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f9691a96640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f968c10fdc0 con 0x7f968c101410 2024-09-15T22:11:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f9691a96640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f968c10fc40 con 0x7f968c106a70 2024-09-15T22:11:35.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f9691a96640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f968c10ff40 con 0x7f968c100a70 2024-09-15T22:11:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f968affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f968c101410 0x7f968c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f968affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f968c101410 0x7f968c19abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60922/0 (socket says 172.21.15.5:60922) 2024-09-15T22:11:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f968affd640 1 -- 172.21.15.5:0/1435392125 learned_addr learned my addr 172.21.15.5:0/1435392125 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f968bfff640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c106a70 0x7f968c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f968b7fe640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c100a70 0x7f968c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f968bfff640 1 -- 172.21.15.5:0/1435392125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c100a70 msgr2=0x7f968c19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f968bfff640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c100a70 0x7f968c19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f968bfff640 1 -- 172.21.15.5:0/1435392125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f968c101410 msgr2=0x7f968c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f968bfff640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f968c101410 0x7f968c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f968bfff640 1 -- 172.21.15.5:0/1435392125 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f968c19f630 con 0x7f968c106a70 2024-09-15T22:11:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.132+0000 7f968b7fe640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c100a70 0x7f968c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.133+0000 7f968bfff640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c106a70 0x7f968c19ef50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f96840098f0 tx=0x7f9684009db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.133+0000 7f9688ff9640 1 -- 172.21.15.5:0/1435392125 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684018070 con 0x7f968c106a70 2024-09-15T22:11:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.133+0000 7f9688ff9640 1 -- 172.21.15.5:0/1435392125 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968400ed10 con 0x7f968c106a70 2024-09-15T22:11:35.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.133+0000 7f9691a96640 1 -- 172.21.15.5:0/1435392125 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f968c19f920 con 0x7f968c106a70 2024-09-15T22:11:35.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.133+0000 7f9688ff9640 1 -- 172.21.15.5:0/1435392125 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96840046a0 con 0x7f968c106a70 2024-09-15T22:11:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.133+0000 7f9691a96640 1 -- 172.21.15.5:0/1435392125 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f968c1a7200 con 0x7f968c106a70 2024-09-15T22:11:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.137+0000 7f9688ff9640 1 -- 172.21.15.5:0/1435392125 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9684013070 con 0x7f968c106a70 2024-09-15T22:11:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.138+0000 7f9693e6f740 1 Processor -- start 2024-09-15T22:11:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.138+0000 7f9693e6f740 1 -- start start 2024-09-15T22:11:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f9688ff9640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f96740a3480 0x7f96740a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab07b60 0x559daab07f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab08480 0x559daab12800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab15130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f9693e6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559daaa6d280 con 0x559daab08480 2024-09-15T22:11:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f9693e6f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559daaa14b10 con 0x559daab07b60 2024-09-15T22:11:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f9693e6f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559daaa69240 con 0x559daab12d40 2024-09-15T22:11:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f9688ff9640 1 -- 172.21.15.5:0/1435392125 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f96840cfa50 con 0x7f968c106a70 2024-09-15T22:11:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f968affd640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f96740a3480 0x7f96740a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f968b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab07b60 0x559daab07f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f968b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab07b60 0x559daab07f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33866/0 (socket says 172.21.15.5:33866) 2024-09-15T22:11:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f968b7fe640 1 -- 172.21.15.5:0/582429220 learned_addr learned my addr 172.21.15.5:0/582429220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f968affd640 1 --2- 172.21.15.5:0/582429220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab08480 0x559daab12800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.139+0000 7f968bfff640 1 --2- 172.21.15.5:0/582429220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab15130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.140+0000 7f968affd640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f96740a3480 0x7f96740a5940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96800097f0 tx=0x7f9680009c40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.140+0000 7f968b7fe640 1 -- 172.21.15.5:0/582429220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 msgr2=0x559daab15130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.140+0000 7f968b7fe640 1 --2- 172.21.15.5:0/582429220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab15130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.140+0000 7f968b7fe640 1 -- 172.21.15.5:0/582429220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab08480 msgr2=0x559daab12800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.140+0000 7f968b7fe640 1 --2- 172.21.15.5:0/582429220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab08480 0x559daab12800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.140+0000 7f968b7fe640 1 -- 172.21.15.5:0/582429220 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559daaa211f0 con 0x559daab07b60 2024-09-15T22:11:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.140+0000 7f968b7fe640 1 --2- 172.21.15.5:0/582429220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab07b60 0x559daab07f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f967c02aa10 tx=0x7f967c0314d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19425e0c43517aa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.142+0000 7f9671ffb640 1 -- 172.21.15.5:0/582429220 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c03f070 con 0x559daab07b60 2024-09-15T22:11:35.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.142+0000 7f9671ffb640 1 -- 172.21.15.5:0/582429220 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f967c005520 con 0x559daab07b60 2024-09-15T22:11:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.142+0000 7f9671ffb640 1 -- 172.21.15.5:0/582429220 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c035040 con 0x559daab07b60 2024-09-15T22:11:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.143+0000 7f9693e6f740 1 -- 172.21.15.5:0/582429220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab07b60 msgr2=0x559daab07f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.143+0000 7f9693e6f740 1 --2- 172.21.15.5:0/582429220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab07b60 0x559daab07f40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f967c02aa10 tx=0x7f967c0314d0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.143+0000 7f9693e6f740 1 -- 172.21.15.5:0/582429220 shutdown_connections 2024-09-15T22:11:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.143+0000 7f9693e6f740 1 --2- 172.21.15.5:0/582429220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab15130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.143+0000 7f9693e6f740 1 --2- 172.21.15.5:0/582429220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab08480 0x559daab12800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.143+0000 7f9693e6f740 1 --2- 172.21.15.5:0/582429220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab07b60 0x559daab07f40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.143+0000 7f9693e6f740 1 -- 172.21.15.5:0/582429220 >> 172.21.15.5:0/582429220 conn(0x559daab040f0 msgr2=0x559daab044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.143+0000 7f9693e6f740 1 -- 172.21.15.5:0/582429220 shutdown_connections 2024-09-15T22:11:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.143+0000 7f9693e6f740 1 -- 172.21.15.5:0/582429220 wait complete. 2024-09-15T22:11:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.144+0000 7f9693e6f740 1 -- start start 2024-09-15T22:11:35.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.144+0000 7f9693e6f740 1 Processor -- start 2024-09-15T22:11:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.147+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab07b60 0x559daab127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.147+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab08480 0x559daab0fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.147+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab12d40 0x559daab10250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.147+0000 7f9693e6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559daab15900 con 0x559daab08480 2024-09-15T22:11:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.147+0000 7f9693e6f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559daaa27f10 con 0x559daab12d40 2024-09-15T22:11:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.147+0000 7f9693e6f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559daab15a80 con 0x559daab07b60 2024-09-15T22:11:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.147+0000 7f9693e6f740 10 client.0 _ll_register_callbacks cb 0x559daa9f0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.147+0000 7f968b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab07b60 0x559daab127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.147+0000 7f968bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab12d40 0x559daab10250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.148+0000 7f968b7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab07b60 0x559daab127c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:52902/0 (socket says 172.21.15.5:52902) 2024-09-15T22:11:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.148+0000 7f968b7fe640 1 -- 172.21.15.5:0/3984499968 learned_addr learned my addr 172.21.15.5:0/3984499968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.148+0000 7f968affd640 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab08480 0x559daab0fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.149+0000 7f968affd640 1 -- 172.21.15.5:0/3984499968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab07b60 msgr2=0x559daab127c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.149+0000 7f968affd640 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab07b60 0x559daab127c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.149+0000 7f968affd640 1 -- 172.21.15.5:0/3984499968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab12d40 msgr2=0x559daab10250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.149+0000 7f968affd640 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab12d40 0x559daab10250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.149+0000 7f968affd640 1 -- 172.21.15.5:0/3984499968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559daa7de6a0 con 0x559daab08480 2024-09-15T22:11:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.150+0000 7f968affd640 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab08480 0x559daab0fb10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x559daab04f50 tx=0x7f9680005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e277efa2db2d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.150+0000 7f966affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.150+0000 7f966affd640 1 -- 172.21.15.5:0/3984499968 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968002c060 con 0x559daab08480 2024-09-15T22:11:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.150+0000 7f966affd640 1 -- 172.21.15.5:0/3984499968 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f968001f700 con 0x559daab08480 2024-09-15T22:11:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.150+0000 7f9693e6f740 1 -- 172.21.15.5:0/3984499968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f967c005520 con 0x559daab08480 2024-09-15T22:11:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.151+0000 7f968b7fe640 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab07b60 0x559daab127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.151+0000 7f966affd640 1 -- 172.21.15.5:0/3984499968 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9680033050 con 0x559daab08480 2024-09-15T22:11:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.151+0000 7f9693e6f740 10 client.15909 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.151+0000 7f9693e6f740 1 -- 172.21.15.5:0/3984499968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559daab21470 con 0x559daab08480 2024-09-15T22:11:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.151+0000 7f966affd640 1 -- 172.21.15.5:0/3984499968 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f967c005520 con 0x559daab08480 2024-09-15T22:11:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.152+0000 7f966affd640 1 -- 172.21.15.5:0/3984499968 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9680026070 con 0x559daab08480 2024-09-15T22:11:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.152+0000 7f9693e6f740 10 client.15909 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.152+0000 7f9693e6f740 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.152+0000 7f9693e6f740 20 client.15909 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.152+0000 7f9693e6f740 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559daab22dd0 0x559daab25250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.152+0000 7f9693e6f740 4 client.15909 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.152+0000 7f9693e6f740 1 -- 172.21.15.5:0/3984499968 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559daaa7a7f0 con 0x559daab22dd0 2024-09-15T22:11:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.153+0000 7f968b7fe640 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559daab22dd0 0x559daab25250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.153+0000 7f968b7fe640 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559daab22dd0 0x559daab25250 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa9dadad21818e8 server_cookie=8caefd36362d84f in_seq=0 out_seq=0 2024-09-15T22:11:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.153+0000 7f966affd640 10 client.15909 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.154+0000 7f966affd640 1 -- 172.21.15.5:0/3984499968 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559daaa7a7f0 con 0x559daab22dd0 2024-09-15T22:11:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.154+0000 7f966affd640 10 client.15909 handle_command_reply: tid=0 2024-09-15T22:11:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.154+0000 7f9693e6f740 1 client.15909 shutdown 2024-09-15T22:11:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.154+0000 7f9693e6f740 1 -- 172.21.15.5:0/3984499968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab08480 msgr2=0x559daab0fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.154+0000 7f9693e6f740 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab08480 0x559daab0fb10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x559daab04f50 tx=0x7f9680005f90 comp rx=0 tx=0).stop 2024-09-15T22:11:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.154+0000 7f9693e6f740 1 -- 172.21.15.5:0/3984499968 shutdown_connections 2024-09-15T22:11:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.154+0000 7f9693e6f740 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559daab22dd0 0x559daab25250 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.154+0000 7f9693e6f740 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab12d40 0x559daab10250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.154+0000 7f9693e6f740 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab08480 0x559daab0fb10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.154+0000 7f9693e6f740 1 --2- 172.21.15.5:0/3984499968 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab07b60 0x559daab127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.154+0000 7f9693e6f740 1 -- 172.21.15.5:0/3984499968 >> 172.21.15.5:0/3984499968 conn(0x559daab040f0 msgr2=0x559daab09e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.155+0000 7f966affd640 0 client.15909 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.155+0000 7f9693e6f740 1 -- 172.21.15.5:0/3984499968 shutdown_connections 2024-09-15T22:11:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.155+0000 7f9693e6f740 1 -- 172.21.15.5:0/3984499968 wait complete. 2024-09-15T22:11:35.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.155+0000 7f9693e6f740 20 client.15909 trim_cache size 0 max 16384 2024-09-15T22:11:35.162 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:35.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.190+0000 7f9693e6f740 1 Processor -- start 2024-09-15T22:11:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.191+0000 7f9693e6f740 1 -- start start 2024-09-15T22:11:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.191+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab07b60 0x559daab25240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.191+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab08480 0x559daab1ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.191+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab0eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.191+0000 7f9693e6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1af7c0 con 0x559daab07b60 2024-09-15T22:11:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.191+0000 7f9693e6f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1ad090 con 0x559daab08480 2024-09-15T22:11:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.191+0000 7f9693e6f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1aaee0 con 0x559daab12d40 2024-09-15T22:11:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.192+0000 7f968b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab07b60 0x559daab25240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.192+0000 7f968b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab07b60 0x559daab25240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60954/0 (socket says 172.21.15.5:60954) 2024-09-15T22:11:35.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.192+0000 7f968b7fe640 1 -- 172.21.15.5:0/84773458 learned_addr learned my addr 172.21.15.5:0/84773458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:35.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.194+0000 7f968bfff640 1 --2- 172.21.15.5:0/84773458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab0eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.194+0000 7f968bfff640 1 -- 172.21.15.5:0/84773458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab08480 msgr2=0x559daab1ebe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:11:35.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.194+0000 7f968bfff640 1 --2- 172.21.15.5:0/84773458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab08480 0x559daab1ebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.195+0000 7f968bfff640 1 -- 172.21.15.5:0/84773458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab07b60 msgr2=0x559daab25240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:35.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.195+0000 7f968bfff640 1 --2- 172.21.15.5:0/84773458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab07b60 0x559daab25240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.195+0000 7f968bfff640 1 -- 172.21.15.5:0/84773458 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559daab082b0 con 0x559daab12d40 2024-09-15T22:11:35.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.201+0000 7f968bfff640 1 --2- 172.21.15.5:0/84773458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab0eaf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f96840cce00 tx=0x7f9684086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8cb42c58defbc0b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.205+0000 7f96717fa640 1 -- 172.21.15.5:0/84773458 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968409b2f0 con 0x559daab12d40 2024-09-15T22:11:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.205+0000 7f96717fa640 1 -- 172.21.15.5:0/84773458 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968409c090 con 0x559daab12d40 2024-09-15T22:11:35.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.205+0000 7f96717fa640 1 -- 172.21.15.5:0/84773458 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684098000 con 0x559daab12d40 2024-09-15T22:11:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.206+0000 7f9693e6f740 1 -- 172.21.15.5:0/84773458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 msgr2=0x559daab0eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.206+0000 7f9693e6f740 1 --2- 172.21.15.5:0/84773458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab0eaf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f96840cce00 tx=0x7f9684086040 comp rx=0 tx=0).stop 2024-09-15T22:11:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.207+0000 7f9693e6f740 1 -- 172.21.15.5:0/84773458 shutdown_connections 2024-09-15T22:11:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.207+0000 7f9693e6f740 1 --2- 172.21.15.5:0/84773458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab0eaf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.207+0000 7f9693e6f740 1 --2- 172.21.15.5:0/84773458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab08480 0x559daab1ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.207+0000 7f9693e6f740 1 --2- 172.21.15.5:0/84773458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab07b60 0x559daab25240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.207+0000 7f9693e6f740 1 -- 172.21.15.5:0/84773458 >> 172.21.15.5:0/84773458 conn(0x559daab040f0 msgr2=0x559daab0e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:35.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.211+0000 7f9693e6f740 1 -- 172.21.15.5:0/84773458 shutdown_connections 2024-09-15T22:11:35.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.211+0000 7f9693e6f740 1 -- 172.21.15.5:0/84773458 wait complete. 2024-09-15T22:11:35.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.212+0000 7f9693e6f740 1 -- start start 2024-09-15T22:11:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.212+0000 7f9693e6f740 1 Processor -- start 2024-09-15T22:11:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.213+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab07b60 0x559daab88dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.213+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab08480 0x559daab89310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.213+0000 7f9693e6f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab1ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.213+0000 7f968affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab08480 0x559daab89310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.213+0000 7f968affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab08480 0x559daab89310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33906/0 (socket says 172.21.15.5:33906) 2024-09-15T22:11:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.213+0000 7f968affd640 1 -- 172.21.15.5:0/622924442 learned_addr learned my addr 172.21.15.5:0/622924442 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.214+0000 7f968b7fe640 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab07b60 0x559daab88dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.214+0000 7f9693e6f740 1 -- 172.21.15.5:0/622924442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1ab550 con 0x559daab07b60 2024-09-15T22:11:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.214+0000 7f968bfff640 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab1ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.214+0000 7f9693e6f740 1 -- 172.21.15.5:0/622924442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1ac9c0 con 0x559daab08480 2024-09-15T22:11:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.214+0000 7f9693e6f740 1 -- 172.21.15.5:0/622924442 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f968c1abc20 con 0x559daab12d40 2024-09-15T22:11:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.214+0000 7f9693e6f740 10 client.0 _ll_register_callbacks cb 0x7f96800099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.215+0000 7f968affd640 1 -- 172.21.15.5:0/622924442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 msgr2=0x559daab1ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:35.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.215+0000 7f968affd640 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab1ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.215+0000 7f968affd640 1 -- 172.21.15.5:0/622924442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab07b60 msgr2=0x559daab88dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:35.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.215+0000 7f968affd640 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab07b60 0x559daab88dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.215+0000 7f968affd640 1 -- 172.21.15.5:0/622924442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559daab1b580 con 0x559daab08480 2024-09-15T22:11:35.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.216+0000 7f968affd640 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab08480 0x559daab89310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9680011fd0 tx=0x7f9680010fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=473485f375bba286 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:35.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.216+0000 7f966affd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:11:35.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.216+0000 7f966affd640 1 -- 172.21.15.5:0/622924442 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f968002c070 con 0x559daab08480 2024-09-15T22:11:35.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.216+0000 7f9693e6f740 1 -- 172.21.15.5:0/622924442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f968409c090 con 0x559daab08480 2024-09-15T22:11:35.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.216+0000 7f966affd640 1 -- 172.21.15.5:0/622924442 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f9680010640 con 0x559daab08480 2024-09-15T22:11:35.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.217+0000 7f966affd640 1 -- 172.21.15.5:0/622924442 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9680005d30 con 0x559daab08480 2024-09-15T22:11:35.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.217+0000 7f9693e6f740 10 client.25517 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:35.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.217+0000 7f9693e6f740 1 -- 172.21.15.5:0/622924442 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559daab86e50 con 0x559daab08480 2024-09-15T22:11:35.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.218+0000 7f966affd640 1 -- 172.21.15.5:0/622924442 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f968000fe60 con 0x559daab08480 2024-09-15T22:11:35.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.218+0000 7f966affd640 1 -- 172.21.15.5:0/622924442 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9680026040 con 0x559daab08480 2024-09-15T22:11:35.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.218+0000 7f9693e6f740 10 client.25517 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:35.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.218+0000 7f9693e6f740 10 client.25517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:35.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.218+0000 7f9693e6f740 20 client.25517 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:35.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.218+0000 7f9693e6f740 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559daab9eec0 0x559daaba1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.218+0000 7f9693e6f740 4 client.25517 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:11:35.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.218+0000 7f9693e6f740 1 -- 172.21.15.5:0/622924442 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f968409c090 con 0x559daab9eec0 2024-09-15T22:11:35.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.220+0000 7f968b7fe640 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559daab9eec0 0x559daaba1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.225+0000 7f966affd640 10 client.25517 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:35.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.225+0000 7f968b7fe640 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559daab9eec0 0x559daaba1320 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3cec286077cc82 server_cookie=44c18f596c4f47d9 in_seq=0 out_seq=0 2024-09-15T22:11:35.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.226+0000 7f966affd640 1 -- 172.21.15.5:0/622924442 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f968409c090 con 0x559daab9eec0 2024-09-15T22:11:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.226+0000 7f966affd640 10 client.25517 handle_command_reply: tid=0 2024-09-15T22:11:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.226+0000 7f9693e6f740 1 client.25517 shutdown 2024-09-15T22:11:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.229+0000 7f9693e6f740 1 -- 172.21.15.5:0/622924442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab08480 msgr2=0x559daab89310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.229+0000 7f9693e6f740 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab08480 0x559daab89310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9680011fd0 tx=0x7f9680010fc0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.229+0000 7f9693e6f740 1 -- 172.21.15.5:0/622924442 shutdown_connections 2024-09-15T22:11:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.229+0000 7f9693e6f740 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559daab12d40 0x559daab1ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.229+0000 7f9693e6f740 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559daab9eec0 0x559daaba1320 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.229+0000 7f9693e6f740 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559daab08480 0x559daab89310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.229+0000 7f9693e6f740 1 --2- 172.21.15.5:0/622924442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559daab07b60 0x559daab88dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.230+0000 7f9693e6f740 1 -- 172.21.15.5:0/622924442 >> 172.21.15.5:0/622924442 conn(0x559daab040f0 msgr2=0x559daab0ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:35.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.230+0000 7f966affd640 0 client.25517 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:35.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.230+0000 7f9693e6f740 1 -- 172.21.15.5:0/622924442 shutdown_connections 2024-09-15T22:11:35.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.230+0000 7f9693e6f740 1 -- 172.21.15.5:0/622924442 wait complete. 2024-09-15T22:11:35.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.230+0000 7f9693e6f740 20 client.25517 trim_cache size 0 max 16384 2024-09-15T22:11:35.232 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:11:35.232 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:11:35.233 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "904f3892-e94e-4c35-b332-aca67f898f51", 2024-09-15T22:11:35.233 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:11:35.233 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:11:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.231+0000 7f9691a96640 1 -- 172.21.15.5:0/1435392125 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f96740a3480 msgr2=0x7f96740a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.231+0000 7f9691a96640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f96740a3480 0x7f96740a5940 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f96800097f0 tx=0x7f9680009c40 comp rx=0 tx=0).stop 2024-09-15T22:11:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.232+0000 7f9691a96640 1 -- 172.21.15.5:0/1435392125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c106a70 msgr2=0x7f968c19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.232+0000 7f9691a96640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c106a70 0x7f968c19ef50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f96840098f0 tx=0x7f9684009db0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.232+0000 7f9691a96640 1 -- 172.21.15.5:0/1435392125 shutdown_connections 2024-09-15T22:11:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.232+0000 7f9691a96640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f96740a3480 0x7f96740a5940 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.232+0000 7f9691a96640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f968c106a70 0x7f968c19ef50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.232+0000 7f9691a96640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f968c101410 0x7f968c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.232+0000 7f9691a96640 1 --2- 172.21.15.5:0/1435392125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f968c100a70 0x7f968c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.232+0000 7f9691a96640 1 -- 172.21.15.5:0/1435392125 >> 172.21.15.5:0/1435392125 conn(0x7f968c0fc7d0 msgr2=0x7f968c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.234+0000 7f9691a96640 1 -- 172.21.15.5:0/1435392125 shutdown_connections 2024-09-15T22:11:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.235+0000 7f9691a96640 1 -- 172.21.15.5:0/1435392125 wait complete. 2024-09-15T22:11:35.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:35 smithi005 ceph-mon[30944]: pgmap v83: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-09-15T22:11:35.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:35 smithi099 ceph-mon[34892]: pgmap v83: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-09-15T22:11:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:35 smithi129 ceph-mon[35314]: pgmap v83: 129 pgs: 129 active+clean; 1.6 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-09-15T22:11:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.759+0000 7fa150307640 1 -- 172.21.15.5:0/670064024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1481036e0 msgr2=0x7fa148107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.760+0000 7fa150307640 1 --2- 172.21.15.5:0/670064024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1481036e0 0x7fa148107b90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa138009900 tx=0x7fa13802a410 comp rx=0 tx=0).stop 2024-09-15T22:11:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.760+0000 7fa150307640 1 -- 172.21.15.5:0/670064024 shutdown_connections 2024-09-15T22:11:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.760+0000 7fa150307640 1 --2- 172.21.15.5:0/670064024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1481082a0 0x7fa14810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.760+0000 7fa150307640 1 --2- 172.21.15.5:0/670064024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa1481036e0 0x7fa148107b90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.760+0000 7fa150307640 1 --2- 172.21.15.5:0/670064024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa148102d30 0x7fa148103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.760+0000 7fa150307640 1 -- 172.21.15.5:0/670064024 >> 172.21.15.5:0/670064024 conn(0x7fa1480fc7d0 msgr2=0x7fa1480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:35.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.760+0000 7fa150307640 1 -- 172.21.15.5:0/670064024 shutdown_connections 2024-09-15T22:11:35.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.760+0000 7fa150307640 1 -- 172.21.15.5:0/670064024 wait complete. 2024-09-15T22:11:35.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.761+0000 7fa150307640 1 Processor -- start 2024-09-15T22:11:35.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.761+0000 7fa150307640 1 -- start start 2024-09-15T22:11:35.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.762+0000 7fa150307640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa148102d30 0x7fa148198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.762+0000 7fa150307640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1481036e0 0x7fa148198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.762+0000 7fa150307640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1481082a0 0x7fa14819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.762+0000 7fa150307640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481120b0 con 0x7fa1481036e0 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.762+0000 7fa150307640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa148111f30 con 0x7fa1481082a0 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.762+0000 7fa14e8a4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1481082a0 0x7fa14819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.762+0000 7fa14e8a4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1481082a0 0x7fa14819ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33912/0 (socket says 172.21.15.5:33912) 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.762+0000 7fa14e8a4640 1 -- 172.21.15.5:0/1067342605 learned_addr learned my addr 172.21.15.5:0/1067342605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.762+0000 7fa150307640 1 -- 172.21.15.5:0/1067342605 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa148112230 con 0x7fa148102d30 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.762+0000 7fa14e0a3640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa148102d30 0x7fa148198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.763+0000 7fa14e8a4640 1 -- 172.21.15.5:0/1067342605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa148102d30 msgr2=0x7fa148198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.763+0000 7fa14e8a4640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa148102d30 0x7fa148198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.763+0000 7fa14e8a4640 1 -- 172.21.15.5:0/1067342605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1481036e0 msgr2=0x7fa148198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.763+0000 7fa14e8a4640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1481036e0 0x7fa148198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:35.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.763+0000 7fa14e8a4640 1 -- 172.21.15.5:0/1067342605 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14819d4e0 con 0x7fa1481082a0 2024-09-15T22:11:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.764+0000 7fa14e8a4640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1481082a0 0x7fa14819ce00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa14000eb10 tx=0x7fa14000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.764+0000 7fa13f7fe640 1 -- 172.21.15.5:0/1067342605 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140004b20 con 0x7fa1481082a0 2024-09-15T22:11:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.764+0000 7fa13f7fe640 1 -- 172.21.15.5:0/1067342605 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa140004cc0 con 0x7fa1481082a0 2024-09-15T22:11:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.765+0000 7fa150307640 1 -- 172.21.15.5:0/1067342605 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa14819d7d0 con 0x7fa1481082a0 2024-09-15T22:11:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.765+0000 7fa13f7fe640 1 -- 172.21.15.5:0/1067342605 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140010790 con 0x7fa1481082a0 2024-09-15T22:11:35.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.765+0000 7fa150307640 1 -- 172.21.15.5:0/1067342605 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1481a50b0 con 0x7fa1481082a0 2024-09-15T22:11:35.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.767+0000 7fa150307640 1 -- 172.21.15.5:0/1067342605 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa148103110 con 0x7fa1481082a0 2024-09-15T22:11:35.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.768+0000 7fa13f7fe640 1 -- 172.21.15.5:0/1067342605 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa140010930 con 0x7fa1481082a0 2024-09-15T22:11:35.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.769+0000 7fa13f7fe640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa1340a3430 0x7fa1340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.769+0000 7fa14e0a3640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa1340a3430 0x7fa1340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.770+0000 7fa13f7fe640 1 -- 172.21.15.5:0/1067342605 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa1400d0980 con 0x7fa1481082a0 2024-09-15T22:11:35.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.770+0000 7fa14e0a3640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa1340a3430 0x7fa1340a58f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa144009b60 tx=0x7fa144005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:35.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:35.772+0000 7fa13f7fe640 1 -- 172.21.15.5:0/1067342605 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa14009cf60 con 0x7fa1481082a0 2024-09-15T22:11:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.155+0000 7fa150307640 1 -- 172.21.15.5:0/1067342605 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa148199890 con 0x7fa1481082a0 2024-09-15T22:11:36.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.155+0000 7fa13f7fe640 1 -- 172.21.15.5:0/1067342605 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fa148199890 con 0x7fa1481082a0 2024-09-15T22:11:36.157 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:11:36.157 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:11:36.157 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:11:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.158+0000 7fa150307640 1 -- 172.21.15.5:0/1067342605 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa1340a3430 msgr2=0x7fa1340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.158+0000 7fa150307640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa1340a3430 0x7fa1340a58f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa144009b60 tx=0x7fa144005f90 comp rx=0 tx=0).stop 2024-09-15T22:11:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.158+0000 7fa150307640 1 -- 172.21.15.5:0/1067342605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1481082a0 msgr2=0x7fa14819ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.158+0000 7fa150307640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1481082a0 0x7fa14819ce00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa14000eb10 tx=0x7fa14000c2d0 comp rx=0 tx=0).stop 2024-09-15T22:11:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.159+0000 7fa150307640 1 -- 172.21.15.5:0/1067342605 shutdown_connections 2024-09-15T22:11:36.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.159+0000 7fa150307640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa1340a3430 0x7fa1340a58f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.159+0000 7fa150307640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa1481082a0 0x7fa14819ce00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.159+0000 7fa150307640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1481036e0 0x7fa148198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.159+0000 7fa150307640 1 --2- 172.21.15.5:0/1067342605 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa148102d30 0x7fa148198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.159+0000 7fa150307640 1 -- 172.21.15.5:0/1067342605 >> 172.21.15.5:0/1067342605 conn(0x7fa1480fc7d0 msgr2=0x7fa148106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.159+0000 7fa150307640 1 -- 172.21.15.5:0/1067342605 shutdown_connections 2024-09-15T22:11:36.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:36.159+0000 7fa150307640 1 -- 172.21.15.5:0/1067342605 wait complete. 2024-09-15T22:11:36.526 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:36 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:11:36.526 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:36 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:36.526 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:36 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:11:36.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:36 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:36.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:36 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1067342605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:36.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:11:36.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:36 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:11:36.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:36 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:36.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:36 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:11:36.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:36 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:36.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:36 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1067342605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:36.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:36 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:11:36.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:36 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:36.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:36 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:11:36.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:36 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:36.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:36 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1067342605' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:37.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:37 smithi005 ceph-mon[30944]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.9 MiB/s wr, 705 op/s 2024-09-15T22:11:37.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:37 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:11:37.459 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0' 2024-09-15T22:11:37.459 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 139.55 --await-for 45.0 2024-09-15T22:11:37.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:37 smithi099 ceph-mon[34892]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.9 MiB/s wr, 705 op/s 2024-09-15T22:11:37.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:37 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:11:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:37.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:37 smithi129 ceph-mon[35314]: pgmap v84: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 7.9 MiB/s wr, 705 op/s 2024-09-15T22:11:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:37 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:11:38.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.202+0000 7ff908fda640 1 -- 172.21.15.5:0/3152172141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904112b90 msgr2=0x7ff904114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:38.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.202+0000 7ff908fda640 1 --2- 172.21.15.5:0/3152172141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904112b90 0x7ff904114f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009860 tx=0x7ff8fc02a3d0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.203+0000 7ff908fda640 1 -- 172.21.15.5:0/3152172141 shutdown_connections 2024-09-15T22:11:38.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.203+0000 7ff908fda640 1 --2- 172.21.15.5:0/3152172141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904112b90 0x7ff904114f80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.203+0000 7ff908fda640 1 --2- 172.21.15.5:0/3152172141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff904074eb0 0x7ff904112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.203+0000 7ff908fda640 1 --2- 172.21.15.5:0/3152172141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff904074590 0x7ff904074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.203+0000 7ff908fda640 1 -- 172.21.15.5:0/3152172141 >> 172.21.15.5:0/3152172141 conn(0x7ff9040fd8e0 msgr2=0x7ff9040ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.203+0000 7ff908fda640 1 -- 172.21.15.5:0/3152172141 shutdown_connections 2024-09-15T22:11:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.203+0000 7ff908fda640 1 -- 172.21.15.5:0/3152172141 wait complete. 2024-09-15T22:11:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.205+0000 7ff908fda640 1 Processor -- start 2024-09-15T22:11:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.205+0000 7ff908fda640 1 -- start start 2024-09-15T22:11:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.206+0000 7ff908fda640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904074590 0x7ff9041a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.206+0000 7ff908fda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff904074eb0 0x7ff9041a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.206+0000 7ff908fda640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff904112b90 0x7ff90419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.206+0000 7ff908fda640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff904118550 con 0x7ff904074eb0 2024-09-15T22:11:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.206+0000 7ff908fda640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041183d0 con 0x7ff904074590 2024-09-15T22:11:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.206+0000 7ff908fda640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041186d0 con 0x7ff904112b90 2024-09-15T22:11:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.207+0000 7ff902d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff904112b90 0x7ff90419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.207+0000 7ff902d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff904112b90 0x7ff90419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:52946/0 (socket says 172.21.15.5:52946) 2024-09-15T22:11:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.207+0000 7ff902d9d640 1 -- 172.21.15.5:0/1510329681 learned_addr learned my addr 172.21.15.5:0/1510329681 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.207+0000 7ff902d9d640 1 -- 172.21.15.5:0/1510329681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904074590 msgr2=0x7ff9041a46a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:11:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.207+0000 7ff902d9d640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904074590 0x7ff9041a46a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.207+0000 7ff902d9d640 1 -- 172.21.15.5:0/1510329681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff904074eb0 msgr2=0x7ff9041a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.207+0000 7ff902d9d640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff904074eb0 0x7ff9041a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.207+0000 7ff902d9d640 1 -- 172.21.15.5:0/1510329681 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff90419efc0 con 0x7ff904112b90 2024-09-15T22:11:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.208+0000 7ff902d9d640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff904112b90 0x7ff90419e820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc005040 tx=0x7ff8fc005070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.211+0000 7ff8f37fe640 1 -- 172.21.15.5:0/1510329681 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc040070 con 0x7ff904112b90 2024-09-15T22:11:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.211+0000 7ff8f37fe640 1 -- 172.21.15.5:0/1510329681 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8fc0055a0 con 0x7ff904112b90 2024-09-15T22:11:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.211+0000 7ff908fda640 1 -- 172.21.15.5:0/1510329681 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff90419f250 con 0x7ff904112b90 2024-09-15T22:11:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.212+0000 7ff8f37fe640 1 -- 172.21.15.5:0/1510329681 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc004e40 con 0x7ff904112b90 2024-09-15T22:11:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.212+0000 7ff908fda640 1 -- 172.21.15.5:0/1510329681 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9041ab500 con 0x7ff904112b90 2024-09-15T22:11:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.215+0000 7ff90b3b3740 1 Processor -- start 2024-09-15T22:11:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.216+0000 7ff90b3b3740 1 -- start start 2024-09-15T22:11:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.216+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e27671e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.216+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671e970 0x55e27671edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.216+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27673a440 0x55e27673c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.216+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e2766842e0 con 0x55e27671e970 2024-09-15T22:11:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.216+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e27662bce0 con 0x55e27671fd70 2024-09-15T22:11:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.216+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e2766802a0 con 0x55e27673a440 2024-09-15T22:11:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.218+0000 7ff8f37fe640 1 -- 172.21.15.5:0/1510329681 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff8fc039040 con 0x7ff904112b90 2024-09-15T22:11:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.218+0000 7ff90259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671e970 0x55e27671edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.218+0000 7ff90259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671e970 0x55e27671edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32768/0 (socket says 172.21.15.5:32768) 2024-09-15T22:11:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.218+0000 7ff90259c640 1 -- 172.21.15.5:0/2545081087 learned_addr learned my addr 172.21.15.5:0/2545081087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.219+0000 7ff901d9b640 1 --2- 172.21.15.5:0/2545081087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e27671e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.219+0000 7ff90259c640 1 -- 172.21.15.5:0/2545081087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27673a440 msgr2=0x55e27673c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.220+0000 7ff901d9b640 1 --2- 172.21.15.5:0/2545081087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27673a440 0x55e27673c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.221+0000 7ff90259c640 1 --2- 172.21.15.5:0/2545081087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27673a440 0x55e27673c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.221+0000 7ff90259c640 1 -- 172.21.15.5:0/2545081087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 msgr2=0x55e27671e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.221+0000 7ff90259c640 1 --2- 172.21.15.5:0/2545081087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e27671e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.221+0000 7ff90259c640 1 -- 172.21.15.5:0/2545081087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e276610440 con 0x55e27671e970 2024-09-15T22:11:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.221+0000 7ff901d9b640 1 --2- 172.21.15.5:0/2545081087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e27671e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.222+0000 7ff8f37fe640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff8ec0a3500 0x7ff8ec0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.223+0000 7ff8f37fe640 1 -- 172.21.15.5:0/1510329681 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7ff8fc0f0c90 con 0x7ff904112b90 2024-09-15T22:11:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.223+0000 7ff901d9b640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff8ec0a3500 0x7ff8ec0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.223+0000 7ff90259c640 1 --2- 172.21.15.5:0/2545081087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671e970 0x55e27671edd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800ba80 tx=0x7ff8f800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3bb33735faa6f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.224+0000 7ff901d9b640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff8ec0a3500 0x7ff8ec0a59c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4005c40 tx=0x7ff8f4005d80 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.226+0000 7ff908fda640 1 -- 172.21.15.5:0/2545081087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8016070 con 0x55e27671e970 2024-09-15T22:11:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.226+0000 7ff908fda640 1 -- 172.21.15.5:0/2545081087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8f8005500 con 0x55e27671e970 2024-09-15T22:11:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.227+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2545081087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671e970 msgr2=0x55e27671edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.227+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2545081087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671e970 0x55e27671edd0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800ba80 tx=0x7ff8f800bf40 comp rx=0 tx=0).stop 2024-09-15T22:11:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.228+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2545081087 shutdown_connections 2024-09-15T22:11:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.228+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2545081087 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27673a440 0x55e27673c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.228+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2545081087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671e970 0x55e27671edd0 secure :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff8f800ba80 tx=0x7ff8f800bf40 comp rx=0 tx=0).stop 2024-09-15T22:11:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.228+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2545081087 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e27671e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.228+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2545081087 >> 172.21.15.5:0/2545081087 conn(0x55e27671ad20 msgr2=0x55e27671b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.229+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2545081087 shutdown_connections 2024-09-15T22:11:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.229+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2545081087 wait complete. 2024-09-15T22:11:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.230+0000 7ff90b3b3740 1 -- start start 2024-09-15T22:11:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.230+0000 7ff90b3b3740 1 Processor -- start 2024-09-15T22:11:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.230+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e2767372f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.230+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 0x55e276737830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.230+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e276737d70 0x55e2767467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.230+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e27673d280 con 0x55e27673a440 2024-09-15T22:11:38.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.230+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e27663f050 con 0x55e27671fd70 2024-09-15T22:11:38.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.230+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e27673d400 con 0x55e276737d70 2024-09-15T22:11:38.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.231+0000 7ff90b3b3740 10 client.0 _ll_register_callbacks cb 0x55e27667f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:38.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.232+0000 7ff90259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e2767372f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.232+0000 7ff901d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 0x55e276737830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.232+0000 7ff90259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e2767372f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33946/0 (socket says 172.21.15.5:33946) 2024-09-15T22:11:38.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.232+0000 7ff90259c640 1 -- 172.21.15.5:0/3973942332 learned_addr learned my addr 172.21.15.5:0/3973942332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.236+0000 7ff901d9b640 1 -- 172.21.15.5:0/3973942332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e276737d70 msgr2=0x55e2767467b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:11:38.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.236+0000 7ff901d9b640 1 --2- 172.21.15.5:0/3973942332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e276737d70 0x55e2767467b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.236+0000 7ff901d9b640 1 -- 172.21.15.5:0/3973942332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 msgr2=0x55e2767372f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.236+0000 7ff901d9b640 1 --2- 172.21.15.5:0/3973942332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e2767372f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.236+0000 7ff901d9b640 1 -- 172.21.15.5:0/3973942332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2763f53c0 con 0x55e27673a440 2024-09-15T22:11:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.236+0000 7ff901d9b640 1 --2- 172.21.15.5:0/3973942332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 0x55e276737830 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4005e30 tx=0x7ff8f4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e69a933bc219a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.237+0000 7ff8e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.241+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/3973942332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f401bde0 con 0x55e27673a440 2024-09-15T22:11:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.241+0000 7ff90b3b3740 1 -- 172.21.15.5:0/3973942332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e276691850 con 0x55e27673a440 2024-09-15T22:11:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.242+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/3973942332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7ff8f4003970 con 0x55e27673a440 2024-09-15T22:11:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.243+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/3973942332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8f402e050 con 0x55e27673a440 2024-09-15T22:11:38.245 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.245+0000 7ff90b3b3740 10 client.15927 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.245+0000 7ff90b3b3740 1 -- 172.21.15.5:0/3973942332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2767493e0 con 0x55e27673a440 2024-09-15T22:11:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.245+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/3973942332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e276691850 con 0x55e27673a440 2024-09-15T22:11:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.245+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/3973942332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff8f4017040 con 0x55e27673a440 2024-09-15T22:11:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.246+0000 7ff90b3b3740 10 client.15927 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.246+0000 7ff90b3b3740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.246+0000 7ff90b3b3740 20 client.15927 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.246+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/3973942332 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e27674ace0 0x55e27674d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.246+0000 7ff90b3b3740 4 client.15927 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:38.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.246+0000 7ff90b3b3740 1 -- 172.21.15.5:0/3973942332 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e27668bd60 con 0x55e27674ace0 2024-09-15T22:11:38.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.246+0000 7ff90259c640 1 --2- 172.21.15.5:0/3973942332 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e27674ace0 0x55e27674d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.252+0000 7ff8e9ffb640 10 client.15927 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:38.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.252+0000 7ff90259c640 1 --2- 172.21.15.5:0/3973942332 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e27674ace0 0x55e27674d160 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66e966b0af05b253 server_cookie=a2647d365d21875f in_seq=0 out_seq=0 2024-09-15T22:11:38.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.253+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/3973942332 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e27668bd60 con 0x55e27674ace0 2024-09-15T22:11:38.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.253+0000 7ff8e9ffb640 10 client.15927 handle_command_reply: tid=0 2024-09-15T22:11:38.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.253+0000 7ff90b3b3740 1 client.15927 shutdown 2024-09-15T22:11:38.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.253+0000 7ff90b3b3740 1 -- 172.21.15.5:0/3973942332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 msgr2=0x55e276737830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:38.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.253+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/3973942332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 0x55e276737830 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4005e30 tx=0x7ff8f4005bd0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.254+0000 7ff90b3b3740 1 -- 172.21.15.5:0/3973942332 shutdown_connections 2024-09-15T22:11:38.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.254+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/3973942332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e276737d70 0x55e2767467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.254+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/3973942332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 0x55e276737830 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.254+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/3973942332 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e27674ace0 0x55e27674d160 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.254+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/3973942332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e2767372f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.254+0000 7ff90b3b3740 1 -- 172.21.15.5:0/3973942332 >> 172.21.15.5:0/3973942332 conn(0x55e27671ad20 msgr2=0x55e276720a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:38.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.254+0000 7ff8e9ffb640 0 client.15927 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:38.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.254+0000 7ff90b3b3740 1 -- 172.21.15.5:0/3973942332 shutdown_connections 2024-09-15T22:11:38.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.255+0000 7ff90b3b3740 1 -- 172.21.15.5:0/3973942332 wait complete. 2024-09-15T22:11:38.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.255+0000 7ff90b3b3740 20 client.15927 trim_cache size 0 max 16384 2024-09-15T22:11:38.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.285+0000 7ff90b3b3740 1 Processor -- start 2024-09-15T22:11:38.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.285+0000 7ff90b3b3740 1 -- start start 2024-09-15T22:11:38.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.286+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671fd70 0x55e27671a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.286+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e276737d70 0x55e276739260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.286+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27673a440 0x55e2767465a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.286+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041af800 con 0x55e27671fd70 2024-09-15T22:11:38.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.286+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041ad0d0 con 0x55e276737d70 2024-09-15T22:11:38.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.286+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff90419fd50 con 0x55e27673a440 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.286+0000 7ff90259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671fd70 0x55e27671a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.286+0000 7ff90259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671fd70 0x55e27671a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32784/0 (socket says 172.21.15.5:32784) 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.286+0000 7ff90259c640 1 -- 172.21.15.5:0/2288480022 learned_addr learned my addr 172.21.15.5:0/2288480022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.286+0000 7ff901d9b640 1 --2- 172.21.15.5:0/2288480022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e276737d70 0x55e276739260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.286+0000 7ff902d9d640 1 --2- 172.21.15.5:0/2288480022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27673a440 0x55e2767465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.287+0000 7ff902d9d640 1 -- 172.21.15.5:0/2288480022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e276737d70 msgr2=0x55e276739260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.287+0000 7ff902d9d640 1 --2- 172.21.15.5:0/2288480022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e276737d70 0x55e276739260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.287+0000 7ff902d9d640 1 -- 172.21.15.5:0/2288480022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671fd70 msgr2=0x55e27671a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.287+0000 7ff902d9d640 1 --2- 172.21.15.5:0/2288480022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671fd70 0x55e27671a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.287+0000 7ff902d9d640 1 -- 172.21.15.5:0/2288480022 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27673cba0 con 0x55e27673a440 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.287+0000 7ff902d9d640 1 --2- 172.21.15.5:0/2288480022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27673a440 0x55e2767465a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0edfe0 tx=0x7ff8fc005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94b987e06e2b8c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.287+0000 7ff8ebfff640 1 -- 172.21.15.5:0/2288480022 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc044ea0 con 0x55e27673a440 2024-09-15T22:11:38.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.287+0000 7ff8ebfff640 1 -- 172.21.15.5:0/2288480022 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8fc0d4030 con 0x55e27673a440 2024-09-15T22:11:38.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.287+0000 7ff8ebfff640 1 -- 172.21.15.5:0/2288480022 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc0b91e0 con 0x55e27673a440 2024-09-15T22:11:38.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.288+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2288480022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27673a440 msgr2=0x55e2767465a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:38.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.288+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2288480022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27673a440 0x55e2767465a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc0edfe0 tx=0x7ff8fc005750 comp rx=0 tx=0).stop 2024-09-15T22:11:38.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.288+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2288480022 shutdown_connections 2024-09-15T22:11:38.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.288+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2288480022 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e27673a440 0x55e2767465a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.289+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2288480022 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e276737d70 0x55e276739260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.289+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2288480022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27671fd70 0x55e27671a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.289+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2288480022 >> 172.21.15.5:0/2288480022 conn(0x55e27671ad20 msgr2=0x55e276738b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:38.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.290+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2288480022 shutdown_connections 2024-09-15T22:11:38.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.290+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2288480022 wait complete. 2024-09-15T22:11:38.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.291+0000 7ff90b3b3740 1 -- start start 2024-09-15T22:11:38.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.291+0000 7ff90b3b3740 1 Processor -- start 2024-09-15T22:11:38.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.292+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e276746390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.292+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e276737d70 0x55e2767468d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.292+0000 7ff90b3b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 0x55e276746e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.292+0000 7ff90259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e276746390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.292+0000 7ff90259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e276746390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33960/0 (socket says 172.21.15.5:33960) 2024-09-15T22:11:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.293+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041ab6d0 con 0x55e27673a440 2024-09-15T22:11:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.293+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041aca00 con 0x55e27671fd70 2024-09-15T22:11:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.293+0000 7ff90b3b3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff9041abc60 con 0x55e276737d70 2024-09-15T22:11:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.293+0000 7ff90b3b3740 10 client.0 _ll_register_callbacks cb 0x7ff8f4010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.293+0000 7ff902d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 0x55e276746e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.293+0000 7ff902d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 0x55e276746e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32800/0 (socket says 172.21.15.5:32800) 2024-09-15T22:11:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.293+0000 7ff902d9d640 1 -- 172.21.15.5:0/2203590734 learned_addr learned my addr 172.21.15.5:0/2203590734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:38.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.293+0000 7ff901d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e276737d70 0x55e2767468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.294+0000 7ff901d9b640 1 -- 172.21.15.5:0/2203590734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 msgr2=0x55e276746390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.294+0000 7ff901d9b640 1 --2- 172.21.15.5:0/2203590734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e276746390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.294+0000 7ff901d9b640 1 -- 172.21.15.5:0/2203590734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 msgr2=0x55e276746e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.294+0000 7ff901d9b640 1 --2- 172.21.15.5:0/2203590734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 0x55e276746e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.294+0000 7ff901d9b640 1 -- 172.21.15.5:0/2203590734 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e276740f80 con 0x55e276737d70 2024-09-15T22:11:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.296+0000 7ff901d9b640 1 --2- 172.21.15.5:0/2203590734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e276737d70 0x55e2767468d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4023850 tx=0x7ff8f40235f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c492b835e895ee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:38.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.296+0000 7ff8e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:11:38.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.296+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/2203590734 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4038020 con 0x55e276737d70 2024-09-15T22:11:38.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.296+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/2203590734 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7ff8f4023ae0 con 0x55e276737d70 2024-09-15T22:11:38.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.297+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2203590734 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8fc0d4030 con 0x55e276737d70 2024-09-15T22:11:38.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.297+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/2203590734 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8f4034440 con 0x55e276737d70 2024-09-15T22:11:38.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.297+0000 7ff90b3b3740 10 client.25525 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:38.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.297+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2203590734 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e2767b5ec0 con 0x55e276737d70 2024-09-15T22:11:38.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.300+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/2203590734 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8f4034ac0 con 0x55e276737d70 2024-09-15T22:11:38.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.300+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/2203590734 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff8f4017040 con 0x55e276737d70 2024-09-15T22:11:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.300+0000 7ff90b3b3740 10 client.25525 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.300+0000 7ff90b3b3740 10 client.25525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.300+0000 7ff90b3b3740 20 client.25525 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.300+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2203590734 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e2767b6560 0x55e2767b8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.300+0000 7ff90b3b3740 4 client.25525 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:11:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.300+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2203590734 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff8fc0d4030 con 0x55e2767b6560 2024-09-15T22:11:38.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.301+0000 7ff90259c640 1 --2- 172.21.15.5:0/2203590734 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e2767b6560 0x55e2767b8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:38.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.301+0000 7ff90259c640 1 --2- 172.21.15.5:0/2203590734 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e2767b6560 0x55e2767b8980 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a873b249e5bc708 server_cookie=2a7b830a7bc2ef69 in_seq=0 out_seq=0 2024-09-15T22:11:38.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.301+0000 7ff8e9ffb640 10 client.25525 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:38.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.301+0000 7ff8e9ffb640 1 -- 172.21.15.5:0/2203590734 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7ff8fc0d4030 con 0x55e2767b6560 2024-09-15T22:11:38.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.301+0000 7ff8e9ffb640 10 client.25525 handle_command_reply: tid=0 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.301+0000 7ff90b3b3740 1 client.25525 shutdown 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.301+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2203590734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e276737d70 msgr2=0x55e2767468d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.301+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2203590734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e276737d70 0x55e2767468d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4023850 tx=0x7ff8f40235f0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.301+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2203590734 shutdown_connections 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.301+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2203590734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e27673a440 0x55e276746e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.301+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2203590734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e276737d70 0x55e2767468d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.302+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2203590734 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e2767b6560 0x55e2767b8980 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.302+0000 7ff90b3b3740 1 --2- 172.21.15.5:0/2203590734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e27671fd70 0x55e276746390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.302+0000 7ff8e9ffb640 0 client.25525 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.302+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2203590734 >> 172.21.15.5:0/2203590734 conn(0x55e27671ad20 msgr2=0x55e276720a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.302+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2203590734 shutdown_connections 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.302+0000 7ff90b3b3740 1 -- 172.21.15.5:0/2203590734 wait complete. 2024-09-15T22:11:38.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.302+0000 7ff90b3b3740 20 client.25525 trim_cache size 0 max 16384 2024-09-15T22:11:38.305 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:11:38.305 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:11:38.305 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:11:38.305 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:11:38.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.303+0000 7ff8f17fa640 1 -- 172.21.15.5:0/1510329681 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff8ec0a3500 msgr2=0x7ff8ec0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:38.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.303+0000 7ff8f17fa640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff8ec0a3500 0x7ff8ec0a59c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff8f4005c40 tx=0x7ff8f4005d80 comp rx=0 tx=0).stop 2024-09-15T22:11:38.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.303+0000 7ff8f17fa640 1 -- 172.21.15.5:0/1510329681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff904112b90 msgr2=0x7ff90419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:38.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.303+0000 7ff8f17fa640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff904112b90 0x7ff90419e820 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc005040 tx=0x7ff8fc005070 comp rx=0 tx=0).stop 2024-09-15T22:11:38.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.303+0000 7ff8f17fa640 1 -- 172.21.15.5:0/1510329681 shutdown_connections 2024-09-15T22:11:38.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.303+0000 7ff8f17fa640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff8ec0a3500 0x7ff8ec0a59c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.303+0000 7ff8f17fa640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff904112b90 0x7ff90419e820 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.303+0000 7ff8f17fa640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff904074eb0 0x7ff9041a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.303+0000 7ff8f17fa640 1 --2- 172.21.15.5:0/1510329681 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff904074590 0x7ff9041a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:38.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.303+0000 7ff8f17fa640 1 -- 172.21.15.5:0/1510329681 >> 172.21.15.5:0/1510329681 conn(0x7ff9040fd8e0 msgr2=0x7ff9040fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:38.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.304+0000 7ff8f17fa640 1 -- 172.21.15.5:0/1510329681 shutdown_connections 2024-09-15T22:11:38.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:38.304+0000 7ff8f17fa640 1 -- 172.21.15.5:0/1510329681 wait complete. 2024-09-15T22:11:39.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:39 smithi005 ceph-mon[30944]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.9 MiB/s wr, 702 op/s 2024-09-15T22:11:39.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:39 smithi099 ceph-mon[34892]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.9 MiB/s wr, 702 op/s 2024-09-15T22:11:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:39 smithi129 ceph-mon[35314]: pgmap v85: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 7.9 MiB/s wr, 702 op/s 2024-09-15T22:11:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 130.0 s, 2439.0 tps, lat 40.794 ms stddev 47.750 2024-09-15T22:11:40.407 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:11:40.407 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:11:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.676+0000 7f1cfa40b640 1 -- 172.21.15.5:0/4195631995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cf4076720 msgr2=0x7f1cf4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.676+0000 7f1cfa40b640 1 --2- 172.21.15.5:0/4195631995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cf4076720 0x7f1cf4076b00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009860 tx=0x7f1ce402a340 comp rx=0 tx=0).stop 2024-09-15T22:11:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.676+0000 7f1cfa40b640 1 -- 172.21.15.5:0/4195631995 shutdown_connections 2024-09-15T22:11:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.676+0000 7f1cfa40b640 1 --2- 172.21.15.5:0/4195631995 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cf4112a30 0x7f1cf4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.676+0000 7f1cfa40b640 1 --2- 172.21.15.5:0/4195631995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf4077040 0x7f1cf41124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.676+0000 7f1cfa40b640 1 --2- 172.21.15.5:0/4195631995 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cf4076720 0x7f1cf4076b00 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009860 tx=0x7f1ce402a340 comp rx=0 tx=0).stop 2024-09-15T22:11:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.676+0000 7f1cfa40b640 1 -- 172.21.15.5:0/4195631995 >> 172.21.15.5:0/4195631995 conn(0x7f1cf40ffa90 msgr2=0x7f1cf4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.676+0000 7f1cfa40b640 1 -- 172.21.15.5:0/4195631995 shutdown_connections 2024-09-15T22:11:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.676+0000 7f1cfa40b640 1 -- 172.21.15.5:0/4195631995 wait complete. 2024-09-15T22:11:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.677+0000 7f1cfa40b640 1 Processor -- start 2024-09-15T22:11:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.677+0000 7f1cfa40b640 1 -- start start 2024-09-15T22:11:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.678+0000 7f1cfa40b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cf4077040 0x7f1cf41a4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.678+0000 7f1cfa40b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cf4112a30 0x7f1cf41a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.678+0000 7f1cfa40b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf419eb00 0x7f1cf419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.678+0000 7f1cfa40b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41185c0 con 0x7f1cf419eb00 2024-09-15T22:11:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.678+0000 7f1cfa40b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4118440 con 0x7f1cf4112a30 2024-09-15T22:11:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.678+0000 7f1cfa40b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4118740 con 0x7f1cf4077040 2024-09-15T22:11:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.678+0000 7f1cf37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cf4112a30 0x7f1cf41a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.678+0000 7f1cf37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cf4112a30 0x7f1cf41a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33984/0 (socket says 172.21.15.5:33984) 2024-09-15T22:11:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.678+0000 7f1cf37fe640 1 -- 172.21.15.5:0/749305283 learned_addr learned my addr 172.21.15.5:0/749305283 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.678+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cf4077040 0x7f1cf41a4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.678+0000 7f1cf89a8640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf419eb00 0x7f1cf419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.679+0000 7f1cf37fe640 1 -- 172.21.15.5:0/749305283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cf4077040 msgr2=0x7f1cf41a4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.679+0000 7f1cf37fe640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cf4077040 0x7f1cf41a4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.679+0000 7f1cf37fe640 1 -- 172.21.15.5:0/749305283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf419eb00 msgr2=0x7f1cf419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.679+0000 7f1cf37fe640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf419eb00 0x7f1cf419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.679+0000 7f1cf37fe640 1 -- 172.21.15.5:0/749305283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf419f620 con 0x7f1cf4112a30 2024-09-15T22:11:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.679+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cf4077040 0x7f1cf41a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.679+0000 7f1cf37fe640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cf4112a30 0x7f1cf41a4e80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800e9f0 tx=0x7f1ce800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.679+0000 7f1cf17fa640 1 -- 172.21.15.5:0/749305283 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce800ccc0 con 0x7f1cf4112a30 2024-09-15T22:11:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.679+0000 7f1cf17fa640 1 -- 172.21.15.5:0/749305283 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce8005500 con 0x7f1cf4112a30 2024-09-15T22:11:40.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.680+0000 7f1cf17fa640 1 -- 172.21.15.5:0/749305283 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8010690 con 0x7f1cf4112a30 2024-09-15T22:11:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.681+0000 7f1cfa40b640 1 -- 172.21.15.5:0/749305283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf419f910 con 0x7f1cf4112a30 2024-09-15T22:11:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.681+0000 7f1cfa40b640 1 -- 172.21.15.5:0/749305283 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf41abb00 con 0x7f1cf4112a30 2024-09-15T22:11:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.684+0000 7f1cfc7e4740 1 Processor -- start 2024-09-15T22:11:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.685+0000 7f1cfc7e4740 1 -- start start 2024-09-15T22:11:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.685+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876831850 0x55f876831c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.685+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f87683c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.685+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f87683ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.685+0000 7f1cfc7e4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f876796160 con 0x55f876831850 2024-09-15T22:11:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.685+0000 7f1cfc7e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f87673d720 con 0x55f87683ca30 2024-09-15T22:11:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.685+0000 7f1cfc7e4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f876792120 con 0x55f876832170 2024-09-15T22:11:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.685+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f87683c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.685+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f87683c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:52992/0 (socket says 172.21.15.5:52992) 2024-09-15T22:11:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.685+0000 7f1cf3fff640 1 -- 172.21.15.5:0/778608957 learned_addr learned my addr 172.21.15.5:0/778608957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:40.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.686+0000 7f1cf3fff640 1 -- 172.21.15.5:0/778608957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 msgr2=0x55f87683ee20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:11:40.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.686+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/778608957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f87683ee20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.686+0000 7f1cf3fff640 1 -- 172.21.15.5:0/778608957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876831850 msgr2=0x55f876831c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:40.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.686+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/778608957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876831850 0x55f876831c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.686+0000 7f1cf3fff640 1 -- 172.21.15.5:0/778608957 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f876507ba0 con 0x55f876832170 2024-09-15T22:11:40.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.686+0000 7f1cf17fa640 1 -- 172.21.15.5:0/749305283 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1ce8004680 con 0x7f1cf4112a30 2024-09-15T22:11:40.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.687+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/778608957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f87683c4f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4005e00 tx=0x7f1ce4030340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=880c543ca2ada615 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:40.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.687+0000 7f1cf17fa640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1cdc0a3460 0x7f1cdc0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.688+0000 7f1cf17fa640 1 -- 172.21.15.5:0/749305283 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f1ce80d0410 con 0x7f1cf4112a30 2024-09-15T22:11:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.688+0000 7f1cf89a8640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1cdc0a3460 0x7f1cdc0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.688+0000 7f1cda7fc640 1 -- 172.21.15.5:0/778608957 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4041020 con 0x55f876832170 2024-09-15T22:11:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.689+0000 7f1cda7fc640 1 -- 172.21.15.5:0/778608957 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce40042c0 con 0x55f876832170 2024-09-15T22:11:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.689+0000 7f1cda7fc640 1 -- 172.21.15.5:0/778608957 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4037040 con 0x55f876832170 2024-09-15T22:11:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.690+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/778608957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 msgr2=0x55f87683c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.690+0000 7f1cf89a8640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1cdc0a3460 0x7f1cdc0a5920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40697e0 tx=0x7f1cec00a2d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.690+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/778608957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f87683c4f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4005e00 tx=0x7f1ce4030340 comp rx=0 tx=0).stop 2024-09-15T22:11:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.690+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/778608957 shutdown_connections 2024-09-15T22:11:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.690+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/778608957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f87683ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.690+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/778608957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f87683c4f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.690+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/778608957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876831850 0x55f876831c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.690+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/778608957 >> 172.21.15.5:0/778608957 conn(0x55f87682dde0 msgr2=0x55f87682e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.690+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/778608957 shutdown_connections 2024-09-15T22:11:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.690+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/778608957 wait complete. 2024-09-15T22:11:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.690+0000 7f1cfc7e4740 1 -- start start 2024-09-15T22:11:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cfc7e4740 1 Processor -- start 2024-09-15T22:11:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 0x55f876839aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876832170 0x55f876839fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f87683a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cfc7e4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f876850610 con 0x55f876832170 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cfc7e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f87671e900 con 0x55f87683ca30 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cfc7e4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f876850790 con 0x55f876831850 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cfc7e4740 10 client.0 _ll_register_callbacks cb 0x55f87671c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 0x55f876839aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cf3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 0x55f876839aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:52996/0 (socket says 172.21.15.5:52996) 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cf3fff640 1 -- 172.21.15.5:0/2544383391 learned_addr learned my addr 172.21.15.5:0/2544383391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.691+0000 7f1cf89a8640 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f87683a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.692+0000 7f1cf37fe640 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876832170 0x55f876839fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.692+0000 7f1cf3fff640 1 -- 172.21.15.5:0/2544383391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 msgr2=0x55f87683a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.692+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f87683a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.692+0000 7f1cf3fff640 1 -- 172.21.15.5:0/2544383391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876832170 msgr2=0x55f876839fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.692+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876832170 0x55f876839fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.692+0000 7f1cf3fff640 1 -- 172.21.15.5:0/2544383391 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f87675ec50 con 0x55f876831850 2024-09-15T22:11:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.692+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 0x55f876839aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009780 tx=0x7f1ce4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60aa4d8896e8ee5b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.692+0000 7f1cd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:11:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.692+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2544383391 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce404c070 con 0x55f876831850 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.692+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2544383391 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f1ce4003d10 con 0x55f876831850 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.693+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2544383391 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce40042c0 con 0x55f876831850 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.693+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2544383391 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ce4058050 con 0x55f876831850 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.693+0000 7f1cfc7e4740 10 client.25537 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.693+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2544383391 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f876850610 con 0x55f876831850 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.693+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2544383391 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce40042c0 con 0x55f876831850 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.694+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2544383391 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1ce403db70 con 0x55f876831850 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.694+0000 7f1cfc7e4740 10 client.25537 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.694+0000 7f1cfc7e4740 10 client.25537 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.694+0000 7f1cfc7e4740 20 client.25537 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.694+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f87685d9a0 0x55f87685fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.694+0000 7f1cfc7e4740 4 client.25537 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.694+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2544383391 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8767a3890 con 0x55f87685d9a0 2024-09-15T22:11:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.694+0000 7f1cf37fe640 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f87685d9a0 0x55f87685fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.695+0000 7f1cd37fe640 10 client.25537 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.695+0000 7f1cf37fe640 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f87685d9a0 0x55f87685fe20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78b38e7a45d1a2f7 server_cookie=1c0fe6fabfcb93fe in_seq=0 out_seq=0 2024-09-15T22:11:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.696+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2544383391 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f8767a3890 con 0x55f87685d9a0 2024-09-15T22:11:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.696+0000 7f1cd37fe640 10 client.25537 handle_command_reply: tid=0 2024-09-15T22:11:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.696+0000 7f1cfc7e4740 1 client.25537 shutdown 2024-09-15T22:11:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.696+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2544383391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 msgr2=0x55f876839aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.696+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 0x55f876839aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4009780 tx=0x7f1ce4005850 comp rx=0 tx=0).stop 2024-09-15T22:11:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.698+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2544383391 shutdown_connections 2024-09-15T22:11:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.698+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f87685d9a0 0x55f87685fe20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.698+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f87683a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.698+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876832170 0x55f876839fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.698+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2544383391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 0x55f876839aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.698+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2544383391 >> 172.21.15.5:0/2544383391 conn(0x55f87682dde0 msgr2=0x55f876833b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:40.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.700+0000 7f1cd37fe640 0 client.25537 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:40.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.700+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2544383391 shutdown_connections 2024-09-15T22:11:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.701+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2544383391 wait complete. 2024-09-15T22:11:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.702+0000 7f1cfc7e4740 20 client.25537 trim_cache size 0 max 16384 2024-09-15T22:11:40.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.751+0000 7f1cfc7e4740 1 Processor -- start 2024-09-15T22:11:40.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.751+0000 7f1cfc7e4740 1 -- start start 2024-09-15T22:11:40.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.752+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 0x55f87683a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.752+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876832170 0x55f8768339b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.752+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f8768656f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.752+0000 7f1cfc7e4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41af990 con 0x55f876832170 2024-09-15T22:11:40.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.752+0000 7f1cfc7e4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41ad260 con 0x55f87683ca30 2024-09-15T22:11:40.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.752+0000 7f1cfc7e4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf40736d0 con 0x55f876831850 2024-09-15T22:11:40.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.753+0000 7f1cf37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876832170 0x55f8768339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.753+0000 7f1cf37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876832170 0x55f8768339b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32856/0 (socket says 172.21.15.5:32856) 2024-09-15T22:11:40.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.753+0000 7f1cf37fe640 1 -- 172.21.15.5:0/1454770362 learned_addr learned my addr 172.21.15.5:0/1454770362 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:40.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.753+0000 7f1cf89a8640 1 --2- 172.21.15.5:0/1454770362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f8768656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.753+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/1454770362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 0x55f87683a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.753+0000 7f1cf89a8640 1 -- 172.21.15.5:0/1454770362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 msgr2=0x55f87683a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:40.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.753+0000 7f1cf89a8640 1 --2- 172.21.15.5:0/1454770362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 0x55f87683a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.753+0000 7f1cf89a8640 1 -- 172.21.15.5:0/1454770362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876832170 msgr2=0x55f8768339b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:40.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.754+0000 7f1cf89a8640 1 --2- 172.21.15.5:0/1454770362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876832170 0x55f8768339b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.754+0000 7f1cf89a8640 1 -- 172.21.15.5:0/1454770362 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f87683f190 con 0x55f87683ca30 2024-09-15T22:11:40.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.754+0000 7f1cf89a8640 1 --2- 172.21.15.5:0/1454770362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f8768656f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55f87683ca00 tx=0x7f1cec011000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89f80303cb3e0293 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:40.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.754+0000 7f1cd9ffb640 1 -- 172.21.15.5:0/1454770362 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec01e840 con 0x55f87683ca30 2024-09-15T22:11:40.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.754+0000 7f1cd9ffb640 1 -- 172.21.15.5:0/1454770362 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cec022070 con 0x55f87683ca30 2024-09-15T22:11:40.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.754+0000 7f1cd9ffb640 1 -- 172.21.15.5:0/1454770362 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cec01d430 con 0x55f87683ca30 2024-09-15T22:11:40.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.755+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/1454770362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 msgr2=0x55f8768656f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:40.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.755+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/1454770362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f8768656f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x55f87683ca00 tx=0x7f1cec011000 comp rx=0 tx=0).stop 2024-09-15T22:11:40.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.756+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/1454770362 shutdown_connections 2024-09-15T22:11:40.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.756+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/1454770362 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f8768656f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.756+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/1454770362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876832170 0x55f8768339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.756+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/1454770362 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876831850 0x55f87683a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.756+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/1454770362 >> 172.21.15.5:0/1454770362 conn(0x55f87682dde0 msgr2=0x55f8768332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:40.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.758+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/1454770362 shutdown_connections 2024-09-15T22:11:40.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.758+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/1454770362 wait complete. 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.758+0000 7f1cfc7e4740 1 -- start start 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.759+0000 7f1cfc7e4740 1 Processor -- start 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.759+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876831850 0x55f876855fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.760+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f876836660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.760+0000 7f1cfc7e4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f876836ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.760+0000 7f1cf37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f876836660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.760+0000 7f1cf37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f876836660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53026/0 (socket says 172.21.15.5:53026) 2024-09-15T22:11:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.760+0000 7f1cf89a8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f876836ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.760+0000 7f1cf89a8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f876836ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:34018/0 (socket says 172.21.15.5:34018) 2024-09-15T22:11:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.760+0000 7f1cf37fe640 1 -- 172.21.15.5:0/2884478135 learned_addr learned my addr 172.21.15.5:0/2884478135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.761+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2884478135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf4073d40 con 0x55f876831850 2024-09-15T22:11:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.761+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2884478135 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41acb90 con 0x55f87683ca30 2024-09-15T22:11:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.761+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2884478135 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf41abdf0 con 0x55f876832170 2024-09-15T22:11:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.761+0000 7f1cfc7e4740 10 client.0 _ll_register_callbacks cb 0x7f1ce40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.762+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876831850 0x55f876855fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.763+0000 7f1cf3fff640 1 -- 172.21.15.5:0/2884478135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 msgr2=0x55f876836660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:40.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.763+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f876836660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.763+0000 7f1cf3fff640 1 -- 172.21.15.5:0/2884478135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 msgr2=0x55f876836ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:40.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.763+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f876836ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:40.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.763+0000 7f1cf3fff640 1 -- 172.21.15.5:0/2884478135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f876837240 con 0x55f876831850 2024-09-15T22:11:40.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.763+0000 7f1cf3fff640 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876831850 0x55f876855fb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4000c00 tx=0x7f1ce4030780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2cd4b47fd09880 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:40.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.764+0000 7f1cf37fe640 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f876836660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:40.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.764+0000 7f1cf89a8640 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f876836ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.769+0000 7f1cd37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.769+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2884478135 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce404c040 con 0x55f876831850 2024-09-15T22:11:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.769+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2884478135 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f1ce4003d10 con 0x55f876831850 2024-09-15T22:11:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.769+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2884478135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cec022070 con 0x55f876831850 2024-09-15T22:11:40.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.770+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2884478135 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1ce4041050 con 0x55f876831850 2024-09-15T22:11:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.771+0000 7f1cfc7e4740 10 client.15948 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.771+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2884478135 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f8768c8530 con 0x55f876831850 2024-09-15T22:11:40.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.776+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2884478135 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce40043f0 con 0x55f876831850 2024-09-15T22:11:40.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.776+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2884478135 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1ce403d730 con 0x55f876831850 2024-09-15T22:11:40.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.776+0000 7f1cfc7e4740 10 client.15948 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:40.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.776+0000 7f1cfc7e4740 10 client.15948 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:11:40.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.776+0000 7f1cfc7e4740 20 client.15948 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.776+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f8768c8c30 0x55f8768cb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.776+0000 7f1cfc7e4740 4 client.15948 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}] 2024-09-15T22:11:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.776+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2884478135 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.55, "await_for": 45.0}) -- 0x7f1cec022070 con 0x55f8768c8c30 2024-09-15T22:11:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.777+0000 7f1cf37fe640 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f8768c8c30 0x55f8768cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.778+0000 7f1cd37fe640 10 client.15948 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:40.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:40.778+0000 7f1cf37fe640 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f8768c8c30 0x55f8768cb050 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7ea286d5c61acb server_cookie=7ba87709f39e4dfa in_seq=0 out_seq=0 2024-09-15T22:11:40.842 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.035+0000 7f1cd37fe640 1 -- 172.21.15.5:0/2884478135 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f1cec022070 con 0x55f8768c8c30 2024-09-15T22:11:41.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.035+0000 7f1cd37fe640 10 client.15948 handle_command_reply: tid=0 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.035+0000 7f1cfc7e4740 1 client.15948 shutdown 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2884478135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876831850 msgr2=0x55f876855fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876831850 0x55f876855fb0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4000c00 tx=0x7f1ce4030780 comp rx=0 tx=0).stop 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2884478135 shutdown_connections 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f8768c8c30 0x55f8768cb050 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f87683ca30 0x55f876836ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f876832170 0x55f876836660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cd37fe640 0 client.15948 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cfc7e4740 1 --2- 172.21.15.5:0/2884478135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f876831850 0x55f876855fb0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2884478135 >> 172.21.15.5:0/2884478135 conn(0x55f87682dde0 msgr2=0x55f876833ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2884478135 shutdown_connections 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cfc7e4740 1 -- 172.21.15.5:0/2884478135 wait complete. 2024-09-15T22:11:41.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.036+0000 7f1cfc7e4740 20 client.15948 trim_cache size 0 max 16384 2024-09-15T22:11:41.039 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:11:41.039 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:11:41.039 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:11:41.039 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 10, 2024-09-15T22:11:41.039 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:11:41.039 INFO:teuthology.orchestra.run.smithi005.stdout: "ffcb6d65": { 2024-09-15T22:11:41.039 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 10, 2024-09-15T22:11:41.039 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:11:41.039 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:11:41.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.6, 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:41.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:41.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:41.041 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:11:41.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 -- 172.21.15.5:0/749305283 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1cdc0a3460 msgr2=0x7f1cdc0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:41.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1cdc0a3460 0x7f1cdc0a5920 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40697e0 tx=0x7f1cec00a2d0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 -- 172.21.15.5:0/749305283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cf4112a30 msgr2=0x7f1cf41a4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:41.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cf4112a30 0x7f1cf41a4e80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800e9f0 tx=0x7f1ce800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 -- 172.21.15.5:0/749305283 shutdown_connections 2024-09-15T22:11:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1cdc0a3460 0x7f1cdc0a5920 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cf419eb00 0x7f1cf419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1cf4112a30 0x7f1cf41a4e80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 --2- 172.21.15.5:0/749305283 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1cf4077040 0x7f1cf41a4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 -- 172.21.15.5:0/749305283 >> 172.21.15.5:0/749305283 conn(0x7f1cf40ffa90 msgr2=0x7f1cf41147f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 -- 172.21.15.5:0/749305283 shutdown_connections 2024-09-15T22:11:41.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.037+0000 7f1cfa40b640 1 -- 172.21.15.5:0/749305283 wait complete. 2024-09-15T22:11:41.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:41 smithi005 ceph-mon[30944]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.6 MiB/s wr, 783 op/s 2024-09-15T22:11:41.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:41 smithi099 ceph-mon[34892]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.6 MiB/s wr, 783 op/s 2024-09-15T22:11:41.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:41 smithi129 ceph-mon[35314]: pgmap v86: 129 pgs: 129 active+clean; 1.6 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 8.6 MiB/s wr, 783 op/s 2024-09-15T22:11:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.609+0000 7f2aa5325640 1 -- 172.21.15.5:0/3117480294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa0102d30 msgr2=0x7f2aa0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.609+0000 7f2aa5325640 1 --2- 172.21.15.5:0/3117480294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa0102d30 0x7f2aa0103110 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2a90009cd0 tx=0x7f2a9002a8e0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.609+0000 7f2aa5325640 1 -- 172.21.15.5:0/3117480294 shutdown_connections 2024-09-15T22:11:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.610+0000 7f2aa5325640 1 --2- 172.21.15.5:0/3117480294 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2aa01082a0 0x7f2aa010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.610+0000 7f2aa5325640 1 --2- 172.21.15.5:0/3117480294 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa01036e0 0x7f2aa0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.610+0000 7f2aa5325640 1 --2- 172.21.15.5:0/3117480294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa0102d30 0x7f2aa0103110 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.610+0000 7f2aa5325640 1 -- 172.21.15.5:0/3117480294 >> 172.21.15.5:0/3117480294 conn(0x7f2aa00fc7d0 msgr2=0x7f2aa00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.610+0000 7f2aa5325640 1 -- 172.21.15.5:0/3117480294 shutdown_connections 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.610+0000 7f2aa5325640 1 -- 172.21.15.5:0/3117480294 wait complete. 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.610+0000 7f2aa5325640 1 Processor -- start 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.610+0000 7f2aa5325640 1 -- start start 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2aa5325640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa0102d30 0x7f2aa01a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2aa5325640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2aa01036e0 0x7f2aa01a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2aa5325640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa01082a0 0x7f2aa019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2aa5325640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa01120a0 con 0x7f2aa01082a0 2024-09-15T22:11:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2aa5325640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0111f20 con 0x7f2aa0102d30 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2aa5325640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0112220 con 0x7f2aa01036e0 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa01082a0 0x7f2aa019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa01082a0 0x7f2aa019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32882/0 (socket says 172.21.15.5:32882) 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9f7fe640 1 -- 172.21.15.5:0/1444995297 learned_addr learned my addr 172.21.15.5:0/1444995297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9e7fc640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2aa01036e0 0x7f2aa01a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9effd640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa0102d30 0x7f2aa01a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9f7fe640 1 -- 172.21.15.5:0/1444995297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2aa01036e0 msgr2=0x7f2aa01a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9f7fe640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2aa01036e0 0x7f2aa01a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9f7fe640 1 -- 172.21.15.5:0/1444995297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa0102d30 msgr2=0x7f2aa01a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9f7fe640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa0102d30 0x7f2aa01a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9f7fe640 1 -- 172.21.15.5:0/1444995297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa019ad20 con 0x7f2aa01082a0 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9e7fc640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2aa01036e0 0x7f2aa01a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9effd640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa0102d30 0x7f2aa01a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a9f7fe640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa01082a0 0x7f2aa019a5f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800c970 tx=0x7f2a9800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a97fff640 1 -- 172.21.15.5:0/1444995297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98007bb0 con 0x7f2aa01082a0 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2aa5325640 1 -- 172.21.15.5:0/1444995297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa019b010 con 0x7f2aa01082a0 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.611+0000 7f2a97fff640 1 -- 172.21.15.5:0/1444995297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a98007d50 con 0x7f2aa01082a0 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.612+0000 7f2aa5325640 1 -- 172.21.15.5:0/1444995297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa01a7300 con 0x7f2aa01082a0 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.612+0000 7f2a97fff640 1 -- 172.21.15.5:0/1444995297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98018800 con 0x7f2aa01082a0 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.613+0000 7f2aa5325640 1 -- 172.21.15.5:0/1444995297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa006a850 con 0x7f2aa01082a0 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.614+0000 7f2a97fff640 1 -- 172.21.15.5:0/1444995297 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2a9800fa50 con 0x7f2aa01082a0 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.614+0000 7f2a97fff640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a8c0a33b0 0x7f2a8c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.615+0000 7f2a97fff640 1 -- 172.21.15.5:0/1444995297 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2a980d01d0 con 0x7f2aa01082a0 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.615+0000 7f2a9effd640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a8c0a33b0 0x7f2a8c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.615+0000 7f2a9effd640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a8c0a33b0 0x7f2a8c0a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2a90000f80 tx=0x7f2a900093f0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.619+0000 7f2a97fff640 1 -- 172.21.15.5:0/1444995297 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2a98014030 con 0x7f2aa01082a0 2024-09-15T22:11:41.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.833+0000 7f2aa5325640 1 -- 172.21.15.5:0/1444995297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2aa010c2c0 con 0x7f2aa01082a0 2024-09-15T22:11:41.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.834+0000 7f2a97fff640 1 -- 172.21.15.5:0/1444995297 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f2a9809c810 con 0x7f2aa01082a0 2024-09-15T22:11:41.836 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:11:41.836 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:11:41.836 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 -- 172.21.15.5:0/1444995297 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a8c0a33b0 msgr2=0x7f2a8c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a8c0a33b0 0x7f2a8c0a5870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2a90000f80 tx=0x7f2a900093f0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 -- 172.21.15.5:0/1444995297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa01082a0 msgr2=0x7f2aa019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa01082a0 0x7f2aa019a5f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800c970 tx=0x7f2a9800ce30 comp rx=0 tx=0).stop 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 -- 172.21.15.5:0/1444995297 shutdown_connections 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a8c0a33b0 0x7f2a8c0a5870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aa01082a0 0x7f2aa019a5f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2aa01036e0 0x7f2aa01a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 --2- 172.21.15.5:0/1444995297 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2aa0102d30 0x7f2aa01a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 -- 172.21.15.5:0/1444995297 >> 172.21.15.5:0/1444995297 conn(0x7f2aa00fc7d0 msgr2=0x7f2aa00fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 -- 172.21.15.5:0/1444995297 shutdown_connections 2024-09-15T22:11:41.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:41.836+0000 7f2aa5325640 1 -- 172.21.15.5:0/1444995297 wait complete. 2024-09-15T22:11:42.268 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ffcb6d65', quiesce took 7.8 seconds. Will release after: 11.75 2024-09-15T22:11:42.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:42 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1444995297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:42.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:42 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1444995297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:42.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:42 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1444995297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:43.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:43 smithi099 ceph-mon[34892]: pgmap v87: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 11 MiB/s wr, 980 op/s 2024-09-15T22:11:43.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:43 smithi129 ceph-mon[35314]: pgmap v87: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 11 MiB/s wr, 980 op/s 2024-09-15T22:11:43.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:43 smithi005 ceph-mon[30944]: pgmap v87: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 11 MiB/s wr, 980 op/s 2024-09-15T22:11:43.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:11:43.972 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:44.664 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 135.0 s, 420.1 tps, lat 52.889 ms stddev 60.135 2024-09-15T22:11:45.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.530+0000 7fc504892640 1 -- 172.21.15.5:0/2376804759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc500106a90 msgr2=0x7fc500106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:45.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.530+0000 7fc504892640 1 --2- 172.21.15.5:0/2376804759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc500106a90 0x7fc500106e70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009930 tx=0x7fc4f002a420 comp rx=0 tx=0).stop 2024-09-15T22:11:45.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.530+0000 7fc504892640 1 -- 172.21.15.5:0/2376804759 shutdown_connections 2024-09-15T22:11:45.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.530+0000 7fc504892640 1 --2- 172.21.15.5:0/2376804759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc500101430 0x7fc50010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.530+0000 7fc504892640 1 --2- 172.21.15.5:0/2376804759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc500100a90 0x7fc500100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.530+0000 7fc504892640 1 --2- 172.21.15.5:0/2376804759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc500106a90 0x7fc500106e70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.530+0000 7fc504892640 1 -- 172.21.15.5:0/2376804759 >> 172.21.15.5:0/2376804759 conn(0x7fc5000fc7d0 msgr2=0x7fc5000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.530+0000 7fc504892640 1 -- 172.21.15.5:0/2376804759 shutdown_connections 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.530+0000 7fc504892640 1 -- 172.21.15.5:0/2376804759 wait complete. 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.531+0000 7fc504892640 1 Processor -- start 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.531+0000 7fc504892640 1 -- start start 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc504892640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc500100a90 0x7fc5001984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc504892640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc500101430 0x7fc500198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc504892640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc500106a90 0x7fc50019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc504892640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc500111fb0 con 0x7fc500101430 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc504892640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc500111e30 con 0x7fc500100a90 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc504892640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc500112130 con 0x7fc500106a90 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4f5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc500101430 0x7fc500198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4f5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc500101430 0x7fc500198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50780/0 (socket says 172.21.15.5:50780) 2024-09-15T22:11:45.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/301525589 learned_addr learned my addr 172.21.15.5:0/301525589 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc500100a90 0x7fc5001984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4fed9d640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc500106a90 0x7fc50019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/301525589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc500106a90 msgr2=0x7fc50019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4f5d9b640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc500106a90 0x7fc50019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/301525589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc500100a90 msgr2=0x7fc5001984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4f5d9b640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc500100a90 0x7fc5001984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/301525589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50019d480 con 0x7fc500101430 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4f5d9b640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc500101430 0x7fc500198a10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800d950 tx=0x7fc4e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.532+0000 7fc4f7fff640 1 -- 172.21.15.5:0/301525589 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8017070 con 0x7fc500101430 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.533+0000 7fc504892640 1 -- 172.21.15.5:0/301525589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50019d770 con 0x7fc500101430 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.533+0000 7fc4f7fff640 1 -- 172.21.15.5:0/301525589 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e8009c30 con 0x7fc500101430 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.533+0000 7fc4f7fff640 1 -- 172.21.15.5:0/301525589 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e800f6b0 con 0x7fc500101430 2024-09-15T22:11:45.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.533+0000 7fc504892640 1 -- 172.21.15.5:0/301525589 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5001a5050 con 0x7fc500101430 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.535+0000 7fc4f7fff640 1 -- 172.21.15.5:0/301525589 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc4e80109d0 con 0x7fc500101430 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc4f7fff640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc4ec0a33b0 0x7fc4ec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc506c6b740 1 Processor -- start 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc4ec0a33b0 0x7fc4ec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc4f7fff640 1 -- 172.21.15.5:0/301525589 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fc4e80cfce0 con 0x7fc500101430 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc506c6b740 1 -- start start 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x5581665f4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x5581665f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x558166601760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc506c6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55816655a2f0 con 0x5581665f5b40 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc506c6b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558166501cf0 con 0x5581665ff370 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc506c6b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5581665562b0 con 0x5581665f45d0 2024-09-15T22:11:45.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc4ec0a33b0 0x7fc4ec0a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009900 tx=0x7fc4f00058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc4fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x5581665f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc4fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x5581665f4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50796/0 (socket says 172.21.15.5:50796) 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.536+0000 7fc4fe59c640 1 -- 172.21.15.5:0/2841222291 learned_addr learned my addr 172.21.15.5:0/2841222291 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4fed9d640 1 --2- 172.21.15.5:0/2841222291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x558166601760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4f5d9b640 1 --2- 172.21.15.5:0/2841222291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x5581665f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4fe59c640 1 -- 172.21.15.5:0/2841222291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 msgr2=0x5581665f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/2841222291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x5581665f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4fe59c640 1 -- 172.21.15.5:0/2841222291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 msgr2=0x558166601760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/2841222291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x558166601760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4fe59c640 1 -- 172.21.15.5:0/2841222291 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581664e63a0 con 0x5581665f5b40 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4f5d9b640 1 --2- 172.21.15.5:0/2841222291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x5581665f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:45.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/2841222291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x5581665f4090 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0005ec0 tx=0x7fc4f003d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baea4d48ed497f05 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4f4d99640 1 -- 172.21.15.5:0/2841222291 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f004d070 con 0x5581665f5b40 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4f4d99640 1 -- 172.21.15.5:0/2841222291 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f0003cd0 con 0x5581665f5b40 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.537+0000 7fc4f4d99640 1 -- 172.21.15.5:0/2841222291 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0035040 con 0x5581665f5b40 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.538+0000 7fc506c6b740 1 -- 172.21.15.5:0/2841222291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 msgr2=0x5581665f4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.538+0000 7fc506c6b740 1 --2- 172.21.15.5:0/2841222291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x5581665f4090 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0005ec0 tx=0x7fc4f003d000 comp rx=0 tx=0).stop 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.538+0000 7fc506c6b740 1 -- 172.21.15.5:0/2841222291 shutdown_connections 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.538+0000 7fc506c6b740 1 --2- 172.21.15.5:0/2841222291 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x558166601760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.538+0000 7fc506c6b740 1 --2- 172.21.15.5:0/2841222291 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x5581665f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.538+0000 7fc506c6b740 1 --2- 172.21.15.5:0/2841222291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x5581665f4090 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.538+0000 7fc506c6b740 1 -- 172.21.15.5:0/2841222291 >> 172.21.15.5:0/2841222291 conn(0x5581665bd310 msgr2=0x5581665f0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.538+0000 7fc506c6b740 1 -- 172.21.15.5:0/2841222291 shutdown_connections 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.538+0000 7fc506c6b740 1 -- 172.21.15.5:0/2841222291 wait complete. 2024-09-15T22:11:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.538+0000 7fc506c6b740 1 -- start start 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc506c6b740 1 Processor -- start 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x5581665fed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665f5b40 0x5581665fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665ff370 0x5581665fc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc506c6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558166602010 con 0x5581665ff370 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc506c6b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558166515060 con 0x5581665f5b40 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc506c6b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558166602190 con 0x5581665f45d0 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc506c6b740 10 client.0 _ll_register_callbacks cb 0x5581662688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4f5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665f5b40 0x5581665fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665ff370 0x5581665fc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4f5d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665f5b40 0x5581665fbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46908/0 (socket says 172.21.15.5:46908) 2024-09-15T22:11:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665ff370 0x5581665fc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50802/0 (socket says 172.21.15.5:50802) 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/2282157144 learned_addr learned my addr 172.21.15.5:0/2282157144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x5581665fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/2282157144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 msgr2=0x5581665fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4f5d9b640 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x5581665fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/2282157144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665ff370 msgr2=0x5581665fc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4f5d9b640 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665ff370 0x5581665fc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/2282157144 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581662cb3c0 con 0x5581665f5b40 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.539+0000 7fc4fed9d640 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665ff370 0x5581665fc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.540+0000 7fc4f5d9b640 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665f5b40 0x5581665fbfe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4e809b970 tx=0x7fc4e8009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f53f8ff6eb0ed019 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.540+0000 7fc4e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.540+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/2282157144 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8020040 con 0x5581665f5b40 2024-09-15T22:11:45.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.540+0000 7fc506c6b740 1 -- 172.21.15.5:0/2282157144 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f0003cd0 con 0x5581665f5b40 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.540+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/2282157144 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fc4e809b0c0 con 0x5581665f5b40 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.540+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/2282157144 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc4e809b2c0 con 0x5581665f5b40 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.540+0000 7fc506c6b740 10 client.25562 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.540+0000 7fc506c6b740 1 -- 172.21.15.5:0/2282157144 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55816660dac0 con 0x5581665f5b40 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.540+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/2282157144 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f0003cd0 con 0x5581665f5b40 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.540+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/2282157144 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc4e809ab30 con 0x5581665f5b40 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.541+0000 7fc506c6b740 10 client.25562 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.541+0000 7fc506c6b740 10 client.25562 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.541+0000 7fc506c6b740 20 client.25562 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.541+0000 7fc506c6b740 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55816660eed0 0x5581666112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.541+0000 7fc506c6b740 4 client.25562 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.541+0000 7fc506c6b740 1 -- 172.21.15.5:0/2282157144 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558166567860 con 0x55816660eed0 2024-09-15T22:11:45.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.541+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55816660eed0 0x5581666112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.541+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55816660eed0 0x5581666112e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4bd2339f7b2edefa server_cookie=2b5f8ace6da6bcb in_seq=0 out_seq=0 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.541+0000 7fc4e5ffb640 10 client.25562 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/2282157144 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558166567860 con 0x55816660eed0 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc4e5ffb640 10 client.25562 handle_command_reply: tid=0 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc506c6b740 1 client.25562 shutdown 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc506c6b740 1 -- 172.21.15.5:0/2282157144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665f5b40 msgr2=0x5581665fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc506c6b740 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665f5b40 0x5581665fbfe0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4e809b970 tx=0x7fc4e8009f10 comp rx=0 tx=0).stop 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc506c6b740 1 -- 172.21.15.5:0/2282157144 shutdown_connections 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc506c6b740 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55816660eed0 0x5581666112e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc506c6b740 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665ff370 0x5581665fc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc506c6b740 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665f5b40 0x5581665fbfe0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc4e5ffb640 0 client.25562 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc506c6b740 1 --2- 172.21.15.5:0/2282157144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x5581665fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc506c6b740 1 -- 172.21.15.5:0/2282157144 >> 172.21.15.5:0/2282157144 conn(0x5581665bd310 msgr2=0x5581665f6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc506c6b740 1 -- 172.21.15.5:0/2282157144 shutdown_connections 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.542+0000 7fc506c6b740 1 -- 172.21.15.5:0/2282157144 wait complete. 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.543+0000 7fc506c6b740 20 client.25562 trim_cache size 0 max 16384 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc506c6b740 1 Processor -- start 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc506c6b740 1 -- start start 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f45d0 0x55816660b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f5b40 0x5581665fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x558166616bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc506c6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5001b1940 con 0x5581665f45d0 2024-09-15T22:11:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc506c6b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc5001af210 con 0x5581665ff370 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc506c6b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5001ad060 con 0x5581665f5b40 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc4fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f45d0 0x55816660b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc4fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f45d0 0x55816660b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50804/0 (socket says 172.21.15.5:50804) 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc4fe59c640 1 -- 172.21.15.5:0/1771675889 learned_addr learned my addr 172.21.15.5:0/1771675889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc4fe59c640 1 -- 172.21.15.5:0/1771675889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f5b40 msgr2=0x5581665fafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/1771675889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f5b40 0x5581665fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc4fe59c640 1 -- 172.21.15.5:0/1771675889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 msgr2=0x558166616bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/1771675889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x558166616bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc4fe59c640 1 -- 172.21.15.5:0/1771675889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581665f4d50 con 0x5581665f45d0 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.556+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/1771675889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f45d0 0x55816660b2b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0049970 tx=0x7fc4f0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82504e654356a968 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc4e7fff640 1 -- 172.21.15.5:0/1771675889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0040050 con 0x5581665f45d0 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc4e7fff640 1 -- 172.21.15.5:0/1771675889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f0049650 con 0x5581665f45d0 2024-09-15T22:11:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc4e7fff640 1 -- 172.21.15.5:0/1771675889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f004d040 con 0x5581665f45d0 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc506c6b740 1 -- 172.21.15.5:0/1771675889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f45d0 msgr2=0x55816660b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc506c6b740 1 --2- 172.21.15.5:0/1771675889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f45d0 0x55816660b2b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0049970 tx=0x7fc4f0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc506c6b740 1 -- 172.21.15.5:0/1771675889 shutdown_connections 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc506c6b740 1 --2- 172.21.15.5:0/1771675889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x558166616bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc506c6b740 1 --2- 172.21.15.5:0/1771675889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f5b40 0x5581665fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc506c6b740 1 --2- 172.21.15.5:0/1771675889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f45d0 0x55816660b2b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc506c6b740 1 -- 172.21.15.5:0/1771675889 >> 172.21.15.5:0/1771675889 conn(0x5581665bd310 msgr2=0x5581665fa8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc506c6b740 1 -- 172.21.15.5:0/1771675889 shutdown_connections 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.557+0000 7fc506c6b740 1 -- 172.21.15.5:0/1771675889 wait complete. 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc506c6b740 1 -- start start 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc506c6b740 1 Processor -- start 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x558166607c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x558166602fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc506c6b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x5581666035b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc506c6b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5001ad6d0 con 0x5581665f5b40 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc506c6b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc5001aeb40 con 0x5581665ff370 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc506c6b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5001adda0 con 0x5581665f45d0 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc506c6b740 10 client.0 _ll_register_callbacks cb 0x7fc4e8097f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc4f5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x558166602fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc4f5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x558166602fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50808/0 (socket says 172.21.15.5:50808) 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/243373625 learned_addr learned my addr 172.21.15.5:0/243373625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x558166607c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc4fed9d640 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x5581666035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/243373625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 msgr2=0x558166607c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:45.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc4f5d9b640 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x558166607c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/243373625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 msgr2=0x5581666035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc4f5d9b640 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x5581666035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc4f5d9b640 1 -- 172.21.15.5:0/243373625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558166603de0 con 0x5581665f5b40 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.558+0000 7fc4fed9d640 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x5581666035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc4f5d9b640 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x558166602fc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8010d20 tx=0x7fc4e809ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade38be8b4612a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc4e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/243373625 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8020070 con 0x5581665f5b40 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/243373625 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fc4e80ba2a0 con 0x5581665f5b40 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc506c6b740 1 -- 172.21.15.5:0/243373625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f0049650 con 0x5581665f5b40 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/243373625 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc4e80b2080 con 0x5581665f5b40 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc506c6b740 10 client.15972 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc506c6b740 1 -- 172.21.15.5:0/243373625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558166679c90 con 0x5581665f5b40 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/243373625 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f0049650 con 0x5581665f5b40 2024-09-15T22:11:45.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/243373625 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc4e80979c0 con 0x5581665f5b40 2024-09-15T22:11:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc506c6b740 10 client.15972 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc506c6b740 10 client.15972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:11:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc506c6b740 20 client.15972 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc506c6b740 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55816667a280 0x55816667c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc506c6b740 4 client.15972 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:11:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.559+0000 7fc506c6b740 1 -- 172.21.15.5:0/243373625 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558166561d70 con 0x55816667a280 2024-09-15T22:11:45.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55816667a280 0x55816667c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:45.482 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:11:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc4e5ffb640 10 client.15972 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:11:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc4fe59c640 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55816667a280 0x55816667c6a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9eb662c5354588b server_cookie=62fb6dbf45d823fc in_seq=0 out_seq=0 2024-09-15T22:11:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc4e5ffb640 1 -- 172.21.15.5:0/243373625 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558166561d70 con 0x55816667a280 2024-09-15T22:11:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc4e5ffb640 10 client.15972 handle_command_reply: tid=0 2024-09-15T22:11:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc506c6b740 1 client.15972 shutdown 2024-09-15T22:11:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc506c6b740 1 -- 172.21.15.5:0/243373625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 msgr2=0x558166602fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc506c6b740 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x558166602fc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8010d20 tx=0x7fc4e809ade0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc506c6b740 1 -- 172.21.15.5:0/243373625 shutdown_connections 2024-09-15T22:11:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc506c6b740 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55816667a280 0x55816667c6a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc506c6b740 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5581665ff370 0x5581666035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc506c6b740 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5581665f5b40 0x558166602fc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc506c6b740 1 --2- 172.21.15.5:0/243373625 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5581665f45d0 0x558166607c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc4e5ffb640 0 client.15972 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc506c6b740 1 -- 172.21.15.5:0/243373625 >> 172.21.15.5:0/243373625 conn(0x5581665bd310 msgr2=0x5581665fb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc506c6b740 1 -- 172.21.15.5:0/243373625 shutdown_connections 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.560+0000 7fc506c6b740 1 -- 172.21.15.5:0/243373625 wait complete. 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc506c6b740 20 client.15972 trim_cache size 0 max 16384 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 -- 172.21.15.5:0/301525589 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc4ec0a33b0 msgr2=0x7fc4ec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc4ec0a33b0 0x7fc4ec0a5870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009900 tx=0x7fc4f00058d0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 -- 172.21.15.5:0/301525589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc500101430 msgr2=0x7fc500198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc500101430 0x7fc500198a10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800d950 tx=0x7fc4e800de10 comp rx=0 tx=0).stop 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 -- 172.21.15.5:0/301525589 shutdown_connections 2024-09-15T22:11:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc4ec0a33b0 0x7fc4ec0a5870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc500106a90 0x7fc50019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc500101430 0x7fc500198a10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 --2- 172.21.15.5:0/301525589 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc500100a90 0x7fc5001984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 -- 172.21.15.5:0/301525589 >> 172.21.15.5:0/301525589 conn(0x7fc5000fc7d0 msgr2=0x7fc5000fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 -- 172.21.15.5:0/301525589 shutdown_connections 2024-09-15T22:11:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:44.561+0000 7fc504892640 1 -- 172.21.15.5:0/301525589 wait complete. 2024-09-15T22:11:45.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:45 smithi099 ceph-mon[34892]: pgmap v88: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.7 MiB/s wr, 826 op/s 2024-09-15T22:11:45.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:45 smithi129 ceph-mon[35314]: pgmap v88: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.7 MiB/s wr, 826 op/s 2024-09-15T22:11:45.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:45 smithi005 ceph-mon[30944]: pgmap v88: 129 pgs: 129 active+clean; 1.7 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 8.7 MiB/s wr, 826 op/s 2024-09-15T22:11:46.563 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:11:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.322+0000 7fea1d313640 1 -- 172.21.15.5:0/2912522931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18106f10 msgr2=0x7fea1810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.322+0000 7fea1d313640 1 --2- 172.21.15.5:0/2912522931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18106f10 0x7fea1810ea90 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea10009ca0 tx=0x7fea1002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.322+0000 7fea1d313640 1 -- 172.21.15.5:0/2912522931 shutdown_connections 2024-09-15T22:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.322+0000 7fea1d313640 1 --2- 172.21.15.5:0/2912522931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18106f10 0x7fea1810ea90 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.322+0000 7fea1d313640 1 --2- 172.21.15.5:0/2912522931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea18102790 0x7fea181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.322+0000 7fea1d313640 1 --2- 172.21.15.5:0/2912522931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea18101de0 0x7fea181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.322+0000 7fea1d313640 1 -- 172.21.15.5:0/2912522931 >> 172.21.15.5:0/2912522931 conn(0x7fea180fd8c0 msgr2=0x7fea180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.322+0000 7fea1d313640 1 -- 172.21.15.5:0/2912522931 shutdown_connections 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.322+0000 7fea1d313640 1 -- 172.21.15.5:0/2912522931 wait complete. 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea1d313640 1 Processor -- start 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea1d313640 1 -- start start 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea1d313640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea18101de0 0x7fea180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea1d313640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea18102790 0x7fea18074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea1d313640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18106f10 0x7fea18074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea1d313640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea18111f70 con 0x7fea18106f10 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea1d313640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fea18111df0 con 0x7fea18101de0 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea1d313640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fea181120f0 con 0x7fea18102790 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18106f10 0x7fea18074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea16ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea18101de0 0x7fea180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18106f10 0x7fea18074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50830/0 (socket says 172.21.15.5:50830) 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.323+0000 7fea177fe640 1 -- 172.21.15.5:0/2887501574 learned_addr learned my addr 172.21.15.5:0/2887501574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea0ffff640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea18102790 0x7fea18074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea177fe640 1 -- 172.21.15.5:0/2887501574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea18102790 msgr2=0x7fea18074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea177fe640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea18102790 0x7fea18074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea177fe640 1 -- 172.21.15.5:0/2887501574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea18101de0 msgr2=0x7fea180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea177fe640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea18101de0 0x7fea180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea177fe640 1 -- 172.21.15.5:0/2887501574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea181a68b0 con 0x7fea18106f10 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea177fe640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18106f10 0x7fea18074f70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea10031a20 tx=0x7fea10031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea14ff9640 1 -- 172.21.15.5:0/2887501574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10003c30 con 0x7fea18106f10 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea1d313640 1 -- 172.21.15.5:0/2887501574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea181a6b40 con 0x7fea18106f10 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea14ff9640 1 -- 172.21.15.5:0/2887501574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea10003dd0 con 0x7fea18106f10 2024-09-15T22:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea14ff9640 1 -- 172.21.15.5:0/2887501574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1002fcf0 con 0x7fea18106f10 2024-09-15T22:11:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.324+0000 7fea1d313640 1 -- 172.21.15.5:0/2887501574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea181a7080 con 0x7fea18106f10 2024-09-15T22:11:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.326+0000 7fea14ff9640 1 -- 172.21.15.5:0/2887501574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fea10004050 con 0x7fea18106f10 2024-09-15T22:11:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea1f6ec740 1 Processor -- start 2024-09-15T22:11:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea1f6ec740 1 -- start start 2024-09-15T22:11:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 0x560099d85a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 0x560099d92760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560099ceb2f0 con 0x560099d86b40 2024-09-15T22:11:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560099c92cf0 con 0x560099d90370 2024-09-15T22:11:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560099ce72b0 con 0x560099d855d0 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d85090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50846/0 (socket says 172.21.15.5:50846) 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea0ffff640 1 -- 172.21.15.5:0/284492774 learned_addr learned my addr 172.21.15.5:0/284492774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea14ff9640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea040a33b0 0x7fea040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.327+0000 7fea0ffff640 1 --2- 172.21.15.5:0/284492774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 0x560099d92760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea16ffd640 1 --2- 172.21.15.5:0/284492774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 0x560099d85a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea177fe640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea040a33b0 0x7fea040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea0ffff640 1 -- 172.21.15.5:0/284492774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 msgr2=0x560099d85a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea0ffff640 1 --2- 172.21.15.5:0/284492774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 0x560099d85a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea0ffff640 1 -- 172.21.15.5:0/284492774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 msgr2=0x560099d92760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea0ffff640 1 --2- 172.21.15.5:0/284492774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 0x560099d92760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea0ffff640 1 -- 172.21.15.5:0/284492774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560099c773a0 con 0x560099d86b40 2024-09-15T22:11:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea16ffd640 1 --2- 172.21.15.5:0/284492774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 0x560099d85a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea14ff9640 1 -- 172.21.15.5:0/2887501574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fea100f1250 con 0x7fea18106f10 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea0ffff640 1 --2- 172.21.15.5:0/284492774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d85090 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fea00004650 tx=0x7fea000041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf8b0f7f873c6af server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea0d7fa640 1 -- 172.21.15.5:0/284492774 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00005020 con 0x560099d86b40 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea177fe640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea040a33b0 0x7fea040a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fea100ee7c0 tx=0x7fea10044f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea0d7fa640 1 -- 172.21.15.5:0/284492774 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea000056a0 con 0x560099d86b40 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.328+0000 7fea0d7fa640 1 -- 172.21.15.5:0/284492774 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000104c0 con 0x560099d86b40 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.329+0000 7fea1f6ec740 1 -- 172.21.15.5:0/284492774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 msgr2=0x560099d85090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.329+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/284492774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d85090 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fea00004650 tx=0x7fea000041d0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.329+0000 7fea1f6ec740 1 -- 172.21.15.5:0/284492774 shutdown_connections 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.329+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/284492774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 0x560099d92760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.329+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/284492774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 0x560099d85a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.329+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/284492774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d85090 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.329+0000 7fea1f6ec740 1 -- 172.21.15.5:0/284492774 >> 172.21.15.5:0/284492774 conn(0x560099d4e310 msgr2=0x560099d81c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.329+0000 7fea1f6ec740 1 -- 172.21.15.5:0/284492774 shutdown_connections 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.329+0000 7fea1f6ec740 1 -- 172.21.15.5:0/284492774 wait complete. 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.329+0000 7fea1f6ec740 1 -- start start 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.329+0000 7fea1f6ec740 1 Processor -- start 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d855d0 0x560099d8fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d8d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d90370 0x560099d8d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560099d93010 con 0x560099d86b40 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560099ca6060 con 0x560099d855d0 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560099d93190 con 0x560099d90370 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea1f6ec740 10 client.0 _ll_register_callbacks cb 0x5600999f98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea16ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d8d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea16ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d8d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50850/0 (socket says 172.21.15.5:50850) 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea16ffd640 1 -- 172.21.15.5:0/2018703842 learned_addr learned my addr 172.21.15.5:0/2018703842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea0ffff640 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d90370 0x560099d8d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea0ffff640 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d855d0 0x560099d8fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea16ffd640 1 -- 172.21.15.5:0/2018703842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d90370 msgr2=0x560099d8d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea16ffd640 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d90370 0x560099d8d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea16ffd640 1 -- 172.21.15.5:0/2018703842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d855d0 msgr2=0x560099d8fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea16ffd640 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d855d0 0x560099d8fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea16ffd640 1 -- 172.21.15.5:0/2018703842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560099a5c3c0 con 0x560099d86b40 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea0ffff640 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d90370 0x560099d8d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea16ffd640 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d8d000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fea0800b980 tx=0x7fea0800be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd068dcb9e77821e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fe9fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2018703842 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08005220 con 0x560099d86b40 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2018703842 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fea080053c0 con 0x560099d86b40 2024-09-15T22:11:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.330+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2018703842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea000056a0 con 0x560099d86b40 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2018703842 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea08003e40 con 0x560099d86b40 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fea1f6ec740 10 client.15984 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2018703842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560099d9ea90 con 0x560099d86b40 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2018703842 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea000056a0 con 0x560099d86b40 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2018703842 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fea08012830 con 0x560099d86b40 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fea1f6ec740 10 client.15984 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fea1f6ec740 10 client.15984 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fea1f6ec740 20 client.15984 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560099d9ffa0 0x560099da2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fea1f6ec740 4 client.15984 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2018703842 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560099cf8860 con 0x560099d9ffa0 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.331+0000 7fea0ffff640 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560099d9ffa0 0x560099da2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.332+0000 7fe9fe7fc640 10 client.15984 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.332+0000 7fea0ffff640 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560099d9ffa0 0x560099da2390 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a15d4668600044c5 server_cookie=f827ffb7b36ee3df in_seq=0 out_seq=0 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.332+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2018703842 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560099cf8860 con 0x560099d9ffa0 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.332+0000 7fe9fe7fc640 10 client.15984 handle_command_reply: tid=0 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 1 client.15984 shutdown 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2018703842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 msgr2=0x560099d8d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d8d000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fea0800b980 tx=0x7fea0800be40 comp rx=0 tx=0).stop 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2018703842 shutdown_connections 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560099d9ffa0 0x560099da2390 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d90370 0x560099d8d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d8d000 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fe9fe7fc640 0 client.15984 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2018703842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d855d0 0x560099d8fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2018703842 >> 172.21.15.5:0/2018703842 conn(0x560099d4e310 msgr2=0x560099d87930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2018703842 shutdown_connections 2024-09-15T22:11:47.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2018703842 wait complete. 2024-09-15T22:11:47.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.333+0000 7fea1f6ec740 20 client.15984 trim_cache size 0 max 16384 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea1f6ec740 1 Processor -- start 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea1f6ec740 1 -- start start 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d855d0 0x560099d8cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d86b40 0x560099d9c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 0x560099da7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea181b3af0 con 0x560099d855d0 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fea181b13c0 con 0x560099d90370 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fea181af210 con 0x560099d86b40 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d855d0 0x560099d8cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea0ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d855d0 0x560099d8cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50854/0 (socket says 172.21.15.5:50854) 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea0ffff640 1 -- 172.21.15.5:0/1468761753 learned_addr learned my addr 172.21.15.5:0/1468761753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea16ffd640 1 --2- 172.21.15.5:0/1468761753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d86b40 0x560099d9c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea0ffff640 1 -- 172.21.15.5:0/1468761753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d86b40 msgr2=0x560099d9c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea0ffff640 1 --2- 172.21.15.5:0/1468761753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d86b40 0x560099d9c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea0ffff640 1 -- 172.21.15.5:0/1468761753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 msgr2=0x560099da7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea0ffff640 1 --2- 172.21.15.5:0/1468761753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 0x560099da7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea0ffff640 1 -- 172.21.15.5:0/1468761753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560099d900c0 con 0x560099d855d0 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.347+0000 7fea16ffd640 1 --2- 172.21.15.5:0/1468761753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d86b40 0x560099d9c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.348+0000 7fea0ffff640 1 --2- 172.21.15.5:0/1468761753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d855d0 0x560099d8cfd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fea00004c30 tx=0x7fea000041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562d13ef7a3cb0b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.348+0000 7fea0cff9640 1 -- 172.21.15.5:0/1468761753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00004d70 con 0x560099d855d0 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.348+0000 7fea0cff9640 1 -- 172.21.15.5:0/1468761753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea000053f0 con 0x560099d855d0 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.348+0000 7fea0cff9640 1 -- 172.21.15.5:0/1468761753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0002c840 con 0x560099d855d0 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 -- 172.21.15.5:0/1468761753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d855d0 msgr2=0x560099d8cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/1468761753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d855d0 0x560099d8cfd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fea00004c30 tx=0x7fea000041d0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 -- 172.21.15.5:0/1468761753 shutdown_connections 2024-09-15T22:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/1468761753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 0x560099da7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/1468761753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d86b40 0x560099d9c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/1468761753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d855d0 0x560099d8cfd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 -- 172.21.15.5:0/1468761753 >> 172.21.15.5:0/1468761753 conn(0x560099d4e310 msgr2=0x560099d8c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 -- 172.21.15.5:0/1468761753 shutdown_connections 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 -- 172.21.15.5:0/1468761753 wait complete. 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 -- start start 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 Processor -- start 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 0x560099d98a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d93d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 0x560099d942c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea181af880 con 0x560099d86b40 2024-09-15T22:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fea181b0cf0 con 0x560099d90370 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.349+0000 7fea1f6ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fea181aff50 con 0x560099d855d0 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea1f6ec740 10 client.0 _ll_register_callbacks cb 0x7fea080040c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea16ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d93d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea16ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d93d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50860/0 (socket says 172.21.15.5:50860) 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea16ffd640 1 -- 172.21.15.5:0/2200810608 learned_addr learned my addr 172.21.15.5:0/2200810608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea0ffff640 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 0x560099d942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea0ffff640 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 0x560099d98a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea16ffd640 1 -- 172.21.15.5:0/2200810608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 msgr2=0x560099d98a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea16ffd640 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 0x560099d98a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea16ffd640 1 -- 172.21.15.5:0/2200810608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 msgr2=0x560099d942c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea16ffd640 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 0x560099d942c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea16ffd640 1 -- 172.21.15.5:0/2200810608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560099d949b0 con 0x560099d86b40 2024-09-15T22:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea0ffff640 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 0x560099d98a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea16ffd640 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d93d80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fea0800c010 tx=0x7fea08012a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9c1a7652b02a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fe9fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2200810608 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08005b90 con 0x560099d86b40 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2200810608 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fea08011860 con 0x560099d86b40 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.350+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2200810608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea000053f0 con 0x560099d86b40 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2200810608 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea080039c0 con 0x560099d86b40 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fea1f6ec740 10 client.15990 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2200810608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560099e0aac0 con 0x560099d86b40 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2200810608 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea000053f0 con 0x560099d86b40 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2200810608 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fea08028bb0 con 0x560099d86b40 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fea1f6ec740 10 client.15990 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fea1f6ec740 10 client.15990 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fea1f6ec740 20 client.15990 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560099e0b290 0x560099e0d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fea1f6ec740 4 client.15990 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2200810608 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560099cf2d70 con 0x560099e0b290 2024-09-15T22:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.351+0000 7fea0ffff640 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560099e0b290 0x560099e0d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea0ffff640 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560099e0b290 0x560099e0d6b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a85c21774fc44b48 server_cookie=d5b3b1b439e4275f in_seq=0 out_seq=0 2024-09-15T22:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fe9fe7fc640 10 client.15990 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fe9fe7fc640 1 -- 172.21.15.5:0/2200810608 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560099cf2d70 con 0x560099e0b290 2024-09-15T22:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fe9fe7fc640 10 client.15990 handle_command_reply: tid=0 2024-09-15T22:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 1 client.15990 shutdown 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2200810608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 msgr2=0x560099d93d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d93d80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fea0800c010 tx=0x7fea08012a70 comp rx=0 tx=0).stop 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2200810608 shutdown_connections 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x560099e0b290 0x560099e0d6b0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560099d90370 0x560099d942c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560099d86b40 0x560099d93d80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 1 --2- 172.21.15.5:0/2200810608 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560099d855d0 0x560099d98a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2200810608 >> 172.21.15.5:0/2200810608 conn(0x560099d4e310 msgr2=0x560099d804e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fe9fe7fc640 0 client.15990 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2200810608 shutdown_connections 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 1 -- 172.21.15.5:0/2200810608 wait complete. 2024-09-15T22:11:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.352+0000 7fea1f6ec740 20 client.15990 trim_cache size 0 max 16384 2024-09-15T22:11:47.365 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 -- 172.21.15.5:0/2887501574 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea040a33b0 msgr2=0x7fea040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea040a33b0 0x7fea040a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fea100ee7c0 tx=0x7fea10044f90 comp rx=0 tx=0).stop 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 -- 172.21.15.5:0/2887501574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18106f10 msgr2=0x7fea18074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18106f10 0x7fea18074f70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea10031a20 tx=0x7fea10031a50 comp rx=0 tx=0).stop 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 -- 172.21.15.5:0/2887501574 shutdown_connections 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea040a33b0 0x7fea040a5870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea18106f10 0x7fea18074f70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fea18102790 0x7fea18074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 --2- 172.21.15.5:0/2887501574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fea18101de0 0x7fea180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 -- 172.21.15.5:0/2887501574 >> 172.21.15.5:0/2887501574 conn(0x7fea180fd8c0 msgr2=0x7fea180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 -- 172.21.15.5:0/2887501574 shutdown_connections 2024-09-15T22:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:47.353+0000 7fea1d313640 1 -- 172.21.15.5:0/2887501574 wait complete. 2024-09-15T22:11:47.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:47 smithi099 ceph-mon[34892]: pgmap v89: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.7 MiB/s wr, 913 op/s 2024-09-15T22:11:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:47 smithi129 ceph-mon[35314]: pgmap v89: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.7 MiB/s wr, 913 op/s 2024-09-15T22:11:47.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:47 smithi005 ceph-mon[30944]: pgmap v89: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 9.7 MiB/s wr, 913 op/s 2024-09-15T22:11:48.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:11:48.397 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:49.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:49 smithi005 ceph-mon[30944]: pgmap v90: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 366 op/s 2024-09-15T22:11:49.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.379+0000 7feab1fb9640 1 -- 172.21.15.5:0/3692713240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaac10c180 msgr2=0x7feaac10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.379+0000 7feab1fb9640 1 --2- 172.21.15.5:0/3692713240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaac10c180 0x7feaac10e600 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009ca0 tx=0x7feaa402a950 comp rx=0 tx=0).stop 2024-09-15T22:11:49.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.379+0000 7feab1fb9640 1 -- 172.21.15.5:0/3692713240 shutdown_connections 2024-09-15T22:11:49.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.379+0000 7feab1fb9640 1 --2- 172.21.15.5:0/3692713240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaac10c180 0x7feaac10e600 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.379+0000 7feab1fb9640 1 --2- 172.21.15.5:0/3692713240 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feaac1036c0 0x7feaac10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.379+0000 7feab1fb9640 1 --2- 172.21.15.5:0/3692713240 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaac102da0 0x7feaac103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.379+0000 7feab1fb9640 1 -- 172.21.15.5:0/3692713240 >> 172.21.15.5:0/3692713240 conn(0x7feaac0fc7a0 msgr2=0x7feaac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.379+0000 7feab1fb9640 1 -- 172.21.15.5:0/3692713240 shutdown_connections 2024-09-15T22:11:49.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.379+0000 7feab1fb9640 1 -- 172.21.15.5:0/3692713240 wait complete. 2024-09-15T22:11:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.380+0000 7feab1fb9640 1 Processor -- start 2024-09-15T22:11:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feab1fb9640 1 -- start start 2024-09-15T22:11:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feab1fb9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaac102da0 0x7feaac198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feab1fb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaac1036c0 0x7feaac198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feab1fb9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feaac10c180 0x7feaac19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feab1fb9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feaac111f60 con 0x7feaac1036c0 2024-09-15T22:11:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feab1fb9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feaac111de0 con 0x7feaac102da0 2024-09-15T22:11:49.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feab1fb9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1120e0 con 0x7feaac10c180 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feaaaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaac1036c0 0x7feaac198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feaaaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaac1036c0 0x7feaac198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50880/0 (socket says 172.21.15.5:50880) 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feaaaffd640 1 -- 172.21.15.5:0/370240184 learned_addr learned my addr 172.21.15.5:0/370240184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feaab7fe640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaac102da0 0x7feaac198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feaabfff640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feaac10c180 0x7feaac19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feaaaffd640 1 -- 172.21.15.5:0/370240184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feaac10c180 msgr2=0x7feaac19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feaaaffd640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feaac10c180 0x7feaac19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feaaaffd640 1 -- 172.21.15.5:0/370240184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaac102da0 msgr2=0x7feaac198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feaaaffd640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaac102da0 0x7feaac198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.381+0000 7feaaaffd640 1 -- 172.21.15.5:0/370240184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaac19d3f0 con 0x7feaac1036c0 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.382+0000 7feaaaffd640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaac1036c0 0x7feaac198980 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feaa000d770 tx=0x7feaa000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.382+0000 7feaa8ff9640 1 -- 172.21.15.5:0/370240184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0017070 con 0x7feaac1036c0 2024-09-15T22:11:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.382+0000 7feaa8ff9640 1 -- 172.21.15.5:0/370240184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa0005500 con 0x7feaac1036c0 2024-09-15T22:11:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.382+0000 7feab1fb9640 1 -- 172.21.15.5:0/370240184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaac19d6e0 con 0x7feaac1036c0 2024-09-15T22:11:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.382+0000 7feaa8ff9640 1 -- 172.21.15.5:0/370240184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0004e00 con 0x7feaac1036c0 2024-09-15T22:11:49.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.382+0000 7feab1fb9640 1 -- 172.21.15.5:0/370240184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaac1a4fc0 con 0x7feaac1036c0 2024-09-15T22:11:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.384+0000 7feaa8ff9640 1 -- 172.21.15.5:0/370240184 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7feaa0004fa0 con 0x7feaac1036c0 2024-09-15T22:11:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feab4392740 1 Processor -- start 2024-09-15T22:11:49.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feab4392740 1 -- start start 2024-09-15T22:11:49.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feab4392740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487ecb40 0x5580487eb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feab4392740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487eba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feab4392740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487f6370 0x5580487f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feab4392740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5580487512f0 con 0x5580487eb5d0 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feab4392740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5580486f8cf0 con 0x5580487ecb40 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feab4392740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55804874d2b0 con 0x5580487f6370 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feaab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487eba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feaab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487eba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50888/0 (socket says 172.21.15.5:50888) 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feaab7fe640 1 -- 172.21.15.5:0/1776231324 learned_addr learned my addr 172.21.15.5:0/1776231324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feaabfff640 1 --2- 172.21.15.5:0/1776231324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487ecb40 0x5580487eb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feaa8ff9640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea940a3430 0x7fea940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.385+0000 7feaaaffd640 1 --2- 172.21.15.5:0/1776231324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487f6370 0x5580487f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7feaabfff640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea940a3430 0x7fea940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7feaab7fe640 1 -- 172.21.15.5:0/1776231324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487f6370 msgr2=0x5580487f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7feaab7fe640 1 --2- 172.21.15.5:0/1776231324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487f6370 0x5580487f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7feaab7fe640 1 -- 172.21.15.5:0/1776231324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487ecb40 msgr2=0x5580487eb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7feaab7fe640 1 --2- 172.21.15.5:0/1776231324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487ecb40 0x5580487eb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7feaab7fe640 1 -- 172.21.15.5:0/1776231324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580486dd3a0 con 0x5580487eb5d0 2024-09-15T22:11:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7feaa8ff9640 1 -- 172.21.15.5:0/370240184 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7feaa00cfa10 con 0x7feaac1036c0 2024-09-15T22:11:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7feaab7fe640 1 --2- 172.21.15.5:0/1776231324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487eba10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00bd10 tx=0x7fea9c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc64719c7b64e77 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7fea91ffb640 1 -- 172.21.15.5:0/1776231324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c004030 con 0x5580487eb5d0 2024-09-15T22:11:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7feaabfff640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea940a3430 0x7fea940a58f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feaa4004ad0 tx=0x7feaa402a950 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7fea91ffb640 1 -- 172.21.15.5:0/1776231324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9c0041d0 con 0x5580487eb5d0 2024-09-15T22:11:49.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.386+0000 7fea91ffb640 1 -- 172.21.15.5:0/1776231324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c0117e0 con 0x5580487eb5d0 2024-09-15T22:11:49.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.387+0000 7feab4392740 1 -- 172.21.15.5:0/1776231324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 msgr2=0x5580487eba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.387+0000 7feab4392740 1 --2- 172.21.15.5:0/1776231324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487eba10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00bd10 tx=0x7fea9c0107a0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.387+0000 7feab4392740 1 -- 172.21.15.5:0/1776231324 shutdown_connections 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.387+0000 7feab4392740 1 --2- 172.21.15.5:0/1776231324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487f6370 0x5580487f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.387+0000 7feab4392740 1 --2- 172.21.15.5:0/1776231324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487eba10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.387+0000 7feab4392740 1 --2- 172.21.15.5:0/1776231324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487ecb40 0x5580487eb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.387+0000 7feab4392740 1 -- 172.21.15.5:0/1776231324 >> 172.21.15.5:0/1776231324 conn(0x5580487b4310 msgr2=0x5580487e7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.387+0000 7feab4392740 1 -- 172.21.15.5:0/1776231324 shutdown_connections 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.387+0000 7feab4392740 1 -- 172.21.15.5:0/1776231324 wait complete. 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.387+0000 7feab4392740 1 -- start start 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feab4392740 1 Processor -- start 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feab4392740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feab4392740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487ecb40 0x5580487f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feab4392740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feab4392740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5580487f9070 con 0x5580487eb5d0 2024-09-15T22:11:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feab4392740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55804870c060 con 0x5580487f6370 2024-09-15T22:11:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feab4392740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5580487f91f0 con 0x5580487ecb40 2024-09-15T22:11:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feab4392740 10 client.0 _ll_register_callbacks cb 0x55804845f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feaab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feaab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50892/0 (socket says 172.21.15.5:50892) 2024-09-15T22:11:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feaab7fe640 1 -- 172.21.15.5:0/88178404 learned_addr learned my addr 172.21.15.5:0/88178404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feaaaffd640 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487ecb40 0x5580487f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feaabfff640 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feaab7fe640 1 -- 172.21.15.5:0/88178404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487ecb40 msgr2=0x5580487f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feaab7fe640 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487ecb40 0x5580487f2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feaab7fe640 1 -- 172.21.15.5:0/88178404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 msgr2=0x5580487f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feaab7fe640 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487f35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.388+0000 7feaab7fe640 1 -- 172.21.15.5:0/88178404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580484c23c0 con 0x5580487eb5d0 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.389+0000 7feaab7fe640 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487f5d40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fea9c003f30 tx=0x7fea9c00b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28928668b558993 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.389+0000 7fea8affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.389+0000 7fea8affd640 1 -- 172.21.15.5:0/88178404 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c029060 con 0x5580487eb5d0 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.389+0000 7fea8affd640 1 -- 172.21.15.5:0/88178404 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fea9c0035f0 con 0x5580487eb5d0 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.389+0000 7feab4392740 1 -- 172.21.15.5:0/88178404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea9c0041d0 con 0x5580487eb5d0 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.389+0000 7fea8affd640 1 -- 172.21.15.5:0/88178404 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea9c030050 con 0x5580487eb5d0 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.389+0000 7feab4392740 10 client.16002 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.389+0000 7feab4392740 1 -- 172.21.15.5:0/88178404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558048804ae0 con 0x5580487eb5d0 2024-09-15T22:11:49.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.389+0000 7fea8affd640 1 -- 172.21.15.5:0/88178404 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9c0041d0 con 0x5580487eb5d0 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.389+0000 7fea8affd640 1 -- 172.21.15.5:0/88178404 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fea9c0046c0 con 0x5580487eb5d0 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.390+0000 7feab4392740 10 client.16002 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.390+0000 7feab4392740 10 client.16002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.390+0000 7feab4392740 20 client.16002 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.390+0000 7feab4392740 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558048805ee0 0x5580488082d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.390+0000 7feab4392740 4 client.16002 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.390+0000 7feab4392740 1 -- 172.21.15.5:0/88178404 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55804875e860 con 0x558048805ee0 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.390+0000 7feaaaffd640 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558048805ee0 0x5580488082d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.390+0000 7fea8affd640 10 client.16002 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.390+0000 7feaaaffd640 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558048805ee0 0x5580488082d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8532560142c5c22 server_cookie=e9289ab1ace94d1b in_seq=0 out_seq=0 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7fea8affd640 1 -- 172.21.15.5:0/88178404 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55804875e860 con 0x558048805ee0 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7fea8affd640 10 client.16002 handle_command_reply: tid=0 2024-09-15T22:11:49.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7feab4392740 1 client.16002 shutdown 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7feab4392740 1 -- 172.21.15.5:0/88178404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 msgr2=0x5580487f5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7feab4392740 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487f5d40 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fea9c003f30 tx=0x7fea9c00b950 comp rx=0 tx=0).stop 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7feab4392740 1 -- 172.21.15.5:0/88178404 shutdown_connections 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7feab4392740 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558048805ee0 0x5580488082d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7feab4392740 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7feab4392740 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487ecb40 0x5580487f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7feab4392740 1 --2- 172.21.15.5:0/88178404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487eb5d0 0x5580487f5d40 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7feab4392740 1 -- 172.21.15.5:0/88178404 >> 172.21.15.5:0/88178404 conn(0x5580487b4310 msgr2=0x5580487ed960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.391+0000 7fea8affd640 0 client.16002 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.392+0000 7feab4392740 1 -- 172.21.15.5:0/88178404 shutdown_connections 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.392+0000 7feab4392740 1 -- 172.21.15.5:0/88178404 wait complete. 2024-09-15T22:11:49.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.392+0000 7feab4392740 20 client.16002 trim_cache size 0 max 16384 2024-09-15T22:11:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feab4392740 1 Processor -- start 2024-09-15T22:11:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feab4392740 1 -- start start 2024-09-15T22:11:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feab4392740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487eb5d0 0x558048802310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feab4392740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487ee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feab4392740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487ee660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feab4392740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1b1870 con 0x5580487ecb40 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feab4392740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1af140 con 0x5580487f6370 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feab4392740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1acf90 con 0x5580487eb5d0 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feaaaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487ee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feaaaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487ee120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50904/0 (socket says 172.21.15.5:50904) 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feaaaffd640 1 -- 172.21.15.5:0/3408136436 learned_addr learned my addr 172.21.15.5:0/3408136436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feaab7fe640 1 --2- 172.21.15.5:0/3408136436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487eb5d0 0x558048802310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feaabfff640 1 --2- 172.21.15.5:0/3408136436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487ee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feaaaffd640 1 -- 172.21.15.5:0/3408136436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487eb5d0 msgr2=0x558048802310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feaaaffd640 1 --2- 172.21.15.5:0/3408136436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487eb5d0 0x558048802310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feaaaffd640 1 -- 172.21.15.5:0/3408136436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 msgr2=0x5580487ee660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feaaaffd640 1 --2- 172.21.15.5:0/3408136436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487ee660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.405+0000 7feaaaffd640 1 -- 172.21.15.5:0/3408136436 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580487ec740 con 0x5580487ecb40 2024-09-15T22:11:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.406+0000 7feaaaffd640 1 --2- 172.21.15.5:0/3408136436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487ee120 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feaa0097e00 tx=0x7feaa006d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30399fab6d5f6a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.406+0000 7fea917fa640 1 -- 172.21.15.5:0/3408136436 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0020040 con 0x5580487ecb40 2024-09-15T22:11:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.406+0000 7fea917fa640 1 -- 172.21.15.5:0/3408136436 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa0099a40 con 0x5580487ecb40 2024-09-15T22:11:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.406+0000 7fea917fa640 1 -- 172.21.15.5:0/3408136436 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa0086160 con 0x5580487ecb40 2024-09-15T22:11:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.406+0000 7feab4392740 1 -- 172.21.15.5:0/3408136436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 msgr2=0x5580487ee120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.406+0000 7feab4392740 1 --2- 172.21.15.5:0/3408136436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487ee120 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feaa0097e00 tx=0x7feaa006d720 comp rx=0 tx=0).stop 2024-09-15T22:11:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.406+0000 7feab4392740 1 -- 172.21.15.5:0/3408136436 shutdown_connections 2024-09-15T22:11:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.406+0000 7feab4392740 1 --2- 172.21.15.5:0/3408136436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487ee660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.406+0000 7feab4392740 1 --2- 172.21.15.5:0/3408136436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487ee120 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.406+0000 7feab4392740 1 --2- 172.21.15.5:0/3408136436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487eb5d0 0x558048802310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.406+0000 7feab4392740 1 -- 172.21.15.5:0/3408136436 >> 172.21.15.5:0/3408136436 conn(0x5580487b4310 msgr2=0x5580487f2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.407+0000 7feab4392740 1 -- 172.21.15.5:0/3408136436 shutdown_connections 2024-09-15T22:11:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.407+0000 7feab4392740 1 -- 172.21.15.5:0/3408136436 wait complete. 2024-09-15T22:11:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.407+0000 7feab4392740 1 -- start start 2024-09-15T22:11:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.407+0000 7feab4392740 1 Processor -- start 2024-09-15T22:11:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.407+0000 7feab4392740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487eb5d0 0x5580487fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.407+0000 7feab4392740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487fed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.407+0000 7feab4392740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487efb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.407+0000 7feab4392740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1ad600 con 0x5580487ecb40 2024-09-15T22:11:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.407+0000 7feab4392740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1aea70 con 0x5580487f6370 2024-09-15T22:11:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.407+0000 7feab4392740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feaac1adcd0 con 0x5580487eb5d0 2024-09-15T22:11:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.407+0000 7feab4392740 10 client.0 _ll_register_callbacks cb 0x7feaa4009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feaaaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feaaaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487fed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50916/0 (socket says 172.21.15.5:50916) 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feaaaffd640 1 -- 172.21.15.5:0/1066770444 learned_addr learned my addr 172.21.15.5:0/1066770444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feaab7fe640 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487eb5d0 0x5580487fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feaabfff640 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feaaaffd640 1 -- 172.21.15.5:0/1066770444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487eb5d0 msgr2=0x5580487fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feaaaffd640 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487eb5d0 0x5580487fe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feaaaffd640 1 -- 172.21.15.5:0/1066770444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 msgr2=0x5580487efb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feaaaffd640 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487efb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feaaaffd640 1 -- 172.21.15.5:0/1066770444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580487f0390 con 0x5580487ecb40 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feaaaffd640 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487fed30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feaa009bb60 tx=0x7feaa0005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7afd176b10f0a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7fea8affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7fea8affd640 1 -- 172.21.15.5:0/1066770444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa00b4060 con 0x5580487ecb40 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7fea8affd640 1 -- 172.21.15.5:0/1066770444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7feaa0009cc0 con 0x5580487ecb40 2024-09-15T22:11:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feab4392740 1 -- 172.21.15.5:0/1066770444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feaa0099a40 con 0x5580487ecb40 2024-09-15T22:11:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7fea8affd640 1 -- 172.21.15.5:0/1066770444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feaa006b070 con 0x5580487ecb40 2024-09-15T22:11:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feab4392740 10 client.16008 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.408+0000 7feab4392740 1 -- 172.21.15.5:0/1066770444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558048874850 con 0x5580487ecb40 2024-09-15T22:11:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.409+0000 7fea8affd640 1 -- 172.21.15.5:0/1066770444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaa0099a40 con 0x5580487ecb40 2024-09-15T22:11:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.409+0000 7fea8affd640 1 -- 172.21.15.5:0/1066770444 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7feaa00cd050 con 0x5580487ecb40 2024-09-15T22:11:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.409+0000 7feab4392740 10 client.16008 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.409+0000 7feab4392740 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.409+0000 7feab4392740 20 client.16008 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.409+0000 7feab4392740 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558048874ed0 0x5580488772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.409+0000 7feab4392740 4 client.16008 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.409+0000 7feab4392740 1 -- 172.21.15.5:0/1066770444 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558048758d70 con 0x558048874ed0 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.409+0000 7feaab7fe640 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558048874ed0 0x5580488772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.409+0000 7feaab7fe640 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558048874ed0 0x5580488772f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=125ff5ea6a9469c9 server_cookie=545c200e0709cb2d in_seq=0 out_seq=0 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.409+0000 7fea8affd640 10 client.16008 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7fea8affd640 1 -- 172.21.15.5:0/1066770444 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558048758d70 con 0x558048874ed0 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7fea8affd640 10 client.16008 handle_command_reply: tid=0 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 1 client.16008 shutdown 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 1 -- 172.21.15.5:0/1066770444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 msgr2=0x5580487fed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487fed30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feaa009bb60 tx=0x7feaa0005b20 comp rx=0 tx=0).stop 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 1 -- 172.21.15.5:0/1066770444 shutdown_connections 2024-09-15T22:11:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558048874ed0 0x5580488772f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5580487f6370 0x5580487efb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580487ecb40 0x5580487fed30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 1 --2- 172.21.15.5:0/1066770444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5580487eb5d0 0x5580487fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 1 -- 172.21.15.5:0/1066770444 >> 172.21.15.5:0/1066770444 conn(0x5580487b4310 msgr2=0x5580487e64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7fea8affd640 0 client.16008 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 1 -- 172.21.15.5:0/1066770444 shutdown_connections 2024-09-15T22:11:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 1 -- 172.21.15.5:0/1066770444 wait complete. 2024-09-15T22:11:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.410+0000 7feab4392740 20 client.16008 trim_cache size 0 max 16384 2024-09-15T22:11:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:11:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 -- 172.21.15.5:0/370240184 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea940a3430 msgr2=0x7fea940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea940a3430 0x7fea940a58f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feaa4004ad0 tx=0x7feaa402a950 comp rx=0 tx=0).stop 2024-09-15T22:11:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 -- 172.21.15.5:0/370240184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaac1036c0 msgr2=0x7feaac198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaac1036c0 0x7feaac198980 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feaa000d770 tx=0x7feaa000dc30 comp rx=0 tx=0).stop 2024-09-15T22:11:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 -- 172.21.15.5:0/370240184 shutdown_connections 2024-09-15T22:11:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fea940a3430 0x7fea940a58f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feaac10c180 0x7feaac19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feaac1036c0 0x7feaac198980 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 --2- 172.21.15.5:0/370240184 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feaac102da0 0x7feaac198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 -- 172.21.15.5:0/370240184 >> 172.21.15.5:0/370240184 conn(0x7feaac0fc7a0 msgr2=0x7feaac0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 -- 172.21.15.5:0/370240184 shutdown_connections 2024-09-15T22:11:49.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:49.411+0000 7feab1fb9640 1 -- 172.21.15.5:0/370240184 wait complete. 2024-09-15T22:11:49.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:49 smithi099 ceph-mon[34892]: pgmap v90: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 366 op/s 2024-09-15T22:11:49.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:49 smithi129 ceph-mon[35314]: pgmap v90: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 3.5 MiB/s wr, 366 op/s 2024-09-15T22:11:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 140.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:11:51.219 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:11:51.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:11:51.491 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:51.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:51 smithi099 ceph-mon[34892]: pgmap v91: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 404 op/s 2024-09-15T22:11:51.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:51 smithi129 ceph-mon[35314]: pgmap v91: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 404 op/s 2024-09-15T22:11:51.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:51 smithi005 ceph-mon[30944]: pgmap v91: 129 pgs: 129 active+clean; 1.7 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 4.0 MiB/s wr, 404 op/s 2024-09-15T22:11:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.963+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/32580726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe4101de0 msgr2=0x7fdbe41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.963+0000 7fdbe8c2b640 1 --2- 172.21.15.5:0/32580726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe4101de0 0x7fdbe41021c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4009900 tx=0x7fdbd402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.964+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/32580726 shutdown_connections 2024-09-15T22:11:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.964+0000 7fdbe8c2b640 1 --2- 172.21.15.5:0/32580726 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4106f10 0x7fdbe410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.964+0000 7fdbe8c2b640 1 --2- 172.21.15.5:0/32580726 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbe4102790 0x7fdbe41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.964+0000 7fdbe8c2b640 1 --2- 172.21.15.5:0/32580726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe4101de0 0x7fdbe41021c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.964+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/32580726 >> 172.21.15.5:0/32580726 conn(0x7fdbe40fd8c0 msgr2=0x7fdbe40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.964+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/32580726 shutdown_connections 2024-09-15T22:11:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.964+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/32580726 wait complete. 2024-09-15T22:11:51.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.964+0000 7fdbe8c2b640 1 Processor -- start 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe8c2b640 1 -- start start 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe8c2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe4101de0 0x7fdbe419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe8c2b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbe4102790 0x7fdbe419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe8c2b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4106f10 0x7fdbe419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe8c2b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4111f90 con 0x7fdbe4101de0 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe8c2b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4111e10 con 0x7fdbe4102790 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe8c2b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe4112110 con 0x7fdbe4106f10 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe4101de0 0x7fdbe419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe4101de0 0x7fdbe419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50944/0 (socket says 172.21.15.5:50944) 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe259c640 1 -- 172.21.15.5:0/3153347030 learned_addr learned my addr 172.21.15.5:0/3153347030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbd9d9b640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbe4102790 0x7fdbe419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4106f10 0x7fdbe419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe259c640 1 -- 172.21.15.5:0/3153347030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4106f10 msgr2=0x7fdbe419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe259c640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4106f10 0x7fdbe419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe259c640 1 -- 172.21.15.5:0/3153347030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbe4102790 msgr2=0x7fdbe419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe259c640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbe4102790 0x7fdbe419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe259c640 1 -- 172.21.15.5:0/3153347030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe419f6a0 con 0x7fdbe4101de0 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.965+0000 7fdbe259c640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe4101de0 0x7fdbe419a6f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4005ab0 tx=0x7fdbd40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.966+0000 7fdbdbfff640 1 -- 172.21.15.5:0/3153347030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd403f030 con 0x7fdbe4101de0 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.966+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/3153347030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe419f930 con 0x7fdbe4101de0 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.966+0000 7fdbdbfff640 1 -- 172.21.15.5:0/3153347030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd4003ce0 con 0x7fdbe4101de0 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.966+0000 7fdbdbfff640 1 -- 172.21.15.5:0/3153347030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4035070 con 0x7fdbe4101de0 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.966+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/3153347030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe41a7270 con 0x7fdbe4101de0 2024-09-15T22:11:51.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.968+0000 7fdbdbfff640 1 -- 172.21.15.5:0/3153347030 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fdbd4042030 con 0x7fdbe4101de0 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.968+0000 7fdbeb004740 1 Processor -- start 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.968+0000 7fdbeb004740 1 -- start start 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1b96f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1b97480 0x5583e1ba1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 0x5583e1ba4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbeb004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5583e1afc280 con 0x5583e1b96b60 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbeb004740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5583e1aa3b10 con 0x5583e1b97480 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbeb004740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5583e1af8240 con 0x5583e1ba1d40 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbe2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1b96f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbe2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1b96f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50958/0 (socket says 172.21.15.5:50958) 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbe2d9d640 1 -- 172.21.15.5:0/2308505602 learned_addr learned my addr 172.21.15.5:0/2308505602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbdbfff640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdbd00a33b0 0x7fdbd00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbd9d9b640 1 --2- 172.21.15.5:0/2308505602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 0x5583e1ba4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbe259c640 1 --2- 172.21.15.5:0/2308505602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1b97480 0x5583e1ba1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbe2d9d640 1 -- 172.21.15.5:0/2308505602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 msgr2=0x5583e1ba4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/2308505602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 0x5583e1ba4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbe2d9d640 1 -- 172.21.15.5:0/2308505602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1b97480 msgr2=0x5583e1ba1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/2308505602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1b97480 0x5583e1ba1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbe2d9d640 1 -- 172.21.15.5:0/2308505602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e1ab01f0 con 0x5583e1b96b60 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbd9d9b640 1 --2- 172.21.15.5:0/2308505602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 0x5583e1ba4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbe259c640 1 --2- 172.21.15.5:0/2308505602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1b97480 0x5583e1ba1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbdbfff640 1 -- 172.21.15.5:0/3153347030 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fdbd40f0a60 con 0x7fdbe4101de0 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.969+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdbd00a33b0 0x7fdbd00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.970+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/2308505602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1b96f40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00c4a0 tx=0x7fdbdc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d257eaddce14b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.970+0000 7fdbd8d99640 1 -- 172.21.15.5:0/2308505602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc018640 con 0x5583e1b96b60 2024-09-15T22:11:51.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.970+0000 7fdbd8d99640 1 -- 172.21.15.5:0/2308505602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbdc01e070 con 0x5583e1b96b60 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.970+0000 7fdbd8d99640 1 -- 172.21.15.5:0/2308505602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00a970 con 0x5583e1b96b60 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 -- 172.21.15.5:0/2308505602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 msgr2=0x5583e1b96f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 --2- 172.21.15.5:0/2308505602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1b96f40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00c4a0 tx=0x7fdbdc00c960 comp rx=0 tx=0).stop 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdbd00a33b0 0x7fdbd00a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc01d000 tx=0x7fdbdc017000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 -- 172.21.15.5:0/2308505602 shutdown_connections 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 --2- 172.21.15.5:0/2308505602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 0x5583e1ba4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 --2- 172.21.15.5:0/2308505602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1b97480 0x5583e1ba1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 --2- 172.21.15.5:0/2308505602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1b96f40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 -- 172.21.15.5:0/2308505602 >> 172.21.15.5:0/2308505602 conn(0x5583e1b930f0 msgr2=0x5583e1b934d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 -- 172.21.15.5:0/2308505602 shutdown_connections 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 -- 172.21.15.5:0/2308505602 wait complete. 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 -- start start 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 Processor -- start 2024-09-15T22:11:51.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1b96b60 0x5583e1b9ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1b9f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 0x5583e1b9f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5583e1ba4910 con 0x5583e1b97480 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5583e1ab6f10 con 0x5583e1ba1d40 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5583e1ba4a90 con 0x5583e1b96b60 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbeb004740 10 client.0 _ll_register_callbacks cb 0x5583e1a7fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.971+0000 7fdbd9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1b9f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbd9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1b9f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50966/0 (socket says 172.21.15.5:50966) 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbd9d9b640 1 -- 172.21.15.5:0/1930613338 learned_addr learned my addr 172.21.15.5:0/1930613338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbe259c640 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1b96b60 0x5583e1b9ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 0x5583e1b9f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbd9d9b640 1 -- 172.21.15.5:0/1930613338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1b96b60 msgr2=0x5583e1b9ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:51.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbd9d9b640 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1b96b60 0x5583e1b9ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbd9d9b640 1 -- 172.21.15.5:0/1930613338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 msgr2=0x5583e1b9f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbd9d9b640 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 0x5583e1b9f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbd9d9b640 1 -- 172.21.15.5:0/1930613338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e186d6a0 con 0x5583e1b97480 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 0x5583e1b9f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbd9d9b640 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1b9f140 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc010810 tx=0x7fdbcc010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0fd4012a12288d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/1930613338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00ed40 con 0x5583e1b97480 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/1930613338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fdbcc01c480 con 0x5583e1b97480 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbeb004740 1 -- 172.21.15.5:0/1930613338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbdc01e070 con 0x5583e1b97480 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/1930613338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdbcc025050 con 0x5583e1b97480 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbeb004740 10 client.16020 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:51.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.972+0000 7fdbeb004740 1 -- 172.21.15.5:0/1930613338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5583e1bb0550 con 0x5583e1b97480 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.973+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/1930613338 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbdc01e070 con 0x5583e1b97480 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.973+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/1930613338 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdbcc0116e0 con 0x5583e1b97480 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.973+0000 7fdbeb004740 10 client.16020 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.973+0000 7fdbeb004740 10 client.16020 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.973+0000 7fdbeb004740 20 client.16020 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.973+0000 7fdbeb004740 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583e1bb1a60 0x5583e1bb3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.973+0000 7fdbeb004740 4 client.16020 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.973+0000 7fdbeb004740 1 -- 172.21.15.5:0/1930613338 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583e1b097f0 con 0x5583e1bb1a60 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.973+0000 7fdbe259c640 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583e1bb1a60 0x5583e1bb3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.974+0000 7fdbc9ffb640 10 client.16020 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.974+0000 7fdbe259c640 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583e1bb1a60 0x5583e1bb3ea0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15abc455291e9773 server_cookie=92d0133869fdd9f8 in_seq=0 out_seq=0 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.974+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/1930613338 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583e1b097f0 con 0x5583e1bb1a60 2024-09-15T22:11:51.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.974+0000 7fdbc9ffb640 10 client.16020 handle_command_reply: tid=0 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.974+0000 7fdbeb004740 1 client.16020 shutdown 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.974+0000 7fdbeb004740 1 -- 172.21.15.5:0/1930613338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 msgr2=0x5583e1b9f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.974+0000 7fdbeb004740 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1b9f140 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc010810 tx=0x7fdbcc010cd0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.975+0000 7fdbeb004740 1 -- 172.21.15.5:0/1930613338 shutdown_connections 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.975+0000 7fdbeb004740 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583e1bb1a60 0x5583e1bb3ea0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.975+0000 7fdbeb004740 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 0x5583e1b9f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.975+0000 7fdbeb004740 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1b9f140 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.975+0000 7fdbeb004740 1 --2- 172.21.15.5:0/1930613338 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1b96b60 0x5583e1b9ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.975+0000 7fdbeb004740 1 -- 172.21.15.5:0/1930613338 >> 172.21.15.5:0/1930613338 conn(0x5583e1b930f0 msgr2=0x5583e1b98e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.975+0000 7fdbc9ffb640 0 client.16020 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.975+0000 7fdbeb004740 1 -- 172.21.15.5:0/1930613338 shutdown_connections 2024-09-15T22:11:51.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.975+0000 7fdbeb004740 1 -- 172.21.15.5:0/1930613338 wait complete. 2024-09-15T22:11:51.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.975+0000 7fdbeb004740 20 client.16020 trim_cache size 0 max 16384 2024-09-15T22:11:51.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.992+0000 7fdbeb004740 1 Processor -- start 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.992+0000 7fdbeb004740 1 -- start start 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.992+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1b96b60 0x5583e1b99610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.992+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1bb9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.992+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 0x5583e1bb9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.992+0000 7fdbeb004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41b3ae0 con 0x5583e1b97480 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.992+0000 7fdbeb004740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41b13b0 con 0x5583e1b96b60 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.992+0000 7fdbeb004740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41af200 con 0x5583e1ba1d40 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbd9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1bb9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbd9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1bb9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50972/0 (socket says 172.21.15.5:50972) 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbd9d9b640 1 -- 172.21.15.5:0/1535545460 learned_addr learned my addr 172.21.15.5:0/1535545460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/1535545460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 0x5583e1bb9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbe259c640 1 --2- 172.21.15.5:0/1535545460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1b96b60 0x5583e1b99610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbd9d9b640 1 -- 172.21.15.5:0/1535545460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 msgr2=0x5583e1bb9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbd9d9b640 1 --2- 172.21.15.5:0/1535545460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 0x5583e1bb9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbd9d9b640 1 -- 172.21.15.5:0/1535545460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1b96b60 msgr2=0x5583e1b99610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbd9d9b640 1 --2- 172.21.15.5:0/1535545460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1b96b60 0x5583e1b99610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbd9d9b640 1 -- 172.21.15.5:0/1535545460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e1b92f20 con 0x5583e1b97480 2024-09-15T22:11:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/1535545460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 0x5583e1bb9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbd9d9b640 1 --2- 172.21.15.5:0/1535545460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1bb9770 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc011ab0 tx=0x7fdbcc010fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f8a73a4f76df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbcbfff640 1 -- 172.21.15.5:0/1535545460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00c5e0 con 0x5583e1b97480 2024-09-15T22:11:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbcbfff640 1 -- 172.21.15.5:0/1535545460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbcc02d070 con 0x5583e1b97480 2024-09-15T22:11:51.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.993+0000 7fdbcbfff640 1 -- 172.21.15.5:0/1535545460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc005370 con 0x5583e1b97480 2024-09-15T22:11:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.994+0000 7fdbeb004740 1 -- 172.21.15.5:0/1535545460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 msgr2=0x5583e1bb9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.994+0000 7fdbeb004740 1 --2- 172.21.15.5:0/1535545460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1bb9770 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc011ab0 tx=0x7fdbcc010fa0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.994+0000 7fdbeb004740 1 -- 172.21.15.5:0/1535545460 shutdown_connections 2024-09-15T22:11:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.994+0000 7fdbeb004740 1 --2- 172.21.15.5:0/1535545460 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1ba1d40 0x5583e1bb9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.994+0000 7fdbeb004740 1 --2- 172.21.15.5:0/1535545460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b97480 0x5583e1bb9770 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.994+0000 7fdbeb004740 1 --2- 172.21.15.5:0/1535545460 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1b96b60 0x5583e1b99610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.994+0000 7fdbeb004740 1 -- 172.21.15.5:0/1535545460 >> 172.21.15.5:0/1535545460 conn(0x5583e1b930f0 msgr2=0x5583e1ba0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:51.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.994+0000 7fdbeb004740 1 -- 172.21.15.5:0/1535545460 shutdown_connections 2024-09-15T22:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.994+0000 7fdbeb004740 1 -- 172.21.15.5:0/1535545460 wait complete. 2024-09-15T22:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.994+0000 7fdbeb004740 1 -- start start 2024-09-15T22:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.994+0000 7fdbeb004740 1 Processor -- start 2024-09-15T22:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1baa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1b97480 0x5583e1b9b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbeb004740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 0x5583e1b9bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbeb004740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41af870 con 0x5583e1b96b60 2024-09-15T22:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbeb004740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41b0ce0 con 0x5583e1ba1d40 2024-09-15T22:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbeb004740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe41aff40 con 0x5583e1b97480 2024-09-15T22:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbe259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1baa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbe259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1baa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50982/0 (socket says 172.21.15.5:50982) 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbe259c640 1 -- 172.21.15.5:0/3301133044 learned_addr learned my addr 172.21.15.5:0/3301133044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbeb004740 10 client.0 _ll_register_callbacks cb 0x7fdbcc00c410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 0x5583e1b9bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbe259c640 1 -- 172.21.15.5:0/3301133044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1b97480 msgr2=0x5583e1b9b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbe259c640 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1b97480 0x5583e1b9b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbe259c640 1 -- 172.21.15.5:0/3301133044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 msgr2=0x5583e1b9bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbe259c640 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 0x5583e1b9bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbe259c640 1 -- 172.21.15.5:0/3301133044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e1b9c560 con 0x5583e1b96b60 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbe2d9d640 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 0x5583e1b9bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbe259c640 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1baa360 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4006010 tx=0x7fdbd40bc830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac21edd11e836f2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/3301133044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4049040 con 0x5583e1b96b60 2024-09-15T22:11:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.995+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/3301133044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fdbd403bd70 con 0x5583e1b96b60 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbeb004740 1 -- 172.21.15.5:0/3301133044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbcc02d070 con 0x5583e1b96b60 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/3301133044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdbd40c0d50 con 0x5583e1b96b60 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbeb004740 10 client.16026 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbeb004740 1 -- 172.21.15.5:0/3301133044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5583e1c1c6c0 con 0x5583e1b96b60 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/3301133044 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbcc02d070 con 0x5583e1b96b60 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/3301133044 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdbd40ee090 con 0x5583e1b96b60 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbeb004740 10 client.16026 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbeb004740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbeb004740 20 client.16026 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbeb004740 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583e1c1ce40 0x5583e1c1f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbeb004740 4 client.16026 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbeb004740 1 -- 172.21.15.5:0/3301133044 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5583e1b03d00 con 0x5583e1c1ce40 2024-09-15T22:11:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.996+0000 7fdbd9d9b640 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583e1c1ce40 0x5583e1c1f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.997+0000 7fdbd9d9b640 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583e1c1ce40 0x5583e1c1f260 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d7599e6b1dc88a server_cookie=2e1f58045f1a7169 in_seq=0 out_seq=0 2024-09-15T22:11:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.997+0000 7fdbc9ffb640 10 client.16026 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbc9ffb640 1 -- 172.21.15.5:0/3301133044 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5583e1b03d00 con 0x5583e1c1ce40 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbc9ffb640 10 client.16026 handle_command_reply: tid=0 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 1 client.16026 shutdown 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 1 -- 172.21.15.5:0/3301133044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 msgr2=0x5583e1baa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1baa360 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4006010 tx=0x7fdbd40bc830 comp rx=0 tx=0).stop 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 1 -- 172.21.15.5:0/3301133044 shutdown_connections 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583e1c1ce40 0x5583e1c1f260 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583e1ba1d40 0x5583e1b9bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583e1b97480 0x5583e1b9b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 1 --2- 172.21.15.5:0/3301133044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583e1b96b60 0x5583e1baa360 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 1 -- 172.21.15.5:0/3301133044 >> 172.21.15.5:0/3301133044 conn(0x5583e1b930f0 msgr2=0x5583e1b9d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbc9ffb640 0 client.16026 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 1 -- 172.21.15.5:0/3301133044 shutdown_connections 2024-09-15T22:11:52.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 1 -- 172.21.15.5:0/3301133044 wait complete. 2024-09-15T22:11:52.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.998+0000 7fdbeb004740 20 client.16026 trim_cache size 0 max 16384 2024-09-15T22:11:52.005 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:11:52.005 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:11:52.005 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "421f8241-a342-4789-80bf-f0cef09b317d", 2024-09-15T22:11:52.005 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:11:52.005 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:11:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/3153347030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdbd00a33b0 msgr2=0x7fdbd00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdbd00a33b0 0x7fdbd00a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc01d000 tx=0x7fdbdc017000 comp rx=0 tx=0).stop 2024-09-15T22:11:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/3153347030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe4101de0 msgr2=0x7fdbe419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe4101de0 0x7fdbe419a6f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4005ab0 tx=0x7fdbd40042f0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/3153347030 shutdown_connections 2024-09-15T22:11:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdbd00a33b0 0x7fdbd00a5870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdbe4106f10 0x7fdbe419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdbe4102790 0x7fdbe419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 --2- 172.21.15.5:0/3153347030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe4101de0 0x7fdbe419a6f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/3153347030 >> 172.21.15.5:0/3153347030 conn(0x7fdbe40fd8c0 msgr2=0x7fdbe40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/3153347030 shutdown_connections 2024-09-15T22:11:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:51.999+0000 7fdbe8c2b640 1 -- 172.21.15.5:0/3153347030 wait complete. 2024-09-15T22:11:52.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:52 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:11:52.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:52 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:52.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:52 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:11:52.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:52 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:11:52.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:52 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:52.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:52 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:11:52.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:52 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:11:52.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:52 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:52.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:52 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:11:52.835 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:11:53.096 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:53.488 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:53 smithi005 ceph-mon[30944]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 403 op/s 2024-09-15T22:11:53.488 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:53 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:53.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:53 smithi099 ceph-mon[34892]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 403 op/s 2024-09-15T22:11:53.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:53 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:53.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:53 smithi129 ceph-mon[35314]: pgmap v92: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 4.9 MiB/s wr, 403 op/s 2024-09-15T22:11:53.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:53 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:11:53.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.571+0000 7f98d06b4640 1 -- 172.21.15.5:0/567796887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8106a80 msgr2=0x7f98c8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.571+0000 7f98d06b4640 1 --2- 172.21.15.5:0/567796887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8106a80 0x7f98c8106e60 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009930 tx=0x7f98c402a420 comp rx=0 tx=0).stop 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.571+0000 7f98d06b4640 1 -- 172.21.15.5:0/567796887 shutdown_connections 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.571+0000 7f98d06b4640 1 --2- 172.21.15.5:0/567796887 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8101420 0x7f98c810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.571+0000 7f98d06b4640 1 --2- 172.21.15.5:0/567796887 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98c8100a80 0x7f98c8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.571+0000 7f98d06b4640 1 --2- 172.21.15.5:0/567796887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8106a80 0x7f98c8106e60 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.571+0000 7f98d06b4640 1 -- 172.21.15.5:0/567796887 >> 172.21.15.5:0/567796887 conn(0x7f98c80fc7a0 msgr2=0x7f98c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.571+0000 7f98d06b4640 1 -- 172.21.15.5:0/567796887 shutdown_connections 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.571+0000 7f98d06b4640 1 -- 172.21.15.5:0/567796887 wait complete. 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98d06b4640 1 Processor -- start 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98d06b4640 1 -- start start 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98d06b4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98c8100a80 0x7f98c8198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98d06b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8101420 0x7f98c8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98d06b4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8106a80 0x7f98c819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98d06b4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8110e40 con 0x7f98c8101420 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98d06b4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8110cc0 con 0x7f98c8106a80 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98d06b4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8110fc0 con 0x7f98c8100a80 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98cdc4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8101420 0x7f98c8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98cdc4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8101420 0x7f98c8198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51316/0 (socket says 172.21.15.5:51316) 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98cdc4f640 1 -- 172.21.15.5:0/2024054823 learned_addr learned my addr 172.21.15.5:0/2024054823 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98ce450640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98c8100a80 0x7f98c8198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.572+0000 7f98cec51640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8106a80 0x7f98c819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.573+0000 7f98cdc4f640 1 -- 172.21.15.5:0/2024054823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98c8100a80 msgr2=0x7f98c8198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.573+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98c8100a80 0x7f98c8198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.573+0000 7f98cdc4f640 1 -- 172.21.15.5:0/2024054823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8106a80 msgr2=0x7f98c819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.573+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8106a80 0x7f98c819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.573+0000 7f98cdc4f640 1 -- 172.21.15.5:0/2024054823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c819d3c0 con 0x7f98c8101420 2024-09-15T22:11:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.573+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8101420 0x7f98c8198950 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f98b800c920 tx=0x7f98b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.573+0000 7f98bf7fe640 1 -- 172.21.15.5:0/2024054823 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8007c70 con 0x7f98c8101420 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.573+0000 7f98d06b4640 1 -- 172.21.15.5:0/2024054823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c819d6b0 con 0x7f98c8101420 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.573+0000 7f98bf7fe640 1 -- 172.21.15.5:0/2024054823 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98b8007e10 con 0x7f98c8101420 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.573+0000 7f98bf7fe640 1 -- 172.21.15.5:0/2024054823 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8018890 con 0x7f98c8101420 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.573+0000 7f98d06b4640 1 -- 172.21.15.5:0/2024054823 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c81a4f90 con 0x7f98c8101420 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.575+0000 7f98bf7fe640 1 -- 172.21.15.5:0/2024054823 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f98b8018a50 con 0x7f98c8101420 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98d2a8d740 1 Processor -- start 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98d2a8d740 1 -- start start 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9acdc0 0x55b41f9b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9b7680 0x55b41f9b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b41f9122e0 con 0x55b41f9ac4a0 2024-09-15T22:11:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b41f8b9ce0 con 0x55b41f9acdc0 2024-09-15T22:11:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b41f90e2a0 con 0x55b41f9b7680 2024-09-15T22:11:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98cec51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98cec51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51322/0 (socket says 172.21.15.5:51322) 2024-09-15T22:11:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98cec51640 1 -- 172.21.15.5:0/1567218772 learned_addr learned my addr 172.21.15.5:0/1567218772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.576+0000 7f98bf7fe640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f98b40a33b0 0x7f98b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/1567218772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9b7680 0x55b41f9b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98cec51640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f98b40a33b0 0x7f98b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98ce450640 1 --2- 172.21.15.5:0/1567218772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9acdc0 0x55b41f9b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98cec51640 1 -- 172.21.15.5:0/1567218772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9b7680 msgr2=0x55b41f9b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98cec51640 1 --2- 172.21.15.5:0/1567218772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9b7680 0x55b41f9b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98cec51640 1 -- 172.21.15.5:0/1567218772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9acdc0 msgr2=0x55b41f9b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98cec51640 1 --2- 172.21.15.5:0/1567218772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9acdc0 0x55b41f9b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98cec51640 1 -- 172.21.15.5:0/1567218772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b41f89e440 con 0x55b41f9ac4a0 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/1567218772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9b7680 0x55b41f9b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98bf7fe640 1 -- 172.21.15.5:0/2024054823 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f98b80d01e0 con 0x7f98c8101420 2024-09-15T22:11:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98cec51640 1 --2- 172.21.15.5:0/1567218772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9ac880 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f98c0005c40 tx=0x7f98c0013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca120a6da4906bea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98cec51640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f98b40a33b0 0x7f98b40a5870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f98c000f850 tx=0x7f98c001f000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98bcff9640 1 -- 172.21.15.5:0/1567218772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c000fde0 con 0x55b41f9ac4a0 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98bcff9640 1 -- 172.21.15.5:0/1567218772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c0022020 con 0x55b41f9ac4a0 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.577+0000 7f98bcff9640 1 -- 172.21.15.5:0/1567218772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c0010670 con 0x55b41f9ac4a0 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.578+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1567218772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 msgr2=0x55b41f9ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.578+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1567218772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9ac880 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f98c0005c40 tx=0x7f98c0013f50 comp rx=0 tx=0).stop 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.578+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1567218772 shutdown_connections 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.578+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1567218772 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9b7680 0x55b41f9b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.578+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1567218772 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9acdc0 0x55b41f9b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.578+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1567218772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9ac880 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.579+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1567218772 >> 172.21.15.5:0/1567218772 conn(0x55b41f9a8a30 msgr2=0x55b41f9a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.579+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1567218772 shutdown_connections 2024-09-15T22:11:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.579+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1567218772 wait complete. 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.579+0000 7f98d2a8d740 1 -- start start 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.579+0000 7f98d2a8d740 1 Processor -- start 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9acdc0 0x55b41f9b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9b7680 0x55b41f9b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b41f9ba250 con 0x55b41f9ac4a0 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b41f8cd050 con 0x55b41f9acdc0 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b41f9ba3d0 con 0x55b41f9b7680 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98d2a8d740 10 client.0 _ll_register_callbacks cb 0x55b41f90d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98ce450640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98ce450640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9b6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51332/0 (socket says 172.21.15.5:51332) 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98ce450640 1 -- 172.21.15.5:0/1408649791 learned_addr learned my addr 172.21.15.5:0/1408649791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9acdc0 0x55b41f9b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98cec51640 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9b7680 0x55b41f9b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98ce450640 1 -- 172.21.15.5:0/1408649791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9b7680 msgr2=0x55b41f9b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98ce450640 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9b7680 0x55b41f9b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98ce450640 1 -- 172.21.15.5:0/1408649791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9acdc0 msgr2=0x55b41f9b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98ce450640 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9acdc0 0x55b41f9b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98ce450640 1 -- 172.21.15.5:0/1408649791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b41f6833c0 con 0x55b41f9ac4a0 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98ce450640 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9b6fc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f98c402a940 tx=0x7f98c40317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63487ececbfc5f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.580+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1408649791 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4035040 con 0x55b41f9ac4a0 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1408649791 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f98c4004240 con 0x55b41f9ac4a0 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1408649791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c0022020 con 0x55b41f9ac4a0 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1408649791 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f98c4004440 con 0x55b41f9ac4a0 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98d2a8d740 10 client.16038 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:53.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1408649791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b41f9c5c00 con 0x55b41f9ac4a0 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1408649791 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c0022020 con 0x55b41f9ac4a0 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1408649791 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f98c403c6e0 con 0x55b41f9ac4a0 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98d2a8d740 10 client.16038 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98d2a8d740 10 client.16038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98d2a8d740 20 client.16038 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b41f9c74f0 0x55b41f9c9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98d2a8d740 4 client.16038 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1408649791 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b41f91f850 con 0x55b41f9c74f0 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.581+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b41f9c74f0 0x55b41f9c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.582+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b41f9c74f0 0x55b41f9c9940 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09c793f9f494d34 server_cookie=7b76b229ed7cc507 in_seq=0 out_seq=0 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.582+0000 7f98b1ffb640 10 client.16038 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1408649791 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b41f91f850 con 0x55b41f9c74f0 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98b1ffb640 10 client.16038 handle_command_reply: tid=0 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 1 client.16038 shutdown 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1408649791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 msgr2=0x55b41f9b6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9b6fc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f98c402a940 tx=0x7f98c40317e0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1408649791 shutdown_connections 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9b7680 0x55b41f9b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b41f9c74f0 0x55b41f9c9940 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9acdc0 0x55b41f9b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1408649791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9b6fc0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1408649791 >> 172.21.15.5:0/1408649791 conn(0x55b41f9a8a30 msgr2=0x55b41f9ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98b1ffb640 0 client.16038 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1408649791 shutdown_connections 2024-09-15T22:11:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1408649791 wait complete. 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.583+0000 7f98d2a8d740 20 client.16038 trim_cache size 0 max 16384 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.600+0000 7f98d2a8d740 1 Processor -- start 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.600+0000 7f98d2a8d740 1 -- start start 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9acdc0 0x55b41f9c9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9b7680 0x55b41f9af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81b19a0 con 0x55b41f9ac4a0 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81af270 con 0x55b41f9b7680 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ad0c0 con 0x55b41f9acdc0 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98ce450640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98ce450640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9c34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51348/0 (socket says 172.21.15.5:51348) 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98ce450640 1 -- 172.21.15.5:0/3601997029 learned_addr learned my addr 172.21.15.5:0/3601997029 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98cec51640 1 --2- 172.21.15.5:0/3601997029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9b7680 0x55b41f9af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/3601997029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9acdc0 0x55b41f9c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98ce450640 1 -- 172.21.15.5:0/3601997029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9acdc0 msgr2=0x55b41f9c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98ce450640 1 --2- 172.21.15.5:0/3601997029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9acdc0 0x55b41f9c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98ce450640 1 -- 172.21.15.5:0/3601997029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9b7680 msgr2=0x55b41f9af7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98ce450640 1 --2- 172.21.15.5:0/3601997029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9b7680 0x55b41f9af7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98ce450640 1 -- 172.21.15.5:0/3601997029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b41f9b3220 con 0x55b41f9ac4a0 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98cec51640 1 --2- 172.21.15.5:0/3601997029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9b7680 0x55b41f9af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98ce450640 1 --2- 172.21.15.5:0/3601997029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9c34b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009850 tx=0x7f98c4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828d0f786bb33224 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98b3fff640 1 -- 172.21.15.5:0/3601997029 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4058070 con 0x55b41f9ac4a0 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98b3fff640 1 -- 172.21.15.5:0/3601997029 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c4005030 con 0x55b41f9ac4a0 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.601+0000 7f98b3fff640 1 -- 172.21.15.5:0/3601997029 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4035070 con 0x55b41f9ac4a0 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.602+0000 7f98d2a8d740 1 -- 172.21.15.5:0/3601997029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 msgr2=0x55b41f9c34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.602+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/3601997029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9c34b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009850 tx=0x7f98c4005fd0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.602+0000 7f98d2a8d740 1 -- 172.21.15.5:0/3601997029 shutdown_connections 2024-09-15T22:11:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.602+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/3601997029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9b7680 0x55b41f9af7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.602+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/3601997029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9acdc0 0x55b41f9c9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.602+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/3601997029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9c34b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.602+0000 7f98d2a8d740 1 -- 172.21.15.5:0/3601997029 >> 172.21.15.5:0/3601997029 conn(0x55b41f9a8a30 msgr2=0x55b41f9af0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.602+0000 7f98d2a8d740 1 -- 172.21.15.5:0/3601997029 shutdown_connections 2024-09-15T22:11:53.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.602+0000 7f98d2a8d740 1 -- 172.21.15.5:0/3601997029 wait complete. 2024-09-15T22:11:53.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.602+0000 7f98d2a8d740 1 -- start start 2024-09-15T22:11:53.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98d2a8d740 1 Processor -- start 2024-09-15T22:11:53.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9bf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9acdc0 0x55b41f9bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98d2a8d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9b7680 0x55b41f9bac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ad730 con 0x55b41f9ac4a0 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81aeba0 con 0x55b41f9b7680 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98d2a8d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81ade00 con 0x55b41f9acdc0 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98d2a8d740 10 client.0 _ll_register_callbacks cb 0x7f98c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98ce450640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9bf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98ce450640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9bf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51364/0 (socket says 172.21.15.5:51364) 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98ce450640 1 -- 172.21.15.5:0/1332891896 learned_addr learned my addr 172.21.15.5:0/1332891896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9acdc0 0x55b41f9bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98cec51640 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9b7680 0x55b41f9bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98ce450640 1 -- 172.21.15.5:0/1332891896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9acdc0 msgr2=0x55b41f9bfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98ce450640 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9acdc0 0x55b41f9bfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98ce450640 1 -- 172.21.15.5:0/1332891896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9b7680 msgr2=0x55b41f9bac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98ce450640 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9b7680 0x55b41f9bac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98ce450640 1 -- 172.21.15.5:0/1332891896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b41f9bb4b0 con 0x55b41f9ac4a0 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98ce450640 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9bf8f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f98c402a460 tx=0x7f98c403ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8de1c9958da3fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.603+0000 7f98b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1332891896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4062060 con 0x55b41f9ac4a0 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1332891896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f98c40053e0 con 0x55b41f9ac4a0 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1332891896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98c4005030 con 0x55b41f9ac4a0 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1332891896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f98c4069050 con 0x55b41f9ac4a0 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98d2a8d740 10 client.16044 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1332891896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b41fa31d80 con 0x55b41f9ac4a0 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1332891896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98c4005030 con 0x55b41f9ac4a0 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1332891896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f98c4004c40 con 0x55b41f9ac4a0 2024-09-15T22:11:53.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98d2a8d740 10 client.16044 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98d2a8d740 10 client.16044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98d2a8d740 20 client.16044 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b41fa321b0 0x55b41fa345a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98d2a8d740 4 client.16044 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1332891896 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b41f9c9cd0 con 0x55b41fa321b0 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.604+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b41fa321b0 0x55b41fa345a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98cdc4f640 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b41fa321b0 0x55b41fa345a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9acf6d5964ecd2b6 server_cookie=d26c9dbf68220030 in_seq=0 out_seq=0 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98b1ffb640 10 client.16044 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98b1ffb640 1 -- 172.21.15.5:0/1332891896 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b41f9c9cd0 con 0x55b41fa321b0 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98b1ffb640 10 client.16044 handle_command_reply: tid=0 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98d2a8d740 1 client.16044 shutdown 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1332891896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 msgr2=0x55b41f9bf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9bf8f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f98c402a460 tx=0x7f98c403ccb0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1332891896 shutdown_connections 2024-09-15T22:11:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b41fa321b0 0x55b41fa345a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b41f9b7680 0x55b41f9bac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b41f9acdc0 0x55b41f9bfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98b1ffb640 0 client.16044 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98d2a8d740 1 --2- 172.21.15.5:0/1332891896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b41f9ac4a0 0x55b41f9bf8f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1332891896 >> 172.21.15.5:0/1332891896 conn(0x55b41f9a8a30 msgr2=0x55b41f9a7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1332891896 shutdown_connections 2024-09-15T22:11:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.605+0000 7f98d2a8d740 1 -- 172.21.15.5:0/1332891896 wait complete. 2024-09-15T22:11:53.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d2a8d740 20 client.16044 trim_cache size 0 max 16384 2024-09-15T22:11:53.677 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stdout: "421f8241-a342-4789-80bf-f0cef09b317d": { 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "421f8241-a342-4789-80bf-f0cef09b317d", 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 -- 172.21.15.5:0/2024054823 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f98b40a33b0 msgr2=0x7f98b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f98b40a33b0 0x7f98b40a5870 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f98c000f850 tx=0x7f98c001f000 comp rx=0 tx=0).stop 2024-09-15T22:11:53.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 -- 172.21.15.5:0/2024054823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8101420 msgr2=0x7f98c8198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:53.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8101420 0x7f98c8198950 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f98b800c920 tx=0x7f98b800cde0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 -- 172.21.15.5:0/2024054823 shutdown_connections 2024-09-15T22:11:53.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f98b40a33b0 0x7f98b40a5870 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f98c8106a80 0x7f98c819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c8101420 0x7f98c8198950 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 --2- 172.21.15.5:0/2024054823 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f98c8100a80 0x7f98c8198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:53.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 -- 172.21.15.5:0/2024054823 >> 172.21.15.5:0/2024054823 conn(0x7f98c80fc7a0 msgr2=0x7f98c806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:53.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 -- 172.21.15.5:0/2024054823 shutdown_connections 2024-09-15T22:11:53.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:53.606+0000 7f98d06b4640 1 -- 172.21.15.5:0/2024054823 wait complete. 2024-09-15T22:11:54.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:11:54.534 INFO:tasks.cephfs.filesystem:scrub status for tag:421f8241-a342-4789-80bf-f0cef09b317d - {'path': '/', 'tag': '421f8241-a342-4789-80bf-f0cef09b317d', 'options': 'recursive,force'} 2024-09-15T22:11:54.606 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 145.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:11:55.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.096+0000 7ff0ab6df640 1 -- 172.21.15.5:0/719751844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0a4106f10 msgr2=0x7ff0a410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:55.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.096+0000 7ff0ab6df640 1 --2- 172.21.15.5:0/719751844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0a4106f10 0x7ff0a410ea90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009ca0 tx=0x7ff09c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:11:55.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.096+0000 7ff0ab6df640 1 -- 172.21.15.5:0/719751844 shutdown_connections 2024-09-15T22:11:55.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.096+0000 7ff0ab6df640 1 --2- 172.21.15.5:0/719751844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0a4106f10 0x7ff0a410ea90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:55.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.096+0000 7ff0ab6df640 1 --2- 172.21.15.5:0/719751844 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0a4102790 0x7ff0a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:55.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.096+0000 7ff0ab6df640 1 --2- 172.21.15.5:0/719751844 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a4101de0 0x7ff0a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:55.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.096+0000 7ff0ab6df640 1 -- 172.21.15.5:0/719751844 >> 172.21.15.5:0/719751844 conn(0x7ff0a40fd8c0 msgr2=0x7ff0a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:55.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.096+0000 7ff0ab6df640 1 -- 172.21.15.5:0/719751844 shutdown_connections 2024-09-15T22:11:55.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.096+0000 7ff0ab6df640 1 -- 172.21.15.5:0/719751844 wait complete. 2024-09-15T22:11:55.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0ab6df640 1 Processor -- start 2024-09-15T22:11:55.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0ab6df640 1 -- start start 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0ab6df640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0a4101de0 0x7ff0a41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0ab6df640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a4102790 0x7ff0a4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0ab6df640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0a4106f10 0x7ff0a419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0ab6df640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4112010 con 0x7ff0a4101de0 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0ab6df640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4111e90 con 0x7ff0a4102790 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0ab6df640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4112190 con 0x7ff0a4106f10 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0a947b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0a4101de0 0x7ff0a41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0a947b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0a4101de0 0x7ff0a41985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51394/0 (socket says 172.21.15.5:51394) 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0a947b640 1 -- 172.21.15.5:0/350512519 learned_addr learned my addr 172.21.15.5:0/350512519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0a8c7a640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a4102790 0x7ff0a4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0a9c7c640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0a4106f10 0x7ff0a419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0a947b640 1 -- 172.21.15.5:0/350512519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0a4106f10 msgr2=0x7ff0a419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:55.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0a947b640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0a4106f10 0x7ff0a419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0a947b640 1 -- 172.21.15.5:0/350512519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a4102790 msgr2=0x7ff0a4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0a947b640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a4102790 0x7ff0a4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0a947b640 1 -- 172.21.15.5:0/350512519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a419d5a0 con 0x7ff0a4101de0 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.097+0000 7ff0a9c7c640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0a4106f10 0x7ff0a419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.098+0000 7ff0a947b640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0a4101de0 0x7ff0a41985f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000b4a0 tx=0x7ff0a000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.098+0000 7ff09a7fc640 1 -- 172.21.15.5:0/350512519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a0015020 con 0x7ff0a4101de0 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.098+0000 7ff0ab6df640 1 -- 172.21.15.5:0/350512519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a419d890 con 0x7ff0a4101de0 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.098+0000 7ff09a7fc640 1 -- 172.21.15.5:0/350512519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0011d10 con 0x7ff0a4101de0 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.098+0000 7ff09a7fc640 1 -- 172.21.15.5:0/350512519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a000f2e0 con 0x7ff0a4101de0 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.098+0000 7ff0ab6df640 1 -- 172.21.15.5:0/350512519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a41a5170 con 0x7ff0a4101de0 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.099+0000 7ff0ab6df640 1 -- 172.21.15.5:0/350512519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0a4102240 con 0x7ff0a4101de0 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.100+0000 7ff09a7fc640 1 -- 172.21.15.5:0/350512519 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff0a0018020 con 0x7ff0a4101de0 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.101+0000 7ff09a7fc640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0900a33b0 0x7ff0900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:55.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.101+0000 7ff0a8c7a640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0900a33b0 0x7ff0900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:55.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.101+0000 7ff09a7fc640 1 -- 172.21.15.5:0/350512519 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7ff0a00cf880 con 0x7ff0a4101de0 2024-09-15T22:11:55.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.101+0000 7ff0a8c7a640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0900a33b0 0x7ff0900a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40ffe70 tx=0x7ff094009110 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:55.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.105+0000 7ff09a7fc640 1 -- 172.21.15.5:0/350512519 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0a001f020 con 0x7ff0a4101de0 2024-09-15T22:11:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.322+0000 7ff0ab6df640 1 -- 172.21.15.5:0/350512519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0a410a0f0 con 0x7ff0a4101de0 2024-09-15T22:11:55.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.322+0000 7ff09a7fc640 1 -- 172.21.15.5:0/350512519 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7ff0a009bfd0 con 0x7ff0a4101de0 2024-09-15T22:11:55.324 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:11:55.324 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:11:55.324 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 -- 172.21.15.5:0/350512519 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0900a33b0 msgr2=0x7ff0900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0900a33b0 0x7ff0900a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40ffe70 tx=0x7ff094009110 comp rx=0 tx=0).stop 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 -- 172.21.15.5:0/350512519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0a4101de0 msgr2=0x7ff0a41985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0a4101de0 0x7ff0a41985f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000b4a0 tx=0x7ff0a000b8a0 comp rx=0 tx=0).stop 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 -- 172.21.15.5:0/350512519 shutdown_connections 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0900a33b0 0x7ff0900a5870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0a4106f10 0x7ff0a419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0a4102790 0x7ff0a4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 --2- 172.21.15.5:0/350512519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0a4101de0 0x7ff0a41985f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 -- 172.21.15.5:0/350512519 >> 172.21.15.5:0/350512519 conn(0x7ff0a40fd8c0 msgr2=0x7ff0a40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 -- 172.21.15.5:0/350512519 shutdown_connections 2024-09-15T22:11:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:55.324+0000 7ff0ab6df640 1 -- 172.21.15.5:0/350512519 wait complete. 2024-09-15T22:11:55.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:55 smithi099 ceph-mon[34892]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.0 MiB/s wr, 205 op/s 2024-09-15T22:11:55.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:55 smithi129 ceph-mon[35314]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.0 MiB/s wr, 205 op/s 2024-09-15T22:11:55.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:55 smithi005 ceph-mon[30944]: pgmap v93: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 3.0 MiB/s wr, 205 op/s 2024-09-15T22:11:56.042 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id ffcb6d65 --release --await' 2024-09-15T22:11:56.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id ffcb6d65 --release --await 2024-09-15T22:11:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:11:56.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/350512519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:56.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/350512519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:56.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/350512519' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:11:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.754+0000 7f675b696640 1 -- 172.21.15.5:0/2361082569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6754102790 msgr2=0x7f67541067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.754+0000 7f675b696640 1 --2- 172.21.15.5:0/2361082569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6754102790 0x7f67541067e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6744009900 tx=0x7f674402a410 comp rx=0 tx=0).stop 2024-09-15T22:11:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.754+0000 7f675b696640 1 -- 172.21.15.5:0/2361082569 shutdown_connections 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.754+0000 7f675b696640 1 --2- 172.21.15.5:0/2361082569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6754106f10 0x7f675410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.754+0000 7f675b696640 1 --2- 172.21.15.5:0/2361082569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6754102790 0x7f67541067e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.754+0000 7f675b696640 1 --2- 172.21.15.5:0/2361082569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6754101de0 0x7f67541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.754+0000 7f675b696640 1 -- 172.21.15.5:0/2361082569 >> 172.21.15.5:0/2361082569 conn(0x7f67540fd8c0 msgr2=0x7f67540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.754+0000 7f675b696640 1 -- 172.21.15.5:0/2361082569 shutdown_connections 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.754+0000 7f675b696640 1 -- 172.21.15.5:0/2361082569 wait complete. 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f675b696640 1 Processor -- start 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f675b696640 1 -- start start 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f675b696640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6754101de0 0x7f675419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f675b696640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6754102790 0x7f675419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f675b696640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6754106f10 0x7f675419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f675b696640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6754111f90 con 0x7f6754101de0 2024-09-15T22:11:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f675b696640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6754111e10 con 0x7f6754102790 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f675b696640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6754112110 con 0x7f6754106f10 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f6759432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6754101de0 0x7f675419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f6759432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6754101de0 0x7f675419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51406/0 (socket says 172.21.15.5:51406) 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f6759432640 1 -- 172.21.15.5:0/4273182590 learned_addr learned my addr 172.21.15.5:0/4273182590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f6758c31640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6754102790 0x7f675419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f6759432640 1 -- 172.21.15.5:0/4273182590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6754106f10 msgr2=0x7f675419f000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f6759432640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6754106f10 0x7f675419f000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f6759432640 1 -- 172.21.15.5:0/4273182590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6754102790 msgr2=0x7f675419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f6759432640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6754102790 0x7f675419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.755+0000 7f6759432640 1 -- 172.21.15.5:0/4273182590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675419f6e0 con 0x7f6754101de0 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.756+0000 7f6759432640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6754101de0 0x7f675419a730 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f675000c920 tx=0x7f675000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.756+0000 7f674a7fc640 1 -- 172.21.15.5:0/4273182590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750007c70 con 0x7f6754101de0 2024-09-15T22:11:56.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.756+0000 7f674a7fc640 1 -- 172.21.15.5:0/4273182590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6750007e10 con 0x7f6754101de0 2024-09-15T22:11:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.756+0000 7f675b696640 1 -- 172.21.15.5:0/4273182590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675419f9d0 con 0x7f6754101de0 2024-09-15T22:11:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.756+0000 7f674a7fc640 1 -- 172.21.15.5:0/4273182590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750018890 con 0x7f6754101de0 2024-09-15T22:11:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.756+0000 7f675b696640 1 -- 172.21.15.5:0/4273182590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67541a72b0 con 0x7f6754101de0 2024-09-15T22:11:56.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.759+0000 7f674a7fc640 1 -- 172.21.15.5:0/4273182590 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6750018a70 con 0x7f6754101de0 2024-09-15T22:11:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.759+0000 7f675da6f740 1 Processor -- start 2024-09-15T22:11:56.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.759+0000 7f675da6f740 1 -- start start 2024-09-15T22:11:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.759+0000 7f675da6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb01440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f675da6f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb0bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f675da6f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb0e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f675da6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55feeea66620 con 0x55feeeb01060 2024-09-15T22:11:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f675da6f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55feeea0de50 con 0x55feeeb0c240 2024-09-15T22:11:56.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f675da6f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55feeea625e0 con 0x55feeeb01980 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f6759c33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb01440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f6759c33640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb01440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51416/0 (socket says 172.21.15.5:51416) 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f6759c33640 1 -- 172.21.15.5:0/4170449922 learned_addr learned my addr 172.21.15.5:0/4170449922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f6758c31640 1 --2- 172.21.15.5:0/4170449922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb0e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f6759432640 1 --2- 172.21.15.5:0/4170449922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb0bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f674a7fc640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f67400a3430 0x7f67400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f6759c33640 1 -- 172.21.15.5:0/4170449922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 msgr2=0x55feeeb0bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f6759c33640 1 --2- 172.21.15.5:0/4170449922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb0bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f6759c33640 1 -- 172.21.15.5:0/4170449922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 msgr2=0x55feeeb0e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f6759c33640 1 --2- 172.21.15.5:0/4170449922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb0e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f6759c33640 1 -- 172.21.15.5:0/4170449922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feee9fed40 con 0x55feeeb01060 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f674a7fc640 1 -- 172.21.15.5:0/4273182590 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f67500d0290 con 0x7f6754101de0 2024-09-15T22:11:56.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.760+0000 7f6759c33640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f67400a3430 0x7f67400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.761+0000 7f6759c33640 1 --2- 172.21.15.5:0/4170449922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb01440 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f674c009ec0 tx=0x7f674c00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e79363dd5cdd605 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.761+0000 7f673f7fe640 1 -- 172.21.15.5:0/4170449922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c015e60 con 0x55feeeb01060 2024-09-15T22:11:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.761+0000 7f673f7fe640 1 -- 172.21.15.5:0/4170449922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f674c0148c0 con 0x55feeeb01060 2024-09-15T22:11:56.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.761+0000 7f673f7fe640 1 -- 172.21.15.5:0/4170449922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c0136c0 con 0x55feeeb01060 2024-09-15T22:11:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.761+0000 7f675da6f740 1 -- 172.21.15.5:0/4170449922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 msgr2=0x55feeeb01440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.761+0000 7f675da6f740 1 --2- 172.21.15.5:0/4170449922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb01440 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f674c009ec0 tx=0x7f674c00b420 comp rx=0 tx=0).stop 2024-09-15T22:11:56.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.761+0000 7f6759c33640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f67400a3430 0x7f67400a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f674c01d000 tx=0x7f674c01e000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.762+0000 7f675da6f740 1 -- 172.21.15.5:0/4170449922 shutdown_connections 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.762+0000 7f675da6f740 1 --2- 172.21.15.5:0/4170449922 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb0e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.762+0000 7f675da6f740 1 --2- 172.21.15.5:0/4170449922 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb0bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.762+0000 7f675da6f740 1 --2- 172.21.15.5:0/4170449922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb01440 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.762+0000 7f675da6f740 1 -- 172.21.15.5:0/4170449922 >> 172.21.15.5:0/4170449922 conn(0x55feeeafd5f0 msgr2=0x55feeeafd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.762+0000 7f675da6f740 1 -- 172.21.15.5:0/4170449922 shutdown_connections 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.762+0000 7f675da6f740 1 -- 172.21.15.5:0/4170449922 wait complete. 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.762+0000 7f675da6f740 1 -- start start 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.762+0000 7f675da6f740 1 Processor -- start 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f675da6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb091b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f675da6f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f675da6f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb09c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f675da6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55feeeb0ee10 con 0x55feeeb01060 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f675da6f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55feee9eef10 con 0x55feeeb0c240 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f675da6f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55feeeb0ef90 con 0x55feeeb01980 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f675da6f740 10 client.0 _ll_register_callbacks cb 0x55feeea12db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f6759432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb091b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f6759432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb091b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51418/0 (socket says 172.21.15.5:51418) 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f6759432640 1 -- 172.21.15.5:0/4270571685 learned_addr learned my addr 172.21.15.5:0/4270571685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f6758c31640 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f6759c33640 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb09c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f6759432640 1 -- 172.21.15.5:0/4270571685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 msgr2=0x55feeeb096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f6759432640 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f6759432640 1 -- 172.21.15.5:0/4270571685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 msgr2=0x55feeeb09c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f6759432640 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb09c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f6759432640 1 -- 172.21.15.5:0/4270571685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feee775580 con 0x55feeeb01060 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f6759432640 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb091b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6750097eb0 tx=0x7f675009c0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b136e95e55d286 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f673cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f673cff9640 1 -- 172.21.15.5:0/4270571685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67500d00d0 con 0x55feeeb01060 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f675da6f740 1 -- 172.21.15.5:0/4270571685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f674c0148c0 con 0x55feeeb01060 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.763+0000 7f673cff9640 1 -- 172.21.15.5:0/4270571685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f675000fb30 con 0x55feeeb01060 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f673cff9640 1 -- 172.21.15.5:0/4270571685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f675009c1d0 con 0x55feeeb01060 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f675da6f740 10 client.16062 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f675da6f740 1 -- 172.21.15.5:0/4270571685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55feee83f960 con 0x55feeeb01060 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f673cff9640 1 -- 172.21.15.5:0/4270571685 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f674c0148c0 con 0x55feeeb01060 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f673cff9640 1 -- 172.21.15.5:0/4270571685 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6750098840 con 0x55feeeb01060 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f675da6f740 10 client.16062 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f675da6f740 10 client.16062 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:11:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f675da6f740 20 client.16062 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f675da6f740 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55feeeb1be30 0x55feeeb1e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f675da6f740 4 client.16062 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:11:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f675da6f740 1 -- 172.21.15.5:0/4270571685 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55feeea73b90 con 0x55feeeb1be30 2024-09-15T22:11:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.764+0000 7f6758c31640 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55feeeb1be30 0x55feeeb1e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.765+0000 7f673cff9640 10 client.16062 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.765+0000 7f6758c31640 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55feeeb1be30 0x55feeeb1e220 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1debab193eade237 server_cookie=54a78458b1272a29 in_seq=0 out_seq=0 2024-09-15T22:11:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.765+0000 7f673cff9640 1 -- 172.21.15.5:0/4270571685 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55feeea73b90 con 0x55feeeb1be30 2024-09-15T22:11:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.765+0000 7f673cff9640 10 client.16062 handle_command_reply: tid=0 2024-09-15T22:11:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 1 client.16062 shutdown 2024-09-15T22:11:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 1 -- 172.21.15.5:0/4270571685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 msgr2=0x55feeeb091b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb091b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6750097eb0 tx=0x7f675009c0d0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 1 -- 172.21.15.5:0/4270571685 shutdown_connections 2024-09-15T22:11:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55feeeb1be30 0x55feeeb1e220 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb09c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f673cff9640 0 client.16062 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 1 --2- 172.21.15.5:0/4270571685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb091b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 1 -- 172.21.15.5:0/4270571685 >> 172.21.15.5:0/4270571685 conn(0x55feeeafd5f0 msgr2=0x55feeeb032d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 1 -- 172.21.15.5:0/4270571685 shutdown_connections 2024-09-15T22:11:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 1 -- 172.21.15.5:0/4270571685 wait complete. 2024-09-15T22:11:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.766+0000 7f675da6f740 20 client.16062 trim_cache size 0 max 16384 2024-09-15T22:11:56.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.783+0000 7f675da6f740 1 Processor -- start 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.783+0000 7f675da6f740 1 -- start start 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.783+0000 7f675da6f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb01060 0x55feeeb0b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.783+0000 7f675da6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01980 0x55feeeb23a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.783+0000 7f675da6f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb0c240 0x55feeeb23f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.783+0000 7f675da6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67541b3ad0 con 0x55feeeb01980 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.783+0000 7f675da6f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67541b13a0 con 0x55feeeb01060 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.783+0000 7f675da6f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67541af1f0 con 0x55feeeb0c240 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f6758c31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01980 0x55feeeb23a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f6758c31640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01980 0x55feeeb23a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51428/0 (socket says 172.21.15.5:51428) 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f6758c31640 1 -- 172.21.15.5:0/1710629616 learned_addr learned my addr 172.21.15.5:0/1710629616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f6759c33640 1 --2- 172.21.15.5:0/1710629616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb0c240 0x55feeeb23f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f6759432640 1 --2- 172.21.15.5:0/1710629616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb01060 0x55feeeb0b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f6758c31640 1 -- 172.21.15.5:0/1710629616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb0c240 msgr2=0x55feeeb23f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f6758c31640 1 --2- 172.21.15.5:0/1710629616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb0c240 0x55feeeb23f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f6758c31640 1 -- 172.21.15.5:0/1710629616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb01060 msgr2=0x55feeeb0b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f6758c31640 1 --2- 172.21.15.5:0/1710629616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb01060 0x55feeeb0b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f6758c31640 1 -- 172.21.15.5:0/1710629616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feeeafbbb0 con 0x55feeeb01980 2024-09-15T22:11:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f6758c31640 1 --2- 172.21.15.5:0/1710629616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01980 0x55feeeb23a50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6744009850 tx=0x7f67440058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7698eb0c88b3f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f673effd640 1 -- 172.21.15.5:0/1710629616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744048020 con 0x55feeeb01980 2024-09-15T22:11:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f673effd640 1 -- 172.21.15.5:0/1710629616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67440327b0 con 0x55feeeb01980 2024-09-15T22:11:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.784+0000 7f673effd640 1 -- 172.21.15.5:0/1710629616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744035040 con 0x55feeeb01980 2024-09-15T22:11:56.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.785+0000 7f675da6f740 1 -- 172.21.15.5:0/1710629616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01980 msgr2=0x55feeeb23a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.785+0000 7f675da6f740 1 --2- 172.21.15.5:0/1710629616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01980 0x55feeeb23a50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6744009850 tx=0x7f67440058d0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.785+0000 7f675da6f740 1 -- 172.21.15.5:0/1710629616 shutdown_connections 2024-09-15T22:11:56.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.785+0000 7f675da6f740 1 --2- 172.21.15.5:0/1710629616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb0c240 0x55feeeb23f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.785+0000 7f675da6f740 1 --2- 172.21.15.5:0/1710629616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01980 0x55feeeb23a50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.785+0000 7f675da6f740 1 --2- 172.21.15.5:0/1710629616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb01060 0x55feeeb0b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.785+0000 7f675da6f740 1 -- 172.21.15.5:0/1710629616 >> 172.21.15.5:0/1710629616 conn(0x55feeeafd5f0 msgr2=0x55feeeb0aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:56.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.785+0000 7f675da6f740 1 -- 172.21.15.5:0/1710629616 shutdown_connections 2024-09-15T22:11:56.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.785+0000 7f675da6f740 1 -- 172.21.15.5:0/1710629616 wait complete. 2024-09-15T22:11:56.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.785+0000 7f675da6f740 1 -- start start 2024-09-15T22:11:56.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.785+0000 7f675da6f740 1 Processor -- start 2024-09-15T22:11:56.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f675da6f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb10b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f675da6f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb11090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f675da6f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb7e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f675da6f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67541af860 con 0x55feeeb01060 2024-09-15T22:11:56.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f675da6f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f67541b0cd0 con 0x55feeeb0c240 2024-09-15T22:11:56.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f675da6f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f67541aff30 con 0x55feeeb01980 2024-09-15T22:11:56.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f675da6f740 10 client.0 _ll_register_callbacks cb 0x7f675000a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:11:56.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f6759432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb10b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f6759432640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb10b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51444/0 (socket says 172.21.15.5:51444) 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f6759432640 1 -- 172.21.15.5:0/1332329005 learned_addr learned my addr 172.21.15.5:0/1332329005 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f6758c31640 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb11090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f6759c33640 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb7e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f6759432640 1 -- 172.21.15.5:0/1332329005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 msgr2=0x55feeeb11090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f6759432640 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb11090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f6759432640 1 -- 172.21.15.5:0/1332329005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 msgr2=0x55feeeb7e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f6759432640 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb7e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f6759432640 1 -- 172.21.15.5:0/1332329005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feeeb7f1e0 con 0x55feeeb01060 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.786+0000 7f6759c33640 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb7e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f6759432640 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb10b50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6750098ba0 tx=0x7f67500d3e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd466c5f3fd30172 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f673cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f673cff9640 1 -- 172.21.15.5:0/1332329005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67500d00d0 con 0x55feeeb01060 2024-09-15T22:11:56.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f673cff9640 1 -- 172.21.15.5:0/1332329005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f675000fb30 con 0x55feeeb01060 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f675da6f740 1 -- 172.21.15.5:0/1332329005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67440327b0 con 0x55feeeb01060 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f673cff9640 1 -- 172.21.15.5:0/1332329005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f67500a0580 con 0x55feeeb01060 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f675da6f740 10 client.16068 fetch_fsmap learned FSMap version 25 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f675da6f740 1 -- 172.21.15.5:0/1332329005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55feeeb875e0 con 0x55feeeb01060 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f673cff9640 1 -- 172.21.15.5:0/1332329005 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67440327b0 con 0x55feeeb01060 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f673cff9640 1 -- 172.21.15.5:0/1332329005 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f67500a4a40 con 0x55feeeb01060 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f675da6f740 10 client.16068 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f675da6f740 10 client.16068 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f675da6f740 20 client.16068 populate_metadata read hostname 'smithi005' 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f675da6f740 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55feeeb87d00 0x55feeeb8a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f675da6f740 4 client.16068 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ffcb6d65", "release": true, "await": true}] 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.787+0000 7f675da6f740 1 -- 172.21.15.5:0/1332329005 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ffcb6d65", "release": true, "await": true}) -- 0x55feeea6e0a0 con 0x55feeeb87d00 2024-09-15T22:11:56.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.788+0000 7f6758c31640 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55feeeb87d00 0x55feeeb8a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:11:56.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.788+0000 7f6758c31640 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55feeeb87d00 0x55feeeb8a0f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fca5e1d1eaa6a6b server_cookie=bf07236fdf376cc in_seq=0 out_seq=0 2024-09-15T22:11:56.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.788+0000 7f673cff9640 10 client.16068 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.834+0000 7f673cff9640 1 -- 172.21.15.5:0/1332329005 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55feeea6e0a0 con 0x55feeeb87d00 2024-09-15T22:11:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.834+0000 7f673cff9640 10 client.16068 handle_command_reply: tid=0 2024-09-15T22:11:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.834+0000 7f675da6f740 1 client.16068 shutdown 2024-09-15T22:11:56.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.845+0000 7f675da6f740 1 -- 172.21.15.5:0/1332329005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 msgr2=0x55feeeb10b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.845+0000 7f675da6f740 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb10b50 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6750098ba0 tx=0x7f67500d3e50 comp rx=0 tx=0).stop 2024-09-15T22:11:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.847+0000 7f675da6f740 1 -- 172.21.15.5:0/1332329005 shutdown_connections 2024-09-15T22:11:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.847+0000 7f675da6f740 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55feeeb87d00 0x55feeeb8a0f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.847+0000 7f675da6f740 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55feeeb0c240 0x55feeeb7e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.847+0000 7f675da6f740 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55feeeb01980 0x55feeeb11090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.847+0000 7f675da6f740 1 --2- 172.21.15.5:0/1332329005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55feeeb01060 0x55feeeb10b50 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.847+0000 7f675da6f740 1 -- 172.21.15.5:0/1332329005 >> 172.21.15.5:0/1332329005 conn(0x55feeeafd5f0 msgr2=0x55feeeafc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.847+0000 7f673cff9640 0 client.16068 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:11:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.847+0000 7f675da6f740 1 -- 172.21.15.5:0/1332329005 shutdown_connections 2024-09-15T22:11:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.847+0000 7f675da6f740 1 -- 172.21.15.5:0/1332329005 wait complete. 2024-09-15T22:11:56.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.847+0000 7f675da6f740 20 client.16068 trim_cache size 0 max 16384 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 12, 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout: "ffcb6d65": { 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 12, 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:11:56.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.6, 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:56.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:11:56.853 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:11:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.848+0000 7f675b696640 1 -- 172.21.15.5:0/4273182590 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f67400a3430 msgr2=0x7f67400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.848+0000 7f675b696640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f67400a3430 0x7f67400a58f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f674c01d000 tx=0x7f674c01e000 comp rx=0 tx=0).stop 2024-09-15T22:11:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.848+0000 7f675b696640 1 -- 172.21.15.5:0/4273182590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6754101de0 msgr2=0x7f675419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:11:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.848+0000 7f675b696640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6754101de0 0x7f675419a730 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f675000c920 tx=0x7f675000cde0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.848+0000 7f675b696640 1 -- 172.21.15.5:0/4273182590 shutdown_connections 2024-09-15T22:11:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.848+0000 7f675b696640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f67400a3430 0x7f67400a58f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.848+0000 7f675b696640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6754106f10 0x7f675419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.848+0000 7f675b696640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6754102790 0x7f675419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.848+0000 7f675b696640 1 --2- 172.21.15.5:0/4273182590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6754101de0 0x7f675419a730 secure :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f675000c920 tx=0x7f675000cde0 comp rx=0 tx=0).stop 2024-09-15T22:11:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.848+0000 7f675b696640 1 -- 172.21.15.5:0/4273182590 >> 172.21.15.5:0/4273182590 conn(0x7f67540fd8c0 msgr2=0x7f67540febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:11:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.849+0000 7f675b696640 1 -- 172.21.15.5:0/4273182590 shutdown_connections 2024-09-15T22:11:56.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:11:56.851+0000 7f675b696640 1 -- 172.21.15.5:0/4273182590 wait complete. 2024-09-15T22:11:57.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:57 smithi099 ceph-mon[34892]: pgmap v94: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.0 MiB/s wr, 208 op/s 2024-09-15T22:11:57.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:57 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:11:57.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:57 smithi129 ceph-mon[35314]: pgmap v94: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.0 MiB/s wr, 208 op/s 2024-09-15T22:11:57.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:57 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:11:57.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:57 smithi005 ceph-mon[30944]: pgmap v94: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 3.0 MiB/s wr, 208 op/s 2024-09-15T22:11:57.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:57 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:11:57.708 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ffcb6d65', total seconds elapsed: 23.2 2024-09-15T22:11:59.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:11:59 smithi099 ceph-mon[34892]: pgmap v95: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-09-15T22:11:59.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:11:59 smithi129 ceph-mon[35314]: pgmap v95: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-09-15T22:11:59.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:11:59 smithi005 ceph-mon[30944]: pgmap v95: 129 pgs: 129 active+clean; 1.7 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.0 MiB/s wr, 120 op/s 2024-09-15T22:11:59.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 150.0 s, 1300.1 tps, lat 290.453 ms stddev 1990.390 2024-09-15T22:12:01.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:01 smithi099 ceph-mon[34892]: pgmap v96: 129 pgs: 129 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-09-15T22:12:01.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:01 smithi129 ceph-mon[35314]: pgmap v96: 129 pgs: 129 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-09-15T22:12:01.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:01 smithi005 ceph-mon[30944]: pgmap v96: 129 pgs: 129 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 2.3 MiB/s wr, 161 op/s 2024-09-15T22:12:03.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:03 smithi099 ceph-mon[34892]: pgmap v97: 129 pgs: 129 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.1 MiB/s wr, 249 op/s 2024-09-15T22:12:03.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:03 smithi129 ceph-mon[35314]: pgmap v97: 129 pgs: 129 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.1 MiB/s wr, 249 op/s 2024-09-15T22:12:03.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:03 smithi005 ceph-mon[30944]: pgmap v97: 129 pgs: 129 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.1 MiB/s wr, 249 op/s 2024-09-15T22:12:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 155.0 s, 881.4 tps, lat 86.658 ms stddev 114.402 2024-09-15T22:12:05.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:05 smithi099 ceph-mon[34892]: pgmap v98: 129 pgs: 129 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-15T22:12:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:05 smithi129 ceph-mon[35314]: pgmap v98: 129 pgs: 129 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-15T22:12:05.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:05 smithi005 ceph-mon[30944]: pgmap v98: 129 pgs: 129 active+clean; 1.7 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-15T22:12:06.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:06 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:06.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:06 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:06.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:06 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:12:07.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:07 smithi005 ceph-mon[30944]: pgmap v99: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-09-15T22:12:07.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:07 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:07.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:07 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:07.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:07 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:12:07.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:07 smithi099 ceph-mon[34892]: pgmap v99: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-09-15T22:12:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:07 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:07 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:07 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:12:07.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:07 smithi129 ceph-mon[35314]: pgmap v99: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-09-15T22:12:07.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:07 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:12:07.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:07 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:12:07.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:07 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:12:09.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:09 smithi005 ceph-mon[30944]: pgmap v100: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-09-15T22:12:09.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:09 smithi099 ceph-mon[34892]: pgmap v100: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-09-15T22:12:09.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:09 smithi129 ceph-mon[35314]: pgmap v100: 129 pgs: 129 active+clean; 1.8 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 MiB/s wr, 1.29k op/s 2024-09-15T22:12:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 160.0 s, 2094.2 tps, lat 59.240 ms stddev 160.815 2024-09-15T22:12:11.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:11 smithi099 ceph-mon[34892]: pgmap v101: 129 pgs: 129 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 1.33k op/s 2024-09-15T22:12:11.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:11 smithi129 ceph-mon[35314]: pgmap v101: 129 pgs: 129 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 1.33k op/s 2024-09-15T22:12:11.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:11 smithi005 ceph-mon[30944]: pgmap v101: 129 pgs: 129 active+clean; 1.8 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 15 MiB/s wr, 1.33k op/s 2024-09-15T22:12:13.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:13 smithi099 ceph-mon[34892]: pgmap v102: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-15T22:12:13.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:13 smithi129 ceph-mon[35314]: pgmap v102: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-15T22:12:13.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:13 smithi005 ceph-mon[30944]: pgmap v102: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 19 MiB/s wr, 1.53k op/s 2024-09-15T22:12:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 165.0 s, 2164.2 tps, lat 46.279 ms stddev 52.165 2024-09-15T22:12:15.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:15 smithi099 ceph-mon[34892]: pgmap v103: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-09-15T22:12:15.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:15 smithi129 ceph-mon[35314]: pgmap v103: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-09-15T22:12:15.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:15 smithi005 ceph-mon[30944]: pgmap v103: 129 pgs: 129 active+clean; 1.8 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 18 MiB/s wr, 1.41k op/s 2024-09-15T22:12:17.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:17 smithi099 ceph-mon[34892]: pgmap v104: 129 pgs: 129 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2024-09-15T22:12:17.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:17 smithi129 ceph-mon[35314]: pgmap v104: 129 pgs: 129 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2024-09-15T22:12:17.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:17 smithi005 ceph-mon[30944]: pgmap v104: 129 pgs: 129 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 20 MiB/s wr, 1.67k op/s 2024-09-15T22:12:19.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:19 smithi099 ceph-mon[34892]: pgmap v105: 129 pgs: 129 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 545 op/s 2024-09-15T22:12:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:19 smithi129 ceph-mon[35314]: pgmap v105: 129 pgs: 129 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 545 op/s 2024-09-15T22:12:19.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:19 smithi005 ceph-mon[30944]: pgmap v105: 129 pgs: 129 active+clean; 1.9 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.6 MiB/s wr, 545 op/s 2024-09-15T22:12:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 170.0 s, 1967.2 tps, lat 50.225 ms stddev 55.334 2024-09-15T22:12:21.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:21 smithi099 ceph-mon[34892]: pgmap v106: 129 pgs: 129 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 605 op/s 2024-09-15T22:12:21.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:21 smithi129 ceph-mon[35314]: pgmap v106: 129 pgs: 129 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 605 op/s 2024-09-15T22:12:21.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:21 smithi005 ceph-mon[30944]: pgmap v106: 129 pgs: 129 active+clean; 1.9 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 7.1 MiB/s wr, 605 op/s 2024-09-15T22:12:23.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:23 smithi099 ceph-mon[34892]: pgmap v107: 129 pgs: 129 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.4 MiB/s wr, 826 op/s 2024-09-15T22:12:23.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:23 smithi129 ceph-mon[35314]: pgmap v107: 129 pgs: 129 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.4 MiB/s wr, 826 op/s 2024-09-15T22:12:23.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:23 smithi005 ceph-mon[30944]: pgmap v107: 129 pgs: 129 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 9.4 MiB/s wr, 826 op/s 2024-09-15T22:12:24.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:12:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 175.0 s, 1709.8 tps, lat 58.297 ms stddev 61.919 2024-09-15T22:12:24.981 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:25.428 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:25 smithi005 ceph-mon[30944]: pgmap v108: 129 pgs: 129 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 MiB/s wr, 581 op/s 2024-09-15T22:12:25.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.563+0000 7faddd8ab640 1 -- 172.21.15.5:0/2583038011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadd81011e0 msgr2=0x7fadd8105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.563+0000 7faddd8ab640 1 --2- 172.21.15.5:0/2583038011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadd81011e0 0x7fadd8105980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009930 tx=0x7fadcc02a440 comp rx=0 tx=0).stop 2024-09-15T22:12:25.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.563+0000 7faddd8ab640 1 -- 172.21.15.5:0/2583038011 shutdown_connections 2024-09-15T22:12:25.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.563+0000 7faddd8ab640 1 --2- 172.21.15.5:0/2583038011 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadd8105ec0 0x7fadd8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.563+0000 7faddd8ab640 1 --2- 172.21.15.5:0/2583038011 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadd81011e0 0x7fadd8105980 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.563+0000 7faddd8ab640 1 --2- 172.21.15.5:0/2583038011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd81008c0 0x7fadd8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.563+0000 7faddd8ab640 1 -- 172.21.15.5:0/2583038011 >> 172.21.15.5:0/2583038011 conn(0x7fadd80fc7d0 msgr2=0x7fadd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:25.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.563+0000 7faddd8ab640 1 -- 172.21.15.5:0/2583038011 shutdown_connections 2024-09-15T22:12:25.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.563+0000 7faddd8ab640 1 -- 172.21.15.5:0/2583038011 wait complete. 2024-09-15T22:12:25.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.564+0000 7faddd8ab640 1 Processor -- start 2024-09-15T22:12:25.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.564+0000 7faddd8ab640 1 -- start start 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.565+0000 7faddd8ab640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadd81008c0 0x7fadd80739e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.565+0000 7faddd8ab640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadd81011e0 0x7fadd806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.565+0000 7fadd6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadd81008c0 0x7fadd80739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.565+0000 7fadd6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadd81008c0 0x7fadd80739e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:49996/0 (socket says 172.21.15.5:49996) 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.565+0000 7fadd6ffd640 1 -- 172.21.15.5:0/2645730437 learned_addr learned my addr 172.21.15.5:0/2645730437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.565+0000 7faddd8ab640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd8105ec0 0x7fadd806cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.565+0000 7faddd8ab640 1 -- 172.21.15.5:0/2645730437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fadd8118660 con 0x7fadd8105ec0 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.565+0000 7faddd8ab640 1 -- 172.21.15.5:0/2645730437 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fadd81184e0 con 0x7fadd81011e0 2024-09-15T22:12:25.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.566+0000 7faddd8ab640 1 -- 172.21.15.5:0/2645730437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fadd81187e0 con 0x7fadd81008c0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.566+0000 7fadd67fc640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadd81011e0 0x7fadd806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.566+0000 7fadd77fe640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd8105ec0 0x7fadd806cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.566+0000 7fadd6ffd640 1 -- 172.21.15.5:0/2645730437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadd81011e0 msgr2=0x7fadd806ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.566+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadd81011e0 0x7fadd806ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.566+0000 7fadd6ffd640 1 -- 172.21.15.5:0/2645730437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd8105ec0 msgr2=0x7fadd806cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.566+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd8105ec0 0x7fadd806cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.566+0000 7fadd6ffd640 1 -- 172.21.15.5:0/2645730437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd806d570 con 0x7fadd81008c0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.566+0000 7fadd67fc640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadd81011e0 0x7fadd806ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.566+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadd81008c0 0x7fadd80739e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fadc800e9f0 tx=0x7fadc800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.567+0000 7faddc8a9640 1 -- 172.21.15.5:0/2645730437 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc800cd80 con 0x7fadd81008c0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.567+0000 7faddc8a9640 1 -- 172.21.15.5:0/2645730437 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc8010910 con 0x7fadd81008c0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.567+0000 7faddd8ab640 1 -- 172.21.15.5:0/2645730437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd806d860 con 0x7fadd81008c0 2024-09-15T22:12:25.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.567+0000 7faddc8a9640 1 -- 172.21.15.5:0/2645730437 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8018990 con 0x7fadd81008c0 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.567+0000 7faddd8ab640 1 -- 172.21.15.5:0/2645730437 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd8071850 con 0x7fadd81008c0 2024-09-15T22:12:25.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.570+0000 7faddc8a9640 1 -- 172.21.15.5:0/2645730437 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fadc8018b30 con 0x7fadd81008c0 2024-09-15T22:12:25.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.571+0000 7faddc8a9640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fadc00a3480 0x7fadc00a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.571+0000 7faddfc84740 1 Processor -- start 2024-09-15T22:12:25.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7faddc8a9640 1 -- 172.21.15.5:0/2645730437 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fadc80d1520 con 0x7fadd81008c0 2024-09-15T22:12:25.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7faddfc84740 1 -- start start 2024-09-15T22:12:25.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7fadd67fc640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fadc00a3480 0x7fadc00a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7faddfc84740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da64a0 0x5579e3da6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7faddfc84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da6dc0 0x5579e3db1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7faddfc84740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3db1680 0x5579e3db3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7faddfc84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5579e3d0c2e0 con 0x5579e3da6dc0 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7faddfc84740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5579e3cb3ce0 con 0x5579e3da64a0 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7faddfc84740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5579e3d082a0 con 0x5579e3db1680 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7fadd6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da64a0 0x5579e3da6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7fadd6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da64a0 0x5579e3da6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:32854/0 (socket says 172.21.15.5:32854) 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.572+0000 7fadd6ffd640 1 -- 172.21.15.5:0/2320532912 learned_addr learned my addr 172.21.15.5:0/2320532912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.573+0000 7fadd6ffd640 1 -- 172.21.15.5:0/2320532912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3db1680 msgr2=0x5579e3db3a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.573+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/2320532912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3db1680 0x5579e3db3a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.573+0000 7fadd6ffd640 1 -- 172.21.15.5:0/2320532912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da6dc0 msgr2=0x5579e3db1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.573+0000 7fadd67fc640 1 --2- 172.21.15.5:0/2320532912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da6dc0 0x5579e3db1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.573+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/2320532912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da6dc0 0x5579e3db1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.573+0000 7fadd6ffd640 1 -- 172.21.15.5:0/2320532912 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e3c98440 con 0x5579e3da64a0 2024-09-15T22:12:25.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.573+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/2320532912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da64a0 0x5579e3da6880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fadc800c6a0 tx=0x7fadc809d360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953a32ceead50173 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:25.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.573+0000 7fadbdffb640 1 -- 172.21.15.5:0/2320532912 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc80a60d0 con 0x5579e3da64a0 2024-09-15T22:12:25.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.573+0000 7fadbdffb640 1 -- 172.21.15.5:0/2320532912 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc80ceb50 con 0x5579e3da64a0 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.573+0000 7fadbdffb640 1 -- 172.21.15.5:0/2320532912 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc809c8b0 con 0x5579e3da64a0 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.574+0000 7faddfc84740 1 -- 172.21.15.5:0/2320532912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da64a0 msgr2=0x5579e3da6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.574+0000 7fadd67fc640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fadc00a3480 0x7fadc00a5940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fadcc006010 tx=0x7fadcc004180 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.574+0000 7faddfc84740 1 --2- 172.21.15.5:0/2320532912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da64a0 0x5579e3da6880 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fadc800c6a0 tx=0x7fadc809d360 comp rx=0 tx=0).stop 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.575+0000 7faddfc84740 1 -- 172.21.15.5:0/2320532912 shutdown_connections 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.575+0000 7faddfc84740 1 --2- 172.21.15.5:0/2320532912 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3db1680 0x5579e3db3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.575+0000 7faddfc84740 1 --2- 172.21.15.5:0/2320532912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da6dc0 0x5579e3db1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.575+0000 7faddfc84740 1 --2- 172.21.15.5:0/2320532912 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da64a0 0x5579e3da6880 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.575+0000 7faddfc84740 1 -- 172.21.15.5:0/2320532912 >> 172.21.15.5:0/2320532912 conn(0x5579e3da2a30 msgr2=0x5579e3da2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.575+0000 7faddfc84740 1 -- 172.21.15.5:0/2320532912 shutdown_connections 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.575+0000 7faddfc84740 1 -- 172.21.15.5:0/2320532912 wait complete. 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.575+0000 7faddfc84740 1 -- start start 2024-09-15T22:12:25.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.576+0000 7faddfc84740 1 Processor -- start 2024-09-15T22:12:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.576+0000 7faddfc84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da64a0 0x5579e3db1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.576+0000 7faddfc84740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da6dc0 0x5579e3dae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.576+0000 7faddfc84740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3db1680 0x5579e3dae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.576+0000 7faddfc84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5579e3db4250 con 0x5579e3da64a0 2024-09-15T22:12:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.576+0000 7faddfc84740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5579e3cc7050 con 0x5579e3db1680 2024-09-15T22:12:25.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.577+0000 7faddfc84740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5579e3db43d0 con 0x5579e3da6dc0 2024-09-15T22:12:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.577+0000 7faddfc84740 10 client.0 _ll_register_callbacks cb 0x5579e3d07160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.577+0000 7fadd77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3db1680 0x5579e3dae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.577+0000 7fadd77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3db1680 0x5579e3dae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:32862/0 (socket says 172.21.15.5:32862) 2024-09-15T22:12:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.577+0000 7fadd77fe640 1 -- 172.21.15.5:0/219702093 learned_addr learned my addr 172.21.15.5:0/219702093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.579+0000 7fadd77fe640 1 -- 172.21.15.5:0/219702093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da6dc0 msgr2=0x5579e3dae310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:12:25.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.579+0000 7fadd77fe640 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da6dc0 0x5579e3dae310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.579+0000 7fadd77fe640 1 -- 172.21.15.5:0/219702093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da64a0 msgr2=0x5579e3db1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.579+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da64a0 0x5579e3db1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.579+0000 7fadd77fe640 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da64a0 0x5579e3db1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.579+0000 7fadd77fe640 1 -- 172.21.15.5:0/219702093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e3a7d3c0 con 0x5579e3db1680 2024-09-15T22:12:25.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.580+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da64a0 0x5579e3db1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.581+0000 7fadd77fe640 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3db1680 0x5579e3dae8e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fadd0010830 tx=0x7fadd0010cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3441f3045a6c45b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.581+0000 7fadb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:12:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.581+0000 7fadb6ffd640 1 -- 172.21.15.5:0/219702093 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0017070 con 0x5579e3db1680 2024-09-15T22:12:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.581+0000 7fadb6ffd640 1 -- 172.21.15.5:0/219702093 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fadd0004d20 con 0x5579e3db1680 2024-09-15T22:12:25.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.581+0000 7faddfc84740 1 -- 172.21.15.5:0/219702093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc80ceb50 con 0x5579e3db1680 2024-09-15T22:12:25.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.582+0000 7fadb6ffd640 1 -- 172.21.15.5:0/219702093 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadd000eac0 con 0x5579e3db1680 2024-09-15T22:12:25.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.582+0000 7faddfc84740 10 client.25649 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:25.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.582+0000 7faddfc84740 1 -- 172.21.15.5:0/219702093 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5579e3dbfcf0 con 0x5579e3db1680 2024-09-15T22:12:25.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.582+0000 7fadb6ffd640 1 -- 172.21.15.5:0/219702093 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc80ceb50 con 0x5579e3db1680 2024-09-15T22:12:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.582+0000 7fadb6ffd640 1 -- 172.21.15.5:0/219702093 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fadd0004430 con 0x5579e3db1680 2024-09-15T22:12:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.582+0000 7faddfc84740 10 client.25649 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.582+0000 7faddfc84740 10 client.25649 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.582+0000 7faddfc84740 20 client.25649 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.582+0000 7faddfc84740 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5579e3dc1100 0x5579e3dc3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.582+0000 7faddfc84740 4 client.25649 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.582+0000 7faddfc84740 1 -- 172.21.15.5:0/219702093 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579e3d19850 con 0x5579e3dc1100 2024-09-15T22:12:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.583+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5579e3dc1100 0x5579e3dc3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.583+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5579e3dc1100 0x5579e3dc3510 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8db5cb407d911f0c server_cookie=ea401290800911b0 in_seq=0 out_seq=0 2024-09-15T22:12:25.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.583+0000 7fadb6ffd640 10 client.25649 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7fadb6ffd640 1 -- 172.21.15.5:0/219702093 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579e3d19850 con 0x5579e3dc1100 2024-09-15T22:12:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7fadb6ffd640 10 client.25649 handle_command_reply: tid=0 2024-09-15T22:12:25.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7faddfc84740 1 client.25649 shutdown 2024-09-15T22:12:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7faddfc84740 1 -- 172.21.15.5:0/219702093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3db1680 msgr2=0x5579e3dae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7faddfc84740 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3db1680 0x5579e3dae8e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fadd0010830 tx=0x7fadd0010cf0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7faddfc84740 1 -- 172.21.15.5:0/219702093 shutdown_connections 2024-09-15T22:12:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7faddfc84740 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5579e3dc1100 0x5579e3dc3510 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7faddfc84740 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3db1680 0x5579e3dae8e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7faddfc84740 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da6dc0 0x5579e3dae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7faddfc84740 1 --2- 172.21.15.5:0/219702093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da64a0 0x5579e3db1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7faddfc84740 1 -- 172.21.15.5:0/219702093 >> 172.21.15.5:0/219702093 conn(0x5579e3da2a30 msgr2=0x5579e3da1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7fadb6ffd640 0 client.25649 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7faddfc84740 1 -- 172.21.15.5:0/219702093 shutdown_connections 2024-09-15T22:12:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.585+0000 7faddfc84740 1 -- 172.21.15.5:0/219702093 wait complete. 2024-09-15T22:12:25.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.586+0000 7faddfc84740 20 client.25649 trim_cache size 0 max 16384 2024-09-15T22:12:25.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:25 smithi099 ceph-mon[34892]: pgmap v108: 129 pgs: 129 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 MiB/s wr, 581 op/s 2024-09-15T22:12:25.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:25 smithi129 ceph-mon[35314]: pgmap v108: 129 pgs: 129 active+clean; 1.9 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.5 MiB/s wr, 581 op/s 2024-09-15T22:12:25.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.615+0000 7faddfc84740 1 Processor -- start 2024-09-15T22:12:25.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.615+0000 7faddfc84740 1 -- start start 2024-09-15T22:12:25.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.615+0000 7faddfc84740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 0x5579e3dbd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.616+0000 7faddfc84740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da6dc0 0x5579e3da2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.616+0000 7faddfc84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3db1680 0x5579e3da97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.616+0000 7fadd6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 0x5579e3dbd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.616+0000 7faddfc84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fadd81c07d0 con 0x5579e3db1680 2024-09-15T22:12:25.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.616+0000 7faddfc84740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fadd81be0a0 con 0x5579e3da6dc0 2024-09-15T22:12:25.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.616+0000 7faddfc84740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fadd81bbef0 con 0x5579e3da64a0 2024-09-15T22:12:25.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.616+0000 7fadd6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 0x5579e3dbd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50034/0 (socket says 172.21.15.5:50034) 2024-09-15T22:12:25.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.616+0000 7fadd6ffd640 1 -- 172.21.15.5:0/2030057023 learned_addr learned my addr 172.21.15.5:0/2030057023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:25.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.616+0000 7fadd67fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da6dc0 0x5579e3da2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.616+0000 7fadd77fe640 1 --2- 172.21.15.5:0/2030057023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3db1680 0x5579e3da97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.617+0000 7fadd77fe640 1 -- 172.21.15.5:0/2030057023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 msgr2=0x5579e3dbd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:25.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.617+0000 7fadd77fe640 1 --2- 172.21.15.5:0/2030057023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 0x5579e3dbd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.617+0000 7fadd77fe640 1 -- 172.21.15.5:0/2030057023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da6dc0 msgr2=0x5579e3da2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:25.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.617+0000 7fadd77fe640 1 --2- 172.21.15.5:0/2030057023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da6dc0 0x5579e3da2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.617+0000 7fadd77fe640 1 -- 172.21.15.5:0/2030057023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e3dad2c0 con 0x5579e3db1680 2024-09-15T22:12:25.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.617+0000 7fadd67fc640 1 --2- 172.21.15.5:0/2030057023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da6dc0 0x5579e3da2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:25.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.617+0000 7fadd77fe640 1 --2- 172.21.15.5:0/2030057023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3db1680 0x5579e3da97f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fadd0006010 tx=0x7fadd000e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140ac4ddcddad862 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:25.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.617+0000 7fadbd7fa640 1 -- 172.21.15.5:0/2030057023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0017040 con 0x5579e3db1680 2024-09-15T22:12:25.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.617+0000 7fadbd7fa640 1 -- 172.21.15.5:0/2030057023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadd001b520 con 0x5579e3db1680 2024-09-15T22:12:25.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.617+0000 7fadbd7fa640 1 -- 172.21.15.5:0/2030057023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd002e6a0 con 0x5579e3db1680 2024-09-15T22:12:25.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.618+0000 7faddfc84740 1 -- 172.21.15.5:0/2030057023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3db1680 msgr2=0x5579e3da97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.618+0000 7faddfc84740 1 --2- 172.21.15.5:0/2030057023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3db1680 0x5579e3da97f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fadd0006010 tx=0x7fadd000e7a0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.619+0000 7faddfc84740 1 -- 172.21.15.5:0/2030057023 shutdown_connections 2024-09-15T22:12:25.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.619+0000 7faddfc84740 1 --2- 172.21.15.5:0/2030057023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3db1680 0x5579e3da97f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.619+0000 7faddfc84740 1 --2- 172.21.15.5:0/2030057023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3da6dc0 0x5579e3da2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.619+0000 7faddfc84740 1 --2- 172.21.15.5:0/2030057023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 0x5579e3dbd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.619+0000 7faddfc84740 1 -- 172.21.15.5:0/2030057023 >> 172.21.15.5:0/2030057023 conn(0x5579e3da2a30 msgr2=0x5579e3da90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:25.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.619+0000 7faddfc84740 1 -- 172.21.15.5:0/2030057023 shutdown_connections 2024-09-15T22:12:25.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.619+0000 7faddfc84740 1 -- 172.21.15.5:0/2030057023 wait complete. 2024-09-15T22:12:25.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.620+0000 7faddfc84740 1 -- start start 2024-09-15T22:12:25.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.620+0000 7faddfc84740 1 Processor -- start 2024-09-15T22:12:25.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.620+0000 7faddfc84740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 0x5579e3db9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.620+0000 7faddfc84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da6dc0 0x5579e3db9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.620+0000 7faddfc84740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3db1680 0x5579e3db4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7faddfc84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fadd81bc560 con 0x5579e3da6dc0 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7faddfc84740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fadd81bd9d0 con 0x5579e3db1680 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7faddfc84740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fadd81bcc30 con 0x5579e3da64a0 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7faddfc84740 10 client.0 _ll_register_callbacks cb 0x7fadd00099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7fadd6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 0x5579e3db9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7fadd6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 0x5579e3db9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50050/0 (socket says 172.21.15.5:50050) 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7fadd6ffd640 1 -- 172.21.15.5:0/718506719 learned_addr learned my addr 172.21.15.5:0/718506719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7fadd67fc640 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da6dc0 0x5579e3db9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7fadd6ffd640 1 -- 172.21.15.5:0/718506719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3db1680 msgr2=0x5579e3db4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3db1680 0x5579e3db4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7fadd6ffd640 1 -- 172.21.15.5:0/718506719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da6dc0 msgr2=0x5579e3db9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da6dc0 0x5579e3db9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.621+0000 7fadd6ffd640 1 -- 172.21.15.5:0/718506719 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e3db13d0 con 0x5579e3da64a0 2024-09-15T22:12:25.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.622+0000 7fadd6ffd640 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 0x5579e3db9970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fadc800c6a0 tx=0x7fadc8018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20b50ed01aeb5cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:25.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.622+0000 7fadb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:12:25.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.622+0000 7fadb6ffd640 1 -- 172.21.15.5:0/718506719 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc80ceb50 con 0x5579e3da64a0 2024-09-15T22:12:25.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.622+0000 7fadb6ffd640 1 -- 172.21.15.5:0/718506719 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fadc8099b50 con 0x5579e3da64a0 2024-09-15T22:12:25.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.623+0000 7faddfc84740 1 -- 172.21.15.5:0/718506719 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd001b520 con 0x5579e3da64a0 2024-09-15T22:12:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.627+0000 7fadb6ffd640 1 -- 172.21.15.5:0/718506719 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fadc80a60d0 con 0x5579e3da64a0 2024-09-15T22:12:25.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.630+0000 7faddfc84740 10 client.25633 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:25.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.630+0000 7faddfc84740 1 -- 172.21.15.5:0/718506719 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5579e3e2b840 con 0x5579e3da64a0 2024-09-15T22:12:25.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.633+0000 7fadb6ffd640 1 -- 172.21.15.5:0/718506719 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadc8010ac0 con 0x5579e3da64a0 2024-09-15T22:12:25.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.633+0000 7fadb6ffd640 1 -- 172.21.15.5:0/718506719 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fadc809c8b0 con 0x5579e3da64a0 2024-09-15T22:12:25.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.635+0000 7faddfc84740 10 client.25633 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:25.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.635+0000 7faddfc84740 10 client.25633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:25.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.635+0000 7faddfc84740 20 client.25633 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:25.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.635+0000 7faddfc84740 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5579e3e2bb30 0x5579e3e2df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:25.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.635+0000 7faddfc84740 4 client.25633 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:12:25.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.635+0000 7faddfc84740 1 -- 172.21.15.5:0/718506719 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fadd001b520 con 0x5579e3e2bb30 2024-09-15T22:12:25.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.645+0000 7fadd67fc640 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5579e3e2bb30 0x5579e3e2df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.655+0000 7fadb6ffd640 10 client.25633 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:25.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.655+0000 7fadd67fc640 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5579e3e2bb30 0x5579e3e2df20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ea0e921d2044a0 server_cookie=5706bb0319a5f7bb in_seq=0 out_seq=0 2024-09-15T22:12:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.657+0000 7fadb6ffd640 1 -- 172.21.15.5:0/718506719 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fadd001b520 con 0x5579e3e2bb30 2024-09-15T22:12:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.657+0000 7fadb6ffd640 10 client.25633 handle_command_reply: tid=0 2024-09-15T22:12:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.657+0000 7faddfc84740 1 client.25633 shutdown 2024-09-15T22:12:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.657+0000 7faddfc84740 1 -- 172.21.15.5:0/718506719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 msgr2=0x5579e3db9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.657+0000 7faddfc84740 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 0x5579e3db9970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fadc800c6a0 tx=0x7fadc8018420 comp rx=0 tx=0).stop 2024-09-15T22:12:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.658+0000 7faddfc84740 1 -- 172.21.15.5:0/718506719 shutdown_connections 2024-09-15T22:12:25.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.658+0000 7faddfc84740 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5579e3e2bb30 0x5579e3e2df20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.658+0000 7fadb6ffd640 0 client.25633 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.658+0000 7faddfc84740 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5579e3db1680 0x5579e3db4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.658+0000 7faddfc84740 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5579e3da6dc0 0x5579e3db9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.658+0000 7faddfc84740 1 --2- 172.21.15.5:0/718506719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5579e3da64a0 0x5579e3db9970 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.658+0000 7faddfc84740 1 -- 172.21.15.5:0/718506719 >> 172.21.15.5:0/718506719 conn(0x5579e3da2a30 msgr2=0x5579e3db0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:25.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.658+0000 7faddfc84740 1 -- 172.21.15.5:0/718506719 shutdown_connections 2024-09-15T22:12:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.659+0000 7faddfc84740 1 -- 172.21.15.5:0/718506719 wait complete. 2024-09-15T22:12:25.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.660+0000 7faddfc84740 20 client.25633 trim_cache size 0 max 16384 2024-09-15T22:12:25.662 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:12:25.662 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:12:25.663 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:12:25.663 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:12:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.661+0000 7faddd8ab640 1 -- 172.21.15.5:0/2645730437 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fadc00a3480 msgr2=0x7fadc00a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.661+0000 7faddd8ab640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fadc00a3480 0x7fadc00a5940 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fadcc006010 tx=0x7fadcc004180 comp rx=0 tx=0).stop 2024-09-15T22:12:25.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.661+0000 7faddd8ab640 1 -- 172.21.15.5:0/2645730437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadd81008c0 msgr2=0x7fadd80739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.661+0000 7faddd8ab640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadd81008c0 0x7fadd80739e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fadc800e9f0 tx=0x7fadc800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.661+0000 7faddd8ab640 1 -- 172.21.15.5:0/2645730437 shutdown_connections 2024-09-15T22:12:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.661+0000 7faddd8ab640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fadc00a3480 0x7fadc00a5940 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.661+0000 7faddd8ab640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadd8105ec0 0x7fadd806cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.661+0000 7faddd8ab640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fadd81011e0 0x7fadd806ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.661+0000 7faddd8ab640 1 --2- 172.21.15.5:0/2645730437 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fadd81008c0 0x7fadd80739e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.661+0000 7faddd8ab640 1 -- 172.21.15.5:0/2645730437 >> 172.21.15.5:0/2645730437 conn(0x7fadd80fc7d0 msgr2=0x7fadd8103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.662+0000 7faddd8ab640 1 -- 172.21.15.5:0/2645730437 shutdown_connections 2024-09-15T22:12:25.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:25.662+0000 7faddd8ab640 1 -- 172.21.15.5:0/2645730437 wait complete. 2024-09-15T22:12:26.529 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:12:26.530 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:12:27.062 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:27.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:27 smithi005 ceph-mon[30944]: pgmap v109: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.8 MiB/s wr, 838 op/s 2024-09-15T22:12:27.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:27 smithi099 ceph-mon[34892]: pgmap v109: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.8 MiB/s wr, 838 op/s 2024-09-15T22:12:27.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:27 smithi129 ceph-mon[35314]: pgmap v109: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 7.8 MiB/s wr, 838 op/s 2024-09-15T22:12:27.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.697+0000 7f80dd981640 1 -- 172.21.15.5:0/3555865117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d8100e20 msgr2=0x7f80d8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:27.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.697+0000 7f80dd981640 1 --2- 172.21.15.5:0/3555865117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d8100e20 0x7f80d8101200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f80d0009cd0 tx=0x7f80d002a9a0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.698+0000 7f80dd981640 1 -- 172.21.15.5:0/3555865117 shutdown_connections 2024-09-15T22:12:27.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.698+0000 7f80dd981640 1 --2- 172.21.15.5:0/3555865117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d8102ef0 0x7f80d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.698+0000 7f80dd981640 1 --2- 172.21.15.5:0/3555865117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8102570 0x7f80d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.698+0000 7f80dd981640 1 --2- 172.21.15.5:0/3555865117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d8100e20 0x7f80d8101200 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.698+0000 7f80dd981640 1 -- 172.21.15.5:0/3555865117 >> 172.21.15.5:0/3555865117 conn(0x7f80d80fc7d0 msgr2=0x7f80d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:27.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.699+0000 7f80dd981640 1 -- 172.21.15.5:0/3555865117 shutdown_connections 2024-09-15T22:12:27.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.699+0000 7f80dd981640 1 -- 172.21.15.5:0/3555865117 wait complete. 2024-09-15T22:12:27.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.700+0000 7f80dd981640 1 Processor -- start 2024-09-15T22:12:27.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.700+0000 7f80dd981640 1 -- start start 2024-09-15T22:12:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.700+0000 7f80dd981640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d8100e20 0x7f80d81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.700+0000 7f80dd981640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d8102570 0x7f80d81a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.701+0000 7f80dd981640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8102ef0 0x7f80d819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.701+0000 7f80dd981640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f80d8111090 con 0x7f80d8102ef0 2024-09-15T22:12:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.701+0000 7f80dd981640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80d8110f10 con 0x7f80d8100e20 2024-09-15T22:12:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.701+0000 7f80dd981640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80d8111210 con 0x7f80d8102570 2024-09-15T22:12:27.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.701+0000 7f80d67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d8102570 0x7f80d81a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.701+0000 7f80d67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d8102570 0x7f80d81a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50066/0 (socket says 172.21.15.5:50066) 2024-09-15T22:12:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.701+0000 7f80d67fc640 1 -- 172.21.15.5:0/1936286967 learned_addr learned my addr 172.21.15.5:0/1936286967 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.701+0000 7f80d77fe640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8102ef0 0x7f80d819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.701+0000 7f80d6ffd640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d8100e20 0x7f80d81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.702+0000 7f80d67fc640 1 -- 172.21.15.5:0/1936286967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d8100e20 msgr2=0x7f80d81a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.702+0000 7f80d67fc640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d8100e20 0x7f80d81a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.702+0000 7f80d67fc640 1 -- 172.21.15.5:0/1936286967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8102ef0 msgr2=0x7f80d819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.702+0000 7f80d67fc640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8102ef0 0x7f80d819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:27.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.702+0000 7f80d67fc640 1 -- 172.21.15.5:0/1936286967 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d819af50 con 0x7f80d8102570 2024-09-15T22:12:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.702+0000 7f80d67fc640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d8102570 0x7f80d81a0b50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80c800b4a0 tx=0x7f80c800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.702+0000 7f80dc97f640 1 -- 172.21.15.5:0/1936286967 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8014070 con 0x7f80d8102570 2024-09-15T22:12:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.702+0000 7f80dc97f640 1 -- 172.21.15.5:0/1936286967 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80c8010af0 con 0x7f80d8102570 2024-09-15T22:12:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.702+0000 7f80dc97f640 1 -- 172.21.15.5:0/1936286967 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8017d10 con 0x7f80d8102570 2024-09-15T22:12:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.703+0000 7f80dd981640 1 -- 172.21.15.5:0/1936286967 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d819b240 con 0x7f80d8102570 2024-09-15T22:12:27.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.703+0000 7f80dd981640 1 -- 172.21.15.5:0/1936286967 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d81a7440 con 0x7f80d8102570 2024-09-15T22:12:27.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.704+0000 7f80dd981640 1 -- 172.21.15.5:0/1936286967 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d81021b0 con 0x7f80d8102570 2024-09-15T22:12:27.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.706+0000 7f80dc97f640 1 -- 172.21.15.5:0/1936286967 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f80c8017070 con 0x7f80d8102570 2024-09-15T22:12:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.708+0000 7f80dc97f640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80c40a3540 0x7f80c40a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.709+0000 7f80dc97f640 1 -- 172.21.15.5:0/1936286967 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f80c80cf210 con 0x7f80d8102570 2024-09-15T22:12:27.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.709+0000 7f80d6ffd640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80c40a3540 0x7f80c40a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:27.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.710+0000 7f80d6ffd640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80c40a3540 0x7f80c40a5a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80d0006010 tx=0x7f80d0000f40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:27.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:27.713+0000 7f80dc97f640 1 -- 172.21.15.5:0/1936286967 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80c809b2b0 con 0x7f80d8102570 2024-09-15T22:12:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.125+0000 7f80dd981640 1 -- 172.21.15.5:0/1936286967 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80d81a7670 con 0x7f80d8102570 2024-09-15T22:12:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.126+0000 7f80dc97f640 1 -- 172.21.15.5:0/1936286967 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f80c8095e20 con 0x7f80d8102570 2024-09-15T22:12:28.129 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:12:28.129 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:12:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:12:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.129+0000 7f80dd981640 1 -- 172.21.15.5:0/1936286967 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80c40a3540 msgr2=0x7f80c40a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.129+0000 7f80dd981640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80c40a3540 0x7f80c40a5a00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80d0006010 tx=0x7f80d0000f40 comp rx=0 tx=0).stop 2024-09-15T22:12:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.129+0000 7f80dd981640 1 -- 172.21.15.5:0/1936286967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d8102570 msgr2=0x7f80d81a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.129+0000 7f80dd981640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d8102570 0x7f80d81a0b50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f80c800b4a0 tx=0x7f80c800b960 comp rx=0 tx=0).stop 2024-09-15T22:12:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.129+0000 7f80dd981640 1 -- 172.21.15.5:0/1936286967 shutdown_connections 2024-09-15T22:12:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.129+0000 7f80dd981640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80c40a3540 0x7f80c40a5a00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.129+0000 7f80dd981640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8102ef0 0x7f80d819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.129+0000 7f80dd981640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80d8102570 0x7f80d81a0b50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.129+0000 7f80dd981640 1 --2- 172.21.15.5:0/1936286967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80d8100e20 0x7f80d81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.129+0000 7f80dd981640 1 -- 172.21.15.5:0/1936286967 >> 172.21.15.5:0/1936286967 conn(0x7f80d80fc7d0 msgr2=0x7f80d806a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.130+0000 7f80dd981640 1 -- 172.21.15.5:0/1936286967 shutdown_connections 2024-09-15T22:12:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:28.130+0000 7f80dd981640 1 -- 172.21.15.5:0/1936286967 wait complete. 2024-09-15T22:12:28.450 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:28 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1936286967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:28.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:28 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1936286967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:28.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:28 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1936286967' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:28.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:12:29.331 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:29.587 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:29 smithi005 ceph-mon[30944]: pgmap v110: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-09-15T22:12:29.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:29 smithi099 ceph-mon[34892]: pgmap v110: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-09-15T22:12:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:29 smithi129 ceph-mon[35314]: pgmap v110: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-09-15T22:12:29.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 180.0 s, 2508.2 tps, lat 40.065 ms stddev 59.022 2024-09-15T22:12:29.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.869+0000 7fabc4bcb640 1 -- 172.21.15.5:0/470780098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabc0112b90 msgr2=0x7fabc0114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.869+0000 7fabc4bcb640 1 --2- 172.21.15.5:0/470780098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabc0112b90 0x7fabc0114f80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009860 tx=0x7fabb402a410 comp rx=0 tx=0).stop 2024-09-15T22:12:29.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.874+0000 7fabc4bcb640 1 -- 172.21.15.5:0/470780098 shutdown_connections 2024-09-15T22:12:29.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.874+0000 7fabc4bcb640 1 --2- 172.21.15.5:0/470780098 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabc0112b90 0x7fabc0114f80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.874+0000 7fabc4bcb640 1 --2- 172.21.15.5:0/470780098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabc0074eb0 0x7fabc0112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.874+0000 7fabc4bcb640 1 --2- 172.21.15.5:0/470780098 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc0074590 0x7fabc0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.874+0000 7fabc4bcb640 1 -- 172.21.15.5:0/470780098 >> 172.21.15.5:0/470780098 conn(0x7fabc00fd8e0 msgr2=0x7fabc00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:29.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.875+0000 7fabc4bcb640 1 -- 172.21.15.5:0/470780098 shutdown_connections 2024-09-15T22:12:29.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.875+0000 7fabc4bcb640 1 -- 172.21.15.5:0/470780098 wait complete. 2024-09-15T22:12:29.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.876+0000 7fabc4bcb640 1 Processor -- start 2024-09-15T22:12:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.877+0000 7fabc4bcb640 1 -- start start 2024-09-15T22:12:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.878+0000 7fabc4bcb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabc0074590 0x7fabc0071080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.878+0000 7fabc4bcb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc0074eb0 0x7fabc00715c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.878+0000 7fabc4bcb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabc0112b90 0x7fabc0071b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.878+0000 7fabbe59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabc0074590 0x7fabc0071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.878+0000 7fabbe59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabc0074590 0x7fabc0071080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50082/0 (socket says 172.21.15.5:50082) 2024-09-15T22:12:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.878+0000 7fabbe59c640 1 -- 172.21.15.5:0/1677934138 learned_addr learned my addr 172.21.15.5:0/1677934138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.878+0000 7fabc4bcb640 1 -- 172.21.15.5:0/1677934138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabc0118550 con 0x7fabc0112b90 2024-09-15T22:12:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.878+0000 7fabc4bcb640 1 -- 172.21.15.5:0/1677934138 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fabc01183d0 con 0x7fabc0074eb0 2024-09-15T22:12:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.878+0000 7fabc4bcb640 1 -- 172.21.15.5:0/1677934138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fabc01186d0 con 0x7fabc0074590 2024-09-15T22:12:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.879+0000 7fabbdd9b640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc0074eb0 0x7fabc00715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.879+0000 7fabbe59c640 1 -- 172.21.15.5:0/1677934138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc0074eb0 msgr2=0x7fabc00715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.879+0000 7fabbe59c640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc0074eb0 0x7fabc00715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.879+0000 7fabbe59c640 1 -- 172.21.15.5:0/1677934138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabc0112b90 msgr2=0x7fabc0071b00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:12:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.879+0000 7fabbe59c640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabc0112b90 0x7fabc0071b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.879+0000 7fabbe59c640 1 -- 172.21.15.5:0/1677934138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc010ee30 con 0x7fabc0074590 2024-09-15T22:12:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.880+0000 7fabbe59c640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabc0074590 0x7fabc0071080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fabb800cd10 tx=0x7fabb800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.880+0000 7fabaf7fe640 1 -- 172.21.15.5:0/1677934138 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8004b20 con 0x7fabc0074590 2024-09-15T22:12:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.880+0000 7fabaf7fe640 1 -- 172.21.15.5:0/1677934138 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb8004cc0 con 0x7fabc0074590 2024-09-15T22:12:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.880+0000 7fabaf7fe640 1 -- 172.21.15.5:0/1677934138 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb8004b20 con 0x7fabc0074590 2024-09-15T22:12:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.881+0000 7fabc4bcb640 1 -- 172.21.15.5:0/1677934138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc010f120 con 0x7fabc0074590 2024-09-15T22:12:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.881+0000 7fabc4bcb640 1 -- 172.21.15.5:0/1677934138 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc010f610 con 0x7fabc0074590 2024-09-15T22:12:29.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.887+0000 7fabc6fa4740 1 Processor -- start 2024-09-15T22:12:29.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.888+0000 7fabc6fa4740 1 -- start start 2024-09-15T22:12:29.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.888+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544cb40 0x555b0544b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.888+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544b5d0 0x555b0544ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.888+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05456370 0x555b05458760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.888+0000 7fabc6fa4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b053b12f0 con 0x555b05456370 2024-09-15T22:12:29.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.888+0000 7fabc6fa4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555b05358cf0 con 0x555b0544cb40 2024-09-15T22:12:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.888+0000 7fabc6fa4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555b053ad2b0 con 0x555b0544b5d0 2024-09-15T22:12:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.888+0000 7fabbed9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544cb40 0x555b0544b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.888+0000 7fabbdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05456370 0x555b05458760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.888+0000 7fabbdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05456370 0x555b05458760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41960/0 (socket says 172.21.15.5:41960) 2024-09-15T22:12:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.888+0000 7fabbdd9b640 1 -- 172.21.15.5:0/3003344517 learned_addr learned my addr 172.21.15.5:0/3003344517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.889+0000 7fabbdd9b640 1 -- 172.21.15.5:0/3003344517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544b5d0 msgr2=0x555b0544ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:12:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.889+0000 7fabbdd9b640 1 --2- 172.21.15.5:0/3003344517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544b5d0 0x555b0544ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.889+0000 7fabbdd9b640 1 -- 172.21.15.5:0/3003344517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544cb40 msgr2=0x555b0544b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.889+0000 7fabbdd9b640 1 --2- 172.21.15.5:0/3003344517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544cb40 0x555b0544b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.889+0000 7fabbdd9b640 1 -- 172.21.15.5:0/3003344517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b0533d3a0 con 0x555b05456370 2024-09-15T22:12:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.889+0000 7fabbdd9b640 1 --2- 172.21.15.5:0/3003344517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05456370 0x555b05458760 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009ca0 tx=0x7fabb00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3d9829eeaf4f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.890+0000 7fabaf7fe640 1 -- 172.21.15.5:0/1677934138 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fabb800f5d0 con 0x7fabc0074590 2024-09-15T22:12:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.890+0000 7fabacff9640 1 -- 172.21.15.5:0/3003344517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb001c480 con 0x555b05456370 2024-09-15T22:12:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.890+0000 7fabacff9640 1 -- 172.21.15.5:0/3003344517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb001cb00 con 0x555b05456370 2024-09-15T22:12:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.890+0000 7fabacff9640 1 -- 172.21.15.5:0/3003344517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0011cb0 con 0x555b05456370 2024-09-15T22:12:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.892+0000 7fabaf7fe640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faba80a3870 0x7faba80a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.892+0000 7fabaf7fe640 1 -- 172.21.15.5:0/1677934138 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fabb800ce00 con 0x7fabc0074590 2024-09-15T22:12:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.893+0000 7fabc6fa4740 1 -- 172.21.15.5:0/3003344517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05456370 msgr2=0x555b05458760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.893+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/3003344517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05456370 0x555b05458760 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009ca0 tx=0x7fabb00107a0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.893+0000 7fabc6fa4740 1 -- 172.21.15.5:0/3003344517 shutdown_connections 2024-09-15T22:12:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.893+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/3003344517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05456370 0x555b05458760 secure :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009ca0 tx=0x7fabb00107a0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.893+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/3003344517 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544b5d0 0x555b0544ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.893+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/3003344517 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544cb40 0x555b0544b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.893+0000 7fabc6fa4740 1 -- 172.21.15.5:0/3003344517 >> 172.21.15.5:0/3003344517 conn(0x555b05414310 msgr2=0x555b05447c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.893+0000 7fabc6fa4740 1 -- 172.21.15.5:0/3003344517 shutdown_connections 2024-09-15T22:12:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.894+0000 7fabc6fa4740 1 -- 172.21.15.5:0/3003344517 wait complete. 2024-09-15T22:12:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.894+0000 7fabc6fa4740 1 -- start start 2024-09-15T22:12:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.894+0000 7fabc6fa4740 1 Processor -- start 2024-09-15T22:12:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.895+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544b5d0 0x555b05453300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.895+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b0544cb40 0x555b05453840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.895+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b05453d80 0x555b054625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.895+0000 7fabc6fa4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b05459070 con 0x555b0544cb40 2024-09-15T22:12:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.895+0000 7fabc6fa4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555b0536c060 con 0x555b0544b5d0 2024-09-15T22:12:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.895+0000 7fabc6fa4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555b054591f0 con 0x555b05453d80 2024-09-15T22:12:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.895+0000 7fabc6fa4740 10 client.0 _ll_register_callbacks cb 0x555b050bf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.895+0000 7fabbed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b05453d80 0x555b054625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.895+0000 7fabbed9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b05453d80 0x555b054625b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50090/0 (socket says 172.21.15.5:50090) 2024-09-15T22:12:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.895+0000 7fabbed9d640 1 -- 172.21.15.5:0/2354483077 learned_addr learned my addr 172.21.15.5:0/2354483077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.895+0000 7fabbdd9b640 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b0544cb40 0x555b05453840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.896+0000 7fabbed9d640 1 -- 172.21.15.5:0/2354483077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544b5d0 msgr2=0x555b05453300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:12:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.896+0000 7fabbed9d640 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544b5d0 0x555b05453300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.896+0000 7fabbed9d640 1 -- 172.21.15.5:0/2354483077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b0544cb40 msgr2=0x555b05453840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.896+0000 7fabbed9d640 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b0544cb40 0x555b05453840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.896+0000 7fabbed9d640 1 -- 172.21.15.5:0/2354483077 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b051223c0 con 0x555b05453d80 2024-09-15T22:12:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.898+0000 7fabbed9d640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faba80a3870 0x7faba80a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.898+0000 7fabbed9d640 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b05453d80 0x555b054625b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fabc0076600 tx=0x7fabb4030690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea097629fe376857 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.898+0000 7faba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:12:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.898+0000 7faba5ffb640 1 -- 172.21.15.5:0/2354483077 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4005320 con 0x555b05453d80 2024-09-15T22:12:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.898+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2354483077 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb001cb00 con 0x555b05453d80 2024-09-15T22:12:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.898+0000 7faba5ffb640 1 -- 172.21.15.5:0/2354483077 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fabb4005540 con 0x555b05453d80 2024-09-15T22:12:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7faba5ffb640 1 -- 172.21.15.5:0/2354483077 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fabb4056050 con 0x555b05453d80 2024-09-15T22:12:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7fabbed9d640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faba80a3870 0x7faba80a5d30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fabb4030a00 tx=0x7fabb4049000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7fabc6fa4740 10 client.25648 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2354483077 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b054652d0 con 0x555b05453d80 2024-09-15T22:12:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7faba5ffb640 1 -- 172.21.15.5:0/2354483077 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb001cb00 con 0x555b05453d80 2024-09-15T22:12:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7faba5ffb640 1 -- 172.21.15.5:0/2354483077 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fabb403ea20 con 0x555b05453d80 2024-09-15T22:12:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7fabc6fa4740 10 client.25648 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7fabc6fa4740 10 client.25648 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:12:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7fabc6fa4740 20 client.25648 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555b05466c80 0x555b05469100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7fabc6fa4740 4 client.25648 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.899+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2354483077 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b053be860 con 0x555b05466c80 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.900+0000 7fabbe59c640 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555b05466c80 0x555b05469100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.900+0000 7fabbe59c640 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555b05466c80 0x555b05469100 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8572e59c5eef2d4 server_cookie=24fb2676b5486afc in_seq=0 out_seq=0 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.900+0000 7faba5ffb640 10 client.25648 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7faba5ffb640 1 -- 172.21.15.5:0/2354483077 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b053be860 con 0x555b05466c80 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7faba5ffb640 10 client.25648 handle_command_reply: tid=0 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7fabc6fa4740 1 client.25648 shutdown 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2354483077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b05453d80 msgr2=0x555b054625b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b05453d80 0x555b054625b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fabc0076600 tx=0x7fabb4030690 comp rx=0 tx=0).stop 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2354483077 shutdown_connections 2024-09-15T22:12:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555b05466c80 0x555b05469100 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b05453d80 0x555b054625b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b0544cb40 0x555b05453840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/2354483077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544b5d0 0x555b05453300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2354483077 >> 172.21.15.5:0/2354483077 conn(0x555b05414310 msgr2=0x555b0544d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.901+0000 7faba5ffb640 0 client.25648 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:12:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.902+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2354483077 shutdown_connections 2024-09-15T22:12:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.902+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2354483077 wait complete. 2024-09-15T22:12:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.902+0000 7fabc6fa4740 20 client.25648 trim_cache size 0 max 16384 2024-09-15T22:12:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.924+0000 7fabc6fa4740 1 Processor -- start 2024-09-15T22:12:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.924+0000 7fabc6fa4740 1 -- start start 2024-09-15T22:12:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.924+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b0544b5d0 0x555b054623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.924+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b05455270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.924+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b05453d80 0x555b0546ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.924+0000 7fabc6fa4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faba0012880 con 0x555b0544b5d0 2024-09-15T22:12:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.924+0000 7fabbdd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b05455270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.924+0000 7fabbe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b0544b5d0 0x555b054623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.924+0000 7fabbdd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b05455270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50096/0 (socket says 172.21.15.5:50096) 2024-09-15T22:12:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.924+0000 7fabbdd9b640 1 -- 172.21.15.5:0/2165205508 learned_addr learned my addr 172.21.15.5:0/2165205508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2165205508 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faba0010150 con 0x555b05453d80 2024-09-15T22:12:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7fabbed9d640 1 --2- 172.21.15.5:0/2165205508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b05453d80 0x555b0546ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2165205508 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faba000dfa0 con 0x555b0544cb40 2024-09-15T22:12:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7fabbed9d640 1 -- 172.21.15.5:0/2165205508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 msgr2=0x555b05455270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7fabbed9d640 1 --2- 172.21.15.5:0/2165205508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b05455270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7fabbed9d640 1 -- 172.21.15.5:0/2165205508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b0544b5d0 msgr2=0x555b054623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7fabbed9d640 1 --2- 172.21.15.5:0/2165205508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b0544b5d0 0x555b054623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7fabbed9d640 1 -- 172.21.15.5:0/2165205508 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b05448dd0 con 0x555b05453d80 2024-09-15T22:12:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7fabbdd9b640 1 --2- 172.21.15.5:0/2165205508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b05455270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7fabbe59c640 1 --2- 172.21.15.5:0/2165205508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b0544b5d0 0x555b054623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7fabbed9d640 1 --2- 172.21.15.5:0/2165205508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b05453d80 0x555b0546ea10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fabb403ee90 tx=0x7fabb402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5064626b9b62cea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.925+0000 7faba7fff640 1 -- 172.21.15.5:0/2165205508 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4004040 con 0x555b05453d80 2024-09-15T22:12:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.926+0000 7faba7fff640 1 -- 172.21.15.5:0/2165205508 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb40055e0 con 0x555b05453d80 2024-09-15T22:12:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.926+0000 7faba7fff640 1 -- 172.21.15.5:0/2165205508 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4004d10 con 0x555b05453d80 2024-09-15T22:12:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.927+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2165205508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b05453d80 msgr2=0x555b0546ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.927+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/2165205508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b05453d80 0x555b0546ea10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fabb403ee90 tx=0x7fabb402a410 comp rx=0 tx=0).stop 2024-09-15T22:12:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.927+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2165205508 shutdown_connections 2024-09-15T22:12:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.927+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/2165205508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b05453d80 0x555b0546ea10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.927+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/2165205508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b05455270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.927+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/2165205508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b0544b5d0 0x555b054623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.927+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2165205508 >> 172.21.15.5:0/2165205508 conn(0x555b05414310 msgr2=0x555b05454b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.927+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2165205508 shutdown_connections 2024-09-15T22:12:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.927+0000 7fabc6fa4740 1 -- 172.21.15.5:0/2165205508 wait complete. 2024-09-15T22:12:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.927+0000 7fabc6fa4740 1 -- start start 2024-09-15T22:12:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabc6fa4740 1 Processor -- start 2024-09-15T22:12:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544b5d0 0x555b0545ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b0544daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabc6fa4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05453d80 0x555b0544e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabbdd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b0544daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabbdd9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b0544daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50104/0 (socket says 172.21.15.5:50104) 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabbdd9b640 1 -- 172.21.15.5:0/4154838350 learned_addr learned my addr 172.21.15.5:0/4154838350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabc6fa4740 1 -- 172.21.15.5:0/4154838350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faba000e610 con 0x555b05453d80 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabc6fa4740 1 -- 172.21.15.5:0/4154838350 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faba000fa80 con 0x555b0544b5d0 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabc6fa4740 1 -- 172.21.15.5:0/4154838350 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faba000ece0 con 0x555b0544cb40 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabbed9d640 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05453d80 0x555b0544e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.928+0000 7fabc6fa4740 10 client.0 _ll_register_callbacks cb 0x7fabb40096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.929+0000 7fabbdd9b640 1 -- 172.21.15.5:0/4154838350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544b5d0 msgr2=0x555b0545ec10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.929+0000 7fabbdd9b640 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544b5d0 0x555b0545ec10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.929+0000 7fabbdd9b640 1 -- 172.21.15.5:0/4154838350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05453d80 msgr2=0x555b0544e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.929+0000 7fabbdd9b640 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05453d80 0x555b0544e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.929+0000 7fabbdd9b640 1 -- 172.21.15.5:0/4154838350 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b0544e830 con 0x555b0544cb40 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.929+0000 7fabbed9d640 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05453d80 0x555b0544e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.929+0000 7fabbdd9b640 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b0544daf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fabb0000c00 tx=0x7fabb0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da536e5dcda5304 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.929+0000 7faba5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.930+0000 7faba5ffb640 1 -- 172.21.15.5:0/4154838350 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0013d30 con 0x555b0544cb40 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.930+0000 7fabc6fa4740 1 -- 172.21.15.5:0/4154838350 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb40055e0 con 0x555b0544cb40 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.930+0000 7faba5ffb640 1 -- 172.21.15.5:0/4154838350 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fabb0011880 con 0x555b0544cb40 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.930+0000 7faba5ffb640 1 -- 172.21.15.5:0/4154838350 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fabb0056050 con 0x555b0544cb40 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.930+0000 7fabc6fa4740 10 client.25654 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.930+0000 7fabc6fa4740 1 -- 172.21.15.5:0/4154838350 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b054d19e0 con 0x555b0544cb40 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.930+0000 7faba5ffb640 1 -- 172.21.15.5:0/4154838350 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb0005040 con 0x555b0544cb40 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.930+0000 7faba5ffb640 1 -- 172.21.15.5:0/4154838350 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fabb0051030 con 0x555b0544cb40 2024-09-15T22:12:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.930+0000 7fabc6fa4740 10 client.25654 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.930+0000 7fabc6fa4740 10 client.25654 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:12:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.930+0000 7fabc6fa4740 20 client.25654 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.931+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555b054d2030 0x555b054d4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.931+0000 7fabc6fa4740 4 client.25654 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:12:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.931+0000 7fabc6fa4740 1 -- 172.21.15.5:0/4154838350 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fabb40055e0 con 0x555b054d2030 2024-09-15T22:12:29.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.931+0000 7fabbe59c640 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555b054d2030 0x555b054d4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.931+0000 7faba5ffb640 10 client.25654 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:12:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.931+0000 7fabbe59c640 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555b054d2030 0x555b054d4450 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aeb2a38d69576fca server_cookie=8219d3225283bed2 in_seq=0 out_seq=0 2024-09-15T22:12:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7faba5ffb640 1 -- 172.21.15.5:0/4154838350 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fabb40055e0 con 0x555b054d2030 2024-09-15T22:12:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7faba5ffb640 10 client.25654 handle_command_reply: tid=0 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 1 client.25654 shutdown 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 1 -- 172.21.15.5:0/4154838350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 msgr2=0x555b0544daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b0544daf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fabb0000c00 tx=0x7fabb0011810 comp rx=0 tx=0).stop 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 1 -- 172.21.15.5:0/4154838350 shutdown_connections 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555b054d2030 0x555b054d4450 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b05453d80 0x555b0544e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555b0544cb40 0x555b0544daf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 1 --2- 172.21.15.5:0/4154838350 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555b0544b5d0 0x555b0545ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 1 -- 172.21.15.5:0/4154838350 >> 172.21.15.5:0/4154838350 conn(0x555b05414310 msgr2=0x555b05454b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7faba5ffb640 0 client.25654 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 1 -- 172.21.15.5:0/4154838350 shutdown_connections 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 1 -- 172.21.15.5:0/4154838350 wait complete. 2024-09-15T22:12:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.932+0000 7fabc6fa4740 20 client.25654 trim_cache size 0 max 16384 2024-09-15T22:12:29.938 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:12:29.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 -- 172.21.15.5:0/1677934138 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faba80a3870 msgr2=0x7faba80a5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faba80a3870 0x7faba80a5d30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fabb4030a00 tx=0x7fabb4049000 comp rx=0 tx=0).stop 2024-09-15T22:12:29.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 -- 172.21.15.5:0/1677934138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabc0074590 msgr2=0x7fabc0071080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:29.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabc0074590 0x7fabc0071080 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fabb800cd10 tx=0x7fabb800a620 comp rx=0 tx=0).stop 2024-09-15T22:12:29.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 -- 172.21.15.5:0/1677934138 shutdown_connections 2024-09-15T22:12:29.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faba80a3870 0x7faba80a5d30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabc0112b90 0x7fabc0071b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabc0074eb0 0x7fabc00715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 --2- 172.21.15.5:0/1677934138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabc0074590 0x7fabc0071080 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 -- 172.21.15.5:0/1677934138 >> 172.21.15.5:0/1677934138 conn(0x7fabc00fd8e0 msgr2=0x7fabc00ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 -- 172.21.15.5:0/1677934138 shutdown_connections 2024-09-15T22:12:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:29.933+0000 7fabc4bcb640 1 -- 172.21.15.5:0/1677934138 wait complete. 2024-09-15T22:12:31.329 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:12:31.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:31 smithi099 ceph-mon[34892]: pgmap v111: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.7 MiB/s wr, 619 op/s 2024-09-15T22:12:31.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:31 smithi129 ceph-mon[35314]: pgmap v111: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.7 MiB/s wr, 619 op/s 2024-09-15T22:12:31.609 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:31 smithi005 ceph-mon[30944]: pgmap v111: 129 pgs: 129 active+clean; 1.9 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 5.7 MiB/s wr, 619 op/s 2024-09-15T22:12:31.743 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.302+0000 7f3826860640 1 -- 172.21.15.5:0/2185964138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3820076720 msgr2=0x7f3820076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.302+0000 7f3826860640 1 --2- 172.21.15.5:0/2185964138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3820076720 0x7f3820076b00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3808009900 tx=0x7f380802a410 comp rx=0 tx=0).stop 2024-09-15T22:12:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.302+0000 7f3826860640 1 -- 172.21.15.5:0/2185964138 shutdown_connections 2024-09-15T22:12:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.302+0000 7f3826860640 1 --2- 172.21.15.5:0/2185964138 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3820112a30 0x7f3820114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.302+0000 7f3826860640 1 --2- 172.21.15.5:0/2185964138 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3820077040 0x7f38201124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.302+0000 7f3826860640 1 --2- 172.21.15.5:0/2185964138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3820076720 0x7f3820076b00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.302+0000 7f3826860640 1 -- 172.21.15.5:0/2185964138 >> 172.21.15.5:0/2185964138 conn(0x7f38200ffa70 msgr2=0x7f3820101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.302+0000 7f3826860640 1 -- 172.21.15.5:0/2185964138 shutdown_connections 2024-09-15T22:12:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.303+0000 7f3826860640 1 -- 172.21.15.5:0/2185964138 wait complete. 2024-09-15T22:12:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.304+0000 7f3826860640 1 Processor -- start 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.304+0000 7f3826860640 1 -- start start 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.304+0000 7f3826860640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3820077040 0x7f382019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.304+0000 7f3826860640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3820112a30 0x7f382019f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.304+0000 7f3826860640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38201a3630 0x7f38201a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.304+0000 7f3826860640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3820118670 con 0x7f3820077040 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.304+0000 7f3826860640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f38201184f0 con 0x7f3820112a30 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.304+0000 7f3826860640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f38201187f0 con 0x7f38201a3630 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.304+0000 7f381ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3820077040 0x7f382019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.304+0000 7f381ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3820077040 0x7f382019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42010/0 (socket says 172.21.15.5:42010) 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.304+0000 7f381ffff640 1 -- 172.21.15.5:0/51312481 learned_addr learned my addr 172.21.15.5:0/51312481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.305+0000 7f3824dfd640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38201a3630 0x7f38201a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.305+0000 7f381f7fe640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3820112a30 0x7f382019f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.305+0000 7f381ffff640 1 -- 172.21.15.5:0/51312481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38201a3630 msgr2=0x7f38201a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.305+0000 7f381ffff640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38201a3630 0x7f38201a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.305+0000 7f381ffff640 1 -- 172.21.15.5:0/51312481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3820112a30 msgr2=0x7f382019f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.305+0000 7f381ffff640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3820112a30 0x7f382019f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.305+0000 7f381ffff640 1 -- 172.21.15.5:0/51312481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38201a40f0 con 0x7f3820077040 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.305+0000 7f3824dfd640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38201a3630 0x7f38201a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.305+0000 7f381ffff640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3820077040 0x7f382019ed60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3808004770 tx=0x7f3808003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.306+0000 7f381d7fa640 1 -- 172.21.15.5:0/51312481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380803f020 con 0x7f3820077040 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.306+0000 7f381d7fa640 1 -- 172.21.15.5:0/51312481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3808005d10 con 0x7f3820077040 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.306+0000 7f381d7fa640 1 -- 172.21.15.5:0/51312481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808035070 con 0x7f3820077040 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.306+0000 7f3826860640 1 -- 172.21.15.5:0/51312481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38201a4380 con 0x7f3820077040 2024-09-15T22:12:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.306+0000 7f3826860640 1 -- 172.21.15.5:0/51312481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38201abab0 con 0x7f3820077040 2024-09-15T22:12:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.310+0000 7f381d7fa640 1 -- 172.21.15.5:0/51312481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3808042020 con 0x7f3820077040 2024-09-15T22:12:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.311+0000 7f3828c39740 1 Processor -- start 2024-09-15T22:12:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.311+0000 7f3828c39740 1 -- start start 2024-09-15T22:12:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.311+0000 7f3828c39740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a5fb40 0x563bf1a5e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.311+0000 7f3828c39740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5e5d0 0x563bf1a5ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.311+0000 7f3824dfd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a5fb40 0x563bf1a5e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.311+0000 7f3824dfd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a5fb40 0x563bf1a5e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50142/0 (socket says 172.21.15.5:50142) 2024-09-15T22:12:32.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.311+0000 7f3828c39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a69370 0x563bf1a6b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.311+0000 7f3828c39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563bf19c42f0 con 0x563bf1a69370 2024-09-15T22:12:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.311+0000 7f3828c39740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563bf196bcf0 con 0x563bf1a5e5d0 2024-09-15T22:12:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.311+0000 7f3828c39740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563bf19c02b0 con 0x563bf1a5fb40 2024-09-15T22:12:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.312+0000 7f381ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5e5d0 0x563bf1a5ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.312+0000 7f381ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5e5d0 0x563bf1a5ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:32976/0 (socket says 172.21.15.5:32976) 2024-09-15T22:12:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.312+0000 7f381ffff640 1 -- 172.21.15.5:0/3016438908 learned_addr learned my addr 172.21.15.5:0/3016438908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.312+0000 7f381ffff640 1 -- 172.21.15.5:0/3016438908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a5fb40 msgr2=0x563bf1a5e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.312+0000 7f381ffff640 1 --2- 172.21.15.5:0/3016438908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a5fb40 0x563bf1a5e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.312+0000 7f381ffff640 1 -- 172.21.15.5:0/3016438908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a69370 msgr2=0x563bf1a6b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.312+0000 7f381f7fe640 1 --2- 172.21.15.5:0/3016438908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a69370 0x563bf1a6b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.312+0000 7f381ffff640 1 --2- 172.21.15.5:0/3016438908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a69370 0x563bf1a6b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.312+0000 7f381ffff640 1 -- 172.21.15.5:0/3016438908 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bf19503a0 con 0x563bf1a5e5d0 2024-09-15T22:12:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.312+0000 7f381f7fe640 1 --2- 172.21.15.5:0/3016438908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a69370 0x563bf1a6b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.313+0000 7f381d7fa640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f380c0a34b0 0x7f380c0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.313+0000 7f3824dfd640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f380c0a34b0 0x7f380c0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.313+0000 7f381d7fa640 1 -- 172.21.15.5:0/51312481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f38080f0800 con 0x7f3820077040 2024-09-15T22:12:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.313+0000 7f381ffff640 1 --2- 172.21.15.5:0/3016438908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5e5d0 0x563bf1a5ea10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f38080eda90 tx=0x7f38080f6cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a1a7d3e59d5baa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.313+0000 7f38167fc640 1 -- 172.21.15.5:0/3016438908 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38080b8980 con 0x563bf1a5e5d0 2024-09-15T22:12:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.314+0000 7f38167fc640 1 -- 172.21.15.5:0/3016438908 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38080bcde0 con 0x563bf1a5e5d0 2024-09-15T22:12:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.314+0000 7f38167fc640 1 -- 172.21.15.5:0/3016438908 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38080bc040 con 0x563bf1a5e5d0 2024-09-15T22:12:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.315+0000 7f3828c39740 1 -- 172.21.15.5:0/3016438908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5e5d0 msgr2=0x563bf1a5ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.315+0000 7f3828c39740 1 --2- 172.21.15.5:0/3016438908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5e5d0 0x563bf1a5ea10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f38080eda90 tx=0x7f38080f6cb0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.315+0000 7f3824dfd640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f380c0a34b0 0x7f380c0a5970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3818009fd0 tx=0x7f381800d040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.315+0000 7f3828c39740 1 -- 172.21.15.5:0/3016438908 shutdown_connections 2024-09-15T22:12:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.315+0000 7f3828c39740 1 --2- 172.21.15.5:0/3016438908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a69370 0x563bf1a6b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.315+0000 7f3828c39740 1 --2- 172.21.15.5:0/3016438908 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5e5d0 0x563bf1a5ea10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.315+0000 7f3828c39740 1 --2- 172.21.15.5:0/3016438908 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a5fb40 0x563bf1a5e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.315+0000 7f3828c39740 1 -- 172.21.15.5:0/3016438908 >> 172.21.15.5:0/3016438908 conn(0x563bf1a27310 msgr2=0x563bf1a5ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.315+0000 7f3828c39740 1 -- 172.21.15.5:0/3016438908 shutdown_connections 2024-09-15T22:12:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.315+0000 7f3828c39740 1 -- 172.21.15.5:0/3016438908 wait complete. 2024-09-15T22:12:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.315+0000 7f3828c39740 1 -- start start 2024-09-15T22:12:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.316+0000 7f3828c39740 1 Processor -- start 2024-09-15T22:12:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.316+0000 7f3828c39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a68d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.316+0000 7f3828c39740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 0x563bf1a66040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f3828c39740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a69370 0x563bf1a66650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f3828c39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563bf1a6be30 con 0x563bf1a5e5d0 2024-09-15T22:12:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f3828c39740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563bf197f060 con 0x563bf1a5fb40 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f3828c39740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563bf1a6bfb0 con 0x563bf1a69370 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f3828c39740 10 client.0 _ll_register_callbacks cb 0x563bf16d28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f381ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a68d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f381ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a68d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42038/0 (socket says 172.21.15.5:42038) 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f381ffff640 1 -- 172.21.15.5:0/3964752840 learned_addr learned my addr 172.21.15.5:0/3964752840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f381f7fe640 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 0x563bf1a66040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f3824dfd640 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a69370 0x563bf1a66650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f381ffff640 1 -- 172.21.15.5:0/3964752840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a69370 msgr2=0x563bf1a66650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f381ffff640 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a69370 0x563bf1a66650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f381ffff640 1 -- 172.21.15.5:0/3964752840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 msgr2=0x563bf1a66040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f381ffff640 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 0x563bf1a66040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f381ffff640 1 -- 172.21.15.5:0/3964752840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bf17353c0 con 0x563bf1a5e5d0 2024-09-15T22:12:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f3824dfd640 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a69370 0x563bf1a66650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f381ffff640 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a68d90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f38080c4e20 tx=0x7f3808031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88b3265730a4282 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.317+0000 7f37ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:12:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.318+0000 7f37ff7fe640 1 -- 172.21.15.5:0/3964752840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38080bc220 con 0x563bf1a5e5d0 2024-09-15T22:12:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.318+0000 7f37ff7fe640 1 -- 172.21.15.5:0/3964752840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f38080b8d50 con 0x563bf1a5e5d0 2024-09-15T22:12:32.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.318+0000 7f3828c39740 1 -- 172.21.15.5:0/3964752840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38080bcde0 con 0x563bf1a5e5d0 2024-09-15T22:12:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.321+0000 7f37ff7fe640 1 -- 172.21.15.5:0/3964752840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f38080bc4b0 con 0x563bf1a5e5d0 2024-09-15T22:12:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.321+0000 7f3828c39740 10 client.16107 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.321+0000 7f3828c39740 1 -- 172.21.15.5:0/3964752840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563bf1a6be30 con 0x563bf1a5e5d0 2024-09-15T22:12:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.321+0000 7f37ff7fe640 1 -- 172.21.15.5:0/3964752840 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38080bcde0 con 0x563bf1a5e5d0 2024-09-15T22:12:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.321+0000 7f37ff7fe640 1 -- 172.21.15.5:0/3964752840 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f38080eddd0 con 0x563bf1a5e5d0 2024-09-15T22:12:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.321+0000 7f3828c39740 10 client.16107 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.321+0000 7f3828c39740 10 client.16107 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:12:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.321+0000 7f3828c39740 20 client.16107 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:32.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.321+0000 7f3828c39740 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563bf1a789a0 0x563bf1a7ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.321+0000 7f3828c39740 4 client.16107 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:32.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.321+0000 7f3828c39740 1 -- 172.21.15.5:0/3964752840 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bf19d1860 con 0x563bf1a789a0 2024-09-15T22:12:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.322+0000 7f381f7fe640 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563bf1a789a0 0x563bf1a7ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.322+0000 7f381f7fe640 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563bf1a789a0 0x563bf1a7ade0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ade308fff13e598e server_cookie=ddce34fea2bd42b8 in_seq=0 out_seq=0 2024-09-15T22:12:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.322+0000 7f37ff7fe640 10 client.16107 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.323+0000 7f37ff7fe640 1 -- 172.21.15.5:0/3964752840 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563bf19d1860 con 0x563bf1a789a0 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.323+0000 7f37ff7fe640 10 client.16107 handle_command_reply: tid=0 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.323+0000 7f3828c39740 1 client.16107 shutdown 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.324+0000 7f3828c39740 1 -- 172.21.15.5:0/3964752840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 msgr2=0x563bf1a68d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.324+0000 7f3828c39740 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a68d90 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f38080c4e20 tx=0x7f3808031490 comp rx=0 tx=0).stop 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.325+0000 7f3828c39740 1 -- 172.21.15.5:0/3964752840 shutdown_connections 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.325+0000 7f3828c39740 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563bf1a789a0 0x563bf1a7ade0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.325+0000 7f3828c39740 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a69370 0x563bf1a66650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.325+0000 7f3828c39740 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 0x563bf1a66040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.325+0000 7f3828c39740 1 --2- 172.21.15.5:0/3964752840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a68d90 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.325+0000 7f3828c39740 1 -- 172.21.15.5:0/3964752840 >> 172.21.15.5:0/3964752840 conn(0x563bf1a27310 msgr2=0x563bf1a60960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.325+0000 7f37ff7fe640 0 client.16107 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:12:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.325+0000 7f3828c39740 1 -- 172.21.15.5:0/3964752840 shutdown_connections 2024-09-15T22:12:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.325+0000 7f3828c39740 1 -- 172.21.15.5:0/3964752840 wait complete. 2024-09-15T22:12:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.325+0000 7f3828c39740 20 client.16107 trim_cache size 0 max 16384 2024-09-15T22:12:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.344+0000 7f3828c39740 1 Processor -- start 2024-09-15T22:12:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.349+0000 7f3828c39740 1 -- start start 2024-09-15T22:12:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.349+0000 7f3828c39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a75090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.349+0000 7f3828c39740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 0x563bf1a61120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.349+0000 7f3828c39740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a69370 0x563bf1a61660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.349+0000 7f3828c39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38201b8600 con 0x563bf1a5e5d0 2024-09-15T22:12:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.349+0000 7f3828c39740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f38201b5ed0 con 0x563bf1a5fb40 2024-09-15T22:12:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.349+0000 7f3828c39740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f38201b3d20 con 0x563bf1a69370 2024-09-15T22:12:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.350+0000 7f381f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 0x563bf1a61120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.350+0000 7f381f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 0x563bf1a61120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:32992/0 (socket says 172.21.15.5:32992) 2024-09-15T22:12:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.350+0000 7f381f7fe640 1 -- 172.21.15.5:0/2248480758 learned_addr learned my addr 172.21.15.5:0/2248480758 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.354+0000 7f381f7fe640 1 -- 172.21.15.5:0/2248480758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a69370 msgr2=0x563bf1a61660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.354+0000 7f381f7fe640 1 --2- 172.21.15.5:0/2248480758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a69370 0x563bf1a61660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.354+0000 7f381f7fe640 1 -- 172.21.15.5:0/2248480758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 msgr2=0x563bf1a75090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.354+0000 7f381f7fe640 1 --2- 172.21.15.5:0/2248480758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a75090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.354+0000 7f381f7fe640 1 -- 172.21.15.5:0/2248480758 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bf1a669c0 con 0x563bf1a5fb40 2024-09-15T22:12:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.356+0000 7f381f7fe640 1 --2- 172.21.15.5:0/2248480758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 0x563bf1a61120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x563bf1a658d0 tx=0x7f38100044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e85a94f38480709 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.359+0000 7f3815ffb640 1 -- 172.21.15.5:0/2248480758 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810022070 con 0x563bf1a5fb40 2024-09-15T22:12:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.359+0000 7f3815ffb640 1 -- 172.21.15.5:0/2248480758 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3810003da0 con 0x563bf1a5fb40 2024-09-15T22:12:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.359+0000 7f3815ffb640 1 -- 172.21.15.5:0/2248480758 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810012a40 con 0x563bf1a5fb40 2024-09-15T22:12:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.362+0000 7f3828c39740 1 -- 172.21.15.5:0/2248480758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 msgr2=0x563bf1a61120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.362+0000 7f3828c39740 1 --2- 172.21.15.5:0/2248480758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 0x563bf1a61120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x563bf1a658d0 tx=0x7f38100044d0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.363+0000 7f3828c39740 1 -- 172.21.15.5:0/2248480758 shutdown_connections 2024-09-15T22:12:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.363+0000 7f3828c39740 1 --2- 172.21.15.5:0/2248480758 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a69370 0x563bf1a61660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.363+0000 7f3828c39740 1 --2- 172.21.15.5:0/2248480758 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a5fb40 0x563bf1a61120 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.363+0000 7f3828c39740 1 --2- 172.21.15.5:0/2248480758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a75090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.363+0000 7f3828c39740 1 -- 172.21.15.5:0/2248480758 >> 172.21.15.5:0/2248480758 conn(0x563bf1a27310 msgr2=0x563bf1a65150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.364+0000 7f3828c39740 1 -- 172.21.15.5:0/2248480758 shutdown_connections 2024-09-15T22:12:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.364+0000 7f3828c39740 1 -- 172.21.15.5:0/2248480758 wait complete. 2024-09-15T22:12:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.365+0000 7f3828c39740 1 -- start start 2024-09-15T22:12:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.365+0000 7f3828c39740 1 Processor -- start 2024-09-15T22:12:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.366+0000 7f3828c39740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.366+0000 7f3828c39740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a5fb40 0x563bf1a71920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.366+0000 7f3828c39740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a69370 0x563bf1a62890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.367+0000 7f3828c39740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38201b4390 con 0x563bf1a5e5d0 2024-09-15T22:12:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.367+0000 7f3824dfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a69370 0x563bf1a62890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.367+0000 7f3824dfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a69370 0x563bf1a62890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:32996/0 (socket says 172.21.15.5:32996) 2024-09-15T22:12:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.367+0000 7f3824dfd640 1 -- 172.21.15.5:0/2333696016 learned_addr learned my addr 172.21.15.5:0/2333696016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.367+0000 7f3828c39740 1 -- 172.21.15.5:0/2333696016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f38201b5800 con 0x563bf1a69370 2024-09-15T22:12:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.368+0000 7f3828c39740 1 -- 172.21.15.5:0/2333696016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f38201b4a60 con 0x563bf1a5fb40 2024-09-15T22:12:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.368+0000 7f3828c39740 10 client.0 _ll_register_callbacks cb 0x7f3818009d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.368+0000 7f381ffff640 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.369+0000 7f3824dfd640 1 -- 172.21.15.5:0/2333696016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a5fb40 msgr2=0x563bf1a71920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.369+0000 7f3824dfd640 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a5fb40 0x563bf1a71920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.369+0000 7f3824dfd640 1 -- 172.21.15.5:0/2333696016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 msgr2=0x563bf1a713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.369+0000 7f3824dfd640 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a713e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.369+0000 7f3824dfd640 1 -- 172.21.15.5:0/2333696016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bf1a63070 con 0x563bf1a69370 2024-09-15T22:12:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.369+0000 7f3824dfd640 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a69370 0x563bf1a62890 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3818018710 tx=0x7f3818009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92d7a3bccb4d47b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.371+0000 7f37ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:12:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.372+0000 7f37ff7fe640 1 -- 172.21.15.5:0/2333696016 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381801e040 con 0x563bf1a69370 2024-09-15T22:12:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.372+0000 7f37ff7fe640 1 -- 172.21.15.5:0/2333696016 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f3818016630 con 0x563bf1a69370 2024-09-15T22:12:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.372+0000 7f381ffff640 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:32.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.372+0000 7f3828c39740 1 -- 172.21.15.5:0/2333696016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3810003da0 con 0x563bf1a69370 2024-09-15T22:12:32.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.375+0000 7f37ff7fe640 1 -- 172.21.15.5:0/2333696016 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3818016830 con 0x563bf1a69370 2024-09-15T22:12:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.375+0000 7f3828c39740 10 client.25679 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.375+0000 7f3828c39740 1 -- 172.21.15.5:0/2333696016 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563bf1ae3160 con 0x563bf1a69370 2024-09-15T22:12:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.376+0000 7f37ff7fe640 1 -- 172.21.15.5:0/2333696016 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3810003da0 con 0x563bf1a69370 2024-09-15T22:12:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.376+0000 7f37ff7fe640 1 -- 172.21.15.5:0/2333696016 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3818023cd0 con 0x563bf1a69370 2024-09-15T22:12:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.376+0000 7f3828c39740 10 client.25679 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.376+0000 7f3828c39740 10 client.25679 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:12:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.376+0000 7f3828c39740 20 client.25679 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.376+0000 7f3828c39740 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563bf1ae38e0 0x563bf1ae5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.376+0000 7f3828c39740 4 client.25679 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:12:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.376+0000 7f3828c39740 1 -- 172.21.15.5:0/2333696016 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563bf19cbd70 con 0x563bf1ae38e0 2024-09-15T22:12:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.377+0000 7f381ffff640 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563bf1ae38e0 0x563bf1ae5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.379+0000 7f381ffff640 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563bf1ae38e0 0x563bf1ae5d00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe10b7847ec1b08b server_cookie=768c7b1da79db285 in_seq=0 out_seq=0 2024-09-15T22:12:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.379+0000 7f37ff7fe640 10 client.25679 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:12:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.379+0000 7f37ff7fe640 1 -- 172.21.15.5:0/2333696016 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563bf19cbd70 con 0x563bf1ae38e0 2024-09-15T22:12:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.379+0000 7f37ff7fe640 10 client.25679 handle_command_reply: tid=0 2024-09-15T22:12:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.379+0000 7f3828c39740 1 client.25679 shutdown 2024-09-15T22:12:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.382+0000 7f3828c39740 1 -- 172.21.15.5:0/2333696016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a69370 msgr2=0x563bf1a62890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.382+0000 7f3828c39740 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a69370 0x563bf1a62890 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3818018710 tx=0x7f3818009650 comp rx=0 tx=0).stop 2024-09-15T22:12:32.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.387+0000 7f3828c39740 1 -- 172.21.15.5:0/2333696016 shutdown_connections 2024-09-15T22:12:32.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.388+0000 7f3828c39740 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563bf1ae38e0 0x563bf1ae5d00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.388+0000 7f3828c39740 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563bf1a69370 0x563bf1a62890 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.388+0000 7f3828c39740 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563bf1a5fb40 0x563bf1a71920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.388+0000 7f37ff7fe640 0 client.25679 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:12:32.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.388+0000 7f3828c39740 1 --2- 172.21.15.5:0/2333696016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bf1a5e5d0 0x563bf1a713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.388+0000 7f3828c39740 1 -- 172.21.15.5:0/2333696016 >> 172.21.15.5:0/2333696016 conn(0x563bf1a27310 msgr2=0x563bf1a594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:32.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.388+0000 7f3828c39740 1 -- 172.21.15.5:0/2333696016 shutdown_connections 2024-09-15T22:12:32.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.388+0000 7f3828c39740 1 -- 172.21.15.5:0/2333696016 wait complete. 2024-09-15T22:12:32.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.389+0000 7f3828c39740 20 client.25679 trim_cache size 0 max 16384 2024-09-15T22:12:32.394 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:12:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.398+0000 7f3826860640 1 -- 172.21.15.5:0/51312481 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f380c0a34b0 msgr2=0x7f380c0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.398+0000 7f3826860640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f380c0a34b0 0x7f380c0a5970 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3818009fd0 tx=0x7f381800d040 comp rx=0 tx=0).stop 2024-09-15T22:12:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.398+0000 7f3826860640 1 -- 172.21.15.5:0/51312481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3820077040 msgr2=0x7f382019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.398+0000 7f3826860640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3820077040 0x7f382019ed60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3808004770 tx=0x7f3808003c10 comp rx=0 tx=0).stop 2024-09-15T22:12:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.398+0000 7f3826860640 1 -- 172.21.15.5:0/51312481 shutdown_connections 2024-09-15T22:12:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.398+0000 7f3826860640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f380c0a34b0 0x7f380c0a5970 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.398+0000 7f3826860640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f38201a3630 0x7f38201a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.398+0000 7f3826860640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3820112a30 0x7f382019f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.398+0000 7f3826860640 1 --2- 172.21.15.5:0/51312481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3820077040 0x7f382019ed60 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.398+0000 7f3826860640 1 -- 172.21.15.5:0/51312481 >> 172.21.15.5:0/51312481 conn(0x7f38200ffa70 msgr2=0x7f3820101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.400+0000 7f3826860640 1 -- 172.21.15.5:0/51312481 shutdown_connections 2024-09-15T22:12:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:32.400+0000 7f3826860640 1 -- 172.21.15.5:0/51312481 wait complete. 2024-09-15T22:12:33.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:33 smithi099 ceph-mon[34892]: pgmap v112: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.0 MiB/s wr, 810 op/s 2024-09-15T22:12:33.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:33 smithi129 ceph-mon[35314]: pgmap v112: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.0 MiB/s wr, 810 op/s 2024-09-15T22:12:33.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:33 smithi005 ceph-mon[30944]: pgmap v112: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 9.0 MiB/s wr, 810 op/s 2024-09-15T22:12:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 185.0 s, 1434.4 tps, lat 69.887 ms stddev 82.079 2024-09-15T22:12:34.997 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:12:35.431 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:35.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:35 smithi099 ceph-mon[34892]: pgmap v113: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-09-15T22:12:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:35 smithi129 ceph-mon[35314]: pgmap v113: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-09-15T22:12:35.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:35 smithi005 ceph-mon[30944]: pgmap v113: 129 pgs: 129 active+clean; 1.9 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.4 MiB/s wr, 551 op/s 2024-09-15T22:12:37.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:37 smithi005 ceph-mon[30944]: pgmap v114: 129 pgs: 129 active+clean; 2.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.4 MiB/s wr, 840 op/s 2024-09-15T22:12:37.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:37 smithi099 ceph-mon[34892]: pgmap v114: 129 pgs: 129 active+clean; 2.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.4 MiB/s wr, 840 op/s 2024-09-15T22:12:37.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:37 smithi129 ceph-mon[35314]: pgmap v114: 129 pgs: 129 active+clean; 2.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 9.4 MiB/s wr, 840 op/s 2024-09-15T22:12:39.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:39 smithi099 ceph-mon[34892]: pgmap v115: 129 pgs: 129 active+clean; 2.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.0 MiB/s wr, 583 op/s 2024-09-15T22:12:39.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:39 smithi129 ceph-mon[35314]: pgmap v115: 129 pgs: 129 active+clean; 2.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.0 MiB/s wr, 583 op/s 2024-09-15T22:12:39.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:39 smithi005 ceph-mon[30944]: pgmap v115: 129 pgs: 129 active+clean; 2.0 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 7.0 MiB/s wr, 583 op/s 2024-09-15T22:12:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 190.0 s, 1599.4 tps, lat 63.115 ms stddev 75.905 2024-09-15T22:12:39.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.967+0000 7fe01e394640 1 -- 172.21.15.5:0/2624809839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe018100e20 msgr2=0x7fe018101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.967+0000 7fe01e394640 1 --2- 172.21.15.5:0/2624809839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe018100e20 0x7fe018101200 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fe008009900 tx=0x7fe00802a480 comp rx=0 tx=0).stop 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.967+0000 7fe01e394640 1 -- 172.21.15.5:0/2624809839 shutdown_connections 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.967+0000 7fe01e394640 1 --2- 172.21.15.5:0/2624809839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018102ef0 0x7fe01810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.967+0000 7fe01e394640 1 --2- 172.21.15.5:0/2624809839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe018102570 0x7fe0181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.967+0000 7fe01e394640 1 --2- 172.21.15.5:0/2624809839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe018100e20 0x7fe018101200 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.967+0000 7fe01e394640 1 -- 172.21.15.5:0/2624809839 >> 172.21.15.5:0/2624809839 conn(0x7fe0180fc7d0 msgr2=0x7fe0180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.967+0000 7fe01e394640 1 -- 172.21.15.5:0/2624809839 shutdown_connections 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.967+0000 7fe01e394640 1 -- 172.21.15.5:0/2624809839 wait complete. 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.968+0000 7fe01e394640 1 Processor -- start 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.968+0000 7fe01e394640 1 -- start start 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe01e394640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018100e20 0x7fe0181a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe01e394640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe018102570 0x7fe0181a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe01e394640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe018102ef0 0x7fe01819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe01e394640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe018110fd0 con 0x7fe018102ef0 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe01e394640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe018110e50 con 0x7fe018100e20 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe01e394640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe018111150 con 0x7fe018102570 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe017fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018100e20 0x7fe0181a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe017fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018100e20 0x7fe0181a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33434/0 (socket says 172.21.15.5:33434) 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe017fff640 1 -- 172.21.15.5:0/1737423873 learned_addr learned my addr 172.21.15.5:0/1737423873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe01c931640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe018102ef0 0x7fe01819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe0177fe640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe018102570 0x7fe0181a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe017fff640 1 -- 172.21.15.5:0/1737423873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe018102570 msgr2=0x7fe0181a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe017fff640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe018102570 0x7fe0181a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe017fff640 1 -- 172.21.15.5:0/1737423873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe018102ef0 msgr2=0x7fe01819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:39.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe017fff640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe018102ef0 0x7fe01819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.969+0000 7fe017fff640 1 -- 172.21.15.5:0/1737423873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01819f440 con 0x7fe018100e20 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.970+0000 7fe017fff640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018100e20 0x7fe0181a4830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe00802aa70 tx=0x7fe008003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.970+0000 7fe0157fa640 1 -- 172.21.15.5:0/1737423873 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00803f070 con 0x7fe018100e20 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.970+0000 7fe0157fa640 1 -- 172.21.15.5:0/1737423873 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe008031c30 con 0x7fe018100e20 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.970+0000 7fe0157fa640 1 -- 172.21.15.5:0/1737423873 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008035030 con 0x7fe018100e20 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.971+0000 7fe01e394640 1 -- 172.21.15.5:0/1737423873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01819f6d0 con 0x7fe018100e20 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.971+0000 7fe01e394640 1 -- 172.21.15.5:0/1737423873 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01819faa0 con 0x7fe018100e20 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.974+0000 7fe0157fa640 1 -- 172.21.15.5:0/1737423873 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe008042070 con 0x7fe018100e20 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.975+0000 7fe0157fa640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0000a3500 0x7fe0000a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.975+0000 7fe0157fa640 1 -- 172.21.15.5:0/1737423873 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe0080f06f0 con 0x7fe018100e20 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.975+0000 7fe0177fe640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0000a3500 0x7fe0000a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.976+0000 7fe0177fe640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0000a3500 0x7fe0000a59c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0180ff020 tx=0x7fe00c005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:39.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.976+0000 7fe02076d740 1 Processor -- start 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.976+0000 7fe02076d740 1 -- start start 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe02076d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e2ab40 0x5625b0e29090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe02076d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 0x5625b0e29a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe02076d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e34370 0x5625b0e36760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe02076d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5625b0d8f2f0 con 0x5625b0e34370 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe02076d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5625b0d36cf0 con 0x5625b0e295d0 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe02076d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5625b0d8b2b0 con 0x5625b0e2ab40 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe01c931640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e34370 0x5625b0e36760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe0177fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 0x5625b0e29a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe01c931640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e34370 0x5625b0e36760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35262/0 (socket says 172.21.15.5:35262) 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe01c931640 1 -- 172.21.15.5:0/499927615 learned_addr learned my addr 172.21.15.5:0/499927615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe017fff640 1 --2- 172.21.15.5:0/499927615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e2ab40 0x5625b0e29090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe0177fe640 1 -- 172.21.15.5:0/499927615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e2ab40 msgr2=0x5625b0e29090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe0177fe640 1 --2- 172.21.15.5:0/499927615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e2ab40 0x5625b0e29090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe0177fe640 1 -- 172.21.15.5:0/499927615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e34370 msgr2=0x5625b0e36760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe0177fe640 1 --2- 172.21.15.5:0/499927615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e34370 0x5625b0e36760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe0177fe640 1 -- 172.21.15.5:0/499927615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b0d1b3a0 con 0x5625b0e295d0 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.977+0000 7fe0177fe640 1 --2- 172.21.15.5:0/499927615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 0x5625b0e29a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0117b0 tx=0x7fe00c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3af38fe6ad0b8d8b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.978+0000 7fdffe7fc640 1 -- 172.21.15.5:0/499927615 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c012e60 con 0x5625b0e295d0 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.978+0000 7fdffe7fc640 1 -- 172.21.15.5:0/499927615 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe00c01fb00 con 0x5625b0e295d0 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.978+0000 7fdffe7fc640 1 -- 172.21.15.5:0/499927615 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c01dba0 con 0x5625b0e295d0 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.978+0000 7fe02076d740 1 -- 172.21.15.5:0/499927615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 msgr2=0x5625b0e29a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.978+0000 7fe02076d740 1 --2- 172.21.15.5:0/499927615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 0x5625b0e29a10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0117b0 tx=0x7fe00c014000 comp rx=0 tx=0).stop 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.979+0000 7fe02076d740 1 -- 172.21.15.5:0/499927615 shutdown_connections 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.979+0000 7fe02076d740 1 --2- 172.21.15.5:0/499927615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e34370 0x5625b0e36760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.979+0000 7fe02076d740 1 --2- 172.21.15.5:0/499927615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 0x5625b0e29a10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.979+0000 7fe02076d740 1 --2- 172.21.15.5:0/499927615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e2ab40 0x5625b0e29090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.979+0000 7fe02076d740 1 -- 172.21.15.5:0/499927615 >> 172.21.15.5:0/499927615 conn(0x5625b0df2310 msgr2=0x5625b0e25c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.979+0000 7fe02076d740 1 -- 172.21.15.5:0/499927615 shutdown_connections 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.979+0000 7fe02076d740 1 -- 172.21.15.5:0/499927615 wait complete. 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.979+0000 7fe02076d740 1 -- start start 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.979+0000 7fe02076d740 1 Processor -- start 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe02076d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 0x5625b0e310f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe02076d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e2ab40 0x5625b0e31630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe017fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 0x5625b0e310f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe017fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 0x5625b0e310f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33454/0 (socket says 172.21.15.5:33454) 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe017fff640 1 -- 172.21.15.5:0/3323177904 learned_addr learned my addr 172.21.15.5:0/3323177904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe02076d740 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e31b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe02076d740 1 -- 172.21.15.5:0/3323177904 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5625b0e37070 con 0x5625b0e2ab40 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe02076d740 1 -- 172.21.15.5:0/3323177904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5625b0d4a060 con 0x5625b0e295d0 2024-09-15T22:12:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe02076d740 1 -- 172.21.15.5:0/3323177904 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5625b0e371f0 con 0x5625b0e34370 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe0177fe640 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e2ab40 0x5625b0e31630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe02076d740 10 client.0 _ll_register_callbacks cb 0x5625b0a9d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe017fff640 1 -- 172.21.15.5:0/3323177904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 msgr2=0x5625b0e31b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe017fff640 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e31b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe017fff640 1 -- 172.21.15.5:0/3323177904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e2ab40 msgr2=0x5625b0e31630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe017fff640 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e2ab40 0x5625b0e31630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.980+0000 7fe017fff640 1 -- 172.21.15.5:0/3323177904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b0b003c0 con 0x5625b0e295d0 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fe017fff640 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 0x5625b0e310f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe008000c00 tx=0x7fe0080bab40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f2548c2f5042926 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fe02076d740 1 -- 172.21.15.5:0/3323177904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe00c01fb00 con 0x5625b0e295d0 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fdff77fe640 10 client.25691 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fdff77fe640 1 -- 172.21.15.5:0/3323177904 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0080d5070 con 0x5625b0e295d0 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fdff77fe640 1 -- 172.21.15.5:0/3323177904 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe00803b460 con 0x5625b0e295d0 2024-09-15T22:12:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fdff77fe640 1 -- 172.21.15.5:0/3323177904 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe008049040 con 0x5625b0e295d0 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fe02076d740 10 client.25691 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fe02076d740 1 -- 172.21.15.5:0/3323177904 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5625b0e42a80 con 0x5625b0e295d0 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fdff77fe640 1 -- 172.21.15.5:0/3323177904 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe0080bc890 con 0x5625b0e295d0 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fe02076d740 10 client.25691 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fe02076d740 10 client.25691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fe02076d740 20 client.25691 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fe02076d740 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625b0e43f30 0x5625b0e463a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fe02076d740 4 client.25691 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.981+0000 7fe02076d740 1 -- 172.21.15.5:0/3323177904 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe00c01fb00 con 0x5625b0e43f30 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.982+0000 7fe0177fe640 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625b0e43f30 0x5625b0e463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.983+0000 7fe0177fe640 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625b0e43f30 0x5625b0e463a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=432c6ce6269277fc server_cookie=9cb0d0a045e0dc2a in_seq=0 out_seq=0 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.983+0000 7fdff77fe640 10 client.25691 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.984+0000 7fdff77fe640 1 -- 172.21.15.5:0/3323177904 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe00c01fb00 con 0x5625b0e43f30 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.984+0000 7fdff77fe640 10 client.25691 handle_command_reply: tid=0 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.984+0000 7fe02076d740 1 client.25691 shutdown 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fe02076d740 1 -- 172.21.15.5:0/3323177904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 msgr2=0x5625b0e310f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fe02076d740 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 0x5625b0e310f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe008000c00 tx=0x7fe0080bab40 comp rx=0 tx=0).stop 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fe02076d740 1 -- 172.21.15.5:0/3323177904 shutdown_connections 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fe02076d740 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e31b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fe02076d740 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e2ab40 0x5625b0e31630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fe02076d740 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625b0e43f30 0x5625b0e463a0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fe02076d740 1 --2- 172.21.15.5:0/3323177904 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e295d0 0x5625b0e310f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fe02076d740 1 -- 172.21.15.5:0/3323177904 >> 172.21.15.5:0/3323177904 conn(0x5625b0df2310 msgr2=0x5625b0e2b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fdff77fe640 0 client.25691 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fe02076d740 1 -- 172.21.15.5:0/3323177904 shutdown_connections 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fe02076d740 1 -- 172.21.15.5:0/3323177904 wait complete. 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:38.986+0000 7fe02076d740 20 client.25691 trim_cache size 0 max 16384 2024-09-15T22:12:39.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.006+0000 7fe02076d740 1 Processor -- start 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.006+0000 7fe02076d740 1 -- start start 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.006+0000 7fe02076d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 0x5625b0e2f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.006+0000 7fe02076d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 0x5625b0e2c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe02076d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e2c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe02076d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181b7ec0 con 0x5625b0e295d0 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe02076d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181b5790 con 0x5625b0e2ab40 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe02076d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181b35e0 con 0x5625b0e34370 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe017fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 0x5625b0e2f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe017fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 0x5625b0e2f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35282/0 (socket says 172.21.15.5:35282) 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe017fff640 1 -- 172.21.15.5:0/1036812169 learned_addr learned my addr 172.21.15.5:0/1036812169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe0177fe640 1 --2- 172.21.15.5:0/1036812169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 0x5625b0e2c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe01c931640 1 --2- 172.21.15.5:0/1036812169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e2c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe0177fe640 1 -- 172.21.15.5:0/1036812169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 msgr2=0x5625b0e2c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe0177fe640 1 --2- 172.21.15.5:0/1036812169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e2c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe0177fe640 1 -- 172.21.15.5:0/1036812169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 msgr2=0x5625b0e2f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe0177fe640 1 --2- 172.21.15.5:0/1036812169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 0x5625b0e2f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe0177fe640 1 -- 172.21.15.5:0/1036812169 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625b0e2a780 con 0x5625b0e2ab40 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.007+0000 7fe01c931640 1 --2- 172.21.15.5:0/1036812169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e2c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.008+0000 7fe0177fe640 1 --2- 172.21.15.5:0/1036812169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 0x5625b0e2c120 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009a20 tx=0x7fe00c01d700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf0a420ff8c819c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.008+0000 7fdffdffb640 1 -- 172.21.15.5:0/1036812169 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c01d3f0 con 0x5625b0e2ab40 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.008+0000 7fdffdffb640 1 -- 172.21.15.5:0/1036812169 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe00c0055a0 con 0x5625b0e2ab40 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.008+0000 7fdffdffb640 1 -- 172.21.15.5:0/1036812169 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c01d9f0 con 0x5625b0e2ab40 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.009+0000 7fe02076d740 1 -- 172.21.15.5:0/1036812169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 msgr2=0x5625b0e2c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.009+0000 7fe02076d740 1 --2- 172.21.15.5:0/1036812169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 0x5625b0e2c120 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe00c009a20 tx=0x7fe00c01d700 comp rx=0 tx=0).stop 2024-09-15T22:12:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.009+0000 7fe02076d740 1 -- 172.21.15.5:0/1036812169 shutdown_connections 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.009+0000 7fe02076d740 1 --2- 172.21.15.5:0/1036812169 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e2c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.009+0000 7fe02076d740 1 --2- 172.21.15.5:0/1036812169 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 0x5625b0e2c120 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.009+0000 7fe02076d740 1 --2- 172.21.15.5:0/1036812169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 0x5625b0e2f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.009+0000 7fe02076d740 1 -- 172.21.15.5:0/1036812169 >> 172.21.15.5:0/1036812169 conn(0x5625b0df2310 msgr2=0x5625b0e24d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.009+0000 7fe02076d740 1 -- 172.21.15.5:0/1036812169 shutdown_connections 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.009+0000 7fe02076d740 1 -- 172.21.15.5:0/1036812169 wait complete. 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.009+0000 7fe02076d740 1 -- start start 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.009+0000 7fe02076d740 1 Processor -- start 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.010+0000 7fe02076d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 0x5625b0e3ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.010+0000 7fe02076d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 0x5625b0e2dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.010+0000 7fe02076d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e2e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.010+0000 7fe017fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 0x5625b0e3ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.010+0000 7fe017fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 0x5625b0e3ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35292/0 (socket says 172.21.15.5:35292) 2024-09-15T22:12:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.010+0000 7fe0177fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 0x5625b0e2dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.010+0000 7fe0177fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 0x5625b0e2dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33482/0 (socket says 172.21.15.5:33482) 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe017fff640 1 -- 172.21.15.5:0/3080569015 learned_addr learned my addr 172.21.15.5:0/3080569015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe02076d740 1 -- 172.21.15.5:0/3080569015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181b3c50 con 0x5625b0e295d0 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe01c931640 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e2e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe02076d740 1 -- 172.21.15.5:0/3080569015 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181b50c0 con 0x5625b0e2ab40 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe02076d740 1 -- 172.21.15.5:0/3080569015 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181b4320 con 0x5625b0e34370 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe02076d740 10 client.0 _ll_register_callbacks cb 0x7fe0080bc400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe0177fe640 1 -- 172.21.15.5:0/3080569015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 msgr2=0x5625b0e2e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe0177fe640 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e2e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe0177fe640 1 -- 172.21.15.5:0/3080569015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 msgr2=0x5625b0e3ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe0177fe640 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 0x5625b0e3ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe0177fe640 1 -- 172.21.15.5:0/3080569015 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625b0e2e810 con 0x5625b0e2ab40 2024-09-15T22:12:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe02076d740 1 -- 172.21.15.5:0/3080569015 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe00c0055a0 con 0x5625b0e2ab40 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.011+0000 7fe017fff640 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 0x5625b0e3ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.012+0000 7fe0177fe640 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 0x5625b0e2dc90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5625b0e43eb0 tx=0x7fe00c011c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe14ff7ab0d1a151 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.012+0000 7fdff77fe640 10 client.25697 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.012+0000 7fdff77fe640 1 -- 172.21.15.5:0/3080569015 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c01d770 con 0x5625b0e2ab40 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.012+0000 7fdff77fe640 1 -- 172.21.15.5:0/3080569015 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe00c01f420 con 0x5625b0e2ab40 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.012+0000 7fdff77fe640 1 -- 172.21.15.5:0/3080569015 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe00c01f6a0 con 0x5625b0e2ab40 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.012+0000 7fe02076d740 10 client.25697 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.012+0000 7fe02076d740 1 -- 172.21.15.5:0/3080569015 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5625b0eae940 con 0x5625b0e2ab40 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.013+0000 7fdff77fe640 1 -- 172.21.15.5:0/3080569015 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe00c005080 con 0x5625b0e2ab40 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.013+0000 7fe02076d740 10 client.25697 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.013+0000 7fe02076d740 10 client.25697 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.013+0000 7fe02076d740 20 client.25697 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.013+0000 7fe02076d740 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625b0eaec20 0x5625b0eb1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.013+0000 7fe02076d740 4 client.25697 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.013+0000 7fe02076d740 1 -- 172.21.15.5:0/3080569015 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625b0d9c860 con 0x5625b0eaec20 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.014+0000 7fe017fff640 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625b0eaec20 0x5625b0eb1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.015+0000 7fe017fff640 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625b0eaec20 0x5625b0eb1010 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e0a172a58d90df server_cookie=79b07fddfe75553d in_seq=0 out_seq=0 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.015+0000 7fdff77fe640 10 client.25697 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.015+0000 7fdff77fe640 1 -- 172.21.15.5:0/3080569015 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625b0d9c860 con 0x5625b0eaec20 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.015+0000 7fdff77fe640 10 client.25697 handle_command_reply: tid=0 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.015+0000 7fe02076d740 1 client.25697 shutdown 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fe02076d740 1 -- 172.21.15.5:0/3080569015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 msgr2=0x5625b0e2dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fe02076d740 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 0x5625b0e2dc90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x5625b0e43eb0 tx=0x7fe00c011c40 comp rx=0 tx=0).stop 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fe02076d740 1 -- 172.21.15.5:0/3080569015 shutdown_connections 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fe02076d740 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625b0e34370 0x5625b0e2e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fe02076d740 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625b0eaec20 0x5625b0eb1010 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fe02076d740 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625b0e2ab40 0x5625b0e2dc90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fe02076d740 1 --2- 172.21.15.5:0/3080569015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625b0e295d0 0x5625b0e3ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fe02076d740 1 -- 172.21.15.5:0/3080569015 >> 172.21.15.5:0/3080569015 conn(0x5625b0df2310 msgr2=0x5625b0e244e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fdff77fe640 0 client.25697 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fe02076d740 1 -- 172.21.15.5:0/3080569015 shutdown_connections 2024-09-15T22:12:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fe02076d740 1 -- 172.21.15.5:0/3080569015 wait complete. 2024-09-15T22:12:39.913 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:12:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.017+0000 7fe02076d740 20 client.25697 trim_cache size 0 max 16384 2024-09-15T22:12:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.018+0000 7fe01e394640 1 -- 172.21.15.5:0/1737423873 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0000a3500 msgr2=0x7fe0000a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.018+0000 7fe01e394640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0000a3500 0x7fe0000a59c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe0180ff020 tx=0x7fe00c005f90 comp rx=0 tx=0).stop 2024-09-15T22:12:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.018+0000 7fe01e394640 1 -- 172.21.15.5:0/1737423873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018100e20 msgr2=0x7fe0181a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.018+0000 7fe01e394640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018100e20 0x7fe0181a4830 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe00802aa70 tx=0x7fe008003c10 comp rx=0 tx=0).stop 2024-09-15T22:12:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.018+0000 7fe01e394640 1 -- 172.21.15.5:0/1737423873 shutdown_connections 2024-09-15T22:12:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.018+0000 7fe01e394640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0000a3500 0x7fe0000a59c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.018+0000 7fe01e394640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe018102ef0 0x7fe01819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.018+0000 7fe01e394640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe018102570 0x7fe0181a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.018+0000 7fe01e394640 1 --2- 172.21.15.5:0/1737423873 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe018100e20 0x7fe0181a4830 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.018+0000 7fe01e394640 1 -- 172.21.15.5:0/1737423873 >> 172.21.15.5:0/1737423873 conn(0x7fe0180fc7d0 msgr2=0x7fe018069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:39.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.019+0000 7fe01e394640 1 -- 172.21.15.5:0/1737423873 shutdown_connections 2024-09-15T22:12:39.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:39.019+0000 7fe01e394640 1 -- 172.21.15.5:0/1737423873 wait complete. 2024-09-15T22:12:41.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:41 smithi129 ceph-mon[35314]: pgmap v116: 129 pgs: 129 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2024-09-15T22:12:41.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:41 smithi005 ceph-mon[30944]: pgmap v116: 129 pgs: 129 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2024-09-15T22:12:41.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:41 smithi099 ceph-mon[34892]: pgmap v116: 129 pgs: 129 active+clean; 2.0 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 7.4 MiB/s wr, 628 op/s 2024-09-15T22:12:42.668 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:12:42.668 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:12:43.152 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:43.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:43 smithi129 ceph-mon[35314]: pgmap v117: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.9 MiB/s wr, 790 op/s 2024-09-15T22:12:43.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:43 smithi005 ceph-mon[30944]: pgmap v117: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.9 MiB/s wr, 790 op/s 2024-09-15T22:12:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.763+0000 7ffbe0112640 1 -- 172.21.15.5:0/3902212548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd8109830 msgr2=0x7ffbd8109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.763+0000 7ffbe0112640 1 --2- 172.21.15.5:0/3902212548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd8109830 0x7ffbd8109c90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009900 tx=0x7ffbc802a400 comp rx=0 tx=0).stop 2024-09-15T22:12:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.764+0000 7ffbe0112640 1 -- 172.21.15.5:0/3902212548 shutdown_connections 2024-09-15T22:12:43.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.764+0000 7ffbe0112640 1 --2- 172.21.15.5:0/3902212548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd811b140 0x7ffbd811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.764+0000 7ffbe0112640 1 --2- 172.21.15.5:0/3902212548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd8109830 0x7ffbd8109c90 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009900 tx=0x7ffbc802a400 comp rx=0 tx=0).stop 2024-09-15T22:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.764+0000 7ffbe0112640 1 --2- 172.21.15.5:0/3902212548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8109450 0x7ffbd8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.764+0000 7ffbe0112640 1 -- 172.21.15.5:0/3902212548 >> 172.21.15.5:0/3902212548 conn(0x7ffbd806e140 msgr2=0x7ffbd806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:43.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.765+0000 7ffbe0112640 1 -- 172.21.15.5:0/3902212548 shutdown_connections 2024-09-15T22:12:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.766+0000 7ffbe0112640 1 -- 172.21.15.5:0/3902212548 wait complete. 2024-09-15T22:12:43.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.766+0000 7ffbe0112640 1 Processor -- start 2024-09-15T22:12:43.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.770+0000 7ffbe0112640 1 -- start start 2024-09-15T22:12:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbe0112640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8109450 0x7ffbd81a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbe0112640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd811b140 0x7ffbd81a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbe0112640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd81abdb0 0x7ffbd81ac260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbe0112640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8120e10 con 0x7ffbd811b140 2024-09-15T22:12:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbe0112640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8120c90 con 0x7ffbd81abdb0 2024-09-15T22:12:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbe0112640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8120f90 con 0x7ffbd8109450 2024-09-15T22:12:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbddeae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8109450 0x7ffbd81a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbddeae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8109450 0x7ffbd81a74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57826/0 (socket says 172.21.15.5:57826) 2024-09-15T22:12:43.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbddeae640 1 -- 172.21.15.5:0/1991849735 learned_addr learned my addr 172.21.15.5:0/1991849735 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbde6af640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd81abdb0 0x7ffbd81ac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbdd6ad640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd811b140 0x7ffbd81a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbddeae640 1 -- 172.21.15.5:0/1991849735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd81abdb0 msgr2=0x7ffbd81ac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.771+0000 7ffbddeae640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd81abdb0 0x7ffbd81ac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.772+0000 7ffbddeae640 1 -- 172.21.15.5:0/1991849735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd811b140 msgr2=0x7ffbd81a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.772+0000 7ffbddeae640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd811b140 0x7ffbd81a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.772+0000 7ffbddeae640 1 -- 172.21.15.5:0/1991849735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd81ac870 con 0x7ffbd8109450 2024-09-15T22:12:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.773+0000 7ffbddeae640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8109450 0x7ffbd81a74e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400e9f0 tx=0x7ffbd400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.774+0000 7ffbceffd640 1 -- 172.21.15.5:0/1991849735 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd400cd80 con 0x7ffbd8109450 2024-09-15T22:12:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.774+0000 7ffbceffd640 1 -- 172.21.15.5:0/1991849735 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd4010910 con 0x7ffbd8109450 2024-09-15T22:12:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.774+0000 7ffbceffd640 1 -- 172.21.15.5:0/1991849735 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd400cd80 con 0x7ffbd8109450 2024-09-15T22:12:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.774+0000 7ffbe0112640 1 -- 172.21.15.5:0/1991849735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd81acb60 con 0x7ffbd8109450 2024-09-15T22:12:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.774+0000 7ffbe0112640 1 -- 172.21.15.5:0/1991849735 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd81b4230 con 0x7ffbd8109450 2024-09-15T22:12:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.778+0000 7ffbe24eb740 1 Processor -- start 2024-09-15T22:12:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.780+0000 7ffbe24eb740 1 -- start start 2024-09-15T22:12:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.780+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da41af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.780+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da41b480 0x5599da4259e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.780+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da428310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.780+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5599da380280 con 0x5599da41b480 2024-09-15T22:12:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.780+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5599da327b10 con 0x5599da425f20 2024-09-15T22:12:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.780+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5599da37c240 con 0x5599da41ab60 2024-09-15T22:12:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.781+0000 7ffbde6af640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da41af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.781+0000 7ffbdd6ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da428310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.782+0000 7ffbdd6ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da428310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:58148/0 (socket says 172.21.15.5:58148) 2024-09-15T22:12:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.782+0000 7ffbdd6ad640 1 -- 172.21.15.5:0/3775253560 learned_addr learned my addr 172.21.15.5:0/3775253560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.783+0000 7ffbdd6ad640 1 -- 172.21.15.5:0/3775253560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 msgr2=0x5599da41af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.783+0000 7ffbdd6ad640 1 --2- 172.21.15.5:0/3775253560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da41af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.783+0000 7ffbdd6ad640 1 -- 172.21.15.5:0/3775253560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da41b480 msgr2=0x5599da4259e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:12:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.783+0000 7ffbdd6ad640 1 --2- 172.21.15.5:0/3775253560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da41b480 0x5599da4259e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.783+0000 7ffbdd6ad640 1 -- 172.21.15.5:0/3775253560 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599da3341f0 con 0x5599da425f20 2024-09-15T22:12:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.783+0000 7ffbdd6ad640 1 --2- 172.21.15.5:0/3775253560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da428310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009900 tx=0x7ffbc8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecf06ed6863b92ae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.784+0000 7ffbe0112640 1 -- 172.21.15.5:0/3775253560 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc803f070 con 0x5599da425f20 2024-09-15T22:12:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.784+0000 7ffbe0112640 1 -- 172.21.15.5:0/3775253560 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc8004430 con 0x5599da425f20 2024-09-15T22:12:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.784+0000 7ffbe0112640 1 -- 172.21.15.5:0/3775253560 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8035070 con 0x5599da425f20 2024-09-15T22:12:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.785+0000 7ffbe24eb740 1 -- 172.21.15.5:0/3775253560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 msgr2=0x5599da428310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.785+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/3775253560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da428310 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009900 tx=0x7ffbc8004040 comp rx=0 tx=0).stop 2024-09-15T22:12:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.786+0000 7ffbe24eb740 1 -- 172.21.15.5:0/3775253560 shutdown_connections 2024-09-15T22:12:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.786+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/3775253560 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da428310 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.786+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/3775253560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da41b480 0x5599da4259e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.786+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/3775253560 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da41af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.786+0000 7ffbe24eb740 1 -- 172.21.15.5:0/3775253560 >> 172.21.15.5:0/3775253560 conn(0x5599da4172d0 msgr2=0x5599da4176b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.786+0000 7ffbceffd640 1 -- 172.21.15.5:0/1991849735 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ffbd4010ac0 con 0x7ffbd8109450 2024-09-15T22:12:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.786+0000 7ffbe24eb740 1 -- 172.21.15.5:0/3775253560 shutdown_connections 2024-09-15T22:12:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.786+0000 7ffbe24eb740 1 -- 172.21.15.5:0/3775253560 wait complete. 2024-09-15T22:12:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.787+0000 7ffbe24eb740 1 -- start start 2024-09-15T22:12:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.787+0000 7ffbe24eb740 1 Processor -- start 2024-09-15T22:12:43.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.787+0000 7ffbceffd640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ffbc40a36c0 0x7ffbc40a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.788+0000 7ffbceffd640 1 -- 172.21.15.5:0/1991849735 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7ffbd400cb80 con 0x7ffbd8109450 2024-09-15T22:12:43.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.788+0000 7ffbde6af640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ffbc40a36c0 0x7ffbc40a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.788+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da41ab60 0x5599da422f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.788+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da425f20 0x5599da4234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.788+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 0x5599da431d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.788+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5599da4288b0 con 0x5599da423a10 2024-09-15T22:12:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.788+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5599da33af10 con 0x5599da41ab60 2024-09-15T22:12:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.788+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5599da428a30 con 0x5599da425f20 2024-09-15T22:12:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.788+0000 7ffbe24eb740 10 client.0 _ll_register_callbacks cb 0x5599da303a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.788+0000 7ffbdd6ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da425f20 0x5599da4234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.788+0000 7ffbdd6ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da425f20 0x5599da4234d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57848/0 (socket says 172.21.15.5:57848) 2024-09-15T22:12:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.789+0000 7ffbdd6ad640 1 -- 172.21.15.5:0/2706017657 learned_addr learned my addr 172.21.15.5:0/2706017657 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:43.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.789+0000 7ffbddeae640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da41ab60 0x5599da422f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.789+0000 7ffbdd6ad640 1 -- 172.21.15.5:0/2706017657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da41ab60 msgr2=0x5599da422f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:43.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.789+0000 7ffbdd6ad640 1 --2- 172.21.15.5:0/2706017657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da41ab60 0x5599da422f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.789+0000 7ffbdd6ad640 1 -- 172.21.15.5:0/2706017657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 msgr2=0x5599da431d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.789+0000 7ffbdd6ad640 1 --2- 172.21.15.5:0/2706017657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 0x5599da431d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.789+0000 7ffbdd6ad640 1 -- 172.21.15.5:0/2706017657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599da0f16a0 con 0x5599da425f20 2024-09-15T22:12:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.789+0000 7ffbdd6ad640 1 --2- 172.21.15.5:0/2706017657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da425f20 0x5599da4234d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8000c00 tx=0x7ffbc80317c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77473c5dee7d4fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.790+0000 7ffbc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:12:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.790+0000 7ffbde6af640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ffbc40a36c0 0x7ffbc40a5b80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0013410 tx=0x7ffbd0001d40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.790+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2706017657 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc804d060 con 0x5599da425f20 2024-09-15T22:12:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.790+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2706017657 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7ffbc8031cc0 con 0x5599da425f20 2024-09-15T22:12:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.790+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2706017657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc8004430 con 0x5599da425f20 2024-09-15T22:12:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.790+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2706017657 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffbc8054050 con 0x5599da425f20 2024-09-15T22:12:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.790+0000 7ffbe24eb740 10 client.25693 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:43.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.790+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2706017657 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599da4288b0 con 0x5599da425f20 2024-09-15T22:12:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.791+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2706017657 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc8004430 con 0x5599da425f20 2024-09-15T22:12:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.791+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2706017657 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ffbc8005080 con 0x5599da425f20 2024-09-15T22:12:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.791+0000 7ffbe24eb740 10 client.25693 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.791+0000 7ffbe24eb740 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.791+0000 7ffbe24eb740 20 client.25693 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.791+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2706017657 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5599da435f60 0x5599da438380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.791+0000 7ffbe24eb740 4 client.25693 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.791+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2706017657 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599da38d7f0 con 0x5599da435f60 2024-09-15T22:12:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.793+0000 7ffbddeae640 1 --2- 172.21.15.5:0/2706017657 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5599da435f60 0x5599da438380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.793+0000 7ffbddeae640 1 --2- 172.21.15.5:0/2706017657 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5599da435f60 0x5599da438380 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c8cf5b2027ad30b server_cookie=31f5b9deaf254c5c in_seq=0 out_seq=0 2024-09-15T22:12:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.793+0000 7ffbc17fa640 10 client.25693 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.794+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2706017657 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599da38d7f0 con 0x5599da435f60 2024-09-15T22:12:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.794+0000 7ffbc17fa640 10 client.25693 handle_command_reply: tid=0 2024-09-15T22:12:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.794+0000 7ffbe24eb740 1 client.25693 shutdown 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.794+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2706017657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da425f20 msgr2=0x5599da4234d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.794+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2706017657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da425f20 0x5599da4234d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8000c00 tx=0x7ffbc80317c0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.795+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2706017657 shutdown_connections 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.795+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2706017657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 0x5599da431d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.795+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2706017657 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da425f20 0x5599da4234d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.795+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2706017657 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5599da435f60 0x5599da438380 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.795+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2706017657 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da41ab60 0x5599da422f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.795+0000 7ffbc17fa640 0 client.25693 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.795+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2706017657 >> 172.21.15.5:0/2706017657 conn(0x5599da4172d0 msgr2=0x5599da41cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.795+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2706017657 shutdown_connections 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.795+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2706017657 wait complete. 2024-09-15T22:12:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.795+0000 7ffbe24eb740 20 client.25693 trim_cache size 0 max 16384 2024-09-15T22:12:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.825+0000 7ffbe24eb740 1 Processor -- start 2024-09-15T22:12:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.825+0000 7ffbe24eb740 1 -- start start 2024-09-15T22:12:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.825+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da431b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.825+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 0x5599da43dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.825+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da43e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.825+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81c0ce0 con 0x5599da423a10 2024-09-15T22:12:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.825+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81be5b0 con 0x5599da425f20 2024-09-15T22:12:43.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.825+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81bc420 con 0x5599da41ab60 2024-09-15T22:12:43.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.826+0000 7ffbde6af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da43e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.826+0000 7ffbde6af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da43e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:58166/0 (socket says 172.21.15.5:58166) 2024-09-15T22:12:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.826+0000 7ffbde6af640 1 -- 172.21.15.5:0/2730043063 learned_addr learned my addr 172.21.15.5:0/2730043063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.826+0000 7ffbddeae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da431b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.826+0000 7ffbde6af640 1 -- 172.21.15.5:0/2730043063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 msgr2=0x5599da431b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.826+0000 7ffbde6af640 1 --2- 172.21.15.5:0/2730043063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da431b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.826+0000 7ffbde6af640 1 -- 172.21.15.5:0/2730043063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 msgr2=0x5599da43dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.826+0000 7ffbde6af640 1 --2- 172.21.15.5:0/2730043063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 0x5599da43dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.826+0000 7ffbde6af640 1 -- 172.21.15.5:0/2730043063 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599da4389e0 con 0x5599da425f20 2024-09-15T22:12:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.828+0000 7ffbde6af640 1 --2- 172.21.15.5:0/2730043063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da43e150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0001c90 tx=0x7ffbd0015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51373392b95a39c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.828+0000 7ffbc37fe640 1 -- 172.21.15.5:0/2730043063 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd001e050 con 0x5599da425f20 2024-09-15T22:12:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.828+0000 7ffbc37fe640 1 -- 172.21.15.5:0/2730043063 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd0028070 con 0x5599da425f20 2024-09-15T22:12:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.828+0000 7ffbc37fe640 1 -- 172.21.15.5:0/2730043063 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0023ae0 con 0x5599da425f20 2024-09-15T22:12:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.829+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2730043063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 msgr2=0x5599da43e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.829+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2730043063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da43e150 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0001c90 tx=0x7ffbd0015000 comp rx=0 tx=0).stop 2024-09-15T22:12:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.829+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2730043063 shutdown_connections 2024-09-15T22:12:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.829+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2730043063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da43e150 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.829+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2730043063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 0x5599da43dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.829+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2730043063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da431b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.829+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2730043063 >> 172.21.15.5:0/2730043063 conn(0x5599da4172d0 msgr2=0x5599da425010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.830+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2730043063 shutdown_connections 2024-09-15T22:12:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.830+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2730043063 wait complete. 2024-09-15T22:12:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.830+0000 7ffbe24eb740 1 -- start start 2024-09-15T22:12:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.830+0000 7ffbe24eb740 1 Processor -- start 2024-09-15T22:12:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da42e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 0x5599da4297e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbe24eb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da429da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81bca70 con 0x5599da423a10 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81bdee0 con 0x5599da425f20 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbe24eb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81bd140 con 0x5599da41ab60 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbe24eb740 10 client.0 _ll_register_callbacks cb 0x7ffbd8110870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbddeae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da42e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbddeae640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da42e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57866/0 (socket says 172.21.15.5:57866) 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbddeae640 1 -- 172.21.15.5:0/2056461331 learned_addr learned my addr 172.21.15.5:0/2056461331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbde6af640 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da429da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbddeae640 1 -- 172.21.15.5:0/2056461331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 msgr2=0x5599da429da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbddeae640 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da429da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbddeae640 1 -- 172.21.15.5:0/2056461331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 msgr2=0x5599da4297e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbddeae640 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 0x5599da4297e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.831+0000 7ffbddeae640 1 -- 172.21.15.5:0/2056461331 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599da42a620 con 0x5599da41ab60 2024-09-15T22:12:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.832+0000 7ffbddeae640 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da42e4b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffbd409bbe0 tx=0x7ffbd409bb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f5b4fe56a49434f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.832+0000 7ffbc17fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:12:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.832+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2056461331 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd400cd80 con 0x5599da41ab60 2024-09-15T22:12:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.832+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2056461331 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7ffbd40cd5d0 con 0x5599da41ab60 2024-09-15T22:12:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.832+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2056461331 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd0028070 con 0x5599da41ab60 2024-09-15T22:12:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.832+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2056461331 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffbd4098550 con 0x5599da41ab60 2024-09-15T22:12:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.832+0000 7ffbe24eb740 10 client.25699 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.832+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2056461331 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5599da4a0d50 con 0x5599da41ab60 2024-09-15T22:12:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.833+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2056461331 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd0028070 con 0x5599da41ab60 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.833+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2056461331 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ffbd4010430 con 0x5599da41ab60 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.833+0000 7ffbe24eb740 10 client.25699 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.833+0000 7ffbe24eb740 10 client.25699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.833+0000 7ffbe24eb740 20 client.25699 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.833+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5599da4a1400 0x5599da4a3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.833+0000 7ffbe24eb740 4 client.25699 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.833+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2056461331 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5599da387d00 con 0x5599da4a1400 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.833+0000 7ffbdd6ad640 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5599da4a1400 0x5599da4a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.834+0000 7ffbc17fa640 10 client.25699 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.834+0000 7ffbdd6ad640 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5599da4a1400 0x5599da4a3840 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5a50d39d7b5e43 server_cookie=46b1349f9d2823b7 in_seq=0 out_seq=0 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbc17fa640 1 -- 172.21.15.5:0/2056461331 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5599da387d00 con 0x5599da4a1400 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbc17fa640 10 client.25699 handle_command_reply: tid=0 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbe24eb740 1 client.25699 shutdown 2024-09-15T22:12:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2056461331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 msgr2=0x5599da42e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da42e4b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffbd409bbe0 tx=0x7ffbd409bb00 comp rx=0 tx=0).stop 2024-09-15T22:12:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2056461331 shutdown_connections 2024-09-15T22:12:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5599da4a1400 0x5599da4a3840 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5599da425f20 0x5599da429da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5599da423a10 0x5599da4297e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbe24eb740 1 --2- 172.21.15.5:0/2056461331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5599da41ab60 0x5599da42e4b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbc17fa640 0 client.25699 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.835+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2056461331 >> 172.21.15.5:0/2056461331 conn(0x5599da4172d0 msgr2=0x5599da415ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.836+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2056461331 shutdown_connections 2024-09-15T22:12:43.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:43 smithi099 ceph-mon[34892]: pgmap v117: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.9 MiB/s wr, 790 op/s 2024-09-15T22:12:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.836+0000 7ffbe24eb740 1 -- 172.21.15.5:0/2056461331 wait complete. 2024-09-15T22:12:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.836+0000 7ffbe24eb740 20 client.25699 trim_cache size 0 max 16384 2024-09-15T22:12:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:12:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:12:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "bf07ca55-f08b-4a2e-8f2e-8ef10801084b", 2024-09-15T22:12:43.842 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:12:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:12:43.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.838+0000 7ffbccff9640 1 -- 172.21.15.5:0/1991849735 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ffbc40a36c0 msgr2=0x7ffbc40a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.838+0000 7ffbccff9640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ffbc40a36c0 0x7ffbc40a5b80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0013410 tx=0x7ffbd0001d40 comp rx=0 tx=0).stop 2024-09-15T22:12:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.838+0000 7ffbccff9640 1 -- 172.21.15.5:0/1991849735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8109450 msgr2=0x7ffbd81a74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.838+0000 7ffbccff9640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8109450 0x7ffbd81a74e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffbd400e9f0 tx=0x7ffbd400eeb0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.838+0000 7ffbccff9640 1 -- 172.21.15.5:0/1991849735 shutdown_connections 2024-09-15T22:12:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.838+0000 7ffbccff9640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ffbc40a36c0 0x7ffbc40a5b80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.838+0000 7ffbccff9640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffbd81abdb0 0x7ffbd81ac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.838+0000 7ffbccff9640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd811b140 0x7ffbd81a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.838+0000 7ffbccff9640 1 --2- 172.21.15.5:0/1991849735 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffbd8109450 0x7ffbd81a74e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.838+0000 7ffbccff9640 1 -- 172.21.15.5:0/1991849735 >> 172.21.15.5:0/1991849735 conn(0x7ffbd806e140 msgr2=0x7ffbd811c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.839+0000 7ffbccff9640 1 -- 172.21.15.5:0/1991849735 shutdown_connections 2024-09-15T22:12:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:43.839+0000 7ffbccff9640 1 -- 172.21.15.5:0/1991849735 wait complete. 2024-09-15T22:12:44.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:44 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:12:44.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:44 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:44.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:44 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:12:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 195.0 s, 2314.0 tps, lat 43.085 ms stddev 74.633 2024-09-15T22:12:44.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:44 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:12:44.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:44 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:44.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:44 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:12:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:44 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:12:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:44 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:44.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:44 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:12:45.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:45 smithi099 ceph-mon[34892]: pgmap v118: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.2 MiB/s wr, 539 op/s 2024-09-15T22:12:45.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:45 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:45 smithi129 ceph-mon[35314]: pgmap v118: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.2 MiB/s wr, 539 op/s 2024-09-15T22:12:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:45 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:45.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:45 smithi005 ceph-mon[30944]: pgmap v118: 129 pgs: 129 active+clean; 2.0 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 5.2 MiB/s wr, 539 op/s 2024-09-15T22:12:45.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:45 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:12:46.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:12:46.945 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.551+0000 7fd18a642640 1 -- 172.21.15.5:0/3430735811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd18411b140 msgr2=0x7fd18411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.551+0000 7fd18a642640 1 --2- 172.21.15.5:0/3430735811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd18411b140 0x7fd18411d530 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd174009900 tx=0x7fd17402a3f0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.552+0000 7fd18a642640 1 -- 172.21.15.5:0/3430735811 shutdown_connections 2024-09-15T22:12:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.552+0000 7fd18a642640 1 --2- 172.21.15.5:0/3430735811 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd18411b140 0x7fd18411d530 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.552+0000 7fd18a642640 1 --2- 172.21.15.5:0/3430735811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd184109830 0x7fd184109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.552+0000 7fd18a642640 1 --2- 172.21.15.5:0/3430735811 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd184109450 0x7fd184110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.552+0000 7fd18a642640 1 -- 172.21.15.5:0/3430735811 >> 172.21.15.5:0/3430735811 conn(0x7fd18406e140 msgr2=0x7fd18406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.555+0000 7fd18a642640 1 -- 172.21.15.5:0/3430735811 shutdown_connections 2024-09-15T22:12:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.557+0000 7fd18a642640 1 -- 172.21.15.5:0/3430735811 wait complete. 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.558+0000 7fd18a642640 1 Processor -- start 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.558+0000 7fd18a642640 1 -- start start 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.559+0000 7fd18a642640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd184109450 0x7fd1841a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.559+0000 7fd18a642640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184109830 0x7fd1841a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.559+0000 7fd18a642640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18411b140 0x7fd1841aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.559+0000 7fd18a642640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd184120df0 con 0x7fd184109450 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.559+0000 7fd18a642640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd184120c70 con 0x7fd184109830 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.559+0000 7fd18a642640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd184120f70 con 0x7fd18411b140 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.559+0000 7fd1837fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184109830 0x7fd1841a76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.559+0000 7fd1837fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184109830 0x7fd1841a76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:58184/0 (socket says 172.21.15.5:58184) 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.559+0000 7fd1837fe640 1 -- 172.21.15.5:0/3274987891 learned_addr learned my addr 172.21.15.5:0/3274987891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.559+0000 7fd188bdf640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18411b140 0x7fd1841aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.560+0000 7fd1837fe640 1 -- 172.21.15.5:0/3274987891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18411b140 msgr2=0x7fd1841aba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.560+0000 7fd1837fe640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18411b140 0x7fd1841aba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.560+0000 7fd1837fe640 1 -- 172.21.15.5:0/3274987891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd184109450 msgr2=0x7fd1841a7190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:12:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.560+0000 7fd1837fe640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd184109450 0x7fd1841a7190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.560+0000 7fd1837fe640 1 -- 172.21.15.5:0/3274987891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1841ac140 con 0x7fd184109830 2024-09-15T22:12:47.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.562+0000 7fd1837fe640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184109830 0x7fd1841a76d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd17800e9f0 tx=0x7fd17800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.562+0000 7fd1817fa640 1 -- 172.21.15.5:0/3274987891 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17800cd80 con 0x7fd184109830 2024-09-15T22:12:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.562+0000 7fd1817fa640 1 -- 172.21.15.5:0/3274987891 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd178010910 con 0x7fd184109830 2024-09-15T22:12:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.562+0000 7fd1817fa640 1 -- 172.21.15.5:0/3274987891 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178018980 con 0x7fd184109830 2024-09-15T22:12:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.563+0000 7fd18a642640 1 -- 172.21.15.5:0/3274987891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1841ac430 con 0x7fd184109830 2024-09-15T22:12:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.563+0000 7fd18a642640 1 -- 172.21.15.5:0/3274987891 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1841b3d10 con 0x7fd184109830 2024-09-15T22:12:47.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.567+0000 7fd18ca1b740 1 Processor -- start 2024-09-15T22:12:47.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.567+0000 7fd18ca1b740 1 -- start start 2024-09-15T22:12:47.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.568+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befaf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.568+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 0x55e4befba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.568+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befba680 0x55e4befbca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.568+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4bef152e0 con 0x55e4befba680 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.568+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e4beebcce0 con 0x55e4befaf4a0 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.568+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e4bef112a0 con 0x55e4befafdc0 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.568+0000 7fd188bdf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befaf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.568+0000 7fd183fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 0x55e4befba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.568+0000 7fd183fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 0x55e4befba140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57892/0 (socket says 172.21.15.5:57892) 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.568+0000 7fd183fff640 1 -- 172.21.15.5:0/1010551231 learned_addr learned my addr 172.21.15.5:0/1010551231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.569+0000 7fd183fff640 1 -- 172.21.15.5:0/1010551231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 msgr2=0x55e4befaf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.569+0000 7fd183fff640 1 --2- 172.21.15.5:0/1010551231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befaf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.569+0000 7fd183fff640 1 -- 172.21.15.5:0/1010551231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befba680 msgr2=0x55e4befbca70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.569+0000 7fd183fff640 1 --2- 172.21.15.5:0/1010551231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befba680 0x55e4befbca70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.569+0000 7fd183fff640 1 -- 172.21.15.5:0/1010551231 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4beea1440 con 0x55e4befafdc0 2024-09-15T22:12:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.569+0000 7fd1817fa640 1 -- 172.21.15.5:0/3274987891 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd178010ac0 con 0x7fd184109830 2024-09-15T22:12:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.569+0000 7fd183fff640 1 --2- 172.21.15.5:0/1010551231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 0x55e4befba140 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009d80 tx=0x7fd17c00e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=415ced5572506336 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.570+0000 7fd16a7fc640 1 -- 172.21.15.5:0/1010551231 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c016070 con 0x55e4befafdc0 2024-09-15T22:12:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.570+0000 7fd16a7fc640 1 -- 172.21.15.5:0/1010551231 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c005020 con 0x55e4befafdc0 2024-09-15T22:12:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.570+0000 7fd16a7fc640 1 -- 172.21.15.5:0/1010551231 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0116b0 con 0x55e4befafdc0 2024-09-15T22:12:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.570+0000 7fd1817fa640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd16c0a3540 0x7fd16c0a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.570+0000 7fd1817fa640 1 -- 172.21.15.5:0/3274987891 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fd1780d03c0 con 0x7fd184109830 2024-09-15T22:12:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.570+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1010551231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 msgr2=0x55e4befba140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.570+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/1010551231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 0x55e4befba140 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009d80 tx=0x7fd17c00e6c0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.571+0000 7fd188bdf640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd16c0a3540 0x7fd16c0a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.571+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1010551231 shutdown_connections 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.571+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/1010551231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befba680 0x55e4befbca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.571+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/1010551231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 0x55e4befba140 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.571+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/1010551231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befaf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.571+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1010551231 >> 172.21.15.5:0/1010551231 conn(0x55e4befaba30 msgr2=0x55e4befabe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.571+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1010551231 shutdown_connections 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.571+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1010551231 wait complete. 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.571+0000 7fd188bdf640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd16c0a3540 0x7fd16c0a5a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd17402a960 tx=0x7fd17402a8f0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd18ca1b740 1 -- start start 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd18ca1b740 1 Processor -- start 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befb9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 0x55e4befb71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befba680 0x55e4befb7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4befbd2f0 con 0x55e4befba680 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e4beed0050 con 0x55e4befaf4a0 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e4befbd470 con 0x55e4befafdc0 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd18ca1b740 10 client.0 _ll_register_callbacks cb 0x55e4bef10160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd183fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befb9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd183fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befb9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:58208/0 (socket says 172.21.15.5:58208) 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.572+0000 7fd183fff640 1 -- 172.21.15.5:0/2111259486 learned_addr learned my addr 172.21.15.5:0/2111259486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.573+0000 7fd188bdf640 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befba680 0x55e4befb7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.573+0000 7fd1837fe640 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 0x55e4befb71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.573+0000 7fd183fff640 1 -- 172.21.15.5:0/2111259486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 msgr2=0x55e4befb71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.573+0000 7fd183fff640 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 0x55e4befb71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.573+0000 7fd183fff640 1 -- 172.21.15.5:0/2111259486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befba680 msgr2=0x55e4befb7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.573+0000 7fd183fff640 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befba680 0x55e4befb7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.573+0000 7fd183fff640 1 -- 172.21.15.5:0/2111259486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4bec863c0 con 0x55e4befaf4a0 2024-09-15T22:12:47.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.573+0000 7fd183fff640 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befb9f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009cd0 tx=0x7fd17c004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=967ce50fe953cc88 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.573+0000 7fd1637fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:12:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.573+0000 7fd1637fe640 1 -- 172.21.15.5:0/2111259486 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c027060 con 0x55e4befaf4a0 2024-09-15T22:12:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.574+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2111259486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd17c005020 con 0x55e4befaf4a0 2024-09-15T22:12:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.574+0000 7fd1637fe640 1 -- 172.21.15.5:0/2111259486 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fd17c003970 con 0x55e4befaf4a0 2024-09-15T22:12:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.574+0000 7fd1637fe640 1 -- 172.21.15.5:0/2111259486 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd17c016070 con 0x55e4befaf4a0 2024-09-15T22:12:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.574+0000 7fd18ca1b740 10 client.25727 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.574+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2111259486 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4befc8ca0 con 0x55e4befaf4a0 2024-09-15T22:12:47.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.574+0000 7fd1637fe640 1 -- 172.21.15.5:0/2111259486 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c005020 con 0x55e4befaf4a0 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.574+0000 7fd1637fe640 1 -- 172.21.15.5:0/2111259486 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd17c00f040 con 0x55e4befaf4a0 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.575+0000 7fd18ca1b740 10 client.25727 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.575+0000 7fd18ca1b740 10 client.25727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.575+0000 7fd18ca1b740 20 client.25727 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.575+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e4befca0b0 0x55e4befcc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.575+0000 7fd18ca1b740 4 client.25727 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.575+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2111259486 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4bef22850 con 0x55e4befca0b0 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.575+0000 7fd1837fe640 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e4befca0b0 0x55e4befcc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.576+0000 7fd1637fe640 10 client.25727 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.577+0000 7fd1837fe640 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e4befca0b0 0x55e4befcc4a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275ef5c4793743fc server_cookie=606ef72b28901e97 in_seq=0 out_seq=0 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd1637fe640 1 -- 172.21.15.5:0/2111259486 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4bef22850 con 0x55e4befca0b0 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd1637fe640 10 client.25727 handle_command_reply: tid=0 2024-09-15T22:12:47.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 1 client.25727 shutdown 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2111259486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 msgr2=0x55e4befb9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befb9f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009cd0 tx=0x7fd17c004c70 comp rx=0 tx=0).stop 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2111259486 shutdown_connections 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befba680 0x55e4befb7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befafdc0 0x55e4befb71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e4befca0b0 0x55e4befcc4a0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/2111259486 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befb9f30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2111259486 >> 172.21.15.5:0/2111259486 conn(0x55e4befaba30 msgr2=0x55e4befb1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd1637fe640 0 client.25727 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2111259486 shutdown_connections 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2111259486 wait complete. 2024-09-15T22:12:47.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.578+0000 7fd18ca1b740 20 client.25727 trim_cache size 0 max 16384 2024-09-15T22:12:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.605+0000 7fd18ca1b740 1 Processor -- start 2024-09-15T22:12:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.606+0000 7fd18ca1b740 1 -- start start 2024-09-15T22:12:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.606+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befc6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.606+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befafdc0 0x55e4befb7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.606+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befba680 0x55e4befb61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.606+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841c0710 con 0x55e4befafdc0 2024-09-15T22:12:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.606+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841bdfe0 con 0x55e4befaf4a0 2024-09-15T22:12:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.606+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841bbe30 con 0x55e4befba680 2024-09-15T22:12:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.606+0000 7fd183fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befc6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.606+0000 7fd183fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befc6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:58212/0 (socket says 172.21.15.5:58212) 2024-09-15T22:12:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.606+0000 7fd183fff640 1 -- 172.21.15.5:0/2053030345 learned_addr learned my addr 172.21.15.5:0/2053030345 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.607+0000 7fd183fff640 1 -- 172.21.15.5:0/2053030345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befba680 msgr2=0x55e4befb61c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:12:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.607+0000 7fd183fff640 1 --2- 172.21.15.5:0/2053030345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befba680 0x55e4befb61c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.607+0000 7fd183fff640 1 -- 172.21.15.5:0/2053030345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befafdc0 msgr2=0x55e4befb7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.607+0000 7fd183fff640 1 --2- 172.21.15.5:0/2053030345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befafdc0 0x55e4befb7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.607+0000 7fd183fff640 1 -- 172.21.15.5:0/2053030345 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4befbcdb0 con 0x55e4befaf4a0 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.607+0000 7fd183fff640 1 --2- 172.21.15.5:0/2053030345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befc6580 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009700 tx=0x7fd17c0056a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0df36ab3982c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.607+0000 7fd169ffb640 1 -- 172.21.15.5:0/2053030345 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c037070 con 0x55e4befaf4a0 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.608+0000 7fd169ffb640 1 -- 172.21.15.5:0/2053030345 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c0099a0 con 0x55e4befaf4a0 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.608+0000 7fd169ffb640 1 -- 172.21.15.5:0/2053030345 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c005be0 con 0x55e4befaf4a0 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.609+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2053030345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 msgr2=0x55e4befc6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.609+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/2053030345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befc6580 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009700 tx=0x7fd17c0056a0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.609+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2053030345 shutdown_connections 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.609+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/2053030345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befba680 0x55e4befb61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.609+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/2053030345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befafdc0 0x55e4befb7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.609+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/2053030345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befaf4a0 0x55e4befc6580 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.609+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2053030345 >> 172.21.15.5:0/2053030345 conn(0x55e4befaba30 msgr2=0x55e4befb5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.609+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2053030345 shutdown_connections 2024-09-15T22:12:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.609+0000 7fd18ca1b740 1 -- 172.21.15.5:0/2053030345 wait complete. 2024-09-15T22:12:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.609+0000 7fd18ca1b740 1 -- start start 2024-09-15T22:12:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.609+0000 7fd18ca1b740 1 Processor -- start 2024-09-15T22:12:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.610+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befaf4a0 0x55e4befbde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.610+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befafdc0 0x55e4befbe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.610+0000 7fd18ca1b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befba680 0x55e4bf02ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.610+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841bc4a0 con 0x55e4befaf4a0 2024-09-15T22:12:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.610+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841bd910 con 0x55e4befafdc0 2024-09-15T22:12:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.610+0000 7fd18ca1b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd1841bcb70 con 0x55e4befba680 2024-09-15T22:12:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.610+0000 7fd18ca1b740 10 client.0 _ll_register_callbacks cb 0x7fd17c009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.610+0000 7fd183fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befaf4a0 0x55e4befbde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.610+0000 7fd183fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befaf4a0 0x55e4befbde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57850/0 (socket says 172.21.15.5:57850) 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.610+0000 7fd183fff640 1 -- 172.21.15.5:0/1570735501 learned_addr learned my addr 172.21.15.5:0/1570735501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.610+0000 7fd1837fe640 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befafdc0 0x55e4befbe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.611+0000 7fd183fff640 1 -- 172.21.15.5:0/1570735501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befba680 msgr2=0x55e4bf02ca00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.611+0000 7fd183fff640 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befba680 0x55e4bf02ca00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.611+0000 7fd183fff640 1 -- 172.21.15.5:0/1570735501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befafdc0 msgr2=0x55e4befbe370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.611+0000 7fd183fff640 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befafdc0 0x55e4befbe370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.611+0000 7fd183fff640 1 -- 172.21.15.5:0/1570735501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4bf02d040 con 0x55e4befaf4a0 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.611+0000 7fd1837fe640 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befafdc0 0x55e4befbe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.611+0000 7fd183fff640 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befaf4a0 0x55e4befbde30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd17c010010 tx=0x7fd17c023f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c5177e1a01a1d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.611+0000 7fd1637fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.611+0000 7fd1637fe640 1 -- 172.21.15.5:0/1570735501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17c037070 con 0x55e4befaf4a0 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.611+0000 7fd1637fe640 1 -- 172.21.15.5:0/1570735501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fd17c023240 con 0x55e4befaf4a0 2024-09-15T22:12:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.611+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1570735501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd17c0099a0 con 0x55e4befaf4a0 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.613+0000 7fd1637fe640 1 -- 172.21.15.5:0/1570735501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd17c027020 con 0x55e4befaf4a0 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.613+0000 7fd18ca1b740 10 client.16140 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.613+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1570735501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4bf0343a0 con 0x55e4befaf4a0 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.614+0000 7fd1637fe640 1 -- 172.21.15.5:0/1570735501 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd17c00ebc0 con 0x55e4befaf4a0 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.614+0000 7fd1637fe640 1 -- 172.21.15.5:0/1570735501 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd17c022b40 con 0x55e4befaf4a0 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.614+0000 7fd18ca1b740 10 client.16140 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.614+0000 7fd18ca1b740 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.614+0000 7fd18ca1b740 20 client.16140 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.614+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e4bf034920 0x55e4bf036d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.614+0000 7fd18ca1b740 4 client.16140 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.614+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1570735501 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fd17c0099a0 con 0x55e4bf034920 2024-09-15T22:12:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.614+0000 7fd1837fe640 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e4bf034920 0x55e4bf036d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd1837fe640 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e4bf034920 0x55e4bf036d10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74418b89255b3018 server_cookie=d68631cb2e48f3dd in_seq=0 out_seq=0 2024-09-15T22:12:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd1637fe640 10 client.16140 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd1637fe640 1 -- 172.21.15.5:0/1570735501 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fd17c0099a0 con 0x55e4bf034920 2024-09-15T22:12:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd1637fe640 10 client.16140 handle_command_reply: tid=0 2024-09-15T22:12:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd18ca1b740 1 client.16140 shutdown 2024-09-15T22:12:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1570735501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befaf4a0 msgr2=0x55e4befbde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befaf4a0 0x55e4befbde30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd17c010010 tx=0x7fd17c023f40 comp rx=0 tx=0).stop 2024-09-15T22:12:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1570735501 shutdown_connections 2024-09-15T22:12:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4befba680 0x55e4bf02ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e4bf034920 0x55e4bf036d10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4befafdc0 0x55e4befbe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd18ca1b740 1 --2- 172.21.15.5:0/1570735501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4befaf4a0 0x55e4befbde30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1570735501 >> 172.21.15.5:0/1570735501 conn(0x55e4befaba30 msgr2=0x55e4befb62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:47.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.615+0000 7fd1637fe640 0 client.16140 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:47.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.616+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1570735501 shutdown_connections 2024-09-15T22:12:47.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.616+0000 7fd18ca1b740 1 -- 172.21.15.5:0/1570735501 wait complete. 2024-09-15T22:12:47.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.616+0000 7fd18ca1b740 20 client.16140 trim_cache size 0 max 16384 2024-09-15T22:12:47.619 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:12:47.619 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:12:47.619 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:12:47.619 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:12:47.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 -- 172.21.15.5:0/3274987891 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd16c0a3540 msgr2=0x7fd16c0a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:47.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd16c0a3540 0x7fd16c0a5a00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd17402a960 tx=0x7fd17402a8f0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 -- 172.21.15.5:0/3274987891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184109830 msgr2=0x7fd1841a76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184109830 0x7fd1841a76d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd17800e9f0 tx=0x7fd17800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 -- 172.21.15.5:0/3274987891 shutdown_connections 2024-09-15T22:12:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd16c0a3540 0x7fd16c0a5a00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd18411b140 0x7fd1841aba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd184109830 0x7fd1841a76d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 --2- 172.21.15.5:0/3274987891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd184109450 0x7fd1841a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 -- 172.21.15.5:0/3274987891 >> 172.21.15.5:0/3274987891 conn(0x7fd18406e140 msgr2=0x7fd18411bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 -- 172.21.15.5:0/3274987891 shutdown_connections 2024-09-15T22:12:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:47.617+0000 7fd18a642640 1 -- 172.21.15.5:0/3274987891 wait complete. 2024-09-15T22:12:47.632 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:47 smithi005 ceph-mon[30944]: pgmap v119: 129 pgs: 129 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.28k op/s 2024-09-15T22:12:47.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:47 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:12:47.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:47 smithi099 ceph-mon[34892]: pgmap v119: 129 pgs: 129 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.28k op/s 2024-09-15T22:12:47.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:47 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:12:47.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:47 smithi129 ceph-mon[35314]: pgmap v119: 129 pgs: 129 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 17 MiB/s wr, 1.28k op/s 2024-09-15T22:12:47.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:47 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:12:48.579 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:12:48.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:12:49.030 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:49.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:49 smithi005 ceph-mon[30944]: pgmap v120: 129 pgs: 129 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 992 op/s 2024-09-15T22:12:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 200.0 s, 1825.2 tps, lat 54.450 ms stddev 62.973 2024-09-15T22:12:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.662+0000 7fbe7d568640 1 -- 172.21.15.5:0/170430345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe78105f50 msgr2=0x7fbe781063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:49.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.662+0000 7fbe7d568640 1 --2- 172.21.15.5:0/170430345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe78105f50 0x7fbe781063e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009ca0 tx=0x7fbe7002a940 comp rx=0 tx=0).stop 2024-09-15T22:12:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.662+0000 7fbe7d568640 1 -- 172.21.15.5:0/170430345 shutdown_connections 2024-09-15T22:12:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.662+0000 7fbe7d568640 1 --2- 172.21.15.5:0/170430345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe78105f50 0x7fbe781063e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.662+0000 7fbe7d568640 1 --2- 172.21.15.5:0/170430345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe78101460 0x7fbe78105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.662+0000 7fbe7d568640 1 --2- 172.21.15.5:0/170430345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe78100b40 0x7fbe78100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.662+0000 7fbe7d568640 1 -- 172.21.15.5:0/170430345 >> 172.21.15.5:0/170430345 conn(0x7fbe780fc7d0 msgr2=0x7fbe780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.663+0000 7fbe7d568640 1 -- 172.21.15.5:0/170430345 shutdown_connections 2024-09-15T22:12:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.663+0000 7fbe7d568640 1 -- 172.21.15.5:0/170430345 wait complete. 2024-09-15T22:12:49.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.663+0000 7fbe7d568640 1 Processor -- start 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe7d568640 1 -- start start 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe7d568640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe78100b40 0x7fbe78114ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe7d568640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe78101460 0x7fbe78115220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe7d568640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe78105f50 0x7fbe7810ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe7d568640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbe78107950 con 0x7fbe78100b40 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe7d568640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe781077d0 con 0x7fbe78105f50 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe7d568640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe78107ad0 con 0x7fbe78101460 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe76ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe78100b40 0x7fbe78114ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe76ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe78100b40 0x7fbe78114ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57882/0 (socket says 172.21.15.5:57882) 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe76ffd640 1 -- 172.21.15.5:0/24306163 learned_addr learned my addr 172.21.15.5:0/24306163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe767fc640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe78101460 0x7fbe78115220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:49.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.664+0000 7fbe777fe640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe78105f50 0x7fbe7810ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.665+0000 7fbe767fc640 1 -- 172.21.15.5:0/24306163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe78105f50 msgr2=0x7fbe7810ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.665+0000 7fbe767fc640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe78105f50 0x7fbe7810ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.665+0000 7fbe767fc640 1 -- 172.21.15.5:0/24306163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe78100b40 msgr2=0x7fbe78114ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.665+0000 7fbe767fc640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe78100b40 0x7fbe78114ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.665+0000 7fbe767fc640 1 -- 172.21.15.5:0/24306163 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7810f850 con 0x7fbe78101460 2024-09-15T22:12:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.665+0000 7fbe767fc640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe78101460 0x7fbe78115220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800e7a0 tx=0x7fbe6800ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:49.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.665+0000 7fbe5ffff640 1 -- 172.21.15.5:0/24306163 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6800ccc0 con 0x7fbe78101460 2024-09-15T22:12:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.665+0000 7fbe5ffff640 1 -- 172.21.15.5:0/24306163 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe68009530 con 0x7fbe78101460 2024-09-15T22:12:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.665+0000 7fbe7d568640 1 -- 172.21.15.5:0/24306163 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7810fb40 con 0x7fbe78101460 2024-09-15T22:12:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.665+0000 7fbe5ffff640 1 -- 172.21.15.5:0/24306163 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68018880 con 0x7fbe78101460 2024-09-15T22:12:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.665+0000 7fbe7d568640 1 -- 172.21.15.5:0/24306163 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe781b3ba0 con 0x7fbe78101460 2024-09-15T22:12:49.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.667+0000 7fbe7d568640 1 -- 172.21.15.5:0/24306163 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7806b070 con 0x7fbe78101460 2024-09-15T22:12:49.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.669+0000 7fbe5ffff640 1 -- 172.21.15.5:0/24306163 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbe68018a20 con 0x7fbe78101460 2024-09-15T22:12:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.670+0000 7fbe5ffff640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe600a33b0 0x7fbe600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.670+0000 7fbe5ffff640 1 -- 172.21.15.5:0/24306163 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fbe680d01d0 con 0x7fbe78101460 2024-09-15T22:12:49.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.670+0000 7fbe76ffd640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe600a33b0 0x7fbe600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:49.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.671+0000 7fbe76ffd640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe600a33b0 0x7fbe600a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0095f0 tx=0x7fbe6c005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:49.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:49.676+0000 7fbe5ffff640 1 -- 172.21.15.5:0/24306163 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe6809c1c0 con 0x7fbe78101460 2024-09-15T22:12:49.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:49 smithi099 ceph-mon[34892]: pgmap v120: 129 pgs: 129 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 992 op/s 2024-09-15T22:12:49.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:49 smithi129 ceph-mon[35314]: pgmap v120: 129 pgs: 129 active+clean; 2.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 14 MiB/s wr, 992 op/s 2024-09-15T22:12:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.063+0000 7fbe7d568640 1 -- 172.21.15.5:0/24306163 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe781b3f80 con 0x7fbe78101460 2024-09-15T22:12:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:12:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.064+0000 7fbe5ffff640 1 -- 172.21.15.5:0/24306163 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fbe680a04c0 con 0x7fbe78101460 2024-09-15T22:12:50.066 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:12:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:12:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.067+0000 7fbe7d568640 1 -- 172.21.15.5:0/24306163 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe600a33b0 msgr2=0x7fbe600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.067+0000 7fbe7d568640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe600a33b0 0x7fbe600a5870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0095f0 tx=0x7fbe6c005f90 comp rx=0 tx=0).stop 2024-09-15T22:12:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.068+0000 7fbe7d568640 1 -- 172.21.15.5:0/24306163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe78101460 msgr2=0x7fbe78115220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.068+0000 7fbe7d568640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe78101460 0x7fbe78115220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800e7a0 tx=0x7fbe6800ec60 comp rx=0 tx=0).stop 2024-09-15T22:12:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.068+0000 7fbe7d568640 1 -- 172.21.15.5:0/24306163 shutdown_connections 2024-09-15T22:12:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.068+0000 7fbe7d568640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe600a33b0 0x7fbe600a5870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.068+0000 7fbe7d568640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe78105f50 0x7fbe7810ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.068+0000 7fbe7d568640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe78101460 0x7fbe78115220 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbe6800e7a0 tx=0x7fbe6800ec60 comp rx=0 tx=0).stop 2024-09-15T22:12:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.068+0000 7fbe7d568640 1 --2- 172.21.15.5:0/24306163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe78100b40 0x7fbe78114ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.068+0000 7fbe7d568640 1 -- 172.21.15.5:0/24306163 >> 172.21.15.5:0/24306163 conn(0x7fbe780fc7d0 msgr2=0x7fbe780fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.068+0000 7fbe7d568640 1 -- 172.21.15.5:0/24306163 shutdown_connections 2024-09-15T22:12:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:50.068+0000 7fbe7d568640 1 -- 172.21.15.5:0/24306163 wait complete. 2024-09-15T22:12:50.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:50 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/24306163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:50 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/24306163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:50.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:50 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/24306163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:12:51.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:51 smithi005 ceph-mon[30944]: pgmap v121: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-09-15T22:12:51.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:12:51.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:51 smithi099 ceph-mon[34892]: pgmap v121: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-09-15T22:12:51.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:51 smithi129 ceph-mon[35314]: pgmap v121: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 14 MiB/s wr, 1.04k op/s 2024-09-15T22:12:52.227 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.825+0000 7f563a342640 1 -- 172.21.15.5:0/2327172304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563411b140 msgr2=0x7f563411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.825+0000 7f563a342640 1 --2- 172.21.15.5:0/2327172304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563411b140 0x7f563411d530 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f562c01acb0 tx=0x7f562c03b8e0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.827+0000 7f563a342640 1 -- 172.21.15.5:0/2327172304 shutdown_connections 2024-09-15T22:12:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.827+0000 7f563a342640 1 --2- 172.21.15.5:0/2327172304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563411b140 0x7f563411d530 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.827+0000 7f563a342640 1 --2- 172.21.15.5:0/2327172304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5634109830 0x7f5634109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.827+0000 7f563a342640 1 --2- 172.21.15.5:0/2327172304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5634109450 0x7f5634110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.827+0000 7f563a342640 1 -- 172.21.15.5:0/2327172304 >> 172.21.15.5:0/2327172304 conn(0x7f563406e140 msgr2=0x7f563406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.828+0000 7f563a342640 1 -- 172.21.15.5:0/2327172304 shutdown_connections 2024-09-15T22:12:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.828+0000 7f563a342640 1 -- 172.21.15.5:0/2327172304 wait complete. 2024-09-15T22:12:52.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.828+0000 7f563a342640 1 Processor -- start 2024-09-15T22:12:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.829+0000 7f563a342640 1 -- start start 2024-09-15T22:12:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.829+0000 7f563a342640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5634109450 0x7f563411a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.829+0000 7f563a342640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5634109830 0x7f5634116860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.830+0000 7f563a342640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563411b140 0x7f5634116da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.830+0000 7f563a342640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56341132b0 con 0x7f5634109830 2024-09-15T22:12:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.830+0000 7f563a342640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5634113130 con 0x7f563411b140 2024-09-15T22:12:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.830+0000 7f563a342640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5634113430 con 0x7f5634109450 2024-09-15T22:12:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.830+0000 7f56337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5634109830 0x7f5634116860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.830+0000 7f56337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5634109830 0x7f5634116860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57894/0 (socket says 172.21.15.5:57894) 2024-09-15T22:12:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.830+0000 7f56337fe640 1 -- 172.21.15.5:0/72253981 learned_addr learned my addr 172.21.15.5:0/72253981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:52.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.830+0000 7f5633fff640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5634109450 0x7f563411a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.830+0000 7f56388df640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563411b140 0x7f5634116da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.831+0000 7f56337fe640 1 -- 172.21.15.5:0/72253981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5634109450 msgr2=0x7f563411a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:52.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.831+0000 7f56337fe640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5634109450 0x7f563411a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.831+0000 7f56337fe640 1 -- 172.21.15.5:0/72253981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563411b140 msgr2=0x7f5634116da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:52.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.831+0000 7f56337fe640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563411b140 0x7f5634116da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.831+0000 7f56337fe640 1 -- 172.21.15.5:0/72253981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5634117360 con 0x7f5634109830 2024-09-15T22:12:52.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.831+0000 7f5633fff640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5634109450 0x7f563411a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:52.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.832+0000 7f56337fe640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5634109830 0x7f5634116860 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f562800d980 tx=0x7f562800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.832+0000 7f56317fa640 1 -- 172.21.15.5:0/72253981 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628017070 con 0x7f5634109830 2024-09-15T22:12:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.832+0000 7f56317fa640 1 -- 172.21.15.5:0/72253981 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5628009bb0 con 0x7f5634109830 2024-09-15T22:12:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.832+0000 7f56317fa640 1 -- 172.21.15.5:0/72253981 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562800f620 con 0x7f5634109830 2024-09-15T22:12:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.832+0000 7f563a342640 1 -- 172.21.15.5:0/72253981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5634117650 con 0x7f5634109830 2024-09-15T22:12:52.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.832+0000 7f563a342640 1 -- 172.21.15.5:0/72253981 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56341ab460 con 0x7f5634109830 2024-09-15T22:12:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.839+0000 7f563c71b740 1 Processor -- start 2024-09-15T22:12:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.839+0000 7f56317fa640 1 -- 172.21.15.5:0/72253981 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5628010460 con 0x7f5634109830 2024-09-15T22:12:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.840+0000 7f563c71b740 1 -- start start 2024-09-15T22:12:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.840+0000 7f563c71b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.840+0000 7f563c71b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 0x559ddc23ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.840+0000 7f563c71b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc247370 0x559ddc249760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.840+0000 7f563c71b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ddc1a22f0 con 0x559ddc23c5d0 2024-09-15T22:12:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.840+0000 7f563c71b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559ddc149cf0 con 0x559ddc247370 2024-09-15T22:12:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.840+0000 7f563c71b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ddc19e2b0 con 0x559ddc23db40 2024-09-15T22:12:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.841+0000 7f56388df640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.841+0000 7f56388df640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57950/0 (socket says 172.21.15.5:57950) 2024-09-15T22:12:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.841+0000 7f56388df640 1 -- 172.21.15.5:0/3911589160 learned_addr learned my addr 172.21.15.5:0/3911589160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.842+0000 7f56337fe640 1 --2- 172.21.15.5:0/3911589160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc247370 0x559ddc249760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.842+0000 7f56388df640 1 -- 172.21.15.5:0/3911589160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc247370 msgr2=0x559ddc249760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.842+0000 7f56317fa640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f561c0a35f0 0x7f561c0a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.842+0000 7f56388df640 1 --2- 172.21.15.5:0/3911589160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc247370 0x559ddc249760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.842+0000 7f56388df640 1 -- 172.21.15.5:0/3911589160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 msgr2=0x559ddc23ca10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:12:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.842+0000 7f56388df640 1 --2- 172.21.15.5:0/3911589160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 0x559ddc23ca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.842+0000 7f56388df640 1 -- 172.21.15.5:0/3911589160 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ddc12e3a0 con 0x559ddc23db40 2024-09-15T22:12:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.842+0000 7f56337fe640 1 --2- 172.21.15.5:0/3911589160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc247370 0x559ddc249760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.843+0000 7f56317fa640 1 -- 172.21.15.5:0/72253981 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f56280cf9f0 con 0x7f5634109830 2024-09-15T22:12:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.843+0000 7f56388df640 1 --2- 172.21.15.5:0/3911589160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23c090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f562c01acb0 tx=0x7f562c01aa30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ae95b44f12a7744 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.843+0000 7f56388df640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f561c0a35f0 0x7f561c0a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.844+0000 7f56388df640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f561c0a35f0 0x7f561c0a5ab0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f562c055000 tx=0x7f562c004790 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.844+0000 7f561a7fc640 1 -- 172.21.15.5:0/3911589160 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c05a040 con 0x559ddc23db40 2024-09-15T22:12:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.844+0000 7f561a7fc640 1 -- 172.21.15.5:0/3911589160 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f562c047070 con 0x559ddc23db40 2024-09-15T22:12:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.844+0000 7f561a7fc640 1 -- 172.21.15.5:0/3911589160 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c004510 con 0x559ddc23db40 2024-09-15T22:12:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.845+0000 7f563c71b740 1 -- 172.21.15.5:0/3911589160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 msgr2=0x559ddc23c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.845+0000 7f563c71b740 1 --2- 172.21.15.5:0/3911589160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23c090 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f562c01acb0 tx=0x7f562c01aa30 comp rx=0 tx=0).stop 2024-09-15T22:12:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.845+0000 7f563c71b740 1 -- 172.21.15.5:0/3911589160 shutdown_connections 2024-09-15T22:12:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.845+0000 7f563c71b740 1 --2- 172.21.15.5:0/3911589160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc247370 0x559ddc249760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.845+0000 7f563c71b740 1 --2- 172.21.15.5:0/3911589160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 0x559ddc23ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.845+0000 7f563c71b740 1 --2- 172.21.15.5:0/3911589160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23c090 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.845+0000 7f563c71b740 1 -- 172.21.15.5:0/3911589160 >> 172.21.15.5:0/3911589160 conn(0x559ddc205310 msgr2=0x559ddc238c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:52.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.846+0000 7f563c71b740 1 -- 172.21.15.5:0/3911589160 shutdown_connections 2024-09-15T22:12:52.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.846+0000 7f563c71b740 1 -- 172.21.15.5:0/3911589160 wait complete. 2024-09-15T22:12:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.846+0000 7f563c71b740 1 -- start start 2024-09-15T22:12:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.846+0000 7f563c71b740 1 Processor -- start 2024-09-15T22:12:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.847+0000 7f563c71b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 0x559ddc246cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.847+0000 7f563c71b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23db40 0x559ddc243f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.847+0000 7f563c71b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc247370 0x559ddc2444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.847+0000 7f563c71b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ddc24a010 con 0x559ddc23c5d0 2024-09-15T22:12:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.847+0000 7f563c71b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559ddc15d060 con 0x559ddc23db40 2024-09-15T22:12:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.847+0000 7f563c71b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559ddc24a190 con 0x559ddc247370 2024-09-15T22:12:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.847+0000 7f563c71b740 10 client.0 _ll_register_callbacks cb 0x559ddbeb08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.847+0000 7f56337fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23db40 0x559ddc243f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.847+0000 7f56337fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23db40 0x559ddc243f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:58292/0 (socket says 172.21.15.5:58292) 2024-09-15T22:12:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.847+0000 7f56337fe640 1 -- 172.21.15.5:0/2726409617 learned_addr learned my addr 172.21.15.5:0/2726409617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.847+0000 7f56388df640 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc247370 0x559ddc2444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.848+0000 7f56337fe640 1 -- 172.21.15.5:0/2726409617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc247370 msgr2=0x559ddc2444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.848+0000 7f56337fe640 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc247370 0x559ddc2444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.848+0000 7f56337fe640 1 -- 172.21.15.5:0/2726409617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 msgr2=0x559ddc246cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.848+0000 7f56337fe640 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 0x559ddc246cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.848+0000 7f56337fe640 1 -- 172.21.15.5:0/2726409617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ddbf133c0 con 0x559ddc23db40 2024-09-15T22:12:52.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.848+0000 7f56337fe640 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23db40 0x559ddc243f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f56280a4010 tx=0x7f562809b2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=489e2b5c6dea2f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.848+0000 7f56137fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.848+0000 7f56137fe640 1 -- 172.21.15.5:0/2726409617 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628020040 con 0x559ddc23db40 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.848+0000 7f563c71b740 1 -- 172.21.15.5:0/2726409617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f562c047070 con 0x559ddc23db40 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.848+0000 7f56137fe640 1 -- 172.21.15.5:0/2726409617 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f562809c8a0 con 0x559ddc23db40 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.849+0000 7f56137fe640 1 -- 172.21.15.5:0/2726409617 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f562809fc80 con 0x559ddc23db40 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.849+0000 7f563c71b740 10 client.25748 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.849+0000 7f563c71b740 1 -- 172.21.15.5:0/2726409617 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ddc255a20 con 0x559ddc23db40 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.849+0000 7f56137fe640 1 -- 172.21.15.5:0/2726409617 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f562c047070 con 0x559ddc23db40 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.849+0000 7f56137fe640 1 -- 172.21.15.5:0/2726409617 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f562809add0 con 0x559ddc23db40 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.849+0000 7f563c71b740 10 client.25748 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.849+0000 7f563c71b740 10 client.25748 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.849+0000 7f563c71b740 20 client.25748 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.850+0000 7f563c71b740 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559ddc257420 0x559ddc2598c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.850+0000 7f563c71b740 4 client.25748 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.850+0000 7f563c71b740 1 -- 172.21.15.5:0/2726409617 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ddc1af860 con 0x559ddc257420 2024-09-15T22:12:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.850+0000 7f5633fff640 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559ddc257420 0x559ddc2598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.852+0000 7f5633fff640 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559ddc257420 0x559ddc2598c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5eb3fd157ac2363 server_cookie=6c8ac81be485e3ad in_seq=0 out_seq=0 2024-09-15T22:12:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.852+0000 7f56137fe640 10 client.25748 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:12:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.853+0000 7f56137fe640 1 -- 172.21.15.5:0/2726409617 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ddc1af860 con 0x559ddc257420 2024-09-15T22:12:52.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.853+0000 7f56137fe640 10 client.25748 handle_command_reply: tid=0 2024-09-15T22:12:52.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.853+0000 7f563c71b740 1 client.25748 shutdown 2024-09-15T22:12:52.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.854+0000 7f563c71b740 1 -- 172.21.15.5:0/2726409617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23db40 msgr2=0x559ddc243f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:52.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.854+0000 7f563c71b740 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23db40 0x559ddc243f80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f56280a4010 tx=0x7f562809b2e0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.854+0000 7f563c71b740 1 -- 172.21.15.5:0/2726409617 shutdown_connections 2024-09-15T22:12:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.854+0000 7f563c71b740 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559ddc257420 0x559ddc2598c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.854+0000 7f563c71b740 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc247370 0x559ddc2444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.854+0000 7f563c71b740 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23db40 0x559ddc243f80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.854+0000 7f563c71b740 1 --2- 172.21.15.5:0/2726409617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 0x559ddc246cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.854+0000 7f563c71b740 1 -- 172.21.15.5:0/2726409617 >> 172.21.15.5:0/2726409617 conn(0x559ddc205310 msgr2=0x559ddc23e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.855+0000 7f56137fe640 0 client.25748 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:12:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.855+0000 7f563c71b740 1 -- 172.21.15.5:0/2726409617 shutdown_connections 2024-09-15T22:12:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.855+0000 7f563c71b740 1 -- 172.21.15.5:0/2726409617 wait complete. 2024-09-15T22:12:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.855+0000 7f563c71b740 20 client.25748 trim_cache size 0 max 16384 2024-09-15T22:12:52.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.875+0000 7f563c71b740 1 Processor -- start 2024-09-15T22:12:52.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.875+0000 7f563c71b740 1 -- start start 2024-09-15T22:12:52.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.876+0000 7f563c71b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23c5d0 0x559ddc253310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.876+0000 7f563c71b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.876+0000 7f563c71b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc247370 0x559ddc23f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.876+0000 7f56337fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.876+0000 7f56337fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57976/0 (socket says 172.21.15.5:57976) 2024-09-15T22:12:52.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.876+0000 7f5633fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23c5d0 0x559ddc253310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.876+0000 7f5633fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23c5d0 0x559ddc253310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:58300/0 (socket says 172.21.15.5:58300) 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f56337fe640 1 -- 172.21.15.5:0/142809492 learned_addr learned my addr 172.21.15.5:0/142809492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f563c71b740 1 -- 172.21.15.5:0/142809492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5620012880 con 0x559ddc247370 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f563c71b740 1 -- 172.21.15.5:0/142809492 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5620010150 con 0x559ddc23c5d0 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f563c71b740 1 -- 172.21.15.5:0/142809492 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f562000dfa0 con 0x559ddc23db40 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f56388df640 1 --2- 172.21.15.5:0/142809492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc247370 0x559ddc23f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f5633fff640 1 -- 172.21.15.5:0/142809492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 msgr2=0x559ddc23f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f5633fff640 1 --2- 172.21.15.5:0/142809492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f5633fff640 1 -- 172.21.15.5:0/142809492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc247370 msgr2=0x559ddc23f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f5633fff640 1 --2- 172.21.15.5:0/142809492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc247370 0x559ddc23f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f5633fff640 1 -- 172.21.15.5:0/142809492 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ddc244850 con 0x559ddc23c5d0 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f56337fe640 1 --2- 172.21.15.5:0/142809492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.877+0000 7f56388df640 1 --2- 172.21.15.5:0/142809492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc247370 0x559ddc23f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:52.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.878+0000 7f5633fff640 1 --2- 172.21.15.5:0/142809492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23c5d0 0x559ddc253310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5624011460 tx=0x7f5624009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b854e8b2bbb7b4d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:52.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.878+0000 7f5619ffb640 1 -- 172.21.15.5:0/142809492 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624003c40 con 0x559ddc23c5d0 2024-09-15T22:12:52.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.878+0000 7f5619ffb640 1 -- 172.21.15.5:0/142809492 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5624003de0 con 0x559ddc23c5d0 2024-09-15T22:12:52.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.878+0000 7f5619ffb640 1 -- 172.21.15.5:0/142809492 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624011aa0 con 0x559ddc23c5d0 2024-09-15T22:12:52.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.879+0000 7f563c71b740 1 -- 172.21.15.5:0/142809492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23c5d0 msgr2=0x559ddc253310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.879+0000 7f563c71b740 1 --2- 172.21.15.5:0/142809492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23c5d0 0x559ddc253310 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5624011460 tx=0x7f5624009b90 comp rx=0 tx=0).stop 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.879+0000 7f563c71b740 1 -- 172.21.15.5:0/142809492 shutdown_connections 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.879+0000 7f563c71b740 1 --2- 172.21.15.5:0/142809492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc247370 0x559ddc23f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.879+0000 7f563c71b740 1 --2- 172.21.15.5:0/142809492 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc23f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.879+0000 7f563c71b740 1 --2- 172.21.15.5:0/142809492 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc23c5d0 0x559ddc253310 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.879+0000 7f563c71b740 1 -- 172.21.15.5:0/142809492 >> 172.21.15.5:0/142809492 conn(0x559ddc205310 msgr2=0x559ddc259430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.879+0000 7f563c71b740 1 -- 172.21.15.5:0/142809492 shutdown_connections 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.880+0000 7f563c71b740 1 -- 172.21.15.5:0/142809492 wait complete. 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.880+0000 7f563c71b740 1 -- start start 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.881+0000 7f563c71b740 1 Processor -- start 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.881+0000 7f563c71b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 0x559ddc24a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.881+0000 7f563c71b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc24a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.881+0000 7f563c71b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc247370 0x559ddc24aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.881+0000 7f563c71b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f562000e610 con 0x559ddc23c5d0 2024-09-15T22:12:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.881+0000 7f563c71b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f562000fa80 con 0x559ddc247370 2024-09-15T22:12:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.881+0000 7f563c71b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f562000ece0 con 0x559ddc23db40 2024-09-15T22:12:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.881+0000 7f563c71b740 10 client.0 _ll_register_callbacks cb 0x7f56280d2f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f56337fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc24a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f56337fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc24a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57988/0 (socket says 172.21.15.5:57988) 2024-09-15T22:12:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f56337fe640 1 -- 172.21.15.5:0/986905532 learned_addr learned my addr 172.21.15.5:0/986905532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f5633fff640 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 0x559ddc24a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f56337fe640 1 -- 172.21.15.5:0/986905532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc247370 msgr2=0x559ddc24aec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:12:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f56337fe640 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc247370 0x559ddc24aec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f56337fe640 1 -- 172.21.15.5:0/986905532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 msgr2=0x559ddc24a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f56337fe640 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 0x559ddc24a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f56337fe640 1 -- 172.21.15.5:0/986905532 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ddc2c02c0 con 0x559ddc23db40 2024-09-15T22:12:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f56337fe640 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc24a980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f562800d950 tx=0x7f5628099e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34223a7695d04287 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f56137fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f5633fff640 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 0x559ddc24a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.882+0000 7f56137fe640 1 -- 172.21.15.5:0/986905532 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628020040 con 0x559ddc23db40 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f56137fe640 1 -- 172.21.15.5:0/986905532 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f562809add0 con 0x559ddc23db40 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f563c71b740 1 -- 172.21.15.5:0/986905532 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5624003de0 con 0x559ddc23db40 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f56137fe640 1 -- 172.21.15.5:0/986905532 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f562809b010 con 0x559ddc23db40 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f563c71b740 10 client.25735 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f563c71b740 1 -- 172.21.15.5:0/986905532 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559ddc2c15d0 con 0x559ddc23db40 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f56137fe640 1 -- 172.21.15.5:0/986905532 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5624003de0 con 0x559ddc23db40 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f56137fe640 1 -- 172.21.15.5:0/986905532 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f562809a9d0 con 0x559ddc23db40 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f563c71b740 10 client.25735 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f563c71b740 10 client.25735 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:12:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f563c71b740 20 client.25735 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f563c71b740 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559ddc2c1c10 0x559ddc2c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.883+0000 7f563c71b740 4 client.25735 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:12:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.884+0000 7f563c71b740 1 -- 172.21.15.5:0/986905532 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ddc1a9d70 con 0x559ddc2c1c10 2024-09-15T22:12:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.884+0000 7f5633fff640 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559ddc2c1c10 0x559ddc2c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f56137fe640 10 client.25735 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:12:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f5633fff640 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559ddc2c1c10 0x559ddc2c4030 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a64ed6566a94538 server_cookie=8fe976ea11587ccd in_seq=0 out_seq=0 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f56137fe640 1 -- 172.21.15.5:0/986905532 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ddc1a9d70 con 0x559ddc2c1c10 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f56137fe640 10 client.25735 handle_command_reply: tid=0 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f563c71b740 1 client.25735 shutdown 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f563c71b740 1 -- 172.21.15.5:0/986905532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 msgr2=0x559ddc24a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f563c71b740 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc24a980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f562800d950 tx=0x7f5628099e90 comp rx=0 tx=0).stop 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f563c71b740 1 -- 172.21.15.5:0/986905532 shutdown_connections 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f563c71b740 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559ddc2c1c10 0x559ddc2c4030 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f56137fe640 0 client.25735 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f563c71b740 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559ddc247370 0x559ddc24aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f563c71b740 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559ddc23db40 0x559ddc24a980 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f563c71b740 1 --2- 172.21.15.5:0/986905532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ddc23c5d0 0x559ddc24a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f563c71b740 1 -- 172.21.15.5:0/986905532 >> 172.21.15.5:0/986905532 conn(0x559ddc205310 msgr2=0x559ddc2436f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f563c71b740 1 -- 172.21.15.5:0/986905532 shutdown_connections 2024-09-15T22:12:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.887+0000 7f563c71b740 1 -- 172.21.15.5:0/986905532 wait complete. 2024-09-15T22:12:52.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.888+0000 7f563c71b740 20 client.25735 trim_cache size 0 max 16384 2024-09-15T22:12:52.892 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:12:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.888+0000 7f563a342640 1 -- 172.21.15.5:0/72253981 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f561c0a35f0 msgr2=0x7f561c0a5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.888+0000 7f563a342640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f561c0a35f0 0x7f561c0a5ab0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f562c055000 tx=0x7f562c004790 comp rx=0 tx=0).stop 2024-09-15T22:12:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.888+0000 7f563a342640 1 -- 172.21.15.5:0/72253981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5634109830 msgr2=0x7f5634116860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.888+0000 7f563a342640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5634109830 0x7f5634116860 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f562800d980 tx=0x7f562800de40 comp rx=0 tx=0).stop 2024-09-15T22:12:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.888+0000 7f563a342640 1 -- 172.21.15.5:0/72253981 shutdown_connections 2024-09-15T22:12:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.888+0000 7f563a342640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f561c0a35f0 0x7f561c0a5ab0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.889+0000 7f563a342640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f563411b140 0x7f5634116da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.889+0000 7f563a342640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5634109830 0x7f5634116860 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.889+0000 7f563a342640 1 --2- 172.21.15.5:0/72253981 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5634109450 0x7f563411a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.889+0000 7f563a342640 1 -- 172.21.15.5:0/72253981 >> 172.21.15.5:0/72253981 conn(0x7f563406e140 msgr2=0x7f5634070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.889+0000 7f563a342640 1 -- 172.21.15.5:0/72253981 shutdown_connections 2024-09-15T22:12:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:52.889+0000 7f563a342640 1 -- 172.21.15.5:0/72253981 wait complete. 2024-09-15T22:12:53.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:53 smithi005 ceph-mon[30944]: pgmap v122: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-15T22:12:53.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:12:53.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:53 smithi099 ceph-mon[34892]: pgmap v122: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-15T22:12:53.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:53 smithi129 ceph-mon[35314]: pgmap v122: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 MiB/s wr, 1.21k op/s 2024-09-15T22:12:54.113 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:54.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 205.0 s, 2059.6 tps, lat 48.689 ms stddev 61.656 2024-09-15T22:12:55.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.596+0000 7f1948c58640 1 -- 172.21.15.5:0/216089931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1944102570 msgr2=0x7f19441029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:55.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.596+0000 7f1948c58640 1 --2- 172.21.15.5:0/216089931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1944102570 0x7f19441029b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1938009900 tx=0x7f193802a480 comp rx=0 tx=0).stop 2024-09-15T22:12:55.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.596+0000 7f1948c58640 1 -- 172.21.15.5:0/216089931 shutdown_connections 2024-09-15T22:12:55.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.596+0000 7f1948c58640 1 --2- 172.21.15.5:0/216089931 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1944102ef0 0x7f194410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.596+0000 7f1948c58640 1 --2- 172.21.15.5:0/216089931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1944102570 0x7f19441029b0 secure :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f1938009900 tx=0x7f193802a480 comp rx=0 tx=0).stop 2024-09-15T22:12:55.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.596+0000 7f1948c58640 1 --2- 172.21.15.5:0/216089931 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1944100e20 0x7f1944101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.596+0000 7f1948c58640 1 -- 172.21.15.5:0/216089931 >> 172.21.15.5:0/216089931 conn(0x7f19440fc7d0 msgr2=0x7f19440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:55.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.596+0000 7f1948c58640 1 -- 172.21.15.5:0/216089931 shutdown_connections 2024-09-15T22:12:55.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.596+0000 7f1948c58640 1 -- 172.21.15.5:0/216089931 wait complete. 2024-09-15T22:12:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.597+0000 7f1948c58640 1 Processor -- start 2024-09-15T22:12:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.597+0000 7f1948c58640 1 -- start start 2024-09-15T22:12:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f1948c58640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1944100e20 0x7f1944198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f1948c58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1944102ef0 0x7f1944198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f1948c58640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f194419d000 0x7f194419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f1948c58640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1944110f80 con 0x7f1944102ef0 2024-09-15T22:12:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f1948c58640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1944110e00 con 0x7f194419d000 2024-09-15T22:12:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f1948c58640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1944111100 con 0x7f1944100e20 2024-09-15T22:12:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f1941d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1944102ef0 0x7f1944198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f194259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1944100e20 0x7f1944198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f194259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1944100e20 0x7f1944198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58364/0 (socket says 172.21.15.5:58364) 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f194259c640 1 -- 172.21.15.5:0/2535645996 learned_addr learned my addr 172.21.15.5:0/2535645996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f1942d9d640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f194419d000 0x7f194419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.598+0000 7f1941d9b640 1 -- 172.21.15.5:0/2535645996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1944100e20 msgr2=0x7f1944198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f1941d9b640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1944100e20 0x7f1944198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f1941d9b640 1 -- 172.21.15.5:0/2535645996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f194419d000 msgr2=0x7f194419d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f1941d9b640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f194419d000 0x7f194419d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f1941d9b640 1 -- 172.21.15.5:0/2535645996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194419dac0 con 0x7f1944102ef0 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f194259c640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1944100e20 0x7f1944198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f1942d9d640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f194419d000 0x7f194419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f1941d9b640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1944102ef0 0x7f1944198c70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1938008000 tx=0x7f1938003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f19337fe640 1 -- 172.21.15.5:0/2535645996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193803f020 con 0x7f1944102ef0 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f19337fe640 1 -- 172.21.15.5:0/2535645996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1938031c30 con 0x7f1944102ef0 2024-09-15T22:12:55.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f19337fe640 1 -- 172.21.15.5:0/2535645996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938035070 con 0x7f1944102ef0 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f1948c58640 1 -- 172.21.15.5:0/2535645996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194419dd50 con 0x7f1944102ef0 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.599+0000 7f1948c58640 1 -- 172.21.15.5:0/2535645996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19441a5480 con 0x7f1944102ef0 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.604+0000 7f194b031740 1 Processor -- start 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.604+0000 7f194b031740 1 -- start start 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194b031740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194b031740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194b031740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194b031740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641f68362f0 con 0x5641f68d1b40 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194b031740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641f67ddcf0 con 0x5641f68db370 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194b031740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5641f68322b0 con 0x5641f68d05d0 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f1942d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f1942d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68d0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36322/0 (socket says 172.21.15.5:36322) 2024-09-15T22:12:55.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f1942d9d640 1 -- 172.21.15.5:0/2436760977 learned_addr learned my addr 172.21.15.5:0/2436760977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194259c640 1 --2- 172.21.15.5:0/2436760977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f19337fe640 1 -- 172.21.15.5:0/2535645996 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1938042020 con 0x7f1944102ef0 2024-09-15T22:12:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194259c640 1 -- 172.21.15.5:0/2436760977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 msgr2=0x5641f68dd760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:12:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194259c640 1 --2- 172.21.15.5:0/2436760977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68dd760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194259c640 1 -- 172.21.15.5:0/2436760977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 msgr2=0x5641f68d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194259c640 1 --2- 172.21.15.5:0/2436760977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.605+0000 7f194259c640 1 -- 172.21.15.5:0/2436760977 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f67c23a0 con 0x5641f68d05d0 2024-09-15T22:12:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.606+0000 7f194259c640 1 --2- 172.21.15.5:0/2436760977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68d0a10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1934011510 tx=0x7f19340119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=553a55cafcc50bb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.606+0000 7f1930ff9640 1 -- 172.21.15.5:0/2436760977 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934023070 con 0x5641f68d05d0 2024-09-15T22:12:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.606+0000 7f1930ff9640 1 -- 172.21.15.5:0/2436760977 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f193400fca0 con 0x5641f68d05d0 2024-09-15T22:12:55.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.606+0000 7f1930ff9640 1 -- 172.21.15.5:0/2436760977 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1934016df0 con 0x5641f68d05d0 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.607+0000 7f19337fe640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f192c0a33b0 0x7f192c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.607+0000 7f19337fe640 1 -- 172.21.15.5:0/2535645996 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f19380efc30 con 0x7f1944102ef0 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.607+0000 7f194b031740 1 -- 172.21.15.5:0/2436760977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 msgr2=0x5641f68d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.607+0000 7f194b031740 1 --2- 172.21.15.5:0/2436760977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68d0a10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1934011510 tx=0x7f19340119d0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.607+0000 7f194b031740 1 -- 172.21.15.5:0/2436760977 shutdown_connections 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.607+0000 7f194b031740 1 --2- 172.21.15.5:0/2436760977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.607+0000 7f194b031740 1 --2- 172.21.15.5:0/2436760977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68d0a10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.607+0000 7f194b031740 1 --2- 172.21.15.5:0/2436760977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.607+0000 7f194b031740 1 -- 172.21.15.5:0/2436760977 >> 172.21.15.5:0/2436760977 conn(0x5641f6899310 msgr2=0x5641f68ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.608+0000 7f1942d9d640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f192c0a33b0 0x7f192c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.608+0000 7f194b031740 1 -- 172.21.15.5:0/2436760977 shutdown_connections 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.608+0000 7f194b031740 1 -- 172.21.15.5:0/2436760977 wait complete. 2024-09-15T22:12:55.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.608+0000 7f1942d9d640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f192c0a33b0 0x7f192c0a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f193c009fd0 tx=0x7f193c009500 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:55.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.608+0000 7f194b031740 1 -- start start 2024-09-15T22:12:55.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.608+0000 7f194b031740 1 Processor -- start 2024-09-15T22:12:55.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194b031740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d05d0 0x5641f68dad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194b031740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d1b40 0x5641f68d8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194b031740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194b031740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641f68de0c0 con 0x5641f68d05d0 2024-09-15T22:12:55.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d05d0 0x5641f68dad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194b031740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5641f67f1060 con 0x5641f68db370 2024-09-15T22:12:55.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194b031740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5641f68de240 con 0x5641f68d1b40 2024-09-15T22:12:55.395 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:12:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d05d0 0x5641f68dad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36336/0 (socket says 172.21.15.5:36336) 2024-09-15T22:12:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194259c640 1 -- 172.21.15.5:0/3397997157 learned_addr learned my addr 172.21.15.5:0/3397997157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f1941d9b640 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d1b40 0x5641f68d8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f1942d9d640 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194259c640 1 -- 172.21.15.5:0/3397997157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d1b40 msgr2=0x5641f68d8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194259c640 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d1b40 0x5641f68d8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194259c640 1 -- 172.21.15.5:0/3397997157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 msgr2=0x5641f68d85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194259c640 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68d85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.609+0000 7f194259c640 1 -- 172.21.15.5:0/3397997157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f65a73c0 con 0x5641f68d05d0 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.611+0000 7f194b031740 10 client.0 _ll_register_callbacks cb 0x5641f65448c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.611+0000 7f194b031740 1 -- 172.21.15.5:0/3397997157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f193400fca0 con 0x5641f68d05d0 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.612+0000 7f194259c640 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d05d0 0x5641f68dad50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f19340169b0 tx=0x7f1934016700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41aed54f73b5f6f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.612+0000 7f1929ffb640 10 client.16170 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.612+0000 7f1929ffb640 1 -- 172.21.15.5:0/3397997157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193402c020 con 0x5641f68d05d0 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.612+0000 7f1929ffb640 1 -- 172.21.15.5:0/3397997157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f193400f9d0 con 0x5641f68d05d0 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.612+0000 7f1929ffb640 1 -- 172.21.15.5:0/3397997157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1934023070 con 0x5641f68d05d0 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.612+0000 7f194b031740 10 client.16170 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.612+0000 7f194b031740 1 -- 172.21.15.5:0/3397997157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5641f68e9960 con 0x5641f68d05d0 2024-09-15T22:12:55.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.612+0000 7f1929ffb640 1 -- 172.21.15.5:0/3397997157 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1934005040 con 0x5641f68d05d0 2024-09-15T22:12:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.613+0000 7f194b031740 10 client.16170 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.613+0000 7f194b031740 10 client.16170 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:12:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.613+0000 7f194b031740 20 client.16170 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.613+0000 7f194b031740 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5641f68eacb0 0x5641f68ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.613+0000 7f194b031740 4 client.16170 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.613+0000 7f194b031740 1 -- 172.21.15.5:0/3397997157 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641f6843860 con 0x5641f68eacb0 2024-09-15T22:12:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.613+0000 7f1941d9b640 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5641f68eacb0 0x5641f68ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.613+0000 7f1941d9b640 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5641f68eacb0 0x5641f68ed0a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ddc9abd6cf3e2523 server_cookie=ed9d2dcc1de266d6 in_seq=0 out_seq=0 2024-09-15T22:12:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.613+0000 7f1929ffb640 10 client.16170 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:12:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f1929ffb640 1 -- 172.21.15.5:0/3397997157 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641f6843860 con 0x5641f68eacb0 2024-09-15T22:12:55.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f1929ffb640 10 client.16170 handle_command_reply: tid=0 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f194b031740 1 client.16170 shutdown 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f194b031740 1 -- 172.21.15.5:0/3397997157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d05d0 msgr2=0x5641f68dad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f194b031740 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d05d0 0x5641f68dad50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f19340169b0 tx=0x7f1934016700 comp rx=0 tx=0).stop 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f194b031740 1 -- 172.21.15.5:0/3397997157 shutdown_connections 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f194b031740 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5641f68eacb0 0x5641f68ed0a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f194b031740 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f1929ffb640 0 client.16170 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f194b031740 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d1b40 0x5641f68d8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f194b031740 1 --2- 172.21.15.5:0/3397997157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d05d0 0x5641f68dad50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f194b031740 1 -- 172.21.15.5:0/3397997157 >> 172.21.15.5:0/3397997157 conn(0x5641f6899310 msgr2=0x5641f68d2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f194b031740 1 -- 172.21.15.5:0/3397997157 shutdown_connections 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.614+0000 7f194b031740 1 -- 172.21.15.5:0/3397997157 wait complete. 2024-09-15T22:12:55.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.615+0000 7f194b031740 20 client.16170 trim_cache size 0 max 16384 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.634+0000 7f194b031740 1 Processor -- start 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.637+0000 7f194b031740 1 -- start start 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.637+0000 7f194b031740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68cce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.638+0000 7f194b031740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68e7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.638+0000 7f194b031740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.638+0000 7f194259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.638+0000 7f194259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68cce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58402/0 (socket says 172.21.15.5:58402) 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.638+0000 7f194259c640 1 -- 172.21.15.5:0/3458244579 learned_addr learned my addr 172.21.15.5:0/3458244579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.638+0000 7f194b031740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19441afc00 con 0x5641f68d1b40 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.638+0000 7f194b031740 1 -- 172.21.15.5:0/3458244579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19441ad4d0 con 0x5641f68db370 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.638+0000 7f194b031740 1 -- 172.21.15.5:0/3458244579 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19441ab320 con 0x5641f68d05d0 2024-09-15T22:12:55.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.638+0000 7f1941d9b640 1 --2- 172.21.15.5:0/3458244579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68e7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.639+0000 7f194259c640 1 -- 172.21.15.5:0/3458244579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 msgr2=0x5641f68f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.639+0000 7f194259c640 1 --2- 172.21.15.5:0/3458244579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.639+0000 7f194259c640 1 -- 172.21.15.5:0/3458244579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 msgr2=0x5641f68e7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.639+0000 7f194259c640 1 --2- 172.21.15.5:0/3458244579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68e7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.639+0000 7f194259c640 1 -- 172.21.15.5:0/3458244579 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641f68d6550 con 0x5641f68d05d0 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.639+0000 7f194259c640 1 --2- 172.21.15.5:0/3458244579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68cce10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1934011460 tx=0x7f1934023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e546c4857d0b6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.640+0000 7f192bfff640 1 -- 172.21.15.5:0/3458244579 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193402c070 con 0x5641f68d05d0 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.640+0000 7f192bfff640 1 -- 172.21.15.5:0/3458244579 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f193400fca0 con 0x5641f68d05d0 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.640+0000 7f192bfff640 1 -- 172.21.15.5:0/3458244579 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193400f7d0 con 0x5641f68d05d0 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.640+0000 7f194b031740 1 -- 172.21.15.5:0/3458244579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 msgr2=0x5641f68cce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.640+0000 7f194b031740 1 --2- 172.21.15.5:0/3458244579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68cce10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1934011460 tx=0x7f1934023000 comp rx=0 tx=0).stop 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.640+0000 7f194b031740 1 -- 172.21.15.5:0/3458244579 shutdown_connections 2024-09-15T22:12:55.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.640+0000 7f194b031740 1 --2- 172.21.15.5:0/3458244579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.640+0000 7f194b031740 1 --2- 172.21.15.5:0/3458244579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68e7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.640+0000 7f194b031740 1 --2- 172.21.15.5:0/3458244579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68cce10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.640+0000 7f194b031740 1 -- 172.21.15.5:0/3458244579 >> 172.21.15.5:0/3458244579 conn(0x5641f6899310 msgr2=0x5641f68cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f194b031740 1 -- 172.21.15.5:0/3458244579 shutdown_connections 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f194b031740 1 -- 172.21.15.5:0/3458244579 wait complete. 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f194b031740 1 -- start start 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f194b031740 1 Processor -- start 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f194b031740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f194b031740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f194b031740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68d59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f194b031740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19441ab990 con 0x5641f68d1b40 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f194b031740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19441ace00 con 0x5641f68db370 2024-09-15T22:12:55.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f194b031740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19441ac060 con 0x5641f68d05d0 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f194b031740 10 client.0 _ll_register_callbacks cb 0x7f1934021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f1941d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f1941d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68d5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36354/0 (socket says 172.21.15.5:36354) 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.641+0000 7f1941d9b640 1 -- 172.21.15.5:0/409522715 learned_addr learned my addr 172.21.15.5:0/409522715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f194259c640 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1942d9d640 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68d59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1941d9b640 1 -- 172.21.15.5:0/409522715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 msgr2=0x5641f68d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1941d9b640 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1941d9b640 1 -- 172.21.15.5:0/409522715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 msgr2=0x5641f68d59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1941d9b640 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68d59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1941d9b640 1 -- 172.21.15.5:0/409522715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641f6954640 con 0x5641f68d1b40 2024-09-15T22:12:55.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f194259c640 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1942d9d640 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68d59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:12:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1941d9b640 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68d5470 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f19380b8080 tx=0x7f193803b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d23cc7c5e2ae6e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1929ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:12:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1929ffb640 1 -- 172.21.15.5:0/409522715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938031de0 con 0x5641f68d1b40 2024-09-15T22:12:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1929ffb640 1 -- 172.21.15.5:0/409522715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f19380b8290 con 0x5641f68d1b40 2024-09-15T22:12:55.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f194b031740 1 -- 172.21.15.5:0/409522715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f193400fca0 con 0x5641f68d1b40 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f1929ffb640 1 -- 172.21.15.5:0/409522715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f19380bbc20 con 0x5641f68d1b40 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f194b031740 10 client.16176 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.642+0000 7f194b031740 1 -- 172.21.15.5:0/409522715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5641f6955880 con 0x5641f68d1b40 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.643+0000 7f1929ffb640 1 -- 172.21.15.5:0/409522715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19380d3060 con 0x5641f68d1b40 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.643+0000 7f1929ffb640 1 -- 172.21.15.5:0/409522715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f19380c4350 con 0x5641f68d1b40 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.643+0000 7f194b031740 10 client.16176 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.643+0000 7f194b031740 10 client.16176 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.643+0000 7f194b031740 20 client.16176 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.643+0000 7f194b031740 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5641f6955fb0 0x5641f69583d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.643+0000 7f194b031740 4 client.16176 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.643+0000 7f194b031740 1 -- 172.21.15.5:0/409522715 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f193400fca0 con 0x5641f6955fb0 2024-09-15T22:12:55.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.647+0000 7f194259c640 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5641f6955fb0 0x5641f69583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.647+0000 7f1929ffb640 10 client.16176 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.647+0000 7f194259c640 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5641f6955fb0 0x5641f69583d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5e2966f267da48e server_cookie=4951f28720cc81d4 in_seq=0 out_seq=0 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.647+0000 7f1929ffb640 1 -- 172.21.15.5:0/409522715 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f193400fca0 con 0x5641f6955fb0 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.647+0000 7f1929ffb640 10 client.16176 handle_command_reply: tid=0 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.647+0000 7f194b031740 1 client.16176 shutdown 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.647+0000 7f194b031740 1 -- 172.21.15.5:0/409522715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 msgr2=0x5641f68d5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.647+0000 7f194b031740 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68d5470 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f19380b8080 tx=0x7f193803b640 comp rx=0 tx=0).stop 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f194b031740 1 -- 172.21.15.5:0/409522715 shutdown_connections 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f194b031740 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5641f6955fb0 0x5641f69583d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f194b031740 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5641f68db370 0x5641f68d59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f194b031740 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641f68d1b40 0x5641f68d5470 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f194b031740 1 --2- 172.21.15.5:0/409522715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5641f68d05d0 0x5641f68d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f194b031740 1 -- 172.21.15.5:0/409522715 >> 172.21.15.5:0/409522715 conn(0x5641f6899310 msgr2=0x5641f68d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1929ffb640 0 client.16176 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f194b031740 1 -- 172.21.15.5:0/409522715 shutdown_connections 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f194b031740 1 -- 172.21.15.5:0/409522715 wait complete. 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f194b031740 20 client.16176 trim_cache size 0 max 16384 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1948c58640 1 -- 172.21.15.5:0/2535645996 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f192c0a33b0 msgr2=0x7f192c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1948c58640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f192c0a33b0 0x7f192c0a5870 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f193c009fd0 tx=0x7f193c009500 comp rx=0 tx=0).stop 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1948c58640 1 -- 172.21.15.5:0/2535645996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1944102ef0 msgr2=0x7f1944198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1948c58640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1944102ef0 0x7f1944198c70 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f1938008000 tx=0x7f1938003c10 comp rx=0 tx=0).stop 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1948c58640 1 -- 172.21.15.5:0/2535645996 shutdown_connections 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1948c58640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f192c0a33b0 0x7f192c0a5870 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1948c58640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f194419d000 0x7f194419d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1948c58640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1944102ef0 0x7f1944198c70 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1948c58640 1 --2- 172.21.15.5:0/2535645996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1944100e20 0x7f1944198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:55.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1948c58640 1 -- 172.21.15.5:0/2535645996 >> 172.21.15.5:0/2535645996 conn(0x7f19440fc7d0 msgr2=0x7f194406a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.648+0000 7f1948c58640 1 -- 172.21.15.5:0/2535645996 shutdown_connections 2024-09-15T22:12:55.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:54.649+0000 7f1948c58640 1 -- 172.21.15.5:0/2535645996 wait complete. 2024-09-15T22:12:55.646 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:55 smithi129 ceph-mon[35314]: pgmap v123: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-09-15T22:12:55.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:55 smithi099 ceph-mon[34892]: pgmap v123: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-09-15T22:12:55.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:55 smithi005 ceph-mon[30944]: pgmap v123: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 1.00k op/s 2024-09-15T22:12:57.798 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:57 smithi005 ceph-mon[30944]: pgmap v124: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 1.26k op/s 2024-09-15T22:12:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:57 smithi099 ceph-mon[34892]: pgmap v124: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 1.26k op/s 2024-09-15T22:12:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:57 smithi129 ceph-mon[35314]: pgmap v124: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 17 MiB/s wr, 1.26k op/s 2024-09-15T22:12:58.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:12:58.758 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:12:59.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.344+0000 7f822b203640 1 -- 172.21.15.5:0/2881584082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f821c0a5460 msgr2=0x7f821c0a9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:59.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.344+0000 7f822b203640 1 --2- 172.21.15.5:0/2881584082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f821c0a5460 0x7f821c0a9000 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8224067960 tx=0x7f8224092760 comp rx=0 tx=0).stop 2024-09-15T22:12:59.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.347+0000 7f822b203640 1 -- 172.21.15.5:0/2881584082 shutdown_connections 2024-09-15T22:12:59.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.347+0000 7f822b203640 1 --2- 172.21.15.5:0/2881584082 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f821c0a9730 0x7f821c00b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.347+0000 7f822b203640 1 --2- 172.21.15.5:0/2881584082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f821c0a5460 0x7f821c0a9000 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.347+0000 7f822b203640 1 --2- 172.21.15.5:0/2881584082 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f821c0a4ab0 0x7f821c0a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.347+0000 7f822b203640 1 -- 172.21.15.5:0/2881584082 >> 172.21.15.5:0/2881584082 conn(0x7f821c01a110 msgr2=0x7f821c01a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.351+0000 7f822b203640 1 -- 172.21.15.5:0/2881584082 shutdown_connections 2024-09-15T22:12:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.356+0000 7f822b203640 1 -- 172.21.15.5:0/2881584082 wait complete. 2024-09-15T22:12:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.357+0000 7f822b203640 1 Processor -- start 2024-09-15T22:12:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.357+0000 7f822b203640 1 -- start start 2024-09-15T22:12:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.357+0000 7f822b203640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f821c0a4ab0 0x7f821c0157b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.357+0000 7f822b203640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f821c0a5460 0x7f821c00e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.357+0000 7f822b203640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f821c0a9730 0x7f821c00edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.357+0000 7f822b203640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f821c0cdec0 con 0x7f821c0a4ab0 2024-09-15T22:12:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.357+0000 7f822b203640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f821c0cdd40 con 0x7f821c0a9730 2024-09-15T22:12:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.357+0000 7f822b203640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f821c0ce040 con 0x7f821c0a5460 2024-09-15T22:12:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.358+0000 7f82297a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f821c0a9730 0x7f821c00edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.358+0000 7f82297a0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f821c0a9730 0x7f821c00edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:41322/0 (socket says 172.21.15.5:41322) 2024-09-15T22:12:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.358+0000 7f82297a0640 1 -- 172.21.15.5:0/3054691972 learned_addr learned my addr 172.21.15.5:0/3054691972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.358+0000 7f8228f9f640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f821c0a4ab0 0x7f821c0157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.358+0000 7f82297a0640 1 -- 172.21.15.5:0/3054691972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f821c0a5460 msgr2=0x7f821c00e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:59.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.358+0000 7f82297a0640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f821c0a5460 0x7f821c00e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.358+0000 7f82297a0640 1 -- 172.21.15.5:0/3054691972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f821c0a4ab0 msgr2=0x7f821c0157b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.358+0000 7f82297a0640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f821c0a4ab0 0x7f821c0157b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.358+0000 7f82297a0640 1 -- 172.21.15.5:0/3054691972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c00f3e0 con 0x7f821c0a9730 2024-09-15T22:12:59.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.359+0000 7f8228f9f640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f821c0a4ab0 0x7f821c0157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:12:59.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.361+0000 7f82297a0640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f821c0a9730 0x7f821c00edd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f821000a850 tx=0x7f821000ad10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:59.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.362+0000 7f8221ffb640 1 -- 172.21.15.5:0/3054691972 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82100079b0 con 0x7f821c0a9730 2024-09-15T22:12:59.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.362+0000 7f8221ffb640 1 -- 172.21.15.5:0/3054691972 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8210005500 con 0x7f821c0a9730 2024-09-15T22:12:59.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.365+0000 7f822b203640 1 -- 172.21.15.5:0/3054691972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821c00f6d0 con 0x7f821c0a9730 2024-09-15T22:12:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.365+0000 7f822b203640 1 -- 172.21.15.5:0/3054691972 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821c013600 con 0x7f821c0a9730 2024-09-15T22:12:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.365+0000 7f8221ffb640 1 -- 172.21.15.5:0/3054691972 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210004de0 con 0x7f821c0a9730 2024-09-15T22:12:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.370+0000 7f822d5dc740 1 Processor -- start 2024-09-15T22:12:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.373+0000 7f822d5dc740 1 -- start start 2024-09-15T22:12:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.373+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abf1cf0 0x55b50abf0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.373+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf0780 0x55b50abf0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.373+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 0x55b50abfd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.373+0000 7f822d5dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b50ab562f0 con 0x55b50abf0780 2024-09-15T22:12:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.373+0000 7f822d5dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b50aafdcf0 con 0x55b50abfb520 2024-09-15T22:12:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.373+0000 7f822d5dc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b50ab522b0 con 0x55b50abf1cf0 2024-09-15T22:12:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.373+0000 7f8221ffb640 1 -- 172.21.15.5:0/3054691972 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8210007b50 con 0x7f821c0a9730 2024-09-15T22:12:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.374+0000 7f8221ffb640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f82140a3540 0x7f82140a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.375+0000 7f8221ffb640 1 -- 172.21.15.5:0/3054691972 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f82100cf470 con 0x7f821c0a9730 2024-09-15T22:12:59.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.377+0000 7f82297a0640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f82140a3540 0x7f82140a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.378+0000 7f82297a0640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f82140a3540 0x7f82140a5a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8210097d20 tx=0x7f821000d040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.378+0000 7f8223fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abf1cf0 0x55b50abf0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.379+0000 7f8223fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 0x55b50abfd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.380+0000 7f8223fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abf1cf0 0x55b50abf0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58450/0 (socket says 172.21.15.5:58450) 2024-09-15T22:12:59.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.380+0000 7f8223fff640 1 -- 172.21.15.5:0/513590674 learned_addr learned my addr 172.21.15.5:0/513590674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:59.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.382+0000 7f8228f9f640 1 --2- 172.21.15.5:0/513590674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf0780 0x55b50abf0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.385+0000 7f8223fff640 1 -- 172.21.15.5:0/513590674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abf1cf0 msgr2=0x55b50abf0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.385+0000 7f8223fff640 1 --2- 172.21.15.5:0/513590674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abf1cf0 0x55b50abf0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.385+0000 7f8223fff640 1 -- 172.21.15.5:0/513590674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf0780 msgr2=0x55b50abf0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.385+0000 7f8223fff640 1 --2- 172.21.15.5:0/513590674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf0780 0x55b50abf0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.385+0000 7f8223fff640 1 -- 172.21.15.5:0/513590674 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50aae23a0 con 0x55b50abfb520 2024-09-15T22:12:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.386+0000 7f8223fff640 1 --2- 172.21.15.5:0/513590674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 0x55b50abfd910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82240637b0 tx=0x7f8224065460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a7a4f081e08c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.389+0000 7f820effd640 1 -- 172.21.15.5:0/513590674 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224099560 con 0x55b50abfb520 2024-09-15T22:12:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.389+0000 7f820effd640 1 -- 172.21.15.5:0/513590674 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82240700d0 con 0x55b50abfb520 2024-09-15T22:12:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.390+0000 7f822d5dc740 1 -- 172.21.15.5:0/513590674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 msgr2=0x55b50abfd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.390+0000 7f822d5dc740 1 --2- 172.21.15.5:0/513590674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 0x55b50abfd910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82240637b0 tx=0x7f8224065460 comp rx=0 tx=0).stop 2024-09-15T22:12:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.390+0000 7f822d5dc740 1 -- 172.21.15.5:0/513590674 shutdown_connections 2024-09-15T22:12:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.390+0000 7f822d5dc740 1 --2- 172.21.15.5:0/513590674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 0x55b50abfd910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.390+0000 7f822d5dc740 1 --2- 172.21.15.5:0/513590674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf0780 0x55b50abf0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.390+0000 7f822d5dc740 1 --2- 172.21.15.5:0/513590674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abf1cf0 0x55b50abf0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.390+0000 7f822d5dc740 1 -- 172.21.15.5:0/513590674 >> 172.21.15.5:0/513590674 conn(0x55b50abb9310 msgr2=0x55b50abece10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.397+0000 7f822d5dc740 1 -- 172.21.15.5:0/513590674 shutdown_connections 2024-09-15T22:12:59.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.401+0000 7f822d5dc740 1 -- 172.21.15.5:0/513590674 wait complete. 2024-09-15T22:12:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.401+0000 7f822d5dc740 1 -- start start 2024-09-15T22:12:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.402+0000 7f822d5dc740 1 Processor -- start 2024-09-15T22:12:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.402+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abf0780 0x55b50abfaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.402+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50abf8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.402+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 0x55b50abf86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.402+0000 7f822d5dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b50abfe1b0 con 0x55b50abf1cf0 2024-09-15T22:12:59.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.402+0000 7f822d5dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b50ab11060 con 0x55b50abfb520 2024-09-15T22:12:59.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.402+0000 7f822d5dc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b50abfe330 con 0x55b50abf0780 2024-09-15T22:12:59.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.402+0000 7f822d5dc740 10 client.0 _ll_register_callbacks cb 0x55b50a8648c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.405+0000 7f8223fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 0x55b50abf86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.405+0000 7f8223fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 0x55b50abf86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:41336/0 (socket says 172.21.15.5:41336) 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.405+0000 7f8223fff640 1 -- 172.21.15.5:0/4088135375 learned_addr learned my addr 172.21.15.5:0/4088135375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.405+0000 7f8223fff640 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abf0780 0x55b50abfaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.406+0000 7f8228f9f640 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50abf8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.407+0000 7f8223fff640 1 -- 172.21.15.5:0/4088135375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abf0780 msgr2=0x55b50abfaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.407+0000 7f8223fff640 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abf0780 0x55b50abfaeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.407+0000 7f8223fff640 1 -- 172.21.15.5:0/4088135375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 msgr2=0x55b50abf8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.407+0000 7f8223fff640 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50abf8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.407+0000 7f8223fff640 1 -- 172.21.15.5:0/4088135375 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50a8c73c0 con 0x55b50abfb520 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.407+0000 7f8223fff640 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 0x55b50abf86e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8224050d60 tx=0x7f8224072040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae735fab77512a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:59.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.408+0000 7f8203fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:12:59.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.410+0000 7f8203fff640 1 -- 172.21.15.5:0/4088135375 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82240ab040 con 0x55b50abfb520 2024-09-15T22:12:59.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.410+0000 7f8203fff640 1 -- 172.21.15.5:0/4088135375 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f822406e250 con 0x55b50abfb520 2024-09-15T22:12:59.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.411+0000 7f822d5dc740 1 -- 172.21.15.5:0/4088135375 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82240700d0 con 0x55b50abfb520 2024-09-15T22:12:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.412+0000 7f8203fff640 1 -- 172.21.15.5:0/4088135375 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f822406e5b0 con 0x55b50abfb520 2024-09-15T22:12:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.412+0000 7f822d5dc740 10 client.25775 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:59.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.412+0000 7f822d5dc740 1 -- 172.21.15.5:0/4088135375 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b50abfe1b0 con 0x55b50abfb520 2024-09-15T22:12:59.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.413+0000 7f8203fff640 1 -- 172.21.15.5:0/4088135375 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82240700d0 con 0x55b50abfb520 2024-09-15T22:12:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.414+0000 7f8203fff640 1 -- 172.21.15.5:0/4088135375 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f822406d710 con 0x55b50abfb520 2024-09-15T22:12:59.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.415+0000 7f822d5dc740 10 client.25775 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.415+0000 7f822d5dc740 10 client.25775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.415+0000 7f822d5dc740 20 client.25775 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.415+0000 7f822d5dc740 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b50ac0ae10 0x55b50ac0d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.415+0000 7f822d5dc740 4 client.25775 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:12:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.415+0000 7f822d5dc740 1 -- 172.21.15.5:0/4088135375 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b50ab63860 con 0x55b50ac0ae10 2024-09-15T22:12:59.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.415+0000 7f8228f9f640 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b50ac0ae10 0x55b50ac0d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.417+0000 7f8228f9f640 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b50ac0ae10 0x55b50ac0d200 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b556b227c97287 server_cookie=af5c4a9feae03351 in_seq=0 out_seq=0 2024-09-15T22:12:59.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.417+0000 7f8203fff640 10 client.25775 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:59.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.421+0000 7f8203fff640 1 -- 172.21.15.5:0/4088135375 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b50ab63860 con 0x55b50ac0ae10 2024-09-15T22:12:59.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.421+0000 7f8203fff640 10 client.25775 handle_command_reply: tid=0 2024-09-15T22:12:59.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.422+0000 7f822d5dc740 1 client.25775 shutdown 2024-09-15T22:12:59.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.424+0000 7f822d5dc740 1 -- 172.21.15.5:0/4088135375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 msgr2=0x55b50abf86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:59.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.424+0000 7f822d5dc740 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 0x55b50abf86e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8224050d60 tx=0x7f8224072040 comp rx=0 tx=0).stop 2024-09-15T22:12:59.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.424+0000 7f822d5dc740 1 -- 172.21.15.5:0/4088135375 shutdown_connections 2024-09-15T22:12:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.424+0000 7f822d5dc740 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b50ac0ae10 0x55b50ac0d200 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.424+0000 7f822d5dc740 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abfb520 0x55b50abf86e0 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8224050d60 tx=0x7f8224072040 comp rx=0 tx=0).stop 2024-09-15T22:12:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.424+0000 7f822d5dc740 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50abf8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.424+0000 7f822d5dc740 1 --2- 172.21.15.5:0/4088135375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abf0780 0x55b50abfaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.424+0000 7f822d5dc740 1 -- 172.21.15.5:0/4088135375 >> 172.21.15.5:0/4088135375 conn(0x55b50abb9310 msgr2=0x55b50abf2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.424+0000 7f8203fff640 0 client.25775 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.424+0000 7f822d5dc740 1 -- 172.21.15.5:0/4088135375 shutdown_connections 2024-09-15T22:12:59.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.424+0000 7f822d5dc740 1 -- 172.21.15.5:0/4088135375 wait complete. 2024-09-15T22:12:59.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.425+0000 7f822d5dc740 20 client.25775 trim_cache size 0 max 16384 2024-09-15T22:12:59.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.448+0000 7f822d5dc740 1 Processor -- start 2024-09-15T22:12:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.449+0000 7f822d5dc740 1 -- start start 2024-09-15T22:12:59.451 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:12:59 smithi005 ceph-mon[30944]: pgmap v125: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-09-15T22:12:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.449+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abf0780 0x55b50ac07500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.449+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50abec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.449+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abecbe0 0x55b50ac127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.449+0000 7f822d5dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8208012730 con 0x55b50abf1cf0 2024-09-15T22:12:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.449+0000 7f822d5dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8208010000 con 0x55b50abf0780 2024-09-15T22:12:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.449+0000 7f822d5dc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f820800df00 con 0x55b50abecbe0 2024-09-15T22:12:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.450+0000 7f8228f9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50abec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.450+0000 7f8228f9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50abec6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36404/0 (socket says 172.21.15.5:36404) 2024-09-15T22:12:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.450+0000 7f8228f9f640 1 -- 172.21.15.5:0/245033975 learned_addr learned my addr 172.21.15.5:0/245033975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.450+0000 7f8228f9f640 1 -- 172.21.15.5:0/245033975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abecbe0 msgr2=0x55b50ac127c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:12:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.450+0000 7f8228f9f640 1 --2- 172.21.15.5:0/245033975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abecbe0 0x55b50ac127c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.450+0000 7f8228f9f640 1 -- 172.21.15.5:0/245033975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abf0780 msgr2=0x55b50ac07500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:12:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.450+0000 7f8228f9f640 1 --2- 172.21.15.5:0/245033975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abf0780 0x55b50ac07500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.450+0000 7f8228f9f640 1 -- 172.21.15.5:0/245033975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b50abf32d0 con 0x55b50abf1cf0 2024-09-15T22:12:59.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.450+0000 7f8228f9f640 1 --2- 172.21.15.5:0/245033975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50abec6a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8218000c00 tx=0x7f82180094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c45d84f49f814d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:59.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.451+0000 7f820e7fc640 1 -- 172.21.15.5:0/245033975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180142d0 con 0x55b50abf1cf0 2024-09-15T22:12:59.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.451+0000 7f820e7fc640 1 -- 172.21.15.5:0/245033975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82180035f0 con 0x55b50abf1cf0 2024-09-15T22:12:59.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.451+0000 7f820e7fc640 1 -- 172.21.15.5:0/245033975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821801b590 con 0x55b50abf1cf0 2024-09-15T22:12:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.452+0000 7f822d5dc740 1 -- 172.21.15.5:0/245033975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 msgr2=0x55b50abec6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.452+0000 7f822d5dc740 1 --2- 172.21.15.5:0/245033975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50abec6a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8218000c00 tx=0x7f82180094a0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.453+0000 7f822d5dc740 1 -- 172.21.15.5:0/245033975 shutdown_connections 2024-09-15T22:12:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.453+0000 7f822d5dc740 1 --2- 172.21.15.5:0/245033975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abecbe0 0x55b50ac127c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.453+0000 7f822d5dc740 1 --2- 172.21.15.5:0/245033975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50abec6a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.453+0000 7f822d5dc740 1 --2- 172.21.15.5:0/245033975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abf0780 0x55b50ac07500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.453+0000 7f822d5dc740 1 -- 172.21.15.5:0/245033975 >> 172.21.15.5:0/245033975 conn(0x55b50abb9310 msgr2=0x55b50abfa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.453+0000 7f822d5dc740 1 -- 172.21.15.5:0/245033975 shutdown_connections 2024-09-15T22:12:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.453+0000 7f822d5dc740 1 -- 172.21.15.5:0/245033975 wait complete. 2024-09-15T22:12:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.454+0000 7f822d5dc740 1 -- start start 2024-09-15T22:12:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.454+0000 7f822d5dc740 1 Processor -- start 2024-09-15T22:12:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.454+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abecbe0 0x55b50abffaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.454+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abf0780 0x55b50ac00030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.454+0000 7f822d5dc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50ac74790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.454+0000 7f822d5dc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f820800e4c0 con 0x55b50abf1cf0 2024-09-15T22:12:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.454+0000 7f822d5dc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f820800f930 con 0x55b50abf0780 2024-09-15T22:12:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.454+0000 7f822d5dc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f820800eb90 con 0x55b50abecbe0 2024-09-15T22:12:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.454+0000 7f822d5dc740 10 client.0 _ll_register_callbacks cb 0x7f8218009340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:12:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8223fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abecbe0 0x55b50abffaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8223fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abecbe0 0x55b50abffaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58470/0 (socket says 172.21.15.5:58470) 2024-09-15T22:12:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8223fff640 1 -- 172.21.15.5:0/3877787534 learned_addr learned my addr 172.21.15.5:0/3877787534 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:12:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8228f9f640 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abf0780 0x55b50ac00030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8223fff640 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50ac74790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8228f9f640 1 -- 172.21.15.5:0/3877787534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abecbe0 msgr2=0x55b50abffaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8228f9f640 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abecbe0 0x55b50abffaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8228f9f640 1 -- 172.21.15.5:0/3877787534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 msgr2=0x55b50ac74790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:12:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8228f9f640 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50ac74790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8228f9f640 1 -- 172.21.15.5:0/3877787534 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b50ac74cd0 con 0x55b50abf0780 2024-09-15T22:12:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8228f9f640 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abf0780 0x55b50ac00030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55b50abf6890 tx=0x7f82180140f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f92a0a2c89792cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:12:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.455+0000 7f8203fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:12:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.456+0000 7f8203fff640 1 -- 172.21.15.5:0/3877787534 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218004020 con 0x55b50abf0780 2024-09-15T22:12:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.456+0000 7f8203fff640 1 -- 172.21.15.5:0/3877787534 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f82180041c0 con 0x55b50abf0780 2024-09-15T22:12:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.456+0000 7f822d5dc740 1 -- 172.21.15.5:0/3877787534 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82180035f0 con 0x55b50abf0780 2024-09-15T22:12:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.457+0000 7f8203fff640 1 -- 172.21.15.5:0/3877787534 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8218032050 con 0x55b50abf0780 2024-09-15T22:12:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.457+0000 7f822d5dc740 10 client.25778 fetch_fsmap learned FSMap version 25 2024-09-15T22:12:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.457+0000 7f822d5dc740 1 -- 172.21.15.5:0/3877787534 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b50ac75fa0 con 0x55b50abf0780 2024-09-15T22:12:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.457+0000 7f8203fff640 1 -- 172.21.15.5:0/3877787534 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82180043e0 con 0x55b50abf0780 2024-09-15T22:12:59.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.457+0000 7f8203fff640 1 -- 172.21.15.5:0/3877787534 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8224070280 con 0x55b50abf0780 2024-09-15T22:12:59.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.457+0000 7f822d5dc740 10 client.25778 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:12:59.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.457+0000 7f822d5dc740 10 client.25778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:12:59.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.457+0000 7f822d5dc740 20 client.25778 populate_metadata read hostname 'smithi005' 2024-09-15T22:12:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.458+0000 7f822d5dc740 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b50ac76700 0x55b50ac78b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:12:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.458+0000 7f822d5dc740 4 client.25778 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:12:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.458+0000 7f822d5dc740 1 -- 172.21.15.5:0/3877787534 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f82180035f0 con 0x55b50ac76700 2024-09-15T22:12:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.458+0000 7f8223fff640 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b50ac76700 0x55b50ac78b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:12:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.458+0000 7f8203fff640 10 client.25778 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.458+0000 7f8223fff640 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b50ac76700 0x55b50ac78b20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5dab723bcf3c842 server_cookie=7d0a47a8e7a5fa5b in_seq=0 out_seq=0 2024-09-15T22:12:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f8203fff640 1 -- 172.21.15.5:0/3877787534 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f82180035f0 con 0x55b50ac76700 2024-09-15T22:12:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f8203fff640 10 client.25778 handle_command_reply: tid=0 2024-09-15T22:12:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 1 client.25778 shutdown 2024-09-15T22:12:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 1 -- 172.21.15.5:0/3877787534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abf0780 msgr2=0x55b50ac00030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:59.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abf0780 0x55b50ac00030 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x55b50abf6890 tx=0x7f82180140f0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 1 -- 172.21.15.5:0/3877787534 shutdown_connections 2024-09-15T22:12:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b50abf1cf0 0x55b50ac74790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b50ac76700 0x55b50ac78b20 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b50abf0780 0x55b50ac00030 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 1 --2- 172.21.15.5:0/3877787534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b50abecbe0 0x55b50abffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 1 -- 172.21.15.5:0/3877787534 >> 172.21.15.5:0/3877787534 conn(0x55b50abb9310 msgr2=0x55b50abf22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f8203fff640 0 client.25778 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:12:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 1 -- 172.21.15.5:0/3877787534 shutdown_connections 2024-09-15T22:12:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 1 -- 172.21.15.5:0/3877787534 wait complete. 2024-09-15T22:12:59.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.459+0000 7f822d5dc740 20 client.25778 trim_cache size 0 max 16384 2024-09-15T22:12:59.466 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:12:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.460+0000 7f822b203640 1 -- 172.21.15.5:0/3054691972 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f82140a3540 msgr2=0x7f82140a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.460+0000 7f822b203640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f82140a3540 0x7f82140a5a00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8210097d20 tx=0x7f821000d040 comp rx=0 tx=0).stop 2024-09-15T22:12:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.460+0000 7f822b203640 1 -- 172.21.15.5:0/3054691972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f821c0a9730 msgr2=0x7f821c00edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:12:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.460+0000 7f822b203640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f821c0a9730 0x7f821c00edd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f821000a850 tx=0x7f821000ad10 comp rx=0 tx=0).stop 2024-09-15T22:12:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.462+0000 7f822b203640 1 -- 172.21.15.5:0/3054691972 shutdown_connections 2024-09-15T22:12:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.462+0000 7f822b203640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f82140a3540 0x7f82140a5a00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.462+0000 7f822b203640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f821c0a9730 0x7f821c00edd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.462+0000 7f822b203640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f821c0a5460 0x7f821c00e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.462+0000 7f822b203640 1 --2- 172.21.15.5:0/3054691972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f821c0a4ab0 0x7f821c0157b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:12:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.462+0000 7f822b203640 1 -- 172.21.15.5:0/3054691972 >> 172.21.15.5:0/3054691972 conn(0x7f821c01a110 msgr2=0x7f821c0a81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:12:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.463+0000 7f822b203640 1 -- 172.21.15.5:0/3054691972 shutdown_connections 2024-09-15T22:12:59.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:12:59.463+0000 7f822b203640 1 -- 172.21.15.5:0/3054691972 wait complete. 2024-09-15T22:12:59.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 210.0 s, 2064.0 tps, lat 48.623 ms stddev 56.488 2024-09-15T22:12:59.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:12:59 smithi099 ceph-mon[34892]: pgmap v125: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-09-15T22:12:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:12:59 smithi129 ceph-mon[35314]: pgmap v125: 129 pgs: 129 active+clean; 2.1 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.8 MiB/s wr, 521 op/s 2024-09-15T22:13:01.298 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:13:01.299 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:13:01.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:01 smithi005 ceph-mon[30944]: pgmap v126: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 597 op/s 2024-09-15T22:13:01.631 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:01.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:01 smithi099 ceph-mon[34892]: pgmap v126: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 597 op/s 2024-09-15T22:13:01.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:01 smithi129 ceph-mon[35314]: pgmap v126: 129 pgs: 129 active+clean; 2.1 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 5.8 MiB/s wr, 597 op/s 2024-09-15T22:13:02.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:02 smithi099 ceph-mon[34892]: pgmap v127: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 839 op/s 2024-09-15T22:13:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:02 smithi129 ceph-mon[35314]: pgmap v127: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 839 op/s 2024-09-15T22:13:02.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:02 smithi005 ceph-mon[30944]: pgmap v127: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.5 MiB/s wr, 839 op/s 2024-09-15T22:13:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 215.0 s, 582.0 tps, lat 165.480 ms stddev 239.483 2024-09-15T22:13:05.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:04 smithi099 ceph-mon[34892]: pgmap v128: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 625 op/s 2024-09-15T22:13:05.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:04 smithi129 ceph-mon[35314]: pgmap v128: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 625 op/s 2024-09-15T22:13:05.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:04 smithi005 ceph-mon[30944]: pgmap v128: 129 pgs: 129 active+clean; 2.1 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.5 MiB/s wr, 625 op/s 2024-09-15T22:13:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.281+0000 7f39952ff640 1 -- 172.21.15.5:0/1929523392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3990102d30 msgr2=0x7f3990103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.281+0000 7f39952ff640 1 --2- 172.21.15.5:0/1929523392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3990102d30 0x7f3990103110 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f3980009930 tx=0x7f398002a420 comp rx=0 tx=0).stop 2024-09-15T22:13:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.281+0000 7f39952ff640 1 -- 172.21.15.5:0/1929523392 shutdown_connections 2024-09-15T22:13:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.281+0000 7f39952ff640 1 --2- 172.21.15.5:0/1929523392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39901082a0 0x7f399010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.281+0000 7f39952ff640 1 --2- 172.21.15.5:0/1929523392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39901036e0 0x7f3990107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.281+0000 7f39952ff640 1 --2- 172.21.15.5:0/1929523392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3990102d30 0x7f3990103110 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.281+0000 7f39952ff640 1 -- 172.21.15.5:0/1929523392 >> 172.21.15.5:0/1929523392 conn(0x7f39900fc7d0 msgr2=0x7f39900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.281+0000 7f39952ff640 1 -- 172.21.15.5:0/1929523392 shutdown_connections 2024-09-15T22:13:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.281+0000 7f39952ff640 1 -- 172.21.15.5:0/1929523392 wait complete. 2024-09-15T22:13:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.282+0000 7f39952ff640 1 Processor -- start 2024-09-15T22:13:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.282+0000 7f39952ff640 1 -- start start 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.282+0000 7f39952ff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3990102d30 0x7f3990100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.282+0000 7f39952ff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39901036e0 0x7f3990100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f39952ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39901082a0 0x7f3990101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f39952ff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39901120a0 con 0x7f39901082a0 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f39952ff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3990111f20 con 0x7f39901036e0 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f39952ff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3990112220 con 0x7f3990102d30 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39901082a0 0x7f3990101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39901082a0 0x7f3990101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35400/0 (socket says 172.21.15.5:35400) 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398f7fe640 1 -- 172.21.15.5:0/1586904907 learned_addr learned my addr 172.21.15.5:0/1586904907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398effd640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3990102d30 0x7f3990100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398e7fc640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39901036e0 0x7f3990100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398f7fe640 1 -- 172.21.15.5:0/1586904907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3990102d30 msgr2=0x7f3990100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398f7fe640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3990102d30 0x7f3990100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398f7fe640 1 -- 172.21.15.5:0/1586904907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39901036e0 msgr2=0x7f3990100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398f7fe640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39901036e0 0x7f3990100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398f7fe640 1 -- 172.21.15.5:0/1586904907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39901a6ab0 con 0x7f39901082a0 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398e7fc640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39901036e0 0x7f3990100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f398f7fe640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39901082a0 0x7f3990101410 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f398800efd0 tx=0x7f398800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f3977fff640 1 -- 172.21.15.5:0/1586904907 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3988003b70 con 0x7f39901082a0 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f3977fff640 1 -- 172.21.15.5:0/1586904907 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3988003d10 con 0x7f39901082a0 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f39952ff640 1 -- 172.21.15.5:0/1586904907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39901a6da0 con 0x7f39901082a0 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f3977fff640 1 -- 172.21.15.5:0/1586904907 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39880109f0 con 0x7f39901082a0 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.283+0000 7f39952ff640 1 -- 172.21.15.5:0/1586904907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39901a7260 con 0x7f39901082a0 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.287+0000 7f3977fff640 1 -- 172.21.15.5:0/1586904907 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3988004020 con 0x7f39901082a0 2024-09-15T22:13:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.287+0000 7f39976d8740 1 Processor -- start 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.287+0000 7f39976d8740 1 -- start start 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.287+0000 7f39976d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fc7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.287+0000 7f39976d8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc8480 0x561471fd2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f39976d8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fd2d40 0x561471fd5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f39976d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561471f2d280 con 0x561471fc7b60 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f39976d8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561471ed4b10 con 0x561471fc8480 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f39976d8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561471f29240 con 0x561471fd2d40 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f3977fff640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39780a33b0 0x7f39780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f398effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc8480 0x561471fd2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f398e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fc7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f398effd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc8480 0x561471fd2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:57494/0 (socket says 172.21.15.5:57494) 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f398effd640 1 -- 172.21.15.5:0/181464686 learned_addr learned my addr 172.21.15.5:0/181464686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f398f7fe640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39780a33b0 0x7f39780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f3977fff640 1 -- 172.21.15.5:0/1586904907 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f39880d0530 con 0x7f39901082a0 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f398e7fc640 1 --2- 172.21.15.5:0/181464686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fd2d40 0x561471fd5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f398effd640 1 -- 172.21.15.5:0/181464686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fd2d40 msgr2=0x561471fd5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f398effd640 1 --2- 172.21.15.5:0/181464686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fd2d40 0x561471fd5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f398effd640 1 -- 172.21.15.5:0/181464686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 msgr2=0x561471fc7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f398effd640 1 --2- 172.21.15.5:0/181464686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fc7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.288+0000 7f398effd640 1 -- 172.21.15.5:0/181464686 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561471ee11f0 con 0x561471fc8480 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.289+0000 7f398f7fe640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39780a33b0 0x7f39780a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f398809c170 tx=0x7f3988091900 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.289+0000 7f398effd640 1 --2- 172.21.15.5:0/181464686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc8480 0x561471fd2800 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f398002a940 tx=0x7f3980005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e8790dcb253650a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.289+0000 7f39757fa640 1 -- 172.21.15.5:0/181464686 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980040070 con 0x561471fc8480 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.289+0000 7f39757fa640 1 -- 172.21.15.5:0/181464686 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3980031870 con 0x561471fc8480 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.289+0000 7f39757fa640 1 -- 172.21.15.5:0/181464686 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980035040 con 0x561471fc8480 2024-09-15T22:13:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.290+0000 7f39976d8740 1 -- 172.21.15.5:0/181464686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc8480 msgr2=0x561471fd2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.290+0000 7f39976d8740 1 --2- 172.21.15.5:0/181464686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc8480 0x561471fd2800 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f398002a940 tx=0x7f3980005bd0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.290+0000 7f39976d8740 1 -- 172.21.15.5:0/181464686 shutdown_connections 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.290+0000 7f39976d8740 1 --2- 172.21.15.5:0/181464686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fd2d40 0x561471fd5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.290+0000 7f39976d8740 1 --2- 172.21.15.5:0/181464686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc8480 0x561471fd2800 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.290+0000 7f39976d8740 1 --2- 172.21.15.5:0/181464686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fc7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.290+0000 7f39976d8740 1 -- 172.21.15.5:0/181464686 >> 172.21.15.5:0/181464686 conn(0x561471fc40f0 msgr2=0x561471fc44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.290+0000 7f39976d8740 1 -- 172.21.15.5:0/181464686 shutdown_connections 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.290+0000 7f39976d8740 1 -- 172.21.15.5:0/181464686 wait complete. 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.290+0000 7f39976d8740 1 -- start start 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.290+0000 7f39976d8740 1 Processor -- start 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f39976d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fcfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f39976d8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fd00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f39976d8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fd2d40 0x561471fd0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f39976d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561471fd5910 con 0x561471fc7b60 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f39976d8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561471ee7f10 con 0x561471fd2d40 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f39976d8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561471fd5a90 con 0x561471fc8480 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fd00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fcfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fcfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35416/0 (socket says 172.21.15.5:35416) 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fd00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:37200/0 (socket says 172.21.15.5:37200) 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398e7fc640 1 -- 172.21.15.5:0/1667403413 learned_addr learned my addr 172.21.15.5:0/1667403413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398e7fc640 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fd2d40 0x561471fd0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f39976d8740 10 client.0 _ll_register_callbacks cb 0x561471eb0a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398effd640 1 -- 172.21.15.5:0/1667403413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fd2d40 msgr2=0x561471fd0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398effd640 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fd2d40 0x561471fd0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398effd640 1 -- 172.21.15.5:0/1667403413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 msgr2=0x561471fcfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398effd640 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fcfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398effd640 1 -- 172.21.15.5:0/1667403413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561471c9e6a0 con 0x561471fc8480 2024-09-15T22:13:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.291+0000 7f398e7fc640 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fcfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.292+0000 7f398effd640 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fd00f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3980006010 tx=0x7f3980005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1418161a226fd48 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.292+0000 7f3972ff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:13:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.292+0000 7f3972ff5640 1 -- 172.21.15.5:0/1667403413 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398004d060 con 0x561471fc8480 2024-09-15T22:13:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.292+0000 7f3972ff5640 1 -- 172.21.15.5:0/1667403413 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f3980004240 con 0x561471fc8480 2024-09-15T22:13:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.292+0000 7f39976d8740 1 -- 172.21.15.5:0/1667403413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3980031870 con 0x561471fc8480 2024-09-15T22:13:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.292+0000 7f3972ff5640 1 -- 172.21.15.5:0/1667403413 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3980054050 con 0x561471fc8480 2024-09-15T22:13:05.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.292+0000 7f39976d8740 10 client.25777 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.292+0000 7f39976d8740 1 -- 172.21.15.5:0/1667403413 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561471fd5910 con 0x561471fc8480 2024-09-15T22:13:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.293+0000 7f3972ff5640 1 -- 172.21.15.5:0/1667403413 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3980031870 con 0x561471fc8480 2024-09-15T22:13:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.293+0000 7f3972ff5640 1 -- 172.21.15.5:0/1667403413 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3980003680 con 0x561471fc8480 2024-09-15T22:13:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.293+0000 7f39976d8740 10 client.25777 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.293+0000 7f39976d8740 10 client.25777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.293+0000 7f39976d8740 20 client.25777 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.293+0000 7f39976d8740 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561471fe2660 0x561471fe4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.293+0000 7f39976d8740 4 client.25777 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.293+0000 7f39976d8740 1 -- 172.21.15.5:0/1667403413 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561471f3a7f0 con 0x561471fe2660 2024-09-15T22:13:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.293+0000 7f398e7fc640 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561471fe2660 0x561471fe4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.293+0000 7f398e7fc640 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561471fe2660 0x561471fe4a50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfaa2cc7bc1ebfba server_cookie=84d96414a50fbf3d in_seq=0 out_seq=0 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.293+0000 7f3972ff5640 10 client.25777 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f3972ff5640 1 -- 172.21.15.5:0/1667403413 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561471f3a7f0 con 0x561471fe2660 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f3972ff5640 10 client.25777 handle_command_reply: tid=0 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f39976d8740 1 client.25777 shutdown 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f39976d8740 1 -- 172.21.15.5:0/1667403413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 msgr2=0x561471fd00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f39976d8740 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fd00f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3980006010 tx=0x7f3980005250 comp rx=0 tx=0).stop 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f39976d8740 1 -- 172.21.15.5:0/1667403413 shutdown_connections 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f39976d8740 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561471fe2660 0x561471fe4a50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f39976d8740 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fd2d40 0x561471fd0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f39976d8740 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fd00f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f3972ff5640 0 client.25777 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f39976d8740 1 --2- 172.21.15.5:0/1667403413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fcfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f39976d8740 1 -- 172.21.15.5:0/1667403413 >> 172.21.15.5:0/1667403413 conn(0x561471fc40f0 msgr2=0x561471fc3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f39976d8740 1 -- 172.21.15.5:0/1667403413 shutdown_connections 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.294+0000 7f39976d8740 1 -- 172.21.15.5:0/1667403413 wait complete. 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.295+0000 7f39976d8740 20 client.25777 trim_cache size 0 max 16384 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f39976d8740 1 Processor -- start 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f39976d8740 1 -- start start 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f39976d8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc7b60 0x561471fc9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f39976d8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f39976d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fd2d40 0x561471fea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f39976d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39901b3ae0 con 0x561471fd2d40 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f39976d8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39901b13b0 con 0x561471fc7b60 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f39976d8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39901af200 con 0x561471fc8480 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f398e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fd2d40 0x561471fea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f398effd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f398e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fd2d40 0x561471fea6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35424/0 (socket says 172.21.15.5:35424) 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f398e7fc640 1 -- 172.21.15.5:0/1664049054 learned_addr learned my addr 172.21.15.5:0/1664049054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.325+0000 7f398e7fc640 1 --2- 172.21.15.5:0/1664049054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc7b60 0x561471fc9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.326+0000 7f398e7fc640 1 -- 172.21.15.5:0/1664049054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 msgr2=0x561471fea190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.326+0000 7f398e7fc640 1 --2- 172.21.15.5:0/1664049054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fea190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.326+0000 7f398e7fc640 1 -- 172.21.15.5:0/1664049054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc7b60 msgr2=0x561471fc9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.326+0000 7f398e7fc640 1 --2- 172.21.15.5:0/1664049054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc7b60 0x561471fc9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.326+0000 7f398e7fc640 1 -- 172.21.15.5:0/1664049054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561471fc78e0 con 0x561471fd2d40 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.326+0000 7f398effd640 1 --2- 172.21.15.5:0/1664049054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.326+0000 7f398e7fc640 1 --2- 172.21.15.5:0/1664049054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fd2d40 0x561471fea6d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3984012a40 tx=0x7f3984005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8b793d2b5f055a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.326+0000 7f3974ff9640 1 -- 172.21.15.5:0/1664049054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984031070 con 0x561471fd2d40 2024-09-15T22:13:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.326+0000 7f3974ff9640 1 -- 172.21.15.5:0/1664049054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3984022b70 con 0x561471fd2d40 2024-09-15T22:13:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.326+0000 7f3974ff9640 1 -- 172.21.15.5:0/1664049054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984003ba0 con 0x561471fd2d40 2024-09-15T22:13:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.327+0000 7f39976d8740 1 -- 172.21.15.5:0/1664049054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fd2d40 msgr2=0x561471fea6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.327+0000 7f39976d8740 1 --2- 172.21.15.5:0/1664049054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fd2d40 0x561471fea6d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f3984012a40 tx=0x7f3984005f90 comp rx=0 tx=0).stop 2024-09-15T22:13:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.327+0000 7f39976d8740 1 -- 172.21.15.5:0/1664049054 shutdown_connections 2024-09-15T22:13:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.327+0000 7f39976d8740 1 --2- 172.21.15.5:0/1664049054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fd2d40 0x561471fea6d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.327+0000 7f39976d8740 1 --2- 172.21.15.5:0/1664049054 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fea190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.327+0000 7f39976d8740 1 --2- 172.21.15.5:0/1664049054 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fc7b60 0x561471fc9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.327+0000 7f39976d8740 1 -- 172.21.15.5:0/1664049054 >> 172.21.15.5:0/1664049054 conn(0x561471fc40f0 msgr2=0x561471fc95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:05.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.327+0000 7f39976d8740 1 -- 172.21.15.5:0/1664049054 shutdown_connections 2024-09-15T22:13:05.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.327+0000 7f39976d8740 1 -- 172.21.15.5:0/1664049054 wait complete. 2024-09-15T22:13:05.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.327+0000 7f39976d8740 1 -- start start 2024-09-15T22:13:05.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.327+0000 7f39976d8740 1 Processor -- start 2024-09-15T22:13:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f39976d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fd69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f39976d8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fd6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f39976d8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fd2d40 0x561471fd7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f39976d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39901af870 con 0x561471fc7b60 2024-09-15T22:13:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f39976d8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39901b0ce0 con 0x561471fd2d40 2024-09-15T22:13:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f39976d8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39901aff40 con 0x561471fc8480 2024-09-15T22:13:05.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f39976d8740 10 client.0 _ll_register_callbacks cb 0x7f3984009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:05.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fd69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fd69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35430/0 (socket says 172.21.15.5:35430) 2024-09-15T22:13:05.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398e7fc640 1 -- 172.21.15.5:0/2806273638 learned_addr learned my addr 172.21.15.5:0/2806273638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:05.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398effd640 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fd6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398e7fc640 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fd2d40 0x561471fd7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398e7fc640 1 -- 172.21.15.5:0/2806273638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 msgr2=0x561471fd6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398e7fc640 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fd6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398e7fc640 1 -- 172.21.15.5:0/2806273638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fd2d40 msgr2=0x561471fd7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398e7fc640 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fd2d40 0x561471fd7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398e7fc640 1 -- 172.21.15.5:0/2806273638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56147204c160 con 0x561471fc7b60 2024-09-15T22:13:05.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398effd640 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fd6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f398e7fc640 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fd69f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3984009ec0 tx=0x7f3984014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3236ee229ff1c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f3972ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.328+0000 7f3972ff5640 1 -- 172.21.15.5:0/2806273638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3984031040 con 0x561471fc7b60 2024-09-15T22:13:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f3972ff5640 1 -- 172.21.15.5:0/2806273638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f39840035f0 con 0x561471fc7b60 2024-09-15T22:13:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f39976d8740 1 -- 172.21.15.5:0/2806273638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3984022b70 con 0x561471fc7b60 2024-09-15T22:13:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f3972ff5640 1 -- 172.21.15.5:0/2806273638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3984004040 con 0x561471fc7b60 2024-09-15T22:13:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f39976d8740 10 client.16209 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f39976d8740 1 -- 172.21.15.5:0/2806273638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56147204d400 con 0x561471fc7b60 2024-09-15T22:13:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f3972ff5640 1 -- 172.21.15.5:0/2806273638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3984022b70 con 0x561471fc7b60 2024-09-15T22:13:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f3972ff5640 1 -- 172.21.15.5:0/2806273638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3984037070 con 0x561471fc7b60 2024-09-15T22:13:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f39976d8740 10 client.16209 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f39976d8740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f39976d8740 20 client.16209 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f39976d8740 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56147204da10 0x56147204fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f39976d8740 4 client.16209 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:13:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f39976d8740 1 -- 172.21.15.5:0/2806273638 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561471f34d00 con 0x56147204da10 2024-09-15T22:13:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.329+0000 7f398effd640 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56147204da10 0x56147204fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.330+0000 7f398effd640 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56147204da10 0x56147204fe30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52cf02eb5e56b61d server_cookie=5d0fa59a68455437 in_seq=0 out_seq=0 2024-09-15T22:13:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.330+0000 7f3972ff5640 10 client.16209 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f3972ff5640 1 -- 172.21.15.5:0/2806273638 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561471f34d00 con 0x56147204da10 2024-09-15T22:13:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f3972ff5640 10 client.16209 handle_command_reply: tid=0 2024-09-15T22:13:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 1 client.16209 shutdown 2024-09-15T22:13:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 1 -- 172.21.15.5:0/2806273638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 msgr2=0x561471fd69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fd69f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3984009ec0 tx=0x7f3984014040 comp rx=0 tx=0).stop 2024-09-15T22:13:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 1 -- 172.21.15.5:0/2806273638 shutdown_connections 2024-09-15T22:13:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56147204da10 0x56147204fe30 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f3972ff5640 0 client.16209 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561471fd2d40 0x561471fd7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561471fc8480 0x561471fd6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 1 --2- 172.21.15.5:0/2806273638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561471fc7b60 0x561471fd69f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 1 -- 172.21.15.5:0/2806273638 >> 172.21.15.5:0/2806273638 conn(0x561471fc40f0 msgr2=0x561471fd0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 1 -- 172.21.15.5:0/2806273638 shutdown_connections 2024-09-15T22:13:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 1 -- 172.21.15.5:0/2806273638 wait complete. 2024-09-15T22:13:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.331+0000 7f39976d8740 20 client.16209 trim_cache size 0 max 16384 2024-09-15T22:13:05.201 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:13:05.202 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:13:05.202 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "55e87cf4-148c-45e2-9b9d-584867077259", 2024-09-15T22:13:05.202 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:13:05.202 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:13:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 -- 172.21.15.5:0/1586904907 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39780a33b0 msgr2=0x7f39780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39780a33b0 0x7f39780a5870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f398809c170 tx=0x7f3988091900 comp rx=0 tx=0).stop 2024-09-15T22:13:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 -- 172.21.15.5:0/1586904907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39901082a0 msgr2=0x7f3990101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39901082a0 0x7f3990101410 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f398800efd0 tx=0x7f398800c6a0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 -- 172.21.15.5:0/1586904907 shutdown_connections 2024-09-15T22:13:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39780a33b0 0x7f39780a5870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39901082a0 0x7f3990101410 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39901036e0 0x7f3990100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 --2- 172.21.15.5:0/1586904907 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3990102d30 0x7f3990100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 -- 172.21.15.5:0/1586904907 >> 172.21.15.5:0/1586904907 conn(0x7f39900fc7d0 msgr2=0x7f3990105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 -- 172.21.15.5:0/1586904907 shutdown_connections 2024-09-15T22:13:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:04.332+0000 7f39952ff640 1 -- 172.21.15.5:0/1586904907 wait complete. 2024-09-15T22:13:05.934 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:05 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:13:05.934 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:05 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:05.934 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:05 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:13:05.934 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:05 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:06.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:05 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:13:06.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:05 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:06.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:05 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:13:06.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:05 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:06.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:05 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:13:06.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:05 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:06.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:05 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:13:06.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:05 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:06.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:13:06.944 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:07.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:06 smithi099 ceph-mon[34892]: pgmap v129: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 776 op/s 2024-09-15T22:13:07.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:06 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:13:07.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:06 smithi129 ceph-mon[35314]: pgmap v129: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 776 op/s 2024-09-15T22:13:07.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:06 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:13:07.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:06 smithi005 ceph-mon[30944]: pgmap v129: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 7.5 MiB/s wr, 776 op/s 2024-09-15T22:13:07.109 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:06 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:13:08.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:07 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:13:08.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:07 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:13:08.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:07 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:13:08.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:07 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:13:08.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:07 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:13:08.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:07 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:13:08.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:07 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:13:08.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:07 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:13:08.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:07 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:13:08.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:07 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:13:08.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:07 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:13:08.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:07 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:13:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.737+0000 7fe4f63db640 1 -- 172.21.15.5:0/2539581992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f01047e0 msgr2=0x7fe4f0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.737+0000 7fe4f63db640 1 --2- 172.21.15.5:0/2539581992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f01047e0 0x7fe4f0108830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec013f60 tx=0x7fe4ec0328f0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.737+0000 7fe4f63db640 1 -- 172.21.15.5:0/2539581992 shutdown_connections 2024-09-15T22:13:08.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.737+0000 7fe4f63db640 1 --2- 172.21.15.5:0/2539581992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f0108f60 0x7fe4f0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.737+0000 7fe4f63db640 1 --2- 172.21.15.5:0/2539581992 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f01047e0 0x7fe4f0108830 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec013f60 tx=0x7fe4ec0328f0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.737+0000 7fe4f63db640 1 --2- 172.21.15.5:0/2539581992 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f0103e30 0x7fe4f0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.737+0000 7fe4f63db640 1 -- 172.21.15.5:0/2539581992 >> 172.21.15.5:0/2539581992 conn(0x7fe4f00ff910 msgr2=0x7fe4f0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.739+0000 7fe4f63db640 1 -- 172.21.15.5:0/2539581992 shutdown_connections 2024-09-15T22:13:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.739+0000 7fe4f63db640 1 -- 172.21.15.5:0/2539581992 wait complete. 2024-09-15T22:13:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.740+0000 7fe4f63db640 1 Processor -- start 2024-09-15T22:13:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.740+0000 7fe4f63db640 1 -- start start 2024-09-15T22:13:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.740+0000 7fe4f63db640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f0103e30 0x7fe4f00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.740+0000 7fe4f63db640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f0108f60 0x7fe4f006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.740+0000 7fe4f63db640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f0073f00 0x7fe4f006c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.740+0000 7fe4f63db640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0079010 con 0x7fe4f0073f00 2024-09-15T22:13:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.740+0000 7fe4f63db640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0078e90 con 0x7fe4f0103e30 2024-09-15T22:13:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.740+0000 7fe4f63db640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f0079190 con 0x7fe4f0108f60 2024-09-15T22:13:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.741+0000 7fe4f5bda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f0073f00 0x7fe4f006c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.741+0000 7fe4f4bd8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f0108f60 0x7fe4f006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.741+0000 7fe4f4bd8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f0108f60 0x7fe4f006bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:37232/0 (socket says 172.21.15.5:37232) 2024-09-15T22:13:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.741+0000 7fe4f4bd8640 1 -- 172.21.15.5:0/3805840033 learned_addr learned my addr 172.21.15.5:0/3805840033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.741+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f0103e30 0x7fe4f00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.741+0000 7fe4f5bda640 1 -- 172.21.15.5:0/3805840033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f0108f60 msgr2=0x7fe4f006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.741+0000 7fe4f5bda640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f0108f60 0x7fe4f006bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.741+0000 7fe4f5bda640 1 -- 172.21.15.5:0/3805840033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f0103e30 msgr2=0x7fe4f00739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.741+0000 7fe4f5bda640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f0103e30 0x7fe4f00739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.741+0000 7fe4f5bda640 1 -- 172.21.15.5:0/3805840033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f006c980 con 0x7fe4f0073f00 2024-09-15T22:13:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.745+0000 7fe4f5bda640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f0073f00 0x7fe4f006c150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009fd0 tx=0x7fe4e000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.745+0000 7fe4e67fc640 1 -- 172.21.15.5:0/3805840033 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0004230 con 0x7fe4f0073f00 2024-09-15T22:13:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.746+0000 7fe4f63db640 1 -- 172.21.15.5:0/3805840033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f01ab3d0 con 0x7fe4f0073f00 2024-09-15T22:13:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.746+0000 7fe4f63db640 1 -- 172.21.15.5:0/3805840033 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f01ab910 con 0x7fe4f0073f00 2024-09-15T22:13:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.748+0000 7fe4e67fc640 1 -- 172.21.15.5:0/3805840033 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e000cd90 con 0x7fe4f0073f00 2024-09-15T22:13:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.748+0000 7fe4e67fc640 1 -- 172.21.15.5:0/3805840033 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0004230 con 0x7fe4f0073f00 2024-09-15T22:13:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.750+0000 7fe4f9a16740 1 Processor -- start 2024-09-15T22:13:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.751+0000 7fe4f9a16740 1 -- start start 2024-09-15T22:13:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73670 0x5653dbf73a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73f90 0x5653dbf7e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf7e850 0x5653dbf80c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5653dbed92e0 con 0x5653dbf73f90 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5653dbe80ce0 con 0x5653dbf7e850 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5653dbed52a0 con 0x5653dbf73670 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f53d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73f90 0x5653dbf7e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f53d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73f90 0x5653dbf7e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35474/0 (socket says 172.21.15.5:35474) 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f53d9640 1 -- 172.21.15.5:0/1131116597 learned_addr learned my addr 172.21.15.5:0/1131116597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f4bd8640 1 --2- 172.21.15.5:0/1131116597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf7e850 0x5653dbf80c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f4bd8640 1 --2- 172.21.15.5:0/1131116597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73670 0x5653dbf73a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f53d9640 1 -- 172.21.15.5:0/1131116597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73670 msgr2=0x5653dbf73a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/1131116597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73670 0x5653dbf73a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f53d9640 1 -- 172.21.15.5:0/1131116597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf7e850 msgr2=0x5653dbf80c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/1131116597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf7e850 0x5653dbf80c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.752+0000 7fe4f53d9640 1 -- 172.21.15.5:0/1131116597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dbe65440 con 0x5653dbf73f90 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.753+0000 7fe4e67fc640 1 -- 172.21.15.5:0/3805840033 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe4e00047a0 con 0x7fe4f0073f00 2024-09-15T22:13:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.753+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/1131116597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73f90 0x5653dbf7e310 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80103d0 tx=0x7fe4e8010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e19fecf60e92d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.754+0000 7fe4e67fc640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe4dc0a36b0 0x7fe4dc0a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.754+0000 7fe4f4bd8640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe4dc0a36b0 0x7fe4dc0a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.754+0000 7fe4e67fc640 1 -- 172.21.15.5:0/3805840033 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe4e00cfc10 con 0x7fe4f0073f00 2024-09-15T22:13:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.754+0000 7fe4db7fe640 1 -- 172.21.15.5:0/1131116597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8011700 con 0x5653dbf73f90 2024-09-15T22:13:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.754+0000 7fe4db7fe640 1 -- 172.21.15.5:0/1131116597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e8011d80 con 0x5653dbf73f90 2024-09-15T22:13:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.755+0000 7fe4f9a16740 1 -- 172.21.15.5:0/1131116597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73f90 msgr2=0x5653dbf7e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.755+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/1131116597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73f90 0x5653dbf7e310 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fe4e80103d0 tx=0x7fe4e8010890 comp rx=0 tx=0).stop 2024-09-15T22:13:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.755+0000 7fe4f4bd8640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe4dc0a36b0 0x7fe4dc0a5b70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0110ee0 tx=0x7fe4ec0328f0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.758+0000 7fe4f9a16740 1 -- 172.21.15.5:0/1131116597 shutdown_connections 2024-09-15T22:13:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.758+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/1131116597 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf7e850 0x5653dbf80c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.758+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/1131116597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73f90 0x5653dbf7e310 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.758+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/1131116597 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73670 0x5653dbf73a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.758+0000 7fe4f9a16740 1 -- 172.21.15.5:0/1131116597 >> 172.21.15.5:0/1131116597 conn(0x5653dbf6fc00 msgr2=0x5653dbf6ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.758+0000 7fe4f9a16740 1 -- 172.21.15.5:0/1131116597 shutdown_connections 2024-09-15T22:13:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.758+0000 7fe4f9a16740 1 -- 172.21.15.5:0/1131116597 wait complete. 2024-09-15T22:13:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.758+0000 7fe4f9a16740 1 -- start start 2024-09-15T22:13:08.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.758+0000 7fe4f9a16740 1 Processor -- start 2024-09-15T22:13:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf73670 0x5653dbf7dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73f90 0x5653dbf7b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf7e850 0x5653dbf7b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5653dbf814c0 con 0x5653dbf7e850 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5653dbe94050 con 0x5653dbf73670 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5653dbf81640 con 0x5653dbf73f90 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f9a16740 10 client.0 _ll_register_callbacks cb 0x5653dbed4160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f5bda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf7e850 0x5653dbf7b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f5bda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf7e850 0x5653dbf7b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35476/0 (socket says 172.21.15.5:35476) 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f5bda640 1 -- 172.21.15.5:0/440500251 learned_addr learned my addr 172.21.15.5:0/440500251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf73670 0x5653dbf7dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.759+0000 7fe4f4bd8640 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73f90 0x5653dbf7b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.760+0000 7fe4f5bda640 1 -- 172.21.15.5:0/440500251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73f90 msgr2=0x5653dbf7b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.760+0000 7fe4f5bda640 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73f90 0x5653dbf7b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.760+0000 7fe4f5bda640 1 -- 172.21.15.5:0/440500251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf73670 msgr2=0x5653dbf7dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.760+0000 7fe4f5bda640 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf73670 0x5653dbf7dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.760+0000 7fe4f5bda640 1 -- 172.21.15.5:0/440500251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653dbc4a3c0 con 0x5653dbf7e850 2024-09-15T22:13:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.760+0000 7fe4f5bda640 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf7e850 0x5653dbf7b7c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00984c0 tx=0x7fe4e009c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621ef4aa34276408 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.760+0000 7fe4d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.760+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/440500251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e00cd1e0 con 0x5653dbf7e850 2024-09-15T22:13:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.760+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/440500251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe4e00cd380 con 0x5653dbf7e850 2024-09-15T22:13:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4f9a16740 1 -- 172.21.15.5:0/440500251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4e8011d80 con 0x5653dbf7e850 2024-09-15T22:13:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/440500251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe4e00ba280 con 0x5653dbf7e850 2024-09-15T22:13:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4f9a16740 10 client.16221 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4f9a16740 1 -- 172.21.15.5:0/440500251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5653dbf8cdb0 con 0x5653dbf7e850 2024-09-15T22:13:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/440500251 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e8011d80 con 0x5653dbf7e850 2024-09-15T22:13:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/440500251 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe4e0020030 con 0x5653dbf7e850 2024-09-15T22:13:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4f9a16740 10 client.16221 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4f9a16740 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4f9a16740 20 client.16221 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5653dbf8e1c0 0x5653dbf905b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4f9a16740 4 client.16221 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.761+0000 7fe4f9a16740 1 -- 172.21.15.5:0/440500251 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653dbee6850 con 0x5653dbf8e1c0 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.762+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5653dbf8e1c0 0x5653dbf905b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.762+0000 7fe4d8ff9640 10 client.16221 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.762+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5653dbf8e1c0 0x5653dbf905b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b50591d82d53f4a server_cookie=feb94de0746e3bd0 in_seq=0 out_seq=0 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/440500251 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653dbee6850 con 0x5653dbf8e1c0 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4d8ff9640 10 client.16221 handle_command_reply: tid=0 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4f9a16740 1 client.16221 shutdown 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4f9a16740 1 -- 172.21.15.5:0/440500251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf7e850 msgr2=0x5653dbf7b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf7e850 0x5653dbf7b7c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe4e00984c0 tx=0x7fe4e009c330 comp rx=0 tx=0).stop 2024-09-15T22:13:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4f9a16740 1 -- 172.21.15.5:0/440500251 shutdown_connections 2024-09-15T22:13:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf7e850 0x5653dbf7b7c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73f90 0x5653dbf7b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5653dbf8e1c0 0x5653dbf905b0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/440500251 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf73670 0x5653dbf7dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4f9a16740 1 -- 172.21.15.5:0/440500251 >> 172.21.15.5:0/440500251 conn(0x5653dbf6fc00 msgr2=0x5653dbf758e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.763+0000 7fe4d8ff9640 0 client.16221 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.764+0000 7fe4f9a16740 1 -- 172.21.15.5:0/440500251 shutdown_connections 2024-09-15T22:13:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.764+0000 7fe4f9a16740 1 -- 172.21.15.5:0/440500251 wait complete. 2024-09-15T22:13:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.764+0000 7fe4f9a16740 20 client.16221 trim_cache size 0 max 16384 2024-09-15T22:13:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.794+0000 7fe4f9a16740 1 Processor -- start 2024-09-15T22:13:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.794+0000 7fe4f9a16740 1 -- start start 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf6ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73f90 0x5653dbf95e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf7e850 0x5653dbf963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d4012910 con 0x5653dbf73670 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d40101e0 con 0x5653dbf7e850 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d400e030 con 0x5653dbf73f90 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f53d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf6ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f53d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf6ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35492/0 (socket says 172.21.15.5:35492) 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f53d9640 1 -- 172.21.15.5:0/20899314 learned_addr learned my addr 172.21.15.5:0/20899314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f4bd8640 1 --2- 172.21.15.5:0/20899314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73f90 0x5653dbf95e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f5bda640 1 --2- 172.21.15.5:0/20899314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf7e850 0x5653dbf963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f53d9640 1 -- 172.21.15.5:0/20899314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73f90 msgr2=0x5653dbf95e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/20899314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73f90 0x5653dbf95e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f53d9640 1 -- 172.21.15.5:0/20899314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf7e850 msgr2=0x5653dbf963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/20899314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf7e850 0x5653dbf963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f53d9640 1 -- 172.21.15.5:0/20899314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653dbf7a230 con 0x5653dbf73670 2024-09-15T22:13:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.795+0000 7fe4f4bd8640 1 --2- 172.21.15.5:0/20899314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73f90 0x5653dbf95e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.796+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/20899314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf6ab90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009f40 tx=0x7fe4e8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df65eaebbab021dc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.796+0000 7fe4daffd640 1 -- 172.21.15.5:0/20899314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e802e070 con 0x5653dbf73670 2024-09-15T22:13:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.796+0000 7fe4daffd640 1 -- 172.21.15.5:0/20899314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e8020030 con 0x5653dbf73670 2024-09-15T22:13:08.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.796+0000 7fe4daffd640 1 -- 172.21.15.5:0/20899314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e8011920 con 0x5653dbf73670 2024-09-15T22:13:08.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.797+0000 7fe4f9a16740 1 -- 172.21.15.5:0/20899314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 msgr2=0x5653dbf6ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.797+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/20899314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf6ab90 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8009f40 tx=0x7fe4e8013040 comp rx=0 tx=0).stop 2024-09-15T22:13:08.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.797+0000 7fe4f9a16740 1 -- 172.21.15.5:0/20899314 shutdown_connections 2024-09-15T22:13:08.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.797+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/20899314 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf7e850 0x5653dbf963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.797+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/20899314 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf73f90 0x5653dbf95e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.797+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/20899314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf6ab90 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.797+0000 7fe4f9a16740 1 -- 172.21.15.5:0/20899314 >> 172.21.15.5:0/20899314 conn(0x5653dbf6fc00 msgr2=0x5653dbf7d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.798+0000 7fe4f9a16740 1 -- 172.21.15.5:0/20899314 shutdown_connections 2024-09-15T22:13:08.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.799+0000 7fe4f9a16740 1 -- 172.21.15.5:0/20899314 wait complete. 2024-09-15T22:13:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.800+0000 7fe4f9a16740 1 -- start start 2024-09-15T22:13:08.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.800+0000 7fe4f9a16740 1 Processor -- start 2024-09-15T22:13:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf86b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf73f90 0x5653dbf87070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f9a16740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf7e850 0x5653dbf81900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d400e6a0 con 0x5653dbf73670 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d400fb10 con 0x5653dbf73f90 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f9a16740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d400ed70 con 0x5653dbf7e850 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f9a16740 10 client.0 _ll_register_callbacks cb 0x7fe4e00046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f53d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf86b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f5bda640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf7e850 0x5653dbf81900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f53d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf86b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35496/0 (socket says 172.21.15.5:35496) 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f53d9640 1 -- 172.21.15.5:0/2288693550 learned_addr learned my addr 172.21.15.5:0/2288693550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f5bda640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf7e850 0x5653dbf81900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:37286/0 (socket says 172.21.15.5:37286) 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f4bd8640 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf73f90 0x5653dbf87070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f5bda640 1 -- 172.21.15.5:0/2288693550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf73f90 msgr2=0x5653dbf87070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:08.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f5bda640 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf73f90 0x5653dbf87070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f5bda640 1 -- 172.21.15.5:0/2288693550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 msgr2=0x5653dbf86b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f5bda640 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf86b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f5bda640 1 -- 172.21.15.5:0/2288693550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653dbf82100 con 0x5653dbf7e850 2024-09-15T22:13:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f4bd8640 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf73f90 0x5653dbf87070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.801+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf86b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.802+0000 7fe4f5bda640 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf7e850 0x5653dbf81900 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0004770 tx=0x7fe4e00094f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=453bb3138fe60bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.802+0000 7fe4d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:13:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.802+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/2288693550 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e007f080 con 0x5653dbf7e850 2024-09-15T22:13:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.802+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/2288693550 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe4e00ba420 con 0x5653dbf7e850 2024-09-15T22:13:08.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.802+0000 7fe4f9a16740 1 -- 172.21.15.5:0/2288693550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4e8020030 con 0x5653dbf7e850 2024-09-15T22:13:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.802+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/2288693550 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe4e009b450 con 0x5653dbf7e850 2024-09-15T22:13:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.802+0000 7fe4f9a16740 10 client.25801 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.802+0000 7fe4f9a16740 1 -- 172.21.15.5:0/2288693550 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5653dbff98e0 con 0x5653dbf7e850 2024-09-15T22:13:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.803+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/2288693550 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4e8020030 con 0x5653dbf7e850 2024-09-15T22:13:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.804+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/2288693550 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe4e0020070 con 0x5653dbf7e850 2024-09-15T22:13:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.804+0000 7fe4f9a16740 10 client.25801 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.804+0000 7fe4f9a16740 10 client.25801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.804+0000 7fe4f9a16740 20 client.25801 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.804+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5653dbffa050 0x5653dbffc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.804+0000 7fe4f9a16740 4 client.25801 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:13:08.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.804+0000 7fe4f9a16740 1 -- 172.21.15.5:0/2288693550 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5653dbee0d60 con 0x5653dbffa050 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.804+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5653dbffa050 0x5653dbffc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4d8ff9640 10 client.25801 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4f53d9640 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5653dbffa050 0x5653dbffc470 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739ecbbcae2a18d4 server_cookie=db4c0f434fcf6572 in_seq=0 out_seq=0 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4d8ff9640 1 -- 172.21.15.5:0/2288693550 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5653dbee0d60 con 0x5653dbffa050 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4d8ff9640 10 client.25801 handle_command_reply: tid=0 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4f9a16740 1 client.25801 shutdown 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4f9a16740 1 -- 172.21.15.5:0/2288693550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf7e850 msgr2=0x5653dbf81900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf7e850 0x5653dbf81900 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0004770 tx=0x7fe4e00094f0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4f9a16740 1 -- 172.21.15.5:0/2288693550 shutdown_connections 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5653dbf7e850 0x5653dbf81900 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5653dbffa050 0x5653dbffc470 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4d8ff9640 0 client.25801 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:08.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5653dbf73f90 0x5653dbf87070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.805+0000 7fe4f9a16740 1 --2- 172.21.15.5:0/2288693550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653dbf73670 0x5653dbf86b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.806+0000 7fe4f9a16740 1 -- 172.21.15.5:0/2288693550 >> 172.21.15.5:0/2288693550 conn(0x5653dbf6fc00 msgr2=0x5653dbf6e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.806+0000 7fe4f9a16740 1 -- 172.21.15.5:0/2288693550 shutdown_connections 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.806+0000 7fe4f9a16740 1 -- 172.21.15.5:0/2288693550 wait complete. 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.806+0000 7fe4f9a16740 20 client.25801 trim_cache size 0 max 16384 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:13:08.810 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.808+0000 7fe4f63db640 1 -- 172.21.15.5:0/3805840033 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe4dc0a36b0 msgr2=0x7fe4dc0a5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.808+0000 7fe4f63db640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe4dc0a36b0 0x7fe4dc0a5b70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0110ee0 tx=0x7fe4ec0328f0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.808+0000 7fe4f63db640 1 -- 172.21.15.5:0/3805840033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f0073f00 msgr2=0x7fe4f006c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.808+0000 7fe4f63db640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f0073f00 0x7fe4f006c150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0009fd0 tx=0x7fe4e000eba0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.808+0000 7fe4f63db640 1 -- 172.21.15.5:0/3805840033 shutdown_connections 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.808+0000 7fe4f63db640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe4dc0a36b0 0x7fe4dc0a5b70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.808+0000 7fe4f63db640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4f0073f00 0x7fe4f006c150 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.808+0000 7fe4f63db640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe4f0108f60 0x7fe4f006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.808+0000 7fe4f63db640 1 --2- 172.21.15.5:0/3805840033 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe4f0103e30 0x7fe4f00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.808+0000 7fe4f63db640 1 -- 172.21.15.5:0/3805840033 >> 172.21.15.5:0/3805840033 conn(0x7fe4f00ff910 msgr2=0x7fe4f0107810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.809+0000 7fe4f63db640 1 -- 172.21.15.5:0/3805840033 shutdown_connections 2024-09-15T22:13:08.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:08.809+0000 7fe4f63db640 1 -- 172.21.15.5:0/3805840033 wait complete. 2024-09-15T22:13:08.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:08 smithi005 ceph-mon[30944]: pgmap v130: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 514 op/s 2024-09-15T22:13:09.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:08 smithi099 ceph-mon[34892]: pgmap v130: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 514 op/s 2024-09-15T22:13:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:08 smithi129 ceph-mon[35314]: pgmap v130: 129 pgs: 129 active+clean; 2.1 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 5.1 MiB/s wr, 514 op/s 2024-09-15T22:13:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 220.0 s, 674.0 tps, lat 154.383 ms stddev 402.829 2024-09-15T22:13:10.323 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.8 2024-09-15T22:13:10.323 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:13:11.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:10 smithi099 ceph-mon[34892]: pgmap v131: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.0 MiB/s wr, 761 op/s 2024-09-15T22:13:11.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:10 smithi129 ceph-mon[35314]: pgmap v131: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.0 MiB/s wr, 761 op/s 2024-09-15T22:13:11.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:10 smithi005 ceph-mon[30944]: pgmap v131: 129 pgs: 129 active+clean; 2.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 9.0 MiB/s wr, 761 op/s 2024-09-15T22:13:13.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:12 smithi099 ceph-mon[34892]: pgmap v132: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2024-09-15T22:13:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:12 smithi129 ceph-mon[35314]: pgmap v132: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2024-09-15T22:13:13.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:12 smithi005 ceph-mon[30944]: pgmap v132: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 1.60k op/s 2024-09-15T22:13:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 225.0 s, 1770.2 tps, lat 55.083 ms stddev 77.002 2024-09-15T22:13:15.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:14 smithi099 ceph-mon[34892]: pgmap v133: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-09-15T22:13:15.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:14 smithi129 ceph-mon[35314]: pgmap v133: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-09-15T22:13:15.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:14 smithi005 ceph-mon[30944]: pgmap v133: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 1.31k op/s 2024-09-15T22:13:16.816 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:13:16.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:13:16.929 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:17.065 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:16 smithi005 ceph-mon[30944]: pgmap v134: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-09-15T22:13:17.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:16 smithi099 ceph-mon[34892]: pgmap v134: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-09-15T22:13:17.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:16 smithi129 ceph-mon[35314]: pgmap v134: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 24 MiB/s wr, 1.64k op/s 2024-09-15T22:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.479+0000 7efdd3fff640 1 -- 172.21.15.5:0/3517611543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd406ea30 msgr2=0x7efdd406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.479+0000 7efdd3fff640 1 --2- 172.21.15.5:0/3517611543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd406ea30 0x7efdd406ee70 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009860 tx=0x7efdc802a340 comp rx=0 tx=0).stop 2024-09-15T22:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.479+0000 7efdd3fff640 1 -- 172.21.15.5:0/3517611543 shutdown_connections 2024-09-15T22:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.479+0000 7efdd3fff640 1 --2- 172.21.15.5:0/3517611543 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd411b140 0x7efdd411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.479+0000 7efdd3fff640 1 --2- 172.21.15.5:0/3517611543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd406ea30 0x7efdd406ee70 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.479+0000 7efdd3fff640 1 --2- 172.21.15.5:0/3517611543 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdd4110190 0x7efdd406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.479+0000 7efdd3fff640 1 -- 172.21.15.5:0/3517611543 >> 172.21.15.5:0/3517611543 conn(0x7efdd4073180 msgr2=0x7efdd4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.481+0000 7efdd3fff640 1 -- 172.21.15.5:0/3517611543 shutdown_connections 2024-09-15T22:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.481+0000 7efdd3fff640 1 -- 172.21.15.5:0/3517611543 wait complete. 2024-09-15T22:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.482+0000 7efdd3fff640 1 Processor -- start 2024-09-15T22:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.482+0000 7efdd3fff640 1 -- start start 2024-09-15T22:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.482+0000 7efdd3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd406ea30 0x7efdd4119770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.483+0000 7efdd3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4110190 0x7efdd4112800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.483+0000 7efdd3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdd411b140 0x7efdd4112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.483+0000 7efdd3fff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdd4120bf0 con 0x7efdd406ea30 2024-09-15T22:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.483+0000 7efdd3fff640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efdd4120a70 con 0x7efdd411b140 2024-09-15T22:13:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.483+0000 7efdd3fff640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efdd4120d70 con 0x7efdd4110190 2024-09-15T22:13:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.483+0000 7efdd37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdd411b140 0x7efdd4112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.484+0000 7efdd37fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdd411b140 0x7efdd4112d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:56534/0 (socket says 172.21.15.5:56534) 2024-09-15T22:13:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.484+0000 7efdd37fe640 1 -- 172.21.15.5:0/2182371076 learned_addr learned my addr 172.21.15.5:0/2182371076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.484+0000 7efdd27fc640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4110190 0x7efdd4112800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.484+0000 7efdd27fc640 1 -- 172.21.15.5:0/2182371076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdd411b140 msgr2=0x7efdd4112d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.484+0000 7efdd27fc640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdd411b140 0x7efdd4112d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.484+0000 7efdd27fc640 1 -- 172.21.15.5:0/2182371076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd406ea30 msgr2=0x7efdd4119770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.484+0000 7efdd27fc640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd406ea30 0x7efdd4119770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.484+0000 7efdd27fc640 1 -- 172.21.15.5:0/2182371076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdd4113350 con 0x7efdd4110190 2024-09-15T22:13:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.485+0000 7efdd27fc640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4110190 0x7efdd4112800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efdc802a860 tx=0x7efdc8033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.485+0000 7efdbbfff640 1 -- 172.21.15.5:0/2182371076 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8037040 con 0x7efdd4110190 2024-09-15T22:13:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.485+0000 7efdbbfff640 1 -- 172.21.15.5:0/2182371076 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdc8030550 con 0x7efdd4110190 2024-09-15T22:13:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.486+0000 7efdbbfff640 1 -- 172.21.15.5:0/2182371076 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8003b90 con 0x7efdd4110190 2024-09-15T22:13:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.486+0000 7efdd3fff640 1 -- 172.21.15.5:0/2182371076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdd41135e0 con 0x7efdd4110190 2024-09-15T22:13:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.486+0000 7efdd3fff640 1 -- 172.21.15.5:0/2182371076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdd4117530 con 0x7efdd4110190 2024-09-15T22:13:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.488+0000 7efdd3fff640 1 -- 172.21.15.5:0/2182371076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdd41096f0 con 0x7efdd4110190 2024-09-15T22:13:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.490+0000 7efdbbfff640 1 -- 172.21.15.5:0/2182371076 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7efdc8042020 con 0x7efdd4110190 2024-09-15T22:13:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.491+0000 7efdbbfff640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efdbc0a34b0 0x7efdbc0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.491+0000 7efdd2ffd640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efdbc0a34b0 0x7efdbc0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.491+0000 7efdbbfff640 1 -- 172.21.15.5:0/2182371076 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7efdc80ef760 con 0x7efdd4110190 2024-09-15T22:13:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.492+0000 7efdd2ffd640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efdbc0a34b0 0x7efdbc0a5970 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efdc4005fd0 tx=0x7efdc4005a00 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:17.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.495+0000 7efdbbfff640 1 -- 172.21.15.5:0/2182371076 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efdc8049020 con 0x7efdd4110190 2024-09-15T22:13:17.540 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:17.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.812+0000 7efdd3fff640 1 -- 172.21.15.5:0/2182371076 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efdd406ee70 con 0x7efdd4110190 2024-09-15T22:13:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.813+0000 7efdbbfff640 1 -- 172.21.15.5:0/2182371076 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7efdc80bbe50 con 0x7efdd4110190 2024-09-15T22:13:17.815 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:13:17.815 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:13:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:13:17.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.815+0000 7efdd3fff640 1 -- 172.21.15.5:0/2182371076 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efdbc0a34b0 msgr2=0x7efdbc0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.815+0000 7efdd3fff640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efdbc0a34b0 0x7efdbc0a5970 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efdc4005fd0 tx=0x7efdc4005a00 comp rx=0 tx=0).stop 2024-09-15T22:13:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.815+0000 7efdd3fff640 1 -- 172.21.15.5:0/2182371076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4110190 msgr2=0x7efdd4112800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.815+0000 7efdd3fff640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4110190 0x7efdd4112800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efdc802a860 tx=0x7efdc8033d90 comp rx=0 tx=0).stop 2024-09-15T22:13:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.815+0000 7efdd3fff640 1 -- 172.21.15.5:0/2182371076 shutdown_connections 2024-09-15T22:13:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.815+0000 7efdd3fff640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efdbc0a34b0 0x7efdbc0a5970 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.815+0000 7efdd3fff640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdd411b140 0x7efdd4112d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.815+0000 7efdd3fff640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdd4110190 0x7efdd4112800 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.815+0000 7efdd3fff640 1 --2- 172.21.15.5:0/2182371076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdd406ea30 0x7efdd4119770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.815+0000 7efdd3fff640 1 -- 172.21.15.5:0/2182371076 >> 172.21.15.5:0/2182371076 conn(0x7efdd4073180 msgr2=0x7efdd406b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.816+0000 7efdd3fff640 1 -- 172.21.15.5:0/2182371076 shutdown_connections 2024-09-15T22:13:17.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:17.816+0000 7efdd3fff640 1 -- 172.21.15.5:0/2182371076 wait complete. 2024-09-15T22:13:18.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.707+0000 7fabd3cd5640 1 -- 172.21.15.5:0/289990725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabcc101450 msgr2=0x7fabcc105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:18.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.707+0000 7fabd3cd5640 1 --2- 172.21.15.5:0/289990725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabcc101450 0x7fabcc105920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009860 tx=0x7fabbc02a410 comp rx=0 tx=0).stop 2024-09-15T22:13:18.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.707+0000 7fabd3cd5640 1 -- 172.21.15.5:0/289990725 shutdown_connections 2024-09-15T22:13:18.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.707+0000 7fabd3cd5640 1 --2- 172.21.15.5:0/289990725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabcc105f40 0x7fabcc1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:18.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.707+0000 7fabd3cd5640 1 --2- 172.21.15.5:0/289990725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabcc101450 0x7fabcc105920 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:18.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.707+0000 7fabd3cd5640 1 --2- 172.21.15.5:0/289990725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabcc100b30 0x7fabcc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:18.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.707+0000 7fabd3cd5640 1 -- 172.21.15.5:0/289990725 >> 172.21.15.5:0/289990725 conn(0x7fabcc0fc7a0 msgr2=0x7fabcc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.708+0000 7fabd3cd5640 1 -- 172.21.15.5:0/289990725 shutdown_connections 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.708+0000 7fabd3cd5640 1 -- 172.21.15.5:0/289990725 wait complete. 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.709+0000 7fabd3cd5640 1 Processor -- start 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.709+0000 7fabd3cd5640 1 -- start start 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.709+0000 7fabd3cd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabcc100b30 0x7fabcc073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.709+0000 7fabd3cd5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabcc101450 0x7fabcc073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.709+0000 7fabd3cd5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabcc105f40 0x7fabcc071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.709+0000 7fabd3cd5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc107940 con 0x7fabcc100b30 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.709+0000 7fabd3cd5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1077c0 con 0x7fabcc105f40 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.709+0000 7fabd3cd5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc107ac0 con 0x7fabcc101450 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.709+0000 7fabd1270640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabcc101450 0x7fabcc073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.709+0000 7fabd1270640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabcc101450 0x7fabcc073ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:47126/0 (socket says 172.21.15.5:47126) 2024-09-15T22:13:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.709+0000 7fabd1270640 1 -- 172.21.15.5:0/803861312 learned_addr learned my addr 172.21.15.5:0/803861312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.710+0000 7fabd2272640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabcc105f40 0x7fabcc071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.710+0000 7fabd1a71640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabcc100b30 0x7fabcc073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.710+0000 7fabd2272640 1 -- 172.21.15.5:0/803861312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabcc101450 msgr2=0x7fabcc073ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.710+0000 7fabd2272640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabcc101450 0x7fabcc073ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.710+0000 7fabd2272640 1 -- 172.21.15.5:0/803861312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabcc100b30 msgr2=0x7fabcc073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.710+0000 7fabd2272640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabcc100b30 0x7fabcc073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.710+0000 7fabd2272640 1 -- 172.21.15.5:0/803861312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabcc072540 con 0x7fabcc105f40 2024-09-15T22:13:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.713+0000 7fabd2272640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabcc105f40 0x7fabcc071c80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009fd0 tx=0x7fabc800eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.714+0000 7fabc2ffd640 1 -- 172.21.15.5:0/803861312 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8004230 con 0x7fabcc105f40 2024-09-15T22:13:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.714+0000 7fabc2ffd640 1 -- 172.21.15.5:0/803861312 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc800cd90 con 0x7fabcc105f40 2024-09-15T22:13:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.714+0000 7fabc2ffd640 1 -- 172.21.15.5:0/803861312 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8004a00 con 0x7fabcc105f40 2024-09-15T22:13:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.714+0000 7fabd3cd5640 1 -- 172.21.15.5:0/803861312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabcc072830 con 0x7fabcc105f40 2024-09-15T22:13:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.714+0000 7fabd3cd5640 1 -- 172.21.15.5:0/803861312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabcc1134a0 con 0x7fabcc105f40 2024-09-15T22:13:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.720+0000 7fabd3cd5640 1 -- 172.21.15.5:0/803861312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabcc105920 con 0x7fabcc105f40 2024-09-15T22:13:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.721+0000 7fabc2ffd640 1 -- 172.21.15.5:0/803861312 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fabc800c8b0 con 0x7fabcc105f40 2024-09-15T22:13:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.722+0000 7fabc2ffd640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fabb80a3580 0x7fabb80a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.722+0000 7fabc2ffd640 1 -- 172.21.15.5:0/803861312 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fabc80d0140 con 0x7fabcc105f40 2024-09-15T22:13:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.724+0000 7fabd1a71640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fabb80a3580 0x7fabb80a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.728+0000 7fabc2ffd640 1 -- 172.21.15.5:0/803861312 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabc809c720 con 0x7fabcc105f40 2024-09-15T22:13:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:18.729+0000 7fabd1a71640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fabb80a3580 0x7fabb80a5a40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009f60 tx=0x7fabc40094f0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:18.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:18 smithi005 ceph-mon[30944]: pgmap v135: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2024-09-15T22:13:18.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:18 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2182371076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:18.907 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0' 2024-09-15T22:13:18.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 133.8 --await-for 45.0 2024-09-15T22:13:19.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.082+0000 7fabd3cd5640 1 -- 172.21.15.5:0/803861312 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fabcc1137c0 con 0x7fabcc105f40 2024-09-15T22:13:19.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.083+0000 7fabc2ffd640 1 -- 172.21.15.5:0/803861312 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fabc809c020 con 0x7fabcc105f40 2024-09-15T22:13:19.085 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:13:19.085 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:13:19.085 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:13:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 -- 172.21.15.5:0/803861312 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fabb80a3580 msgr2=0x7fabb80a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fabb80a3580 0x7fabb80a5a40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fabc4009f60 tx=0x7fabc40094f0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 -- 172.21.15.5:0/803861312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabcc105f40 msgr2=0x7fabcc071c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabcc105f40 0x7fabcc071c80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009fd0 tx=0x7fabc800eba0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 -- 172.21.15.5:0/803861312 shutdown_connections 2024-09-15T22:13:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fabb80a3580 0x7fabb80a5a40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fabcc105f40 0x7fabcc071c80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fabcc101450 0x7fabcc073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 --2- 172.21.15.5:0/803861312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabcc100b30 0x7fabcc073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 -- 172.21.15.5:0/803861312 >> 172.21.15.5:0/803861312 conn(0x7fabcc0fc7a0 msgr2=0x7fabcc108100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:19.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 -- 172.21.15.5:0/803861312 shutdown_connections 2024-09-15T22:13:19.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:19.085+0000 7fabd3cd5640 1 -- 172.21.15.5:0/803861312 wait complete. 2024-09-15T22:13:19.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:18 smithi099 ceph-mon[34892]: pgmap v135: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2024-09-15T22:13:19.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:18 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2182371076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:19.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:18 smithi129 ceph-mon[35314]: pgmap v135: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 MiB/s wr, 1.49k op/s 2024-09-15T22:13:19.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:18 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2182371076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:19.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 230.0 s, 1518.2 tps, lat 67.009 ms stddev 92.405 2024-09-15T22:13:19.826 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:13:19.888 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:20.019 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:19 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/803861312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:20.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:19 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/803861312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:20.094 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:19 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/803861312' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:20.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.537+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/1781169241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd81011e0 msgr2=0x7f9dd8105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.537+0000 7f9ddd9ba640 1 --2- 172.21.15.5:0/1781169241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd81011e0 0x7f9dd8105980 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8009930 tx=0x7f9dc802a460 comp rx=0 tx=0).stop 2024-09-15T22:13:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.537+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/1781169241 shutdown_connections 2024-09-15T22:13:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.537+0000 7f9ddd9ba640 1 --2- 172.21.15.5:0/1781169241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dd8105ec0 0x7f9dd8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.537+0000 7f9ddd9ba640 1 --2- 172.21.15.5:0/1781169241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd81011e0 0x7f9dd8105980 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.537+0000 7f9ddd9ba640 1 --2- 172.21.15.5:0/1781169241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd81008c0 0x7f9dd8100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.537+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/1781169241 >> 172.21.15.5:0/1781169241 conn(0x7f9dd80fc7d0 msgr2=0x7f9dd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.537+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/1781169241 shutdown_connections 2024-09-15T22:13:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.537+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/1781169241 wait complete. 2024-09-15T22:13:20.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.538+0000 7f9ddd9ba640 1 Processor -- start 2024-09-15T22:13:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.538+0000 7f9ddd9ba640 1 -- start start 2024-09-15T22:13:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.539+0000 7f9ddd9ba640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd81008c0 0x7f9dd81a4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.539+0000 7f9ddd9ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd81011e0 0x7f9dd81a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.539+0000 7f9ddd9ba640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dd8105ec0 0x7f9dd819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.539+0000 7f9dd67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd81011e0 0x7f9dd81a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.539+0000 7f9dd67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd81011e0 0x7f9dd81a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56370/0 (socket says 172.21.15.5:56370) 2024-09-15T22:13:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.539+0000 7f9dd67fc640 1 -- 172.21.15.5:0/4118936438 learned_addr learned my addr 172.21.15.5:0/4118936438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:20.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.539+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/4118936438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd81186b0 con 0x7f9dd81011e0 2024-09-15T22:13:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.539+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/4118936438 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd8118530 con 0x7f9dd81008c0 2024-09-15T22:13:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.539+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/4118936438 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd8118830 con 0x7f9dd8105ec0 2024-09-15T22:13:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.540+0000 7f9dd77fe640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dd8105ec0 0x7f9dd819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.541+0000 7f9dd67fc640 1 -- 172.21.15.5:0/4118936438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dd8105ec0 msgr2=0x7f9dd819ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.541+0000 7f9dd67fc640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dd8105ec0 0x7f9dd819ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.541+0000 7f9dd67fc640 1 -- 172.21.15.5:0/4118936438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd81008c0 msgr2=0x7f9dd81a4840 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:13:20.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.541+0000 7f9dd67fc640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd81008c0 0x7f9dd81a4840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.541+0000 7f9dd67fc640 1 -- 172.21.15.5:0/4118936438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd819f130 con 0x7f9dd81011e0 2024-09-15T22:13:20.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.541+0000 7f9dd67fc640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd81011e0 0x7f9dd81a4d80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8005880 tx=0x7f9dc80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.542+0000 7f9ddc9b8640 1 -- 172.21.15.5:0/4118936438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8040070 con 0x7f9dd81011e0 2024-09-15T22:13:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.542+0000 7f9ddc9b8640 1 -- 172.21.15.5:0/4118936438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dc8031c80 con 0x7f9dd81011e0 2024-09-15T22:13:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.542+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/4118936438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd819f3c0 con 0x7f9dd81011e0 2024-09-15T22:13:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.542+0000 7f9ddc9b8640 1 -- 172.21.15.5:0/4118936438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc8035020 con 0x7f9dd81011e0 2024-09-15T22:13:20.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.542+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/4118936438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd81ab710 con 0x7f9dd81011e0 2024-09-15T22:13:20.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.546+0000 7f9ddc9b8640 1 -- 172.21.15.5:0/4118936438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9dc8043070 con 0x7f9dd81011e0 2024-09-15T22:13:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9ddfd93740 1 Processor -- start 2024-09-15T22:13:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9ddfd93740 1 -- start start 2024-09-15T22:13:20.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9ddc9b8640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9dc00a3500 0x7f9dc00a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 0x5583924ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924cf080 0x5583924d9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 0x5583924dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5583924342a0 con 0x5583924cf080 2024-09-15T22:13:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5583923dbb10 con 0x5583924ce760 2024-09-15T22:13:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558392430260 con 0x5583924d9940 2024-09-15T22:13:20.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9ddc9b8640 1 -- 172.21.15.5:0/4118936438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f9dc80f0a00 con 0x7f9dd81011e0 2024-09-15T22:13:20.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9dd77fe640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9dc00a3500 0x7f9dc00a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9dd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 0x5583924dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9dd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 0x5583924dbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:47156/0 (socket says 172.21.15.5:47156) 2024-09-15T22:13:20.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9dd77fe640 1 -- 172.21.15.5:0/165260966 learned_addr learned my addr 172.21.15.5:0/165260966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:20.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.547+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/165260966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 0x5583924ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.548+0000 7f9dd67fc640 1 --2- 172.21.15.5:0/165260966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924cf080 0x5583924d9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.548+0000 7f9dd77fe640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9dc00a3500 0x7f9dc00a59c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009e70 tx=0x7f9dcc0156d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.548+0000 7f9dd6ffd640 1 -- 172.21.15.5:0/165260966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 msgr2=0x5583924dbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.548+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/165260966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 0x5583924dbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.548+0000 7f9dd6ffd640 1 -- 172.21.15.5:0/165260966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924cf080 msgr2=0x5583924d9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.548+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/165260966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924cf080 0x5583924d9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.548+0000 7f9dd6ffd640 1 -- 172.21.15.5:0/165260966 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583921a5b90 con 0x5583924ce760 2024-09-15T22:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.548+0000 7f9dd77fe640 1 --2- 172.21.15.5:0/165260966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 0x5583924dbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.548+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/165260966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 0x5583924ceb40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0005fd0 tx=0x7f9dd000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a43479ea5197df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.548+0000 7f9dbdffb640 1 -- 172.21.15.5:0/165260966 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0004b20 con 0x5583924ce760 2024-09-15T22:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.549+0000 7f9dbdffb640 1 -- 172.21.15.5:0/165260966 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dd0003e50 con 0x5583924ce760 2024-09-15T22:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.549+0000 7f9dbdffb640 1 -- 172.21.15.5:0/165260966 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0005020 con 0x5583924ce760 2024-09-15T22:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.550+0000 7f9ddfd93740 1 -- 172.21.15.5:0/165260966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 msgr2=0x5583924ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.550+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/165260966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 0x5583924ceb40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0005fd0 tx=0x7f9dd000e720 comp rx=0 tx=0).stop 2024-09-15T22:13:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.551+0000 7f9ddfd93740 1 -- 172.21.15.5:0/165260966 shutdown_connections 2024-09-15T22:13:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.551+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/165260966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 0x5583924dbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.551+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/165260966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924cf080 0x5583924d9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.551+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/165260966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 0x5583924ceb40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.551+0000 7f9ddfd93740 1 -- 172.21.15.5:0/165260966 >> 172.21.15.5:0/165260966 conn(0x5583924cacf0 msgr2=0x5583924cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.551+0000 7f9ddfd93740 1 -- 172.21.15.5:0/165260966 shutdown_connections 2024-09-15T22:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.551+0000 7f9ddfd93740 1 -- 172.21.15.5:0/165260966 wait complete. 2024-09-15T22:13:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.552+0000 7f9ddfd93740 1 -- start start 2024-09-15T22:13:20.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.552+0000 7f9ddfd93740 1 Processor -- start 2024-09-15T22:13:20.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.552+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924ce760 0x5583924d67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924cf080 0x5583924d6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924d9940 0x5583924d7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5583924dc510 con 0x5583924ce760 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5583923bc6e0 con 0x5583924d9940 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9dd67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924cf080 0x5583924d6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9dd67fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924cf080 0x5583924d6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:47158/0 (socket says 172.21.15.5:47158) 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9dd67fc640 1 -- 172.21.15.5:0/1894548567 learned_addr learned my addr 172.21.15.5:0/1894548567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9dd77fe640 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924d9940 0x5583924d7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924ce760 0x5583924d67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5583924dc690 con 0x5583924cf080 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9dd67fc640 1 -- 172.21.15.5:0/1894548567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924d9940 msgr2=0x5583924d7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9dd67fc640 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924d9940 0x5583924d7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9dd67fc640 1 -- 172.21.15.5:0/1894548567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924ce760 msgr2=0x5583924d67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9dd67fc640 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924ce760 0x5583924d67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.553+0000 7f9dd67fc640 1 -- 172.21.15.5:0/1894548567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583924e59e0 con 0x5583924cf080 2024-09-15T22:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.554+0000 7f9ddfd93740 10 client.0 _ll_register_callbacks cb 0x5583923e0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.554+0000 7f9dd67fc640 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924cf080 0x5583924d6ce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8006010 tx=0x7f9dc80bc500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4008184a26c62bab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.554+0000 7f9ddfd93740 1 -- 172.21.15.5:0/1894548567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd0003e50 con 0x5583924cf080 2024-09-15T22:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.555+0000 7f9db6ffd640 10 client.25816 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.555+0000 7f9db6ffd640 1 -- 172.21.15.5:0/1894548567 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc80d4020 con 0x5583924cf080 2024-09-15T22:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.555+0000 7f9db6ffd640 1 -- 172.21.15.5:0/1894548567 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f9dc80db120 con 0x5583924cf080 2024-09-15T22:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.555+0000 7f9db6ffd640 1 -- 172.21.15.5:0/1894548567 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9dc80d0e60 con 0x5583924cf080 2024-09-15T22:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.555+0000 7f9ddfd93740 10 client.25816 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.555+0000 7f9ddfd93740 1 -- 172.21.15.5:0/1894548567 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5583924e7fe0 con 0x5583924cf080 2024-09-15T22:13:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.556+0000 7f9db6ffd640 1 -- 172.21.15.5:0/1894548567 <== mon.1 v2:172.21.15.129:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9dc803a040 con 0x5583924cf080 2024-09-15T22:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.556+0000 7f9ddfd93740 10 client.25816 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.556+0000 7f9ddfd93740 10 client.25816 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.556+0000 7f9ddfd93740 20 client.25816 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.556+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583924e9810 0x5583924ebcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.556+0000 7f9ddfd93740 4 client.25816 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.556+0000 7f9ddfd93740 1 -- 172.21.15.5:0/1894548567 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583924419d0 con 0x5583924e9810 2024-09-15T22:13:20.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.556+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583924e9810 0x5583924ebcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.557+0000 7f9db6ffd640 10 client.25816 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.557+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583924e9810 0x5583924ebcd0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fa3ff86712c0f43 server_cookie=4044e4c9d4fe8744 in_seq=0 out_seq=0 2024-09-15T22:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.557+0000 7f9db6ffd640 1 -- 172.21.15.5:0/1894548567 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583924419d0 con 0x5583924e9810 2024-09-15T22:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.557+0000 7f9db6ffd640 10 client.25816 handle_command_reply: tid=0 2024-09-15T22:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.557+0000 7f9ddfd93740 1 client.25816 shutdown 2024-09-15T22:13:20.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.557+0000 7f9ddfd93740 1 -- 172.21.15.5:0/1894548567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924cf080 msgr2=0x5583924d6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.557+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924cf080 0x5583924d6ce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8006010 tx=0x7f9dc80bc500 comp rx=0 tx=0).stop 2024-09-15T22:13:20.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.560+0000 7f9ddfd93740 1 -- 172.21.15.5:0/1894548567 shutdown_connections 2024-09-15T22:13:20.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.560+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5583924e9810 0x5583924ebcd0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.560+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924d9940 0x5583924d7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.560+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924cf080 0x5583924d6ce0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.560+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/1894548567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924ce760 0x5583924d67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.560+0000 7f9db6ffd640 0 client.25816 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:20.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.560+0000 7f9ddfd93740 1 -- 172.21.15.5:0/1894548567 >> 172.21.15.5:0/1894548567 conn(0x5583924cacf0 msgr2=0x5583924d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.560+0000 7f9ddfd93740 1 -- 172.21.15.5:0/1894548567 shutdown_connections 2024-09-15T22:13:20.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.561+0000 7f9ddfd93740 1 -- 172.21.15.5:0/1894548567 wait complete. 2024-09-15T22:13:20.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.561+0000 7f9ddfd93740 20 client.25816 trim_cache size 0 max 16384 2024-09-15T22:13:20.574 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:20.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.594+0000 7f9ddfd93740 1 Processor -- start 2024-09-15T22:13:20.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.598+0000 7f9ddfd93740 1 -- start start 2024-09-15T22:13:20.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.598+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924ce760 0x5583924c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.598+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924cf080 0x5583924d5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.598+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924d9940 0x5583924d7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.598+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4012870 con 0x5583924d9940 2024-09-15T22:13:20.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.598+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4010140 con 0x5583924cf080 2024-09-15T22:13:20.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.598+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc400df90 con 0x5583924ce760 2024-09-15T22:13:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.599+0000 7f9dd6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924ce760 0x5583924c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.599+0000 7f9dd6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924ce760 0x5583924c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:47174/0 (socket says 172.21.15.5:47174) 2024-09-15T22:13:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.599+0000 7f9dd6ffd640 1 -- 172.21.15.5:0/3396729857 learned_addr learned my addr 172.21.15.5:0/3396729857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.599+0000 7f9dd67fc640 1 --2- 172.21.15.5:0/3396729857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924cf080 0x5583924d5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.599+0000 7f9dd77fe640 1 --2- 172.21.15.5:0/3396729857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924d9940 0x5583924d7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.600+0000 7f9dd6ffd640 1 -- 172.21.15.5:0/3396729857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924cf080 msgr2=0x5583924d5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.600+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/3396729857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924cf080 0x5583924d5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.600+0000 7f9dd6ffd640 1 -- 172.21.15.5:0/3396729857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924d9940 msgr2=0x5583924d7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.600+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/3396729857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924d9940 0x5583924d7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.600+0000 7f9dd6ffd640 1 -- 172.21.15.5:0/3396729857 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583924cab20 con 0x5583924ce760 2024-09-15T22:13:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.600+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/3396729857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924ce760 0x5583924c9cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0000f80 tx=0x7f9dd00044d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146ab8e983794697 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.601+0000 7f9dbd7fa640 1 -- 172.21.15.5:0/3396729857 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0004000 con 0x5583924ce760 2024-09-15T22:13:20.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.601+0000 7f9dbd7fa640 1 -- 172.21.15.5:0/3396729857 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dd00041a0 con 0x5583924ce760 2024-09-15T22:13:20.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.601+0000 7f9dbd7fa640 1 -- 172.21.15.5:0/3396729857 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0004cd0 con 0x5583924ce760 2024-09-15T22:13:20.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.604+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3396729857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924ce760 msgr2=0x5583924c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.604+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/3396729857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924ce760 0x5583924c9cc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0000f80 tx=0x7f9dd00044d0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.610+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3396729857 shutdown_connections 2024-09-15T22:13:20.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.610+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/3396729857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924d9940 0x5583924d7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.610+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/3396729857 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924cf080 0x5583924d5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.610+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/3396729857 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924ce760 0x5583924c9cc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.610+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3396729857 >> 172.21.15.5:0/3396729857 conn(0x5583924cacf0 msgr2=0x5583924d8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.616+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3396729857 shutdown_connections 2024-09-15T22:13:20.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.618+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3396729857 wait complete. 2024-09-15T22:13:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.619+0000 7f9ddfd93740 1 -- start start 2024-09-15T22:13:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.620+0000 7f9ddfd93740 1 Processor -- start 2024-09-15T22:13:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.620+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 0x5583924ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.620+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924cf080 0x5583924de460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.621+0000 7f9ddfd93740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 0x55839254d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.621+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc400e600 con 0x5583924cf080 2024-09-15T22:13:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.621+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc400fa70 con 0x5583924ce760 2024-09-15T22:13:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.621+0000 7f9ddfd93740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc400ecd0 con 0x5583924d9940 2024-09-15T22:13:20.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.621+0000 7f9ddfd93740 10 client.0 _ll_register_callbacks cb 0x7f9dc80bc710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.622+0000 7f9dd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 0x55839254d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.622+0000 7f9dd6ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 0x5583924ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.622+0000 7f9dd77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 0x55839254d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:47176/0 (socket says 172.21.15.5:47176) 2024-09-15T22:13:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.622+0000 7f9dd77fe640 1 -- 172.21.15.5:0/3690959359 learned_addr learned my addr 172.21.15.5:0/3690959359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.623+0000 7f9dd6ffd640 1 -- 172.21.15.5:0/3690959359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 msgr2=0x55839254d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.623+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/3690959359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 0x55839254d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.623+0000 7f9dd6ffd640 1 -- 172.21.15.5:0/3690959359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924cf080 msgr2=0x5583924de460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.623+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/3690959359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924cf080 0x5583924de460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.623+0000 7f9dd6ffd640 1 -- 172.21.15.5:0/3690959359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583924cb980 con 0x5583924ce760 2024-09-15T22:13:20.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.627+0000 7f9dd6ffd640 1 --2- 172.21.15.5:0/3690959359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 0x5583924ddf20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0025040 tx=0x7f9dd0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc2b28f11abd7a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:20.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.628+0000 7f9db6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:13:20.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.634+0000 7f9db6ffd640 1 -- 172.21.15.5:0/3690959359 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0004000 con 0x5583924ce760 2024-09-15T22:13:20.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.634+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3690959359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd00041a0 con 0x5583924ce760 2024-09-15T22:13:20.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.634+0000 7f9db6ffd640 1 -- 172.21.15.5:0/3690959359 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f9dd0004950 con 0x5583924ce760 2024-09-15T22:13:20.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.637+0000 7f9db6ffd640 1 -- 172.21.15.5:0/3690959359 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9dd0036050 con 0x5583924ce760 2024-09-15T22:13:20.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.638+0000 7f9ddfd93740 10 client.25841 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:20.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.638+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3690959359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5583924c6260 con 0x5583924ce760 2024-09-15T22:13:20.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.640+0000 7f9db6ffd640 1 -- 172.21.15.5:0/3690959359 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dd00041a0 con 0x5583924ce760 2024-09-15T22:13:20.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.640+0000 7f9db6ffd640 1 -- 172.21.15.5:0/3690959359 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9dd0017410 con 0x5583924ce760 2024-09-15T22:13:20.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.640+0000 7f9ddfd93740 10 client.25841 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:20.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.640+0000 7f9ddfd93740 10 client.25841 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:13:20.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.640+0000 7f9ddfd93740 20 client.25841 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:20.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.641+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/3690959359 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558392555250 0x558392557640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:20.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.641+0000 7f9ddfd93740 4 client.25841 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}] 2024-09-15T22:13:20.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.641+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3690959359 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.8, "await_for": 45.0}) -- 0x55839243bd20 con 0x558392555250 2024-09-15T22:13:20.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.642+0000 7f9dd67fc640 1 --2- 172.21.15.5:0/3690959359 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558392555250 0x558392557640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:20.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.650+0000 7f9db6ffd640 10 client.25841 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:20.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.650+0000 7f9dd67fc640 1 --2- 172.21.15.5:0/3690959359 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558392555250 0x558392557640 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56cb360313a19c server_cookie=92270f5c3c2927b3 in_seq=0 out_seq=0 2024-09-15T22:13:20.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.905+0000 7f9db6ffd640 1 -- 172.21.15.5:0/3690959359 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55839243bd20 con 0x558392555250 2024-09-15T22:13:20.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.905+0000 7f9db6ffd640 10 client.25841 handle_command_reply: tid=0 2024-09-15T22:13:20.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.905+0000 7f9ddfd93740 1 client.25841 shutdown 2024-09-15T22:13:20.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:20 smithi005 ceph-mon[30944]: pgmap v136: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 1.53k op/s 2024-09-15T22:13:20.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3690959359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 msgr2=0x5583924ddf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/3690959359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 0x5583924ddf20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0025040 tx=0x7f9dd0004770 comp rx=0 tx=0).stop 2024-09-15T22:13:20.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3690959359 shutdown_connections 2024-09-15T22:13:20.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/3690959359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5583924d9940 0x55839254d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/3690959359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5583924cf080 0x5583924de460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/3690959359 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558392555250 0x558392557640 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9ddfd93740 1 --2- 172.21.15.5:0/3690959359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5583924ce760 0x5583924ddf20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9db6ffd640 0 client.25841 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:20.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3690959359 >> 172.21.15.5:0/3690959359 conn(0x5583924cacf0 msgr2=0x5583924e47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3690959359 shutdown_connections 2024-09-15T22:13:20.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9ddfd93740 1 -- 172.21.15.5:0/3690959359 wait complete. 2024-09-15T22:13:20.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.906+0000 7f9ddfd93740 20 client.25841 trim_cache size 0 max 16384 2024-09-15T22:13:20.909 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 14, 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "61222043": { 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 14, 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 133.8, 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:13:20.910 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:13:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:13:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:13:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:13:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:13:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:20.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:20.911 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/4118936438 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9dc00a3500 msgr2=0x7f9dc00a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9dc00a3500 0x7f9dc00a59c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc009e70 tx=0x7f9dcc0156d0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/4118936438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd81011e0 msgr2=0x7f9dd81a4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd81011e0 0x7f9dd81a4d80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8005880 tx=0x7f9dc80045e0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/4118936438 shutdown_connections 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9dc00a3500 0x7f9dc00a59c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9dd8105ec0 0x7f9dd819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dd81011e0 0x7f9dd81a4d80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 --2- 172.21.15.5:0/4118936438 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9dd81008c0 0x7f9dd81a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/4118936438 >> 172.21.15.5:0/4118936438 conn(0x7f9dd80fc7d0 msgr2=0x7f9dd8103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/4118936438 shutdown_connections 2024-09-15T22:13:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:20.907+0000 7f9ddd9ba640 1 -- 172.21.15.5:0/4118936438 wait complete. 2024-09-15T22:13:21.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:20 smithi099 ceph-mon[34892]: pgmap v136: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 1.53k op/s 2024-09-15T22:13:21.094 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:20 smithi129 ceph-mon[35314]: pgmap v136: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 1.53k op/s 2024-09-15T22:13:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.680+0000 7fa37d6c7640 1 -- 172.21.15.5:0/503837635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa378102d30 msgr2=0x7fa378103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.680+0000 7fa37d6c7640 1 --2- 172.21.15.5:0/503837635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa378102d30 0x7fa378103110 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa368009930 tx=0x7fa36802a420 comp rx=0 tx=0).stop 2024-09-15T22:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.680+0000 7fa37d6c7640 1 -- 172.21.15.5:0/503837635 shutdown_connections 2024-09-15T22:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.680+0000 7fa37d6c7640 1 --2- 172.21.15.5:0/503837635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3781082a0 0x7fa37810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.680+0000 7fa37d6c7640 1 --2- 172.21.15.5:0/503837635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3781036e0 0x7fa378107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.680+0000 7fa37d6c7640 1 --2- 172.21.15.5:0/503837635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa378102d30 0x7fa378103110 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.680+0000 7fa37d6c7640 1 -- 172.21.15.5:0/503837635 >> 172.21.15.5:0/503837635 conn(0x7fa3780fc7d0 msgr2=0x7fa3780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.680+0000 7fa37d6c7640 1 -- 172.21.15.5:0/503837635 shutdown_connections 2024-09-15T22:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.680+0000 7fa37d6c7640 1 -- 172.21.15.5:0/503837635 wait complete. 2024-09-15T22:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.680+0000 7fa37d6c7640 1 Processor -- start 2024-09-15T22:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa37d6c7640 1 -- start start 2024-09-15T22:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa37d6c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa378102d30 0x7fa3781a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa37d6c7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3781036e0 0x7fa3781a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa37d6c7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3781082a0 0x7fa37819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa37d6c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781120a0 con 0x7fa378102d30 2024-09-15T22:13:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa37d6c7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa378111f20 con 0x7fa3781036e0 2024-09-15T22:13:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa37d6c7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa378112220 con 0x7fa3781082a0 2024-09-15T22:13:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa376ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa378102d30 0x7fa3781a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa376ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa378102d30 0x7fa3781a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56414/0 (socket says 172.21.15.5:56414) 2024-09-15T22:13:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa376ffd640 1 -- 172.21.15.5:0/3662081028 learned_addr learned my addr 172.21.15.5:0/3662081028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa3767fc640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3781036e0 0x7fa3781a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.681+0000 7fa3777fe640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3781082a0 0x7fa37819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.682+0000 7fa376ffd640 1 -- 172.21.15.5:0/3662081028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3781082a0 msgr2=0x7fa37819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.682+0000 7fa376ffd640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3781082a0 0x7fa37819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.682+0000 7fa376ffd640 1 -- 172.21.15.5:0/3662081028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3781036e0 msgr2=0x7fa3781a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.682+0000 7fa376ffd640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3781036e0 0x7fa3781a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.682+0000 7fa376ffd640 1 -- 172.21.15.5:0/3662081028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37819ad10 con 0x7fa378102d30 2024-09-15T22:13:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.682+0000 7fa376ffd640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa378102d30 0x7fa3781a0420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa368009230 tx=0x7fa368004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.683+0000 7fa35ffff640 1 -- 172.21.15.5:0/3662081028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36803f030 con 0x7fa378102d30 2024-09-15T22:13:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.683+0000 7fa35ffff640 1 -- 172.21.15.5:0/3662081028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa368003ad0 con 0x7fa378102d30 2024-09-15T22:13:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.683+0000 7fa37d6c7640 1 -- 172.21.15.5:0/3662081028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37819afa0 con 0x7fa378102d30 2024-09-15T22:13:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.683+0000 7fa35ffff640 1 -- 172.21.15.5:0/3662081028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368035070 con 0x7fa378102d30 2024-09-15T22:13:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.683+0000 7fa37d6c7640 1 -- 172.21.15.5:0/3662081028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3781a72f0 con 0x7fa378102d30 2024-09-15T22:13:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.685+0000 7fa35ffff640 1 -- 172.21.15.5:0/3662081028 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa368042030 con 0x7fa378102d30 2024-09-15T22:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa37faa0740 1 Processor -- start 2024-09-15T22:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa37faa0740 1 -- start start 2024-09-15T22:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7d0b40 0x562c7c7cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7cf5d0 0x562c7c7cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7da370 0x562c7c7dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa37faa0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562c7c7352f0 con 0x562c7c7cf5d0 2024-09-15T22:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa37faa0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562c7c6dccf0 con 0x562c7c7da370 2024-09-15T22:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa37faa0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c7c7312b0 con 0x562c7c7d0b40 2024-09-15T22:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa376ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7cf5d0 0x562c7c7cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa376ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7cf5d0 0x562c7c7cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56422/0 (socket says 172.21.15.5:56422) 2024-09-15T22:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa376ffd640 1 -- 172.21.15.5:0/3653334621 learned_addr learned my addr 172.21.15.5:0/3653334621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa3777fe640 1 --2- 172.21.15.5:0/3653334621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7d0b40 0x562c7c7cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.686+0000 7fa3767fc640 1 --2- 172.21.15.5:0/3653334621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7da370 0x562c7c7dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa35ffff640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3600a3430 0x7fa3600a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa376ffd640 1 -- 172.21.15.5:0/3653334621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7d0b40 msgr2=0x562c7c7cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa376ffd640 1 --2- 172.21.15.5:0/3653334621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7d0b40 0x562c7c7cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa376ffd640 1 -- 172.21.15.5:0/3653334621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7da370 msgr2=0x562c7c7dc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa376ffd640 1 --2- 172.21.15.5:0/3653334621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7da370 0x562c7c7dc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa376ffd640 1 -- 172.21.15.5:0/3653334621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c7c6c13a0 con 0x562c7c7cf5d0 2024-09-15T22:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa3777fe640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3600a3430 0x7fa3600a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa3767fc640 1 --2- 172.21.15.5:0/3653334621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7da370 0x562c7c7dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa35ffff640 1 -- 172.21.15.5:0/3662081028 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa3680f04a0 con 0x7fa378102d30 2024-09-15T22:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa376ffd640 1 --2- 172.21.15.5:0/3653334621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7cf5d0 0x562c7c7cfa10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa3680ed7c0 tx=0x7fa368049bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58fcef4727ba731 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa35d7fa640 1 -- 172.21.15.5:0/3653334621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3680bbdf0 con 0x562c7c7cf5d0 2024-09-15T22:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa35d7fa640 1 -- 172.21.15.5:0/3653334621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36808af90 con 0x562c7c7cf5d0 2024-09-15T22:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa35d7fa640 1 -- 172.21.15.5:0/3653334621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368086ef0 con 0x562c7c7cf5d0 2024-09-15T22:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.687+0000 7fa3777fe640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3600a3430 0x7fa3600a58f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa370005fd0 tx=0x7fa370004a40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.688+0000 7fa37faa0740 1 -- 172.21.15.5:0/3653334621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7cf5d0 msgr2=0x562c7c7cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.688+0000 7fa37faa0740 1 --2- 172.21.15.5:0/3653334621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7cf5d0 0x562c7c7cfa10 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa3680ed7c0 tx=0x7fa368049bd0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.688+0000 7fa37faa0740 1 -- 172.21.15.5:0/3653334621 shutdown_connections 2024-09-15T22:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.688+0000 7fa37faa0740 1 --2- 172.21.15.5:0/3653334621 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7da370 0x562c7c7dc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.688+0000 7fa37faa0740 1 --2- 172.21.15.5:0/3653334621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7cf5d0 0x562c7c7cfa10 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.688+0000 7fa37faa0740 1 --2- 172.21.15.5:0/3653334621 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7d0b40 0x562c7c7cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.688+0000 7fa37faa0740 1 -- 172.21.15.5:0/3653334621 >> 172.21.15.5:0/3653334621 conn(0x562c7c798310 msgr2=0x562c7c7cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.688+0000 7fa37faa0740 1 -- 172.21.15.5:0/3653334621 shutdown_connections 2024-09-15T22:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.688+0000 7fa37faa0740 1 -- 172.21.15.5:0/3653334621 wait complete. 2024-09-15T22:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa37faa0740 1 -- start start 2024-09-15T22:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa37faa0740 1 Processor -- start 2024-09-15T22:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7cf5d0 0x562c7c7d9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7d0b40 0x562c7c7d6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7da370 0x562c7c7d74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa37faa0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562c7c7dce30 con 0x562c7c7d0b40 2024-09-15T22:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa37faa0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562c7c6f0060 con 0x562c7c7cf5d0 2024-09-15T22:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa37faa0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562c7c7dcfb0 con 0x562c7c7da370 2024-09-15T22:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa37faa0740 10 client.0 _ll_register_callbacks cb 0x562c7c4438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa3767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7d0b40 0x562c7c7d6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa3767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7d0b40 0x562c7c7d6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56428/0 (socket says 172.21.15.5:56428) 2024-09-15T22:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa3767fc640 1 -- 172.21.15.5:0/2261074304 learned_addr learned my addr 172.21.15.5:0/2261074304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa3777fe640 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7da370 0x562c7c7d74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.689+0000 7fa376ffd640 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7cf5d0 0x562c7c7d9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa3767fc640 1 -- 172.21.15.5:0/2261074304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7da370 msgr2=0x562c7c7d74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa3767fc640 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7da370 0x562c7c7d74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa3767fc640 1 -- 172.21.15.5:0/2261074304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7cf5d0 msgr2=0x562c7c7d9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa3767fc640 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7cf5d0 0x562c7c7d9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa3767fc640 1 -- 172.21.15.5:0/2261074304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c7c4a63c0 con 0x562c7c7d0b40 2024-09-15T22:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa3767fc640 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7d0b40 0x562c7c7d6f60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00baa0 tx=0x7fa36c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad0443f9321ff13 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa35aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa35aff5640 1 -- 172.21.15.5:0/2261074304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c003d30 con 0x562c7c7d0b40 2024-09-15T22:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa35aff5640 1 -- 172.21.15.5:0/2261074304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa36c005020 con 0x562c7c7d0b40 2024-09-15T22:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa37faa0740 1 -- 172.21.15.5:0/2261074304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa36808af90 con 0x562c7c7d0b40 2024-09-15T22:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa35aff5640 1 -- 172.21.15.5:0/2261074304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa36c005220 con 0x562c7c7d0b40 2024-09-15T22:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa37faa0740 10 client.16263 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa37faa0740 1 -- 172.21.15.5:0/2261074304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562c7c7e87b0 con 0x562c7c7d0b40 2024-09-15T22:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.690+0000 7fa35aff5640 1 -- 172.21.15.5:0/2261074304 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa36808af90 con 0x562c7c7d0b40 2024-09-15T22:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.691+0000 7fa35aff5640 1 -- 172.21.15.5:0/2261074304 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa36c01b470 con 0x562c7c7d0b40 2024-09-15T22:13:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.691+0000 7fa37faa0740 10 client.16263 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.691+0000 7fa37faa0740 10 client.16263 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:13:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.691+0000 7fa37faa0740 20 client.16263 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.691+0000 7fa37faa0740 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562c7c7e9cc0 0x562c7c7ec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.691+0000 7fa37faa0740 4 client.16263 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.691+0000 7fa37faa0740 1 -- 172.21.15.5:0/2261074304 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c7c742860 con 0x562c7c7e9cc0 2024-09-15T22:13:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.691+0000 7fa376ffd640 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562c7c7e9cc0 0x562c7c7ec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.692+0000 7fa376ffd640 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562c7c7e9cc0 0x562c7c7ec120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56b25508d1118f21 server_cookie=975cc6aaba7d553c in_seq=0 out_seq=0 2024-09-15T22:13:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.692+0000 7fa35aff5640 10 client.16263 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:13:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.692+0000 7fa35aff5640 1 -- 172.21.15.5:0/2261074304 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c7c742860 con 0x562c7c7e9cc0 2024-09-15T22:13:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.692+0000 7fa35aff5640 10 client.16263 handle_command_reply: tid=0 2024-09-15T22:13:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 1 client.16263 shutdown 2024-09-15T22:13:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 1 -- 172.21.15.5:0/2261074304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7d0b40 msgr2=0x562c7c7d6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7d0b40 0x562c7c7d6f60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00baa0 tx=0x7fa36c00bea0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 1 -- 172.21.15.5:0/2261074304 shutdown_connections 2024-09-15T22:13:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562c7c7e9cc0 0x562c7c7ec120 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7da370 0x562c7c7d74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7d0b40 0x562c7c7d6f60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 1 --2- 172.21.15.5:0/2261074304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7cf5d0 0x562c7c7d9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 1 -- 172.21.15.5:0/2261074304 >> 172.21.15.5:0/2261074304 conn(0x562c7c798310 msgr2=0x562c7c7d1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa35aff5640 0 client.16263 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:13:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 1 -- 172.21.15.5:0/2261074304 shutdown_connections 2024-09-15T22:13:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 1 -- 172.21.15.5:0/2261074304 wait complete. 2024-09-15T22:13:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.693+0000 7fa37faa0740 20 client.16263 trim_cache size 0 max 16384 2024-09-15T22:13:21.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.706+0000 7fa37faa0740 1 Processor -- start 2024-09-15T22:13:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.706+0000 7fa37faa0740 1 -- start start 2024-09-15T22:13:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7cf5d0 0x562c7c7cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7d0b40 0x562c7c7d7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7e5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa37faa0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781b3b60 con 0x562c7c7da370 2024-09-15T22:13:21.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa37faa0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781b1430 con 0x562c7c7cf5d0 2024-09-15T22:13:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa37faa0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781af280 con 0x562c7c7d0b40 2024-09-15T22:13:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa3777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7e5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa3777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7e5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56444/0 (socket says 172.21.15.5:56444) 2024-09-15T22:13:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa3777fe640 1 -- 172.21.15.5:0/3680818198 learned_addr learned my addr 172.21.15.5:0/3680818198 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:21.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa3767fc640 1 --2- 172.21.15.5:0/3680818198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7d0b40 0x562c7c7d7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa376ffd640 1 --2- 172.21.15.5:0/3680818198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7cf5d0 0x562c7c7cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa3777fe640 1 -- 172.21.15.5:0/3680818198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7d0b40 msgr2=0x562c7c7d7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa3777fe640 1 --2- 172.21.15.5:0/3680818198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7d0b40 0x562c7c7d7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa3777fe640 1 -- 172.21.15.5:0/3680818198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7cf5d0 msgr2=0x562c7c7cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa3777fe640 1 --2- 172.21.15.5:0/3680818198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7cf5d0 0x562c7c7cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa3777fe640 1 -- 172.21.15.5:0/3680818198 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c7c7e6410 con 0x562c7c7da370 2024-09-15T22:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa376ffd640 1 --2- 172.21.15.5:0/3680818198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7cf5d0 0x562c7c7cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.707+0000 7fa3777fe640 1 --2- 172.21.15.5:0/3680818198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7e5ed0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa370012630 tx=0x7fa370016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535931593c0a6c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa35cff9640 1 -- 172.21.15.5:0/3680818198 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370021810 con 0x562c7c7da370 2024-09-15T22:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa35cff9640 1 -- 172.21.15.5:0/3680818198 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa370010400 con 0x562c7c7da370 2024-09-15T22:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa35cff9640 1 -- 172.21.15.5:0/3680818198 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370011640 con 0x562c7c7da370 2024-09-15T22:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa37faa0740 1 -- 172.21.15.5:0/3680818198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 msgr2=0x562c7c7e5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa37faa0740 1 --2- 172.21.15.5:0/3680818198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7e5ed0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa370012630 tx=0x7fa370016000 comp rx=0 tx=0).stop 2024-09-15T22:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa37faa0740 1 -- 172.21.15.5:0/3680818198 shutdown_connections 2024-09-15T22:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa37faa0740 1 --2- 172.21.15.5:0/3680818198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7e5ed0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa37faa0740 1 --2- 172.21.15.5:0/3680818198 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7d0b40 0x562c7c7d7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa37faa0740 1 --2- 172.21.15.5:0/3680818198 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7cf5d0 0x562c7c7cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa37faa0740 1 -- 172.21.15.5:0/3680818198 >> 172.21.15.5:0/3680818198 conn(0x562c7c798310 msgr2=0x562c7c7d6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa37faa0740 1 -- 172.21.15.5:0/3680818198 shutdown_connections 2024-09-15T22:13:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.708+0000 7fa37faa0740 1 -- 172.21.15.5:0/3680818198 wait complete. 2024-09-15T22:13:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa37faa0740 1 -- start start 2024-09-15T22:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa37faa0740 1 Processor -- start 2024-09-15T22:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7cf5d0 0x562c7c7e23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7d0b40 0x562c7c7e28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa37faa0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7d3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa37faa0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781af8f0 con 0x562c7c7da370 2024-09-15T22:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa37faa0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781b0d60 con 0x562c7c7d0b40 2024-09-15T22:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa37faa0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa3781affc0 con 0x562c7c7cf5d0 2024-09-15T22:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa37faa0740 10 client.0 _ll_register_callbacks cb 0x7fa36c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa3777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7d3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa3777fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7d3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56450/0 (socket says 172.21.15.5:56450) 2024-09-15T22:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa3777fe640 1 -- 172.21.15.5:0/374339990 learned_addr learned my addr 172.21.15.5:0/374339990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa376ffd640 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7cf5d0 0x562c7c7e23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.709+0000 7fa3767fc640 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7d0b40 0x562c7c7e28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa3777fe640 1 -- 172.21.15.5:0/374339990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7cf5d0 msgr2=0x562c7c7e23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa3777fe640 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7cf5d0 0x562c7c7e23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa3777fe640 1 -- 172.21.15.5:0/374339990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7d0b40 msgr2=0x562c7c7e28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa3777fe640 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7d0b40 0x562c7c7e28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa3777fe640 1 -- 172.21.15.5:0/374339990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c7c7d4390 con 0x562c7c7da370 2024-09-15T22:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa3777fe640 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7d3b60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa370004410 tx=0x7fa370010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b73ec20419657 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa35aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa35aff5640 1 -- 172.21.15.5:0/374339990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370021a20 con 0x562c7c7da370 2024-09-15T22:13:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa37faa0740 1 -- 172.21.15.5:0/374339990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa370010400 con 0x562c7c7da370 2024-09-15T22:13:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa35aff5640 1 -- 172.21.15.5:0/374339990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa370021bc0 con 0x562c7c7da370 2024-09-15T22:13:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa35aff5640 1 -- 172.21.15.5:0/374339990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa3700047d0 con 0x562c7c7da370 2024-09-15T22:13:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa37faa0740 10 client.16269 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.710+0000 7fa37faa0740 1 -- 172.21.15.5:0/374339990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562c7c854650 con 0x562c7c7da370 2024-09-15T22:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.711+0000 7fa35aff5640 1 -- 172.21.15.5:0/374339990 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa370010400 con 0x562c7c7da370 2024-09-15T22:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.711+0000 7fa35aff5640 1 -- 172.21.15.5:0/374339990 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa37000f040 con 0x562c7c7da370 2024-09-15T22:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.711+0000 7fa37faa0740 10 client.16269 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.711+0000 7fa37faa0740 10 client.16269 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.711+0000 7fa37faa0740 20 client.16269 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.711+0000 7fa37faa0740 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562c7c854d10 0x562c7c857130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.711+0000 7fa37faa0740 4 client.16269 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.711+0000 7fa37faa0740 1 -- 172.21.15.5:0/374339990 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562c7c73cd70 con 0x562c7c854d10 2024-09-15T22:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.711+0000 7fa376ffd640 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562c7c854d10 0x562c7c857130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa35aff5640 10 client.16269 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa376ffd640 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562c7c854d10 0x562c7c857130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9cfb75b88df01f0c server_cookie=4c3deff53bb3109f in_seq=0 out_seq=0 2024-09-15T22:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa35aff5640 1 -- 172.21.15.5:0/374339990 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562c7c73cd70 con 0x562c7c854d10 2024-09-15T22:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa35aff5640 10 client.16269 handle_command_reply: tid=0 2024-09-15T22:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 1 client.16269 shutdown 2024-09-15T22:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 1 -- 172.21.15.5:0/374339990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 msgr2=0x562c7c7d3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7d3b60 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa370004410 tx=0x7fa370010b60 comp rx=0 tx=0).stop 2024-09-15T22:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 1 -- 172.21.15.5:0/374339990 shutdown_connections 2024-09-15T22:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562c7c854d10 0x562c7c857130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c7c7da370 0x562c7c7d3b60 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562c7c7d0b40 0x562c7c7e28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa35aff5640 0 client.16269 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 1 --2- 172.21.15.5:0/374339990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562c7c7cf5d0 0x562c7c7e23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 1 -- 172.21.15.5:0/374339990 >> 172.21.15.5:0/374339990 conn(0x562c7c798310 msgr2=0x562c7c7ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 1 -- 172.21.15.5:0/374339990 shutdown_connections 2024-09-15T22:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 1 -- 172.21.15.5:0/374339990 wait complete. 2024-09-15T22:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.712+0000 7fa37faa0740 20 client.16269 trim_cache size 0 max 16384 2024-09-15T22:13:21.738 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:13:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 -- 172.21.15.5:0/3662081028 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3600a3430 msgr2=0x7fa3600a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3600a3430 0x7fa3600a58f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa370005fd0 tx=0x7fa370004a40 comp rx=0 tx=0).stop 2024-09-15T22:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 -- 172.21.15.5:0/3662081028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa378102d30 msgr2=0x7fa3781a0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa378102d30 0x7fa3781a0420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa368009230 tx=0x7fa368004270 comp rx=0 tx=0).stop 2024-09-15T22:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 -- 172.21.15.5:0/3662081028 shutdown_connections 2024-09-15T22:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3600a3430 0x7fa3600a58f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa3781082a0 0x7fa37819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa3781036e0 0x7fa3781a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 --2- 172.21.15.5:0/3662081028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa378102d30 0x7fa3781a0420 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 -- 172.21.15.5:0/3662081028 >> 172.21.15.5:0/3662081028 conn(0x7fa3780fc7d0 msgr2=0x7fa378100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 -- 172.21.15.5:0/3662081028 shutdown_connections 2024-09-15T22:13:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:21.713+0000 7fa37d6c7640 1 -- 172.21.15.5:0/3662081028 wait complete. 2024-09-15T22:13:22.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:13:22.846 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:22 smithi005 ceph-mon[30944]: pgmap v137: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-09-15T22:13:22.877 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '61222043', quiesce took 12.6 seconds. Will release after: 1.200000000000001 2024-09-15T22:13:22.920 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:23.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:22 smithi099 ceph-mon[34892]: pgmap v137: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-09-15T22:13:23.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:22 smithi129 ceph-mon[35314]: pgmap v137: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 21 MiB/s wr, 1.49k op/s 2024-09-15T22:13:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.398+0000 7f1163b2c640 1 -- 172.21.15.5:0/4133839431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115c100ca0 msgr2=0x7f115c106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.398+0000 7f1163b2c640 1 --2- 172.21.15.5:0/4133839431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115c100ca0 0x7f115c106840 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f114c009930 tx=0x7f114c02a420 comp rx=0 tx=0).stop 2024-09-15T22:13:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.398+0000 7f1163b2c640 1 -- 172.21.15.5:0/4133839431 shutdown_connections 2024-09-15T22:13:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.398+0000 7f1163b2c640 1 --2- 172.21.15.5:0/4133839431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c106f50 0x7f115c10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.398+0000 7f1163b2c640 1 --2- 172.21.15.5:0/4133839431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115c100ca0 0x7f115c106840 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.398+0000 7f1163b2c640 1 --2- 172.21.15.5:0/4133839431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c1002f0 0x7f115c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.398+0000 7f1163b2c640 1 -- 172.21.15.5:0/4133839431 >> 172.21.15.5:0/4133839431 conn(0x7f115c077210 msgr2=0x7f115c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.398+0000 7f1163b2c640 1 -- 172.21.15.5:0/4133839431 shutdown_connections 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.398+0000 7f1163b2c640 1 -- 172.21.15.5:0/4133839431 wait complete. 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f1163b2c640 1 Processor -- start 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f1163b2c640 1 -- start start 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f1163b2c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c1002f0 0x7f115c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f1163b2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115c100ca0 0x7f115c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f1163b2c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c106f50 0x7f115c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f1163b2c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f115c111fa0 con 0x7f115c100ca0 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f1163b2c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f115c111e20 con 0x7f115c1002f0 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f1163b2c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f115c112120 con 0x7f115c106f50 2024-09-15T22:13:23.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f11610c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115c100ca0 0x7f115c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f11610c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115c100ca0 0x7f115c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59960/0 (socket says 172.21.15.5:59960) 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f11610c7640 1 -- 172.21.15.5:0/3976343620 learned_addr learned my addr 172.21.15.5:0/3976343620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f11620c9640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c106f50 0x7f115c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f11618c8640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c1002f0 0x7f115c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f11610c7640 1 -- 172.21.15.5:0/3976343620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c106f50 msgr2=0x7f115c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f11610c7640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c106f50 0x7f115c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f11610c7640 1 -- 172.21.15.5:0/3976343620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c1002f0 msgr2=0x7f115c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f11610c7640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c1002f0 0x7f115c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f11610c7640 1 -- 172.21.15.5:0/3976343620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115c19d450 con 0x7f115c100ca0 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.399+0000 7f11618c8640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c1002f0 0x7f115c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.400+0000 7f11610c7640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115c100ca0 0x7f115c1989e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f114c009230 tx=0x7f114c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.400+0000 7f1152ffd640 1 -- 172.21.15.5:0/3976343620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c03f030 con 0x7f115c100ca0 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.400+0000 7f1163b2c640 1 -- 172.21.15.5:0/3976343620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115c19d6e0 con 0x7f115c100ca0 2024-09-15T22:13:23.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.400+0000 7f1152ffd640 1 -- 172.21.15.5:0/3976343620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114c003ce0 con 0x7f115c100ca0 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.400+0000 7f1152ffd640 1 -- 172.21.15.5:0/3976343620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c035070 con 0x7f115c100ca0 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.400+0000 7f1163b2c640 1 -- 172.21.15.5:0/3976343620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f115c1a5020 con 0x7f115c100ca0 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.402+0000 7f1152ffd640 1 -- 172.21.15.5:0/3976343620 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f114c042030 con 0x7f115c100ca0 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.402+0000 7f1165f05740 1 Processor -- start 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f1165f05740 1 -- start start 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f1165f05740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b8b40 0x5596337b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f1165f05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f1165f05740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f1165f05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55963371d2f0 con 0x5596337b75d0 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f1165f05740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5596336c4cf0 con 0x5596337b8b40 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f1165f05740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5596337192b0 con 0x5596337c2370 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f11618c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f1152ffd640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f11480a33b0 0x7f11480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f11618c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337b7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59962/0 (socket says 172.21.15.5:59962) 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f11618c8640 1 -- 172.21.15.5:0/2194136943 learned_addr learned my addr 172.21.15.5:0/2194136943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f11620c9640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f11480a33b0 0x7f11480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f1152ffd640 1 -- 172.21.15.5:0/3976343620 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f114c0f03d0 con 0x7f115c100ca0 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f11620c9640 1 --2- 172.21.15.5:0/2194136943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b8b40 0x5596337b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f11610c7640 1 --2- 172.21.15.5:0/2194136943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f11618c8640 1 -- 172.21.15.5:0/2194136943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 msgr2=0x5596337c4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f11618c8640 1 --2- 172.21.15.5:0/2194136943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f11618c8640 1 -- 172.21.15.5:0/2194136943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b8b40 msgr2=0x5596337b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f11618c8640 1 --2- 172.21.15.5:0/2194136943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b8b40 0x5596337b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.403+0000 7f11618c8640 1 -- 172.21.15.5:0/2194136943 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596336a93a0 con 0x5596337b75d0 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.404+0000 7f11618c8640 1 --2- 172.21.15.5:0/2194136943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337b7a10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f115800bb70 tx=0x7f115800d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59e7690967e0559 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.404+0000 7f11620c9640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f11480a33b0 0x7f11480a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1154005cc0 tx=0x7f115400b3e0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:23.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.404+0000 7f1147fff640 1 -- 172.21.15.5:0/2194136943 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158017620 con 0x5596337b75d0 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.404+0000 7f1147fff640 1 -- 172.21.15.5:0/2194136943 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1158017ca0 con 0x5596337b75d0 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.404+0000 7f1147fff640 1 -- 172.21.15.5:0/2194136943 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115800fc00 con 0x5596337b75d0 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 -- 172.21.15.5:0/2194136943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 msgr2=0x5596337b7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 --2- 172.21.15.5:0/2194136943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337b7a10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f115800bb70 tx=0x7f115800d710 comp rx=0 tx=0).stop 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 -- 172.21.15.5:0/2194136943 shutdown_connections 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 --2- 172.21.15.5:0/2194136943 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337c4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 --2- 172.21.15.5:0/2194136943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337b7a10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 --2- 172.21.15.5:0/2194136943 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b8b40 0x5596337b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 -- 172.21.15.5:0/2194136943 >> 172.21.15.5:0/2194136943 conn(0x559633780310 msgr2=0x5596337b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 -- 172.21.15.5:0/2194136943 shutdown_connections 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 -- 172.21.15.5:0/2194136943 wait complete. 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 -- start start 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 Processor -- start 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b8b40 0x5596337bef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.405+0000 7f1165f05740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337bf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f1165f05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5596337c5070 con 0x5596337b75d0 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f1165f05740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5596336d8060 con 0x5596337b8b40 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f1165f05740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5596337c51f0 con 0x5596337c2370 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f1165f05740 10 client.0 _ll_register_callbacks cb 0x55963342b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11618c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11618c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337c1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59976/0 (socket says 172.21.15.5:59976) 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11618c8640 1 -- 172.21.15.5:0/1526211642 learned_addr learned my addr 172.21.15.5:0/1526211642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11610c7640 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b8b40 0x5596337bef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11620c9640 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337bf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11618c8640 1 -- 172.21.15.5:0/1526211642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 msgr2=0x5596337bf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11618c8640 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337bf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11618c8640 1 -- 172.21.15.5:0/1526211642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b8b40 msgr2=0x5596337bef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:23.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11618c8640 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b8b40 0x5596337bef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11618c8640 1 -- 172.21.15.5:0/1526211642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55963348e3c0 con 0x5596337b75d0 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11618c8640 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337c1cb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1158019040 tx=0x7f1158017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a14325c9f9deb5a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11457fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11457fa640 1 -- 172.21.15.5:0/1526211642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1158026070 con 0x5596337b75d0 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11457fa640 1 -- 172.21.15.5:0/1526211642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f1158022440 con 0x5596337b75d0 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f1165f05740 1 -- 172.21.15.5:0/1526211642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1158017ca0 con 0x5596337b75d0 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f11457fa640 1 -- 172.21.15.5:0/1526211642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1158032050 con 0x5596337b75d0 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.406+0000 7f1165f05740 10 client.16281 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.407+0000 7f1165f05740 1 -- 172.21.15.5:0/1526211642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5596337d4e70 con 0x5596337b75d0 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.407+0000 7f11457fa640 1 -- 172.21.15.5:0/1526211642 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1158017ca0 con 0x5596337b75d0 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.407+0000 7f11457fa640 1 -- 172.21.15.5:0/1526211642 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1158022b20 con 0x5596337b75d0 2024-09-15T22:13:23.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.407+0000 7f1165f05740 10 client.16281 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.407+0000 7f1165f05740 10 client.16281 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.407+0000 7f1165f05740 20 client.16281 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.407+0000 7f1165f05740 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5596337d62d0 0x5596337d86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.407+0000 7f1165f05740 4 client.16281 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.407+0000 7f1165f05740 1 -- 172.21.15.5:0/1526211642 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55963372a860 con 0x5596337d62d0 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.407+0000 7f11610c7640 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5596337d62d0 0x5596337d86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.408+0000 7f11457fa640 10 client.16281 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.408+0000 7f11610c7640 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5596337d62d0 0x5596337d86c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a8a9911c864841ef server_cookie=ecc04e2bd79ae1b4 in_seq=0 out_seq=0 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.408+0000 7f11457fa640 1 -- 172.21.15.5:0/1526211642 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55963372a860 con 0x5596337d62d0 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.408+0000 7f11457fa640 10 client.16281 handle_command_reply: tid=0 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.408+0000 7f1165f05740 1 client.16281 shutdown 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.408+0000 7f1165f05740 1 -- 172.21.15.5:0/1526211642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 msgr2=0x5596337c1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.409+0000 7f1165f05740 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337c1cb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1158019040 tx=0x7f1158017650 comp rx=0 tx=0).stop 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.409+0000 7f1165f05740 1 -- 172.21.15.5:0/1526211642 shutdown_connections 2024-09-15T22:13:23.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.409+0000 7f1165f05740 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5596337d62d0 0x5596337d86c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.409+0000 7f1165f05740 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337bf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.409+0000 7f1165f05740 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b8b40 0x5596337bef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.409+0000 7f1165f05740 1 --2- 172.21.15.5:0/1526211642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b75d0 0x5596337c1cb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.409+0000 7f11457fa640 0 client.16281 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:13:23.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.409+0000 7f1165f05740 1 -- 172.21.15.5:0/1526211642 >> 172.21.15.5:0/1526211642 conn(0x559633780310 msgr2=0x5596337b9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:23.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.409+0000 7f1165f05740 1 -- 172.21.15.5:0/1526211642 shutdown_connections 2024-09-15T22:13:23.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.409+0000 7f1165f05740 1 -- 172.21.15.5:0/1526211642 wait complete. 2024-09-15T22:13:23.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.409+0000 7f1165f05740 20 client.16281 trim_cache size 0 max 16384 2024-09-15T22:13:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.422+0000 7f1165f05740 1 Processor -- start 2024-09-15T22:13:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f1165f05740 1 -- start start 2024-09-15T22:13:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f1165f05740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337b75d0 0x5596337d27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f1165f05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337be840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f1165f05740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337c2370 0x5596337ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f1165f05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1af770 con 0x5596337b8b40 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f1165f05740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1ad040 con 0x5596337c2370 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f1165f05740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1aae90 con 0x5596337b75d0 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11610c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11610c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337be840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59978/0 (socket says 172.21.15.5:59978) 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11610c7640 1 -- 172.21.15.5:0/2104555591 learned_addr learned my addr 172.21.15.5:0/2104555591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11620c9640 1 --2- 172.21.15.5:0/2104555591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337c2370 0x5596337ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11618c8640 1 --2- 172.21.15.5:0/2104555591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337b75d0 0x5596337d27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11610c7640 1 -- 172.21.15.5:0/2104555591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337b75d0 msgr2=0x5596337d27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11610c7640 1 --2- 172.21.15.5:0/2104555591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337b75d0 0x5596337d27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11610c7640 1 -- 172.21.15.5:0/2104555591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337c2370 msgr2=0x5596337ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11610c7640 1 --2- 172.21.15.5:0/2104555591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337c2370 0x5596337ddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11610c7640 1 -- 172.21.15.5:0/2104555591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596337c51e0 con 0x5596337b8b40 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11618c8640 1 --2- 172.21.15.5:0/2104555591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337b75d0 0x5596337d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:23.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.423+0000 7f11610c7640 1 --2- 172.21.15.5:0/2104555591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337be840 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f114c0bc010 tx=0x7f114c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c02e5a6d47c6b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.424+0000 7f11477fe640 1 -- 172.21.15.5:0/2104555591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c049040 con 0x5596337b8b40 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.424+0000 7f11477fe640 1 -- 172.21.15.5:0/2104555591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114c0c4f80 con 0x5596337b8b40 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.424+0000 7f11477fe640 1 -- 172.21.15.5:0/2104555591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c0eda00 con 0x5596337b8b40 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.424+0000 7f1165f05740 1 -- 172.21.15.5:0/2104555591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 msgr2=0x5596337be840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.424+0000 7f1165f05740 1 --2- 172.21.15.5:0/2104555591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337be840 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f114c0bc010 tx=0x7f114c0dc040 comp rx=0 tx=0).stop 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.424+0000 7f1165f05740 1 -- 172.21.15.5:0/2104555591 shutdown_connections 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.424+0000 7f1165f05740 1 --2- 172.21.15.5:0/2104555591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337c2370 0x5596337ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.424+0000 7f1165f05740 1 --2- 172.21.15.5:0/2104555591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337be840 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.424+0000 7f1165f05740 1 --2- 172.21.15.5:0/2104555591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337b75d0 0x5596337d27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.425+0000 7f1165f05740 1 -- 172.21.15.5:0/2104555591 >> 172.21.15.5:0/2104555591 conn(0x559633780310 msgr2=0x5596337b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.425+0000 7f1165f05740 1 -- 172.21.15.5:0/2104555591 shutdown_connections 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.425+0000 7f1165f05740 1 -- 172.21.15.5:0/2104555591 wait complete. 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.425+0000 7f1165f05740 1 -- start start 2024-09-15T22:13:23.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.425+0000 7f1165f05740 1 Processor -- start 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f1165f05740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b75d0 0x5596337cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f1165f05740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f1165f05740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337cac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f1165f05740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1ab500 con 0x5596337b8b40 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f1165f05740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1ac970 con 0x5596337b75d0 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f1165f05740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1abbd0 con 0x5596337c2370 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f1165f05740 10 client.0 _ll_register_callbacks cb 0x7f115800b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11610c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11610c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59980/0 (socket says 172.21.15.5:59980) 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11610c7640 1 -- 172.21.15.5:0/3991872217 learned_addr learned my addr 172.21.15.5:0/3991872217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11618c8640 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b75d0 0x5596337cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11620c9640 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11610c7640 1 -- 172.21.15.5:0/3991872217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 msgr2=0x5596337cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:23.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11610c7640 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337cac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11610c7640 1 -- 172.21.15.5:0/3991872217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b75d0 msgr2=0x5596337cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11610c7640 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b75d0 0x5596337cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11610c7640 1 -- 172.21.15.5:0/3991872217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55963383fc60 con 0x5596337b8b40 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11618c8640 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b75d0 0x5596337cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11610c7640 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337ca520 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f114c0b8770 tx=0x7f114c0b85b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b663a982b88fd125 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11457fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.426+0000 7f11457fa640 1 -- 172.21.15.5:0/3991872217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c0d51b0 con 0x5596337b8b40 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f11457fa640 1 -- 172.21.15.5:0/3991872217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f114c0eda00 con 0x5596337b8b40 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f1165f05740 1 -- 172.21.15.5:0/3991872217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114c0c4f80 con 0x5596337b8b40 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f11457fa640 1 -- 172.21.15.5:0/3991872217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f114c0b8980 con 0x5596337b8b40 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f1165f05740 10 client.16287 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f1165f05740 1 -- 172.21.15.5:0/3991872217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559633840d70 con 0x5596337b8b40 2024-09-15T22:13:23.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f11457fa640 1 -- 172.21.15.5:0/3991872217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f114c0c4f80 con 0x5596337b8b40 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f11457fa640 1 -- 172.21.15.5:0/3991872217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f114c031bd0 con 0x5596337b8b40 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f1165f05740 10 client.16287 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f1165f05740 10 client.16287 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f1165f05740 20 client.16287 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f1165f05740 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x559633841460 0x559633843880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f1165f05740 4 client.16287 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.427+0000 7f1165f05740 1 -- 172.21.15.5:0/3991872217 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596337bf830 con 0x559633841460 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f11618c8640 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x559633841460 0x559633843880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f11618c8640 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x559633841460 0x559633843880 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb79b07076795f83 server_cookie=bd1ff3ee05152fd in_seq=0 out_seq=0 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f11457fa640 10 client.16287 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f11457fa640 1 -- 172.21.15.5:0/3991872217 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596337bf830 con 0x559633841460 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f11457fa640 10 client.16287 handle_command_reply: tid=0 2024-09-15T22:13:23.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f1165f05740 1 client.16287 shutdown 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f1165f05740 1 -- 172.21.15.5:0/3991872217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 msgr2=0x5596337ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f1165f05740 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337ca520 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f114c0b8770 tx=0x7f114c0b85b0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f1165f05740 1 -- 172.21.15.5:0/3991872217 shutdown_connections 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f1165f05740 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x559633841460 0x559633843880 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f1165f05740 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5596337c2370 0x5596337cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f1165f05740 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596337b8b40 0x5596337ca520 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f11457fa640 0 client.16287 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f1165f05740 1 --2- 172.21.15.5:0/3991872217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5596337b75d0 0x5596337cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.428+0000 7f1165f05740 1 -- 172.21.15.5:0/3991872217 >> 172.21.15.5:0/3991872217 conn(0x559633780310 msgr2=0x5596337b9110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1165f05740 1 -- 172.21.15.5:0/3991872217 shutdown_connections 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1165f05740 1 -- 172.21.15.5:0/3991872217 wait complete. 2024-09-15T22:13:23.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1165f05740 20 client.16287 trim_cache size 0 max 16384 2024-09-15T22:13:23.439 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:13:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 -- 172.21.15.5:0/3976343620 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f11480a33b0 msgr2=0x7f11480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f11480a33b0 0x7f11480a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1154005cc0 tx=0x7f115400b3e0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 -- 172.21.15.5:0/3976343620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115c100ca0 msgr2=0x7f115c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115c100ca0 0x7f115c1989e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f114c009230 tx=0x7f114c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 -- 172.21.15.5:0/3976343620 shutdown_connections 2024-09-15T22:13:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f11480a33b0 0x7f11480a5870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f115c106f50 0x7f115c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f115c100ca0 0x7f115c1989e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 --2- 172.21.15.5:0/3976343620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f115c1002f0 0x7f115c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 -- 172.21.15.5:0/3976343620 >> 172.21.15.5:0/3976343620 conn(0x7f115c077210 msgr2=0x7f115c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 -- 172.21.15.5:0/3976343620 shutdown_connections 2024-09-15T22:13:23.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:23.429+0000 7f1163b2c640 1 -- 172.21.15.5:0/3976343620 wait complete. 2024-09-15T22:13:24.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:13:24.175 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:13:24.362 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:24.664 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 235.0 s, 402.5 tps, lat 48.018 ms stddev 55.744 2024-09-15T22:13:24.733 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:24 smithi005 ceph-mon[30944]: pgmap v138: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 581 op/s 2024-09-15T22:13:24.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.813+0000 7fc365772640 1 -- 172.21.15.5:0/563538479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc360102790 msgr2=0x7fc3601067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.813+0000 7fc365772640 1 --2- 172.21.15.5:0/563538479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc360102790 0x7fc3601067e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc354009ca0 tx=0x7fc35402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.813+0000 7fc365772640 1 -- 172.21.15.5:0/563538479 shutdown_connections 2024-09-15T22:13:24.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.813+0000 7fc365772640 1 --2- 172.21.15.5:0/563538479 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc360106f10 0x7fc36010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.813+0000 7fc365772640 1 --2- 172.21.15.5:0/563538479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc360102790 0x7fc3601067e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.813+0000 7fc365772640 1 --2- 172.21.15.5:0/563538479 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc360101de0 0x7fc3601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.813+0000 7fc365772640 1 -- 172.21.15.5:0/563538479 >> 172.21.15.5:0/563538479 conn(0x7fc3600fd8c0 msgr2=0x7fc3600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:24.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.813+0000 7fc365772640 1 -- 172.21.15.5:0/563538479 shutdown_connections 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.814+0000 7fc365772640 1 -- 172.21.15.5:0/563538479 wait complete. 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.814+0000 7fc365772640 1 Processor -- start 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.814+0000 7fc365772640 1 -- start start 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.814+0000 7fc365772640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc360101de0 0x7fc3600744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.814+0000 7fc365772640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc360102790 0x7fc360074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc365772640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc360106f10 0x7fc360074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc365772640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc360111f30 con 0x7fc360101de0 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc365772640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc360111db0 con 0x7fc360102790 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc365772640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3601120b0 con 0x7fc360106f10 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc35effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc360101de0 0x7fc3600744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc35effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc360101de0 0x7fc3600744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59992/0 (socket says 172.21.15.5:59992) 2024-09-15T22:13:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc35effd640 1 -- 172.21.15.5:0/892535990 learned_addr learned my addr 172.21.15.5:0/892535990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc35f7fe640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc360106f10 0x7fc360074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc35e7fc640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc360102790 0x7fc360074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc35effd640 1 -- 172.21.15.5:0/892535990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc360106f10 msgr2=0x7fc360074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc35effd640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc360106f10 0x7fc360074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc35effd640 1 -- 172.21.15.5:0/892535990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc360102790 msgr2=0x7fc360074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc35effd640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc360102790 0x7fc360074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc35effd640 1 -- 172.21.15.5:0/892535990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3601a6840 con 0x7fc360101de0 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc35effd640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc360101de0 0x7fc3600744f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc35000c920 tx=0x7fc35000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc347fff640 1 -- 172.21.15.5:0/892535990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350007c70 con 0x7fc360101de0 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc365772640 1 -- 172.21.15.5:0/892535990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3601a6b30 con 0x7fc360101de0 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc347fff640 1 -- 172.21.15.5:0/892535990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc350007e10 con 0x7fc360101de0 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc347fff640 1 -- 172.21.15.5:0/892535990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc350018890 con 0x7fc360101de0 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.815+0000 7fc365772640 1 -- 172.21.15.5:0/892535990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3601a7070 con 0x7fc360101de0 2024-09-15T22:13:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.816+0000 7fc365772640 1 -- 172.21.15.5:0/892535990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3601021c0 con 0x7fc360101de0 2024-09-15T22:13:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.818+0000 7fc347fff640 1 -- 172.21.15.5:0/892535990 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc350018a50 con 0x7fc360101de0 2024-09-15T22:13:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.818+0000 7fc347fff640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3480a33b0 0x7fc3480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.819+0000 7fc35e7fc640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3480a33b0 0x7fc3480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.819+0000 7fc347fff640 1 -- 172.21.15.5:0/892535990 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fc3500d04f0 con 0x7fc360101de0 2024-09-15T22:13:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.819+0000 7fc35e7fc640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3480a33b0 0x7fc3480a5870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc354000f80 tx=0x7fc3540093f0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:24.822+0000 7fc347fff640 1 -- 172.21.15.5:0/892535990 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc350014030 con 0x7fc360101de0 2024-09-15T22:13:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:25.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.042+0000 7fc365772640 1 -- 172.21.15.5:0/892535990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc360075a20 con 0x7fc360101de0 2024-09-15T22:13:25.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.042+0000 7fc347fff640 1 -- 172.21.15.5:0/892535990 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fc360075a20 con 0x7fc360101de0 2024-09-15T22:13:25.046 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:13:25.049 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:13:25.050 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:13:25.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 -- 172.21.15.5:0/892535990 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3480a33b0 msgr2=0x7fc3480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:25.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3480a33b0 0x7fc3480a5870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc354000f80 tx=0x7fc3540093f0 comp rx=0 tx=0).stop 2024-09-15T22:13:25.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 -- 172.21.15.5:0/892535990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc360101de0 msgr2=0x7fc3600744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:25.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc360101de0 0x7fc3600744f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc35000c920 tx=0x7fc35000cde0 comp rx=0 tx=0).stop 2024-09-15T22:13:25.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 -- 172.21.15.5:0/892535990 shutdown_connections 2024-09-15T22:13:25.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3480a33b0 0x7fc3480a5870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:25.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc360106f10 0x7fc360074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:25.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc360102790 0x7fc360074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:25.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 --2- 172.21.15.5:0/892535990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc360101de0 0x7fc3600744f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:25.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 -- 172.21.15.5:0/892535990 >> 172.21.15.5:0/892535990 conn(0x7fc3600fd8c0 msgr2=0x7fc3600fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:25.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 -- 172.21.15.5:0/892535990 shutdown_connections 2024-09-15T22:13:25.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:25.045+0000 7fc365772640 1 -- 172.21.15.5:0/892535990 wait complete. 2024-09-15T22:13:25.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:24 smithi099 ceph-mon[34892]: pgmap v138: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 581 op/s 2024-09-15T22:13:25.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:24 smithi129 ceph-mon[35314]: pgmap v138: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 5.2 MiB/s wr, 581 op/s 2024-09-15T22:13:25.934 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id 61222043 --release --await' 2024-09-15T22:13:25.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id 61222043 --release --await 2024-09-15T22:13:25.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:25 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/892535990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:25.996 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:25 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/892535990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:26.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:25 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/892535990' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:26.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.085+0000 7f6649897640 1 -- 172.21.15.5:0/3032380739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644106f10 msgr2=0x7f664410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.085+0000 7f6649897640 1 --2- 172.21.15.5:0/3032380739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644106f10 0x7f664410ea90 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f663c009ca0 tx=0x7f663c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.085+0000 7f6649897640 1 -- 172.21.15.5:0/3032380739 shutdown_connections 2024-09-15T22:13:26.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.085+0000 7f6649897640 1 --2- 172.21.15.5:0/3032380739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644106f10 0x7f664410ea90 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.085+0000 7f6649897640 1 --2- 172.21.15.5:0/3032380739 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6644102790 0x7f66441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.085+0000 7f6649897640 1 --2- 172.21.15.5:0/3032380739 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6644101de0 0x7f66441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.085+0000 7f6649897640 1 -- 172.21.15.5:0/3032380739 >> 172.21.15.5:0/3032380739 conn(0x7f66440fd8c0 msgr2=0x7f66440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.085+0000 7f6649897640 1 -- 172.21.15.5:0/3032380739 shutdown_connections 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.085+0000 7f6649897640 1 -- 172.21.15.5:0/3032380739 wait complete. 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f6649897640 1 Processor -- start 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f6649897640 1 -- start start 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f6649897640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6644101de0 0x7f664419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f6649897640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6644102790 0x7f664419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f6649897640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644106f10 0x7f664419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f6649897640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6644111f70 con 0x7f6644106f10 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f6649897640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6644111df0 con 0x7f6644102790 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f6649897640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66441120f0 con 0x7f6644101de0 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f66437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644106f10 0x7f664419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f66437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644106f10 0x7f664419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60012/0 (socket says 172.21.15.5:60012) 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f66437fe640 1 -- 172.21.15.5:0/2192303158 learned_addr learned my addr 172.21.15.5:0/2192303158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f6642ffd640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6644101de0 0x7f664419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f66427fc640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6644102790 0x7f664419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f66437fe640 1 -- 172.21.15.5:0/2192303158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6644101de0 msgr2=0x7f664419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f66437fe640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6644101de0 0x7f664419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f66437fe640 1 -- 172.21.15.5:0/2192303158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6644102790 msgr2=0x7f664419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f66437fe640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6644102790 0x7f664419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.086+0000 7f66437fe640 1 -- 172.21.15.5:0/2192303158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664419f5f0 con 0x7f6644106f10 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.087+0000 7f66437fe640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644106f10 0x7f664419ef10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f663c031a20 tx=0x7f663c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.087+0000 7f6648895640 1 -- 172.21.15.5:0/2192303158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c003c30 con 0x7f6644106f10 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.087+0000 7f6648895640 1 -- 172.21.15.5:0/2192303158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c003dd0 con 0x7f6644106f10 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.087+0000 7f6649897640 1 -- 172.21.15.5:0/2192303158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664419f880 con 0x7f6644106f10 2024-09-15T22:13:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.087+0000 7f6648895640 1 -- 172.21.15.5:0/2192303158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c02fcf0 con 0x7f6644106f10 2024-09-15T22:13:26.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.087+0000 7f6649897640 1 -- 172.21.15.5:0/2192303158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66441a71c0 con 0x7f6644106f10 2024-09-15T22:13:26.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.089+0000 7f6648895640 1 -- 172.21.15.5:0/2192303158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f663c004050 con 0x7f6644106f10 2024-09-15T22:13:26.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.090+0000 7f664bc70740 1 Processor -- start 2024-09-15T22:13:26.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.090+0000 7f664bc70740 1 -- start start 2024-09-15T22:13:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.090+0000 7f6648895640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 0x7f662c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.091+0000 7f6648895640 1 -- 172.21.15.5:0/2192303158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f663c0f1300 con 0x7f6644106f10 2024-09-15T22:13:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.091+0000 7f66427fc640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 0x7f662c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.091+0000 7f664bc70740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa991090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.091+0000 7f664bc70740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa9915d0 0x55c9aa991a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.091+0000 7f664bc70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa99c370 0x55c9aa99e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.091+0000 7f664bc70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa8f72f0 con 0x55c9aa99c370 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.091+0000 7f664bc70740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa89ecf0 con 0x55c9aa9915d0 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.091+0000 7f664bc70740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa8f32b0 con 0x55c9aa992b40 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.091+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa991090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.091+0000 7f66437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa99c370 0x55c9aa99e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa991090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53436/0 (socket says 172.21.15.5:53436) 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f6642ffd640 1 -- 172.21.15.5:0/3177933991 learned_addr learned my addr 172.21.15.5:0/3177933991 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f66427fc640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 0x7f662c0a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6638005fd0 tx=0x7f663800b510 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f66427fc640 1 --2- 172.21.15.5:0/3177933991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa9915d0 0x55c9aa991a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f66437fe640 1 -- 172.21.15.5:0/3177933991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 msgr2=0x55c9aa991090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f66437fe640 1 --2- 172.21.15.5:0/3177933991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa991090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f66437fe640 1 -- 172.21.15.5:0/3177933991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa9915d0 msgr2=0x55c9aa991a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f66437fe640 1 --2- 172.21.15.5:0/3177933991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa9915d0 0x55c9aa991a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f66437fe640 1 -- 172.21.15.5:0/3177933991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9aa8833a0 con 0x55c9aa99c370 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f6642ffd640 1 --2- 172.21.15.5:0/3177933991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa991090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f66437fe640 1 --2- 172.21.15.5:0/3177933991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa99c370 0x55c9aa99e760 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f663c0ee7c0 tx=0x7f663c0bd270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925ad732ca63af69 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f6629ffb640 1 -- 172.21.15.5:0/3177933991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0f1160 con 0x55c9aa99c370 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f6629ffb640 1 -- 172.21.15.5:0/3177933991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c036070 con 0x55c9aa99c370 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.092+0000 7f6629ffb640 1 -- 172.21.15.5:0/3177933991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c0ee930 con 0x55c9aa99c370 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.093+0000 7f664bc70740 1 -- 172.21.15.5:0/3177933991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa99c370 msgr2=0x55c9aa99e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.093+0000 7f664bc70740 1 --2- 172.21.15.5:0/3177933991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa99c370 0x55c9aa99e760 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f663c0ee7c0 tx=0x7f663c0bd270 comp rx=0 tx=0).stop 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.093+0000 7f664bc70740 1 -- 172.21.15.5:0/3177933991 shutdown_connections 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.093+0000 7f664bc70740 1 --2- 172.21.15.5:0/3177933991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa99c370 0x55c9aa99e760 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.093+0000 7f664bc70740 1 --2- 172.21.15.5:0/3177933991 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa9915d0 0x55c9aa991a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.093+0000 7f664bc70740 1 --2- 172.21.15.5:0/3177933991 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa991090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.093+0000 7f664bc70740 1 -- 172.21.15.5:0/3177933991 >> 172.21.15.5:0/3177933991 conn(0x55c9aa95a310 msgr2=0x55c9aa98dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.093+0000 7f664bc70740 1 -- 172.21.15.5:0/3177933991 shutdown_connections 2024-09-15T22:13:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.093+0000 7f664bc70740 1 -- 172.21.15.5:0/3177933991 wait complete. 2024-09-15T22:13:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.093+0000 7f664bc70740 1 -- start start 2024-09-15T22:13:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f664bc70740 1 Processor -- start 2024-09-15T22:13:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f664bc70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa99bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f664bc70740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa992b40 0x55c9aa999070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f664bc70740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa99c370 0x55c9aa999800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f664bc70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa8f72f0 con 0x55c9aa9915d0 2024-09-15T22:13:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f664bc70740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa8b2060 con 0x55c9aa992b40 2024-09-15T22:13:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f664bc70740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c9aa99ee30 con 0x55c9aa99c370 2024-09-15T22:13:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f664bc70740 10 client.0 _ll_register_callbacks cb 0x55c9aa6058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa99bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa99bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60028/0 (socket says 172.21.15.5:60028) 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f6642ffd640 1 -- 172.21.15.5:0/4069192825 learned_addr learned my addr 172.21.15.5:0/4069192825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f66427fc640 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa992b40 0x55c9aa999070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.094+0000 7f66437fe640 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa99c370 0x55c9aa999800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f6642ffd640 1 -- 172.21.15.5:0/4069192825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa99c370 msgr2=0x55c9aa999800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f6642ffd640 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa99c370 0x55c9aa999800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f6642ffd640 1 -- 172.21.15.5:0/4069192825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa992b40 msgr2=0x55c9aa999070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f6642ffd640 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa992b40 0x55c9aa999070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f6642ffd640 1 -- 172.21.15.5:0/4069192825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9aa6683c0 con 0x55c9aa9915d0 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f66427fc640 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa992b40 0x55c9aa999070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f6642ffd640 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa99bdc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6634010a50 tx=0x7f6634010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42c2ac60eaa64d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f6622ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f6622ffd640 1 -- 172.21.15.5:0/4069192825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663401e060 con 0x55c9aa9915d0 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f6622ffd640 1 -- 172.21.15.5:0/4069192825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f663400fbd0 con 0x55c9aa9915d0 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f664bc70740 1 -- 172.21.15.5:0/4069192825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663c036070 con 0x55c9aa9915d0 2024-09-15T22:13:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f6622ffd640 1 -- 172.21.15.5:0/4069192825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6634025050 con 0x55c9aa9915d0 2024-09-15T22:13:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f664bc70740 10 client.16305 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f664bc70740 1 -- 172.21.15.5:0/4069192825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9aa9aa760 con 0x55c9aa9915d0 2024-09-15T22:13:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.095+0000 7f6622ffd640 1 -- 172.21.15.5:0/4069192825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c036070 con 0x55c9aa9915d0 2024-09-15T22:13:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.096+0000 7f6622ffd640 1 -- 172.21.15.5:0/4069192825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f663400e3b0 con 0x55c9aa9915d0 2024-09-15T22:13:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.096+0000 7f664bc70740 10 client.16305 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.096+0000 7f664bc70740 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.096+0000 7f664bc70740 20 client.16305 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.096+0000 7f664bc70740 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c9aa9abc70 0x55c9aa9ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.096+0000 7f664bc70740 4 client.16305 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.096+0000 7f664bc70740 1 -- 172.21.15.5:0/4069192825 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9aa904860 con 0x55c9aa9abc70 2024-09-15T22:13:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.096+0000 7f66427fc640 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c9aa9abc70 0x55c9aa9ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.096+0000 7f6622ffd640 10 client.16305 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.096+0000 7f66427fc640 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c9aa9abc70 0x55c9aa9ae0f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e32a2575bb5daf9d server_cookie=8d7f8d18ac14af44 in_seq=0 out_seq=0 2024-09-15T22:13:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.097+0000 7f6622ffd640 1 -- 172.21.15.5:0/4069192825 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9aa904860 con 0x55c9aa9abc70 2024-09-15T22:13:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.097+0000 7f6622ffd640 10 client.16305 handle_command_reply: tid=0 2024-09-15T22:13:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.097+0000 7f664bc70740 1 client.16305 shutdown 2024-09-15T22:13:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.097+0000 7f664bc70740 1 -- 172.21.15.5:0/4069192825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 msgr2=0x55c9aa99bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.097+0000 7f664bc70740 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa99bdc0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6634010a50 tx=0x7f6634010f10 comp rx=0 tx=0).stop 2024-09-15T22:13:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.097+0000 7f664bc70740 1 -- 172.21.15.5:0/4069192825 shutdown_connections 2024-09-15T22:13:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.097+0000 7f664bc70740 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa99c370 0x55c9aa999800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.097+0000 7f664bc70740 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c9aa9abc70 0x55c9aa9ae0f0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.098+0000 7f6622ffd640 0 client.16305 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.098+0000 7f664bc70740 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa992b40 0x55c9aa999070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.098+0000 7f664bc70740 1 --2- 172.21.15.5:0/4069192825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa99bdc0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.098+0000 7f664bc70740 1 -- 172.21.15.5:0/4069192825 >> 172.21.15.5:0/4069192825 conn(0x55c9aa95a310 msgr2=0x55c9aa993990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.098+0000 7f664bc70740 1 -- 172.21.15.5:0/4069192825 shutdown_connections 2024-09-15T22:13:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.098+0000 7f664bc70740 1 -- 172.21.15.5:0/4069192825 wait complete. 2024-09-15T22:13:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.098+0000 7f664bc70740 20 client.16305 trim_cache size 0 max 16384 2024-09-15T22:13:26.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f664bc70740 1 Processor -- start 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f664bc70740 1 -- start start 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f664bc70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa992300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f664bc70740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa9b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f664bc70740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa99c370 0x55c9aa9b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f664bc70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66441b3af0 con 0x55c9aa9915d0 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f664bc70740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66441b13c0 con 0x55c9aa99c370 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f664bc70740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66441af210 con 0x55c9aa992b40 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa992300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa992300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60040/0 (socket says 172.21.15.5:60040) 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f6642ffd640 1 -- 172.21.15.5:0/2879832469 learned_addr learned my addr 172.21.15.5:0/2879832469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f66427fc640 1 --2- 172.21.15.5:0/2879832469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa9b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f66437fe640 1 --2- 172.21.15.5:0/2879832469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa99c370 0x55c9aa9b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f6642ffd640 1 -- 172.21.15.5:0/2879832469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 msgr2=0x55c9aa9b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f6642ffd640 1 --2- 172.21.15.5:0/2879832469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa9b39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f6642ffd640 1 -- 172.21.15.5:0/2879832469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa99c370 msgr2=0x55c9aa9b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.111+0000 7f6642ffd640 1 --2- 172.21.15.5:0/2879832469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa99c370 0x55c9aa9b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.112+0000 7f6642ffd640 1 -- 172.21.15.5:0/2879832469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9aa994120 con 0x55c9aa9915d0 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.112+0000 7f66427fc640 1 --2- 172.21.15.5:0/2879832469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa9b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.112+0000 7f6642ffd640 1 --2- 172.21.15.5:0/2879832469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa992300 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f66340058c0 tx=0x7f663400a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9fe5d54ed71b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.112+0000 7f66297fa640 1 -- 172.21.15.5:0/2879832469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663402e070 con 0x55c9aa9915d0 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.112+0000 7f66297fa640 1 -- 172.21.15.5:0/2879832469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663400e470 con 0x55c9aa9915d0 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.112+0000 7f66297fa640 1 -- 172.21.15.5:0/2879832469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634005490 con 0x55c9aa9915d0 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.113+0000 7f664bc70740 1 -- 172.21.15.5:0/2879832469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 msgr2=0x55c9aa992300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.113+0000 7f664bc70740 1 --2- 172.21.15.5:0/2879832469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa992300 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f66340058c0 tx=0x7f663400a4c0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.113+0000 7f664bc70740 1 -- 172.21.15.5:0/2879832469 shutdown_connections 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.113+0000 7f664bc70740 1 --2- 172.21.15.5:0/2879832469 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa99c370 0x55c9aa9b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.113+0000 7f664bc70740 1 --2- 172.21.15.5:0/2879832469 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa9b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.113+0000 7f664bc70740 1 --2- 172.21.15.5:0/2879832469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa992300 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.113+0000 7f664bc70740 1 -- 172.21.15.5:0/2879832469 >> 172.21.15.5:0/2879832469 conn(0x55c9aa95a310 msgr2=0x55c9aa988a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.113+0000 7f664bc70740 1 -- 172.21.15.5:0/2879832469 shutdown_connections 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.113+0000 7f664bc70740 1 -- 172.21.15.5:0/2879832469 wait complete. 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.113+0000 7f664bc70740 1 -- start start 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.113+0000 7f664bc70740 1 Processor -- start 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f664bc70740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa9a4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f664bc70740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa9a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f664bc70740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa99c370 0x55c9aa9958a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f664bc70740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66441af880 con 0x55c9aa9915d0 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f664bc70740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66441b0cf0 con 0x55c9aa99c370 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f664bc70740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66441aff50 con 0x55c9aa992b40 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa9a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f664bc70740 10 client.0 _ll_register_callbacks cb 0x7f6634010660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa9a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60042/0 (socket says 172.21.15.5:60042) 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f6642ffd640 1 -- 172.21.15.5:0/681237190 learned_addr learned my addr 172.21.15.5:0/681237190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f66437fe640 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa99c370 0x55c9aa9958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f66427fc640 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa9a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f6642ffd640 1 -- 172.21.15.5:0/681237190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 msgr2=0x55c9aa9a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f6642ffd640 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa9a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f6642ffd640 1 -- 172.21.15.5:0/681237190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa99c370 msgr2=0x55c9aa9958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f6642ffd640 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa99c370 0x55c9aa9958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f6642ffd640 1 -- 172.21.15.5:0/681237190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9aa996080 con 0x55c9aa9915d0 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f66437fe640 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa99c370 0x55c9aa9958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f66427fc640 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa9a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f6642ffd640 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa9a4260 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6634001eb0 tx=0x7f663400fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fcfab4c36b06e5e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.114+0000 7f6622ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f6622ffd640 1 -- 172.21.15.5:0/681237190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634037060 con 0x55c9aa9915d0 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f6622ffd640 1 -- 172.21.15.5:0/681237190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f663400e940 con 0x55c9aa9915d0 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f664bc70740 1 -- 172.21.15.5:0/681237190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663400e470 con 0x55c9aa9915d0 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f6622ffd640 1 -- 172.21.15.5:0/681237190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f663403e050 con 0x55c9aa9915d0 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f664bc70740 10 client.16311 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f664bc70740 1 -- 172.21.15.5:0/681237190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9aaa163f0 con 0x55c9aa9915d0 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f6622ffd640 1 -- 172.21.15.5:0/681237190 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663400e470 con 0x55c9aa9915d0 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f6622ffd640 1 -- 172.21.15.5:0/681237190 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6634004820 con 0x55c9aa9915d0 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f664bc70740 10 client.16311 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f664bc70740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f664bc70740 20 client.16311 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f664bc70740 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c9aaa16ad0 0x55c9aaa18ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f664bc70740 4 client.16311 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:13:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.115+0000 7f664bc70740 1 -- 172.21.15.5:0/681237190 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c9aa8fed70 con 0x55c9aaa16ad0 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.116+0000 7f66427fc640 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c9aaa16ad0 0x55c9aaa18ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.116+0000 7f66427fc640 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c9aaa16ad0 0x55c9aaa18ef0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=367afe622fbaab28 server_cookie=addc21fb670844ba in_seq=0 out_seq=0 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.116+0000 7f6622ffd640 10 client.16311 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.116+0000 7f6622ffd640 1 -- 172.21.15.5:0/681237190 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c9aa8fed70 con 0x55c9aaa16ad0 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.116+0000 7f6622ffd640 10 client.16311 handle_command_reply: tid=0 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.116+0000 7f664bc70740 1 client.16311 shutdown 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.116+0000 7f664bc70740 1 -- 172.21.15.5:0/681237190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 msgr2=0x55c9aa9a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.116+0000 7f664bc70740 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa9a4260 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6634001eb0 tx=0x7f663400fe00 comp rx=0 tx=0).stop 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.116+0000 7f664bc70740 1 -- 172.21.15.5:0/681237190 shutdown_connections 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f664bc70740 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c9aaa16ad0 0x55c9aaa18ef0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f664bc70740 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c9aa99c370 0x55c9aa9958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f6622ffd640 0 client.16311 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f664bc70740 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c9aa992b40 0x55c9aa9a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f664bc70740 1 --2- 172.21.15.5:0/681237190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9aa9915d0 0x55c9aa9a4260 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f664bc70740 1 -- 172.21.15.5:0/681237190 >> 172.21.15.5:0/681237190 conn(0x55c9aa95a310 msgr2=0x55c9aa997940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f664bc70740 1 -- 172.21.15.5:0/681237190 shutdown_connections 2024-09-15T22:13:26.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f664bc70740 1 -- 172.21.15.5:0/681237190 wait complete. 2024-09-15T22:13:26.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f664bc70740 20 client.16311 trim_cache size 0 max 16384 2024-09-15T22:13:26.126 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f6649897640 1 -- 172.21.15.5:0/2192303158 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 msgr2=0x7f662c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f6649897640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 0x7f662c0a5870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6638005fd0 tx=0x7f663800b510 comp rx=0 tx=0).stop 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f6649897640 1 -- 172.21.15.5:0/2192303158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644106f10 msgr2=0x7f664419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f6649897640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644106f10 0x7f664419ef10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f663c031a20 tx=0x7f663c031a50 comp rx=0 tx=0).stop 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f6649897640 1 -- 172.21.15.5:0/2192303158 shutdown_connections 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f6649897640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 0x7f662c0a5870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f6649897640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644106f10 0x7f664419ef10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f6649897640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6644102790 0x7f664419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f6649897640 1 --2- 172.21.15.5:0/2192303158 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6644101de0 0x7f664419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.117+0000 7f6649897640 1 -- 172.21.15.5:0/2192303158 >> 172.21.15.5:0/2192303158 conn(0x7f66440fd8c0 msgr2=0x7f66440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.118+0000 7f6649897640 1 -- 172.21.15.5:0/2192303158 shutdown_connections 2024-09-15T22:13:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:26.118+0000 7f6649897640 1 -- 172.21.15.5:0/2192303158 wait complete. 2024-09-15T22:13:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:26.994 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:26 smithi005 ceph-mon[30944]: pgmap v139: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.6 MiB/s wr, 629 op/s 2024-09-15T22:13:27.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:26 smithi099 ceph-mon[34892]: pgmap v139: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.6 MiB/s wr, 629 op/s 2024-09-15T22:13:27.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:26 smithi129 ceph-mon[35314]: pgmap v139: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 5.6 MiB/s wr, 629 op/s 2024-09-15T22:13:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.366+0000 7f245d43c640 1 -- 172.21.15.5:0/456038584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2458102570 msgr2=0x7f24581029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.366+0000 7f245d43c640 1 --2- 172.21.15.5:0/456038584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2458102570 0x7f24581029b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2440009900 tx=0x7f244002a480 comp rx=0 tx=0).stop 2024-09-15T22:13:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.366+0000 7f245d43c640 1 -- 172.21.15.5:0/456038584 shutdown_connections 2024-09-15T22:13:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.366+0000 7f245d43c640 1 --2- 172.21.15.5:0/456038584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2458102ef0 0x7f245810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.366+0000 7f245d43c640 1 --2- 172.21.15.5:0/456038584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2458102570 0x7f24581029b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.366+0000 7f245d43c640 1 --2- 172.21.15.5:0/456038584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2458100e20 0x7f2458101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.366+0000 7f245d43c640 1 -- 172.21.15.5:0/456038584 >> 172.21.15.5:0/456038584 conn(0x7f24580fc7d0 msgr2=0x7f24580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.366+0000 7f245d43c640 1 -- 172.21.15.5:0/456038584 shutdown_connections 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.366+0000 7f245d43c640 1 -- 172.21.15.5:0/456038584 wait complete. 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f245d43c640 1 Processor -- start 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f245d43c640 1 -- start start 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f245d43c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2458100e20 0x7f24580fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f245d43c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2458102570 0x7f24580fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f245d43c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2458102ef0 0x7f24580fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f245d43c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2458110f80 con 0x7f2458102570 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f245d43c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2458110e00 con 0x7f2458100e20 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f245d43c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2458111100 con 0x7f2458102ef0 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f244ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2458102570 0x7f24580fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f2456ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2458100e20 0x7f24580fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f244ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2458102570 0x7f24580fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60058/0 (socket says 172.21.15.5:60058) 2024-09-15T22:13:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f244ffff640 1 -- 172.21.15.5:0/3190093417 learned_addr learned my addr 172.21.15.5:0/3190093417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:27.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f24577fe640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2458102ef0 0x7f24580fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f2456ffd640 1 -- 172.21.15.5:0/3190093417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2458102ef0 msgr2=0x7f24580fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f2456ffd640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2458102ef0 0x7f24580fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f2456ffd640 1 -- 172.21.15.5:0/3190093417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2458102570 msgr2=0x7f24580fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f2456ffd640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2458102570 0x7f24580fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f2456ffd640 1 -- 172.21.15.5:0/3190093417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24581a6990 con 0x7f2458100e20 2024-09-15T22:13:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.367+0000 7f244ffff640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2458102570 0x7f24580fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.368+0000 7f2456ffd640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2458100e20 0x7f24580fcc80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f244800d720 tx=0x7f244800dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.368+0000 7f2454ff9640 1 -- 172.21.15.5:0/3190093417 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2448017070 con 0x7f2458100e20 2024-09-15T22:13:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.368+0000 7f2454ff9640 1 -- 172.21.15.5:0/3190093417 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f244800de30 con 0x7f2458100e20 2024-09-15T22:13:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.368+0000 7f245d43c640 1 -- 172.21.15.5:0/3190093417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24581a6c80 con 0x7f2458100e20 2024-09-15T22:13:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.368+0000 7f2454ff9640 1 -- 172.21.15.5:0/3190093417 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24480047a0 con 0x7f2458100e20 2024-09-15T22:13:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.368+0000 7f245d43c640 1 -- 172.21.15.5:0/3190093417 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24581a7170 con 0x7f2458100e20 2024-09-15T22:13:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.370+0000 7f2454ff9640 1 -- 172.21.15.5:0/3190093417 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f24480099d0 con 0x7f2458100e20 2024-09-15T22:13:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.370+0000 7f245f815740 1 Processor -- start 2024-09-15T22:13:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f245f815740 1 -- start start 2024-09-15T22:13:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f245f815740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 0x5643c700b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f245f815740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c7015d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f245f815740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c7016240 0x5643c7018630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f245f815740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5643c6f70620 con 0x5643c700b980 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f245f815740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5643c6f17e50 con 0x5643c700b060 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f245f815740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5643c6f6c5e0 con 0x5643c7016240 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f24577fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 0x5643c700b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f24577fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 0x5643c700b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:39078/0 (socket says 172.21.15.5:39078) 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f24577fe640 1 -- 172.21.15.5:0/2935799015 learned_addr learned my addr 172.21.15.5:0/2935799015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f2456ffd640 1 --2- 172.21.15.5:0/2935799015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c7015d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f244ffff640 1 --2- 172.21.15.5:0/2935799015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c7016240 0x5643c7018630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.371+0000 7f2454ff9640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24440a33b0 0x7f24440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f24577fe640 1 -- 172.21.15.5:0/2935799015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c7016240 msgr2=0x5643c7018630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f24577fe640 1 --2- 172.21.15.5:0/2935799015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c7016240 0x5643c7018630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f24577fe640 1 -- 172.21.15.5:0/2935799015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 msgr2=0x5643c7015d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f24577fe640 1 --2- 172.21.15.5:0/2935799015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c7015d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f24577fe640 1 -- 172.21.15.5:0/2935799015 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c6f08d40 con 0x5643c700b060 2024-09-15T22:13:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f244ffff640 1 --2- 172.21.15.5:0/2935799015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c7016240 0x5643c7018630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f24577fe640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24440a33b0 0x7f24440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f2454ff9640 1 -- 172.21.15.5:0/3190093417 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f24480cfcd0 con 0x7f2458100e20 2024-09-15T22:13:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f24577fe640 1 --2- 172.21.15.5:0/2935799015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 0x5643c700b440 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f24500104a0 tx=0x7f2450010960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9524a8315eac5756 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f244d7fa640 1 -- 172.21.15.5:0/2935799015 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245001d910 con 0x5643c700b060 2024-09-15T22:13:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f244d7fa640 1 -- 172.21.15.5:0/2935799015 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2450011440 con 0x5643c700b060 2024-09-15T22:13:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.372+0000 7f244d7fa640 1 -- 172.21.15.5:0/2935799015 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245001c6a0 con 0x5643c700b060 2024-09-15T22:13:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f245f815740 1 -- 172.21.15.5:0/2935799015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 msgr2=0x5643c700b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f245f815740 1 --2- 172.21.15.5:0/2935799015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 0x5643c700b440 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f24500104a0 tx=0x7f2450010960 comp rx=0 tx=0).stop 2024-09-15T22:13:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f24577fe640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24440a33b0 0x7f24440a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2450021000 tx=0x7f245001e000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f245f815740 1 -- 172.21.15.5:0/2935799015 shutdown_connections 2024-09-15T22:13:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f245f815740 1 --2- 172.21.15.5:0/2935799015 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c7016240 0x5643c7018630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f245f815740 1 --2- 172.21.15.5:0/2935799015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c7015d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f245f815740 1 --2- 172.21.15.5:0/2935799015 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 0x5643c700b440 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f245f815740 1 -- 172.21.15.5:0/2935799015 >> 172.21.15.5:0/2935799015 conn(0x5643c70075f0 msgr2=0x5643c70079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f245f815740 1 -- 172.21.15.5:0/2935799015 shutdown_connections 2024-09-15T22:13:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f245f815740 1 -- 172.21.15.5:0/2935799015 wait complete. 2024-09-15T22:13:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f245f815740 1 -- start start 2024-09-15T22:13:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.373+0000 7f245f815740 1 Processor -- start 2024-09-15T22:13:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f245f815740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c700b060 0x5643c7013110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f245f815740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b980 0x5643c7013650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f245f815740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c7016240 0x5643c7013b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f245f815740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5643c7018e10 con 0x5643c7016240 2024-09-15T22:13:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f245f815740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5643c6ef8f10 con 0x5643c700b980 2024-09-15T22:13:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f245f815740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5643c7018f90 con 0x5643c700b060 2024-09-15T22:13:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f245f815740 10 client.0 _ll_register_callbacks cb 0x5643c6f1cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f24577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c7016240 0x5643c7013b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f24577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c7016240 0x5643c7013b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60074/0 (socket says 172.21.15.5:60074) 2024-09-15T22:13:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f24577fe640 1 -- 172.21.15.5:0/252813686 learned_addr learned my addr 172.21.15.5:0/252813686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f244ffff640 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b980 0x5643c7013650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f2456ffd640 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c700b060 0x5643c7013110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f24577fe640 1 -- 172.21.15.5:0/252813686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c700b060 msgr2=0x5643c7013110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f24577fe640 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c700b060 0x5643c7013110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f24577fe640 1 -- 172.21.15.5:0/252813686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b980 msgr2=0x5643c7013650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f24577fe640 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b980 0x5643c7013650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f24577fe640 1 -- 172.21.15.5:0/252813686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c6c7f580 con 0x5643c7016240 2024-09-15T22:13:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f24577fe640 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c7016240 0x5643c7013b90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2450011ea0 tx=0x7f245001c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f029c48ccbcf6c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.374+0000 7f243e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f243e7fc640 1 -- 172.21.15.5:0/252813686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24500136c0 con 0x5643c7016240 2024-09-15T22:13:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f243e7fc640 1 -- 172.21.15.5:0/252813686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2450023040 con 0x5643c7016240 2024-09-15T22:13:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f245f815740 1 -- 172.21.15.5:0/252813686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2450011440 con 0x5643c7016240 2024-09-15T22:13:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f243e7fc640 1 -- 172.21.15.5:0/252813686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2450013260 con 0x5643c7016240 2024-09-15T22:13:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f245f815740 10 client.16323 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f245f815740 1 -- 172.21.15.5:0/252813686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643c6d49960 con 0x5643c7016240 2024-09-15T22:13:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f243e7fc640 1 -- 172.21.15.5:0/252813686 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2450011440 con 0x5643c7016240 2024-09-15T22:13:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f243e7fc640 1 -- 172.21.15.5:0/252813686 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f245001d890 con 0x5643c7016240 2024-09-15T22:13:27.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f245f815740 10 client.16323 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:27.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f245f815740 10 client.16323 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:13:27.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f245f815740 20 client.16323 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:27.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f245f815740 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5643c7025c50 0x5643c7028040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f245f815740 4 client.16323 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:27.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.375+0000 7f245f815740 1 -- 172.21.15.5:0/252813686 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643c6f7db90 con 0x5643c7025c50 2024-09-15T22:13:27.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.376+0000 7f2456ffd640 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5643c7025c50 0x5643c7028040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.376+0000 7f243e7fc640 10 client.16323 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.376+0000 7f2456ffd640 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5643c7025c50 0x5643c7028040 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=308f1d83c579c944 server_cookie=277390920a632bf8 in_seq=0 out_seq=0 2024-09-15T22:13:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f243e7fc640 1 -- 172.21.15.5:0/252813686 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643c6f7db90 con 0x5643c7025c50 2024-09-15T22:13:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f243e7fc640 10 client.16323 handle_command_reply: tid=0 2024-09-15T22:13:27.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 1 client.16323 shutdown 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 1 -- 172.21.15.5:0/252813686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c7016240 msgr2=0x5643c7013b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c7016240 0x5643c7013b90 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2450011ea0 tx=0x7f245001c670 comp rx=0 tx=0).stop 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 1 -- 172.21.15.5:0/252813686 shutdown_connections 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c7016240 0x5643c7013b90 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5643c7025c50 0x5643c7028040 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b980 0x5643c7013650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 1 --2- 172.21.15.5:0/252813686 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c700b060 0x5643c7013110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 1 -- 172.21.15.5:0/252813686 >> 172.21.15.5:0/252813686 conn(0x5643c70075f0 msgr2=0x5643c700d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f243e7fc640 0 client.16323 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 1 -- 172.21.15.5:0/252813686 shutdown_connections 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 1 -- 172.21.15.5:0/252813686 wait complete. 2024-09-15T22:13:27.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.377+0000 7f245f815740 20 client.16323 trim_cache size 0 max 16384 2024-09-15T22:13:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.394+0000 7f245f815740 1 Processor -- start 2024-09-15T22:13:27.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.394+0000 7f245f815740 1 -- start start 2024-09-15T22:13:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.394+0000 7f245f815740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c700b060 0x5643c700dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.394+0000 7f245f815740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c7011fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.394+0000 7f245f815740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c7016240 0x5643c702d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.394+0000 7f245f815740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24581b3a80 con 0x5643c700b980 2024-09-15T22:13:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.394+0000 7f245f815740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24581b1350 con 0x5643c7016240 2024-09-15T22:13:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.394+0000 7f245f815740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f24581af1a0 con 0x5643c700b060 2024-09-15T22:13:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.394+0000 7f244ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c7011fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.394+0000 7f244ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c7011fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60090/0 (socket says 172.21.15.5:60090) 2024-09-15T22:13:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.394+0000 7f244ffff640 1 -- 172.21.15.5:0/234700085 learned_addr learned my addr 172.21.15.5:0/234700085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:27.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.395+0000 7f24577fe640 1 --2- 172.21.15.5:0/234700085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c7016240 0x5643c702d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.395+0000 7f2456ffd640 1 --2- 172.21.15.5:0/234700085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c700b060 0x5643c700dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.395+0000 7f244ffff640 1 -- 172.21.15.5:0/234700085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c700b060 msgr2=0x5643c700dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.395+0000 7f244ffff640 1 --2- 172.21.15.5:0/234700085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c700b060 0x5643c700dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.395+0000 7f244ffff640 1 -- 172.21.15.5:0/234700085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c7016240 msgr2=0x5643c702d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.395+0000 7f244ffff640 1 --2- 172.21.15.5:0/234700085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c7016240 0x5643c702d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.395+0000 7f244ffff640 1 -- 172.21.15.5:0/234700085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c7005bb0 con 0x5643c700b980 2024-09-15T22:13:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.395+0000 7f244ffff640 1 --2- 172.21.15.5:0/234700085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c7011fc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2440005770 tx=0x7f2440004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4703e39984d77b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.395+0000 7f244cff9640 1 -- 172.21.15.5:0/234700085 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440035040 con 0x5643c700b980 2024-09-15T22:13:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.395+0000 7f244cff9640 1 -- 172.21.15.5:0/234700085 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2440031540 con 0x5643c700b980 2024-09-15T22:13:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.395+0000 7f244cff9640 1 -- 172.21.15.5:0/234700085 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244003c6a0 con 0x5643c700b980 2024-09-15T22:13:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.396+0000 7f245f815740 1 -- 172.21.15.5:0/234700085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 msgr2=0x5643c7011fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.396+0000 7f245f815740 1 --2- 172.21.15.5:0/234700085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c7011fc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2440005770 tx=0x7f2440004610 comp rx=0 tx=0).stop 2024-09-15T22:13:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.396+0000 7f245f815740 1 -- 172.21.15.5:0/234700085 shutdown_connections 2024-09-15T22:13:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.396+0000 7f245f815740 1 --2- 172.21.15.5:0/234700085 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c7016240 0x5643c702d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.396+0000 7f245f815740 1 --2- 172.21.15.5:0/234700085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c7011fc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.396+0000 7f245f815740 1 --2- 172.21.15.5:0/234700085 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c700b060 0x5643c700dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.396+0000 7f245f815740 1 -- 172.21.15.5:0/234700085 >> 172.21.15.5:0/234700085 conn(0x5643c70075f0 msgr2=0x5643c70118b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.396+0000 7f245f815740 1 -- 172.21.15.5:0/234700085 shutdown_connections 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.396+0000 7f245f815740 1 -- 172.21.15.5:0/234700085 wait complete. 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.396+0000 7f245f815740 1 -- start start 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.396+0000 7f245f815740 1 Processor -- start 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f245f815740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 0x5643c701e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f245f815740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c701ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f245f815740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c7016240 0x5643c7019b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f245f815740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24581af810 con 0x5643c700b980 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f245f815740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f24581b0c80 con 0x5643c700b060 2024-09-15T22:13:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f245f815740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f24581afee0 con 0x5643c7016240 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f245f815740 10 client.0 _ll_register_callbacks cb 0x7f2450009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f244ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c701ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f244ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c701ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60094/0 (socket says 172.21.15.5:60094) 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f244ffff640 1 -- 172.21.15.5:0/3843079759 learned_addr learned my addr 172.21.15.5:0/3843079759 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f2456ffd640 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 0x5643c701e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f24577fe640 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c7016240 0x5643c7019b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f244ffff640 1 -- 172.21.15.5:0/3843079759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c7016240 msgr2=0x5643c7019b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f244ffff640 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c7016240 0x5643c7019b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f244ffff640 1 -- 172.21.15.5:0/3843079759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 msgr2=0x5643c701e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f244ffff640 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 0x5643c701e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f244ffff640 1 -- 172.21.15.5:0/3843079759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c701a340 con 0x5643c700b980 2024-09-15T22:13:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f2456ffd640 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 0x5643c701e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f244ffff640 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c701ead0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2440006010 tx=0x7f2440004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a34c4c0f3f0f581 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.397+0000 7f243e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f243e7fc640 1 -- 172.21.15.5:0/3843079759 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440053060 con 0x5643c700b980 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f243e7fc640 1 -- 172.21.15.5:0/3843079759 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2440003a60 con 0x5643c700b980 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f245f815740 1 -- 172.21.15.5:0/3843079759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2440031540 con 0x5643c700b980 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f243e7fc640 1 -- 172.21.15.5:0/3843079759 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f244005a050 con 0x5643c700b980 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f245f815740 10 client.16329 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f245f815740 1 -- 172.21.15.5:0/3843079759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643c7090480 con 0x5643c700b980 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f243e7fc640 1 -- 172.21.15.5:0/3843079759 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2440031540 con 0x5643c700b980 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f243e7fc640 1 -- 172.21.15.5:0/3843079759 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2440044070 con 0x5643c700b980 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f245f815740 10 client.16329 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f245f815740 10 client.16329 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f245f815740 20 client.16329 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f245f815740 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5643c7090bd0 0x5643c7092ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f245f815740 4 client.16329 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "61222043", "release": true, "await": true}] 2024-09-15T22:13:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.398+0000 7f245f815740 1 -- 172.21.15.5:0/3843079759 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "61222043", "release": true, "await": true}) -- 0x5643c6f780a0 con 0x5643c7090bd0 2024-09-15T22:13:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.399+0000 7f2456ffd640 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5643c7090bd0 0x5643c7092ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.399+0000 7f243e7fc640 10 client.16329 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.399+0000 7f2456ffd640 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5643c7090bd0 0x5643c7092ff0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01b8e3dc1c43de1 server_cookie=acfc99834395277f in_seq=0 out_seq=0 2024-09-15T22:13:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f243e7fc640 1 -- 172.21.15.5:0/3843079759 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5643c6f780a0 con 0x5643c7090bd0 2024-09-15T22:13:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f243e7fc640 10 client.16329 handle_command_reply: tid=0 2024-09-15T22:13:27.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 1 client.16329 shutdown 2024-09-15T22:13:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 1 -- 172.21.15.5:0/3843079759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 msgr2=0x5643c701ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c701ead0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2440006010 tx=0x7f2440004510 comp rx=0 tx=0).stop 2024-09-15T22:13:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 1 -- 172.21.15.5:0/3843079759 shutdown_connections 2024-09-15T22:13:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5643c7016240 0x5643c7019b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c700b980 0x5643c701ead0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5643c7090bd0 0x5643c7092ff0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 1 --2- 172.21.15.5:0/3843079759 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5643c700b060 0x5643c701e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 1 -- 172.21.15.5:0/3843079759 >> 172.21.15.5:0/3843079759 conn(0x5643c70075f0 msgr2=0x5643c7013880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f243e7fc640 0 client.16329 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 1 -- 172.21.15.5:0/3843079759 shutdown_connections 2024-09-15T22:13:27.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 1 -- 172.21.15.5:0/3843079759 wait complete. 2024-09-15T22:13:27.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.476+0000 7f245f815740 20 client.16329 trim_cache size 0 max 16384 2024-09-15T22:13:27.483 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:13:27.483 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:13:27.484 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:13:27.484 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 16, 2024-09-15T22:13:27.484 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:13:27.484 INFO:teuthology.orchestra.run.smithi005.stdout: "61222043": { 2024-09-15T22:13:27.484 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 16, 2024-09-15T22:13:27.485 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:13:27.485 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:13:27.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:13:27.485 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:13:27.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:13:27.486 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:13:27.486 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 133.8, 2024-09-15T22:13:27.486 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:13:27.486 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:13:27.486 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:13:27.486 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:13:27.487 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:13:27.487 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:13:27.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:27.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:27.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:27.487 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:27.488 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:27.488 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:13:27.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.477+0000 7f245d43c640 1 -- 172.21.15.5:0/3190093417 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24440a33b0 msgr2=0x7f24440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.477+0000 7f245d43c640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24440a33b0 0x7f24440a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2450021000 tx=0x7f245001e000 comp rx=0 tx=0).stop 2024-09-15T22:13:27.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.477+0000 7f245d43c640 1 -- 172.21.15.5:0/3190093417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2458100e20 msgr2=0x7f24580fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:27.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.477+0000 7f245d43c640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2458100e20 0x7f24580fcc80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f244800d720 tx=0x7f244800dbe0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.477+0000 7f245d43c640 1 -- 172.21.15.5:0/3190093417 shutdown_connections 2024-09-15T22:13:27.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.477+0000 7f245d43c640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f24440a33b0 0x7f24440a5870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.477+0000 7f245d43c640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2458102ef0 0x7f24580fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.477+0000 7f245d43c640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2458102570 0x7f24580fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.477+0000 7f245d43c640 1 --2- 172.21.15.5:0/3190093417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2458100e20 0x7f24580fcc80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:27.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.477+0000 7f245d43c640 1 -- 172.21.15.5:0/3190093417 >> 172.21.15.5:0/3190093417 conn(0x7f24580fc7d0 msgr2=0x7f245806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:27.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.477+0000 7f245d43c640 1 -- 172.21.15.5:0/3190093417 shutdown_connections 2024-09-15T22:13:27.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:27.480+0000 7f245d43c640 1 -- 172.21.15.5:0/3190093417 wait complete. 2024-09-15T22:13:27.838 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:13:27.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:13:28.455 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '61222043', total seconds elapsed: 18.1 2024-09-15T22:13:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:29.054 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:28 smithi005 ceph-mon[30944]: pgmap v140: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-09-15T22:13:29.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:28 smithi099 ceph-mon[34892]: pgmap v140: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-09-15T22:13:29.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:28 smithi129 ceph-mon[35314]: pgmap v140: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-09-15T22:13:29.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.212+0000 7f2dbb28d640 1 -- 172.21.15.5:0/583160493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2db411b140 msgr2=0x7f2db411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.212+0000 7f2dbb28d640 1 --2- 172.21.15.5:0/583160493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2db411b140 0x7f2db411d530 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009ca0 tx=0x7f2dac02a8f0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.213+0000 7f2dbb28d640 1 -- 172.21.15.5:0/583160493 shutdown_connections 2024-09-15T22:13:29.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.213+0000 7f2dbb28d640 1 --2- 172.21.15.5:0/583160493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2db411b140 0x7f2db411d530 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.213+0000 7f2dbb28d640 1 --2- 172.21.15.5:0/583160493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db4109830 0x7f2db4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.213+0000 7f2dbb28d640 1 --2- 172.21.15.5:0/583160493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2db4109450 0x7f2db4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.213+0000 7f2dbb28d640 1 -- 172.21.15.5:0/583160493 >> 172.21.15.5:0/583160493 conn(0x7f2db406e140 msgr2=0x7f2db406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.217+0000 7f2dbb28d640 1 -- 172.21.15.5:0/583160493 shutdown_connections 2024-09-15T22:13:29.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.217+0000 7f2dbb28d640 1 -- 172.21.15.5:0/583160493 wait complete. 2024-09-15T22:13:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.218+0000 7f2dbb28d640 1 Processor -- start 2024-09-15T22:13:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.218+0000 7f2dbb28d640 1 -- start start 2024-09-15T22:13:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.218+0000 7f2dbb28d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2db4109450 0x7f2db41a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.218+0000 7f2dbb28d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2db4109830 0x7f2db41a7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.218+0000 7f2dbb28d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db411b140 0x7f2db41abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.218+0000 7f2dbb28d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4120de0 con 0x7f2db4109830 2024-09-15T22:13:29.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.218+0000 7f2dbb28d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4120c60 con 0x7f2db411b140 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.218+0000 7f2dbb28d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2db4120f60 con 0x7f2db4109450 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.219+0000 7f2db982a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db411b140 0x7f2db41abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.219+0000 7f2db982a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db411b140 0x7f2db41abbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:39126/0 (socket says 172.21.15.5:39126) 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.219+0000 7f2db982a640 1 -- 172.21.15.5:0/1917747731 learned_addr learned my addr 172.21.15.5:0/1917747731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.219+0000 7f2db8828640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2db4109830 0x7f2db41a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.219+0000 7f2db982a640 1 -- 172.21.15.5:0/1917747731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2db4109450 msgr2=0x7f2db41a72e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.219+0000 7f2db982a640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2db4109450 0x7f2db41a72e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.219+0000 7f2db982a640 1 -- 172.21.15.5:0/1917747731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2db4109830 msgr2=0x7f2db41a7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.219+0000 7f2db982a640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2db4109830 0x7f2db41a7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.219+0000 7f2db982a640 1 -- 172.21.15.5:0/1917747731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2db41ac2b0 con 0x7f2db411b140 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.219+0000 7f2db8828640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2db4109830 0x7f2db41a7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:29.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.219+0000 7f2db982a640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db411b140 0x7f2db41abbb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2dac031000 tx=0x7f2dac0307c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.220+0000 7f2daa7fc640 1 -- 172.21.15.5:0/1917747731 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac031890 con 0x7f2db411b140 2024-09-15T22:13:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.220+0000 7f2dbb28d640 1 -- 172.21.15.5:0/1917747731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2db41ac540 con 0x7f2db411b140 2024-09-15T22:13:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.220+0000 7f2daa7fc640 1 -- 172.21.15.5:0/1917747731 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dac02d070 con 0x7f2db411b140 2024-09-15T22:13:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.220+0000 7f2daa7fc640 1 -- 172.21.15.5:0/1917747731 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac03b820 con 0x7f2db411b140 2024-09-15T22:13:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.220+0000 7f2dbb28d640 1 -- 172.21.15.5:0/1917747731 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2db4110920 con 0x7f2db411b140 2024-09-15T22:13:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.226+0000 7f2daa7fc640 1 -- 172.21.15.5:0/1917747731 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2dac03b9c0 con 0x7f2db411b140 2024-09-15T22:13:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.227+0000 7f2daa7fc640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2da00a35c0 0x7f2da00a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.228+0000 7f2daa7fc640 1 -- 172.21.15.5:0/1917747731 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2dac0f1f40 con 0x7f2db411b140 2024-09-15T22:13:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.228+0000 7f2db9029640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2da00a35c0 0x7f2da00a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.231+0000 7f2db9029640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2da00a35c0 0x7f2da00a5a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009670 tx=0x7f2da4006bd0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.232+0000 7f2dbd666740 1 Processor -- start 2024-09-15T22:13:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.232+0000 7f2dbd666740 1 -- start start 2024-09-15T22:13:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.232+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc766f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.232+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 0x5625fc771800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.232+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc771d40 0x5625fc774130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.232+0000 7f2db9029640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc766f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.233+0000 7f2db9029640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc766f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:39138/0 (socket says 172.21.15.5:39138) 2024-09-15T22:13:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.233+0000 7f2db9029640 1 -- 172.21.15.5:0/260384207 learned_addr learned my addr 172.21.15.5:0/260384207 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.233+0000 7f2dbd666740 1 -- 172.21.15.5:0/260384207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5625fc6cc280 con 0x5625fc771d40 2024-09-15T22:13:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.233+0000 7f2dbd666740 1 -- 172.21.15.5:0/260384207 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5625fc673b10 con 0x5625fc766b60 2024-09-15T22:13:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.233+0000 7f2dbd666740 1 -- 172.21.15.5:0/260384207 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5625fc6c8240 con 0x5625fc767480 2024-09-15T22:13:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.239+0000 7f2db8828640 1 --2- 172.21.15.5:0/260384207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 0x5625fc771800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.239+0000 7f2db982a640 1 --2- 172.21.15.5:0/260384207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc771d40 0x5625fc774130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.240+0000 7f2db8828640 1 -- 172.21.15.5:0/260384207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 msgr2=0x5625fc766f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.240+0000 7f2db8828640 1 --2- 172.21.15.5:0/260384207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc766f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.240+0000 7f2db8828640 1 -- 172.21.15.5:0/260384207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc771d40 msgr2=0x5625fc774130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.240+0000 7f2db8828640 1 --2- 172.21.15.5:0/260384207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc771d40 0x5625fc774130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.240+0000 7f2db8828640 1 -- 172.21.15.5:0/260384207 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625fc6801f0 con 0x5625fc767480 2024-09-15T22:13:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.240+0000 7f2db8828640 1 --2- 172.21.15.5:0/260384207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 0x5625fc771800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2db000bcc0 tx=0x7f2db000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3114b80dccd39e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.240+0000 7f2d9f7fe640 1 -- 172.21.15.5:0/260384207 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0011440 con 0x5625fc767480 2024-09-15T22:13:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.240+0000 7f2d9f7fe640 1 -- 172.21.15.5:0/260384207 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db0011ac0 con 0x5625fc767480 2024-09-15T22:13:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.240+0000 7f2d9f7fe640 1 -- 172.21.15.5:0/260384207 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db001bbc0 con 0x5625fc767480 2024-09-15T22:13:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.241+0000 7f2dbd666740 1 -- 172.21.15.5:0/260384207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 msgr2=0x5625fc771800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.241+0000 7f2dbd666740 1 --2- 172.21.15.5:0/260384207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 0x5625fc771800 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2db000bcc0 tx=0x7f2db000f730 comp rx=0 tx=0).stop 2024-09-15T22:13:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.245+0000 7f2dbd666740 1 -- 172.21.15.5:0/260384207 shutdown_connections 2024-09-15T22:13:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.245+0000 7f2dbd666740 1 --2- 172.21.15.5:0/260384207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc771d40 0x5625fc774130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.245+0000 7f2dbd666740 1 --2- 172.21.15.5:0/260384207 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 0x5625fc771800 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.245+0000 7f2dbd666740 1 --2- 172.21.15.5:0/260384207 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc766f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.245+0000 7f2dbd666740 1 -- 172.21.15.5:0/260384207 >> 172.21.15.5:0/260384207 conn(0x5625fc7630f0 msgr2=0x5625fc7634d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.246+0000 7f2dbd666740 1 -- 172.21.15.5:0/260384207 shutdown_connections 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.246+0000 7f2dbd666740 1 -- 172.21.15.5:0/260384207 wait complete. 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.247+0000 7f2dbd666740 1 -- start start 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.247+0000 7f2dbd666740 1 Processor -- start 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.247+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc76eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.247+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 0x5625fc76f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.248+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc771d40 0x5625fc76f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.248+0000 7f2dbd666740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5625fc774910 con 0x5625fc767480 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.248+0000 7f2dbd666740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5625fc686f10 con 0x5625fc766b60 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.248+0000 7f2dbd666740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5625fc774a90 con 0x5625fc771d40 2024-09-15T22:13:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.248+0000 7f2dbd666740 10 client.0 _ll_register_callbacks cb 0x5625fc64fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.248+0000 7f2db9029640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc76eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.248+0000 7f2db9029640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc76eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:39152/0 (socket says 172.21.15.5:39152) 2024-09-15T22:13:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.248+0000 7f2db9029640 1 -- 172.21.15.5:0/243787898 learned_addr learned my addr 172.21.15.5:0/243787898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.248+0000 7f2db8828640 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 0x5625fc76f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.249+0000 7f2db982a640 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc771d40 0x5625fc76f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.249+0000 7f2db8828640 1 -- 172.21.15.5:0/243787898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc771d40 msgr2=0x5625fc76f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.249+0000 7f2db8828640 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc771d40 0x5625fc76f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.249+0000 7f2db8828640 1 -- 172.21.15.5:0/243787898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 msgr2=0x5625fc76eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.249+0000 7f2db8828640 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc76eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.249+0000 7f2db8828640 1 -- 172.21.15.5:0/243787898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625fc43d6a0 con 0x5625fc767480 2024-09-15T22:13:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.249+0000 7f2db982a640 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc771d40 0x5625fc76f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:13:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.249+0000 7f2db8828640 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 0x5625fc76f0e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2db0000c00 tx=0x7f2db000b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d3725f40a8045a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.249+0000 7f2d9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.251+0000 7f2d9cff9640 1 -- 172.21.15.5:0/243787898 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db0011c70 con 0x5625fc767480 2024-09-15T22:13:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.251+0000 7f2d9cff9640 1 -- 172.21.15.5:0/243787898 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2db0004020 con 0x5625fc767480 2024-09-15T22:13:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.251+0000 7f2dbd666740 1 -- 172.21.15.5:0/243787898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2db0011ac0 con 0x5625fc767480 2024-09-15T22:13:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.252+0000 7f2d9cff9640 1 -- 172.21.15.5:0/243787898 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2db0031050 con 0x5625fc767480 2024-09-15T22:13:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.252+0000 7f2dbd666740 10 client.16341 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.252+0000 7f2dbd666740 1 -- 172.21.15.5:0/243787898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5625fc7a2470 con 0x5625fc767480 2024-09-15T22:13:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.253+0000 7f2d9cff9640 1 -- 172.21.15.5:0/243787898 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2db0011ac0 con 0x5625fc767480 2024-09-15T22:13:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.253+0000 7f2d9cff9640 1 -- 172.21.15.5:0/243787898 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2db001bb40 con 0x5625fc767480 2024-09-15T22:13:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.253+0000 7f2dbd666740 10 client.16341 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.253+0000 7f2dbd666740 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.253+0000 7f2dbd666740 20 client.16341 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.253+0000 7f2dbd666740 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625fc7a38d0 0x5625fc7a5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.253+0000 7f2dbd666740 4 client.16341 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.253+0000 7f2dbd666740 1 -- 172.21.15.5:0/243787898 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625fc6d97f0 con 0x5625fc7a38d0 2024-09-15T22:13:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.254+0000 7f2db9029640 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625fc7a38d0 0x5625fc7a5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.262+0000 7f2d9cff9640 10 client.16341 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.262+0000 7f2db9029640 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625fc7a38d0 0x5625fc7a5cc0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d14310add347c server_cookie=fb7b8549e6ed397c in_seq=0 out_seq=0 2024-09-15T22:13:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.262+0000 7f2d9cff9640 1 -- 172.21.15.5:0/243787898 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625fc6d97f0 con 0x5625fc7a38d0 2024-09-15T22:13:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.262+0000 7f2d9cff9640 10 client.16341 handle_command_reply: tid=0 2024-09-15T22:13:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.262+0000 7f2dbd666740 1 client.16341 shutdown 2024-09-15T22:13:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2dbd666740 1 -- 172.21.15.5:0/243787898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 msgr2=0x5625fc76f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2dbd666740 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 0x5625fc76f0e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2db0000c00 tx=0x7f2db000b800 comp rx=0 tx=0).stop 2024-09-15T22:13:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2dbd666740 1 -- 172.21.15.5:0/243787898 shutdown_connections 2024-09-15T22:13:29.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2dbd666740 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc771d40 0x5625fc76f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2dbd666740 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 0x5625fc76f0e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2dbd666740 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625fc7a38d0 0x5625fc7a5cc0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2dbd666740 1 --2- 172.21.15.5:0/243787898 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc76eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2dbd666740 1 -- 172.21.15.5:0/243787898 >> 172.21.15.5:0/243787898 conn(0x5625fc7630f0 msgr2=0x5625fc762270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2d9cff9640 0 client.16341 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:29.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2dbd666740 1 -- 172.21.15.5:0/243787898 shutdown_connections 2024-09-15T22:13:29.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2dbd666740 1 -- 172.21.15.5:0/243787898 wait complete. 2024-09-15T22:13:29.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.263+0000 7f2dbd666740 20 client.16341 trim_cache size 0 max 16384 2024-09-15T22:13:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.292+0000 7f2dbd666740 1 Processor -- start 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.292+0000 7f2dbd666740 1 -- start start 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.292+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc768cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.292+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 0x5625fc7ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.292+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc771d40 0x5625fc7ab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db9029640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc768cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2dbd666740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d980128d0 con 0x5625fc767480 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2dbd666740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d980101a0 con 0x5625fc766b60 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2dbd666740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9800dff0 con 0x5625fc771d40 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db9029640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc768cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:39160/0 (socket says 172.21.15.5:39160) 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db9029640 1 -- 172.21.15.5:0/3563720502 learned_addr learned my addr 172.21.15.5:0/3563720502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db8828640 1 --2- 172.21.15.5:0/3563720502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 0x5625fc7ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db982a640 1 --2- 172.21.15.5:0/3563720502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc771d40 0x5625fc7ab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db9029640 1 -- 172.21.15.5:0/3563720502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc771d40 msgr2=0x5625fc7ab830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db9029640 1 --2- 172.21.15.5:0/3563720502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc771d40 0x5625fc7ab830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db9029640 1 -- 172.21.15.5:0/3563720502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 msgr2=0x5625fc7ab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db9029640 1 --2- 172.21.15.5:0/3563720502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 0x5625fc7ab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db9029640 1 -- 172.21.15.5:0/3563720502 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625fc762f20 con 0x5625fc766b60 2024-09-15T22:13:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db8828640 1 --2- 172.21.15.5:0/3563720502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 0x5625fc7ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.293+0000 7f2db9029640 1 --2- 172.21.15.5:0/3563720502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc768cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009f00 tx=0x7f2da4012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a34b4ef0bddbbe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.294+0000 7f2d9effd640 1 -- 172.21.15.5:0/3563720502 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da4030070 con 0x5625fc766b60 2024-09-15T22:13:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.294+0000 7f2d9effd640 1 -- 172.21.15.5:0/3563720502 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da4022c50 con 0x5625fc766b60 2024-09-15T22:13:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.294+0000 7f2d9effd640 1 -- 172.21.15.5:0/3563720502 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da400b710 con 0x5625fc766b60 2024-09-15T22:13:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.295+0000 7f2dbd666740 1 -- 172.21.15.5:0/3563720502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 msgr2=0x5625fc768cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.295+0000 7f2dbd666740 1 --2- 172.21.15.5:0/3563720502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc768cc0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009f00 tx=0x7f2da4012000 comp rx=0 tx=0).stop 2024-09-15T22:13:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.295+0000 7f2dbd666740 1 -- 172.21.15.5:0/3563720502 shutdown_connections 2024-09-15T22:13:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.295+0000 7f2dbd666740 1 --2- 172.21.15.5:0/3563720502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc771d40 0x5625fc7ab830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.295+0000 7f2dbd666740 1 --2- 172.21.15.5:0/3563720502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc767480 0x5625fc7ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.295+0000 7f2dbd666740 1 --2- 172.21.15.5:0/3563720502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc766b60 0x5625fc768cc0 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009f00 tx=0x7f2da4012000 comp rx=0 tx=0).stop 2024-09-15T22:13:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.295+0000 7f2dbd666740 1 -- 172.21.15.5:0/3563720502 >> 172.21.15.5:0/3563720502 conn(0x5625fc7630f0 msgr2=0x5625fc7685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.295+0000 7f2dbd666740 1 -- 172.21.15.5:0/3563720502 shutdown_connections 2024-09-15T22:13:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.295+0000 7f2dbd666740 1 -- 172.21.15.5:0/3563720502 wait complete. 2024-09-15T22:13:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.295+0000 7f2dbd666740 1 -- start start 2024-09-15T22:13:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.295+0000 7f2dbd666740 1 Processor -- start 2024-09-15T22:13:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.297+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 0x5625fc775da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.297+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc771d40 0x5625fc7762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.297+0000 7f2dbd666740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc776870 0x5625fc80d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.297+0000 7f2dbd666740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9800e660 con 0x5625fc771d40 2024-09-15T22:13:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.297+0000 7f2db9029640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 0x5625fc775da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.297+0000 7f2db982a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc776870 0x5625fc80d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.297+0000 7f2db9029640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 0x5625fc775da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53534/0 (socket says 172.21.15.5:53534) 2024-09-15T22:13:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.297+0000 7f2db9029640 1 -- 172.21.15.5:0/3421206940 learned_addr learned my addr 172.21.15.5:0/3421206940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.297+0000 7f2dbd666740 1 -- 172.21.15.5:0/3421206940 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9800fad0 con 0x5625fc776870 2024-09-15T22:13:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.298+0000 7f2db8828640 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc771d40 0x5625fc7762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.298+0000 7f2dbd666740 1 -- 172.21.15.5:0/3421206940 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2d9800ed30 con 0x5625fc767480 2024-09-15T22:13:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.298+0000 7f2dbd666740 10 client.0 _ll_register_callbacks cb 0x7f2db0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.298+0000 7f2db9029640 1 -- 172.21.15.5:0/3421206940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc776870 msgr2=0x5625fc80d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.298+0000 7f2db9029640 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc776870 0x5625fc80d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.298+0000 7f2db9029640 1 -- 172.21.15.5:0/3421206940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc771d40 msgr2=0x5625fc7762e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:29.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.298+0000 7f2db9029640 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc771d40 0x5625fc7762e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.298+0000 7f2db9029640 1 -- 172.21.15.5:0/3421206940 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625fc80da30 con 0x5625fc767480 2024-09-15T22:13:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.298+0000 7f2db982a640 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc776870 0x5625fc80d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.299+0000 7f2db9029640 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 0x5625fc775da0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2da4006a50 tx=0x7f2da402bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41d2b2f3d87b9f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.299+0000 7f2d9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:13:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.299+0000 7f2d9cff9640 1 -- 172.21.15.5:0/3421206940 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2da4030070 con 0x5625fc767480 2024-09-15T22:13:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.299+0000 7f2d9cff9640 1 -- 172.21.15.5:0/3421206940 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2da402b690 con 0x5625fc767480 2024-09-15T22:13:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.299+0000 7f2dbd666740 1 -- 172.21.15.5:0/3421206940 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2da4022c50 con 0x5625fc767480 2024-09-15T22:13:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.299+0000 7f2d9cff9640 1 -- 172.21.15.5:0/3421206940 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2da402b890 con 0x5625fc767480 2024-09-15T22:13:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.299+0000 7f2dbd666740 10 client.25888 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.299+0000 7f2dbd666740 1 -- 172.21.15.5:0/3421206940 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5625fc80ed10 con 0x5625fc767480 2024-09-15T22:13:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.300+0000 7f2d9cff9640 1 -- 172.21.15.5:0/3421206940 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2da4022c50 con 0x5625fc767480 2024-09-15T22:13:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.300+0000 7f2d9cff9640 1 -- 172.21.15.5:0/3421206940 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2da403e030 con 0x5625fc767480 2024-09-15T22:13:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.301+0000 7f2dbd666740 10 client.25888 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.301+0000 7f2dbd666740 10 client.25888 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.301+0000 7f2dbd666740 20 client.25888 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.301+0000 7f2dbd666740 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625fc80f2d0 0x5625fc8116c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.301+0000 7f2dbd666740 4 client.25888 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.301+0000 7f2db8828640 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625fc80f2d0 0x5625fc8116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.301+0000 7f2dbd666740 1 -- 172.21.15.5:0/3421206940 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5625fc6d3d00 con 0x5625fc80f2d0 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.301+0000 7f2d9cff9640 10 client.25888 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.302+0000 7f2db8828640 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625fc80f2d0 0x5625fc8116c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76d05a0b764fabf1 server_cookie=31f6c90fba3d9c5 in_seq=0 out_seq=0 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2d9cff9640 1 -- 172.21.15.5:0/3421206940 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5625fc6d3d00 con 0x5625fc80f2d0 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2d9cff9640 10 client.25888 handle_command_reply: tid=0 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 1 client.25888 shutdown 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 1 -- 172.21.15.5:0/3421206940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 msgr2=0x5625fc775da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 0x5625fc775da0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2da4006a50 tx=0x7f2da402bca0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 1 -- 172.21.15.5:0/3421206940 shutdown_connections 2024-09-15T22:13:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5625fc80f2d0 0x5625fc8116c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5625fc776870 0x5625fc80d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625fc771d40 0x5625fc7762e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2d9cff9640 0 client.25888 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 1 --2- 172.21.15.5:0/3421206940 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5625fc767480 0x5625fc775da0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 1 -- 172.21.15.5:0/3421206940 >> 172.21.15.5:0/3421206940 conn(0x5625fc7630f0 msgr2=0x5625fc768dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 1 -- 172.21.15.5:0/3421206940 shutdown_connections 2024-09-15T22:13:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 1 -- 172.21.15.5:0/3421206940 wait complete. 2024-09-15T22:13:29.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.303+0000 7f2dbd666740 20 client.25888 trim_cache size 0 max 16384 2024-09-15T22:13:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:13:29.309 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:13:29.309 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "86931505-5925-4184-9e02-36cc1cb8054b", 2024-09-15T22:13:29.309 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:13:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.304+0000 7f2dbb28d640 1 -- 172.21.15.5:0/1917747731 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2da00a35c0 msgr2=0x7f2da00a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.304+0000 7f2dbb28d640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2da00a35c0 0x7f2da00a5a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2da4009670 tx=0x7f2da4006bd0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.304+0000 7f2dbb28d640 1 -- 172.21.15.5:0/1917747731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db411b140 msgr2=0x7f2db41abbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.304+0000 7f2dbb28d640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db411b140 0x7f2db41abbb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2dac031000 tx=0x7f2dac0307c0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.304+0000 7f2dbb28d640 1 -- 172.21.15.5:0/1917747731 shutdown_connections 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.304+0000 7f2dbb28d640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2da00a35c0 0x7f2da00a5a80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.304+0000 7f2dbb28d640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2db411b140 0x7f2db41abbb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.304+0000 7f2dbb28d640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2db4109830 0x7f2db41a7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.304+0000 7f2dbb28d640 1 --2- 172.21.15.5:0/1917747731 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2db4109450 0x7f2db41a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.304+0000 7f2dbb28d640 1 -- 172.21.15.5:0/1917747731 >> 172.21.15.5:0/1917747731 conn(0x7f2db406e140 msgr2=0x7f2db411bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.305+0000 7f2dbb28d640 1 -- 172.21.15.5:0/1917747731 shutdown_connections 2024-09-15T22:13:29.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:29.305+0000 7f2dbb28d640 1 -- 172.21.15.5:0/1917747731 wait complete. 2024-09-15T22:13:29.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 240.0 s, 916.1 tps, lat 196.941 ms stddev 1014.258 2024-09-15T22:13:30.234 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:13:30.763 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:31.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:30 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:13:31.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:30 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:31.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:30 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:13:31.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:30 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:31.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:30 smithi005 ceph-mon[30944]: pgmap v141: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-09-15T22:13:31.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:30 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:13:31.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:30 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:31.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:30 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:13:31.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:30 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:31.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:30 smithi099 ceph-mon[34892]: pgmap v141: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-09-15T22:13:31.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:30 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:13:31.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:30 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:31.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:30 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:13:31.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:30 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:13:31.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:30 smithi129 ceph-mon[35314]: pgmap v141: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 2.7 MiB/s wr, 313 op/s 2024-09-15T22:13:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.365+0000 7fecde581640 1 -- 172.21.15.5:0/2730061699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd806ea30 msgr2=0x7fecd806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.366+0000 7fecde581640 1 --2- 172.21.15.5:0/2730061699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd806ea30 0x7fecd806ee70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009900 tx=0x7fecd402a440 comp rx=0 tx=0).stop 2024-09-15T22:13:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.369+0000 7fecde581640 1 -- 172.21.15.5:0/2730061699 shutdown_connections 2024-09-15T22:13:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.369+0000 7fecde581640 1 --2- 172.21.15.5:0/2730061699 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fecd811b140 0x7fecd811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.369+0000 7fecde581640 1 --2- 172.21.15.5:0/2730061699 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd806ea30 0x7fecd806ee70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.369+0000 7fecde581640 1 --2- 172.21.15.5:0/2730061699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecd8110190 0x7fecd806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.369+0000 7fecde581640 1 -- 172.21.15.5:0/2730061699 >> 172.21.15.5:0/2730061699 conn(0x7fecd8073180 msgr2=0x7fecd8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.373+0000 7fecde581640 1 -- 172.21.15.5:0/2730061699 shutdown_connections 2024-09-15T22:13:31.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.373+0000 7fecde581640 1 -- 172.21.15.5:0/2730061699 wait complete. 2024-09-15T22:13:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.374+0000 7fecde581640 1 Processor -- start 2024-09-15T22:13:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.375+0000 7fecde581640 1 -- start start 2024-09-15T22:13:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.375+0000 7fecde581640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecd806ea30 0x7fecd81a7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.375+0000 7fecde581640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fecd8110190 0x7fecd81a7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.375+0000 7fecde581640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd811b140 0x7fecd81ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.375+0000 7fecde581640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fecd8120c60 con 0x7fecd806ea30 2024-09-15T22:13:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.375+0000 7fecde581640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fecd8120ae0 con 0x7fecd811b140 2024-09-15T22:13:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.375+0000 7fecde581640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fecd8120de0 con 0x7fecd8110190 2024-09-15T22:13:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.376+0000 7fecddd80640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd811b140 0x7fecd81ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.376+0000 7fecddd80640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd811b140 0x7fecd81ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:39192/0 (socket says 172.21.15.5:39192) 2024-09-15T22:13:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.376+0000 7fecddd80640 1 -- 172.21.15.5:0/2473005427 learned_addr learned my addr 172.21.15.5:0/2473005427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.378+0000 7fecddd80640 1 -- 172.21.15.5:0/2473005427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fecd8110190 msgr2=0x7fecd81a7550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:13:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.378+0000 7fecddd80640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fecd8110190 0x7fecd81a7550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.378+0000 7fecddd80640 1 -- 172.21.15.5:0/2473005427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecd806ea30 msgr2=0x7fecd81a7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.378+0000 7fecddd80640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecd806ea30 0x7fecd81a7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.378+0000 7fecddd80640 1 -- 172.21.15.5:0/2473005427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd81abfe0 con 0x7fecd811b140 2024-09-15T22:13:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.380+0000 7fecddd80640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd811b140 0x7fecd81ab8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fecd000ca00 tx=0x7fecd000cec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.380+0000 7fecce7fc640 1 -- 172.21.15.5:0/2473005427 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd000ad60 con 0x7fecd811b140 2024-09-15T22:13:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.380+0000 7fecce7fc640 1 -- 172.21.15.5:0/2473005427 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd0005020 con 0x7fecd811b140 2024-09-15T22:13:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.380+0000 7fecce7fc640 1 -- 172.21.15.5:0/2473005427 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd000f690 con 0x7fecd811b140 2024-09-15T22:13:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.380+0000 7fecde581640 1 -- 172.21.15.5:0/2473005427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd81ac2d0 con 0x7fecd811b140 2024-09-15T22:13:31.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.381+0000 7fecde581640 1 -- 172.21.15.5:0/2473005427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd8110920 con 0x7fecd811b140 2024-09-15T22:13:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.391+0000 7fece1bbc740 1 Processor -- start 2024-09-15T22:13:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.391+0000 7fece1bbc740 1 -- start start 2024-09-15T22:13:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.391+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a901da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.391+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9028310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.391+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a902ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.391+0000 7fece1bbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617a8f832e0 con 0x5617a901d670 2024-09-15T22:13:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.391+0000 7fece1bbc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5617a8f2ace0 con 0x5617a9028850 2024-09-15T22:13:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.391+0000 7fece1bbc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5617a8f7f2a0 con 0x5617a901df90 2024-09-15T22:13:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.392+0000 7fecdcd7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a901da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.392+0000 7fecdcd7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a901da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60172/0 (socket says 172.21.15.5:60172) 2024-09-15T22:13:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.392+0000 7fecdcd7e640 1 -- 172.21.15.5:0/1216122306 learned_addr learned my addr 172.21.15.5:0/1216122306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.392+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/1216122306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a902ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.392+0000 7fecce7fc640 1 -- 172.21.15.5:0/2473005427 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fecd000f850 con 0x7fecd811b140 2024-09-15T22:13:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.393+0000 7fecce7fc640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fecc40a35c0 0x7fecc40a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.393+0000 7fecce7fc640 1 -- 172.21.15.5:0/2473005427 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fecd00d1480 con 0x7fecd811b140 2024-09-15T22:13:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.396+0000 7fecdd57f640 1 --2- 172.21.15.5:0/1216122306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9028310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.396+0000 7fecdcd7e640 1 -- 172.21.15.5:0/1216122306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 msgr2=0x5617a9028310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.396+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/1216122306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9028310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.396+0000 7fecdcd7e640 1 -- 172.21.15.5:0/1216122306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 msgr2=0x5617a901da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.396+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/1216122306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a901da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.396+0000 7fecdcd7e640 1 -- 172.21.15.5:0/1216122306 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a8f0f440 con 0x5617a9028850 2024-09-15T22:13:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.396+0000 7fecddd80640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fecc40a35c0 0x7fecc40a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.396+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/1216122306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a902ac40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fecd402aa30 tx=0x7fecd4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a96a4175d838c76e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.397+0000 7fecc37fe640 1 -- 172.21.15.5:0/1216122306 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4035040 con 0x5617a9028850 2024-09-15T22:13:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.397+0000 7fecc37fe640 1 -- 172.21.15.5:0/1216122306 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd4004230 con 0x5617a9028850 2024-09-15T22:13:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.397+0000 7fecc37fe640 1 -- 172.21.15.5:0/1216122306 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd40038e0 con 0x5617a9028850 2024-09-15T22:13:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.398+0000 7fece1bbc740 1 -- 172.21.15.5:0/1216122306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 msgr2=0x5617a902ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.398+0000 7fece1bbc740 1 --2- 172.21.15.5:0/1216122306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a902ac40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fecd402aa30 tx=0x7fecd4005fd0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.399+0000 7fecdd57f640 1 --2- 172.21.15.5:0/1216122306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9028310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.401+0000 7fece1bbc740 1 -- 172.21.15.5:0/1216122306 shutdown_connections 2024-09-15T22:13:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.401+0000 7fece1bbc740 1 --2- 172.21.15.5:0/1216122306 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a902ac40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.401+0000 7fece1bbc740 1 --2- 172.21.15.5:0/1216122306 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9028310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.401+0000 7fece1bbc740 1 --2- 172.21.15.5:0/1216122306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a901da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.401+0000 7fece1bbc740 1 -- 172.21.15.5:0/1216122306 >> 172.21.15.5:0/1216122306 conn(0x5617a9019c00 msgr2=0x5617a9019fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.401+0000 7fecddd80640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fecc40a35c0 0x7fecc40a5a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fecd0099d30 tx=0x7fecd00996f0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.402+0000 7fece1bbc740 1 -- 172.21.15.5:0/1216122306 shutdown_connections 2024-09-15T22:13:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.402+0000 7fece1bbc740 1 -- 172.21.15.5:0/1216122306 wait complete. 2024-09-15T22:13:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.402+0000 7fece1bbc740 1 -- start start 2024-09-15T22:13:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.403+0000 7fece1bbc740 1 Processor -- start 2024-09-15T22:13:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.404+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a9028190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.404+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9025440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.404+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a9025980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.404+0000 7fece1bbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617a902b3c0 con 0x5617a901d670 2024-09-15T22:13:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.404+0000 7fecdd57f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9025440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.404+0000 7fecdd57f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9025440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53558/0 (socket says 172.21.15.5:53558) 2024-09-15T22:13:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.404+0000 7fecdd57f640 1 -- 172.21.15.5:0/2834817709 learned_addr learned my addr 172.21.15.5:0/2834817709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.404+0000 7fece1bbc740 1 -- 172.21.15.5:0/2834817709 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5617a8f3e050 con 0x5617a9028850 2024-09-15T22:13:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.404+0000 7fece1bbc740 1 -- 172.21.15.5:0/2834817709 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5617a902b540 con 0x5617a901df90 2024-09-15T22:13:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.405+0000 7fece1bbc740 10 client.0 _ll_register_callbacks cb 0x5617a8f7e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.405+0000 7fecdd57f640 1 -- 172.21.15.5:0/2834817709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 msgr2=0x5617a9025980 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:13:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.405+0000 7fecdd57f640 1 --2- 172.21.15.5:0/2834817709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a9025980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.405+0000 7fecdd57f640 1 -- 172.21.15.5:0/2834817709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 msgr2=0x5617a9028190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.405+0000 7fecdd57f640 1 --2- 172.21.15.5:0/2834817709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a9028190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.405+0000 7fecdd57f640 1 -- 172.21.15.5:0/2834817709 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a8cf43c0 con 0x5617a901df90 2024-09-15T22:13:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.405+0000 7fecdd57f640 1 --2- 172.21.15.5:0/2834817709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9025440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fecc800d230 tx=0x7fecc800d6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab4c5fe71929aaac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.405+0000 7fecc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:13:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.406+0000 7fecc0ff9640 1 -- 172.21.15.5:0/2834817709 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc80125f0 con 0x5617a901df90 2024-09-15T22:13:31.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.406+0000 7fece1bbc740 1 -- 172.21.15.5:0/2834817709 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecd4004230 con 0x5617a901df90 2024-09-15T22:13:31.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.406+0000 7fecc0ff9640 1 -- 172.21.15.5:0/2834817709 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fecc8012790 con 0x5617a901df90 2024-09-15T22:13:31.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.406+0000 7fecc0ff9640 1 -- 172.21.15.5:0/2834817709 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fecc8021050 con 0x5617a901df90 2024-09-15T22:13:31.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.406+0000 7fece1bbc740 10 client.25894 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:31.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.406+0000 7fece1bbc740 1 -- 172.21.15.5:0/2834817709 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617a9036e70 con 0x5617a901df90 2024-09-15T22:13:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.407+0000 7fecc0ff9640 1 -- 172.21.15.5:0/2834817709 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd4004230 con 0x5617a901df90 2024-09-15T22:13:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.407+0000 7fecc0ff9640 1 -- 172.21.15.5:0/2834817709 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fecc8013c40 con 0x5617a901df90 2024-09-15T22:13:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.407+0000 7fece1bbc740 10 client.25894 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.407+0000 7fece1bbc740 10 client.25894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.407+0000 7fece1bbc740 20 client.25894 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.407+0000 7fece1bbc740 1 --2- 172.21.15.5:0/2834817709 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5617a9038850 0x5617a903ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.407+0000 7fece1bbc740 4 client.25894 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.407+0000 7fece1bbc740 1 -- 172.21.15.5:0/2834817709 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617a8f90850 con 0x5617a9038850 2024-09-15T22:13:31.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.414+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/2834817709 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5617a9038850 0x5617a903ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.416+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/2834817709 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5617a9038850 0x5617a903ad10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a546da8608507375 server_cookie=f0f355e02b4c3c0a in_seq=0 out_seq=0 2024-09-15T22:13:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.416+0000 7fecc0ff9640 10 client.25894 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.416+0000 7fecc0ff9640 1 -- 172.21.15.5:0/2834817709 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617a8f90850 con 0x5617a9038850 2024-09-15T22:13:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.416+0000 7fecc0ff9640 10 client.25894 handle_command_reply: tid=0 2024-09-15T22:13:31.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.417+0000 7fece1bbc740 1 client.25894 shutdown 2024-09-15T22:13:31.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.417+0000 7fece1bbc740 1 -- 172.21.15.5:0/2834817709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 msgr2=0x5617a9025440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.417+0000 7fece1bbc740 1 --2- 172.21.15.5:0/2834817709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9025440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fecc800d230 tx=0x7fecc800d6f0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.418+0000 7fece1bbc740 1 -- 172.21.15.5:0/2834817709 shutdown_connections 2024-09-15T22:13:31.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.418+0000 7fece1bbc740 1 --2- 172.21.15.5:0/2834817709 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5617a9038850 0x5617a903ad10 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.418+0000 7fece1bbc740 1 --2- 172.21.15.5:0/2834817709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a9025980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.418+0000 7fece1bbc740 1 --2- 172.21.15.5:0/2834817709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9025440 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.418+0000 7fece1bbc740 1 --2- 172.21.15.5:0/2834817709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a9028190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.418+0000 7fece1bbc740 1 -- 172.21.15.5:0/2834817709 >> 172.21.15.5:0/2834817709 conn(0x5617a9019c00 msgr2=0x5617a901f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.418+0000 7fecc0ff9640 0 client.25894 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:31.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.418+0000 7fece1bbc740 1 -- 172.21.15.5:0/2834817709 shutdown_connections 2024-09-15T22:13:31.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.418+0000 7fece1bbc740 1 -- 172.21.15.5:0/2834817709 wait complete. 2024-09-15T22:13:31.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.420+0000 7fece1bbc740 20 client.25894 trim_cache size 0 max 16384 2024-09-15T22:13:31.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fece1bbc740 1 Processor -- start 2024-09-15T22:13:31.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fece1bbc740 1 -- start start 2024-09-15T22:13:31.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a90346b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9040620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a9040b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fece1bbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc0128c0 con 0x5617a901d670 2024-09-15T22:13:31.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fece1bbc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc010190 con 0x5617a9028850 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fece1bbc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc00dfe0 con 0x5617a901df90 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fecdcd7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a90346b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fecdcd7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a90346b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60190/0 (socket says 172.21.15.5:60190) 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fecdcd7e640 1 -- 172.21.15.5:0/2269754210 learned_addr learned my addr 172.21.15.5:0/2269754210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.447+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/2269754210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a9040b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.448+0000 7fecdd57f640 1 --2- 172.21.15.5:0/2269754210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9040620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.448+0000 7fecdcd7e640 1 -- 172.21.15.5:0/2269754210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 msgr2=0x5617a9040620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.448+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/2269754210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9040620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.448+0000 7fecdcd7e640 1 -- 172.21.15.5:0/2269754210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 msgr2=0x5617a9040b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.448+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/2269754210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a9040b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.448+0000 7fecdcd7e640 1 -- 172.21.15.5:0/2269754210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617a901ad70 con 0x5617a901d670 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.448+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/2269754210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a90346b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fecd4005480 tx=0x7fecd4004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee247ca1828418 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.448+0000 7fecc2ffd640 1 -- 172.21.15.5:0/2269754210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4035040 con 0x5617a901d670 2024-09-15T22:13:31.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.448+0000 7fecc2ffd640 1 -- 172.21.15.5:0/2269754210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd4004ba0 con 0x5617a901d670 2024-09-15T22:13:31.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.448+0000 7fecc2ffd640 1 -- 172.21.15.5:0/2269754210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4031ab0 con 0x5617a901d670 2024-09-15T22:13:31.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.449+0000 7fece1bbc740 1 -- 172.21.15.5:0/2269754210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 msgr2=0x5617a90346b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.449+0000 7fece1bbc740 1 --2- 172.21.15.5:0/2269754210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a90346b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fecd4005480 tx=0x7fecd4004ec0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.451+0000 7fece1bbc740 1 -- 172.21.15.5:0/2269754210 shutdown_connections 2024-09-15T22:13:31.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.451+0000 7fece1bbc740 1 --2- 172.21.15.5:0/2269754210 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a9040b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.451+0000 7fece1bbc740 1 --2- 172.21.15.5:0/2269754210 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a9040620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.451+0000 7fece1bbc740 1 --2- 172.21.15.5:0/2269754210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a90346b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.451+0000 7fece1bbc740 1 -- 172.21.15.5:0/2269754210 >> 172.21.15.5:0/2269754210 conn(0x5617a9019c00 msgr2=0x5617a9020a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.451+0000 7fece1bbc740 1 -- 172.21.15.5:0/2269754210 shutdown_connections 2024-09-15T22:13:31.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.451+0000 7fece1bbc740 1 -- 172.21.15.5:0/2269754210 wait complete. 2024-09-15T22:13:31.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.452+0000 7fece1bbc740 1 -- start start 2024-09-15T22:13:31.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.452+0000 7fece1bbc740 1 Processor -- start 2024-09-15T22:13:31.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a902c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a902cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fece1bbc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a902d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fece1bbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc00e650 con 0x5617a901d670 2024-09-15T22:13:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fece1bbc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc00fac0 con 0x5617a9028850 2024-09-15T22:13:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fece1bbc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc00ed20 con 0x5617a901df90 2024-09-15T22:13:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fece1bbc740 10 client.0 _ll_register_callbacks cb 0x7fecc800b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fecdcd7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a902c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fecdcd7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a902c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60194/0 (socket says 172.21.15.5:60194) 2024-09-15T22:13:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fecdcd7e640 1 -- 172.21.15.5:0/3376077516 learned_addr learned my addr 172.21.15.5:0/3376077516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a902d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fecdd57f640 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a902cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fecdcd7e640 1 -- 172.21.15.5:0/3376077516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 msgr2=0x5617a902cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a902cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fecdcd7e640 1 -- 172.21.15.5:0/3376077516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 msgr2=0x5617a902d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a902d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.453+0000 7fecdcd7e640 1 -- 172.21.15.5:0/3376077516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617a90a2670 con 0x5617a901d670 2024-09-15T22:13:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.454+0000 7fecdcd7e640 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a902c5f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fecd4004df0 tx=0x7fecd4009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84f44a5970df6b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.454+0000 7fecc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:13:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.454+0000 7fecc0ff9640 1 -- 172.21.15.5:0/3376077516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd4035070 con 0x5617a901d670 2024-09-15T22:13:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.454+0000 7fecc0ff9640 1 -- 172.21.15.5:0/3376077516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fecd4004ff0 con 0x5617a901d670 2024-09-15T22:13:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.454+0000 7fece1bbc740 1 -- 172.21.15.5:0/3376077516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecd4004ba0 con 0x5617a901d670 2024-09-15T22:13:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.454+0000 7fecc0ff9640 1 -- 172.21.15.5:0/3376077516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fecd405e040 con 0x5617a901d670 2024-09-15T22:13:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.454+0000 7fece1bbc740 10 client.16356 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:31.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.454+0000 7fece1bbc740 1 -- 172.21.15.5:0/3376077516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617a90a3960 con 0x5617a901d670 2024-09-15T22:13:31.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.456+0000 7fecc0ff9640 1 -- 172.21.15.5:0/3376077516 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd4004ba0 con 0x5617a901d670 2024-09-15T22:13:31.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.456+0000 7fecc0ff9640 1 -- 172.21.15.5:0/3376077516 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fecd4065470 con 0x5617a901d670 2024-09-15T22:13:31.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.456+0000 7fece1bbc740 10 client.16356 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:31.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.456+0000 7fece1bbc740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:13:31.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.456+0000 7fece1bbc740 20 client.16356 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:31.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.457+0000 7fece1bbc740 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5617a90a3b30 0x5617a90a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:31.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.457+0000 7fece1bbc740 4 client.16356 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:13:31.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.457+0000 7fece1bbc740 1 -- 172.21.15.5:0/3376077516 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5617a8f8ad60 con 0x5617a90a3b30 2024-09-15T22:13:31.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.457+0000 7fecdd57f640 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5617a90a3b30 0x5617a90a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:31.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.457+0000 7fecc0ff9640 10 client.16356 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.457+0000 7fecdd57f640 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5617a90a3b30 0x5617a90a5f20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a7429cb770f9eb server_cookie=75450941b9b3339d in_seq=0 out_seq=0 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fecc0ff9640 1 -- 172.21.15.5:0/3376077516 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5617a8f8ad60 con 0x5617a90a3b30 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fecc0ff9640 10 client.16356 handle_command_reply: tid=0 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 1 client.16356 shutdown 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 1 -- 172.21.15.5:0/3376077516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 msgr2=0x5617a902c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a902c5f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fecd4004df0 tx=0x7fecd4009520 comp rx=0 tx=0).stop 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 1 -- 172.21.15.5:0/3376077516 shutdown_connections 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5617a90a3b30 0x5617a90a5f20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5617a9028850 0x5617a902d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5617a901df90 0x5617a902cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 1 --2- 172.21.15.5:0/3376077516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617a901d670 0x5617a902c5f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fecc0ff9640 0 client.16356 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:31.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 1 -- 172.21.15.5:0/3376077516 >> 172.21.15.5:0/3376077516 conn(0x5617a9019c00 msgr2=0x5617a9018500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 1 -- 172.21.15.5:0/3376077516 shutdown_connections 2024-09-15T22:13:31.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 1 -- 172.21.15.5:0/3376077516 wait complete. 2024-09-15T22:13:31.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.458+0000 7fece1bbc740 20 client.16356 trim_cache size 0 max 16384 2024-09-15T22:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "86931505-5925-4184-9e02-36cc1cb8054b": { 2024-09-15T22:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:13:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "86931505-5925-4184-9e02-36cc1cb8054b", 2024-09-15T22:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:31.464 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:13:31.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.459+0000 7fecde581640 1 -- 172.21.15.5:0/2473005427 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fecc40a35c0 msgr2=0x7fecc40a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.459+0000 7fecde581640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fecc40a35c0 0x7fecc40a5a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fecd0099d30 tx=0x7fecd00996f0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.459+0000 7fecde581640 1 -- 172.21.15.5:0/2473005427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd811b140 msgr2=0x7fecd81ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:31.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.459+0000 7fecde581640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd811b140 0x7fecd81ab8e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fecd000ca00 tx=0x7fecd000cec0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.460+0000 7fecde581640 1 -- 172.21.15.5:0/2473005427 shutdown_connections 2024-09-15T22:13:31.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.460+0000 7fecde581640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fecc40a35c0 0x7fecc40a5a80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.460+0000 7fecde581640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fecd811b140 0x7fecd81ab8e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.460+0000 7fecde581640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fecd8110190 0x7fecd81a7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.460+0000 7fecde581640 1 --2- 172.21.15.5:0/2473005427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecd806ea30 0x7fecd81a7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.460+0000 7fecde581640 1 -- 172.21.15.5:0/2473005427 >> 172.21.15.5:0/2473005427 conn(0x7fecd8073180 msgr2=0x7fecd8119130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.460+0000 7fecde581640 1 -- 172.21.15.5:0/2473005427 shutdown_connections 2024-09-15T22:13:31.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:31.460+0000 7fecde581640 1 -- 172.21.15.5:0/2473005427 wait complete. 2024-09-15T22:13:32.392 INFO:tasks.cephfs.filesystem:scrub status for tag:86931505-5925-4184-9e02-36cc1cb8054b - {'path': '/', 'tag': '86931505-5925-4184-9e02-36cc1cb8054b', 'options': 'recursive,force'} 2024-09-15T22:13:33.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:32 smithi099 ceph-mon[34892]: pgmap v142: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-09-15T22:13:33.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:32 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:13:33.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:32 smithi129 ceph-mon[35314]: pgmap v142: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-09-15T22:13:33.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:32 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:13:33.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:32 smithi005 ceph-mon[30944]: pgmap v142: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 3.8 MiB/s wr, 427 op/s 2024-09-15T22:13:33.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:32 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:13:34.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 245.0 s, 1618.2 tps, lat 62.143 ms stddev 75.053 2024-09-15T22:13:35.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:34 smithi099 ceph-mon[34892]: pgmap v143: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-15T22:13:35.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:34 smithi129 ceph-mon[35314]: pgmap v143: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-15T22:13:35.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:34 smithi005 ceph-mon[30944]: pgmap v143: 129 pgs: 129 active+clean; 2.3 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-15T22:13:37.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:36 smithi099 ceph-mon[34892]: pgmap v144: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-09-15T22:13:37.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:36 smithi129 ceph-mon[35314]: pgmap v144: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-09-15T22:13:37.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:36 smithi005 ceph-mon[30944]: pgmap v144: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-09-15T22:13:39.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:38 smithi099 ceph-mon[34892]: pgmap v145: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.0 MiB/s wr, 425 op/s 2024-09-15T22:13:39.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:38 smithi129 ceph-mon[35314]: pgmap v145: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.0 MiB/s wr, 425 op/s 2024-09-15T22:13:39.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:38 smithi005 ceph-mon[30944]: pgmap v145: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.0 MiB/s wr, 425 op/s 2024-09-15T22:13:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 250.0 s, 2047.6 tps, lat 49.024 ms stddev 60.331 2024-09-15T22:13:41.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:40 smithi099 ceph-mon[34892]: pgmap v146: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.1 MiB/s wr, 444 op/s 2024-09-15T22:13:41.094 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:40 smithi129 ceph-mon[35314]: pgmap v146: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.1 MiB/s wr, 444 op/s 2024-09-15T22:13:41.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:40 smithi005 ceph-mon[30944]: pgmap v146: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.1 MiB/s wr, 444 op/s 2024-09-15T22:13:43.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:42 smithi099 ceph-mon[34892]: pgmap v147: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.6 MiB/s wr, 673 op/s 2024-09-15T22:13:43.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:42 smithi129 ceph-mon[35314]: pgmap v147: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.6 MiB/s wr, 673 op/s 2024-09-15T22:13:43.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:42 smithi005 ceph-mon[30944]: pgmap v147: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 7.6 MiB/s wr, 673 op/s 2024-09-15T22:13:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 255.0 s, 1549.8 tps, lat 64.255 ms stddev 75.666 2024-09-15T22:13:45.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:44 smithi099 ceph-mon[34892]: pgmap v148: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 513 op/s 2024-09-15T22:13:45.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:44 smithi129 ceph-mon[35314]: pgmap v148: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 513 op/s 2024-09-15T22:13:45.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:44 smithi005 ceph-mon[30944]: pgmap v148: 129 pgs: 129 active+clean; 2.4 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 6.2 MiB/s wr, 513 op/s 2024-09-15T22:13:47.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:46 smithi099 ceph-mon[34892]: pgmap v149: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 990 op/s 2024-09-15T22:13:47.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:46 smithi129 ceph-mon[35314]: pgmap v149: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 990 op/s 2024-09-15T22:13:47.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:46 smithi005 ceph-mon[30944]: pgmap v149: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 14 MiB/s wr, 990 op/s 2024-09-15T22:13:49.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:48 smithi099 ceph-mon[34892]: pgmap v150: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 737 op/s 2024-09-15T22:13:49.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:48 smithi129 ceph-mon[35314]: pgmap v150: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 737 op/s 2024-09-15T22:13:49.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:48 smithi005 ceph-mon[30944]: pgmap v150: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 12 MiB/s wr, 737 op/s 2024-09-15T22:13:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 260.0 s, 1262.8 tps, lat 79.235 ms stddev 100.759 2024-09-15T22:13:51.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:50 smithi099 ceph-mon[34892]: pgmap v151: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 778 op/s 2024-09-15T22:13:51.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:50 smithi129 ceph-mon[35314]: pgmap v151: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 778 op/s 2024-09-15T22:13:51.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:50 smithi005 ceph-mon[30944]: pgmap v151: 129 pgs: 129 active+clean; 2.5 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 12 MiB/s wr, 778 op/s 2024-09-15T22:13:53.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:52 smithi099 ceph-mon[34892]: pgmap v152: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-09-15T22:13:53.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:52 smithi129 ceph-mon[35314]: pgmap v152: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-09-15T22:13:53.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:52 smithi005 ceph-mon[30944]: pgmap v152: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 18 MiB/s wr, 1.13k op/s 2024-09-15T22:13:54.085 INFO:tasks.quiescer.fs.[cephfs]:Total duration (50.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:13:54.085 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-15T22:13:54.085 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:13:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:54.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 265.0 s, 1211.2 tps, lat 82.160 ms stddev 88.267 2024-09-15T22:13:55.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:54 smithi099 ceph-mon[34892]: pgmap v153: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 885 op/s 2024-09-15T22:13:55.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:54 smithi129 ceph-mon[35314]: pgmap v153: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 885 op/s 2024-09-15T22:13:55.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:54 smithi005 ceph-mon[30944]: pgmap v153: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 885 op/s 2024-09-15T22:13:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.003+0000 7f19f7e87640 1 -- 172.21.15.5:0/2329306249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19f0101df0 msgr2=0x7f19f01021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.003+0000 7f19f7e87640 1 --2- 172.21.15.5:0/2329306249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19f0101df0 0x7f19f01021d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009900 tx=0x7f19ec02a410 comp rx=0 tx=0).stop 2024-09-15T22:13:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.003+0000 7f19f7e87640 1 -- 172.21.15.5:0/2329306249 shutdown_connections 2024-09-15T22:13:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.003+0000 7f19f7e87640 1 --2- 172.21.15.5:0/2329306249 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19f0106f20 0x7f19f010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.003+0000 7f19f7e87640 1 --2- 172.21.15.5:0/2329306249 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f01027a0 0x7f19f01067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.003+0000 7f19f7e87640 1 --2- 172.21.15.5:0/2329306249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19f0101df0 0x7f19f01021d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:56.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.003+0000 7f19f7e87640 1 -- 172.21.15.5:0/2329306249 >> 172.21.15.5:0/2329306249 conn(0x7f19f00fd8f0 msgr2=0x7f19f00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.003+0000 7f19f7e87640 1 -- 172.21.15.5:0/2329306249 shutdown_connections 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.003+0000 7f19f7e87640 1 -- 172.21.15.5:0/2329306249 wait complete. 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.004+0000 7f19f7e87640 1 Processor -- start 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.004+0000 7f19f7e87640 1 -- start start 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.004+0000 7f19f7e87640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19f0101df0 0x7f19f019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.004+0000 7f19f7e87640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f01027a0 0x7f19f019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.004+0000 7f19f7e87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19f0106f20 0x7f19f019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.004+0000 7f19f7e87640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0111f80 con 0x7f19f0106f20 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.004+0000 7f19f7e87640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0111e00 con 0x7f19f01027a0 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.004+0000 7f19f7e87640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0112100 con 0x7f19f0101df0 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f5422640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f01027a0 0x7f19f019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f5422640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f01027a0 0x7f19f019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:54572/0 (socket says 172.21.15.5:54572) 2024-09-15T22:13:56.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f5c23640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19f0101df0 0x7f19f019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f5422640 1 -- 172.21.15.5:0/2599520523 learned_addr learned my addr 172.21.15.5:0/2599520523 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f6424640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19f0106f20 0x7f19f019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f5422640 1 -- 172.21.15.5:0/2599520523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19f0101df0 msgr2=0x7f19f019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f5422640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19f0101df0 0x7f19f019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f5422640 1 -- 172.21.15.5:0/2599520523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19f0106f20 msgr2=0x7f19f019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f5422640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19f0106f20 0x7f19f019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f5422640 1 -- 172.21.15.5:0/2599520523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f019f650 con 0x7f19f01027a0 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f5422640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f01027a0 0x7f19f019abe0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e9f0 tx=0x7f19e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19e6ffd640 1 -- 172.21.15.5:0/2599520523 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e000cd80 con 0x7f19f01027a0 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f7e87640 1 -- 172.21.15.5:0/2599520523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f019f940 con 0x7f19f01027a0 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19e6ffd640 1 -- 172.21.15.5:0/2599520523 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e0010910 con 0x7f19f01027a0 2024-09-15T22:13:56.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19f7e87640 1 -- 172.21.15.5:0/2599520523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f01a7220 con 0x7f19f01027a0 2024-09-15T22:13:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.005+0000 7f19e6ffd640 1 -- 172.21.15.5:0/2599520523 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e00188f0 con 0x7f19f01027a0 2024-09-15T22:13:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.007+0000 7f19f7e87640 1 -- 172.21.15.5:0/2599520523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f01021d0 con 0x7f19f01027a0 2024-09-15T22:13:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.008+0000 7f19e6ffd640 1 -- 172.21.15.5:0/2599520523 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f19e0010ac0 con 0x7f19f01027a0 2024-09-15T22:13:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.009+0000 7f19e6ffd640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19dc0a33b0 0x7f19dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.009+0000 7f19f5c23640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19dc0a33b0 0x7f19dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.009+0000 7f19e6ffd640 1 -- 172.21.15.5:0/2599520523 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f19e00d0450 con 0x7f19f01027a0 2024-09-15T22:13:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.010+0000 7f19f5c23640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19dc0a33b0 0x7f19dc0a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009230 tx=0x7f19ec0058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.014+0000 7f19e6ffd640 1 -- 172.21.15.5:0/2599520523 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19e009c440 con 0x7f19f01027a0 2024-09-15T22:13:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.384+0000 7f19f7e87640 1 -- 172.21.15.5:0/2599520523 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19f01a73f0 con 0x7f19f01027a0 2024-09-15T22:13:56.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.385+0000 7f19e6ffd640 1 -- 172.21.15.5:0/2599520523 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f19e00a0740 con 0x7f19f01027a0 2024-09-15T22:13:56.387 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:13:56.387 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:13:56.387 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:13:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 -- 172.21.15.5:0/2599520523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19dc0a33b0 msgr2=0x7f19dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19dc0a33b0 0x7f19dc0a5870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009230 tx=0x7f19ec0058d0 comp rx=0 tx=0).stop 2024-09-15T22:13:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 -- 172.21.15.5:0/2599520523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f01027a0 msgr2=0x7f19f019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f01027a0 0x7f19f019abe0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f19e000e9f0 tx=0x7f19e000eeb0 comp rx=0 tx=0).stop 2024-09-15T22:13:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 -- 172.21.15.5:0/2599520523 shutdown_connections 2024-09-15T22:13:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19dc0a33b0 0x7f19dc0a5870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19f0106f20 0x7f19f019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:56.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19f01027a0 0x7f19f019abe0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 --2- 172.21.15.5:0/2599520523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19f0101df0 0x7f19f019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 -- 172.21.15.5:0/2599520523 >> 172.21.15.5:0/2599520523 conn(0x7f19f00fd8f0 msgr2=0x7f19f00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 -- 172.21.15.5:0/2599520523 shutdown_connections 2024-09-15T22:13:56.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:56.388+0000 7f19f7e87640 1 -- 172.21.15.5:0/2599520523 wait complete. 2024-09-15T22:13:57.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:56 smithi099 ceph-mon[34892]: pgmap v154: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-09-15T22:13:57.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2599520523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:57.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:56 smithi129 ceph-mon[35314]: pgmap v154: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-09-15T22:13:57.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2599520523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:57.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:56 smithi005 ceph-mon[30944]: pgmap v154: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 16 MiB/s wr, 1.12k op/s 2024-09-15T22:13:57.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2599520523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:13:57.159 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-15T22:13:57.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-15T22:13:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:13:58.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.385+0000 7f0911913640 1 -- 172.21.15.5:0/3173221914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f090c106f00 msgr2=0x7f090c10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:58.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.385+0000 7f0911913640 1 --2- 172.21.15.5:0/3173221914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f090c106f00 0x7f090c10ea80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f090401acb0 tx=0x7f090403b920 comp rx=0 tx=0).stop 2024-09-15T22:13:58.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.388+0000 7f0911913640 1 -- 172.21.15.5:0/3173221914 shutdown_connections 2024-09-15T22:13:58.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.388+0000 7f0911913640 1 --2- 172.21.15.5:0/3173221914 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f090c106f00 0x7f090c10ea80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.388+0000 7f0911913640 1 --2- 172.21.15.5:0/3173221914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f090c102780 0x7f090c1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.388+0000 7f0911913640 1 --2- 172.21.15.5:0/3173221914 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c101dd0 0x7f090c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.388+0000 7f0911913640 1 -- 172.21.15.5:0/3173221914 >> 172.21.15.5:0/3173221914 conn(0x7f090c0fd8b0 msgr2=0x7f090c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:58.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.388+0000 7f0911913640 1 -- 172.21.15.5:0/3173221914 shutdown_connections 2024-09-15T22:13:58.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.388+0000 7f0911913640 1 -- 172.21.15.5:0/3173221914 wait complete. 2024-09-15T22:13:58.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.389+0000 7f0911913640 1 Processor -- start 2024-09-15T22:13:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.393+0000 7f0911913640 1 -- start start 2024-09-15T22:13:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.393+0000 7f0911913640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c101dd0 0x7f090c10e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.393+0000 7f0911913640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f090c102780 0x7f090c108a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.393+0000 7f0911913640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f090c106f00 0x7f090c108f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.393+0000 7f0911913640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f090c06d750 con 0x7f090c106f00 2024-09-15T22:13:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.393+0000 7f0911913640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f090c06d5d0 con 0x7f090c101dd0 2024-09-15T22:13:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.393+0000 7f0911913640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f090c06d8d0 con 0x7f090c102780 2024-09-15T22:13:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.394+0000 7f090affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c101dd0 0x7f090c10e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.394+0000 7f090affd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c101dd0 0x7f090c10e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:54588/0 (socket says 172.21.15.5:54588) 2024-09-15T22:13:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.394+0000 7f090affd640 1 -- 172.21.15.5:0/2985496593 learned_addr learned my addr 172.21.15.5:0/2985496593 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.394+0000 7f090a7fc640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f090c102780 0x7f090c108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.394+0000 7f090affd640 1 -- 172.21.15.5:0/2985496593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f090c102780 msgr2=0x7f090c108a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.394+0000 7f090affd640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f090c102780 0x7f090c108a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.394+0000 7f090affd640 1 -- 172.21.15.5:0/2985496593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f090c106f00 msgr2=0x7f090c108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.394+0000 7f090affd640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f090c106f00 0x7f090c108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.394+0000 7f090affd640 1 -- 172.21.15.5:0/2985496593 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090c1096a0 con 0x7f090c101dd0 2024-09-15T22:13:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.395+0000 7f090affd640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c101dd0 0x7f090c10e6d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00b450 tx=0x7f08fc00b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.395+0000 7f0910911640 1 -- 172.21.15.5:0/2985496593 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc015020 con 0x7f090c101dd0 2024-09-15T22:13:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.395+0000 7f0911913640 1 -- 172.21.15.5:0/2985496593 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090c115600 con 0x7f090c101dd0 2024-09-15T22:13:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.395+0000 7f0911913640 1 -- 172.21.15.5:0/2985496593 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f090c115b90 con 0x7f090c101dd0 2024-09-15T22:13:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.399+0000 7f0913cec740 1 Processor -- start 2024-09-15T22:13:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.399+0000 7f0913cec740 1 -- start start 2024-09-15T22:13:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.399+0000 7f0913cec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6e180 0x55c7bff6c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f0913cec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6ce90 0x55c7bff6d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f0913cec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff777f0 0x55c7bff79be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f0913cec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c7bfed1160 con 0x55c7bff6e180 2024-09-15T22:13:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f0913cec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c7bfe78720 con 0x55c7bff6ce90 2024-09-15T22:13:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f0913cec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c7bfecd120 con 0x55c7bff777f0 2024-09-15T22:13:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f090b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6e180 0x55c7bff6c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f090b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6e180 0x55c7bff6c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52994/0 (socket says 172.21.15.5:52994) 2024-09-15T22:13:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f090b7fe640 1 -- 172.21.15.5:0/3733527534 learned_addr learned my addr 172.21.15.5:0/3733527534 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f090b7fe640 1 -- 172.21.15.5:0/3733527534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff777f0 msgr2=0x55c7bff79be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:13:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f090b7fe640 1 --2- 172.21.15.5:0/3733527534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff777f0 0x55c7bff79be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f090b7fe640 1 -- 172.21.15.5:0/3733527534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6ce90 msgr2=0x55c7bff6d2d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:13:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f090b7fe640 1 --2- 172.21.15.5:0/3733527534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6ce90 0x55c7bff6d2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.400+0000 7f090b7fe640 1 -- 172.21.15.5:0/3733527534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7bfc42ba0 con 0x55c7bff6e180 2024-09-15T22:13:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.401+0000 7f090b7fe640 1 --2- 172.21.15.5:0/3733527534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6e180 0x55c7bff6c950 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0904040670 tx=0x7f09040406a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f085b9ad74af3d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.401+0000 7f0910911640 1 -- 172.21.15.5:0/2985496593 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08fc011d10 con 0x7f090c101dd0 2024-09-15T22:13:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.401+0000 7f0911913640 1 -- 172.21.15.5:0/3733527534 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904045070 con 0x55c7bff6e180 2024-09-15T22:13:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.402+0000 7f0911913640 1 -- 172.21.15.5:0/3733527534 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0904040d50 con 0x55c7bff6e180 2024-09-15T22:13:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.402+0000 7f0911913640 1 -- 172.21.15.5:0/3733527534 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090406d440 con 0x55c7bff6e180 2024-09-15T22:13:58.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.405+0000 7f0913cec740 1 -- 172.21.15.5:0/3733527534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6e180 msgr2=0x55c7bff6c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:58.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.405+0000 7f0913cec740 1 --2- 172.21.15.5:0/3733527534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6e180 0x55c7bff6c950 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0904040670 tx=0x7f09040406a0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.405+0000 7f0910911640 1 -- 172.21.15.5:0/2985496593 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00f2e0 con 0x7f090c101dd0 2024-09-15T22:13:58.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.405+0000 7f0910911640 1 -- 172.21.15.5:0/2985496593 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f08fc018020 con 0x7f090c101dd0 2024-09-15T22:13:58.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.406+0000 7f0913cec740 1 -- 172.21.15.5:0/3733527534 shutdown_connections 2024-09-15T22:13:58.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.406+0000 7f0913cec740 1 --2- 172.21.15.5:0/3733527534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff777f0 0x55c7bff79be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.406+0000 7f0913cec740 1 --2- 172.21.15.5:0/3733527534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6ce90 0x55c7bff6d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.406+0000 7f0913cec740 1 --2- 172.21.15.5:0/3733527534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6e180 0x55c7bff6c950 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.406+0000 7f0913cec740 1 -- 172.21.15.5:0/3733527534 >> 172.21.15.5:0/3733527534 conn(0x55c7bff690e0 msgr2=0x55c7bff694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:58.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.407+0000 7f0910911640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f08f40a32f0 0x7f08f40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.407+0000 7f090a7fc640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f08f40a32f0 0x7f08f40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.407+0000 7f0910911640 1 -- 172.21.15.5:0/2985496593 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f08fc0d08b0 con 0x7f090c101dd0 2024-09-15T22:13:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.407+0000 7f0913cec740 1 -- 172.21.15.5:0/3733527534 shutdown_connections 2024-09-15T22:13:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.407+0000 7f0913cec740 1 -- 172.21.15.5:0/3733527534 wait complete. 2024-09-15T22:13:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.408+0000 7f090a7fc640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f08f40a32f0 0x7f08f40a57b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55c7bff6a460 tx=0x7f0900009110 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.408+0000 7f0913cec740 1 -- start start 2024-09-15T22:13:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.408+0000 7f0913cec740 1 Processor -- start 2024-09-15T22:13:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.409+0000 7f0913cec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6ce90 0x55c7bff74720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.409+0000 7f0913cec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff6e180 0x55c7bff74c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.409+0000 7f0913cec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff777f0 0x55c7bff751a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.409+0000 7f0913cec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c7bff7a600 con 0x55c7bff6ce90 2024-09-15T22:13:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.409+0000 7f0913cec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c7bfe59900 con 0x55c7bff777f0 2024-09-15T22:13:58.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.409+0000 7f0913cec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c7bff7a780 con 0x55c7bff6e180 2024-09-15T22:13:58.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.409+0000 7f0913cec740 10 client.0 _ll_register_callbacks cb 0x55c7bfe57780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:58.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.410+0000 7f090b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff777f0 0x55c7bff751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.410+0000 7f090affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6ce90 0x55c7bff74720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.410+0000 7f090affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6ce90 0x55c7bff74720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52996/0 (socket says 172.21.15.5:52996) 2024-09-15T22:13:58.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.410+0000 7f090affd640 1 -- 172.21.15.5:0/4263360434 learned_addr learned my addr 172.21.15.5:0/4263360434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:58.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.411+0000 7f090a7fc640 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff6e180 0x55c7bff74c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.411+0000 7f090a7fc640 1 -- 172.21.15.5:0/4263360434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff777f0 msgr2=0x55c7bff751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.411+0000 7f090a7fc640 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff777f0 0x55c7bff751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.411+0000 7f090a7fc640 1 -- 172.21.15.5:0/4263360434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6ce90 msgr2=0x55c7bff74720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.411+0000 7f090a7fc640 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6ce90 0x55c7bff74720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.411+0000 7f090a7fc640 1 -- 172.21.15.5:0/4263360434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7bfe99c50 con 0x55c7bff6e180 2024-09-15T22:13:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.411+0000 7f090a7fc640 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff6e180 0x55c7bff74c60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f090c076550 tx=0x7f09000047d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f54355f05bc631 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.411+0000 7f08eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:13:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.412+0000 7f08eaffd640 1 -- 172.21.15.5:0/4263360434 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900023020 con 0x55c7bff6e180 2024-09-15T22:13:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.412+0000 7f0913cec740 1 -- 172.21.15.5:0/4263360434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0904040d50 con 0x55c7bff6e180 2024-09-15T22:13:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.412+0000 7f08eaffd640 1 -- 172.21.15.5:0/4263360434 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f0900013450 con 0x55c7bff6e180 2024-09-15T22:13:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.412+0000 7f08eaffd640 1 -- 172.21.15.5:0/4263360434 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f090002e050 con 0x55c7bff6e180 2024-09-15T22:13:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.412+0000 7f0913cec740 10 client.25915 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.412+0000 7f0913cec740 1 -- 172.21.15.5:0/4263360434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7bff86400 con 0x55c7bff6e180 2024-09-15T22:13:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.413+0000 7f08eaffd640 1 -- 172.21.15.5:0/4263360434 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0904040d50 con 0x55c7bff6e180 2024-09-15T22:13:58.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.413+0000 7f08eaffd640 1 -- 172.21.15.5:0/4263360434 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f090001e030 con 0x55c7bff6e180 2024-09-15T22:13:58.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.413+0000 7f0913cec740 10 client.25915 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:58.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.413+0000 7f0913cec740 10 client.25915 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:13:58.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.413+0000 7f0913cec740 20 client.25915 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:58.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.413+0000 7f0913cec740 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c7bff87d60 0x55c7bff8a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.413+0000 7f0913cec740 4 client.25915 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:13:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.413+0000 7f0913cec740 1 -- 172.21.15.5:0/4263360434 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7bfede890 con 0x55c7bff87d60 2024-09-15T22:13:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.415+0000 7f090affd640 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c7bff87d60 0x55c7bff8a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.415+0000 7f090affd640 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c7bff87d60 0x55c7bff8a180 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63f8e2814214de0 server_cookie=b2526f0dbaab0827 in_seq=0 out_seq=0 2024-09-15T22:13:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.415+0000 7f08eaffd640 10 client.25915 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:58.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.416+0000 7f08eaffd640 1 -- 172.21.15.5:0/4263360434 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7bfede890 con 0x55c7bff87d60 2024-09-15T22:13:58.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.416+0000 7f08eaffd640 10 client.25915 handle_command_reply: tid=0 2024-09-15T22:13:58.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.416+0000 7f0913cec740 1 client.25915 shutdown 2024-09-15T22:13:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f0913cec740 1 -- 172.21.15.5:0/4263360434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff6e180 msgr2=0x55c7bff74c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f0913cec740 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff6e180 0x55c7bff74c60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f090c076550 tx=0x7f09000047d0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f0913cec740 1 -- 172.21.15.5:0/4263360434 shutdown_connections 2024-09-15T22:13:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f0913cec740 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c7bff87d60 0x55c7bff8a180 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f0913cec740 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff777f0 0x55c7bff751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f0913cec740 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff6e180 0x55c7bff74c60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f0913cec740 1 --2- 172.21.15.5:0/4263360434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6ce90 0x55c7bff74720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f0913cec740 1 -- 172.21.15.5:0/4263360434 >> 172.21.15.5:0/4263360434 conn(0x55c7bff690e0 msgr2=0x55c7bff6e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f08eaffd640 0 client.25915 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f0913cec740 1 -- 172.21.15.5:0/4263360434 shutdown_connections 2024-09-15T22:13:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f0913cec740 1 -- 172.21.15.5:0/4263360434 wait complete. 2024-09-15T22:13:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.417+0000 7f0913cec740 20 client.25915 trim_cache size 0 max 16384 2024-09-15T22:13:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.440+0000 7f0913cec740 1 Processor -- start 2024-09-15T22:13:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.440+0000 7f0913cec740 1 -- start start 2024-09-15T22:13:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.440+0000 7f0913cec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff690e0 0x55c7bff6fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.440+0000 7f0913cec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff6ce90 0x55c7bff68390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.440+0000 7f0913cec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 0x55c7bff8fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.440+0000 7f0913cec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f090c122120 con 0x55c7bff690e0 2024-09-15T22:13:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.440+0000 7f0913cec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f090c11f9f0 con 0x55c7bff6e180 2024-09-15T22:13:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.440+0000 7f0913cec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f090c11d840 con 0x55c7bff6ce90 2024-09-15T22:13:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff690e0 0x55c7bff6fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff690e0 0x55c7bff6fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53006/0 (socket says 172.21.15.5:53006) 2024-09-15T22:13:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090affd640 1 -- 172.21.15.5:0/689952387 learned_addr learned my addr 172.21.15.5:0/689952387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090b7fe640 1 --2- 172.21.15.5:0/689952387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 0x55c7bff8fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090a7fc640 1 --2- 172.21.15.5:0/689952387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff6ce90 0x55c7bff68390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090b7fe640 1 -- 172.21.15.5:0/689952387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff6ce90 msgr2=0x55c7bff68390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090b7fe640 1 --2- 172.21.15.5:0/689952387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff6ce90 0x55c7bff68390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090b7fe640 1 -- 172.21.15.5:0/689952387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff690e0 msgr2=0x55c7bff6fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090b7fe640 1 --2- 172.21.15.5:0/689952387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff690e0 0x55c7bff6fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090b7fe640 1 -- 172.21.15.5:0/689952387 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7bff736d0 con 0x55c7bff6e180 2024-09-15T22:13:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090affd640 1 --2- 172.21.15.5:0/689952387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff690e0 0x55c7bff6fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:13:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.441+0000 7f090b7fe640 1 --2- 172.21.15.5:0/689952387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 0x55c7bff8fa50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f090401ac00 tx=0x7f090401a940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c14c5b01f2a82eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.442+0000 7f08f17fa640 1 -- 172.21.15.5:0/689952387 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904045040 con 0x55c7bff6e180 2024-09-15T22:13:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.442+0000 7f08f17fa640 1 -- 172.21.15.5:0/689952387 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0904003970 con 0x55c7bff6e180 2024-09-15T22:13:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.442+0000 7f08f17fa640 1 -- 172.21.15.5:0/689952387 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904040870 con 0x55c7bff6e180 2024-09-15T22:13:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.442+0000 7f0913cec740 1 -- 172.21.15.5:0/689952387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 msgr2=0x55c7bff8fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.442+0000 7f0913cec740 1 --2- 172.21.15.5:0/689952387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 0x55c7bff8fa50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f090401ac00 tx=0x7f090401a940 comp rx=0 tx=0).stop 2024-09-15T22:13:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.443+0000 7f0913cec740 1 -- 172.21.15.5:0/689952387 shutdown_connections 2024-09-15T22:13:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.443+0000 7f0913cec740 1 --2- 172.21.15.5:0/689952387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 0x55c7bff8fa50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.443+0000 7f0913cec740 1 --2- 172.21.15.5:0/689952387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff6ce90 0x55c7bff68390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.443+0000 7f0913cec740 1 --2- 172.21.15.5:0/689952387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff690e0 0x55c7bff6fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.443+0000 7f0913cec740 1 -- 172.21.15.5:0/689952387 >> 172.21.15.5:0/689952387 conn(0x55c7bff70110 msgr2=0x55c7bff67c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.444+0000 7f0913cec740 1 -- 172.21.15.5:0/689952387 shutdown_connections 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.444+0000 7f0913cec740 1 -- 172.21.15.5:0/689952387 wait complete. 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.444+0000 7f0913cec740 1 -- start start 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.445+0000 7f0913cec740 1 Processor -- start 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.445+0000 7f0913cec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff690e0 0x55c7bff80270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.445+0000 7f0913cec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6ce90 0x55c7bff7b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.446+0000 7f090affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff690e0 0x55c7bff80270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.446+0000 7f090affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff690e0 0x55c7bff80270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:54520/0 (socket says 172.21.15.5:54520) 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.446+0000 7f090affd640 1 -- 172.21.15.5:0/4080950464 learned_addr learned my addr 172.21.15.5:0/4080950464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.446+0000 7f0913cec740 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 0x55c7bff7bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.446+0000 7f090a7fc640 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6ce90 0x55c7bff7b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.446+0000 7f0913cec740 1 -- 172.21.15.5:0/4080950464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f090c11deb0 con 0x55c7bff6ce90 2024-09-15T22:13:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.446+0000 7f090b7fe640 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 0x55c7bff7bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.446+0000 7f0913cec740 1 -- 172.21.15.5:0/4080950464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f090c11f320 con 0x55c7bff6e180 2024-09-15T22:13:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.446+0000 7f0913cec740 1 -- 172.21.15.5:0/4080950464 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f090c11e580 con 0x55c7bff690e0 2024-09-15T22:13:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.446+0000 7f0913cec740 10 client.0 _ll_register_callbacks cb 0x7f09000093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:13:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.447+0000 7f090b7fe640 1 -- 172.21.15.5:0/4080950464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff690e0 msgr2=0x55c7bff80270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.447+0000 7f090b7fe640 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff690e0 0x55c7bff80270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.447+0000 7f090b7fe640 1 -- 172.21.15.5:0/4080950464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6ce90 msgr2=0x55c7bff7b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:13:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.447+0000 7f090b7fe640 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6ce90 0x55c7bff7b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.447+0000 7f090b7fe640 1 -- 172.21.15.5:0/4080950464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7bff7c350 con 0x55c7bff6e180 2024-09-15T22:13:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.447+0000 7f090affd640 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff690e0 0x55c7bff80270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:13:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.447+0000 7f090b7fe640 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 0x55c7bff7bb20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f090401ac00 tx=0x7f090401aa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29ff71595ad83ec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:13:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.447+0000 7f08eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:13:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.447+0000 7f08eaffd640 1 -- 172.21.15.5:0/4080950464 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904045030 con 0x55c7bff6e180 2024-09-15T22:13:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f08eaffd640 1 -- 172.21.15.5:0/4080950464 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f0904019040 con 0x55c7bff6e180 2024-09-15T22:13:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f0913cec740 1 -- 172.21.15.5:0/4080950464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0904003970 con 0x55c7bff6e180 2024-09-15T22:13:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f08eaffd640 1 -- 172.21.15.5:0/4080950464 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0904003d30 con 0x55c7bff6e180 2024-09-15T22:13:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f0913cec740 10 client.25958 fetch_fsmap learned FSMap version 25 2024-09-15T22:13:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f0913cec740 1 -- 172.21.15.5:0/4080950464 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7bfff2a60 con 0x55c7bff6e180 2024-09-15T22:13:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f08eaffd640 1 -- 172.21.15.5:0/4080950464 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0904003970 con 0x55c7bff6e180 2024-09-15T22:13:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f08eaffd640 1 -- 172.21.15.5:0/4080950464 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0904004020 con 0x55c7bff6e180 2024-09-15T22:13:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f0913cec740 10 client.25958 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:13:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f0913cec740 10 client.25958 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:13:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f0913cec740 20 client.25958 populate_metadata read hostname 'smithi005' 2024-09-15T22:13:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f0913cec740 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c7bff777f0 0x55c7bfff4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:13:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f0913cec740 4 client.25958 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-15T22:13:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.448+0000 7f0913cec740 1 -- 172.21.15.5:0/4080950464 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x55c7bfed8be0 con 0x55c7bff777f0 2024-09-15T22:13:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.449+0000 7f090affd640 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c7bff777f0 0x55c7bfff4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:13:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.449+0000 7f08eaffd640 10 client.25958 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.449+0000 7f090affd640 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c7bff777f0 0x55c7bfff4d40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12329909a6f06d90 server_cookie=b71bf432bc0d78c2 in_seq=0 out_seq=0 2024-09-15T22:13:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f08eaffd640 1 -- 172.21.15.5:0/4080950464 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c7bfed8be0 con 0x55c7bff777f0 2024-09-15T22:13:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f08eaffd640 10 client.25958 handle_command_reply: tid=0 2024-09-15T22:13:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 1 client.25958 shutdown 2024-09-15T22:13:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 1 -- 172.21.15.5:0/4080950464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 msgr2=0x55c7bff7bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 0x55c7bff7bb20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f090401ac00 tx=0x7f090401aa30 comp rx=0 tx=0).stop 2024-09-15T22:13:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 1 -- 172.21.15.5:0/4080950464 shutdown_connections 2024-09-15T22:13:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c7bff777f0 0x55c7bfff4d40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c7bff6e180 0x55c7bff7bb20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7bff6ce90 0x55c7bff7b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f08eaffd640 0 client.25958 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:13:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 1 --2- 172.21.15.5:0/4080950464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c7bff690e0 0x55c7bff80270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 1 -- 172.21.15.5:0/4080950464 >> 172.21.15.5:0/4080950464 conn(0x55c7bff70110 msgr2=0x55c7bff746e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 1 -- 172.21.15.5:0/4080950464 shutdown_connections 2024-09-15T22:13:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 1 -- 172.21.15.5:0/4080950464 wait complete. 2024-09-15T22:13:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.763+0000 7f0913cec740 20 client.25958 trim_cache size 0 max 16384 2024-09-15T22:13:58.766 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:13:58.766 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 18, 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "27b49ad6": { 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 18, 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 145.3, 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:13:58.767 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:13:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:13:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:13:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:13:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:58.768 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:13:58.768 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:13:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 -- 172.21.15.5:0/2985496593 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f08f40a32f0 msgr2=0x7f08f40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f08f40a32f0 0x7f08f40a57b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55c7bff6a460 tx=0x7f0900009110 comp rx=0 tx=0).stop 2024-09-15T22:13:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 -- 172.21.15.5:0/2985496593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c101dd0 msgr2=0x7f090c10e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:13:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c101dd0 0x7f090c10e6d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00b450 tx=0x7f08fc00b850 comp rx=0 tx=0).stop 2024-09-15T22:13:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 -- 172.21.15.5:0/2985496593 shutdown_connections 2024-09-15T22:13:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f08f40a32f0 0x7f08f40a57b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f090c106f00 0x7f090c108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f090c102780 0x7f090c108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 --2- 172.21.15.5:0/2985496593 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f090c101dd0 0x7f090c10e6d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:13:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 -- 172.21.15.5:0/2985496593 >> 172.21.15.5:0/2985496593 conn(0x7f090c0fd8b0 msgr2=0x7f090c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:13:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 -- 172.21.15.5:0/2985496593 shutdown_connections 2024-09-15T22:13:58.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:13:58.764+0000 7f08f27fc640 1 -- 172.21.15.5:0/2985496593 wait complete. 2024-09-15T22:13:59.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:13:58 smithi005 ceph-mon[30944]: pgmap v155: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 646 op/s 2024-09-15T22:13:59.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:13:58 smithi099 ceph-mon[34892]: pgmap v155: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 646 op/s 2024-09-15T22:13:59.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:13:58 smithi129 ceph-mon[35314]: pgmap v155: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 8.5 MiB/s wr, 646 op/s 2024-09-15T22:13:59.480 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '27b49ad6', quiesce took 5.4 seconds. Will release after: 19.950000000000003 2024-09-15T22:13:59.662 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 270.0 s, 764.4 tps, lat 99.807 ms stddev 110.278 2024-09-15T22:14:01.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:00 smithi099 ceph-mon[34892]: pgmap v156: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 674 op/s 2024-09-15T22:14:01.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:00 smithi129 ceph-mon[35314]: pgmap v156: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 674 op/s 2024-09-15T22:14:01.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:00 smithi005 ceph-mon[30944]: pgmap v156: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 8.7 MiB/s wr, 674 op/s 2024-09-15T22:14:02.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:14:02.845 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:02.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:02 smithi005 ceph-mon[30944]: pgmap v157: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.5 MiB/s wr, 783 op/s 2024-09-15T22:14:03.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:02 smithi099 ceph-mon[34892]: pgmap v157: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.5 MiB/s wr, 783 op/s 2024-09-15T22:14:03.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:02 smithi129 ceph-mon[35314]: pgmap v157: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 9.5 MiB/s wr, 783 op/s 2024-09-15T22:14:03.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.321+0000 7f52cb4be640 1 -- 172.21.15.5:0/1178495642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c4102ef0 msgr2=0x7f52c410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:03.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.321+0000 7f52cb4be640 1 --2- 172.21.15.5:0/1178495642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c4102ef0 0x7f52c410da40 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009ca0 tx=0x7f52bc02a950 comp rx=0 tx=0).stop 2024-09-15T22:14:03.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.321+0000 7f52cb4be640 1 -- 172.21.15.5:0/1178495642 shutdown_connections 2024-09-15T22:14:03.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.321+0000 7f52cb4be640 1 --2- 172.21.15.5:0/1178495642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c4102ef0 0x7f52c410da40 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.321+0000 7f52cb4be640 1 --2- 172.21.15.5:0/1178495642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c4102570 0x7f52c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.321+0000 7f52cb4be640 1 --2- 172.21.15.5:0/1178495642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52c4100e20 0x7f52c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.321+0000 7f52cb4be640 1 -- 172.21.15.5:0/1178495642 >> 172.21.15.5:0/1178495642 conn(0x7f52c40fc7d0 msgr2=0x7f52c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:03.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.321+0000 7f52cb4be640 1 -- 172.21.15.5:0/1178495642 shutdown_connections 2024-09-15T22:14:03.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.321+0000 7f52cb4be640 1 -- 172.21.15.5:0/1178495642 wait complete. 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.322+0000 7f52cb4be640 1 Processor -- start 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.322+0000 7f52cb4be640 1 -- start start 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.322+0000 7f52cb4be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c4100e20 0x7f52c41971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.322+0000 7f52cb4be640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c4102570 0x7f52c41976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.322+0000 7f52cb4be640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52c4102ef0 0x7f52c4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.322+0000 7f52cb4be640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52c4110fa0 con 0x7f52c4100e20 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.322+0000 7f52cb4be640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f52c4110e20 con 0x7f52c4102570 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.322+0000 7f52cb4be640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52c4111120 con 0x7f52c4102ef0 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.322+0000 7f52c925a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c4100e20 0x7f52c41971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52c925a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c4100e20 0x7f52c41971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53032/0 (socket says 172.21.15.5:53032) 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52c925a640 1 -- 172.21.15.5:0/1600349064 learned_addr learned my addr 172.21.15.5:0/1600349064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:03.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52c8a59640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c4102570 0x7f52c41976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52c9a5b640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52c4102ef0 0x7f52c4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52c925a640 1 -- 172.21.15.5:0/1600349064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52c4102ef0 msgr2=0x7f52c4197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52c925a640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52c4102ef0 0x7f52c4197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52c925a640 1 -- 172.21.15.5:0/1600349064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c4102570 msgr2=0x7f52c41976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52c925a640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c4102570 0x7f52c41976e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52c925a640 1 -- 172.21.15.5:0/1600349064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c41a6a10 con 0x7f52c4100e20 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52c9a5b640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52c4102ef0 0x7f52c4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52c925a640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c4100e20 0x7f52c41971a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b4a0 tx=0x7f52c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52ba7fc640 1 -- 172.21.15.5:0/1600349064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0014070 con 0x7f52c4100e20 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52ba7fc640 1 -- 172.21.15.5:0/1600349064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52c0010af0 con 0x7f52c4100e20 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52cb4be640 1 -- 172.21.15.5:0/1600349064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c41a6d00 con 0x7f52c4100e20 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52ba7fc640 1 -- 172.21.15.5:0/1600349064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0017d10 con 0x7f52c4100e20 2024-09-15T22:14:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.323+0000 7f52cb4be640 1 -- 172.21.15.5:0/1600349064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c41a7190 con 0x7f52c4100e20 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.326+0000 7f52ba7fc640 1 -- 172.21.15.5:0/1600349064 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f52c0017070 con 0x7f52c4100e20 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.326+0000 7f52cd897740 1 Processor -- start 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.326+0000 7f52cd897740 1 -- start start 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.326+0000 7f52cd897740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.326+0000 7f52cd897740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7b4dc0 0x55c88e7bf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.326+0000 7f52cd897740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.326+0000 7f52cd897740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c88e71a2e0 con 0x55c88e7bf680 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.326+0000 7f52cd897740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c88e6c1ce0 con 0x55c88e7b4dc0 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.326+0000 7f52cd897740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c88e7162a0 con 0x55c88e7b44a0 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c8a59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c8a59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7c1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53034/0 (socket says 172.21.15.5:53034) 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c8a59640 1 -- 172.21.15.5:0/3143680050 learned_addr learned my addr 172.21.15.5:0/3143680050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:03.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c925a640 1 --2- 172.21.15.5:0/3143680050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7b4dc0 0x55c88e7bf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c9a5b640 1 --2- 172.21.15.5:0/3143680050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c8a59640 1 -- 172.21.15.5:0/3143680050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 msgr2=0x55c88e7b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c8a59640 1 --2- 172.21.15.5:0/3143680050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c8a59640 1 -- 172.21.15.5:0/3143680050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7b4dc0 msgr2=0x55c88e7bf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c8a59640 1 --2- 172.21.15.5:0/3143680050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7b4dc0 0x55c88e7bf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52ba7fc640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f52b00a33b0 0x7f52b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c8a59640 1 -- 172.21.15.5:0/3143680050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c88e6a6440 con 0x55c88e7bf680 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c9a5b640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f52b00a33b0 0x7f52b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52ba7fc640 1 -- 172.21.15.5:0/1600349064 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f52c00cf380 con 0x7f52c4100e20 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.327+0000 7f52c8a59640 1 --2- 172.21.15.5:0/3143680050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7c1a70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f52b400abf0 tx=0x7f52b400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=644f3c918ae9b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.328+0000 7f52af7fe640 1 -- 172.21.15.5:0/3143680050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b401d070 con 0x55c88e7bf680 2024-09-15T22:14:03.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.328+0000 7f52af7fe640 1 -- 172.21.15.5:0/3143680050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b4018b20 con 0x55c88e7bf680 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.328+0000 7f52af7fe640 1 -- 172.21.15.5:0/3143680050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4010aa0 con 0x55c88e7bf680 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.328+0000 7f52cd897740 1 -- 172.21.15.5:0/3143680050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 msgr2=0x55c88e7c1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.328+0000 7f52cd897740 1 --2- 172.21.15.5:0/3143680050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7c1a70 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f52b400abf0 tx=0x7f52b400f790 comp rx=0 tx=0).stop 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.328+0000 7f52c9a5b640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f52b00a33b0 0x7f52b00a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009430 tx=0x7f52bc009360 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.328+0000 7f52cd897740 1 -- 172.21.15.5:0/3143680050 shutdown_connections 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.328+0000 7f52cd897740 1 --2- 172.21.15.5:0/3143680050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7c1a70 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.328+0000 7f52cd897740 1 --2- 172.21.15.5:0/3143680050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7b4dc0 0x55c88e7bf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.328+0000 7f52cd897740 1 --2- 172.21.15.5:0/3143680050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.328+0000 7f52cd897740 1 -- 172.21.15.5:0/3143680050 >> 172.21.15.5:0/3143680050 conn(0x55c88e7b0a30 msgr2=0x55c88e7b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.329+0000 7f52cd897740 1 -- 172.21.15.5:0/3143680050 shutdown_connections 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.329+0000 7f52cd897740 1 -- 172.21.15.5:0/3143680050 wait complete. 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.329+0000 7f52cd897740 1 -- start start 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.329+0000 7f52cd897740 1 Processor -- start 2024-09-15T22:14:03.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.329+0000 7f52cd897740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7beef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.329+0000 7f52cd897740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7b4dc0 0x55c88e7bc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.329+0000 7f52cd897740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7bc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.329+0000 7f52cd897740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c88e7c2250 con 0x55c88e7bf680 2024-09-15T22:14:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.329+0000 7f52cd897740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c88e6d5050 con 0x55c88e7b4dc0 2024-09-15T22:14:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.329+0000 7f52cd897740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c88e7c23d0 con 0x55c88e7b44a0 2024-09-15T22:14:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.329+0000 7f52cd897740 10 client.0 _ll_register_callbacks cb 0x55c88e715160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52c9a5b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7bc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52c9a5b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7bc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53042/0 (socket says 172.21.15.5:53042) 2024-09-15T22:14:03.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52c9a5b640 1 -- 172.21.15.5:0/47189257 learned_addr learned my addr 172.21.15.5:0/47189257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52c925a640 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7beef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52c8a59640 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7b4dc0 0x55c88e7bc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52c9a5b640 1 -- 172.21.15.5:0/47189257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 msgr2=0x55c88e7beef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52c9a5b640 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7beef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52c9a5b640 1 -- 172.21.15.5:0/47189257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7b4dc0 msgr2=0x55c88e7bc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52c9a5b640 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7b4dc0 0x55c88e7bc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52c9a5b640 1 -- 172.21.15.5:0/47189257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c88e48b3c0 con 0x55c88e7bf680 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52c9a5b640 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7bc6e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f52bc035000 tx=0x7f52bc02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a683d39017608f9e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52acff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52acff9640 1 -- 172.21.15.5:0/47189257 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc034040 con 0x55c88e7bf680 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52acff9640 1 -- 172.21.15.5:0/47189257 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f52bc004250 con 0x55c88e7bf680 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.330+0000 7f52cd897740 1 -- 172.21.15.5:0/47189257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b4018b20 con 0x55c88e7bf680 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52acff9640 1 -- 172.21.15.5:0/47189257 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f52bc004450 con 0x55c88e7bf680 2024-09-15T22:14:03.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52cd897740 10 client.16386 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52cd897740 1 -- 172.21.15.5:0/47189257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c88e7cdb70 con 0x55c88e7bf680 2024-09-15T22:14:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52acff9640 1 -- 172.21.15.5:0/47189257 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b4018b20 con 0x55c88e7bf680 2024-09-15T22:14:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52acff9640 1 -- 172.21.15.5:0/47189257 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f52bc052c50 con 0x55c88e7bf680 2024-09-15T22:14:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52cd897740 10 client.16386 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52cd897740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52cd897740 20 client.16386 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52cd897740 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c88e7cf030 0x55c88e7d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52cd897740 4 client.16386 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52cd897740 1 -- 172.21.15.5:0/47189257 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c88e727850 con 0x55c88e7cf030 2024-09-15T22:14:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.331+0000 7f52c925a640 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c88e7cf030 0x55c88e7d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.332+0000 7f52c925a640 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c88e7cf030 0x55c88e7d14a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a4303eba261a548 server_cookie=bc73aefeabffe135 in_seq=0 out_seq=0 2024-09-15T22:14:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.332+0000 7f52acff9640 10 client.16386 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.332+0000 7f52acff9640 1 -- 172.21.15.5:0/47189257 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c88e727850 con 0x55c88e7cf030 2024-09-15T22:14:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.332+0000 7f52acff9640 10 client.16386 handle_command_reply: tid=0 2024-09-15T22:14:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.332+0000 7f52cd897740 1 client.16386 shutdown 2024-09-15T22:14:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52cd897740 1 -- 172.21.15.5:0/47189257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 msgr2=0x55c88e7bc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52cd897740 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7bc6e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f52bc035000 tx=0x7f52bc02a950 comp rx=0 tx=0).stop 2024-09-15T22:14:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52cd897740 1 -- 172.21.15.5:0/47189257 shutdown_connections 2024-09-15T22:14:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52cd897740 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7bf680 0x55c88e7bc6e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52cd897740 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c88e7cf030 0x55c88e7d14a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52cd897740 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7b4dc0 0x55c88e7bc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52cd897740 1 --2- 172.21.15.5:0/47189257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7beef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52cd897740 1 -- 172.21.15.5:0/47189257 >> 172.21.15.5:0/47189257 conn(0x55c88e7b0a30 msgr2=0x55c88e7b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52acff9640 0 client.16386 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52cd897740 1 -- 172.21.15.5:0/47189257 shutdown_connections 2024-09-15T22:14:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52cd897740 1 -- 172.21.15.5:0/47189257 wait complete. 2024-09-15T22:14:03.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.333+0000 7f52cd897740 20 client.16386 trim_cache size 0 max 16384 2024-09-15T22:14:03.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52cd897740 1 Processor -- start 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52cd897740 1 -- start start 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52cd897740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7be110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52cd897740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7d6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52cd897740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7bf680 0x55c88e7d72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52cd897740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52c41b3b30 con 0x55c88e7b4dc0 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52cd897740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f52c41b1400 con 0x55c88e7bf680 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52cd897740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52c41af250 con 0x55c88e7b44a0 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52c8a59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7d6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52c8a59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7d6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53044/0 (socket says 172.21.15.5:53044) 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52c8a59640 1 -- 172.21.15.5:0/2589859471 learned_addr learned my addr 172.21.15.5:0/2589859471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52c9a5b640 1 --2- 172.21.15.5:0/2589859471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7bf680 0x55c88e7d72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52c925a640 1 --2- 172.21.15.5:0/2589859471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52c8a59640 1 -- 172.21.15.5:0/2589859471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 msgr2=0x55c88e7be110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52c8a59640 1 --2- 172.21.15.5:0/2589859471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7be110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52c8a59640 1 -- 172.21.15.5:0/2589859471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7bf680 msgr2=0x55c88e7d72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.351+0000 7f52c8a59640 1 --2- 172.21.15.5:0/2589859471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7bf680 0x55c88e7d72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.352+0000 7f52c8a59640 1 -- 172.21.15.5:0/2589859471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c88e7b0860 con 0x55c88e7b4dc0 2024-09-15T22:14:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.352+0000 7f52c8a59640 1 --2- 172.21.15.5:0/2589859471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7d6d70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f52b400abc0 tx=0x7f52b40058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea548937feab14b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.352+0000 7f52aeffd640 1 -- 172.21.15.5:0/2589859471 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b401d040 con 0x55c88e7b4dc0 2024-09-15T22:14:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.352+0000 7f52aeffd640 1 -- 172.21.15.5:0/2589859471 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b401e040 con 0x55c88e7b4dc0 2024-09-15T22:14:03.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.352+0000 7f52aeffd640 1 -- 172.21.15.5:0/2589859471 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b40055a0 con 0x55c88e7b4dc0 2024-09-15T22:14:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.353+0000 7f52cd897740 1 -- 172.21.15.5:0/2589859471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 msgr2=0x55c88e7d6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.353+0000 7f52cd897740 1 --2- 172.21.15.5:0/2589859471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7d6d70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f52b400abc0 tx=0x7f52b40058a0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.353+0000 7f52cd897740 1 -- 172.21.15.5:0/2589859471 shutdown_connections 2024-09-15T22:14:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.353+0000 7f52cd897740 1 --2- 172.21.15.5:0/2589859471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7bf680 0x55c88e7d72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.353+0000 7f52cd897740 1 --2- 172.21.15.5:0/2589859471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7d6d70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.353+0000 7f52cd897740 1 --2- 172.21.15.5:0/2589859471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7be110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.353+0000 7f52cd897740 1 -- 172.21.15.5:0/2589859471 >> 172.21.15.5:0/2589859471 conn(0x55c88e7b0a30 msgr2=0x55c88e7bda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.353+0000 7f52cd897740 1 -- 172.21.15.5:0/2589859471 shutdown_connections 2024-09-15T22:14:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.353+0000 7f52cd897740 1 -- 172.21.15.5:0/2589859471 wait complete. 2024-09-15T22:14:03.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.353+0000 7f52cd897740 1 -- start start 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.353+0000 7f52cd897740 1 Processor -- start 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52cd897740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52cd897740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7c5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52cd897740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7bf680 0x55c88e7c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52cd897740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52c41af8c0 con 0x55c88e7b4dc0 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52cd897740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f52c41b0d30 con 0x55c88e7bf680 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52cd897740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52c41aff90 con 0x55c88e7b44a0 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52cd897740 10 client.0 _ll_register_callbacks cb 0x7f52bc009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52c8a59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52c8a59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7c5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53060/0 (socket says 172.21.15.5:53060) 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52c8a59640 1 -- 172.21.15.5:0/2152620490 learned_addr learned my addr 172.21.15.5:0/2152620490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52c925a640 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52c9a5b640 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7bf680 0x55c88e7c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52c8a59640 1 -- 172.21.15.5:0/2152620490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 msgr2=0x55c88e7c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52c8a59640 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52c8a59640 1 -- 172.21.15.5:0/2152620490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7bf680 msgr2=0x55c88e7c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52c8a59640 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7bf680 0x55c88e7c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.354+0000 7f52c8a59640 1 -- 172.21.15.5:0/2152620490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c88e838d50 con 0x55c88e7b4dc0 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52c8a59640 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7c5880 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f52b401b040 tx=0x7f52b4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a9a18cbcb4920d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52acff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52acff9640 1 -- 172.21.15.5:0/2152620490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4031070 con 0x55c88e7b4dc0 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52acff9640 1 -- 172.21.15.5:0/2152620490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f52b40053f0 con 0x55c88e7b4dc0 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52cd897740 1 -- 172.21.15.5:0/2152620490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b401e040 con 0x55c88e7b4dc0 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52acff9640 1 -- 172.21.15.5:0/2152620490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f52b401d040 con 0x55c88e7b4dc0 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52cd897740 10 client.16392 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52cd897740 1 -- 172.21.15.5:0/2152620490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c88e839f60 con 0x55c88e7b4dc0 2024-09-15T22:14:03.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52acff9640 1 -- 172.21.15.5:0/2152620490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52b401e040 con 0x55c88e7b4dc0 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52acff9640 1 -- 172.21.15.5:0/2152620490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f52b40049d0 con 0x55c88e7b4dc0 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52cd897740 10 client.16392 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.355+0000 7f52cd897740 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.356+0000 7f52cd897740 20 client.16392 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.356+0000 7f52cd897740 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c88e83a490 0x55c88e83c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.356+0000 7f52cd897740 4 client.16392 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.356+0000 7f52cd897740 1 -- 172.21.15.5:0/2152620490 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c88e721d60 con 0x55c88e83a490 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.356+0000 7f52c925a640 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c88e83a490 0x55c88e83c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.356+0000 7f52c925a640 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c88e83a490 0x55c88e83c8b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cccd4160bbcca9f5 server_cookie=54d3abf3562eac7f in_seq=0 out_seq=0 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.356+0000 7f52acff9640 10 client.16392 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.356+0000 7f52acff9640 1 -- 172.21.15.5:0/2152620490 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c88e721d60 con 0x55c88e83a490 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.356+0000 7f52acff9640 10 client.16392 handle_command_reply: tid=0 2024-09-15T22:14:03.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.356+0000 7f52cd897740 1 client.16392 shutdown 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cd897740 1 -- 172.21.15.5:0/2152620490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 msgr2=0x55c88e7c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cd897740 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7c5880 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f52b401b040 tx=0x7f52b4005030 comp rx=0 tx=0).stop 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cd897740 1 -- 172.21.15.5:0/2152620490 shutdown_connections 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cd897740 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c88e83a490 0x55c88e83c8b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cd897740 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c88e7bf680 0x55c88e7c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cd897740 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c88e7b4dc0 0x55c88e7c5880 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cd897740 1 --2- 172.21.15.5:0/2152620490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c88e7b44a0 0x55c88e7c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cd897740 1 -- 172.21.15.5:0/2152620490 >> 172.21.15.5:0/2152620490 conn(0x55c88e7b0a30 msgr2=0x55c88e7af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52acff9640 0 client.16392 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cd897740 1 -- 172.21.15.5:0/2152620490 shutdown_connections 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cd897740 1 -- 172.21.15.5:0/2152620490 wait complete. 2024-09-15T22:14:03.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cd897740 20 client.16392 trim_cache size 0 max 16384 2024-09-15T22:14:03.364 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:14:03.364 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:14:03.364 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:14:03.364 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cb4be640 1 -- 172.21.15.5:0/1600349064 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f52b00a33b0 msgr2=0x7f52b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cb4be640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f52b00a33b0 0x7f52b00a5870 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009430 tx=0x7f52bc009360 comp rx=0 tx=0).stop 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cb4be640 1 -- 172.21.15.5:0/1600349064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c4100e20 msgr2=0x7f52c41971a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.357+0000 7f52cb4be640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c4100e20 0x7f52c41971a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b4a0 tx=0x7f52c000b960 comp rx=0 tx=0).stop 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.358+0000 7f52cb4be640 1 -- 172.21.15.5:0/1600349064 shutdown_connections 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.358+0000 7f52cb4be640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f52b00a33b0 0x7f52b00a5870 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.358+0000 7f52cb4be640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f52c4102ef0 0x7f52c4197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.358+0000 7f52cb4be640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52c4102570 0x7f52c41976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.358+0000 7f52cb4be640 1 --2- 172.21.15.5:0/1600349064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52c4100e20 0x7f52c41971a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.358+0000 7f52cb4be640 1 -- 172.21.15.5:0/1600349064 >> 172.21.15.5:0/1600349064 conn(0x7f52c40fc7d0 msgr2=0x7f52c4106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.358+0000 7f52cb4be640 1 -- 172.21.15.5:0/1600349064 shutdown_connections 2024-09-15T22:14:03.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:03.358+0000 7f52cb4be640 1 -- 172.21.15.5:0/1600349064 wait complete. 2024-09-15T22:14:04.262 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:14:04.262 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:14:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 275.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:14:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.037+0000 7f324dae1640 1 -- 172.21.15.5:0/1314063990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3248100e20 msgr2=0x7f3248101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.037+0000 7f324dae1640 1 --2- 172.21.15.5:0/1314063990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3248100e20 0x7f3248101200 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3238009900 tx=0x7f323802a480 comp rx=0 tx=0).stop 2024-09-15T22:14:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.037+0000 7f324dae1640 1 -- 172.21.15.5:0/1314063990 shutdown_connections 2024-09-15T22:14:05.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.037+0000 7f324dae1640 1 --2- 172.21.15.5:0/1314063990 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3248102ef0 0x7f324810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.037+0000 7f324dae1640 1 --2- 172.21.15.5:0/1314063990 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3248102570 0x7f32481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.037+0000 7f324dae1640 1 --2- 172.21.15.5:0/1314063990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3248100e20 0x7f3248101200 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.037+0000 7f324dae1640 1 -- 172.21.15.5:0/1314063990 >> 172.21.15.5:0/1314063990 conn(0x7f32480fc7d0 msgr2=0x7f32480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.037+0000 7f324dae1640 1 -- 172.21.15.5:0/1314063990 shutdown_connections 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.037+0000 7f324dae1640 1 -- 172.21.15.5:0/1314063990 wait complete. 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.037+0000 7f324dae1640 1 Processor -- start 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f324dae1640 1 -- start start 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f324dae1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3248100e20 0x7f3248198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f324dae1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3248102570 0x7f3248198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f324dae1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3248102ef0 0x7f324819cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f324dae1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3248110f50 con 0x7f3248102570 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f324dae1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3248110dd0 con 0x7f3248102ef0 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f324dae1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f32481110d0 con 0x7f3248100e20 2024-09-15T22:14:05.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f3246ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3248102570 0x7f3248198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f3246ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3248102570 0x7f3248198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52494/0 (socket says 172.21.15.5:52494) 2024-09-15T22:14:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f3246ffd640 1 -- 172.21.15.5:0/4231329651 learned_addr learned my addr 172.21.15.5:0/4231329651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f32477fe640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3248100e20 0x7f3248198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f3247fff640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3248102ef0 0x7f324819cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f3246ffd640 1 -- 172.21.15.5:0/4231329651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3248100e20 msgr2=0x7f3248198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f3246ffd640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3248100e20 0x7f3248198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f3246ffd640 1 -- 172.21.15.5:0/4231329651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3248102ef0 msgr2=0x7f324819cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f3246ffd640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3248102ef0 0x7f324819cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.038+0000 7f3246ffd640 1 -- 172.21.15.5:0/4231329651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324819d4d0 con 0x7f3248102570 2024-09-15T22:14:05.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.039+0000 7f3246ffd640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3248102570 0x7f3248198a60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f323c00d950 tx=0x7f323c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:05.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.039+0000 7f3244ff9640 1 -- 172.21.15.5:0/4231329651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c017070 con 0x7f3248102570 2024-09-15T22:14:05.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.039+0000 7f324dae1640 1 -- 172.21.15.5:0/4231329651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324819d7c0 con 0x7f3248102570 2024-09-15T22:14:05.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.039+0000 7f3244ff9640 1 -- 172.21.15.5:0/4231329651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f323c009c30 con 0x7f3248102570 2024-09-15T22:14:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.039+0000 7f3244ff9640 1 -- 172.21.15.5:0/4231329651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c00f6b0 con 0x7f3248102570 2024-09-15T22:14:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.039+0000 7f324dae1640 1 -- 172.21.15.5:0/4231329651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32481a50a0 con 0x7f3248102570 2024-09-15T22:14:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.040+0000 7f324dae1640 1 -- 172.21.15.5:0/4231329651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3248101200 con 0x7f3248102570 2024-09-15T22:14:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.041+0000 7f3244ff9640 1 -- 172.21.15.5:0/4231329651 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f323c0109d0 con 0x7f3248102570 2024-09-15T22:14:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.042+0000 7f3244ff9640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f32300a33b0 0x7f32300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.042+0000 7f3244ff9640 1 -- 172.21.15.5:0/4231329651 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f323c0cfde0 con 0x7f3248102570 2024-09-15T22:14:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.042+0000 7f32477fe640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f32300a33b0 0x7f32300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:05.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.042+0000 7f32477fe640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f32300a33b0 0x7f32300a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3238005ab0 tx=0x7f3238005a20 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.046+0000 7f3244ff9640 1 -- 172.21.15.5:0/4231329651 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f323c009de0 con 0x7f3248102570 2024-09-15T22:14:05.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:04 smithi099 ceph-mon[34892]: pgmap v158: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-09-15T22:14:05.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:04 smithi129 ceph-mon[35314]: pgmap v158: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-09-15T22:14:05.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:04 smithi005 ceph-mon[30944]: pgmap v158: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 3.6 MiB/s wr, 415 op/s 2024-09-15T22:14:05.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.261+0000 7f324dae1640 1 -- 172.21.15.5:0/4231329651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f324810a0a0 con 0x7f3248102570 2024-09-15T22:14:05.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.262+0000 7f3244ff9640 1 -- 172.21.15.5:0/4231329651 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f323c09c420 con 0x7f3248102570 2024-09-15T22:14:05.264 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:14:05.264 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:14:05.264 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:14:05.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.264+0000 7f324dae1640 1 -- 172.21.15.5:0/4231329651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f32300a33b0 msgr2=0x7f32300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:05.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.264+0000 7f324dae1640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f32300a33b0 0x7f32300a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3238005ab0 tx=0x7f3238005a20 comp rx=0 tx=0).stop 2024-09-15T22:14:05.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.264+0000 7f324dae1640 1 -- 172.21.15.5:0/4231329651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3248102570 msgr2=0x7f3248198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:05.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.264+0000 7f324dae1640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3248102570 0x7f3248198a60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f323c00d950 tx=0x7f323c00de10 comp rx=0 tx=0).stop 2024-09-15T22:14:05.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.264+0000 7f324dae1640 1 -- 172.21.15.5:0/4231329651 shutdown_connections 2024-09-15T22:14:05.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.264+0000 7f324dae1640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f32300a33b0 0x7f32300a5870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:05.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.265+0000 7f324dae1640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3248102ef0 0x7f324819cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:05.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.265+0000 7f324dae1640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3248102570 0x7f3248198a60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:05.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.265+0000 7f324dae1640 1 --2- 172.21.15.5:0/4231329651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3248100e20 0x7f3248198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:05.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.265+0000 7f324dae1640 1 -- 172.21.15.5:0/4231329651 >> 172.21.15.5:0/4231329651 conn(0x7f32480fc7d0 msgr2=0x7f324806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:05.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.265+0000 7f324dae1640 1 -- 172.21.15.5:0/4231329651 shutdown_connections 2024-09-15T22:14:05.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:05.265+0000 7f324dae1640 1 -- 172.21.15.5:0/4231329651 wait complete. 2024-09-15T22:14:05.993 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:05 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4231329651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:06.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:14:06.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:05 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4231329651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:06.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:05 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4231329651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:06.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.821+0000 7f4204caf640 1 -- 172.21.15.5:0/3060608023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102570 msgr2=0x7f42001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:06.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.821+0000 7f4204caf640 1 --2- 172.21.15.5:0/3060608023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102570 0x7f42001029b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009900 tx=0x7f41f402a480 comp rx=0 tx=0).stop 2024-09-15T22:14:06.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.821+0000 7f4204caf640 1 -- 172.21.15.5:0/3060608023 shutdown_connections 2024-09-15T22:14:06.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.821+0000 7f4204caf640 1 --2- 172.21.15.5:0/3060608023 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4200102ef0 0x7f420010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.821+0000 7f4204caf640 1 --2- 172.21.15.5:0/3060608023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102570 0x7f42001029b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.821+0000 7f4204caf640 1 --2- 172.21.15.5:0/3060608023 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4200100e20 0x7f4200101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.821+0000 7f4204caf640 1 -- 172.21.15.5:0/3060608023 >> 172.21.15.5:0/3060608023 conn(0x7f42000fc7d0 msgr2=0x7f42000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.821+0000 7f4204caf640 1 -- 172.21.15.5:0/3060608023 shutdown_connections 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.821+0000 7f4204caf640 1 -- 172.21.15.5:0/3060608023 wait complete. 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f4204caf640 1 Processor -- start 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f4204caf640 1 -- start start 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f4204caf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4200100e20 0x7f42000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f4204caf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102570 0x7f42000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f4204caf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4200102ef0 0x7f42000fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f4204caf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4200110fa0 con 0x7f4200102570 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f4204caf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4200110e20 con 0x7f4200102ef0 2024-09-15T22:14:06.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f4204caf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4200111120 con 0x7f4200100e20 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f41fdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102570 0x7f42000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f41fdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102570 0x7f42000fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52522/0 (socket says 172.21.15.5:52522) 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f41fdd9b640 1 -- 172.21.15.5:0/91013388 learned_addr learned my addr 172.21.15.5:0/91013388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.822+0000 7f41fe59c640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4200100e20 0x7f42000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f41fed9d640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4200102ef0 0x7f42000fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f41fdd9b640 1 -- 172.21.15.5:0/91013388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4200100e20 msgr2=0x7f42000fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f41fdd9b640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4200100e20 0x7f42000fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f41fdd9b640 1 -- 172.21.15.5:0/91013388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4200102ef0 msgr2=0x7f42000fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f41fdd9b640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4200102ef0 0x7f42000fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f41fdd9b640 1 -- 172.21.15.5:0/91013388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42001a6a00 con 0x7f4200102570 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f41fed9d640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4200102ef0 0x7f42000fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f41fdd9b640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102570 0x7f42000fd1c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f41f4006010 tx=0x7f41f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f41ef7fe640 1 -- 172.21.15.5:0/91013388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f403f070 con 0x7f4200102570 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f4204caf640 1 -- 172.21.15.5:0/91013388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42001a6c90 con 0x7f4200102570 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f41ef7fe640 1 -- 172.21.15.5:0/91013388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f4031c30 con 0x7f4200102570 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f41ef7fe640 1 -- 172.21.15.5:0/91013388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4035030 con 0x7f4200102570 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.823+0000 7f4204caf640 1 -- 172.21.15.5:0/91013388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001a7170 con 0x7f4200102570 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.825+0000 7f41ef7fe640 1 -- 172.21.15.5:0/91013388 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f41f4042070 con 0x7f4200102570 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.825+0000 7f4207088740 1 Processor -- start 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f4207088740 1 -- start start 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f4207088740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a2030ab40 0x562a20309090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f4207088740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a203095d0 0x562a20309a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f4207088740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a20314370 0x562a20316760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f4207088740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a2026f2f0 con 0x562a2030ab40 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f4207088740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562a20216cf0 con 0x562a203095d0 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f4207088740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562a2026b2b0 con 0x562a20314370 2024-09-15T22:14:06.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f41fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a2030ab40 0x562a20309090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f41fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a2030ab40 0x562a20309090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52524/0 (socket says 172.21.15.5:52524) 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f41fed9d640 1 -- 172.21.15.5:0/3198291718 learned_addr learned my addr 172.21.15.5:0/3198291718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f41ef7fe640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f41e80a33b0 0x7f41e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.826+0000 7f41fe59c640 1 --2- 172.21.15.5:0/3198291718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a203095d0 0x562a20309a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41fdd9b640 1 --2- 172.21.15.5:0/3198291718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a20314370 0x562a20316760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41fed9d640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f41e80a33b0 0x7f41e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41ef7fe640 1 -- 172.21.15.5:0/91013388 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f41f40f0a30 con 0x7f4200102570 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41fe59c640 1 -- 172.21.15.5:0/3198291718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a20314370 msgr2=0x562a20316760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41fe59c640 1 --2- 172.21.15.5:0/3198291718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a20314370 0x562a20316760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41fe59c640 1 -- 172.21.15.5:0/3198291718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a2030ab40 msgr2=0x562a20309090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41fe59c640 1 --2- 172.21.15.5:0/3198291718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a2030ab40 0x562a20309090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41fe59c640 1 -- 172.21.15.5:0/3198291718 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a201fb3a0 con 0x562a203095d0 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41fed9d640 1 --2- 172.21.15.5:0/3198291718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a2030ab40 0x562a20309090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:06.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41fe59c640 1 --2- 172.21.15.5:0/3198291718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a203095d0 0x562a20309a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f41f000dbe0 tx=0x7f41f000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15befce4049e1ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41fed9d640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f41e80a33b0 0x7f41e80a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f41f8010b90 tx=0x7f41f8004a40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41ecff9640 1 -- 172.21.15.5:0/3198291718 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0005020 con 0x562a203095d0 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41ecff9640 1 -- 172.21.15.5:0/3198291718 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f00047a0 con 0x562a203095d0 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.827+0000 7f41ecff9640 1 -- 172.21.15.5:0/3198291718 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00116a0 con 0x562a203095d0 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.828+0000 7f4207088740 1 -- 172.21.15.5:0/3198291718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a203095d0 msgr2=0x562a20309a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.828+0000 7f4207088740 1 --2- 172.21.15.5:0/3198291718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a203095d0 0x562a20309a10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f41f000dbe0 tx=0x7f41f000f730 comp rx=0 tx=0).stop 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.828+0000 7f4207088740 1 -- 172.21.15.5:0/3198291718 shutdown_connections 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.828+0000 7f4207088740 1 --2- 172.21.15.5:0/3198291718 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a20314370 0x562a20316760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.828+0000 7f4207088740 1 --2- 172.21.15.5:0/3198291718 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a203095d0 0x562a20309a10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.828+0000 7f4207088740 1 --2- 172.21.15.5:0/3198291718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a2030ab40 0x562a20309090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.828+0000 7f4207088740 1 -- 172.21.15.5:0/3198291718 >> 172.21.15.5:0/3198291718 conn(0x562a202d2310 msgr2=0x562a20305c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.828+0000 7f4207088740 1 -- 172.21.15.5:0/3198291718 shutdown_connections 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.828+0000 7f4207088740 1 -- 172.21.15.5:0/3198291718 wait complete. 2024-09-15T22:14:06.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f4207088740 1 -- start start 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f4207088740 1 Processor -- start 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f4207088740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a203095d0 0x562a20313d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f4207088740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 0x562a20311000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f4207088740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f4207088740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a20317070 con 0x562a20314370 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f4207088740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562a2022a060 con 0x562a2030ab40 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f4207088740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562a203171f0 con 0x562a203095d0 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f4207088740 10 client.0 _ll_register_callbacks cb 0x562a1ff7d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203115d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52530/0 (socket says 172.21.15.5:52530) 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fed9d640 1 -- 172.21.15.5:0/1115541353 learned_addr learned my addr 172.21.15.5:0/1115541353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fdd9b640 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 0x562a20311000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fed9d640 1 -- 172.21.15.5:0/1115541353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a203095d0 msgr2=0x562a20313d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fe59c640 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a203095d0 0x562a20313d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fed9d640 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a203095d0 0x562a20313d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fed9d640 1 -- 172.21.15.5:0/1115541353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 msgr2=0x562a20311000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fed9d640 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 0x562a20311000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fed9d640 1 -- 172.21.15.5:0/1115541353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a1ffe03c0 con 0x562a20314370 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fdd9b640 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 0x562a20311000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.829+0000 7f41fe59c640 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a203095d0 0x562a20313d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f41fed9d640 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203115d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f41f800f6e0 tx=0x7f41f8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99060604510ad0d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f41e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1115541353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8022020 con 0x562a20314370 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1115541353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f41f801e700 con 0x562a20314370 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f4207088740 1 -- 172.21.15.5:0/1115541353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f00047a0 con 0x562a20314370 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1115541353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f41f802d050 con 0x562a20314370 2024-09-15T22:14:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f4207088740 10 client.16410 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f4207088740 1 -- 172.21.15.5:0/1115541353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a20322ba0 con 0x562a20314370 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1115541353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f00047a0 con 0x562a20314370 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1115541353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f41f801da30 con 0x562a20314370 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f4207088740 10 client.16410 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f4207088740 10 client.16410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f4207088740 20 client.16410 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f4207088740 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a20324060 0x562a20326520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f4207088740 4 client.16410 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.830+0000 7f4207088740 1 -- 172.21.15.5:0/1115541353 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a2027c860 con 0x562a20324060 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.831+0000 7f41fe59c640 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a20324060 0x562a20326520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.831+0000 7f41e5ffb640 10 client.16410 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.831+0000 7f41fe59c640 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a20324060 0x562a20326520 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c893d5c0a120a960 server_cookie=a0e737d5546ad4ce in_seq=0 out_seq=0 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1115541353 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a2027c860 con 0x562a20324060 2024-09-15T22:14:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f41e5ffb640 10 client.16410 handle_command_reply: tid=0 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f4207088740 1 client.16410 shutdown 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f4207088740 1 -- 172.21.15.5:0/1115541353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 msgr2=0x562a203115d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f4207088740 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203115d0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f41f800f6e0 tx=0x7f41f8004b50 comp rx=0 tx=0).stop 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f4207088740 1 -- 172.21.15.5:0/1115541353 shutdown_connections 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f4207088740 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a20324060 0x562a20326520 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f4207088740 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203115d0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f4207088740 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 0x562a20311000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f4207088740 1 --2- 172.21.15.5:0/1115541353 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a203095d0 0x562a20313d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f4207088740 1 -- 172.21.15.5:0/1115541353 >> 172.21.15.5:0/1115541353 conn(0x562a202d2310 msgr2=0x562a2030b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f41e5ffb640 0 client.16410 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.832+0000 7f4207088740 1 -- 172.21.15.5:0/1115541353 shutdown_connections 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.833+0000 7f4207088740 1 -- 172.21.15.5:0/1115541353 wait complete. 2024-09-15T22:14:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.833+0000 7f4207088740 20 client.16410 trim_cache size 0 max 16384 2024-09-15T22:14:06.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f4207088740 1 Processor -- start 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f4207088740 1 -- start start 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f4207088740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a203095d0 0x562a20305e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f4207088740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2030ab40 0x562a20320310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f4207088740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a20314370 0x562a2030c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f4207088740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b3ae0 con 0x562a203095d0 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f4207088740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b13b0 con 0x562a20314370 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f4207088740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f42001af200 con 0x562a2030ab40 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f41fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a203095d0 0x562a20305e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f41fe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a203095d0 0x562a20305e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52546/0 (socket says 172.21.15.5:52546) 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f41fe59c640 1 -- 172.21.15.5:0/1931436216 learned_addr learned my addr 172.21.15.5:0/1931436216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f41fdd9b640 1 --2- 172.21.15.5:0/1931436216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2030ab40 0x562a20320310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f41fe59c640 1 -- 172.21.15.5:0/1931436216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2030ab40 msgr2=0x562a20320310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:06.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f41fe59c640 1 --2- 172.21.15.5:0/1931436216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2030ab40 0x562a20320310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f41fe59c640 1 -- 172.21.15.5:0/1931436216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a20314370 msgr2=0x562a2030c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f41fe59c640 1 --2- 172.21.15.5:0/1931436216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a20314370 0x562a2030c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f41fe59c640 1 -- 172.21.15.5:0/1931436216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a2030f550 con 0x562a203095d0 2024-09-15T22:14:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.846+0000 7f41fe59c640 1 --2- 172.21.15.5:0/1931436216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a203095d0 0x562a20305e10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f41f000fc30 tx=0x7f41f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5007c7391d331a2a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f41e7fff640 1 -- 172.21.15.5:0/1931436216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0004240 con 0x562a203095d0 2024-09-15T22:14:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f41e7fff640 1 -- 172.21.15.5:0/1931436216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f0005510 con 0x562a203095d0 2024-09-15T22:14:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f41e7fff640 1 -- 172.21.15.5:0/1931436216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00035f0 con 0x562a203095d0 2024-09-15T22:14:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f4207088740 1 -- 172.21.15.5:0/1931436216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a203095d0 msgr2=0x562a20305e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f4207088740 1 --2- 172.21.15.5:0/1931436216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a203095d0 0x562a20305e10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f41f000fc30 tx=0x7f41f000d3b0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f4207088740 1 -- 172.21.15.5:0/1931436216 shutdown_connections 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f4207088740 1 --2- 172.21.15.5:0/1931436216 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a20314370 0x562a2030c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f4207088740 1 --2- 172.21.15.5:0/1931436216 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a2030ab40 0x562a20320310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f4207088740 1 --2- 172.21.15.5:0/1931436216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a203095d0 0x562a20305e10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f4207088740 1 -- 172.21.15.5:0/1931436216 >> 172.21.15.5:0/1931436216 conn(0x562a202d2310 msgr2=0x562a20305700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f4207088740 1 -- 172.21.15.5:0/1931436216 shutdown_connections 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.847+0000 7f4207088740 1 -- 172.21.15.5:0/1931436216 wait complete. 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f4207088740 1 -- start start 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f4207088740 1 Processor -- start 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f4207088740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a203095d0 0x562a2031cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f4207088740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 0x562a20317f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f4207088740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f4207088740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42001af870 con 0x562a20314370 2024-09-15T22:14:06.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f4207088740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b0ce0 con 0x562a2030ab40 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f4207088740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f42001aff40 con 0x562a203095d0 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f4207088740 10 client.0 _ll_register_callbacks cb 0x7f41f8010f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f41fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f41fed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203184d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52562/0 (socket says 172.21.15.5:52562) 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f41fed9d640 1 -- 172.21.15.5:0/1672084771 learned_addr learned my addr 172.21.15.5:0/1672084771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f41fe59c640 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a203095d0 0x562a2031cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.848+0000 7f41fdd9b640 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 0x562a20317f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41fed9d640 1 -- 172.21.15.5:0/1672084771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a203095d0 msgr2=0x562a2031cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41fed9d640 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a203095d0 0x562a2031cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41fed9d640 1 -- 172.21.15.5:0/1672084771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 msgr2=0x562a20317f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41fed9d640 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 0x562a20317f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41fed9d640 1 -- 172.21.15.5:0/1672084771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a20318d00 con 0x562a20314370 2024-09-15T22:14:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41fdd9b640 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 0x562a20317f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41fed9d640 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203184d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f41f8007bf0 tx=0x7f41f800e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e5fa018d6acb23 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1672084771 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8035020 con 0x562a20314370 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1672084771 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f41f8007280 con 0x562a20314370 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f4207088740 1 -- 172.21.15.5:0/1672084771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f0005510 con 0x562a20314370 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1672084771 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f41f8040050 con 0x562a20314370 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f4207088740 10 client.16416 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f4207088740 1 -- 172.21.15.5:0/1672084771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a2038eea0 con 0x562a20314370 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.849+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1672084771 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f0005510 con 0x562a20314370 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1672084771 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f41f801d8f0 con 0x562a20314370 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f4207088740 10 client.16416 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f4207088740 10 client.16416 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f4207088740 20 client.16416 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f4207088740 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a2038f5b0 0x562a203919a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f4207088740 4 client.16416 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f4207088740 1 -- 172.21.15.5:0/1672084771 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a20276d70 con 0x562a2038f5b0 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f41fe59c640 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a2038f5b0 0x562a203919a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f41fe59c640 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a2038f5b0 0x562a203919a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6a3e0cde76bd234 server_cookie=c33770a7ccaf9fa6 in_seq=0 out_seq=0 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f41e5ffb640 10 client.16416 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f41e5ffb640 1 -- 172.21.15.5:0/1672084771 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a20276d70 con 0x562a2038f5b0 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f41e5ffb640 10 client.16416 handle_command_reply: tid=0 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.850+0000 7f4207088740 1 client.16416 shutdown 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4207088740 1 -- 172.21.15.5:0/1672084771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 msgr2=0x562a203184d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4207088740 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203184d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f41f8007bf0 tx=0x7f41f800e690 comp rx=0 tx=0).stop 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4207088740 1 -- 172.21.15.5:0/1672084771 shutdown_connections 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4207088740 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x562a2038f5b0 0x562a203919a0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4207088740 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a20314370 0x562a203184d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4207088740 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562a2030ab40 0x562a20317f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4207088740 1 --2- 172.21.15.5:0/1672084771 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562a203095d0 0x562a2031cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4207088740 1 -- 172.21.15.5:0/1672084771 >> 172.21.15.5:0/1672084771 conn(0x562a202d2310 msgr2=0x562a20313080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f41e5ffb640 0 client.16416 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:14:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4207088740 1 -- 172.21.15.5:0/1672084771 shutdown_connections 2024-09-15T22:14:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4207088740 1 -- 172.21.15.5:0/1672084771 wait complete. 2024-09-15T22:14:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4207088740 20 client.16416 trim_cache size 0 max 16384 2024-09-15T22:14:06.860 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:14:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4204caf640 1 -- 172.21.15.5:0/91013388 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f41e80a33b0 msgr2=0x7f41e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4204caf640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f41e80a33b0 0x7f41e80a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f41f8010b90 tx=0x7f41f8004a40 comp rx=0 tx=0).stop 2024-09-15T22:14:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4204caf640 1 -- 172.21.15.5:0/91013388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102570 msgr2=0x7f42000fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.851+0000 7f4204caf640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102570 0x7f42000fd1c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f41f4006010 tx=0x7f41f4003c10 comp rx=0 tx=0).stop 2024-09-15T22:14:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.852+0000 7f4204caf640 1 -- 172.21.15.5:0/91013388 shutdown_connections 2024-09-15T22:14:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.852+0000 7f4204caf640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f41e80a33b0 0x7f41e80a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.852+0000 7f4204caf640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4200102ef0 0x7f42000fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.852+0000 7f4204caf640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102570 0x7f42000fd1c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.852+0000 7f4204caf640 1 --2- 172.21.15.5:0/91013388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4200100e20 0x7f42000fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.852+0000 7f4204caf640 1 -- 172.21.15.5:0/91013388 >> 172.21.15.5:0/91013388 conn(0x7f42000fc7d0 msgr2=0x7f4200069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.852+0000 7f4204caf640 1 -- 172.21.15.5:0/91013388 shutdown_connections 2024-09-15T22:14:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:06.852+0000 7f4204caf640 1 -- 172.21.15.5:0/91013388 wait complete. 2024-09-15T22:14:07.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:06 smithi099 ceph-mon[34892]: pgmap v159: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.6 MiB/s wr, 433 op/s 2024-09-15T22:14:07.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:06 smithi129 ceph-mon[35314]: pgmap v159: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.6 MiB/s wr, 433 op/s 2024-09-15T22:14:07.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:06 smithi005 ceph-mon[30944]: pgmap v159: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 3.6 MiB/s wr, 433 op/s 2024-09-15T22:14:07.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:14:07.945 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:08.066 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:07 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:14:08.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:07 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:14:08.096 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:07 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:14:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.487+0000 7f55759e6640 1 -- 172.21.15.5:0/3096302717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5570102ef0 msgr2=0x7f557010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.487+0000 7f55759e6640 1 --2- 172.21.15.5:0/3096302717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5570102ef0 0x7f557010da40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5568009900 tx=0x7f556802a480 comp rx=0 tx=0).stop 2024-09-15T22:14:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.487+0000 7f55759e6640 1 -- 172.21.15.5:0/3096302717 shutdown_connections 2024-09-15T22:14:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.487+0000 7f55759e6640 1 --2- 172.21.15.5:0/3096302717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5570102ef0 0x7f557010da40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.487+0000 7f55759e6640 1 --2- 172.21.15.5:0/3096302717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5570102570 0x7f55701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.487+0000 7f55759e6640 1 --2- 172.21.15.5:0/3096302717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570100e20 0x7f5570101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.487+0000 7f55759e6640 1 -- 172.21.15.5:0/3096302717 >> 172.21.15.5:0/3096302717 conn(0x7f55700fc7d0 msgr2=0x7f55700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.487+0000 7f55759e6640 1 -- 172.21.15.5:0/3096302717 shutdown_connections 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.487+0000 7f55759e6640 1 -- 172.21.15.5:0/3096302717 wait complete. 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.487+0000 7f55759e6640 1 Processor -- start 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f55759e6640 1 -- start start 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f55759e6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5570100e20 0x7f557010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f55759e6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570102570 0x7f557010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f55759e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5570102ef0 0x7f557010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f55759e6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55701120a0 con 0x7f5570102ef0 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f55759e6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5570111f20 con 0x7f5570102570 2024-09-15T22:14:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f55759e6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5570112220 con 0x7f5570100e20 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f556f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5570102ef0 0x7f557010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f556f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5570102ef0 0x7f557010ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52584/0 (socket says 172.21.15.5:52584) 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f556f7fe640 1 -- 172.21.15.5:0/3871708010 learned_addr learned my addr 172.21.15.5:0/3871708010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f556effd640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5570100e20 0x7f557010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f556e7fc640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570102570 0x7f557010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f556f7fe640 1 -- 172.21.15.5:0/3871708010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5570100e20 msgr2=0x7f557010e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f556f7fe640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5570100e20 0x7f557010e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f556f7fe640 1 -- 172.21.15.5:0/3871708010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570102570 msgr2=0x7f557010ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f556f7fe640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570102570 0x7f557010ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f556f7fe640 1 -- 172.21.15.5:0/3871708010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55701a6960 con 0x7f5570102ef0 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.488+0000 7f556e7fc640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570102570 0x7f557010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.489+0000 7f556f7fe640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5570102ef0 0x7f557010ef50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5568004770 tx=0x7f5568003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.489+0000 7f55749e4640 1 -- 172.21.15.5:0/3871708010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556803f070 con 0x7f5570102ef0 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.489+0000 7f55759e6640 1 -- 172.21.15.5:0/3871708010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55701a6bf0 con 0x7f5570102ef0 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.489+0000 7f55749e4640 1 -- 172.21.15.5:0/3871708010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5568031c20 con 0x7f5570102ef0 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.489+0000 7f55749e4640 1 -- 172.21.15.5:0/3871708010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568035030 con 0x7f5570102ef0 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.489+0000 7f55759e6640 1 -- 172.21.15.5:0/3871708010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55701a7120 con 0x7f5570102ef0 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.491+0000 7f55749e4640 1 -- 172.21.15.5:0/3871708010 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5568042070 con 0x7f5570102ef0 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.491+0000 7f5577dbf740 1 Processor -- start 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.491+0000 7f5577dbf740 1 -- start start 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858d9b40 0x55e4858d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858e3370 0x55e4858e5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f5577dbf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e48583e2f0 con 0x55e4858d85d0 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f5577dbf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e4857e5cf0 con 0x55e4858d9b40 2024-09-15T22:14:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f5577dbf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e48583a2b0 con 0x55e4858e3370 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556e7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858d9b40 0x55e4858d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858d8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52594/0 (socket says 172.21.15.5:52594) 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556effd640 1 -- 172.21.15.5:0/82072159 learned_addr learned my addr 172.21.15.5:0/82072159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556e7fc640 1 --2- 172.21.15.5:0/82072159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858e3370 0x55e4858e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556effd640 1 -- 172.21.15.5:0/82072159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858e3370 msgr2=0x55e4858e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556effd640 1 --2- 172.21.15.5:0/82072159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858e3370 0x55e4858e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556effd640 1 -- 172.21.15.5:0/82072159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858d9b40 msgr2=0x55e4858d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556effd640 1 --2- 172.21.15.5:0/82072159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858d9b40 0x55e4858d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556effd640 1 -- 172.21.15.5:0/82072159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4857ca3a0 con 0x55e4858d85d0 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f55749e4640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f555c0a33b0 0x7f555c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556e7fc640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f555c0a33b0 0x7f555c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f556effd640 1 --2- 172.21.15.5:0/82072159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858d8a10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f55600114f0 tx=0x7f55600119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab5f6e155f02554 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.492+0000 7f55749e4640 1 -- 172.21.15.5:0/3871708010 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f55680f0a30 con 0x7f5570102ef0 2024-09-15T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.493+0000 7f5565ffb640 1 -- 172.21.15.5:0/82072159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556001e070 con 0x55e4858d85d0 2024-09-15T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.493+0000 7f5565ffb640 1 -- 172.21.15.5:0/82072159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f556000fa80 con 0x55e4858d85d0 2024-09-15T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.493+0000 7f5565ffb640 1 -- 172.21.15.5:0/82072159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560019aa0 con 0x55e4858d85d0 2024-09-15T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.493+0000 7f5577dbf740 1 -- 172.21.15.5:0/82072159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 msgr2=0x55e4858d8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.493+0000 7f5577dbf740 1 --2- 172.21.15.5:0/82072159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858d8a10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f55600114f0 tx=0x7f55600119b0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.493+0000 7f556e7fc640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f555c0a33b0 0x7f555c0a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55e4858d9070 tx=0x7f55580096a0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.494+0000 7f5577dbf740 1 -- 172.21.15.5:0/82072159 shutdown_connections 2024-09-15T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.494+0000 7f5577dbf740 1 --2- 172.21.15.5:0/82072159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858e3370 0x55e4858e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.494+0000 7f5577dbf740 1 --2- 172.21.15.5:0/82072159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858d8a10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.494+0000 7f5577dbf740 1 --2- 172.21.15.5:0/82072159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858d9b40 0x55e4858d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.494+0000 7f5577dbf740 1 -- 172.21.15.5:0/82072159 >> 172.21.15.5:0/82072159 conn(0x55e4858a1310 msgr2=0x55e4858d4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.494+0000 7f5577dbf740 1 -- 172.21.15.5:0/82072159 shutdown_connections 2024-09-15T22:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.494+0000 7f5577dbf740 1 -- 172.21.15.5:0/82072159 wait complete. 2024-09-15T22:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.494+0000 7f5577dbf740 1 -- start start 2024-09-15T22:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.494+0000 7f5577dbf740 1 Processor -- start 2024-09-15T22:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 0x55e4858e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d9b40 0x55e4858dffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858e3370 0x55e4858e0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f5577dbf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4858e6070 con 0x55e4858d9b40 2024-09-15T22:14:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f5577dbf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e4857f9060 con 0x55e4858e3370 2024-09-15T22:14:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f5577dbf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e4858e61f0 con 0x55e4858d85d0 2024-09-15T22:14:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f5577dbf740 10 client.0 _ll_register_callbacks cb 0x55e48554c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d9b40 0x55e4858dffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d9b40 0x55e4858dffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52604/0 (socket says 172.21.15.5:52604) 2024-09-15T22:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556e7fc640 1 -- 172.21.15.5:0/3362878832 learned_addr learned my addr 172.21.15.5:0/3362878832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556effd640 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 0x55e4858e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556f7fe640 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858e3370 0x55e4858e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556effd640 1 -- 172.21.15.5:0/3362878832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858e3370 msgr2=0x55e4858e0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556effd640 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858e3370 0x55e4858e0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556effd640 1 -- 172.21.15.5:0/3362878832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d9b40 msgr2=0x55e4858dffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556effd640 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d9b40 0x55e4858dffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556effd640 1 -- 172.21.15.5:0/3362878832 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4855af3c0 con 0x55e4858d85d0 2024-09-15T22:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556e7fc640 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d9b40 0x55e4858dffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:08.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f556effd640 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 0x55e4858e2d10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5560000f80 tx=0x7f5560019600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1198465081ffd878 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.495+0000 7f554effd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f554effd640 1 -- 172.21.15.5:0/3362878832 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556001e040 con 0x55e4858d85d0 2024-09-15T22:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f5577dbf740 1 -- 172.21.15.5:0/3362878832 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f556000fa80 con 0x55e4858d85d0 2024-09-15T22:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f554effd640 1 -- 172.21.15.5:0/3362878832 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f5560016040 con 0x55e4858d85d0 2024-09-15T22:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f554effd640 1 -- 172.21.15.5:0/3362878832 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5560031050 con 0x55e4858d85d0 2024-09-15T22:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f5577dbf740 10 client.25966 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f5577dbf740 1 -- 172.21.15.5:0/3362878832 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e4858f1af0 con 0x55e4858d85d0 2024-09-15T22:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f554effd640 1 -- 172.21.15.5:0/3362878832 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f556000fa80 con 0x55e4858d85d0 2024-09-15T22:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f554effd640 1 -- 172.21.15.5:0/3362878832 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5560019c60 con 0x55e4858d85d0 2024-09-15T22:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f5577dbf740 10 client.25966 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:08.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f5577dbf740 10 client.25966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f5577dbf740 20 client.25966 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f5577dbf740 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e4858f2ea0 0x55e4858f5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f5577dbf740 4 client.25966 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.496+0000 7f5577dbf740 1 -- 172.21.15.5:0/3362878832 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e48584b860 con 0x55e4858f2ea0 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.497+0000 7f556e7fc640 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e4858f2ea0 0x55e4858f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.497+0000 7f554effd640 10 client.25966 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.497+0000 7f556e7fc640 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e4858f2ea0 0x55e4858f5290 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f86b6a20465f1fd9 server_cookie=6bd2fc71222c8282 in_seq=0 out_seq=0 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f554effd640 1 -- 172.21.15.5:0/3362878832 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e48584b860 con 0x55e4858f2ea0 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f554effd640 10 client.25966 handle_command_reply: tid=0 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 1 client.25966 shutdown 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 1 -- 172.21.15.5:0/3362878832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 msgr2=0x55e4858e2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 0x55e4858e2d10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5560000f80 tx=0x7f5560019600 comp rx=0 tx=0).stop 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 1 -- 172.21.15.5:0/3362878832 shutdown_connections 2024-09-15T22:14:08.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e4858f2ea0 0x55e4858f5290 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858e3370 0x55e4858e0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f554effd640 0 client.25966 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:14:08.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d9b40 0x55e4858dffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 1 --2- 172.21.15.5:0/3362878832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 0x55e4858e2d10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 1 -- 172.21.15.5:0/3362878832 >> 172.21.15.5:0/3362878832 conn(0x55e4858a1310 msgr2=0x55e4858da930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:08.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 1 -- 172.21.15.5:0/3362878832 shutdown_connections 2024-09-15T22:14:08.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 1 -- 172.21.15.5:0/3362878832 wait complete. 2024-09-15T22:14:08.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.498+0000 7f5577dbf740 20 client.25966 trim_cache size 0 max 16384 2024-09-15T22:14:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f5577dbf740 1 Processor -- start 2024-09-15T22:14:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f5577dbf740 1 -- start start 2024-09-15T22:14:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 0x55e4858ef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858d9b40 0x55e4858dee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858e3370 0x55e4858faa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f5577dbf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55701b3b10 con 0x55e4858e3370 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f5577dbf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55701b13e0 con 0x55e4858d9b40 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f5577dbf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55701af230 con 0x55e4858d85d0 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f556f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858e3370 0x55e4858faa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f556f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858e3370 0x55e4858faa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52614/0 (socket says 172.21.15.5:52614) 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f556f7fe640 1 -- 172.21.15.5:0/4050423710 learned_addr learned my addr 172.21.15.5:0/4050423710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f556effd640 1 --2- 172.21.15.5:0/4050423710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 0x55e4858ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.512+0000 7f556e7fc640 1 --2- 172.21.15.5:0/4050423710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858d9b40 0x55e4858dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.513+0000 7f556f7fe640 1 -- 172.21.15.5:0/4050423710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 msgr2=0x55e4858ef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.513+0000 7f556f7fe640 1 --2- 172.21.15.5:0/4050423710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 0x55e4858ef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.513+0000 7f556f7fe640 1 -- 172.21.15.5:0/4050423710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858d9b40 msgr2=0x55e4858dee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.513+0000 7f556f7fe640 1 --2- 172.21.15.5:0/4050423710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858d9b40 0x55e4858dee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.513+0000 7f556f7fe640 1 -- 172.21.15.5:0/4050423710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4858d5960 con 0x55e4858e3370 2024-09-15T22:14:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.513+0000 7f556effd640 1 --2- 172.21.15.5:0/4050423710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 0x55e4858ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.513+0000 7f556e7fc640 1 --2- 172.21.15.5:0/4050423710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858d9b40 0x55e4858dee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.513+0000 7f556f7fe640 1 --2- 172.21.15.5:0/4050423710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858e3370 0x55e4858faa50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f55680bc9b0 tx=0x7f55680bc620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2bb1c5a09a2432 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.513+0000 7f55657fa640 1 -- 172.21.15.5:0/4050423710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568049040 con 0x55e4858e3370 2024-09-15T22:14:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.513+0000 7f55657fa640 1 -- 172.21.15.5:0/4050423710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55680c0d20 con 0x55e4858e3370 2024-09-15T22:14:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.513+0000 7f55657fa640 1 -- 172.21.15.5:0/4050423710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55680b9090 con 0x55e4858e3370 2024-09-15T22:14:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.514+0000 7f5577dbf740 1 -- 172.21.15.5:0/4050423710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858e3370 msgr2=0x55e4858faa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.514+0000 7f5577dbf740 1 --2- 172.21.15.5:0/4050423710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858e3370 0x55e4858faa50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f55680bc9b0 tx=0x7f55680bc620 comp rx=0 tx=0).stop 2024-09-15T22:14:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.514+0000 7f5577dbf740 1 -- 172.21.15.5:0/4050423710 shutdown_connections 2024-09-15T22:14:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.514+0000 7f5577dbf740 1 --2- 172.21.15.5:0/4050423710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858e3370 0x55e4858faa50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.514+0000 7f5577dbf740 1 --2- 172.21.15.5:0/4050423710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858d9b40 0x55e4858dee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.514+0000 7f5577dbf740 1 --2- 172.21.15.5:0/4050423710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d85d0 0x55e4858ef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.514+0000 7f5577dbf740 1 -- 172.21.15.5:0/4050423710 >> 172.21.15.5:0/4050423710 conn(0x55e4858a1310 msgr2=0x55e4858de760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.514+0000 7f5577dbf740 1 -- 172.21.15.5:0/4050423710 shutdown_connections 2024-09-15T22:14:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.514+0000 7f5577dbf740 1 -- 172.21.15.5:0/4050423710 wait complete. 2024-09-15T22:14:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.514+0000 7f5577dbf740 1 -- start start 2024-09-15T22:14:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.514+0000 7f5577dbf740 1 Processor -- start 2024-09-15T22:14:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858eb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d9b40 0x55e4858ebbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f5577dbf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858e3370 0x55e4858dc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f5577dbf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55701af8a0 con 0x55e4858d85d0 2024-09-15T22:14:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f5577dbf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55701b0d10 con 0x55e4858e3370 2024-09-15T22:14:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f5577dbf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55701aff70 con 0x55e4858d9b40 2024-09-15T22:14:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f5577dbf740 10 client.0 _ll_register_callbacks cb 0x7f556001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f556effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858eb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f556effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858eb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52618/0 (socket says 172.21.15.5:52618) 2024-09-15T22:14:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f556effd640 1 -- 172.21.15.5:0/2283443755 learned_addr learned my addr 172.21.15.5:0/2283443755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f556f7fe640 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858e3370 0x55e4858dc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f556e7fc640 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d9b40 0x55e4858ebbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f556effd640 1 -- 172.21.15.5:0/2283443755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d9b40 msgr2=0x55e4858ebbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f556effd640 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d9b40 0x55e4858ebbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f556effd640 1 -- 172.21.15.5:0/2283443755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858e3370 msgr2=0x55e4858dc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f556effd640 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858e3370 0x55e4858dc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f556effd640 1 -- 172.21.15.5:0/2283443755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4858dd000 con 0x55e4858d85d0 2024-09-15T22:14:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f556effd640 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858eb690 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5560005230 tx=0x7f5560005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedf3e214956e158 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.515+0000 7f554effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f554effd640 1 -- 172.21.15.5:0/2283443755 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560015020 con 0x55e4858d85d0 2024-09-15T22:14:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f554effd640 1 -- 172.21.15.5:0/2283443755 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f5560005960 con 0x55e4858d85d0 2024-09-15T22:14:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f5577dbf740 1 -- 172.21.15.5:0/2283443755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55680c0d20 con 0x55e4858d85d0 2024-09-15T22:14:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f554effd640 1 -- 172.21.15.5:0/2283443755 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f556003e050 con 0x55e4858d85d0 2024-09-15T22:14:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f5577dbf740 10 client.16434 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f5577dbf740 1 -- 172.21.15.5:0/2283443755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e48595d460 con 0x55e4858d85d0 2024-09-15T22:14:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f554effd640 1 -- 172.21.15.5:0/2283443755 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55680c0d20 con 0x55e4858d85d0 2024-09-15T22:14:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f554effd640 1 -- 172.21.15.5:0/2283443755 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5560019670 con 0x55e4858d85d0 2024-09-15T22:14:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f5577dbf740 10 client.16434 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f5577dbf740 10 client.16434 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:14:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f5577dbf740 20 client.16434 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f5577dbf740 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e4858f2ea0 0x55e48595fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f5577dbf740 4 client.16434 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.516+0000 7f5577dbf740 1 -- 172.21.15.5:0/2283443755 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e485845d70 con 0x55e4858f2ea0 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.517+0000 7f556e7fc640 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e4858f2ea0 0x55e48595fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.517+0000 7f556e7fc640 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e4858f2ea0 0x55e48595fad0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db3d8ad45dd8504d server_cookie=bf7f0e785f24a9f3 in_seq=0 out_seq=0 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.517+0000 7f554effd640 10 client.16434 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.517+0000 7f554effd640 1 -- 172.21.15.5:0/2283443755 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e485845d70 con 0x55e4858f2ea0 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.517+0000 7f554effd640 10 client.16434 handle_command_reply: tid=0 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.517+0000 7f5577dbf740 1 client.16434 shutdown 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f5577dbf740 1 -- 172.21.15.5:0/2283443755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 msgr2=0x55e4858eb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f5577dbf740 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858eb690 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5560005230 tx=0x7f5560005540 comp rx=0 tx=0).stop 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f5577dbf740 1 -- 172.21.15.5:0/2283443755 shutdown_connections 2024-09-15T22:14:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f5577dbf740 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e4858f2ea0 0x55e48595fad0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f5577dbf740 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e4858e3370 0x55e4858dc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f5577dbf740 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e4858d9b40 0x55e4858ebbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f5577dbf740 1 --2- 172.21.15.5:0/2283443755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4858d85d0 0x55e4858eb690 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f5577dbf740 1 -- 172.21.15.5:0/2283443755 >> 172.21.15.5:0/2283443755 conn(0x55e4858a1310 msgr2=0x55e4858def80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f554effd640 0 client.16434 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:14:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f5577dbf740 1 -- 172.21.15.5:0/2283443755 shutdown_connections 2024-09-15T22:14:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f5577dbf740 1 -- 172.21.15.5:0/2283443755 wait complete. 2024-09-15T22:14:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f5577dbf740 20 client.16434 trim_cache size 0 max 16384 2024-09-15T22:14:08.533 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:14:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f55759e6640 1 -- 172.21.15.5:0/3871708010 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f555c0a33b0 msgr2=0x7f555c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f55759e6640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f555c0a33b0 0x7f555c0a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55e4858d9070 tx=0x7f55580096a0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f55759e6640 1 -- 172.21.15.5:0/3871708010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5570102ef0 msgr2=0x7f557010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.518+0000 7f55759e6640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5570102ef0 0x7f557010ef50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5568004770 tx=0x7f5568003b90 comp rx=0 tx=0).stop 2024-09-15T22:14:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.519+0000 7f55759e6640 1 -- 172.21.15.5:0/3871708010 shutdown_connections 2024-09-15T22:14:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.519+0000 7f55759e6640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f555c0a33b0 0x7f555c0a5870 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.519+0000 7f55759e6640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5570102ef0 0x7f557010ef50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.519+0000 7f55759e6640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5570102570 0x7f557010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.519+0000 7f55759e6640 1 --2- 172.21.15.5:0/3871708010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5570100e20 0x7f557010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.519+0000 7f55759e6640 1 -- 172.21.15.5:0/3871708010 >> 172.21.15.5:0/3871708010 conn(0x7f55700fc7d0 msgr2=0x7f5570106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.519+0000 7f55759e6640 1 -- 172.21.15.5:0/3871708010 shutdown_connections 2024-09-15T22:14:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:08.519+0000 7f55759e6640 1 -- 172.21.15.5:0/3871708010 wait complete. 2024-09-15T22:14:09.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:08 smithi099 ceph-mon[34892]: pgmap v160: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-09-15T22:14:09.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:08 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:14:09.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:08 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:14:09.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:08 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:14:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:08 smithi129 ceph-mon[35314]: pgmap v160: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-09-15T22:14:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:08 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:14:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:08 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:14:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:08 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:14:09.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:08 smithi005 ceph-mon[30944]: pgmap v160: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-09-15T22:14:09.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:08 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:14:09.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:08 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:14:09.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:08 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:14:09.496 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:14:09.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 280.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:14:09.762 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.256+0000 7fb24dc79640 1 -- 172.21.15.5:0/4027834279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb248103660 msgr2=0x7fb24810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.256+0000 7fb24dc79640 1 --2- 172.21.15.5:0/4027834279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb248103660 0x7fb24810bb50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb230009900 tx=0x7fb23002a480 comp rx=0 tx=0).stop 2024-09-15T22:14:10.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.256+0000 7fb24dc79640 1 -- 172.21.15.5:0/4027834279 shutdown_connections 2024-09-15T22:14:10.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.256+0000 7fb24dc79640 1 --2- 172.21.15.5:0/4027834279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb24810c170 0x7fb24810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.256+0000 7fb24dc79640 1 --2- 172.21.15.5:0/4027834279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb248103660 0x7fb24810bb50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.256+0000 7fb24dc79640 1 --2- 172.21.15.5:0/4027834279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb248102d40 0x7fb248103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.256+0000 7fb24dc79640 1 -- 172.21.15.5:0/4027834279 >> 172.21.15.5:0/4027834279 conn(0x7fb2480fc7a0 msgr2=0x7fb2480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:10.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.257+0000 7fb24dc79640 1 -- 172.21.15.5:0/4027834279 shutdown_connections 2024-09-15T22:14:10.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.257+0000 7fb24dc79640 1 -- 172.21.15.5:0/4027834279 wait complete. 2024-09-15T22:14:10.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.257+0000 7fb24dc79640 1 Processor -- start 2024-09-15T22:14:10.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.257+0000 7fb24dc79640 1 -- start start 2024-09-15T22:14:10.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb24dc79640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb248102d40 0x7fb248198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb24dc79640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb248103660 0x7fb2481988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb24dc79640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24810c170 0x7fb24819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb24dc79640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb248111f70 con 0x7fb24810c170 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb24dc79640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb248111df0 con 0x7fb248102d40 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb24dc79640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481120f0 con 0x7fb248103660 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb247fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24810c170 0x7fb24819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb247fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24810c170 0x7fb24819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52632/0 (socket says 172.21.15.5:52632) 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb247fff640 1 -- 172.21.15.5:0/394405978 learned_addr learned my addr 172.21.15.5:0/394405978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb2477fe640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb248102d40 0x7fb248198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb23effd640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb248103660 0x7fb2481988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb247fff640 1 -- 172.21.15.5:0/394405978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb248103660 msgr2=0x7fb2481988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb247fff640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb248103660 0x7fb2481988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb247fff640 1 -- 172.21.15.5:0/394405978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb248102d40 msgr2=0x7fb248198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb247fff640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb248102d40 0x7fb248198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb247fff640 1 -- 172.21.15.5:0/394405978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24819d320 con 0x7fb24810c170 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb2477fe640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb248102d40 0x7fb248198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb23effd640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb248103660 0x7fb2481988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.258+0000 7fb247fff640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24810c170 0x7fb24819cc40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb24000cce0 tx=0x7fb24000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.259+0000 7fb2457fa640 1 -- 172.21.15.5:0/394405978 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24000f450 con 0x7fb24810c170 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.259+0000 7fb2457fa640 1 -- 172.21.15.5:0/394405978 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb24000fad0 con 0x7fb24810c170 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.259+0000 7fb24dc79640 1 -- 172.21.15.5:0/394405978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24819d610 con 0x7fb24810c170 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.259+0000 7fb2457fa640 1 -- 172.21.15.5:0/394405978 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240018c10 con 0x7fb24810c170 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.259+0000 7fb24dc79640 1 -- 172.21.15.5:0/394405978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2481a4ef0 con 0x7fb24810c170 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.261+0000 7fb2457fa640 1 -- 172.21.15.5:0/394405978 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb24000f5f0 con 0x7fb24810c170 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.261+0000 7fb250052740 1 Processor -- start 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.261+0000 7fb250052740 1 -- start start 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb250052740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8fb40 0x556cd5c8e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb250052740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5c8ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb250052740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c9b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb250052740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cd5bf42f0 con 0x556cd5c99370 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb250052740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cd5b9bcf0 con 0x556cd5c8fb40 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb250052740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556cd5bf02b0 con 0x556cd5c8e5d0 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb23effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c9b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb23effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c9b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52636/0 (socket says 172.21.15.5:52636) 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb23effd640 1 -- 172.21.15.5:0/319411706 learned_addr learned my addr 172.21.15.5:0/319411706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb23effd640 1 --2- 172.21.15.5:0/319411706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8fb40 0x556cd5c8e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb2477fe640 1 --2- 172.21.15.5:0/319411706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5c8ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb23effd640 1 -- 172.21.15.5:0/319411706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 msgr2=0x556cd5c8ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb23effd640 1 --2- 172.21.15.5:0/319411706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5c8ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb2457fa640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb2340a3430 0x7fb2340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb23effd640 1 -- 172.21.15.5:0/319411706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8fb40 msgr2=0x556cd5c8e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb23effd640 1 --2- 172.21.15.5:0/319411706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8fb40 0x556cd5c8e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb23effd640 1 -- 172.21.15.5:0/319411706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd5b803a0 con 0x556cd5c99370 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.262+0000 7fb247fff640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb2340a3430 0x7fb2340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.263+0000 7fb23effd640 1 --2- 172.21.15.5:0/319411706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c9b760 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb230005ec0 tx=0x7fb230000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d194006751007be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.263+0000 7fb2457fa640 1 -- 172.21.15.5:0/394405978 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fb2400d0750 con 0x7fb24810c170 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.263+0000 7fb23dffb640 1 -- 172.21.15.5:0/319411706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230035040 con 0x556cd5c99370 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.263+0000 7fb23dffb640 1 -- 172.21.15.5:0/319411706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb230005050 con 0x556cd5c99370 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.263+0000 7fb23dffb640 1 -- 172.21.15.5:0/319411706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230004920 con 0x556cd5c99370 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 -- 172.21.15.5:0/319411706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 msgr2=0x556cd5c9b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 --2- 172.21.15.5:0/319411706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c9b760 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb230005ec0 tx=0x7fb230000bc0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb247fff640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb2340a3430 0x7fb2340a58f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb24001b000 tx=0x7fb24006e3b0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:10.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 -- 172.21.15.5:0/319411706 shutdown_connections 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 --2- 172.21.15.5:0/319411706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c9b760 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 --2- 172.21.15.5:0/319411706 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5c8ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 --2- 172.21.15.5:0/319411706 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8fb40 0x556cd5c8e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 -- 172.21.15.5:0/319411706 >> 172.21.15.5:0/319411706 conn(0x556cd5c57310 msgr2=0x556cd5c8ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 -- 172.21.15.5:0/319411706 shutdown_connections 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 -- 172.21.15.5:0/319411706 wait complete. 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 -- start start 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 Processor -- start 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5c98d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8fb40 0x556cd5c96030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c96640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cd5c9c010 con 0x556cd5c99370 2024-09-15T22:14:10.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556cd5baf060 con 0x556cd5c8fb40 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.264+0000 7fb250052740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556cd5c9c190 con 0x556cd5c8e5d0 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb250052740 10 client.0 _ll_register_callbacks cb 0x556cd59028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb23effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c96640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb23effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c96640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52646/0 (socket says 172.21.15.5:52646) 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb23effd640 1 -- 172.21.15.5:0/3250827641 learned_addr learned my addr 172.21.15.5:0/3250827641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb23effd640 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5c98d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb2477fe640 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8fb40 0x556cd5c96030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb23effd640 1 -- 172.21.15.5:0/3250827641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 msgr2=0x556cd5c98d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb23effd640 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5c98d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb23effd640 1 -- 172.21.15.5:0/3250827641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8fb40 msgr2=0x556cd5c96030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb23effd640 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8fb40 0x556cd5c96030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb23effd640 1 -- 172.21.15.5:0/3250827641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd59653c0 con 0x556cd5c99370 2024-09-15T22:14:10.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb23effd640 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c96640 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb230009900 tx=0x7fb23002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2b2488ec2b5d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb226ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb226ffd640 1 -- 172.21.15.5:0/3250827641 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230035030 con 0x556cd5c99370 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb226ffd640 1 -- 172.21.15.5:0/3250827641 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fb230004af0 con 0x556cd5c99370 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb250052740 1 -- 172.21.15.5:0/3250827641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb230005050 con 0x556cd5c99370 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb226ffd640 1 -- 172.21.15.5:0/3250827641 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb230004cf0 con 0x556cd5c99370 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb250052740 10 client.16446 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.265+0000 7fb250052740 1 -- 172.21.15.5:0/3250827641 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556cd5ca7b10 con 0x556cd5c99370 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.266+0000 7fb226ffd640 1 -- 172.21.15.5:0/3250827641 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb230005050 con 0x556cd5c99370 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.266+0000 7fb226ffd640 1 -- 172.21.15.5:0/3250827641 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb23004a040 con 0x556cd5c99370 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.266+0000 7fb250052740 10 client.16446 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.266+0000 7fb250052740 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.266+0000 7fb250052740 20 client.16446 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.266+0000 7fb250052740 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556cd5ca8e60 0x556cd5cab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.266+0000 7fb250052740 4 client.16446 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.266+0000 7fb250052740 1 -- 172.21.15.5:0/3250827641 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cd5c01860 con 0x556cd5ca8e60 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.266+0000 7fb2477fe640 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556cd5ca8e60 0x556cd5cab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.267+0000 7fb226ffd640 10 client.16446 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.267+0000 7fb2477fe640 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556cd5ca8e60 0x556cd5cab250 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14bb561a5b36ca14 server_cookie=db8caf3774577e3c in_seq=0 out_seq=0 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.267+0000 7fb226ffd640 1 -- 172.21.15.5:0/3250827641 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cd5c01860 con 0x556cd5ca8e60 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.267+0000 7fb226ffd640 10 client.16446 handle_command_reply: tid=0 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.267+0000 7fb250052740 1 client.16446 shutdown 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb250052740 1 -- 172.21.15.5:0/3250827641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 msgr2=0x556cd5c96640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb250052740 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c96640 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb230009900 tx=0x7fb23002a480 comp rx=0 tx=0).stop 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb250052740 1 -- 172.21.15.5:0/3250827641 shutdown_connections 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb250052740 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c96640 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb250052740 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556cd5ca8e60 0x556cd5cab250 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb250052740 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8fb40 0x556cd5c96030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb250052740 1 --2- 172.21.15.5:0/3250827641 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5c98d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb226ffd640 0 client.16446 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb250052740 1 -- 172.21.15.5:0/3250827641 >> 172.21.15.5:0/3250827641 conn(0x556cd5c57310 msgr2=0x556cd5c90960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb250052740 1 -- 172.21.15.5:0/3250827641 shutdown_connections 2024-09-15T22:14:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb250052740 1 -- 172.21.15.5:0/3250827641 wait complete. 2024-09-15T22:14:10.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.268+0000 7fb250052740 20 client.16446 trim_cache size 0 max 16384 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb250052740 1 Processor -- start 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb250052740 1 -- start start 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb250052740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8e5d0 0x556cd5ca5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb250052740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8fb40 0x556cd5c8ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb250052740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c958f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb250052740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481b1840 con 0x556cd5c99370 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb250052740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481af110 con 0x556cd5c8e5d0 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb250052740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481acf60 con 0x556cd5c8fb40 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb23effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c958f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb23effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c958f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52656/0 (socket says 172.21.15.5:52656) 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb23effd640 1 -- 172.21.15.5:0/3890185620 learned_addr learned my addr 172.21.15.5:0/3890185620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb23effd640 1 --2- 172.21.15.5:0/3890185620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8e5d0 0x556cd5ca5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb23effd640 1 -- 172.21.15.5:0/3890185620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8fb40 msgr2=0x556cd5c8ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb23effd640 1 --2- 172.21.15.5:0/3890185620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8fb40 0x556cd5c8ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb23effd640 1 -- 172.21.15.5:0/3890185620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8e5d0 msgr2=0x556cd5ca5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.281+0000 7fb23effd640 1 --2- 172.21.15.5:0/3890185620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8e5d0 0x556cd5ca5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.282+0000 7fb23effd640 1 -- 172.21.15.5:0/3890185620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cd5c95020 con 0x556cd5c99370 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.282+0000 7fb23effd640 1 --2- 172.21.15.5:0/3890185620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c958f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb2300318d0 tx=0x7fb2300056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29f8bcfe6aebc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.282+0000 7fb23d7fa640 1 -- 172.21.15.5:0/3890185620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb230035070 con 0x556cd5c99370 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.282+0000 7fb23d7fa640 1 -- 172.21.15.5:0/3890185620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb230032e60 con 0x556cd5c99370 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.282+0000 7fb23d7fa640 1 -- 172.21.15.5:0/3890185620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2300048d0 con 0x556cd5c99370 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.283+0000 7fb250052740 1 -- 172.21.15.5:0/3890185620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 msgr2=0x556cd5c958f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.283+0000 7fb250052740 1 --2- 172.21.15.5:0/3890185620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c958f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb2300318d0 tx=0x7fb2300056e0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.283+0000 7fb250052740 1 -- 172.21.15.5:0/3890185620 shutdown_connections 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.283+0000 7fb250052740 1 --2- 172.21.15.5:0/3890185620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c99370 0x556cd5c958f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.283+0000 7fb250052740 1 --2- 172.21.15.5:0/3890185620 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8fb40 0x556cd5c8ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.283+0000 7fb250052740 1 --2- 172.21.15.5:0/3890185620 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c8e5d0 0x556cd5ca5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.283+0000 7fb250052740 1 -- 172.21.15.5:0/3890185620 >> 172.21.15.5:0/3890185620 conn(0x556cd5c57310 msgr2=0x556cd5c8a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.283+0000 7fb250052740 1 -- 172.21.15.5:0/3890185620 shutdown_connections 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.283+0000 7fb250052740 1 -- 172.21.15.5:0/3890185620 wait complete. 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.283+0000 7fb250052740 1 -- start start 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb250052740 1 Processor -- start 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb250052740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5cae760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb250052740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c8fb40 0x556cd5cabb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb250052740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c99370 0x556cd5cac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb250052740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481ad5d0 con 0x556cd5c8fb40 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb250052740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481aea40 con 0x556cd5c99370 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb250052740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481adca0 con 0x556cd5c8e5d0 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb250052740 10 client.0 _ll_register_callbacks cb 0x7fb230009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:10.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb2477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c8fb40 0x556cd5cabb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb2477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c8fb40 0x556cd5cabb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52672/0 (socket says 172.21.15.5:52672) 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb2477fe640 1 -- 172.21.15.5:0/125791125 learned_addr learned my addr 172.21.15.5:0/125791125 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb23effd640 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c99370 0x556cd5cac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb23effd640 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5cae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb2477fe640 1 -- 172.21.15.5:0/125791125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 msgr2=0x556cd5cae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb2477fe640 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5cae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb2477fe640 1 -- 172.21.15.5:0/125791125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c99370 msgr2=0x556cd5cac090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb2477fe640 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c99370 0x556cd5cac090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb2477fe640 1 -- 172.21.15.5:0/125791125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cd5cac650 con 0x556cd5c8fb40 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.284+0000 7fb23effd640 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c99370 0x556cd5cac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb2477fe640 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c8fb40 0x556cd5cabb50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x556cd5c958c0 tx=0x7fb238012b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6da5a20173f3600 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb226ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:10.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb226ffd640 1 -- 172.21.15.5:0/125791125 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238021070 con 0x556cd5c8fb40 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb226ffd640 1 -- 172.21.15.5:0/125791125 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fb23800fa10 con 0x556cd5c8fb40 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb250052740 1 -- 172.21.15.5:0/125791125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb230032e60 con 0x556cd5c8fb40 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb226ffd640 1 -- 172.21.15.5:0/125791125 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb23800fc10 con 0x556cd5c8fb40 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb250052740 10 client.16452 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb250052740 1 -- 172.21.15.5:0/125791125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556cd5d135c0 con 0x556cd5c8fb40 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb226ffd640 1 -- 172.21.15.5:0/125791125 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb230032e60 con 0x556cd5c8fb40 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb226ffd640 1 -- 172.21.15.5:0/125791125 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb238024020 con 0x556cd5c8fb40 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb250052740 10 client.16452 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb250052740 10 client.16452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb250052740 20 client.16452 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb250052740 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556cd5d13dc0 0x556cd5d161e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:10.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb250052740 4 client.16452 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.285+0000 7fb250052740 1 -- 172.21.15.5:0/125791125 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cd5bfbd70 con 0x556cd5d13dc0 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.286+0000 7fb23effd640 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556cd5d13dc0 0x556cd5d161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.286+0000 7fb23effd640 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556cd5d13dc0 0x556cd5d161e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e04c6003ebdeed server_cookie=fa213f4b5103c108 in_seq=0 out_seq=0 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.286+0000 7fb226ffd640 10 client.16452 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.286+0000 7fb226ffd640 1 -- 172.21.15.5:0/125791125 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cd5bfbd70 con 0x556cd5d13dc0 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.286+0000 7fb226ffd640 10 client.16452 handle_command_reply: tid=0 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.286+0000 7fb250052740 1 client.16452 shutdown 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.286+0000 7fb250052740 1 -- 172.21.15.5:0/125791125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c8fb40 msgr2=0x556cd5cabb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.286+0000 7fb250052740 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c8fb40 0x556cd5cabb50 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x556cd5c958c0 tx=0x7fb238012b10 comp rx=0 tx=0).stop 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb250052740 1 -- 172.21.15.5:0/125791125 shutdown_connections 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb250052740 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556cd5d13dc0 0x556cd5d161e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb250052740 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556cd5c99370 0x556cd5cac090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb250052740 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cd5c8fb40 0x556cd5cabb50 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb250052740 1 --2- 172.21.15.5:0/125791125 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556cd5c8e5d0 0x556cd5cae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb250052740 1 -- 172.21.15.5:0/125791125 >> 172.21.15.5:0/125791125 conn(0x556cd5c57310 msgr2=0x556cd5c97890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:10.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb226ffd640 0 client.16452 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:10.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb250052740 1 -- 172.21.15.5:0/125791125 shutdown_connections 2024-09-15T22:14:10.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb250052740 1 -- 172.21.15.5:0/125791125 wait complete. 2024-09-15T22:14:10.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb250052740 20 client.16452 trim_cache size 0 max 16384 2024-09-15T22:14:10.306 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:14:10.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb24dc79640 1 -- 172.21.15.5:0/394405978 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb2340a3430 msgr2=0x7fb2340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:10.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb24dc79640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb2340a3430 0x7fb2340a58f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb24001b000 tx=0x7fb24006e3b0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb24dc79640 1 -- 172.21.15.5:0/394405978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24810c170 msgr2=0x7fb24819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:10.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb24dc79640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24810c170 0x7fb24819cc40 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb24000cce0 tx=0x7fb24000a620 comp rx=0 tx=0).stop 2024-09-15T22:14:10.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb24dc79640 1 -- 172.21.15.5:0/394405978 shutdown_connections 2024-09-15T22:14:10.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb24dc79640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb2340a3430 0x7fb2340a58f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb24dc79640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb24810c170 0x7fb24819cc40 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb24dc79640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb248103660 0x7fb2481988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb24dc79640 1 --2- 172.21.15.5:0/394405978 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb248102d40 0x7fb248198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:10.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.287+0000 7fb24dc79640 1 -- 172.21.15.5:0/394405978 >> 172.21.15.5:0/394405978 conn(0x7fb2480fc7a0 msgr2=0x7fb2480fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:10.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.288+0000 7fb24dc79640 1 -- 172.21.15.5:0/394405978 shutdown_connections 2024-09-15T22:14:10.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:10.288+0000 7fb24dc79640 1 -- 172.21.15.5:0/394405978 wait complete. 2024-09-15T22:14:10.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:10 smithi005 ceph-mon[30944]: pgmap v161: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-09-15T22:14:11.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:10 smithi099 ceph-mon[34892]: pgmap v161: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-09-15T22:14:11.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:10 smithi129 ceph-mon[35314]: pgmap v161: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-09-15T22:14:11.995 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:14:11.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:14:12.272 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.770+0000 7f2a46a2b640 1 -- 172.21.15.5:0/3884812719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4010c170 msgr2=0x7f2a4010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.770+0000 7f2a46a2b640 1 --2- 172.21.15.5:0/3884812719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4010c170 0x7f2a4010e610 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009ca0 tx=0x7f2a3802a950 comp rx=0 tx=0).stop 2024-09-15T22:14:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.770+0000 7f2a46a2b640 1 -- 172.21.15.5:0/3884812719 shutdown_connections 2024-09-15T22:14:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.770+0000 7f2a46a2b640 1 --2- 172.21.15.5:0/3884812719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4010c170 0x7f2a4010e610 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.770+0000 7f2a46a2b640 1 --2- 172.21.15.5:0/3884812719 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a40103660 0x7f2a4010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.770+0000 7f2a46a2b640 1 --2- 172.21.15.5:0/3884812719 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a40102d40 0x7f2a40103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.770+0000 7f2a46a2b640 1 -- 172.21.15.5:0/3884812719 >> 172.21.15.5:0/3884812719 conn(0x7f2a400fc7a0 msgr2=0x7f2a400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.771+0000 7f2a46a2b640 1 -- 172.21.15.5:0/3884812719 shutdown_connections 2024-09-15T22:14:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.771+0000 7f2a46a2b640 1 -- 172.21.15.5:0/3884812719 wait complete. 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.771+0000 7f2a46a2b640 1 Processor -- start 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.771+0000 7f2a46a2b640 1 -- start start 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.771+0000 7f2a46a2b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a40102d40 0x7f2a401009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.771+0000 7f2a46a2b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a40103660 0x7f2a40100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.771+0000 7f2a46a2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4010c170 0x7f2a40101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a46a2b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a40111f50 con 0x7f2a4010c170 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a46a2b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a40111dd0 con 0x7f2a40103660 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a46a2b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401120d0 con 0x7f2a40102d40 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a44fc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4010c170 0x7f2a40101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a44fc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4010c170 0x7f2a40101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52686/0 (socket says 172.21.15.5:52686) 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a44fc8640 1 -- 172.21.15.5:0/936124569 learned_addr learned my addr 172.21.15.5:0/936124569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a40102d40 0x7f2a401009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a40103660 0x7f2a40100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a44fc8640 1 -- 172.21.15.5:0/936124569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a40102d40 msgr2=0x7f2a401009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a44fc8640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a40102d40 0x7f2a401009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a44fc8640 1 -- 172.21.15.5:0/936124569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a40103660 msgr2=0x7f2a40100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a44fc8640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a40103660 0x7f2a40100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a44fc8640 1 -- 172.21.15.5:0/936124569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a401a6920 con 0x7f2a4010c170 2024-09-15T22:14:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a40102d40 0x7f2a401009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a44fc8640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4010c170 0x7f2a40101420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2a38004af0 tx=0x7f2a3802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a3d7fa640 1 -- 172.21.15.5:0/936124569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38034040 con 0x7f2a4010c170 2024-09-15T22:14:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a3d7fa640 1 -- 172.21.15.5:0/936124569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a38030920 con 0x7f2a4010c170 2024-09-15T22:14:12.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a46a2b640 1 -- 172.21.15.5:0/936124569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a401a6bb0 con 0x7f2a4010c170 2024-09-15T22:14:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a3d7fa640 1 -- 172.21.15.5:0/936124569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380047b0 con 0x7f2a4010c170 2024-09-15T22:14:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.772+0000 7f2a46a2b640 1 -- 172.21.15.5:0/936124569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a401a7090 con 0x7f2a4010c170 2024-09-15T22:14:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.775+0000 7f2a3d7fa640 1 -- 172.21.15.5:0/936124569 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2a38037020 con 0x7f2a4010c170 2024-09-15T22:14:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.775+0000 7f2a48e04740 1 Processor -- start 2024-09-15T22:14:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.775+0000 7f2a48e04740 1 -- start start 2024-09-15T22:14:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.775+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a466db60 0x5587a466df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.775+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466e480 0x5587a4678800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.775+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a467b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.775+0000 7f2a48e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5587a45d3280 con 0x5587a4678d40 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.775+0000 7f2a48e04740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5587a457ab10 con 0x5587a466db60 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.775+0000 7f2a48e04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5587a45cf240 con 0x5587a466e480 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.775+0000 7f2a3d7fa640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a280a33b0 0x7f2a280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.775+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a467b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a467b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52694/0 (socket says 172.21.15.5:52694) 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a3f7fe640 1 -- 172.21.15.5:0/2002010936 learned_addr learned my addr 172.21.15.5:0/2002010936 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a44fc8640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a280a33b0 0x7f2a280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/2002010936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466e480 0x5587a4678800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/2002010936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a466db60 0x5587a466df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a3d7fa640 1 -- 172.21.15.5:0/936124569 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2a380f0e90 con 0x7f2a4010c170 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a3f7fe640 1 -- 172.21.15.5:0/2002010936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466e480 msgr2=0x5587a4678800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/2002010936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466e480 0x5587a4678800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a3f7fe640 1 -- 172.21.15.5:0/2002010936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a466db60 msgr2=0x5587a466df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/2002010936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a466db60 0x5587a466df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a3f7fe640 1 -- 172.21.15.5:0/2002010936 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a45871f0 con 0x5587a4678d40 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/2002010936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a467b130 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2a34012520 tx=0x7f2a34009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13290560879c44b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a44fc8640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a280a33b0 0x7f2a280a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a380ee400 tx=0x7f2a3802a950 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a267fc640 1 -- 172.21.15.5:0/2002010936 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a34010690 con 0x5587a4678d40 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a267fc640 1 -- 172.21.15.5:0/2002010936 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a34010d10 con 0x5587a4678d40 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.776+0000 7f2a267fc640 1 -- 172.21.15.5:0/2002010936 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3401ee30 con 0x5587a4678d40 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.777+0000 7f2a48e04740 1 -- 172.21.15.5:0/2002010936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 msgr2=0x5587a467b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.777+0000 7f2a48e04740 1 --2- 172.21.15.5:0/2002010936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a467b130 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2a34012520 tx=0x7f2a34009720 comp rx=0 tx=0).stop 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.777+0000 7f2a48e04740 1 -- 172.21.15.5:0/2002010936 shutdown_connections 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.777+0000 7f2a48e04740 1 --2- 172.21.15.5:0/2002010936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a467b130 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.777+0000 7f2a48e04740 1 --2- 172.21.15.5:0/2002010936 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466e480 0x5587a4678800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.777+0000 7f2a48e04740 1 --2- 172.21.15.5:0/2002010936 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a466db60 0x5587a466df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.777+0000 7f2a48e04740 1 -- 172.21.15.5:0/2002010936 >> 172.21.15.5:0/2002010936 conn(0x5587a466a0f0 msgr2=0x5587a466a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.777+0000 7f2a48e04740 1 -- 172.21.15.5:0/2002010936 shutdown_connections 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.777+0000 7f2a48e04740 1 -- 172.21.15.5:0/2002010936 wait complete. 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.777+0000 7f2a48e04740 1 -- start start 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a48e04740 1 Processor -- start 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466db60 0x5587a4675cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a466e480 0x5587a46761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4678d40 0x5587a4676730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a48e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5587a467b910 con 0x5587a466e480 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a48e04740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5587a458df10 con 0x5587a4678d40 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a48e04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5587a467ba90 con 0x5587a466db60 2024-09-15T22:14:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a48e04740 10 client.0 _ll_register_callbacks cb 0x5587a4556a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a466e480 0x5587a46761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a466e480 0x5587a46761f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52698/0 (socket says 172.21.15.5:52698) 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3770520732 learned_addr learned my addr 172.21.15.5:0/3770520732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466db60 0x5587a4675cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3770520732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466db60 msgr2=0x5587a4675cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466db60 0x5587a4675cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4678d40 0x5587a4676730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3770520732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4678d40 msgr2=0x5587a4676730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.778+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4678d40 0x5587a4676730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3770520732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a43446a0 con 0x5587a466e480 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4678d40 0x5587a4676730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a466e480 0x5587a46761f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000bb70 tx=0x7f2a300039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d928aafe95955d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3770520732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30005220 con 0x5587a466e480 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3770520732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2a300053c0 con 0x5587a466e480 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a48e04740 1 -- 172.21.15.5:0/3770520732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a34010d10 con 0x5587a466e480 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3770520732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a30023050 con 0x5587a466e480 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a48e04740 10 client.16464 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a48e04740 1 -- 172.21.15.5:0/3770520732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5587a4687630 con 0x5587a466e480 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3770520732 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a34010d10 con 0x5587a466e480 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3770520732 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2a30019030 con 0x5587a466e480 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a48e04740 10 client.16464 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a48e04740 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.779+0000 7f2a48e04740 20 client.16464 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.780+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587a4688b40 0x5587a468af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.780+0000 7f2a48e04740 4 client.16464 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.780+0000 7f2a48e04740 1 -- 172.21.15.5:0/3770520732 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587a45e07f0 con 0x5587a4688b40 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.780+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587a4688b40 0x5587a468af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.780+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587a4688b40 0x5587a468af80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d433fdc071e4454d server_cookie=bc9ff46ee05443c2 in_seq=0 out_seq=0 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.780+0000 7f2a1f7fe640 10 client.16464 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3770520732 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587a45e07f0 con 0x5587a4688b40 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a1f7fe640 10 client.16464 handle_command_reply: tid=0 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a48e04740 1 client.16464 shutdown 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a48e04740 1 -- 172.21.15.5:0/3770520732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a466e480 msgr2=0x5587a46761f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a466e480 0x5587a46761f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000bb70 tx=0x7f2a300039c0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a48e04740 1 -- 172.21.15.5:0/3770520732 shutdown_connections 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587a4688b40 0x5587a468af80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4678d40 0x5587a4676730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a466e480 0x5587a46761f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3770520732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466db60 0x5587a4675cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a48e04740 1 -- 172.21.15.5:0/3770520732 >> 172.21.15.5:0/3770520732 conn(0x5587a466a0f0 msgr2=0x5587a466fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:12.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a1f7fe640 0 client.16464 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:12.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a48e04740 1 -- 172.21.15.5:0/3770520732 shutdown_connections 2024-09-15T22:14:12.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.781+0000 7f2a48e04740 1 -- 172.21.15.5:0/3770520732 wait complete. 2024-09-15T22:14:12.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.782+0000 7f2a48e04740 20 client.16464 trim_cache size 0 max 16384 2024-09-15T22:14:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a48e04740 1 Processor -- start 2024-09-15T22:14:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a48e04740 1 -- start start 2024-09-15T22:14:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466db60 0x5587a466fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a4690850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4690d90 0x5587a4672f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a48e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401b3af0 con 0x5587a4678d40 2024-09-15T22:14:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a48e04740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401b13c0 con 0x5587a4690d90 2024-09-15T22:14:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a48e04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401af210 con 0x5587a466db60 2024-09-15T22:14:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a4690850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a4690850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52700/0 (socket says 172.21.15.5:52700) 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3223523192 learned_addr learned my addr 172.21.15.5:0/3223523192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3223523192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466db60 0x5587a466fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3223523192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466db60 msgr2=0x5587a466fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/3223523192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466db60 0x5587a466fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3223523192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4690d90 0x5587a4672f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3223523192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4690d90 msgr2=0x5587a4672f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/3223523192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4690d90 0x5587a4672f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3223523192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a468b600 con 0x5587a4678d40 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.799+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3223523192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4690d90 0x5587a4672f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.800+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/3223523192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a4690850 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2a30000c00 tx=0x7f2a3000bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfff38d539ded269 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.800+0000 7f2a25ffb640 1 -- 172.21.15.5:0/3223523192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3002b020 con 0x5587a4678d40 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.800+0000 7f2a25ffb640 1 -- 172.21.15.5:0/3223523192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a30003f00 con 0x5587a4678d40 2024-09-15T22:14:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.800+0000 7f2a25ffb640 1 -- 172.21.15.5:0/3223523192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a300055f0 con 0x5587a4678d40 2024-09-15T22:14:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.800+0000 7f2a48e04740 1 -- 172.21.15.5:0/3223523192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 msgr2=0x5587a4690850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.800+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3223523192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a4690850 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f2a30000c00 tx=0x7f2a3000bf30 comp rx=0 tx=0).stop 2024-09-15T22:14:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.801+0000 7f2a48e04740 1 -- 172.21.15.5:0/3223523192 shutdown_connections 2024-09-15T22:14:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.801+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3223523192 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a4690d90 0x5587a4672f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.801+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3223523192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4678d40 0x5587a4690850 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.801+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3223523192 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a466db60 0x5587a466fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.801+0000 7f2a48e04740 1 -- 172.21.15.5:0/3223523192 >> 172.21.15.5:0/3223523192 conn(0x5587a466a0f0 msgr2=0x5587a466f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.801+0000 7f2a48e04740 1 -- 172.21.15.5:0/3223523192 shutdown_connections 2024-09-15T22:14:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.801+0000 7f2a48e04740 1 -- 172.21.15.5:0/3223523192 wait complete. 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.801+0000 7f2a48e04740 1 -- start start 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.801+0000 7f2a48e04740 1 Processor -- start 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a466db60 0x5587a4681540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a4678d40 0x5587a467dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a48e04740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4690d90 0x5587a467e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a48e04740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401af880 con 0x5587a4690d90 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a48e04740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401b0cf0 con 0x5587a466db60 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a48e04740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401aff50 con 0x5587a4678d40 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a48e04740 10 client.0 _ll_register_callbacks cb 0x7f2a300043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a4678d40 0x5587a467dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a3ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a4678d40 0x5587a467dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58778/0 (socket says 172.21.15.5:58778) 2024-09-15T22:14:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3746046822 learned_addr learned my addr 172.21.15.5:0/3746046822 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4690d90 0x5587a467e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a466db60 0x5587a4681540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3746046822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a466db60 msgr2=0x5587a4681540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a466db60 0x5587a4681540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3746046822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4690d90 msgr2=0x5587a467e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4690d90 0x5587a467e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a3ffff640 1 -- 172.21.15.5:0/3746046822 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a467eb00 con 0x5587a4678d40 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.802+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4690d90 0x5587a467e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.803+0000 7f2a3ffff640 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a4678d40 0x5587a467dce0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a30006010 tx=0x7f2a30011c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30750638e403ac7b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.803+0000 7f2a1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.803+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3746046822 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30034020 con 0x5587a4678d40 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.803+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3746046822 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2a30005250 con 0x5587a4678d40 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.803+0000 7f2a48e04740 1 -- 172.21.15.5:0/3746046822 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a30003f00 con 0x5587a4678d40 2024-09-15T22:14:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.803+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3746046822 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a30011e20 con 0x5587a4678d40 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.803+0000 7f2a48e04740 10 client.25999 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.803+0000 7f2a48e04740 1 -- 172.21.15.5:0/3746046822 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5587a46f48c0 con 0x5587a4678d40 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.803+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3746046822 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a30003f00 con 0x5587a4678d40 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.804+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3746046822 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2a30019070 con 0x5587a4678d40 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.804+0000 7f2a48e04740 10 client.25999 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.804+0000 7f2a48e04740 10 client.25999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.804+0000 7f2a48e04740 20 client.25999 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.804+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587a46f4f50 0x5587a46f7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.804+0000 7f2a48e04740 4 client.25999 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.804+0000 7f2a48e04740 1 -- 172.21.15.5:0/3746046822 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5587a45dad00 con 0x5587a46f4f50 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.804+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587a46f4f50 0x5587a46f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.804+0000 7f2a3f7fe640 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587a46f4f50 0x5587a46f7370 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a051d01235c41f server_cookie=c88f1c3234cf1aaa in_seq=0 out_seq=0 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.804+0000 7f2a1f7fe640 10 client.25999 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.805+0000 7f2a1f7fe640 1 -- 172.21.15.5:0/3746046822 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5587a45dad00 con 0x5587a46f4f50 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.805+0000 7f2a1f7fe640 10 client.25999 handle_command_reply: tid=0 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.805+0000 7f2a48e04740 1 client.25999 shutdown 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.805+0000 7f2a48e04740 1 -- 172.21.15.5:0/3746046822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a4678d40 msgr2=0x5587a467dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.805+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a4678d40 0x5587a467dce0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2a30006010 tx=0x7f2a30011c30 comp rx=0 tx=0).stop 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.805+0000 7f2a48e04740 1 -- 172.21.15.5:0/3746046822 shutdown_connections 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.805+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a4690d90 0x5587a467e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.805+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5587a4678d40 0x5587a467dce0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.805+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587a46f4f50 0x5587a46f7370 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.805+0000 7f2a48e04740 1 --2- 172.21.15.5:0/3746046822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5587a466db60 0x5587a4681540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.805+0000 7f2a48e04740 1 -- 172.21.15.5:0/3746046822 >> 172.21.15.5:0/3746046822 conn(0x5587a466a0f0 msgr2=0x5587a4674cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.806+0000 7f2a1f7fe640 0 client.25999 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.806+0000 7f2a48e04740 1 -- 172.21.15.5:0/3746046822 shutdown_connections 2024-09-15T22:14:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.806+0000 7f2a48e04740 1 -- 172.21.15.5:0/3746046822 wait complete. 2024-09-15T22:14:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.806+0000 7f2a48e04740 20 client.25999 trim_cache size 0 max 16384 2024-09-15T22:14:12.814 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:14:12.814 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:14:12.814 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "3a4af246-8746-4f34-9581-975ea07df83a", 2024-09-15T22:14:12.814 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:14:12.814 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:14:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.806+0000 7f2a46a2b640 1 -- 172.21.15.5:0/936124569 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a280a33b0 msgr2=0x7f2a280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.806+0000 7f2a46a2b640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a280a33b0 0x7f2a280a5870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2a380ee400 tx=0x7f2a3802a950 comp rx=0 tx=0).stop 2024-09-15T22:14:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.806+0000 7f2a46a2b640 1 -- 172.21.15.5:0/936124569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4010c170 msgr2=0x7f2a40101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.806+0000 7f2a46a2b640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4010c170 0x7f2a40101420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2a38004af0 tx=0x7f2a3802f5b0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.807+0000 7f2a46a2b640 1 -- 172.21.15.5:0/936124569 shutdown_connections 2024-09-15T22:14:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.807+0000 7f2a46a2b640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2a280a33b0 0x7f2a280a5870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.807+0000 7f2a46a2b640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a4010c170 0x7f2a40101420 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.807+0000 7f2a46a2b640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a40103660 0x7f2a40100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.807+0000 7f2a46a2b640 1 --2- 172.21.15.5:0/936124569 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a40102d40 0x7f2a401009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:12.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.807+0000 7f2a46a2b640 1 -- 172.21.15.5:0/936124569 >> 172.21.15.5:0/936124569 conn(0x7f2a400fc7a0 msgr2=0x7f2a400fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:12.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.807+0000 7f2a46a2b640 1 -- 172.21.15.5:0/936124569 shutdown_connections 2024-09-15T22:14:12.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:12.807+0000 7f2a46a2b640 1 -- 172.21.15.5:0/936124569 wait complete. 2024-09-15T22:14:13.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:12 smithi099 ceph-mon[34892]: pgmap v162: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-09-15T22:14:13.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:12 smithi129 ceph-mon[35314]: pgmap v162: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-09-15T22:14:13.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:12 smithi005 ceph-mon[30944]: pgmap v162: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-09-15T22:14:13.874 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:14:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:13 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:14:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:13 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:13 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:14:14.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:13 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:13 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:14:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:13 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:13 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:14:14.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:13 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:14.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:13 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:14:14.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:13 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:14.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:13 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:14:14.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:13 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:14:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 285.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:14:14.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.735+0000 7f5044a38640 1 -- 172.21.15.5:0/1277792906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5040101420 msgr2=0x7f504010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.735+0000 7f5044a38640 1 --2- 172.21.15.5:0/1277792906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5040101420 0x7f504010d950 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5038009cd0 tx=0x7f503802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.735+0000 7f5044a38640 1 -- 172.21.15.5:0/1277792906 shutdown_connections 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.735+0000 7f5044a38640 1 --2- 172.21.15.5:0/1277792906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5040101420 0x7f504010d950 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.735+0000 7f5044a38640 1 --2- 172.21.15.5:0/1277792906 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5040100a80 0x7f5040100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.735+0000 7f5044a38640 1 --2- 172.21.15.5:0/1277792906 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5040106a80 0x7f5040106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.735+0000 7f5044a38640 1 -- 172.21.15.5:0/1277792906 >> 172.21.15.5:0/1277792906 conn(0x7f50400fc7a0 msgr2=0x7f50400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.735+0000 7f5044a38640 1 -- 172.21.15.5:0/1277792906 shutdown_connections 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.735+0000 7f5044a38640 1 -- 172.21.15.5:0/1277792906 wait complete. 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f5044a38640 1 Processor -- start 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f5044a38640 1 -- start start 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f5044a38640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5040100a80 0x7f5040102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f5044a38640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5040101420 0x7f5040102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f5044a38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5040106a80 0x7f50401030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f5044a38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5040110e80 con 0x7f5040106a80 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f5044a38640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5040110d00 con 0x7f5040101420 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f5044a38640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5040111000 con 0x7f5040100a80 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f503ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5040106a80 0x7f50401030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f503ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5040106a80 0x7f50401030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57140/0 (socket says 172.21.15.5:57140) 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f503ed9d640 1 -- 172.21.15.5:0/4038567594 learned_addr learned my addr 172.21.15.5:0/4038567594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.736+0000 7f503e59c640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5040100a80 0x7f5040102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f5035d9b640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5040101420 0x7f5040102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f503ed9d640 1 -- 172.21.15.5:0/4038567594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5040100a80 msgr2=0x7f5040102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f503ed9d640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5040100a80 0x7f5040102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f503ed9d640 1 -- 172.21.15.5:0/4038567594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5040101420 msgr2=0x7f5040102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:14.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f503ed9d640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5040101420 0x7f5040102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f503ed9d640 1 -- 172.21.15.5:0/4038567594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50401a68d0 con 0x7f5040106a80 2024-09-15T22:14:14.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f5035d9b640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5040101420 0x7f5040102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:14.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f503ed9d640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5040106a80 0x7f50401030d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5038005cc0 tx=0x7f5038003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:14.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f5037fff640 1 -- 172.21.15.5:0/4038567594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038004040 con 0x7f5040106a80 2024-09-15T22:14:14.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f5037fff640 1 -- 172.21.15.5:0/4038567594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50380046c0 con 0x7f5040106a80 2024-09-15T22:14:14.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f5044a38640 1 -- 172.21.15.5:0/4038567594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50401a6b60 con 0x7f5040106a80 2024-09-15T22:14:14.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f5037fff640 1 -- 172.21.15.5:0/4038567594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5038046480 con 0x7f5040106a80 2024-09-15T22:14:14.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.737+0000 7f5044a38640 1 -- 172.21.15.5:0/4038567594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50401a70a0 con 0x7f5040106a80 2024-09-15T22:14:14.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.740+0000 7f5037fff640 1 -- 172.21.15.5:0/4038567594 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f50380305a0 con 0x7f5040106a80 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.740+0000 7f5046e11740 1 Processor -- start 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.740+0000 7f5046e11740 1 -- start start 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.740+0000 7f5046e11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa7867b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.740+0000 7f5046e11740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867bdc0 0x55fa78686140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.740+0000 7f5046e11740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa78686680 0x55fa78688a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.740+0000 7f5046e11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa785e12e0 con 0x55fa7867b4a0 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.740+0000 7f5046e11740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fa78588ce0 con 0x55fa78686680 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.740+0000 7f5046e11740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fa785dd2a0 con 0x55fa7867bdc0 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5035d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa7867b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5037fff640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f502c0a33b0 0x7f502c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5035d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa7867b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57156/0 (socket says 172.21.15.5:57156) 2024-09-15T22:14:14.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5035d9b640 1 -- 172.21.15.5:0/1620764587 learned_addr learned my addr 172.21.15.5:0/1620764587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f503e59c640 1 --2- 172.21.15.5:0/1620764587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867bdc0 0x55fa78686140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f503ed9d640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f502c0a33b0 0x7f502c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5035d9b640 1 --2- 172.21.15.5:0/1620764587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa78686680 0x55fa78688a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5035d9b640 1 -- 172.21.15.5:0/1620764587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867bdc0 msgr2=0x55fa78686140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5035d9b640 1 --2- 172.21.15.5:0/1620764587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867bdc0 0x55fa78686140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5037fff640 1 -- 172.21.15.5:0/4038567594 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f503802e030 con 0x7f5040106a80 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5035d9b640 1 -- 172.21.15.5:0/1620764587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa78686680 msgr2=0x55fa78688a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5035d9b640 1 --2- 172.21.15.5:0/1620764587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa78686680 0x55fa78688a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5035d9b640 1 -- 172.21.15.5:0/1620764587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7856d440 con 0x55fa7867b4a0 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f503e59c640 1 --2- 172.21.15.5:0/1620764587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867bdc0 0x55fa78686140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5035d9b640 1 --2- 172.21.15.5:0/1620764587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa7867b880 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f50280058c0 tx=0x7f502800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3d6a25c115425a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f503ed9d640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f502c0a33b0 0x7f502c0a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50380ef180 tx=0x7f503809e040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:14.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5034d99640 1 -- 172.21.15.5:0/1620764587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028019030 con 0x55fa7867b4a0 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5034d99640 1 -- 172.21.15.5:0/1620764587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5028011a20 con 0x55fa7867b4a0 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.741+0000 7f5034d99640 1 -- 172.21.15.5:0/1620764587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502801d950 con 0x55fa7867b4a0 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.742+0000 7f5046e11740 1 -- 172.21.15.5:0/1620764587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 msgr2=0x55fa7867b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.742+0000 7f5046e11740 1 --2- 172.21.15.5:0/1620764587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa7867b880 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f50280058c0 tx=0x7f502800b730 comp rx=0 tx=0).stop 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.742+0000 7f5046e11740 1 -- 172.21.15.5:0/1620764587 shutdown_connections 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.742+0000 7f5046e11740 1 --2- 172.21.15.5:0/1620764587 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa78686680 0x55fa78688a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.742+0000 7f5046e11740 1 --2- 172.21.15.5:0/1620764587 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867bdc0 0x55fa78686140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.742+0000 7f5046e11740 1 --2- 172.21.15.5:0/1620764587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa7867b880 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.742+0000 7f5046e11740 1 -- 172.21.15.5:0/1620764587 >> 172.21.15.5:0/1620764587 conn(0x55fa78677a30 msgr2=0x55fa78677e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.742+0000 7f5046e11740 1 -- 172.21.15.5:0/1620764587 shutdown_connections 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.742+0000 7f5046e11740 1 -- 172.21.15.5:0/1620764587 wait complete. 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5046e11740 1 -- start start 2024-09-15T22:14:14.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5046e11740 1 Processor -- start 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5046e11740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867b4a0 0x55fa78685f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5046e11740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867bdc0 0x55fa78683200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5046e11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa78683740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5046e11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fa78689250 con 0x55fa78686680 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5046e11740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55fa7859c050 con 0x55fa7867b4a0 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5046e11740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55fa786893d0 con 0x55fa7867bdc0 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5046e11740 10 client.0 _ll_register_callbacks cb 0x55fa785dc160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5035d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa78683740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5035d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa78683740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57164/0 (socket says 172.21.15.5:57164) 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5035d9b640 1 -- 172.21.15.5:0/3068331142 learned_addr learned my addr 172.21.15.5:0/3068331142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f5035d9b640 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867b4a0 0x55fa78685f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.743+0000 7f503e59c640 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867bdc0 0x55fa78683200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5035d9b640 1 -- 172.21.15.5:0/3068331142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867bdc0 msgr2=0x55fa78683200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5035d9b640 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867bdc0 0x55fa78683200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5035d9b640 1 -- 172.21.15.5:0/3068331142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867b4a0 msgr2=0x55fa78685f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5035d9b640 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867b4a0 0x55fa78685f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5035d9b640 1 -- 172.21.15.5:0/3068331142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa783523c0 con 0x55fa78686680 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5035d9b640 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa78683740 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f502801d690 tx=0x7f502800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5653485212163cd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5025ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5025ffb640 1 -- 172.21.15.5:0/3068331142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028019040 con 0x55fa78686680 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5025ffb640 1 -- 172.21.15.5:0/3068331142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f5028011540 con 0x55fa78686680 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5046e11740 1 -- 172.21.15.5:0/3068331142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5028011a20 con 0x55fa78686680 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5025ffb640 1 -- 172.21.15.5:0/3068331142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5028033050 con 0x55fa78686680 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5046e11740 10 client.16482 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:14.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5046e11740 1 -- 172.21.15.5:0/3068331142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa78694c00 con 0x55fa78686680 2024-09-15T22:14:14.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5025ffb640 1 -- 172.21.15.5:0/3068331142 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5028011a20 con 0x55fa78686680 2024-09-15T22:14:14.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.744+0000 7f5025ffb640 1 -- 172.21.15.5:0/3068331142 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f502802b020 con 0x55fa78686680 2024-09-15T22:14:14.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.745+0000 7f5046e11740 10 client.16482 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:14.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.745+0000 7f5046e11740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:14.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.745+0000 7f5046e11740 20 client.16482 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:14.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.745+0000 7f5046e11740 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55fa78696060 0x55fa78698450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.745+0000 7f5046e11740 4 client.16482 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:14.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.745+0000 7f5046e11740 1 -- 172.21.15.5:0/3068331142 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa785ee850 con 0x55fa78696060 2024-09-15T22:14:14.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.745+0000 7f503e59c640 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55fa78696060 0x55fa78698450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.745+0000 7f503e59c640 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55fa78696060 0x55fa78698450 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc4c8aff994502ef server_cookie=8b1fff3396d979a3 in_seq=0 out_seq=0 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.745+0000 7f5025ffb640 10 client.16482 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5025ffb640 1 -- 172.21.15.5:0/3068331142 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa785ee850 con 0x55fa78696060 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5025ffb640 10 client.16482 handle_command_reply: tid=0 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5046e11740 1 client.16482 shutdown 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5046e11740 1 -- 172.21.15.5:0/3068331142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 msgr2=0x55fa78683740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5046e11740 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa78683740 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f502801d690 tx=0x7f502800b690 comp rx=0 tx=0).stop 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5046e11740 1 -- 172.21.15.5:0/3068331142 shutdown_connections 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5046e11740 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa78683740 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5046e11740 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867bdc0 0x55fa78683200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5046e11740 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55fa78696060 0x55fa78698450 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5046e11740 1 --2- 172.21.15.5:0/3068331142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867b4a0 0x55fa78685f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5025ffb640 0 client.16482 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5046e11740 1 -- 172.21.15.5:0/3068331142 >> 172.21.15.5:0/3068331142 conn(0x55fa78677a30 msgr2=0x55fa7867d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5046e11740 1 -- 172.21.15.5:0/3068331142 shutdown_connections 2024-09-15T22:14:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.746+0000 7f5046e11740 1 -- 172.21.15.5:0/3068331142 wait complete. 2024-09-15T22:14:14.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.747+0000 7f5046e11740 20 client.16482 trim_cache size 0 max 16384 2024-09-15T22:14:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5046e11740 1 Processor -- start 2024-09-15T22:14:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5046e11740 1 -- start start 2024-09-15T22:14:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5046e11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa78692510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5046e11740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867bdc0 0x55fa7869dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5046e11740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa78686680 0x55fa7869e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5046e11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50401b3a00 con 0x55fa7867b4a0 2024-09-15T22:14:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5046e11740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50401b12d0 con 0x55fa7867bdc0 2024-09-15T22:14:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5046e11740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50401af120 con 0x55fa78686680 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5035d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa78692510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5035d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa78692510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57180/0 (socket says 172.21.15.5:57180) 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5035d9b640 1 -- 172.21.15.5:0/4048325115 learned_addr learned my addr 172.21.15.5:0/4048325115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f503e59c640 1 --2- 172.21.15.5:0/4048325115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867bdc0 0x55fa7869dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5035d9b640 1 -- 172.21.15.5:0/4048325115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa78686680 msgr2=0x55fa7869e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5035d9b640 1 --2- 172.21.15.5:0/4048325115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa78686680 0x55fa7869e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5035d9b640 1 -- 172.21.15.5:0/4048325115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867bdc0 msgr2=0x55fa7869dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5035d9b640 1 --2- 172.21.15.5:0/4048325115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867bdc0 0x55fa7869dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5035d9b640 1 -- 172.21.15.5:0/4048325115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa78677860 con 0x55fa7867b4a0 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.764+0000 7f5035d9b640 1 --2- 172.21.15.5:0/4048325115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa78692510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f50280059f0 tx=0x7f5028005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9eb23a6113a1279 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5027fff640 1 -- 172.21.15.5:0/4048325115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028019040 con 0x55fa7867b4a0 2024-09-15T22:14:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5027fff640 1 -- 172.21.15.5:0/4048325115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5028013470 con 0x55fa7867b4a0 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5027fff640 1 -- 172.21.15.5:0/4048325115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028011540 con 0x55fa7867b4a0 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5046e11740 1 -- 172.21.15.5:0/4048325115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 msgr2=0x55fa78692510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5046e11740 1 --2- 172.21.15.5:0/4048325115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa78692510 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f50280059f0 tx=0x7f5028005bf0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5046e11740 1 -- 172.21.15.5:0/4048325115 shutdown_connections 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5046e11740 1 --2- 172.21.15.5:0/4048325115 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa78686680 0x55fa7869e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5046e11740 1 --2- 172.21.15.5:0/4048325115 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867bdc0 0x55fa7869dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5046e11740 1 --2- 172.21.15.5:0/4048325115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa7867b4a0 0x55fa78692510 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5046e11740 1 -- 172.21.15.5:0/4048325115 >> 172.21.15.5:0/4048325115 conn(0x55fa78677a30 msgr2=0x55fa786822f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5046e11740 1 -- 172.21.15.5:0/4048325115 shutdown_connections 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.765+0000 7f5046e11740 1 -- 172.21.15.5:0/4048325115 wait complete. 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5046e11740 1 -- start start 2024-09-15T22:14:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5046e11740 1 Processor -- start 2024-09-15T22:14:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5046e11740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867b4a0 0x55fa7868bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5046e11740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867bdc0 0x55fa7868c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5046e11740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa786febb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5046e11740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50401af790 con 0x55fa78686680 2024-09-15T22:14:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5046e11740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50401b0c00 con 0x55fa7867bdc0 2024-09-15T22:14:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5046e11740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50401afe60 con 0x55fa7867b4a0 2024-09-15T22:14:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5046e11740 10 client.0 _ll_register_callbacks cb 0x7f5028013620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5035d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa786febb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5035d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa786febb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57192/0 (socket says 172.21.15.5:57192) 2024-09-15T22:14:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5035d9b640 1 -- 172.21.15.5:0/1246791840 learned_addr learned my addr 172.21.15.5:0/1246791840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f503e59c640 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867bdc0 0x55fa7868c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5035d9b640 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867b4a0 0x55fa7868bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5035d9b640 1 -- 172.21.15.5:0/1246791840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867b4a0 msgr2=0x55fa7868bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5035d9b640 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867b4a0 0x55fa7868bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5035d9b640 1 -- 172.21.15.5:0/1246791840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867bdc0 msgr2=0x55fa7868c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5035d9b640 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867bdc0 0x55fa7868c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.766+0000 7f5035d9b640 1 -- 172.21.15.5:0/1246791840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa786ff1f0 con 0x55fa78686680 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5035d9b640 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa786febb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f502801dde0 tx=0x7f502800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa119cec2c7d6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5025ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5025ffb640 1 -- 172.21.15.5:0/1246791840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502803c070 con 0x55fa78686680 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5025ffb640 1 -- 172.21.15.5:0/1246791840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f5028013220 con 0x55fa78686680 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5046e11740 1 -- 172.21.15.5:0/1246791840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5028013470 con 0x55fa78686680 2024-09-15T22:14:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5025ffb640 1 -- 172.21.15.5:0/1246791840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5028019070 con 0x55fa78686680 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5046e11740 10 client.16488 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5046e11740 1 -- 172.21.15.5:0/1246791840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa787004c0 con 0x55fa78686680 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5025ffb640 1 -- 172.21.15.5:0/1246791840 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5028013470 con 0x55fa78686680 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5025ffb640 1 -- 172.21.15.5:0/1246791840 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f502803f020 con 0x55fa78686680 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5046e11740 10 client.16488 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5046e11740 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5046e11740 20 client.16488 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5046e11740 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55fa78700b20 0x55fa78702f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5046e11740 4 client.16488 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.767+0000 7f5046e11740 1 -- 172.21.15.5:0/1246791840 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa785e8d60 con 0x55fa78700b20 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f503e59c640 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55fa78700b20 0x55fa78702f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f503e59c640 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55fa78700b20 0x55fa78702f10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39db7c0f09ee47fb server_cookie=6708eca3d9adc3b1 in_seq=0 out_seq=0 2024-09-15T22:14:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5025ffb640 10 client.16488 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5025ffb640 1 -- 172.21.15.5:0/1246791840 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55fa785e8d60 con 0x55fa78700b20 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5025ffb640 10 client.16488 handle_command_reply: tid=0 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5046e11740 1 client.16488 shutdown 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5046e11740 1 -- 172.21.15.5:0/1246791840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 msgr2=0x55fa786febb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5046e11740 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa786febb0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f502801dde0 tx=0x7f502800b8a0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5046e11740 1 -- 172.21.15.5:0/1246791840 shutdown_connections 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5046e11740 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fa78686680 0x55fa786febb0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5046e11740 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55fa78700b20 0x55fa78702f10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5046e11740 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55fa7867bdc0 0x55fa7868c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5046e11740 1 --2- 172.21.15.5:0/1246791840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55fa7867b4a0 0x55fa7868bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.768+0000 7f5046e11740 1 -- 172.21.15.5:0/1246791840 >> 172.21.15.5:0/1246791840 conn(0x55fa78677a30 msgr2=0x55fa78684a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5025ffb640 0 client.16488 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5046e11740 1 -- 172.21.15.5:0/1246791840 shutdown_connections 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5046e11740 1 -- 172.21.15.5:0/1246791840 wait complete. 2024-09-15T22:14:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5046e11740 20 client.16488 trim_cache size 0 max 16384 2024-09-15T22:14:14.779 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:14:14.779 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:14:14.779 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:14:14.779 INFO:teuthology.orchestra.run.smithi005.stdout: "3a4af246-8746-4f34-9581-975ea07df83a": { 2024-09-15T22:14:14.779 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:14:14.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "3a4af246-8746-4f34-9581-975ea07df83a", 2024-09-15T22:14:14.779 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:14:14.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:14.779 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:14.779 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:14:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 -- 172.21.15.5:0/4038567594 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f502c0a33b0 msgr2=0x7f502c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f502c0a33b0 0x7f502c0a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50380ef180 tx=0x7f503809e040 comp rx=0 tx=0).stop 2024-09-15T22:14:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 -- 172.21.15.5:0/4038567594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5040106a80 msgr2=0x7f50401030d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5040106a80 0x7f50401030d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f5038005cc0 tx=0x7f5038003970 comp rx=0 tx=0).stop 2024-09-15T22:14:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 -- 172.21.15.5:0/4038567594 shutdown_connections 2024-09-15T22:14:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f502c0a33b0 0x7f502c0a5870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5040106a80 0x7f50401030d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5040101420 0x7f5040102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 --2- 172.21.15.5:0/4038567594 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5040100a80 0x7f5040102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 -- 172.21.15.5:0/4038567594 >> 172.21.15.5:0/4038567594 conn(0x7f50400fc7a0 msgr2=0x7f504006a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:14.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 -- 172.21.15.5:0/4038567594 shutdown_connections 2024-09-15T22:14:14.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:14.769+0000 7f5044a38640 1 -- 172.21.15.5:0/4038567594 wait complete. 2024-09-15T22:14:15.082 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:14 smithi005 ceph-mon[30944]: pgmap v163: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-09-15T22:14:15.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:14 smithi099 ceph-mon[34892]: pgmap v163: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-09-15T22:14:15.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:14 smithi129 ceph-mon[35314]: pgmap v163: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-09-15T22:14:15.608 INFO:tasks.cephfs.filesystem:scrub status for tag:3a4af246-8746-4f34-9581-975ea07df83a - {'path': '/', 'tag': '3a4af246-8746-4f34-9581-975ea07df83a', 'options': 'recursive,force'} 2024-09-15T22:14:17.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:16 smithi099 ceph-mon[34892]: pgmap v164: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 448 KiB/s wr, 42 op/s 2024-09-15T22:14:17.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:16 smithi129 ceph-mon[35314]: pgmap v164: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 448 KiB/s wr, 42 op/s 2024-09-15T22:14:17.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:16 smithi005 ceph-mon[30944]: pgmap v164: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 448 KiB/s wr, 42 op/s 2024-09-15T22:14:18.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:17 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:14:18.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:17 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:14:18.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:17 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:14:19.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:18 smithi099 ceph-mon[34892]: pgmap v165: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 425 KiB/s wr, 25 op/s 2024-09-15T22:14:19.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:18 smithi129 ceph-mon[35314]: pgmap v165: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 425 KiB/s wr, 25 op/s 2024-09-15T22:14:19.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:18 smithi005 ceph-mon[30944]: pgmap v165: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 425 KiB/s wr, 25 op/s 2024-09-15T22:14:19.432 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:14:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 290.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:14:19.704 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.861+0000 7ff21c0fc640 1 -- 172.21.15.5:0/1355078161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2141008d0 msgr2=0x7ff214100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.861+0000 7ff21c0fc640 1 --2- 172.21.15.5:0/1355078161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2141008d0 0x7ff214100cb0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff210009900 tx=0x7ff21002a3f0 comp rx=0 tx=0).stop 2024-09-15T22:14:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.862+0000 7ff21c0fc640 1 -- 172.21.15.5:0/1355078161 shutdown_connections 2024-09-15T22:14:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.862+0000 7ff21c0fc640 1 --2- 172.21.15.5:0/1355078161 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff21410e480 0x7ff214110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.862+0000 7ff21c0fc640 1 --2- 172.21.15.5:0/1355078161 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff214101280 0x7ff21410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.862+0000 7ff21c0fc640 1 --2- 172.21.15.5:0/1355078161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2141008d0 0x7ff214100cb0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:20.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.862+0000 7ff21c0fc640 1 -- 172.21.15.5:0/1355078161 >> 172.21.15.5:0/1355078161 conn(0x7ff2140fc7a0 msgr2=0x7ff2140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.862+0000 7ff21c0fc640 1 -- 172.21.15.5:0/1355078161 shutdown_connections 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.862+0000 7ff21c0fc640 1 -- 172.21.15.5:0/1355078161 wait complete. 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.862+0000 7ff21c0fc640 1 Processor -- start 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.862+0000 7ff21c0fc640 1 -- start start 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21c0fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2141008d0 0x7ff21419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21c0fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff214101280 0x7ff21419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21c0fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff21410e480 0x7ff21419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21c0fc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff214114190 con 0x7ff21410e480 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21c0fc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff214114010 con 0x7ff214101280 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21c0fc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff214114310 con 0x7ff2141008d0 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21a699640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff21410e480 0x7ff21419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:20.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21a699640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff21410e480 0x7ff21419ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57224/0 (socket says 172.21.15.5:57224) 2024-09-15T22:14:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21a699640 1 -- 172.21.15.5:0/3542696219 learned_addr learned my addr 172.21.15.5:0/3542696219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff219e98640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2141008d0 0x7ff21419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff219697640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff214101280 0x7ff21419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21a699640 1 -- 172.21.15.5:0/3542696219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2141008d0 msgr2=0x7ff21419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21a699640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2141008d0 0x7ff21419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21a699640 1 -- 172.21.15.5:0/3542696219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff214101280 msgr2=0x7ff21419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21a699640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff214101280 0x7ff21419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21a699640 1 -- 172.21.15.5:0/3542696219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21419f5e0 con 0x7ff21410e480 2024-09-15T22:14:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff21a699640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff21410e480 0x7ff21419ef00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00cce0 tx=0x7ff20c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff20affd640 1 -- 172.21.15.5:0/3542696219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c00f450 con 0x7ff21410e480 2024-09-15T22:14:20.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.863+0000 7ff20affd640 1 -- 172.21.15.5:0/3542696219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff20c00fad0 con 0x7ff21410e480 2024-09-15T22:14:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.864+0000 7ff21c0fc640 1 -- 172.21.15.5:0/3542696219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21419f8d0 con 0x7ff21410e480 2024-09-15T22:14:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.864+0000 7ff20affd640 1 -- 172.21.15.5:0/3542696219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c018bd0 con 0x7ff21410e480 2024-09-15T22:14:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.864+0000 7ff21c0fc640 1 -- 172.21.15.5:0/3542696219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2141a71b0 con 0x7ff21410e480 2024-09-15T22:14:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.864+0000 7ff21c0fc640 1 -- 172.21.15.5:0/3542696219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff214100cb0 con 0x7ff21410e480 2024-09-15T22:14:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.866+0000 7ff20affd640 1 -- 172.21.15.5:0/3542696219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff20c00fc80 con 0x7ff21410e480 2024-09-15T22:14:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.866+0000 7ff20affd640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff2000a33b0 0x7ff2000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.866+0000 7ff219e98640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff2000a33b0 0x7ff2000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.867+0000 7ff20affd640 1 -- 172.21.15.5:0/3542696219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7ff20c0d0690 con 0x7ff21410e480 2024-09-15T22:14:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.867+0000 7ff219e98640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff2000a33b0 0x7ff2000a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff210009230 tx=0x7ff2100058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:20.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:20.871+0000 7ff20affd640 1 -- 172.21.15.5:0/3542696219 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff20c014030 con 0x7ff21410e480 2024-09-15T22:14:21.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.081+0000 7ff21c0fc640 1 -- 172.21.15.5:0/3542696219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff21402d7c0 con 0x7ff21410e480 2024-09-15T22:14:21.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.081+0000 7ff20affd640 1 -- 172.21.15.5:0/3542696219 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7ff20c09ccd0 con 0x7ff21410e480 2024-09-15T22:14:21.084 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:14:21.085 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:14:21.085 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:14:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.083+0000 7ff21c0fc640 1 -- 172.21.15.5:0/3542696219 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff2000a33b0 msgr2=0x7ff2000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.083+0000 7ff21c0fc640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff2000a33b0 0x7ff2000a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff210009230 tx=0x7ff2100058d0 comp rx=0 tx=0).stop 2024-09-15T22:14:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.083+0000 7ff21c0fc640 1 -- 172.21.15.5:0/3542696219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff21410e480 msgr2=0x7ff21419ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.083+0000 7ff21c0fc640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff21410e480 0x7ff21419ef00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00cce0 tx=0x7ff20c00a620 comp rx=0 tx=0).stop 2024-09-15T22:14:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.083+0000 7ff21c0fc640 1 -- 172.21.15.5:0/3542696219 shutdown_connections 2024-09-15T22:14:21.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.083+0000 7ff21c0fc640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff2000a33b0 0x7ff2000a5870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.083+0000 7ff21c0fc640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff21410e480 0x7ff21419ef00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.083+0000 7ff21c0fc640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff214101280 0x7ff21419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.083+0000 7ff21c0fc640 1 --2- 172.21.15.5:0/3542696219 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff2141008d0 0x7ff21419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.084+0000 7ff21c0fc640 1 -- 172.21.15.5:0/3542696219 >> 172.21.15.5:0/3542696219 conn(0x7ff2140fc7a0 msgr2=0x7ff2140feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.084+0000 7ff21c0fc640 1 -- 172.21.15.5:0/3542696219 shutdown_connections 2024-09-15T22:14:21.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:21.084+0000 7ff21c0fc640 1 -- 172.21.15.5:0/3542696219 wait complete. 2024-09-15T22:14:21.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:20 smithi129 ceph-mon[35314]: pgmap v166: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-15T22:14:21.095 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:20 smithi005 ceph-mon[30944]: pgmap v166: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-15T22:14:21.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:20 smithi099 ceph-mon[34892]: pgmap v166: 129 pgs: 129 active+clean; 2.5 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-15T22:14:22.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:21 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3542696219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:22.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:21 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3542696219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:22.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:21 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3542696219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:23.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:22 smithi005 ceph-mon[30944]: pgmap v167: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 MiB/s wr, 320 op/s 2024-09-15T22:14:23.105 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id 27b49ad6 --release --await' 2024-09-15T22:14:23.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id 27b49ad6 --release --await 2024-09-15T22:14:23.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:22 smithi099 ceph-mon[34892]: pgmap v167: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 MiB/s wr, 320 op/s 2024-09-15T22:14:23.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:22 smithi129 ceph-mon[35314]: pgmap v167: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 MiB/s wr, 320 op/s 2024-09-15T22:14:23.369 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.818+0000 7f4aed91e640 1 -- 172.21.15.5:0/4227659117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae810e440 msgr2=0x7f4ae8110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.818+0000 7f4aed91e640 1 --2- 172.21.15.5:0/4227659117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae810e440 0x7f4ae8110830 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009ca0 tx=0x7f4ae002a950 comp rx=0 tx=0).stop 2024-09-15T22:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.818+0000 7f4aed91e640 1 -- 172.21.15.5:0/4227659117 shutdown_connections 2024-09-15T22:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.818+0000 7f4aed91e640 1 --2- 172.21.15.5:0/4227659117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae810e440 0x7f4ae8110830 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.818+0000 7f4aed91e640 1 --2- 172.21.15.5:0/4227659117 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8074e30 0x7f4ae810df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.818+0000 7f4aed91e640 1 --2- 172.21.15.5:0/4227659117 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ae8074510 0x7f4ae80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.818+0000 7f4aed91e640 1 -- 172.21.15.5:0/4227659117 >> 172.21.15.5:0/4227659117 conn(0x7f4ae80fd8c0 msgr2=0x7f4ae80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.818+0000 7f4aed91e640 1 -- 172.21.15.5:0/4227659117 shutdown_connections 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.818+0000 7f4aed91e640 1 -- 172.21.15.5:0/4227659117 wait complete. 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4aed91e640 1 Processor -- start 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4aed91e640 1 -- start start 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4aed91e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ae8074510 0x7f4ae81a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4aed91e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8074e30 0x7f4ae81a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4aed91e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae810e440 0x7f4ae819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4aed91e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae8114230 con 0x7f4ae810e440 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4aed91e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81140b0 con 0x7f4ae8074510 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4aed91e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81143b0 con 0x7f4ae8074e30 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4ae77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae810e440 0x7f4ae819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4ae77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae810e440 0x7f4ae819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46898/0 (socket says 172.21.15.5:46898) 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4ae77fe640 1 -- 172.21.15.5:0/2452146136 learned_addr learned my addr 172.21.15.5:0/2452146136 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.819+0000 7f4ae6ffd640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ae8074510 0x7f4ae81a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4adffff640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8074e30 0x7f4ae81a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4ae77fe640 1 -- 172.21.15.5:0/2452146136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8074e30 msgr2=0x7f4ae81a0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4ae77fe640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8074e30 0x7f4ae81a0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4ae77fe640 1 -- 172.21.15.5:0/2452146136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ae8074510 msgr2=0x7f4ae81a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4ae77fe640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ae8074510 0x7f4ae81a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4ae77fe640 1 -- 172.21.15.5:0/2452146136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae819acd0 con 0x7f4ae810e440 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4ae77fe640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae810e440 0x7f4ae819a5a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006010 tx=0x7f4ae002f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4aec91c640 1 -- 172.21.15.5:0/2452146136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0034040 con 0x7f4ae810e440 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4aec91c640 1 -- 172.21.15.5:0/2452146136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ae0003e00 con 0x7f4ae810e440 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4aed91e640 1 -- 172.21.15.5:0/2452146136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae819af60 con 0x7f4ae810e440 2024-09-15T22:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4aec91c640 1 -- 172.21.15.5:0/2452146136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0045480 con 0x7f4ae810e440 2024-09-15T22:14:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.820+0000 7f4aed91e640 1 -- 172.21.15.5:0/2452146136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae81a72b0 con 0x7f4ae810e440 2024-09-15T22:14:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.823+0000 7f4aec91c640 1 -- 172.21.15.5:0/2452146136 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4ae0037020 con 0x7f4ae810e440 2024-09-15T22:14:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.823+0000 7f4aefcf7740 1 Processor -- start 2024-09-15T22:14:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.823+0000 7f4aefcf7740 1 -- start start 2024-09-15T22:14:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.823+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be1b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.823+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be1b980 0x55a14be25d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.823+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be26240 0x55a14be28630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.823+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a14bd80620 con 0x55a14be1b060 2024-09-15T22:14:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.823+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a14bd27e50 con 0x55a14be26240 2024-09-15T22:14:23.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.823+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a14bd7c5e0 con 0x55a14be1b980 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4adffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be1b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4adffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be1b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46900/0 (socket says 172.21.15.5:46900) 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4adffff640 1 -- 172.21.15.5:0/375013653 learned_addr learned my addr 172.21.15.5:0/375013653 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4ae6ffd640 1 --2- 172.21.15.5:0/375013653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be1b980 0x55a14be25d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4adffff640 1 --2- 172.21.15.5:0/375013653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be26240 0x55a14be28630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4adffff640 1 -- 172.21.15.5:0/375013653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be1b980 msgr2=0x55a14be25d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4adffff640 1 --2- 172.21.15.5:0/375013653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be1b980 0x55a14be25d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4adffff640 1 -- 172.21.15.5:0/375013653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be26240 msgr2=0x55a14be28630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4adffff640 1 --2- 172.21.15.5:0/375013653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be26240 0x55a14be28630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4aec91c640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4ad40a3430 0x7f4ad40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4adffff640 1 -- 172.21.15.5:0/375013653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a14bd18d40 con 0x55a14be1b060 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4ae77fe640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4ad40a3430 0x7f4ad40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4adffff640 1 --2- 172.21.15.5:0/375013653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be1b440 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0006e10 tx=0x7f4ad000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118350db425812d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4aec91c640 1 -- 172.21.15.5:0/2452146136 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4ae00f10b0 con 0x7f4ae810e440 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4addffb640 1 -- 172.21.15.5:0/375013653 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00173f0 con 0x55a14be1b060 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4addffb640 1 -- 172.21.15.5:0/375013653 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad0010040 con 0x55a14be1b060 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.824+0000 7f4addffb640 1 -- 172.21.15.5:0/375013653 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0016950 con 0x55a14be1b060 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.825+0000 7f4aefcf7740 1 -- 172.21.15.5:0/375013653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 msgr2=0x55a14be1b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.825+0000 7f4ae77fe640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4ad40a3430 0x7f4ad40a58f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00bd0c0 tx=0x7f4ae008ff60 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.825+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/375013653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be1b440 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0006e10 tx=0x7f4ad000b400 comp rx=0 tx=0).stop 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.825+0000 7f4aefcf7740 1 -- 172.21.15.5:0/375013653 shutdown_connections 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.825+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/375013653 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be26240 0x55a14be28630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.825+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/375013653 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be1b980 0x55a14be25d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.825+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/375013653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be1b440 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.825+0000 7f4aefcf7740 1 -- 172.21.15.5:0/375013653 >> 172.21.15.5:0/375013653 conn(0x55a14be175f0 msgr2=0x55a14be179d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.825+0000 7f4aefcf7740 1 -- 172.21.15.5:0/375013653 shutdown_connections 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.825+0000 7f4aefcf7740 1 -- 172.21.15.5:0/375013653 wait complete. 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4aefcf7740 1 -- start start 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4aefcf7740 1 Processor -- start 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be1b060 0x55a14be23180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be236c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be26240 0x55a14be23c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a14be28e10 con 0x55a14be1b980 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a14bd08f10 con 0x55a14be26240 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a14be28f90 con 0x55a14be1b060 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4aefcf7740 10 client.0 _ll_register_callbacks cb 0x55a14bd2cdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4ae6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be236c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4ae6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be236c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46914/0 (socket says 172.21.15.5:46914) 2024-09-15T22:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4ae6ffd640 1 -- 172.21.15.5:0/648869691 learned_addr learned my addr 172.21.15.5:0/648869691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4adffff640 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be1b060 0x55a14be23180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.826+0000 7f4adffff640 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be26240 0x55a14be23c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ae6ffd640 1 -- 172.21.15.5:0/648869691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be1b060 msgr2=0x55a14be23180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ae6ffd640 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be1b060 0x55a14be23180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ae6ffd640 1 -- 172.21.15.5:0/648869691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be26240 msgr2=0x55a14be23c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ae6ffd640 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be26240 0x55a14be23c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ae6ffd640 1 -- 172.21.15.5:0/648869691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a14ba8f580 con 0x55a14be1b980 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4adffff640 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be26240 0x55a14be23c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ae6ffd640 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be236c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8011570 tx=0x7f4ad8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77bc47bbf523b368 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ac6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/648869691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad800f6d0 con 0x55a14be1b980 2024-09-15T22:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/648869691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4ad800f870 con 0x55a14be1b980 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4aefcf7740 1 -- 172.21.15.5:0/648869691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad0010040 con 0x55a14be1b980 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/648869691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ad8026050 con 0x55a14be1b980 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4aefcf7740 10 client.16506 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4aefcf7740 1 -- 172.21.15.5:0/648869691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a14bb59960 con 0x55a14be1b980 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/648869691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad0010040 con 0x55a14be1b980 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.827+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/648869691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4ad801d400 con 0x55a14be1b980 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.828+0000 7f4aefcf7740 10 client.16506 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.828+0000 7f4aefcf7740 10 client.16506 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.828+0000 7f4aefcf7740 20 client.16506 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.828+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a14be35ee0 0x55a14be38300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.828+0000 7f4aefcf7740 4 client.16506 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.828+0000 7f4aefcf7740 1 -- 172.21.15.5:0/648869691 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a14bd8db90 con 0x55a14be35ee0 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.828+0000 7f4adffff640 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a14be35ee0 0x55a14be38300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.828+0000 7f4ac6ffd640 10 client.16506 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.828+0000 7f4adffff640 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a14be35ee0 0x55a14be38300 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18d270f7dca3eacd server_cookie=e14ac17a60a1563 in_seq=0 out_seq=0 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/648869691 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a14bd8db90 con 0x55a14be35ee0 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4ac6ffd640 10 client.16506 handle_command_reply: tid=0 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4aefcf7740 1 client.16506 shutdown 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4aefcf7740 1 -- 172.21.15.5:0/648869691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 msgr2=0x55a14be236c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be236c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8011570 tx=0x7f4ad8011a30 comp rx=0 tx=0).stop 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4aefcf7740 1 -- 172.21.15.5:0/648869691 shutdown_connections 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a14be35ee0 0x55a14be38300 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be26240 0x55a14be23c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be236c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/648869691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be1b060 0x55a14be23180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4ac6ffd640 0 client.16506 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:23.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4aefcf7740 1 -- 172.21.15.5:0/648869691 >> 172.21.15.5:0/648869691 conn(0x55a14be175f0 msgr2=0x55a14be1d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:23.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4aefcf7740 1 -- 172.21.15.5:0/648869691 shutdown_connections 2024-09-15T22:14:23.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.829+0000 7f4aefcf7740 1 -- 172.21.15.5:0/648869691 wait complete. 2024-09-15T22:14:23.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.830+0000 7f4aefcf7740 20 client.16506 trim_cache size 0 max 16384 2024-09-15T22:14:23.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4aefcf7740 1 Processor -- start 2024-09-15T22:14:23.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4aefcf7740 1 -- start start 2024-09-15T22:14:23.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be1b060 0x55a14be3dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be3e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be26240 0x55a14be203e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b3ba0 con 0x55a14be1b980 2024-09-15T22:14:23.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b1470 con 0x55a14be1b060 2024-09-15T22:14:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81af2c0 con 0x55a14be26240 2024-09-15T22:14:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4ae6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be3e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4ae6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be3e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46918/0 (socket says 172.21.15.5:46918) 2024-09-15T22:14:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4ae6ffd640 1 -- 172.21.15.5:0/756283928 learned_addr learned my addr 172.21.15.5:0/756283928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4adffff640 1 --2- 172.21.15.5:0/756283928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be1b060 0x55a14be3dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4adffff640 1 --2- 172.21.15.5:0/756283928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be26240 0x55a14be203e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4ae6ffd640 1 -- 172.21.15.5:0/756283928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be26240 msgr2=0x55a14be203e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4ae6ffd640 1 --2- 172.21.15.5:0/756283928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be26240 0x55a14be203e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4ae6ffd640 1 -- 172.21.15.5:0/756283928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be1b060 msgr2=0x55a14be3dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4ae6ffd640 1 --2- 172.21.15.5:0/756283928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be1b060 0x55a14be3dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4ae6ffd640 1 -- 172.21.15.5:0/756283928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a14be17090 con 0x55a14be1b980 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.847+0000 7f4adffff640 1 --2- 172.21.15.5:0/756283928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be26240 0x55a14be203e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.848+0000 7f4ae6ffd640 1 --2- 172.21.15.5:0/756283928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be3e0f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4ad801d9c0 tx=0x7f4ad8011fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90621e37f1ff0d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.848+0000 7f4add7fa640 1 -- 172.21.15.5:0/756283928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad802e020 con 0x55a14be1b980 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.848+0000 7f4add7fa640 1 -- 172.21.15.5:0/756283928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad801d400 con 0x55a14be1b980 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.848+0000 7f4add7fa640 1 -- 172.21.15.5:0/756283928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8014440 con 0x55a14be1b980 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.849+0000 7f4aefcf7740 1 -- 172.21.15.5:0/756283928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 msgr2=0x55a14be3e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.849+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/756283928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be3e0f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4ad801d9c0 tx=0x7f4ad8011fb0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.849+0000 7f4aefcf7740 1 -- 172.21.15.5:0/756283928 shutdown_connections 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.849+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/756283928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be26240 0x55a14be203e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.849+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/756283928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b980 0x55a14be3e0f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.849+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/756283928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be1b060 0x55a14be3dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.849+0000 7f4aefcf7740 1 -- 172.21.15.5:0/756283928 >> 172.21.15.5:0/756283928 conn(0x55a14be175f0 msgr2=0x55a14be25200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.849+0000 7f4aefcf7740 1 -- 172.21.15.5:0/756283928 shutdown_connections 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.849+0000 7f4aefcf7740 1 -- 172.21.15.5:0/756283928 wait complete. 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.849+0000 7f4aefcf7740 1 -- start start 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.849+0000 7f4aefcf7740 1 Processor -- start 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be2e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be1b980 0x55a14be2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4aefcf7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be26240 0x55a14be2af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81af930 con 0x55a14be1b060 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b0da0 con 0x55a14be1b980 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4aefcf7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae81b0000 con 0x55a14be26240 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4aefcf7740 10 client.0 _ll_register_callbacks cb 0x7f4ad8009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4adffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4adffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be2e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46922/0 (socket says 172.21.15.5:46922) 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4adffff640 1 -- 172.21.15.5:0/3240318322 learned_addr learned my addr 172.21.15.5:0/3240318322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4ae6ffd640 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be1b980 0x55a14be2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4adffff640 1 -- 172.21.15.5:0/3240318322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be26240 msgr2=0x55a14be2af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4adffff640 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be26240 0x55a14be2af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4adffff640 1 -- 172.21.15.5:0/3240318322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be1b980 msgr2=0x55a14be2aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4adffff640 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be1b980 0x55a14be2aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4adffff640 1 -- 172.21.15.5:0/3240318322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a14be2b490 con 0x55a14be1b060 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4adffff640 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be2e680 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0016520 tx=0x7f4ad00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ee12ebec83b57b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4ac6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/3240318322 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0037060 con 0x55a14be1b060 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/3240318322 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f4ad00167d0 con 0x55a14be1b060 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.850+0000 7f4aefcf7740 1 -- 172.21.15.5:0/3240318322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ad801d400 con 0x55a14be1b060 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/3240318322 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ad003e050 con 0x55a14be1b060 2024-09-15T22:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4aefcf7740 10 client.16512 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4aefcf7740 1 -- 172.21.15.5:0/3240318322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a14bea18e0 con 0x55a14be1b060 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/3240318322 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ad801d400 con 0x55a14be1b060 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/3240318322 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4ad00177d0 con 0x55a14be1b060 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4aefcf7740 10 client.16512 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4aefcf7740 10 client.16512 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4aefcf7740 20 client.16512 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a14bea1dc0 0x55a14bea41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4aefcf7740 4 client.16512 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "27b49ad6", "release": true, "await": true}] 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4aefcf7740 1 -- 172.21.15.5:0/3240318322 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "27b49ad6", "release": true, "await": true}) -- 0x55a14bd880a0 con 0x55a14bea1dc0 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.851+0000 7f4ae6ffd640 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a14bea1dc0 0x55a14bea41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.852+0000 7f4ac6ffd640 10 client.16512 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.852+0000 7f4ae6ffd640 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a14bea1dc0 0x55a14bea41b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8190476a48437b6 server_cookie=62f1b5f8c9fe7111 in_seq=0 out_seq=0 2024-09-15T22:14:23.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4ac6ffd640 1 -- 172.21.15.5:0/3240318322 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55a14bd880a0 con 0x55a14bea1dc0 2024-09-15T22:14:23.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4ac6ffd640 10 client.16512 handle_command_reply: tid=0 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4aefcf7740 1 client.16512 shutdown 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4aefcf7740 1 -- 172.21.15.5:0/3240318322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 msgr2=0x55a14be2e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be2e680 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0016520 tx=0x7f4ad00047d0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4aefcf7740 1 -- 172.21.15.5:0/3240318322 shutdown_connections 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a14be26240 0x55a14be2af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a14bea1dc0 0x55a14bea41b0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a14be1b980 0x55a14be2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4aefcf7740 1 --2- 172.21.15.5:0/3240318322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a14be1b060 0x55a14be2e680 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4aefcf7740 1 -- 172.21.15.5:0/3240318322 >> 172.21.15.5:0/3240318322 conn(0x55a14be175f0 msgr2=0x55a14be15f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.898+0000 7f4ac6ffd640 0 client.16512 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aefcf7740 1 -- 172.21.15.5:0/3240318322 shutdown_connections 2024-09-15T22:14:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aefcf7740 1 -- 172.21.15.5:0/3240318322 wait complete. 2024-09-15T22:14:23.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aefcf7740 20 client.16512 trim_cache size 0 max 16384 2024-09-15T22:14:23.903 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:14:23.903 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:14:23.903 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:14:23.904 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 20, 2024-09-15T22:14:23.904 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:14:23.904 INFO:teuthology.orchestra.run.smithi005.stdout: "27b49ad6": { 2024-09-15T22:14:23.904 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 20, 2024-09-15T22:14:23.904 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:14:23.904 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 145.3, 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:23.905 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:23.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:23.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:23.906 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:14:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aed91e640 1 -- 172.21.15.5:0/2452146136 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4ad40a3430 msgr2=0x7f4ad40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aed91e640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4ad40a3430 0x7f4ad40a58f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00bd0c0 tx=0x7f4ae008ff60 comp rx=0 tx=0).stop 2024-09-15T22:14:23.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aed91e640 1 -- 172.21.15.5:0/2452146136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae810e440 msgr2=0x7f4ae819a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aed91e640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae810e440 0x7f4ae819a5a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006010 tx=0x7f4ae002f710 comp rx=0 tx=0).stop 2024-09-15T22:14:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aed91e640 1 -- 172.21.15.5:0/2452146136 shutdown_connections 2024-09-15T22:14:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aed91e640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4ad40a3430 0x7f4ad40a58f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aed91e640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ae810e440 0x7f4ae819a5a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aed91e640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4ae8074e30 0x7f4ae81a0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aed91e640 1 --2- 172.21.15.5:0/2452146136 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4ae8074510 0x7f4ae81a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.899+0000 7f4aed91e640 1 -- 172.21.15.5:0/2452146136 >> 172.21.15.5:0/2452146136 conn(0x7f4ae80fd8c0 msgr2=0x7f4ae80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.900+0000 7f4aed91e640 1 -- 172.21.15.5:0/2452146136 shutdown_connections 2024-09-15T22:14:23.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:23.900+0000 7f4aed91e640 1 -- 172.21.15.5:0/2452146136 wait complete. 2024-09-15T22:14:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 295.0 s, 42.8 tps, lat 10783.861 ms stddev 12589.207 2024-09-15T22:14:25.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:24 smithi005 ceph-mon[30944]: pgmap v168: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.1 MiB/s wr, 318 op/s 2024-09-15T22:14:25.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:24 smithi099 ceph-mon[34892]: pgmap v168: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.1 MiB/s wr, 318 op/s 2024-09-15T22:14:25.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:24 smithi129 ceph-mon[35314]: pgmap v168: 129 pgs: 129 active+clean; 2.6 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 7.1 MiB/s wr, 318 op/s 2024-09-15T22:14:25.372 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '27b49ad6', total seconds elapsed: 31.3 2024-09-15T22:14:25.372 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.35 2024-09-15T22:14:25.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:14:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.850+0000 7fb12da29640 1 -- 172.21.15.5:0/1380350278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128101de0 msgr2=0x7fb1281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.850+0000 7fb12da29640 1 --2- 172.21.15.5:0/1380350278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128101de0 0x7fb1281021c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fb118009900 tx=0x7fb11802a410 comp rx=0 tx=0).stop 2024-09-15T22:14:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.851+0000 7fb12da29640 1 -- 172.21.15.5:0/1380350278 shutdown_connections 2024-09-15T22:14:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.851+0000 7fb12da29640 1 --2- 172.21.15.5:0/1380350278 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb128106f10 0x7fb12810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.851+0000 7fb12da29640 1 --2- 172.21.15.5:0/1380350278 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128102790 0x7fb1281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.851+0000 7fb12da29640 1 --2- 172.21.15.5:0/1380350278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128101de0 0x7fb1281021c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.851+0000 7fb12da29640 1 -- 172.21.15.5:0/1380350278 >> 172.21.15.5:0/1380350278 conn(0x7fb1280fd8e0 msgr2=0x7fb1280ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.854+0000 7fb12da29640 1 -- 172.21.15.5:0/1380350278 shutdown_connections 2024-09-15T22:14:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.854+0000 7fb12da29640 1 -- 172.21.15.5:0/1380350278 wait complete. 2024-09-15T22:14:26.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.858+0000 7fb12da29640 1 Processor -- start 2024-09-15T22:14:26.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.859+0000 7fb12da29640 1 -- start start 2024-09-15T22:14:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.859+0000 7fb12da29640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb128101de0 0x7fb1280739e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.859+0000 7fb12da29640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128102790 0x7fb12806ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.859+0000 7fb12da29640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128106f10 0x7fb12806cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.859+0000 7fb12da29640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb128111f70 con 0x7fb128106f10 2024-09-15T22:14:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.859+0000 7fb12da29640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb128111df0 con 0x7fb128101de0 2024-09-15T22:14:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.859+0000 7fb12da29640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb1281120f0 con 0x7fb128102790 2024-09-15T22:14:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.859+0000 7fb1277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128106f10 0x7fb12806cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.859+0000 7fb1277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128106f10 0x7fb12806cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46940/0 (socket says 172.21.15.5:46940) 2024-09-15T22:14:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.859+0000 7fb1277fe640 1 -- 172.21.15.5:0/3733212523 learned_addr learned my addr 172.21.15.5:0/3733212523 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.863+0000 7fb1267fc640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128102790 0x7fb12806ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:26.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.865+0000 7fb126ffd640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb128101de0 0x7fb1280739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.865+0000 7fb1277fe640 1 -- 172.21.15.5:0/3733212523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128102790 msgr2=0x7fb12806ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.865+0000 7fb1277fe640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128102790 0x7fb12806ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.865+0000 7fb1277fe640 1 -- 172.21.15.5:0/3733212523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb128101de0 msgr2=0x7fb1280739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.865+0000 7fb1277fe640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb128101de0 0x7fb1280739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.865+0000 7fb1277fe640 1 -- 172.21.15.5:0/3733212523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb128074480 con 0x7fb128106f10 2024-09-15T22:14:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.870+0000 7fb1277fe640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128106f10 0x7fb12806cfb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb12000eaa0 tx=0x7fb12000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.874+0000 7fb12ca27640 1 -- 172.21.15.5:0/3733212523 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12000cdb0 con 0x7fb128106f10 2024-09-15T22:14:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.874+0000 7fb12ca27640 1 -- 172.21.15.5:0/3733212523 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb120005020 con 0x7fb128106f10 2024-09-15T22:14:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.874+0000 7fb12ca27640 1 -- 172.21.15.5:0/3733212523 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1200116b0 con 0x7fb128106f10 2024-09-15T22:14:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.874+0000 7fb12da29640 1 -- 172.21.15.5:0/3733212523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb12806d550 con 0x7fb128106f10 2024-09-15T22:14:26.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.874+0000 7fb12da29640 1 -- 172.21.15.5:0/3733212523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb128071510 con 0x7fb128106f10 2024-09-15T22:14:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.878+0000 7fb12ca27640 1 -- 172.21.15.5:0/3733212523 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb120004a40 con 0x7fb128106f10 2024-09-15T22:14:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.879+0000 7fb12da29640 1 -- 172.21.15.5:0/3733212523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb114005290 con 0x7fb128106f10 2024-09-15T22:14:26.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.879+0000 7fb12ca27640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb1100a3580 0x7fb1100a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:26.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.880+0000 7fb12ca27640 1 -- 172.21.15.5:0/3733212523 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fb1200d0230 con 0x7fb128106f10 2024-09-15T22:14:26.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.880+0000 7fb126ffd640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb1100a3580 0x7fb1100a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:26.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.889+0000 7fb126ffd640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb1100a3580 0x7fb1100a5a40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb118005ab0 tx=0x7fb118005a20 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:26.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:26.891+0000 7fb12ca27640 1 -- 172.21.15.5:0/3733212523 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb120016030 con 0x7fb128106f10 2024-09-15T22:14:27.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:26 smithi005 ceph-mon[30944]: pgmap v169: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 649 op/s 2024-09-15T22:14:27.145 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:26 smithi129 ceph-mon[35314]: pgmap v169: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 649 op/s 2024-09-15T22:14:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.279+0000 7fb12da29640 1 -- 172.21.15.5:0/3733212523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb114005850 con 0x7fb128106f10 2024-09-15T22:14:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.282+0000 7fb12ca27640 1 -- 172.21.15.5:0/3733212523 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fb12009c810 con 0x7fb128106f10 2024-09-15T22:14:27.285 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:14:27.285 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:14:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:14:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.286+0000 7fb12da29640 1 -- 172.21.15.5:0/3733212523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb1100a3580 msgr2=0x7fb1100a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.286+0000 7fb12da29640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb1100a3580 0x7fb1100a5a40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb118005ab0 tx=0x7fb118005a20 comp rx=0 tx=0).stop 2024-09-15T22:14:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.286+0000 7fb12da29640 1 -- 172.21.15.5:0/3733212523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128106f10 msgr2=0x7fb12806cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.286+0000 7fb12da29640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128106f10 0x7fb12806cfb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb12000eaa0 tx=0x7fb12000ef60 comp rx=0 tx=0).stop 2024-09-15T22:14:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.286+0000 7fb12da29640 1 -- 172.21.15.5:0/3733212523 shutdown_connections 2024-09-15T22:14:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.286+0000 7fb12da29640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb1100a3580 0x7fb1100a5a40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.286+0000 7fb12da29640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb128106f10 0x7fb12806cfb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.286+0000 7fb12da29640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb128102790 0x7fb12806ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.286+0000 7fb12da29640 1 --2- 172.21.15.5:0/3733212523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb128101de0 0x7fb1280739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:27.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.286+0000 7fb12da29640 1 -- 172.21.15.5:0/3733212523 >> 172.21.15.5:0/3733212523 conn(0x7fb1280fd8e0 msgr2=0x7fb1280fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:27.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.292+0000 7fb12da29640 1 -- 172.21.15.5:0/3733212523 shutdown_connections 2024-09-15T22:14:27.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:27.293+0000 7fb12da29640 1 -- 172.21.15.5:0/3733212523 wait complete. 2024-09-15T22:14:27.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:26 smithi099 ceph-mon[34892]: pgmap v169: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 15 MiB/s wr, 649 op/s 2024-09-15T22:14:28.240 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:27 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3733212523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:28.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:27 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3733212523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:27 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3733212523' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:28.777 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0' 2024-09-15T22:14:28.778 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 145.35 --await-for 45.0 2024-09-15T22:14:29.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:28 smithi005 ceph-mon[30944]: pgmap v170: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 627 op/s 2024-09-15T22:14:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:29.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:28 smithi099 ceph-mon[34892]: pgmap v170: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 627 op/s 2024-09-15T22:14:29.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:28 smithi129 ceph-mon[35314]: pgmap v170: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 627 op/s 2024-09-15T22:14:29.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 300.0 s, 655.6 tps, lat 246.989 ms stddev 1534.519 2024-09-15T22:14:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.945+0000 7effd7cb9640 1 -- 172.21.15.5:0/4012535775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effd010c160 msgr2=0x7effd010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.945+0000 7effd7cb9640 1 --2- 172.21.15.5:0/4012535775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effd010c160 0x7effd010e600 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7effc8009be0 tx=0x7effc802a750 comp rx=0 tx=0).stop 2024-09-15T22:14:29.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.945+0000 7effd7cb9640 1 -- 172.21.15.5:0/4012535775 shutdown_connections 2024-09-15T22:14:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.945+0000 7effd7cb9640 1 --2- 172.21.15.5:0/4012535775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effd010c160 0x7effd010e600 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.945+0000 7effd7cb9640 1 --2- 172.21.15.5:0/4012535775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd0103650 0x7effd010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.945+0000 7effd7cb9640 1 --2- 172.21.15.5:0/4012535775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd0102d30 0x7effd0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.945+0000 7effd7cb9640 1 -- 172.21.15.5:0/4012535775 >> 172.21.15.5:0/4012535775 conn(0x7effd00fc790 msgr2=0x7effd00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.945+0000 7effd7cb9640 1 -- 172.21.15.5:0/4012535775 shutdown_connections 2024-09-15T22:14:29.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.945+0000 7effd7cb9640 1 -- 172.21.15.5:0/4012535775 wait complete. 2024-09-15T22:14:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.946+0000 7effd7cb9640 1 Processor -- start 2024-09-15T22:14:29.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.947+0000 7effd7cb9640 1 -- start start 2024-09-15T22:14:29.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.948+0000 7effd7cb9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effd0102d30 0x7effd0196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:29.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.948+0000 7effd7cb9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd0103650 0x7effd01967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:29.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.948+0000 7effd7cb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd010c160 0x7effd019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:29.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.949+0000 7effd6256640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd010c160 0x7effd019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:29.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.949+0000 7effd7cb9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7effd010fd20 con 0x7effd010c160 2024-09-15T22:14:29.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.949+0000 7effd5254640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd0103650 0x7effd01967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:29.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.949+0000 7effd7cb9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effd010fba0 con 0x7effd0103650 2024-09-15T22:14:29.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.949+0000 7effd7cb9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7effd010fea0 con 0x7effd0102d30 2024-09-15T22:14:29.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.949+0000 7effd5254640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd0103650 0x7effd01967c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:51694/0 (socket says 172.21.15.5:51694) 2024-09-15T22:14:29.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.949+0000 7effd5a55640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effd0102d30 0x7effd0196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:29.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.949+0000 7effd5a55640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effd0102d30 0x7effd0196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:46128/0 (socket says 172.21.15.5:46128) 2024-09-15T22:14:29.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.949+0000 7effd5254640 1 -- 172.21.15.5:0/44471139 learned_addr learned my addr 172.21.15.5:0/44471139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:29.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.950+0000 7effd5254640 1 -- 172.21.15.5:0/44471139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effd0102d30 msgr2=0x7effd0196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:29.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.950+0000 7effd5254640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effd0102d30 0x7effd0196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.950+0000 7effd5254640 1 -- 172.21.15.5:0/44471139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd010c160 msgr2=0x7effd019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:29.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.950+0000 7effd5254640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd010c160 0x7effd019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.950+0000 7effd5254640 1 -- 172.21.15.5:0/44471139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd019b230 con 0x7effd0103650 2024-09-15T22:14:29.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.950+0000 7effd5254640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd0103650 0x7effd01967c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7effc0012b80 tx=0x7effc0012f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:29.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.951+0000 7effc6ffd640 1 -- 172.21.15.5:0/44471139 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0010880 con 0x7effd0103650 2024-09-15T22:14:29.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.951+0000 7effc6ffd640 1 -- 172.21.15.5:0/44471139 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc001d070 con 0x7effd0103650 2024-09-15T22:14:29.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.951+0000 7effd7cb9640 1 -- 172.21.15.5:0/44471139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd019b4c0 con 0x7effd0103650 2024-09-15T22:14:29.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.951+0000 7effd7cb9640 1 -- 172.21.15.5:0/44471139 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd01a2e00 con 0x7effd0103650 2024-09-15T22:14:29.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.952+0000 7effc6ffd640 1 -- 172.21.15.5:0/44471139 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0018480 con 0x7effd0103650 2024-09-15T22:14:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.957+0000 7effda092740 1 Processor -- start 2024-09-15T22:14:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effda092740 1 -- start start 2024-09-15T22:14:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effc6ffd640 1 -- 172.21.15.5:0/44471139 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7effc0016050 con 0x7effd0103650 2024-09-15T22:14:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effda092740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba00c060 0x5642ba00c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effda092740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642ba016d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effda092740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 0x5642ba019630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effda092740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5642b9f68160 con 0x5642ba00c060 2024-09-15T22:14:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effda092740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5642b9f0f720 con 0x5642ba00c980 2024-09-15T22:14:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effda092740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5642b9f64120 con 0x5642ba017240 2024-09-15T22:14:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effd5a55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642ba016d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effd5254640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 0x5642ba019630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effd5254640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 0x5642ba019630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:46132/0 (socket says 172.21.15.5:46132) 2024-09-15T22:14:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.958+0000 7effd5254640 1 -- 172.21.15.5:0/1386538159 learned_addr learned my addr 172.21.15.5:0/1386538159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.959+0000 7effd6256640 1 --2- 172.21.15.5:0/1386538159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba00c060 0x5642ba00c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.959+0000 7effd5a55640 1 -- 172.21.15.5:0/1386538159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 msgr2=0x5642ba019630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.959+0000 7effd5a55640 1 --2- 172.21.15.5:0/1386538159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 0x5642ba019630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.959+0000 7effd5a55640 1 -- 172.21.15.5:0/1386538159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba00c060 msgr2=0x5642ba00c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:29.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.959+0000 7effd5a55640 1 --2- 172.21.15.5:0/1386538159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba00c060 0x5642ba00c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.959+0000 7effd5a55640 1 -- 172.21.15.5:0/1386538159 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642b9cd9ba0 con 0x5642ba00c980 2024-09-15T22:14:29.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.960+0000 7effd6256640 1 --2- 172.21.15.5:0/1386538159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba00c060 0x5642ba00c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:29.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.960+0000 7effc6ffd640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7effbc0a34b0 0x7effbc0a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:29.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.961+0000 7effd5a55640 1 --2- 172.21.15.5:0/1386538159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642ba016d00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effcc009c30 tx=0x7effcc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f61079658ee5d660 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:29.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.961+0000 7effc6ffd640 1 -- 172.21.15.5:0/44471139 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7effc00d3d40 con 0x7effd0103650 2024-09-15T22:14:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.961+0000 7effd6256640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7effbc0a34b0 0x7effbc0a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.961+0000 7effbbfff640 1 -- 172.21.15.5:0/1386538159 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc01e070 con 0x5642ba00c980 2024-09-15T22:14:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.961+0000 7effbbfff640 1 -- 172.21.15.5:0/1386538159 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effcc009de0 con 0x5642ba00c980 2024-09-15T22:14:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.962+0000 7effbbfff640 1 -- 172.21.15.5:0/1386538159 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc019410 con 0x5642ba00c980 2024-09-15T22:14:29.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.962+0000 7effda092740 1 -- 172.21.15.5:0/1386538159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 msgr2=0x5642ba016d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.962+0000 7effda092740 1 --2- 172.21.15.5:0/1386538159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642ba016d00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effcc009c30 tx=0x7effcc011810 comp rx=0 tx=0).stop 2024-09-15T22:14:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.962+0000 7effd6256640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7effbc0a34b0 0x7effbc0a5970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7effc8005cc0 tx=0x7effc8009840 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.962+0000 7effda092740 1 -- 172.21.15.5:0/1386538159 shutdown_connections 2024-09-15T22:14:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.962+0000 7effda092740 1 --2- 172.21.15.5:0/1386538159 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 0x5642ba019630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.962+0000 7effda092740 1 --2- 172.21.15.5:0/1386538159 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642ba016d00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.962+0000 7effda092740 1 --2- 172.21.15.5:0/1386538159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba00c060 0x5642ba00c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.962+0000 7effda092740 1 -- 172.21.15.5:0/1386538159 >> 172.21.15.5:0/1386538159 conn(0x5642ba0085f0 msgr2=0x5642ba0089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:29.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.964+0000 7effda092740 1 -- 172.21.15.5:0/1386538159 shutdown_connections 2024-09-15T22:14:29.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.964+0000 7effda092740 1 -- 172.21.15.5:0/1386538159 wait complete. 2024-09-15T22:14:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.964+0000 7effda092740 1 -- start start 2024-09-15T22:14:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.964+0000 7effda092740 1 Processor -- start 2024-09-15T22:14:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.965+0000 7effda092740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c060 0x5642ba014040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.965+0000 7effda092740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba00c980 0x5642ba014580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:29.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.965+0000 7effda092740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba017240 0x5642ba014ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:29.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.965+0000 7effda092740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5642ba019e60 con 0x5642ba017240 2024-09-15T22:14:29.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.965+0000 7effda092740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5642b9ef0900 con 0x5642ba00c060 2024-09-15T22:14:29.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.965+0000 7effda092740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5642ba019fe0 con 0x5642ba00c980 2024-09-15T22:14:29.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.965+0000 7effda092740 10 client.0 _ll_register_callbacks cb 0x5642b9eee780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:29.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.965+0000 7effd5a55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c060 0x5642ba014040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:29.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.965+0000 7effd5a55640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c060 0x5642ba014040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:51712/0 (socket says 172.21.15.5:51712) 2024-09-15T22:14:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.965+0000 7effd5a55640 1 -- 172.21.15.5:0/3840621387 learned_addr learned my addr 172.21.15.5:0/3840621387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effd5254640 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba00c980 0x5642ba014580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effd5a55640 1 -- 172.21.15.5:0/3840621387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba00c980 msgr2=0x5642ba014580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effd5a55640 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba00c980 0x5642ba014580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effd6256640 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba017240 0x5642ba014ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effd5a55640 1 -- 172.21.15.5:0/3840621387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba017240 msgr2=0x5642ba014ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effd5a55640 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba017240 0x5642ba014ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effd5a55640 1 -- 172.21.15.5:0/3840621387 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642b9f30c50 con 0x5642ba00c060 2024-09-15T22:14:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effd6256640 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba017240 0x5642ba014ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effd5a55640 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c060 0x5642ba014040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7effcc01c040 tx=0x7effcc009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6a6c17bc1397c74 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:14:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effb97fa640 1 -- 172.21.15.5:0/3840621387 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc01e040 con 0x5642ba00c060 2024-09-15T22:14:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effb97fa640 1 -- 172.21.15.5:0/3840621387 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7effcc0035f0 con 0x5642ba00c060 2024-09-15T22:14:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.966+0000 7effda092740 1 -- 172.21.15.5:0/3840621387 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effcc009de0 con 0x5642ba00c060 2024-09-15T22:14:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.967+0000 7effb97fa640 1 -- 172.21.15.5:0/3840621387 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7effcc032050 con 0x5642ba00c060 2024-09-15T22:14:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.967+0000 7effda092740 10 client.26087 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.967+0000 7effda092740 1 -- 172.21.15.5:0/3840621387 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5642ba025a80 con 0x5642ba00c060 2024-09-15T22:14:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.967+0000 7effb97fa640 1 -- 172.21.15.5:0/3840621387 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effcc009de0 con 0x5642ba00c060 2024-09-15T22:14:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.967+0000 7effb97fa640 1 -- 172.21.15.5:0/3840621387 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7effcc021030 con 0x5642ba00c060 2024-09-15T22:14:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.967+0000 7effda092740 10 client.26087 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.967+0000 7effda092740 10 client.26087 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:14:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.967+0000 7effda092740 20 client.26087 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.967+0000 7effda092740 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642ba027270 0x5642ba029710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.967+0000 7effda092740 4 client.26087 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.967+0000 7effda092740 1 -- 172.21.15.5:0/3840621387 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642b9f75890 con 0x5642ba027270 2024-09-15T22:14:29.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.968+0000 7effd5254640 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642ba027270 0x5642ba029710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:29.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.968+0000 7effd5254640 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642ba027270 0x5642ba029710 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b4368bc6015199b server_cookie=658296df2af8bf5 in_seq=0 out_seq=0 2024-09-15T22:14:29.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.968+0000 7effb97fa640 10 client.26087 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:29.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effb97fa640 1 -- 172.21.15.5:0/3840621387 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642b9f75890 con 0x5642ba027270 2024-09-15T22:14:29.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effb97fa640 10 client.26087 handle_command_reply: tid=0 2024-09-15T22:14:29.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effda092740 1 client.26087 shutdown 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effda092740 1 -- 172.21.15.5:0/3840621387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c060 msgr2=0x5642ba014040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effda092740 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c060 0x5642ba014040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7effcc01c040 tx=0x7effcc009700 comp rx=0 tx=0).stop 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effda092740 1 -- 172.21.15.5:0/3840621387 shutdown_connections 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effda092740 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba017240 0x5642ba014ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effda092740 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba00c980 0x5642ba014580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effda092740 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642ba027270 0x5642ba029710 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effda092740 1 --2- 172.21.15.5:0/3840621387 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c060 0x5642ba014040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effda092740 1 -- 172.21.15.5:0/3840621387 >> 172.21.15.5:0/3840621387 conn(0x5642ba0085f0 msgr2=0x5642b9ffeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effb97fa640 0 client.26087 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.970+0000 7effda092740 1 -- 172.21.15.5:0/3840621387 shutdown_connections 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.971+0000 7effda092740 1 -- 172.21.15.5:0/3840621387 wait complete. 2024-09-15T22:14:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:29.971+0000 7effda092740 20 client.26087 trim_cache size 0 max 16384 2024-09-15T22:14:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.004+0000 7effda092740 1 Processor -- start 2024-09-15T22:14:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.008+0000 7effda092740 1 -- start start 2024-09-15T22:14:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.009+0000 7effda092740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba00c060 0x5642ba00eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.009+0000 7effda092740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642b9ffed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.009+0000 7effda092740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba017240 0x5642b9fff260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.009+0000 7effda092740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7effb40127b0 con 0x5642ba017240 2024-09-15T22:14:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.009+0000 7effda092740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effb4010080 con 0x5642ba00c980 2024-09-15T22:14:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.009+0000 7effda092740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7effb400ded0 con 0x5642ba00c060 2024-09-15T22:14:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.010+0000 7effd5254640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642b9ffed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.010+0000 7effd5254640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642b9ffed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:51718/0 (socket says 172.21.15.5:51718) 2024-09-15T22:14:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.010+0000 7effd5254640 1 -- 172.21.15.5:0/2646879196 learned_addr learned my addr 172.21.15.5:0/2646879196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.010+0000 7effd5a55640 1 --2- 172.21.15.5:0/2646879196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba00c060 0x5642ba00eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.010+0000 7effd6256640 1 --2- 172.21.15.5:0/2646879196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba017240 0x5642b9fff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.010+0000 7effd5254640 1 -- 172.21.15.5:0/2646879196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba00c060 msgr2=0x5642ba00eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.010+0000 7effd5254640 1 --2- 172.21.15.5:0/2646879196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba00c060 0x5642ba00eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.010+0000 7effd5254640 1 -- 172.21.15.5:0/2646879196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba017240 msgr2=0x5642b9fff260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.010+0000 7effd5254640 1 --2- 172.21.15.5:0/2646879196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba017240 0x5642b9fff260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.010+0000 7effd5254640 1 -- 172.21.15.5:0/2646879196 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642ba00c7b0 con 0x5642ba00c980 2024-09-15T22:14:30.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.011+0000 7effd5254640 1 --2- 172.21.15.5:0/2646879196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642b9ffed20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effc0000c00 tx=0x7effc0009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a40fa50d6e13197 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:30.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.011+0000 7effbb7fe640 1 -- 172.21.15.5:0/2646879196 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc00a0280 con 0x5642ba00c980 2024-09-15T22:14:30.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.012+0000 7effbb7fe640 1 -- 172.21.15.5:0/2646879196 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc009fb80 con 0x5642ba00c980 2024-09-15T22:14:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.013+0000 7effda092740 1 -- 172.21.15.5:0/2646879196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 msgr2=0x5642b9ffed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.013+0000 7effda092740 1 --2- 172.21.15.5:0/2646879196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642b9ffed20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7effc0000c00 tx=0x7effc0009280 comp rx=0 tx=0).stop 2024-09-15T22:14:30.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.017+0000 7effda092740 1 -- 172.21.15.5:0/2646879196 shutdown_connections 2024-09-15T22:14:30.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.017+0000 7effda092740 1 --2- 172.21.15.5:0/2646879196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba017240 0x5642b9fff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.017+0000 7effda092740 1 --2- 172.21.15.5:0/2646879196 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642b9ffed20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.017+0000 7effda092740 1 --2- 172.21.15.5:0/2646879196 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba00c060 0x5642ba00eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.017+0000 7effda092740 1 -- 172.21.15.5:0/2646879196 >> 172.21.15.5:0/2646879196 conn(0x5642ba0085f0 msgr2=0x5642b9ffe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:30.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.017+0000 7effda092740 1 -- 172.21.15.5:0/2646879196 shutdown_connections 2024-09-15T22:14:30.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.021+0000 7effda092740 1 -- 172.21.15.5:0/2646879196 wait complete. 2024-09-15T22:14:30.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.021+0000 7effda092740 1 -- start start 2024-09-15T22:14:30.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.021+0000 7effda092740 1 Processor -- start 2024-09-15T22:14:30.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.022+0000 7effda092740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba00c060 0x5642ba017da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.022+0000 7effda092740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642ba0182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.022+0000 7effda092740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 0x5642ba01e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.022+0000 7effda092740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7effb400e540 con 0x5642ba00c060 2024-09-15T22:14:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.022+0000 7effda092740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7effb400f9b0 con 0x5642ba00c980 2024-09-15T22:14:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.022+0000 7effda092740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7effb400ec10 con 0x5642ba017240 2024-09-15T22:14:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.022+0000 7effda092740 10 client.0 _ll_register_callbacks cb 0x7effcc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:30.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.023+0000 7effd6256640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 0x5642ba01e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.024+0000 7effd6256640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 0x5642ba01e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:46158/0 (socket says 172.21.15.5:46158) 2024-09-15T22:14:30.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.024+0000 7effd6256640 1 -- 172.21.15.5:0/271519228 learned_addr learned my addr 172.21.15.5:0/271519228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.024+0000 7effd6256640 1 -- 172.21.15.5:0/271519228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 msgr2=0x5642ba0182e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:14:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.025+0000 7effd6256640 1 --2- 172.21.15.5:0/271519228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642ba0182e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.025+0000 7effd6256640 1 -- 172.21.15.5:0/271519228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba00c060 msgr2=0x5642ba017da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.025+0000 7effd6256640 1 --2- 172.21.15.5:0/271519228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba00c060 0x5642ba017da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.025+0000 7effd6256640 1 -- 172.21.15.5:0/271519228 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ba01e850 con 0x5642ba017240 2024-09-15T22:14:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.025+0000 7effd6256640 1 --2- 172.21.15.5:0/271519228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 0x5642ba01e310 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7effc8033000 tx=0x7effc802a750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c64f8f3e09637e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.025+0000 7effb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:14:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.025+0000 7effb97fa640 1 -- 172.21.15.5:0/271519228 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8036040 con 0x5642ba017240 2024-09-15T22:14:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.025+0000 7effda092740 1 -- 172.21.15.5:0/271519228 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc009fb80 con 0x5642ba017240 2024-09-15T22:14:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.025+0000 7effb97fa640 1 -- 172.21.15.5:0/271519228 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7effc8004450 con 0x5642ba017240 2024-09-15T22:14:30.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.026+0000 7effb97fa640 1 -- 172.21.15.5:0/271519228 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7effc805e050 con 0x5642ba017240 2024-09-15T22:14:30.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.026+0000 7effda092740 10 client.26050 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:30.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.026+0000 7effda092740 1 -- 172.21.15.5:0/271519228 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5642ba01fb00 con 0x5642ba017240 2024-09-15T22:14:30.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.030+0000 7effb97fa640 1 -- 172.21.15.5:0/271519228 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc009fb80 con 0x5642ba017240 2024-09-15T22:14:30.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.030+0000 7effb97fa640 1 -- 172.21.15.5:0/271519228 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7effc803f480 con 0x5642ba017240 2024-09-15T22:14:30.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.031+0000 7effda092740 10 client.26050 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:30.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.031+0000 7effda092740 10 client.26050 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:14:30.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.031+0000 7effda092740 20 client.26050 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:30.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.031+0000 7effda092740 1 --2- 172.21.15.5:0/271519228 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642ba0a3180 0x5642ba0a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.031+0000 7effda092740 4 client.26050 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}] 2024-09-15T22:14:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.031+0000 7effda092740 1 -- 172.21.15.5:0/271519228 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.35, "await_for": 45.0}) -- 0x5642b9f6fbe0 con 0x5642ba0a3180 2024-09-15T22:14:30.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.031+0000 7effd5a55640 1 --2- 172.21.15.5:0/271519228 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642ba0a3180 0x5642ba0a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:30.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.036+0000 7effd5a55640 1 --2- 172.21.15.5:0/271519228 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642ba0a3180 0x5642ba0a55b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f590b70c89841d23 server_cookie=9ead477889adcde in_seq=0 out_seq=0 2024-09-15T22:14:30.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.039+0000 7effb97fa640 10 client.26050 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.348+0000 7effb97fa640 1 -- 172.21.15.5:0/271519228 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5642b9f6fbe0 con 0x5642ba0a3180 2024-09-15T22:14:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.348+0000 7effb97fa640 10 client.26050 handle_command_reply: tid=0 2024-09-15T22:14:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.348+0000 7effda092740 1 client.26050 shutdown 2024-09-15T22:14:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.348+0000 7effda092740 1 -- 172.21.15.5:0/271519228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 msgr2=0x5642ba01e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.348+0000 7effda092740 1 --2- 172.21.15.5:0/271519228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 0x5642ba01e310 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7effc8033000 tx=0x7effc802a750 comp rx=0 tx=0).stop 2024-09-15T22:14:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effda092740 1 -- 172.21.15.5:0/271519228 shutdown_connections 2024-09-15T22:14:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effda092740 1 --2- 172.21.15.5:0/271519228 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5642ba017240 0x5642ba01e310 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effda092740 1 --2- 172.21.15.5:0/271519228 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642ba0a3180 0x5642ba0a55b0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effda092740 1 --2- 172.21.15.5:0/271519228 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5642ba00c980 0x5642ba0182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effda092740 1 --2- 172.21.15.5:0/271519228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642ba00c060 0x5642ba017da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effb97fa640 0 client.26050 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effda092740 1 -- 172.21.15.5:0/271519228 >> 172.21.15.5:0/271519228 conn(0x5642ba0085f0 msgr2=0x5642b9ffee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effda092740 1 -- 172.21.15.5:0/271519228 shutdown_connections 2024-09-15T22:14:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effda092740 1 -- 172.21.15.5:0/271519228 wait complete. 2024-09-15T22:14:30.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effda092740 20 client.26050 trim_cache size 0 max 16384 2024-09-15T22:14:30.352 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:14:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:14:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:14:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 22, 2024-09-15T22:14:30.352 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "b5407ac4": { 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 22, 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 145.3, 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:14:30.353 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:14:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:14:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:14:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:30.354 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:30.354 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:14:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effd7cb9640 1 -- 172.21.15.5:0/44471139 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7effbc0a34b0 msgr2=0x7effbc0a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effd7cb9640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7effbc0a34b0 0x7effbc0a5970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7effc8005cc0 tx=0x7effc8009840 comp rx=0 tx=0).stop 2024-09-15T22:14:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effd7cb9640 1 -- 172.21.15.5:0/44471139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd0103650 msgr2=0x7effd01967c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effd7cb9640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd0103650 0x7effd01967c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7effc0012b80 tx=0x7effc0012f80 comp rx=0 tx=0).stop 2024-09-15T22:14:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effd7cb9640 1 -- 172.21.15.5:0/44471139 shutdown_connections 2024-09-15T22:14:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effd7cb9640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7effbc0a34b0 0x7effbc0a5970 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effd7cb9640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd010c160 0x7effd019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effd7cb9640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7effd0103650 0x7effd01967c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effd7cb9640 1 --2- 172.21.15.5:0/44471139 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7effd0102d30 0x7effd0196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.349+0000 7effd7cb9640 1 -- 172.21.15.5:0/44471139 >> 172.21.15.5:0/44471139 conn(0x7effd00fc790 msgr2=0x7effd00fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:30.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.350+0000 7effd7cb9640 1 -- 172.21.15.5:0/44471139 shutdown_connections 2024-09-15T22:14:30.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:30.350+0000 7effd7cb9640 1 -- 172.21.15.5:0/44471139 wait complete. 2024-09-15T22:14:30.989 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:30 smithi005 ceph-mon[30944]: pgmap v171: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 627 op/s 2024-09-15T22:14:31.022 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b5407ac4', quiesce took 5.6 seconds. Will release after: 19.75 2024-09-15T22:14:31.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:30 smithi099 ceph-mon[34892]: pgmap v171: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 627 op/s 2024-09-15T22:14:31.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:30 smithi129 ceph-mon[35314]: pgmap v171: 129 pgs: 129 active+clean; 2.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 14 MiB/s wr, 627 op/s 2024-09-15T22:14:33.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:32 smithi099 ceph-mon[34892]: pgmap v172: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2024-09-15T22:14:33.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:32 smithi129 ceph-mon[35314]: pgmap v172: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2024-09-15T22:14:33.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:32 smithi005 ceph-mon[30944]: pgmap v172: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 23 MiB/s wr, 1.06k op/s 2024-09-15T22:14:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 305.0 s, 96.3 tps, lat 83.245 ms stddev 92.577 2024-09-15T22:14:35.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:34 smithi099 ceph-mon[34892]: pgmap v173: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 17 MiB/s wr, 816 op/s 2024-09-15T22:14:35.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:34 smithi129 ceph-mon[35314]: pgmap v173: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 17 MiB/s wr, 816 op/s 2024-09-15T22:14:35.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:34 smithi005 ceph-mon[30944]: pgmap v173: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 17 MiB/s wr, 816 op/s 2024-09-15T22:14:37.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:36 smithi099 ceph-mon[34892]: pgmap v174: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 17 MiB/s wr, 839 op/s 2024-09-15T22:14:37.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:36 smithi129 ceph-mon[35314]: pgmap v174: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 17 MiB/s wr, 839 op/s 2024-09-15T22:14:37.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:36 smithi005 ceph-mon[30944]: pgmap v174: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 17 MiB/s wr, 839 op/s 2024-09-15T22:14:39.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:38 smithi005 ceph-mon[30944]: pgmap v175: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.9 MiB/s wr, 508 op/s 2024-09-15T22:14:39.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:38 smithi099 ceph-mon[34892]: pgmap v175: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.9 MiB/s wr, 508 op/s 2024-09-15T22:14:39.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:38 smithi129 ceph-mon[35314]: pgmap v175: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.9 MiB/s wr, 508 op/s 2024-09-15T22:14:39.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 310.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:14:41.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:40 smithi099 ceph-mon[34892]: pgmap v176: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.9 MiB/s wr, 508 op/s 2024-09-15T22:14:41.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:40 smithi129 ceph-mon[35314]: pgmap v176: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.9 MiB/s wr, 508 op/s 2024-09-15T22:14:41.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:40 smithi005 ceph-mon[30944]: pgmap v176: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 9.9 MiB/s wr, 508 op/s 2024-09-15T22:14:43.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:42 smithi099 ceph-mon[34892]: pgmap v177: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 9.9 MiB/s wr, 511 op/s 2024-09-15T22:14:43.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:42 smithi129 ceph-mon[35314]: pgmap v177: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 9.9 MiB/s wr, 511 op/s 2024-09-15T22:14:43.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:42 smithi005 ceph-mon[30944]: pgmap v177: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 9.9 MiB/s wr, 511 op/s 2024-09-15T22:14:44.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 315.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:14:45.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:44 smithi099 ceph-mon[34892]: pgmap v178: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 176 KiB/s wr, 25 op/s 2024-09-15T22:14:45.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:44 smithi129 ceph-mon[35314]: pgmap v178: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 176 KiB/s wr, 25 op/s 2024-09-15T22:14:45.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:44 smithi005 ceph-mon[30944]: pgmap v178: 129 pgs: 129 active+clean; 2.7 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 176 KiB/s wr, 25 op/s 2024-09-15T22:14:45.610 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:14:45.884 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:47.191 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:46 smithi005 ceph-mon[30944]: pgmap v179: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-15T22:14:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.189+0000 7f31a10af640 1 -- 172.21.15.5:0/1335710422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f319c102790 msgr2=0x7f319c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.189+0000 7f31a10af640 1 --2- 172.21.15.5:0/1335710422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f319c102790 0x7f319c1067e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3190009900 tx=0x7f319002a410 comp rx=0 tx=0).stop 2024-09-15T22:14:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.189+0000 7f31a10af640 1 -- 172.21.15.5:0/1335710422 shutdown_connections 2024-09-15T22:14:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.189+0000 7f31a10af640 1 --2- 172.21.15.5:0/1335710422 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f319c106f10 0x7f319c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.189+0000 7f31a10af640 1 --2- 172.21.15.5:0/1335710422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f319c102790 0x7f319c1067e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.189+0000 7f31a10af640 1 --2- 172.21.15.5:0/1335710422 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c101de0 0x7f319c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.189+0000 7f31a10af640 1 -- 172.21.15.5:0/1335710422 >> 172.21.15.5:0/1335710422 conn(0x7f319c0fd8c0 msgr2=0x7f319c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.189+0000 7f31a10af640 1 -- 172.21.15.5:0/1335710422 shutdown_connections 2024-09-15T22:14:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.189+0000 7f31a10af640 1 -- 172.21.15.5:0/1335710422 wait complete. 2024-09-15T22:14:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.190+0000 7f31a10af640 1 Processor -- start 2024-09-15T22:14:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.190+0000 7f31a10af640 1 -- start start 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.190+0000 7f31a10af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f319c101de0 0x7f319c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.190+0000 7f31a10af640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f319c102790 0x7f319c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.190+0000 7f31a10af640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106f10 0x7f319c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.190+0000 7f31a10af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f319c111f70 con 0x7f319c101de0 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.190+0000 7f31a10af640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f319c111df0 con 0x7f319c102790 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.190+0000 7f31a10af640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1120f0 con 0x7f319c106f10 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f319c101de0 0x7f319c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f319c101de0 0x7f319c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58270/0 (socket says 172.21.15.5:58270) 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319ad9d640 1 -- 172.21.15.5:0/2188218142 learned_addr learned my addr 172.21.15.5:0/2188218142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319a59c640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f319c102790 0x7f319c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319ad9d640 1 -- 172.21.15.5:0/2188218142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106f10 msgr2=0x7f319c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319b59e640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106f10 0x7f319c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319ad9d640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106f10 0x7f319c19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319ad9d640 1 -- 172.21.15.5:0/2188218142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f319c102790 msgr2=0x7f319c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319ad9d640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f319c102790 0x7f319c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319ad9d640 1 -- 172.21.15.5:0/2188218142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f319c19d460 con 0x7f319c101de0 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319a59c640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f319c102790 0x7f319c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f319ad9d640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f319c101de0 0x7f319c1984b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f318c00d770 tx=0x7f318c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f3183fff640 1 -- 172.21.15.5:0/2188218142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c017070 con 0x7f319c101de0 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f31a10af640 1 -- 172.21.15.5:0/2188218142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f319c19d750 con 0x7f319c101de0 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f3183fff640 1 -- 172.21.15.5:0/2188218142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c005500 con 0x7f319c101de0 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.191+0000 7f3183fff640 1 -- 172.21.15.5:0/2188218142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c004e00 con 0x7f319c101de0 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.192+0000 7f31a10af640 1 -- 172.21.15.5:0/2188218142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f319c1a5030 con 0x7f319c101de0 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.194+0000 7f3183fff640 1 -- 172.21.15.5:0/2188218142 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f318c004fa0 con 0x7f319c101de0 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.194+0000 7f31a3488740 1 Processor -- start 2024-09-15T22:14:47.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.194+0000 7f31a3488740 1 -- start start 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f31a3488740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561c0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f31a3488740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561c0dc0 0x55e8561cb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f31a3488740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561cb680 0x55e8561cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f31a3488740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e8561262e0 con 0x55e8561c04a0 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f31a3488740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e8560cdce0 con 0x55e8561c0dc0 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f31a3488740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e8561222a0 con 0x55e8561cb680 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561c0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561c0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58282/0 (socket says 172.21.15.5:58282) 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319b59e640 1 -- 172.21.15.5:0/926812631 learned_addr learned my addr 172.21.15.5:0/926812631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319ad9d640 1 --2- 172.21.15.5:0/926812631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561c0dc0 0x55e8561cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f3183fff640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31840a33b0 0x7f31840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319b59e640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31840a33b0 0x7f31840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319a59c640 1 --2- 172.21.15.5:0/926812631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561cb680 0x55e8561cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319b59e640 1 -- 172.21.15.5:0/926812631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561cb680 msgr2=0x55e8561cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319b59e640 1 --2- 172.21.15.5:0/926812631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561cb680 0x55e8561cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319b59e640 1 -- 172.21.15.5:0/926812631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561c0dc0 msgr2=0x55e8561cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319b59e640 1 --2- 172.21.15.5:0/926812631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561c0dc0 0x55e8561cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319b59e640 1 -- 172.21.15.5:0/926812631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8560b2440 con 0x55e8561c04a0 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f3183fff640 1 -- 172.21.15.5:0/2188218142 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f318c0cfbf0 con 0x7f319c101de0 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.195+0000 7f319b59e640 1 --2- 172.21.15.5:0/926812631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561c0880 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3194013840 tx=0x7f3194013d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d89177ab0a62119 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.196+0000 7f31817fa640 1 -- 172.21.15.5:0/926812631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194013250 con 0x55e8561c04a0 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.196+0000 7f31817fa640 1 -- 172.21.15.5:0/926812631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3194003e50 con 0x55e8561c04a0 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.196+0000 7f31817fa640 1 -- 172.21.15.5:0/926812631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3194004800 con 0x55e8561c04a0 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.196+0000 7f31a3488740 1 -- 172.21.15.5:0/926812631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 msgr2=0x55e8561c0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.196+0000 7f319b59e640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31840a33b0 0x7f31840a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3194020000 tx=0x7f319401e000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:47.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.196+0000 7f31a3488740 1 --2- 172.21.15.5:0/926812631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561c0880 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3194013840 tx=0x7f3194013d00 comp rx=0 tx=0).stop 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 -- 172.21.15.5:0/926812631 shutdown_connections 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 --2- 172.21.15.5:0/926812631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561cb680 0x55e8561cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 --2- 172.21.15.5:0/926812631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561c0dc0 0x55e8561cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 --2- 172.21.15.5:0/926812631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561c0880 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 -- 172.21.15.5:0/926812631 >> 172.21.15.5:0/926812631 conn(0x55e8561bca30 msgr2=0x55e8561bce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 -- 172.21.15.5:0/926812631 shutdown_connections 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 -- 172.21.15.5:0/926812631 wait complete. 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 -- start start 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 Processor -- start 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561caf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c0dc0 0x55e8561c8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561cb680 0x55e8561c8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e8561ce250 con 0x55e8561c04a0 2024-09-15T22:14:47.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e8560e1050 con 0x55e8561cb680 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e8561ce3d0 con 0x55e8561c0dc0 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f31a3488740 10 client.0 _ll_register_callbacks cb 0x55e856121160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f319ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f319ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58288/0 (socket says 172.21.15.5:58288) 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.197+0000 7f319ad9d640 1 -- 172.21.15.5:0/2994377060 learned_addr learned my addr 172.21.15.5:0/2994377060 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f319a59c640 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c0dc0 0x55e8561c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f319b59e640 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561cb680 0x55e8561c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f319ad9d640 1 -- 172.21.15.5:0/2994377060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c0dc0 msgr2=0x55e8561c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f319ad9d640 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c0dc0 0x55e8561c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f319ad9d640 1 -- 172.21.15.5:0/2994377060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561cb680 msgr2=0x55e8561c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f319ad9d640 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561cb680 0x55e8561c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f319ad9d640 1 -- 172.21.15.5:0/2994377060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e855e973c0 con 0x55e8561c04a0 2024-09-15T22:14:47.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f319ad9d640 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561caf70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f318c0cd000 tx=0x7f318c0048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d3d3a7ac8a8aea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f317eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f317eff5640 1 -- 172.21.15.5:0/2994377060 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c09ce60 con 0x55e8561c04a0 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f317eff5640 1 -- 172.21.15.5:0/2994377060 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f318c0cd220 con 0x55e8561c04a0 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f31a3488740 1 -- 172.21.15.5:0/2994377060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3194003e50 con 0x55e8561c04a0 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f317eff5640 1 -- 172.21.15.5:0/2994377060 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f318c020040 con 0x55e8561c04a0 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f31a3488740 10 client.16539 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.198+0000 7f31a3488740 1 -- 172.21.15.5:0/2994377060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8561d9c10 con 0x55e8561c04a0 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.199+0000 7f317eff5640 1 -- 172.21.15.5:0/2994377060 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3194003e50 con 0x55e8561c04a0 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.199+0000 7f317eff5640 1 -- 172.21.15.5:0/2994377060 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f318c0981a0 con 0x55e8561c04a0 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.199+0000 7f31a3488740 10 client.16539 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.199+0000 7f31a3488740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.199+0000 7f31a3488740 20 client.16539 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:47.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.199+0000 7f31a3488740 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e8561db020 0x55e8561dd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.199+0000 7f31a3488740 4 client.16539 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.199+0000 7f31a3488740 1 -- 172.21.15.5:0/2994377060 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e856133850 con 0x55e8561db020 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.199+0000 7f319a59c640 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e8561db020 0x55e8561dd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.199+0000 7f319a59c640 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e8561db020 0x55e8561dd490 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11262163781df1c server_cookie=d815142b3f6d8536 in_seq=0 out_seq=0 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.199+0000 7f317eff5640 10 client.16539 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f317eff5640 1 -- 172.21.15.5:0/2994377060 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e856133850 con 0x55e8561db020 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f317eff5640 10 client.16539 handle_command_reply: tid=0 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f31a3488740 1 client.16539 shutdown 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f31a3488740 1 -- 172.21.15.5:0/2994377060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 msgr2=0x55e8561caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f31a3488740 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561caf70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f318c0cd000 tx=0x7f318c0048a0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f31a3488740 1 -- 172.21.15.5:0/2994377060 shutdown_connections 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f31a3488740 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e8561db020 0x55e8561dd490 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f31a3488740 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561cb680 0x55e8561c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f31a3488740 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c0dc0 0x55e8561c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f31a3488740 1 --2- 172.21.15.5:0/2994377060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561caf70 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f31a3488740 1 -- 172.21.15.5:0/2994377060 >> 172.21.15.5:0/2994377060 conn(0x55e8561bca30 msgr2=0x55e8561c2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.200+0000 7f317eff5640 0 client.16539 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.201+0000 7f31a3488740 1 -- 172.21.15.5:0/2994377060 shutdown_connections 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.201+0000 7f31a3488740 1 -- 172.21.15.5:0/2994377060 wait complete. 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.201+0000 7f31a3488740 20 client.16539 trim_cache size 0 max 16384 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.219+0000 7f31a3488740 1 Processor -- start 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.219+0000 7f31a3488740 1 -- start start 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.219+0000 7f31a3488740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561bca30 0x55e8561bba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.219+0000 7f31a3488740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c04a0 0x55e8561e2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.219+0000 7f31a3488740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561c0dc0 0x55e8561e3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.219+0000 7f31a3488740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1b17c0 con 0x55e8561bca30 2024-09-15T22:14:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.219+0000 7f31a3488740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1af090 con 0x55e8561c0dc0 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.219+0000 7f31a3488740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1acee0 con 0x55e8561c04a0 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.219+0000 7f319ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561bca30 0x55e8561bba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.219+0000 7f319ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561bca30 0x55e8561bba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58304/0 (socket says 172.21.15.5:58304) 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.219+0000 7f319ad9d640 1 -- 172.21.15.5:0/1944501537 learned_addr learned my addr 172.21.15.5:0/1944501537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.220+0000 7f319b59e640 1 --2- 172.21.15.5:0/1944501537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561c0dc0 0x55e8561e3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.220+0000 7f319a59c640 1 --2- 172.21.15.5:0/1944501537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c04a0 0x55e8561e2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.220+0000 7f319ad9d640 1 -- 172.21.15.5:0/1944501537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c04a0 msgr2=0x55e8561e2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.220+0000 7f319ad9d640 1 --2- 172.21.15.5:0/1944501537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c04a0 0x55e8561e2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.220+0000 7f319ad9d640 1 -- 172.21.15.5:0/1944501537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561c0dc0 msgr2=0x55e8561e3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.220+0000 7f319ad9d640 1 --2- 172.21.15.5:0/1944501537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561c0dc0 0x55e8561e3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.220+0000 7f319ad9d640 1 -- 172.21.15.5:0/1944501537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8561c02d0 con 0x55e8561bca30 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.220+0000 7f319ad9d640 1 --2- 172.21.15.5:0/1944501537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561bca30 0x55e8561bba60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f318c09b830 tx=0x7f318c09af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb15fea8d5afeb8a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:47.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.220+0000 7f3180ff9640 1 -- 172.21.15.5:0/1944501537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c014070 con 0x55e8561bca30 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.220+0000 7f3180ff9640 1 -- 172.21.15.5:0/1944501537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c0b9070 con 0x55e8561bca30 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.220+0000 7f3180ff9640 1 -- 172.21.15.5:0/1944501537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c0a47a0 con 0x55e8561bca30 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 -- 172.21.15.5:0/1944501537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561bca30 msgr2=0x55e8561bba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 --2- 172.21.15.5:0/1944501537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561bca30 0x55e8561bba60 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f318c09b830 tx=0x7f318c09af80 comp rx=0 tx=0).stop 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 -- 172.21.15.5:0/1944501537 shutdown_connections 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 --2- 172.21.15.5:0/1944501537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561c0dc0 0x55e8561e3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 --2- 172.21.15.5:0/1944501537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c04a0 0x55e8561e2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 --2- 172.21.15.5:0/1944501537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561bca30 0x55e8561bba60 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 -- 172.21.15.5:0/1944501537 >> 172.21.15.5:0/1944501537 conn(0x55e8561c88c0 msgr2=0x55e8561bb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 -- 172.21.15.5:0/1944501537 shutdown_connections 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 -- 172.21.15.5:0/1944501537 wait complete. 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 -- start start 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 Processor -- start 2024-09-15T22:14:47.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561bca30 0x55e8561d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c0dc0 0x55e8561d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ad550 con 0x55e8561c04a0 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1ae9c0 con 0x55e8561bca30 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.221+0000 7f31a3488740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f319c1adc20 con 0x55e8561c0dc0 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f31a3488740 10 client.0 _ll_register_callbacks cb 0x7f318c09b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561d0ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58318/0 (socket says 172.21.15.5:58318) 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319a59c640 1 -- 172.21.15.5:0/2328699204 learned_addr learned my addr 172.21.15.5:0/2328699204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319ad9d640 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561bca30 0x55e8561d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319b59e640 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c0dc0 0x55e8561d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319a59c640 1 -- 172.21.15.5:0/2328699204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c0dc0 msgr2=0x55e8561d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:47.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319a59c640 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c0dc0 0x55e8561d1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319a59c640 1 -- 172.21.15.5:0/2328699204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561bca30 msgr2=0x55e8561d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319a59c640 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561bca30 0x55e8561d3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319a59c640 1 -- 172.21.15.5:0/2328699204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8561dd800 con 0x55e8561c04a0 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319b59e640 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c0dc0 0x55e8561d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f319a59c640 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561d0ed0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3190009900 tx=0x7f3190009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a6deb41435d222 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f317eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f317eff5640 1 -- 172.21.15.5:0/2328699204 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3190035040 con 0x55e8561c04a0 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f317eff5640 1 -- 172.21.15.5:0/2328699204 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f3190005040 con 0x55e8561c04a0 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f31a3488740 1 -- 172.21.15.5:0/2328699204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f318c0b9070 con 0x55e8561c04a0 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f317eff5640 1 -- 172.21.15.5:0/2328699204 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3190051050 con 0x55e8561c04a0 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.222+0000 7f31a3488740 10 client.16545 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.223+0000 7f31a3488740 1 -- 172.21.15.5:0/2328699204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e8562459b0 con 0x55e8561c04a0 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.223+0000 7f317eff5640 1 -- 172.21.15.5:0/2328699204 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f318c0b9070 con 0x55e8561c04a0 2024-09-15T22:14:47.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.223+0000 7f317eff5640 1 -- 172.21.15.5:0/2328699204 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f319004c030 con 0x55e8561c04a0 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.223+0000 7f31a3488740 10 client.16545 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.223+0000 7f31a3488740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.223+0000 7f31a3488740 20 client.16545 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.223+0000 7f31a3488740 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e8561cb680 0x55e8562481f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.223+0000 7f31a3488740 4 client.16545 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.223+0000 7f31a3488740 1 -- 172.21.15.5:0/2328699204 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e85612dd60 con 0x55e8561cb680 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.223+0000 7f319ad9d640 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e8561cb680 0x55e8562481f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f319ad9d640 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e8561cb680 0x55e8562481f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6920d3392f94ed8e server_cookie=7e8a8cb37a0e506e in_seq=0 out_seq=0 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f317eff5640 10 client.16545 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f317eff5640 1 -- 172.21.15.5:0/2328699204 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e85612dd60 con 0x55e8561cb680 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f317eff5640 10 client.16545 handle_command_reply: tid=0 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f31a3488740 1 client.16545 shutdown 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f31a3488740 1 -- 172.21.15.5:0/2328699204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 msgr2=0x55e8561d0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:47.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f31a3488740 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561d0ed0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3190009900 tx=0x7f3190009670 comp rx=0 tx=0).stop 2024-09-15T22:14:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f31a3488740 1 -- 172.21.15.5:0/2328699204 shutdown_connections 2024-09-15T22:14:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f31a3488740 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e8561c0dc0 0x55e8561d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f31a3488740 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8561c04a0 0x55e8561d0ed0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f31a3488740 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e8561cb680 0x55e8562481f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f31a3488740 1 --2- 172.21.15.5:0/2328699204 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e8561bca30 0x55e8561d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f31a3488740 1 -- 172.21.15.5:0/2328699204 >> 172.21.15.5:0/2328699204 conn(0x55e8561c88c0 msgr2=0x55e8561bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f317eff5640 0 client.16545 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f31a3488740 1 -- 172.21.15.5:0/2328699204 shutdown_connections 2024-09-15T22:14:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.224+0000 7f31a3488740 1 -- 172.21.15.5:0/2328699204 wait complete. 2024-09-15T22:14:47.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a3488740 20 client.16545 trim_cache size 0 max 16384 2024-09-15T22:14:47.249 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:14:47.249 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:14:47.249 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:14:47.249 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:14:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 -- 172.21.15.5:0/2188218142 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31840a33b0 msgr2=0x7f31840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31840a33b0 0x7f31840a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3194020000 tx=0x7f319401e000 comp rx=0 tx=0).stop 2024-09-15T22:14:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 -- 172.21.15.5:0/2188218142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f319c101de0 msgr2=0x7f319c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f319c101de0 0x7f319c1984b0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f318c00d770 tx=0x7f318c00dc30 comp rx=0 tx=0).stop 2024-09-15T22:14:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 -- 172.21.15.5:0/2188218142 shutdown_connections 2024-09-15T22:14:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31840a33b0 0x7f31840a5870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f319c106f10 0x7f319c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f319c102790 0x7f319c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 --2- 172.21.15.5:0/2188218142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f319c101de0 0x7f319c1984b0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 -- 172.21.15.5:0/2188218142 >> 172.21.15.5:0/2188218142 conn(0x7f319c0fd8c0 msgr2=0x7f319c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 -- 172.21.15.5:0/2188218142 shutdown_connections 2024-09-15T22:14:47.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:47.225+0000 7f31a10af640 1 -- 172.21.15.5:0/2188218142 wait complete. 2024-09-15T22:14:47.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:46 smithi099 ceph-mon[34892]: pgmap v179: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-15T22:14:47.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:46 smithi129 ceph-mon[35314]: pgmap v179: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-15T22:14:48.540 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:14:48.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:14:48.803 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:49.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:48 smithi005 ceph-mon[30944]: pgmap v180: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-15T22:14:49.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.325+0000 7f676690c640 1 -- 172.21.15.5:0/2422005795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6760106f30 msgr2=0x7f676010eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:49.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.325+0000 7f676690c640 1 --2- 172.21.15.5:0/2422005795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6760106f30 0x7f676010eab0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6758009ca0 tx=0x7f675802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.325+0000 7f676690c640 1 -- 172.21.15.5:0/2422005795 shutdown_connections 2024-09-15T22:14:49.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.325+0000 7f676690c640 1 --2- 172.21.15.5:0/2422005795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6760106f30 0x7f676010eab0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.325+0000 7f676690c640 1 --2- 172.21.15.5:0/2422005795 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6760100cc0 0x7f6760106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.325+0000 7f676690c640 1 --2- 172.21.15.5:0/2422005795 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6760100310 0x7f67601006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.325+0000 7f676690c640 1 -- 172.21.15.5:0/2422005795 >> 172.21.15.5:0/2422005795 conn(0x7f6760077210 msgr2=0x7f67600fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:49.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.325+0000 7f676690c640 1 -- 172.21.15.5:0/2422005795 shutdown_connections 2024-09-15T22:14:49.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.325+0000 7f676690c640 1 -- 172.21.15.5:0/2422005795 wait complete. 2024-09-15T22:14:49.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.325+0000 7f676690c640 1 Processor -- start 2024-09-15T22:14:49.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f676690c640 1 -- start start 2024-09-15T22:14:49.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f676690c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6760100310 0x7f6760198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f676690c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6760100cc0 0x7f6760198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f676690c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6760106f30 0x7f676019cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f676690c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6760111fb0 con 0x7f6760100310 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f676690c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6760111e30 con 0x7f6760100cc0 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f676690c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6760112130 con 0x7f6760106f30 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f675ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6760100310 0x7f6760198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f675ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6760100310 0x7f6760198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58338/0 (socket says 172.21.15.5:58338) 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f675ffff640 1 -- 172.21.15.5:0/1341297245 learned_addr learned my addr 172.21.15.5:0/1341297245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f6764ea9640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6760106f30 0x7f676019cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f6757fff640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6760100cc0 0x7f6760198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f675ffff640 1 -- 172.21.15.5:0/1341297245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6760106f30 msgr2=0x7f676019cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f675ffff640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6760106f30 0x7f676019cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f675ffff640 1 -- 172.21.15.5:0/1341297245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6760100cc0 msgr2=0x7f6760198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:49.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f675ffff640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6760100cc0 0x7f6760198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f675ffff640 1 -- 172.21.15.5:0/1341297245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676019d400 con 0x7f6760100310 2024-09-15T22:14:49.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f6764ea9640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6760106f30 0x7f676019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:49.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.326+0000 7f6757fff640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6760100cc0 0x7f6760198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:49.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.327+0000 7f675ffff640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6760100310 0x7f6760198450 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f675000b4a0 tx=0x7f675000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:49.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.327+0000 7f675dffb640 1 -- 172.21.15.5:0/1341297245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750014070 con 0x7f6760100310 2024-09-15T22:14:49.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.327+0000 7f675dffb640 1 -- 172.21.15.5:0/1341297245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6750010af0 con 0x7f6760100310 2024-09-15T22:14:49.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.327+0000 7f676690c640 1 -- 172.21.15.5:0/1341297245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676019d6f0 con 0x7f6760100310 2024-09-15T22:14:49.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.327+0000 7f675dffb640 1 -- 172.21.15.5:0/1341297245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750017d10 con 0x7f6760100310 2024-09-15T22:14:49.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.327+0000 7f676690c640 1 -- 172.21.15.5:0/1341297245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67601a4fd0 con 0x7f6760100310 2024-09-15T22:14:49.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.328+0000 7f676690c640 1 -- 172.21.15.5:0/1341297245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67601006f0 con 0x7f6760100310 2024-09-15T22:14:49.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.329+0000 7f675dffb640 1 -- 172.21.15.5:0/1341297245 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6750017070 con 0x7f6760100310 2024-09-15T22:14:49.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.330+0000 7f675dffb640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f674c0a3430 0x7f674c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:49.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.330+0000 7f6757fff640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f674c0a3430 0x7f674c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:49.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.330+0000 7f675dffb640 1 -- 172.21.15.5:0/1341297245 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f67500cf770 con 0x7f6760100310 2024-09-15T22:14:49.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.330+0000 7f6757fff640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f674c0a3430 0x7f674c0a58f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6760104710 tx=0x7f6748005e50 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:49.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.333+0000 7f675dffb640 1 -- 172.21.15.5:0/1341297245 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f675009bec0 con 0x7f6760100310 2024-09-15T22:14:49.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:48 smithi099 ceph-mon[34892]: pgmap v180: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-15T22:14:49.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:48 smithi129 ceph-mon[35314]: pgmap v180: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-15T22:14:49.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.544+0000 7f676690c640 1 -- 172.21.15.5:0/1341297245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f676002d7c0 con 0x7f6760100310 2024-09-15T22:14:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.545+0000 7f675dffb640 1 -- 172.21.15.5:0/1341297245 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f675009b7c0 con 0x7f6760100310 2024-09-15T22:14:49.547 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:14:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:14:49.548 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:14:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.547+0000 7f676690c640 1 -- 172.21.15.5:0/1341297245 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f674c0a3430 msgr2=0x7f674c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:49.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.547+0000 7f676690c640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f674c0a3430 0x7f674c0a58f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6760104710 tx=0x7f6748005e50 comp rx=0 tx=0).stop 2024-09-15T22:14:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.547+0000 7f676690c640 1 -- 172.21.15.5:0/1341297245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6760100310 msgr2=0x7f6760198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.547+0000 7f676690c640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6760100310 0x7f6760198450 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f675000b4a0 tx=0x7f675000b960 comp rx=0 tx=0).stop 2024-09-15T22:14:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.547+0000 7f676690c640 1 -- 172.21.15.5:0/1341297245 shutdown_connections 2024-09-15T22:14:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.547+0000 7f676690c640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f674c0a3430 0x7f674c0a58f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.547+0000 7f676690c640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6760106f30 0x7f676019cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.547+0000 7f676690c640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6760100cc0 0x7f6760198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.547+0000 7f676690c640 1 --2- 172.21.15.5:0/1341297245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6760100310 0x7f6760198450 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.547+0000 7f676690c640 1 -- 172.21.15.5:0/1341297245 >> 172.21.15.5:0/1341297245 conn(0x7f6760077210 msgr2=0x7f67601057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.548+0000 7f676690c640 1 -- 172.21.15.5:0/1341297245 shutdown_connections 2024-09-15T22:14:49.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:49.548+0000 7f676690c640 1 -- 172.21.15.5:0/1341297245 wait complete. 2024-09-15T22:14:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 320.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:14:50.324 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:49 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1341297245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:50.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:49 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1341297245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:50.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:49 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1341297245' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:50.357 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:14:50.620 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:50.773 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:14:51.020 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:50 smithi005 ceph-mon[30944]: pgmap v181: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-15T22:14:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.111+0000 7f6649ce7640 1 -- 172.21.15.5:0/2534129507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644103660 msgr2=0x7f664410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.111+0000 7f6649ce7640 1 --2- 172.21.15.5:0/2534129507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644103660 0x7f664410bb50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6638009900 tx=0x7f663802a480 comp rx=0 tx=0).stop 2024-09-15T22:14:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.111+0000 7f6649ce7640 1 -- 172.21.15.5:0/2534129507 shutdown_connections 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.111+0000 7f6649ce7640 1 --2- 172.21.15.5:0/2534129507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f664410c170 0x7f664410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.111+0000 7f6649ce7640 1 --2- 172.21.15.5:0/2534129507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644103660 0x7f664410bb50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.111+0000 7f6649ce7640 1 --2- 172.21.15.5:0/2534129507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6644102d40 0x7f6644103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.111+0000 7f6649ce7640 1 -- 172.21.15.5:0/2534129507 >> 172.21.15.5:0/2534129507 conn(0x7f66440fc7a0 msgr2=0x7f66440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.111+0000 7f6649ce7640 1 -- 172.21.15.5:0/2534129507 shutdown_connections 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.111+0000 7f6649ce7640 1 -- 172.21.15.5:0/2534129507 wait complete. 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.112+0000 7f6649ce7640 1 Processor -- start 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.112+0000 7f6649ce7640 1 -- start start 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.112+0000 7f6649ce7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644102d40 0x7f66441009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.112+0000 7f6649ce7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6644103660 0x7f6644100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.112+0000 7f6649ce7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f664410c170 0x7f6644101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.112+0000 7f6649ce7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6644111f70 con 0x7f6644102d40 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.112+0000 7f6649ce7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6644111df0 con 0x7f664410c170 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.112+0000 7f6649ce7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66441120f0 con 0x7f6644103660 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f66437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644102d40 0x7f66441009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f66437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644102d40 0x7f66441009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58360/0 (socket says 172.21.15.5:58360) 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f66437fe640 1 -- 172.21.15.5:0/711758675 learned_addr learned my addr 172.21.15.5:0/711758675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f6642ffd640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6644103660 0x7f6644100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f6643fff640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f664410c170 0x7f6644101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f66437fe640 1 -- 172.21.15.5:0/711758675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6644103660 msgr2=0x7f6644100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f66437fe640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6644103660 0x7f6644100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f66437fe640 1 -- 172.21.15.5:0/711758675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f664410c170 msgr2=0x7f6644101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f66437fe640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f664410c170 0x7f6644101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f66437fe640 1 -- 172.21.15.5:0/711758675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66441a6890 con 0x7f6644102d40 2024-09-15T22:14:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f66437fe640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644102d40 0x7f66441009a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f663400c920 tx=0x7f663400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f6640ff9640 1 -- 172.21.15.5:0/711758675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634007c70 con 0x7f6644102d40 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f6640ff9640 1 -- 172.21.15.5:0/711758675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6634007e10 con 0x7f6644102d40 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f6649ce7640 1 -- 172.21.15.5:0/711758675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66441a6b80 con 0x7f6644102d40 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f6640ff9640 1 -- 172.21.15.5:0/711758675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6634018890 con 0x7f6644102d40 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.113+0000 7f6649ce7640 1 -- 172.21.15.5:0/711758675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66441a7090 con 0x7f6644102d40 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.115+0000 7f6640ff9640 1 -- 172.21.15.5:0/711758675 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6634018a70 con 0x7f6644102d40 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.116+0000 7f664c0c0740 1 Processor -- start 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.116+0000 7f664c0c0740 1 -- start start 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.116+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.116+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.116+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c35370 0x556397c37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.116+0000 7f664c0c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556397b902f0 con 0x556397c2bb40 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.116+0000 7f664c0c0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556397b37cf0 con 0x556397c35370 2024-09-15T22:14:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.116+0000 7f664c0c0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556397b8c2b0 con 0x556397c2a5d0 2024-09-15T22:14:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.116+0000 7f6643fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.116+0000 7f6643fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c2a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58370/0 (socket says 172.21.15.5:58370) 2024-09-15T22:14:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.116+0000 7f6643fff640 1 -- 172.21.15.5:0/1046958770 learned_addr learned my addr 172.21.15.5:0/1046958770 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6640ff9640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 0x7f662c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6643fff640 1 -- 172.21.15.5:0/1046958770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 msgr2=0x556397c2aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6642ffd640 1 --2- 172.21.15.5:0/1046958770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c35370 0x556397c37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6643fff640 1 --2- 172.21.15.5:0/1046958770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6643fff640 1 -- 172.21.15.5:0/1046958770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c35370 msgr2=0x556397c37760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6643fff640 1 --2- 172.21.15.5:0/1046958770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c35370 0x556397c37760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6643fff640 1 -- 172.21.15.5:0/1046958770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556397b1c3a0 con 0x556397c2bb40 2024-09-15T22:14:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6640ff9640 1 -- 172.21.15.5:0/711758675 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f66340d0270 con 0x7f6644102d40 2024-09-15T22:14:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6643fff640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 0x7f662c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6643fff640 1 --2- 172.21.15.5:0/1046958770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c2a090 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f663c009fc0 tx=0x7f663c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314d3a8b454904b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6629ffb640 1 -- 172.21.15.5:0/1046958770 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c01c900 con 0x556397c2bb40 2024-09-15T22:14:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6629ffb640 1 -- 172.21.15.5:0/1046958770 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c01b3d0 con 0x556397c2bb40 2024-09-15T22:14:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.117+0000 7f6629ffb640 1 -- 172.21.15.5:0/1046958770 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c025640 con 0x556397c2bb40 2024-09-15T22:14:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.118+0000 7f664c0c0740 1 -- 172.21.15.5:0/1046958770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 msgr2=0x556397c2a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.118+0000 7f6643fff640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 0x7f662c0a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f663c020000 tx=0x7f663c01d000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.118+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1046958770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c2a090 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f663c009fc0 tx=0x7f663c010800 comp rx=0 tx=0).stop 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.118+0000 7f664c0c0740 1 -- 172.21.15.5:0/1046958770 shutdown_connections 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.118+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1046958770 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c35370 0x556397c37760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.118+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1046958770 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.118+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1046958770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c2a090 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.118+0000 7f664c0c0740 1 -- 172.21.15.5:0/1046958770 >> 172.21.15.5:0/1046958770 conn(0x556397bf3310 msgr2=0x556397c26c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.118+0000 7f664c0c0740 1 -- 172.21.15.5:0/1046958770 shutdown_connections 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.118+0000 7f664c0c0740 1 -- 172.21.15.5:0/1046958770 wait complete. 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.118+0000 7f664c0c0740 1 -- start start 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f664c0c0740 1 Processor -- start 2024-09-15T22:14:51.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c32050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c35370 0x556397c327e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f664c0c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556397c38070 con 0x556397c2bb40 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f664c0c0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556397b4b060 con 0x556397c35370 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f664c0c0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556397c381f0 con 0x556397c2a5d0 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f664c0c0740 10 client.0 _ll_register_callbacks cb 0x55639789e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c32050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c32050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58384/0 (socket says 172.21.15.5:58384) 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f6642ffd640 1 -- 172.21.15.5:0/851876389 learned_addr learned my addr 172.21.15.5:0/851876389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f6643fff640 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c35370 0x556397c327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f66437fe640 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f6642ffd640 1 -- 172.21.15.5:0/851876389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 msgr2=0x556397c34da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:51.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f6642ffd640 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c34da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.119+0000 7f6642ffd640 1 -- 172.21.15.5:0/851876389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c35370 msgr2=0x556397c327e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f6642ffd640 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c35370 0x556397c327e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f6642ffd640 1 -- 172.21.15.5:0/851876389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563979013c0 con 0x556397c2bb40 2024-09-15T22:14:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f6643fff640 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c35370 0x556397c327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f66437fe640 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c34da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f6642ffd640 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c32050 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6638009900 tx=0x7f6638031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1092c4d8652802bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f6622ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f6622ffd640 1 -- 172.21.15.5:0/851876389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638040070 con 0x556397c2bb40 2024-09-15T22:14:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f6622ffd640 1 -- 172.21.15.5:0/851876389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f6638004450 con 0x556397c2bb40 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f664c0c0740 1 -- 172.21.15.5:0/851876389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663c01b3d0 con 0x556397c2bb40 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f6622ffd640 1 -- 172.21.15.5:0/851876389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f663804d050 con 0x556397c2bb40 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f664c0c0740 10 client.16563 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f664c0c0740 1 -- 172.21.15.5:0/851876389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556397c43b50 con 0x556397c2bb40 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f6622ffd640 1 -- 172.21.15.5:0/851876389 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c01b3d0 con 0x556397c2bb40 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.120+0000 7f6622ffd640 1 -- 172.21.15.5:0/851876389 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f663803c820 con 0x556397c2bb40 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.121+0000 7f664c0c0740 10 client.16563 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.121+0000 7f664c0c0740 10 client.16563 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.121+0000 7f664c0c0740 20 client.16563 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.121+0000 7f664c0c0740 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556397c45010 0x556397c47430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.121+0000 7f664c0c0740 4 client.16563 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.121+0000 7f664c0c0740 1 -- 172.21.15.5:0/851876389 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556397b9d860 con 0x556397c45010 2024-09-15T22:14:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.121+0000 7f66437fe640 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556397c45010 0x556397c47430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.121+0000 7f66437fe640 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556397c45010 0x556397c47430 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6d8373506c033f59 server_cookie=c8839d954fb22a3c in_seq=0 out_seq=0 2024-09-15T22:14:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.121+0000 7f6622ffd640 10 client.16563 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:14:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f6622ffd640 1 -- 172.21.15.5:0/851876389 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556397b9d860 con 0x556397c45010 2024-09-15T22:14:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f6622ffd640 10 client.16563 handle_command_reply: tid=0 2024-09-15T22:14:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f664c0c0740 1 client.16563 shutdown 2024-09-15T22:14:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f664c0c0740 1 -- 172.21.15.5:0/851876389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 msgr2=0x556397c32050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f664c0c0740 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c32050 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6638009900 tx=0x7f6638031850 comp rx=0 tx=0).stop 2024-09-15T22:14:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f664c0c0740 1 -- 172.21.15.5:0/851876389 shutdown_connections 2024-09-15T22:14:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f664c0c0740 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556397c45010 0x556397c47430 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f664c0c0740 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c35370 0x556397c327e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f664c0c0740 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c32050 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f6622ffd640 0 client.16563 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:14:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f664c0c0740 1 --2- 172.21.15.5:0/851876389 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c34da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f664c0c0740 1 -- 172.21.15.5:0/851876389 >> 172.21.15.5:0/851876389 conn(0x556397bf3310 msgr2=0x556397c2c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f664c0c0740 1 -- 172.21.15.5:0/851876389 shutdown_connections 2024-09-15T22:14:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.122+0000 7f664c0c0740 1 -- 172.21.15.5:0/851876389 wait complete. 2024-09-15T22:14:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.123+0000 7f664c0c0740 20 client.16563 trim_cache size 0 max 16384 2024-09-15T22:14:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.136+0000 7f664c0c0740 1 Processor -- start 2024-09-15T22:14:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.136+0000 7f664c0c0740 1 -- start start 2024-09-15T22:14:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.136+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c4cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.136+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c2bb40 0x556397c4d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.136+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c35370 0x556397c2f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.136+0000 7f664c0c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66441b3a50 con 0x556397c35370 2024-09-15T22:14:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.136+0000 7f664c0c0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66441b1320 con 0x556397c2bb40 2024-09-15T22:14:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.136+0000 7f664c0c0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66441af170 con 0x556397c2a5d0 2024-09-15T22:14:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f6643fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c35370 0x556397c2f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f66437fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c4cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f6643fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c35370 0x556397c2f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58396/0 (socket says 172.21.15.5:58396) 2024-09-15T22:14:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f66437fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c4cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:39726/0 (socket says 172.21.15.5:39726) 2024-09-15T22:14:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f6643fff640 1 -- 172.21.15.5:0/1687670275 learned_addr learned my addr 172.21.15.5:0/1687670275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f6642ffd640 1 --2- 172.21.15.5:0/1687670275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c2bb40 0x556397c4d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f66437fe640 1 -- 172.21.15.5:0/1687670275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c2bb40 msgr2=0x556397c4d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f66437fe640 1 --2- 172.21.15.5:0/1687670275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c2bb40 0x556397c4d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f66437fe640 1 -- 172.21.15.5:0/1687670275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c35370 msgr2=0x556397c2f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f66437fe640 1 --2- 172.21.15.5:0/1687670275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c35370 0x556397c2f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f66437fe640 1 -- 172.21.15.5:0/1687670275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556397c27960 con 0x556397c2a5d0 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f6643fff640 1 --2- 172.21.15.5:0/1687670275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c35370 0x556397c2f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.137+0000 7f66437fe640 1 --2- 172.21.15.5:0/1687670275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c4cce0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f66340cd5c0 tx=0x7f6634000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ded365639600eff6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.138+0000 7f66297fa640 1 -- 172.21.15.5:0/1687670275 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340a4dc0 con 0x556397c2a5d0 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.138+0000 7f66297fa640 1 -- 172.21.15.5:0/1687670275 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663409a450 con 0x556397c2a5d0 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.138+0000 7f66297fa640 1 -- 172.21.15.5:0/1687670275 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66340bc0b0 con 0x556397c2a5d0 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.138+0000 7f664c0c0740 1 -- 172.21.15.5:0/1687670275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 msgr2=0x556397c4cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.138+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1687670275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c4cce0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f66340cd5c0 tx=0x7f6634000bc0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 -- 172.21.15.5:0/1687670275 shutdown_connections 2024-09-15T22:14:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1687670275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c35370 0x556397c2f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1687670275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c2bb40 0x556397c4d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1687670275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c2a5d0 0x556397c4cce0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 -- 172.21.15.5:0/1687670275 >> 172.21.15.5:0/1687670275 conn(0x556397bf3310 msgr2=0x556397c31140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 -- 172.21.15.5:0/1687670275 shutdown_connections 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 -- 172.21.15.5:0/1687670275 wait complete. 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 -- start start 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 Processor -- start 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c2a5d0 0x556397c39b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c3a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c35370 0x556397ca8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.139+0000 7f664c0c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66441af7e0 con 0x556397c2bb40 2024-09-15T22:14:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f664c0c0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f66441b0c50 con 0x556397c2a5d0 2024-09-15T22:14:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f664c0c0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f66441afeb0 con 0x556397c35370 2024-09-15T22:14:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f664c0c0740 10 client.0 _ll_register_callbacks cb 0x7f6638009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c3a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6642ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c3a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58404/0 (socket says 172.21.15.5:58404) 2024-09-15T22:14:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6642ffd640 1 -- 172.21.15.5:0/1264216734 learned_addr learned my addr 172.21.15.5:0/1264216734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f66437fe640 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c2a5d0 0x556397c39b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6643fff640 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c35370 0x556397ca8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6642ffd640 1 -- 172.21.15.5:0/1264216734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c35370 msgr2=0x556397ca8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6642ffd640 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c35370 0x556397ca8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6642ffd640 1 -- 172.21.15.5:0/1264216734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c2a5d0 msgr2=0x556397c39b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6642ffd640 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c2a5d0 0x556397c39b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6642ffd640 1 -- 172.21.15.5:0/1264216734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556397c264f0 con 0x556397c2bb40 2024-09-15T22:14:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6643fff640 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c35370 0x556397ca8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6642ffd640 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c3a0a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f663802a9a0 tx=0x7f663803b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ba48df292486dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6622ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6622ffd640 1 -- 172.21.15.5:0/1264216734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638040040 con 0x556397c2bb40 2024-09-15T22:14:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.140+0000 7f6622ffd640 1 -- 172.21.15.5:0/1264216734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f663803c850 con 0x556397c2bb40 2024-09-15T22:14:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f664c0c0740 1 -- 172.21.15.5:0/1264216734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f663409a450 con 0x556397c2bb40 2024-09-15T22:14:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f6622ffd640 1 -- 172.21.15.5:0/1264216734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f663805f050 con 0x556397c2bb40 2024-09-15T22:14:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f664c0c0740 10 client.16569 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f664c0c0740 1 -- 172.21.15.5:0/1264216734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556397c27dd0 con 0x556397c2bb40 2024-09-15T22:14:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f6622ffd640 1 -- 172.21.15.5:0/1264216734 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663409a450 con 0x556397c2bb40 2024-09-15T22:14:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f6622ffd640 1 -- 172.21.15.5:0/1264216734 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6638050070 con 0x556397c2bb40 2024-09-15T22:14:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f664c0c0740 10 client.16569 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f664c0c0740 10 client.16569 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:14:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f664c0c0740 20 client.16569 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556397cb0a90 0x556397cb2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f664c0c0740 4 client.16569 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:14:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.141+0000 7f664c0c0740 1 -- 172.21.15.5:0/1264216734 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556397b97d70 con 0x556397cb0a90 2024-09-15T22:14:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.142+0000 7f66437fe640 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556397cb0a90 0x556397cb2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.142+0000 7f66437fe640 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556397cb0a90 0x556397cb2eb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26c2d0fae541849d server_cookie=660c80abe576a4fe in_seq=0 out_seq=0 2024-09-15T22:14:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.142+0000 7f6622ffd640 10 client.16569 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:14:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.142+0000 7f6622ffd640 1 -- 172.21.15.5:0/1264216734 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556397b97d70 con 0x556397cb0a90 2024-09-15T22:14:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.142+0000 7f6622ffd640 10 client.16569 handle_command_reply: tid=0 2024-09-15T22:14:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.142+0000 7f664c0c0740 1 client.16569 shutdown 2024-09-15T22:14:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f664c0c0740 1 -- 172.21.15.5:0/1264216734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 msgr2=0x556397c3a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c3a0a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f663802a9a0 tx=0x7f663803b3f0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f664c0c0740 1 -- 172.21.15.5:0/1264216734 shutdown_connections 2024-09-15T22:14:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556397cb0a90 0x556397cb2eb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556397c35370 0x556397ca8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556397c2bb40 0x556397c3a0a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f664c0c0740 1 --2- 172.21.15.5:0/1264216734 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556397c2a5d0 0x556397c39b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f664c0c0740 1 -- 172.21.15.5:0/1264216734 >> 172.21.15.5:0/1264216734 conn(0x556397bf3310 msgr2=0x556397c254e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f6622ffd640 0 client.16569 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:14:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f664c0c0740 1 -- 172.21.15.5:0/1264216734 shutdown_connections 2024-09-15T22:14:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f664c0c0740 1 -- 172.21.15.5:0/1264216734 wait complete. 2024-09-15T22:14:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f664c0c0740 20 client.16569 trim_cache size 0 max 16384 2024-09-15T22:14:51.152 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:14:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f6649ce7640 1 -- 172.21.15.5:0/711758675 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 msgr2=0x7f662c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f6649ce7640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 0x7f662c0a5870 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f663c020000 tx=0x7f663c01d000 comp rx=0 tx=0).stop 2024-09-15T22:14:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f6649ce7640 1 -- 172.21.15.5:0/711758675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644102d40 msgr2=0x7f66441009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.143+0000 7f6649ce7640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644102d40 0x7f66441009a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f663400c920 tx=0x7f663400cde0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.144+0000 7f6649ce7640 1 -- 172.21.15.5:0/711758675 shutdown_connections 2024-09-15T22:14:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.144+0000 7f6649ce7640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f662c0a33b0 0x7f662c0a5870 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.144+0000 7f6649ce7640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f664410c170 0x7f6644101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.144+0000 7f6649ce7640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6644103660 0x7f6644100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.144+0000 7f6649ce7640 1 --2- 172.21.15.5:0/711758675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6644102d40 0x7f66441009a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.144+0000 7f6649ce7640 1 -- 172.21.15.5:0/711758675 >> 172.21.15.5:0/711758675 conn(0x7f66440fc7a0 msgr2=0x7f66440fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.144+0000 7f6649ce7640 1 -- 172.21.15.5:0/711758675 shutdown_connections 2024-09-15T22:14:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:51.144+0000 7f6649ce7640 1 -- 172.21.15.5:0/711758675 wait complete. 2024-09-15T22:14:51.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:50 smithi099 ceph-mon[34892]: pgmap v181: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-15T22:14:51.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:50 smithi129 ceph-mon[35314]: pgmap v181: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-15T22:14:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:51.861 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:14:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.350+0000 7f499d61a640 1 -- 172.21.15.5:0/2186010490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499810c170 msgr2=0x7f499810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.350+0000 7f499d61a640 1 --2- 172.21.15.5:0/2186010490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499810c170 0x7f499810e610 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4990009ca0 tx=0x7f499002a950 comp rx=0 tx=0).stop 2024-09-15T22:14:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.350+0000 7f499d61a640 1 -- 172.21.15.5:0/2186010490 shutdown_connections 2024-09-15T22:14:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.350+0000 7f499d61a640 1 --2- 172.21.15.5:0/2186010490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f499810c170 0x7f499810e610 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.350+0000 7f499d61a640 1 --2- 172.21.15.5:0/2186010490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4998103660 0x7f499810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.350+0000 7f499d61a640 1 --2- 172.21.15.5:0/2186010490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4998102d40 0x7f4998103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.350+0000 7f499d61a640 1 -- 172.21.15.5:0/2186010490 >> 172.21.15.5:0/2186010490 conn(0x7f49980fc7a0 msgr2=0x7f49980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:52.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.350+0000 7f499d61a640 1 -- 172.21.15.5:0/2186010490 shutdown_connections 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.351+0000 7f499d61a640 1 -- 172.21.15.5:0/2186010490 wait complete. 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.351+0000 7f499d61a640 1 Processor -- start 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.351+0000 7f499d61a640 1 -- start start 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.351+0000 7f499d61a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4998102d40 0x7f49981009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.351+0000 7f499d61a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4998103660 0x7f4998100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f499d61a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f499810c170 0x7f4998101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f499d61a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4998111f30 con 0x7f4998103660 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f499d61a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4998111db0 con 0x7f4998102d40 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f499d61a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f49981120b0 con 0x7f499810c170 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f49967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4998103660 0x7f4998100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f49967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4998103660 0x7f4998100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58426/0 (socket says 172.21.15.5:58426) 2024-09-15T22:14:52.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f49967fc640 1 -- 172.21.15.5:0/610084282 learned_addr learned my addr 172.21.15.5:0/610084282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f4996ffd640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4998102d40 0x7f49981009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f49977fe640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f499810c170 0x7f4998101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f49967fc640 1 -- 172.21.15.5:0/610084282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f499810c170 msgr2=0x7f4998101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f49967fc640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f499810c170 0x7f4998101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f49967fc640 1 -- 172.21.15.5:0/610084282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4998102d40 msgr2=0x7f49981009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f49967fc640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4998102d40 0x7f49981009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f49967fc640 1 -- 172.21.15.5:0/610084282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49981a6890 con 0x7f4998103660 2024-09-15T22:14:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f49967fc640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4998103660 0x7f4998100ee0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f498c00c940 tx=0x7f498c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f497ffff640 1 -- 172.21.15.5:0/610084282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c004220 con 0x7f4998103660 2024-09-15T22:14:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f497ffff640 1 -- 172.21.15.5:0/610084282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498c0043c0 con 0x7f4998103660 2024-09-15T22:14:52.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f499d61a640 1 -- 172.21.15.5:0/610084282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49981a6b80 con 0x7f4998103660 2024-09-15T22:14:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.352+0000 7f497ffff640 1 -- 172.21.15.5:0/610084282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c004a00 con 0x7f4998103660 2024-09-15T22:14:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.353+0000 7f499d61a640 1 -- 172.21.15.5:0/610084282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49981a7110 con 0x7f4998103660 2024-09-15T22:14:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.354+0000 7f499d61a640 1 -- 172.21.15.5:0/610084282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4998103120 con 0x7f4998103660 2024-09-15T22:14:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.356+0000 7f497ffff640 1 -- 172.21.15.5:0/610084282 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f498c0079b0 con 0x7f4998103660 2024-09-15T22:14:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.356+0000 7f497ffff640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f49800a3430 0x7f49800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.356+0000 7f4996ffd640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f49800a3430 0x7f49800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.356+0000 7f497ffff640 1 -- 172.21.15.5:0/610084282 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f498c0cfec0 con 0x7f4998103660 2024-09-15T22:14:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.357+0000 7f4996ffd640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f49800a3430 0x7f49800a58f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4988005d00 tx=0x7f498800b410 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:52.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.359+0000 7f497ffff640 1 -- 172.21.15.5:0/610084282 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f498c014030 con 0x7f4998103660 2024-09-15T22:14:52.403 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:52.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.581+0000 7f499d61a640 1 -- 172.21.15.5:0/610084282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4998101ed0 con 0x7f4998103660 2024-09-15T22:14:52.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.582+0000 7f497ffff640 1 -- 172.21.15.5:0/610084282 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f4998101ed0 con 0x7f4998103660 2024-09-15T22:14:52.583 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:14:52.584 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:14:52.584 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:14:52.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 -- 172.21.15.5:0/610084282 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f49800a3430 msgr2=0x7f49800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f49800a3430 0x7f49800a58f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4988005d00 tx=0x7f498800b410 comp rx=0 tx=0).stop 2024-09-15T22:14:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 -- 172.21.15.5:0/610084282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4998103660 msgr2=0x7f4998100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4998103660 0x7f4998100ee0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f498c00c940 tx=0x7f498c00ce00 comp rx=0 tx=0).stop 2024-09-15T22:14:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 -- 172.21.15.5:0/610084282 shutdown_connections 2024-09-15T22:14:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f49800a3430 0x7f49800a58f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f499810c170 0x7f4998101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4998103660 0x7f4998100ee0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 --2- 172.21.15.5:0/610084282 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4998102d40 0x7f49981009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 -- 172.21.15.5:0/610084282 >> 172.21.15.5:0/610084282 conn(0x7f49980fc7a0 msgr2=0x7f49980fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 -- 172.21.15.5:0/610084282 shutdown_connections 2024-09-15T22:14:52.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:52.584+0000 7f499d61a640 1 -- 172.21.15.5:0/610084282 wait complete. 2024-09-15T22:14:53.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.049+0000 7f1958c8a640 1 -- 172.21.15.5:0/384918106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19541082e0 msgr2=0x7f195410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:53.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.049+0000 7f1958c8a640 1 --2- 172.21.15.5:0/384918106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19541082e0 0x7f195410ec50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f194c009930 tx=0x7f194c02a420 comp rx=0 tx=0).stop 2024-09-15T22:14:53.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.049+0000 7f1958c8a640 1 -- 172.21.15.5:0/384918106 shutdown_connections 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.049+0000 7f1958c8a640 1 --2- 172.21.15.5:0/384918106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19541082e0 0x7f195410ec50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.049+0000 7f1958c8a640 1 --2- 172.21.15.5:0/384918106 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1954103760 0x7f1954107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.049+0000 7f1958c8a640 1 --2- 172.21.15.5:0/384918106 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1954102db0 0x7f1954103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.049+0000 7f1958c8a640 1 -- 172.21.15.5:0/384918106 >> 172.21.15.5:0/384918106 conn(0x7f19540fc7d0 msgr2=0x7f19540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.049+0000 7f1958c8a640 1 -- 172.21.15.5:0/384918106 shutdown_connections 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.049+0000 7f1958c8a640 1 -- 172.21.15.5:0/384918106 wait complete. 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f1958c8a640 1 Processor -- start 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f1958c8a640 1 -- start start 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f1958c8a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1954102db0 0x7f19541985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f1958c8a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1954103760 0x7f1954198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f1958c8a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19541082e0 0x7f195419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f1958c8a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19541120b0 con 0x7f1954102db0 2024-09-15T22:14:53.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f1958c8a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1954111f30 con 0x7f1954103760 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f1958c8a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1954112230 con 0x7f19541082e0 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f195259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1954102db0 0x7f19541985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f195259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1954102db0 0x7f19541985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58448/0 (socket says 172.21.15.5:58448) 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f195259c640 1 -- 172.21.15.5:0/2129490549 learned_addr learned my addr 172.21.15.5:0/2129490549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f195259c640 1 -- 172.21.15.5:0/2129490549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19541082e0 msgr2=0x7f195419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f195259c640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19541082e0 0x7f195419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f195259c640 1 -- 172.21.15.5:0/2129490549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1954103760 msgr2=0x7f1954198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f1951d9b640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1954103760 0x7f1954198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f195259c640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1954103760 0x7f1954198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f195259c640 1 -- 172.21.15.5:0/2129490549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195419d560 con 0x7f1954102db0 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f1951d9b640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1954103760 0x7f1954198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.050+0000 7f195259c640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1954102db0 0x7f19541985b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f194400d770 tx=0x7f194400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:53.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.051+0000 7f19437fe640 1 -- 172.21.15.5:0/2129490549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944017070 con 0x7f1954102db0 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.051+0000 7f19437fe640 1 -- 172.21.15.5:0/2129490549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1944005500 con 0x7f1954102db0 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.051+0000 7f1958c8a640 1 -- 172.21.15.5:0/2129490549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195419d850 con 0x7f1954102db0 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.051+0000 7f19437fe640 1 -- 172.21.15.5:0/2129490549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944004e00 con 0x7f1954102db0 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.051+0000 7f1958c8a640 1 -- 172.21.15.5:0/2129490549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19541a5130 con 0x7f1954102db0 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.053+0000 7f19437fe640 1 -- 172.21.15.5:0/2129490549 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1944004fa0 con 0x7f1954102db0 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.053+0000 7f195b063740 1 Processor -- start 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.053+0000 7f195b063740 1 -- start start 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f195b063740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3f0b40 0x55dcab3ef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f195b063740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3ef5d0 0x55dcab3efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f195b063740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab400b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f195b063740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dcab3552f0 con 0x55dcab3fe780 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f195b063740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dcab2fccf0 con 0x55dcab3ef5d0 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f195b063740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dcab3512b0 con 0x55dcab3f0b40 2024-09-15T22:14:53.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f19437fe640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f193c0a3430 0x7f193c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f1951d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab400b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f1951d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab400b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58460/0 (socket says 172.21.15.5:58460) 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f1951d9b640 1 -- 172.21.15.5:0/2050597344 learned_addr learned my addr 172.21.15.5:0/2050597344 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f195259c640 1 --2- 172.21.15.5:0/2050597344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3ef5d0 0x55dcab3efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f1952d9d640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f193c0a3430 0x7f193c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f19437fe640 1 -- 172.21.15.5:0/2129490549 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f19440cfb40 con 0x7f1954102db0 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f1952d9d640 1 --2- 172.21.15.5:0/2050597344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3f0b40 0x55dcab3ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f1951d9b640 1 -- 172.21.15.5:0/2050597344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3f0b40 msgr2=0x55dcab3ef090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f1951d9b640 1 --2- 172.21.15.5:0/2050597344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3f0b40 0x55dcab3ef090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f1951d9b640 1 -- 172.21.15.5:0/2050597344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3ef5d0 msgr2=0x55dcab3efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f1951d9b640 1 --2- 172.21.15.5:0/2050597344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3ef5d0 0x55dcab3efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f1951d9b640 1 -- 172.21.15.5:0/2050597344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcab2e13a0 con 0x55dcab3fe780 2024-09-15T22:14:53.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.054+0000 7f1951d9b640 1 --2- 172.21.15.5:0/2050597344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab400b70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1948009e30 tx=0x7f194800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2bc7b909dd20c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.055+0000 7f1952d9d640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f193c0a3430 0x7f193c0a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f194c005770 tx=0x7f194c0058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.055+0000 7f1940ff9640 1 -- 172.21.15.5:0/2050597344 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948003b70 con 0x55dcab3fe780 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.055+0000 7f1940ff9640 1 -- 172.21.15.5:0/2050597344 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1948003d10 con 0x55dcab3fe780 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.055+0000 7f1940ff9640 1 -- 172.21.15.5:0/2050597344 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19480119a0 con 0x55dcab3fe780 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.055+0000 7f195b063740 1 -- 172.21.15.5:0/2050597344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 msgr2=0x55dcab400b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.055+0000 7f195b063740 1 --2- 172.21.15.5:0/2050597344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab400b70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f1948009e30 tx=0x7f194800f730 comp rx=0 tx=0).stop 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 -- 172.21.15.5:0/2050597344 shutdown_connections 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 --2- 172.21.15.5:0/2050597344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab400b70 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 --2- 172.21.15.5:0/2050597344 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3ef5d0 0x55dcab3efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 --2- 172.21.15.5:0/2050597344 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3f0b40 0x55dcab3ef090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 -- 172.21.15.5:0/2050597344 >> 172.21.15.5:0/2050597344 conn(0x55dcab3b8310 msgr2=0x55dcab3ebc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 -- 172.21.15.5:0/2050597344 shutdown_connections 2024-09-15T22:14:53.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 -- 172.21.15.5:0/2050597344 wait complete. 2024-09-15T22:14:53.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 -- start start 2024-09-15T22:14:53.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 Processor -- start 2024-09-15T22:14:53.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3ef5d0 0x55dcab3fe140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3f0b40 0x55dcab3fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab3fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dcab401480 con 0x55dcab3fe780 2024-09-15T22:14:53.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dcab310060 con 0x55dcab3ef5d0 2024-09-15T22:14:53.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.056+0000 7f195b063740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dcab401600 con 0x55dcab3f0b40 2024-09-15T22:14:53.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f195b063740 10 client.0 _ll_register_callbacks cb 0x55dcab0638c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1952d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab3fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1952d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab3fb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58470/0 (socket says 172.21.15.5:58470) 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1952d9d640 1 -- 172.21.15.5:0/3207784034 learned_addr learned my addr 172.21.15.5:0/3207784034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1951d9b640 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3f0b40 0x55dcab3fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f195259c640 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3ef5d0 0x55dcab3fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1952d9d640 1 -- 172.21.15.5:0/3207784034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3f0b40 msgr2=0x55dcab3fb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1952d9d640 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3f0b40 0x55dcab3fb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1952d9d640 1 -- 172.21.15.5:0/3207784034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3ef5d0 msgr2=0x55dcab3fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1952d9d640 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3ef5d0 0x55dcab3fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1952d9d640 1 -- 172.21.15.5:0/3207784034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcab0c63c0 con 0x55dcab3fe780 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f195259c640 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3ef5d0 0x55dcab3fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1952d9d640 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab3fb9e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f194c033000 tx=0x7f194c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3d7629f33f6993 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:53.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1939ffb640 1 -- 172.21.15.5:0/3207784034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c044040 con 0x55dcab3fe780 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f1939ffb640 1 -- 172.21.15.5:0/3207784034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f194c0046c0 con 0x55dcab3fe780 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.057+0000 7f195b063740 1 -- 172.21.15.5:0/3207784034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1948003d10 con 0x55dcab3fe780 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f1939ffb640 1 -- 172.21.15.5:0/3207784034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f194c056050 con 0x55dcab3fe780 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f195b063740 10 client.16587 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f195b063740 1 -- 172.21.15.5:0/3207784034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dcab40cf30 con 0x55dcab3fe780 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f1939ffb640 1 -- 172.21.15.5:0/3207784034 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1948003d10 con 0x55dcab3fe780 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f1939ffb640 1 -- 172.21.15.5:0/3207784034 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f194c03a040 con 0x55dcab3fe780 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f195b063740 10 client.16587 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f195b063740 10 client.16587 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f195b063740 20 client.16587 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f195b063740 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55dcab40e3f0 0x55dcab410890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f195b063740 4 client.16587 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f195b063740 1 -- 172.21.15.5:0/3207784034 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcab362860 con 0x55dcab40e3f0 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.058+0000 7f195259c640 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55dcab40e3f0 0x55dcab410890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.059+0000 7f1939ffb640 10 client.16587 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.059+0000 7f195259c640 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55dcab40e3f0 0x55dcab410890 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2098d4e921216b16 server_cookie=4c2082a2a4403f28 in_seq=0 out_seq=0 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.059+0000 7f1939ffb640 1 -- 172.21.15.5:0/3207784034 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dcab362860 con 0x55dcab40e3f0 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.059+0000 7f1939ffb640 10 client.16587 handle_command_reply: tid=0 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 1 client.16587 shutdown 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 1 -- 172.21.15.5:0/3207784034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 msgr2=0x55dcab3fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab3fb9e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f194c033000 tx=0x7f194c005fd0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 1 -- 172.21.15.5:0/3207784034 shutdown_connections 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55dcab40e3f0 0x55dcab410890 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab3fb9e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3f0b40 0x55dcab3fb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f1939ffb640 0 client.16587 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:14:53.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 1 --2- 172.21.15.5:0/3207784034 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3ef5d0 0x55dcab3fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 1 -- 172.21.15.5:0/3207784034 >> 172.21.15.5:0/3207784034 conn(0x55dcab3b8310 msgr2=0x55dcab3f1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:53.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 1 -- 172.21.15.5:0/3207784034 shutdown_connections 2024-09-15T22:14:53.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 1 -- 172.21.15.5:0/3207784034 wait complete. 2024-09-15T22:14:53.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.060+0000 7f195b063740 20 client.16587 trim_cache size 0 max 16384 2024-09-15T22:14:53.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f195b063740 1 Processor -- start 2024-09-15T22:14:53.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f195b063740 1 -- start start 2024-09-15T22:14:53.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f195b063740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3ef5d0 0x55dcab40a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f195b063740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3f0b40 0x55dcab3eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f195b063740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3fe780 0x55dcab3ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f195b063740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19541af6a0 con 0x55dcab3f0b40 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f195b063740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19541acf70 con 0x55dcab3fe780 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f195b063740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19541aadc0 con 0x55dcab3ef5d0 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f1951d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3f0b40 0x55dcab3eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f1951d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3f0b40 0x55dcab3eb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58474/0 (socket says 172.21.15.5:58474) 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f1951d9b640 1 -- 172.21.15.5:0/1117190004 learned_addr learned my addr 172.21.15.5:0/1117190004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f195259c640 1 --2- 172.21.15.5:0/1117190004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3ef5d0 0x55dcab40a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f1952d9d640 1 --2- 172.21.15.5:0/1117190004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3fe780 0x55dcab3ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f1951d9b640 1 -- 172.21.15.5:0/1117190004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3ef5d0 msgr2=0x55dcab40a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:53.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f1951d9b640 1 --2- 172.21.15.5:0/1117190004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3ef5d0 0x55dcab40a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f1951d9b640 1 -- 172.21.15.5:0/1117190004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3fe780 msgr2=0x55dcab3ffb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:53.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.074+0000 7f1951d9b640 1 --2- 172.21.15.5:0/1117190004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3fe780 0x55dcab3ffb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.075+0000 7f1951d9b640 1 -- 172.21.15.5:0/1117190004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcab3ebed0 con 0x55dcab3f0b40 2024-09-15T22:14:53.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.075+0000 7f1951d9b640 1 --2- 172.21.15.5:0/1117190004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3f0b40 0x55dcab3eb700 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1948009e30 tx=0x7f194800ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fd2ee4123061d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:53.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.075+0000 7f193bfff640 1 -- 172.21.15.5:0/1117190004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948004cc0 con 0x55dcab3f0b40 2024-09-15T22:14:53.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.075+0000 7f193bfff640 1 -- 172.21.15.5:0/1117190004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1948003e50 con 0x55dcab3f0b40 2024-09-15T22:14:53.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.075+0000 7f193bfff640 1 -- 172.21.15.5:0/1117190004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19480115f0 con 0x55dcab3f0b40 2024-09-15T22:14:53.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 -- 172.21.15.5:0/1117190004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3f0b40 msgr2=0x55dcab3eb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:53.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 --2- 172.21.15.5:0/1117190004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3f0b40 0x55dcab3eb700 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1948009e30 tx=0x7f194800ffb0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 -- 172.21.15.5:0/1117190004 shutdown_connections 2024-09-15T22:14:53.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 --2- 172.21.15.5:0/1117190004 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3fe780 0x55dcab3ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 --2- 172.21.15.5:0/1117190004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3f0b40 0x55dcab3eb700 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 --2- 172.21.15.5:0/1117190004 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3ef5d0 0x55dcab40a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 -- 172.21.15.5:0/1117190004 >> 172.21.15.5:0/1117190004 conn(0x55dcab3b8310 msgr2=0x55dcab3f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 -- 172.21.15.5:0/1117190004 shutdown_connections 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 -- 172.21.15.5:0/1117190004 wait complete. 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 -- start start 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 Processor -- start 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.076+0000 7f195b063740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3ef5d0 0x55dcab4068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f195b063740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3f0b40 0x55dcab406de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f195b063740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab401850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f195b063740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19541ab430 con 0x55dcab3fe780 2024-09-15T22:14:53.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f195b063740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19541ac8a0 con 0x55dcab3f0b40 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f195b063740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19541abb00 con 0x55dcab3ef5d0 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f195b063740 10 client.0 _ll_register_callbacks cb 0x7f194c041040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1952d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab401850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1952d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab401850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58476/0 (socket says 172.21.15.5:58476) 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1952d9d640 1 -- 172.21.15.5:0/268526866 learned_addr learned my addr 172.21.15.5:0/268526866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f195259c640 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3ef5d0 0x55dcab4068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1951d9b640 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3f0b40 0x55dcab406de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1952d9d640 1 -- 172.21.15.5:0/268526866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3ef5d0 msgr2=0x55dcab4068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1952d9d640 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3ef5d0 0x55dcab4068a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1952d9d640 1 -- 172.21.15.5:0/268526866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3f0b40 msgr2=0x55dcab406de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1952d9d640 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3f0b40 0x55dcab406de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1952d9d640 1 -- 172.21.15.5:0/268526866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcab407420 con 0x55dcab3fe780 2024-09-15T22:14:53.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f195259c640 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3ef5d0 0x55dcab4068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1951d9b640 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3f0b40 0x55dcab406de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1952d9d640 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab401850 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f194c048970 tx=0x7f194c031f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8947d2e7efb2076c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.077+0000 7f1939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f1939ffb640 1 -- 172.21.15.5:0/268526866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194c044040 con 0x55dcab3fe780 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f1939ffb640 1 -- 172.21.15.5:0/268526866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f194c048b60 con 0x55dcab3fe780 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f195b063740 1 -- 172.21.15.5:0/268526866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1948003e50 con 0x55dcab3fe780 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f1939ffb640 1 -- 172.21.15.5:0/268526866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f194c068050 con 0x55dcab3fe780 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f195b063740 10 client.16593 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f195b063740 1 -- 172.21.15.5:0/268526866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dcab4796f0 con 0x55dcab3fe780 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f1939ffb640 1 -- 172.21.15.5:0/268526866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1948003e50 con 0x55dcab3fe780 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f1939ffb640 1 -- 172.21.15.5:0/268526866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f194c052450 con 0x55dcab3fe780 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f195b063740 10 client.16593 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f195b063740 10 client.16593 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:14:53.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f195b063740 20 client.16593 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f195b063740 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55dcab479e30 0x55dcab47c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f195b063740 4 client.16593 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.078+0000 7f195b063740 1 -- 172.21.15.5:0/268526866 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcab35cd70 con 0x55dcab479e30 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.079+0000 7f195259c640 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55dcab479e30 0x55dcab47c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.079+0000 7f1939ffb640 10 client.16593 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.079+0000 7f195259c640 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55dcab479e30 0x55dcab47c250 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3954c842f4fcea79 server_cookie=6c7262b061929001 in_seq=0 out_seq=0 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.079+0000 7f1939ffb640 1 -- 172.21.15.5:0/268526866 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcab35cd70 con 0x55dcab479e30 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.079+0000 7f1939ffb640 10 client.16593 handle_command_reply: tid=0 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.079+0000 7f195b063740 1 client.16593 shutdown 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.079+0000 7f195b063740 1 -- 172.21.15.5:0/268526866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 msgr2=0x55dcab401850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.079+0000 7f195b063740 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab401850 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f194c048970 tx=0x7f194c031f20 comp rx=0 tx=0).stop 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.079+0000 7f195b063740 1 -- 172.21.15.5:0/268526866 shutdown_connections 2024-09-15T22:14:53.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.079+0000 7f195b063740 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55dcab479e30 0x55dcab47c250 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f195b063740 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dcab3fe780 0x55dcab401850 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1939ffb640 0 client.16593 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:14:53.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f195b063740 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dcab3f0b40 0x55dcab406de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f195b063740 1 --2- 172.21.15.5:0/268526866 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dcab3ef5d0 0x55dcab4068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f195b063740 1 -- 172.21.15.5:0/268526866 >> 172.21.15.5:0/268526866 conn(0x55dcab3b8310 msgr2=0x55dcab3fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:53.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f195b063740 1 -- 172.21.15.5:0/268526866 shutdown_connections 2024-09-15T22:14:53.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f195b063740 1 -- 172.21.15.5:0/268526866 wait complete. 2024-09-15T22:14:53.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f195b063740 20 client.16593 trim_cache size 0 max 16384 2024-09-15T22:14:53.089 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:14:53.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1958c8a640 1 -- 172.21.15.5:0/2129490549 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f193c0a3430 msgr2=0x7f193c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:53.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1958c8a640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f193c0a3430 0x7f193c0a58f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f194c005770 tx=0x7f194c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1958c8a640 1 -- 172.21.15.5:0/2129490549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1954102db0 msgr2=0x7f19541985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:53.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1958c8a640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1954102db0 0x7f19541985b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f194400d770 tx=0x7f194400dc30 comp rx=0 tx=0).stop 2024-09-15T22:14:53.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1958c8a640 1 -- 172.21.15.5:0/2129490549 shutdown_connections 2024-09-15T22:14:53.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1958c8a640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f193c0a3430 0x7f193c0a58f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1958c8a640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19541082e0 0x7f195419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1958c8a640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1954103760 0x7f1954198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1958c8a640 1 --2- 172.21.15.5:0/2129490549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1954102db0 0x7f19541985b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1958c8a640 1 -- 172.21.15.5:0/2129490549 >> 172.21.15.5:0/2129490549 conn(0x7f19540fc7d0 msgr2=0x7f1954106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.080+0000 7f1958c8a640 1 -- 172.21.15.5:0/2129490549 shutdown_connections 2024-09-15T22:14:53.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:53.081+0000 7f1958c8a640 1 -- 172.21.15.5:0/2129490549 wait complete. 2024-09-15T22:14:53.187 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:52 smithi005 ceph-mon[30944]: pgmap v182: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-15T22:14:53.187 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:52 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/610084282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:52 smithi099 ceph-mon[34892]: pgmap v182: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-15T22:14:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:52 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/610084282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:52 smithi129 ceph-mon[35314]: pgmap v182: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-09-15T22:14:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:52 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/610084282' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:14:53.541 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id b5407ac4 --release --await' 2024-09-15T22:14:53.542 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id b5407ac4 --release --await 2024-09-15T22:14:53.806 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:54.481 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:14:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 325.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:14:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.695+0000 7f58328d4640 1 -- 172.21.15.5:0/777850181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f582c102570 msgr2=0x7f582c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.695+0000 7f58328d4640 1 --2- 172.21.15.5:0/777850181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f582c102570 0x7f582c1029b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5820009900 tx=0x7f582002a480 comp rx=0 tx=0).stop 2024-09-15T22:14:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.695+0000 7f58328d4640 1 -- 172.21.15.5:0/777850181 shutdown_connections 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.695+0000 7f58328d4640 1 --2- 172.21.15.5:0/777850181 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f582c102ef0 0x7f582c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.695+0000 7f58328d4640 1 --2- 172.21.15.5:0/777850181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f582c102570 0x7f582c1029b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.695+0000 7f58328d4640 1 --2- 172.21.15.5:0/777850181 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f582c100e20 0x7f582c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.695+0000 7f58328d4640 1 -- 172.21.15.5:0/777850181 >> 172.21.15.5:0/777850181 conn(0x7f582c0fc7d0 msgr2=0x7f582c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.695+0000 7f58328d4640 1 -- 172.21.15.5:0/777850181 shutdown_connections 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.695+0000 7f58328d4640 1 -- 172.21.15.5:0/777850181 wait complete. 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f58328d4640 1 Processor -- start 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f58328d4640 1 -- start start 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f58328d4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f582c100e20 0x7f582c10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f58328d4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f582c102570 0x7f582c10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f58328d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f582c102ef0 0x7f582c10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f58328d4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1120f0 con 0x7f582c102ef0 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f58328d4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f582c111f70 con 0x7f582c100e20 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f58328d4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f582c112270 con 0x7f582c102570 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f5830e71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f582c102ef0 0x7f582c10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f5830e71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f582c102ef0 0x7f582c10ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33166/0 (socket says 172.21.15.5:33166) 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f5830e71640 1 -- 172.21.15.5:0/2558697112 learned_addr learned my addr 172.21.15.5:0/2558697112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f582b7fe640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f582c102570 0x7f582c10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.696+0000 7f582bfff640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f582c100e20 0x7f582c10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.697+0000 7f5830e71640 1 -- 172.21.15.5:0/2558697112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f582c102570 msgr2=0x7f582c10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.697+0000 7f5830e71640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f582c102570 0x7f582c10ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.697+0000 7f5830e71640 1 -- 172.21.15.5:0/2558697112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f582c100e20 msgr2=0x7f582c10e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.697+0000 7f5830e71640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f582c100e20 0x7f582c10e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.697+0000 7f5830e71640 1 -- 172.21.15.5:0/2558697112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f582c1a69f0 con 0x7f582c102ef0 2024-09-15T22:14:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.697+0000 7f5830e71640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f582c102ef0 0x7f582c10ef50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f582400cce0 tx=0x7f582400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.697+0000 7f58297fa640 1 -- 172.21.15.5:0/2558697112 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f582400f450 con 0x7f582c102ef0 2024-09-15T22:14:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.697+0000 7f58328d4640 1 -- 172.21.15.5:0/2558697112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f582c1a6ce0 con 0x7f582c102ef0 2024-09-15T22:14:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.697+0000 7f58297fa640 1 -- 172.21.15.5:0/2558697112 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f582400fad0 con 0x7f582c102ef0 2024-09-15T22:14:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.697+0000 7f58297fa640 1 -- 172.21.15.5:0/2558697112 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5824018b60 con 0x7f582c102ef0 2024-09-15T22:14:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.697+0000 7f58328d4640 1 -- 172.21.15.5:0/2558697112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f582c1a71a0 con 0x7f582c102ef0 2024-09-15T22:14:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.699+0000 7f58297fa640 1 -- 172.21.15.5:0/2558697112 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f582400fc80 con 0x7f582c102ef0 2024-09-15T22:14:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f5834cad740 1 Processor -- start 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f5834cad740 1 -- start start 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f5834cad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9060 0x561c392b9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f5834cad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f5834cad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392c4240 0x561c392c6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f5834cad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c3921e620 con 0x561c392b9980 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f5834cad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561c391c5e50 con 0x561c392b9060 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f5834cad740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561c3921a5e0 con 0x561c392c4240 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f582bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f582bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392c3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33180/0 (socket says 172.21.15.5:33180) 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f582bfff640 1 -- 172.21.15.5:0/2198303603 learned_addr learned my addr 172.21.15.5:0/2198303603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f58297fa640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f58140a33b0 0x7f58140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f582b7fe640 1 --2- 172.21.15.5:0/2198303603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392c4240 0x561c392c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.700+0000 7f5830e71640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f58140a33b0 0x7f58140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f58297fa640 1 -- 172.21.15.5:0/2558697112 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f58240d0550 con 0x7f582c102ef0 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f582b7fe640 1 --2- 172.21.15.5:0/2198303603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9060 0x561c392b9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f582bfff640 1 -- 172.21.15.5:0/2198303603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392c4240 msgr2=0x561c392c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f582bfff640 1 --2- 172.21.15.5:0/2198303603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392c4240 0x561c392c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f582bfff640 1 -- 172.21.15.5:0/2198303603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9060 msgr2=0x561c392b9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f582bfff640 1 --2- 172.21.15.5:0/2198303603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9060 0x561c392b9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f582bfff640 1 -- 172.21.15.5:0/2198303603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c391b6d40 con 0x561c392b9980 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f582b7fe640 1 --2- 172.21.15.5:0/2198303603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9060 0x561c392b9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f582bfff640 1 --2- 172.21.15.5:0/2198303603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392c3d00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f581c009a70 tx=0x7f581c009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73661dfa9cdd9886 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f5830e71640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f58140a33b0 0x7f58140a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58240cd9b0 tx=0x7f5824091920 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:54.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f58127fc640 1 -- 172.21.15.5:0/2198303603 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c017070 con 0x561c392b9980 2024-09-15T22:14:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f58127fc640 1 -- 172.21.15.5:0/2198303603 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f581c013470 con 0x561c392b9980 2024-09-15T22:14:54.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.701+0000 7f58127fc640 1 -- 172.21.15.5:0/2198303603 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c012630 con 0x561c392b9980 2024-09-15T22:14:54.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.702+0000 7f5834cad740 1 -- 172.21.15.5:0/2198303603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 msgr2=0x561c392c3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:54.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.702+0000 7f5834cad740 1 --2- 172.21.15.5:0/2198303603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392c3d00 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f581c009a70 tx=0x7f581c009f30 comp rx=0 tx=0).stop 2024-09-15T22:14:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.702+0000 7f5834cad740 1 -- 172.21.15.5:0/2198303603 shutdown_connections 2024-09-15T22:14:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.702+0000 7f5834cad740 1 --2- 172.21.15.5:0/2198303603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392c4240 0x561c392c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.702+0000 7f5834cad740 1 --2- 172.21.15.5:0/2198303603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392c3d00 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.702+0000 7f5834cad740 1 --2- 172.21.15.5:0/2198303603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9060 0x561c392b9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.702+0000 7f5834cad740 1 -- 172.21.15.5:0/2198303603 >> 172.21.15.5:0/2198303603 conn(0x561c392b55f0 msgr2=0x561c392b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.702+0000 7f5834cad740 1 -- 172.21.15.5:0/2198303603 shutdown_connections 2024-09-15T22:14:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.702+0000 7f5834cad740 1 -- 172.21.15.5:0/2198303603 wait complete. 2024-09-15T22:14:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.702+0000 7f5834cad740 1 -- start start 2024-09-15T22:14:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f5834cad740 1 Processor -- start 2024-09-15T22:14:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f5834cad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9060 0x561c392c1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f5834cad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392b9980 0x561c392c1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f5834cad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392c4240 0x561c392c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f5834cad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c392c6e10 con 0x561c392b9060 2024-09-15T22:14:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f5834cad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561c391a6f10 con 0x561c392c4240 2024-09-15T22:14:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f5834cad740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561c392c6f90 con 0x561c392b9980 2024-09-15T22:14:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f5834cad740 10 client.0 _ll_register_callbacks cb 0x561c391cadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f582b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9060 0x561c392c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f582b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9060 0x561c392c1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33190/0 (socket says 172.21.15.5:33190) 2024-09-15T22:14:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f582b7fe640 1 -- 172.21.15.5:0/1966362447 learned_addr learned my addr 172.21.15.5:0/1966362447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f582bfff640 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392b9980 0x561c392c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f582b7fe640 1 -- 172.21.15.5:0/1966362447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392b9980 msgr2=0x561c392c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f582b7fe640 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392b9980 0x561c392c1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f582b7fe640 1 -- 172.21.15.5:0/1966362447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392c4240 msgr2=0x561c392c1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.703+0000 7f582b7fe640 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392c4240 0x561c392c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.704+0000 7f582b7fe640 1 -- 172.21.15.5:0/1966362447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c38f2d580 con 0x561c392b9060 2024-09-15T22:14:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.704+0000 7f582b7fe640 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9060 0x561c392c1130 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5820009900 tx=0x7f58200035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26341fdd375ff75e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.704+0000 7f580b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.704+0000 7f580b7fe640 1 -- 172.21.15.5:0/1966362447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820035040 con 0x561c392b9060 2024-09-15T22:14:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.704+0000 7f5834cad740 1 -- 172.21.15.5:0/1966362447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f581c013470 con 0x561c392b9060 2024-09-15T22:14:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.704+0000 7f580b7fe640 1 -- 172.21.15.5:0/1966362447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f5820003c30 con 0x561c392b9060 2024-09-15T22:14:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.704+0000 7f580b7fe640 1 -- 172.21.15.5:0/1966362447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5820003e30 con 0x561c392b9060 2024-09-15T22:14:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.704+0000 7f5834cad740 10 client.16605 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.704+0000 7f5834cad740 1 -- 172.21.15.5:0/1966362447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c38ff7960 con 0x561c392b9060 2024-09-15T22:14:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.704+0000 7f580b7fe640 1 -- 172.21.15.5:0/1966362447 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f581c013470 con 0x561c392b9060 2024-09-15T22:14:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.705+0000 7f580b7fe640 1 -- 172.21.15.5:0/1966362447 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f582003d7d0 con 0x561c392b9060 2024-09-15T22:14:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.705+0000 7f5834cad740 10 client.16605 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.705+0000 7f5834cad740 10 client.16605 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:14:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.705+0000 7f5834cad740 20 client.16605 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.705+0000 7f5834cad740 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561c392d3cf0 0x561c392d60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.705+0000 7f5834cad740 4 client.16605 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.705+0000 7f5834cad740 1 -- 172.21.15.5:0/1966362447 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c3922bb90 con 0x561c392d3cf0 2024-09-15T22:14:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.705+0000 7f582bfff640 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561c392d3cf0 0x561c392d60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.706+0000 7f582bfff640 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561c392d3cf0 0x561c392d60e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3607bfdf156d4e2e server_cookie=4fb30cc2a94a9012 in_seq=0 out_seq=0 2024-09-15T22:14:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.706+0000 7f580b7fe640 10 client.16605 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.706+0000 7f580b7fe640 1 -- 172.21.15.5:0/1966362447 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c3922bb90 con 0x561c392d3cf0 2024-09-15T22:14:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.706+0000 7f580b7fe640 10 client.16605 handle_command_reply: tid=0 2024-09-15T22:14:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.706+0000 7f5834cad740 1 client.16605 shutdown 2024-09-15T22:14:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f5834cad740 1 -- 172.21.15.5:0/1966362447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9060 msgr2=0x561c392c1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f5834cad740 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9060 0x561c392c1130 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5820009900 tx=0x7f58200035f0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f5834cad740 1 -- 172.21.15.5:0/1966362447 shutdown_connections 2024-09-15T22:14:54.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f5834cad740 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561c392d3cf0 0x561c392d60e0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f580b7fe640 0 client.16605 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:54.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f5834cad740 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392c4240 0x561c392c1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f5834cad740 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392b9980 0x561c392c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f5834cad740 1 --2- 172.21.15.5:0/1966362447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9060 0x561c392c1130 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f5834cad740 1 -- 172.21.15.5:0/1966362447 >> 172.21.15.5:0/1966362447 conn(0x561c392b55f0 msgr2=0x561c392bb320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:54.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f5834cad740 1 -- 172.21.15.5:0/1966362447 shutdown_connections 2024-09-15T22:14:54.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f5834cad740 1 -- 172.21.15.5:0/1966362447 wait complete. 2024-09-15T22:14:54.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.707+0000 7f5834cad740 20 client.16605 trim_cache size 0 max 16384 2024-09-15T22:14:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.725+0000 7f5834cad740 1 Processor -- start 2024-09-15T22:14:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.725+0000 7f5834cad740 1 -- start start 2024-09-15T22:14:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f5834cad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392b9060 0x561c392bbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f5834cad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9980 0x561c392b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f5834cad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392c4240 0x561c392db970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f5834cad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b3aa0 con 0x561c392c4240 2024-09-15T22:14:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f5834cad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b1370 con 0x561c392b9980 2024-09-15T22:14:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f5834cad740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1af1c0 con 0x561c392b9060 2024-09-15T22:14:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f582b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392c4240 0x561c392db970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f582b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392c4240 0x561c392db970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33206/0 (socket says 172.21.15.5:33206) 2024-09-15T22:14:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f582b7fe640 1 -- 172.21.15.5:0/1731813075 learned_addr learned my addr 172.21.15.5:0/1731813075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f582b7fe640 1 --2- 172.21.15.5:0/1731813075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392b9060 0x561c392bbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f582bfff640 1 --2- 172.21.15.5:0/1731813075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9980 0x561c392b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f582b7fe640 1 -- 172.21.15.5:0/1731813075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392b9060 msgr2=0x561c392bbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f582b7fe640 1 --2- 172.21.15.5:0/1731813075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392b9060 0x561c392bbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f582b7fe640 1 -- 172.21.15.5:0/1731813075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9980 msgr2=0x561c392b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:54.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f582b7fe640 1 --2- 172.21.15.5:0/1731813075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9980 0x561c392b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f582b7fe640 1 -- 172.21.15.5:0/1731813075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c392b97b0 con 0x561c392c4240 2024-09-15T22:14:54.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.726+0000 7f582bfff640 1 --2- 172.21.15.5:0/1731813075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9980 0x561c392b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.727+0000 7f582b7fe640 1 --2- 172.21.15.5:0/1731813075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392c4240 0x561c392db970 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f58200040a0 tx=0x7f5820005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2bb38b1f8620b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.727+0000 7f5811ffb640 1 -- 172.21.15.5:0/1731813075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820035040 con 0x561c392c4240 2024-09-15T22:14:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.727+0000 7f5811ffb640 1 -- 172.21.15.5:0/1731813075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5820003c70 con 0x561c392c4240 2024-09-15T22:14:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.727+0000 7f5811ffb640 1 -- 172.21.15.5:0/1731813075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58200049e0 con 0x561c392c4240 2024-09-15T22:14:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.728+0000 7f5834cad740 1 -- 172.21.15.5:0/1731813075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392c4240 msgr2=0x561c392db970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.728+0000 7f5834cad740 1 --2- 172.21.15.5:0/1731813075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392c4240 0x561c392db970 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f58200040a0 tx=0x7f5820005fd0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.728+0000 7f5834cad740 1 -- 172.21.15.5:0/1731813075 shutdown_connections 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.728+0000 7f5834cad740 1 --2- 172.21.15.5:0/1731813075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392c4240 0x561c392db970 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.728+0000 7f5834cad740 1 --2- 172.21.15.5:0/1731813075 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9980 0x561c392b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.728+0000 7f5834cad740 1 --2- 172.21.15.5:0/1731813075 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392b9060 0x561c392bbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.728+0000 7f5834cad740 1 -- 172.21.15.5:0/1731813075 >> 172.21.15.5:0/1731813075 conn(0x561c392b55f0 msgr2=0x561c392c1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.728+0000 7f5834cad740 1 -- 172.21.15.5:0/1731813075 shutdown_connections 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.728+0000 7f5834cad740 1 -- 172.21.15.5:0/1731813075 wait complete. 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.728+0000 7f5834cad740 1 -- start start 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f5834cad740 1 Processor -- start 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f5834cad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9060 0x561c392cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f5834cad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f5834cad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392c4240 0x561c392c91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f5834cad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1af830 con 0x561c392b9980 2024-09-15T22:14:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f5834cad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1b0ca0 con 0x561c392b9060 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f582bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f582bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33214/0 (socket says 172.21.15.5:33214) 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f582bfff640 1 -- 172.21.15.5:0/400154331 learned_addr learned my addr 172.21.15.5:0/400154331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f582b7fe640 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9060 0x561c392cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f5834cad740 1 -- 172.21.15.5:0/400154331 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f582c1aff00 con 0x561c392c4240 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.729+0000 7f5834cad740 10 client.0 _ll_register_callbacks cb 0x7f5820004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f582b7fe640 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392c4240 0x561c392c91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f582bfff640 1 -- 172.21.15.5:0/400154331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392c4240 msgr2=0x561c392c91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f582bfff640 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392c4240 0x561c392c91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f582bfff640 1 -- 172.21.15.5:0/400154331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9060 msgr2=0x561c392cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f582bfff640 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9060 0x561c392cc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f582bfff640 1 -- 172.21.15.5:0/400154331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c392c9a10 con 0x561c392b9980 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f582b7fe640 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392c4240 0x561c392c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:14:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f582bfff640 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392cc930 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f581c009a70 tx=0x7f581c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d695332d60548040 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f580b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f580b7fe640 1 -- 172.21.15.5:0/400154331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581c02a030 con 0x561c392b9980 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f5834cad740 1 -- 172.21.15.5:0/400154331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c392260a0 con 0x561c392b9980 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f580b7fe640 1 -- 172.21.15.5:0/400154331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f581c00ed50 con 0x561c392b9980 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f580b7fe640 1 -- 172.21.15.5:0/400154331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f581c037050 con 0x561c392b9980 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.730+0000 7f5834cad740 10 client.16611 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.731+0000 7f5834cad740 1 -- 172.21.15.5:0/400154331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561c39347cb0 con 0x561c392b9980 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.731+0000 7f580b7fe640 1 -- 172.21.15.5:0/400154331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561c392260a0 con 0x561c392b9980 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.731+0000 7f580b7fe640 1 -- 172.21.15.5:0/400154331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f581c006040 con 0x561c392b9980 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.731+0000 7f5834cad740 10 client.16611 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.731+0000 7f5834cad740 10 client.16611 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:14:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.731+0000 7f5834cad740 20 client.16611 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.731+0000 7f5834cad740 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561c39348180 0x561c3934a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.731+0000 7f5834cad740 4 client.16611 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b5407ac4", "release": true, "await": true}] 2024-09-15T22:14:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.731+0000 7f5834cad740 1 -- 172.21.15.5:0/400154331 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b5407ac4", "release": true, "await": true}) -- 0x561c391a25a0 con 0x561c39348180 2024-09-15T22:14:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.732+0000 7f582b7fe640 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561c39348180 0x561c3934a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.732+0000 7f580b7fe640 10 client.16611 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.732+0000 7f582b7fe640 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561c39348180 0x561c3934a5a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d58e01bd66178a0 server_cookie=6809ffbd30c35698 in_seq=0 out_seq=0 2024-09-15T22:14:54.777 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:54.803 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 24, 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "b5407ac4": { 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 24, 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 145.3, 2024-09-15T22:14:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:14:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:14:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:14:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:14:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:14:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:14:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:14:54.805 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.787+0000 7f580b7fe640 1 -- 172.21.15.5:0/400154331 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561c391a25a0 con 0x561c39348180 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.787+0000 7f580b7fe640 10 client.16611 handle_command_reply: tid=0 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.787+0000 7f5834cad740 1 client.16611 shutdown 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.788+0000 7f5834cad740 1 -- 172.21.15.5:0/400154331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 msgr2=0x561c392cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.788+0000 7f5834cad740 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392cc930 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f581c009a70 tx=0x7f581c008040 comp rx=0 tx=0).stop 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.791+0000 7f5834cad740 1 -- 172.21.15.5:0/400154331 shutdown_connections 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.791+0000 7f5834cad740 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561c392c4240 0x561c392c91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.791+0000 7f5834cad740 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c392b9980 0x561c392cc930 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.791+0000 7f5834cad740 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561c39348180 0x561c3934a5a0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.791+0000 7f580b7fe640 0 client.16611 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.791+0000 7f5834cad740 1 --2- 172.21.15.5:0/400154331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561c392b9060 0x561c392cc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.791+0000 7f5834cad740 1 -- 172.21.15.5:0/400154331 >> 172.21.15.5:0/400154331 conn(0x561c392b55f0 msgr2=0x561c392cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.791+0000 7f5834cad740 1 -- 172.21.15.5:0/400154331 shutdown_connections 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.791+0000 7f5834cad740 1 -- 172.21.15.5:0/400154331 wait complete. 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.792+0000 7f5834cad740 20 client.16611 trim_cache size 0 max 16384 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.793+0000 7f58328d4640 1 -- 172.21.15.5:0/2558697112 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f58140a33b0 msgr2=0x7f58140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.793+0000 7f58328d4640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f58140a33b0 0x7f58140a5870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58240cd9b0 tx=0x7f5824091920 comp rx=0 tx=0).stop 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.793+0000 7f58328d4640 1 -- 172.21.15.5:0/2558697112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f582c102ef0 msgr2=0x7f582c10ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.793+0000 7f58328d4640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f582c102ef0 0x7f582c10ef50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f582400cce0 tx=0x7f582400a620 comp rx=0 tx=0).stop 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.795+0000 7f58328d4640 1 -- 172.21.15.5:0/2558697112 shutdown_connections 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.795+0000 7f58328d4640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f58140a33b0 0x7f58140a5870 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.795+0000 7f58328d4640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f582c102ef0 0x7f582c10ef50 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.795+0000 7f58328d4640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f582c102570 0x7f582c10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.795+0000 7f58328d4640 1 --2- 172.21.15.5:0/2558697112 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f582c100e20 0x7f582c10e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.795+0000 7f58328d4640 1 -- 172.21.15.5:0/2558697112 >> 172.21.15.5:0/2558697112 conn(0x7f582c0fc7d0 msgr2=0x7f582c106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.795+0000 7f58328d4640 1 -- 172.21.15.5:0/2558697112 shutdown_connections 2024-09-15T22:14:54.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:54.795+0000 7f58328d4640 1 -- 172.21.15.5:0/2558697112 wait complete. 2024-09-15T22:14:55.051 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:54 smithi005 ceph-mon[30944]: pgmap v183: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-15T22:14:55.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:54 smithi099 ceph-mon[34892]: pgmap v183: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-15T22:14:55.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:54 smithi129 ceph-mon[35314]: pgmap v183: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-09-15T22:14:55.573 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b5407ac4', total seconds elapsed: 30.2 2024-09-15T22:14:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.156+0000 7fa91b59e640 1 -- 172.21.15.5:0/3624113165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91c10e970 msgr2=0x7fa91c115310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.156+0000 7fa91b59e640 1 --2- 172.21.15.5:0/3624113165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91c10e970 0x7fa91c115310 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa914009ca0 tx=0x7fa91402a8f0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.156+0000 7fa91b59e640 1 -- 172.21.15.5:0/3624113165 shutdown_connections 2024-09-15T22:14:56.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.157+0000 7fa91b59e640 1 --2- 172.21.15.5:0/3624113165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91c10e970 0x7fa91c115310 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.157+0000 7fa91b59e640 1 --2- 172.21.15.5:0/3624113165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c109e00 0x7fa91c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.157+0000 7fa91b59e640 1 --2- 172.21.15.5:0/3624113165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c109450 0x7fa91c109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.160+0000 7fa91b59e640 1 -- 172.21.15.5:0/3624113165 >> 172.21.15.5:0/3624113165 conn(0x7fa91c073180 msgr2=0x7fa91c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:56.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.161+0000 7fa91b59e640 1 -- 172.21.15.5:0/3624113165 shutdown_connections 2024-09-15T22:14:56.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.161+0000 7fa91b59e640 1 -- 172.21.15.5:0/3624113165 wait complete. 2024-09-15T22:14:56.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.162+0000 7fa91b59e640 1 Processor -- start 2024-09-15T22:14:56.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.162+0000 7fa91b59e640 1 -- start start 2024-09-15T22:14:56.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.163+0000 7fa91b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91c109450 0x7fa91c1a7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.163+0000 7fa91b59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c109e00 0x7fa91c1a7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.163+0000 7fa91b59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c10e970 0x7fa91c1ab8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.163+0000 7fa91b59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1187b0 con 0x7fa91c109450 2024-09-15T22:14:56.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.163+0000 7fa91ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c10e970 0x7fa91c1ab8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.163+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91c109450 0x7fa91c1a7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.164+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91c109450 0x7fa91c1a7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33236/0 (socket says 172.21.15.5:33236) 2024-09-15T22:14:56.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.164+0000 7fa91a59c640 1 -- 172.21.15.5:0/1478897544 learned_addr learned my addr 172.21.15.5:0/1478897544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:56.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.164+0000 7fa919d9b640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c109e00 0x7fa91c1a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.164+0000 7fa91b59e640 1 -- 172.21.15.5:0/1478897544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c118630 con 0x7fa91c10e970 2024-09-15T22:14:56.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.164+0000 7fa91b59e640 1 -- 172.21.15.5:0/1478897544 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c118930 con 0x7fa91c109e00 2024-09-15T22:14:56.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.165+0000 7fa91ad9d640 1 -- 172.21.15.5:0/1478897544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c109e00 msgr2=0x7fa91c1a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:56.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.165+0000 7fa91ad9d640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c109e00 0x7fa91c1a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.165+0000 7fa91ad9d640 1 -- 172.21.15.5:0/1478897544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91c109450 msgr2=0x7fa91c1a7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:56.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.165+0000 7fa91ad9d640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91c109450 0x7fa91c1a7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.165+0000 7fa91ad9d640 1 -- 172.21.15.5:0/1478897544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c1abfd0 con 0x7fa91c10e970 2024-09-15T22:14:56.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.165+0000 7fa91ad9d640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c10e970 0x7fa91c1ab8f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa914005e30 tx=0x7fa9140316b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:56.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.165+0000 7fa90b7fe640 1 -- 172.21.15.5:0/1478897544 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91402f840 con 0x7fa91c10e970 2024-09-15T22:14:56.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.165+0000 7fa90b7fe640 1 -- 172.21.15.5:0/1478897544 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa91402d070 con 0x7fa91c10e970 2024-09-15T22:14:56.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.165+0000 7fa91b59e640 1 -- 172.21.15.5:0/1478897544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c1ac260 con 0x7fa91c10e970 2024-09-15T22:14:56.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.166+0000 7fa90b7fe640 1 -- 172.21.15.5:0/1478897544 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914046480 con 0x7fa91c10e970 2024-09-15T22:14:56.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.166+0000 7fa91b59e640 1 -- 172.21.15.5:0/1478897544 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c1ac730 con 0x7fa91c10e970 2024-09-15T22:14:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.169+0000 7fa90b7fe640 1 -- 172.21.15.5:0/1478897544 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa91403b400 con 0x7fa91c10e970 2024-09-15T22:14:56.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.169+0000 7fa923161740 1 Processor -- start 2024-09-15T22:14:56.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa923161740 1 -- start start 2024-09-15T22:14:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa923161740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4dacf0 0x55c40d4d9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa923161740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4d9780 0x55c40d4d9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa923161740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4e4520 0x55c40d4e6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa923161740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c40d43f2f0 con 0x55c40d4e4520 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa923161740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c40d3e6cf0 con 0x55c40d4dacf0 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa923161740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c40d43b2b0 con 0x55c40d4d9780 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa90b7fe640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9040a3480 0x7fa9040a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4d9780 0x55c40d4d9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4d9780 0x55c40d4d9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:34490/0 (socket says 172.21.15.5:34490) 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa91a59c640 1 -- 172.21.15.5:0/2185354130 learned_addr learned my addr 172.21.15.5:0/2185354130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa91a59c640 1 -- 172.21.15.5:0/2185354130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4dacf0 msgr2=0x55c40d4d9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa91a59c640 1 --2- 172.21.15.5:0/2185354130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4dacf0 0x55c40d4d9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa91a59c640 1 -- 172.21.15.5:0/2185354130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4e4520 msgr2=0x55c40d4e6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa91a59c640 1 --2- 172.21.15.5:0/2185354130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4e4520 0x55c40d4e6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa91ad9d640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9040a3480 0x7fa9040a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.170+0000 7fa91a59c640 1 -- 172.21.15.5:0/2185354130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c40d3cb3a0 con 0x55c40d4d9780 2024-09-15T22:14:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.171+0000 7fa91a59c640 1 --2- 172.21.15.5:0/2185354130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4d9780 0x55c40d4d9bc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009dc0 tx=0x7fa90c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e3418c6e3bf6c58 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.171+0000 7fa90b7fe640 1 -- 172.21.15.5:0/1478897544 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa9140f2010 con 0x7fa91c10e970 2024-09-15T22:14:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.171+0000 7fa908ff9640 1 -- 172.21.15.5:0/2185354130 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c005220 con 0x55c40d4d9780 2024-09-15T22:14:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.171+0000 7fa908ff9640 1 -- 172.21.15.5:0/2185354130 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c004c80 con 0x55c40d4d9780 2024-09-15T22:14:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.171+0000 7fa908ff9640 1 -- 172.21.15.5:0/2185354130 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0198d0 con 0x55c40d4d9780 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.172+0000 7fa923161740 1 -- 172.21.15.5:0/2185354130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4d9780 msgr2=0x55c40d4d9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.172+0000 7fa923161740 1 --2- 172.21.15.5:0/2185354130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4d9780 0x55c40d4d9bc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009dc0 tx=0x7fa90c011810 comp rx=0 tx=0).stop 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.172+0000 7fa91ad9d640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9040a3480 0x7fa9040a5940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa91402f570 tx=0x7fa9140305f0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.172+0000 7fa923161740 1 -- 172.21.15.5:0/2185354130 shutdown_connections 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.172+0000 7fa923161740 1 --2- 172.21.15.5:0/2185354130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4e4520 0x55c40d4e6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.172+0000 7fa923161740 1 --2- 172.21.15.5:0/2185354130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4d9780 0x55c40d4d9bc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.172+0000 7fa923161740 1 --2- 172.21.15.5:0/2185354130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4dacf0 0x55c40d4d9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.172+0000 7fa923161740 1 -- 172.21.15.5:0/2185354130 >> 172.21.15.5:0/2185354130 conn(0x55c40d4a2310 msgr2=0x55c40d4d5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.173+0000 7fa923161740 1 -- 172.21.15.5:0/2185354130 shutdown_connections 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.173+0000 7fa923161740 1 -- 172.21.15.5:0/2185354130 wait complete. 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.174+0000 7fa923161740 1 -- start start 2024-09-15T22:14:56.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.174+0000 7fa923161740 1 Processor -- start 2024-09-15T22:14:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.174+0000 7fa923161740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4d9780 0x55c40d4e3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.174+0000 7fa923161740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4dacf0 0x55c40d4e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.174+0000 7fa923161740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4e4520 0x55c40d4e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.174+0000 7fa923161740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c40d4e7220 con 0x55c40d4d9780 2024-09-15T22:14:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.174+0000 7fa923161740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c40d3fa060 con 0x55c40d4e4520 2024-09-15T22:14:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.174+0000 7fa923161740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c40d4e73a0 con 0x55c40d4dacf0 2024-09-15T22:14:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.174+0000 7fa923161740 10 client.0 _ll_register_callbacks cb 0x55c40d14d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.175+0000 7fa919d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4d9780 0x55c40d4e3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.175+0000 7fa919d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4d9780 0x55c40d4e3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33258/0 (socket says 172.21.15.5:33258) 2024-09-15T22:14:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.175+0000 7fa919d9b640 1 -- 172.21.15.5:0/519864049 learned_addr learned my addr 172.21.15.5:0/519864049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.175+0000 7fa919d9b640 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4e4520 0x55c40d4e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.175+0000 7fa91a59c640 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4dacf0 0x55c40d4e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.175+0000 7fa919d9b640 1 -- 172.21.15.5:0/519864049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4dacf0 msgr2=0x55c40d4e1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.175+0000 7fa919d9b640 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4dacf0 0x55c40d4e1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.175+0000 7fa919d9b640 1 -- 172.21.15.5:0/519864049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4d9780 msgr2=0x55c40d4e3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.175+0000 7fa919d9b640 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4d9780 0x55c40d4e3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.175+0000 7fa919d9b640 1 -- 172.21.15.5:0/519864049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c40d1b03c0 con 0x55c40d4e4520 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.176+0000 7fa91a59c640 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4dacf0 0x55c40d4e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.176+0000 7fa919d9b640 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4e4520 0x55c40d4e16a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa910011b30 tx=0x7fa910005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0b0e3f977cd02fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.176+0000 7fa901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.176+0000 7fa901ffb640 1 -- 172.21.15.5:0/519864049 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91000fb50 con 0x55c40d4e4520 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.176+0000 7fa923161740 1 -- 172.21.15.5:0/519864049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90c004c80 con 0x55c40d4e4520 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.176+0000 7fa901ffb640 1 -- 172.21.15.5:0/519864049 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa91000fcf0 con 0x55c40d4e4520 2024-09-15T22:14:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.176+0000 7fa901ffb640 1 -- 172.21.15.5:0/519864049 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa910028050 con 0x55c40d4e4520 2024-09-15T22:14:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.176+0000 7fa923161740 10 client.26156 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.176+0000 7fa923161740 1 -- 172.21.15.5:0/519864049 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c40d4f2ce0 con 0x55c40d4e4520 2024-09-15T22:14:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.177+0000 7fa901ffb640 1 -- 172.21.15.5:0/519864049 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c004c80 con 0x55c40d4e4520 2024-09-15T22:14:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.177+0000 7fa901ffb640 1 -- 172.21.15.5:0/519864049 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa91001d020 con 0x55c40d4e4520 2024-09-15T22:14:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.177+0000 7fa923161740 10 client.26156 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.177+0000 7fa923161740 10 client.26156 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.177+0000 7fa923161740 20 client.26156 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.177+0000 7fa923161740 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c40d4f40f0 0x55c40d4f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.177+0000 7fa923161740 4 client.26156 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.177+0000 7fa923161740 1 -- 172.21.15.5:0/519864049 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c40d44c860 con 0x55c40d4f40f0 2024-09-15T22:14:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.178+0000 7fa91a59c640 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c40d4f40f0 0x55c40d4f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.180+0000 7fa91a59c640 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c40d4f40f0 0x55c40d4f64e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1ca85540703cee server_cookie=18cfbdec7a3d46f7 in_seq=0 out_seq=0 2024-09-15T22:14:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.180+0000 7fa901ffb640 10 client.26156 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa901ffb640 1 -- 172.21.15.5:0/519864049 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c40d44c860 con 0x55c40d4f40f0 2024-09-15T22:14:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa901ffb640 10 client.26156 handle_command_reply: tid=0 2024-09-15T22:14:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa923161740 1 client.26156 shutdown 2024-09-15T22:14:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa923161740 1 -- 172.21.15.5:0/519864049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4e4520 msgr2=0x55c40d4e16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa923161740 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4e4520 0x55c40d4e16a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa910011b30 tx=0x7fa910005f90 comp rx=0 tx=0).stop 2024-09-15T22:14:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa923161740 1 -- 172.21.15.5:0/519864049 shutdown_connections 2024-09-15T22:14:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa923161740 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c40d4f40f0 0x55c40d4f64e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa923161740 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4e4520 0x55c40d4e16a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa923161740 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4dacf0 0x55c40d4e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa923161740 1 --2- 172.21.15.5:0/519864049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4d9780 0x55c40d4e3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa923161740 1 -- 172.21.15.5:0/519864049 >> 172.21.15.5:0/519864049 conn(0x55c40d4a2310 msgr2=0x55c40d4dbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.181+0000 7fa901ffb640 0 client.26156 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.182+0000 7fa923161740 1 -- 172.21.15.5:0/519864049 shutdown_connections 2024-09-15T22:14:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.182+0000 7fa923161740 1 -- 172.21.15.5:0/519864049 wait complete. 2024-09-15T22:14:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.182+0000 7fa923161740 20 client.26156 trim_cache size 0 max 16384 2024-09-15T22:14:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.203+0000 7fa923161740 1 Processor -- start 2024-09-15T22:14:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.203+0000 7fa923161740 1 -- start start 2024-09-15T22:14:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.204+0000 7fa923161740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4d9780 0x55c40d4f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.204+0000 7fa923161740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4dacf0 0x55c40d4e5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.204+0000 7fa923161740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4e4520 0x55c40d4e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.204+0000 7fa923161740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1c0540 con 0x55c40d4e4520 2024-09-15T22:14:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.204+0000 7fa923161740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1bde10 con 0x55c40d4dacf0 2024-09-15T22:14:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.204+0000 7fa923161740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1bbc60 con 0x55c40d4d9780 2024-09-15T22:14:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.204+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4dacf0 0x55c40d4e5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.205+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4dacf0 0x55c40d4e5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:59502/0 (socket says 172.21.15.5:59502) 2024-09-15T22:14:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.205+0000 7fa91a59c640 1 -- 172.21.15.5:0/3613930091 learned_addr learned my addr 172.21.15.5:0/3613930091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.205+0000 7fa91a59c640 1 -- 172.21.15.5:0/3613930091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4d9780 msgr2=0x55c40d4f04c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:14:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.205+0000 7fa91a59c640 1 --2- 172.21.15.5:0/3613930091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4d9780 0x55c40d4f04c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.205+0000 7fa91a59c640 1 -- 172.21.15.5:0/3613930091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4e4520 msgr2=0x55c40d4e5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.205+0000 7fa91a59c640 1 --2- 172.21.15.5:0/3613930091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4e4520 0x55c40d4e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.205+0000 7fa91a59c640 1 -- 172.21.15.5:0/3613930091 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c40d4d6080 con 0x55c40d4dacf0 2024-09-15T22:14:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.206+0000 7fa91a59c640 1 --2- 172.21.15.5:0/3613930091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4dacf0 0x55c40d4e5910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009380 tx=0x7fa90c0194f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2debc722ed49a36 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.206+0000 7fa903fff640 1 -- 172.21.15.5:0/3613930091 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c032070 con 0x55c40d4dacf0 2024-09-15T22:14:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.206+0000 7fa903fff640 1 -- 172.21.15.5:0/3613930091 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c0053b0 con 0x55c40d4dacf0 2024-09-15T22:14:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.206+0000 7fa903fff640 1 -- 172.21.15.5:0/3613930091 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0198e0 con 0x55c40d4dacf0 2024-09-15T22:14:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.207+0000 7fa923161740 1 -- 172.21.15.5:0/3613930091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4dacf0 msgr2=0x55c40d4e5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.207+0000 7fa923161740 1 --2- 172.21.15.5:0/3613930091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4dacf0 0x55c40d4e5910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009380 tx=0x7fa90c0194f0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.207+0000 7fa923161740 1 -- 172.21.15.5:0/3613930091 shutdown_connections 2024-09-15T22:14:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.207+0000 7fa923161740 1 --2- 172.21.15.5:0/3613930091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4e4520 0x55c40d4e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.207+0000 7fa923161740 1 --2- 172.21.15.5:0/3613930091 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4dacf0 0x55c40d4e5910 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009380 tx=0x7fa90c0194f0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.207+0000 7fa923161740 1 --2- 172.21.15.5:0/3613930091 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4d9780 0x55c40d4f04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.207+0000 7fa923161740 1 -- 172.21.15.5:0/3613930091 >> 172.21.15.5:0/3613930091 conn(0x55c40d4a2310 msgr2=0x55c40d4d4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.208+0000 7fa923161740 1 -- 172.21.15.5:0/3613930091 shutdown_connections 2024-09-15T22:14:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.208+0000 7fa923161740 1 -- 172.21.15.5:0/3613930091 wait complete. 2024-09-15T22:14:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.209+0000 7fa923161740 1 -- start start 2024-09-15T22:14:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.209+0000 7fa923161740 1 Processor -- start 2024-09-15T22:14:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.209+0000 7fa923161740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4d9780 0x55c40d4deba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.210+0000 7fa923161740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4e4520 0x55c40d4ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.210+0000 7fa919d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4d9780 0x55c40d4deba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.210+0000 7fa923161740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4ecb70 0x55c40d4dd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.210+0000 7fa923161740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1bc2d0 con 0x55c40d4d9780 2024-09-15T22:14:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.210+0000 7fa923161740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1bd740 con 0x55c40d4ecb70 2024-09-15T22:14:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.210+0000 7fa923161740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1bc9a0 con 0x55c40d4e4520 2024-09-15T22:14:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.210+0000 7fa923161740 10 client.0 _ll_register_callbacks cb 0x7fa9100099f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.210+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4e4520 0x55c40d4ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.210+0000 7fa91a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4e4520 0x55c40d4ec630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:34512/0 (socket says 172.21.15.5:34512) 2024-09-15T22:14:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.210+0000 7fa91a59c640 1 -- 172.21.15.5:0/3769989529 learned_addr learned my addr 172.21.15.5:0/3769989529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.212+0000 7fa919d9b640 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4ecb70 0x55c40d4dd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.212+0000 7fa91a59c640 1 -- 172.21.15.5:0/3769989529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4ecb70 msgr2=0x55c40d4dd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.212+0000 7fa91a59c640 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4ecb70 0x55c40d4dd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.212+0000 7fa91a59c640 1 -- 172.21.15.5:0/3769989529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4d9780 msgr2=0x55c40d4deba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.212+0000 7fa91a59c640 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4d9780 0x55c40d4deba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.212+0000 7fa91a59c640 1 -- 172.21.15.5:0/3769989529 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c40d4ed0f0 con 0x55c40d4e4520 2024-09-15T22:14:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.212+0000 7fa91a59c640 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4e4520 0x55c40d4ec630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00a010 tx=0x7fa90c005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b400d6af761ca30f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.212+0000 7fa901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:14:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.213+0000 7fa901ffb640 1 -- 172.21.15.5:0/3769989529 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c03b060 con 0x55c40d4e4520 2024-09-15T22:14:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.213+0000 7fa923161740 1 -- 172.21.15.5:0/3769989529 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90c0053b0 con 0x55c40d4e4520 2024-09-15T22:14:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.213+0000 7fa901ffb640 1 -- 172.21.15.5:0/3769989529 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa90c0035f0 con 0x55c40d4e4520 2024-09-15T22:14:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.214+0000 7fa901ffb640 1 -- 172.21.15.5:0/3769989529 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa90c042050 con 0x55c40d4e4520 2024-09-15T22:14:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.214+0000 7fa923161740 10 client.26125 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.214+0000 7fa923161740 1 -- 172.21.15.5:0/3769989529 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c40d55f1a0 con 0x55c40d4e4520 2024-09-15T22:14:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.214+0000 7fa901ffb640 1 -- 172.21.15.5:0/3769989529 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c005200 con 0x55c40d4e4520 2024-09-15T22:14:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.214+0000 7fa901ffb640 1 -- 172.21.15.5:0/3769989529 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa90c035070 con 0x55c40d4e4520 2024-09-15T22:14:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.215+0000 7fa923161740 10 client.26125 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.215+0000 7fa923161740 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.215+0000 7fa923161740 20 client.26125 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.215+0000 7fa923161740 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c40d55f520 0x55c40d561910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.215+0000 7fa923161740 4 client.26125 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:14:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.215+0000 7fa923161740 1 -- 172.21.15.5:0/3769989529 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa90c0053b0 con 0x55c40d55f520 2024-09-15T22:14:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.216+0000 7fa919d9b640 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c40d55f520 0x55c40d561910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:56.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa901ffb640 10 client.26125 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:56.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa919d9b640 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c40d55f520 0x55c40d561910 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72c3ab73cd4da97f server_cookie=12dda2d8de6672bb in_seq=0 out_seq=0 2024-09-15T22:14:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa901ffb640 1 -- 172.21.15.5:0/3769989529 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa90c0053b0 con 0x55c40d55f520 2024-09-15T22:14:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa901ffb640 10 client.26125 handle_command_reply: tid=0 2024-09-15T22:14:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa923161740 1 client.26125 shutdown 2024-09-15T22:14:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa923161740 1 -- 172.21.15.5:0/3769989529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4e4520 msgr2=0x55c40d4ec630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa923161740 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4e4520 0x55c40d4ec630 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00a010 tx=0x7fa90c005720 comp rx=0 tx=0).stop 2024-09-15T22:14:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa923161740 1 -- 172.21.15.5:0/3769989529 shutdown_connections 2024-09-15T22:14:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa923161740 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c40d55f520 0x55c40d561910 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa923161740 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c40d4ecb70 0x55c40d4dd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa923161740 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c40d4e4520 0x55c40d4ec630 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa923161740 1 --2- 172.21.15.5:0/3769989529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c40d4d9780 0x55c40d4deba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa923161740 1 -- 172.21.15.5:0/3769989529 >> 172.21.15.5:0/3769989529 conn(0x55c40d4a2310 msgr2=0x55c40d4d4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.221+0000 7fa901ffb640 0 client.26125 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.222+0000 7fa923161740 1 -- 172.21.15.5:0/3769989529 shutdown_connections 2024-09-15T22:14:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.222+0000 7fa923161740 1 -- 172.21.15.5:0/3769989529 wait complete. 2024-09-15T22:14:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.222+0000 7fa923161740 20 client.26125 trim_cache size 0 max 16384 2024-09-15T22:14:56.225 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.223+0000 7fa91b59e640 1 -- 172.21.15.5:0/1478897544 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9040a3480 msgr2=0x7fa9040a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.223+0000 7fa91b59e640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9040a3480 0x7fa9040a5940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa91402f570 tx=0x7fa9140305f0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.223+0000 7fa91b59e640 1 -- 172.21.15.5:0/1478897544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c10e970 msgr2=0x7fa91c1ab8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.223+0000 7fa91b59e640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c10e970 0x7fa91c1ab8f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa914005e30 tx=0x7fa9140316b0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.223+0000 7fa91b59e640 1 -- 172.21.15.5:0/1478897544 shutdown_connections 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.223+0000 7fa91b59e640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa9040a3480 0x7fa9040a5940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.223+0000 7fa91b59e640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa91c10e970 0x7fa91c1ab8f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.223+0000 7fa91b59e640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa91c109e00 0x7fa91c1a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.223+0000 7fa91b59e640 1 --2- 172.21.15.5:0/1478897544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa91c109450 0x7fa91c1a7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.223+0000 7fa91b59e640 1 -- 172.21.15.5:0/1478897544 >> 172.21.15.5:0/1478897544 conn(0x7fa91c073180 msgr2=0x7fa91c10c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.224+0000 7fa91b59e640 1 -- 172.21.15.5:0/1478897544 shutdown_connections 2024-09-15T22:14:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:56.224+0000 7fa91b59e640 1 -- 172.21.15.5:0/1478897544 wait complete. 2024-09-15T22:14:57.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:56 smithi099 ceph-mon[34892]: pgmap v184: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-15T22:14:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:56 smithi129 ceph-mon[35314]: pgmap v184: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-15T22:14:57.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:56 smithi005 ceph-mon[30944]: pgmap v184: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-09-15T22:14:58.158 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:14:58.159 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:14:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:14:59.075 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:14:58 smithi005 ceph-mon[30944]: pgmap v185: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 260 KiB/s wr, 34 op/s 2024-09-15T22:14:59.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.207+0000 7f54fc761640 1 -- 172.21.15.5:0/3811922540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54f407af80 msgr2=0x7f54f407b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:59.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.207+0000 7f54fc761640 1 --2- 172.21.15.5:0/3811922540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54f407af80 0x7f54f407b3c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009ca0 tx=0x7f54ec02a980 comp rx=0 tx=0).stop 2024-09-15T22:14:59.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.207+0000 7f54fc761640 1 -- 172.21.15.5:0/3811922540 shutdown_connections 2024-09-15T22:14:59.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.207+0000 7f54fc761640 1 --2- 172.21.15.5:0/3811922540 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f411b040 0x7f54f411d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.207+0000 7f54fc761640 1 --2- 172.21.15.5:0/3811922540 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54f407af80 0x7f54f407b3c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.207+0000 7f54fc761640 1 --2- 172.21.15.5:0/3811922540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f4082d50 0x7f54f407aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.207+0000 7f54fc761640 1 -- 172.21.15.5:0/3811922540 >> 172.21.15.5:0/3811922540 conn(0x7f54f4077a10 msgr2=0x7f54f4079e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:59.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.207+0000 7f54fc761640 1 -- 172.21.15.5:0/3811922540 shutdown_connections 2024-09-15T22:14:59.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.208+0000 7f54fc761640 1 -- 172.21.15.5:0/3811922540 wait complete. 2024-09-15T22:14:59.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.208+0000 7f54fc761640 1 Processor -- start 2024-09-15T22:14:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.209+0000 7f54fc761640 1 -- start start 2024-09-15T22:14:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.209+0000 7f54fc761640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f407af80 0x7f54f4073b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.209+0000 7f54fc761640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f4082d50 0x7f54f406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.209+0000 7f54fc761640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54f411b040 0x7f54f406e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.209+0000 7f54f9cfc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f4082d50 0x7f54f406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.209+0000 7f54f9cfc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f4082d50 0x7f54f406dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:59522/0 (socket says 172.21.15.5:59522) 2024-09-15T22:14:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.209+0000 7f54f9cfc640 1 -- 172.21.15.5:0/1615752671 learned_addr learned my addr 172.21.15.5:0/1615752671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:59.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.210+0000 7f54fc761640 1 -- 172.21.15.5:0/1615752671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54f406c660 con 0x7f54f407af80 2024-09-15T22:14:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.210+0000 7f54fc761640 1 -- 172.21.15.5:0/1615752671 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f54f406c4e0 con 0x7f54f4082d50 2024-09-15T22:14:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.210+0000 7f54fc761640 1 -- 172.21.15.5:0/1615752671 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f54f406c7e0 con 0x7f54f411b040 2024-09-15T22:14:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.210+0000 7f54facfe640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54f411b040 0x7f54f406e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.210+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f407af80 0x7f54f4073b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.211+0000 7f54fa4fd640 1 -- 172.21.15.5:0/1615752671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54f411b040 msgr2=0x7f54f406e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.211+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54f411b040 0x7f54f406e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.211+0000 7f54fa4fd640 1 -- 172.21.15.5:0/1615752671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f4082d50 msgr2=0x7f54f406dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.211+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f4082d50 0x7f54f406dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.211+0000 7f54fa4fd640 1 -- 172.21.15.5:0/1615752671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f406e8c0 con 0x7f54f407af80 2024-09-15T22:14:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.211+0000 7f54f9cfc640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f4082d50 0x7f54f406dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:14:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.212+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f407af80 0x7f54f4073b80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f54f000b3f0 tx=0x7f54f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.212+0000 7f54eb7fe640 1 -- 172.21.15.5:0/1615752671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f0015020 con 0x7f54f407af80 2024-09-15T22:14:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.212+0000 7f54fc761640 1 -- 172.21.15.5:0/1615752671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f41bc130 con 0x7f54f407af80 2024-09-15T22:14:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.212+0000 7f54fc761640 1 -- 172.21.15.5:0/1615752671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f41bc630 con 0x7f54f407af80 2024-09-15T22:14:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.212+0000 7f54eb7fe640 1 -- 172.21.15.5:0/1615752671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f0011d10 con 0x7f54f407af80 2024-09-15T22:14:59.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.212+0000 7f54eb7fe640 1 -- 172.21.15.5:0/1615752671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f000f2e0 con 0x7f54f407af80 2024-09-15T22:14:59.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.217+0000 7f54eb7fe640 1 -- 172.21.15.5:0/1615752671 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f54f0018020 con 0x7f54f407af80 2024-09-15T22:14:59.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.218+0000 7f54eb7fe640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f54e00a3500 0x7f54e00a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.218+0000 7f54eb7fe640 1 -- 172.21.15.5:0/1615752671 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f54f00d0810 con 0x7f54f407af80 2024-09-15T22:14:59.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.219+0000 7f54feb3a740 1 Processor -- start 2024-09-15T22:14:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.219+0000 7f54feb3a740 1 -- start start 2024-09-15T22:14:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.219+0000 7f54feb3a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f1f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.219+0000 7f54f9cfc640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f54e00a3500 0x7f54e00a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.219+0000 7f54feb3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f1f690 0x55ca08f29a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54feb3a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f29f50 0x55ca08f2c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54feb3a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca08e84280 con 0x55ca08f1f690 2024-09-15T22:14:59.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54feb3a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ca08e2bb10 con 0x55ca08f1ed70 2024-09-15T22:14:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54feb3a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ca08e80240 con 0x55ca08f29f50 2024-09-15T22:14:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54fa4fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f1f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54fa4fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f1f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:59526/0 (socket says 172.21.15.5:59526) 2024-09-15T22:14:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54fa4fd640 1 -- 172.21.15.5:0/337417308 learned_addr learned my addr 172.21.15.5:0/337417308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54facfe640 1 --2- 172.21.15.5:0/337417308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f29f50 0x55ca08f2c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54fa4fd640 1 -- 172.21.15.5:0/337417308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f29f50 msgr2=0x55ca08f2c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/337417308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f29f50 0x55ca08f2c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54fa4fd640 1 -- 172.21.15.5:0/337417308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f1f690 msgr2=0x55ca08f29a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/337417308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f1f690 0x55ca08f29a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54fa4fd640 1 -- 172.21.15.5:0/337417308 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca08e381f0 con 0x55ca08f1ed70 2024-09-15T22:14:59.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.220+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/337417308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f1f150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f54f0000c00 tx=0x7f54f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93c9b102ede41701 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.222+0000 7f54e8ff9640 1 -- 172.21.15.5:0/337417308 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f009c800 con 0x55ca08f1ed70 2024-09-15T22:14:59.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.222+0000 7f54e8ff9640 1 -- 172.21.15.5:0/337417308 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f00a0bb0 con 0x55ca08f1ed70 2024-09-15T22:14:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.227+0000 7f54f9cfc640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f54e00a3500 0x7f54e00a59c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f54f406f450 tx=0x7f54ec02ff70 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.227+0000 7f54feb3a740 1 -- 172.21.15.5:0/337417308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 msgr2=0x55ca08f1f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.227+0000 7f54feb3a740 1 --2- 172.21.15.5:0/337417308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f1f150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f54f0000c00 tx=0x7f54f00046c0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.229+0000 7f54feb3a740 1 -- 172.21.15.5:0/337417308 shutdown_connections 2024-09-15T22:14:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.229+0000 7f54feb3a740 1 --2- 172.21.15.5:0/337417308 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f29f50 0x55ca08f2c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.229+0000 7f54feb3a740 1 --2- 172.21.15.5:0/337417308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f1f690 0x55ca08f29a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.229+0000 7f54feb3a740 1 --2- 172.21.15.5:0/337417308 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f1f150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.229+0000 7f54feb3a740 1 -- 172.21.15.5:0/337417308 >> 172.21.15.5:0/337417308 conn(0x55ca08f1b300 msgr2=0x55ca08f1b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:59.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.231+0000 7f54feb3a740 1 -- 172.21.15.5:0/337417308 shutdown_connections 2024-09-15T22:14:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.232+0000 7f54feb3a740 1 -- 172.21.15.5:0/337417308 wait complete. 2024-09-15T22:14:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.233+0000 7f54feb3a740 1 -- start start 2024-09-15T22:14:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.233+0000 7f54feb3a740 1 Processor -- start 2024-09-15T22:14:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.234+0000 7f54feb3a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 0x55ca08f29a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.234+0000 7f54feb3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f1f690 0x55ca08f26d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.234+0000 7f54fa4fd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 0x55ca08f29a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.234+0000 7f54fa4fd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 0x55ca08f29a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:34572/0 (socket says 172.21.15.5:34572) 2024-09-15T22:14:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.234+0000 7f54fa4fd640 1 -- 172.21.15.5:0/315779423 learned_addr learned my addr 172.21.15.5:0/315779423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.234+0000 7f54feb3a740 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f29f50 0x55ca08f274e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.234+0000 7f54feb3a740 1 -- 172.21.15.5:0/315779423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ca08f2cb70 con 0x55ca08f1f690 2024-09-15T22:14:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.234+0000 7f54feb3a740 1 -- 172.21.15.5:0/315779423 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ca08e3ef10 con 0x55ca08f29f50 2024-09-15T22:14:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.234+0000 7f54feb3a740 1 -- 172.21.15.5:0/315779423 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ca08f2ccf0 con 0x55ca08f1ed70 2024-09-15T22:14:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.235+0000 7f54f9cfc640 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f1f690 0x55ca08f26d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.235+0000 7f54feb3a740 10 client.0 _ll_register_callbacks cb 0x55ca08e07a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.235+0000 7f54facfe640 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f29f50 0x55ca08f274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.235+0000 7f54fa4fd640 1 -- 172.21.15.5:0/315779423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f29f50 msgr2=0x55ca08f274e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.235+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f29f50 0x55ca08f274e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.235+0000 7f54fa4fd640 1 -- 172.21.15.5:0/315779423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f1f690 msgr2=0x55ca08f26d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.235+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f1f690 0x55ca08f26d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.236+0000 7f54fa4fd640 1 -- 172.21.15.5:0/315779423 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca08bf56a0 con 0x55ca08f1ed70 2024-09-15T22:14:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.236+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 0x55ca08f29a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54f0004810 tx=0x7f54f0099080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2d4ded71f9333c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.236+0000 7f54ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:14:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.237+0000 7f54ddffb640 1 -- 172.21.15.5:0/315779423 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f00b4030 con 0x55ca08f1ed70 2024-09-15T22:14:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.237+0000 7f54ddffb640 1 -- 172.21.15.5:0/315779423 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f54f009ab00 con 0x55ca08f1ed70 2024-09-15T22:14:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.237+0000 7f54feb3a740 1 -- 172.21.15.5:0/315779423 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f00a0bb0 con 0x55ca08f1ed70 2024-09-15T22:14:59.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.241+0000 7f54ddffb640 1 -- 172.21.15.5:0/315779423 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54f00d05f0 con 0x55ca08f1ed70 2024-09-15T22:14:59.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.241+0000 7f54feb3a740 10 client.26134 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:59.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.241+0000 7f54feb3a740 1 -- 172.21.15.5:0/315779423 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ca08f2cb70 con 0x55ca08f1ed70 2024-09-15T22:14:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.242+0000 7f54ddffb640 1 -- 172.21.15.5:0/315779423 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f00a0bb0 con 0x55ca08f1ed70 2024-09-15T22:14:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.242+0000 7f54ddffb640 1 -- 172.21.15.5:0/315779423 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f54f0085b00 con 0x55ca08f1ed70 2024-09-15T22:14:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.242+0000 7f54feb3a740 10 client.26134 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.242+0000 7f54feb3a740 10 client.26134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.242+0000 7f54feb3a740 20 client.26134 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.242+0000 7f54feb3a740 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ca08f39b80 0x55ca08f3bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.242+0000 7f54feb3a740 4 client.26134 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:14:59.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.242+0000 7f54feb3a740 1 -- 172.21.15.5:0/315779423 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca08e917f0 con 0x55ca08f39b80 2024-09-15T22:14:59.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.242+0000 7f54f9cfc640 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ca08f39b80 0x55ca08f3bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.243+0000 7f54f9cfc640 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ca08f39b80 0x55ca08f3bfe0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821f3c0d18c453c6 server_cookie=ffa4bf53338ae100 in_seq=0 out_seq=0 2024-09-15T22:14:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.243+0000 7f54ddffb640 10 client.26134 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.243+0000 7f54ddffb640 1 -- 172.21.15.5:0/315779423 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca08e917f0 con 0x55ca08f39b80 2024-09-15T22:14:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.243+0000 7f54ddffb640 10 client.26134 handle_command_reply: tid=0 2024-09-15T22:14:59.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.243+0000 7f54feb3a740 1 client.26134 shutdown 2024-09-15T22:14:59.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.244+0000 7f54feb3a740 1 -- 172.21.15.5:0/315779423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 msgr2=0x55ca08f29a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:59.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.244+0000 7f54feb3a740 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 0x55ca08f29a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f54f0004810 tx=0x7f54f0099080 comp rx=0 tx=0).stop 2024-09-15T22:14:59.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.245+0000 7f54feb3a740 1 -- 172.21.15.5:0/315779423 shutdown_connections 2024-09-15T22:14:59.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.245+0000 7f54feb3a740 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ca08f39b80 0x55ca08f3bfe0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.245+0000 7f54feb3a740 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f29f50 0x55ca08f274e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.245+0000 7f54ddffb640 0 client.26134 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.245+0000 7f54feb3a740 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f1f690 0x55ca08f26d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.245+0000 7f54feb3a740 1 --2- 172.21.15.5:0/315779423 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 0x55ca08f29a00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.245+0000 7f54feb3a740 1 -- 172.21.15.5:0/315779423 >> 172.21.15.5:0/315779423 conn(0x55ca08f1b300 msgr2=0x55ca08f21030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.245+0000 7f54feb3a740 1 -- 172.21.15.5:0/315779423 shutdown_connections 2024-09-15T22:14:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.245+0000 7f54feb3a740 1 -- 172.21.15.5:0/315779423 wait complete. 2024-09-15T22:14:59.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.246+0000 7f54feb3a740 20 client.26134 trim_cache size 0 max 16384 2024-09-15T22:14:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.270+0000 7f54feb3a740 1 Processor -- start 2024-09-15T22:14:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.270+0000 7f54feb3a740 1 -- start start 2024-09-15T22:14:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.270+0000 7f54feb3a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f21740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.270+0000 7f54feb3a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1f690 0x55ca08f418b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.270+0000 7f54feb3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f29f50 0x55ca08f41df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.271+0000 7f54fa4fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f21740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.271+0000 7f54fa4fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f21740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:59548/0 (socket says 172.21.15.5:59548) 2024-09-15T22:14:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.271+0000 7f54f9cfc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1f690 0x55ca08f418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.271+0000 7f54f9cfc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1f690 0x55ca08f418b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:34582/0 (socket says 172.21.15.5:34582) 2024-09-15T22:14:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.271+0000 7f54facfe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f29f50 0x55ca08f41df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.271+0000 7f54facfe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f29f50 0x55ca08f41df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33326/0 (socket says 172.21.15.5:33326) 2024-09-15T22:14:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.271+0000 7f54fa4fd640 1 -- 172.21.15.5:0/2463735079 learned_addr learned my addr 172.21.15.5:0/2463735079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.271+0000 7f54feb3a740 1 -- 172.21.15.5:0/2463735079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54d80128c0 con 0x55ca08f29f50 2024-09-15T22:14:59.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.271+0000 7f54feb3a740 1 -- 172.21.15.5:0/2463735079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f54d8010190 con 0x55ca08f1ed70 2024-09-15T22:14:59.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.271+0000 7f54feb3a740 1 -- 172.21.15.5:0/2463735079 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f54d800dfe0 con 0x55ca08f1f690 2024-09-15T22:14:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.272+0000 7f54fa4fd640 1 -- 172.21.15.5:0/2463735079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1f690 msgr2=0x55ca08f418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.272+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/2463735079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1f690 0x55ca08f418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.272+0000 7f54fa4fd640 1 -- 172.21.15.5:0/2463735079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f29f50 msgr2=0x55ca08f41df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.272+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/2463735079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f29f50 0x55ca08f41df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.272+0000 7f54fa4fd640 1 -- 172.21.15.5:0/2463735079 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca08f27850 con 0x55ca08f1ed70 2024-09-15T22:14:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.272+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/2463735079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f21740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f54f000bdb0 tx=0x7f54f00ce100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d05ebd7636598c55 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.272+0000 7f54dffff640 1 -- 172.21.15.5:0/2463735079 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f00b4070 con 0x55ca08f1ed70 2024-09-15T22:14:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.272+0000 7f54dffff640 1 -- 172.21.15.5:0/2463735079 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f00a0bb0 con 0x55ca08f1ed70 2024-09-15T22:14:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.273+0000 7f54dffff640 1 -- 172.21.15.5:0/2463735079 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f00987b0 con 0x55ca08f1ed70 2024-09-15T22:14:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.273+0000 7f54feb3a740 1 -- 172.21.15.5:0/2463735079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 msgr2=0x55ca08f21740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:59.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.273+0000 7f54feb3a740 1 --2- 172.21.15.5:0/2463735079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f21740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f54f000bdb0 tx=0x7f54f00ce100 comp rx=0 tx=0).stop 2024-09-15T22:14:59.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.273+0000 7f54feb3a740 1 -- 172.21.15.5:0/2463735079 shutdown_connections 2024-09-15T22:14:59.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.273+0000 7f54feb3a740 1 --2- 172.21.15.5:0/2463735079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f29f50 0x55ca08f41df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.273+0000 7f54feb3a740 1 --2- 172.21.15.5:0/2463735079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1f690 0x55ca08f418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.273+0000 7f54feb3a740 1 --2- 172.21.15.5:0/2463735079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1ed70 0x55ca08f21740 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.273+0000 7f54feb3a740 1 -- 172.21.15.5:0/2463735079 >> 172.21.15.5:0/2463735079 conn(0x55ca08f1b300 msgr2=0x55ca08f25620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:59.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.273+0000 7f54feb3a740 1 -- 172.21.15.5:0/2463735079 shutdown_connections 2024-09-15T22:14:59.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.274+0000 7f54feb3a740 1 -- 172.21.15.5:0/2463735079 wait complete. 2024-09-15T22:14:59.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.274+0000 7f54feb3a740 1 -- start start 2024-09-15T22:14:59.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.275+0000 7f54feb3a740 1 Processor -- start 2024-09-15T22:14:59.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.275+0000 7f54feb3a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 0x55ca08f324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.275+0000 7f54feb3a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1f690 0x55ca08f2d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.275+0000 7f54feb3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f29f50 0x55ca08f2dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.275+0000 7f54feb3a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54d800e650 con 0x55ca08f29f50 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.275+0000 7f54feb3a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f54d800fac0 con 0x55ca08f1f690 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.275+0000 7f54feb3a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f54d800ed20 con 0x55ca08f1ed70 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.275+0000 7f54feb3a740 10 client.0 _ll_register_callbacks cb 0x7f54f009c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.275+0000 7f54f9cfc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1f690 0x55ca08f2d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.275+0000 7f54f9cfc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1f690 0x55ca08f2d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:59560/0 (socket says 172.21.15.5:59560) 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.275+0000 7f54f9cfc640 1 -- 172.21.15.5:0/2322107326 learned_addr learned my addr 172.21.15.5:0/2322107326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.276+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 0x55ca08f324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.276+0000 7f54facfe640 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f29f50 0x55ca08f2dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.276+0000 7f54f9cfc640 1 -- 172.21.15.5:0/2322107326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 msgr2=0x55ca08f324b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.276+0000 7f54f9cfc640 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 0x55ca08f324b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.276+0000 7f54f9cfc640 1 -- 172.21.15.5:0/2322107326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f29f50 msgr2=0x55ca08f2dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.276+0000 7f54f9cfc640 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f29f50 0x55ca08f2dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.276+0000 7f54f9cfc640 1 -- 172.21.15.5:0/2322107326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca08f2e2e0 con 0x55ca08f1f690 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.277+0000 7f54f9cfc640 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1f690 0x55ca08f2d7e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f54ec030040 tx=0x7f54ec02a980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=714173813b08645a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.277+0000 7f54ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.277+0000 7f54ddffb640 1 -- 172.21.15.5:0/2322107326 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec034040 con 0x55ca08f1f690 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.277+0000 7f54ddffb640 1 -- 172.21.15.5:0/2322107326 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f54ec004240 con 0x55ca08f1f690 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.277+0000 7f54feb3a740 1 -- 172.21.15.5:0/2322107326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f00a0bb0 con 0x55ca08f1f690 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.277+0000 7f54ddffb640 1 -- 172.21.15.5:0/2322107326 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54ec061050 con 0x55ca08f1f690 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.277+0000 7f54feb3a740 10 client.26177 fetch_fsmap learned FSMap version 25 2024-09-15T22:14:59.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.277+0000 7f54feb3a740 1 -- 172.21.15.5:0/2322107326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ca08fa46e0 con 0x55ca08f1f690 2024-09-15T22:14:59.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.278+0000 7f54ddffb640 1 -- 172.21.15.5:0/2322107326 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ec052410 con 0x55ca08f1f690 2024-09-15T22:14:59.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.278+0000 7f54ddffb640 1 -- 172.21.15.5:0/2322107326 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f54ec046430 con 0x55ca08f1f690 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.278+0000 7f54feb3a740 10 client.26177 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.278+0000 7f54feb3a740 10 client.26177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.278+0000 7f54feb3a740 20 client.26177 populate_metadata read hostname 'smithi005' 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.278+0000 7f54feb3a740 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ca08f39b80 0x55ca08fa6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.278+0000 7f54feb3a740 4 client.26177 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.278+0000 7f54feb3a740 1 -- 172.21.15.5:0/2322107326 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f54f00a0bb0 con 0x55ca08f39b80 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.278+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ca08f39b80 0x55ca08fa6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.279+0000 7f54fa4fd640 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ca08f39b80 0x55ca08fa6ed0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e0a81cd0fa8d4d server_cookie=8cff64b0f25c0a56 in_seq=0 out_seq=0 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.279+0000 7f54ddffb640 10 client.26177 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.280+0000 7f54ddffb640 1 -- 172.21.15.5:0/2322107326 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f54f00a0bb0 con 0x55ca08f39b80 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.280+0000 7f54ddffb640 10 client.26177 handle_command_reply: tid=0 2024-09-15T22:14:59.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.280+0000 7f54feb3a740 1 client.26177 shutdown 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54feb3a740 1 -- 172.21.15.5:0/2322107326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1f690 msgr2=0x55ca08f2d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54feb3a740 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1f690 0x55ca08f2d7e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f54ec030040 tx=0x7f54ec02a980 comp rx=0 tx=0).stop 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54feb3a740 1 -- 172.21.15.5:0/2322107326 shutdown_connections 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54feb3a740 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca08f29f50 0x55ca08f2dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54feb3a740 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ca08f39b80 0x55ca08fa6ed0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54feb3a740 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ca08f1f690 0x55ca08f2d7e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54ddffb640 0 client.26177 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54feb3a740 1 --2- 172.21.15.5:0/2322107326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ca08f1ed70 0x55ca08f324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54feb3a740 1 -- 172.21.15.5:0/2322107326 >> 172.21.15.5:0/2322107326 conn(0x55ca08f1b300 msgr2=0x55ca08f19bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54feb3a740 1 -- 172.21.15.5:0/2322107326 shutdown_connections 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54feb3a740 1 -- 172.21.15.5:0/2322107326 wait complete. 2024-09-15T22:14:59.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.281+0000 7f54feb3a740 20 client.26177 trim_cache size 0 max 16384 2024-09-15T22:14:59.286 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:14:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:14:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "776d4c79-9925-4f0b-a7c1-b1d387f8ef9a", 2024-09-15T22:14:59.286 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:14:59.286 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:14:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 -- 172.21.15.5:0/1615752671 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f54e00a3500 msgr2=0x7f54e00a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f54e00a3500 0x7f54e00a59c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f54f406f450 tx=0x7f54ec02ff70 comp rx=0 tx=0).stop 2024-09-15T22:14:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 -- 172.21.15.5:0/1615752671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f407af80 msgr2=0x7f54f4073b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:14:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f407af80 0x7f54f4073b80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f54f000b3f0 tx=0x7f54f000b8b0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 -- 172.21.15.5:0/1615752671 shutdown_connections 2024-09-15T22:14:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f54e00a3500 0x7f54e00a59c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f54f411b040 0x7f54f406e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f54f4082d50 0x7f54f406dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 --2- 172.21.15.5:0/1615752671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54f407af80 0x7f54f4073b80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:14:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 -- 172.21.15.5:0/1615752671 >> 172.21.15.5:0/1615752671 conn(0x7f54f4077a10 msgr2=0x7f54f4081240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:14:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 -- 172.21.15.5:0/1615752671 shutdown_connections 2024-09-15T22:14:59.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:14:59.282+0000 7f54fc761640 1 -- 172.21.15.5:0/1615752671 wait complete. 2024-09-15T22:14:59.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:14:58 smithi099 ceph-mon[34892]: pgmap v185: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 260 KiB/s wr, 34 op/s 2024-09-15T22:14:59.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:14:58 smithi129 ceph-mon[35314]: pgmap v185: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 260 KiB/s wr, 34 op/s 2024-09-15T22:14:59.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 330.0 s, 1094.5 tps, lat 541.391 ms stddev 3391.588 2024-09-15T22:15:00.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:15:01.168 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:01.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:00 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:15:01.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:00 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:01.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:00 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:15:01.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:00 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:01.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:00 smithi005 ceph-mon[30944]: pgmap v186: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 260 KiB/s wr, 34 op/s 2024-09-15T22:15:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:00 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:15:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:00 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:00 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:15:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:00 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:00 smithi099 ceph-mon[34892]: pgmap v186: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 260 KiB/s wr, 34 op/s 2024-09-15T22:15:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:00 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:15:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:00 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:00 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:15:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:00 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:15:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:00 smithi129 ceph-mon[35314]: pgmap v186: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 260 KiB/s wr, 34 op/s 2024-09-15T22:15:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.780+0000 7fe0b76b7640 1 -- 172.21.15.5:0/3384879071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b0102ef0 msgr2=0x7fe0b010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.780+0000 7fe0b76b7640 1 --2- 172.21.15.5:0/3384879071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b0102ef0 0x7fe0b010da40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009900 tx=0x7fe0a002a480 comp rx=0 tx=0).stop 2024-09-15T22:15:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.780+0000 7fe0b76b7640 1 -- 172.21.15.5:0/3384879071 shutdown_connections 2024-09-15T22:15:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.780+0000 7fe0b76b7640 1 --2- 172.21.15.5:0/3384879071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b0102ef0 0x7fe0b010da40 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.780+0000 7fe0b76b7640 1 --2- 172.21.15.5:0/3384879071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b0102570 0x7fe0b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.780+0000 7fe0b76b7640 1 --2- 172.21.15.5:0/3384879071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0b0100e20 0x7fe0b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.780+0000 7fe0b76b7640 1 -- 172.21.15.5:0/3384879071 >> 172.21.15.5:0/3384879071 conn(0x7fe0b00fc7d0 msgr2=0x7fe0b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.781+0000 7fe0b76b7640 1 -- 172.21.15.5:0/3384879071 shutdown_connections 2024-09-15T22:15:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.781+0000 7fe0b76b7640 1 -- 172.21.15.5:0/3384879071 wait complete. 2024-09-15T22:15:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.781+0000 7fe0b76b7640 1 Processor -- start 2024-09-15T22:15:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.781+0000 7fe0b76b7640 1 -- start start 2024-09-15T22:15:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.782+0000 7fe0b76b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b0100e20 0x7fe0b0196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.782+0000 7fe0b76b7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0b0102570 0x7fe0b0196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.782+0000 7fe0b76b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b0102ef0 0x7fe0b01a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.782+0000 7fe0b76b7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0110fd0 con 0x7fe0b0102ef0 2024-09-15T22:15:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.782+0000 7fe0b76b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0110e50 con 0x7fe0b0100e20 2024-09-15T22:15:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.782+0000 7fe0b76b7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b0111150 con 0x7fe0b0102570 2024-09-15T22:15:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.783+0000 7fe0b5c54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b0102ef0 0x7fe0b01a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.783+0000 7fe0b5c54640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b0102ef0 0x7fe0b01a20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33366/0 (socket says 172.21.15.5:33366) 2024-09-15T22:15:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.783+0000 7fe0b5c54640 1 -- 172.21.15.5:0/3270551265 learned_addr learned my addr 172.21.15.5:0/3270551265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.783+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0b0102570 0x7fe0b0196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.783+0000 7fe0b5453640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b0100e20 0x7fe0b0196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.784+0000 7fe0b5c54640 1 -- 172.21.15.5:0/3270551265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0b0102570 msgr2=0x7fe0b0196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.784+0000 7fe0b5c54640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0b0102570 0x7fe0b0196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.784+0000 7fe0b5c54640 1 -- 172.21.15.5:0/3270551265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b0100e20 msgr2=0x7fe0b0196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.784+0000 7fe0b5c54640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b0100e20 0x7fe0b0196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.784+0000 7fe0b5c54640 1 -- 172.21.15.5:0/3270551265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b01a2630 con 0x7fe0b0102ef0 2024-09-15T22:15:01.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.791+0000 7fe0b5c54640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b0102ef0 0x7fe0b01a20f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009230 tx=0x7fe0a0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:01.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.794+0000 7fe0a67fc640 1 -- 172.21.15.5:0/3270551265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a003f070 con 0x7fe0b0102ef0 2024-09-15T22:15:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.794+0000 7fe0a67fc640 1 -- 172.21.15.5:0/3270551265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0a0031c20 con 0x7fe0b0102ef0 2024-09-15T22:15:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.794+0000 7fe0b76b7640 1 -- 172.21.15.5:0/3270551265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b01a28c0 con 0x7fe0b0102ef0 2024-09-15T22:15:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.794+0000 7fe0b76b7640 1 -- 172.21.15.5:0/3270551265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b01a2ce0 con 0x7fe0b0102ef0 2024-09-15T22:15:01.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.795+0000 7fe0a67fc640 1 -- 172.21.15.5:0/3270551265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0035030 con 0x7fe0b0102ef0 2024-09-15T22:15:01.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.799+0000 7fe0b9a90740 1 Processor -- start 2024-09-15T22:15:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.800+0000 7fe0b9a90740 1 -- start start 2024-09-15T22:15:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.801+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b31790 0x55a898b31b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.801+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b320b0 0x55a898b3c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.801+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b3c970 0x55a898b3ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.801+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a898a972e0 con 0x55a898b3c970 2024-09-15T22:15:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.801+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a898a3ece0 con 0x55a898b31790 2024-09-15T22:15:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.801+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a898a932a0 con 0x55a898b320b0 2024-09-15T22:15:01.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.801+0000 7fe0a67fc640 1 -- 172.21.15.5:0/3270551265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe0a0042070 con 0x7fe0b0102ef0 2024-09-15T22:15:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.802+0000 7fe0a67fc640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe09c0a35c0 0x7fe09c0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.802+0000 7fe0a67fc640 1 -- 172.21.15.5:0/3270551265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe0a00f0cc0 con 0x7fe0b0102ef0 2024-09-15T22:15:01.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.807+0000 7fe0b5c54640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe09c0a35c0 0x7fe09c0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.807+0000 7fe0b5453640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b320b0 0x55a898b3c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.807+0000 7fe0b5453640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b320b0 0x55a898b3c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:34616/0 (socket says 172.21.15.5:34616) 2024-09-15T22:15:01.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.807+0000 7fe0b5453640 1 -- 172.21.15.5:0/3373786660 learned_addr learned my addr 172.21.15.5:0/3373786660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:01.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.807+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/3373786660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b3c970 0x55a898b3ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.807+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/3373786660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b31790 0x55a898b31b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.807+0000 7fe0b5453640 1 -- 172.21.15.5:0/3373786660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b31790 msgr2=0x55a898b31b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:01.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.807+0000 7fe0b5453640 1 --2- 172.21.15.5:0/3373786660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b31790 0x55a898b31b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.807+0000 7fe0b5453640 1 -- 172.21.15.5:0/3373786660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b3c970 msgr2=0x55a898b3ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:01.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.807+0000 7fe0b5453640 1 --2- 172.21.15.5:0/3373786660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b3c970 0x55a898b3ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.807+0000 7fe0b5453640 1 -- 172.21.15.5:0/3373786660 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a898a23440 con 0x55a898b320b0 2024-09-15T22:15:01.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.808+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/3373786660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b31790 0x55a898b31b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:15:01.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.808+0000 7fe0b5453640 1 --2- 172.21.15.5:0/3373786660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b320b0 0x55a898b3c430 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80039f0 tx=0x7fe0a8003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda0dd7121d3e703 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.811+0000 7fe09affd640 1 -- 172.21.15.5:0/3373786660 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8004b20 con 0x55a898b320b0 2024-09-15T22:15:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.812+0000 7fe09affd640 1 -- 172.21.15.5:0/3373786660 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0a8004cc0 con 0x55a898b320b0 2024-09-15T22:15:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.812+0000 7fe0b9a90740 1 -- 172.21.15.5:0/3373786660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b320b0 msgr2=0x55a898b3c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.812+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/3373786660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b320b0 0x55a898b3c430 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80039f0 tx=0x7fe0a8003eb0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.814+0000 7fe0b5c54640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe09c0a35c0 0x7fe09c0a5a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00ee070 tx=0x7fe0a0005fd0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.814+0000 7fe0b9a90740 1 -- 172.21.15.5:0/3373786660 shutdown_connections 2024-09-15T22:15:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.814+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/3373786660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b3c970 0x55a898b3ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.814+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/3373786660 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b320b0 0x55a898b3c430 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80039f0 tx=0x7fe0a8003eb0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.814+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/3373786660 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b31790 0x55a898b31b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.814+0000 7fe0b9a90740 1 -- 172.21.15.5:0/3373786660 >> 172.21.15.5:0/3373786660 conn(0x55a898b2dd20 msgr2=0x55a898b2e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.814+0000 7fe0b9a90740 1 -- 172.21.15.5:0/3373786660 shutdown_connections 2024-09-15T22:15:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.817+0000 7fe0b9a90740 1 -- 172.21.15.5:0/3373786660 wait complete. 2024-09-15T22:15:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.818+0000 7fe0b9a90740 1 -- start start 2024-09-15T22:15:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.818+0000 7fe0b9a90740 1 Processor -- start 2024-09-15T22:15:01.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.820+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b31790 0x55a898b3c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.820+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b3c970 0x55a898b39600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.820+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b39b90 0x55a898b39fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.820+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a898b3f4b0 con 0x55a898b31790 2024-09-15T22:15:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.820+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a898a52050 con 0x55a898b39b90 2024-09-15T22:15:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.820+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a898b3f630 con 0x55a898b3c970 2024-09-15T22:15:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.820+0000 7fe0b9a90740 10 client.0 _ll_register_callbacks cb 0x55a898a92160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.820+0000 7fe0b5453640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b3c970 0x55a898b39600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.820+0000 7fe0b5453640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b3c970 0x55a898b39600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:34618/0 (socket says 172.21.15.5:34618) 2024-09-15T22:15:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.820+0000 7fe0b5453640 1 -- 172.21.15.5:0/4107116579 learned_addr learned my addr 172.21.15.5:0/4107116579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.823+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b39b90 0x55a898b39fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.826+0000 7fe0b5453640 1 -- 172.21.15.5:0/4107116579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b39b90 msgr2=0x55a898b39fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.826+0000 7fe0b5453640 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b39b90 0x55a898b39fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.826+0000 7fe0b5453640 1 -- 172.21.15.5:0/4107116579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b31790 msgr2=0x55a898b3c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:01.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.826+0000 7fe0b5453640 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b31790 0x55a898b3c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.826+0000 7fe0b5453640 1 -- 172.21.15.5:0/4107116579 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8988083c0 con 0x55a898b3c970 2024-09-15T22:15:01.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.831+0000 7fe0b5453640 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b3c970 0x55a898b39600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55a898b2eb80 tx=0x7fe0a8005980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16896ae5d56cf141 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:01.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.832+0000 7fe098ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:15:01.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.835+0000 7fe098ff9640 1 -- 172.21.15.5:0/4107116579 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8005bd0 con 0x55a898b3c970 2024-09-15T22:15:01.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.835+0000 7fe098ff9640 1 -- 172.21.15.5:0/4107116579 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe0a8012ca0 con 0x55a898b3c970 2024-09-15T22:15:01.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.836+0000 7fe0b9a90740 1 -- 172.21.15.5:0/4107116579 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a898aa4850 con 0x55a898b3c970 2024-09-15T22:15:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.841+0000 7fe098ff9640 1 -- 172.21.15.5:0/4107116579 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe0a8024400 con 0x55a898b3c970 2024-09-15T22:15:01.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.841+0000 7fe0b9a90740 10 client.26152 fetch_fsmap learned FSMap version 25 2024-09-15T22:15:01.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.841+0000 7fe0b9a90740 1 -- 172.21.15.5:0/4107116579 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a898b4b320 con 0x55a898b3c970 2024-09-15T22:15:01.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.842+0000 7fe098ff9640 1 -- 172.21.15.5:0/4107116579 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a898aa4850 con 0x55a898b3c970 2024-09-15T22:15:01.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.842+0000 7fe098ff9640 1 -- 172.21.15.5:0/4107116579 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe0a800f040 con 0x55a898b3c970 2024-09-15T22:15:01.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.842+0000 7fe0b9a90740 10 client.26152 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:15:01.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.842+0000 7fe0b9a90740 10 client.26152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:01.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.842+0000 7fe0b9a90740 20 client.26152 populate_metadata read hostname 'smithi005' 2024-09-15T22:15:01.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.842+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a898b4cb40 0x55a898b4ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.842+0000 7fe0b9a90740 4 client.26152 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:01.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.842+0000 7fe0b9a90740 1 -- 172.21.15.5:0/4107116579 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a898a9ed60 con 0x55a898b4cb40 2024-09-15T22:15:01.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.848+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a898b4cb40 0x55a898b4ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.850+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a898b4cb40 0x55a898b4ef60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b373c8d41f45762 server_cookie=b3b7020b98f53f5b in_seq=0 out_seq=0 2024-09-15T22:15:01.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.850+0000 7fe098ff9640 10 client.26152 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:01.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.851+0000 7fe098ff9640 1 -- 172.21.15.5:0/4107116579 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a898a9ed60 con 0x55a898b4cb40 2024-09-15T22:15:01.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.851+0000 7fe098ff9640 10 client.26152 handle_command_reply: tid=0 2024-09-15T22:15:01.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.851+0000 7fe0b9a90740 1 client.26152 shutdown 2024-09-15T22:15:01.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.855+0000 7fe0b9a90740 1 -- 172.21.15.5:0/4107116579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b3c970 msgr2=0x55a898b39600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:01.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.855+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b3c970 0x55a898b39600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55a898b2eb80 tx=0x7fe0a8005980 comp rx=0 tx=0).stop 2024-09-15T22:15:01.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.855+0000 7fe0b9a90740 1 -- 172.21.15.5:0/4107116579 shutdown_connections 2024-09-15T22:15:01.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.855+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a898b4cb40 0x55a898b4ef60 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.855+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b39b90 0x55a898b39fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.855+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b3c970 0x55a898b39600 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x55a898b2eb80 tx=0x7fe0a8005980 comp rx=0 tx=0).stop 2024-09-15T22:15:01.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.855+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/4107116579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b31790 0x55a898b3c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.855+0000 7fe0b9a90740 1 -- 172.21.15.5:0/4107116579 >> 172.21.15.5:0/4107116579 conn(0x55a898b2dd20 msgr2=0x55a898b33a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:01.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.855+0000 7fe098ff9640 0 client.26152 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:01.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.856+0000 7fe0b9a90740 1 -- 172.21.15.5:0/4107116579 shutdown_connections 2024-09-15T22:15:01.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.856+0000 7fe0b9a90740 1 -- 172.21.15.5:0/4107116579 wait complete. 2024-09-15T22:15:01.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.856+0000 7fe0b9a90740 20 client.26152 trim_cache size 0 max 16384 2024-09-15T22:15:01.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.890+0000 7fe0b9a90740 1 Processor -- start 2024-09-15T22:15:01.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.890+0000 7fe0b9a90740 1 -- start start 2024-09-15T22:15:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.891+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b31790 0x55a898b34ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.891+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b39b90 0x55a898b54830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.891+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b48760 0x55a898b54d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.891+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b01af860 con 0x55a898b31790 2024-09-15T22:15:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.891+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b01ad170 con 0x55a898b39b90 2024-09-15T22:15:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.891+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b01975d0 con 0x55a898b48760 2024-09-15T22:15:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.893+0000 7fe0b5453640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b39b90 0x55a898b54830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.893+0000 7fe0b5453640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b39b90 0x55a898b54830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:59610/0 (socket says 172.21.15.5:59610) 2024-09-15T22:15:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.893+0000 7fe0b5453640 1 -- 172.21.15.5:0/1142548711 learned_addr learned my addr 172.21.15.5:0/1142548711 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.893+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/1142548711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b48760 0x55a898b54d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.893+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/1142548711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b31790 0x55a898b34ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.896+0000 7fe0b4c52640 1 -- 172.21.15.5:0/1142548711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b39b90 msgr2=0x55a898b54830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:01.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.896+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/1142548711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b39b90 0x55a898b54830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.896+0000 7fe0b4c52640 1 -- 172.21.15.5:0/1142548711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b31790 msgr2=0x55a898b34ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:01.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.896+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/1142548711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b31790 0x55a898b34ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.896+0000 7fe0b4c52640 1 -- 172.21.15.5:0/1142548711 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a898b3a320 con 0x55a898b48760 2024-09-15T22:15:01.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.897+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/1142548711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b48760 0x55a898b54d70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55a898b4f770 tx=0x7fe0ac012040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e75664b114836d98 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.900+0000 7fe09a7fc640 1 -- 172.21.15.5:0/1142548711 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac005020 con 0x55a898b48760 2024-09-15T22:15:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.900+0000 7fe09a7fc640 1 -- 172.21.15.5:0/1142548711 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac004730 con 0x55a898b48760 2024-09-15T22:15:01.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.900+0000 7fe09a7fc640 1 -- 172.21.15.5:0/1142548711 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac010690 con 0x55a898b48760 2024-09-15T22:15:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.902+0000 7fe0b9a90740 1 -- 172.21.15.5:0/1142548711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b48760 msgr2=0x55a898b54d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.902+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/1142548711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b48760 0x55a898b54d70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x55a898b4f770 tx=0x7fe0ac012040 comp rx=0 tx=0).stop 2024-09-15T22:15:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.902+0000 7fe0b9a90740 1 -- 172.21.15.5:0/1142548711 shutdown_connections 2024-09-15T22:15:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.902+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/1142548711 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b48760 0x55a898b54d70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.902+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/1142548711 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b39b90 0x55a898b54830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.902+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/1142548711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b31790 0x55a898b34ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.902+0000 7fe0b9a90740 1 -- 172.21.15.5:0/1142548711 >> 172.21.15.5:0/1142548711 conn(0x55a898b2dd20 msgr2=0x55a898b343d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.905+0000 7fe0b9a90740 1 -- 172.21.15.5:0/1142548711 shutdown_connections 2024-09-15T22:15:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.905+0000 7fe0b9a90740 1 -- 172.21.15.5:0/1142548711 wait complete. 2024-09-15T22:15:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.906+0000 7fe0b9a90740 1 -- start start 2024-09-15T22:15:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.906+0000 7fe0b9a90740 1 Processor -- start 2024-09-15T22:15:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.906+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b31790 0x55a898b44ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.906+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b39b90 0x55a898b44ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.906+0000 7fe0b9a90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b48760 0x55a898b418a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.906+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b01ab630 con 0x55a898b48760 2024-09-15T22:15:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.906+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b01acaa0 con 0x55a898b31790 2024-09-15T22:15:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.907+0000 7fe0b9a90740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b01abd00 con 0x55a898b39b90 2024-09-15T22:15:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.907+0000 7fe0b9a90740 10 client.0 _ll_register_callbacks cb 0x7fe0a8009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.907+0000 7fe0b5453640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b39b90 0x55a898b44ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.907+0000 7fe0b5453640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b39b90 0x55a898b44ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:34646/0 (socket says 172.21.15.5:34646) 2024-09-15T22:15:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.907+0000 7fe0b5453640 1 -- 172.21.15.5:0/2931407473 learned_addr learned my addr 172.21.15.5:0/2931407473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.908+0000 7fe0b5453640 1 -- 172.21.15.5:0/2931407473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b31790 msgr2=0x55a898b44ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.908+0000 7fe0b5453640 1 --2- 172.21.15.5:0/2931407473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b31790 0x55a898b44ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.908+0000 7fe0b5453640 1 -- 172.21.15.5:0/2931407473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b48760 msgr2=0x55a898b418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.908+0000 7fe0b5453640 1 --2- 172.21.15.5:0/2931407473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b48760 0x55a898b418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.908+0000 7fe0b5453640 1 -- 172.21.15.5:0/2931407473 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a898b4f2d0 con 0x55a898b39b90 2024-09-15T22:15:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.909+0000 7fe0b5453640 1 --2- 172.21.15.5:0/2931407473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b39b90 0x55a898b44ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55a898b38e90 tx=0x7fe0a8024cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57b1f0d6e7af03b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.909+0000 7fe08ffff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:15:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.909+0000 7fe08ffff640 1 -- 172.21.15.5:0/2931407473 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a803d060 con 0x55a898b39b90 2024-09-15T22:15:01.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.909+0000 7fe08ffff640 1 -- 172.21.15.5:0/2931407473 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fe0a8021040 con 0x55a898b39b90 2024-09-15T22:15:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.909+0000 7fe0b9a90740 1 -- 172.21.15.5:0/2931407473 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ac004730 con 0x55a898b39b90 2024-09-15T22:15:01.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.909+0000 7fe08ffff640 1 -- 172.21.15.5:0/2931407473 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe0a8012450 con 0x55a898b39b90 2024-09-15T22:15:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.913+0000 7fe0b9a90740 10 client.26158 fetch_fsmap learned FSMap version 25 2024-09-15T22:15:01.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.913+0000 7fe0b9a90740 1 -- 172.21.15.5:0/2931407473 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a898b3c6c0 con 0x55a898b39b90 2024-09-15T22:15:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.917+0000 7fe08ffff640 1 -- 172.21.15.5:0/2931407473 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ac004730 con 0x55a898b39b90 2024-09-15T22:15:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.917+0000 7fe08ffff640 1 -- 172.21.15.5:0/2931407473 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe0a8011940 con 0x55a898b39b90 2024-09-15T22:15:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.917+0000 7fe0b9a90740 10 client.26158 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:15:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.917+0000 7fe0b9a90740 10 client.26158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.917+0000 7fe0b9a90740 20 client.26158 populate_metadata read hostname 'smithi005' 2024-09-15T22:15:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.917+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/2931407473 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a898bb84c0 0x55a898bba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:01.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.917+0000 7fe0b9a90740 4 client.26158 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:15:01.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.917+0000 7fe0b9a90740 1 -- 172.21.15.5:0/2931407473 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a898a1b210 con 0x55a898bb84c0 2024-09-15T22:15:01.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.921+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/2931407473 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a898bb84c0 0x55a898bba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:01.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.923+0000 7fe0b4c52640 1 --2- 172.21.15.5:0/2931407473 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a898bb84c0 0x55a898bba8e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41283e22ed6ee6e7 server_cookie=dc62ccfcc3782aaf in_seq=0 out_seq=0 2024-09-15T22:15:01.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.923+0000 7fe08ffff640 10 client.26158 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:01.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.926+0000 7fe08ffff640 1 -- 172.21.15.5:0/2931407473 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a898a1b210 con 0x55a898bb84c0 2024-09-15T22:15:01.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.926+0000 7fe08ffff640 10 client.26158 handle_command_reply: tid=0 2024-09-15T22:15:01.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.926+0000 7fe0b9a90740 1 client.26158 shutdown 2024-09-15T22:15:01.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.926+0000 7fe0b9a90740 1 -- 172.21.15.5:0/2931407473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b39b90 msgr2=0x55a898b44ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:01.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.926+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/2931407473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b39b90 0x55a898b44ff0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55a898b38e90 tx=0x7fe0a8024cf0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.926+0000 7fe0b9a90740 1 -- 172.21.15.5:0/2931407473 shutdown_connections 2024-09-15T22:15:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.926+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/2931407473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a898b48760 0x55a898b418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.926+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/2931407473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a898b39b90 0x55a898b44ff0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.926+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/2931407473 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a898bb84c0 0x55a898bba8e0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.926+0000 7fe0b9a90740 1 --2- 172.21.15.5:0/2931407473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a898b31790 0x55a898b44ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.926+0000 7fe0b9a90740 1 -- 172.21.15.5:0/2931407473 >> 172.21.15.5:0/2931407473 conn(0x55a898b2dd20 msgr2=0x55a898b47f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.927+0000 7fe08ffff640 0 client.26158 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:01.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.928+0000 7fe0b9a90740 1 -- 172.21.15.5:0/2931407473 shutdown_connections 2024-09-15T22:15:01.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.928+0000 7fe0b9a90740 1 -- 172.21.15.5:0/2931407473 wait complete. 2024-09-15T22:15:01.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.928+0000 7fe0b9a90740 20 client.26158 trim_cache size 0 max 16384 2024-09-15T22:15:01.935 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:15:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:15:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:15:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "776d4c79-9925-4f0b-a7c1-b1d387f8ef9a": { 2024-09-15T22:15:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:15:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "776d4c79-9925-4f0b-a7c1-b1d387f8ef9a", 2024-09-15T22:15:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:15:01.935 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:15:01.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:15:01.936 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:15:01.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.933+0000 7fe09bfff640 1 -- 172.21.15.5:0/3270551265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe09c0a35c0 msgr2=0x7fe09c0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:01.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.933+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe09c0a35c0 0x7fe09c0a5a80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe0a00ee070 tx=0x7fe0a0005fd0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.933+0000 7fe09bfff640 1 -- 172.21.15.5:0/3270551265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b0102ef0 msgr2=0x7fe0b01a20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:01.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.933+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b0102ef0 0x7fe0b01a20f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009230 tx=0x7fe0a0003b90 comp rx=0 tx=0).stop 2024-09-15T22:15:01.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.933+0000 7fe09bfff640 1 -- 172.21.15.5:0/3270551265 shutdown_connections 2024-09-15T22:15:01.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.933+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe09c0a35c0 0x7fe09c0a5a80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.933+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0b0102ef0 0x7fe0b01a20f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.933+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe0b0102570 0x7fe0b0196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.933+0000 7fe09bfff640 1 --2- 172.21.15.5:0/3270551265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe0b0100e20 0x7fe0b0196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:01.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.933+0000 7fe09bfff640 1 -- 172.21.15.5:0/3270551265 >> 172.21.15.5:0/3270551265 conn(0x7fe0b00fc7d0 msgr2=0x7fe0b0069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:01.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.936+0000 7fe09bfff640 1 -- 172.21.15.5:0/3270551265 shutdown_connections 2024-09-15T22:15:01.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:01.937+0000 7fe09bfff640 1 -- 172.21.15.5:0/3270551265 wait complete. 2024-09-15T22:15:02.670 INFO:tasks.cephfs.filesystem:scrub status for tag:776d4c79-9925-4f0b-a7c1-b1d387f8ef9a - {'path': '/', 'tag': '776d4c79-9925-4f0b-a7c1-b1d387f8ef9a', 'options': 'recursive,force'} 2024-09-15T22:15:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:02 smithi099 ceph-mon[34892]: pgmap v187: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 316 op/s 2024-09-15T22:15:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:02 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:15:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:02 smithi129 ceph-mon[35314]: pgmap v187: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 316 op/s 2024-09-15T22:15:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:02 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:15:03.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:02 smithi005 ceph-mon[30944]: pgmap v187: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 10 MiB/s wr, 316 op/s 2024-09-15T22:15:03.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:02 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:15:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 335.0 s, 1200.6 tps, lat 83.613 ms stddev 113.202 2024-09-15T22:15:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:04 smithi099 ceph-mon[34892]: pgmap v188: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 313 op/s 2024-09-15T22:15:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:04 smithi129 ceph-mon[35314]: pgmap v188: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 313 op/s 2024-09-15T22:15:05.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:04 smithi005 ceph-mon[30944]: pgmap v188: 129 pgs: 129 active+clean; 2.8 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 313 op/s 2024-09-15T22:15:07.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:06 smithi099 ceph-mon[34892]: pgmap v189: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 644 op/s 2024-09-15T22:15:07.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:06 smithi129 ceph-mon[35314]: pgmap v189: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 644 op/s 2024-09-15T22:15:07.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:06 smithi005 ceph-mon[30944]: pgmap v189: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 19 MiB/s wr, 644 op/s 2024-09-15T22:15:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:08 smithi099 ceph-mon[34892]: pgmap v190: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 20 MiB/s wr, 679 op/s 2024-09-15T22:15:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:08 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:15:09.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:08 smithi129 ceph-mon[35314]: pgmap v190: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 20 MiB/s wr, 679 op/s 2024-09-15T22:15:09.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:08 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:15:09.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:08 smithi005 ceph-mon[30944]: pgmap v190: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 20 MiB/s wr, 679 op/s 2024-09-15T22:15:09.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:08 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:15:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 340.0 s, 1060.8 tps, lat 92.281 ms stddev 117.968 2024-09-15T22:15:10.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:09 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:15:10.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:09 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:15:10.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:09 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:15:10.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:09 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:15:10.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:09 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:15:10.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:09 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:15:10.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:09 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:15:10.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:09 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:15:10.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:09 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:15:11.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:11 smithi099 ceph-mon[34892]: pgmap v191: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 20 MiB/s wr, 682 op/s 2024-09-15T22:15:11.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:11 smithi129 ceph-mon[35314]: pgmap v191: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 20 MiB/s wr, 682 op/s 2024-09-15T22:15:11.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:11 smithi005 ceph-mon[30944]: pgmap v191: 129 pgs: 129 active+clean; 2.9 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 20 MiB/s wr, 682 op/s 2024-09-15T22:15:13.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:13 smithi099 ceph-mon[34892]: pgmap v192: 129 pgs: 129 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 27 MiB/s wr, 910 op/s 2024-09-15T22:15:13.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:13 smithi129 ceph-mon[35314]: pgmap v192: 129 pgs: 129 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 27 MiB/s wr, 910 op/s 2024-09-15T22:15:13.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:13 smithi005 ceph-mon[30944]: pgmap v192: 129 pgs: 129 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 27 MiB/s wr, 910 op/s 2024-09-15T22:15:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 345.0 s, 1274.8 tps, lat 79.062 ms stddev 99.127 2024-09-15T22:15:15.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:15 smithi099 ceph-mon[34892]: pgmap v193: 129 pgs: 129 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 17 MiB/s wr, 628 op/s 2024-09-15T22:15:15.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:15 smithi129 ceph-mon[35314]: pgmap v193: 129 pgs: 129 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 17 MiB/s wr, 628 op/s 2024-09-15T22:15:15.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:15 smithi005 ceph-mon[30944]: pgmap v193: 129 pgs: 129 active+clean; 3.0 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 17 MiB/s wr, 628 op/s 2024-09-15T22:15:17.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:17 smithi099 ceph-mon[34892]: pgmap v194: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 MiB/s wr, 850 op/s 2024-09-15T22:15:17.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:17 smithi129 ceph-mon[35314]: pgmap v194: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 MiB/s wr, 850 op/s 2024-09-15T22:15:17.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:17 smithi005 ceph-mon[30944]: pgmap v194: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 MiB/s wr, 850 op/s 2024-09-15T22:15:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:19 smithi099 ceph-mon[34892]: pgmap v195: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 544 op/s 2024-09-15T22:15:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:19 smithi129 ceph-mon[35314]: pgmap v195: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 544 op/s 2024-09-15T22:15:19.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:19 smithi005 ceph-mon[30944]: pgmap v195: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 16 MiB/s wr, 544 op/s 2024-09-15T22:15:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 350.0 s, 1339.8 tps, lat 74.488 ms stddev 100.488 2024-09-15T22:15:21.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:21 smithi099 ceph-mon[34892]: pgmap v196: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 485 op/s 2024-09-15T22:15:21.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:21 smithi129 ceph-mon[35314]: pgmap v196: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 485 op/s 2024-09-15T22:15:21.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:21 smithi005 ceph-mon[30944]: pgmap v196: 129 pgs: 129 active+clean; 3.0 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 15 MiB/s wr, 485 op/s 2024-09-15T22:15:23.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:23 smithi099 ceph-mon[34892]: pgmap v197: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 718 op/s 2024-09-15T22:15:23.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:23 smithi129 ceph-mon[35314]: pgmap v197: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 718 op/s 2024-09-15T22:15:23.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:23 smithi005 ceph-mon[30944]: pgmap v197: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 22 MiB/s wr, 718 op/s 2024-09-15T22:15:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 355.0 s, 1110.0 tps, lat 90.509 ms stddev 103.851 2024-09-15T22:15:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:25 smithi099 ceph-mon[34892]: pgmap v198: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 490 op/s 2024-09-15T22:15:25.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:25 smithi129 ceph-mon[35314]: pgmap v198: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 490 op/s 2024-09-15T22:15:25.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:25 smithi005 ceph-mon[30944]: pgmap v198: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 15 MiB/s wr, 490 op/s 2024-09-15T22:15:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:27 smithi099 ceph-mon[34892]: pgmap v199: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 733 op/s 2024-09-15T22:15:27.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:27 smithi129 ceph-mon[35314]: pgmap v199: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 733 op/s 2024-09-15T22:15:27.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:27 smithi005 ceph-mon[30944]: pgmap v199: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 19 MiB/s wr, 733 op/s 2024-09-15T22:15:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:29 smithi099 ceph-mon[34892]: pgmap v200: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 13 MiB/s wr, 562 op/s 2024-09-15T22:15:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:29 smithi129 ceph-mon[35314]: pgmap v200: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 13 MiB/s wr, 562 op/s 2024-09-15T22:15:29.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:29 smithi005 ceph-mon[30944]: pgmap v200: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 13 MiB/s wr, 562 op/s 2024-09-15T22:15:29.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 360.0 s, 1325.8 tps, lat 76.075 ms stddev 99.586 2024-09-15T22:15:31.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:31 smithi099 ceph-mon[34892]: pgmap v201: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 537 op/s 2024-09-15T22:15:31.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:31 smithi129 ceph-mon[35314]: pgmap v201: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 537 op/s 2024-09-15T22:15:31.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:31 smithi005 ceph-mon[30944]: pgmap v201: 129 pgs: 129 active+clean; 3.1 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 13 MiB/s wr, 537 op/s 2024-09-15T22:15:32.671 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:15:33.131 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:33.441 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:33 smithi005 ceph-mon[30944]: pgmap v202: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 20 MiB/s wr, 847 op/s 2024-09-15T22:15:33.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:33 smithi099 ceph-mon[34892]: pgmap v202: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 20 MiB/s wr, 847 op/s 2024-09-15T22:15:33.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:33 smithi129 ceph-mon[35314]: pgmap v202: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 20 MiB/s wr, 847 op/s 2024-09-15T22:15:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.779+0000 7f1c25563640 1 -- 172.21.15.5:0/3175956495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2011b140 msgr2=0x7f1c2011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.779+0000 7f1c25563640 1 --2- 172.21.15.5:0/3175956495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2011b140 0x7f1c2011d530 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f1c1001a910 tx=0x7f1c1003b470 comp rx=0 tx=0).stop 2024-09-15T22:15:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.779+0000 7f1c25563640 1 -- 172.21.15.5:0/3175956495 shutdown_connections 2024-09-15T22:15:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.779+0000 7f1c25563640 1 --2- 172.21.15.5:0/3175956495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2011b140 0x7f1c2011d530 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.779+0000 7f1c25563640 1 --2- 172.21.15.5:0/3175956495 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c2006ea30 0x7f1c2006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.779+0000 7f1c25563640 1 --2- 172.21.15.5:0/3175956495 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c20110190 0x7f1c2006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.779+0000 7f1c25563640 1 -- 172.21.15.5:0/3175956495 >> 172.21.15.5:0/3175956495 conn(0x7f1c20073180 msgr2=0x7f1c20073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.780+0000 7f1c25563640 1 -- 172.21.15.5:0/3175956495 shutdown_connections 2024-09-15T22:15:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.780+0000 7f1c25563640 1 -- 172.21.15.5:0/3175956495 wait complete. 2024-09-15T22:15:33.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.780+0000 7f1c25563640 1 Processor -- start 2024-09-15T22:15:33.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.781+0000 7f1c25563640 1 -- start start 2024-09-15T22:15:33.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.781+0000 7f1c25563640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c2006ea30 0x7f1c2011a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.781+0000 7f1c25563640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c20110190 0x7f1c20116a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.781+0000 7f1c25563640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2011b140 0x7f1c20116fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.781+0000 7f1c25563640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c201133b0 con 0x7f1c2011b140 2024-09-15T22:15:33.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.781+0000 7f1c25563640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20113230 con 0x7f1c20110190 2024-09-15T22:15:33.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.781+0000 7f1c25563640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1c20113530 con 0x7f1c2006ea30 2024-09-15T22:15:33.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.781+0000 7f1c1ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c2006ea30 0x7f1c2011a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.781+0000 7f1c1ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c2006ea30 0x7f1c2011a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:42530/0 (socket says 172.21.15.5:42530) 2024-09-15T22:15:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.781+0000 7f1c1ffff640 1 -- 172.21.15.5:0/843201255 learned_addr learned my addr 172.21.15.5:0/843201255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.782+0000 7f1c1f7fe640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c20110190 0x7f1c20116a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.782+0000 7f1c24d62640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2011b140 0x7f1c20116fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.782+0000 7f1c24d62640 1 -- 172.21.15.5:0/843201255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c2006ea30 msgr2=0x7f1c2011a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.782+0000 7f1c24d62640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c2006ea30 0x7f1c2011a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.782+0000 7f1c24d62640 1 -- 172.21.15.5:0/843201255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c20110190 msgr2=0x7f1c20116a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.782+0000 7f1c24d62640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c20110190 0x7f1c20116a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.782+0000 7f1c24d62640 1 -- 172.21.15.5:0/843201255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c20117830 con 0x7f1c2011b140 2024-09-15T22:15:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.782+0000 7f1c24d62640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2011b140 0x7f1c20116fc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1c1003b990 tx=0x7f1c10040fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:33.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.784+0000 7f1c1d7fa640 1 -- 172.21.15.5:0/843201255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1004f070 con 0x7f1c2011b140 2024-09-15T22:15:33.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.784+0000 7f1c1d7fa640 1 -- 172.21.15.5:0/843201255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c10005260 con 0x7f1c2011b140 2024-09-15T22:15:33.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.784+0000 7f1c1d7fa640 1 -- 172.21.15.5:0/843201255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10046030 con 0x7f1c2011b140 2024-09-15T22:15:33.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.784+0000 7f1c25563640 1 -- 172.21.15.5:0/843201255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c201aaf60 con 0x7f1c2011b140 2024-09-15T22:15:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.784+0000 7f1c25563640 1 -- 172.21.15.5:0/843201255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c201ab360 con 0x7f1c2011b140 2024-09-15T22:15:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.791+0000 7f1c1d7fa640 1 -- 172.21.15.5:0/843201255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1c10052070 con 0x7f1c2011b140 2024-09-15T22:15:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.791+0000 7f1c28b9e740 1 Processor -- start 2024-09-15T22:15:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.792+0000 7f1c1d7fa640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c080a3580 0x7f1c080a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.793+0000 7f1c1d7fa640 1 -- 172.21.15.5:0/843201255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f1c10101240 con 0x7f1c2011b140 2024-09-15T22:15:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.793+0000 7f1c1f7fe640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c080a3580 0x7f1c080a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.793+0000 7f1c28b9e740 1 -- start start 2024-09-15T22:15:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4dc670 0x5612da4dca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4e7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4e7850 0x5612da4e9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c28b9e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5612da4422e0 con 0x5612da4e7850 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c28b9e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5612da3e9ce0 con 0x5612da4dcf90 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c28b9e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5612da43e2a0 con 0x5612da4dc670 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c1f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c1f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4e7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:39242/0 (socket says 172.21.15.5:39242) 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c1f7fe640 1 -- 172.21.15.5:0/846466890 learned_addr learned my addr 172.21.15.5:0/846466890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c24d62640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4e7850 0x5612da4e9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c24d62640 1 -- 172.21.15.5:0/846466890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4dc670 msgr2=0x5612da4dca50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c24d62640 1 --2- 172.21.15.5:0/846466890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4dc670 0x5612da4dca50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c24d62640 1 -- 172.21.15.5:0/846466890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 msgr2=0x5612da4e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c24d62640 1 --2- 172.21.15.5:0/846466890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4e7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.794+0000 7f1c24d62640 1 -- 172.21.15.5:0/846466890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612da3ce440 con 0x5612da4e7850 2024-09-15T22:15:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.795+0000 7f1c24d62640 1 --2- 172.21.15.5:0/846466890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4e7850 0x5612da4e9c40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1c100fe590 tx=0x7f1c1001a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a663458ce56c496 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.797+0000 7f1c067fc640 1 -- 172.21.15.5:0/846466890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10059040 con 0x5612da4e7850 2024-09-15T22:15:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.797+0000 7f1c067fc640 1 -- 172.21.15.5:0/846466890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c100d5d90 con 0x5612da4e7850 2024-09-15T22:15:33.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.797+0000 7f1c1f7fe640 1 --2- 172.21.15.5:0/846466890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.798+0000 7f1c28b9e740 1 -- 172.21.15.5:0/846466890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4e7850 msgr2=0x5612da4e9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.798+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/846466890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4e7850 0x5612da4e9c40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f1c100fe590 tx=0x7f1c1001a270 comp rx=0 tx=0).stop 2024-09-15T22:15:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.799+0000 7f1c1f7fe640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c080a3580 0x7f1c080a5a40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5612da4d9a60 tx=0x7f1c180098b0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.799+0000 7f1c28b9e740 1 -- 172.21.15.5:0/846466890 shutdown_connections 2024-09-15T22:15:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.799+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/846466890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4e7850 0x5612da4e9c40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.799+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/846466890 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.799+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/846466890 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4dc670 0x5612da4dca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.799+0000 7f1c28b9e740 1 -- 172.21.15.5:0/846466890 >> 172.21.15.5:0/846466890 conn(0x5612da4d8c00 msgr2=0x5612da4d8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:33.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.805+0000 7f1c28b9e740 1 -- 172.21.15.5:0/846466890 shutdown_connections 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.807+0000 7f1c28b9e740 1 -- 172.21.15.5:0/846466890 wait complete. 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.807+0000 7f1c28b9e740 1 -- start start 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.807+0000 7f1c28b9e740 1 Processor -- start 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.808+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 0x5612da4e9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.808+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dcf90 0x5612da4e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.808+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da4e74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.808+0000 7f1c28b9e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5612da4ea1e0 con 0x5612da4dcf90 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.808+0000 7f1c28b9e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5612da3fd050 con 0x5612da4dc670 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.808+0000 7f1c28b9e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5612da4ea360 con 0x5612da4e7030 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.808+0000 7f1c28b9e740 10 client.0 _ll_register_callbacks cb 0x5612da43d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.809+0000 7f1c1ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 0x5612da4e9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.809+0000 7f1c24d62640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da4e74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.809+0000 7f1c24d62640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da4e74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:42548/0 (socket says 172.21.15.5:42548) 2024-09-15T22:15:33.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.809+0000 7f1c24d62640 1 -- 172.21.15.5:0/1898870359 learned_addr learned my addr 172.21.15.5:0/1898870359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:33.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.809+0000 7f1c24d62640 1 -- 172.21.15.5:0/1898870359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 msgr2=0x5612da4e9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:33.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.809+0000 7f1c24d62640 1 --2- 172.21.15.5:0/1898870359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 0x5612da4e9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.809+0000 7f1c24d62640 1 -- 172.21.15.5:0/1898870359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dcf90 msgr2=0x5612da4e6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:33.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.809+0000 7f1c24d62640 1 --2- 172.21.15.5:0/1898870359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dcf90 0x5612da4e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.809+0000 7f1c24d62640 1 -- 172.21.15.5:0/1898870359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612da1b33c0 con 0x5612da4e7030 2024-09-15T22:15:33.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.810+0000 7f1c24d62640 1 --2- 172.21.15.5:0/1898870359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da4e74b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1c100c9af0 tx=0x7f1c10101160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50e504aa9397f25c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:33.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.810+0000 7f1bff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:15:33.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.810+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1898870359 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10059040 con 0x5612da4e7030 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.810+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1898870359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612da44f850 con 0x5612da4e7030 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.810+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1898870359 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f1c100fe810 con 0x5612da4e7030 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.811+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1898870359 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c100de050 con 0x5612da4e7030 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.811+0000 7f1c28b9e740 10 client.26164 fetch_fsmap learned FSMap version 25 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.811+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1898870359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5612da4ea1e0 con 0x5612da4e7030 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.812+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1898870359 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5612da44f850 con 0x5612da4e7030 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.812+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1898870359 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1c100fece0 con 0x5612da4e7030 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.812+0000 7f1c28b9e740 10 client.26164 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.812+0000 7f1c28b9e740 10 client.26164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.812+0000 7f1c28b9e740 20 client.26164 populate_metadata read hostname 'smithi005' 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.812+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1898870359 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5612da5190b0 0x5612da51b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.812+0000 7f1c28b9e740 4 client.26164 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:33.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.812+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1898870359 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612da449d60 con 0x5612da5190b0 2024-09-15T22:15:33.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.813+0000 7f1c1ffff640 1 --2- 172.21.15.5:0/1898870359 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5612da5190b0 0x5612da51b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.815+0000 7f1c1ffff640 1 --2- 172.21.15.5:0/1898870359 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5612da5190b0 0x5612da51b510 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b439ac759caab9 server_cookie=45849fb2f61010a6 in_seq=0 out_seq=0 2024-09-15T22:15:33.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.815+0000 7f1bff7fe640 10 client.26164 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:33.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.816+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1898870359 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612da449d60 con 0x5612da5190b0 2024-09-15T22:15:33.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.816+0000 7f1bff7fe640 10 client.26164 handle_command_reply: tid=0 2024-09-15T22:15:33.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.816+0000 7f1c28b9e740 1 client.26164 shutdown 2024-09-15T22:15:33.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.817+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1898870359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 msgr2=0x5612da4e74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:33.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.817+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1898870359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da4e74b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1c100c9af0 tx=0x7f1c10101160 comp rx=0 tx=0).stop 2024-09-15T22:15:33.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.817+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1898870359 shutdown_connections 2024-09-15T22:15:33.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.817+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1898870359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da4e74b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.817+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1898870359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dcf90 0x5612da4e6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.817+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1898870359 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5612da5190b0 0x5612da51b510 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.817+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1898870359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 0x5612da4e9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.817+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1898870359 >> 172.21.15.5:0/1898870359 conn(0x5612da4d8c00 msgr2=0x5612da4de8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:33.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.817+0000 7f1bff7fe640 0 client.26164 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:33.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.819+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1898870359 shutdown_connections 2024-09-15T22:15:33.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.820+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1898870359 wait complete. 2024-09-15T22:15:33.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.820+0000 7f1c28b9e740 20 client.26164 trim_cache size 0 max 16384 2024-09-15T22:15:33.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.856+0000 7f1c28b9e740 1 Processor -- start 2024-09-15T22:15:33.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.856+0000 7f1c28b9e740 1 -- start start 2024-09-15T22:15:33.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.856+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 0x5612da4e2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.856+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dcf90 0x5612da4f3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.856+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da520e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c1ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 0x5612da4e2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c1ffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 0x5612da4e2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:39254/0 (socket says 172.21.15.5:39254) 2024-09-15T22:15:33.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c1ffff640 1 -- 172.21.15.5:0/87212809 learned_addr learned my addr 172.21.15.5:0/87212809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:33.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c28b9e740 1 -- 172.21.15.5:0/87212809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c012770 con 0x5612da4dcf90 2024-09-15T22:15:33.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c28b9e740 1 -- 172.21.15.5:0/87212809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c010040 con 0x5612da4dc670 2024-09-15T22:15:33.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c28b9e740 1 -- 172.21.15.5:0/87212809 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c00de90 con 0x5612da4e7030 2024-09-15T22:15:33.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c24d62640 1 --2- 172.21.15.5:0/87212809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da520e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c1ffff640 1 -- 172.21.15.5:0/87212809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 msgr2=0x5612da520e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:33.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c1ffff640 1 --2- 172.21.15.5:0/87212809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da520e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c1f7fe640 1 --2- 172.21.15.5:0/87212809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dcf90 0x5612da4f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c1ffff640 1 -- 172.21.15.5:0/87212809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dcf90 msgr2=0x5612da4f3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:33.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c1ffff640 1 --2- 172.21.15.5:0/87212809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dcf90 0x5612da4f3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c1ffff640 1 -- 172.21.15.5:0/87212809 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612da4e9bb0 con 0x5612da4dc670 2024-09-15T22:15:33.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.857+0000 7f1c24d62640 1 --2- 172.21.15.5:0/87212809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da520e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:33.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.858+0000 7f1c1ffff640 1 --2- 172.21.15.5:0/87212809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 0x5612da4e2280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5612da51bdc0 tx=0x7f1c14009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45860dafb1c21124 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:33.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.858+0000 7f1c05ffb640 1 -- 172.21.15.5:0/87212809 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c140169b0 con 0x5612da4dc670 2024-09-15T22:15:33.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.858+0000 7f1c05ffb640 1 -- 172.21.15.5:0/87212809 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c14016b50 con 0x5612da4dc670 2024-09-15T22:15:33.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.858+0000 7f1c05ffb640 1 -- 172.21.15.5:0/87212809 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14014670 con 0x5612da4dc670 2024-09-15T22:15:33.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.859+0000 7f1c28b9e740 1 -- 172.21.15.5:0/87212809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 msgr2=0x5612da4e2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:33.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.859+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/87212809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 0x5612da4e2280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x5612da51bdc0 tx=0x7f1c14009190 comp rx=0 tx=0).stop 2024-09-15T22:15:33.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.859+0000 7f1c28b9e740 1 -- 172.21.15.5:0/87212809 shutdown_connections 2024-09-15T22:15:33.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.859+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/87212809 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da520e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.859+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/87212809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dcf90 0x5612da4f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.859+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/87212809 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dc670 0x5612da4e2280 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.859+0000 7f1c28b9e740 1 -- 172.21.15.5:0/87212809 >> 172.21.15.5:0/87212809 conn(0x5612da4d8c00 msgr2=0x5612da4d7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.860+0000 7f1c28b9e740 1 -- 172.21.15.5:0/87212809 shutdown_connections 2024-09-15T22:15:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.860+0000 7f1c28b9e740 1 -- 172.21.15.5:0/87212809 wait complete. 2024-09-15T22:15:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.860+0000 7f1c28b9e740 1 -- start start 2024-09-15T22:15:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.860+0000 7f1c28b9e740 1 Processor -- start 2024-09-15T22:15:33.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dc670 0x5612da4efc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c28b9e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da4eb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c28b9e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c00e500 con 0x5612da4dc670 2024-09-15T22:15:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c28b9e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c00f970 con 0x5612da4dcf90 2024-09-15T22:15:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c28b9e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c00ebd0 con 0x5612da4e7030 2024-09-15T22:15:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c1f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c24d62640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da4eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c28b9e740 10 client.0 _ll_register_callbacks cb 0x7f1c100ccc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c1f7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4eaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:39268/0 (socket says 172.21.15.5:39268) 2024-09-15T22:15:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c1f7fe640 1 -- 172.21.15.5:0/1811469029 learned_addr learned my addr 172.21.15.5:0/1811469029 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c24d62640 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da4eb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:42576/0 (socket says 172.21.15.5:42576) 2024-09-15T22:15:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c1ffff640 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dc670 0x5612da4efc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c1f7fe640 1 -- 172.21.15.5:0/1811469029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 msgr2=0x5612da4eb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c1f7fe640 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da4eb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c1f7fe640 1 -- 172.21.15.5:0/1811469029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dc670 msgr2=0x5612da4efc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.861+0000 7f1c1f7fe640 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dc670 0x5612da4efc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.862+0000 7f1c1f7fe640 1 -- 172.21.15.5:0/1811469029 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612da4ebb70 con 0x5612da4dcf90 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.862+0000 7f1c1f7fe640 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4eaf40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5612da4e6380 tx=0x7f1c18009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b255b886ea373024 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.862+0000 7f1bff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.862+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1811469029 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c18021070 con 0x5612da4dcf90 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.862+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1811469029 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f1c1801b660 con 0x5612da4dcf90 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.862+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1811469029 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c14016b50 con 0x5612da4dcf90 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.862+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1811469029 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c18007e70 con 0x5612da4dcf90 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.862+0000 7f1c28b9e740 10 client.26201 fetch_fsmap learned FSMap version 25 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.863+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1811469029 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5612da583ca0 con 0x5612da4dcf90 2024-09-15T22:15:33.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.863+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1811469029 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c14016b50 con 0x5612da4dcf90 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.863+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1811469029 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1c180259a0 con 0x5612da4dcf90 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.863+0000 7f1c28b9e740 10 client.26201 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.863+0000 7f1c28b9e740 10 client.26201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.863+0000 7f1c28b9e740 20 client.26201 populate_metadata read hostname 'smithi005' 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.863+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5612da5844a0 0x5612da5868c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.863+0000 7f1c28b9e740 4 client.26201 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.863+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1811469029 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5612da3c6210 con 0x5612da5844a0 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.863+0000 7f1c1ffff640 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5612da5844a0 0x5612da5868c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.864+0000 7f1c1ffff640 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5612da5844a0 0x5612da5868c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e3fab40330b9df4 server_cookie=47a335af39a761aa in_seq=0 out_seq=0 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.864+0000 7f1bff7fe640 10 client.26201 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.864+0000 7f1bff7fe640 1 -- 172.21.15.5:0/1811469029 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5612da3c6210 con 0x5612da5844a0 2024-09-15T22:15:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.864+0000 7f1bff7fe640 10 client.26201 handle_command_reply: tid=0 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.864+0000 7f1c28b9e740 1 client.26201 shutdown 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.864+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1811469029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 msgr2=0x5612da4eaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.864+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4eaf40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x5612da4e6380 tx=0x7f1c18009f20 comp rx=0 tx=0).stop 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.864+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1811469029 shutdown_connections 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.864+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5612da4e7030 0x5612da4eb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.865+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5612da5844a0 0x5612da5868c0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.865+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5612da4dcf90 0x5612da4eaf40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.865+0000 7f1bff7fe640 0 client.26201 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.865+0000 7f1c28b9e740 1 --2- 172.21.15.5:0/1811469029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5612da4dc670 0x5612da4efc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.865+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1811469029 >> 172.21.15.5:0/1811469029 conn(0x5612da4d8c00 msgr2=0x5612da4d7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.865+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1811469029 shutdown_connections 2024-09-15T22:15:33.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.865+0000 7f1c28b9e740 1 -- 172.21.15.5:0/1811469029 wait complete. 2024-09-15T22:15:33.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.868+0000 7f1c28b9e740 20 client.26201 trim_cache size 0 max 16384 2024-09-15T22:15:33.871 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:15:33.871 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:15:33.871 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:15:33.871 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 -- 172.21.15.5:0/843201255 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c080a3580 msgr2=0x7f1c080a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c080a3580 0x7f1c080a5a40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5612da4d9a60 tx=0x7f1c180098b0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 -- 172.21.15.5:0/843201255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2011b140 msgr2=0x7f1c20116fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2011b140 0x7f1c20116fc0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f1c1003b990 tx=0x7f1c10040fa0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 -- 172.21.15.5:0/843201255 shutdown_connections 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c080a3580 0x7f1c080a5a40 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5612da4d9a60 tx=0x7f1c180098b0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c2011b140 0x7f1c20116fc0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c20110190 0x7f1c20116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 --2- 172.21.15.5:0/843201255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c2006ea30 0x7f1c2011a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 -- 172.21.15.5:0/843201255 >> 172.21.15.5:0/843201255 conn(0x7f1c20073180 msgr2=0x7f1c2011b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 -- 172.21.15.5:0/843201255 shutdown_connections 2024-09-15T22:15:33.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:33.869+0000 7f1c25563640 1 -- 172.21.15.5:0/843201255 wait complete. 2024-09-15T22:15:34.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 365.0 s, 1153.8 tps, lat 85.066 ms stddev 98.859 2024-09-15T22:15:34.679 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:15:34.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:15:35.161 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:35.481 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:35 smithi005 ceph-mon[30944]: pgmap v203: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 13 MiB/s wr, 604 op/s 2024-09-15T22:15:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:35 smithi099 ceph-mon[34892]: pgmap v203: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 13 MiB/s wr, 604 op/s 2024-09-15T22:15:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:35 smithi129 ceph-mon[35314]: pgmap v203: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 13 MiB/s wr, 604 op/s 2024-09-15T22:15:35.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.780+0000 7f5a23b2c640 1 -- 172.21.15.5:0/2242810302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a1c106f10 msgr2=0x7f5a1c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.780+0000 7f5a23b2c640 1 --2- 172.21.15.5:0/2242810302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a1c106f10 0x7f5a1c10ea90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009860 tx=0x7f5a0c02a360 comp rx=0 tx=0).stop 2024-09-15T22:15:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.781+0000 7f5a23b2c640 1 -- 172.21.15.5:0/2242810302 shutdown_connections 2024-09-15T22:15:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.781+0000 7f5a23b2c640 1 --2- 172.21.15.5:0/2242810302 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a1c106f10 0x7f5a1c10ea90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.781+0000 7f5a23b2c640 1 --2- 172.21.15.5:0/2242810302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a1c102790 0x7f5a1c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.781+0000 7f5a23b2c640 1 --2- 172.21.15.5:0/2242810302 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a1c101de0 0x7f5a1c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.781+0000 7f5a23b2c640 1 -- 172.21.15.5:0/2242810302 >> 172.21.15.5:0/2242810302 conn(0x7f5a1c0fd8e0 msgr2=0x7f5a1c0ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.781+0000 7f5a23b2c640 1 -- 172.21.15.5:0/2242810302 shutdown_connections 2024-09-15T22:15:35.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.782+0000 7f5a23b2c640 1 -- 172.21.15.5:0/2242810302 wait complete. 2024-09-15T22:15:35.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.783+0000 7f5a23b2c640 1 Processor -- start 2024-09-15T22:15:35.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.783+0000 7f5a23b2c640 1 -- start start 2024-09-15T22:15:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.783+0000 7f5a23b2c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a1c101de0 0x7f5a1c1abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.783+0000 7f5a23b2c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a1c102790 0x7f5a1c1ac310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.783+0000 7f5a23b2c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a1c106f10 0x7f5a1c1a7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.783+0000 7f5a23b2c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c06c660 con 0x7f5a1c101de0 2024-09-15T22:15:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.783+0000 7f5a23b2c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c06c4e0 con 0x7f5a1c102790 2024-09-15T22:15:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.783+0000 7f5a23b2c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5a1c06c7e0 con 0x7f5a1c106f10 2024-09-15T22:15:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a218c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a1c101de0 0x7f5a1c1abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a218c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a1c101de0 0x7f5a1c1abdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46520/0 (socket says 172.21.15.5:46520) 2024-09-15T22:15:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a218c8640 1 -- 172.21.15.5:0/1284697335 learned_addr learned my addr 172.21.15.5:0/1284697335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:35.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a210c7640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a1c102790 0x7f5a1c1ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a220c9640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a1c106f10 0x7f5a1c1a7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a210c7640 1 -- 172.21.15.5:0/1284697335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a1c106f10 msgr2=0x7f5a1c1a7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a210c7640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a1c106f10 0x7f5a1c1a7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a210c7640 1 -- 172.21.15.5:0/1284697335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a1c101de0 msgr2=0x7f5a1c1abdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a210c7640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a1c101de0 0x7f5a1c1abdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a210c7640 1 -- 172.21.15.5:0/1284697335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a1c1a79b0 con 0x7f5a1c102790 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a210c7640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a1c102790 0x7f5a1c1ac310 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800c970 tx=0x7f5a1800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a12ffd640 1 -- 172.21.15.5:0/1284697335 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18007bb0 con 0x7f5a1c102790 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a12ffd640 1 -- 172.21.15.5:0/1284697335 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a18007d50 con 0x7f5a1c102790 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a23b2c640 1 -- 172.21.15.5:0/1284697335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a1c1a7ca0 con 0x7f5a1c102790 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.784+0000 7f5a12ffd640 1 -- 172.21.15.5:0/1284697335 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1800f450 con 0x7f5a1c102790 2024-09-15T22:15:35.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.785+0000 7f5a23b2c640 1 -- 172.21.15.5:0/1284697335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a1c1b3d50 con 0x7f5a1c102790 2024-09-15T22:15:35.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.786+0000 7f5a23b2c640 1 -- 172.21.15.5:0/1284697335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a1c1021c0 con 0x7f5a1c102790 2024-09-15T22:15:35.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.787+0000 7f5a12ffd640 1 -- 172.21.15.5:0/1284697335 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5a18004ef0 con 0x7f5a1c102790 2024-09-15T22:15:35.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.788+0000 7f5a12ffd640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a3480 0x7f5a080a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.789+0000 7f5a12ffd640 1 -- 172.21.15.5:0/1284697335 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f5a180cfc40 con 0x7f5a1c102790 2024-09-15T22:15:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.789+0000 7f5a218c8640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a3480 0x7f5a080a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:35.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.790+0000 7f5a218c8640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a3480 0x7f5a080a5940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c1a8870 tx=0x7f5a14010040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:35.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:35.793+0000 7f5a12ffd640 1 -- 172.21.15.5:0/1284697335 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a18014030 con 0x7f5a1c102790 2024-09-15T22:15:36.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.169+0000 7f5a23b2c640 1 -- 172.21.15.5:0/1284697335 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a1c1a8550 con 0x7f5a1c102790 2024-09-15T22:15:36.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.173+0000 7f5a12ffd640 1 -- 172.21.15.5:0/1284697335 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f5a1c1a8550 con 0x7f5a1c102790 2024-09-15T22:15:36.175 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:15:36.175 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:15:36.175 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:15:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.176+0000 7f5a23b2c640 1 -- 172.21.15.5:0/1284697335 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a3480 msgr2=0x7f5a080a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.176+0000 7f5a23b2c640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a3480 0x7f5a080a5940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c1a8870 tx=0x7f5a14010040 comp rx=0 tx=0).stop 2024-09-15T22:15:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.176+0000 7f5a23b2c640 1 -- 172.21.15.5:0/1284697335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a1c102790 msgr2=0x7f5a1c1ac310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:36.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.176+0000 7f5a23b2c640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a1c102790 0x7f5a1c1ac310 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800c970 tx=0x7f5a1800ce30 comp rx=0 tx=0).stop 2024-09-15T22:15:36.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.180+0000 7f5a23b2c640 1 -- 172.21.15.5:0/1284697335 shutdown_connections 2024-09-15T22:15:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.180+0000 7f5a23b2c640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5a080a3480 0x7f5a080a5940 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.180+0000 7f5a23b2c640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5a1c106f10 0x7f5a1c1a7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.180+0000 7f5a23b2c640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5a1c102790 0x7f5a1c1ac310 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.180+0000 7f5a23b2c640 1 --2- 172.21.15.5:0/1284697335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a1c101de0 0x7f5a1c1abdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:36.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.180+0000 7f5a23b2c640 1 -- 172.21.15.5:0/1284697335 >> 172.21.15.5:0/1284697335 conn(0x7f5a1c0fd8e0 msgr2=0x7f5a1c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:36.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.183+0000 7f5a23b2c640 1 -- 172.21.15.5:0/1284697335 shutdown_connections 2024-09-15T22:15:36.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:36.184+0000 7f5a23b2c640 1 -- 172.21.15.5:0/1284697335 wait complete. 2024-09-15T22:15:36.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:36 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1284697335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:36.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:36 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1284697335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:36.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:36 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1284697335' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:15:37.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:37 smithi099 ceph-mon[34892]: pgmap v204: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 18 MiB/s wr, 827 op/s 2024-09-15T22:15:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:37 smithi129 ceph-mon[35314]: pgmap v204: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 18 MiB/s wr, 827 op/s 2024-09-15T22:15:37.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:37 smithi005 ceph-mon[30944]: pgmap v204: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 18 MiB/s wr, 827 op/s 2024-09-15T22:15:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:39 smithi099 ceph-mon[34892]: pgmap v205: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 14 MiB/s wr, 607 op/s 2024-09-15T22:15:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:39 smithi129 ceph-mon[35314]: pgmap v205: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 14 MiB/s wr, 607 op/s 2024-09-15T22:15:39.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:39 smithi005 ceph-mon[30944]: pgmap v205: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 14 MiB/s wr, 607 op/s 2024-09-15T22:15:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 370.0 s, 1148.4 tps, lat 88.518 ms stddev 100.869 2024-09-15T22:15:41.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:41 smithi099 ceph-mon[34892]: pgmap v206: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 12 MiB/s wr, 555 op/s 2024-09-15T22:15:41.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:41 smithi129 ceph-mon[35314]: pgmap v206: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 12 MiB/s wr, 555 op/s 2024-09-15T22:15:41.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:41 smithi005 ceph-mon[30944]: pgmap v206: 129 pgs: 129 active+clean; 3.2 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 12 MiB/s wr, 555 op/s 2024-09-15T22:15:43.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:43 smithi099 ceph-mon[34892]: pgmap v207: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 17 MiB/s wr, 777 op/s 2024-09-15T22:15:43.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:43 smithi129 ceph-mon[35314]: pgmap v207: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 17 MiB/s wr, 777 op/s 2024-09-15T22:15:43.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:43 smithi005 ceph-mon[30944]: pgmap v207: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 17 MiB/s wr, 777 op/s 2024-09-15T22:15:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 375.0 s, 277.0 tps, lat 350.011 ms stddev 860.105 2024-09-15T22:15:44.680 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:15:45.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:45 smithi099 ceph-mon[34892]: pgmap v208: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 532 op/s 2024-09-15T22:15:45.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:45 smithi129 ceph-mon[35314]: pgmap v208: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 532 op/s 2024-09-15T22:15:45.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:45 smithi005 ceph-mon[30944]: pgmap v208: 129 pgs: 129 active+clean; 3.2 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 11 MiB/s wr, 532 op/s 2024-09-15T22:15:47.087 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:47.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:47 smithi099 ceph-mon[34892]: pgmap v209: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-09-15T22:15:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:47 smithi129 ceph-mon[35314]: pgmap v209: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-09-15T22:15:47.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:47 smithi005 ceph-mon[30944]: pgmap v209: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 21 MiB/s wr, 1.38k op/s 2024-09-15T22:15:49.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:49 smithi099 ceph-mon[34892]: pgmap v210: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-09-15T22:15:49.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:49 smithi129 ceph-mon[35314]: pgmap v210: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-09-15T22:15:49.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:49 smithi005 ceph-mon[30944]: pgmap v210: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 19 MiB/s wr, 1.36k op/s 2024-09-15T22:15:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 380.0 s, 662.2 tps, lat 145.451 ms stddev 200.524 2024-09-15T22:15:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.344+0000 7f520a228640 1 -- 172.21.15.5:0/890105502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5204114ab0 msgr2=0x7f5204116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.344+0000 7f520a228640 1 --2- 172.21.15.5:0/890105502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5204114ab0 0x7f5204116ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009ca0 tx=0x7f51fc02a960 comp rx=0 tx=0).stop 2024-09-15T22:15:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.348+0000 7f520a228640 1 -- 172.21.15.5:0/890105502 shutdown_connections 2024-09-15T22:15:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.348+0000 7f520a228640 1 --2- 172.21.15.5:0/890105502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5204114ab0 0x7f5204116ea0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.348+0000 7f520a228640 1 --2- 172.21.15.5:0/890105502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5204109b10 0x7f5204114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.348+0000 7f520a228640 1 --2- 172.21.15.5:0/890105502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52041091f0 0x7f52041095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.348+0000 7f520a228640 1 -- 172.21.15.5:0/890105502 >> 172.21.15.5:0/890105502 conn(0x7f520406e140 msgr2=0x7f520406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.351+0000 7f520a228640 1 -- 172.21.15.5:0/890105502 shutdown_connections 2024-09-15T22:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.355+0000 7f520a228640 1 -- 172.21.15.5:0/890105502 wait complete. 2024-09-15T22:15:50.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.355+0000 7f520a228640 1 Processor -- start 2024-09-15T22:15:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.360+0000 7f520a228640 1 -- start start 2024-09-15T22:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.360+0000 7f520a228640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52041091f0 0x7f520410a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.360+0000 7f520a228640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5204109b10 0x7f520410a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.360+0000 7f520a228640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5204114ab0 0x7f520410ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.360+0000 7f520a228640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f520411a8c0 con 0x7f5204114ab0 2024-09-15T22:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.360+0000 7f520a228640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f520411a740 con 0x7f52041091f0 2024-09-15T22:15:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.360+0000 7f520a228640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f520411aa40 con 0x7f5204109b10 2024-09-15T22:15:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.362+0000 7f5202ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5204109b10 0x7f520410a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.362+0000 7f52037fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52041091f0 0x7f520410a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.362+0000 7f52037fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52041091f0 0x7f520410a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46698/0 (socket says 172.21.15.5:46698) 2024-09-15T22:15:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.362+0000 7f52037fe640 1 -- 172.21.15.5:0/1660315881 learned_addr learned my addr 172.21.15.5:0/1660315881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.363+0000 7f52037fe640 1 -- 172.21.15.5:0/1660315881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5204109b10 msgr2=0x7f520410a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.363+0000 7f52037fe640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5204109b10 0x7f520410a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.363+0000 7f52037fe640 1 -- 172.21.15.5:0/1660315881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5204114ab0 msgr2=0x7f520410ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.363+0000 7f52037fe640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5204114ab0 0x7f520410ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.363+0000 7f52037fe640 1 -- 172.21.15.5:0/1660315881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52041a4e70 con 0x7f52041091f0 2024-09-15T22:15:50.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.363+0000 7f52037fe640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52041091f0 0x7f520410a3d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51f400b4a0 tx=0x7f51f400b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:50.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.365+0000 7f5200ff9640 1 -- 172.21.15.5:0/1660315881 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4014070 con 0x7f52041091f0 2024-09-15T22:15:50.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.365+0000 7f5200ff9640 1 -- 172.21.15.5:0/1660315881 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f4010af0 con 0x7f52041091f0 2024-09-15T22:15:50.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.365+0000 7f5200ff9640 1 -- 172.21.15.5:0/1660315881 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4017c60 con 0x7f52041091f0 2024-09-15T22:15:50.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.369+0000 7f520a228640 1 -- 172.21.15.5:0/1660315881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52041a5160 con 0x7f52041091f0 2024-09-15T22:15:50.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.369+0000 7f520a228640 1 -- 172.21.15.5:0/1660315881 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52041a55c0 con 0x7f52041091f0 2024-09-15T22:15:50.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.378+0000 7f520c601740 1 Processor -- start 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.378+0000 7f520c601740 1 -- start start 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.379+0000 7f520c601740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 0x557827e11090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.379+0000 7f520c601740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e115d0 0x557827e11a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.379+0000 7f520c601740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e1c370 0x557827e1e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.379+0000 7f520c601740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557827d772f0 con 0x557827e115d0 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.379+0000 7f520c601740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557827d1ecf0 con 0x557827e1c370 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.379+0000 7f520c601740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557827d732b0 con 0x557827e12b40 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.381+0000 7f5203fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 0x557827e11090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.381+0000 7f5203fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 0x557827e11090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:40148/0 (socket says 172.21.15.5:40148) 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.381+0000 7f5203fff640 1 -- 172.21.15.5:0/1649357616 learned_addr learned my addr 172.21.15.5:0/1649357616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.381+0000 7f5200ff9640 1 -- 172.21.15.5:0/1660315881 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f51f4017070 con 0x7f52041091f0 2024-09-15T22:15:50.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.382+0000 7f5202ffd640 1 --2- 172.21.15.5:0/1649357616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e1c370 0x557827e1e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.383+0000 7f5200ff9640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51ec0a35c0 0x7f51ec0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.383+0000 7f5200ff9640 1 -- 172.21.15.5:0/1660315881 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f51f40cf350 con 0x7f52041091f0 2024-09-15T22:15:50.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.383+0000 7f52037fe640 1 --2- 172.21.15.5:0/1649357616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e115d0 0x557827e11a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.385+0000 7f5203fff640 1 -- 172.21.15.5:0/1649357616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e1c370 msgr2=0x557827e1e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.385+0000 7f5203fff640 1 --2- 172.21.15.5:0/1649357616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e1c370 0x557827e1e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.385+0000 7f5203fff640 1 -- 172.21.15.5:0/1649357616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e115d0 msgr2=0x557827e11a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.385+0000 7f5203fff640 1 --2- 172.21.15.5:0/1649357616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e115d0 0x557827e11a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.385+0000 7f5203fff640 1 -- 172.21.15.5:0/1649357616 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557827d033a0 con 0x557827e12b40 2024-09-15T22:15:50.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.385+0000 7f5203fff640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51ec0a35c0 0x7f51ec0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.387+0000 7f5203fff640 1 --2- 172.21.15.5:0/1649357616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 0x557827e11090 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009bf0 tx=0x7f51fc02fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acf27c6213c0d8eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:50.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.390+0000 7f51e9ffb640 1 -- 172.21.15.5:0/1649357616 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51fc034040 con 0x557827e12b40 2024-09-15T22:15:50.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.391+0000 7f51e9ffb640 1 -- 172.21.15.5:0/1649357616 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51fc003e50 con 0x557827e12b40 2024-09-15T22:15:50.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.392+0000 7f520c601740 1 -- 172.21.15.5:0/1649357616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 msgr2=0x557827e11090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:50.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.392+0000 7f520c601740 1 --2- 172.21.15.5:0/1649357616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 0x557827e11090 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009bf0 tx=0x7f51fc02fc00 comp rx=0 tx=0).stop 2024-09-15T22:15:50.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.393+0000 7f5203fff640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51ec0a35c0 0x7f51ec0a5a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51fc04a000 tx=0x7f51fc0046f0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:50.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.393+0000 7f520c601740 1 -- 172.21.15.5:0/1649357616 shutdown_connections 2024-09-15T22:15:50.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.393+0000 7f520c601740 1 --2- 172.21.15.5:0/1649357616 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e1c370 0x557827e1e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.393+0000 7f520c601740 1 --2- 172.21.15.5:0/1649357616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e115d0 0x557827e11a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.393+0000 7f520c601740 1 --2- 172.21.15.5:0/1649357616 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 0x557827e11090 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.393+0000 7f520c601740 1 -- 172.21.15.5:0/1649357616 >> 172.21.15.5:0/1649357616 conn(0x557827dda310 msgr2=0x557827e0dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:50.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.398+0000 7f520c601740 1 -- 172.21.15.5:0/1649357616 shutdown_connections 2024-09-15T22:15:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.399+0000 7f520c601740 1 -- 172.21.15.5:0/1649357616 wait complete. 2024-09-15T22:15:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.399+0000 7f520c601740 1 -- start start 2024-09-15T22:15:50.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.400+0000 7f520c601740 1 Processor -- start 2024-09-15T22:15:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.400+0000 7f520c601740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e1bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.400+0000 7f520c601740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 0x557827e18e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.400+0000 7f520c601740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e1c370 0x557827e19390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.400+0000 7f520c601740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557827e1f0c0 con 0x557827e1c370 2024-09-15T22:15:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.400+0000 7f520c601740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557827d32060 con 0x557827e115d0 2024-09-15T22:15:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.400+0000 7f520c601740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557827e1f240 con 0x557827e12b40 2024-09-15T22:15:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.400+0000 7f520c601740 10 client.0 _ll_register_callbacks cb 0x557827a858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:50.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.401+0000 7f52037fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e1bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.401+0000 7f52037fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e1bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46720/0 (socket says 172.21.15.5:46720) 2024-09-15T22:15:50.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.401+0000 7f52037fe640 1 -- 172.21.15.5:0/2832273153 learned_addr learned my addr 172.21.15.5:0/2832273153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:50.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.401+0000 7f5202ffd640 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 0x557827e18e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.401+0000 7f5202ffd640 1 -- 172.21.15.5:0/2832273153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 msgr2=0x557827e1bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:50.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.401+0000 7f5202ffd640 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e1bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.401+0000 7f5202ffd640 1 -- 172.21.15.5:0/2832273153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e1c370 msgr2=0x557827e19390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:50.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.401+0000 7f5202ffd640 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e1c370 0x557827e19390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.401+0000 7f5202ffd640 1 -- 172.21.15.5:0/2832273153 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557827ae83c0 con 0x557827e12b40 2024-09-15T22:15:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.408+0000 7f5202ffd640 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 0x557827e18e50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f520410bb90 tx=0x7f51f800b590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2df698277924e45d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.412+0000 7f51e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:15:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.412+0000 7f51e2ffd640 1 -- 172.21.15.5:0/2832273153 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8016020 con 0x557827e12b40 2024-09-15T22:15:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.412+0000 7f520c601740 1 -- 172.21.15.5:0/2832273153 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51fc003e50 con 0x557827e12b40 2024-09-15T22:15:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.412+0000 7f51e2ffd640 1 -- 172.21.15.5:0/2832273153 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f51f8012680 con 0x557827e12b40 2024-09-15T22:15:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.413+0000 7f51e2ffd640 1 -- 172.21.15.5:0/2832273153 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f51f8021050 con 0x557827e12b40 2024-09-15T22:15:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.414+0000 7f520c601740 10 client.26182 fetch_fsmap learned FSMap version 25 2024-09-15T22:15:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.414+0000 7f520c601740 1 -- 172.21.15.5:0/2832273153 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557827e2aa60 con 0x557827e12b40 2024-09-15T22:15:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.417+0000 7f51e2ffd640 1 -- 172.21.15.5:0/2832273153 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51fc003e50 con 0x557827e12b40 2024-09-15T22:15:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.417+0000 7f51e2ffd640 1 -- 172.21.15.5:0/2832273153 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f51f8011c50 con 0x557827e12b40 2024-09-15T22:15:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.417+0000 7f520c601740 10 client.26182 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:15:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.417+0000 7f520c601740 10 client.26182 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:15:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.417+0000 7f520c601740 20 client.26182 populate_metadata read hostname 'smithi005' 2024-09-15T22:15:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.417+0000 7f520c601740 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x557827e2c3e0 0x557827e2e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.417+0000 7f520c601740 4 client.26182 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.417+0000 7f520c601740 1 -- 172.21.15.5:0/2832273153 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557827d84860 con 0x557827e2c3e0 2024-09-15T22:15:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.418+0000 7f52037fe640 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x557827e2c3e0 0x557827e2e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.420+0000 7f52037fe640 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x557827e2c3e0 0x557827e2e840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=79ea1943889a9d4f server_cookie=b08bcba991a35dc7 in_seq=0 out_seq=0 2024-09-15T22:15:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.420+0000 7f51e2ffd640 10 client.26182 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:15:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.421+0000 7f51e2ffd640 1 -- 172.21.15.5:0/2832273153 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557827d84860 con 0x557827e2c3e0 2024-09-15T22:15:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.421+0000 7f51e2ffd640 10 client.26182 handle_command_reply: tid=0 2024-09-15T22:15:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.421+0000 7f520c601740 1 client.26182 shutdown 2024-09-15T22:15:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.422+0000 7f520c601740 1 -- 172.21.15.5:0/2832273153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 msgr2=0x557827e18e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.422+0000 7f520c601740 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 0x557827e18e50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f520410bb90 tx=0x7f51f800b590 comp rx=0 tx=0).stop 2024-09-15T22:15:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.422+0000 7f520c601740 1 -- 172.21.15.5:0/2832273153 shutdown_connections 2024-09-15T22:15:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.422+0000 7f520c601740 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x557827e2c3e0 0x557827e2e840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.422+0000 7f520c601740 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e1c370 0x557827e19390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.422+0000 7f520c601740 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e12b40 0x557827e18e50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.422+0000 7f520c601740 1 --2- 172.21.15.5:0/2832273153 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e1bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.422+0000 7f520c601740 1 -- 172.21.15.5:0/2832273153 >> 172.21.15.5:0/2832273153 conn(0x557827dda310 msgr2=0x557827e13960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.422+0000 7f51e2ffd640 0 client.26182 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:15:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.423+0000 7f520c601740 1 -- 172.21.15.5:0/2832273153 shutdown_connections 2024-09-15T22:15:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.423+0000 7f520c601740 1 -- 172.21.15.5:0/2832273153 wait complete. 2024-09-15T22:15:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.423+0000 7f520c601740 20 client.26182 trim_cache size 0 max 16384 2024-09-15T22:15:50.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f520c601740 1 Processor -- start 2024-09-15T22:15:50.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f520c601740 1 -- start start 2024-09-15T22:15:50.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f520c601740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e14120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f520c601740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 0x557827e14660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f520c601740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e1c370 0x557827e34150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f5202ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 0x557827e14660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f520c601740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52041c9050 con 0x557827e12b40 2024-09-15T22:15:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f5202ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 0x557827e14660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45386/0 (socket says 172.21.15.5:45386) 2024-09-15T22:15:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f5202ffd640 1 -- 172.21.15.5:0/536105934 learned_addr learned my addr 172.21.15.5:0/536105934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f520c601740 1 -- 172.21.15.5:0/536105934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f52041c6920 con 0x557827e115d0 2024-09-15T22:15:50.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f520c601740 1 -- 172.21.15.5:0/536105934 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52041095d0 con 0x557827e1c370 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.448+0000 7f52037fe640 1 --2- 172.21.15.5:0/536105934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e14120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.449+0000 7f5203fff640 1 --2- 172.21.15.5:0/536105934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e1c370 0x557827e34150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.449+0000 7f5202ffd640 1 -- 172.21.15.5:0/536105934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e1c370 msgr2=0x557827e34150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.449+0000 7f5202ffd640 1 --2- 172.21.15.5:0/536105934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e1c370 0x557827e34150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.449+0000 7f5202ffd640 1 -- 172.21.15.5:0/536105934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 msgr2=0x557827e14120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.449+0000 7f5202ffd640 1 --2- 172.21.15.5:0/536105934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e14120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.449+0000 7f5202ffd640 1 -- 172.21.15.5:0/536105934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557827e1ead0 con 0x557827e12b40 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.449+0000 7f52037fe640 1 --2- 172.21.15.5:0/536105934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e14120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.449+0000 7f5202ffd640 1 --2- 172.21.15.5:0/536105934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 0x557827e14660 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f51f8001f70 tx=0x7f51f801d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f3cc408bfe2be4d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.450+0000 7f51e97fa640 1 -- 172.21.15.5:0/536105934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8029020 con 0x557827e12b40 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.450+0000 7f51e97fa640 1 -- 172.21.15.5:0/536105934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f8006c10 con 0x557827e12b40 2024-09-15T22:15:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.450+0000 7f51e97fa640 1 -- 172.21.15.5:0/536105934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8016030 con 0x557827e12b40 2024-09-15T22:15:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.451+0000 7f520c601740 1 -- 172.21.15.5:0/536105934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 msgr2=0x557827e14660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.451+0000 7f520c601740 1 --2- 172.21.15.5:0/536105934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 0x557827e14660 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f51f8001f70 tx=0x7f51f801d8c0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.451+0000 7f520c601740 1 -- 172.21.15.5:0/536105934 shutdown_connections 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.451+0000 7f520c601740 1 --2- 172.21.15.5:0/536105934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e1c370 0x557827e34150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.451+0000 7f520c601740 1 --2- 172.21.15.5:0/536105934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 0x557827e14660 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.451+0000 7f520c601740 1 --2- 172.21.15.5:0/536105934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e14120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.451+0000 7f520c601740 1 -- 172.21.15.5:0/536105934 >> 172.21.15.5:0/536105934 conn(0x557827dda310 msgr2=0x557827e17720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.451+0000 7f520c601740 1 -- 172.21.15.5:0/536105934 shutdown_connections 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.451+0000 7f520c601740 1 -- 172.21.15.5:0/536105934 wait complete. 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.451+0000 7f520c601740 1 -- start start 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.452+0000 7f520c601740 1 Processor -- start 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.452+0000 7f520c601740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.452+0000 7f520c601740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 0x557827e20830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.452+0000 7f52037fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f520c601740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e1c370 0x557827e20d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f52037fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e202f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46726/0 (socket says 172.21.15.5:46726) 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f52037fe640 1 -- 172.21.15.5:0/4206296408 learned_addr learned my addr 172.21.15.5:0/4206296408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f520c601740 1 -- 172.21.15.5:0/4206296408 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52041ad500 con 0x557827e12b40 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f520c601740 1 -- 172.21.15.5:0/4206296408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f52041c6250 con 0x557827e115d0 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f520c601740 1 -- 172.21.15.5:0/4206296408 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f52041c54b0 con 0x557827e1c370 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f5203fff640 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e1c370 0x557827e20d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f520c601740 10 client.0 _ll_register_callbacks cb 0x7f51f8006dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f5202ffd640 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 0x557827e20830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f52037fe640 1 -- 172.21.15.5:0/4206296408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e1c370 msgr2=0x557827e20d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f52037fe640 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e1c370 0x557827e20d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f52037fe640 1 -- 172.21.15.5:0/4206296408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 msgr2=0x557827e20830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f52037fe640 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 0x557827e20830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.453+0000 7f52037fe640 1 -- 172.21.15.5:0/4206296408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557827e961a0 con 0x557827e115d0 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.454+0000 7f52037fe640 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e202f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51f400be60 tx=0x7f51f4017520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b72b5fb7a4879c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.454+0000 7f51e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.454+0000 7f51e2ffd640 1 -- 172.21.15.5:0/4206296408 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f409f6f0 con 0x557827e115d0 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.454+0000 7f51e2ffd640 1 -- 172.21.15.5:0/4206296408 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f51f4017e00 con 0x557827e115d0 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.454+0000 7f520c601740 1 -- 172.21.15.5:0/4206296408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557827d7ed70 con 0x557827e115d0 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.454+0000 7f51e2ffd640 1 -- 172.21.15.5:0/4206296408 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f51f40a3fb0 con 0x557827e115d0 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.454+0000 7f520c601740 10 client.26222 fetch_fsmap learned FSMap version 25 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.454+0000 7f520c601740 1 -- 172.21.15.5:0/4206296408 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557827e97480 con 0x557827e115d0 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.455+0000 7f51e2ffd640 1 -- 172.21.15.5:0/4206296408 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557827d7ed70 con 0x557827e115d0 2024-09-15T22:15:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.455+0000 7f51e2ffd640 1 -- 172.21.15.5:0/4206296408 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f51f40979b0 con 0x557827e115d0 2024-09-15T22:15:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.455+0000 7f520c601740 10 client.26222 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:15:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.455+0000 7f520c601740 10 client.26222 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:15:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.455+0000 7f520c601740 20 client.26222 populate_metadata read hostname 'smithi005' 2024-09-15T22:15:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.455+0000 7f520c601740 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x557827e97ba0 0x557827e99fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.455+0000 7f520c601740 4 client.26222 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:15:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.455+0000 7f520c601740 1 -- 172.21.15.5:0/4206296408 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557827cfb170 con 0x557827e97ba0 2024-09-15T22:15:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.455+0000 7f5202ffd640 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x557827e97ba0 0x557827e99fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.455+0000 7f51e2ffd640 10 client.26222 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:15:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.455+0000 7f5202ffd640 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x557827e97ba0 0x557827e99fc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b07c670aacd482c server_cookie=41c4e716c52c9a10 in_seq=0 out_seq=0 2024-09-15T22:15:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f51e2ffd640 1 -- 172.21.15.5:0/4206296408 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557827cfb170 con 0x557827e97ba0 2024-09-15T22:15:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f51e2ffd640 10 client.26222 handle_command_reply: tid=0 2024-09-15T22:15:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f520c601740 1 client.26222 shutdown 2024-09-15T22:15:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f520c601740 1 -- 172.21.15.5:0/4206296408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 msgr2=0x557827e202f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f520c601740 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e202f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f51f400be60 tx=0x7f51f4017520 comp rx=0 tx=0).stop 2024-09-15T22:15:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f520c601740 1 -- 172.21.15.5:0/4206296408 shutdown_connections 2024-09-15T22:15:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f520c601740 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x557827e97ba0 0x557827e99fc0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f520c601740 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557827e1c370 0x557827e20d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f520c601740 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557827e12b40 0x557827e20830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f520c601740 1 --2- 172.21.15.5:0/4206296408 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557827e115d0 0x557827e202f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f520c601740 1 -- 172.21.15.5:0/4206296408 >> 172.21.15.5:0/4206296408 conn(0x557827dda310 msgr2=0x557827e1a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f51e2ffd640 0 client.26222 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:15:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f520c601740 1 -- 172.21.15.5:0/4206296408 shutdown_connections 2024-09-15T22:15:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.456+0000 7f520c601740 1 -- 172.21.15.5:0/4206296408 wait complete. 2024-09-15T22:15:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520c601740 20 client.26222 trim_cache size 0 max 16384 2024-09-15T22:15:50.465 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:15:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 -- 172.21.15.5:0/1660315881 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51ec0a35c0 msgr2=0x7f51ec0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51ec0a35c0 0x7f51ec0a5a80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f51fc04a000 tx=0x7f51fc0046f0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 -- 172.21.15.5:0/1660315881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52041091f0 msgr2=0x7f520410a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52041091f0 0x7f520410a3d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51f400b4a0 tx=0x7f51f400b960 comp rx=0 tx=0).stop 2024-09-15T22:15:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 -- 172.21.15.5:0/1660315881 shutdown_connections 2024-09-15T22:15:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51ec0a35c0 0x7f51ec0a5a80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5204114ab0 0x7f520410ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5204109b10 0x7f520410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 --2- 172.21.15.5:0/1660315881 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f52041091f0 0x7f520410a3d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 -- 172.21.15.5:0/1660315881 >> 172.21.15.5:0/1660315881 conn(0x7f520406e140 msgr2=0x7f5204070840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 -- 172.21.15.5:0/1660315881 shutdown_connections 2024-09-15T22:15:50.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:50.457+0000 7f520a228640 1 -- 172.21.15.5:0/1660315881 wait complete. 2024-09-15T22:15:51.335 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:15:51.509 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:51 smithi005 ceph-mon[30944]: pgmap v211: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-09-15T22:15:51.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:51 smithi099 ceph-mon[34892]: pgmap v211: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-09-15T22:15:51.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:51 smithi129 ceph-mon[35314]: pgmap v211: 129 pgs: 129 active+clean; 3.3 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 19 MiB/s wr, 1.34k op/s 2024-09-15T22:15:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.746+0000 7fa321862640 1 -- 172.21.15.5:0/652193843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa31c102390 msgr2=0x7fa31c10e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.746+0000 7fa321862640 1 --2- 172.21.15.5:0/652193843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa31c102390 0x7fa31c10e8c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa314009cd0 tx=0x7fa31402a8f0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.747+0000 7fa321862640 1 -- 172.21.15.5:0/652193843 shutdown_connections 2024-09-15T22:15:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.747+0000 7fa321862640 1 --2- 172.21.15.5:0/652193843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa31c102390 0x7fa31c10e8c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.747+0000 7fa321862640 1 --2- 172.21.15.5:0/652193843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa31c1019f0 0x7fa31c101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.747+0000 7fa321862640 1 --2- 172.21.15.5:0/652193843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa31c1079f0 0x7fa31c107dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.747+0000 7fa321862640 1 -- 172.21.15.5:0/652193843 >> 172.21.15.5:0/652193843 conn(0x7fa31c0fd730 msgr2=0x7fa31c0ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.747+0000 7fa321862640 1 -- 172.21.15.5:0/652193843 shutdown_connections 2024-09-15T22:15:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.747+0000 7fa321862640 1 -- 172.21.15.5:0/652193843 wait complete. 2024-09-15T22:15:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.747+0000 7fa321862640 1 Processor -- start 2024-09-15T22:15:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa321862640 1 -- start start 2024-09-15T22:15:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa321862640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa31c1019f0 0x7fa31c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa321862640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa31c102390 0x7fa31c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa321862640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa31c1079f0 0x7fa31c19cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa321862640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c111e40 con 0x7fa31c1079f0 2024-09-15T22:15:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa321862640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c111cc0 con 0x7fa31c102390 2024-09-15T22:15:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa321862640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c111fc0 con 0x7fa31c1019f0 2024-09-15T22:15:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa31affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa31c1019f0 0x7fa31c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa31a7fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa31c102390 0x7fa31c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa31affd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa31c1019f0 0x7fa31c1982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:40170/0 (socket says 172.21.15.5:40170) 2024-09-15T22:15:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa31affd640 1 -- 172.21.15.5:0/2430729036 learned_addr learned my addr 172.21.15.5:0/2430729036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.748+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa31c1079f0 0x7fa31c19cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.749+0000 7fa31affd640 1 -- 172.21.15.5:0/2430729036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa31c102390 msgr2=0x7fa31c198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.749+0000 7fa31affd640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa31c102390 0x7fa31c198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.749+0000 7fa31affd640 1 -- 172.21.15.5:0/2430729036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa31c1079f0 msgr2=0x7fa31c19cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.749+0000 7fa31affd640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa31c1079f0 0x7fa31c19cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.749+0000 7fa31affd640 1 -- 172.21.15.5:0/2430729036 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c19d270 con 0x7fa31c1019f0 2024-09-15T22:15:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.749+0000 7fa31affd640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa31c1019f0 0x7fa31c1982c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00ea60 tx=0x7fa30c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.749+0000 7fa320860640 1 -- 172.21.15.5:0/2430729036 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00cde0 con 0x7fa31c1019f0 2024-09-15T22:15:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.749+0000 7fa320860640 1 -- 172.21.15.5:0/2430729036 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c0035f0 con 0x7fa31c1019f0 2024-09-15T22:15:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.749+0000 7fa320860640 1 -- 172.21.15.5:0/2430729036 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c010670 con 0x7fa31c1019f0 2024-09-15T22:15:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.749+0000 7fa321862640 1 -- 172.21.15.5:0/2430729036 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c19d560 con 0x7fa31c1019f0 2024-09-15T22:15:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.749+0000 7fa321862640 1 -- 172.21.15.5:0/2430729036 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c074420 con 0x7fa31c1019f0 2024-09-15T22:15:52.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.751+0000 7fa320860640 1 -- 172.21.15.5:0/2430729036 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa30c004020 con 0x7fa31c1019f0 2024-09-15T22:15:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.753+0000 7fa323c3b740 1 Processor -- start 2024-09-15T22:15:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.753+0000 7fa323c3b740 1 -- start start 2024-09-15T22:15:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132c0cf0 0x5564132bf240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132bf780 0x5564132bfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 0x5564132cc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa323c3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5564132252f0 con 0x5564132bf780 2024-09-15T22:15:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa323c3b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5564131cccf0 con 0x5564132c0cf0 2024-09-15T22:15:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa323c3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5564132212b0 con 0x5564132ca520 2024-09-15T22:15:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa31b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132c0cf0 0x5564132bf240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa31b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132c0cf0 0x5564132bf240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46750/0 (socket says 172.21.15.5:46750) 2024-09-15T22:15:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa31b7fe640 1 -- 172.21.15.5:0/3118433092 learned_addr learned my addr 172.21.15.5:0/3118433092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:52.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa31b7fe640 1 -- 172.21.15.5:0/3118433092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 msgr2=0x5564132cc910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/3118433092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 0x5564132cc910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa31b7fe640 1 -- 172.21.15.5:0/3118433092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132bf780 msgr2=0x5564132bfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/3118433092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132bf780 0x5564132bfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa31b7fe640 1 -- 172.21.15.5:0/3118433092 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564131b13a0 con 0x5564132c0cf0 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.754+0000 7fa320860640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3040a34b0 0x7fa3040a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.755+0000 7fa31affd640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3040a34b0 0x7fa3040a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.755+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/3118433092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132c0cf0 0x5564132bf240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa314009cd0 tx=0x7fa3140093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=741be5ff56c3f9b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.755+0000 7fa320860640 1 -- 172.21.15.5:0/2430729036 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa30c0d0320 con 0x7fa31c1019f0 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.755+0000 7fa301ffb640 1 -- 172.21.15.5:0/3118433092 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314030720 con 0x5564132c0cf0 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.755+0000 7fa301ffb640 1 -- 172.21.15.5:0/3118433092 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314036030 con 0x5564132c0cf0 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.755+0000 7fa301ffb640 1 -- 172.21.15.5:0/3118433092 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140043c0 con 0x5564132c0cf0 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.756+0000 7fa323c3b740 1 -- 172.21.15.5:0/3118433092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132c0cf0 msgr2=0x5564132bf240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.756+0000 7fa323c3b740 1 --2- 172.21.15.5:0/3118433092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132c0cf0 0x5564132bf240 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa314009cd0 tx=0x7fa3140093e0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.756+0000 7fa31affd640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3040a34b0 0x7fa3040a5970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5564132bcc70 tx=0x7fa30c066bf0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.756+0000 7fa323c3b740 1 -- 172.21.15.5:0/3118433092 shutdown_connections 2024-09-15T22:15:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.756+0000 7fa323c3b740 1 --2- 172.21.15.5:0/3118433092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 0x5564132cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.756+0000 7fa323c3b740 1 --2- 172.21.15.5:0/3118433092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132bf780 0x5564132bfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.756+0000 7fa323c3b740 1 --2- 172.21.15.5:0/3118433092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132c0cf0 0x5564132bf240 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.756+0000 7fa323c3b740 1 -- 172.21.15.5:0/3118433092 >> 172.21.15.5:0/3118433092 conn(0x556413288310 msgr2=0x5564132bbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.756+0000 7fa323c3b740 1 -- 172.21.15.5:0/3118433092 shutdown_connections 2024-09-15T22:15:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.756+0000 7fa323c3b740 1 -- 172.21.15.5:0/3118433092 wait complete. 2024-09-15T22:15:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.757+0000 7fa323c3b740 1 -- start start 2024-09-15T22:15:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.757+0000 7fa323c3b740 1 Processor -- start 2024-09-15T22:15:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.758+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.758+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132c0cf0 0x5564132c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.758+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132ca520 0x5564132c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.758+0000 7fa323c3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5564132d5a20 con 0x5564132ca520 2024-09-15T22:15:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.758+0000 7fa323c3b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5564131e0060 con 0x5564132bf780 2024-09-15T22:15:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.758+0000 7fa323c3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5564132d5ba0 con 0x5564132c0cf0 2024-09-15T22:15:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.759+0000 7fa323c3b740 10 client.0 _ll_register_callbacks cb 0x556412f338c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.759+0000 7fa31b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.759+0000 7fa31a7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132c0cf0 0x5564132c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.759+0000 7fa31b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46762/0 (socket says 172.21.15.5:46762) 2024-09-15T22:15:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.759+0000 7fa31b7fe640 1 -- 172.21.15.5:0/2008061370 learned_addr learned my addr 172.21.15.5:0/2008061370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.759+0000 7fa31b7fe640 1 -- 172.21.15.5:0/2008061370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132c0cf0 msgr2=0x5564132c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.759+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/2008061370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132c0cf0 0x5564132c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.759+0000 7fa31b7fe640 1 -- 172.21.15.5:0/2008061370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132ca520 msgr2=0x5564132c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.759+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/2008061370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132ca520 0x5564132c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.759+0000 7fa31b7fe640 1 -- 172.21.15.5:0/2008061370 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556412f963c0 con 0x5564132bf780 2024-09-15T22:15:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.760+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/2008061370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132c9ed0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa314009a60 tx=0x7fa3140099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2960c97e7d16a7cf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.760+0000 7fa2faffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:15:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.760+0000 7fa2faffd640 1 -- 172.21.15.5:0/2008061370 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314003e40 con 0x5564132bf780 2024-09-15T22:15:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.760+0000 7fa323c3b740 1 -- 172.21.15.5:0/2008061370 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa314036030 con 0x5564132bf780 2024-09-15T22:15:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.760+0000 7fa2faffd640 1 -- 172.21.15.5:0/2008061370 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa314004160 con 0x5564132bf780 2024-09-15T22:15:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.761+0000 7fa2faffd640 1 -- 172.21.15.5:0/2008061370 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa31405e050 con 0x5564132bf780 2024-09-15T22:15:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.761+0000 7fa323c3b740 10 client.26234 fetch_fsmap learned FSMap version 25 2024-09-15T22:15:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.761+0000 7fa323c3b740 1 -- 172.21.15.5:0/2008061370 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5564132d5a20 con 0x5564132bf780 2024-09-15T22:15:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.761+0000 7fa2faffd640 1 -- 172.21.15.5:0/2008061370 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314036030 con 0x5564132bf780 2024-09-15T22:15:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.761+0000 7fa2faffd640 1 -- 172.21.15.5:0/2008061370 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa314043070 con 0x5564132bf780 2024-09-15T22:15:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.761+0000 7fa323c3b740 10 client.26234 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:15:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.761+0000 7fa323c3b740 10 client.26234 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:15:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.761+0000 7fa323c3b740 20 client.26234 populate_metadata read hostname 'smithi005' 2024-09-15T22:15:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.761+0000 7fa323c3b740 1 --2- 172.21.15.5:0/2008061370 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5564132e2610 0x5564132e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.761+0000 7fa323c3b740 4 client.26234 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.761+0000 7fa323c3b740 1 -- 172.21.15.5:0/2008061370 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556413232860 con 0x5564132e2610 2024-09-15T22:15:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.762+0000 7fa31a7fc640 1 --2- 172.21.15.5:0/2008061370 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5564132e2610 0x5564132e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.762+0000 7fa2faffd640 10 client.26234 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:15:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.762+0000 7fa31a7fc640 1 --2- 172.21.15.5:0/2008061370 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5564132e2610 0x5564132e4a00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8f4730d8d8bf54f server_cookie=72c55bc42b0455a in_seq=0 out_seq=0 2024-09-15T22:15:52.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa2faffd640 1 -- 172.21.15.5:0/2008061370 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556413232860 con 0x5564132e2610 2024-09-15T22:15:52.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa2faffd640 10 client.26234 handle_command_reply: tid=0 2024-09-15T22:15:52.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 1 client.26234 shutdown 2024-09-15T22:15:52.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 1 -- 172.21.15.5:0/2008061370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 msgr2=0x5564132c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:52.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 1 --2- 172.21.15.5:0/2008061370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132c9ed0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa314009a60 tx=0x7fa3140099c0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 1 -- 172.21.15.5:0/2008061370 shutdown_connections 2024-09-15T22:15:52.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 1 --2- 172.21.15.5:0/2008061370 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5564132e2610 0x5564132e4a00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 1 --2- 172.21.15.5:0/2008061370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132ca520 0x5564132c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 1 --2- 172.21.15.5:0/2008061370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132c0cf0 0x5564132c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 1 --2- 172.21.15.5:0/2008061370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132c9ed0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa2faffd640 0 client.26234 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:15:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 1 -- 172.21.15.5:0/2008061370 >> 172.21.15.5:0/2008061370 conn(0x556413288310 msgr2=0x5564132c1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 1 -- 172.21.15.5:0/2008061370 shutdown_connections 2024-09-15T22:15:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 1 -- 172.21.15.5:0/2008061370 wait complete. 2024-09-15T22:15:52.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.763+0000 7fa323c3b740 20 client.26234 trim_cache size 0 max 16384 2024-09-15T22:15:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.785+0000 7fa323c3b740 1 Processor -- start 2024-09-15T22:15:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.786+0000 7fa323c3b740 1 -- start start 2024-09-15T22:15:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.786+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132dece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.786+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132c0cf0 0x5564132c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.786+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 0x5564132ea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.786+0000 7fa323c3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1af540 con 0x5564132c0cf0 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.786+0000 7fa323c3b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1ace10 con 0x5564132bf780 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.786+0000 7fa323c3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1aac60 con 0x5564132ca520 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.786+0000 7fa31b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132dece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.786+0000 7fa31b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132dece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46768/0 (socket says 172.21.15.5:46768) 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.786+0000 7fa31b7fe640 1 -- 172.21.15.5:0/4102115979 learned_addr learned my addr 172.21.15.5:0/4102115979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.786+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/4102115979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 0x5564132ea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.787+0000 7fa31b7fe640 1 -- 172.21.15.5:0/4102115979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 msgr2=0x5564132ea060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.787+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/4102115979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 0x5564132ea060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.787+0000 7fa31b7fe640 1 -- 172.21.15.5:0/4102115979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132c0cf0 msgr2=0x5564132c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.787+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/4102115979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132c0cf0 0x5564132c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.787+0000 7fa31b7fe640 1 -- 172.21.15.5:0/4102115979 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564132c56d0 con 0x5564132bf780 2024-09-15T22:15:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.787+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/4102115979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132dece0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa314005de0 tx=0x7fa314047040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ed548ea6df86abf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.787+0000 7fa3017fa640 1 -- 172.21.15.5:0/4102115979 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314003e40 con 0x5564132bf780 2024-09-15T22:15:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.787+0000 7fa3017fa640 1 -- 172.21.15.5:0/4102115979 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314036070 con 0x5564132bf780 2024-09-15T22:15:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.788+0000 7fa3017fa640 1 -- 172.21.15.5:0/4102115979 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3140046a0 con 0x5564132bf780 2024-09-15T22:15:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.788+0000 7fa323c3b740 1 -- 172.21.15.5:0/4102115979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 msgr2=0x5564132dece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.788+0000 7fa323c3b740 1 --2- 172.21.15.5:0/4102115979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132dece0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa314005de0 tx=0x7fa314047040 comp rx=0 tx=0).stop 2024-09-15T22:15:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.789+0000 7fa323c3b740 1 -- 172.21.15.5:0/4102115979 shutdown_connections 2024-09-15T22:15:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.789+0000 7fa323c3b740 1 --2- 172.21.15.5:0/4102115979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 0x5564132ea060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.789+0000 7fa323c3b740 1 --2- 172.21.15.5:0/4102115979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132c0cf0 0x5564132c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.789+0000 7fa323c3b740 1 --2- 172.21.15.5:0/4102115979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132dece0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.789+0000 7fa323c3b740 1 -- 172.21.15.5:0/4102115979 >> 172.21.15.5:0/4102115979 conn(0x556413288310 msgr2=0x5564132c5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.789+0000 7fa323c3b740 1 -- 172.21.15.5:0/4102115979 shutdown_connections 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.789+0000 7fa323c3b740 1 -- 172.21.15.5:0/4102115979 wait complete. 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.789+0000 7fa323c3b740 1 -- start start 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa323c3b740 1 Processor -- start 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132db440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132c0cf0 0x5564132c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa323c3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 0x5564132c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa323c3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1ab2d0 con 0x5564132c0cf0 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa323c3b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1ac740 con 0x5564132bf780 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa323c3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1ab9a0 con 0x5564132ca520 2024-09-15T22:15:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa323c3b740 10 client.0 _ll_register_callbacks cb 0x7fa314030cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa31b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132db440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa31b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132db440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46782/0 (socket says 172.21.15.5:46782) 2024-09-15T22:15:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa31b7fe640 1 -- 172.21.15.5:0/1036114727 learned_addr learned my addr 172.21.15.5:0/1036114727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa31a7fc640 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132c0cf0 0x5564132c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.790+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 0x5564132c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.791+0000 7fa31b7fe640 1 -- 172.21.15.5:0/1036114727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 msgr2=0x5564132c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.791+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 0x5564132c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.791+0000 7fa31b7fe640 1 -- 172.21.15.5:0/1036114727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132c0cf0 msgr2=0x5564132c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.791+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132c0cf0 0x5564132c3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.791+0000 7fa31b7fe640 1 -- 172.21.15.5:0/1036114727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564132c48b0 con 0x5564132bf780 2024-09-15T22:15:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.791+0000 7fa31b7fe640 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132db440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa314050ba0 tx=0x7fa314005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db883e6ead584ac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.791+0000 7fa2faffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:15:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.791+0000 7fa2faffd640 1 -- 172.21.15.5:0/1036114727 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314003e40 con 0x5564132bf780 2024-09-15T22:15:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.791+0000 7fa323c3b740 1 -- 172.21.15.5:0/1036114727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa314036070 con 0x5564132bf780 2024-09-15T22:15:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.791+0000 7fa2faffd640 1 -- 172.21.15.5:0/1036114727 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7fa314004440 con 0x5564132bf780 2024-09-15T22:15:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.791+0000 7fa2faffd640 1 -- 172.21.15.5:0/1036114727 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa314004640 con 0x5564132bf780 2024-09-15T22:15:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.792+0000 7fa323c3b740 10 client.26240 fetch_fsmap learned FSMap version 25 2024-09-15T22:15:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.792+0000 7fa323c3b740 1 -- 172.21.15.5:0/1036114727 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5564132d4d10 con 0x5564132bf780 2024-09-15T22:15:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.793+0000 7fa2faffd640 1 -- 172.21.15.5:0/1036114727 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314036070 con 0x5564132bf780 2024-09-15T22:15:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.793+0000 7fa2faffd640 1 -- 172.21.15.5:0/1036114727 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa31406c020 con 0x5564132bf780 2024-09-15T22:15:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.793+0000 7fa323c3b740 10 client.26240 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:15:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.793+0000 7fa323c3b740 10 client.26240 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:15:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.793+0000 7fa323c3b740 20 client.26240 populate_metadata read hostname 'smithi005' 2024-09-15T22:15:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.793+0000 7fa323c3b740 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5564132d5260 0x556413358010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.793+0000 7fa323c3b740 4 client.26240 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:15:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.793+0000 7fa323c3b740 1 -- 172.21.15.5:0/1036114727 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55641322cd70 con 0x5564132d5260 2024-09-15T22:15:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.793+0000 7fa31a7fc640 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5564132d5260 0x556413358010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.794+0000 7fa2faffd640 10 client.26240 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.794+0000 7fa31a7fc640 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5564132d5260 0x556413358010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67cc79f0a38303bc server_cookie=1d5a5e273a4dcdae in_seq=0 out_seq=0 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa2faffd640 1 -- 172.21.15.5:0/1036114727 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55641322cd70 con 0x5564132d5260 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa2faffd640 10 client.26240 handle_command_reply: tid=0 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 1 client.26240 shutdown 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 1 -- 172.21.15.5:0/1036114727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 msgr2=0x5564132db440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132db440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa314050ba0 tx=0x7fa314005bd0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 1 -- 172.21.15.5:0/1036114727 shutdown_connections 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5564132d5260 0x556413358010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5564132ca520 0x5564132c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa2faffd640 0 client.26240 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564132c0cf0 0x5564132c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 1 --2- 172.21.15.5:0/1036114727 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5564132bf780 0x5564132db440 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 1 -- 172.21.15.5:0/1036114727 >> 172.21.15.5:0/1036114727 conn(0x556413288310 msgr2=0x5564132c6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 1 -- 172.21.15.5:0/1036114727 shutdown_connections 2024-09-15T22:15:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 1 -- 172.21.15.5:0/1036114727 wait complete. 2024-09-15T22:15:52.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.795+0000 7fa323c3b740 20 client.26240 trim_cache size 0 max 16384 2024-09-15T22:15:52.801 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:15:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.803+0000 7fa321862640 1 -- 172.21.15.5:0/2430729036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3040a34b0 msgr2=0x7fa3040a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.803+0000 7fa321862640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3040a34b0 0x7fa3040a5970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5564132bcc70 tx=0x7fa30c066bf0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.803+0000 7fa321862640 1 -- 172.21.15.5:0/2430729036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa31c1019f0 msgr2=0x7fa31c1982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.803+0000 7fa321862640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa31c1019f0 0x7fa31c1982c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00ea60 tx=0x7fa30c00ef20 comp rx=0 tx=0).stop 2024-09-15T22:15:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.803+0000 7fa321862640 1 -- 172.21.15.5:0/2430729036 shutdown_connections 2024-09-15T22:15:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.803+0000 7fa321862640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa3040a34b0 0x7fa3040a5970 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.803+0000 7fa321862640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa31c1079f0 0x7fa31c19cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.803+0000 7fa321862640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa31c102390 0x7fa31c198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.803+0000 7fa321862640 1 --2- 172.21.15.5:0/2430729036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa31c1019f0 0x7fa31c1982c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.803+0000 7fa321862640 1 -- 172.21.15.5:0/2430729036 >> 172.21.15.5:0/2430729036 conn(0x7fa31c0fd730 msgr2=0x7fa31c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.803+0000 7fa321862640 1 -- 172.21.15.5:0/2430729036 shutdown_connections 2024-09-15T22:15:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:52.804+0000 7fa321862640 1 -- 172.21.15.5:0/2430729036 wait complete. 2024-09-15T22:15:53.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:53 smithi099 ceph-mon[34892]: pgmap v212: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2024-09-15T22:15:53.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:53 smithi129 ceph-mon[35314]: pgmap v212: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2024-09-15T22:15:53.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:53 smithi005 ceph-mon[30944]: pgmap v212: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 21 MiB/s wr, 1.53k op/s 2024-09-15T22:15:54.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 385.0 s, 941.8 tps, lat 104.141 ms stddev 171.601 2024-09-15T22:15:55.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:55 smithi099 ceph-mon[34892]: pgmap v213: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.33k op/s 2024-09-15T22:15:55.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:55 smithi129 ceph-mon[35314]: pgmap v213: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.33k op/s 2024-09-15T22:15:55.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:55 smithi005 ceph-mon[30944]: pgmap v213: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 16 MiB/s wr, 1.33k op/s 2024-09-15T22:15:55.734 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:15:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:15:57.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:57 smithi005 ceph-mon[30944]: pgmap v214: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2024-09-15T22:15:57.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:57 smithi099 ceph-mon[34892]: pgmap v214: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2024-09-15T22:15:57.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:57 smithi129 ceph-mon[35314]: pgmap v214: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 20 MiB/s wr, 1.60k op/s 2024-09-15T22:15:57.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.895+0000 7f2a1840d640 1 -- 172.21.15.5:0/1730849067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a10108450 msgr2=0x7f2a10108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:57.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.895+0000 7f2a1840d640 1 --2- 172.21.15.5:0/1730849067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a10108450 0x7f2a10108830 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009900 tx=0x7f2a0c02a410 comp rx=0 tx=0).stop 2024-09-15T22:15:57.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.895+0000 7f2a1840d640 1 -- 172.21.15.5:0/1730849067 shutdown_connections 2024-09-15T22:15:57.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.895+0000 7f2a1840d640 1 --2- 172.21.15.5:0/1730849067 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a1010d530 0x7f2a101150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.895+0000 7f2a1840d640 1 --2- 172.21.15.5:0/1730849067 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a10108e00 0x7f2a1010ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.895+0000 7f2a1840d640 1 --2- 172.21.15.5:0/1730849067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a10108450 0x7f2a10108830 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.895+0000 7f2a1840d640 1 -- 172.21.15.5:0/1730849067 >> 172.21.15.5:0/1730849067 conn(0x7f2a10077ba0 msgr2=0x7f2a10079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.895+0000 7f2a1840d640 1 -- 172.21.15.5:0/1730849067 shutdown_connections 2024-09-15T22:15:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.895+0000 7f2a1840d640 1 -- 172.21.15.5:0/1730849067 wait complete. 2024-09-15T22:15:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.896+0000 7f2a1840d640 1 Processor -- start 2024-09-15T22:15:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.896+0000 7f2a1840d640 1 -- start start 2024-09-15T22:15:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.896+0000 7f2a1840d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a10108450 0x7f2a101a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.896+0000 7f2a1840d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a10108e00 0x7f2a101a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.896+0000 7f2a1840d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a1010d530 0x7f2a1019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.896+0000 7f2a1840d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a10118580 con 0x7f2a10108e00 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.896+0000 7f2a1840d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a10118400 con 0x7f2a10108450 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.896+0000 7f2a1840d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a10118700 con 0x7f2a1010d530 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.896+0000 7f2a161a9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a10108450 0x7f2a101a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a161a9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a10108450 0x7f2a101a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:41918/0 (socket says 172.21.15.5:41918) 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a161a9640 1 -- 172.21.15.5:0/875906445 learned_addr learned my addr 172.21.15.5:0/875906445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a169aa640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a1010d530 0x7f2a1019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a159a8640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a10108e00 0x7f2a101a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a161a9640 1 -- 172.21.15.5:0/875906445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a1010d530 msgr2=0x7f2a1019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a161a9640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a1010d530 0x7f2a1019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a161a9640 1 -- 172.21.15.5:0/875906445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a10108e00 msgr2=0x7f2a101a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a161a9640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a10108e00 0x7f2a101a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a161a9640 1 -- 172.21.15.5:0/875906445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1019efd0 con 0x7f2a10108450 2024-09-15T22:15:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a169aa640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a1010d530 0x7f2a1019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a159a8640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a10108e00 0x7f2a101a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a161a9640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a10108450 0x7f2a101a46b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c004770 tx=0x7f2a0c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.897+0000 7f2a077fe640 1 -- 172.21.15.5:0/875906445 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c03f070 con 0x7f2a10108450 2024-09-15T22:15:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.898+0000 7f2a1840d640 1 -- 172.21.15.5:0/875906445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1019f260 con 0x7f2a10108450 2024-09-15T22:15:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.898+0000 7f2a077fe640 1 -- 172.21.15.5:0/875906445 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0c031ca0 con 0x7f2a10108450 2024-09-15T22:15:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.898+0000 7f2a077fe640 1 -- 172.21.15.5:0/875906445 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c035030 con 0x7f2a10108450 2024-09-15T22:15:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.898+0000 7f2a1840d640 1 -- 172.21.15.5:0/875906445 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1006bc10 con 0x7f2a10108450 2024-09-15T22:15:57.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.903+0000 7f2a077fe640 1 -- 172.21.15.5:0/875906445 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2a0c042070 con 0x7f2a10108450 2024-09-15T22:15:57.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.903+0000 7f2a1a7e6740 1 Processor -- start 2024-09-15T22:15:57.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a1a7e6740 1 -- start start 2024-09-15T22:15:57.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a3b40 0x5630f52a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 0x5630f52a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630f52082f0 con 0x5630f52a25d0 2024-09-15T22:15:57.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5630f51afcf0 con 0x5630f52ad370 2024-09-15T22:15:57.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5630f52042b0 con 0x5630f52a3b40 2024-09-15T22:15:57.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a161a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 0x5630f52a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a169aa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a3b40 0x5630f52a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a169aa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a3b40 0x5630f52a2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:51086/0 (socket says 172.21.15.5:51086) 2024-09-15T22:15:57.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a169aa640 1 -- 172.21.15.5:0/1480231822 learned_addr learned my addr 172.21.15.5:0/1480231822 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:57.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.904+0000 7f2a159a8640 1 --2- 172.21.15.5:0/1480231822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a159a8640 1 -- 172.21.15.5:0/1480231822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a3b40 msgr2=0x5630f52a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:57.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a159a8640 1 --2- 172.21.15.5:0/1480231822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a3b40 0x5630f52a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a159a8640 1 -- 172.21.15.5:0/1480231822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 msgr2=0x5630f52a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a159a8640 1 --2- 172.21.15.5:0/1480231822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 0x5630f52a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a159a8640 1 -- 172.21.15.5:0/1480231822 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f51943a0 con 0x5630f52ad370 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a161a9640 1 --2- 172.21.15.5:0/1480231822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 0x5630f52a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a077fe640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f29fc0a33b0 0x7f29fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a159a8640 1 --2- 172.21.15.5:0/1480231822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52af760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a1019fb80 tx=0x7f2a0000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a284400426ff410c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a169aa640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f29fc0a33b0 0x7f29fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a04dea640 1 -- 172.21.15.5:0/1480231822 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00011700 con 0x5630f52ad370 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a077fe640 1 -- 172.21.15.5:0/875906445 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2a0c0f07c0 con 0x7f2a10108450 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a04dea640 1 -- 172.21.15.5:0/1480231822 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a00011d80 con 0x5630f52ad370 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.905+0000 7f2a04dea640 1 -- 172.21.15.5:0/1480231822 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0001a380 con 0x5630f52ad370 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.906+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/1480231822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 msgr2=0x5630f52af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:57.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.906+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/1480231822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52af760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a1019fb80 tx=0x7f2a0000bf90 comp rx=0 tx=0).stop 2024-09-15T22:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.906+0000 7f2a169aa640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f29fc0a33b0 0x7f29fc0a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a080129e0 tx=0x7f2a08009500 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.906+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/1480231822 shutdown_connections 2024-09-15T22:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.906+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/1480231822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52af760 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.906+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/1480231822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 0x5630f52a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.906+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/1480231822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a3b40 0x5630f52a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.906+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/1480231822 >> 172.21.15.5:0/1480231822 conn(0x5630f526b310 msgr2=0x5630f529ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.906+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/1480231822 shutdown_connections 2024-09-15T22:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.906+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/1480231822 wait complete. 2024-09-15T22:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.907+0000 7f2a1a7e6740 1 -- start start 2024-09-15T22:15:57.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.907+0000 7f2a1a7e6740 1 Processor -- start 2024-09-15T22:15:57.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.907+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a25d0 0x5630f52acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.907+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a3b40 0x5630f52a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.907+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52aa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.907+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630f52b0110 con 0x5630f52a3b40 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.907+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5630f51c3060 con 0x5630f52ad370 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.907+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5630f52b0290 con 0x5630f52a25d0 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.907+0000 7f2a1a7e6740 10 client.0 _ll_register_callbacks cb 0x5630f4f168c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.908+0000 7f2a161a9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a25d0 0x5630f52acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.908+0000 7f2a161a9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a25d0 0x5630f52acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:51096/0 (socket says 172.21.15.5:51096) 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.908+0000 7f2a161a9640 1 -- 172.21.15.5:0/2825733975 learned_addr learned my addr 172.21.15.5:0/2825733975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.908+0000 7f2a159a8640 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a3b40 0x5630f52a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.908+0000 7f2a169aa640 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.908+0000 7f2a169aa640 1 -- 172.21.15.5:0/2825733975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a25d0 msgr2=0x5630f52acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.909+0000 7f2a169aa640 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a25d0 0x5630f52acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.909+0000 7f2a169aa640 1 -- 172.21.15.5:0/2825733975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a3b40 msgr2=0x5630f52a9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.909+0000 7f2a169aa640 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a3b40 0x5630f52a9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.909+0000 7f2a169aa640 1 -- 172.21.15.5:0/2825733975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630f4f793c0 con 0x5630f52ad370 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.909+0000 7f2a159a8640 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a3b40 0x5630f52a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.909+0000 7f2a161a9640 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a25d0 0x5630f52acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.909+0000 7f2a169aa640 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52aa560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a08005fd0 tx=0x7f2a08004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9b7855f3888fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.909+0000 7f29f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.909+0000 7f29f9ffb640 1 -- 172.21.15.5:0/2825733975 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a08003c40 con 0x5630f52ad370 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.909+0000 7f29f9ffb640 1 -- 172.21.15.5:0/2825733975 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2a08003de0 con 0x5630f52ad370 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.909+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/2825733975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a00011d80 con 0x5630f52ad370 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f29f9ffb640 1 -- 172.21.15.5:0/2825733975 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a08011910 con 0x5630f52ad370 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f2a1a7e6740 10 client.26252 fetch_fsmap learned FSMap version 25 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/2825733975 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630f52bbbc0 con 0x5630f52ad370 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f29f9ffb640 1 -- 172.21.15.5:0/2825733975 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a00011d80 con 0x5630f52ad370 2024-09-15T22:15:57.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f29f9ffb640 1 -- 172.21.15.5:0/2825733975 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2a08027020 con 0x5630f52ad370 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f2a1a7e6740 10 client.26252 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f2a1a7e6740 10 client.26252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f2a1a7e6740 20 client.26252 populate_metadata read hostname 'smithi005' 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630f52bd080 0x5630f52bf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f2a1a7e6740 4 client.26252 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/2825733975 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630f5215860 con 0x5630f52bd080 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.910+0000 7f2a161a9640 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630f52bd080 0x5630f52bf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.911+0000 7f2a161a9640 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630f52bd080 0x5630f52bf4d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17feb60ec6caa79a server_cookie=9913a35ccc483872 in_seq=0 out_seq=0 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.911+0000 7f29f9ffb640 10 client.26252 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.911+0000 7f29f9ffb640 1 -- 172.21.15.5:0/2825733975 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630f5215860 con 0x5630f52bd080 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.912+0000 7f29f9ffb640 10 client.26252 handle_command_reply: tid=0 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.912+0000 7f2a1a7e6740 1 client.26252 shutdown 2024-09-15T22:15:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.912+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/2825733975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 msgr2=0x5630f52aa560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.912+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52aa560 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a08005fd0 tx=0x7f2a08004ab0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.915+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/2825733975 shutdown_connections 2024-09-15T22:15:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.915+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630f52bd080 0x5630f52bf4d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.915+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52aa560 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.915+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a3b40 0x5630f52a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.915+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/2825733975 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a25d0 0x5630f52acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.915+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/2825733975 >> 172.21.15.5:0/2825733975 conn(0x5630f526b310 msgr2=0x5630f52a4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.915+0000 7f29f9ffb640 0 client.26252 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.915+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/2825733975 shutdown_connections 2024-09-15T22:15:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.915+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/2825733975 wait complete. 2024-09-15T22:15:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.915+0000 7f2a1a7e6740 20 client.26252 trim_cache size 0 max 16384 2024-09-15T22:15:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.934+0000 7f2a1a7e6740 1 Processor -- start 2024-09-15T22:15:57.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.935+0000 7f2a1a7e6740 1 -- start start 2024-09-15T22:15:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.935+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 0x5630f52abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.935+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a3b40 0x5630f52a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.935+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.935+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101c0740 con 0x5630f52a25d0 2024-09-15T22:15:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.935+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101be010 con 0x5630f52ad370 2024-09-15T22:15:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.935+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a10073d60 con 0x5630f52a3b40 2024-09-15T22:15:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.935+0000 7f2a161a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 0x5630f52abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.935+0000 7f2a161a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 0x5630f52abf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59128/0 (socket says 172.21.15.5:59128) 2024-09-15T22:15:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.935+0000 7f2a161a9640 1 -- 172.21.15.5:0/1140204478 learned_addr learned my addr 172.21.15.5:0/1140204478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.936+0000 7f2a169aa640 1 --2- 172.21.15.5:0/1140204478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.936+0000 7f2a161a9640 1 -- 172.21.15.5:0/1140204478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a3b40 msgr2=0x5630f52a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.936+0000 7f2a161a9640 1 --2- 172.21.15.5:0/1140204478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a3b40 0x5630f52a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.936+0000 7f2a161a9640 1 -- 172.21.15.5:0/1140204478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 msgr2=0x5630f52a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.936+0000 7f2a161a9640 1 --2- 172.21.15.5:0/1140204478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.936+0000 7f2a161a9640 1 -- 172.21.15.5:0/1140204478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f529f960 con 0x5630f52a25d0 2024-09-15T22:15:57.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.936+0000 7f2a161a9640 1 --2- 172.21.15.5:0/1140204478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 0x5630f52abf30 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0b8bb0 tx=0x7f2a0c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f508d0ec579e24c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:57.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.936+0000 7f29fbfff640 1 -- 172.21.15.5:0/1140204478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c049040 con 0x5630f52a25d0 2024-09-15T22:15:57.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.936+0000 7f29fbfff640 1 -- 172.21.15.5:0/1140204478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0c0c0a50 con 0x5630f52a25d0 2024-09-15T22:15:57.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.936+0000 7f29fbfff640 1 -- 172.21.15.5:0/1140204478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0edd90 con 0x5630f52a25d0 2024-09-15T22:15:57.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.937+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/1140204478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 msgr2=0x5630f52abf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:57.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.937+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/1140204478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 0x5630f52abf30 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0b8bb0 tx=0x7f2a0c009260 comp rx=0 tx=0).stop 2024-09-15T22:15:57.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.945+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/1140204478 shutdown_connections 2024-09-15T22:15:57.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.945+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/1140204478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52ad370 0x5630f52a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.945+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/1140204478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52a3b40 0x5630f52a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.945+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/1140204478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a25d0 0x5630f52abf30 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.945+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/1140204478 >> 172.21.15.5:0/1140204478 conn(0x5630f526b310 msgr2=0x5630f52ab820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:57.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.952+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/1140204478 shutdown_connections 2024-09-15T22:15:57.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.952+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/1140204478 wait complete. 2024-09-15T22:15:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.953+0000 7f2a1a7e6740 1 -- start start 2024-09-15T22:15:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.953+0000 7f2a1a7e6740 1 Processor -- start 2024-09-15T22:15:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.953+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52a25d0 0x5630f52b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.953+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a3b40 0x5630f52b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.954+0000 7f2a1a7e6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52ad370 0x5630f52b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.954+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101bc520 con 0x5630f52a3b40 2024-09-15T22:15:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.954+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101bd940 con 0x5630f52a25d0 2024-09-15T22:15:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.954+0000 7f2a1a7e6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2a101bcba0 con 0x5630f52ad370 2024-09-15T22:15:57.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.954+0000 7f2a1a7e6740 10 client.0 _ll_register_callbacks cb 0x7f2a080126f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:15:57.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.954+0000 7f2a169aa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52ad370 0x5630f52b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.954+0000 7f2a161a9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52a25d0 0x5630f52b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.954+0000 7f2a161a9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52a25d0 0x5630f52b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:41944/0 (socket says 172.21.15.5:41944) 2024-09-15T22:15:57.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.954+0000 7f2a161a9640 1 -- 172.21.15.5:0/3651997194 learned_addr learned my addr 172.21.15.5:0/3651997194 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:15:57.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.957+0000 7f2a159a8640 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a3b40 0x5630f52b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.958+0000 7f2a161a9640 1 -- 172.21.15.5:0/3651997194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52ad370 msgr2=0x5630f52b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:57.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.958+0000 7f2a161a9640 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52ad370 0x5630f52b24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.958+0000 7f2a161a9640 1 -- 172.21.15.5:0/3651997194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a3b40 msgr2=0x5630f52b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:15:57.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.958+0000 7f2a161a9640 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a3b40 0x5630f52b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.958+0000 7f2a161a9640 1 -- 172.21.15.5:0/3651997194 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630f52b2a30 con 0x5630f52a25d0 2024-09-15T22:15:57.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.961+0000 7f2a161a9640 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52a25d0 0x5630f52b5520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0c4de0 tx=0x7f2a0c0baa50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b95502ca131fb05 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:15:57.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.961+0000 7f29f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:15:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.965+0000 7f29f9ffb640 1 -- 172.21.15.5:0/3651997194 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0d5060 con 0x5630f52a25d0 2024-09-15T22:15:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.965+0000 7f29f9ffb640 1 -- 172.21.15.5:0/3651997194 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f2a0c03b4f0 con 0x5630f52a25d0 2024-09-15T22:15:57.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.966+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/3651997194 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a0c0c0a50 con 0x5630f52a25d0 2024-09-15T22:15:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.969+0000 7f29f9ffb640 1 -- 172.21.15.5:0/3651997194 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2a0c0da050 con 0x5630f52a25d0 2024-09-15T22:15:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.970+0000 7f2a1a7e6740 10 client.26258 fetch_fsmap learned FSMap version 25 2024-09-15T22:15:57.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.970+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/3651997194 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630f5328490 con 0x5630f52a25d0 2024-09-15T22:15:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.978+0000 7f29f9ffb640 1 -- 172.21.15.5:0/3651997194 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a0c0c0a50 con 0x5630f52a25d0 2024-09-15T22:15:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.978+0000 7f29f9ffb640 1 -- 172.21.15.5:0/3651997194 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2a0c0bcac0 con 0x5630f52a25d0 2024-09-15T22:15:57.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.979+0000 7f2a1a7e6740 10 client.26258 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:15:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.979+0000 7f2a1a7e6740 10 client.26258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:15:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.979+0000 7f2a1a7e6740 20 client.26258 populate_metadata read hostname 'smithi005' 2024-09-15T22:15:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.979+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630f5328a60 0x5630f532ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:15:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.979+0000 7f2a1a7e6740 4 client.26258 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:15:57.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.979+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/3651997194 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630f520fd70 con 0x5630f5328a60 2024-09-15T22:15:57.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.979+0000 7f2a159a8640 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630f5328a60 0x5630f532ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:15:57.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.986+0000 7f2a159a8640 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630f5328a60 0x5630f532ae50 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=365bd82fb62dc8ef server_cookie=fec9563f2d594457 in_seq=0 out_seq=0 2024-09-15T22:15:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.988+0000 7f29f9ffb640 10 client.26258 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.988+0000 7f29f9ffb640 1 -- 172.21.15.5:0/3651997194 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630f520fd70 con 0x5630f5328a60 2024-09-15T22:15:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.988+0000 7f29f9ffb640 10 client.26258 handle_command_reply: tid=0 2024-09-15T22:15:57.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.988+0000 7f2a1a7e6740 1 client.26258 shutdown 2024-09-15T22:15:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.989+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/3651997194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52a25d0 msgr2=0x5630f52b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.989+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52a25d0 0x5630f52b5520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0c4de0 tx=0x7f2a0c0baa50 comp rx=0 tx=0).stop 2024-09-15T22:15:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.991+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/3651997194 shutdown_connections 2024-09-15T22:15:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.991+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630f52ad370 0x5630f52b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.991+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630f52a3b40 0x5630f52b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.991+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630f5328a60 0x5630f532ae50 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.991+0000 7f2a1a7e6740 1 --2- 172.21.15.5:0/3651997194 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630f52a25d0 0x5630f52b5520 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.991+0000 7f29f9ffb640 0 client.26258 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:15:57.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.991+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/3651997194 >> 172.21.15.5:0/3651997194 conn(0x5630f526b310 msgr2=0x5630f52ac040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.994+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/3651997194 shutdown_connections 2024-09-15T22:15:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.994+0000 7f2a1a7e6740 1 -- 172.21.15.5:0/3651997194 wait complete. 2024-09-15T22:15:57.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.994+0000 7f2a1a7e6740 20 client.26258 trim_cache size 0 max 16384 2024-09-15T22:15:58.000 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:15:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.998+0000 7f2a1840d640 1 -- 172.21.15.5:0/875906445 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f29fc0a33b0 msgr2=0x7f29fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.998+0000 7f2a1840d640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f29fc0a33b0 0x7f29fc0a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a080129e0 tx=0x7f2a08009500 comp rx=0 tx=0).stop 2024-09-15T22:15:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.999+0000 7f2a1840d640 1 -- 172.21.15.5:0/875906445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a10108450 msgr2=0x7f2a101a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:15:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.999+0000 7f2a1840d640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a10108450 0x7f2a101a46b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c004770 tx=0x7f2a0c003c10 comp rx=0 tx=0).stop 2024-09-15T22:15:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.999+0000 7f2a1840d640 1 -- 172.21.15.5:0/875906445 shutdown_connections 2024-09-15T22:15:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.999+0000 7f2a1840d640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f29fc0a33b0 0x7f29fc0a5870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.999+0000 7f2a1840d640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2a1010d530 0x7f2a1019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.999+0000 7f2a1840d640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a10108e00 0x7f2a101a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.999+0000 7f2a1840d640 1 --2- 172.21.15.5:0/875906445 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2a10108450 0x7f2a101a46b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:15:58.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.999+0000 7f2a1840d640 1 -- 172.21.15.5:0/875906445 >> 172.21.15.5:0/875906445 conn(0x7f2a10077ba0 msgr2=0x7f2a10078e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:15:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.999+0000 7f2a1840d640 1 -- 172.21.15.5:0/875906445 shutdown_connections 2024-09-15T22:15:58.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:15:57.999+0000 7f2a1840d640 1 -- 172.21.15.5:0/875906445 wait complete. 2024-09-15T22:15:59.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:15:59 smithi099 ceph-mon[34892]: pgmap v215: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 845 op/s 2024-09-15T22:15:59.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:15:59 smithi129 ceph-mon[35314]: pgmap v215: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 845 op/s 2024-09-15T22:15:59.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:15:59 smithi005 ceph-mon[30944]: pgmap v215: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 11 MiB/s wr, 845 op/s 2024-09-15T22:15:59.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 390.0 s, 843.0 tps, lat 123.851 ms stddev 183.408 2024-09-15T22:16:01.472 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:16:01.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:16:01.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:01 smithi099 ceph-mon[34892]: pgmap v216: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.0 MiB/s wr, 637 op/s 2024-09-15T22:16:01.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:01 smithi129 ceph-mon[35314]: pgmap v216: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.0 MiB/s wr, 637 op/s 2024-09-15T22:16:01.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:01 smithi005 ceph-mon[30944]: pgmap v216: 129 pgs: 129 active+clean; 3.4 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 8.0 MiB/s wr, 637 op/s 2024-09-15T22:16:02.098 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:03 smithi099 ceph-mon[34892]: pgmap v217: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 11 MiB/s wr, 930 op/s 2024-09-15T22:16:03.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:03 smithi129 ceph-mon[35314]: pgmap v217: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 11 MiB/s wr, 930 op/s 2024-09-15T22:16:03.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:03 smithi005 ceph-mon[30944]: pgmap v217: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 11 MiB/s wr, 930 op/s 2024-09-15T22:16:04.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:04 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:16:04.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:04 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:04.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:04 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:16:04.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:04 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:16:04.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:04 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:04.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:04 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:16:04.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:04 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:16:04.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:04 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:04.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:04 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:16:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 395.0 s, 734.3 tps, lat 136.084 ms stddev 224.470 2024-09-15T22:16:04.839 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:16:04.839 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:16:04.839 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "a0a613df-6b35-41d7-889b-ed709c0a9fb5", 2024-09-15T22:16:04.839 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:16:04.839 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:16:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.875+0000 7f6899077640 1 -- 172.21.15.5:0/410423949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6894102570 msgr2=0x7f68941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.875+0000 7f6899077640 1 --2- 172.21.15.5:0/410423949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6894102570 0x7f68941029b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f687c009900 tx=0x7f687c02a480 comp rx=0 tx=0).stop 2024-09-15T22:16:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.875+0000 7f6899077640 1 -- 172.21.15.5:0/410423949 shutdown_connections 2024-09-15T22:16:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.875+0000 7f6899077640 1 --2- 172.21.15.5:0/410423949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6894102ef0 0x7f689410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.875+0000 7f6899077640 1 --2- 172.21.15.5:0/410423949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6894102570 0x7f68941029b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.875+0000 7f6899077640 1 --2- 172.21.15.5:0/410423949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6894100e20 0x7f6894101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.876+0000 7f6899077640 1 -- 172.21.15.5:0/410423949 >> 172.21.15.5:0/410423949 conn(0x7f68940fc7d0 msgr2=0x7f68940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.876+0000 7f6899077640 1 -- 172.21.15.5:0/410423949 shutdown_connections 2024-09-15T22:16:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.876+0000 7f6899077640 1 -- 172.21.15.5:0/410423949 wait complete. 2024-09-15T22:16:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.876+0000 7f6899077640 1 Processor -- start 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.876+0000 7f6899077640 1 -- start start 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f6899077640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6894100e20 0x7f68941993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f6899077640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6894102570 0x7f6894199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f6899077640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6894102ef0 0x7f68941a4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f6899077640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6894110f80 con 0x7f6894102570 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f6899077640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6894110e00 con 0x7f6894102ef0 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f6899077640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6894111100 con 0x7f6894100e20 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f688a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6894102570 0x7f6894199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f688a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6894102570 0x7f6894199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37022/0 (socket says 172.21.15.5:37022) 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f688a59c640 1 -- 172.21.15.5:0/4174183791 learned_addr learned my addr 172.21.15.5:0/4174183791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f6892d9d640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6894100e20 0x7f68941993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f689359e640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6894102ef0 0x7f68941a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f688a59c640 1 -- 172.21.15.5:0/4174183791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6894100e20 msgr2=0x7f68941993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f688a59c640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6894100e20 0x7f68941993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f688a59c640 1 -- 172.21.15.5:0/4174183791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6894102ef0 msgr2=0x7f68941a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f688a59c640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6894102ef0 0x7f68941a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.877+0000 7f688a59c640 1 -- 172.21.15.5:0/4174183791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68941a4980 con 0x7f6894102570 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.878+0000 7f688a59c640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6894102570 0x7f6894199900 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f687c006010 tx=0x7f687c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.878+0000 7f6890d99640 1 -- 172.21.15.5:0/4174183791 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c03f070 con 0x7f6894102570 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.878+0000 7f6899077640 1 -- 172.21.15.5:0/4174183791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68941a4b50 con 0x7f6894102570 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.878+0000 7f6890d99640 1 -- 172.21.15.5:0/4174183791 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687c031c30 con 0x7f6894102570 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.878+0000 7f6890d99640 1 -- 172.21.15.5:0/4174183791 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c035030 con 0x7f6894102570 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.878+0000 7f6899077640 1 -- 172.21.15.5:0/4174183791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68941a5000 con 0x7f6894102570 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.880+0000 7f6890d99640 1 -- 172.21.15.5:0/4174183791 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f687c042070 con 0x7f6894102570 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.880+0000 7f689b450740 1 Processor -- start 2024-09-15T22:16:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f689b450740 1 -- start start 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f689b450740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79edb60 0x55d3e79edf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f689b450740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79ee480 0x55d3e79f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f689b450740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79f8d40 0x55d3e79fb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f689b450740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d3e7953280 con 0x55d3e79f8d40 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f689b450740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d3e78fab10 con 0x55d3e79ee480 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f689b450740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d3e794f240 con 0x55d3e79edb60 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f688a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79f8d40 0x55d3e79fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f6892d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79ee480 0x55d3e79f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f688a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79f8d40 0x55d3e79fb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37026/0 (socket says 172.21.15.5:37026) 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f688a59c640 1 -- 172.21.15.5:0/3332107835 learned_addr learned my addr 172.21.15.5:0/3332107835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f6892d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79ee480 0x55d3e79f8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:47694/0 (socket says 172.21.15.5:47694) 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.881+0000 7f689359e640 1 --2- 172.21.15.5:0/3332107835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79edb60 0x55d3e79edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f6892d9d640 1 -- 172.21.15.5:0/3332107835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79edb60 msgr2=0x55d3e79edf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f6892d9d640 1 --2- 172.21.15.5:0/3332107835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79edb60 0x55d3e79edf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f6892d9d640 1 -- 172.21.15.5:0/3332107835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79f8d40 msgr2=0x55d3e79fb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f6892d9d640 1 --2- 172.21.15.5:0/3332107835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79f8d40 0x55d3e79fb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f6892d9d640 1 -- 172.21.15.5:0/3332107835 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e79071f0 con 0x55d3e79ee480 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f6890d99640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68800a33b0 0x7f68800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f689359e640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68800a33b0 0x7f68800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f6892d9d640 1 --2- 172.21.15.5:0/3332107835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79ee480 0x55d3e79f8800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6884009c20 tx=0x7f6884011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e72f143e6afb94d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f6890d99640 1 -- 172.21.15.5:0/4174183791 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f687c0f0780 con 0x7f6894102570 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f688959a640 1 -- 172.21.15.5:0/3332107835 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884004050 con 0x55d3e79ee480 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f688959a640 1 -- 172.21.15.5:0/3332107835 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6884005500 con 0x55d3e79ee480 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.882+0000 7f688959a640 1 -- 172.21.15.5:0/3332107835 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884004e40 con 0x55d3e79ee480 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689b450740 1 -- 172.21.15.5:0/3332107835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79ee480 msgr2=0x55d3e79f8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689b450740 1 --2- 172.21.15.5:0/3332107835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79ee480 0x55d3e79f8800 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6884009c20 tx=0x7f6884011810 comp rx=0 tx=0).stop 2024-09-15T22:16:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689359e640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68800a33b0 0x7f68800a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f688c005fd0 tx=0x7f688c0096a0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689b450740 1 -- 172.21.15.5:0/3332107835 shutdown_connections 2024-09-15T22:16:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689b450740 1 --2- 172.21.15.5:0/3332107835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79f8d40 0x55d3e79fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689b450740 1 --2- 172.21.15.5:0/3332107835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79ee480 0x55d3e79f8800 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689b450740 1 --2- 172.21.15.5:0/3332107835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79edb60 0x55d3e79edf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689b450740 1 -- 172.21.15.5:0/3332107835 >> 172.21.15.5:0/3332107835 conn(0x55d3e79ea0f0 msgr2=0x55d3e79ea4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689b450740 1 -- 172.21.15.5:0/3332107835 shutdown_connections 2024-09-15T22:16:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689b450740 1 -- 172.21.15.5:0/3332107835 wait complete. 2024-09-15T22:16:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689b450740 1 -- start start 2024-09-15T22:16:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.883+0000 7f689b450740 1 Processor -- start 2024-09-15T22:16:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f689b450740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79edb60 0x55d3e79f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f689b450740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e79f6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f689b450740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 0x55d3e79f6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f689b450740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d3e79fb910 con 0x55d3e79ee480 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f689b450740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d3e790df10 con 0x55d3e79edb60 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f689b450740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d3e79fba90 con 0x55d3e79f8d40 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f689b450740 10 client.0 _ll_register_callbacks cb 0x55d3e78d6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f688a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e79f6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f6892d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79edb60 0x55d3e79f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f688a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e79f6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37036/0 (socket says 172.21.15.5:37036) 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f6892d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79edb60 0x55d3e79f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:47700/0 (socket says 172.21.15.5:47700) 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f688a59c640 1 -- 172.21.15.5:0/3477178331 learned_addr learned my addr 172.21.15.5:0/3477178331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f689359e640 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 0x55d3e79f6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f688a59c640 1 -- 172.21.15.5:0/3477178331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 msgr2=0x55d3e79f6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f688a59c640 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 0x55d3e79f6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f688a59c640 1 -- 172.21.15.5:0/3477178331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79edb60 msgr2=0x55d3e79f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f688a59c640 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79edb60 0x55d3e79f5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f688a59c640 1 -- 172.21.15.5:0/3477178331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e76c46a0 con 0x55d3e79ee480 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f6892d9d640 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79edb60 0x55d3e79f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f688a59c640 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e79f6100 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f687c0f62d0 tx=0x7f687c0b8370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d0a95c42fec155 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f687a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.884+0000 7f687a7fc640 1 -- 172.21.15.5:0/3477178331 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c049070 con 0x55d3e79ee480 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f687a7fc640 1 -- 172.21.15.5:0/3477178331 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f687c0b8d80 con 0x55d3e79ee480 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f689b450740 1 -- 172.21.15.5:0/3477178331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6884005500 con 0x55d3e79ee480 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f687a7fc640 1 -- 172.21.15.5:0/3477178331 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f687c0bc4e0 con 0x55d3e79ee480 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f689b450740 10 client.16710 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f689b450740 1 -- 172.21.15.5:0/3477178331 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d3e7a074f0 con 0x55d3e79ee480 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f687a7fc640 1 -- 172.21.15.5:0/3477178331 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6884005500 con 0x55d3e79ee480 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f687a7fc640 1 -- 172.21.15.5:0/3477178331 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f687c0edd50 con 0x55d3e79ee480 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f689b450740 10 client.16710 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f689b450740 10 client.16710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f689b450740 20 client.16710 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f689b450740 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d3e7a08950 0x55d3e7a0ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f689b450740 4 client.16710 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f689b450740 1 -- 172.21.15.5:0/3477178331 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3e79607f0 con 0x55d3e7a08950 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.885+0000 7f6892d9d640 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d3e7a08950 0x55d3e7a0ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.887+0000 7f6892d9d640 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d3e7a08950 0x55d3e7a0ade0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7520295fd079d623 server_cookie=a7819197b8f9a43 in_seq=0 out_seq=0 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.887+0000 7f687a7fc640 10 client.16710 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.892+0000 7f687a7fc640 1 -- 172.21.15.5:0/3477178331 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3e79607f0 con 0x55d3e7a08950 2024-09-15T22:16:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.892+0000 7f687a7fc640 10 client.16710 handle_command_reply: tid=0 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.895+0000 7f689b450740 1 client.16710 shutdown 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.896+0000 7f689b450740 1 -- 172.21.15.5:0/3477178331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 msgr2=0x55d3e79f6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.896+0000 7f689b450740 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e79f6100 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f687c0f62d0 tx=0x7f687c0b8370 comp rx=0 tx=0).stop 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.897+0000 7f689b450740 1 -- 172.21.15.5:0/3477178331 shutdown_connections 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.897+0000 7f689b450740 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 0x55d3e79f6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.897+0000 7f689b450740 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e79f6100 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.897+0000 7f689b450740 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d3e7a08950 0x55d3e7a0ade0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.897+0000 7f687a7fc640 0 client.16710 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.897+0000 7f689b450740 1 --2- 172.21.15.5:0/3477178331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79edb60 0x55d3e79f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.897+0000 7f689b450740 1 -- 172.21.15.5:0/3477178331 >> 172.21.15.5:0/3477178331 conn(0x55d3e79ea0f0 msgr2=0x55d3e79efdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.897+0000 7f689b450740 1 -- 172.21.15.5:0/3477178331 shutdown_connections 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.897+0000 7f689b450740 1 -- 172.21.15.5:0/3477178331 wait complete. 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.897+0000 7f689b450740 20 client.16710 trim_cache size 0 max 16384 2024-09-15T22:16:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.926+0000 7f689b450740 1 Processor -- start 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.926+0000 7f689b450740 1 -- start start 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.927+0000 7f689b450740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79edb60 0x55d3e79f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.927+0000 7f689b450740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e7a010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.927+0000 7f689b450740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79f8d40 0x55d3e7a01630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.927+0000 7f689b450740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68941b1980 con 0x55d3e79ee480 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.927+0000 7f689b450740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f68941af250 con 0x55d3e79f8d40 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.927+0000 7f689b450740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f68941ad0a0 con 0x55d3e79edb60 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.928+0000 7f688a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e7a010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.928+0000 7f688a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e7a010f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37046/0 (socket says 172.21.15.5:37046) 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.928+0000 7f688a59c640 1 -- 172.21.15.5:0/3941933105 learned_addr learned my addr 172.21.15.5:0/3941933105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.928+0000 7f6892d9d640 1 --2- 172.21.15.5:0/3941933105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79edb60 0x55d3e79f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.928+0000 7f689359e640 1 --2- 172.21.15.5:0/3941933105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79f8d40 0x55d3e7a01630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.929+0000 7f688a59c640 1 -- 172.21.15.5:0/3941933105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79edb60 msgr2=0x55d3e79f7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.929+0000 7f688a59c640 1 --2- 172.21.15.5:0/3941933105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79edb60 0x55d3e79f7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.929+0000 7f688a59c640 1 -- 172.21.15.5:0/3941933105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79f8d40 msgr2=0x55d3e7a01630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.929+0000 7f688a59c640 1 --2- 172.21.15.5:0/3941933105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79f8d40 0x55d3e7a01630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.929+0000 7f688a59c640 1 -- 172.21.15.5:0/3941933105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e79e9f20 con 0x55d3e79ee480 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.929+0000 7f688a59c640 1 --2- 172.21.15.5:0/3941933105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e7a010f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f687c0c4da0 tx=0x7f687c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d274e502a84456 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.930+0000 7f6888d99640 1 -- 172.21.15.5:0/3941933105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c0da070 con 0x55d3e79ee480 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.930+0000 7f6888d99640 1 -- 172.21.15.5:0/3941933105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687c0bc710 con 0x55d3e79ee480 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.931+0000 7f6888d99640 1 -- 172.21.15.5:0/3941933105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c049070 con 0x55d3e79ee480 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.931+0000 7f689b450740 1 -- 172.21.15.5:0/3941933105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 msgr2=0x55d3e7a010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.931+0000 7f689b450740 1 --2- 172.21.15.5:0/3941933105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e7a010f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f687c0c4da0 tx=0x7f687c02a480 comp rx=0 tx=0).stop 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.939+0000 7f689b450740 1 -- 172.21.15.5:0/3941933105 shutdown_connections 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.939+0000 7f689b450740 1 --2- 172.21.15.5:0/3941933105 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79f8d40 0x55d3e7a01630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.939+0000 7f689b450740 1 --2- 172.21.15.5:0/3941933105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e7a010f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.939+0000 7f689b450740 1 --2- 172.21.15.5:0/3941933105 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79edb60 0x55d3e79f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.939+0000 7f689b450740 1 -- 172.21.15.5:0/3941933105 >> 172.21.15.5:0/3941933105 conn(0x55d3e79ea0f0 msgr2=0x55d3e79f7310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.945+0000 7f689b450740 1 -- 172.21.15.5:0/3941933105 shutdown_connections 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.945+0000 7f689b450740 1 -- 172.21.15.5:0/3941933105 wait complete. 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.945+0000 7f689b450740 1 -- start start 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.945+0000 7f689b450740 1 Processor -- start 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.947+0000 7f689b450740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79edb60 0x55d3e7a00ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.947+0000 7f689b450740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e7a01430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.947+0000 7f689b450740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 0x55d3e79fdce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.947+0000 7f689b450740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68941ad710 con 0x55d3e79ee480 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.948+0000 7f689b450740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f68941aeb80 con 0x55d3e79edb60 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.948+0000 7f689b450740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f68941adde0 con 0x55d3e79f8d40 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.948+0000 7f689b450740 10 client.0 _ll_register_callbacks cb 0x7f687c0f6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.948+0000 7f689359e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 0x55d3e79fdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.948+0000 7f689359e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 0x55d3e79fdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50550/0 (socket says 172.21.15.5:50550) 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.948+0000 7f689359e640 1 -- 172.21.15.5:0/1161172146 learned_addr learned my addr 172.21.15.5:0/1161172146 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.949+0000 7f689359e640 1 -- 172.21.15.5:0/1161172146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79edb60 msgr2=0x55d3e7a00ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.949+0000 7f689359e640 1 --2- 172.21.15.5:0/1161172146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79edb60 0x55d3e7a00ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.949+0000 7f689359e640 1 -- 172.21.15.5:0/1161172146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 msgr2=0x55d3e7a01430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.949+0000 7f689359e640 1 --2- 172.21.15.5:0/1161172146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e7a01430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.949+0000 7f689359e640 1 -- 172.21.15.5:0/1161172146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e79f09d0 con 0x55d3e79f8d40 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.950+0000 7f689359e640 1 --2- 172.21.15.5:0/1161172146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 0x55d3e79fdce0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f688c010a60 tx=0x7f688c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbdbd2086a08718e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.950+0000 7f687a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.950+0000 7f687a7fc640 1 -- 172.21.15.5:0/1161172146 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688c003b80 con 0x55d3e79f8d40 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.950+0000 7f689b450740 1 -- 172.21.15.5:0/1161172146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f687c0bc710 con 0x55d3e79f8d40 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.950+0000 7f687a7fc640 1 -- 172.21.15.5:0/1161172146 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 9453+0+0 (secure 0 0 0) 0x7f688c003d20 con 0x55d3e79f8d40 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.950+0000 7f687a7fc640 1 -- 172.21.15.5:0/1161172146 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f688c02e650 con 0x55d3e79f8d40 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.950+0000 7f689b450740 10 client.26236 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.951+0000 7f689b450740 1 -- 172.21.15.5:0/1161172146 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d3e7a8d750 con 0x55d3e79f8d40 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.952+0000 7f687a7fc640 1 -- 172.21.15.5:0/1161172146 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f687c0bc710 con 0x55d3e79f8d40 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.952+0000 7f687a7fc640 1 -- 172.21.15.5:0/1161172146 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f688c004250 con 0x55d3e79f8d40 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.952+0000 7f689b450740 10 client.26236 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.952+0000 7f689b450740 10 client.26236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.952+0000 7f689b450740 20 client.26236 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.952+0000 7f689b450740 1 --2- 172.21.15.5:0/1161172146 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d3e7a8df30 0x55d3e7a903b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.952+0000 7f689b450740 4 client.26236 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.952+0000 7f689b450740 1 -- 172.21.15.5:0/1161172146 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d3e795ad00 con 0x55d3e7a8df30 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.953+0000 7f6892d9d640 1 --2- 172.21.15.5:0/1161172146 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d3e7a8df30 0x55d3e7a903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.954+0000 7f6892d9d640 1 --2- 172.21.15.5:0/1161172146 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d3e7a8df30 0x55d3e7a903b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c5284b3a9c33935 server_cookie=ca29d7add979c228 in_seq=0 out_seq=0 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.954+0000 7f687a7fc640 10 client.26236 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.956+0000 7f687a7fc640 1 -- 172.21.15.5:0/1161172146 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d3e795ad00 con 0x55d3e7a8df30 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.956+0000 7f687a7fc640 10 client.26236 handle_command_reply: tid=0 2024-09-15T22:16:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.956+0000 7f689b450740 1 client.26236 shutdown 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.961+0000 7f689b450740 1 -- 172.21.15.5:0/1161172146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 msgr2=0x55d3e79fdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.961+0000 7f689b450740 1 --2- 172.21.15.5:0/1161172146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 0x55d3e79fdce0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f688c010a60 tx=0x7f688c004a40 comp rx=0 tx=0).stop 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.963+0000 7f689b450740 1 -- 172.21.15.5:0/1161172146 shutdown_connections 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.963+0000 7f689b450740 1 --2- 172.21.15.5:0/1161172146 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d3e79f8d40 0x55d3e79fdce0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.963+0000 7f689b450740 1 --2- 172.21.15.5:0/1161172146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3e79ee480 0x55d3e7a01430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.963+0000 7f689b450740 1 --2- 172.21.15.5:0/1161172146 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d3e7a8df30 0x55d3e7a903b0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.963+0000 7f689b450740 1 --2- 172.21.15.5:0/1161172146 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d3e79edb60 0x55d3e7a00ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.963+0000 7f689b450740 1 -- 172.21.15.5:0/1161172146 >> 172.21.15.5:0/1161172146 conn(0x55d3e79ea0f0 msgr2=0x55d3e79efdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.963+0000 7f687a7fc640 0 client.26236 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.964+0000 7f689b450740 1 -- 172.21.15.5:0/1161172146 shutdown_connections 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.964+0000 7f689b450740 1 -- 172.21.15.5:0/1161172146 wait complete. 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.965+0000 7f689b450740 20 client.26236 trim_cache size 0 max 16384 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.967+0000 7f6899077640 1 -- 172.21.15.5:0/4174183791 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68800a33b0 msgr2=0x7f68800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.967+0000 7f6899077640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68800a33b0 0x7f68800a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f688c005fd0 tx=0x7f688c0096a0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.967+0000 7f6899077640 1 -- 172.21.15.5:0/4174183791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6894102570 msgr2=0x7f6894199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.967+0000 7f6899077640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6894102570 0x7f6894199900 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f687c006010 tx=0x7f687c003c10 comp rx=0 tx=0).stop 2024-09-15T22:16:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.967+0000 7f6899077640 1 -- 172.21.15.5:0/4174183791 shutdown_connections 2024-09-15T22:16:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.967+0000 7f6899077640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f68800a33b0 0x7f68800a5870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.967+0000 7f6899077640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6894102ef0 0x7f68941a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.967+0000 7f6899077640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6894102570 0x7f6894199900 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.967+0000 7f6899077640 1 --2- 172.21.15.5:0/4174183791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6894100e20 0x7f68941993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.967+0000 7f6899077640 1 -- 172.21.15.5:0/4174183791 >> 172.21.15.5:0/4174183791 conn(0x7f68940fc7d0 msgr2=0x7f68940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.967+0000 7f6899077640 1 -- 172.21.15.5:0/4174183791 shutdown_connections 2024-09-15T22:16:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:03.968+0000 7f6899077640 1 -- 172.21.15.5:0/4174183791 wait complete. 2024-09-15T22:16:05.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:05 smithi099 ceph-mon[34892]: pgmap v218: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 9.5 MiB/s wr, 789 op/s 2024-09-15T22:16:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:05 smithi129 ceph-mon[35314]: pgmap v218: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 9.5 MiB/s wr, 789 op/s 2024-09-15T22:16:05.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:05 smithi005 ceph-mon[30944]: pgmap v218: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 9.5 MiB/s wr, 789 op/s 2024-09-15T22:16:06.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:16:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:07.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:07 smithi005 ceph-mon[30944]: pgmap v219: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-15T22:16:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:07 smithi099 ceph-mon[34892]: pgmap v219: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-15T22:16:07.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:07 smithi129 ceph-mon[35314]: pgmap v219: 129 pgs: 129 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-15T22:16:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.924+0000 7f3a90e48640 1 -- 172.21.15.5:0/3189964835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c1027a0 msgr2=0x7f3a8c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:07.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.924+0000 7f3a90e48640 1 --2- 172.21.15.5:0/3189964835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c1027a0 0x7f3a8c1067f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009900 tx=0x7f3a7c02a440 comp rx=0 tx=0).stop 2024-09-15T22:16:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.926+0000 7f3a90e48640 1 -- 172.21.15.5:0/3189964835 shutdown_connections 2024-09-15T22:16:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.926+0000 7f3a90e48640 1 --2- 172.21.15.5:0/3189964835 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a8c106f20 0x7f3a8c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.926+0000 7f3a90e48640 1 --2- 172.21.15.5:0/3189964835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c1027a0 0x7f3a8c1067f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.926+0000 7f3a90e48640 1 --2- 172.21.15.5:0/3189964835 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a8c101df0 0x7f3a8c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.926+0000 7f3a90e48640 1 -- 172.21.15.5:0/3189964835 >> 172.21.15.5:0/3189964835 conn(0x7f3a8c0fd8f0 msgr2=0x7f3a8c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.926+0000 7f3a90e48640 1 -- 172.21.15.5:0/3189964835 shutdown_connections 2024-09-15T22:16:07.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.926+0000 7f3a90e48640 1 -- 172.21.15.5:0/3189964835 wait complete. 2024-09-15T22:16:07.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.927+0000 7f3a90e48640 1 Processor -- start 2024-09-15T22:16:07.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.927+0000 7f3a90e48640 1 -- start start 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.927+0000 7f3a90e48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c101df0 0x7f3a8c073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.927+0000 7f3a90e48640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a8c1027a0 0x7f3a8c06cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.927+0000 7f3a90e48640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a8c106f20 0x7f3a8c06d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.927+0000 7f3a90e48640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c10fde0 con 0x7f3a8c101df0 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.927+0000 7f3a90e48640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c10fc60 con 0x7f3a8c106f20 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.927+0000 7f3a90e48640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c10ff60 con 0x7f3a8c1027a0 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.928+0000 7f3a8ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a8c106f20 0x7f3a8c06d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.928+0000 7f3a8ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a8c106f20 0x7f3a8c06d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:47738/0 (socket says 172.21.15.5:47738) 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.928+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/3498986309 learned_addr learned my addr 172.21.15.5:0/3498986309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.928+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/3498986309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a8c1027a0 msgr2=0x7f3a8c06cae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.928+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a8c1027a0 0x7f3a8c06cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.928+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/3498986309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c101df0 msgr2=0x7f3a8c073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:07.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.928+0000 7f3a8a59c640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c101df0 0x7f3a8c073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:07.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.928+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c101df0 0x7f3a8c073a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.928+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/3498986309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c06d5e0 con 0x7f3a8c106f20 2024-09-15T22:16:07.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.930+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a8c106f20 0x7f3a8c06d020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000c970 tx=0x7f3a8000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:07.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.931+0000 7f3a7b7fe640 1 -- 172.21.15.5:0/3498986309 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80007bb0 con 0x7f3a8c106f20 2024-09-15T22:16:07.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.931+0000 7f3a7b7fe640 1 -- 172.21.15.5:0/3498986309 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a80007d50 con 0x7f3a8c106f20 2024-09-15T22:16:07.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.931+0000 7f3a7b7fe640 1 -- 172.21.15.5:0/3498986309 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000f450 con 0x7f3a8c106f20 2024-09-15T22:16:07.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.933+0000 7f3a90e48640 1 -- 172.21.15.5:0/3498986309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c06d8d0 con 0x7f3a8c106f20 2024-09-15T22:16:07.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.933+0000 7f3a90e48640 1 -- 172.21.15.5:0/3498986309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c071820 con 0x7f3a8c106f20 2024-09-15T22:16:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.948+0000 7f3a93221740 1 Processor -- start 2024-09-15T22:16:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.949+0000 7f3a7b7fe640 1 -- 172.21.15.5:0/3498986309 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3a80004ee0 con 0x7f3a8c106f20 2024-09-15T22:16:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.950+0000 7f3a7b7fe640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3a740a79d0 0x7f3a740a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:07.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.950+0000 7f3a7b7fe640 1 -- 172.21.15.5:0/3498986309 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9629+0+0 (secure 0 0 0) 0x7f3a800d00d0 con 0x7f3a8c106f20 2024-09-15T22:16:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.951+0000 7f3a93221740 1 -- start start 2024-09-15T22:16:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.951+0000 7f3a93221740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2f4a0 0x55c436c2f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.951+0000 7f3a93221740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2fdc0 0x55c436c3a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.951+0000 7f3a93221740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c3ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.951+0000 7f3a93221740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c436b952e0 con 0x55c436c3a680 2024-09-15T22:16:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.951+0000 7f3a93221740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c436b3cce0 con 0x55c436c2f4a0 2024-09-15T22:16:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.951+0000 7f3a93221740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c436b912a0 con 0x55c436c2fdc0 2024-09-15T22:16:07.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.952+0000 7f3a89d9b640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3a740a79d0 0x7f3a740a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:07.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.955+0000 7f3a8a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2f4a0 0x55c436c2f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:07.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.956+0000 7f3a8a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2f4a0 0x55c436c2f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:47744/0 (socket says 172.21.15.5:47744) 2024-09-15T22:16:07.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.956+0000 7f3a8a59c640 1 -- 172.21.15.5:0/4158070027 learned_addr learned my addr 172.21.15.5:0/4158070027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:07.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.956+0000 7f3a89d9b640 1 --2- 172.21.15.5:0/4158070027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2fdc0 0x55c436c3a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:07.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.959+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/4158070027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c3ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.963+0000 7f3a8a59c640 1 -- 172.21.15.5:0/4158070027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2fdc0 msgr2=0x55c436c3a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:07.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.963+0000 7f3a8a59c640 1 --2- 172.21.15.5:0/4158070027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2fdc0 0x55c436c3a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.963+0000 7f3a8a59c640 1 -- 172.21.15.5:0/4158070027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 msgr2=0x55c436c3ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.963+0000 7f3a8a59c640 1 --2- 172.21.15.5:0/4158070027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c3ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.963+0000 7f3a8a59c640 1 -- 172.21.15.5:0/4158070027 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c436b21440 con 0x55c436c2f4a0 2024-09-15T22:16:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.964+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/4158070027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c3ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:07.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.965+0000 7f3a89d9b640 1 --2- 172.21.15.5:0/4158070027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2fdc0 0x55c436c3a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.969+0000 7f3a8a59c640 1 --2- 172.21.15.5:0/4158070027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2f4a0 0x55c436c2f880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3a84008d90 tx=0x7f3a8400d710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bd8f92414381d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:07.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.970+0000 7f3a89d9b640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3a740a79d0 0x7f3a740a9e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c033040 tx=0x7f3a7c0035f0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.972+0000 7f3a78ff9640 1 -- 172.21.15.5:0/4158070027 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a84004d20 con 0x55c436c2f4a0 2024-09-15T22:16:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.972+0000 7f3a78ff9640 1 -- 172.21.15.5:0/4158070027 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8400de20 con 0x55c436c2f4a0 2024-09-15T22:16:07.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.972+0000 7f3a78ff9640 1 -- 172.21.15.5:0/4158070027 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840055f0 con 0x55c436c2f4a0 2024-09-15T22:16:07.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.974+0000 7f3a93221740 1 -- 172.21.15.5:0/4158070027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2f4a0 msgr2=0x55c436c2f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:07.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.974+0000 7f3a93221740 1 --2- 172.21.15.5:0/4158070027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2f4a0 0x55c436c2f880 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3a84008d90 tx=0x7f3a8400d710 comp rx=0 tx=0).stop 2024-09-15T22:16:07.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.976+0000 7f3a93221740 1 -- 172.21.15.5:0/4158070027 shutdown_connections 2024-09-15T22:16:07.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.976+0000 7f3a93221740 1 --2- 172.21.15.5:0/4158070027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c3ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.976+0000 7f3a93221740 1 --2- 172.21.15.5:0/4158070027 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2fdc0 0x55c436c3a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.976+0000 7f3a93221740 1 --2- 172.21.15.5:0/4158070027 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2f4a0 0x55c436c2f880 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.976+0000 7f3a93221740 1 -- 172.21.15.5:0/4158070027 >> 172.21.15.5:0/4158070027 conn(0x55c436c2ba30 msgr2=0x55c436c2be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:07.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.982+0000 7f3a93221740 1 -- 172.21.15.5:0/4158070027 shutdown_connections 2024-09-15T22:16:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.983+0000 7f3a93221740 1 -- 172.21.15.5:0/4158070027 wait complete. 2024-09-15T22:16:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.984+0000 7f3a93221740 1 -- start start 2024-09-15T22:16:07.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.984+0000 7f3a93221740 1 Processor -- start 2024-09-15T22:16:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.985+0000 7f3a93221740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c3a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.985+0000 7f3a93221740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2fdc0 0x55c436c372b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.985+0000 7f3a93221740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c37830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.985+0000 7f3a93221740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c436c3d250 con 0x55c436c3a680 2024-09-15T22:16:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.985+0000 7f3a93221740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c436b50050 con 0x55c436c2fdc0 2024-09-15T22:16:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.985+0000 7f3a93221740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c436c3d3d0 con 0x55c436c2f4a0 2024-09-15T22:16:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.985+0000 7f3a93221740 10 client.0 _ll_register_callbacks cb 0x55c436b90160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.985+0000 7f3a8ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c37830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.986+0000 7f3a8ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c37830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37080/0 (socket says 172.21.15.5:37080) 2024-09-15T22:16:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.986+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/2944544447 learned_addr learned my addr 172.21.15.5:0/2944544447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:07.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.986+0000 7f3a8a59c640 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c3a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:07.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.990+0000 7f3a89d9b640 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2fdc0 0x55c436c372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.990+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/2944544447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 msgr2=0x55c436c3a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.990+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c3a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.990+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/2944544447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2fdc0 msgr2=0x55c436c372b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.990+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2fdc0 0x55c436c372b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.990+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/2944544447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4369063c0 con 0x55c436c3a680 2024-09-15T22:16:07.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.995+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c37830 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3a80098950 tx=0x7f3a8009c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702d6a042a626c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:08.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.995+0000 7f3a71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:08.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.998+0000 7f3a71ffb640 1 -- 172.21.15.5:0/2944544447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a800053d0 con 0x55c436c3a680 2024-09-15T22:16:08.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:07.998+0000 7f3a71ffb640 1 -- 172.21.15.5:0/2944544447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9629+0+0 (secure 0 0 0) 0x7f3a800cd5f0 con 0x55c436c3a680 2024-09-15T22:16:08.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.000+0000 7f3a93221740 1 -- 172.21.15.5:0/2944544447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a8400de20 con 0x55c436c3a680 2024-09-15T22:16:08.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.001+0000 7f3a71ffb640 1 -- 172.21.15.5:0/2944544447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a800cd7f0 con 0x55c436c3a680 2024-09-15T22:16:08.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.002+0000 7f3a93221740 10 client.16719 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:08.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.002+0000 7f3a93221740 1 -- 172.21.15.5:0/2944544447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c436c46ba0 con 0x55c436c3a680 2024-09-15T22:16:08.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.006+0000 7f3a71ffb640 1 -- 172.21.15.5:0/2944544447 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a8400de20 con 0x55c436c3a680 2024-09-15T22:16:08.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.006+0000 7f3a71ffb640 1 -- 172.21.15.5:0/2944544447 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3a80097cd0 con 0x55c436c3a680 2024-09-15T22:16:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.008+0000 7f3a93221740 10 client.16719 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:08.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.008+0000 7f3a93221740 10 client.16719 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.008+0000 7f3a93221740 20 client.16719 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.008+0000 7f3a93221740 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c436c48060 0x55c436c4a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.008+0000 7f3a93221740 4 client.16719 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:08.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.008+0000 7f3a93221740 1 -- 172.21.15.5:0/2944544447 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c436ba2850 con 0x55c436c48060 2024-09-15T22:16:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.016+0000 7f3a8a59c640 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c436c48060 0x55c436c4a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.029+0000 7f3a8a59c640 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c436c48060 0x55c436c4a4b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae6359ce3faf1f9d server_cookie=e41a954c8a1909ba in_seq=0 out_seq=0 2024-09-15T22:16:08.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.029+0000 7f3a71ffb640 10 client.16719 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.031+0000 7f3a71ffb640 1 -- 172.21.15.5:0/2944544447 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c436ba2850 con 0x55c436c48060 2024-09-15T22:16:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.031+0000 7f3a71ffb640 10 client.16719 handle_command_reply: tid=0 2024-09-15T22:16:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.031+0000 7f3a93221740 1 client.16719 shutdown 2024-09-15T22:16:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a93221740 1 -- 172.21.15.5:0/2944544447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 msgr2=0x55c436c37830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a93221740 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c37830 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3a80098950 tx=0x7f3a8009c7c0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a93221740 1 -- 172.21.15.5:0/2944544447 shutdown_connections 2024-09-15T22:16:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a93221740 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c37830 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a93221740 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c436c48060 0x55c436c4a4b0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a93221740 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2fdc0 0x55c436c372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a93221740 1 --2- 172.21.15.5:0/2944544447 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c3a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a93221740 1 -- 172.21.15.5:0/2944544447 >> 172.21.15.5:0/2944544447 conn(0x55c436c2ba30 msgr2=0x55c436c31710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a71ffb640 0 client.16719 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a93221740 1 -- 172.21.15.5:0/2944544447 shutdown_connections 2024-09-15T22:16:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a93221740 1 -- 172.21.15.5:0/2944544447 wait complete. 2024-09-15T22:16:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.032+0000 7f3a93221740 20 client.16719 trim_cache size 0 max 16384 2024-09-15T22:16:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a93221740 1 Processor -- start 2024-09-15T22:16:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a93221740 1 -- start start 2024-09-15T22:16:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a93221740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c32b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a93221740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2fdc0 0x55c436c4fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a93221740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c502c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a93221740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b82a0 con 0x55c436c3a680 2024-09-15T22:16:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a93221740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b5b70 con 0x55c436c2fdc0 2024-09-15T22:16:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a93221740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b39e0 con 0x55c436c2f4a0 2024-09-15T22:16:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a89d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2fdc0 0x55c436c4fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a89d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2fdc0 0x55c436c4fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:47754/0 (socket says 172.21.15.5:47754) 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a89d9b640 1 -- 172.21.15.5:0/2799875631 learned_addr learned my addr 172.21.15.5:0/2799875631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.069+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/2799875631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.070+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/2799875631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 msgr2=0x55c436c32b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.070+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/2799875631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c32b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.070+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/2799875631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2fdc0 msgr2=0x55c436c4fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.070+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/2799875631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2fdc0 0x55c436c4fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.070+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/2799875631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c436c2b700 con 0x55c436c3a680 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.070+0000 7f3a8ad9d640 1 --2- 172.21.15.5:0/2799875631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c502c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3a8009c140 tx=0x7f3a80005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c12a2dc06d7f68 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.070+0000 7f3a73fff640 1 -- 172.21.15.5:0/2799875631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a800b0120 con 0x55c436c3a680 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.071+0000 7f3a73fff640 1 -- 172.21.15.5:0/2799875631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a800b6030 con 0x55c436c3a680 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.071+0000 7f3a73fff640 1 -- 172.21.15.5:0/2799875631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8009b4f0 con 0x55c436c3a680 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.072+0000 7f3a93221740 1 -- 172.21.15.5:0/2799875631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 msgr2=0x55c436c502c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.072+0000 7f3a93221740 1 --2- 172.21.15.5:0/2799875631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c502c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f3a8009c140 tx=0x7f3a80005870 comp rx=0 tx=0).stop 2024-09-15T22:16:08.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.072+0000 7f3a93221740 1 -- 172.21.15.5:0/2799875631 shutdown_connections 2024-09-15T22:16:08.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.072+0000 7f3a93221740 1 --2- 172.21.15.5:0/2799875631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c3a680 0x55c436c502c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.072+0000 7f3a93221740 1 --2- 172.21.15.5:0/2799875631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c2fdc0 0x55c436c4fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.072+0000 7f3a93221740 1 --2- 172.21.15.5:0/2799875631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c32b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.072+0000 7f3a93221740 1 -- 172.21.15.5:0/2799875631 >> 172.21.15.5:0/2799875631 conn(0x55c436c2ba30 msgr2=0x55c436c39330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:08.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.072+0000 7f3a93221740 1 -- 172.21.15.5:0/2799875631 shutdown_connections 2024-09-15T22:16:08.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.072+0000 7f3a93221740 1 -- 172.21.15.5:0/2799875631 wait complete. 2024-09-15T22:16:08.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.073+0000 7f3a93221740 1 -- start start 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.074+0000 7f3a93221740 1 Processor -- start 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.074+0000 7f3a93221740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c44410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.074+0000 7f3a93221740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c2fdc0 0x55c436c3f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.074+0000 7f3a93221740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c3a680 0x55c436c3fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.075+0000 7f3a93221740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b4030 con 0x55c436c2fdc0 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.075+0000 7f3a93221740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b54a0 con 0x55c436c3a680 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.075+0000 7f3a93221740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c1b4700 con 0x55c436c2f4a0 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.075+0000 7f3a93221740 10 client.0 _ll_register_callbacks cb 0x7f3a800046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.075+0000 7f3a8ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c3a680 0x55c436c3fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.075+0000 7f3a8ad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c3a680 0x55c436c3fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:47762/0 (socket says 172.21.15.5:47762) 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.075+0000 7f3a8ad9d640 1 -- 172.21.15.5:0/1480265753 learned_addr learned my addr 172.21.15.5:0/1480265753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.076+0000 7f3a89d9b640 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c2fdc0 0x55c436c3f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.076+0000 7f3a8a59c640 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c44410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.076+0000 7f3a8a59c640 1 -- 172.21.15.5:0/1480265753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c3a680 msgr2=0x55c436c3fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.076+0000 7f3a8a59c640 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c3a680 0x55c436c3fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.076+0000 7f3a8a59c640 1 -- 172.21.15.5:0/1480265753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c2fdc0 msgr2=0x55c436c3f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.076+0000 7f3a8a59c640 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c2fdc0 0x55c436c3f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.076+0000 7f3a8a59c640 1 -- 172.21.15.5:0/1480265753 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c436c403c0 con 0x55c436c2f4a0 2024-09-15T22:16:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.077+0000 7f3a8a59c640 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c44410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400e010 tx=0x7f3a84008630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cd11ecf373930d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.077+0000 7f3a71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:16:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.077+0000 7f3a71ffb640 1 -- 172.21.15.5:0/1480265753 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a840049e0 con 0x55c436c2f4a0 2024-09-15T22:16:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.077+0000 7f3a93221740 1 -- 172.21.15.5:0/1480265753 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a800b6030 con 0x55c436c2f4a0 2024-09-15T22:16:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.078+0000 7f3a71ffb640 1 -- 172.21.15.5:0/1480265753 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9629+0+0 (secure 0 0 0) 0x7f3a84004b80 con 0x55c436c2f4a0 2024-09-15T22:16:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.078+0000 7f3a71ffb640 1 -- 172.21.15.5:0/1480265753 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a84037050 con 0x55c436c2f4a0 2024-09-15T22:16:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.078+0000 7f3a93221740 10 client.26239 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.078+0000 7f3a93221740 1 -- 172.21.15.5:0/1480265753 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c436cd70b0 con 0x55c436c2f4a0 2024-09-15T22:16:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.079+0000 7f3a71ffb640 1 -- 172.21.15.5:0/1480265753 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a800b6030 con 0x55c436c2f4a0 2024-09-15T22:16:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.080+0000 7f3a71ffb640 1 -- 172.21.15.5:0/1480265753 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3a84022050 con 0x55c436c2f4a0 2024-09-15T22:16:08.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.080+0000 7f3a93221740 10 client.26239 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.080+0000 7f3a93221740 10 client.26239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.080+0000 7f3a93221740 20 client.26239 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.080+0000 7f3a93221740 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c436cd7730 0x55c436cd9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.080+0000 7f3a93221740 4 client.26239 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:16:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.080+0000 7f3a93221740 1 -- 172.21.15.5:0/1480265753 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c436b9cd60 con 0x55c436cd7730 2024-09-15T22:16:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.081+0000 7f3a89d9b640 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c436cd7730 0x55c436cd9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:08.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.083+0000 7f3a89d9b640 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c436cd7730 0x55c436cd9b20 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370767fbb25abe97 server_cookie=60d4ec85f2965aa8 in_seq=0 out_seq=0 2024-09-15T22:16:08.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.083+0000 7f3a71ffb640 10 client.26239 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:08.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.083+0000 7f3a71ffb640 1 -- 172.21.15.5:0/1480265753 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x55c436b9cd60 con 0x55c436cd7730 2024-09-15T22:16:08.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.083+0000 7f3a71ffb640 10 client.26239 handle_command_reply: tid=0 2024-09-15T22:16:08.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.083+0000 7f3a93221740 1 client.26239 shutdown 2024-09-15T22:16:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.088+0000 7f3a93221740 1 -- 172.21.15.5:0/1480265753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 msgr2=0x55c436c44410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.088+0000 7f3a93221740 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c44410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400e010 tx=0x7f3a84008630 comp rx=0 tx=0).stop 2024-09-15T22:16:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.102+0000 7f3a93221740 1 -- 172.21.15.5:0/1480265753 shutdown_connections 2024-09-15T22:16:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.102+0000 7f3a93221740 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c436cd7730 0x55c436cd9b20 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.102+0000 7f3a93221740 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c436c3a680 0x55c436c3fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.102+0000 7f3a93221740 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c436c2fdc0 0x55c436c3f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.102+0000 7f3a93221740 1 --2- 172.21.15.5:0/1480265753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c436c2f4a0 0x55c436c44410 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.102+0000 7f3a93221740 1 -- 172.21.15.5:0/1480265753 >> 172.21.15.5:0/1480265753 conn(0x55c436c2ba30 msgr2=0x55c436c30ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.102+0000 7f3a71ffb640 0 client.26239 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:08.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.102+0000 7f3a93221740 1 -- 172.21.15.5:0/1480265753 shutdown_connections 2024-09-15T22:16:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.102+0000 7f3a93221740 1 -- 172.21.15.5:0/1480265753 wait complete. 2024-09-15T22:16:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.102+0000 7f3a93221740 20 client.26239 trim_cache size 0 max 16384 2024-09-15T22:16:08.105 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:16:08.105 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2024-09-15T22:16:08.105 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:16:08.105 INFO:teuthology.orchestra.run.smithi005.stdout: "a0a613df-6b35-41d7-889b-ed709c0a9fb5": { 2024-09-15T22:16:08.105 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:16:08.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "a0a613df-6b35-41d7-889b-ed709c0a9fb5", 2024-09-15T22:16:08.105 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:16:08.106 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:08.106 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:08.106 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:16:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 -- 172.21.15.5:0/3498986309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3a740a79d0 msgr2=0x7f3a740a9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3a740a79d0 0x7f3a740a9e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c033040 tx=0x7f3a7c0035f0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 -- 172.21.15.5:0/3498986309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a8c106f20 msgr2=0x7f3a8c06d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a8c106f20 0x7f3a8c06d020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000c970 tx=0x7f3a8000ce30 comp rx=0 tx=0).stop 2024-09-15T22:16:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 -- 172.21.15.5:0/3498986309 shutdown_connections 2024-09-15T22:16:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3a740a79d0 0x7f3a740a9e90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3a8c106f20 0x7f3a8c06d020 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3a8c1027a0 0x7f3a8c06cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 --2- 172.21.15.5:0/3498986309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c101df0 0x7f3a8c073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 -- 172.21.15.5:0/3498986309 >> 172.21.15.5:0/3498986309 conn(0x7f3a8c0fd8f0 msgr2=0x7f3a8c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 -- 172.21.15.5:0/3498986309 shutdown_connections 2024-09-15T22:16:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:08.104+0000 7f3a90e48640 1 -- 172.21.15.5:0/3498986309 wait complete. 2024-09-15T22:16:08.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:08 smithi005 ceph-mon[30944]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T22:16:08.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:08 smithi099 ceph-mon[34892]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T22:16:08.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:08 smithi129 ceph-mon[35314]: osdmap e90: 12 total, 12 up, 12 in 2024-09-15T22:16:08.861 INFO:tasks.cephfs.filesystem:scrub status for tag:a0a613df-6b35-41d7-889b-ed709c0a9fb5 - {'path': '/', 'tag': 'a0a613df-6b35-41d7-889b-ed709c0a9fb5', 'options': 'recursive,force'} 2024-09-15T22:16:09.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:09 smithi005 ceph-mon[30944]: pgmap v221: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 8.5 MiB/s wr, 795 op/s 2024-09-15T22:16:09.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:09 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:16:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 400.0 s, 626.3 tps, lat 161.815 ms stddev 236.816 2024-09-15T22:16:09.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:09 smithi099 ceph-mon[34892]: pgmap v221: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 8.5 MiB/s wr, 795 op/s 2024-09-15T22:16:09.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:09 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:16:09.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:09 smithi129 ceph-mon[35314]: pgmap v221: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 8.5 MiB/s wr, 795 op/s 2024-09-15T22:16:09.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:09 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:16:10.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:10 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:16:10.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:10 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:16:10.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:10 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:16:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:10 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:16:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:10 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:16:10.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:10 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:16:10.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:10 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:16:10.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:10 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:16:10.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:10 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:16:11.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:11 smithi005 ceph-mon[30944]: pgmap v222: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 8.5 MiB/s wr, 795 op/s 2024-09-15T22:16:11.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:11 smithi099 ceph-mon[34892]: pgmap v222: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 8.5 MiB/s wr, 795 op/s 2024-09-15T22:16:11.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:11 smithi129 ceph-mon[35314]: pgmap v222: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.4 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 8.5 MiB/s wr, 795 op/s 2024-09-15T22:16:12.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:12 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:12.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:12 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:12.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:12 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:13.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:13 smithi099 ceph-mon[34892]: pgmap v223: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 7.0 MiB/s wr, 671 op/s 2024-09-15T22:16:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:13 smithi129 ceph-mon[35314]: pgmap v223: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 7.0 MiB/s wr, 671 op/s 2024-09-15T22:16:13.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:13 smithi005 ceph-mon[30944]: pgmap v223: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 180 KiB/s rd, 7.0 MiB/s wr, 671 op/s 2024-09-15T22:16:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 405.0 s, 842.2 tps, lat 120.634 ms stddev 169.369 2024-09-15T22:16:15.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:15 smithi099 ceph-mon[34892]: pgmap v224: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.8 MiB/s wr, 641 op/s 2024-09-15T22:16:15.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:15 smithi129 ceph-mon[35314]: pgmap v224: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.8 MiB/s wr, 641 op/s 2024-09-15T22:16:15.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:15 smithi005 ceph-mon[30944]: pgmap v224: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 6.8 MiB/s wr, 641 op/s 2024-09-15T22:16:17.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:17 smithi099 ceph-mon[34892]: pgmap v225: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.3 MiB/s wr, 663 op/s 2024-09-15T22:16:17.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:17 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:16:17.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:17 smithi129 ceph-mon[35314]: pgmap v225: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.3 MiB/s wr, 663 op/s 2024-09-15T22:16:17.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:17 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:16:17.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:17 smithi005 ceph-mon[30944]: pgmap v225: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 7.3 MiB/s wr, 663 op/s 2024-09-15T22:16:17.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:17 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:16:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 410.0 s, 1161.8 tps, lat 88.120 ms stddev 103.582 2024-09-15T22:16:19.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:19 smithi099 ceph-mon[34892]: pgmap v226: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 7.1 MiB/s wr, 646 op/s 2024-09-15T22:16:19.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:19 smithi099 ceph-mon[34892]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T22:16:19.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:19 smithi129 ceph-mon[35314]: pgmap v226: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 7.1 MiB/s wr, 646 op/s 2024-09-15T22:16:19.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:19 smithi129 ceph-mon[35314]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T22:16:19.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:19 smithi005 ceph-mon[30944]: pgmap v226: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 7.1 MiB/s wr, 646 op/s 2024-09-15T22:16:19.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:19 smithi005 ceph-mon[30944]: osdmap e91: 12 total, 12 up, 12 in 2024-09-15T22:16:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:21 smithi099 ceph-mon[34892]: pgmap v228: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 6.9 MiB/s wr, 662 op/s 2024-09-15T22:16:21.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:21 smithi129 ceph-mon[35314]: pgmap v228: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 6.9 MiB/s wr, 662 op/s 2024-09-15T22:16:21.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:21 smithi005 ceph-mon[30944]: pgmap v228: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 6.9 MiB/s wr, 662 op/s 2024-09-15T22:16:23.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:23 smithi099 ceph-mon[34892]: pgmap v229: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 7.8 MiB/s wr, 692 op/s 2024-09-15T22:16:23.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:23 smithi129 ceph-mon[35314]: pgmap v229: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 7.8 MiB/s wr, 692 op/s 2024-09-15T22:16:23.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:23 smithi005 ceph-mon[30944]: pgmap v229: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 7.8 MiB/s wr, 692 op/s 2024-09-15T22:16:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 415.0 s, 1107.6 tps, lat 86.970 ms stddev 114.654 2024-09-15T22:16:25.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:25 smithi099 ceph-mon[34892]: pgmap v230: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 7.7 MiB/s wr, 688 op/s 2024-09-15T22:16:25.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:25 smithi129 ceph-mon[35314]: pgmap v230: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 7.7 MiB/s wr, 688 op/s 2024-09-15T22:16:25.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:25 smithi005 ceph-mon[30944]: pgmap v230: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 182 KiB/s rd, 7.7 MiB/s wr, 688 op/s 2024-09-15T22:16:27.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:27 smithi099 ceph-mon[34892]: pgmap v231: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.7 MiB/s wr, 587 op/s 2024-09-15T22:16:27.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:27 smithi129 ceph-mon[35314]: pgmap v231: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.7 MiB/s wr, 587 op/s 2024-09-15T22:16:27.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:27 smithi005 ceph-mon[30944]: pgmap v231: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.7 MiB/s wr, 587 op/s 2024-09-15T22:16:29.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 420.0 s, 903.8 tps, lat 114.768 ms stddev 144.541 2024-09-15T22:16:29.732 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.7 2024-09-15T22:16:29.733 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:16:29.797 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:29 smithi005 ceph-mon[30944]: pgmap v232: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.6 MiB/s wr, 577 op/s 2024-09-15T22:16:29.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:29 smithi099 ceph-mon[34892]: pgmap v232: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.6 MiB/s wr, 577 op/s 2024-09-15T22:16:29.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:29 smithi129 ceph-mon[35314]: pgmap v232: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 6.6 MiB/s wr, 577 op/s 2024-09-15T22:16:30.296 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:31 smithi005 ceph-mon[30944]: pgmap v233: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.5 MiB/s wr, 534 op/s 2024-09-15T22:16:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:31 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3948295231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:16:31.807 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:16:31.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.956+0000 7efcc6937640 1 -- 172.21.15.5:0/1910002271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102570 msgr2=0x7efcc01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.956+0000 7efcc6937640 1 --2- 172.21.15.5:0/1910002271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102570 0x7efcc01029b0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efcb4009900 tx=0x7efcb402a480 comp rx=0 tx=0).stop 2024-09-15T22:16:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.956+0000 7efcc6937640 1 -- 172.21.15.5:0/1910002271 shutdown_connections 2024-09-15T22:16:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.956+0000 7efcc6937640 1 --2- 172.21.15.5:0/1910002271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0102ef0 0x7efcc010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.956+0000 7efcc6937640 1 --2- 172.21.15.5:0/1910002271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102570 0x7efcc01029b0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.956+0000 7efcc6937640 1 --2- 172.21.15.5:0/1910002271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc0100e20 0x7efcc0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.956+0000 7efcc6937640 1 -- 172.21.15.5:0/1910002271 >> 172.21.15.5:0/1910002271 conn(0x7efcc00fc7d0 msgr2=0x7efcc00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.956+0000 7efcc6937640 1 -- 172.21.15.5:0/1910002271 shutdown_connections 2024-09-15T22:16:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.956+0000 7efcc6937640 1 -- 172.21.15.5:0/1910002271 wait complete. 2024-09-15T22:16:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.957+0000 7efcc6937640 1 Processor -- start 2024-09-15T22:16:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.957+0000 7efcc6937640 1 -- start start 2024-09-15T22:16:31.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.957+0000 7efcc6937640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc0100e20 0x7efcc00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:31.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.957+0000 7efcc6937640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0102570 0x7efcc00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:31.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.957+0000 7efcc6937640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102ef0 0x7efcc00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:31.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.957+0000 7efcc6937640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0110f50 con 0x7efcc0102ef0 2024-09-15T22:16:31.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.957+0000 7efcc6937640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efcc0110dd0 con 0x7efcc0100e20 2024-09-15T22:16:31.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.957+0000 7efcc6937640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efcc01110d0 con 0x7efcc0102570 2024-09-15T22:16:31.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.957+0000 7efcc4ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102ef0 0x7efcc00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:31.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.957+0000 7efcc4ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102ef0 0x7efcc00fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42030/0 (socket says 172.21.15.5:42030) 2024-09-15T22:16:31.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.957+0000 7efcc4ed4640 1 -- 172.21.15.5:0/3948295231 learned_addr learned my addr 172.21.15.5:0/3948295231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:31.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcbffff640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc0100e20 0x7efcc00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:31.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcbf7fe640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0102570 0x7efcc00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:31.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcc4ed4640 1 -- 172.21.15.5:0/3948295231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0102570 msgr2=0x7efcc00fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcc4ed4640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0102570 0x7efcc00fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcc4ed4640 1 -- 172.21.15.5:0/3948295231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc0100e20 msgr2=0x7efcc00fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcc4ed4640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc0100e20 0x7efcc00fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcc4ed4640 1 -- 172.21.15.5:0/3948295231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc01a69b0 con 0x7efcc0102ef0 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcbffff640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc0100e20 0x7efcc00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcc4ed4640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102ef0 0x7efcc00fd700 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efcb800eaa0 tx=0x7efcb800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcbd7fa640 1 -- 172.21.15.5:0/3948295231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb800cdb0 con 0x7efcc0102ef0 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcc6937640 1 -- 172.21.15.5:0/3948295231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc01a6ca0 con 0x7efcc0102ef0 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcbd7fa640 1 -- 172.21.15.5:0/3948295231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcb8005020 con 0x7efcc0102ef0 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcbd7fa640 1 -- 172.21.15.5:0/3948295231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb80116b0 con 0x7efcc0102ef0 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.958+0000 7efcc6937640 1 -- 172.21.15.5:0/3948295231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc01a7190 con 0x7efcc0102ef0 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.959+0000 7efcc6937640 1 -- 172.21.15.5:0/3948295231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc0101200 con 0x7efcc0102ef0 2024-09-15T22:16:31.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.961+0000 7efcbd7fa640 1 -- 172.21.15.5:0/3948295231 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7efcb8005560 con 0x7efcc0102ef0 2024-09-15T22:16:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.961+0000 7efcbd7fa640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efca80a3430 0x7efca80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.961+0000 7efcbffff640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efca80a3430 0x7efca80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.961+0000 7efcbd7fa640 1 -- 172.21.15.5:0/3948295231 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7efcb80d0440 con 0x7efcc0102ef0 2024-09-15T22:16:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.962+0000 7efcbffff640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efca80a3430 0x7efca80a58f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efcb00058c0 tx=0x7efcb0005850 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:30.966+0000 7efcbd7fa640 1 -- 172.21.15.5:0/3948295231 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcb8016030 con 0x7efcc0102ef0 2024-09-15T22:16:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.350+0000 7efcc6937640 1 -- 172.21.15.5:0/3948295231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcc010a0a0 con 0x7efcc0102ef0 2024-09-15T22:16:31.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.356+0000 7efcbd7fa640 1 -- 172.21.15.5:0/3948295231 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7efcb809c9c0 con 0x7efcc0102ef0 2024-09-15T22:16:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:16:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.362+0000 7efcc6937640 1 -- 172.21.15.5:0/3948295231 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efca80a3430 msgr2=0x7efca80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.362+0000 7efcc6937640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efca80a3430 0x7efca80a58f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efcb00058c0 tx=0x7efcb0005850 comp rx=0 tx=0).stop 2024-09-15T22:16:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.362+0000 7efcc6937640 1 -- 172.21.15.5:0/3948295231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102ef0 msgr2=0x7efcc00fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.362+0000 7efcc6937640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102ef0 0x7efcc00fd700 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efcb800eaa0 tx=0x7efcb800ef60 comp rx=0 tx=0).stop 2024-09-15T22:16:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.364+0000 7efcc6937640 1 -- 172.21.15.5:0/3948295231 shutdown_connections 2024-09-15T22:16:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.364+0000 7efcc6937640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efca80a3430 0x7efca80a58f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.364+0000 7efcc6937640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcc0102ef0 0x7efcc00fd700 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.364+0000 7efcc6937640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efcc0102570 0x7efcc00fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.364+0000 7efcc6937640 1 --2- 172.21.15.5:0/3948295231 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efcc0100e20 0x7efcc00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.364+0000 7efcc6937640 1 -- 172.21.15.5:0/3948295231 >> 172.21.15.5:0/3948295231 conn(0x7efcc00fc7d0 msgr2=0x7efcc006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.370+0000 7efcc6937640 1 -- 172.21.15.5:0/3948295231 shutdown_connections 2024-09-15T22:16:31.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:31.372+0000 7efcc6937640 1 -- 172.21.15.5:0/3948295231 wait complete. 2024-09-15T22:16:31.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:31 smithi099 ceph-mon[34892]: pgmap v233: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.5 MiB/s wr, 534 op/s 2024-09-15T22:16:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:31 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3948295231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:31.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:31 smithi129 ceph-mon[35314]: pgmap v233: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 6.5 MiB/s wr, 534 op/s 2024-09-15T22:16:31.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:31 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3948295231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:33.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:33 smithi099 ceph-mon[34892]: pgmap v234: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 8.2 MiB/s wr, 705 op/s 2024-09-15T22:16:33.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:33 smithi129 ceph-mon[35314]: pgmap v234: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 8.2 MiB/s wr, 705 op/s 2024-09-15T22:16:33.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:33 smithi005 ceph-mon[30944]: pgmap v234: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 8.2 MiB/s wr, 705 op/s 2024-09-15T22:16:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 425.0 s, 678.8 tps, lat 144.845 ms stddev 356.654 2024-09-15T22:16:35.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:35 smithi099 ceph-mon[34892]: pgmap v235: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.0 MiB/s wr, 559 op/s 2024-09-15T22:16:35.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:35 smithi129 ceph-mon[35314]: pgmap v235: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.0 MiB/s wr, 559 op/s 2024-09-15T22:16:35.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:35 smithi005 ceph-mon[30944]: pgmap v235: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 6.0 MiB/s wr, 559 op/s 2024-09-15T22:16:35.918 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 143.7 --await-for 45.0' 2024-09-15T22:16:35.918 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 143.7 --await-for 45.0 2024-09-15T22:16:36.522 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:37.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:37 smithi099 ceph-mon[34892]: pgmap v236: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-15T22:16:37.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:37 smithi129 ceph-mon[35314]: pgmap v236: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-15T22:16:37.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:37 smithi005 ceph-mon[30944]: pgmap v236: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 12 MiB/s wr, 1.12k op/s 2024-09-15T22:16:38.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:16:39.662 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 430.0 s, 556.8 tps, lat 176.588 ms stddev 207.912 2024-09-15T22:16:39.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:39 smithi099 ceph-mon[34892]: pgmap v237: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-09-15T22:16:39.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:39 smithi129 ceph-mon[35314]: pgmap v237: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-09-15T22:16:39.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:39 smithi005 ceph-mon[30944]: pgmap v237: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 10 MiB/s wr, 998 op/s 2024-09-15T22:16:41.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.061+0000 7fddc2030640 1 -- 172.21.15.5:0/3782544063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddbc1023e0 msgr2=0x7fddbc106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:41.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.061+0000 7fddc2030640 1 --2- 172.21.15.5:0/3782544063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddbc1023e0 0x7fddbc106890 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009900 tx=0x7fddb002a410 comp rx=0 tx=0).stop 2024-09-15T22:16:41.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.061+0000 7fddc2030640 1 -- 172.21.15.5:0/3782544063 shutdown_connections 2024-09-15T22:16:41.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.061+0000 7fddc2030640 1 --2- 172.21.15.5:0/3782544063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddbc106fc0 0x7fddbc10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.061+0000 7fddc2030640 1 --2- 172.21.15.5:0/3782544063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddbc1023e0 0x7fddbc106890 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.061+0000 7fddc2030640 1 --2- 172.21.15.5:0/3782544063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddbc101a30 0x7fddbc101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.061+0000 7fddc2030640 1 -- 172.21.15.5:0/3782544063 >> 172.21.15.5:0/3782544063 conn(0x7fddbc075f70 msgr2=0x7fddbc076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:41.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.061+0000 7fddc2030640 1 -- 172.21.15.5:0/3782544063 shutdown_connections 2024-09-15T22:16:41.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.061+0000 7fddc2030640 1 -- 172.21.15.5:0/3782544063 wait complete. 2024-09-15T22:16:41.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.062+0000 7fddc2030640 1 Processor -- start 2024-09-15T22:16:41.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.062+0000 7fddc2030640 1 -- start start 2024-09-15T22:16:41.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.063+0000 7fddc2030640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddbc101a30 0x7fddbc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.063+0000 7fddc2030640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddbc1023e0 0x7fddbc1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.063+0000 7fddc2030640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddbc106fc0 0x7fddbc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.063+0000 7fddc2030640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc110e60 con 0x7fddbc101a30 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.063+0000 7fddc2030640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc110ce0 con 0x7fddbc106fc0 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.063+0000 7fddbb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddbc101a30 0x7fddbc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.063+0000 7fddbb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddbc101a30 0x7fddbc198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44546/0 (socket says 172.21.15.5:44546) 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.063+0000 7fddbb7fe640 1 -- 172.21.15.5:0/2471298006 learned_addr learned my addr 172.21.15.5:0/2471298006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.063+0000 7fddc2030640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc110fe0 con 0x7fddbc1023e0 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.063+0000 7fddbbfff640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddbc106fc0 0x7fddbc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.063+0000 7fddbaffd640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddbc1023e0 0x7fddbc1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.064+0000 7fddbaffd640 1 -- 172.21.15.5:0/2471298006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddbc106fc0 msgr2=0x7fddbc19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.064+0000 7fddbaffd640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddbc106fc0 0x7fddbc19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.064+0000 7fddbaffd640 1 -- 172.21.15.5:0/2471298006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddbc101a30 msgr2=0x7fddbc198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.064+0000 7fddbaffd640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddbc101a30 0x7fddbc198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.064+0000 7fddbaffd640 1 -- 172.21.15.5:0/2471298006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc19d410 con 0x7fddbc1023e0 2024-09-15T22:16:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.064+0000 7fddbbfff640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddbc106fc0 0x7fddbc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.064+0000 7fddbaffd640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddbc1023e0 0x7fddbc1989a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fddb0000c00 tx=0x7fddb00329e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.065+0000 7fddb8ff9640 1 -- 172.21.15.5:0/2471298006 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb003f030 con 0x7fddbc1023e0 2024-09-15T22:16:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.065+0000 7fddb8ff9640 1 -- 172.21.15.5:0/2471298006 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb0032de0 con 0x7fddbc1023e0 2024-09-15T22:16:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.065+0000 7fddc2030640 1 -- 172.21.15.5:0/2471298006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc19d6a0 con 0x7fddbc1023e0 2024-09-15T22:16:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.066+0000 7fddb8ff9640 1 -- 172.21.15.5:0/2471298006 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0035070 con 0x7fddbc1023e0 2024-09-15T22:16:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.066+0000 7fddc2030640 1 -- 172.21.15.5:0/2471298006 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc102850 con 0x7fddbc1023e0 2024-09-15T22:16:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.069+0000 7fddb8ff9640 1 -- 172.21.15.5:0/2471298006 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fddb0042030 con 0x7fddbc1023e0 2024-09-15T22:16:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.069+0000 7fddb8ff9640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdda40a3580 0x7fdda40a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.069+0000 7fddb8ff9640 1 -- 172.21.15.5:0/2471298006 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fddb00f03d0 con 0x7fddbc1023e0 2024-09-15T22:16:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.069+0000 7fddbb7fe640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdda40a3580 0x7fdda40a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.070+0000 7fddbb7fe640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdda40a3580 0x7fdda40a5a40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fddac009c90 tx=0x7fddac005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.072+0000 7fddc4409740 1 Processor -- start 2024-09-15T22:16:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.072+0000 7fddc4409740 1 -- start start 2024-09-15T22:16:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.072+0000 7fddc4409740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac0cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.072+0000 7fddc4409740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac0d080 0x55e9dac17400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.072+0000 7fddc4409740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac17940 0x55e9dac19d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.072+0000 7fddc4409740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e9dab722a0 con 0x55e9dac17940 2024-09-15T22:16:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.072+0000 7fddc4409740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e9dab19b10 con 0x55e9dac0d080 2024-09-15T22:16:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.072+0000 7fddc4409740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e9dab6e260 con 0x55e9dac0c760 2024-09-15T22:16:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.072+0000 7fddbb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac0cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.072+0000 7fddbaffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac0d080 0x55e9dac17400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.073+0000 7fddbbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac17940 0x55e9dac19d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.073+0000 7fddbaffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac0d080 0x55e9dac17400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33802/0 (socket says 172.21.15.5:33802) 2024-09-15T22:16:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.073+0000 7fddbaffd640 1 -- 172.21.15.5:0/4052153725 learned_addr learned my addr 172.21.15.5:0/4052153725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.073+0000 7fddbb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac0cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:41302/0 (socket says 172.21.15.5:41302) 2024-09-15T22:16:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.073+0000 7fddbb7fe640 1 -- 172.21.15.5:0/4052153725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac0d080 msgr2=0x55e9dac17400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.073+0000 7fddbb7fe640 1 --2- 172.21.15.5:0/4052153725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac0d080 0x55e9dac17400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.073+0000 7fddbb7fe640 1 -- 172.21.15.5:0/4052153725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac17940 msgr2=0x55e9dac19d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.073+0000 7fddbb7fe640 1 --2- 172.21.15.5:0/4052153725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac17940 0x55e9dac19d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.073+0000 7fddbb7fe640 1 -- 172.21.15.5:0/4052153725 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9da8e3b90 con 0x55e9dac0c760 2024-09-15T22:16:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.074+0000 7fddbb7fe640 1 --2- 172.21.15.5:0/4052153725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac0cb40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fddac009b50 tx=0x7fddac014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a0c91eef651e5a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.074+0000 7fdda1ffb640 1 -- 172.21.15.5:0/4052153725 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac02e420 con 0x55e9dac0c760 2024-09-15T22:16:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.074+0000 7fdda1ffb640 1 -- 172.21.15.5:0/4052153725 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddac02eaa0 con 0x55e9dac0c760 2024-09-15T22:16:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.074+0000 7fdda1ffb640 1 -- 172.21.15.5:0/4052153725 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac01abe0 con 0x55e9dac0c760 2024-09-15T22:16:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.075+0000 7fddc4409740 1 -- 172.21.15.5:0/4052153725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 msgr2=0x55e9dac0cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.075+0000 7fddc4409740 1 --2- 172.21.15.5:0/4052153725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac0cb40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fddac009b50 tx=0x7fddac014000 comp rx=0 tx=0).stop 2024-09-15T22:16:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.075+0000 7fddc4409740 1 -- 172.21.15.5:0/4052153725 shutdown_connections 2024-09-15T22:16:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.075+0000 7fddc4409740 1 --2- 172.21.15.5:0/4052153725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac17940 0x55e9dac19d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.075+0000 7fddc4409740 1 --2- 172.21.15.5:0/4052153725 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac0d080 0x55e9dac17400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.075+0000 7fddc4409740 1 --2- 172.21.15.5:0/4052153725 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac0cb40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.075+0000 7fddc4409740 1 -- 172.21.15.5:0/4052153725 >> 172.21.15.5:0/4052153725 conn(0x55e9dac08cf0 msgr2=0x55e9dac090d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.075+0000 7fddc4409740 1 -- 172.21.15.5:0/4052153725 shutdown_connections 2024-09-15T22:16:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.075+0000 7fddc4409740 1 -- 172.21.15.5:0/4052153725 wait complete. 2024-09-15T22:16:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.075+0000 7fddc4409740 1 -- start start 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.076+0000 7fddc4409740 1 Processor -- start 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.076+0000 7fddc4409740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac14830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.076+0000 7fddc4409740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac0d080 0x55e9dac14d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.076+0000 7fddc4409740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac17940 0x55e9dac152b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.076+0000 7fddc4409740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e9dac1a510 con 0x55e9dac0d080 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.076+0000 7fddc4409740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e9daafa6e0 con 0x55e9dac17940 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddc4409740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e9dac1a690 con 0x55e9dac0c760 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddbb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac14830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddbb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac14830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:41310/0 (socket says 172.21.15.5:41310) 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddbb7fe640 1 -- 172.21.15.5:0/1445668412 learned_addr learned my addr 172.21.15.5:0/1445668412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddc4409740 10 client.0 _ll_register_callbacks cb 0x55e9dab1ea80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:41.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddbaffd640 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac0d080 0x55e9dac14d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddbb7fe640 1 -- 172.21.15.5:0/1445668412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac17940 msgr2=0x55e9dac152b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddbb7fe640 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac17940 0x55e9dac152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddbb7fe640 1 -- 172.21.15.5:0/1445668412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac0d080 msgr2=0x55e9dac14d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddbb7fe640 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac0d080 0x55e9dac14d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddbb7fe640 1 -- 172.21.15.5:0/1445668412 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9dac23a00 con 0x55e9dac0c760 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddbaffd640 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac0d080 0x55e9dac14d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fddbb7fe640 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac14830 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fddac01a670 tx=0x7fddac01ae60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a18486410104c4a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.077+0000 7fdd9affd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.078+0000 7fdd9affd640 1 -- 172.21.15.5:0/1445668412 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac03b060 con 0x55e9dac0c760 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.078+0000 7fdd9affd640 1 -- 172.21.15.5:0/1445668412 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fddac00edc0 con 0x55e9dac0c760 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.078+0000 7fddc4409740 1 -- 172.21.15.5:0/1445668412 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddac02eaa0 con 0x55e9dac0c760 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.078+0000 7fdd9affd640 1 -- 172.21.15.5:0/1445668412 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fddac042050 con 0x55e9dac0c760 2024-09-15T22:16:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.078+0000 7fddc4409740 10 client.26257 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.078+0000 7fddc4409740 1 -- 172.21.15.5:0/1445668412 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9dac26370 con 0x55e9dac0c760 2024-09-15T22:16:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.079+0000 7fdd9affd640 1 -- 172.21.15.5:0/1445668412 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddac01a960 con 0x55e9dac0c760 2024-09-15T22:16:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.079+0000 7fdd9affd640 1 -- 172.21.15.5:0/1445668412 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fddac0054a0 con 0x55e9dac0c760 2024-09-15T22:16:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.079+0000 7fddc4409740 10 client.26257 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.079+0000 7fddc4409740 10 client.26257 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:16:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.079+0000 7fddc4409740 20 client.26257 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.079+0000 7fddc4409740 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e9dac276c0 0x55e9dac29ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.079+0000 7fddc4409740 4 client.26257 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.079+0000 7fddc4409740 1 -- 172.21.15.5:0/1445668412 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fddac02eaa0 con 0x55e9dac276c0 2024-09-15T22:16:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.080+0000 7fddbaffd640 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e9dac276c0 0x55e9dac29ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.080+0000 7fdd9affd640 10 client.26257 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.080+0000 7fddbaffd640 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e9dac276c0 0x55e9dac29ab0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b666d746d9e657d0 server_cookie=c1ca83d4a6033032 in_seq=0 out_seq=0 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.081+0000 7fdd9affd640 1 -- 172.21.15.5:0/1445668412 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fddac02eaa0 con 0x55e9dac276c0 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.081+0000 7fdd9affd640 10 client.26257 handle_command_reply: tid=0 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.081+0000 7fddc4409740 1 client.26257 shutdown 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.081+0000 7fddc4409740 1 -- 172.21.15.5:0/1445668412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 msgr2=0x55e9dac14830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.081+0000 7fddc4409740 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac14830 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fddac01a670 tx=0x7fddac01ae60 comp rx=0 tx=0).stop 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.082+0000 7fddc4409740 1 -- 172.21.15.5:0/1445668412 shutdown_connections 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.082+0000 7fddc4409740 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e9dac276c0 0x55e9dac29ab0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.082+0000 7fdd9affd640 0 client.26257 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.082+0000 7fddc4409740 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac17940 0x55e9dac152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.082+0000 7fddc4409740 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac0d080 0x55e9dac14d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.082+0000 7fddc4409740 1 --2- 172.21.15.5:0/1445668412 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac14830 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.082+0000 7fddc4409740 1 -- 172.21.15.5:0/1445668412 >> 172.21.15.5:0/1445668412 conn(0x55e9dac08cf0 msgr2=0x55e9dac0e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.082+0000 7fddc4409740 1 -- 172.21.15.5:0/1445668412 shutdown_connections 2024-09-15T22:16:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.082+0000 7fddc4409740 1 -- 172.21.15.5:0/1445668412 wait complete. 2024-09-15T22:16:41.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.082+0000 7fddc4409740 20 client.26257 trim_cache size 0 max 16384 2024-09-15T22:16:41.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.112+0000 7fddc4409740 1 Processor -- start 2024-09-15T22:16:41.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.112+0000 7fddc4409740 1 -- start start 2024-09-15T22:16:41.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.113+0000 7fddc4409740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac0c760 0x55e9dac0f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.113+0000 7fddc4409740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0d080 0x55e9dac22e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.113+0000 7fddc4409740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac17940 0x55e9dac233b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.113+0000 7fddbaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0d080 0x55e9dac22e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.113+0000 7fddbaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0d080 0x55e9dac22e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:41318/0 (socket says 172.21.15.5:41318) 2024-09-15T22:16:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.113+0000 7fddbaffd640 1 -- 172.21.15.5:0/4229739791 learned_addr learned my addr 172.21.15.5:0/4229739791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.113+0000 7fddc4409740 1 -- 172.21.15.5:0/4229739791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1af7b0 con 0x55e9dac17940 2024-09-15T22:16:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.113+0000 7fddc4409740 1 -- 172.21.15.5:0/4229739791 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1ad080 con 0x55e9dac0c760 2024-09-15T22:16:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.113+0000 7fddc4409740 1 -- 172.21.15.5:0/4229739791 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1aaef0 con 0x55e9dac0d080 2024-09-15T22:16:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.114+0000 7fddbaffd640 1 -- 172.21.15.5:0/4229739791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac0c760 msgr2=0x55e9dac0f1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:16:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.114+0000 7fddbaffd640 1 --2- 172.21.15.5:0/4229739791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac0c760 0x55e9dac0f1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.114+0000 7fddbaffd640 1 -- 172.21.15.5:0/4229739791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac17940 msgr2=0x55e9dac233b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.114+0000 7fddbaffd640 1 --2- 172.21.15.5:0/4229739791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac17940 0x55e9dac233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.114+0000 7fddbaffd640 1 -- 172.21.15.5:0/4229739791 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9dac04260 con 0x55e9dac0d080 2024-09-15T22:16:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.115+0000 7fddbaffd640 1 --2- 172.21.15.5:0/4229739791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0d080 0x55e9dac22e70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fddb0005900 tx=0x7fddb00f0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb60543f74f8acd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.115+0000 7fdda17fa640 1 -- 172.21.15.5:0/4229739791 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0049040 con 0x55e9dac0d080 2024-09-15T22:16:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.116+0000 7fdda17fa640 1 -- 172.21.15.5:0/4229739791 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb00c4ec0 con 0x55e9dac0d080 2024-09-15T22:16:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.116+0000 7fdda17fa640 1 -- 172.21.15.5:0/4229739791 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb00b8b00 con 0x55e9dac0d080 2024-09-15T22:16:41.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.117+0000 7fddc4409740 1 -- 172.21.15.5:0/4229739791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0d080 msgr2=0x55e9dac22e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:41.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.117+0000 7fddc4409740 1 --2- 172.21.15.5:0/4229739791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0d080 0x55e9dac22e70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fddb0005900 tx=0x7fddb00f0040 comp rx=0 tx=0).stop 2024-09-15T22:16:41.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.117+0000 7fddc4409740 1 -- 172.21.15.5:0/4229739791 shutdown_connections 2024-09-15T22:16:41.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.117+0000 7fddc4409740 1 --2- 172.21.15.5:0/4229739791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac17940 0x55e9dac233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.117+0000 7fddc4409740 1 --2- 172.21.15.5:0/4229739791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0d080 0x55e9dac22e70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.117+0000 7fddc4409740 1 --2- 172.21.15.5:0/4229739791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac0c760 0x55e9dac0f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.117+0000 7fddc4409740 1 -- 172.21.15.5:0/4229739791 >> 172.21.15.5:0/4229739791 conn(0x55e9dac08cf0 msgr2=0x55e9dac22760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:41.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.117+0000 7fddc4409740 1 -- 172.21.15.5:0/4229739791 shutdown_connections 2024-09-15T22:16:41.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.117+0000 7fddc4409740 1 -- 172.21.15.5:0/4229739791 wait complete. 2024-09-15T22:16:41.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.118+0000 7fddc4409740 1 -- start start 2024-09-15T22:16:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.118+0000 7fddc4409740 1 Processor -- start 2024-09-15T22:16:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.118+0000 7fddc4409740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac1f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.118+0000 7fddc4409740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac0d080 0x55e9dac1fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.118+0000 7fddc4409740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac17940 0x55e9dac1c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.118+0000 7fddc4409740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1ab540 con 0x55e9dac0d080 2024-09-15T22:16:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.118+0000 7fddc4409740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1ac9b0 con 0x55e9dac17940 2024-09-15T22:16:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.118+0000 7fddc4409740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc1abc10 con 0x55e9dac0c760 2024-09-15T22:16:41.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.118+0000 7fddc4409740 10 client.0 _ll_register_callbacks cb 0x7fddac009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fddbb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac1f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fddbb7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac1f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:41332/0 (socket says 172.21.15.5:41332) 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fddbb7fe640 1 -- 172.21.15.5:0/3819039457 learned_addr learned my addr 172.21.15.5:0/3819039457 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fddbbfff640 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac17940 0x55e9dac1c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fddbaffd640 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac0d080 0x55e9dac1fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fddbb7fe640 1 -- 172.21.15.5:0/3819039457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac17940 msgr2=0x55e9dac1c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fddbb7fe640 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac17940 0x55e9dac1c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fddbb7fe640 1 -- 172.21.15.5:0/3819039457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac0d080 msgr2=0x55e9dac1fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fddbb7fe640 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac0d080 0x55e9dac1fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fddbb7fe640 1 -- 172.21.15.5:0/3819039457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9dac09980 con 0x55e9dac0c760 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fddbb7fe640 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac1f8e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fddac004770 tx=0x7fddac011df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6194b35e4cc69f82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:41.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fdd9affd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.119+0000 7fdd9affd640 1 -- 172.21.15.5:0/3819039457 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac04a020 con 0x55e9dac0c760 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.120+0000 7fdd9affd640 1 -- 172.21.15.5:0/3819039457 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fddac00ec10 con 0x55e9dac0c760 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.120+0000 7fddc4409740 1 -- 172.21.15.5:0/3819039457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddb00c4ec0 con 0x55e9dac0c760 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.120+0000 7fdd9affd640 1 -- 172.21.15.5:0/3819039457 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fddac052050 con 0x55e9dac0c760 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.120+0000 7fddc4409740 10 client.26263 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.120+0000 7fddc4409740 1 -- 172.21.15.5:0/3819039457 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e9dac92560 con 0x55e9dac0c760 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.120+0000 7fdd9affd640 1 -- 172.21.15.5:0/3819039457 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb00c4ec0 con 0x55e9dac0c760 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.120+0000 7fdd9affd640 1 -- 172.21.15.5:0/3819039457 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fddac004db0 con 0x55e9dac0c760 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.120+0000 7fddc4409740 10 client.26263 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.120+0000 7fddc4409740 10 client.26263 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.121+0000 7fddc4409740 20 client.26263 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.121+0000 7fddc4409740 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e9dac276c0 0x55e9dac94c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:41.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.121+0000 7fddc4409740 4 client.26263 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.7, "await_for": 45.0}] 2024-09-15T22:16:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.121+0000 7fddc4409740 1 -- 172.21.15.5:0/3819039457 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.7, "await_for": 45.0}) -- 0x55e9dab7f9d0 con 0x55e9dac276c0 2024-09-15T22:16:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.121+0000 7fddbaffd640 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e9dac276c0 0x55e9dac94c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.121+0000 7fdd9affd640 10 client.26263 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:41.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.121+0000 7fddbaffd640 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e9dac276c0 0x55e9dac94c20 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a889c0d9358383cb server_cookie=96f7ca55702ca39d in_seq=0 out_seq=0 2024-09-15T22:16:41.239 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:41.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fdd9affd640 1 -- 172.21.15.5:0/3819039457 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e9dab7f9d0 con 0x55e9dac276c0 2024-09-15T22:16:41.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fdd9affd640 10 client.26263 handle_command_reply: tid=0 2024-09-15T22:16:41.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fddc4409740 1 client.26263 shutdown 2024-09-15T22:16:41.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fddc4409740 1 -- 172.21.15.5:0/3819039457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 msgr2=0x55e9dac1f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:41.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fddc4409740 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac1f8e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fddac004770 tx=0x7fddac011df0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fddc4409740 1 -- 172.21.15.5:0/3819039457 shutdown_connections 2024-09-15T22:16:41.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fddc4409740 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55e9dac276c0 0x55e9dac94c20 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fddc4409740 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e9dac17940 0x55e9dac1c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fdd9affd640 0 client.26263 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:41.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fddc4409740 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9dac0d080 0x55e9dac1fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fddc4409740 1 --2- 172.21.15.5:0/3819039457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e9dac0c760 0x55e9dac1f8e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.539+0000 7fddc4409740 1 -- 172.21.15.5:0/3819039457 >> 172.21.15.5:0/3819039457 conn(0x55e9dac08cf0 msgr2=0x55e9dac22f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:41.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc4409740 1 -- 172.21.15.5:0/3819039457 shutdown_connections 2024-09-15T22:16:41.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc4409740 1 -- 172.21.15.5:0/3819039457 wait complete. 2024-09-15T22:16:41.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc4409740 20 client.26263 trim_cache size 0 max 16384 2024-09-15T22:16:41.544 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:16:41.544 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:16:41.544 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 26, 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "ee493997": { 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 26, 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 143.7, 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:16:41.545 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:16:41.546 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:16:41.546 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:16:41.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:16:41.546 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:16:41.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:41.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:41.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:41.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:41.546 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:41.546 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:16:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc2030640 1 -- 172.21.15.5:0/2471298006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdda40a3580 msgr2=0x7fdda40a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc2030640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdda40a3580 0x7fdda40a5a40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fddac009c90 tx=0x7fddac005f90 comp rx=0 tx=0).stop 2024-09-15T22:16:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc2030640 1 -- 172.21.15.5:0/2471298006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddbc1023e0 msgr2=0x7fddbc1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc2030640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddbc1023e0 0x7fddbc1989a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fddb0000c00 tx=0x7fddb00329e0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc2030640 1 -- 172.21.15.5:0/2471298006 shutdown_connections 2024-09-15T22:16:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc2030640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdda40a3580 0x7fdda40a5a40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc2030640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fddbc106fc0 0x7fddbc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc2030640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fddbc1023e0 0x7fddbc1989a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc2030640 1 --2- 172.21.15.5:0/2471298006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddbc101a30 0x7fddbc198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:41.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.540+0000 7fddc2030640 1 -- 172.21.15.5:0/2471298006 >> 172.21.15.5:0/2471298006 conn(0x7fddbc075f70 msgr2=0x7fddbc1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:41.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.541+0000 7fddc2030640 1 -- 172.21.15.5:0/2471298006 shutdown_connections 2024-09-15T22:16:41.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:41.541+0000 7fddc2030640 1 -- 172.21.15.5:0/2471298006 wait complete. 2024-09-15T22:16:41.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:41 smithi005 ceph-mon[30944]: pgmap v238: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-09-15T22:16:41.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:41 smithi099 ceph-mon[34892]: pgmap v238: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-09-15T22:16:41.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:41 smithi129 ceph-mon[35314]: pgmap v238: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-09-15T22:16:42.449 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ee493997', quiesce took 12.7 seconds. Will release after: 11.0 2024-09-15T22:16:42.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.705+0000 7f4d905b4640 1 -- 172.21.15.5:0/3269207958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d88100e20 msgr2=0x7f4d88101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:42.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.705+0000 7f4d905b4640 1 --2- 172.21.15.5:0/3269207958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d88100e20 0x7f4d88101200 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009900 tx=0x7f4d8402a480 comp rx=0 tx=0).stop 2024-09-15T22:16:42.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.705+0000 7f4d905b4640 1 -- 172.21.15.5:0/3269207958 shutdown_connections 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.705+0000 7f4d905b4640 1 --2- 172.21.15.5:0/3269207958 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d88102ef0 0x7f4d8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.705+0000 7f4d905b4640 1 --2- 172.21.15.5:0/3269207958 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d88102570 0x7f4d881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.705+0000 7f4d905b4640 1 --2- 172.21.15.5:0/3269207958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d88100e20 0x7f4d88101200 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.705+0000 7f4d905b4640 1 -- 172.21.15.5:0/3269207958 >> 172.21.15.5:0/3269207958 conn(0x7f4d880fc7d0 msgr2=0x7f4d880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.705+0000 7f4d905b4640 1 -- 172.21.15.5:0/3269207958 shutdown_connections 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.705+0000 7f4d905b4640 1 -- 172.21.15.5:0/3269207958 wait complete. 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d905b4640 1 Processor -- start 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d905b4640 1 -- start start 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d905b4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d88100e20 0x7f4d88197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d905b4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d88102570 0x7f4d881978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d905b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d88102ef0 0x7f4d8819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d905b4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88110f80 con 0x7f4d88102ef0 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d905b4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88110e00 con 0x7f4d88102570 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d905b4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88111100 con 0x7f4d88100e20 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d8eb51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d88102ef0 0x7f4d8819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d8db4f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d88102570 0x7f4d881978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d8eb51640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d88102ef0 0x7f4d8819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44586/0 (socket says 172.21.15.5:44586) 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.706+0000 7f4d8eb51640 1 -- 172.21.15.5:0/869162586 learned_addr learned my addr 172.21.15.5:0/869162586 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d8e350640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d88100e20 0x7f4d88197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d8eb51640 1 -- 172.21.15.5:0/869162586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d88100e20 msgr2=0x7f4d88197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d8eb51640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d88100e20 0x7f4d88197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d8eb51640 1 -- 172.21.15.5:0/869162586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d88102570 msgr2=0x7f4d881978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d8eb51640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d88102570 0x7f4d881978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d8eb51640 1 -- 172.21.15.5:0/869162586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8819c310 con 0x7f4d88102ef0 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d8eb51640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d88102ef0 0x7f4d8819bc30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000cce0 tx=0x7f4d8000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d7f7fe640 1 -- 172.21.15.5:0/869162586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d8000f450 con 0x7f4d88102ef0 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d7f7fe640 1 -- 172.21.15.5:0/869162586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d8000fad0 con 0x7f4d88102ef0 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d905b4640 1 -- 172.21.15.5:0/869162586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8819c600 con 0x7f4d88102ef0 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d7f7fe640 1 -- 172.21.15.5:0/869162586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80018c10 con 0x7f4d88102ef0 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.707+0000 7f4d905b4640 1 -- 172.21.15.5:0/869162586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d881a3ee0 con 0x7f4d88102ef0 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.710+0000 7f4d7f7fe640 1 -- 172.21.15.5:0/869162586 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4d8000f5f0 con 0x7f4d88102ef0 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.710+0000 7f4d9298d740 1 Processor -- start 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d9298d740 1 -- start start 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc54a0 0x55cce5cc5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5cd2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d9298d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cce5c2b2e0 con 0x55cce5cc5dc0 2024-09-15T22:16:42.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d9298d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cce5bd2ce0 con 0x55cce5cc54a0 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d9298d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cce5c272a0 con 0x55cce5cd0680 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d8e350640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d8e350640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44598/0 (socket says 172.21.15.5:44598) 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d8e350640 1 -- 172.21.15.5:0/566578185 learned_addr learned my addr 172.21.15.5:0/566578185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/566578185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc54a0 0x55cce5cc5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d7f7fe640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d740a33b0 0x7f4d740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.711+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/566578185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5cd2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d8eb51640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d740a33b0 0x7f4d740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d8db4f640 1 -- 172.21.15.5:0/566578185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 msgr2=0x55cce5cd2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/566578185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5cd2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d8db4f640 1 -- 172.21.15.5:0/566578185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 msgr2=0x55cce5cd0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/566578185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d8db4f640 1 -- 172.21.15.5:0/566578185 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cce5bb7440 con 0x55cce5cc54a0 2024-09-15T22:16:42.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d7f7fe640 1 -- 172.21.15.5:0/869162586 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f4d800d0760 con 0x7f4d88102ef0 2024-09-15T22:16:42.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d8e350640 1 --2- 172.21.15.5:0/566578185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:42.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/566578185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc54a0 0x55cce5cc5880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d78005d00 tx=0x7f4d78005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e7de45cfc7e2a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:42.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d8eb51640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d740a33b0 0x7f4d740a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4d800cdb60 tx=0x7f4d80051930 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:42.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d7cff9640 1 -- 172.21.15.5:0/566578185 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78005220 con 0x55cce5cc54a0 2024-09-15T22:16:42.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d7cff9640 1 -- 172.21.15.5:0/566578185 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d78004c80 con 0x55cce5cc54a0 2024-09-15T22:16:42.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.712+0000 7f4d7cff9640 1 -- 172.21.15.5:0/566578185 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d780109b0 con 0x55cce5cc54a0 2024-09-15T22:16:42.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.713+0000 7f4d9298d740 1 -- 172.21.15.5:0/566578185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc54a0 msgr2=0x55cce5cc5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.713+0000 7f4d9298d740 1 --2- 172.21.15.5:0/566578185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc54a0 0x55cce5cc5880 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4d78005d00 tx=0x7f4d78005c90 comp rx=0 tx=0).stop 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.713+0000 7f4d9298d740 1 -- 172.21.15.5:0/566578185 shutdown_connections 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.713+0000 7f4d9298d740 1 --2- 172.21.15.5:0/566578185 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5cd2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.713+0000 7f4d9298d740 1 --2- 172.21.15.5:0/566578185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.713+0000 7f4d9298d740 1 --2- 172.21.15.5:0/566578185 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc54a0 0x55cce5cc5880 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.713+0000 7f4d9298d740 1 -- 172.21.15.5:0/566578185 >> 172.21.15.5:0/566578185 conn(0x55cce5cc1a30 msgr2=0x55cce5cc1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.713+0000 7f4d9298d740 1 -- 172.21.15.5:0/566578185 shutdown_connections 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.713+0000 7f4d9298d740 1 -- 172.21.15.5:0/566578185 wait complete. 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.713+0000 7f4d9298d740 1 -- start start 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.713+0000 7f4d9298d740 1 Processor -- start 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cc54a0 0x55cce5ccffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5ccd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cd0680 0x55cce5ccd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d9298d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cce5cd5460 con 0x55cce5cc5dc0 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d9298d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cce5be6050 con 0x55cce5cd0680 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d9298d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cce5cd55e0 con 0x55cce5cc54a0 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d9298d740 10 client.0 _ll_register_callbacks cb 0x55cce5c26160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d8e350640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5ccd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d8e350640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5ccd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44604/0 (socket says 172.21.15.5:44604) 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d8e350640 1 -- 172.21.15.5:0/435365715 learned_addr learned my addr 172.21.15.5:0/435365715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cd0680 0x55cce5ccd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cc54a0 0x55cce5ccffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d8e350640 1 -- 172.21.15.5:0/435365715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cc54a0 msgr2=0x55cce5ccffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d8e350640 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cc54a0 0x55cce5ccffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d8e350640 1 -- 172.21.15.5:0/435365715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cd0680 msgr2=0x55cce5ccd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d8e350640 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cd0680 0x55cce5ccd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d8e350640 1 -- 172.21.15.5:0/435365715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cce599c3c0 con 0x55cce5cc5dc0 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.714+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cc54a0 0x55cce5ccffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d8e350640 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5ccd280 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009230 tx=0x7f4d840092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783b2128831112c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d71ffb640 1 -- 172.21.15.5:0/435365715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84035030 con 0x55cce5cc5dc0 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d71ffb640 1 -- 172.21.15.5:0/435365715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f4d84003c40 con 0x55cce5cc5dc0 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d9298d740 1 -- 172.21.15.5:0/435365715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d78004c80 con 0x55cce5cc5dc0 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d71ffb640 1 -- 172.21.15.5:0/435365715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d84003e40 con 0x55cce5cc5dc0 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d9298d740 10 client.16755 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d9298d740 1 -- 172.21.15.5:0/435365715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cce5ce0e30 con 0x55cce5cc5dc0 2024-09-15T22:16:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d71ffb640 1 -- 172.21.15.5:0/435365715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d78004c80 con 0x55cce5cc5dc0 2024-09-15T22:16:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d71ffb640 1 -- 172.21.15.5:0/435365715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4d8403c810 con 0x55cce5cc5dc0 2024-09-15T22:16:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d9298d740 10 client.16755 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.715+0000 7f4d9298d740 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.716+0000 7f4d9298d740 20 client.16755 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.716+0000 7f4d9298d740 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cce5ce22f0 0x55cce5ce46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.716+0000 7f4d9298d740 4 client.16755 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.716+0000 7f4d9298d740 1 -- 172.21.15.5:0/435365715 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cce5c38850 con 0x55cce5ce22f0 2024-09-15T22:16:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.716+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cce5ce22f0 0x55cce5ce46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.716+0000 7f4d71ffb640 10 client.16755 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:42.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.716+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cce5ce22f0 0x55cce5ce46e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ef535282d50d7 server_cookie=6bafcb5195dd30c3 in_seq=0 out_seq=0 2024-09-15T22:16:42.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d71ffb640 1 -- 172.21.15.5:0/435365715 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cce5c38850 con 0x55cce5ce22f0 2024-09-15T22:16:42.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d71ffb640 10 client.16755 handle_command_reply: tid=0 2024-09-15T22:16:42.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 1 client.16755 shutdown 2024-09-15T22:16:42.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 1 -- 172.21.15.5:0/435365715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 msgr2=0x55cce5ccd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:42.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5ccd280 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009230 tx=0x7f4d840092f0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 1 -- 172.21.15.5:0/435365715 shutdown_connections 2024-09-15T22:16:42.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cce5ce22f0 0x55cce5ce46e0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cd0680 0x55cce5ccd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d71ffb640 0 client.16755 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5ccd280 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 1 --2- 172.21.15.5:0/435365715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cc54a0 0x55cce5ccffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 1 -- 172.21.15.5:0/435365715 >> 172.21.15.5:0/435365715 conn(0x55cce5cc1a30 msgr2=0x55cce5cc7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 1 -- 172.21.15.5:0/435365715 shutdown_connections 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 1 -- 172.21.15.5:0/435365715 wait complete. 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.717+0000 7f4d9298d740 20 client.16755 trim_cache size 0 max 16384 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.735+0000 7f4d9298d740 1 Processor -- start 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.735+0000 7f4d9298d740 1 -- start start 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.735+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc54a0 0x55cce5ccd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.735+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc5dc0 0x55cce5cde6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.735+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5ce9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.735+0000 7f4d9298d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881af640 con 0x55cce5cc54a0 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.735+0000 7f4d9298d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881acf10 con 0x55cce5cc5dc0 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.735+0000 7f4d9298d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881aad80 con 0x55cce5cd0680 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.735+0000 7f4d8db4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc54a0 0x55cce5ccd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.735+0000 7f4d8db4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc54a0 0x55cce5ccd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44612/0 (socket says 172.21.15.5:44612) 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.735+0000 7f4d8db4f640 1 -- 172.21.15.5:0/3290754172 learned_addr learned my addr 172.21.15.5:0/3290754172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.736+0000 7f4d8e350640 1 --2- 172.21.15.5:0/3290754172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc5dc0 0x55cce5cde6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.736+0000 7f4d8db4f640 1 -- 172.21.15.5:0/3290754172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 msgr2=0x55cce5ce9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.736+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/3290754172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5ce9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.736+0000 7f4d8db4f640 1 -- 172.21.15.5:0/3290754172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc5dc0 msgr2=0x55cce5cde6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.736+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/3290754172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc5dc0 0x55cce5cde6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.736+0000 7f4d8db4f640 1 -- 172.21.15.5:0/3290754172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cce5ccc230 con 0x55cce5cc54a0 2024-09-15T22:16:42.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.736+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/3290754172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc54a0 0x55cce5ccd250 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4d78010540 tx=0x7f4d780104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f448e991ec9d06ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.736+0000 7f4d73fff640 1 -- 172.21.15.5:0/3290754172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d780049c0 con 0x55cce5cc54a0 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.736+0000 7f4d73fff640 1 -- 172.21.15.5:0/3290754172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d78021040 con 0x55cce5cc54a0 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.736+0000 7f4d73fff640 1 -- 172.21.15.5:0/3290754172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78010bd0 con 0x55cce5cc54a0 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.737+0000 7f4d9298d740 1 -- 172.21.15.5:0/3290754172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc54a0 msgr2=0x55cce5ccd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.737+0000 7f4d9298d740 1 --2- 172.21.15.5:0/3290754172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc54a0 0x55cce5ccd250 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f4d78010540 tx=0x7f4d780104d0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.737+0000 7f4d9298d740 1 -- 172.21.15.5:0/3290754172 shutdown_connections 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.737+0000 7f4d9298d740 1 --2- 172.21.15.5:0/3290754172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5ce9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.737+0000 7f4d9298d740 1 --2- 172.21.15.5:0/3290754172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc5dc0 0x55cce5cde6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.737+0000 7f4d9298d740 1 --2- 172.21.15.5:0/3290754172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc54a0 0x55cce5ccd250 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.737+0000 7f4d9298d740 1 -- 172.21.15.5:0/3290754172 >> 172.21.15.5:0/3290754172 conn(0x55cce5cc1a30 msgr2=0x55cce5cccb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.737+0000 7f4d9298d740 1 -- 172.21.15.5:0/3290754172 shutdown_connections 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.737+0000 7f4d9298d740 1 -- 172.21.15.5:0/3290754172 wait complete. 2024-09-15T22:16:42.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.737+0000 7f4d9298d740 1 -- start start 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.737+0000 7f4d9298d740 1 Processor -- start 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc54a0 0x55cce5cd6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d9298d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5cd7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d9298d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881ab3d0 con 0x55cce5cc5dc0 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d9298d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881ac840 con 0x55cce5cc54a0 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d9298d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881abaa0 con 0x55cce5cd0680 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d9298d740 10 client.0 _ll_register_callbacks cb 0x7f4d84004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8e350640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8e350640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44618/0 (socket says 172.21.15.5:44618) 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8e350640 1 -- 172.21.15.5:0/48118503 learned_addr learned my addr 172.21.15.5:0/48118503 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:42.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc54a0 0x55cce5cd6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5cd7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8e350640 1 -- 172.21.15.5:0/48118503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 msgr2=0x55cce5cd7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8e350640 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5cd7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8e350640 1 -- 172.21.15.5:0/48118503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc54a0 msgr2=0x55cce5cd6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8e350640 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc54a0 0x55cce5cd6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8e350640 1 -- 172.21.15.5:0/48118503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cce5ce4a50 con 0x55cce5cc5dc0 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5cd7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d8e350640 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd6d40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009850 tx=0x7f4d8402ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc5e71ce07a0c2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.738+0000 7f4d71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d71ffb640 1 -- 172.21.15.5:0/48118503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84035070 con 0x55cce5cc5dc0 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d71ffb640 1 -- 172.21.15.5:0/48118503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f4d8403c650 con 0x55cce5cc5dc0 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d9298d740 1 -- 172.21.15.5:0/48118503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d78021040 con 0x55cce5cc5dc0 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d71ffb640 1 -- 172.21.15.5:0/48118503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4d8403c870 con 0x55cce5cc5dc0 2024-09-15T22:16:42.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d9298d740 10 client.16761 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:42.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d9298d740 1 -- 172.21.15.5:0/48118503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cce5cd0340 con 0x55cce5cc5dc0 2024-09-15T22:16:42.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d71ffb640 1 -- 172.21.15.5:0/48118503 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d78021040 con 0x55cce5cc5dc0 2024-09-15T22:16:42.763 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:16:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:16:42.763 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:16:42.763 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:16:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d71ffb640 1 -- 172.21.15.5:0/48118503 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4d84004b80 con 0x55cce5cc5dc0 2024-09-15T22:16:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d9298d740 10 client.16761 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d9298d740 10 client.16761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d9298d740 20 client.16761 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d9298d740 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cce5ce22f0 0x55cce5d4f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d9298d740 4 client.16761 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:16:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.739+0000 7f4d9298d740 1 -- 172.21.15.5:0/48118503 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cce5c32d60 con 0x55cce5ce22f0 2024-09-15T22:16:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cce5ce22f0 0x55cce5d4f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:42.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d71ffb640 10 client.16761 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d8db4f640 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cce5ce22f0 0x55cce5d4f8d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a184b8099ce723 server_cookie=8e11845707cb236b in_seq=0 out_seq=0 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d71ffb640 1 -- 172.21.15.5:0/48118503 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cce5c32d60 con 0x55cce5ce22f0 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d71ffb640 10 client.16761 handle_command_reply: tid=0 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d9298d740 1 client.16761 shutdown 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d9298d740 1 -- 172.21.15.5:0/48118503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 msgr2=0x55cce5cd6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d9298d740 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd6d40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f4d84009850 tx=0x7f4d8402ac00 comp rx=0 tx=0).stop 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d9298d740 1 -- 172.21.15.5:0/48118503 shutdown_connections 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d9298d740 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cce5cd0680 0x55cce5cd7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d9298d740 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cce5cc5dc0 0x55cce5cd6d40 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d9298d740 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cce5ce22f0 0x55cce5d4f8d0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d9298d740 1 --2- 172.21.15.5:0/48118503 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cce5cc54a0 0x55cce5cd6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d71ffb640 0 client.16761 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:42.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.740+0000 7f4d9298d740 1 -- 172.21.15.5:0/48118503 >> 172.21.15.5:0/48118503 conn(0x55cce5cc1a30 msgr2=0x55cce5ccf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d9298d740 1 -- 172.21.15.5:0/48118503 shutdown_connections 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d9298d740 1 -- 172.21.15.5:0/48118503 wait complete. 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d9298d740 20 client.16761 trim_cache size 0 max 16384 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 -- 172.21.15.5:0/869162586 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d740a33b0 msgr2=0x7f4d740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d740a33b0 0x7f4d740a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4d800cdb60 tx=0x7f4d80051930 comp rx=0 tx=0).stop 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 -- 172.21.15.5:0/869162586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d88102ef0 msgr2=0x7f4d8819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d88102ef0 0x7f4d8819bc30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4d8000cce0 tx=0x7f4d8000a620 comp rx=0 tx=0).stop 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 -- 172.21.15.5:0/869162586 shutdown_connections 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d740a33b0 0x7f4d740a5870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d88102ef0 0x7f4d8819bc30 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d88102570 0x7f4d881978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 --2- 172.21.15.5:0/869162586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d88100e20 0x7f4d88197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 -- 172.21.15.5:0/869162586 >> 172.21.15.5:0/869162586 conn(0x7f4d880fc7d0 msgr2=0x7f4d880fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:42.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 -- 172.21.15.5:0/869162586 shutdown_connections 2024-09-15T22:16:42.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:42.741+0000 7f4d905b4640 1 -- 172.21.15.5:0/869162586 wait complete. 2024-09-15T22:16:43.422 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:16:43.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:16:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:43.812 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:43 smithi005 ceph-mon[30944]: pgmap v239: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-09-15T22:16:43.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:43 smithi099 ceph-mon[34892]: pgmap v239: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-09-15T22:16:43.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:43 smithi129 ceph-mon[35314]: pgmap v239: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 12 MiB/s wr, 1.18k op/s 2024-09-15T22:16:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.269+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/1626215382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c94106f10 msgr2=0x7f4c9410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.269+0000 7f4c9c6f4640 1 --2- 172.21.15.5:0/1626215382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c94106f10 0x7f4c9410ea90 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009ca0 tx=0x7f4c8c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:16:44.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.269+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/1626215382 shutdown_connections 2024-09-15T22:16:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.269+0000 7f4c9c6f4640 1 --2- 172.21.15.5:0/1626215382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c94106f10 0x7f4c9410ea90 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.269+0000 7f4c9c6f4640 1 --2- 172.21.15.5:0/1626215382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c94102790 0x7f4c941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.269+0000 7f4c9c6f4640 1 --2- 172.21.15.5:0/1626215382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94101de0 0x7f4c941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.269+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/1626215382 >> 172.21.15.5:0/1626215382 conn(0x7f4c940fd8c0 msgr2=0x7f4c940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.269+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/1626215382 shutdown_connections 2024-09-15T22:16:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.269+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/1626215382 wait complete. 2024-09-15T22:16:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9c6f4640 1 Processor -- start 2024-09-15T22:16:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9c6f4640 1 -- start start 2024-09-15T22:16:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9c6f4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c94101de0 0x7f4c94198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9c6f4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94102790 0x7f4c941987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:44.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9c6f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c94106f10 0x7f4c9419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9c6f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c94111f70 con 0x7f4c94106f10 2024-09-15T22:16:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9c6f4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4c94111df0 con 0x7f4c94102790 2024-09-15T22:16:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9c6f4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4c941120f0 con 0x7f4c94101de0 2024-09-15T22:16:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9ac91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c94106f10 0x7f4c9419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9ac91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c94106f10 0x7f4c9419cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54804/0 (socket says 172.21.15.5:54804) 2024-09-15T22:16:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9ac91640 1 -- 172.21.15.5:0/2675799528 learned_addr learned my addr 172.21.15.5:0/2675799528 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c99c8f640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94102790 0x7f4c941987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.270+0000 7f4c9a490640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c94101de0 0x7f4c94198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.271+0000 7f4c9ac91640 1 -- 172.21.15.5:0/2675799528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c94101de0 msgr2=0x7f4c94198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.271+0000 7f4c9ac91640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c94101de0 0x7f4c94198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:44.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.271+0000 7f4c9ac91640 1 -- 172.21.15.5:0/2675799528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94102790 msgr2=0x7f4c941987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.271+0000 7f4c9ac91640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94102790 0x7f4c941987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.271+0000 7f4c9ac91640 1 -- 172.21.15.5:0/2675799528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9419d230 con 0x7f4c94106f10 2024-09-15T22:16:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.271+0000 7f4c9ac91640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c94106f10 0x7f4c9419cb50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c031a60 tx=0x7f4c8c031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.271+0000 7f4c8b7fe640 1 -- 172.21.15.5:0/2675799528 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c003b70 con 0x7f4c94106f10 2024-09-15T22:16:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.271+0000 7f4c8b7fe640 1 -- 172.21.15.5:0/2675799528 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c8c003d10 con 0x7f4c94106f10 2024-09-15T22:16:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.271+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/2675799528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9419d4c0 con 0x7f4c94106f10 2024-09-15T22:16:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.271+0000 7f4c8b7fe640 1 -- 172.21.15.5:0/2675799528 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c004770 con 0x7f4c94106f10 2024-09-15T22:16:44.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.271+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/2675799528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c941a4e00 con 0x7f4c94106f10 2024-09-15T22:16:44.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.272+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/2675799528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c941021c0 con 0x7f4c94106f10 2024-09-15T22:16:44.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.273+0000 7f4c8b7fe640 1 -- 172.21.15.5:0/2675799528 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4c8c02f790 con 0x7f4c94106f10 2024-09-15T22:16:44.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.274+0000 7f4c8b7fe640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4c800a33b0 0x7f4c800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:44.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.274+0000 7f4c8b7fe640 1 -- 172.21.15.5:0/2675799528 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f4c8c0f1490 con 0x7f4c94106f10 2024-09-15T22:16:44.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.274+0000 7f4c9a490640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4c800a33b0 0x7f4c800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:44.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.275+0000 7f4c9a490640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4c800a33b0 0x7f4c800a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000abe0 tx=0x7f4c9000a2d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:44.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.277+0000 7f4c8b7fe640 1 -- 172.21.15.5:0/2675799528 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c8c0bdb20 con 0x7f4c94106f10 2024-09-15T22:16:44.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.488+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/2675799528 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c941a5000 con 0x7f4c94106f10 2024-09-15T22:16:44.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.489+0000 7f4c8b7fe640 1 -- 172.21.15.5:0/2675799528 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f4c8c0bd420 con 0x7f4c94106f10 2024-09-15T22:16:44.492 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:16:44.492 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:16:44.492 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:16:44.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/2675799528 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4c800a33b0 msgr2=0x7f4c800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:44.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4c800a33b0 0x7f4c800a5870 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c9000abe0 tx=0x7f4c9000a2d0 comp rx=0 tx=0).stop 2024-09-15T22:16:44.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/2675799528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c94106f10 msgr2=0x7f4c9419cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:44.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c94106f10 0x7f4c9419cb50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c031a60 tx=0x7f4c8c031a90 comp rx=0 tx=0).stop 2024-09-15T22:16:44.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/2675799528 shutdown_connections 2024-09-15T22:16:44.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4c800a33b0 0x7f4c800a5870 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:44.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c94106f10 0x7f4c9419cb50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:44.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4c94102790 0x7f4c941987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:44.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 --2- 172.21.15.5:0/2675799528 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4c94101de0 0x7f4c94198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:44.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/2675799528 >> 172.21.15.5:0/2675799528 conn(0x7f4c940fd8c0 msgr2=0x7f4c940fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:44.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/2675799528 shutdown_connections 2024-09-15T22:16:44.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:44.491+0000 7f4c9c6f4640 1 -- 172.21.15.5:0/2675799528 wait complete. 2024-09-15T22:16:44.664 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 435.0 s, 121.7 tps, lat 259.894 ms stddev 295.906 2024-09-15T22:16:45.125 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:16:45.391 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:45.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:45 smithi005 ceph-mon[30944]: pgmap v240: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.6 MiB/s wr, 969 op/s 2024-09-15T22:16:45.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:45 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2675799528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:45.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:45 smithi099 ceph-mon[34892]: pgmap v240: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.6 MiB/s wr, 969 op/s 2024-09-15T22:16:45.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:45 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2675799528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:45 smithi129 ceph-mon[35314]: pgmap v240: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 9.6 MiB/s wr, 969 op/s 2024-09-15T22:16:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:45 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2675799528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:45.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.914+0000 7fa633c57640 1 -- 172.21.15.5:0/911393685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa62c106a80 msgr2=0x7fa62c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:45.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.914+0000 7fa633c57640 1 --2- 172.21.15.5:0/911393685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa62c106a80 0x7fa62c106e60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fa620009930 tx=0x7fa62002a420 comp rx=0 tx=0).stop 2024-09-15T22:16:45.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.914+0000 7fa633c57640 1 -- 172.21.15.5:0/911393685 shutdown_connections 2024-09-15T22:16:45.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.914+0000 7fa633c57640 1 --2- 172.21.15.5:0/911393685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa62c101420 0x7fa62c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.914+0000 7fa633c57640 1 --2- 172.21.15.5:0/911393685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa62c100a80 0x7fa62c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.914+0000 7fa633c57640 1 --2- 172.21.15.5:0/911393685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa62c106a80 0x7fa62c106e60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.914+0000 7fa633c57640 1 -- 172.21.15.5:0/911393685 >> 172.21.15.5:0/911393685 conn(0x7fa62c0fc7a0 msgr2=0x7fa62c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:45.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.914+0000 7fa633c57640 1 -- 172.21.15.5:0/911393685 shutdown_connections 2024-09-15T22:16:45.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa633c57640 1 -- 172.21.15.5:0/911393685 wait complete. 2024-09-15T22:16:45.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa633c57640 1 Processor -- start 2024-09-15T22:16:45.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa633c57640 1 -- start start 2024-09-15T22:16:45.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa633c57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa62c100a80 0x7fa62c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa633c57640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa62c101420 0x7fa62c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa633c57640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa62c106a80 0x7fa62c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa633c57640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c110e80 con 0x7fa62c100a80 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa633c57640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c110d00 con 0x7fa62c106a80 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa633c57640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c111000 con 0x7fa62c101420 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa6319f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa62c100a80 0x7fa62c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa6319f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa62c100a80 0x7fa62c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54828/0 (socket says 172.21.15.5:54828) 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.915+0000 7fa6319f3640 1 -- 172.21.15.5:0/955394869 learned_addr learned my addr 172.21.15.5:0/955394869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa6311f2640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa62c101420 0x7fa62c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa6321f4640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa62c106a80 0x7fa62c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa6319f3640 1 -- 172.21.15.5:0/955394869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa62c101420 msgr2=0x7fa62c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa6319f3640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa62c101420 0x7fa62c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa6319f3640 1 -- 172.21.15.5:0/955394869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa62c106a80 msgr2=0x7fa62c19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa6319f3640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa62c106a80 0x7fa62c19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa6319f3640 1 -- 172.21.15.5:0/955394869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c19d410 con 0x7fa62c100a80 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa6319f3640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa62c100a80 0x7fa62c198460 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa620009230 tx=0x7fa620004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa61effd640 1 -- 172.21.15.5:0/955394869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62003f030 con 0x7fa62c100a80 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa61effd640 1 -- 172.21.15.5:0/955394869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa620003ad0 con 0x7fa62c100a80 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa633c57640 1 -- 172.21.15.5:0/955394869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c19d6a0 con 0x7fa62c100a80 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa61effd640 1 -- 172.21.15.5:0/955394869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620035070 con 0x7fa62c100a80 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.916+0000 7fa633c57640 1 -- 172.21.15.5:0/955394869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c1a4fe0 con 0x7fa62c100a80 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.919+0000 7fa61effd640 1 -- 172.21.15.5:0/955394869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa620042030 con 0x7fa62c100a80 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.919+0000 7fa636030740 1 Processor -- start 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.919+0000 7fa636030740 1 -- start start 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.919+0000 7fa636030740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a92090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.919+0000 7fa636030740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a925d0 0x560a45a92a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa636030740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a9d370 0x560a45a9f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa636030740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560a459f82f0 con 0x560a45a93b40 2024-09-15T22:16:45.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa636030740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560a4599fcf0 con 0x560a45a925d0 2024-09-15T22:16:45.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa636030740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560a459f42b0 con 0x560a45a9d370 2024-09-15T22:16:45.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6321f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a92090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6321f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a92090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54830/0 (socket says 172.21.15.5:54830) 2024-09-15T22:16:45.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6321f4640 1 -- 172.21.15.5:0/4156358798 learned_addr learned my addr 172.21.15.5:0/4156358798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:45.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa61effd640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6180a33b0 0x7fa6180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6319f3640 1 --2- 172.21.15.5:0/4156358798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a925d0 0x560a45a92a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6311f2640 1 --2- 172.21.15.5:0/4156358798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a9d370 0x560a45a9f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6321f4640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6180a33b0 0x7fa6180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6321f4640 1 -- 172.21.15.5:0/4156358798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a9d370 msgr2=0x560a45a9f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6321f4640 1 --2- 172.21.15.5:0/4156358798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a9d370 0x560a45a9f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6321f4640 1 -- 172.21.15.5:0/4156358798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a925d0 msgr2=0x560a45a92a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6321f4640 1 --2- 172.21.15.5:0/4156358798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a925d0 0x560a45a92a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6321f4640 1 -- 172.21.15.5:0/4156358798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a459843a0 con 0x560a45a93b40 2024-09-15T22:16:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6311f2640 1 --2- 172.21.15.5:0/4156358798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a9d370 0x560a45a9f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa61effd640 1 -- 172.21.15.5:0/955394869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fa6200f04d0 con 0x7fa62c100a80 2024-09-15T22:16:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.920+0000 7fa6321f4640 1 --2- 172.21.15.5:0/4156358798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a92090 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa624014a60 tx=0x7fa624014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c27971445ce8ba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:45.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa617fff640 1 -- 172.21.15.5:0/4156358798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62401ec50 con 0x560a45a93b40 2024-09-15T22:16:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa617fff640 1 -- 172.21.15.5:0/4156358798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa624023070 con 0x560a45a93b40 2024-09-15T22:16:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa617fff640 1 -- 172.21.15.5:0/4156358798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62400e360 con 0x560a45a93b40 2024-09-15T22:16:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa6321f4640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6180a33b0 0x7fa6180a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa624022000 tx=0x7fa62401e000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa636030740 1 -- 172.21.15.5:0/4156358798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 msgr2=0x560a45a92090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa636030740 1 --2- 172.21.15.5:0/4156358798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a92090 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa624014a60 tx=0x7fa624014f20 comp rx=0 tx=0).stop 2024-09-15T22:16:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa636030740 1 -- 172.21.15.5:0/4156358798 shutdown_connections 2024-09-15T22:16:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa636030740 1 --2- 172.21.15.5:0/4156358798 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a9d370 0x560a45a9f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa636030740 1 --2- 172.21.15.5:0/4156358798 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a925d0 0x560a45a92a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa636030740 1 --2- 172.21.15.5:0/4156358798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a92090 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa636030740 1 -- 172.21.15.5:0/4156358798 >> 172.21.15.5:0/4156358798 conn(0x560a45a5b310 msgr2=0x560a45a8ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.921+0000 7fa636030740 1 -- 172.21.15.5:0/4156358798 shutdown_connections 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.922+0000 7fa636030740 1 -- 172.21.15.5:0/4156358798 wait complete. 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.922+0000 7fa636030740 1 -- start start 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.922+0000 7fa636030740 1 Processor -- start 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.922+0000 7fa636030740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a925d0 0x560a45a9cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.922+0000 7fa636030740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a93b40 0x560a45a9a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.922+0000 7fa636030740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a9d370 0x560a45a9a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.922+0000 7fa636030740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560a45aa0070 con 0x560a45a9d370 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.922+0000 7fa636030740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560a459b3060 con 0x560a45a93b40 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.922+0000 7fa636030740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560a45aa01f0 con 0x560a45a925d0 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa636030740 10 client.0 _ll_register_callbacks cb 0x560a457068c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6321f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a9d370 0x560a45a9a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6321f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a9d370 0x560a45a9a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54834/0 (socket says 172.21.15.5:54834) 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6321f4640 1 -- 172.21.15.5:0/1355633145 learned_addr learned my addr 172.21.15.5:0/1355633145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6319f3640 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a925d0 0x560a45a9cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6311f2640 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a93b40 0x560a45a9a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6321f4640 1 -- 172.21.15.5:0/1355633145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a925d0 msgr2=0x560a45a9cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6321f4640 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a925d0 0x560a45a9cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6321f4640 1 -- 172.21.15.5:0/1355633145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a93b40 msgr2=0x560a45a9a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6321f4640 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a93b40 0x560a45a9a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6321f4640 1 -- 172.21.15.5:0/1355633145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a457693c0 con 0x560a45a9d370 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6321f4640 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a9d370 0x560a45a9a600 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa624009d30 tx=0x7fa62401df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d96c04cf1fcc653 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6157fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6157fa640 1 -- 172.21.15.5:0/1355633145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624036060 con 0x560a45a9d370 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6157fa640 1 -- 172.21.15.5:0/1355633145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fa62401ec50 con 0x560a45a9d370 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa636030740 1 -- 172.21.15.5:0/1355633145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa624023070 con 0x560a45a9d370 2024-09-15T22:16:45.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa6157fa640 1 -- 172.21.15.5:0/1355633145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa62403d050 con 0x560a45a9d370 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa636030740 10 client.16779 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.923+0000 7fa636030740 1 -- 172.21.15.5:0/1355633145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a45aabb10 con 0x560a45a9d370 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.924+0000 7fa6157fa640 1 -- 172.21.15.5:0/1355633145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa624023070 con 0x560a45a9d370 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.924+0000 7fa6157fa640 1 -- 172.21.15.5:0/1355633145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa62400f040 con 0x560a45a9d370 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.924+0000 7fa636030740 10 client.16779 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.924+0000 7fa636030740 10 client.16779 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.924+0000 7fa636030740 20 client.16779 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.924+0000 7fa636030740 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x560a45aacf70 0x560a45aaf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.924+0000 7fa636030740 4 client.16779 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.924+0000 7fa636030740 1 -- 172.21.15.5:0/1355633145 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a45a05860 con 0x560a45aacf70 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.924+0000 7fa6319f3640 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x560a45aacf70 0x560a45aaf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.925+0000 7fa6157fa640 10 client.16779 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:16:45.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.925+0000 7fa6319f3640 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x560a45aacf70 0x560a45aaf360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa275e56f4ab9ce1 server_cookie=1f61e768abb80132 in_seq=0 out_seq=0 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.925+0000 7fa6157fa640 1 -- 172.21.15.5:0/1355633145 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a45a05860 con 0x560a45aacf70 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.925+0000 7fa6157fa640 10 client.16779 handle_command_reply: tid=0 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.925+0000 7fa636030740 1 client.16779 shutdown 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa636030740 1 -- 172.21.15.5:0/1355633145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a9d370 msgr2=0x560a45a9a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa636030740 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a9d370 0x560a45a9a600 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fa624009d30 tx=0x7fa62401df30 comp rx=0 tx=0).stop 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa636030740 1 -- 172.21.15.5:0/1355633145 shutdown_connections 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa636030740 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x560a45aacf70 0x560a45aaf360 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa636030740 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a9d370 0x560a45a9a600 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa636030740 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a93b40 0x560a45a9a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa6157fa640 0 client.16779 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa636030740 1 --2- 172.21.15.5:0/1355633145 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a925d0 0x560a45a9cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa636030740 1 -- 172.21.15.5:0/1355633145 >> 172.21.15.5:0/1355633145 conn(0x560a45a5b310 msgr2=0x560a45a94960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa636030740 1 -- 172.21.15.5:0/1355633145 shutdown_connections 2024-09-15T22:16:45.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa636030740 1 -- 172.21.15.5:0/1355633145 wait complete. 2024-09-15T22:16:45.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.926+0000 7fa636030740 20 client.16779 trim_cache size 0 max 16384 2024-09-15T22:16:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.939+0000 7fa636030740 1 Processor -- start 2024-09-15T22:16:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.939+0000 7fa636030740 1 -- start start 2024-09-15T22:16:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.939+0000 7fa636030740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a925d0 0x560a45a9bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.939+0000 7fa636030740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a93b40 0x560a45a95120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.939+0000 7fa636030740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a9d370 0x560a45a95660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.939+0000 7fa636030740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1b1a30 con 0x560a45a925d0 2024-09-15T22:16:45.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.939+0000 7fa636030740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1af300 con 0x560a45a93b40 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.939+0000 7fa636030740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1ad170 con 0x560a45a9d370 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.939+0000 7fa6319f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a925d0 0x560a45a9bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6319f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a925d0 0x560a45a9bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54838/0 (socket says 172.21.15.5:54838) 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6319f3640 1 -- 172.21.15.5:0/2135313114 learned_addr learned my addr 172.21.15.5:0/2135313114 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6321f4640 1 --2- 172.21.15.5:0/2135313114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a9d370 0x560a45a95660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6311f2640 1 --2- 172.21.15.5:0/2135313114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a93b40 0x560a45a95120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6319f3640 1 -- 172.21.15.5:0/2135313114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a9d370 msgr2=0x560a45a95660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6319f3640 1 --2- 172.21.15.5:0/2135313114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a9d370 0x560a45a95660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6319f3640 1 -- 172.21.15.5:0/2135313114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a93b40 msgr2=0x560a45a95120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6319f3640 1 --2- 172.21.15.5:0/2135313114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a93b40 0x560a45a95120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6319f3640 1 -- 172.21.15.5:0/2135313114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a45a985c0 con 0x560a45a925d0 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6319f3640 1 --2- 172.21.15.5:0/2135313114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a925d0 0x560a45a9bf80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa620009a50 tx=0x7fa6200f2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e3f86550027a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6177fe640 1 -- 172.21.15.5:0/2135313114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620049040 con 0x560a45a925d0 2024-09-15T22:16:45.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6177fe640 1 -- 172.21.15.5:0/2135313114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6200bc500 con 0x560a45a925d0 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.940+0000 7fa6177fe640 1 -- 172.21.15.5:0/2135313114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6200edb40 con 0x560a45a925d0 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.941+0000 7fa636030740 1 -- 172.21.15.5:0/2135313114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a925d0 msgr2=0x560a45a9bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.941+0000 7fa636030740 1 --2- 172.21.15.5:0/2135313114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a925d0 0x560a45a9bf80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa620009a50 tx=0x7fa6200f2040 comp rx=0 tx=0).stop 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.941+0000 7fa636030740 1 -- 172.21.15.5:0/2135313114 shutdown_connections 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.941+0000 7fa636030740 1 --2- 172.21.15.5:0/2135313114 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a9d370 0x560a45a95660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.941+0000 7fa636030740 1 --2- 172.21.15.5:0/2135313114 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a93b40 0x560a45a95120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.941+0000 7fa636030740 1 --2- 172.21.15.5:0/2135313114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a925d0 0x560a45a9bf80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.941+0000 7fa636030740 1 -- 172.21.15.5:0/2135313114 >> 172.21.15.5:0/2135313114 conn(0x560a45a5b310 msgr2=0x560a45a9b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.941+0000 7fa636030740 1 -- 172.21.15.5:0/2135313114 shutdown_connections 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.941+0000 7fa636030740 1 -- 172.21.15.5:0/2135313114 wait complete. 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.941+0000 7fa636030740 1 -- start start 2024-09-15T22:16:45.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.941+0000 7fa636030740 1 Processor -- start 2024-09-15T22:16:45.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa636030740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a925d0 0x560a45aa5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa636030740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a96ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa636030740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a9d370 0x560a45a97670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa636030740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1ad7c0 con 0x560a45a93b40 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa636030740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1aec30 con 0x560a45a9d370 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa636030740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1ade90 con 0x560a45a925d0 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa636030740 10 client.0 _ll_register_callbacks cb 0x7fa6200b8b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6311f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a96ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6311f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a96ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54840/0 (socket says 172.21.15.5:54840) 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6311f2640 1 -- 172.21.15.5:0/1482240226 learned_addr learned my addr 172.21.15.5:0/1482240226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6321f4640 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a9d370 0x560a45a97670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6319f3640 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a925d0 0x560a45aa5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6311f2640 1 -- 172.21.15.5:0/1482240226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a925d0 msgr2=0x560a45aa5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6311f2640 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a925d0 0x560a45aa5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6311f2640 1 -- 172.21.15.5:0/1482240226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a9d370 msgr2=0x560a45a97670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6311f2640 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a9d370 0x560a45a97670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6311f2640 1 -- 172.21.15.5:0/1482240226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a45b16990 con 0x560a45a93b40 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6321f4640 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a9d370 0x560a45a97670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6319f3640 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a925d0 0x560a45aa5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6311f2640 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a96ee0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x560a45a93070 tx=0x7fa62800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bea68372516f53e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.942+0000 7fa6157fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa6157fa640 1 -- 172.21.15.5:0/1482240226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628005020 con 0x560a45a93b40 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa6157fa640 1 -- 172.21.15.5:0/1482240226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fa6280051c0 con 0x560a45a93b40 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa636030740 1 -- 172.21.15.5:0/1482240226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6200bc500 con 0x560a45a93b40 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa6157fa640 1 -- 172.21.15.5:0/1482240226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa628026050 con 0x560a45a93b40 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa636030740 10 client.16785 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa636030740 1 -- 172.21.15.5:0/1482240226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560a45b17b70 con 0x560a45a93b40 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa6157fa640 1 -- 172.21.15.5:0/1482240226 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6200bc500 con 0x560a45a93b40 2024-09-15T22:16:45.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa6157fa640 1 -- 172.21.15.5:0/1482240226 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa62801a070 con 0x560a45a93b40 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa636030740 10 client.16785 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa636030740 10 client.16785 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa636030740 20 client.16785 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa636030740 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x560a45b183a0 0x560a45b1a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa636030740 4 client.16785 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.943+0000 7fa636030740 1 -- 172.21.15.5:0/1482240226 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a459ffd70 con 0x560a45b183a0 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa6319f3640 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x560a45b183a0 0x560a45b1a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa6319f3640 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x560a45b183a0 0x560a45b1a7c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76a31a9241c067c9 server_cookie=35990054c96651d9 in_seq=0 out_seq=0 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa6157fa640 10 client.16785 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa6157fa640 1 -- 172.21.15.5:0/1482240226 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a459ffd70 con 0x560a45b183a0 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa6157fa640 10 client.16785 handle_command_reply: tid=0 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa636030740 1 client.16785 shutdown 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa636030740 1 -- 172.21.15.5:0/1482240226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 msgr2=0x560a45a96ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:45.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa636030740 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a96ee0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x560a45a93070 tx=0x7fa62800be80 comp rx=0 tx=0).stop 2024-09-15T22:16:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa636030740 1 -- 172.21.15.5:0/1482240226 shutdown_connections 2024-09-15T22:16:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa636030740 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x560a45b183a0 0x560a45b1a7c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa636030740 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560a45a9d370 0x560a45a97670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa636030740 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a45a93b40 0x560a45a96ee0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa6157fa640 0 client.16785 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:16:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa636030740 1 --2- 172.21.15.5:0/1482240226 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560a45a925d0 0x560a45aa5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa636030740 1 -- 172.21.15.5:0/1482240226 >> 172.21.15.5:0/1482240226 conn(0x560a45a5b310 msgr2=0x560a45a9c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa636030740 1 -- 172.21.15.5:0/1482240226 shutdown_connections 2024-09-15T22:16:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.944+0000 7fa636030740 1 -- 172.21.15.5:0/1482240226 wait complete. 2024-09-15T22:16:45.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa636030740 20 client.16785 trim_cache size 0 max 16384 2024-09-15T22:16:45.953 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:16:45.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 -- 172.21.15.5:0/955394869 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6180a33b0 msgr2=0x7fa6180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:45.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6180a33b0 0x7fa6180a5870 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa624022000 tx=0x7fa62401e000 comp rx=0 tx=0).stop 2024-09-15T22:16:45.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 -- 172.21.15.5:0/955394869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa62c100a80 msgr2=0x7fa62c198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:45.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa62c100a80 0x7fa62c198460 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa620009230 tx=0x7fa620004270 comp rx=0 tx=0).stop 2024-09-15T22:16:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 -- 172.21.15.5:0/955394869 shutdown_connections 2024-09-15T22:16:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa6180a33b0 0x7fa6180a5870 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa62c106a80 0x7fa62c19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa62c101420 0x7fa62c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 --2- 172.21.15.5:0/955394869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa62c100a80 0x7fa62c198460 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 -- 172.21.15.5:0/955394869 >> 172.21.15.5:0/955394869 conn(0x7fa62c0fc7a0 msgr2=0x7fa62c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 -- 172.21.15.5:0/955394869 shutdown_connections 2024-09-15T22:16:45.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:45.945+0000 7fa633c57640 1 -- 172.21.15.5:0/955394869 wait complete. 2024-09-15T22:16:46.741 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:16:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:47.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.472+0000 7f507b2ed640 1 -- 172.21.15.5:0/2710439538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074100a80 msgr2=0x7f5074100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.472+0000 7f507b2ed640 1 --2- 172.21.15.5:0/2710439538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074100a80 0x7f5074100ee0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5064009930 tx=0x7f506402a420 comp rx=0 tx=0).stop 2024-09-15T22:16:47.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.472+0000 7f507b2ed640 1 -- 172.21.15.5:0/2710439538 shutdown_connections 2024-09-15T22:16:47.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.472+0000 7f507b2ed640 1 --2- 172.21.15.5:0/2710439538 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5074101420 0x7f507410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.472+0000 7f507b2ed640 1 --2- 172.21.15.5:0/2710439538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074100a80 0x7f5074100ee0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.472+0000 7f507b2ed640 1 --2- 172.21.15.5:0/2710439538 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5074106a80 0x7f5074106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.472+0000 7f507b2ed640 1 -- 172.21.15.5:0/2710439538 >> 172.21.15.5:0/2710439538 conn(0x7f50740fc7a0 msgr2=0x7f50740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:47.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.472+0000 7f507b2ed640 1 -- 172.21.15.5:0/2710439538 shutdown_connections 2024-09-15T22:16:47.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.472+0000 7f507b2ed640 1 -- 172.21.15.5:0/2710439538 wait complete. 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.473+0000 7f507b2ed640 1 Processor -- start 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.473+0000 7f507b2ed640 1 -- start start 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.473+0000 7f507b2ed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074100a80 0x7f5074197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.473+0000 7f507b2ed640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5074101420 0x7f5074197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.473+0000 7f507b2ed640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5074106a80 0x7f507419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.473+0000 7f507b2ed640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5074110e80 con 0x7f5074100a80 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.473+0000 7f507b2ed640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5074110d00 con 0x7f5074106a80 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.473+0000 7f507b2ed640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5074111000 con 0x7f5074101420 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f5079089640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074100a80 0x7f5074197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f5079089640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074100a80 0x7f5074197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54854/0 (socket says 172.21.15.5:54854) 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f5079089640 1 -- 172.21.15.5:0/239250631 learned_addr learned my addr 172.21.15.5:0/239250631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f5078888640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5074101420 0x7f5074197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f507988a640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5074106a80 0x7f507419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f5079089640 1 -- 172.21.15.5:0/239250631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5074101420 msgr2=0x7f5074197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f5079089640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5074101420 0x7f5074197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f5079089640 1 -- 172.21.15.5:0/239250631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5074106a80 msgr2=0x7f507419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f5079089640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5074106a80 0x7f507419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f5079089640 1 -- 172.21.15.5:0/239250631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f507419c2b0 con 0x7f5074100a80 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f5079089640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074100a80 0x7f5074197300 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f507000d770 tx=0x7f507000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f506a7fc640 1 -- 172.21.15.5:0/239250631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070017070 con 0x7f5074100a80 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f506a7fc640 1 -- 172.21.15.5:0/239250631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5070005500 con 0x7f5074100a80 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f507b2ed640 1 -- 172.21.15.5:0/239250631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f507419c5a0 con 0x7f5074100a80 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f506a7fc640 1 -- 172.21.15.5:0/239250631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070004e00 con 0x7f5074100a80 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.474+0000 7f507b2ed640 1 -- 172.21.15.5:0/239250631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50741a3e80 con 0x7f5074100a80 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.477+0000 7f506a7fc640 1 -- 172.21.15.5:0/239250631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5070004fa0 con 0x7f5074100a80 2024-09-15T22:16:47.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.477+0000 7f507d6c6740 1 Processor -- start 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.477+0000 7f507d6c6740 1 -- start start 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.477+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcdb40 0x555c23bcc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.477+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23bcca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.477+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bdb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f507d6c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c23b322f0 con 0x555c23bd9580 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f507d6c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555c23ad9cf0 con 0x555c23bcc5d0 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f507d6c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555c23b2e2b0 con 0x555c23bcdb40 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f5078888640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bdb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f5078888640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bdb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54860/0 (socket says 172.21.15.5:54860) 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f5078888640 1 -- 172.21.15.5:0/765244217 learned_addr learned my addr 172.21.15.5:0/765244217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f507988a640 1 --2- 172.21.15.5:0/765244217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcdb40 0x555c23bcc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f506a7fc640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f50600a33b0 0x7f50600a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f5079089640 1 --2- 172.21.15.5:0/765244217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23bcca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f507988a640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f50600a33b0 0x7f50600a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f506a7fc640 1 -- 172.21.15.5:0/239250631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f50700cfa30 con 0x7f5074100a80 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f5078888640 1 -- 172.21.15.5:0/765244217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcdb40 msgr2=0x555c23bcc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f5078888640 1 --2- 172.21.15.5:0/765244217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcdb40 0x555c23bcc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f5078888640 1 -- 172.21.15.5:0/765244217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 msgr2=0x555c23bcca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f5078888640 1 --2- 172.21.15.5:0/765244217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23bcca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f5078888640 1 -- 172.21.15.5:0/765244217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c23abe3a0 con 0x555c23bd9580 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f5078888640 1 --2- 172.21.15.5:0/765244217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bdb970 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5064009930 tx=0x7f5064031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bd95b9046c3c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.478+0000 7f507988a640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f50600a33b0 0x7f50600a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f506c007310 tx=0x7f506c004ab0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f505f7fe640 1 -- 172.21.15.5:0/765244217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064040070 con 0x555c23bd9580 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f505f7fe640 1 -- 172.21.15.5:0/765244217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5064005020 con 0x555c23bd9580 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f505f7fe640 1 -- 172.21.15.5:0/765244217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064035040 con 0x555c23bd9580 2024-09-15T22:16:47.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f507d6c6740 1 -- 172.21.15.5:0/765244217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 msgr2=0x555c23bdb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f507d6c6740 1 --2- 172.21.15.5:0/765244217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bdb970 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5064009930 tx=0x7f5064031630 comp rx=0 tx=0).stop 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f507d6c6740 1 -- 172.21.15.5:0/765244217 shutdown_connections 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f507d6c6740 1 --2- 172.21.15.5:0/765244217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bdb970 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f507d6c6740 1 --2- 172.21.15.5:0/765244217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23bcca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f507d6c6740 1 --2- 172.21.15.5:0/765244217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcdb40 0x555c23bcc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f507d6c6740 1 -- 172.21.15.5:0/765244217 >> 172.21.15.5:0/765244217 conn(0x555c23b95310 msgr2=0x555c23bc8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f507d6c6740 1 -- 172.21.15.5:0/765244217 shutdown_connections 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.479+0000 7f507d6c6740 1 -- 172.21.15.5:0/765244217 wait complete. 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507d6c6740 1 -- start start 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507d6c6740 1 Processor -- start 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23bd8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcdb40 0x555c23bd6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bd6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507d6c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c23bdc220 con 0x555c23bd9580 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507d6c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555c23aed060 con 0x555c23bcc5d0 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507d6c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555c23bdc3a0 con 0x555c23bcdb40 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507d6c6740 10 client.0 _ll_register_callbacks cb 0x555c238408c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507988a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bd6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507988a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bd6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54868/0 (socket says 172.21.15.5:54868) 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f507988a640 1 -- 172.21.15.5:0/3458606156 learned_addr learned my addr 172.21.15.5:0/3458606156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.480+0000 7f5078888640 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcdb40 0x555c23bd6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f5079089640 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23bd8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f507988a640 1 -- 172.21.15.5:0/3458606156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcdb40 msgr2=0x555c23bd6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f507988a640 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcdb40 0x555c23bd6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f507988a640 1 -- 172.21.15.5:0/3458606156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 msgr2=0x555c23bd8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:47.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f507988a640 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23bd8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f507988a640 1 -- 172.21.15.5:0/3458606156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c238a33c0 con 0x555c23bd9580 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f5079089640 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23bd8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f507988a640 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bd6660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f506c0135b0 tx=0x7f506c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=119ac221c813d78a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f505cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f505cff9640 1 -- 172.21.15.5:0/3458606156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c025070 con 0x555c23bd9580 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f507d6c6740 1 -- 172.21.15.5:0/3458606156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5064005020 con 0x555c23bd9580 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f505cff9640 1 -- 172.21.15.5:0/3458606156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f506c011440 con 0x555c23bd9580 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f505cff9640 1 -- 172.21.15.5:0/3458606156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f506c013020 con 0x555c23bd9580 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f507d6c6740 10 client.16797 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.481+0000 7f507d6c6740 1 -- 172.21.15.5:0/3458606156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c23be7c80 con 0x555c23bd9580 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.482+0000 7f505cff9640 1 -- 172.21.15.5:0/3458606156 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5064005020 con 0x555c23bd9580 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.482+0000 7f505cff9640 1 -- 172.21.15.5:0/3458606156 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f506c012450 con 0x555c23bd9580 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.482+0000 7f507d6c6740 10 client.16797 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:47.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.482+0000 7f507d6c6740 10 client.16797 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:16:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.482+0000 7f507d6c6740 20 client.16797 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.482+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555c23be9090 0x555c23beb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.482+0000 7f507d6c6740 4 client.16797 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.482+0000 7f507d6c6740 1 -- 172.21.15.5:0/3458606156 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c23b3f860 con 0x555c23be9090 2024-09-15T22:16:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.482+0000 7f5079089640 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555c23be9090 0x555c23beb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f5079089640 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555c23be9090 0x555c23beb4e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ef4dc1e3f6b9db9 server_cookie=7da29558836d3e32 in_seq=0 out_seq=0 2024-09-15T22:16:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f505cff9640 10 client.16797 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:16:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f505cff9640 1 -- 172.21.15.5:0/3458606156 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c23b3f860 con 0x555c23be9090 2024-09-15T22:16:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f505cff9640 10 client.16797 handle_command_reply: tid=0 2024-09-15T22:16:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f507d6c6740 1 client.16797 shutdown 2024-09-15T22:16:47.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f507d6c6740 1 -- 172.21.15.5:0/3458606156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 msgr2=0x555c23bd6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bd6660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f506c0135b0 tx=0x7f506c004a40 comp rx=0 tx=0).stop 2024-09-15T22:16:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f507d6c6740 1 -- 172.21.15.5:0/3458606156 shutdown_connections 2024-09-15T22:16:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555c23be9090 0x555c23beb4e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bd9580 0x555c23bd6660 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcdb40 0x555c23bd6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3458606156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23bd8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.483+0000 7f505cff9640 0 client.16797 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:16:47.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.484+0000 7f507d6c6740 1 -- 172.21.15.5:0/3458606156 >> 172.21.15.5:0/3458606156 conn(0x555c23b95310 msgr2=0x555c23bce960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:47.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.484+0000 7f507d6c6740 1 -- 172.21.15.5:0/3458606156 shutdown_connections 2024-09-15T22:16:47.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.484+0000 7f507d6c6740 1 -- 172.21.15.5:0/3458606156 wait complete. 2024-09-15T22:16:47.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.484+0000 7f507d6c6740 20 client.16797 trim_cache size 0 max 16384 2024-09-15T22:16:47.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f507d6c6740 1 Processor -- start 2024-09-15T22:16:47.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f507d6c6740 1 -- start start 2024-09-15T22:16:47.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcc5d0 0x555c23be54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bda970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bd9580 0x555c23bdaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f507d6c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50741af730 con 0x555c23bcdb40 2024-09-15T22:16:47.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f507d6c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50741ad000 con 0x555c23bd9580 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f507d6c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50741aae70 con 0x555c23bcc5d0 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f5078888640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bda970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f5078888640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bda970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54880/0 (socket says 172.21.15.5:54880) 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f5078888640 1 -- 172.21.15.5:0/1365016042 learned_addr learned my addr 172.21.15.5:0/1365016042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.497+0000 7f5079089640 1 --2- 172.21.15.5:0/1365016042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcc5d0 0x555c23be54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.498+0000 7f507988a640 1 --2- 172.21.15.5:0/1365016042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bd9580 0x555c23bdaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.498+0000 7f5078888640 1 -- 172.21.15.5:0/1365016042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcc5d0 msgr2=0x555c23be54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.498+0000 7f5078888640 1 --2- 172.21.15.5:0/1365016042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcc5d0 0x555c23be54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.498+0000 7f5078888640 1 -- 172.21.15.5:0/1365016042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bd9580 msgr2=0x555c23bdaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.498+0000 7f5078888640 1 --2- 172.21.15.5:0/1365016042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bd9580 0x555c23bdaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.498+0000 7f5078888640 1 -- 172.21.15.5:0/1365016042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c23bc9960 con 0x555c23bcdb40 2024-09-15T22:16:47.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.498+0000 7f5078888640 1 --2- 172.21.15.5:0/1365016042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bda970 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x555c23bcd070 tx=0x7f506402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc01d4bd487c0ae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.498+0000 7f505effd640 1 -- 172.21.15.5:0/1365016042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064052070 con 0x555c23bcdb40 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.498+0000 7f505effd640 1 -- 172.21.15.5:0/1365016042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f506403c920 con 0x555c23bcdb40 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.498+0000 7f505effd640 1 -- 172.21.15.5:0/1365016042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064035070 con 0x555c23bcdb40 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.499+0000 7f507d6c6740 1 -- 172.21.15.5:0/1365016042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 msgr2=0x555c23bda970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.499+0000 7f507d6c6740 1 --2- 172.21.15.5:0/1365016042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bda970 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x555c23bcd070 tx=0x7f506402a420 comp rx=0 tx=0).stop 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.499+0000 7f507d6c6740 1 -- 172.21.15.5:0/1365016042 shutdown_connections 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.499+0000 7f507d6c6740 1 --2- 172.21.15.5:0/1365016042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bd9580 0x555c23bdaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.499+0000 7f507d6c6740 1 --2- 172.21.15.5:0/1365016042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bda970 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.499+0000 7f507d6c6740 1 --2- 172.21.15.5:0/1365016042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bcc5d0 0x555c23be54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.499+0000 7f507d6c6740 1 -- 172.21.15.5:0/1365016042 >> 172.21.15.5:0/1365016042 conn(0x555c23b95310 msgr2=0x555c23bd60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.499+0000 7f507d6c6740 1 -- 172.21.15.5:0/1365016042 shutdown_connections 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.499+0000 7f507d6c6740 1 -- 172.21.15.5:0/1365016042 wait complete. 2024-09-15T22:16:47.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.499+0000 7f507d6c6740 1 -- start start 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.499+0000 7f507d6c6740 1 Processor -- start 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23be1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bf0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f507d6c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bd9580 0x555c23bf0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f507d6c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50741ab4c0 con 0x555c23bcdb40 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f507d6c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50741ac930 con 0x555c23bcc5d0 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f507d6c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50741abb90 con 0x555c23bd9580 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f507d6c6740 10 client.0 _ll_register_callbacks cb 0x7f506c013e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f5078888640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bf0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f5078888640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bf0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54896/0 (socket says 172.21.15.5:54896) 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f5078888640 1 -- 172.21.15.5:0/3662723126 learned_addr learned my addr 172.21.15.5:0/3662723126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f5079089640 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23be1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f507988a640 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bd9580 0x555c23bf0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f5078888640 1 -- 172.21.15.5:0/3662723126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bd9580 msgr2=0x555c23bf0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:47.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f5078888640 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bd9580 0x555c23bf0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f5078888640 1 -- 172.21.15.5:0/3662723126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 msgr2=0x555c23be1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:47.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f5078888640 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23be1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.500+0000 7f5078888640 1 -- 172.21.15.5:0/3662723126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c23bc8ed0 con 0x555c23bcdb40 2024-09-15T22:16:47.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f5078888640 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bf0840 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f506402a940 tx=0x7f50640046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f641267e88f183 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:47.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f505cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:47.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f505cff9640 1 -- 172.21.15.5:0/3662723126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5064035070 con 0x555c23bcdb40 2024-09-15T22:16:47.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f505cff9640 1 -- 172.21.15.5:0/3662723126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f506403ccd0 con 0x555c23bcdb40 2024-09-15T22:16:47.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f507d6c6740 1 -- 172.21.15.5:0/3662723126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f506403c920 con 0x555c23bcdb40 2024-09-15T22:16:47.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f505cff9640 1 -- 172.21.15.5:0/3662723126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5064060050 con 0x555c23bcdb40 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f507d6c6740 10 client.16803 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f507d6c6740 1 -- 172.21.15.5:0/3662723126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c23c53d80 con 0x555c23bcdb40 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f505cff9640 1 -- 172.21.15.5:0/3662723126 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f506403c920 con 0x555c23bcdb40 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f505cff9640 1 -- 172.21.15.5:0/3662723126 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5064055020 con 0x555c23bcdb40 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f507d6c6740 10 client.16803 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f507d6c6740 10 client.16803 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f507d6c6740 20 client.16803 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555c23c54000 0x555c23c563f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f507d6c6740 4 client.16803 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.501+0000 7f507d6c6740 1 -- 172.21.15.5:0/3662723126 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c23b39d70 con 0x555c23c54000 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f5079089640 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555c23c54000 0x555c23c563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f5079089640 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555c23c54000 0x555c23c563f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e99e4da209801899 server_cookie=caae10e3a8f8aca6 in_seq=0 out_seq=0 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f505cff9640 10 client.16803 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:16:47.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f505cff9640 1 -- 172.21.15.5:0/3662723126 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c23b39d70 con 0x555c23c54000 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f505cff9640 10 client.16803 handle_command_reply: tid=0 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f507d6c6740 1 client.16803 shutdown 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f507d6c6740 1 -- 172.21.15.5:0/3662723126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 msgr2=0x555c23bf0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bf0840 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f506402a940 tx=0x7f50640046e0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f507d6c6740 1 -- 172.21.15.5:0/3662723126 shutdown_connections 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x555c23c54000 0x555c23c563f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c23bd9580 0x555c23bf0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c23bcdb40 0x555c23bf0840 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f507d6c6740 1 --2- 172.21.15.5:0/3662723126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c23bcc5d0 0x555c23be1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f507d6c6740 1 -- 172.21.15.5:0/3662723126 >> 172.21.15.5:0/3662723126 conn(0x555c23b95310 msgr2=0x555c23bd45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.502+0000 7f505cff9640 0 client.16803 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507d6c6740 1 -- 172.21.15.5:0/3662723126 shutdown_connections 2024-09-15T22:16:47.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507d6c6740 1 -- 172.21.15.5:0/3662723126 wait complete. 2024-09-15T22:16:47.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507d6c6740 20 client.16803 trim_cache size 0 max 16384 2024-09-15T22:16:47.513 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:16:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507b2ed640 1 -- 172.21.15.5:0/239250631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f50600a33b0 msgr2=0x7f50600a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507b2ed640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f50600a33b0 0x7f50600a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f506c007310 tx=0x7f506c004ab0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507b2ed640 1 -- 172.21.15.5:0/239250631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074100a80 msgr2=0x7f5074197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507b2ed640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074100a80 0x7f5074197300 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f507000d770 tx=0x7f507000dc30 comp rx=0 tx=0).stop 2024-09-15T22:16:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507b2ed640 1 -- 172.21.15.5:0/239250631 shutdown_connections 2024-09-15T22:16:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507b2ed640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f50600a33b0 0x7f50600a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507b2ed640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5074106a80 0x7f507419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507b2ed640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5074101420 0x7f5074197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507b2ed640 1 --2- 172.21.15.5:0/239250631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5074100a80 0x7f5074197300 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.503+0000 7f507b2ed640 1 -- 172.21.15.5:0/239250631 >> 172.21.15.5:0/239250631 conn(0x7f50740fc7a0 msgr2=0x7f50740fcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.504+0000 7f507b2ed640 1 -- 172.21.15.5:0/239250631 shutdown_connections 2024-09-15T22:16:47.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:47.504+0000 7f507b2ed640 1 -- 172.21.15.5:0/239250631 wait complete. 2024-09-15T22:16:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:47 smithi099 ceph-mon[34892]: pgmap v241: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.2 MiB/s wr, 931 op/s 2024-09-15T22:16:47.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:47 smithi129 ceph-mon[35314]: pgmap v241: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.2 MiB/s wr, 931 op/s 2024-09-15T22:16:47.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:47 smithi005 ceph-mon[30944]: pgmap v241: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 9.2 MiB/s wr, 931 op/s 2024-09-15T22:16:48.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:16:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.066+0000 7f929a912640 1 -- 172.21.15.5:0/3758435600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294106f30 msgr2=0x7f929410eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.066+0000 7f929a912640 1 --2- 172.21.15.5:0/3758435600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294106f30 0x7f929410eab0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f928c009ca0 tx=0x7f928c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.066+0000 7f929a912640 1 -- 172.21.15.5:0/3758435600 shutdown_connections 2024-09-15T22:16:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.066+0000 7f929a912640 1 --2- 172.21.15.5:0/3758435600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294106f30 0x7f929410eab0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.066+0000 7f929a912640 1 --2- 172.21.15.5:0/3758435600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9294100cc0 0x7f9294106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.066+0000 7f929a912640 1 --2- 172.21.15.5:0/3758435600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9294100310 0x7f92941006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.066+0000 7f929a912640 1 -- 172.21.15.5:0/3758435600 >> 172.21.15.5:0/3758435600 conn(0x7f9294077210 msgr2=0x7f92940fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.066+0000 7f929a912640 1 -- 172.21.15.5:0/3758435600 shutdown_connections 2024-09-15T22:16:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.066+0000 7f929a912640 1 -- 172.21.15.5:0/3758435600 wait complete. 2024-09-15T22:16:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f929a912640 1 Processor -- start 2024-09-15T22:16:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f929a912640 1 -- start start 2024-09-15T22:16:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f929a912640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294100310 0x7f92941983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f929a912640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9294100cc0 0x7f92941988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f929a912640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9294106f30 0x7f929419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f929a912640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9294111f90 con 0x7f9294100310 2024-09-15T22:16:49.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f929a912640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9294111e10 con 0x7f9294106f30 2024-09-15T22:16:49.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f929a912640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9294112110 con 0x7f9294100cc0 2024-09-15T22:16:49.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f9293fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294100310 0x7f92941983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f9293fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294100310 0x7f92941983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54912/0 (socket says 172.21.15.5:54912) 2024-09-15T22:16:49.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f9293fff640 1 -- 172.21.15.5:0/901984279 learned_addr learned my addr 172.21.15.5:0/901984279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:49.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f9298eaf640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9294106f30 0x7f929419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f928bfff640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9294100cc0 0x7f92941988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f9293fff640 1 -- 172.21.15.5:0/901984279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9294100cc0 msgr2=0x7f92941988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f9293fff640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9294100cc0 0x7f92941988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f9293fff640 1 -- 172.21.15.5:0/901984279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9294106f30 msgr2=0x7f929419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f9293fff640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9294106f30 0x7f929419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f9293fff640 1 -- 172.21.15.5:0/901984279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929419d360 con 0x7f9294100310 2024-09-15T22:16:49.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f928bfff640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9294100cc0 0x7f92941988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:49.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f9298eaf640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9294106f30 0x7f929419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:49.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.067+0000 7f9293fff640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294100310 0x7f92941983b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f928400d950 tx=0x7f928400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.068+0000 7f9291ffb640 1 -- 172.21.15.5:0/901984279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9284017030 con 0x7f9294100310 2024-09-15T22:16:49.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.068+0000 7f9291ffb640 1 -- 172.21.15.5:0/901984279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9284004020 con 0x7f9294100310 2024-09-15T22:16:49.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.068+0000 7f929a912640 1 -- 172.21.15.5:0/901984279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f929419d650 con 0x7f9294100310 2024-09-15T22:16:49.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.068+0000 7f9291ffb640 1 -- 172.21.15.5:0/901984279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92840047a0 con 0x7f9294100310 2024-09-15T22:16:49.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.068+0000 7f929a912640 1 -- 172.21.15.5:0/901984279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92941a4f30 con 0x7f9294100310 2024-09-15T22:16:49.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.070+0000 7f929cceb740 1 Processor -- start 2024-09-15T22:16:49.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.070+0000 7f9291ffb640 1 -- 172.21.15.5:0/901984279 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f92840041d0 con 0x7f9294100310 2024-09-15T22:16:49.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.070+0000 7f929cceb740 1 -- start start 2024-09-15T22:16:49.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f929cceb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762cb40 0x55b22762b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f929cceb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b22762b5d0 0x55b22762ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f929cceb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227636370 0x55b227638760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f929cceb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b2275912f0 con 0x55b227636370 2024-09-15T22:16:49.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f929cceb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b227538cf0 con 0x55b22762b5d0 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f929cceb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b22758d2b0 con 0x55b22762cb40 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f928bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227636370 0x55b227638760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f928bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227636370 0x55b227638760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54916/0 (socket says 172.21.15.5:54916) 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f928bfff640 1 -- 172.21.15.5:0/1918784622 learned_addr learned my addr 172.21.15.5:0/1918784622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f9293fff640 1 --2- 172.21.15.5:0/1918784622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b22762b5d0 0x55b22762ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f9298eaf640 1 --2- 172.21.15.5:0/1918784622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762cb40 0x55b22762b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f928bfff640 1 -- 172.21.15.5:0/1918784622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762cb40 msgr2=0x55b22762b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f928bfff640 1 --2- 172.21.15.5:0/1918784622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762cb40 0x55b22762b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f928bfff640 1 -- 172.21.15.5:0/1918784622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b22762b5d0 msgr2=0x55b22762ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f928bfff640 1 --2- 172.21.15.5:0/1918784622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b22762b5d0 0x55b22762ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f9291ffb640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f92800a33b0 0x7f92800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f928bfff640 1 -- 172.21.15.5:0/1918784622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b22751d3a0 con 0x55b227636370 2024-09-15T22:16:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f9298eaf640 1 --2- 172.21.15.5:0/1918784622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762cb40 0x55b22762b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f9291ffb640 1 -- 172.21.15.5:0/901984279 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f92840cfbe0 con 0x7f9294100310 2024-09-15T22:16:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f9298eaf640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f92800a33b0 0x7f92800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.071+0000 7f928bfff640 1 --2- 172.21.15.5:0/1918784622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227636370 0x55b227638760 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f927c009c50 tx=0x7f927c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29036c969b8f145f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.072+0000 7f928a7fc640 1 -- 172.21.15.5:0/1918784622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c004030 con 0x55b227636370 2024-09-15T22:16:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.072+0000 7f928a7fc640 1 -- 172.21.15.5:0/1918784622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f927c0041d0 con 0x55b227636370 2024-09-15T22:16:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.072+0000 7f928a7fc640 1 -- 172.21.15.5:0/1918784622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c012740 con 0x55b227636370 2024-09-15T22:16:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.072+0000 7f929cceb740 1 -- 172.21.15.5:0/1918784622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227636370 msgr2=0x55b227638760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.072+0000 7f9298eaf640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f92800a33b0 0x7f92800a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f928c005cc0 tx=0x7f928c005f80 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.072+0000 7f929cceb740 1 --2- 172.21.15.5:0/1918784622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227636370 0x55b227638760 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f927c009c50 tx=0x7f927c010740 comp rx=0 tx=0).stop 2024-09-15T22:16:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.072+0000 7f929cceb740 1 -- 172.21.15.5:0/1918784622 shutdown_connections 2024-09-15T22:16:49.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.072+0000 7f929cceb740 1 --2- 172.21.15.5:0/1918784622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b227636370 0x55b227638760 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.072+0000 7f929cceb740 1 --2- 172.21.15.5:0/1918784622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b22762b5d0 0x55b22762ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 --2- 172.21.15.5:0/1918784622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762cb40 0x55b22762b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 -- 172.21.15.5:0/1918784622 >> 172.21.15.5:0/1918784622 conn(0x55b2275f4310 msgr2=0x55b227627c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 -- 172.21.15.5:0/1918784622 shutdown_connections 2024-09-15T22:16:49.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 -- 172.21.15.5:0/1918784622 wait complete. 2024-09-15T22:16:49.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 -- start start 2024-09-15T22:16:49.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 Processor -- start 2024-09-15T22:16:49.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762b5d0 0x55b227635d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762cb40 0x55b227633020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b227636370 0x55b227633610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b227639070 con 0x55b22762cb40 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b22754c060 con 0x55b227636370 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b2276391f0 con 0x55b22762b5d0 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.073+0000 7f929cceb740 10 client.0 _ll_register_callbacks cb 0x55b22729f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f928bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762cb40 0x55b227633020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f928bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762cb40 0x55b227633020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54930/0 (socket says 172.21.15.5:54930) 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f928bfff640 1 -- 172.21.15.5:0/559172133 learned_addr learned my addr 172.21.15.5:0/559172133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f9298eaf640 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b227636370 0x55b227633610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f9293fff640 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762b5d0 0x55b227635d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f928bfff640 1 -- 172.21.15.5:0/559172133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762b5d0 msgr2=0x55b227635d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f928bfff640 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762b5d0 0x55b227635d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f928bfff640 1 -- 172.21.15.5:0/559172133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b227636370 msgr2=0x55b227633610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f928bfff640 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b227636370 0x55b227633610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f928bfff640 1 -- 172.21.15.5:0/559172133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2273023c0 con 0x55b22762cb40 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f9293fff640 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762b5d0 0x55b227635d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f928bfff640 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762cb40 0x55b227633020 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f927c003fa0 tx=0x7f927c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9badcb5b6f4f062e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f92737fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f92737fe640 1 -- 172.21.15.5:0/559172133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c004380 con 0x55b22762cb40 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f92737fe640 1 -- 172.21.15.5:0/559172133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f927c009d40 con 0x55b22762cb40 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f929cceb740 1 -- 172.21.15.5:0/559172133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f927c0041d0 con 0x55b22762cb40 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f92737fe640 1 -- 172.21.15.5:0/559172133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f927c02f050 con 0x55b22762cb40 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f929cceb740 10 client.16815 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.074+0000 7f929cceb740 1 -- 172.21.15.5:0/559172133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b227644a80 con 0x55b22762cb40 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.075+0000 7f92737fe640 1 -- 172.21.15.5:0/559172133 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f927c0041d0 con 0x55b22762cb40 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.075+0000 7f92737fe640 1 -- 172.21.15.5:0/559172133 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f927c00f040 con 0x55b22762cb40 2024-09-15T22:16:49.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.075+0000 7f929cceb740 10 client.16815 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.075+0000 7f929cceb740 10 client.16815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.075+0000 7f929cceb740 20 client.16815 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.075+0000 7f929cceb740 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b227645dd0 0x55b2276481c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.075+0000 7f929cceb740 4 client.16815 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.075+0000 7f929cceb740 1 -- 172.21.15.5:0/559172133 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b22759e860 con 0x55b227645dd0 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.075+0000 7f9293fff640 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b227645dd0 0x55b2276481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.076+0000 7f9293fff640 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b227645dd0 0x55b2276481c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f9abaac5041a407 server_cookie=69d07edbc730b2bb in_seq=0 out_seq=0 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.076+0000 7f92737fe640 10 client.16815 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.076+0000 7f92737fe640 1 -- 172.21.15.5:0/559172133 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b22759e860 con 0x55b227645dd0 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.076+0000 7f92737fe640 10 client.16815 handle_command_reply: tid=0 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.076+0000 7f929cceb740 1 client.16815 shutdown 2024-09-15T22:16:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f929cceb740 1 -- 172.21.15.5:0/559172133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762cb40 msgr2=0x55b227633020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f929cceb740 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762cb40 0x55b227633020 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f927c003fa0 tx=0x7f927c005db0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f929cceb740 1 -- 172.21.15.5:0/559172133 shutdown_connections 2024-09-15T22:16:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f929cceb740 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b227645dd0 0x55b2276481c0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f929cceb740 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b227636370 0x55b227633610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f929cceb740 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762cb40 0x55b227633020 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f92737fe640 0 client.16815 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f929cceb740 1 --2- 172.21.15.5:0/559172133 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b22762b5d0 0x55b227635d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f929cceb740 1 -- 172.21.15.5:0/559172133 >> 172.21.15.5:0/559172133 conn(0x55b2275f4310 msgr2=0x55b22762d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f929cceb740 1 -- 172.21.15.5:0/559172133 shutdown_connections 2024-09-15T22:16:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f929cceb740 1 -- 172.21.15.5:0/559172133 wait complete. 2024-09-15T22:16:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.077+0000 7f929cceb740 20 client.16815 trim_cache size 0 max 16384 2024-09-15T22:16:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f929cceb740 1 Processor -- start 2024-09-15T22:16:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f929cceb740 1 -- start start 2024-09-15T22:16:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f929cceb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b22762b5d0 0x55b227642310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f929cceb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b227636370 0x55b227637760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f929cceb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762ea20 0x55b227637ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f929cceb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92941b1b00 con 0x55b22762ea20 2024-09-15T22:16:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f929cceb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92941af3d0 con 0x55b22762b5d0 2024-09-15T22:16:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f929cceb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f92941ad240 con 0x55b227636370 2024-09-15T22:16:49.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f9298eaf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762ea20 0x55b227637ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f9298eaf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762ea20 0x55b227637ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54932/0 (socket says 172.21.15.5:54932) 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f9298eaf640 1 -- 172.21.15.5:0/4214100909 learned_addr learned my addr 172.21.15.5:0/4214100909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f928bfff640 1 --2- 172.21.15.5:0/4214100909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b227636370 0x55b227637760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.092+0000 7f9293fff640 1 --2- 172.21.15.5:0/4214100909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b22762b5d0 0x55b227642310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.093+0000 7f9298eaf640 1 -- 172.21.15.5:0/4214100909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b227636370 msgr2=0x55b227637760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.093+0000 7f9298eaf640 1 --2- 172.21.15.5:0/4214100909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b227636370 0x55b227637760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.093+0000 7f9298eaf640 1 -- 172.21.15.5:0/4214100909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b22762b5d0 msgr2=0x55b227642310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.093+0000 7f9298eaf640 1 --2- 172.21.15.5:0/4214100909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b22762b5d0 0x55b227642310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.093+0000 7f9298eaf640 1 -- 172.21.15.5:0/4214100909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2276381e0 con 0x55b22762ea20 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.093+0000 7f9298eaf640 1 --2- 172.21.15.5:0/4214100909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762ea20 0x55b227637ca0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f928c031e10 tx=0x7f928c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eded5eff3e408b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.093+0000 7f9289ffb640 1 -- 172.21.15.5:0/4214100909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c050070 con 0x55b22762ea20 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.093+0000 7f9289ffb640 1 -- 172.21.15.5:0/4214100909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f928c036040 con 0x55b22762ea20 2024-09-15T22:16:49.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.093+0000 7f9289ffb640 1 -- 172.21.15.5:0/4214100909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c02faa0 con 0x55b22762ea20 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 -- 172.21.15.5:0/4214100909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762ea20 msgr2=0x55b227637ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 --2- 172.21.15.5:0/4214100909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762ea20 0x55b227637ca0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f928c031e10 tx=0x7f928c03f000 comp rx=0 tx=0).stop 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 -- 172.21.15.5:0/4214100909 shutdown_connections 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 --2- 172.21.15.5:0/4214100909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762ea20 0x55b227637ca0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 --2- 172.21.15.5:0/4214100909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b227636370 0x55b227637760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 --2- 172.21.15.5:0/4214100909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b22762b5d0 0x55b227642310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 -- 172.21.15.5:0/4214100909 >> 172.21.15.5:0/4214100909 conn(0x55b2275f4310 msgr2=0x55b22762e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 -- 172.21.15.5:0/4214100909 shutdown_connections 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 -- 172.21.15.5:0/4214100909 wait complete. 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 -- start start 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 Processor -- start 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762b5d0 0x55b22763e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b227636370 0x55b2276303f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.094+0000 7f929cceb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b227630930 0x55b22762f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f929cceb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92941ad890 con 0x55b22762b5d0 2024-09-15T22:16:49.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f929cceb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f92941aed00 con 0x55b227630930 2024-09-15T22:16:49.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f929cceb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f92941adf60 con 0x55b227636370 2024-09-15T22:16:49.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f929cceb740 10 client.0 _ll_register_callbacks cb 0x7f927c009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:49.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f9293fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762b5d0 0x55b22763e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f9293fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762b5d0 0x55b22763e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54936/0 (socket says 172.21.15.5:54936) 2024-09-15T22:16:49.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f9293fff640 1 -- 172.21.15.5:0/86035785 learned_addr learned my addr 172.21.15.5:0/86035785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f928bfff640 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b227636370 0x55b2276303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f9298eaf640 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b227630930 0x55b22762f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f9293fff640 1 -- 172.21.15.5:0/86035785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b227636370 msgr2=0x55b2276303f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f9293fff640 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b227636370 0x55b2276303f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f9293fff640 1 -- 172.21.15.5:0/86035785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b227630930 msgr2=0x55b22762f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f9293fff640 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b227630930 0x55b22762f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f9293fff640 1 -- 172.21.15.5:0/86035785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b227630db0 con 0x55b22762b5d0 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f9298eaf640 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b227630930 0x55b22762f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f9293fff640 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762b5d0 0x55b22763e960 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9284098490 tx=0x7f928409b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee068cc8282c830 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f92737fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f92737fe640 1 -- 172.21.15.5:0/86035785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928409bac0 con 0x55b22762b5d0 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f92737fe640 1 -- 172.21.15.5:0/86035785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f92840cd1b0 con 0x55b22762b5d0 2024-09-15T22:16:49.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f929cceb740 1 -- 172.21.15.5:0/86035785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f928c036040 con 0x55b22762b5d0 2024-09-15T22:16:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f92737fe640 1 -- 172.21.15.5:0/86035785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f928409fe70 con 0x55b22762b5d0 2024-09-15T22:16:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f929cceb740 10 client.16821 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.095+0000 7f929cceb740 1 -- 172.21.15.5:0/86035785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b2276b1ad0 con 0x55b22762b5d0 2024-09-15T22:16:49.146 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:16:49.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.096+0000 7f92737fe640 1 -- 172.21.15.5:0/86035785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f928c036040 con 0x55b22762b5d0 2024-09-15T22:16:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.096+0000 7f92737fe640 1 -- 172.21.15.5:0/86035785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f92840ba2a0 con 0x55b22762b5d0 2024-09-15T22:16:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.096+0000 7f929cceb740 10 client.16821 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.096+0000 7f929cceb740 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.096+0000 7f929cceb740 20 client.16821 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.096+0000 7f929cceb740 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b2276b2150 0x55b2276b4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.096+0000 7f929cceb740 4 client.16821 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.096+0000 7f929cceb740 1 -- 172.21.15.5:0/86035785 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b227598d70 con 0x55b2276b2150 2024-09-15T22:16:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.096+0000 7f928bfff640 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b2276b2150 0x55b2276b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:49.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.096+0000 7f92737fe640 10 client.16821 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:49.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.096+0000 7f928bfff640 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b2276b2150 0x55b2276b4540 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60b858bc4aeaa84 server_cookie=479ee84b84cf1863 in_seq=0 out_seq=0 2024-09-15T22:16:49.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f92737fe640 1 -- 172.21.15.5:0/86035785 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b227598d70 con 0x55b2276b2150 2024-09-15T22:16:49.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f92737fe640 10 client.16821 handle_command_reply: tid=0 2024-09-15T22:16:49.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 1 client.16821 shutdown 2024-09-15T22:16:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 1 -- 172.21.15.5:0/86035785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762b5d0 msgr2=0x55b22763e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762b5d0 0x55b22763e960 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f9284098490 tx=0x7f928409b600 comp rx=0 tx=0).stop 2024-09-15T22:16:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 1 -- 172.21.15.5:0/86035785 shutdown_connections 2024-09-15T22:16:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55b2276b2150 0x55b2276b4540 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b227630930 0x55b22762f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b227636370 0x55b2276303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 1 --2- 172.21.15.5:0/86035785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b22762b5d0 0x55b22763e960 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f92737fe640 0 client.16821 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 1 -- 172.21.15.5:0/86035785 >> 172.21.15.5:0/86035785 conn(0x55b2275f4310 msgr2=0x55b2276350a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 1 -- 172.21.15.5:0/86035785 shutdown_connections 2024-09-15T22:16:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 1 -- 172.21.15.5:0/86035785 wait complete. 2024-09-15T22:16:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.097+0000 7f929cceb740 20 client.16821 trim_cache size 0 max 16384 2024-09-15T22:16:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 -- 172.21.15.5:0/901984279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f92800a33b0 msgr2=0x7f92800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f92800a33b0 0x7f92800a5870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f928c005cc0 tx=0x7f928c005f80 comp rx=0 tx=0).stop 2024-09-15T22:16:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 -- 172.21.15.5:0/901984279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294100310 msgr2=0x7f92941983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294100310 0x7f92941983b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f928400d950 tx=0x7f928400de10 comp rx=0 tx=0).stop 2024-09-15T22:16:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 -- 172.21.15.5:0/901984279 shutdown_connections 2024-09-15T22:16:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f92800a33b0 0x7f92800a5870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9294106f30 0x7f929419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9294100cc0 0x7f92941988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 --2- 172.21.15.5:0/901984279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9294100310 0x7f92941983b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:49.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 -- 172.21.15.5:0/901984279 >> 172.21.15.5:0/901984279 conn(0x7f9294077210 msgr2=0x7f9294105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:49.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 -- 172.21.15.5:0/901984279 shutdown_connections 2024-09-15T22:16:49.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:49.098+0000 7f929a912640 1 -- 172.21.15.5:0/901984279 wait complete. 2024-09-15T22:16:49.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 440.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:16:49.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:49 smithi099 ceph-mon[34892]: pgmap v242: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 3.1 MiB/s wr, 350 op/s 2024-09-15T22:16:49.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:49 smithi129 ceph-mon[35314]: pgmap v242: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 3.1 MiB/s wr, 350 op/s 2024-09-15T22:16:49.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:49 smithi005 ceph-mon[30944]: pgmap v242: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 3.1 MiB/s wr, 350 op/s 2024-09-15T22:16:50.947 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:16:50.947 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:16:51.220 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:51.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:51 smithi005 ceph-mon[30944]: pgmap v243: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 2.5 MiB/s wr, 284 op/s 2024-09-15T22:16:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.694+0000 7fe8ca80b640 1 -- 172.21.15.5:0/2098966480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4102570 msgr2=0x7fe8c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.694+0000 7fe8ca80b640 1 --2- 172.21.15.5:0/2098966480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4102570 0x7fe8c41029b0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009900 tx=0x7fe8ac02a480 comp rx=0 tx=0).stop 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.694+0000 7fe8ca80b640 1 -- 172.21.15.5:0/2098966480 shutdown_connections 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.694+0000 7fe8ca80b640 1 --2- 172.21.15.5:0/2098966480 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8c4102ef0 0x7fe8c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.694+0000 7fe8ca80b640 1 --2- 172.21.15.5:0/2098966480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4102570 0x7fe8c41029b0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.694+0000 7fe8ca80b640 1 --2- 172.21.15.5:0/2098966480 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8c4100e20 0x7fe8c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.694+0000 7fe8ca80b640 1 -- 172.21.15.5:0/2098966480 >> 172.21.15.5:0/2098966480 conn(0x7fe8c40fc7d0 msgr2=0x7fe8c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.694+0000 7fe8ca80b640 1 -- 172.21.15.5:0/2098966480 shutdown_connections 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.694+0000 7fe8ca80b640 1 -- 172.21.15.5:0/2098966480 wait complete. 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.694+0000 7fe8ca80b640 1 Processor -- start 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.694+0000 7fe8ca80b640 1 -- start start 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8ca80b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8c4100e20 0x7fe8c4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8ca80b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8c4102570 0x7fe8c4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8ca80b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4102ef0 0x7fe8c419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8ca80b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c4110fa0 con 0x7fe8c4102ef0 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8ca80b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c4110e20 con 0x7fe8c4100e20 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8ca80b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c4111120 con 0x7fe8c4102570 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8c8da8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4102ef0 0x7fe8c419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8c8da8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4102ef0 0x7fe8c419ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54954/0 (socket says 172.21.15.5:54954) 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8c8da8640 1 -- 172.21.15.5:0/1805865279 learned_addr learned my addr 172.21.15.5:0/1805865279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8c4100e20 0x7fe8c4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8c4102570 0x7fe8c4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8c8da8640 1 -- 172.21.15.5:0/1805865279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8c4102570 msgr2=0x7fe8c4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8c8da8640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8c4102570 0x7fe8c4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8c8da8640 1 -- 172.21.15.5:0/1805865279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8c4100e20 msgr2=0x7fe8c4198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8c8da8640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8c4100e20 0x7fe8c4198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8c8da8640 1 -- 172.21.15.5:0/1805865279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c419d510 con 0x7fe8c4102ef0 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8c8da8640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4102ef0 0x7fe8c419ce30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00eaa0 tx=0x7fe8bc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.695+0000 7fe8c1ffb640 1 -- 172.21.15.5:0/1805865279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc004d20 con 0x7fe8c4102ef0 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.696+0000 7fe8c1ffb640 1 -- 172.21.15.5:0/1805865279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc005020 con 0x7fe8c4102ef0 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.696+0000 7fe8ca80b640 1 -- 172.21.15.5:0/1805865279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c419d800 con 0x7fe8c4102ef0 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.696+0000 7fe8c1ffb640 1 -- 172.21.15.5:0/1805865279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0116d0 con 0x7fe8c4102ef0 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.696+0000 7fe8ca80b640 1 -- 172.21.15.5:0/1805865279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c41a50e0 con 0x7fe8c4102ef0 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8ccbe4740 1 Processor -- start 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8c1ffb640 1 -- 172.21.15.5:0/1805865279 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe8bc0051d0 con 0x7fe8c4102ef0 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8ccbe4740 1 -- start start 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 0x5574c4d67f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d72800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d72d40 0x5574c4d75130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574c4ccd280 con 0x5574c4d68480 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5574c4c74b10 con 0x5574c4d67b60 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5574c4cc9240 con 0x5574c4d72d40 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d72800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8c3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d72800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54958/0 (socket says 172.21.15.5:54958) 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8c3fff640 1 -- 172.21.15.5:0/4138555255 learned_addr learned my addr 172.21.15.5:0/4138555255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/4138555255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 0x5574c4d67f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/4138555255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d72d40 0x5574c4d75130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8c3fff640 1 -- 172.21.15.5:0/4138555255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d72d40 msgr2=0x5574c4d75130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/4138555255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d72d40 0x5574c4d75130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8c3fff640 1 -- 172.21.15.5:0/4138555255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 msgr2=0x5574c4d67f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.699+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/4138555255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 0x5574c4d67f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.700+0000 7fe8c3fff640 1 -- 172.21.15.5:0/4138555255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c4c811f0 con 0x5574c4d68480 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.700+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/4138555255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 0x5574c4d67f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.700+0000 7fe8c1ffb640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe8b00a33b0 0x7fe8b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.700+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/4138555255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d72800 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009a70 tx=0x7fe8b4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d856adeb641136f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.700+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe8b00a33b0 0x7fe8b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.700+0000 7fe8c1ffb640 1 -- 172.21.15.5:0/1805865279 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fe8bc0d03b0 con 0x7fe8c4102ef0 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.700+0000 7fe8ba7fc640 1 -- 172.21.15.5:0/4138555255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4017070 con 0x5574c4d68480 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.700+0000 7fe8ba7fc640 1 -- 172.21.15.5:0/4138555255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b4013470 con 0x5574c4d68480 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.700+0000 7fe8ba7fc640 1 -- 172.21.15.5:0/4138555255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4012630 con 0x5574c4d68480 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.701+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/4138555255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 msgr2=0x5574c4d72800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.701+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe8b00a33b0 0x7fe8b00a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40fffe0 tx=0x7fe8ac02a480 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:51.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.701+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/4138555255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d72800 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009a70 tx=0x7fe8b4009f30 comp rx=0 tx=0).stop 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.701+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/4138555255 shutdown_connections 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.701+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/4138555255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d72d40 0x5574c4d75130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.701+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/4138555255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d72800 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.701+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/4138555255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 0x5574c4d67f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.701+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/4138555255 >> 172.21.15.5:0/4138555255 conn(0x5574c4d640f0 msgr2=0x5574c4d644d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.701+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/4138555255 shutdown_connections 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.701+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/4138555255 wait complete. 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.701+0000 7fe8ccbe4740 1 -- start start 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8ccbe4740 1 Processor -- start 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d67b60 0x5574c4d6fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d72d40 0x5574c4d706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574c4d75910 con 0x5574c4d68480 2024-09-15T22:16:51.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5574c4c87f10 con 0x5574c4d72d40 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5574c4d75a90 con 0x5574c4d67b60 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8ccbe4740 10 client.0 _ll_register_callbacks cb 0x5574c4c50a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54964/0 (socket says 172.21.15.5:54964) 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8bbfff640 1 -- 172.21.15.5:0/1308691427 learned_addr learned my addr 172.21.15.5:0/1308691427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8c8da8640 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d72d40 0x5574c4d706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d67b60 0x5574c4d6fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8bbfff640 1 -- 172.21.15.5:0/1308691427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d67b60 msgr2=0x5574c4d6fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d67b60 0x5574c4d6fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8bbfff640 1 -- 172.21.15.5:0/1308691427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d72d40 msgr2=0x5574c4d706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d72d40 0x5574c4d706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8bbfff640 1 -- 172.21.15.5:0/1308691427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c4a3e6a0 con 0x5574c4d68480 2024-09-15T22:16:51.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d67b60 0x5574c4d6fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.702+0000 7fe8c8da8640 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d72d40 0x5574c4d706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.703+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d701a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009900 tx=0x7fe8ac009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44096fa27edbb8dc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.703+0000 7fe8a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.703+0000 7fe8a37fe640 1 -- 172.21.15.5:0/1308691427 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac04e020 con 0x5574c4d68480 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.703+0000 7fe8a37fe640 1 -- 172.21.15.5:0/1308691427 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fe8ac005250 con 0x5574c4d68480 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.703+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1308691427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b4013470 con 0x5574c4d68480 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.703+0000 7fe8a37fe640 1 -- 172.21.15.5:0/1308691427 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe8ac035040 con 0x5574c4d68480 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.703+0000 7fe8ccbe4740 10 client.16833 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.703+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1308691427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5574c4d814c0 con 0x5574c4d68480 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.703+0000 7fe8a37fe640 1 -- 172.21.15.5:0/1308691427 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8b4013470 con 0x5574c4d68480 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.703+0000 7fe8a37fe640 1 -- 172.21.15.5:0/1308691427 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe8ac0317a0 con 0x5574c4d68480 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.704+0000 7fe8ccbe4740 10 client.16833 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:51.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.704+0000 7fe8ccbe4740 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.704+0000 7fe8ccbe4740 20 client.16833 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.704+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574c4d82980 0x5574c4d84d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.704+0000 7fe8ccbe4740 4 client.16833 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.704+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1308691427 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c4cda7f0 con 0x5574c4d82980 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.704+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574c4d82980 0x5574c4d84d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.704+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574c4d82980 0x5574c4d84d70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48b9f95fb05f454d server_cookie=a9bf1ac1dbf1bd63 in_seq=0 out_seq=0 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.704+0000 7fe8a37fe640 10 client.16833 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8a37fe640 1 -- 172.21.15.5:0/1308691427 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574c4cda7f0 con 0x5574c4d82980 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8a37fe640 10 client.16833 handle_command_reply: tid=0 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 1 client.16833 shutdown 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1308691427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 msgr2=0x5574c4d701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d701a0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac009900 tx=0x7fe8ac009260 comp rx=0 tx=0).stop 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1308691427 shutdown_connections 2024-09-15T22:16:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574c4d82980 0x5574c4d84d70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d72d40 0x5574c4d706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d68480 0x5574c4d701a0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/1308691427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d67b60 0x5574c4d6fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8a37fe640 0 client.16833 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1308691427 >> 172.21.15.5:0/1308691427 conn(0x5574c4d640f0 msgr2=0x5574c4d63270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1308691427 shutdown_connections 2024-09-15T22:16:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1308691427 wait complete. 2024-09-15T22:16:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.705+0000 7fe8ccbe4740 20 client.16833 trim_cache size 0 max 16384 2024-09-15T22:16:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8ccbe4740 1 Processor -- start 2024-09-15T22:16:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8ccbe4740 1 -- start start 2024-09-15T22:16:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d67b60 0x5574c4d6a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d68480 0x5574c4d71ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d8a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41b1b00 con 0x5574c4d72d40 2024-09-15T22:16:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41af3d0 con 0x5574c4d68480 2024-09-15T22:16:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41ad240 con 0x5574c4d67b60 2024-09-15T22:16:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8c8da8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d8a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8c8da8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d8a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54968/0 (socket says 172.21.15.5:54968) 2024-09-15T22:16:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8c8da8640 1 -- 172.21.15.5:0/1184906129 learned_addr learned my addr 172.21.15.5:0/1184906129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.723+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/1184906129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d67b60 0x5574c4d6a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.724+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/1184906129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d68480 0x5574c4d71ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.724+0000 7fe8c8da8640 1 -- 172.21.15.5:0/1184906129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d67b60 msgr2=0x5574c4d6a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.724+0000 7fe8c8da8640 1 --2- 172.21.15.5:0/1184906129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d67b60 0x5574c4d6a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.724+0000 7fe8c8da8640 1 -- 172.21.15.5:0/1184906129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d68480 msgr2=0x5574c4d71ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.724+0000 7fe8c8da8640 1 --2- 172.21.15.5:0/1184906129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d68480 0x5574c4d71ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.724+0000 7fe8c8da8640 1 -- 172.21.15.5:0/1184906129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c4d850e0 con 0x5574c4d72d40 2024-09-15T22:16:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.724+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/1184906129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d68480 0x5574c4d71ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.724+0000 7fe8c8da8640 1 --2- 172.21.15.5:0/1184906129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d8a600 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0cd7b0 tx=0x7fe8bc09c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3659b9339fb59e6f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.724+0000 7fe8b9ffb640 1 -- 172.21.15.5:0/1184906129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc01f030 con 0x5574c4d72d40 2024-09-15T22:16:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.724+0000 7fe8b9ffb640 1 -- 172.21.15.5:0/1184906129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc0cde50 con 0x5574c4d72d40 2024-09-15T22:16:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.724+0000 7fe8b9ffb640 1 -- 172.21.15.5:0/1184906129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0cdb10 con 0x5574c4d72d40 2024-09-15T22:16:51.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.725+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1184906129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 msgr2=0x5574c4d8a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:51.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.725+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/1184906129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d8a600 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0cd7b0 tx=0x7fe8bc09c290 comp rx=0 tx=0).stop 2024-09-15T22:16:51.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.725+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1184906129 shutdown_connections 2024-09-15T22:16:51.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.725+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/1184906129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d8a600 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.725+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/1184906129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d68480 0x5574c4d71ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.725+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/1184906129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d67b60 0x5574c4d6a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.725+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1184906129 >> 172.21.15.5:0/1184906129 conn(0x5574c4d640f0 msgr2=0x5574c4d713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:51.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.725+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1184906129 shutdown_connections 2024-09-15T22:16:51.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.725+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/1184906129 wait complete. 2024-09-15T22:16:51.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.725+0000 7fe8ccbe4740 1 -- start start 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.725+0000 7fe8ccbe4740 1 Processor -- start 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 0x5574c4d6cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d68480 0x5574c4d6d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8ccbe4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d6d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41ad890 con 0x5574c4d72d40 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41aed00 con 0x5574c4d67b60 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8ccbe4740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c41adf60 con 0x5574c4d68480 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8ccbe4740 10 client.0 _ll_register_callbacks cb 0x7fe8ac041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 0x5574c4d6cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c8da8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d6d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 0x5574c4d6cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46636/0 (socket says 172.21.15.5:46636) 2024-09-15T22:16:51.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c8da8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d6d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54976/0 (socket says 172.21.15.5:54976) 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c3fff640 1 -- 172.21.15.5:0/2233028173 learned_addr learned my addr 172.21.15.5:0/2233028173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8bbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d68480 0x5574c4d6d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c3fff640 1 -- 172.21.15.5:0/2233028173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d68480 msgr2=0x5574c4d6d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d68480 0x5574c4d6d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c3fff640 1 -- 172.21.15.5:0/2233028173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 msgr2=0x5574c4d6d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d6d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c3fff640 1 -- 172.21.15.5:0/2233028173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c4dec730 con 0x5574c4d67b60 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c8da8640 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d6d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8c3fff640 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 0x5574c4d6cde0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4004810 tx=0x7fe8b4009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1dd6eb959ae03d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.726+0000 7fe8a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8a37fe640 1 -- 172.21.15.5:0/2233028173 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4017040 con 0x5574c4d67b60 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8a37fe640 1 -- 172.21.15.5:0/2233028173 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fe8b400ed40 con 0x5574c4d67b60 2024-09-15T22:16:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/2233028173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8bc0cde50 con 0x5574c4d67b60 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8a37fe640 1 -- 172.21.15.5:0/2233028173 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe8b4034050 con 0x5574c4d67b60 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8ccbe4740 10 client.26378 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/2233028173 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5574c4deda20 con 0x5574c4d67b60 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8a37fe640 1 -- 172.21.15.5:0/2233028173 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8bc0cde50 con 0x5574c4d67b60 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8a37fe640 1 -- 172.21.15.5:0/2233028173 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe8b402f030 con 0x5574c4d67b60 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8ccbe4740 10 client.26378 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8ccbe4740 10 client.26378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8ccbe4740 20 client.26378 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574c4dedf20 0x5574c4df0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8ccbe4740 4 client.26378 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.727+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/2233028173 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5574c4cd4d00 con 0x5574c4dedf20 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.728+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574c4dedf20 0x5574c4df0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.728+0000 7fe8a37fe640 10 client.26378 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:51.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.728+0000 7fe8bbfff640 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574c4dedf20 0x5574c4df0340 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b3202aaf3d2a3fa server_cookie=ecc7a25519ca3095 in_seq=0 out_seq=0 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8a37fe640 1 -- 172.21.15.5:0/2233028173 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5574c4cd4d00 con 0x5574c4dedf20 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8a37fe640 10 client.26378 handle_command_reply: tid=0 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8ccbe4740 1 client.26378 shutdown 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/2233028173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 msgr2=0x5574c4d6cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 0x5574c4d6cde0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4004810 tx=0x7fe8b4009f30 comp rx=0 tx=0).stop 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/2233028173 shutdown_connections 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574c4d72d40 0x5574c4d6d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574c4d68480 0x5574c4d6d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574c4dedf20 0x5574c4df0340 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8ccbe4740 1 --2- 172.21.15.5:0/2233028173 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574c4d67b60 0x5574c4d6cde0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8a37fe640 0 client.26378 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.729+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/2233028173 >> 172.21.15.5:0/2233028173 conn(0x5574c4d640f0 msgr2=0x5574c4d63270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:51.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/2233028173 shutdown_connections 2024-09-15T22:16:51.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ccbe4740 1 -- 172.21.15.5:0/2233028173 wait complete. 2024-09-15T22:16:51.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ccbe4740 20 client.26378 trim_cache size 0 max 16384 2024-09-15T22:16:51.740 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:16:51.740 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:16:51.740 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "c1d9c638-4eaa-4b71-b870-64454f6d4edd", 2024-09-15T22:16:51.740 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:16:51.740 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:16:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ca80b640 1 -- 172.21.15.5:0/1805865279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe8b00a33b0 msgr2=0x7fe8b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ca80b640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe8b00a33b0 0x7fe8b00a5870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40fffe0 tx=0x7fe8ac02a480 comp rx=0 tx=0).stop 2024-09-15T22:16:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ca80b640 1 -- 172.21.15.5:0/1805865279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4102ef0 msgr2=0x7fe8c419ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ca80b640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4102ef0 0x7fe8c419ce30 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00eaa0 tx=0x7fe8bc00eea0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ca80b640 1 -- 172.21.15.5:0/1805865279 shutdown_connections 2024-09-15T22:16:51.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ca80b640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe8b00a33b0 0x7fe8b00a5870 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ca80b640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8c4102ef0 0x7fe8c419ce30 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ca80b640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe8c4102570 0x7fe8c4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ca80b640 1 --2- 172.21.15.5:0/1805865279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe8c4100e20 0x7fe8c4198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ca80b640 1 -- 172.21.15.5:0/1805865279 >> 172.21.15.5:0/1805865279 conn(0x7fe8c40fc7d0 msgr2=0x7fe8c406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.730+0000 7fe8ca80b640 1 -- 172.21.15.5:0/1805865279 shutdown_connections 2024-09-15T22:16:51.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:51.731+0000 7fe8ca80b640 1 -- 172.21.15.5:0/1805865279 wait complete. 2024-09-15T22:16:51.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:51 smithi099 ceph-mon[34892]: pgmap v243: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 2.5 MiB/s wr, 284 op/s 2024-09-15T22:16:51.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:51 smithi129 ceph-mon[35314]: pgmap v243: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 2.5 MiB/s wr, 284 op/s 2024-09-15T22:16:52.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:16:52.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:52 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:16:52.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:52 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:52.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:52 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:16:52.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:52 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:52.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:52 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:16:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:52.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:52 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:16:52.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:52 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:52.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:52 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:16:52.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:52 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:52.841 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:52 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:16:52.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:52 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:16:52.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:52 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:52.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:52 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:16:52.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:52 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:16:52.846 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:52 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:16:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.304+0000 7feddd293640 1 -- 172.21.15.5:0/4219594926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd8101360 msgr2=0x7fedd810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.304+0000 7feddd293640 1 --2- 172.21.15.5:0/4219594926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd8101360 0x7fedd810df90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fedcc009900 tx=0x7fedcc02a410 comp rx=0 tx=0).stop 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.304+0000 7feddd293640 1 -- 172.21.15.5:0/4219594926 shutdown_connections 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.304+0000 7feddd293640 1 --2- 172.21.15.5:0/4219594926 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedd810e4d0 0x7fedd81108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.304+0000 7feddd293640 1 --2- 172.21.15.5:0/4219594926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd8101360 0x7fedd810df90 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.304+0000 7feddd293640 1 --2- 172.21.15.5:0/4219594926 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd81009b0 0x7fedd8100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.304+0000 7feddd293640 1 -- 172.21.15.5:0/4219594926 >> 172.21.15.5:0/4219594926 conn(0x7fedd80fc7d0 msgr2=0x7fedd80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.304+0000 7feddd293640 1 -- 172.21.15.5:0/4219594926 shutdown_connections 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.304+0000 7feddd293640 1 -- 172.21.15.5:0/4219594926 wait complete. 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7feddd293640 1 Processor -- start 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7feddd293640 1 -- start start 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7feddd293640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd81009b0 0x7fedd819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7feddd293640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd8101360 0x7fedd819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7feddd293640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedd810e4d0 0x7fedd819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7feddd293640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedd8114220 con 0x7fedd8101360 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7feddd293640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81140a0 con 0x7fedd810e4d0 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7feddd293640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fedd81143a0 con 0x7fedd81009b0 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7fedd67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd8101360 0x7fedd819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7fedd67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd8101360 0x7fedd819ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54986/0 (socket says 172.21.15.5:54986) 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7fedd67fc640 1 -- 172.21.15.5:0/3324121750 learned_addr learned my addr 172.21.15.5:0/3324121750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7fedd6ffd640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd81009b0 0x7fedd819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.305+0000 7fedd77fe640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedd810e4d0 0x7fedd819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7fedd67fc640 1 -- 172.21.15.5:0/3324121750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd81009b0 msgr2=0x7fedd819a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7fedd67fc640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd81009b0 0x7fedd819a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7fedd67fc640 1 -- 172.21.15.5:0/3324121750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedd810e4d0 msgr2=0x7fedd819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7fedd67fc640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedd810e4d0 0x7fedd819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7fedd67fc640 1 -- 172.21.15.5:0/3324121750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd819f6f0 con 0x7fedd8101360 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7fedd77fe640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedd810e4d0 0x7fedd819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7fedd67fc640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd8101360 0x7fedd819ac80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fedcc006010 tx=0x7fedcc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7fedbffff640 1 -- 172.21.15.5:0/3324121750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc03f030 con 0x7fedd8101360 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7feddd293640 1 -- 172.21.15.5:0/3324121750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd819f980 con 0x7fedd8101360 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7feddd293640 1 -- 172.21.15.5:0/3324121750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd81a72c0 con 0x7fedd8101360 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7fedbffff640 1 -- 172.21.15.5:0/3324121750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc003ce0 con 0x7fedd8101360 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.306+0000 7fedbffff640 1 -- 172.21.15.5:0/3324121750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc035070 con 0x7fedd8101360 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7fedbffff640 1 -- 172.21.15.5:0/3324121750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fedcc042030 con 0x7fedd8101360 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7feddf66c740 1 Processor -- start 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7feddf66c740 1 -- start start 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7feddf66c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585ba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7feddf66c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7feddf66c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 0x5585585c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7feddf66c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585585202e0 con 0x5585585c5680 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7feddf66c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5585584c7ce0 con 0x5585585ba4a0 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7feddf66c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55855851c2a0 con 0x5585585badc0 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7fedd67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 0x5585585c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7fedd77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585ba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7fedd6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7fedd6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:54070/0 (socket says 172.21.15.5:54070) 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7fedd77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585ba880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46660/0 (socket says 172.21.15.5:46660) 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.309+0000 7fedd6ffd640 1 -- 172.21.15.5:0/3489990287 learned_addr learned my addr 172.21.15.5:0/3489990287 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.310+0000 7fedd67fc640 1 -- 172.21.15.5:0/3489990287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 msgr2=0x5585585c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.310+0000 7fedd67fc640 1 --2- 172.21.15.5:0/3489990287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.310+0000 7fedd67fc640 1 -- 172.21.15.5:0/3489990287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 msgr2=0x5585585ba880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.310+0000 7fedd67fc640 1 --2- 172.21.15.5:0/3489990287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585ba880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.310+0000 7fedd67fc640 1 -- 172.21.15.5:0/3489990287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585584ac440 con 0x5585585c5680 2024-09-15T22:16:53.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.311+0000 7fedbffff640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fedc00a33b0 0x7fedc00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.311+0000 7fedd77fe640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fedc00a33b0 0x7fedc00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.311+0000 7fedbffff640 1 -- 172.21.15.5:0/3324121750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fedcc0f07b0 con 0x7fedd8101360 2024-09-15T22:16:53.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.311+0000 7fedd67fc640 1 --2- 172.21.15.5:0/3489990287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 0x5585585c7a70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0edb00 tx=0x7fedcc0f6cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510bf8a98db5f73a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:53.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.311+0000 7fedbd7fa640 1 -- 172.21.15.5:0/3489990287 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0bc460 con 0x5585585c5680 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.311+0000 7fedbd7fa640 1 -- 172.21.15.5:0/3489990287 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc0bcd90 con 0x5585585c5680 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.311+0000 7fedbd7fa640 1 -- 172.21.15.5:0/3489990287 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0044e0 con 0x5585585c5680 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.312+0000 7feddf66c740 1 -- 172.21.15.5:0/3489990287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 msgr2=0x5585585c7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.312+0000 7fedd77fe640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fedc00a33b0 0x7fedc00a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fedd0005fd0 tx=0x7fedd0009500 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.312+0000 7feddf66c740 1 --2- 172.21.15.5:0/3489990287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 0x5585585c7a70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0edb00 tx=0x7fedcc0f6cd0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.312+0000 7feddf66c740 1 -- 172.21.15.5:0/3489990287 shutdown_connections 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.312+0000 7feddf66c740 1 --2- 172.21.15.5:0/3489990287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 0x5585585c7a70 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.312+0000 7feddf66c740 1 --2- 172.21.15.5:0/3489990287 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.312+0000 7feddf66c740 1 --2- 172.21.15.5:0/3489990287 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585ba880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.312+0000 7feddf66c740 1 -- 172.21.15.5:0/3489990287 >> 172.21.15.5:0/3489990287 conn(0x5585585b6a30 msgr2=0x5585585b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.312+0000 7feddf66c740 1 -- 172.21.15.5:0/3489990287 shutdown_connections 2024-09-15T22:16:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.312+0000 7feddf66c740 1 -- 172.21.15.5:0/3489990287 wait complete. 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.312+0000 7feddf66c740 1 -- start start 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7feddf66c740 1 Processor -- start 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7feddf66c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7feddf66c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7feddf66c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 0x5585585c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7feddf66c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585585c8010 con 0x5585585c5680 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7feddf66c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5585584db050 con 0x5585585ba4a0 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7feddf66c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5585585c8190 con 0x5585585badc0 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7feddf66c740 10 client.0 _ll_register_callbacks cb 0x55855851b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7fedd77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 0x5585585c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7fedd77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 0x5585585c26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55002/0 (socket says 172.21.15.5:55002) 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7fedd77fe640 1 -- 172.21.15.5:0/3768749713 learned_addr learned my addr 172.21.15.5:0/3768749713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7fedd6ffd640 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7fedd67fc640 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7fedd77fe640 1 -- 172.21.15.5:0/3768749713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 msgr2=0x5585585c21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7fedd77fe640 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7fedd77fe640 1 -- 172.21.15.5:0/3768749713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 msgr2=0x5585585c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7fedd77fe640 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585c4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.313+0000 7fedd77fe640 1 -- 172.21.15.5:0/3768749713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585582913c0 con 0x5585585c5680 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7fedd77fe640 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 0x5585585c26f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fedd0012010 tx=0x7fedd0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2586b1c0c98932 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7fedbaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7fedbaff5640 1 -- 172.21.15.5:0/3768749713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd002c060 con 0x5585585c5680 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7fedbaff5640 1 -- 172.21.15.5:0/3768749713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fedd0017030 con 0x5585585c5680 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7feddf66c740 1 -- 172.21.15.5:0/3768749713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedcc0bcd90 con 0x5585585c5680 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7fedbaff5640 1 -- 172.21.15.5:0/3768749713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fedd0033050 con 0x5585585c5680 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7feddf66c740 10 client.16851 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7feddf66c740 1 -- 172.21.15.5:0/3768749713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5585585d3940 con 0x5585585c5680 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7fedbaff5640 1 -- 172.21.15.5:0/3768749713 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedcc0bcd90 con 0x5585585c5680 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7fedbaff5640 1 -- 172.21.15.5:0/3768749713 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fedd001c3f0 con 0x5585585c5680 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7feddf66c740 10 client.16851 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7feddf66c740 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7feddf66c740 20 client.16851 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7feddf66c740 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5585585d4e50 0x5585585d72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7feddf66c740 4 client.16851 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.314+0000 7feddf66c740 1 -- 172.21.15.5:0/3768749713 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55855852d850 con 0x5585585d4e50 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.315+0000 7fedd6ffd640 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5585585d4e50 0x5585585d72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.315+0000 7fedd6ffd640 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5585585d4e50 0x5585585d72b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fc36ab62e19a515 server_cookie=87e86b9f3843cfe9 in_seq=0 out_seq=0 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.315+0000 7fedbaff5640 10 client.16851 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7fedbaff5640 1 -- 172.21.15.5:0/3768749713 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55855852d850 con 0x5585585d4e50 2024-09-15T22:16:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7fedbaff5640 10 client.16851 handle_command_reply: tid=0 2024-09-15T22:16:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 1 client.16851 shutdown 2024-09-15T22:16:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 1 -- 172.21.15.5:0/3768749713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 msgr2=0x5585585c26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 0x5585585c26f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fedd0012010 tx=0x7fedd0004ab0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 1 -- 172.21.15.5:0/3768749713 shutdown_connections 2024-09-15T22:16:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585c5680 0x5585585c26f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5585585d4e50 0x5585585d72b0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7fedbaff5640 0 client.16851 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 1 --2- 172.21.15.5:0/3768749713 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 1 -- 172.21.15.5:0/3768749713 >> 172.21.15.5:0/3768749713 conn(0x5585585b6a30 msgr2=0x5585585bc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 1 -- 172.21.15.5:0/3768749713 shutdown_connections 2024-09-15T22:16:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 1 -- 172.21.15.5:0/3768749713 wait complete. 2024-09-15T22:16:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.316+0000 7feddf66c740 20 client.16851 trim_cache size 0 max 16384 2024-09-15T22:16:53.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.333+0000 7feddf66c740 1 Processor -- start 2024-09-15T22:16:53.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.333+0000 7feddf66c740 1 -- start start 2024-09-15T22:16:53.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7feddf66c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585dcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7feddf66c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585dd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7feddf66c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585bf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7feddf66c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedc40128d0 con 0x5585585b6a30 2024-09-15T22:16:53.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7feddf66c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fedc40101a0 con 0x5585585ba4a0 2024-09-15T22:16:53.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7feddf66c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fedc400e010 con 0x5585585badc0 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585dcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585dcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55012/0 (socket says 172.21.15.5:55012) 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedd6ffd640 1 -- 172.21.15.5:0/207318374 learned_addr learned my addr 172.21.15.5:0/207318374 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedd6ffd640 1 -- 172.21.15.5:0/207318374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 msgr2=0x5585585bf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedd6ffd640 1 --2- 172.21.15.5:0/207318374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585bf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedd6ffd640 1 -- 172.21.15.5:0/207318374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 msgr2=0x5585585dd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedd6ffd640 1 --2- 172.21.15.5:0/207318374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585dd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedd6ffd640 1 -- 172.21.15.5:0/207318374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585585c5270 con 0x5585585b6a30 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedd6ffd640 1 --2- 172.21.15.5:0/207318374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585dcb80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005850 tx=0x7fedc800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b186559d10fa4e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedbcff9640 1 -- 172.21.15.5:0/207318374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc800be60 con 0x5585585b6a30 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedbcff9640 1 -- 172.21.15.5:0/207318374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc80047a0 con 0x5585585b6a30 2024-09-15T22:16:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.334+0000 7fedbcff9640 1 -- 172.21.15.5:0/207318374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc80117c0 con 0x5585585b6a30 2024-09-15T22:16:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.335+0000 7feddf66c740 1 -- 172.21.15.5:0/207318374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 msgr2=0x5585585dcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.335+0000 7feddf66c740 1 --2- 172.21.15.5:0/207318374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585dcb80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005850 tx=0x7fedc800b390 comp rx=0 tx=0).stop 2024-09-15T22:16:53.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.335+0000 7feddf66c740 1 -- 172.21.15.5:0/207318374 shutdown_connections 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.335+0000 7feddf66c740 1 --2- 172.21.15.5:0/207318374 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585bf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.335+0000 7feddf66c740 1 --2- 172.21.15.5:0/207318374 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585dd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.335+0000 7feddf66c740 1 --2- 172.21.15.5:0/207318374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585dcb80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.335+0000 7feddf66c740 1 -- 172.21.15.5:0/207318374 >> 172.21.15.5:0/207318374 conn(0x5585585c2850 msgr2=0x5585585c0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.335+0000 7feddf66c740 1 -- 172.21.15.5:0/207318374 shutdown_connections 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.335+0000 7feddf66c740 1 -- 172.21.15.5:0/207318374 wait complete. 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.335+0000 7feddf66c740 1 -- start start 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.335+0000 7feddf66c740 1 Processor -- start 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7feddf66c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7feddf66c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7feddf66c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7feddf66c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedc400e660 con 0x5585585b6a30 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7feddf66c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fedc400fad0 con 0x5585585ba4a0 2024-09-15T22:16:53.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7feddf66c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fedc400ed30 con 0x5585585badc0 2024-09-15T22:16:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7feddf66c740 10 client.0 _ll_register_callbacks cb 0x7fedd0009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedd6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55026/0 (socket says 172.21.15.5:55026) 2024-09-15T22:16:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedd6ffd640 1 -- 172.21.15.5:0/1539588375 learned_addr learned my addr 172.21.15.5:0/1539588375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedd67fc640 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedd77fe640 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedd6ffd640 1 -- 172.21.15.5:0/1539588375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 msgr2=0x5585585c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedd6ffd640 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedd6ffd640 1 -- 172.21.15.5:0/1539588375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 msgr2=0x5585585c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedd6ffd640 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedd6ffd640 1 -- 172.21.15.5:0/1539588375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55855863eda0 con 0x5585585b6a30 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedd6ffd640 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585c93c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fedc80043c0 tx=0x7fedc8005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f59916fe56b260 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.336+0000 7fedbaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7fedbaff5640 1 -- 172.21.15.5:0/1539588375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8004d20 con 0x5585585b6a30 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7fedbaff5640 1 -- 172.21.15.5:0/1539588375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fedc8005020 con 0x5585585b6a30 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7feddf66c740 1 -- 172.21.15.5:0/1539588375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedc80047a0 con 0x5585585b6a30 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7fedbaff5640 1 -- 172.21.15.5:0/1539588375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fedc8005220 con 0x5585585b6a30 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7feddf66c740 10 client.16857 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7feddf66c740 1 -- 172.21.15.5:0/1539588375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558558640070 con 0x5585585b6a30 2024-09-15T22:16:53.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7fedbaff5640 1 -- 172.21.15.5:0/1539588375 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedc80047a0 con 0x5585585b6a30 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7fedbaff5640 1 -- 172.21.15.5:0/1539588375 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fedc8026410 con 0x5585585b6a30 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7feddf66c740 10 client.16857 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7feddf66c740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7feddf66c740 20 client.16857 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7feddf66c740 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5585585c5680 0x5585586426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7feddf66c740 4 client.16857 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7feddf66c740 1 -- 172.21.15.5:0/1539588375 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558558527d60 con 0x5585585c5680 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.337+0000 7fedd67fc640 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5585585c5680 0x5585586426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7fedd67fc640 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5585585c5680 0x5585586426e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae0467728e3f2ad server_cookie=642f4146a411a0e3 in_seq=0 out_seq=0 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7fedbaff5640 10 client.16857 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7fedbaff5640 1 -- 172.21.15.5:0/1539588375 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558558527d60 con 0x5585585c5680 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7fedbaff5640 10 client.16857 handle_command_reply: tid=0 2024-09-15T22:16:53.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 1 client.16857 shutdown 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 1 -- 172.21.15.5:0/1539588375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 msgr2=0x5585585c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585c93c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fedc80043c0 tx=0x7fedc8005bb0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 1 -- 172.21.15.5:0/1539588375 shutdown_connections 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5585585badc0 0x5585585c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5585585c5680 0x5585586426e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5585585ba4a0 0x5585585c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 1 --2- 172.21.15.5:0/1539588375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585585b6a30 0x5585585c93c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7fedbaff5640 0 client.16857 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 1 -- 172.21.15.5:0/1539588375 >> 172.21.15.5:0/1539588375 conn(0x5585585c2850 msgr2=0x5585585c3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 1 -- 172.21.15.5:0/1539588375 shutdown_connections 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 1 -- 172.21.15.5:0/1539588375 wait complete. 2024-09-15T22:16:53.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.338+0000 7feddf66c740 20 client.16857 trim_cache size 0 max 16384 2024-09-15T22:16:53.348 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:16:53.348 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:16:53.348 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:16:53.348 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:16:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 -- 172.21.15.5:0/3324121750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fedc00a33b0 msgr2=0x7fedc00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fedc00a33b0 0x7fedc00a5870 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fedd0005fd0 tx=0x7fedd0009500 comp rx=0 tx=0).stop 2024-09-15T22:16:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 -- 172.21.15.5:0/3324121750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd8101360 msgr2=0x7fedd819ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd8101360 0x7fedd819ac80 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fedcc006010 tx=0x7fedcc0042f0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 -- 172.21.15.5:0/3324121750 shutdown_connections 2024-09-15T22:16:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fedc00a33b0 0x7fedc00a5870 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fedd810e4d0 0x7fedd819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd8101360 0x7fedd819ac80 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 --2- 172.21.15.5:0/3324121750 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fedd81009b0 0x7fedd819a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:53.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 -- 172.21.15.5:0/3324121750 >> 172.21.15.5:0/3324121750 conn(0x7fedd80fc7d0 msgr2=0x7fedd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:53.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 -- 172.21.15.5:0/3324121750 shutdown_connections 2024-09-15T22:16:53.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:53.339+0000 7feddd293640 1 -- 172.21.15.5:0/3324121750 wait complete. 2024-09-15T22:16:53.449 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:16:53.472 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:53 smithi005 ceph-mon[30944]: pgmap v244: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-09-15T22:16:53.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:53 smithi099 ceph-mon[34892]: pgmap v244: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-09-15T22:16:53.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:53 smithi129 ceph-mon[35314]: pgmap v244: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.2 MiB/s wr, 243 op/s 2024-09-15T22:16:54.107 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:16:54.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:16:54.159 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:54.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.634+0000 7f81cd491640 1 -- 172.21.15.5:0/1044323498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81c8106f10 msgr2=0x7f81c810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:54.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.634+0000 7f81cd491640 1 --2- 172.21.15.5:0/1044323498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81c8106f10 0x7f81c810ea90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f81c0009ca0 tx=0x7f81c002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.634+0000 7f81cd491640 1 -- 172.21.15.5:0/1044323498 shutdown_connections 2024-09-15T22:16:54.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.634+0000 7f81cd491640 1 --2- 172.21.15.5:0/1044323498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81c8106f10 0x7f81c810ea90 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.634+0000 7f81cd491640 1 --2- 172.21.15.5:0/1044323498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 0x7f81c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.634+0000 7f81cd491640 1 --2- 172.21.15.5:0/1044323498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c8101de0 0x7f81c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.634+0000 7f81cd491640 1 -- 172.21.15.5:0/1044323498 >> 172.21.15.5:0/1044323498 conn(0x7f81c80fd8c0 msgr2=0x7f81c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.634+0000 7f81cd491640 1 -- 172.21.15.5:0/1044323498 shutdown_connections 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.634+0000 7f81cd491640 1 -- 172.21.15.5:0/1044323498 wait complete. 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.635+0000 7f81cd491640 1 Processor -- start 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.635+0000 7f81cd491640 1 -- start start 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.635+0000 7f81cd491640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c8101de0 0x7f81c8198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.635+0000 7f81cd491640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 0x7f81c8198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.635+0000 7f81cd491640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81c8106f10 0x7f81c819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.635+0000 7f81cd491640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8111f70 con 0x7f81c8106f10 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.635+0000 7f81cd491640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f81c8111df0 con 0x7f81c8102790 2024-09-15T22:16:54.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.635+0000 7f81cd491640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f81c81120f0 con 0x7f81c8101de0 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.635+0000 7f81c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81c8106f10 0x7f81c819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.635+0000 7f81c77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81c8106f10 0x7f81c819cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37056/0 (socket says 172.21.15.5:37056) 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.635+0000 7f81c77fe640 1 -- 172.21.15.5:0/4113114464 learned_addr learned my addr 172.21.15.5:0/4113114464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81c6ffd640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c8101de0 0x7f81c8198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81bffff640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 0x7f81c8198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81c77fe640 1 -- 172.21.15.5:0/4113114464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c8101de0 msgr2=0x7f81c8198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81c77fe640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c8101de0 0x7f81c8198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81c77fe640 1 -- 172.21.15.5:0/4113114464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 msgr2=0x7f81c8198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81c77fe640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 0x7f81c8198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81c77fe640 1 -- 172.21.15.5:0/4113114464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c819d300 con 0x7f81c8106f10 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81c6ffd640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c8101de0 0x7f81c8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:54.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81c77fe640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81c8106f10 0x7f81c819cc20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f81c0031a20 tx=0x7f81c0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81c4ff9640 1 -- 172.21.15.5:0/4113114464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c0003c30 con 0x7f81c8106f10 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81c4ff9640 1 -- 172.21.15.5:0/4113114464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81c0003dd0 con 0x7f81c8106f10 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81cd491640 1 -- 172.21.15.5:0/4113114464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c819d590 con 0x7f81c8106f10 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81c4ff9640 1 -- 172.21.15.5:0/4113114464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c002fcf0 con 0x7f81c8106f10 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.636+0000 7f81cd491640 1 -- 172.21.15.5:0/4113114464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c81a4ed0 con 0x7f81c8106f10 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.637+0000 7f81cd491640 1 -- 172.21.15.5:0/4113114464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c81021c0 con 0x7f81c8106f10 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.638+0000 7f81c4ff9640 1 -- 172.21.15.5:0/4113114464 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f81c0004050 con 0x7f81c8106f10 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.639+0000 7f81c4ff9640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f81b40a3430 0x7f81b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.639+0000 7f81c6ffd640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f81b40a3430 0x7f81b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.639+0000 7f81c4ff9640 1 -- 172.21.15.5:0/4113114464 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f81c00f13e0 con 0x7f81c8106f10 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.639+0000 7f81c6ffd640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f81b40a3430 0x7f81b40a58f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009b40 tx=0x7f81b8005e50 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:54.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.643+0000 7f81c4ff9640 1 -- 172.21.15.5:0/4113114464 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81c00bdc80 con 0x7f81c8106f10 2024-09-15T22:16:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 445.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:16:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:54.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.860+0000 7f81cd491640 1 -- 172.21.15.5:0/4113114464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81c81a5100 con 0x7f81c8106f10 2024-09-15T22:16:54.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.861+0000 7f81c4ff9640 1 -- 172.21.15.5:0/4113114464 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f81c002fe90 con 0x7f81c8106f10 2024-09-15T22:16:54.864 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:16:54.864 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:16:54.864 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:16:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 -- 172.21.15.5:0/4113114464 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f81b40a3430 msgr2=0x7f81b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f81b40a3430 0x7f81b40a58f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81b8009b40 tx=0x7f81b8005e50 comp rx=0 tx=0).stop 2024-09-15T22:16:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 -- 172.21.15.5:0/4113114464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81c8106f10 msgr2=0x7f81c819cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81c8106f10 0x7f81c819cc20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f81c0031a20 tx=0x7f81c0031a50 comp rx=0 tx=0).stop 2024-09-15T22:16:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 -- 172.21.15.5:0/4113114464 shutdown_connections 2024-09-15T22:16:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f81b40a3430 0x7f81b40a58f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81c8106f10 0x7f81c819cc20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f81c8102790 0x7f81c8198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 --2- 172.21.15.5:0/4113114464 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f81c8101de0 0x7f81c8198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 -- 172.21.15.5:0/4113114464 >> 172.21.15.5:0/4113114464 conn(0x7f81c80fd8c0 msgr2=0x7f81c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 -- 172.21.15.5:0/4113114464 shutdown_connections 2024-09-15T22:16:54.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:54.863+0000 7f81cd491640 1 -- 172.21.15.5:0/4113114464 wait complete. 2024-09-15T22:16:55.644 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id ee493997 --release --await' 2024-09-15T22:16:55.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id ee493997 --release --await 2024-09-15T22:16:55.710 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:55 smithi005 ceph-mon[30944]: pgmap v245: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 332 KiB/s wr, 45 op/s 2024-09-15T22:16:55.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:55 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4113114464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:55 smithi099 ceph-mon[34892]: pgmap v245: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 332 KiB/s wr, 45 op/s 2024-09-15T22:16:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:55 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4113114464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:55.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:55 smithi129 ceph-mon[35314]: pgmap v245: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 332 KiB/s wr, 45 op/s 2024-09-15T22:16:55.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:55 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4113114464' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.931+0000 7f002cf91640 1 -- 172.21.15.5:0/280812848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0028100e20 msgr2=0x7f0028101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.931+0000 7f002cf91640 1 --2- 172.21.15.5:0/280812848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0028100e20 0x7f0028101200 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f0020009ca0 tx=0x7f002002a950 comp rx=0 tx=0).stop 2024-09-15T22:16:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.931+0000 7f002cf91640 1 -- 172.21.15.5:0/280812848 shutdown_connections 2024-09-15T22:16:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.931+0000 7f002cf91640 1 --2- 172.21.15.5:0/280812848 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028102ef0 0x7f002810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.931+0000 7f002cf91640 1 --2- 172.21.15.5:0/280812848 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0028102570 0x7f00281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.931+0000 7f002cf91640 1 --2- 172.21.15.5:0/280812848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0028100e20 0x7f0028101200 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.931+0000 7f002cf91640 1 -- 172.21.15.5:0/280812848 >> 172.21.15.5:0/280812848 conn(0x7f00280fc7d0 msgr2=0x7f00280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.931+0000 7f002cf91640 1 -- 172.21.15.5:0/280812848 shutdown_connections 2024-09-15T22:16:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.931+0000 7f002cf91640 1 -- 172.21.15.5:0/280812848 wait complete. 2024-09-15T22:16:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f002cf91640 1 Processor -- start 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f002cf91640 1 -- start start 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f002cf91640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028100e20 0x7f00281984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f002cf91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0028102570 0x7f00281989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f002cf91640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0028102ef0 0x7f002819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f002cf91640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0028110f80 con 0x7f0028102570 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f002cf91640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0028110e00 con 0x7f0028100e20 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f002cf91640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0028111100 con 0x7f0028102ef0 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f0025d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0028102570 0x7f00281989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f0025d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0028102570 0x7f00281989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37066/0 (socket says 172.21.15.5:37066) 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f0025d9b640 1 -- 172.21.15.5:0/2004658099 learned_addr learned my addr 172.21.15.5:0/2004658099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f002659c640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028100e20 0x7f00281984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.932+0000 7f0026d9d640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0028102ef0 0x7f002819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f0025d9b640 1 -- 172.21.15.5:0/2004658099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0028102ef0 msgr2=0x7f002819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f0025d9b640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0028102ef0 0x7f002819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f0025d9b640 1 -- 172.21.15.5:0/2004658099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028100e20 msgr2=0x7f00281984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f0025d9b640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028100e20 0x7f00281984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f0025d9b640 1 -- 172.21.15.5:0/2004658099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f002819d460 con 0x7f0028102570 2024-09-15T22:16:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f0026d9d640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0028102ef0 0x7f002819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f0025d9b640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0028102570 0x7f00281989f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f001000c890 tx=0x7f001000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f001f7fe640 1 -- 172.21.15.5:0/2004658099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0010004220 con 0x7f0028102570 2024-09-15T22:16:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f001f7fe640 1 -- 172.21.15.5:0/2004658099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00100043c0 con 0x7f0028102570 2024-09-15T22:16:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f002cf91640 1 -- 172.21.15.5:0/2004658099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f002819d750 con 0x7f0028102570 2024-09-15T22:16:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f001f7fe640 1 -- 172.21.15.5:0/2004658099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00100049e0 con 0x7f0028102570 2024-09-15T22:16:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.933+0000 7f002cf91640 1 -- 172.21.15.5:0/2004658099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00281a5030 con 0x7f0028102570 2024-09-15T22:16:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.934+0000 7f002cf91640 1 -- 172.21.15.5:0/2004658099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0028101200 con 0x7f0028102570 2024-09-15T22:16:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.935+0000 7f001f7fe640 1 -- 172.21.15.5:0/2004658099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f00100079b0 con 0x7f0028102570 2024-09-15T22:16:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.936+0000 7f001f7fe640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f00140a33b0 0x7f00140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.936+0000 7f002659c640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f00140a33b0 0x7f00140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.936+0000 7f001f7fe640 1 -- 172.21.15.5:0/2004658099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f00100d01d0 con 0x7f0028102570 2024-09-15T22:16:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.937+0000 7f002659c640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f00140a33b0 0x7f00140a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f00200093d0 tx=0x7f0020009360 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:55.940+0000 7f001f7fe640 1 -- 172.21.15.5:0/2004658099 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0010014030 con 0x7f0028102570 2024-09-15T22:16:56.030 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.168+0000 7f002cf91640 1 -- 172.21.15.5:0/2004658099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0028199810 con 0x7f0028102570 2024-09-15T22:16:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.169+0000 7f001f7fe640 1 -- 172.21.15.5:0/2004658099 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f0028199810 con 0x7f0028102570 2024-09-15T22:16:56.171 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:16:56.171 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:16:56.171 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 -- 172.21.15.5:0/2004658099 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f00140a33b0 msgr2=0x7f00140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f00140a33b0 0x7f00140a5870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f00200093d0 tx=0x7f0020009360 comp rx=0 tx=0).stop 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 -- 172.21.15.5:0/2004658099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0028102570 msgr2=0x7f00281989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0028102570 0x7f00281989f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f001000c890 tx=0x7f001000cd50 comp rx=0 tx=0).stop 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 -- 172.21.15.5:0/2004658099 shutdown_connections 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f00140a33b0 0x7f00140a5870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0028102ef0 0x7f002819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0028102570 0x7f00281989f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 --2- 172.21.15.5:0/2004658099 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0028100e20 0x7f00281984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 -- 172.21.15.5:0/2004658099 >> 172.21.15.5:0/2004658099 conn(0x7f00280fc7d0 msgr2=0x7f002806a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 -- 172.21.15.5:0/2004658099 shutdown_connections 2024-09-15T22:16:56.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.171+0000 7f002cf91640 1 -- 172.21.15.5:0/2004658099 wait complete. 2024-09-15T22:16:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.691+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/2079818000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eb810c170 msgr2=0x7f8eb810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.691+0000 7f8ebf0cc640 1 --2- 172.21.15.5:0/2079818000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eb810c170 0x7f8eb810e610 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0009ca0 tx=0x7f8eb002a950 comp rx=0 tx=0).stop 2024-09-15T22:16:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.691+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/2079818000 shutdown_connections 2024-09-15T22:16:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.691+0000 7f8ebf0cc640 1 --2- 172.21.15.5:0/2079818000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eb810c170 0x7f8eb810e610 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.691+0000 7f8ebf0cc640 1 --2- 172.21.15.5:0/2079818000 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eb8103660 0x7f8eb810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.691+0000 7f8ebf0cc640 1 --2- 172.21.15.5:0/2079818000 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8eb8102d40 0x7f8eb8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.691+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/2079818000 >> 172.21.15.5:0/2079818000 conn(0x7f8eb80fc7a0 msgr2=0x7f8eb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.691+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/2079818000 shutdown_connections 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.691+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/2079818000 wait complete. 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8ebf0cc640 1 Processor -- start 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8ebf0cc640 1 -- start start 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8ebf0cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eb8102d40 0x7f8eb81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8ebf0cc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eb8103660 0x7f8eb81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8ebf0cc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8eb810c170 0x7f8eb819cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8ebf0cc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8111f30 con 0x7f8eb8102d40 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8ebf0cc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8111db0 con 0x7f8eb8103660 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8ebf0cc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81120b0 con 0x7f8eb810c170 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8ebce68640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eb8102d40 0x7f8eb81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8ebce68640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eb8102d40 0x7f8eb81983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37086/0 (socket says 172.21.15.5:37086) 2024-09-15T22:16:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8ebce68640 1 -- 172.21.15.5:0/1423673821 learned_addr learned my addr 172.21.15.5:0/1423673821 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.692+0000 7f8eb7fff640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eb8103660 0x7f8eb81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.693+0000 7f8ebd669640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8eb810c170 0x7f8eb819cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.693+0000 7f8ebce68640 1 -- 172.21.15.5:0/1423673821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8eb810c170 msgr2=0x7f8eb819cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.693+0000 7f8ebce68640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8eb810c170 0x7f8eb819cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.693+0000 7f8ebce68640 1 -- 172.21.15.5:0/1423673821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eb8103660 msgr2=0x7f8eb81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.693+0000 7f8ebce68640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eb8103660 0x7f8eb81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.693+0000 7f8ebce68640 1 -- 172.21.15.5:0/1423673821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb819d360 con 0x7f8eb8102d40 2024-09-15T22:16:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.693+0000 7f8ebd669640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8eb810c170 0x7f8eb819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.693+0000 7f8ebce68640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eb8102d40 0x7f8eb81983b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c920 tx=0x7f8ea800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.693+0000 7f8eb5ffb640 1 -- 172.21.15.5:0/1423673821 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8007c70 con 0x7f8eb8102d40 2024-09-15T22:16:56.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.693+0000 7f8eb5ffb640 1 -- 172.21.15.5:0/1423673821 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ea8007e10 con 0x7f8eb8102d40 2024-09-15T22:16:56.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.693+0000 7f8eb5ffb640 1 -- 172.21.15.5:0/1423673821 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8018890 con 0x7f8eb8102d40 2024-09-15T22:16:56.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.694+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/1423673821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb819d650 con 0x7f8eb8102d40 2024-09-15T22:16:56.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.694+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/1423673821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb81a4f30 con 0x7f8eb8102d40 2024-09-15T22:16:56.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.696+0000 7f8eb5ffb640 1 -- 172.21.15.5:0/1423673821 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8ea8018a70 con 0x7f8eb8102d40 2024-09-15T22:16:56.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8ec14a5740 1 Processor -- start 2024-09-15T22:16:56.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8ec14a5740 1 -- start start 2024-09-15T22:16:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60060 0x55747ac60440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60980 0x55747ac6ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac6d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55747abc5620 con 0x55747ac6b240 2024-09-15T22:16:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55747ab6ce50 con 0x55747ac60060 2024-09-15T22:16:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55747abc15e0 con 0x55747ac60980 2024-09-15T22:16:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8eb5ffb640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8ea00a3430 0x7f8ea00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8eb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac6d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8eb7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac6d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37100/0 (socket says 172.21.15.5:37100) 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8eb7fff640 1 -- 172.21.15.5:0/1196123658 learned_addr learned my addr 172.21.15.5:0/1196123658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8ebd669640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8ea00a3430 0x7f8ea00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8eb5ffb640 1 -- 172.21.15.5:0/1423673821 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f8ea80d0240 con 0x7f8eb8102d40 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.697+0000 7f8ebce68640 1 --2- 172.21.15.5:0/1196123658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60980 0x55747ac6ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.698+0000 7f8ebd669640 1 --2- 172.21.15.5:0/1196123658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60060 0x55747ac60440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.698+0000 7f8eb7fff640 1 -- 172.21.15.5:0/1196123658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60980 msgr2=0x55747ac6ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.698+0000 7f8eb7fff640 1 --2- 172.21.15.5:0/1196123658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60980 0x55747ac6ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.698+0000 7f8eb7fff640 1 -- 172.21.15.5:0/1196123658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60060 msgr2=0x55747ac60440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.698+0000 7f8eb7fff640 1 --2- 172.21.15.5:0/1196123658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60060 0x55747ac60440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.698+0000 7f8eb7fff640 1 -- 172.21.15.5:0/1196123658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747ab5dd40 con 0x55747ac6b240 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.698+0000 7f8ebd669640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8ea00a3430 0x7f8ea00a58f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0030040 tx=0x7f8eb0009880 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.698+0000 7f8eb7fff640 1 --2- 172.21.15.5:0/1196123658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac6d630 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ab60 tx=0x7f8eac00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8680c35cc5a2f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.698+0000 7f8e9effd640 1 -- 172.21.15.5:0/1196123658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac01d070 con 0x55747ac6b240 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.698+0000 7f8e9effd640 1 -- 172.21.15.5:0/1196123658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eac018b20 con 0x55747ac6b240 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.698+0000 7f8e9effd640 1 -- 172.21.15.5:0/1196123658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac010a00 con 0x55747ac6b240 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.699+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1196123658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 msgr2=0x55747ac6d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.699+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/1196123658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac6d630 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8eac00ab60 tx=0x7f8eac00f790 comp rx=0 tx=0).stop 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.699+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1196123658 shutdown_connections 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.699+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/1196123658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac6d630 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.699+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/1196123658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60980 0x55747ac6ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.699+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/1196123658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60060 0x55747ac60440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.699+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1196123658 >> 172.21.15.5:0/1196123658 conn(0x55747ac5c5f0 msgr2=0x55747ac5c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.699+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1196123658 shutdown_connections 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.699+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1196123658 wait complete. 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.699+0000 7f8ec14a5740 1 -- start start 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ec14a5740 1 Processor -- start 2024-09-15T22:16:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac60060 0x55747ac68100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac68640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac6b240 0x55747ac68b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55747ac6de10 con 0x55747ac60060 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55747ab4df10 con 0x55747ac60980 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55747ac6df90 con 0x55747ac6b240 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ec14a5740 10 client.0 _ll_register_callbacks cb 0x55747ab71db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ebce68640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac60060 0x55747ac68100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ebce68640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac60060 0x55747ac68100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37114/0 (socket says 172.21.15.5:37114) 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ebce68640 1 -- 172.21.15.5:0/1899427265 learned_addr learned my addr 172.21.15.5:0/1899427265 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ebd669640 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac6b240 0x55747ac68b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8eb7fff640 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac68640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ebce68640 1 -- 172.21.15.5:0/1899427265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac6b240 msgr2=0x55747ac68b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.700+0000 7f8ebce68640 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac6b240 0x55747ac68b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8ebce68640 1 -- 172.21.15.5:0/1899427265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 msgr2=0x55747ac68640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8ebce68640 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac68640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8ebce68640 1 -- 172.21.15.5:0/1899427265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747a8d4580 con 0x55747ac60060 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8eb7fff640 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac68640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8ebce68640 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac60060 0x55747ac68100 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8ea809bdc0 tx=0x7f8ea809c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9657ecf07716ee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8e97fff640 1 -- 172.21.15.5:0/1899427265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80bc0b0 con 0x55747ac60060 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8e97fff640 1 -- 172.21.15.5:0/1899427265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f8ea800fb30 con 0x55747ac60060 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1899427265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eac018b20 con 0x55747ac60060 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8e97fff640 1 -- 172.21.15.5:0/1899427265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8ea80cd7b0 con 0x55747ac60060 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8ec14a5740 10 client.16881 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1899427265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55747a99e960 con 0x55747ac60060 2024-09-15T22:16:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8e97fff640 1 -- 172.21.15.5:0/1899427265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eac018b20 con 0x55747ac60060 2024-09-15T22:16:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.701+0000 7f8e97fff640 1 -- 172.21.15.5:0/1899427265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8ea80a48a0 con 0x55747ac60060 2024-09-15T22:16:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.702+0000 7f8ec14a5740 10 client.16881 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.702+0000 7f8ec14a5740 10 client.16881 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:16:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.702+0000 7f8ec14a5740 20 client.16881 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.702+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55747ac7ae30 0x55747ac7d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.702+0000 7f8ec14a5740 4 client.16881 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.702+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1899427265 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55747abd2b90 con 0x55747ac7ae30 2024-09-15T22:16:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.702+0000 7f8eb7fff640 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55747ac7ae30 0x55747ac7d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.702+0000 7f8eb7fff640 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55747ac7ae30 0x55747ac7d2a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7f214dbaca9b788 server_cookie=35a79384453a4 in_seq=0 out_seq=0 2024-09-15T22:16:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.702+0000 7f8e97fff640 10 client.16881 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8e97fff640 1 -- 172.21.15.5:0/1899427265 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55747abd2b90 con 0x55747ac7ae30 2024-09-15T22:16:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8e97fff640 10 client.16881 handle_command_reply: tid=0 2024-09-15T22:16:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 1 client.16881 shutdown 2024-09-15T22:16:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1899427265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac60060 msgr2=0x55747ac68100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac60060 0x55747ac68100 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8ea809bdc0 tx=0x7f8ea809c180 comp rx=0 tx=0).stop 2024-09-15T22:16:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1899427265 shutdown_connections 2024-09-15T22:16:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac6b240 0x55747ac68b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55747ac7ae30 0x55747ac7d2a0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac68640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/1899427265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac60060 0x55747ac68100 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8e97fff640 0 client.16881 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1899427265 >> 172.21.15.5:0/1899427265 conn(0x55747ac5c5f0 msgr2=0x55747ac622d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1899427265 shutdown_connections 2024-09-15T22:16:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 1 -- 172.21.15.5:0/1899427265 wait complete. 2024-09-15T22:16:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.703+0000 7f8ec14a5740 20 client.16881 trim_cache size 0 max 16384 2024-09-15T22:16:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.720+0000 7f8ec14a5740 1 Processor -- start 2024-09-15T22:16:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.720+0000 7f8ec14a5740 1 -- start start 2024-09-15T22:16:56.722 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2004658099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60060 0x55747ac62ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac5cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac5b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81af750 con 0x55747ac6b240 2024-09-15T22:16:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81ad020 con 0x55747ac60980 2024-09-15T22:16:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81aae90 con 0x55747ac60060 2024-09-15T22:16:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ebd669640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac5b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ebd669640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac5b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37120/0 (socket says 172.21.15.5:37120) 2024-09-15T22:16:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ebd669640 1 -- 172.21.15.5:0/525483281 learned_addr learned my addr 172.21.15.5:0/525483281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ebce68640 1 --2- 172.21.15.5:0/525483281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60060 0x55747ac62ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8eb7fff640 1 --2- 172.21.15.5:0/525483281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac5cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ebd669640 1 -- 172.21.15.5:0/525483281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60060 msgr2=0x55747ac62ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ebd669640 1 --2- 172.21.15.5:0/525483281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60060 0x55747ac62ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ebd669640 1 -- 172.21.15.5:0/525483281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 msgr2=0x55747ac5cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ebd669640 1 --2- 172.21.15.5:0/525483281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac5cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ebd669640 1 -- 172.21.15.5:0/525483281 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747ac6a040 con 0x55747ac6b240 2024-09-15T22:16:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8ebce68640 1 --2- 172.21.15.5:0/525483281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60060 0x55747ac62ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.721+0000 7f8eb7fff640 1 --2- 172.21.15.5:0/525483281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac5cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.722+0000 7f8ebd669640 1 --2- 172.21.15.5:0/525483281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac5b640 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0035000 tx=0x7f8eb00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61db4243fc4ba047 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.722+0000 7f8e9e7fc640 1 -- 172.21.15.5:0/525483281 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0034040 con 0x55747ac6b240 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.722+0000 7f8e9e7fc640 1 -- 172.21.15.5:0/525483281 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb004f070 con 0x55747ac6b240 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.722+0000 7f8e9e7fc640 1 -- 172.21.15.5:0/525483281 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb0005030 con 0x55747ac6b240 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.723+0000 7f8ec14a5740 1 -- 172.21.15.5:0/525483281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 msgr2=0x55747ac5b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.723+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/525483281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac5b640 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0035000 tx=0x7f8eb00095c0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.723+0000 7f8ec14a5740 1 -- 172.21.15.5:0/525483281 shutdown_connections 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.723+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/525483281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac5b640 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.723+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/525483281 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac5cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.723+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/525483281 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60060 0x55747ac62ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.723+0000 7f8ec14a5740 1 -- 172.21.15.5:0/525483281 >> 172.21.15.5:0/525483281 conn(0x55747ac5c5f0 msgr2=0x55747ac5af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.723+0000 7f8ec14a5740 1 -- 172.21.15.5:0/525483281 shutdown_connections 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.723+0000 7f8ec14a5740 1 -- 172.21.15.5:0/525483281 wait complete. 2024-09-15T22:16:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.723+0000 7f8ec14a5740 1 -- start start 2024-09-15T22:16:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.723+0000 7f8ec14a5740 1 Processor -- start 2024-09-15T22:16:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60060 0x55747ac731e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac73720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ec14a5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac701b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81ab4e0 con 0x55747ac6b240 2024-09-15T22:16:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81ac950 con 0x55747ac60980 2024-09-15T22:16:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ec14a5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb81abbb0 con 0x55747ac60060 2024-09-15T22:16:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ec14a5740 10 client.0 _ll_register_callbacks cb 0x7f8ea809c7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ebd669640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac701b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ebd669640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac701b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37136/0 (socket says 172.21.15.5:37136) 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ebd669640 1 -- 172.21.15.5:0/2201683984 learned_addr learned my addr 172.21.15.5:0/2201683984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8eb7fff640 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac73720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ebce68640 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60060 0x55747ac731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ebd669640 1 -- 172.21.15.5:0/2201683984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60060 msgr2=0x55747ac731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ebd669640 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60060 0x55747ac731e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ebd669640 1 -- 172.21.15.5:0/2201683984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 msgr2=0x55747ac73720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ebd669640 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac73720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.724+0000 7f8ebd669640 1 -- 172.21.15.5:0/2201683984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747ac706f0 con 0x55747ac6b240 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.725+0000 7f8ebd669640 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac701b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8eb002fd90 tx=0x7f8eb00047f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0041cc23c0a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.725+0000 7f8e97fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.725+0000 7f8e97fff640 1 -- 172.21.15.5:0/2201683984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb005c060 con 0x55747ac6b240 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.725+0000 7f8e97fff640 1 -- 172.21.15.5:0/2201683984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f8eb0003d80 con 0x55747ac6b240 2024-09-15T22:16:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.725+0000 7f8ec14a5740 1 -- 172.21.15.5:0/2201683984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8eb004f070 con 0x55747ac6b240 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.725+0000 7f8e97fff640 1 -- 172.21.15.5:0/2201683984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8eb0063050 con 0x55747ac6b240 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.725+0000 7f8ec14a5740 10 client.16887 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.725+0000 7f8ec14a5740 1 -- 172.21.15.5:0/2201683984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55747ace6190 con 0x55747ac6b240 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.725+0000 7f8e97fff640 1 -- 172.21.15.5:0/2201683984 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8eb004f070 con 0x55747ac6b240 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.725+0000 7f8e97fff640 1 -- 172.21.15.5:0/2201683984 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8eb0054ba0 con 0x55747ac6b240 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.726+0000 7f8ec14a5740 10 client.16887 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.726+0000 7f8ec14a5740 10 client.16887 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.726+0000 7f8ec14a5740 20 client.16887 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.726+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55747ace6860 0x55747ace8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.726+0000 7f8ec14a5740 4 client.16887 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ee493997", "release": true, "await": true}] 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.726+0000 7f8ec14a5740 1 -- 172.21.15.5:0/2201683984 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ee493997", "release": true, "await": true}) -- 0x55747abcd0a0 con 0x55747ace6860 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.726+0000 7f8ebce68640 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55747ace6860 0x55747ace8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.726+0000 7f8e97fff640 10 client.16887 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.726+0000 7f8ebce68640 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55747ace6860 0x55747ace8c80 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20a80127e92830b6 server_cookie=fea4158912776d95 in_seq=0 out_seq=0 2024-09-15T22:16:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.775+0000 7f8e97fff640 1 -- 172.21.15.5:0/2201683984 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55747abcd0a0 con 0x55747ace6860 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8e97fff640 10 client.16887 handle_command_reply: tid=0 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 1 client.16887 shutdown 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 1 -- 172.21.15.5:0/2201683984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 msgr2=0x55747ac701b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac701b0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8eb002fd90 tx=0x7f8eb00047f0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 1 -- 172.21.15.5:0/2201683984 shutdown_connections 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747ac6b240 0x55747ac701b0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55747ace6860 0x55747ace8c80 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55747ac60980 0x55747ac73720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 1 --2- 172.21.15.5:0/2201683984 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55747ac60060 0x55747ac731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 1 -- 172.21.15.5:0/2201683984 >> 172.21.15.5:0/2201683984 conn(0x55747ac5c5f0 msgr2=0x55747ac688e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8e97fff640 0 client.16887 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 1 -- 172.21.15.5:0/2201683984 shutdown_connections 2024-09-15T22:16:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 1 -- 172.21.15.5:0/2201683984 wait complete. 2024-09-15T22:16:56.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.776+0000 7f8ec14a5740 20 client.16887 trim_cache size 0 max 16384 2024-09-15T22:16:56.780 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:16:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:16:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:16:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 28, 2024-09-15T22:16:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:16:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "ee493997": { 2024-09-15T22:16:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 28, 2024-09-15T22:16:56.780 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 143.7, 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:16:56.781 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:56.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:16:56.782 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/1423673821 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8ea00a3430 msgr2=0x7f8ea00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8ea00a3430 0x7f8ea00a58f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8eb0030040 tx=0x7f8eb0009880 comp rx=0 tx=0).stop 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/1423673821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eb8102d40 msgr2=0x7f8eb81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eb8102d40 0x7f8eb81983b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c920 tx=0x7f8ea800cde0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/1423673821 shutdown_connections 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8ea00a3430 0x7f8ea00a58f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8eb810c170 0x7f8eb819cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8eb8103660 0x7f8eb81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 --2- 172.21.15.5:0/1423673821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eb8102d40 0x7f8eb81983b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/1423673821 >> 172.21.15.5:0/1423673821 conn(0x7f8eb80fc7a0 msgr2=0x7f8eb80fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/1423673821 shutdown_connections 2024-09-15T22:16:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:56.777+0000 7f8ebf0cc640 1 -- 172.21.15.5:0/1423673821 wait complete. 2024-09-15T22:16:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2004658099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2004658099' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:16:57.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:16:57.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:57 smithi005 ceph-mon[30944]: pgmap v246: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 254 KiB/s wr, 37 op/s 2024-09-15T22:16:57.745 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:57 smithi129 ceph-mon[35314]: pgmap v246: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 254 KiB/s wr, 37 op/s 2024-09-15T22:16:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:57 smithi099 ceph-mon[34892]: pgmap v246: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 254 KiB/s wr, 37 op/s 2024-09-15T22:16:58.026 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ee493997', total seconds elapsed: 28.3 2024-09-15T22:16:58.123 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:16:58.525 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:16:58 smithi005 ceph-mon[30944]: pgmap v247: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-09-15T22:16:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.670+0000 7ff0f1d96640 1 -- 172.21.15.5:0/1264283843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0ec1008c0 msgr2=0x7ff0ec100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.670+0000 7ff0f1d96640 1 --2- 172.21.15.5:0/1264283843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0ec1008c0 0x7ff0ec100ca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009890 tx=0x7ff0dc02a360 comp rx=0 tx=0).stop 2024-09-15T22:16:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.680+0000 7ff0f1d96640 1 -- 172.21.15.5:0/1264283843 shutdown_connections 2024-09-15T22:16:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.680+0000 7ff0f1d96640 1 --2- 172.21.15.5:0/1264283843 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0ec105ec0 0x7ff0ec106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.680+0000 7ff0f1d96640 1 --2- 172.21.15.5:0/1264283843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ec1011e0 0x7ff0ec105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.680+0000 7ff0f1d96640 1 --2- 172.21.15.5:0/1264283843 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0ec1008c0 0x7ff0ec100ca0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.680+0000 7ff0f1d96640 1 -- 172.21.15.5:0/1264283843 >> 172.21.15.5:0/1264283843 conn(0x7ff0ec0fc7d0 msgr2=0x7ff0ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:58.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.682+0000 7ff0f1d96640 1 -- 172.21.15.5:0/1264283843 shutdown_connections 2024-09-15T22:16:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.682+0000 7ff0f1d96640 1 -- 172.21.15.5:0/1264283843 wait complete. 2024-09-15T22:16:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.682+0000 7ff0f1d96640 1 Processor -- start 2024-09-15T22:16:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.683+0000 7ff0f1d96640 1 -- start start 2024-09-15T22:16:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.683+0000 7ff0f1d96640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0ec1008c0 0x7ff0ec114d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.683+0000 7ff0f1d96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ec1011e0 0x7ff0ec10ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.683+0000 7ff0f1d96640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0ec105ec0 0x7ff0ec10e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.683+0000 7ff0f1d96640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec118660 con 0x7ff0ec1011e0 2024-09-15T22:16:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.683+0000 7ff0f1d96640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1184e0 con 0x7ff0ec105ec0 2024-09-15T22:16:58.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.683+0000 7ff0f1d96640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1187e0 con 0x7ff0ec1008c0 2024-09-15T22:16:58.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.685+0000 7ff0eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ec1011e0 0x7ff0ec10ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.685+0000 7ff0eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ec1011e0 0x7ff0ec10ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37150/0 (socket says 172.21.15.5:37150) 2024-09-15T22:16:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.685+0000 7ff0eaffd640 1 -- 172.21.15.5:0/433598363 learned_addr learned my addr 172.21.15.5:0/433598363 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.685+0000 7ff0eaffd640 1 -- 172.21.15.5:0/433598363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0ec1008c0 msgr2=0x7ff0ec114d60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:16:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.685+0000 7ff0eaffd640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0ec1008c0 0x7ff0ec114d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.685+0000 7ff0eaffd640 1 -- 172.21.15.5:0/433598363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0ec105ec0 msgr2=0x7ff0ec10e330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:16:58.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.685+0000 7ff0eaffd640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0ec105ec0 0x7ff0ec10e330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.685+0000 7ff0eaffd640 1 -- 172.21.15.5:0/433598363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ec10e970 con 0x7ff0ec1011e0 2024-09-15T22:16:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.686+0000 7ff0eaffd640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ec1011e0 0x7ff0ec10ddf0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400c420 tx=0x7ff0e4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:58.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.686+0000 7ff0e8ff9640 1 -- 172.21.15.5:0/433598363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4003970 con 0x7ff0ec1011e0 2024-09-15T22:16:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.686+0000 7ff0e8ff9640 1 -- 172.21.15.5:0/433598363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e4004020 con 0x7ff0ec1011e0 2024-09-15T22:16:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.686+0000 7ff0e8ff9640 1 -- 172.21.15.5:0/433598363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e400bad0 con 0x7ff0ec1011e0 2024-09-15T22:16:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.687+0000 7ff0f1d96640 1 -- 172.21.15.5:0/433598363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ec112890 con 0x7ff0ec1011e0 2024-09-15T22:16:58.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.687+0000 7ff0f1d96640 1 -- 172.21.15.5:0/433598363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ec112d80 con 0x7ff0ec1011e0 2024-09-15T22:16:58.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.696+0000 7ff0f416f740 1 Processor -- start 2024-09-15T22:16:58.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.696+0000 7ff0e8ff9640 1 -- 172.21.15.5:0/433598363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff0e401d020 con 0x7ff0ec1011e0 2024-09-15T22:16:58.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.697+0000 7ff0e8ff9640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0d40a3540 0x7ff0d40a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.697+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0d40a3540 0x7ff0d40a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.697+0000 7ff0f416f740 1 -- start start 2024-09-15T22:16:58.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.698+0000 7ff0f416f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a013090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.698+0000 7ff0f416f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a0135d0 0x55c63a013a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.698+0000 7ff0eb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a013090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.698+0000 7ff0eb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a013090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:59660/0 (socket says 172.21.15.5:59660) 2024-09-15T22:16:58.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.698+0000 7ff0eb7fe640 1 -- 172.21.15.5:0/720360414 learned_addr learned my addr 172.21.15.5:0/720360414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:58.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.698+0000 7ff0f416f740 1 --2- 172.21.15.5:0/720360414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a020760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.698+0000 7ff0f416f740 1 -- 172.21.15.5:0/720360414 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c639f792f0 con 0x55c63a0135d0 2024-09-15T22:16:58.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.698+0000 7ff0f416f740 1 -- 172.21.15.5:0/720360414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c639f20cf0 con 0x55c63a014b40 2024-09-15T22:16:58.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.698+0000 7ff0f416f740 1 -- 172.21.15.5:0/720360414 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c639f752b0 con 0x55c63a01e370 2024-09-15T22:16:58.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.699+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/720360414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a020760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.699+0000 7ff0e8ff9640 1 -- 172.21.15.5:0/433598363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7ff0e4099a10 con 0x7ff0ec1011e0 2024-09-15T22:16:58.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.699+0000 7ff0eaffd640 1 --2- 172.21.15.5:0/720360414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a0135d0 0x55c63a013a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.699+0000 7ff0eb7fe640 1 -- 172.21.15.5:0/720360414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 msgr2=0x55c63a020760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:58.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.699+0000 7ff0eb7fe640 1 --2- 172.21.15.5:0/720360414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a020760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.699+0000 7ff0eb7fe640 1 -- 172.21.15.5:0/720360414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a0135d0 msgr2=0x55c63a013a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:58.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.699+0000 7ff0eb7fe640 1 --2- 172.21.15.5:0/720360414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a0135d0 0x55c63a013a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.699+0000 7ff0eb7fe640 1 -- 172.21.15.5:0/720360414 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c639f053a0 con 0x55c63a014b40 2024-09-15T22:16:58.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.700+0000 7ff0eb7fe640 1 --2- 172.21.15.5:0/720360414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a013090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc02a8d0 tx=0x7ff0dc009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c72aed4409d5be71 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:58.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.701+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0d40a3540 0x7ff0d40a5a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009b20 tx=0x7ff0e0009f20 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:58.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.701+0000 7ff0d1ffb640 1 -- 172.21.15.5:0/720360414 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc037040 con 0x55c63a014b40 2024-09-15T22:16:58.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.701+0000 7ff0d1ffb640 1 -- 172.21.15.5:0/720360414 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0dc030e80 con 0x55c63a014b40 2024-09-15T22:16:58.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.701+0000 7ff0d1ffb640 1 -- 172.21.15.5:0/720360414 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc005270 con 0x55c63a014b40 2024-09-15T22:16:58.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.702+0000 7ff0f416f740 1 -- 172.21.15.5:0/720360414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 msgr2=0x55c63a013090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:58.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.702+0000 7ff0f416f740 1 --2- 172.21.15.5:0/720360414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a013090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc02a8d0 tx=0x7ff0dc009450 comp rx=0 tx=0).stop 2024-09-15T22:16:58.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.703+0000 7ff0f416f740 1 -- 172.21.15.5:0/720360414 shutdown_connections 2024-09-15T22:16:58.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.703+0000 7ff0f416f740 1 --2- 172.21.15.5:0/720360414 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a020760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.703+0000 7ff0f416f740 1 --2- 172.21.15.5:0/720360414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a0135d0 0x55c63a013a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.703+0000 7ff0f416f740 1 --2- 172.21.15.5:0/720360414 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a013090 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.703+0000 7ff0f416f740 1 -- 172.21.15.5:0/720360414 >> 172.21.15.5:0/720360414 conn(0x55c639fdc310 msgr2=0x55c63a00fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:58.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.704+0000 7ff0f416f740 1 -- 172.21.15.5:0/720360414 shutdown_connections 2024-09-15T22:16:58.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.704+0000 7ff0f416f740 1 -- 172.21.15.5:0/720360414 wait complete. 2024-09-15T22:16:58.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.704+0000 7ff0f416f740 1 -- start start 2024-09-15T22:16:58.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.705+0000 7ff0f416f740 1 Processor -- start 2024-09-15T22:16:58.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.705+0000 7ff0f416f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a0135d0 0x55c63a01dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.706+0000 7ff0f416f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a01afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.706+0000 7ff0f416f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a01e370 0x55c63a01b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.706+0000 7ff0f416f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c63a021060 con 0x55c63a01e370 2024-09-15T22:16:58.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.706+0000 7ff0f416f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c639f34060 con 0x55c63a014b40 2024-09-15T22:16:58.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.706+0000 7ff0f416f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c63a0211e0 con 0x55c63a0135d0 2024-09-15T22:16:58.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.706+0000 7ff0f416f740 10 client.0 _ll_register_callbacks cb 0x55c639c878c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:58.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.707+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a01e370 0x55c63a01b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.707+0000 7ff0ebfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a01e370 0x55c63a01b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37172/0 (socket says 172.21.15.5:37172) 2024-09-15T22:16:58.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.707+0000 7ff0ebfff640 1 -- 172.21.15.5:0/297518869 learned_addr learned my addr 172.21.15.5:0/297518869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.708+0000 7ff0ebfff640 1 -- 172.21.15.5:0/297518869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a0135d0 msgr2=0x55c63a01dcf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.708+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/297518869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a0135d0 0x55c63a01dcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.708+0000 7ff0ebfff640 1 -- 172.21.15.5:0/297518869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 msgr2=0x55c63a01afa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.708+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/297518869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a01afa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.708+0000 7ff0ebfff640 1 -- 172.21.15.5:0/297518869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c639cea3c0 con 0x55c63a01e370 2024-09-15T22:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.709+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/297518869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a01e370 0x55c63a01b520 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000f5b0 tx=0x7ff0e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7394abab7cd420e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:58.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.709+0000 7ff0caffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.710+0000 7ff0caffd640 1 -- 172.21.15.5:0/297518869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0022070 con 0x55c63a01e370 2024-09-15T22:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.710+0000 7ff0caffd640 1 -- 172.21.15.5:0/297518869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7ff0e001e680 con 0x55c63a01e370 2024-09-15T22:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.710+0000 7ff0f416f740 1 -- 172.21.15.5:0/297518869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0dc030e80 con 0x55c63a01e370 2024-09-15T22:16:58.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.710+0000 7ff0caffd640 1 -- 172.21.15.5:0/297518869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0e002c050 con 0x55c63a01e370 2024-09-15T22:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.711+0000 7ff0f416f740 10 client.16899 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:58.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.711+0000 7ff0f416f740 1 -- 172.21.15.5:0/297518869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c63a02cb90 con 0x55c63a01e370 2024-09-15T22:16:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.711+0000 7ff0caffd640 1 -- 172.21.15.5:0/297518869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e001ed60 con 0x55c63a01e370 2024-09-15T22:16:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.711+0000 7ff0caffd640 1 -- 172.21.15.5:0/297518869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff0e001b050 con 0x55c63a01e370 2024-09-15T22:16:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.711+0000 7ff0f416f740 10 client.16899 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.711+0000 7ff0f416f740 10 client.16899 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:16:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.711+0000 7ff0f416f740 20 client.16899 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.711+0000 7ff0f416f740 1 --2- 172.21.15.5:0/297518869 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55c63a02e050 0x55c63a0304a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.712+0000 7ff0f416f740 4 client.16899 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:16:58.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.712+0000 7ff0f416f740 1 -- 172.21.15.5:0/297518869 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff0dc030e80 con 0x55c63a02e050 2024-09-15T22:16:58.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.714+0000 7ff0eb7fe640 1 --2- 172.21.15.5:0/297518869 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55c63a02e050 0x55c63a0304a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.716+0000 7ff0eb7fe640 1 --2- 172.21.15.5:0/297518869 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55c63a02e050 0x55c63a0304a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8c80d8158bd3d261 server_cookie=60e4ba428d6f8a5b in_seq=0 out_seq=0 2024-09-15T22:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.716+0000 7ff0caffd640 10 client.16899 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:16:58.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.717+0000 7ff0caffd640 1 -- 172.21.15.5:0/297518869 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff0dc030e80 con 0x55c63a02e050 2024-09-15T22:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.717+0000 7ff0caffd640 10 client.16899 handle_command_reply: tid=0 2024-09-15T22:16:58.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.717+0000 7ff0f416f740 1 client.16899 shutdown 2024-09-15T22:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.717+0000 7ff0f416f740 1 -- 172.21.15.5:0/297518869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a01e370 msgr2=0x55c63a01b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.717+0000 7ff0f416f740 1 --2- 172.21.15.5:0/297518869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a01e370 0x55c63a01b520 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff0e000f5b0 tx=0x7ff0e00046c0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.717+0000 7ff0f416f740 1 -- 172.21.15.5:0/297518869 shutdown_connections 2024-09-15T22:16:58.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.717+0000 7ff0f416f740 1 --2- 172.21.15.5:0/297518869 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55c63a02e050 0x55c63a0304a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.718+0000 7ff0f416f740 1 --2- 172.21.15.5:0/297518869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a01e370 0x55c63a01b520 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.718+0000 7ff0caffd640 0 client.16899 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:16:58.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.718+0000 7ff0f416f740 1 --2- 172.21.15.5:0/297518869 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a01afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.718+0000 7ff0f416f740 1 --2- 172.21.15.5:0/297518869 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a0135d0 0x55c63a01dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.718+0000 7ff0f416f740 1 -- 172.21.15.5:0/297518869 >> 172.21.15.5:0/297518869 conn(0x55c639fdc310 msgr2=0x55c63a015960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:58.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.718+0000 7ff0f416f740 1 -- 172.21.15.5:0/297518869 shutdown_connections 2024-09-15T22:16:58.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.718+0000 7ff0f416f740 1 -- 172.21.15.5:0/297518869 wait complete. 2024-09-15T22:16:58.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.718+0000 7ff0f416f740 20 client.16899 trim_cache size 0 max 16384 2024-09-15T22:16:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.735+0000 7ff0f416f740 1 Processor -- start 2024-09-15T22:16:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0f416f740 1 -- start start 2024-09-15T22:16:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0f416f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a0135d0 0x55c63a02a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0f416f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a019870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0f416f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a019db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0f416f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1c9230 con 0x55c63a0135d0 2024-09-15T22:16:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0f416f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1c6b00 con 0x55c63a014b40 2024-09-15T22:16:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0f416f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec105980 con 0x55c63a01e370 2024-09-15T22:16:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a0135d0 0x55c63a02a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a0135d0 0x55c63a02a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37186/0 (socket says 172.21.15.5:37186) 2024-09-15T22:16:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0eb7fe640 1 -- 172.21.15.5:0/100883840 learned_addr learned my addr 172.21.15.5:0/100883840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/100883840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a019db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.736+0000 7ff0eaffd640 1 --2- 172.21.15.5:0/100883840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a019870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.737+0000 7ff0eaffd640 1 -- 172.21.15.5:0/100883840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 msgr2=0x55c63a019db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.737+0000 7ff0eaffd640 1 --2- 172.21.15.5:0/100883840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a019db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.737+0000 7ff0eaffd640 1 -- 172.21.15.5:0/100883840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a0135d0 msgr2=0x55c63a02a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.737+0000 7ff0eaffd640 1 --2- 172.21.15.5:0/100883840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a0135d0 0x55c63a02a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.737+0000 7ff0eaffd640 1 -- 172.21.15.5:0/100883840 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c639f8ce70 con 0x55c63a014b40 2024-09-15T22:16:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.737+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/100883840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a019db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:16:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.737+0000 7ff0eaffd640 1 --2- 172.21.15.5:0/100883840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a019870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40ccde0 tx=0x7ff0e409b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab621ca73f087a42 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.738+0000 7ff0d17fa640 1 -- 172.21.15.5:0/100883840 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e40d4060 con 0x55c63a014b40 2024-09-15T22:16:58.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.738+0000 7ff0d17fa640 1 -- 172.21.15.5:0/100883840 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e409c070 con 0x55c63a014b40 2024-09-15T22:16:58.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.738+0000 7ff0d17fa640 1 -- 172.21.15.5:0/100883840 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e40cd060 con 0x55c63a014b40 2024-09-15T22:16:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.739+0000 7ff0f416f740 1 -- 172.21.15.5:0/100883840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 msgr2=0x55c63a019870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.739+0000 7ff0f416f740 1 --2- 172.21.15.5:0/100883840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a019870 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40ccde0 tx=0x7ff0e409b850 comp rx=0 tx=0).stop 2024-09-15T22:16:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.739+0000 7ff0f416f740 1 -- 172.21.15.5:0/100883840 shutdown_connections 2024-09-15T22:16:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.739+0000 7ff0f416f740 1 --2- 172.21.15.5:0/100883840 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a019db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.739+0000 7ff0f416f740 1 --2- 172.21.15.5:0/100883840 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a014b40 0x55c63a019870 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.739+0000 7ff0f416f740 1 --2- 172.21.15.5:0/100883840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a0135d0 0x55c63a02a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.739+0000 7ff0f416f740 1 -- 172.21.15.5:0/100883840 >> 172.21.15.5:0/100883840 conn(0x55c639fdc310 msgr2=0x55c63a01af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.739+0000 7ff0f416f740 1 -- 172.21.15.5:0/100883840 shutdown_connections 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.739+0000 7ff0f416f740 1 -- 172.21.15.5:0/100883840 wait complete. 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.740+0000 7ff0f416f740 1 -- start start 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.740+0000 7ff0f416f740 1 Processor -- start 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0f416f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a0135d0 0x55c63a0267c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0f416f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a014b40 0x55c63a026d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0f416f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a021bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0f416f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec0738a0 con 0x55c63a014b40 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0f416f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec1c6430 con 0x55c63a0135d0 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0f416f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ec073f50 con 0x55c63a01e370 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a014b40 0x55c63a026d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0f416f740 10 client.0 _ll_register_callbacks cb 0x7ff0e0012b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0eaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a014b40 0x55c63a026d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37202/0 (socket says 172.21.15.5:37202) 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0eaffd640 1 -- 172.21.15.5:0/1346524404 learned_addr learned my addr 172.21.15.5:0/1346524404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a021bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0ebfff640 1 -- 172.21.15.5:0/1346524404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a0135d0 msgr2=0x55c63a0267c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a0135d0 0x55c63a0267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0ebfff640 1 -- 172.21.15.5:0/1346524404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a014b40 msgr2=0x55c63a026d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a014b40 0x55c63a026d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.741+0000 7ff0ebfff640 1 -- 172.21.15.5:0/1346524404 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c63a0223f0 con 0x55c63a01e370 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.742+0000 7ff0ebfff640 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a021bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00123c0 tx=0x7ff0e0027670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a17c905b6be2fe23 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.742+0000 7ff0caffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.742+0000 7ff0caffd640 1 -- 172.21.15.5:0/1346524404 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0035070 con 0x55c63a01e370 2024-09-15T22:16:58.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.742+0000 7ff0caffd640 1 -- 172.21.15.5:0/1346524404 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7ff0e0027c40 con 0x55c63a01e370 2024-09-15T22:16:58.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.742+0000 7ff0f416f740 1 -- 172.21.15.5:0/1346524404 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0e409c070 con 0x55c63a01e370 2024-09-15T22:16:58.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.742+0000 7ff0caffd640 1 -- 172.21.15.5:0/1346524404 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0e0041050 con 0x55c63a01e370 2024-09-15T22:16:58.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.742+0000 7ff0f416f740 10 client.26368 fetch_fsmap learned FSMap version 25 2024-09-15T22:16:58.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.742+0000 7ff0f416f740 1 -- 172.21.15.5:0/1346524404 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c63a0988e0 con 0x55c63a01e370 2024-09-15T22:16:58.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.743+0000 7ff0caffd640 1 -- 172.21.15.5:0/1346524404 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e0009840 con 0x55c63a01e370 2024-09-15T22:16:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.744+0000 7ff0caffd640 1 -- 172.21.15.5:0/1346524404 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff0e001b040 con 0x55c63a01e370 2024-09-15T22:16:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.744+0000 7ff0f416f740 10 client.26368 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:16:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.744+0000 7ff0f416f740 10 client.26368 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:16:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.744+0000 7ff0f416f740 20 client.26368 populate_metadata read hostname 'smithi005' 2024-09-15T22:16:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.744+0000 7ff0f416f740 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55c63a02e050 0x55c63a09afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:16:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.744+0000 7ff0f416f740 4 client.26368 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:16:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.744+0000 7ff0f416f740 1 -- 172.21.15.5:0/1346524404 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff0e409c070 con 0x55c63a02e050 2024-09-15T22:16:58.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.744+0000 7ff0eb7fe640 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55c63a02e050 0x55c63a09afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.744+0000 7ff0eb7fe640 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55c63a02e050 0x55c63a09afb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=832e4475986a3c91 server_cookie=9f91d60640afa09a in_seq=0 out_seq=0 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.744+0000 7ff0caffd640 10 client.26368 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0caffd640 1 -- 172.21.15.5:0/1346524404 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff0e409c070 con 0x55c63a02e050 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0caffd640 10 client.26368 handle_command_reply: tid=0 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0f416f740 1 client.26368 shutdown 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0f416f740 1 -- 172.21.15.5:0/1346524404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 msgr2=0x55c63a021bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0f416f740 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a021bf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff0e00123c0 tx=0x7ff0e0027670 comp rx=0 tx=0).stop 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0f416f740 1 -- 172.21.15.5:0/1346524404 shutdown_connections 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0f416f740 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55c63a02e050 0x55c63a09afb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0f416f740 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c63a01e370 0x55c63a021bf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0caffd640 0 client.26368 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0f416f740 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c63a014b40 0x55c63a026d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0f416f740 1 --2- 172.21.15.5:0/1346524404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c63a0135d0 0x55c63a0267c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0f416f740 1 -- 172.21.15.5:0/1346524404 >> 172.21.15.5:0/1346524404 conn(0x55c639fdc310 msgr2=0x55c63a019830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0f416f740 1 -- 172.21.15.5:0/1346524404 shutdown_connections 2024-09-15T22:16:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.745+0000 7ff0f416f740 1 -- 172.21.15.5:0/1346524404 wait complete. 2024-09-15T22:16:58.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.746+0000 7ff0f416f740 20 client.26368 trim_cache size 0 max 16384 2024-09-15T22:16:58.751 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:16:58.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.746+0000 7ff0f1d96640 1 -- 172.21.15.5:0/433598363 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0d40a3540 msgr2=0x7ff0d40a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.747+0000 7ff0f1d96640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0d40a3540 0x7ff0d40a5a00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009b20 tx=0x7ff0e0009f20 comp rx=0 tx=0).stop 2024-09-15T22:16:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.747+0000 7ff0f1d96640 1 -- 172.21.15.5:0/433598363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ec1011e0 msgr2=0x7ff0ec10ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:16:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.747+0000 7ff0f1d96640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ec1011e0 0x7ff0ec10ddf0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400c420 tx=0x7ff0e4009f90 comp rx=0 tx=0).stop 2024-09-15T22:16:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.747+0000 7ff0f1d96640 1 -- 172.21.15.5:0/433598363 shutdown_connections 2024-09-15T22:16:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.747+0000 7ff0f1d96640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff0d40a3540 0x7ff0d40a5a00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.747+0000 7ff0f1d96640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff0ec105ec0 0x7ff0ec10e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.747+0000 7ff0f1d96640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0ec1011e0 0x7ff0ec10ddf0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.747+0000 7ff0f1d96640 1 --2- 172.21.15.5:0/433598363 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff0ec1008c0 0x7ff0ec114d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:16:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.747+0000 7ff0f1d96640 1 -- 172.21.15.5:0/433598363 >> 172.21.15.5:0/433598363 conn(0x7ff0ec0fc7d0 msgr2=0x7ff0ec103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:16:58.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.747+0000 7ff0f1d96640 1 -- 172.21.15.5:0/433598363 shutdown_connections 2024-09-15T22:16:58.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:16:58.747+0000 7ff0f1d96640 1 -- 172.21.15.5:0/433598363 wait complete. 2024-09-15T22:16:58.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:16:58 smithi099 ceph-mon[34892]: pgmap v247: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-09-15T22:16:58.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:16:58 smithi129 ceph-mon[35314]: pgmap v247: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-09-15T22:16:59.661 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:16:59.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 450.0 s, 353.3 tps, lat 1044.724 ms stddev 3628.880 2024-09-15T22:17:00.160 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:00 smithi005 ceph-mon[30944]: pgmap v248: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-09-15T22:17:01.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:00 smithi099 ceph-mon[34892]: pgmap v248: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-09-15T22:17:01.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:00 smithi129 ceph-mon[35314]: pgmap v248: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 44 KiB/s wr, 10 op/s 2024-09-15T22:17:03.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:02 smithi099 ceph-mon[34892]: pgmap v249: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-09-15T22:17:03.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:02 smithi129 ceph-mon[35314]: pgmap v249: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-09-15T22:17:03.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:02 smithi005 ceph-mon[30944]: pgmap v249: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-09-15T22:17:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.336+0000 7fc0ecf39640 1 -- 172.21.15.5:0/1241320101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0030d30 con 0x7fc0e81019c0 2024-09-15T22:17:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.337+0000 7fc0eef3d640 1 -- 172.21.15.5:0/1241320101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e81019c0 msgr2=0x7fc0e8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.337+0000 7fc0eef3d640 1 --2- 172.21.15.5:0/1241320101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e81019c0 0x7fc0e8101e20 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009cd0 tx=0x7fc0e002a8f0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.337+0000 7fc0eef3d640 1 -- 172.21.15.5:0/1241320101 shutdown_connections 2024-09-15T22:17:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.337+0000 7fc0eef3d640 1 --2- 172.21.15.5:0/1241320101 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0e8102360 0x7fc0e810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.337+0000 7fc0eef3d640 1 --2- 172.21.15.5:0/1241320101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e81019c0 0x7fc0e8101e20 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.337+0000 7fc0eef3d640 1 --2- 172.21.15.5:0/1241320101 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0e81079c0 0x7fc0e8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.337+0000 7fc0eef3d640 1 -- 172.21.15.5:0/1241320101 >> 172.21.15.5:0/1241320101 conn(0x7fc0e80fd740 msgr2=0x7fc0e80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.337+0000 7fc0eef3d640 1 -- 172.21.15.5:0/1241320101 shutdown_connections 2024-09-15T22:17:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.337+0000 7fc0eef3d640 1 -- 172.21.15.5:0/1241320101 wait complete. 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.338+0000 7fc0eef3d640 1 Processor -- start 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.338+0000 7fc0eef3d640 1 -- start start 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.338+0000 7fc0eef3d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0e81019c0 0x7fc0e8198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.338+0000 7fc0eef3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e8102360 0x7fc0e8198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0eef3d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0e81079c0 0x7fc0e819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0ed73a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e8102360 0x7fc0e8198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0edf3b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0e81019c0 0x7fc0e8198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0ed73a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e8102360 0x7fc0e8198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37216/0 (socket says 172.21.15.5:37216) 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0ed73a640 1 -- 172.21.15.5:0/2855389299 learned_addr learned my addr 172.21.15.5:0/2855389299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0eef3d640 1 -- 172.21.15.5:0/2855389299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8111e30 con 0x7fc0e8102360 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0eef3d640 1 -- 172.21.15.5:0/2855389299 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8111cb0 con 0x7fc0e81079c0 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0eef3d640 1 -- 172.21.15.5:0/2855389299 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e8111fb0 con 0x7fc0e81019c0 2024-09-15T22:17:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0ee73c640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0e81079c0 0x7fc0e819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0ee73c640 1 -- 172.21.15.5:0/2855389299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0e81019c0 msgr2=0x7fc0e8198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0ee73c640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0e81019c0 0x7fc0e8198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0ee73c640 1 -- 172.21.15.5:0/2855389299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e8102360 msgr2=0x7fc0e8198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0ee73c640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e8102360 0x7fc0e8198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.339+0000 7fc0ee73c640 1 -- 172.21.15.5:0/2855389299 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0e819d2b0 con 0x7fc0e81079c0 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.340+0000 7fc0ee73c640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0e81079c0 0x7fc0e819cbd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400b410 tx=0x7fc0e400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.340+0000 7fc0deffd640 1 -- 172.21.15.5:0/2855389299 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4015020 con 0x7fc0e81079c0 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.340+0000 7fc0deffd640 1 -- 172.21.15.5:0/2855389299 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e4011d10 con 0x7fc0e81079c0 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.340+0000 7fc0deffd640 1 -- 172.21.15.5:0/2855389299 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400f2e0 con 0x7fc0e81079c0 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.340+0000 7fc0eef3d640 1 -- 172.21.15.5:0/2855389299 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0e819d5a0 con 0x7fc0e81079c0 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.340+0000 7fc0eef3d640 1 -- 172.21.15.5:0/2855389299 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0e8074420 con 0x7fc0e81079c0 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.343+0000 7fc0deffd640 1 -- 172.21.15.5:0/2855389299 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc0e4018020 con 0x7fc0e81079c0 2024-09-15T22:17:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.344+0000 7fc0f2578740 1 Processor -- start 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.345+0000 7fc0deffd640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0d40a34b0 0x7fc0d40a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.345+0000 7fc0f2578740 1 -- start start 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.345+0000 7fc0ed73a640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0d40a34b0 0x7fc0d40a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251f8b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561251f8bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251f98910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0f2578740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561251ef12f0 con 0x561251f8b780 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0f2578740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561251e98cf0 con 0x561251f96520 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0f2578740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561251eed2b0 con 0x561251f8ccf0 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0deffd640 1 -- 172.21.15.5:0/2855389299 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fc0e40cf770 con 0x7fc0e81079c0 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0ee73c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251f98910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0ee73c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251f98910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:59718/0 (socket says 172.21.15.5:59718) 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0ee73c640 1 -- 172.21.15.5:0/1508719536 learned_addr learned my addr 172.21.15.5:0/1508719536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0edf3b640 1 --2- 172.21.15.5:0/1508719536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251f8b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0ed73a640 1 --2- 172.21.15.5:0/1508719536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561251f8bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0ee73c640 1 -- 172.21.15.5:0/1508719536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 msgr2=0x561251f8b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0ed73a640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0d40a34b0 0x7fc0d40a5970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009a60 tx=0x7fc0e0009820 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0ee73c640 1 --2- 172.21.15.5:0/1508719536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251f8b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0ee73c640 1 -- 172.21.15.5:0/1508719536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 msgr2=0x561251f8bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0ee73c640 1 --2- 172.21.15.5:0/1508719536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561251f8bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0ee73c640 1 -- 172.21.15.5:0/1508719536 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561251e7d3a0 con 0x561251f96520 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0edf3b640 1 --2- 172.21.15.5:0/1508719536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251f8b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.346+0000 7fc0ee73c640 1 --2- 172.21.15.5:0/1508719536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251f98910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40a3e40 tx=0x7fc0e409b300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea70fe73971323c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.347+0000 7fc0d3fff640 1 -- 172.21.15.5:0/1508719536 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e409b700 con 0x561251f96520 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.347+0000 7fc0d3fff640 1 -- 172.21.15.5:0/1508719536 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e409fab0 con 0x561251f96520 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.347+0000 7fc0d3fff640 1 -- 172.21.15.5:0/1508719536 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40cf260 con 0x561251f96520 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.348+0000 7fc0f2578740 1 -- 172.21.15.5:0/1508719536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 msgr2=0x561251f98910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.348+0000 7fc0f2578740 1 --2- 172.21.15.5:0/1508719536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251f98910 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc0e40a3e40 tx=0x7fc0e409b300 comp rx=0 tx=0).stop 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.348+0000 7fc0f2578740 1 -- 172.21.15.5:0/1508719536 shutdown_connections 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.348+0000 7fc0f2578740 1 --2- 172.21.15.5:0/1508719536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251f98910 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.348+0000 7fc0f2578740 1 --2- 172.21.15.5:0/1508719536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561251f8bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.348+0000 7fc0f2578740 1 --2- 172.21.15.5:0/1508719536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251f8b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.348+0000 7fc0f2578740 1 -- 172.21.15.5:0/1508719536 >> 172.21.15.5:0/1508719536 conn(0x561251f54310 msgr2=0x561251f87e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.349+0000 7fc0f2578740 1 -- 172.21.15.5:0/1508719536 shutdown_connections 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.349+0000 7fc0f2578740 1 -- 172.21.15.5:0/1508719536 wait complete. 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.349+0000 7fc0f2578740 1 -- start start 2024-09-15T22:17:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.349+0000 7fc0f2578740 1 Processor -- start 2024-09-15T22:17:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.350+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8b780 0x561251f95f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.350+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f8ccf0 0x561251f93230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f96520 0x561251f939c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0f2578740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561251f991c0 con 0x561251f96520 2024-09-15T22:17:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0ed73a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f8ccf0 0x561251f93230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0f2578740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561251eac060 con 0x561251f8ccf0 2024-09-15T22:17:04.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0ed73a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f8ccf0 0x561251f93230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:59728/0 (socket says 172.21.15.5:59728) 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0ed73a640 1 -- 172.21.15.5:0/3843056189 learned_addr learned my addr 172.21.15.5:0/3843056189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0f2578740 1 -- 172.21.15.5:0/3843056189 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561251f99340 con 0x561251f8b780 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0f2578740 10 client.0 _ll_register_callbacks cb 0x561251bff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0ee73c640 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f96520 0x561251f939c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0edf3b640 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8b780 0x561251f95f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0ed73a640 1 -- 172.21.15.5:0/3843056189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8b780 msgr2=0x561251f95f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0ed73a640 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8b780 0x561251f95f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0ed73a640 1 -- 172.21.15.5:0/3843056189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f96520 msgr2=0x561251f939c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0ed73a640 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f96520 0x561251f939c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0ed73a640 1 -- 172.21.15.5:0/3843056189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561251c623c0 con 0x561251f8ccf0 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0edf3b640 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8b780 0x561251f95f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0ed73a640 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f8ccf0 0x561251f93230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0006010 tx=0x7fc0e002a8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38479af4e9312a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.351+0000 7fc0d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:17:04.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.352+0000 7fc0d17fa640 1 -- 172.21.15.5:0/3843056189 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0003e00 con 0x561251f8ccf0 2024-09-15T22:17:04.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.352+0000 7fc0d17fa640 1 -- 172.21.15.5:0/3843056189 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fc0e0004040 con 0x561251f8ccf0 2024-09-15T22:17:04.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.352+0000 7fc0f2578740 1 -- 172.21.15.5:0/3843056189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0e409fab0 con 0x561251f8ccf0 2024-09-15T22:17:04.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.352+0000 7fc0d17fa640 1 -- 172.21.15.5:0/3843056189 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc0e0004280 con 0x561251f8ccf0 2024-09-15T22:17:04.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.352+0000 7fc0f2578740 10 client.26426 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:04.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.352+0000 7fc0f2578740 1 -- 172.21.15.5:0/3843056189 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561251fad4f0 con 0x561251f8ccf0 2024-09-15T22:17:04.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.352+0000 7fc0d17fa640 1 -- 172.21.15.5:0/3843056189 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0e0031440 con 0x561251f8ccf0 2024-09-15T22:17:04.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.352+0000 7fc0d17fa640 1 -- 172.21.15.5:0/3843056189 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc0e0043040 con 0x561251f8ccf0 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.352+0000 7fc0f2578740 10 client.26426 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.352+0000 7fc0f2578740 10 client.26426 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.353+0000 7fc0f2578740 20 client.26426 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.353+0000 7fc0f2578740 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561251fae900 0x561251fb0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.353+0000 7fc0f2578740 4 client.26426 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.353+0000 7fc0f2578740 1 -- 172.21.15.5:0/3843056189 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc0e409fab0 con 0x561251fae900 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.353+0000 7fc0edf3b640 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561251fae900 0x561251fb0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.353+0000 7fc0d17fa640 10 client.26426 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.353+0000 7fc0edf3b640 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561251fae900 0x561251fb0da0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7268996098ba74fe server_cookie=20416e50ce5db927 in_seq=0 out_seq=0 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0d17fa640 1 -- 172.21.15.5:0/3843056189 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc0e409fab0 con 0x561251fae900 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0d17fa640 10 client.26426 handle_command_reply: tid=0 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0f2578740 1 client.26426 shutdown 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0f2578740 1 -- 172.21.15.5:0/3843056189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f8ccf0 msgr2=0x561251f93230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0f2578740 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f8ccf0 0x561251f93230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0006010 tx=0x7fc0e002a8f0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0f2578740 1 -- 172.21.15.5:0/3843056189 shutdown_connections 2024-09-15T22:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0f2578740 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561251fae900 0x561251fb0da0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0f2578740 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f96520 0x561251f939c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0f2578740 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f8ccf0 0x561251f93230 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0f2578740 1 --2- 172.21.15.5:0/3843056189 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8b780 0x561251f95f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0f2578740 1 -- 172.21.15.5:0/3843056189 >> 172.21.15.5:0/3843056189 conn(0x561251f54310 msgr2=0x561251f8db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.354+0000 7fc0d17fa640 0 client.26426 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.355+0000 7fc0f2578740 1 -- 172.21.15.5:0/3843056189 shutdown_connections 2024-09-15T22:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.355+0000 7fc0f2578740 1 -- 172.21.15.5:0/3843056189 wait complete. 2024-09-15T22:17:04.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.355+0000 7fc0f2578740 20 client.26426 trim_cache size 0 max 16384 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.379+0000 7fc0f2578740 1 Processor -- start 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.383+0000 7fc0f2578740 1 -- start start 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.383+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561251f92af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.383+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251f8e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.383+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251f8e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.383+0000 7fc0f2578740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81af780 con 0x561251f8b780 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.383+0000 7fc0f2578740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81ad050 con 0x561251f96520 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.383+0000 7fc0f2578740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81aaec0 con 0x561251f8ccf0 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.383+0000 7fc0ee73c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251f8e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.386+0000 7fc0edf3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561251f92af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.386+0000 7fc0edf3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561251f92af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36942/0 (socket says 172.21.15.5:36942) 2024-09-15T22:17:04.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.386+0000 7fc0edf3b640 1 -- 172.21.15.5:0/2518632516 learned_addr learned my addr 172.21.15.5:0/2518632516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.390+0000 7fc0ed73a640 1 --2- 172.21.15.5:0/2518632516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251f8e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.392+0000 7fc0edf3b640 1 -- 172.21.15.5:0/2518632516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 msgr2=0x561251f8e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.392+0000 7fc0edf3b640 1 --2- 172.21.15.5:0/2518632516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251f8e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.392+0000 7fc0edf3b640 1 -- 172.21.15.5:0/2518632516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 msgr2=0x561251f8e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.392+0000 7fc0edf3b640 1 --2- 172.21.15.5:0/2518632516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251f8e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.392+0000 7fc0edf3b640 1 -- 172.21.15.5:0/2518632516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561251f8c5a0 con 0x561251f8b780 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.392+0000 7fc0ee73c640 1 --2- 172.21.15.5:0/2518632516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251f8e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.393+0000 7fc0edf3b640 1 --2- 172.21.15.5:0/2518632516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561251f92af0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800a520 tx=0x7fc0d800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b525586079d393 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.394+0000 7fc0d37fe640 1 -- 172.21.15.5:0/2518632516 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8004020 con 0x561251f8b780 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.394+0000 7fc0d37fe640 1 -- 172.21.15.5:0/2518632516 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d80041c0 con 0x561251f8b780 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.394+0000 7fc0d37fe640 1 -- 172.21.15.5:0/2518632516 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0d801b660 con 0x561251f8b780 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.395+0000 7fc0f2578740 1 -- 172.21.15.5:0/2518632516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 msgr2=0x561251f92af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.395+0000 7fc0f2578740 1 --2- 172.21.15.5:0/2518632516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561251f92af0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc0d800a520 tx=0x7fc0d800f7f0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.396+0000 7fc0f2578740 1 -- 172.21.15.5:0/2518632516 shutdown_connections 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.396+0000 7fc0f2578740 1 --2- 172.21.15.5:0/2518632516 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251f8e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.396+0000 7fc0f2578740 1 --2- 172.21.15.5:0/2518632516 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251f8e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.396+0000 7fc0f2578740 1 --2- 172.21.15.5:0/2518632516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561251f92af0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.396+0000 7fc0f2578740 1 -- 172.21.15.5:0/2518632516 >> 172.21.15.5:0/2518632516 conn(0x561251f54310 msgr2=0x561251f878b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.396+0000 7fc0f2578740 1 -- 172.21.15.5:0/2518632516 shutdown_connections 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.400+0000 7fc0f2578740 1 -- 172.21.15.5:0/2518632516 wait complete. 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.400+0000 7fc0f2578740 1 -- start start 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.401+0000 7fc0f2578740 1 Processor -- start 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.403+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561252014c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.403+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251fa6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.403+0000 7fc0f2578740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251fa73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.403+0000 7fc0f2578740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81ab510 con 0x561251f8b780 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.403+0000 7fc0ed73a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251fa6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.403+0000 7fc0ed73a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251fa6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:46848/0 (socket says 172.21.15.5:46848) 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.403+0000 7fc0ed73a640 1 -- 172.21.15.5:0/401609427 learned_addr learned my addr 172.21.15.5:0/401609427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.404+0000 7fc0f2578740 1 -- 172.21.15.5:0/401609427 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81ac980 con 0x561251f96520 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.404+0000 7fc0f2578740 1 -- 172.21.15.5:0/401609427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc0e81abbe0 con 0x561251f8ccf0 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.404+0000 7fc0f2578740 10 client.0 _ll_register_callbacks cb 0x7fc0e0030da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.405+0000 7fc0ee73c640 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251fa73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.410+0000 7fc0ed73a640 1 -- 172.21.15.5:0/401609427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 msgr2=0x561251fa73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.410+0000 7fc0ed73a640 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251fa73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.410+0000 7fc0ed73a640 1 -- 172.21.15.5:0/401609427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 msgr2=0x561252014c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.410+0000 7fc0ed73a640 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561252014c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.410+0000 7fc0ed73a640 1 -- 172.21.15.5:0/401609427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561251fa7960 con 0x561251f8ccf0 2024-09-15T22:17:04.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.411+0000 7fc0ed73a640 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251fa6e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x561251f94870 tx=0x7fc0e0004650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a30a0bf4c508469a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.411+0000 7fc0d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.411+0000 7fc0d17fa640 1 -- 172.21.15.5:0/401609427 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e003bc00 con 0x561251f8ccf0 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.411+0000 7fc0f2578740 1 -- 172.21.15.5:0/401609427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0d80041c0 con 0x561251f8ccf0 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.412+0000 7fc0d17fa640 1 -- 172.21.15.5:0/401609427 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7fc0e003bda0 con 0x561251f8ccf0 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.412+0000 7fc0d17fa640 1 -- 172.21.15.5:0/401609427 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc0e0065050 con 0x561251f8ccf0 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.412+0000 7fc0f2578740 10 client.26380 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.412+0000 7fc0f2578740 1 -- 172.21.15.5:0/401609427 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56125202a650 con 0x561251f8ccf0 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.413+0000 7fc0d17fa640 1 -- 172.21.15.5:0/401609427 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0d80041c0 con 0x561251f8ccf0 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.413+0000 7fc0d17fa640 1 -- 172.21.15.5:0/401609427 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc0e0043070 con 0x561251f8ccf0 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.416+0000 7fc0f2578740 10 client.26380 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.416+0000 7fc0f2578740 10 client.26380 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:17:04.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.416+0000 7fc0f2578740 20 client.26380 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.416+0000 7fc0f2578740 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56125202ad20 0x56125202d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.416+0000 7fc0f2578740 4 client.26380 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.416+0000 7fc0f2578740 1 -- 172.21.15.5:0/401609427 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561251efe860 con 0x56125202ad20 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.424+0000 7fc0edf3b640 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56125202ad20 0x56125202d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.428+0000 7fc0edf3b640 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56125202ad20 0x56125202d140 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4205a7bb7a78b905 server_cookie=123720ec79c8de33 in_seq=0 out_seq=0 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.428+0000 7fc0d17fa640 10 client.26380 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.429+0000 7fc0d17fa640 1 -- 172.21.15.5:0/401609427 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561251efe860 con 0x56125202ad20 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.429+0000 7fc0d17fa640 10 client.26380 handle_command_reply: tid=0 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.430+0000 7fc0f2578740 1 client.26380 shutdown 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.436+0000 7fc0f2578740 1 -- 172.21.15.5:0/401609427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 msgr2=0x561251fa6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.436+0000 7fc0f2578740 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251fa6e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x561251f94870 tx=0x7fc0e0004650 comp rx=0 tx=0).stop 2024-09-15T22:17:04.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.437+0000 7fc0f2578740 1 -- 172.21.15.5:0/401609427 shutdown_connections 2024-09-15T22:17:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.437+0000 7fc0f2578740 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56125202ad20 0x56125202d140 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.175 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:17:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.437+0000 7fc0f2578740 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561251f96520 0x561251fa73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.437+0000 7fc0f2578740 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561251f8ccf0 0x561251fa6e60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.437+0000 7fc0f2578740 1 --2- 172.21.15.5:0/401609427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561251f8b780 0x561252014c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.437+0000 7fc0f2578740 1 -- 172.21.15.5:0/401609427 >> 172.21.15.5:0/401609427 conn(0x561251f54310 msgr2=0x561251f952b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.437+0000 7fc0d17fa640 0 client.26380 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.439+0000 7fc0f2578740 1 -- 172.21.15.5:0/401609427 shutdown_connections 2024-09-15T22:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.439+0000 7fc0f2578740 1 -- 172.21.15.5:0/401609427 wait complete. 2024-09-15T22:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.440+0000 7fc0f2578740 20 client.26380 trim_cache size 0 max 16384 2024-09-15T22:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.445+0000 7fc0eef3d640 1 -- 172.21.15.5:0/2855389299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0d40a34b0 msgr2=0x7fc0d40a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.445+0000 7fc0eef3d640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0d40a34b0 0x7fc0d40a5970 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009a60 tx=0x7fc0e0009820 comp rx=0 tx=0).stop 2024-09-15T22:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.445+0000 7fc0eef3d640 1 -- 172.21.15.5:0/2855389299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0e81079c0 msgr2=0x7fc0e819cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.445+0000 7fc0eef3d640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0e81079c0 0x7fc0e819cbd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400b410 tx=0x7fc0e400b8d0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.445+0000 7fc0eef3d640 1 -- 172.21.15.5:0/2855389299 shutdown_connections 2024-09-15T22:17:04.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.445+0000 7fc0eef3d640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0d40a34b0 0x7fc0d40a5970 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0009a60 tx=0x7fc0e0009820 comp rx=0 tx=0).stop 2024-09-15T22:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.445+0000 7fc0eef3d640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0e81079c0 0x7fc0e819cbd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.445+0000 7fc0eef3d640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0e8102360 0x7fc0e8198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.445+0000 7fc0eef3d640 1 --2- 172.21.15.5:0/2855389299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0e81019c0 0x7fc0e8198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.445+0000 7fc0eef3d640 1 -- 172.21.15.5:0/2855389299 >> 172.21.15.5:0/2855389299 conn(0x7fc0e80fd740 msgr2=0x7fc0e80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.446+0000 7fc0eef3d640 1 -- 172.21.15.5:0/2855389299 shutdown_connections 2024-09-15T22:17:04.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:03.446+0000 7fc0eef3d640 1 -- 172.21.15.5:0/2855389299 wait complete. 2024-09-15T22:17:04.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 455.0 s, 572.4 tps, lat 174.843 ms stddev 221.459 2024-09-15T22:17:04.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:04 smithi005 ceph-mon[30944]: pgmap v250: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-09-15T22:17:04.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:17:05.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:04 smithi099 ceph-mon[34892]: pgmap v250: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-09-15T22:17:05.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:04 smithi129 ceph-mon[35314]: pgmap v250: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 1.6 MiB/s wr, 182 op/s 2024-09-15T22:17:05.345 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.970+0000 7f9bded44640 1 -- 172.21.15.5:0/2602966911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bd8074990 msgr2=0x7f9bd807b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.970+0000 7f9bded44640 1 --2- 172.21.15.5:0/2602966911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bd8074990 0x7f9bd807b3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009ca0 tx=0x7f9bd002a8f0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.970+0000 7f9bded44640 1 -- 172.21.15.5:0/2602966911 shutdown_connections 2024-09-15T22:17:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.970+0000 7f9bded44640 1 --2- 172.21.15.5:0/2602966911 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bd8074990 0x7f9bd807b3c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.970+0000 7f9bded44640 1 --2- 172.21.15.5:0/2602966911 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd8109e00 0x7f9bd80742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.970+0000 7f9bded44640 1 --2- 172.21.15.5:0/2602966911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8109450 0x7f9bd8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.970+0000 7f9bded44640 1 -- 172.21.15.5:0/2602966911 >> 172.21.15.5:0/2602966911 conn(0x7f9bd806e140 msgr2=0x7f9bd806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.970+0000 7f9bded44640 1 -- 172.21.15.5:0/2602966911 shutdown_connections 2024-09-15T22:17:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.970+0000 7f9bded44640 1 -- 172.21.15.5:0/2602966911 wait complete. 2024-09-15T22:17:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.971+0000 7f9bded44640 1 Processor -- start 2024-09-15T22:17:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.971+0000 7f9bded44640 1 -- start start 2024-09-15T22:17:05.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.972+0000 7f9bded44640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd8074990 0x7f9bd807b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.972+0000 7f9bded44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8109450 0x7f9bd80761b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.972+0000 7f9bded44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bd8109e00 0x7f9bd80766f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.972+0000 7f9bded44640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd807e7b0 con 0x7f9bd8109450 2024-09-15T22:17:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.972+0000 7f9bded44640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd807e630 con 0x7f9bd8074990 2024-09-15T22:17:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.972+0000 7f9bded44640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd807e930 con 0x7f9bd8109e00 2024-09-15T22:17:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.972+0000 7f9bdcae0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd8074990 0x7f9bd807b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.972+0000 7f9bdcae0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd8074990 0x7f9bd807b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:57264/0 (socket says 172.21.15.5:57264) 2024-09-15T22:17:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.972+0000 7f9bdcae0640 1 -- 172.21.15.5:0/391592384 learned_addr learned my addr 172.21.15.5:0/391592384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:05.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.972+0000 7f9bdd2e1640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bd8109e00 0x7f9bd80766f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.973+0000 7f9bd7fff640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8109450 0x7f9bd80761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.973+0000 7f9bdcae0640 1 -- 172.21.15.5:0/391592384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bd8109e00 msgr2=0x7f9bd80766f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.973+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bd8109e00 0x7f9bd80766f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.973+0000 7f9bdcae0640 1 -- 172.21.15.5:0/391592384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8109450 msgr2=0x7f9bd80761b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.973+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8109450 0x7f9bd80761b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.973+0000 7f9bdcae0640 1 -- 172.21.15.5:0/391592384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd8076fb0 con 0x7f9bd8074990 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.973+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd8074990 0x7f9bd807b120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800b4a0 tx=0x7f9bc800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.974+0000 7f9bd5ffb640 1 -- 172.21.15.5:0/391592384 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8014070 con 0x7f9bd8074990 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.974+0000 7f9bd5ffb640 1 -- 172.21.15.5:0/391592384 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bc8010af0 con 0x7f9bd8074990 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.974+0000 7f9bded44640 1 -- 172.21.15.5:0/391592384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd81bbfa0 con 0x7f9bd8074990 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.974+0000 7f9bded44640 1 -- 172.21.15.5:0/391592384 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd81bc560 con 0x7f9bd8074990 2024-09-15T22:17:05.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.975+0000 7f9bd5ffb640 1 -- 172.21.15.5:0/391592384 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8017d40 con 0x7f9bd8074990 2024-09-15T22:17:05.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.983+0000 7f9bd5ffb640 1 -- 172.21.15.5:0/391592384 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9bc8017070 con 0x7f9bd8074990 2024-09-15T22:17:05.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.984+0000 7f9bd5ffb640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9bc00a35c0 0x7f9bc00a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.984+0000 7f9bd5ffb640 1 -- 172.21.15.5:0/391592384 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f9bc80cf340 con 0x7f9bd8074990 2024-09-15T22:17:05.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.989+0000 7f9be111d740 1 Processor -- start 2024-09-15T22:17:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.991+0000 7f9be111d740 1 -- start start 2024-09-15T22:17:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.991+0000 7f9be111d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93390b40 0x564b9338f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.991+0000 7f9be111d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b9338f5d0 0x564b9338fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.991+0000 7f9be111d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b9339a370 0x564b9339c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.991+0000 7f9be111d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564b932f52f0 con 0x564b9338f5d0 2024-09-15T22:17:05.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.991+0000 7f9be111d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564b9329ccf0 con 0x564b9339a370 2024-09-15T22:17:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.991+0000 7f9be111d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564b932f12b0 con 0x564b93390b40 2024-09-15T22:17:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.991+0000 7f9bdcae0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93390b40 0x564b9338f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.991+0000 7f9bdcae0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93390b40 0x564b9338f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:46882/0 (socket says 172.21.15.5:46882) 2024-09-15T22:17:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.991+0000 7f9bdcae0640 1 -- 172.21.15.5:0/3430479952 learned_addr learned my addr 172.21.15.5:0/3430479952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:05.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.992+0000 7f9bd7fff640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9bc00a35c0 0x7f9bc00a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.993+0000 7f9bdd2e1640 1 --2- 172.21.15.5:0/3430479952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b9339a370 0x564b9339c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:05.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.993+0000 7f9bdcae0640 1 -- 172.21.15.5:0/3430479952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b9339a370 msgr2=0x564b9339c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:05.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.993+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/3430479952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b9339a370 0x564b9339c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.993+0000 7f9bdcae0640 1 -- 172.21.15.5:0/3430479952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b9338f5d0 msgr2=0x564b9338fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:05.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.993+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/3430479952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b9338f5d0 0x564b9338fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:05.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.993+0000 7f9bdcae0640 1 -- 172.21.15.5:0/3430479952 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b932813a0 con 0x564b93390b40 2024-09-15T22:17:05.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.994+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/3430479952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93390b40 0x564b9338f090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80a3a10 tx=0x7f9bc800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dec947fb660cd59 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:05.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.994+0000 7f9bbeffd640 1 -- 172.21.15.5:0/3430479952 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc801e070 con 0x564b93390b40 2024-09-15T22:17:05.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.994+0000 7f9bbeffd640 1 -- 172.21.15.5:0/3430479952 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bc809f680 con 0x564b93390b40 2024-09-15T22:17:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.995+0000 7f9be111d740 1 -- 172.21.15.5:0/3430479952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93390b40 msgr2=0x564b9338f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:05.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.995+0000 7f9be111d740 1 --2- 172.21.15.5:0/3430479952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93390b40 0x564b9338f090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80a3a10 tx=0x7f9bc800bf90 comp rx=0 tx=0).stop 2024-09-15T22:17:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.999+0000 7f9be111d740 1 -- 172.21.15.5:0/3430479952 shutdown_connections 2024-09-15T22:17:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.999+0000 7f9be111d740 1 --2- 172.21.15.5:0/3430479952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b9339a370 0x564b9339c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.999+0000 7f9be111d740 1 --2- 172.21.15.5:0/3430479952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b9338f5d0 0x564b9338fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.999+0000 7f9be111d740 1 --2- 172.21.15.5:0/3430479952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93390b40 0x564b9338f090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:05.999+0000 7f9be111d740 1 -- 172.21.15.5:0/3430479952 >> 172.21.15.5:0/3430479952 conn(0x564b93358310 msgr2=0x564b9338bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:06.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.006+0000 7f9be111d740 1 -- 172.21.15.5:0/3430479952 shutdown_connections 2024-09-15T22:17:06.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.010+0000 7f9be111d740 1 -- 172.21.15.5:0/3430479952 wait complete. 2024-09-15T22:17:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.010+0000 7f9be111d740 1 -- start start 2024-09-15T22:17:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.010+0000 7f9be111d740 1 Processor -- start 2024-09-15T22:17:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.011+0000 7f9be111d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 0x564b93399db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:06.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.012+0000 7f9be111d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b9339a370 0x564b93397060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:06.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.012+0000 7f9be111d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93397750 0x564b933a65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:06.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.014+0000 7f9bdcae0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 0x564b93399db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.014+0000 7f9bdcae0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 0x564b93399db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:57286/0 (socket says 172.21.15.5:57286) 2024-09-15T22:17:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.015+0000 7f9bdcae0640 1 -- 172.21.15.5:0/172834404 learned_addr learned my addr 172.21.15.5:0/172834404 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:06.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.015+0000 7f9bdd2e1640 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93397750 0x564b933a65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.015+0000 7f9be111d740 1 -- 172.21.15.5:0/172834404 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564b9339d060 con 0x564b9339a370 2024-09-15T22:17:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.015+0000 7f9be111d740 1 -- 172.21.15.5:0/172834404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564b932b0060 con 0x564b93390b40 2024-09-15T22:17:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.016+0000 7f9be111d740 1 -- 172.21.15.5:0/172834404 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564b9339d1e0 con 0x564b93397750 2024-09-15T22:17:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.016+0000 7f9bdcae0640 1 -- 172.21.15.5:0/172834404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93397750 msgr2=0x564b933a65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:06.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.016+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93397750 0x564b933a65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.016+0000 7f9bdcae0640 1 -- 172.21.15.5:0/172834404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b9339a370 msgr2=0x564b93397060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.016+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b9339a370 0x564b93397060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.016+0000 7f9bdcae0640 1 -- 172.21.15.5:0/172834404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b930663c0 con 0x564b93390b40 2024-09-15T22:17:06.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.017+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 0x564b93399db0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800be60 tx=0x7f9bc8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9836db58921c18c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.017+0000 7f9be111d740 10 client.0 _ll_register_callbacks cb 0x564b930038c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:06.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.017+0000 7f9be111d740 1 -- 172.21.15.5:0/172834404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc809f680 con 0x564b93390b40 2024-09-15T22:17:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.021+0000 7f9bb7fff640 10 client.26444 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:17:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.021+0000 7f9bdd2e1640 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93397750 0x564b933a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.021+0000 7f9bb7fff640 1 -- 172.21.15.5:0/172834404 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80b9060 con 0x564b93390b40 2024-09-15T22:17:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.021+0000 7f9bb7fff640 1 -- 172.21.15.5:0/172834404 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f9bc80bb0e0 con 0x564b93390b40 2024-09-15T22:17:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.021+0000 7f9bb7fff640 1 -- 172.21.15.5:0/172834404 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9bc801e070 con 0x564b93390b40 2024-09-15T22:17:06.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.022+0000 7f9be111d740 10 client.26444 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:06.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.022+0000 7f9be111d740 1 -- 172.21.15.5:0/172834404 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=25}) -- 0x564b933a8c40 con 0x564b93390b40 2024-09-15T22:17:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.024+0000 7f9bb7fff640 1 -- 172.21.15.5:0/172834404 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9bc809a720 con 0x564b93390b40 2024-09-15T22:17:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.024+0000 7f9be111d740 10 client.26444 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.024+0000 7f9be111d740 10 client.26444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.024+0000 7f9be111d740 20 client.26444 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.024+0000 7f9be111d740 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564b933aa310 0x564b933ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.024+0000 7f9be111d740 4 client.26444 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:06.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.024+0000 7f9be111d740 1 -- 172.21.15.5:0/172834404 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b93302860 con 0x564b933aa310 2024-09-15T22:17:06.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.030+0000 7f9bd7fff640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9bc00a35c0 0x7f9bc00a5a80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x564b9338cac0 tx=0x7f9bcc005850 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:06.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.031+0000 7f9bd7fff640 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564b933aa310 0x564b933ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:06.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.032+0000 7f9bd7fff640 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564b933aa310 0x564b933ac7d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3ada3870b2a92 server_cookie=7ec53ef0943b4ef2 in_seq=0 out_seq=0 2024-09-15T22:17:06.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.033+0000 7f9bb7fff640 10 client.26444 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.037+0000 7f9bb7fff640 1 -- 172.21.15.5:0/172834404 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b93302860 con 0x564b933aa310 2024-09-15T22:17:06.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.037+0000 7f9bb7fff640 10 client.26444 handle_command_reply: tid=0 2024-09-15T22:17:06.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.037+0000 7f9be111d740 1 client.26444 shutdown 2024-09-15T22:17:06.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.040+0000 7f9be111d740 1 -- 172.21.15.5:0/172834404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 msgr2=0x564b93399db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:06.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.040+0000 7f9be111d740 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 0x564b93399db0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800be60 tx=0x7f9bc8000bc0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.040+0000 7f9be111d740 1 -- 172.21.15.5:0/172834404 shutdown_connections 2024-09-15T22:17:06.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.040+0000 7f9be111d740 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b93397750 0x564b933a65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.040+0000 7f9be111d740 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b9339a370 0x564b93397060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.040+0000 7f9be111d740 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564b933aa310 0x564b933ac7d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.040+0000 7f9be111d740 1 --2- 172.21.15.5:0/172834404 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 0x564b93399db0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.040+0000 7f9be111d740 1 -- 172.21.15.5:0/172834404 >> 172.21.15.5:0/172834404 conn(0x564b93358310 msgr2=0x564b93391930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.041+0000 7f9bb7fff640 0 client.26444 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.041+0000 7f9be111d740 1 -- 172.21.15.5:0/172834404 shutdown_connections 2024-09-15T22:17:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.041+0000 7f9be111d740 1 -- 172.21.15.5:0/172834404 wait complete. 2024-09-15T22:17:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.045+0000 7f9be111d740 20 client.26444 trim_cache size 0 max 16384 2024-09-15T22:17:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.071+0000 7f9be111d740 1 Processor -- start 2024-09-15T22:17:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.071+0000 7f9be111d740 1 -- start start 2024-09-15T22:17:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.071+0000 7f9be111d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 0x564b9338bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.071+0000 7f9be111d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b93397750 0x564b9338aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.071+0000 7f9be111d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b9339a370 0x564b9338af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.071+0000 7f9be111d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc40129e0 con 0x564b93397750 2024-09-15T22:17:06.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.071+0000 7f9be111d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc40102b0 con 0x564b93390b40 2024-09-15T22:17:06.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.071+0000 7f9be111d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc400e120 con 0x564b9339a370 2024-09-15T22:17:06.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.071+0000 7f9bdd2e1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b9339a370 0x564b9338af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:06.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bdd2e1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b9339a370 0x564b9338af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:46894/0 (socket says 172.21.15.5:46894) 2024-09-15T22:17:06.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bdd2e1640 1 -- 172.21.15.5:0/3694442053 learned_addr learned my addr 172.21.15.5:0/3694442053 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:06.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/3694442053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 0x564b9338bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:06.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bdd2e1640 1 -- 172.21.15.5:0/3694442053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 msgr2=0x564b9338bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:06.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bdd2e1640 1 --2- 172.21.15.5:0/3694442053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 0x564b9338bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bd7fff640 1 --2- 172.21.15.5:0/3694442053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b93397750 0x564b9338aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bdd2e1640 1 -- 172.21.15.5:0/3694442053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b93397750 msgr2=0x564b9338aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bdd2e1640 1 --2- 172.21.15.5:0/3694442053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b93397750 0x564b9338aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bdd2e1640 1 -- 172.21.15.5:0/3694442053 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b933984f0 con 0x564b9339a370 2024-09-15T22:17:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/3694442053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 0x564b9338bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bd7fff640 1 --2- 172.21.15.5:0/3694442053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b93397750 0x564b9338aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.072+0000 7f9bdd2e1640 1 --2- 172.21.15.5:0/3694442053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b9339a370 0x564b9338af60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009ca0 tx=0x7f9bd002f420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7596053f420ec3b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.073+0000 7f9bbe7fc640 1 -- 172.21.15.5:0/3694442053 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0049070 con 0x564b9339a370 2024-09-15T22:17:06.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.073+0000 7f9bbe7fc640 1 -- 172.21.15.5:0/3694442053 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bd0036040 con 0x564b9339a370 2024-09-15T22:17:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.073+0000 7f9bbe7fc640 1 -- 172.21.15.5:0/3694442053 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd002fb40 con 0x564b9339a370 2024-09-15T22:17:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.074+0000 7f9be111d740 1 -- 172.21.15.5:0/3694442053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b9339a370 msgr2=0x564b9338af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.074+0000 7f9be111d740 1 --2- 172.21.15.5:0/3694442053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b9339a370 0x564b9338af60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9bd0009ca0 tx=0x7f9bd002f420 comp rx=0 tx=0).stop 2024-09-15T22:17:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.074+0000 7f9be111d740 1 -- 172.21.15.5:0/3694442053 shutdown_connections 2024-09-15T22:17:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.074+0000 7f9be111d740 1 --2- 172.21.15.5:0/3694442053 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b9339a370 0x564b9338af60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.074+0000 7f9be111d740 1 --2- 172.21.15.5:0/3694442053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b93397750 0x564b9338aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.074+0000 7f9be111d740 1 --2- 172.21.15.5:0/3694442053 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93390b40 0x564b9338bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.074+0000 7f9be111d740 1 -- 172.21.15.5:0/3694442053 >> 172.21.15.5:0/3694442053 conn(0x564b93358310 msgr2=0x564b9338a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.074+0000 7f9be111d740 1 -- 172.21.15.5:0/3694442053 shutdown_connections 2024-09-15T22:17:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.074+0000 7f9be111d740 1 -- 172.21.15.5:0/3694442053 wait complete. 2024-09-15T22:17:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.074+0000 7f9be111d740 1 -- start start 2024-09-15T22:17:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.074+0000 7f9be111d740 1 Processor -- start 2024-09-15T22:17:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9be111d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b93390b40 0x564b933a2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9be111d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93397750 0x564b933a2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9be111d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b9339a370 0x564b933933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9be111d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc400e770 con 0x564b93390b40 2024-09-15T22:17:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9be111d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc400fbe0 con 0x564b93397750 2024-09-15T22:17:06.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9be111d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc400ee40 con 0x564b9339a370 2024-09-15T22:17:06.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9be111d740 10 client.0 _ll_register_callbacks cb 0x7f9bd0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:06.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9bd7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93397750 0x564b933a2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:06.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9bd7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93397750 0x564b933a2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:57298/0 (socket says 172.21.15.5:57298) 2024-09-15T22:17:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9bd7fff640 1 -- 172.21.15.5:0/3218450431 learned_addr learned my addr 172.21.15.5:0/3218450431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b93390b40 0x564b933a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9bd7fff640 1 -- 172.21.15.5:0/3218450431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b9339a370 msgr2=0x564b933933f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9bd7fff640 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b9339a370 0x564b933933f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9bd7fff640 1 -- 172.21.15.5:0/3218450431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b93390b40 msgr2=0x564b933a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9bd7fff640 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b93390b40 0x564b933a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.075+0000 7f9bd7fff640 1 -- 172.21.15.5:0/3218450431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b93393930 con 0x564b93397750 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.076+0000 7f9bd7fff640 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93397750 0x564b933a2a90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8077a00 tx=0x7f9bcc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbe44dee43e2b26d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.076+0000 7f9bb7fff640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.076+0000 7f9bb7fff640 1 -- 172.21.15.5:0/3218450431 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc01f420 con 0x564b93397750 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.076+0000 7f9bb7fff640 1 -- 172.21.15.5:0/3218450431 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9541+0+0 (secure 0 0 0) 0x7f9bcc01f5c0 con 0x564b93397750 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.076+0000 7f9be111d740 1 -- 172.21.15.5:0/3218450431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bd0036040 con 0x564b93397750 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.077+0000 7f9bb7fff640 1 -- 172.21.15.5:0/3218450431 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9bcc034050 con 0x564b93397750 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.077+0000 7f9be111d740 10 client.26450 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.077+0000 7f9be111d740 1 -- 172.21.15.5:0/3218450431 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564b934146f0 con 0x564b93397750 2024-09-15T22:17:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.078+0000 7f9bb7fff640 1 -- 172.21.15.5:0/3218450431 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bcc01fca0 con 0x564b93397750 2024-09-15T22:17:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.078+0000 7f9bb7fff640 1 -- 172.21.15.5:0/3218450431 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9bcc02f9d0 con 0x564b93397750 2024-09-15T22:17:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.078+0000 7f9be111d740 10 client.26450 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.078+0000 7f9be111d740 10 client.26450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.078+0000 7f9be111d740 20 client.26450 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.078+0000 7f9be111d740 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564b93414e50 0x564b93417270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.079+0000 7f9be111d740 4 client.26450 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.079+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564b93414e50 0x564b93417270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.079+0000 7f9be111d740 1 -- 172.21.15.5:0/3218450431 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9bd0036040 con 0x564b93414e50 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.079+0000 7f9bdcae0640 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564b93414e50 0x564b93417270 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e05489db9afee431 server_cookie=eb584ec7b7da9fb5 in_seq=0 out_seq=0 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.079+0000 7f9bb7fff640 10 client.26450 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.080+0000 7f9bb7fff640 1 -- 172.21.15.5:0/3218450431 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9bd0036040 con 0x564b93414e50 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.080+0000 7f9bb7fff640 10 client.26450 handle_command_reply: tid=0 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.080+0000 7f9be111d740 1 client.26450 shutdown 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.080+0000 7f9be111d740 1 -- 172.21.15.5:0/3218450431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93397750 msgr2=0x564b933a2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.080+0000 7f9be111d740 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93397750 0x564b933a2a90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8077a00 tx=0x7f9bcc0046c0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.081+0000 7f9be111d740 1 -- 172.21.15.5:0/3218450431 shutdown_connections 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.081+0000 7f9be111d740 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564b9339a370 0x564b933933f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.081+0000 7f9be111d740 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564b93414e50 0x564b93417270 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.081+0000 7f9bb7fff640 0 client.26450 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.081+0000 7f9be111d740 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564b93397750 0x564b933a2a90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.081+0000 7f9be111d740 1 --2- 172.21.15.5:0/3218450431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564b93390b40 0x564b933a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.081+0000 7f9be111d740 1 -- 172.21.15.5:0/3218450431 >> 172.21.15.5:0/3218450431 conn(0x564b93358310 msgr2=0x564b933990e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.081+0000 7f9be111d740 1 -- 172.21.15.5:0/3218450431 shutdown_connections 2024-09-15T22:17:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.081+0000 7f9be111d740 1 -- 172.21.15.5:0/3218450431 wait complete. 2024-09-15T22:17:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.082+0000 7f9be111d740 20 client.26450 trim_cache size 0 max 16384 2024-09-15T22:17:06.088 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.082+0000 7f9bded44640 1 -- 172.21.15.5:0/391592384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9bc00a35c0 msgr2=0x7f9bc00a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.082+0000 7f9bded44640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9bc00a35c0 0x7f9bc00a5a80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x564b9338cac0 tx=0x7f9bcc005850 comp rx=0 tx=0).stop 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.082+0000 7f9bded44640 1 -- 172.21.15.5:0/391592384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd8074990 msgr2=0x7f9bd807b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.082+0000 7f9bded44640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd8074990 0x7f9bd807b120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800b4a0 tx=0x7f9bc800b960 comp rx=0 tx=0).stop 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.082+0000 7f9bded44640 1 -- 172.21.15.5:0/391592384 shutdown_connections 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.082+0000 7f9bded44640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9bc00a35c0 0x7f9bc00a5a80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.082+0000 7f9bded44640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9bd8109e00 0x7f9bd80766f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.083+0000 7f9bded44640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd8109450 0x7f9bd80761b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.083+0000 7f9bded44640 1 --2- 172.21.15.5:0/391592384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9bd8074990 0x7f9bd807b120 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.083+0000 7f9bded44640 1 -- 172.21.15.5:0/391592384 >> 172.21.15.5:0/391592384 conn(0x7f9bd806e140 msgr2=0x7f9bd8072440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.083+0000 7f9bded44640 1 -- 172.21.15.5:0/391592384 shutdown_connections 2024-09-15T22:17:06.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:06.083+0000 7f9bded44640 1 -- 172.21.15.5:0/391592384 wait complete. 2024-09-15T22:17:06.836 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:06 smithi005 ceph-mon[30944]: pgmap v251: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 3.5 MiB/s wr, 367 op/s 2024-09-15T22:17:07.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:06 smithi099 ceph-mon[34892]: pgmap v251: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 3.5 MiB/s wr, 367 op/s 2024-09-15T22:17:07.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:06 smithi129 ceph-mon[35314]: pgmap v251: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 3.5 MiB/s wr, 367 op/s 2024-09-15T22:17:07.836 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:17:07.837 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:17:08.405 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:08.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:08 smithi005 ceph-mon[30944]: pgmap v252: 129 pgs: 129 active+clean; 3.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.6 MiB/s wr, 386 op/s 2024-09-15T22:17:08.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:08 smithi005 ceph-mon[30944]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T22:17:09.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:08 smithi099 ceph-mon[34892]: pgmap v252: 129 pgs: 129 active+clean; 3.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.6 MiB/s wr, 386 op/s 2024-09-15T22:17:09.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:08 smithi099 ceph-mon[34892]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T22:17:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:08 smithi129 ceph-mon[35314]: pgmap v252: 129 pgs: 129 active+clean; 3.3 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 3.6 MiB/s wr, 386 op/s 2024-09-15T22:17:09.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:08 smithi129 ceph-mon[35314]: osdmap e92: 12 total, 12 up, 12 in 2024-09-15T22:17:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 460.0 s, 801.4 tps, lat 127.117 ms stddev 206.740 2024-09-15T22:17:11.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:10 smithi129 ceph-mon[35314]: pgmap v254: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-09-15T22:17:11.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:10 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:17:11.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:10 smithi005 ceph-mon[30944]: pgmap v254: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-09-15T22:17:11.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:10 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:17:11.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:10 smithi099 ceph-mon[34892]: pgmap v254: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 4.6 MiB/s wr, 493 op/s 2024-09-15T22:17:11.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:10 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:17:13.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:12 smithi005 ceph-mon[30944]: pgmap v255: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.1 MiB/s wr, 656 op/s 2024-09-15T22:17:13.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:12 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:17:13.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:12 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:17:13.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:12 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:17:13.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:12 smithi099 ceph-mon[34892]: pgmap v255: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.1 MiB/s wr, 656 op/s 2024-09-15T22:17:13.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:12 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:17:13.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:12 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:17:13.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:12 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:17:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:12 smithi129 ceph-mon[35314]: pgmap v255: 129 pgs: 2 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 123 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 6.1 MiB/s wr, 656 op/s 2024-09-15T22:17:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:12 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:17:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:12 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:17:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:12 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:17:13.475 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:17:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:17:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "08b02715-8614-4c34-a7cc-4c311d2c5afe", 2024-09-15T22:17:13.476 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:17:13.476 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.187+0000 7f44b101c640 1 -- 172.21.15.5:0/3024302838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac102ef0 msgr2=0x7f44ac10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.187+0000 7f44b101c640 1 --2- 172.21.15.5:0/3024302838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac102ef0 0x7f44ac10da40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009cd0 tx=0x7f44a402a970 comp rx=0 tx=0).stop 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.190+0000 7f44b101c640 1 -- 172.21.15.5:0/3024302838 shutdown_connections 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.190+0000 7f44b101c640 1 --2- 172.21.15.5:0/3024302838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac102ef0 0x7f44ac10da40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.190+0000 7f44b101c640 1 --2- 172.21.15.5:0/3024302838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44ac102570 0x7f44ac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.190+0000 7f44b101c640 1 --2- 172.21.15.5:0/3024302838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac100e20 0x7f44ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.190+0000 7f44b101c640 1 -- 172.21.15.5:0/3024302838 >> 172.21.15.5:0/3024302838 conn(0x7f44ac0fc7d0 msgr2=0x7f44ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.190+0000 7f44b101c640 1 -- 172.21.15.5:0/3024302838 shutdown_connections 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.190+0000 7f44b101c640 1 -- 172.21.15.5:0/3024302838 wait complete. 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44b101c640 1 Processor -- start 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44b101c640 1 -- start start 2024-09-15T22:17:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44b101c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac100e20 0x7f44ac1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44b101c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac102570 0x7f44ac1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44b101c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44ac102ef0 0x7f44ac19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44b101c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac110f80 con 0x7f44ac100e20 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44b101c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac110e00 con 0x7f44ac102570 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44b101c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac111100 con 0x7f44ac102ef0 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44a3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac102570 0x7f44ac1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44aad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac100e20 0x7f44ac1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44a3fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac102570 0x7f44ac1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:57322/0 (socket says 172.21.15.5:57322) 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44aad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac100e20 0x7f44ac1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37008/0 (socket says 172.21.15.5:37008) 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.191+0000 7f44a3fff640 1 -- 172.21.15.5:0/793865637 learned_addr learned my addr 172.21.15.5:0/793865637 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.192+0000 7f44ab59e640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44ac102ef0 0x7f44ac19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.192+0000 7f44a3fff640 1 -- 172.21.15.5:0/793865637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44ac102ef0 msgr2=0x7f44ac19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.192+0000 7f44a3fff640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44ac102ef0 0x7f44ac19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.192+0000 7f44a3fff640 1 -- 172.21.15.5:0/793865637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac100e20 msgr2=0x7f44ac1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.192+0000 7f44a3fff640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac100e20 0x7f44ac1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.192+0000 7f44a3fff640 1 -- 172.21.15.5:0/793865637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac19d460 con 0x7f44ac102570 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.192+0000 7f44a3fff640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac102570 0x7f44ac1989f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f449400ea60 tx=0x7f449400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.193+0000 7f44a8d99640 1 -- 172.21.15.5:0/793865637 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449400cde0 con 0x7f44ac102570 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.193+0000 7f44a8d99640 1 -- 172.21.15.5:0/793865637 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44940035f0 con 0x7f44ac102570 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.193+0000 7f44a8d99640 1 -- 172.21.15.5:0/793865637 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4494010670 con 0x7f44ac102570 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.193+0000 7f44b101c640 1 -- 172.21.15.5:0/793865637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac19d750 con 0x7f44ac102570 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.193+0000 7f44b101c640 1 -- 172.21.15.5:0/793865637 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac1a5030 con 0x7f44ac102570 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.195+0000 7f44a8d99640 1 -- 172.21.15.5:0/793865637 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f44940037a0 con 0x7f44ac102570 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.196+0000 7f44a8d99640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f44980a3430 0x7f44980a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.196+0000 7f44a8d99640 1 -- 172.21.15.5:0/793865637 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9629+0+0 (secure 0 0 0) 0x7f44940d0440 con 0x7f44ac102570 2024-09-15T22:17:13.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.196+0000 7f44aad9d640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f44980a3430 0x7f44980a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.196+0000 7f44b33f5740 1 Processor -- start 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.196+0000 7f44b33f5740 1 -- start start 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.197+0000 7f44aad9d640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f44980a3430 0x7f44980a58f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f449c0095f0 tx=0x7f449c005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.197+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x564212304e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.197+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212305340 0x564212313c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.197+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212314150 0x564212316540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.197+0000 7f44b33f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56421226a280 con 0x564212314150 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.197+0000 7f44b33f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564212211b10 con 0x564212304a20 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.197+0000 7f44b33f5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564212266240 con 0x564212305340 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.199+0000 7f44aad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x564212304e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.199+0000 7f44aad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x564212304e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:57330/0 (socket says 172.21.15.5:57330) 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.199+0000 7f44aad9d640 1 -- 172.21.15.5:0/3900342164 learned_addr learned my addr 172.21.15.5:0/3900342164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:13.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.200+0000 7f44a3fff640 1 --2- 172.21.15.5:0/3900342164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212305340 0x564212313c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.200+0000 7f44ab59e640 1 --2- 172.21.15.5:0/3900342164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212314150 0x564212316540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.200+0000 7f44aad9d640 1 -- 172.21.15.5:0/3900342164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212305340 msgr2=0x564212313c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.200+0000 7f44aad9d640 1 --2- 172.21.15.5:0/3900342164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212305340 0x564212313c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.200+0000 7f44aad9d640 1 -- 172.21.15.5:0/3900342164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212314150 msgr2=0x564212316540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.200+0000 7f44aad9d640 1 --2- 172.21.15.5:0/3900342164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212314150 0x564212316540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.200+0000 7f44aad9d640 1 -- 172.21.15.5:0/3900342164 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56421221e1f0 con 0x564212304a20 2024-09-15T22:17:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.200+0000 7f44a3fff640 1 --2- 172.21.15.5:0/3900342164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212305340 0x564212313c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.200+0000 7f44aad9d640 1 --2- 172.21.15.5:0/3900342164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x564212304e00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f449c0097e0 tx=0x7f449c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48437326676d4d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.201+0000 7f44a17fa640 1 -- 172.21.15.5:0/3900342164 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c004020 con 0x564212304a20 2024-09-15T22:17:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.201+0000 7f44a17fa640 1 -- 172.21.15.5:0/3900342164 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449c0041c0 con 0x564212304a20 2024-09-15T22:17:13.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.201+0000 7f44a17fa640 1 -- 172.21.15.5:0/3900342164 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c01c780 con 0x564212304a20 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.201+0000 7f44b33f5740 1 -- 172.21.15.5:0/3900342164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 msgr2=0x564212304e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.201+0000 7f44b33f5740 1 --2- 172.21.15.5:0/3900342164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x564212304e00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f449c0097e0 tx=0x7f449c014000 comp rx=0 tx=0).stop 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 -- 172.21.15.5:0/3900342164 shutdown_connections 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 --2- 172.21.15.5:0/3900342164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212314150 0x564212316540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 --2- 172.21.15.5:0/3900342164 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212305340 0x564212313c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 --2- 172.21.15.5:0/3900342164 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x564212304e00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 -- 172.21.15.5:0/3900342164 >> 172.21.15.5:0/3900342164 conn(0x5642123010f0 msgr2=0x5642123014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 -- 172.21.15.5:0/3900342164 shutdown_connections 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 -- 172.21.15.5:0/3900342164 wait complete. 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 -- start start 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 Processor -- start 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x5642123110d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x564212311610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 0x564212311b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564212316d70 con 0x564212305340 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564212224f10 con 0x564212304a20 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564212316ef0 con 0x564212314150 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.202+0000 7f44b33f5740 10 client.0 _ll_register_callbacks cb 0x5642121eda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x564212311610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x564212311610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37014/0 (socket says 172.21.15.5:37014) 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44a3fff640 1 -- 172.21.15.5:0/452809551 learned_addr learned my addr 172.21.15.5:0/452809551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44ab59e640 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 0x564212311b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44aad9d640 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x5642123110d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44a3fff640 1 -- 172.21.15.5:0/452809551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 msgr2=0x564212311b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44a3fff640 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 0x564212311b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44a3fff640 1 -- 172.21.15.5:0/452809551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 msgr2=0x5642123110d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44a3fff640 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x5642123110d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44a3fff640 1 -- 172.21.15.5:0/452809551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564211fdb6a0 con 0x564212305340 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44a3fff640 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x564212311610 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f44940986d0 tx=0x7f449409cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ba91b7b6800093 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.203+0000 7f44927fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.204+0000 7f44927fc640 1 -- 172.21.15.5:0/452809551 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44940b5060 con 0x564212305340 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.204+0000 7f44927fc640 1 -- 172.21.15.5:0/452809551 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9629+0+0 (secure 0 0 0) 0x7f4494003b30 con 0x564212305340 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.204+0000 7f44b33f5740 1 -- 172.21.15.5:0/452809551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f449c0041c0 con 0x564212305340 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.204+0000 7f44927fc640 1 -- 172.21.15.5:0/452809551 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44940ba050 con 0x564212305340 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.204+0000 7f44b33f5740 10 client.16938 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.204+0000 7f44b33f5740 1 -- 172.21.15.5:0/452809551 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5642123229d0 con 0x564212305340 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.204+0000 7f44927fc640 1 -- 172.21.15.5:0/452809551 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449407ae50 con 0x564212305340 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.204+0000 7f44927fc640 1 -- 172.21.15.5:0/452809551 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f44940bc0d0 con 0x564212305340 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.205+0000 7f44b33f5740 10 client.16938 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.205+0000 7f44b33f5740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.205+0000 7f44b33f5740 20 client.16938 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:13.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.205+0000 7f44b33f5740 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564212323e90 0x564212326280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.205+0000 7f44b33f5740 4 client.16938 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.205+0000 7f44b33f5740 1 -- 172.21.15.5:0/452809551 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642122777f0 con 0x564212323e90 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.205+0000 7f44aad9d640 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564212323e90 0x564212326280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.205+0000 7f44aad9d640 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564212323e90 0x564212326280 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9588147d5bd489b4 server_cookie=b92b78e8c572f97b in_seq=0 out_seq=0 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.205+0000 7f44927fc640 10 client.16938 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44927fc640 1 -- 172.21.15.5:0/452809551 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642122777f0 con 0x564212323e90 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44927fc640 10 client.16938 handle_command_reply: tid=0 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44b33f5740 1 client.16938 shutdown 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44b33f5740 1 -- 172.21.15.5:0/452809551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 msgr2=0x564212311610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44b33f5740 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x564212311610 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f44940986d0 tx=0x7f449409cab0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44b33f5740 1 -- 172.21.15.5:0/452809551 shutdown_connections 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44b33f5740 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 0x564212311b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44b33f5740 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x564212311610 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44b33f5740 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564212323e90 0x564212326280 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44b33f5740 1 --2- 172.21.15.5:0/452809551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x5642123110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44b33f5740 1 -- 172.21.15.5:0/452809551 >> 172.21.15.5:0/452809551 conn(0x5642123010f0 msgr2=0x564212306e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44927fc640 0 client.16938 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44b33f5740 1 -- 172.21.15.5:0/452809551 shutdown_connections 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.206+0000 7f44b33f5740 1 -- 172.21.15.5:0/452809551 wait complete. 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.207+0000 7f44b33f5740 20 client.16938 trim_cache size 0 max 16384 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.233+0000 7f44b33f5740 1 Processor -- start 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.233+0000 7f44b33f5740 1 -- start start 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.233+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x56421232ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.233+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x56421232bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.233+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 0x56421230e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.233+0000 7f44b33f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1afa70 con 0x564212305340 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.233+0000 7f44b33f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1ad340 con 0x564212304a20 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.233+0000 7f44b33f5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1ab1b0 con 0x564212314150 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44ab59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 0x56421230e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44aad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x56421232ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44aad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x56421232ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:57350/0 (socket says 172.21.15.5:57350) 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44ab59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 0x56421230e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:46966/0 (socket says 172.21.15.5:46966) 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44aad9d640 1 -- 172.21.15.5:0/210171698 learned_addr learned my addr 172.21.15.5:0/210171698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44a3fff640 1 --2- 172.21.15.5:0/210171698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x56421232bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44ab59e640 1 -- 172.21.15.5:0/210171698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 msgr2=0x56421232ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44ab59e640 1 --2- 172.21.15.5:0/210171698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x56421232ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44ab59e640 1 -- 172.21.15.5:0/210171698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 msgr2=0x56421232bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44ab59e640 1 --2- 172.21.15.5:0/210171698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x56421232bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44ab59e640 1 -- 172.21.15.5:0/210171698 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56421230f620 con 0x564212314150 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44ab59e640 1 --2- 172.21.15.5:0/210171698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 0x56421230e330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f44a4005cc0 tx=0x7f44a4005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c96372701a534d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44a0ff9640 1 -- 172.21.15.5:0/210171698 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4034040 con 0x564212314150 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44a0ff9640 1 -- 172.21.15.5:0/210171698 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a4003e50 con 0x564212314150 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.234+0000 7f44a0ff9640 1 -- 172.21.15.5:0/210171698 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4004040 con 0x564212314150 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.235+0000 7f44b33f5740 1 -- 172.21.15.5:0/210171698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 msgr2=0x56421230e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.235+0000 7f44b33f5740 1 --2- 172.21.15.5:0/210171698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 0x56421230e330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f44a4005cc0 tx=0x7f44a4005c50 comp rx=0 tx=0).stop 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.235+0000 7f44b33f5740 1 -- 172.21.15.5:0/210171698 shutdown_connections 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.235+0000 7f44b33f5740 1 --2- 172.21.15.5:0/210171698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212314150 0x56421230e330 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.235+0000 7f44b33f5740 1 --2- 172.21.15.5:0/210171698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x56421232bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.235+0000 7f44b33f5740 1 --2- 172.21.15.5:0/210171698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212304a20 0x56421232ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.235+0000 7f44b33f5740 1 -- 172.21.15.5:0/210171698 >> 172.21.15.5:0/210171698 conn(0x5642123010f0 msgr2=0x564212300270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.235+0000 7f44b33f5740 1 -- 172.21.15.5:0/210171698 shutdown_connections 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.235+0000 7f44b33f5740 1 -- 172.21.15.5:0/210171698 wait complete. 2024-09-15T22:17:13.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.236+0000 7f44b33f5740 1 -- start start 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.236+0000 7f44b33f5740 1 Processor -- start 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.236+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212304a20 0x56421231c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.236+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x56421231c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.237+0000 7f44b33f5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212314150 0x564212318680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.237+0000 7f44b33f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1ab800 con 0x564212305340 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.237+0000 7f44b33f5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1acc70 con 0x564212314150 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.237+0000 7f44b33f5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac1abed0 con 0x564212304a20 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.237+0000 7f44b33f5740 10 client.0 _ll_register_callbacks cb 0x7f4494003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.237+0000 7f44aad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212304a20 0x56421231c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.237+0000 7f44ab59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212314150 0x564212318680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.237+0000 7f44aad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212304a20 0x56421231c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:46970/0 (socket says 172.21.15.5:46970) 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.237+0000 7f44aad9d640 1 -- 172.21.15.5:0/1681735944 learned_addr learned my addr 172.21.15.5:0/1681735944 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.237+0000 7f44a3fff640 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x56421231c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.238+0000 7f44ab59e640 1 -- 172.21.15.5:0/1681735944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212304a20 msgr2=0x56421231c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.238+0000 7f44ab59e640 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212304a20 0x56421231c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.238+0000 7f44ab59e640 1 -- 172.21.15.5:0/1681735944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 msgr2=0x56421231c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.238+0000 7f44ab59e640 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x56421231c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.238+0000 7f44ab59e640 1 -- 172.21.15.5:0/1681735944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564212318eb0 con 0x564212314150 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.238+0000 7f44ab59e640 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212314150 0x564212318680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009bf0 tx=0x7f44a40099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8dc0f726b87cf97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.238+0000 7f44927fc640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.238+0000 7f44927fc640 1 -- 172.21.15.5:0/1681735944 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a4034040 con 0x564212314150 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.239+0000 7f44b33f5740 1 -- 172.21.15.5:0/1681735944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a4003e50 con 0x564212314150 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.239+0000 7f44927fc640 1 -- 172.21.15.5:0/1681735944 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9629+0+0 (secure 0 0 0) 0x7f44a4003b70 con 0x564212314150 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.239+0000 7f44927fc640 1 -- 172.21.15.5:0/1681735944 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44a405c050 con 0x564212314150 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.239+0000 7f44b33f5740 10 client.26468 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.239+0000 7f44b33f5740 1 -- 172.21.15.5:0/1681735944 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5642123079a0 con 0x564212314150 2024-09-15T22:17:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.240+0000 7f44927fc640 1 -- 172.21.15.5:0/1681735944 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44a4003e50 con 0x564212314150 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.240+0000 7f44927fc640 1 -- 172.21.15.5:0/1681735944 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f44a4004000 con 0x564212314150 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.240+0000 7f44b33f5740 10 client.26468 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.240+0000 7f44b33f5740 10 client.26468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.240+0000 7f44b33f5740 20 client.26468 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.240+0000 7f44b33f5740 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642123080f0 0x56421230a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.240+0000 7f44b33f5740 4 client.26468 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.240+0000 7f44b33f5740 1 -- 172.21.15.5:0/1681735944 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564212271d00 con 0x5642123080f0 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.240+0000 7f44aad9d640 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642123080f0 0x56421230a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.241+0000 7f44927fc640 10 client.26468 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.241+0000 7f44aad9d640 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642123080f0 0x56421230a510 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30fab40ff29cf22a server_cookie=c86e0dc4c0dee661 in_seq=0 out_seq=0 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44927fc640 1 -- 172.21.15.5:0/1681735944 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564212271d00 con 0x5642123080f0 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44927fc640 10 client.26468 handle_command_reply: tid=0 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44b33f5740 1 client.26468 shutdown 2024-09-15T22:17:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44b33f5740 1 -- 172.21.15.5:0/1681735944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212314150 msgr2=0x564212318680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44b33f5740 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212314150 0x564212318680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f44a4009bf0 tx=0x7f44a40099c0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44b33f5740 1 -- 172.21.15.5:0/1681735944 shutdown_connections 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44b33f5740 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5642123080f0 0x56421230a510 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44b33f5740 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564212314150 0x564212318680 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44b33f5740 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564212305340 0x56421231c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44b33f5740 1 --2- 172.21.15.5:0/1681735944 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564212304a20 0x56421231c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44b33f5740 1 -- 172.21.15.5:0/1681735944 >> 172.21.15.5:0/1681735944 conn(0x5642123010f0 msgr2=0x564212312740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.242+0000 7f44927fc640 0 client.26468 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b33f5740 1 -- 172.21.15.5:0/1681735944 shutdown_connections 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b33f5740 1 -- 172.21.15.5:0/1681735944 wait complete. 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b33f5740 20 client.26468 trim_cache size 0 max 16384 2024-09-15T22:17:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b101c640 1 -- 172.21.15.5:0/793865637 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f44980a3430 msgr2=0x7f44980a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b101c640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f44980a3430 0x7f44980a58f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f449c0095f0 tx=0x7f449c005f90 comp rx=0 tx=0).stop 2024-09-15T22:17:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b101c640 1 -- 172.21.15.5:0/793865637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac102570 msgr2=0x7f44ac1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b101c640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac102570 0x7f44ac1989f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f449400ea60 tx=0x7f449400ef20 comp rx=0 tx=0).stop 2024-09-15T22:17:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b101c640 1 -- 172.21.15.5:0/793865637 shutdown_connections 2024-09-15T22:17:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b101c640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f44980a3430 0x7f44980a58f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b101c640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f44ac102ef0 0x7f44ac19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b101c640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f44ac102570 0x7f44ac1989f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b101c640 1 --2- 172.21.15.5:0/793865637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44ac100e20 0x7f44ac1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.243+0000 7f44b101c640 1 -- 172.21.15.5:0/793865637 >> 172.21.15.5:0/793865637 conn(0x7f44ac0fc7d0 msgr2=0x7f44ac06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.244+0000 7f44b101c640 1 -- 172.21.15.5:0/793865637 shutdown_connections 2024-09-15T22:17:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:13.244+0000 7f44b101c640 1 -- 172.21.15.5:0/793865637 wait complete. 2024-09-15T22:17:14.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:13 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:17:14.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:13 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:14.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:13 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:17:14.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:17:14.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:13 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:17:14.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:13 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:14.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:13 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:17:14.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:13 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:17:14.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:13 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:14.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:13 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:17:14.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 465.0 s, 727.5 tps, lat 133.380 ms stddev 185.150 2024-09-15T22:17:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:15.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:14 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:15.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:14 smithi099 ceph-mon[34892]: pgmap v256: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.2 MiB/s wr, 659 op/s 2024-09-15T22:17:15.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:14 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:15.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:14 smithi129 ceph-mon[35314]: pgmap v256: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.2 MiB/s wr, 659 op/s 2024-09-15T22:17:15.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:14 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:15.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:14 smithi005 ceph-mon[30944]: pgmap v256: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 6.2 MiB/s wr, 659 op/s 2024-09-15T22:17:17.167 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:16 smithi005 ceph-mon[30944]: pgmap v257: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.8 MiB/s wr, 719 op/s 2024-09-15T22:17:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:16 smithi099 ceph-mon[34892]: pgmap v257: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.8 MiB/s wr, 719 op/s 2024-09-15T22:17:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:16 smithi129 ceph-mon[35314]: pgmap v257: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.8 MiB/s wr, 719 op/s 2024-09-15T22:17:17.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.503+0000 7f8e12252640 1 -- 172.21.15.5:0/2184327462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e0c109b10 msgr2=0x7f8e0c114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:17.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.503+0000 7f8e12252640 1 --2- 172.21.15.5:0/2184327462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e0c109b10 0x7f8e0c114570 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8df4009900 tx=0x7f8df402a4a0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.503+0000 7f8e12252640 1 -- 172.21.15.5:0/2184327462 shutdown_connections 2024-09-15T22:17:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.503+0000 7f8e12252640 1 --2- 172.21.15.5:0/2184327462 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e0c114ab0 0x7f8e0c116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.503+0000 7f8e12252640 1 --2- 172.21.15.5:0/2184327462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e0c109b10 0x7f8e0c114570 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.503+0000 7f8e12252640 1 --2- 172.21.15.5:0/2184327462 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e0c1091f0 0x7f8e0c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.503+0000 7f8e12252640 1 -- 172.21.15.5:0/2184327462 >> 172.21.15.5:0/2184327462 conn(0x7f8e0c06e140 msgr2=0x7f8e0c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.503+0000 7f8e12252640 1 -- 172.21.15.5:0/2184327462 shutdown_connections 2024-09-15T22:17:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.504+0000 7f8e12252640 1 -- 172.21.15.5:0/2184327462 wait complete. 2024-09-15T22:17:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.504+0000 7f8e12252640 1 Processor -- start 2024-09-15T22:17:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.504+0000 7f8e12252640 1 -- start start 2024-09-15T22:17:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.505+0000 7f8e12252640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e0c1091f0 0x7f8e0c1a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.505+0000 7f8e12252640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e0c109b10 0x7f8e0c1a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.505+0000 7f8e12252640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e0c114ab0 0x7f8e0c1a7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.505+0000 7f8e12252640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c11a7e0 con 0x7f8e0c109b10 2024-09-15T22:17:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.505+0000 7f8e12252640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c11a660 con 0x7f8e0c1091f0 2024-09-15T22:17:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.505+0000 7f8e12252640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c11a960 con 0x7f8e0c114ab0 2024-09-15T22:17:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.506+0000 7f8e03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e0c109b10 0x7f8e0c1a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.506+0000 7f8e03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e0c109b10 0x7f8e0c1a3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58332/0 (socket says 172.21.15.5:58332) 2024-09-15T22:17:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.506+0000 7f8e03fff640 1 -- 172.21.15.5:0/3476908635 learned_addr learned my addr 172.21.15.5:0/3476908635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.506+0000 7f8e0bfff640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e0c114ab0 0x7f8e0c1a7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.507+0000 7f8e03fff640 1 -- 172.21.15.5:0/3476908635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e0c114ab0 msgr2=0x7f8e0c1a7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.507+0000 7f8e03fff640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e0c114ab0 0x7f8e0c1a7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.507+0000 7f8e03fff640 1 -- 172.21.15.5:0/3476908635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e0c1091f0 msgr2=0x7f8e0c1a2ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:17:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.507+0000 7f8e03fff640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e0c1091f0 0x7f8e0c1a2ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.507+0000 7f8e03fff640 1 -- 172.21.15.5:0/3476908635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c1a7e70 con 0x7f8e0c109b10 2024-09-15T22:17:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.508+0000 7f8e03fff640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e0c109b10 0x7f8e0c1a3400 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8df4004650 tx=0x7f8df40052a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.514+0000 7f8e097fa640 1 -- 172.21.15.5:0/3476908635 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4040030 con 0x7f8e0c109b10 2024-09-15T22:17:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.514+0000 7f8e12252640 1 -- 172.21.15.5:0/3476908635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c1a8100 con 0x7f8e0c109b10 2024-09-15T22:17:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.514+0000 7f8e12252640 1 -- 172.21.15.5:0/3476908635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c1afa40 con 0x7f8e0c109b10 2024-09-15T22:17:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.517+0000 7f8e1462b740 1 Processor -- start 2024-09-15T22:17:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.518+0000 7f8e097fa640 1 -- 172.21.15.5:0/3476908635 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df403c920 con 0x7f8e0c109b10 2024-09-15T22:17:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.518+0000 7f8e097fa640 1 -- 172.21.15.5:0/3476908635 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df4031b10 con 0x7f8e0c109b10 2024-09-15T22:17:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.518+0000 7f8e1462b740 1 -- start start 2024-09-15T22:17:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.519+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebb9cb0 0x55896ebba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.519+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.519+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 0x55896ebc7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.519+0000 7f8e1462b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55896eb172e0 con 0x55896ebba5d0 2024-09-15T22:17:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.519+0000 7f8e1462b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55896eabece0 con 0x55896ebc4e90 2024-09-15T22:17:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.519+0000 7f8e1462b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55896eb132a0 con 0x55896ebb9cb0 2024-09-15T22:17:17.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.520+0000 7f8e0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.520+0000 7f8e0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58348/0 (socket says 172.21.15.5:58348) 2024-09-15T22:17:17.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.520+0000 7f8e0b7fe640 1 -- 172.21.15.5:0/2587429598 learned_addr learned my addr 172.21.15.5:0/2587429598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:17.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.520+0000 7f8e0bfff640 1 --2- 172.21.15.5:0/2587429598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebb9cb0 0x55896ebba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.522+0000 7f8e0b7fe640 1 -- 172.21.15.5:0/2587429598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebb9cb0 msgr2=0x55896ebba090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:17.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.522+0000 7f8e0b7fe640 1 --2- 172.21.15.5:0/2587429598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebb9cb0 0x55896ebba090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.522+0000 7f8e0b7fe640 1 -- 172.21.15.5:0/2587429598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 msgr2=0x55896ebc7280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:17:17.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.522+0000 7f8e0b7fe640 1 --2- 172.21.15.5:0/2587429598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 0x55896ebc7280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.522+0000 7f8e0b7fe640 1 -- 172.21.15.5:0/2587429598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55896eaa3440 con 0x55896ebba5d0 2024-09-15T22:17:17.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.523+0000 7f8e0b7fe640 1 --2- 172.21.15.5:0/2587429598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc4950 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009bd0 tx=0x7f8dfc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0ea12836b52060 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.526+0000 7f8e01ffb640 1 -- 172.21.15.5:0/2587429598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc005020 con 0x55896ebba5d0 2024-09-15T22:17:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.526+0000 7f8e01ffb640 1 -- 172.21.15.5:0/2587429598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dfc0047a0 con 0x55896ebba5d0 2024-09-15T22:17:17.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.526+0000 7f8e01ffb640 1 -- 172.21.15.5:0/2587429598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc010690 con 0x55896ebba5d0 2024-09-15T22:17:17.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.528+0000 7f8e097fa640 1 -- 172.21.15.5:0/3476908635 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8df4043030 con 0x7f8e0c109b10 2024-09-15T22:17:17.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.529+0000 7f8e1462b740 1 -- 172.21.15.5:0/2587429598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 msgr2=0x55896ebc4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:17.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.529+0000 7f8e1462b740 1 --2- 172.21.15.5:0/2587429598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc4950 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009bd0 tx=0x7f8dfc00e720 comp rx=0 tx=0).stop 2024-09-15T22:17:17.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.529+0000 7f8e097fa640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8df80a35c0 0x7f8df80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.529+0000 7f8e097fa640 1 -- 172.21.15.5:0/3476908635 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9629+0+0 (secure 0 0 0) 0x7f8df40f06b0 con 0x7f8e0c109b10 2024-09-15T22:17:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.530+0000 7f8e1462b740 1 -- 172.21.15.5:0/2587429598 shutdown_connections 2024-09-15T22:17:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.530+0000 7f8e1462b740 1 --2- 172.21.15.5:0/2587429598 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 0x55896ebc7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.530+0000 7f8e1462b740 1 --2- 172.21.15.5:0/2587429598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc4950 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.530+0000 7f8e1462b740 1 --2- 172.21.15.5:0/2587429598 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebb9cb0 0x55896ebba090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.530+0000 7f8e1462b740 1 -- 172.21.15.5:0/2587429598 >> 172.21.15.5:0/2587429598 conn(0x55896ebb6240 msgr2=0x55896ebb6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.534+0000 7f8e0bfff640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8df80a35c0 0x7f8df80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.535+0000 7f8e1462b740 1 -- 172.21.15.5:0/2587429598 shutdown_connections 2024-09-15T22:17:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.539+0000 7f8e1462b740 1 -- 172.21.15.5:0/2587429598 wait complete. 2024-09-15T22:17:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.539+0000 7f8e1462b740 1 -- start start 2024-09-15T22:17:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.539+0000 7f8e1462b740 1 Processor -- start 2024-09-15T22:17:17.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.540+0000 7f8e0bfff640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8df80a35c0 0x7f8df80a5a80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e040092f0 tx=0x7f8e04009280 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.542+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebb9cb0 0x55896ebc4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.542+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.542+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 0x55896ebc2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.542+0000 7f8e1462b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55896ebc7a60 con 0x55896ebba5d0 2024-09-15T22:17:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.542+0000 7f8e1462b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55896ead2050 con 0x55896ebc4e90 2024-09-15T22:17:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.542+0000 7f8e1462b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55896ebc7be0 con 0x55896ebb9cb0 2024-09-15T22:17:17.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.542+0000 7f8e1462b740 10 client.0 _ll_register_callbacks cb 0x55896eb12160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:17.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.545+0000 7f8e03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.545+0000 7f8e03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc1af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58362/0 (socket says 172.21.15.5:58362) 2024-09-15T22:17:17.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.545+0000 7f8e03fff640 1 -- 172.21.15.5:0/3148636752 learned_addr learned my addr 172.21.15.5:0/3148636752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:17.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.548+0000 7f8e0b7fe640 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebb9cb0 0x55896ebc4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.548+0000 7f8e0bfff640 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 0x55896ebc2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.549+0000 7f8e03fff640 1 -- 172.21.15.5:0/3148636752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebb9cb0 msgr2=0x55896ebc4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:17.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.549+0000 7f8e03fff640 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebb9cb0 0x55896ebc4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.549+0000 7f8e03fff640 1 -- 172.21.15.5:0/3148636752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 msgr2=0x55896ebc2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:17.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.549+0000 7f8e03fff640 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 0x55896ebc2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.549+0000 7f8e03fff640 1 -- 172.21.15.5:0/3148636752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55896e8883c0 con 0x55896ebba5d0 2024-09-15T22:17:17.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.559+0000 7f8e03fff640 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc1af0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8df4031760 tx=0x7f8df40bc0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579c5dedd7455143 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:17.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.559+0000 7f8deaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:17:17.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.560+0000 7f8deaffd640 1 -- 172.21.15.5:0/3148636752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df404a070 con 0x55896ebba5d0 2024-09-15T22:17:17.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.560+0000 7f8e1462b740 1 -- 172.21.15.5:0/3148636752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dfc0047a0 con 0x55896ebba5d0 2024-09-15T22:17:17.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.560+0000 7f8deaffd640 1 -- 172.21.15.5:0/3148636752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9629+0+0 (secure 0 0 0) 0x7f8df40edbd0 con 0x55896ebba5d0 2024-09-15T22:17:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.564+0000 7f8deaffd640 1 -- 172.21.15.5:0/3148636752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8df40eddd0 con 0x55896ebba5d0 2024-09-15T22:17:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.564+0000 7f8e1462b740 10 client.16956 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.564+0000 7f8e1462b740 1 -- 172.21.15.5:0/3148636752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55896ebd3500 con 0x55896ebba5d0 2024-09-15T22:17:17.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.568+0000 7f8deaffd640 1 -- 172.21.15.5:0/3148636752 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dfc0047a0 con 0x55896ebba5d0 2024-09-15T22:17:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.569+0000 7f8deaffd640 1 -- 172.21.15.5:0/3148636752 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8df40bc740 con 0x55896ebba5d0 2024-09-15T22:17:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.569+0000 7f8e1462b740 10 client.16956 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.569+0000 7f8e1462b740 10 client.16956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.569+0000 7f8e1462b740 20 client.16956 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.569+0000 7f8e1462b740 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55896ebd4cf0 0x55896ebd7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.569+0000 7f8e1462b740 4 client.16956 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.569+0000 7f8e1462b740 1 -- 172.21.15.5:0/3148636752 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55896eb24850 con 0x55896ebd4cf0 2024-09-15T22:17:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.569+0000 7f8e0b7fe640 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55896ebd4cf0 0x55896ebd7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.571+0000 7f8e0b7fe640 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55896ebd4cf0 0x55896ebd7110 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e3e4478fb56b4 server_cookie=5909b822360a4367 in_seq=0 out_seq=0 2024-09-15T22:17:17.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.572+0000 7f8deaffd640 10 client.16956 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:17.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.573+0000 7f8deaffd640 1 -- 172.21.15.5:0/3148636752 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55896eb24850 con 0x55896ebd4cf0 2024-09-15T22:17:17.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.573+0000 7f8deaffd640 10 client.16956 handle_command_reply: tid=0 2024-09-15T22:17:17.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.573+0000 7f8e1462b740 1 client.16956 shutdown 2024-09-15T22:17:17.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.576+0000 7f8e1462b740 1 -- 172.21.15.5:0/3148636752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 msgr2=0x55896ebc1af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:17.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.576+0000 7f8e1462b740 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc1af0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8df4031760 tx=0x7f8df40bc0e0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.576+0000 7f8e1462b740 1 -- 172.21.15.5:0/3148636752 shutdown_connections 2024-09-15T22:17:17.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.576+0000 7f8e1462b740 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55896ebd4cf0 0x55896ebd7110 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.576+0000 7f8e1462b740 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 0x55896ebc2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.576+0000 7f8e1462b740 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebba5d0 0x55896ebc1af0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.576+0000 7f8e1462b740 1 --2- 172.21.15.5:0/3148636752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebb9cb0 0x55896ebc4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.576+0000 7f8deaffd640 0 client.16956 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:17.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.576+0000 7f8e1462b740 1 -- 172.21.15.5:0/3148636752 >> 172.21.15.5:0/3148636752 conn(0x55896ebb6240 msgr2=0x55896ebacb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:17.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.577+0000 7f8e1462b740 1 -- 172.21.15.5:0/3148636752 shutdown_connections 2024-09-15T22:17:17.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.577+0000 7f8e1462b740 1 -- 172.21.15.5:0/3148636752 wait complete. 2024-09-15T22:17:17.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.577+0000 7f8e1462b740 20 client.16956 trim_cache size 0 max 16384 2024-09-15T22:17:17.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.620+0000 7f8e1462b740 1 Processor -- start 2024-09-15T22:17:17.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.621+0000 7f8e1462b740 1 -- start start 2024-09-15T22:17:17.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.622+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebb9cb0 0x55896ebd0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.622+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebba5d0 0x55896ebc13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.622+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 0x55896ebaca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.622+0000 7f8e03fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebba5d0 0x55896ebc13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.622+0000 7f8e03fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebba5d0 0x55896ebc13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50364/0 (socket says 172.21.15.5:50364) 2024-09-15T22:17:17.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.622+0000 7f8e03fff640 1 -- 172.21.15.5:0/4206646217 learned_addr learned my addr 172.21.15.5:0/4206646217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:17.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.623+0000 7f8e1462b740 1 -- 172.21.15.5:0/4206646217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec0128b0 con 0x55896ebb9cb0 2024-09-15T22:17:17.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.623+0000 7f8e0bfff640 1 --2- 172.21.15.5:0/4206646217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 0x55896ebaca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.623+0000 7f8e1462b740 1 -- 172.21.15.5:0/4206646217 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec010180 con 0x55896ebc4e90 2024-09-15T22:17:17.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.623+0000 7f8e1462b740 1 -- 172.21.15.5:0/4206646217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec00e080 con 0x55896ebba5d0 2024-09-15T22:17:17.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.625+0000 7f8e03fff640 1 -- 172.21.15.5:0/4206646217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 msgr2=0x55896ebaca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:17.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.625+0000 7f8e03fff640 1 --2- 172.21.15.5:0/4206646217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 0x55896ebaca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.625+0000 7f8e03fff640 1 -- 172.21.15.5:0/4206646217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebb9cb0 msgr2=0x55896ebd0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:17.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.625+0000 7f8e03fff640 1 --2- 172.21.15.5:0/4206646217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebb9cb0 0x55896ebd0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.625+0000 7f8e03fff640 1 -- 172.21.15.5:0/4206646217 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55896ebc0aa0 con 0x55896ebba5d0 2024-09-15T22:17:17.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.625+0000 7f8e03fff640 1 --2- 172.21.15.5:0/4206646217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebba5d0 0x55896ebc13b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8df40c4c20 tx=0x7f8df40bc150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f9c77d0148c70fa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.629+0000 7f8e017fa640 1 -- 172.21.15.5:0/4206646217 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df40bc2b0 con 0x55896ebba5d0 2024-09-15T22:17:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.629+0000 7f8e017fa640 1 -- 172.21.15.5:0/4206646217 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8df40bcbe0 con 0x55896ebba5d0 2024-09-15T22:17:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.629+0000 7f8e017fa640 1 -- 172.21.15.5:0/4206646217 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df40edbd0 con 0x55896ebba5d0 2024-09-15T22:17:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.630+0000 7f8e1462b740 1 -- 172.21.15.5:0/4206646217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebba5d0 msgr2=0x55896ebc13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.630+0000 7f8e1462b740 1 --2- 172.21.15.5:0/4206646217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebba5d0 0x55896ebc13b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8df40c4c20 tx=0x7f8df40bc150 comp rx=0 tx=0).stop 2024-09-15T22:17:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.631+0000 7f8e1462b740 1 -- 172.21.15.5:0/4206646217 shutdown_connections 2024-09-15T22:17:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.631+0000 7f8e1462b740 1 --2- 172.21.15.5:0/4206646217 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebc4e90 0x55896ebaca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.631+0000 7f8e1462b740 1 --2- 172.21.15.5:0/4206646217 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebba5d0 0x55896ebc13b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.631+0000 7f8e1462b740 1 --2- 172.21.15.5:0/4206646217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebb9cb0 0x55896ebd0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.631+0000 7f8e1462b740 1 -- 172.21.15.5:0/4206646217 >> 172.21.15.5:0/4206646217 conn(0x55896ebb6240 msgr2=0x55896ebac350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.632+0000 7f8e1462b740 1 -- 172.21.15.5:0/4206646217 shutdown_connections 2024-09-15T22:17:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.632+0000 7f8e1462b740 1 -- 172.21.15.5:0/4206646217 wait complete. 2024-09-15T22:17:17.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.632+0000 7f8e1462b740 1 -- start start 2024-09-15T22:17:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.633+0000 7f8e1462b740 1 Processor -- start 2024-09-15T22:17:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.633+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebb9cb0 0x55896ebc8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.634+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebba5d0 0x55896ebc90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.634+0000 7f8e0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebb9cb0 0x55896ebc8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.634+0000 7f8e0b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebb9cb0 0x55896ebc8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58382/0 (socket says 172.21.15.5:58382) 2024-09-15T22:17:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.634+0000 7f8e1462b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebc4e90 0x55896ebc95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.634+0000 7f8e0b7fe640 1 -- 172.21.15.5:0/1739319942 learned_addr learned my addr 172.21.15.5:0/1739319942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.634+0000 7f8e03fff640 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebba5d0 0x55896ebc90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.634+0000 7f8e1462b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec00e640 con 0x55896ebb9cb0 2024-09-15T22:17:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.634+0000 7f8e1462b740 1 -- 172.21.15.5:0/1739319942 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec00fab0 con 0x55896ebba5d0 2024-09-15T22:17:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.634+0000 7f8e1462b740 1 -- 172.21.15.5:0/1739319942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8dec00ed10 con 0x55896ebc4e90 2024-09-15T22:17:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.634+0000 7f8e0bfff640 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebc4e90 0x55896ebc95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.634+0000 7f8e1462b740 10 client.0 _ll_register_callbacks cb 0x7f8df40b8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.635+0000 7f8e0bfff640 1 -- 172.21.15.5:0/1739319942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebba5d0 msgr2=0x55896ebc90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:17.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.635+0000 7f8e0bfff640 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebba5d0 0x55896ebc90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.635+0000 7f8e0bfff640 1 -- 172.21.15.5:0/1739319942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebb9cb0 msgr2=0x55896ebc8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.635+0000 7f8e0bfff640 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebb9cb0 0x55896ebc8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.635+0000 7f8e0bfff640 1 -- 172.21.15.5:0/1739319942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55896ec3e990 con 0x55896ebc4e90 2024-09-15T22:17:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.635+0000 7f8e0bfff640 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebc4e90 0x55896ebc95e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009fd0 tx=0x7f8e04004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62baada890d519b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:17.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.636+0000 7f8deaffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:17:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.636+0000 7f8deaffd640 1 -- 172.21.15.5:0/1739319942 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0402e060 con 0x55896ebc4e90 2024-09-15T22:17:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.636+0000 7f8deaffd640 1 -- 172.21.15.5:0/1739319942 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9629+0+0 (secure 0 0 0) 0x7f8e04020b10 con 0x55896ebc4e90 2024-09-15T22:17:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.636+0000 7f8e1462b740 1 -- 172.21.15.5:0/1739319942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df40bcbe0 con 0x55896ebc4e90 2024-09-15T22:17:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.637+0000 7f8deaffd640 1 -- 172.21.15.5:0/1739319942 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8e04020d10 con 0x55896ebc4e90 2024-09-15T22:17:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.637+0000 7f8e1462b740 10 client.26422 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.637+0000 7f8e1462b740 1 -- 172.21.15.5:0/1739319942 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55896ec3fbe0 con 0x55896ebc4e90 2024-09-15T22:17:17.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.637+0000 7f8deaffd640 1 -- 172.21.15.5:0/1739319942 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e040298f0 con 0x55896ebc4e90 2024-09-15T22:17:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.638+0000 7f8deaffd640 1 -- 172.21.15.5:0/1739319942 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8e0400f040 con 0x55896ebc4e90 2024-09-15T22:17:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.638+0000 7f8e1462b740 10 client.26422 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.638+0000 7f8e1462b740 10 client.26422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.638+0000 7f8e1462b740 20 client.26422 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.638+0000 7f8e1462b740 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55896ec40260 0x55896ec42680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.638+0000 7f8e1462b740 4 client.26422 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:17:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.638+0000 7f8e1462b740 1 -- 172.21.15.5:0/1739319942 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f8df40bcbe0 con 0x55896ec40260 2024-09-15T22:17:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.641+0000 7f8e0b7fe640 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55896ec40260 0x55896ec42680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:17.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.642+0000 7f8e0b7fe640 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55896ec40260 0x55896ec42680 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a62c4c59b3e29603 server_cookie=16542b2d066cda90 in_seq=0 out_seq=0 2024-09-15T22:17:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.642+0000 7f8deaffd640 10 client.26422 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.642+0000 7f8deaffd640 1 -- 172.21.15.5:0/1739319942 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f8df40bcbe0 con 0x55896ec40260 2024-09-15T22:17:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.642+0000 7f8deaffd640 10 client.26422 handle_command_reply: tid=0 2024-09-15T22:17:17.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.643+0000 7f8e1462b740 1 client.26422 shutdown 2024-09-15T22:17:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.644+0000 7f8e1462b740 1 -- 172.21.15.5:0/1739319942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebc4e90 msgr2=0x55896ebc95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:17.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.644+0000 7f8e1462b740 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebc4e90 0x55896ebc95e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009fd0 tx=0x7f8e04004a40 comp rx=0 tx=0).stop 2024-09-15T22:17:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.644+0000 7f8e1462b740 1 -- 172.21.15.5:0/1739319942 shutdown_connections 2024-09-15T22:17:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.644+0000 7f8e1462b740 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55896ebc4e90 0x55896ebc95e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.644+0000 7f8e1462b740 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55896ec40260 0x55896ec42680 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.644+0000 7f8e1462b740 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55896ebba5d0 0x55896ebc90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.644+0000 7f8deaffd640 0 client.26422 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.644+0000 7f8e1462b740 1 --2- 172.21.15.5:0/1739319942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55896ebb9cb0 0x55896ebc8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.644+0000 7f8e1462b740 1 -- 172.21.15.5:0/1739319942 >> 172.21.15.5:0/1739319942 conn(0x55896ebb6240 msgr2=0x55896ebc3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.645+0000 7f8e1462b740 1 -- 172.21.15.5:0/1739319942 shutdown_connections 2024-09-15T22:17:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.645+0000 7f8e1462b740 1 -- 172.21.15.5:0/1739319942 wait complete. 2024-09-15T22:17:17.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.645+0000 7f8e1462b740 20 client.26422 trim_cache size 0 max 16384 2024-09-15T22:17:17.647 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:17:17.648 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:17:17.648 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:17:17.648 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:17:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 -- 172.21.15.5:0/3476908635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8df80a35c0 msgr2=0x7f8df80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:17.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8df80a35c0 0x7f8df80a5a80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e040092f0 tx=0x7f8e04009280 comp rx=0 tx=0).stop 2024-09-15T22:17:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 -- 172.21.15.5:0/3476908635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e0c109b10 msgr2=0x7f8e0c1a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e0c109b10 0x7f8e0c1a3400 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8df4004650 tx=0x7f8df40052a0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 -- 172.21.15.5:0/3476908635 shutdown_connections 2024-09-15T22:17:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8df80a35c0 0x7f8df80a5a80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8e0c114ab0 0x7f8e0c1a7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e0c109b10 0x7f8e0c1a3400 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 --2- 172.21.15.5:0/3476908635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8e0c1091f0 0x7f8e0c1a2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 -- 172.21.15.5:0/3476908635 >> 172.21.15.5:0/3476908635 conn(0x7f8e0c06e140 msgr2=0x7f8e0c070800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 -- 172.21.15.5:0/3476908635 shutdown_connections 2024-09-15T22:17:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:17.646+0000 7f8e12252640 1 -- 172.21.15.5:0/3476908635 wait complete. 2024-09-15T22:17:18.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:17 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:17:18.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:17 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:17:18.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:17 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:17:18.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:18 smithi005 ceph-mon[30944]: pgmap v258: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.8 MiB/s wr, 726 op/s 2024-09-15T22:17:18.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:18 smithi005 ceph-mon[30944]: osdmap e93: 12 total, 12 up, 12 in 2024-09-15T22:17:19.020 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:17:19.020 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:17:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:18 smithi099 ceph-mon[34892]: pgmap v258: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.8 MiB/s wr, 726 op/s 2024-09-15T22:17:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:18 smithi099 ceph-mon[34892]: osdmap e93: 12 total, 12 up, 12 in 2024-09-15T22:17:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:18 smithi129 ceph-mon[35314]: pgmap v258: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.8 MiB/s wr, 726 op/s 2024-09-15T22:17:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:18 smithi129 ceph-mon[35314]: osdmap e93: 12 total, 12 up, 12 in 2024-09-15T22:17:19.542 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:19.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 470.0 s, 558.5 tps, lat 178.587 ms stddev 253.924 2024-09-15T22:17:20.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.613+0000 7fd532a38640 1 -- 172.21.15.5:0/473444504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd52c1008d0 msgr2=0x7fd52c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:20.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.613+0000 7fd532a38640 1 --2- 172.21.15.5:0/473444504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd52c1008d0 0x7fd52c100d30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd51c009930 tx=0x7fd51c02a440 comp rx=0 tx=0).stop 2024-09-15T22:17:20.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.614+0000 7fd532a38640 1 -- 172.21.15.5:0/473444504 shutdown_connections 2024-09-15T22:17:20.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.614+0000 7fd532a38640 1 --2- 172.21.15.5:0/473444504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101270 0x7fd52c10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.614+0000 7fd532a38640 1 --2- 172.21.15.5:0/473444504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd52c1008d0 0x7fd52c100d30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.614+0000 7fd532a38640 1 --2- 172.21.15.5:0/473444504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd52c1068d0 0x7fd52c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.614+0000 7fd532a38640 1 -- 172.21.15.5:0/473444504 >> 172.21.15.5:0/473444504 conn(0x7fd52c0fc610 msgr2=0x7fd52c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.614+0000 7fd532a38640 1 -- 172.21.15.5:0/473444504 shutdown_connections 2024-09-15T22:17:20.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.614+0000 7fd532a38640 1 -- 172.21.15.5:0/473444504 wait complete. 2024-09-15T22:17:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.615+0000 7fd532a38640 1 Processor -- start 2024-09-15T22:17:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.615+0000 7fd532a38640 1 -- start start 2024-09-15T22:17:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd532a38640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd52c1008d0 0x7fd52c195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd532a38640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101270 0x7fd52c196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd532a38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd52c1068d0 0x7fd52c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd532a38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c10fc20 con 0x7fd52c1068d0 2024-09-15T22:17:20.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd532a38640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c10faa0 con 0x7fd52c101270 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd532a38640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c10fda0 con 0x7fd52c1008d0 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd531235640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101270 0x7fd52c196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd531a36640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd52c1008d0 0x7fd52c195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd531a36640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd52c1008d0 0x7fd52c195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:50398/0 (socket says 172.21.15.5:50398) 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd531a36640 1 -- 172.21.15.5:0/3071619899 learned_addr learned my addr 172.21.15.5:0/3071619899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd531235640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101270 0x7fd52c196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:41488/0 (socket says 172.21.15.5:41488) 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd531a36640 1 -- 172.21.15.5:0/3071619899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101270 msgr2=0x7fd52c196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd531a36640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101270 0x7fd52c196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd531a36640 1 -- 172.21.15.5:0/3071619899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd52c1068d0 msgr2=0x7fd52c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd531a36640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd52c1068d0 0x7fd52c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.616+0000 7fd531a36640 1 -- 172.21.15.5:0/3071619899 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c19ae90 con 0x7fd52c1008d0 2024-09-15T22:17:20.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.617+0000 7fd531a36640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd52c1008d0 0x7fd52c195ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd5280097d0 tx=0x7fd528009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:20.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.617+0000 7fd522ffd640 1 -- 172.21.15.5:0/3071619899 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52800eab0 con 0x7fd52c1008d0 2024-09-15T22:17:20.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.617+0000 7fd522ffd640 1 -- 172.21.15.5:0/3071619899 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd528019070 con 0x7fd52c1008d0 2024-09-15T22:17:20.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.617+0000 7fd532a38640 1 -- 172.21.15.5:0/3071619899 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c19b180 con 0x7fd52c1008d0 2024-09-15T22:17:20.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.617+0000 7fd532a38640 1 -- 172.21.15.5:0/3071619899 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c1a6e70 con 0x7fd52c1008d0 2024-09-15T22:17:20.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.617+0000 7fd522ffd640 1 -- 172.21.15.5:0/3071619899 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528013380 con 0x7fd52c1008d0 2024-09-15T22:17:20.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.618+0000 7fd532a38640 1 -- 172.21.15.5:0/3071619899 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52c197240 con 0x7fd52c1008d0 2024-09-15T22:17:20.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.619+0000 7fd522ffd640 1 -- 172.21.15.5:0/3071619899 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd528014420 con 0x7fd52c1008d0 2024-09-15T22:17:20.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.620+0000 7fd522ffd640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd5180a3430 0x7fd5180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.620+0000 7fd531235640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd5180a3430 0x7fd5180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:20.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.621+0000 7fd522ffd640 1 -- 172.21.15.5:0/3071619899 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7fd5280cfec0 con 0x7fd52c1008d0 2024-09-15T22:17:20.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.621+0000 7fd531235640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd5180a3430 0x7fd5180a58f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd51c004770 tx=0x7fd51c00b040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:20.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:20.626+0000 7fd522ffd640 1 -- 172.21.15.5:0/3071619899 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd52809c4a0 con 0x7fd52c1008d0 2024-09-15T22:17:21.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.010+0000 7fd532a38640 1 -- 172.21.15.5:0/3071619899 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd52c1a7290 con 0x7fd52c1008d0 2024-09-15T22:17:21.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.011+0000 7fd522ffd640 1 -- 172.21.15.5:0/3071619899 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fd52809bda0 con 0x7fd52c1008d0 2024-09-15T22:17:21.013 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:17:21.014 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:17:21.014 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:17:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 -- 172.21.15.5:0/3071619899 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd5180a3430 msgr2=0x7fd5180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd5180a3430 0x7fd5180a58f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd51c004770 tx=0x7fd51c00b040 comp rx=0 tx=0).stop 2024-09-15T22:17:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 -- 172.21.15.5:0/3071619899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd52c1008d0 msgr2=0x7fd52c195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd52c1008d0 0x7fd52c195ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd5280097d0 tx=0x7fd528009c90 comp rx=0 tx=0).stop 2024-09-15T22:17:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 -- 172.21.15.5:0/3071619899 shutdown_connections 2024-09-15T22:17:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd5180a3430 0x7fd5180a58f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd52c1068d0 0x7fd52c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:21.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd52c101270 0x7fd52c196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 --2- 172.21.15.5:0/3071619899 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd52c1008d0 0x7fd52c195ee0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 -- 172.21.15.5:0/3071619899 >> 172.21.15.5:0/3071619899 conn(0x7fd52c0fc610 msgr2=0x7fd52c0fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 -- 172.21.15.5:0/3071619899 shutdown_connections 2024-09-15T22:17:21.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:21.015+0000 7fd532a38640 1 -- 172.21.15.5:0/3071619899 wait complete. 2024-09-15T22:17:21.181 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:20 smithi005 ceph-mon[30944]: pgmap v260: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.8 MiB/s wr, 727 op/s 2024-09-15T22:17:21.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:20 smithi099 ceph-mon[34892]: pgmap v260: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.8 MiB/s wr, 727 op/s 2024-09-15T22:17:21.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:20 smithi129 ceph-mon[35314]: pgmap v260: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 6.8 MiB/s wr, 727 op/s 2024-09-15T22:17:22.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:21 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3071619899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:22.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:21 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3071619899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:22.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:21 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3071619899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:23.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:22 smithi005 ceph-mon[30944]: pgmap v261: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 645 op/s 2024-09-15T22:17:23.263 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:17:23.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:22 smithi099 ceph-mon[34892]: pgmap v261: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 645 op/s 2024-09-15T22:17:23.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:22 smithi129 ceph-mon[35314]: pgmap v261: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.1 MiB/s wr, 645 op/s 2024-09-15T22:17:23.760 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:24.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.359+0000 7f39e6925640 1 -- 172.21.15.5:0/2166807193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e0109450 msgr2=0x7f39e0109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:24.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.359+0000 7f39e6925640 1 --2- 172.21.15.5:0/2166807193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e0109450 0x7f39e0109830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f39c800af00 tx=0x7f39c80327b0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.360+0000 7f39e6925640 1 -- 172.21.15.5:0/2166807193 shutdown_connections 2024-09-15T22:17:24.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.360+0000 7f39e6925640 1 --2- 172.21.15.5:0/2166807193 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e010ea80 0x7f39e01154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.360+0000 7f39e6925640 1 --2- 172.21.15.5:0/2166807193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e0109e00 0x7f39e010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.360+0000 7f39e6925640 1 --2- 172.21.15.5:0/2166807193 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e0109450 0x7f39e0109830 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.360+0000 7f39e6925640 1 -- 172.21.15.5:0/2166807193 >> 172.21.15.5:0/2166807193 conn(0x7f39e006e2d0 msgr2=0x7f39e006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.365+0000 7f39e6925640 1 -- 172.21.15.5:0/2166807193 shutdown_connections 2024-09-15T22:17:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.365+0000 7f39e6925640 1 -- 172.21.15.5:0/2166807193 wait complete. 2024-09-15T22:17:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.366+0000 7f39e6925640 1 Processor -- start 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.366+0000 7f39e6925640 1 -- start start 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.366+0000 7f39e6925640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e0109450 0x7f39e01b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.366+0000 7f39e6925640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e0109e00 0x7f39e01b5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.366+0000 7f39e6925640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e010ea80 0x7f39e01af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.366+0000 7f39e6925640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01185b0 con 0x7f39e0109e00 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.366+0000 7f39e6925640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0118430 con 0x7f39e0109450 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.366+0000 7f39e6925640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0118730 con 0x7f39e010ea80 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.367+0000 7f39dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e0109450 0x7f39e01b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.367+0000 7f39dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e0109450 0x7f39e01b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46356/0 (socket says 172.21.15.5:46356) 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.367+0000 7f39dffff640 1 -- 172.21.15.5:0/4154231052 learned_addr learned my addr 172.21.15.5:0/4154231052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.367+0000 7f39e4ec2640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e010ea80 0x7f39e01af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.367+0000 7f39dffff640 1 -- 172.21.15.5:0/4154231052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e010ea80 msgr2=0x7f39e01af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.367+0000 7f39dffff640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e010ea80 0x7f39e01af890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.367+0000 7f39dffff640 1 -- 172.21.15.5:0/4154231052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e0109e00 msgr2=0x7f39e01b5c10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:17:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.367+0000 7f39dffff640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e0109e00 0x7f39e01b5c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.367+0000 7f39dffff640 1 -- 172.21.15.5:0/4154231052 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e01affc0 con 0x7f39e0109450 2024-09-15T22:17:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.368+0000 7f39dffff640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e0109450 0x7f39e01b56d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f39c80084c0 tx=0x7f39c800a670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.368+0000 7f39ddffb640 1 -- 172.21.15.5:0/4154231052 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8018410 con 0x7f39e0109450 2024-09-15T22:17:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.368+0000 7f39ddffb640 1 -- 172.21.15.5:0/4154231052 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c80185b0 con 0x7f39e0109450 2024-09-15T22:17:24.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.368+0000 7f39e6925640 1 -- 172.21.15.5:0/4154231052 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e01b0250 con 0x7f39e0109450 2024-09-15T22:17:24.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.368+0000 7f39e6925640 1 -- 172.21.15.5:0/4154231052 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e01bc5a0 con 0x7f39e0109450 2024-09-15T22:17:24.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.369+0000 7f39ddffb640 1 -- 172.21.15.5:0/4154231052 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8003910 con 0x7f39e0109450 2024-09-15T22:17:24.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.374+0000 7f39e8cfe740 1 Processor -- start 2024-09-15T22:17:24.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.374+0000 7f39e8cfe740 1 -- start start 2024-09-15T22:17:24.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.375+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80cfb40 0x559aa80ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.375+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80ce5d0 0x559aa80cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.375+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80d9370 0x559aa80db760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.375+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559aa80342f0 con 0x559aa80ce5d0 2024-09-15T22:17:24.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.375+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559aa7fdbcf0 con 0x559aa80d9370 2024-09-15T22:17:24.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.375+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559aa80302b0 con 0x559aa80cfb40 2024-09-15T22:17:24.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.375+0000 7f39d77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80d9370 0x559aa80db760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.375+0000 7f39d77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80d9370 0x559aa80db760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46362/0 (socket says 172.21.15.5:46362) 2024-09-15T22:17:24.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.375+0000 7f39d77fe640 1 -- 172.21.15.5:0/524736188 learned_addr learned my addr 172.21.15.5:0/524736188 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:24.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.375+0000 7f39ddffb640 1 -- 172.21.15.5:0/4154231052 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f39c8003ab0 con 0x7f39e0109450 2024-09-15T22:17:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.377+0000 7f39ddffb640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39cc0a3540 0x7f39cc0a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.377+0000 7f39dffff640 1 --2- 172.21.15.5:0/524736188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80ce5d0 0x559aa80cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.377+0000 7f39d77fe640 1 -- 172.21.15.5:0/524736188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80cfb40 msgr2=0x559aa80ce090 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:17:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.377+0000 7f39d77fe640 1 --2- 172.21.15.5:0/524736188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80cfb40 0x559aa80ce090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.377+0000 7f39d77fe640 1 -- 172.21.15.5:0/524736188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80ce5d0 msgr2=0x559aa80cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.377+0000 7f39d77fe640 1 --2- 172.21.15.5:0/524736188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80ce5d0 0x559aa80cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.377+0000 7f39d77fe640 1 -- 172.21.15.5:0/524736188 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa7fc03a0 con 0x559aa80d9370 2024-09-15T22:17:24.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.377+0000 7f39dffff640 1 --2- 172.21.15.5:0/524736188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80ce5d0 0x559aa80cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:24.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.377+0000 7f39ddffb640 1 -- 172.21.15.5:0/4154231052 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f39c80f07e0 con 0x7f39e0109450 2024-09-15T22:17:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.381+0000 7f39d77fe640 1 --2- 172.21.15.5:0/524736188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80d9370 0x559aa80db760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f39d00096e0 tx=0x7f39d0009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb44ef61a89c689 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.384+0000 7f39e4ec2640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39cc0a3540 0x7f39cc0a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.384+0000 7f39d67fc640 1 -- 172.21.15.5:0/524736188 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0015020 con 0x559aa80d9370 2024-09-15T22:17:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.384+0000 7f39d67fc640 1 -- 172.21.15.5:0/524736188 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d000bda0 con 0x559aa80d9370 2024-09-15T22:17:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.385+0000 7f39e8cfe740 1 -- 172.21.15.5:0/524736188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80d9370 msgr2=0x559aa80db760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.385+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/524736188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80d9370 0x559aa80db760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f39d00096e0 tx=0x7f39d0009ba0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.385+0000 7f39e8cfe740 1 -- 172.21.15.5:0/524736188 shutdown_connections 2024-09-15T22:17:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.385+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/524736188 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80d9370 0x559aa80db760 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.385+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/524736188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80ce5d0 0x559aa80cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.385+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/524736188 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80cfb40 0x559aa80ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.385+0000 7f39e8cfe740 1 -- 172.21.15.5:0/524736188 >> 172.21.15.5:0/524736188 conn(0x559aa8097310 msgr2=0x559aa80cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:24.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.388+0000 7f39e8cfe740 1 -- 172.21.15.5:0/524736188 shutdown_connections 2024-09-15T22:17:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.393+0000 7f39e8cfe740 1 -- 172.21.15.5:0/524736188 wait complete. 2024-09-15T22:17:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.393+0000 7f39e8cfe740 1 -- start start 2024-09-15T22:17:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.393+0000 7f39e8cfe740 1 Processor -- start 2024-09-15T22:17:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.394+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80ce5d0 0x559aa80d8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.394+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80cfb40 0x559aa80d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.394+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.394+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559aa80dc0c0 con 0x559aa80ce5d0 2024-09-15T22:17:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.394+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559aa7fef060 con 0x559aa80cfb40 2024-09-15T22:17:24.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.394+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559aa80dc240 con 0x559aa80d9370 2024-09-15T22:17:24.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.394+0000 7f39e8cfe740 10 client.0 _ll_register_callbacks cb 0x559aa7d428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.398+0000 7f39d77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80cfb40 0x559aa80d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.398+0000 7f39e4ec2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.398+0000 7f39d77fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80cfb40 0x559aa80d5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46372/0 (socket says 172.21.15.5:46372) 2024-09-15T22:17:24.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.398+0000 7f39d77fe640 1 -- 172.21.15.5:0/1289962574 learned_addr learned my addr 172.21.15.5:0/1289962574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.399+0000 7f39e4ec2640 1 -- 172.21.15.5:0/1289962574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80cfb40 msgr2=0x559aa80d5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.399+0000 7f39e4ec2640 1 --2- 172.21.15.5:0/1289962574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80cfb40 0x559aa80d5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.399+0000 7f39e4ec2640 1 -- 172.21.15.5:0/1289962574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80ce5d0 msgr2=0x559aa80d8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.399+0000 7f39e4ec2640 1 --2- 172.21.15.5:0/1289962574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80ce5d0 0x559aa80d8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.399+0000 7f39e4ec2640 1 -- 172.21.15.5:0/1289962574 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa7da53c0 con 0x559aa80d9370 2024-09-15T22:17:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.399+0000 7f39e4ec2640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39cc0a3540 0x7f39cc0a5a00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39d8005fd0 tx=0x7f39d8009930 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:24.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.399+0000 7f39e4ec2640 1 --2- 172.21.15.5:0/1289962574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d6460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f39d801d000 tx=0x7f39d801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f06bc74ca6278a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:24.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.399+0000 7f39bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:17:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.401+0000 7f39bf7fe640 1 -- 172.21.15.5:0/1289962574 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d801bc40 con 0x559aa80d9370 2024-09-15T22:17:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.401+0000 7f39e8cfe740 1 -- 172.21.15.5:0/1289962574 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d000bda0 con 0x559aa80d9370 2024-09-15T22:17:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.401+0000 7f39bf7fe640 1 -- 172.21.15.5:0/1289962574 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f39d801bde0 con 0x559aa80d9370 2024-09-15T22:17:24.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.402+0000 7f39bf7fe640 1 -- 172.21.15.5:0/1289962574 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f39d802e050 con 0x559aa80d9370 2024-09-15T22:17:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.408+0000 7f39e8cfe740 10 client.26437 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:24.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.408+0000 7f39e8cfe740 1 -- 172.21.15.5:0/1289962574 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559aa80e7b20 con 0x559aa80d9370 2024-09-15T22:17:24.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.413+0000 7f39bf7fe640 1 -- 172.21.15.5:0/1289962574 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d000bda0 con 0x559aa80d9370 2024-09-15T22:17:24.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.413+0000 7f39bf7fe640 1 -- 172.21.15.5:0/1289962574 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f39d8025410 con 0x559aa80d9370 2024-09-15T22:17:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.414+0000 7f39e8cfe740 10 client.26437 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.414+0000 7f39e8cfe740 10 client.26437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:17:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.414+0000 7f39e8cfe740 20 client.26437 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:24.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.414+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/1289962574 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559aa80e9090 0x559aa80eb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.414+0000 7f39e8cfe740 4 client.26437 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.414+0000 7f39e8cfe740 1 -- 172.21.15.5:0/1289962574 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aa8041860 con 0x559aa80e9090 2024-09-15T22:17:24.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.414+0000 7f39dffff640 1 --2- 172.21.15.5:0/1289962574 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559aa80e9090 0x559aa80eb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.416+0000 7f39dffff640 1 --2- 172.21.15.5:0/1289962574 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559aa80e9090 0x559aa80eb530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28f16d0a609691d5 server_cookie=b82580f76bd8b81f in_seq=0 out_seq=0 2024-09-15T22:17:24.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.416+0000 7f39bf7fe640 10 client.26437 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:17:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.417+0000 7f39bf7fe640 1 -- 172.21.15.5:0/1289962574 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559aa8041860 con 0x559aa80e9090 2024-09-15T22:17:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.417+0000 7f39bf7fe640 10 client.26437 handle_command_reply: tid=0 2024-09-15T22:17:24.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.418+0000 7f39e8cfe740 1 client.26437 shutdown 2024-09-15T22:17:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.418+0000 7f39e8cfe740 1 -- 172.21.15.5:0/1289962574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 msgr2=0x559aa80d6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.418+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/1289962574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d6460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f39d801d000 tx=0x7f39d801e000 comp rx=0 tx=0).stop 2024-09-15T22:17:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.419+0000 7f39e8cfe740 1 -- 172.21.15.5:0/1289962574 shutdown_connections 2024-09-15T22:17:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.419+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/1289962574 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559aa80e9090 0x559aa80eb530 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.419+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/1289962574 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d6460 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.419+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/1289962574 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80cfb40 0x559aa80d5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.419+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/1289962574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80ce5d0 0x559aa80d8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.419+0000 7f39e8cfe740 1 -- 172.21.15.5:0/1289962574 >> 172.21.15.5:0/1289962574 conn(0x559aa8097310 msgr2=0x559aa80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.419+0000 7f39bf7fe640 0 client.26437 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:17:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.419+0000 7f39e8cfe740 1 -- 172.21.15.5:0/1289962574 shutdown_connections 2024-09-15T22:17:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.419+0000 7f39e8cfe740 1 -- 172.21.15.5:0/1289962574 wait complete. 2024-09-15T22:17:24.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.419+0000 7f39e8cfe740 20 client.26437 trim_cache size 0 max 16384 2024-09-15T22:17:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.451+0000 7f39e8cfe740 1 Processor -- start 2024-09-15T22:17:24.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.451+0000 7f39e8cfe740 1 -- start start 2024-09-15T22:17:24.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.451+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 0x559aa80e5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.451+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80cfb40 0x559aa80d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.451+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.451+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39c0012a20 con 0x559aa80cfb40 2024-09-15T22:17:24.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.451+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39c00102f0 con 0x559aa80ce5d0 2024-09-15T22:17:24.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.451+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39c000e160 con 0x559aa80d9370 2024-09-15T22:17:24.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.453+0000 7f39e4ec2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.453+0000 7f39e4ec2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58184/0 (socket says 172.21.15.5:58184) 2024-09-15T22:17:24.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.453+0000 7f39e4ec2640 1 -- 172.21.15.5:0/3845608867 learned_addr learned my addr 172.21.15.5:0/3845608867 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:24.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.454+0000 7f39dffff640 1 --2- 172.21.15.5:0/3845608867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 0x559aa80e5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.454+0000 7f39e4ec2640 1 -- 172.21.15.5:0/3845608867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 msgr2=0x559aa80e5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:24.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.454+0000 7f39e4ec2640 1 --2- 172.21.15.5:0/3845608867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 0x559aa80e5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.454+0000 7f39e4ec2640 1 -- 172.21.15.5:0/3845608867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80cfb40 msgr2=0x559aa80d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:24.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.454+0000 7f39e4ec2640 1 --2- 172.21.15.5:0/3845608867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80cfb40 0x559aa80d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.454+0000 7f39e4ec2640 1 -- 172.21.15.5:0/3845608867 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aa80ced50 con 0x559aa80d9370 2024-09-15T22:17:24.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.455+0000 7f39e4ec2640 1 --2- 172.21.15.5:0/3845608867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d1660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f39d800bd40 tx=0x7f39d8025c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=747fffa3acd24f81 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:24.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.455+0000 7f39d5ffb640 1 -- 172.21.15.5:0/3845608867 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d80253e0 con 0x559aa80d9370 2024-09-15T22:17:24.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.455+0000 7f39d5ffb640 1 -- 172.21.15.5:0/3845608867 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d8025580 con 0x559aa80d9370 2024-09-15T22:17:24.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.456+0000 7f39d5ffb640 1 -- 172.21.15.5:0/3845608867 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d80050f0 con 0x559aa80d9370 2024-09-15T22:17:24.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.456+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3845608867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 msgr2=0x559aa80d1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:24.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.456+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/3845608867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d1660 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f39d800bd40 tx=0x7f39d8025c80 comp rx=0 tx=0).stop 2024-09-15T22:17:24.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.457+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3845608867 shutdown_connections 2024-09-15T22:17:24.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.457+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/3845608867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d1660 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.457+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/3845608867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80cfb40 0x559aa80d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.457+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/3845608867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 0x559aa80e5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.457+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3845608867 >> 172.21.15.5:0/3845608867 conn(0x559aa8097310 msgr2=0x559aa80d5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:24.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.457+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3845608867 shutdown_connections 2024-09-15T22:17:24.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.457+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3845608867 wait complete. 2024-09-15T22:17:24.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.457+0000 7f39e8cfe740 1 -- start start 2024-09-15T22:17:24.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.457+0000 7f39e8cfe740 1 Processor -- start 2024-09-15T22:17:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 0x559aa80d2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80cfb40 0x559aa80d3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39e8cfe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39c000e7b0 con 0x559aa80cfb40 2024-09-15T22:17:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f39c000fc20 con 0x559aa80ce5d0 2024-09-15T22:17:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39e8cfe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f39c000ee80 con 0x559aa80d9370 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39e8cfe740 10 client.0 _ll_register_callbacks cb 0x7f39d801a6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 0x559aa80d2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 0x559aa80d2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46386/0 (socket says 172.21.15.5:46386) 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39dffff640 1 -- 172.21.15.5:0/3823789551 learned_addr learned my addr 172.21.15.5:0/3823789551 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39e4ec2640 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.458+0000 7f39d77fe640 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80cfb40 0x559aa80d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.459+0000 7f39dffff640 1 -- 172.21.15.5:0/3823789551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 msgr2=0x559aa80d3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.459+0000 7f39dffff640 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.459+0000 7f39dffff640 1 -- 172.21.15.5:0/3823789551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80cfb40 msgr2=0x559aa80d3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.459+0000 7f39dffff640 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80cfb40 0x559aa80d3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.459+0000 7f39dffff640 1 -- 172.21.15.5:0/3823789551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aa8152dd0 con 0x559aa80ce5d0 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.460+0000 7f39dffff640 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 0x559aa80d2ec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f39c80edb90 tx=0x7f39c8004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb4fd1d89da015f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.460+0000 7f39bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:17:24.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.460+0000 7f39bf7fe640 1 -- 172.21.15.5:0/3823789551 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80d0070 con 0x559aa80ce5d0 2024-09-15T22:17:24.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.460+0000 7f39bf7fe640 1 -- 172.21.15.5:0/3823789551 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f39c80bb590 con 0x559aa80ce5d0 2024-09-15T22:17:24.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.460+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3823789551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d8025580 con 0x559aa80ce5d0 2024-09-15T22:17:24.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.465+0000 7f39bf7fe640 1 -- 172.21.15.5:0/3823789551 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f39c80d6050 con 0x559aa80ce5d0 2024-09-15T22:17:24.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.465+0000 7f39e8cfe740 10 client.26504 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:24.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.465+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3823789551 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559aa81540b0 con 0x559aa80ce5d0 2024-09-15T22:17:24.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.466+0000 7f39bf7fe640 1 -- 172.21.15.5:0/3823789551 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d8025580 con 0x559aa80ce5d0 2024-09-15T22:17:24.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.466+0000 7f39bf7fe640 1 -- 172.21.15.5:0/3823789551 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f39c80f0040 con 0x559aa80ce5d0 2024-09-15T22:17:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.467+0000 7f39e8cfe740 10 client.26504 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.467+0000 7f39e8cfe740 10 client.26504 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:17:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.467+0000 7f39e8cfe740 20 client.26504 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.467+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559aa8154730 0x559aa8156b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.467+0000 7f39e8cfe740 4 client.26504 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:24.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.467+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3823789551 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559aa803bd70 con 0x559aa8154730 2024-09-15T22:17:24.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.470+0000 7f39d77fe640 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559aa8154730 0x559aa8156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.472+0000 7f39d77fe640 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559aa8154730 0x559aa8156b20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6bb241815937f90 server_cookie=bd507c27e8cf2af1 in_seq=0 out_seq=0 2024-09-15T22:17:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.472+0000 7f39bf7fe640 10 client.26504 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:17:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.472+0000 7f39bf7fe640 1 -- 172.21.15.5:0/3823789551 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559aa803bd70 con 0x559aa8154730 2024-09-15T22:17:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.472+0000 7f39bf7fe640 10 client.26504 handle_command_reply: tid=0 2024-09-15T22:17:24.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.472+0000 7f39e8cfe740 1 client.26504 shutdown 2024-09-15T22:17:24.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.473+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3823789551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 msgr2=0x559aa80d2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:24.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.473+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 0x559aa80d2ec0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f39c80edb90 tx=0x7f39c8004700 comp rx=0 tx=0).stop 2024-09-15T22:17:24.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.473+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3823789551 shutdown_connections 2024-09-15T22:17:24.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.473+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x559aa8154730 0x559aa8156b20 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.473+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559aa80d9370 0x559aa80d3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.473+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559aa80cfb40 0x559aa80d3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.473+0000 7f39e8cfe740 1 --2- 172.21.15.5:0/3823789551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559aa80ce5d0 0x559aa80d2ec0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.473+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3823789551 >> 172.21.15.5:0/3823789551 conn(0x559aa8097310 msgr2=0x559aa80d7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:24.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.473+0000 7f39bf7fe640 0 client.26504 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:17:24.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.474+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3823789551 shutdown_connections 2024-09-15T22:17:24.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.474+0000 7f39e8cfe740 1 -- 172.21.15.5:0/3823789551 wait complete. 2024-09-15T22:17:24.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.477+0000 7f39e8cfe740 20 client.26504 trim_cache size 0 max 16384 2024-09-15T22:17:24.479 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 -- 172.21.15.5:0/4154231052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39cc0a3540 msgr2=0x7f39cc0a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39cc0a3540 0x7f39cc0a5a00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f39d8005fd0 tx=0x7f39d8009930 comp rx=0 tx=0).stop 2024-09-15T22:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 -- 172.21.15.5:0/4154231052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e0109450 msgr2=0x7f39e01b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e0109450 0x7f39e01b56d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f39c80084c0 tx=0x7f39c800a670 comp rx=0 tx=0).stop 2024-09-15T22:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 -- 172.21.15.5:0/4154231052 shutdown_connections 2024-09-15T22:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f39cc0a3540 0x7f39cc0a5a00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f39e010ea80 0x7f39e01af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39e0109e00 0x7f39e01b5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 --2- 172.21.15.5:0/4154231052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f39e0109450 0x7f39e01b56d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 -- 172.21.15.5:0/4154231052 >> 172.21.15.5:0/4154231052 conn(0x7f39e006e2d0 msgr2=0x7f39e010c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:24.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 -- 172.21.15.5:0/4154231052 shutdown_connections 2024-09-15T22:17:24.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:24.478+0000 7f39e6925640 1 -- 172.21.15.5:0/4154231052 wait complete. 2024-09-15T22:17:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 475.0 s, 651.3 tps, lat 154.928 ms stddev 226.611 2024-09-15T22:17:25.175 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:24 smithi005 ceph-mon[30944]: pgmap v262: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 635 op/s 2024-09-15T22:17:25.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:17:25.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:24 smithi099 ceph-mon[34892]: pgmap v262: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 635 op/s 2024-09-15T22:17:25.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:24 smithi129 ceph-mon[35314]: pgmap v262: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 6.0 MiB/s wr, 635 op/s 2024-09-15T22:17:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:26.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.304+0000 7f5c10b9b640 1 -- 172.21.15.5:0/2159000635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0c109830 msgr2=0x7f5c0c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:26.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.304+0000 7f5c10b9b640 1 --2- 172.21.15.5:0/2159000635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0c109830 0x7f5c0c109c90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009860 tx=0x7f5c0002a360 comp rx=0 tx=0).stop 2024-09-15T22:17:26.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.309+0000 7f5c10b9b640 1 -- 172.21.15.5:0/2159000635 shutdown_connections 2024-09-15T22:17:26.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.309+0000 7f5c10b9b640 1 --2- 172.21.15.5:0/2159000635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c0c11b140 0x7f5c0c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.309+0000 7f5c10b9b640 1 --2- 172.21.15.5:0/2159000635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0c109830 0x7f5c0c109c90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.309+0000 7f5c10b9b640 1 --2- 172.21.15.5:0/2159000635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c109450 0x7f5c0c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.309+0000 7f5c10b9b640 1 -- 172.21.15.5:0/2159000635 >> 172.21.15.5:0/2159000635 conn(0x7f5c0c06e140 msgr2=0x7f5c0c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:26.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.311+0000 7f5c10b9b640 1 -- 172.21.15.5:0/2159000635 shutdown_connections 2024-09-15T22:17:26.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.311+0000 7f5c10b9b640 1 -- 172.21.15.5:0/2159000635 wait complete. 2024-09-15T22:17:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.312+0000 7f5c10b9b640 1 Processor -- start 2024-09-15T22:17:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.313+0000 7f5c10b9b640 1 -- start start 2024-09-15T22:17:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.313+0000 7f5c10b9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c109450 0x7f5c0c118a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.313+0000 7f5c10b9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0c109830 0x7f5c0c1109f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.313+0000 7f5c10b9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c0c11b140 0x7f5c0c110f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.313+0000 7f5c10b9b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c120e30 con 0x7f5c0c109450 2024-09-15T22:17:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.313+0000 7f5c10b9b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c120cb0 con 0x7f5c0c11b140 2024-09-15T22:17:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.313+0000 7f5c10b9b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c120fb0 con 0x7f5c0c109830 2024-09-15T22:17:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.313+0000 7f5c0a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c109450 0x7f5c0c118a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.313+0000 7f5c0a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c109450 0x7f5c0c118a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56250/0 (socket says 172.21.15.5:56250) 2024-09-15T22:17:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.313+0000 7f5c0a59c640 1 -- 172.21.15.5:0/298970690 learned_addr learned my addr 172.21.15.5:0/298970690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.313+0000 7f5c09d9b640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0c109830 0x7f5c0c1109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.314+0000 7f5c0ad9d640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c0c11b140 0x7f5c0c110f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.315+0000 7f5c0a59c640 1 -- 172.21.15.5:0/298970690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0c109830 msgr2=0x7f5c0c1109f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.315+0000 7f5c0a59c640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0c109830 0x7f5c0c1109f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.315+0000 7f5c0a59c640 1 -- 172.21.15.5:0/298970690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c0c11b140 msgr2=0x7f5c0c110f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.315+0000 7f5c0a59c640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c0c11b140 0x7f5c0c110f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.315+0000 7f5c0a59c640 1 -- 172.21.15.5:0/298970690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0c111750 con 0x7f5c0c109450 2024-09-15T22:17:26.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.320+0000 7f5c0a59c640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c109450 0x7f5c0c118a20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0309d0 tx=0x7f5bfc030e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.322+0000 7f5bfb7fe640 1 -- 172.21.15.5:0/298970690 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc02ecd0 con 0x7f5c0c109450 2024-09-15T22:17:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.322+0000 7f5c10b9b640 1 -- 172.21.15.5:0/298970690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0c1a7130 con 0x7f5c0c109450 2024-09-15T22:17:26.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.322+0000 7f5c10b9b640 1 -- 172.21.15.5:0/298970690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0c1a7640 con 0x7f5c0c109450 2024-09-15T22:17:26.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.322+0000 7f5bfb7fe640 1 -- 172.21.15.5:0/298970690 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bfc005500 con 0x7f5c0c109450 2024-09-15T22:17:26.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.322+0000 7f5bfb7fe640 1 -- 172.21.15.5:0/298970690 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0326a0 con 0x7f5c0c109450 2024-09-15T22:17:26.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.326+0000 7f5bfb7fe640 1 -- 172.21.15.5:0/298970690 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5bfc032840 con 0x7f5c0c109450 2024-09-15T22:17:26.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.330+0000 7f5c12f74740 1 Processor -- start 2024-09-15T22:17:26.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.333+0000 7f5c12f74740 1 -- start start 2024-09-15T22:17:26.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.333+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 0x55f178383090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.333+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1783835d0 0x55f178383a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.333+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f17838e370 0x55f178390760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.333+0000 7f5c12f74740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f1782e92f0 con 0x55f1783835d0 2024-09-15T22:17:26.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.333+0000 7f5c12f74740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f178290cf0 con 0x55f17838e370 2024-09-15T22:17:26.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.333+0000 7f5c12f74740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f1782e52b0 con 0x55f178384b40 2024-09-15T22:17:26.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.333+0000 7f5c0ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 0x55f178383090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.333+0000 7f5c0ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 0x55f178383090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58232/0 (socket says 172.21.15.5:58232) 2024-09-15T22:17:26.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.333+0000 7f5c0ad9d640 1 -- 172.21.15.5:0/2922636909 learned_addr learned my addr 172.21.15.5:0/2922636909 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:26.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.333+0000 7f5c0a59c640 1 --2- 172.21.15.5:0/2922636909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1783835d0 0x55f178383a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.334+0000 7f5bfb7fe640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5bf40a35c0 0x7f5bf40a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.334+0000 7f5c0ad9d640 1 -- 172.21.15.5:0/2922636909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f17838e370 msgr2=0x55f178390760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:17:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.334+0000 7f5c0ad9d640 1 --2- 172.21.15.5:0/2922636909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f17838e370 0x55f178390760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.334+0000 7f5c0ad9d640 1 -- 172.21.15.5:0/2922636909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1783835d0 msgr2=0x55f178383a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.334+0000 7f5c0ad9d640 1 --2- 172.21.15.5:0/2922636909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1783835d0 0x55f178383a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.334+0000 7f5c0ad9d640 1 -- 172.21.15.5:0/2922636909 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1782753a0 con 0x55f178384b40 2024-09-15T22:17:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.334+0000 7f5c0ad9d640 1 --2- 172.21.15.5:0/2922636909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 0x55f178383090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009f60 tx=0x7f5c04009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77a794b8d78d6b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.335+0000 7f5bfb7fe640 1 -- 172.21.15.5:0/298970690 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f5bfc0f34c0 con 0x7f5c0c109450 2024-09-15T22:17:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.335+0000 7f5c0ad9d640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5bf40a35c0 0x7f5bf40a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.335+0000 7f5bf8ff9640 1 -- 172.21.15.5:0/2922636909 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0401c070 con 0x55f178384b40 2024-09-15T22:17:26.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.335+0000 7f5bf8ff9640 1 -- 172.21.15.5:0/2922636909 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c0400ab30 con 0x55f178384b40 2024-09-15T22:17:26.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.335+0000 7f5bf8ff9640 1 -- 172.21.15.5:0/2922636909 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04016790 con 0x55f178384b40 2024-09-15T22:17:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.336+0000 7f5c12f74740 1 -- 172.21.15.5:0/2922636909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 msgr2=0x55f178383090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.336+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2922636909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 0x55f178383090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c04009f60 tx=0x7f5c04009f90 comp rx=0 tx=0).stop 2024-09-15T22:17:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.336+0000 7f5c0ad9d640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5bf40a35c0 0x7f5bf40a5a80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5c040169c0 tx=0x7f5c04016930 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.336+0000 7f5c12f74740 1 -- 172.21.15.5:0/2922636909 shutdown_connections 2024-09-15T22:17:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.336+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2922636909 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f17838e370 0x55f178390760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.336+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2922636909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1783835d0 0x55f178383a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.336+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2922636909 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 0x55f178383090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.336+0000 7f5c12f74740 1 -- 172.21.15.5:0/2922636909 >> 172.21.15.5:0/2922636909 conn(0x55f17834c310 msgr2=0x55f17837fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.336+0000 7f5c12f74740 1 -- 172.21.15.5:0/2922636909 shutdown_connections 2024-09-15T22:17:26.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.336+0000 7f5c12f74740 1 -- 172.21.15.5:0/2922636909 wait complete. 2024-09-15T22:17:26.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.337+0000 7f5c12f74740 1 -- start start 2024-09-15T22:17:26.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.337+0000 7f5c12f74740 1 Processor -- start 2024-09-15T22:17:26.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.338+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1783835d0 0x55f17838dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.338+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f178384b40 0x55f17838aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.338+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f17838e370 0x55f17838b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.338+0000 7f5c09d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f178384b40 0x55f17838aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.338+0000 7f5c09d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f178384b40 0x55f17838aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46420/0 (socket says 172.21.15.5:46420) 2024-09-15T22:17:26.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.338+0000 7f5c09d9b640 1 -- 172.21.15.5:0/2657297882 learned_addr learned my addr 172.21.15.5:0/2657297882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:26.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.338+0000 7f5c0a59c640 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1783835d0 0x55f17838dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.338+0000 7f5c12f74740 1 -- 172.21.15.5:0/2657297882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f178391110 con 0x55f1783835d0 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.338+0000 7f5c12f74740 1 -- 172.21.15.5:0/2657297882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f1782a4060 con 0x55f178384b40 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.338+0000 7f5c12f74740 1 -- 172.21.15.5:0/2657297882 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f178391290 con 0x55f17838e370 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5c0ad9d640 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f17838e370 0x55f17838b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5c12f74740 10 client.0 _ll_register_callbacks cb 0x55f177ff78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5c09d9b640 1 -- 172.21.15.5:0/2657297882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f17838e370 msgr2=0x55f17838b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5c09d9b640 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f17838e370 0x55f17838b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5c09d9b640 1 -- 172.21.15.5:0/2657297882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1783835d0 msgr2=0x55f17838dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5c09d9b640 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1783835d0 0x55f17838dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5c09d9b640 1 -- 172.21.15.5:0/2657297882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f17805a3c0 con 0x55f178384b40 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5c0a59c640 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1783835d0 0x55f17838dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5c09d9b640 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f178384b40 0x55f17838aef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c00035040 tx=0x7f5c00004250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba8d288a28bc449 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:26.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5bf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5c12f74740 1 -- 172.21.15.5:0/2657297882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c0400ab30 con 0x55f178384b40 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5bf1ffb640 1 -- 172.21.15.5:0/2657297882 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00040020 con 0x55f178384b40 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.339+0000 7f5bf1ffb640 1 -- 172.21.15.5:0/2657297882 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f5c00030780 con 0x55f178384b40 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.340+0000 7f5bf1ffb640 1 -- 172.21.15.5:0/2657297882 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5c0004b050 con 0x55f178384b40 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.340+0000 7f5c12f74740 10 client.26510 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.340+0000 7f5c12f74740 1 -- 172.21.15.5:0/2657297882 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55f17839ca00 con 0x55f178384b40 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.340+0000 7f5bf1ffb640 1 -- 172.21.15.5:0/2657297882 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5c00005040 con 0x55f178384b40 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.340+0000 7f5c12f74740 10 client.26510 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.340+0000 7f5c12f74740 10 client.26510 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.340+0000 7f5c12f74740 20 client.26510 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.340+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f17839ddb0 0x55f1783a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.340+0000 7f5c12f74740 4 client.26510 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.340+0000 7f5c12f74740 1 -- 172.21.15.5:0/2657297882 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1782f6860 con 0x55f17839ddb0 2024-09-15T22:17:26.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.340+0000 7f5c0a59c640 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f17839ddb0 0x55f1783a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.341+0000 7f5c0a59c640 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f17839ddb0 0x55f1783a0270 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d19a132f6e5e41b8 server_cookie=7399800c76e727ef in_seq=0 out_seq=0 2024-09-15T22:17:26.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.341+0000 7f5bf1ffb640 10 client.26510 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:17:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5bf1ffb640 1 -- 172.21.15.5:0/2657297882 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f1782f6860 con 0x55f17839ddb0 2024-09-15T22:17:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5bf1ffb640 10 client.26510 handle_command_reply: tid=0 2024-09-15T22:17:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5c12f74740 1 client.26510 shutdown 2024-09-15T22:17:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5c12f74740 1 -- 172.21.15.5:0/2657297882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f178384b40 msgr2=0x55f17838aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f178384b40 0x55f17838aef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c00035040 tx=0x7f5c00004250 comp rx=0 tx=0).stop 2024-09-15T22:17:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5c12f74740 1 -- 172.21.15.5:0/2657297882 shutdown_connections 2024-09-15T22:17:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f17839ddb0 0x55f1783a0270 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5bf1ffb640 0 client.26510 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:17:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f17838e370 0x55f17838b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f178384b40 0x55f17838aef0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2657297882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f1783835d0 0x55f17838dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.342+0000 7f5c12f74740 1 -- 172.21.15.5:0/2657297882 >> 172.21.15.5:0/2657297882 conn(0x55f17834c310 msgr2=0x55f178385930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.343+0000 7f5c12f74740 1 -- 172.21.15.5:0/2657297882 shutdown_connections 2024-09-15T22:17:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.343+0000 7f5c12f74740 1 -- 172.21.15.5:0/2657297882 wait complete. 2024-09-15T22:17:26.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.343+0000 7f5c12f74740 20 client.26510 trim_cache size 0 max 16384 2024-09-15T22:17:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.366+0000 7f5c12f74740 1 Processor -- start 2024-09-15T22:17:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.366+0000 7f5c12f74740 1 -- start start 2024-09-15T22:17:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.367+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1783835d0 0x55f17839a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.367+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f178384b40 0x55f1783a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.367+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f17838e370 0x55f1783a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.367+0000 7f5c12f74740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec012840 con 0x55f178384b40 2024-09-15T22:17:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.367+0000 7f5c12f74740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec010110 con 0x55f17838e370 2024-09-15T22:17:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.367+0000 7f5c12f74740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec00df80 con 0x55f1783835d0 2024-09-15T22:17:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.367+0000 7f5c09d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f178384b40 0x55f1783a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.367+0000 7f5c09d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f178384b40 0x55f1783a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56294/0 (socket says 172.21.15.5:56294) 2024-09-15T22:17:26.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.367+0000 7f5c09d9b640 1 -- 172.21.15.5:0/2115552382 learned_addr learned my addr 172.21.15.5:0/2115552382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:26.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.367+0000 7f5c0a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1783835d0 0x55f17839a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.367+0000 7f5c0ad9d640 1 --2- 172.21.15.5:0/2115552382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f17838e370 0x55f1783a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.368+0000 7f5c09d9b640 1 -- 172.21.15.5:0/2115552382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1783835d0 msgr2=0x55f17839a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.368+0000 7f5c09d9b640 1 --2- 172.21.15.5:0/2115552382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1783835d0 0x55f17839a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.368+0000 7f5c09d9b640 1 -- 172.21.15.5:0/2115552382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f17838e370 msgr2=0x55f1783a6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.368+0000 7f5c09d9b640 1 --2- 172.21.15.5:0/2115552382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f17838e370 0x55f1783a6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.368+0000 7f5c09d9b640 1 -- 172.21.15.5:0/2115552382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17838ba30 con 0x55f178384b40 2024-09-15T22:17:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.368+0000 7f5c09d9b640 1 --2- 172.21.15.5:0/2115552382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f178384b40 0x55f1783a5b40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5c0002a950 tx=0x7f5c00037000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3958adccf4bae39 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.368+0000 7f5bf3fff640 1 -- 172.21.15.5:0/2115552382 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00055070 con 0x55f178384b40 2024-09-15T22:17:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.368+0000 7f5bf3fff640 1 -- 172.21.15.5:0/2115552382 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c0400ab30 con 0x55f178384b40 2024-09-15T22:17:26.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.368+0000 7f5bf3fff640 1 -- 172.21.15.5:0/2115552382 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c00040070 con 0x55f178384b40 2024-09-15T22:17:26.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.369+0000 7f5c12f74740 1 -- 172.21.15.5:0/2115552382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f178384b40 msgr2=0x55f1783a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:26.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.369+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2115552382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f178384b40 0x55f1783a5b40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f5c0002a950 tx=0x7f5c00037000 comp rx=0 tx=0).stop 2024-09-15T22:17:26.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.370+0000 7f5c12f74740 1 -- 172.21.15.5:0/2115552382 shutdown_connections 2024-09-15T22:17:26.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.370+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2115552382 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f17838e370 0x55f1783a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.370+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2115552382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f178384b40 0x55f1783a5b40 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.370+0000 7f5c12f74740 1 --2- 172.21.15.5:0/2115552382 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f1783835d0 0x55f17839a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.370+0000 7f5c12f74740 1 -- 172.21.15.5:0/2115552382 >> 172.21.15.5:0/2115552382 conn(0x55f17834c310 msgr2=0x55f178389fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.372+0000 7f5c12f74740 1 -- 172.21.15.5:0/2115552382 shutdown_connections 2024-09-15T22:17:26.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.372+0000 7f5c12f74740 1 -- 172.21.15.5:0/2115552382 wait complete. 2024-09-15T22:17:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.372+0000 7f5c12f74740 1 -- start start 2024-09-15T22:17:26.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.373+0000 7f5c12f74740 1 Processor -- start 2024-09-15T22:17:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.373+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1783835d0 0x55f178392980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.373+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 0x55f178392ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.373+0000 7f5c12f74740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17838e370 0x55f178407b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c0a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1783835d0 0x55f178392980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c0a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1783835d0 0x55f178392980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46438/0 (socket says 172.21.15.5:46438) 2024-09-15T22:17:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c0a59c640 1 -- 172.21.15.5:0/3576691552 learned_addr learned my addr 172.21.15.5:0/3576691552 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:26.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c09d9b640 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 0x55f178392ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c12f74740 1 -- 172.21.15.5:0/3576691552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec00e5d0 con 0x55f17838e370 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c12f74740 1 -- 172.21.15.5:0/3576691552 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec00fa40 con 0x55f1783835d0 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c12f74740 1 -- 172.21.15.5:0/3576691552 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec00eca0 con 0x55f178384b40 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c12f74740 10 client.0 _ll_register_callbacks cb 0x7f5c00035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c0ad9d640 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17838e370 0x55f178407b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c0a59c640 1 -- 172.21.15.5:0/3576691552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 msgr2=0x55f178392ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c0a59c640 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 0x55f178392ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c0a59c640 1 -- 172.21.15.5:0/3576691552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17838e370 msgr2=0x55f178407b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c0a59c640 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17838e370 0x55f178407b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.374+0000 7f5c0a59c640 1 -- 172.21.15.5:0/3576691552 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f178380960 con 0x55f1783835d0 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.375+0000 7f5c0a59c640 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1783835d0 0x55f178392980 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0bf040 tx=0x7f5bfc035000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1110e70a151d3a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.375+0000 7f5bf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:17:26.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.375+0000 7f5bf1ffb640 1 -- 172.21.15.5:0/3576691552 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bfc0d7060 con 0x55f1783835d0 2024-09-15T22:17:26.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.375+0000 7f5bf1ffb640 1 -- 172.21.15.5:0/3576691552 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f5bfc0f0a30 con 0x55f1783835d0 2024-09-15T22:17:26.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.375+0000 7f5c12f74740 1 -- 172.21.15.5:0/3576691552 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c0400ab30 con 0x55f1783835d0 2024-09-15T22:17:26.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.375+0000 7f5bf1ffb640 1 -- 172.21.15.5:0/3576691552 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5bfc0bfa40 con 0x55f1783835d0 2024-09-15T22:17:26.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.375+0000 7f5c12f74740 10 client.26516 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:26.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.375+0000 7f5c12f74740 1 -- 172.21.15.5:0/3576691552 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f178409100 con 0x55f1783835d0 2024-09-15T22:17:26.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.376+0000 7f5bf1ffb640 1 -- 172.21.15.5:0/3576691552 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c0400ab30 con 0x55f1783835d0 2024-09-15T22:17:26.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.376+0000 7f5bf1ffb640 1 -- 172.21.15.5:0/3576691552 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5bfc0bb9b0 con 0x55f1783835d0 2024-09-15T22:17:26.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.376+0000 7f5c12f74740 10 client.26516 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:26.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.376+0000 7f5c12f74740 10 client.26516 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:17:26.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.376+0000 7f5c12f74740 20 client.26516 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:26.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.376+0000 7f5c12f74740 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f17839ddb0 0x55f17840b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:26.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.376+0000 7f5c12f74740 4 client.26516 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:26.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.376+0000 7f5c12f74740 1 -- 172.21.15.5:0/3576691552 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1782f0d70 con 0x55f17839ddb0 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.376+0000 7f5c09d9b640 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f17839ddb0 0x55f17840b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c09d9b640 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f17839ddb0 0x55f17840b7e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=182c256193e40d21 server_cookie=a139a24d18b903d3 in_seq=0 out_seq=0 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5bf1ffb640 10 client.26516 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5bf1ffb640 1 -- 172.21.15.5:0/3576691552 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1782f0d70 con 0x55f17839ddb0 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5bf1ffb640 10 client.26516 handle_command_reply: tid=0 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 1 client.26516 shutdown 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 1 -- 172.21.15.5:0/3576691552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1783835d0 msgr2=0x55f178392980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1783835d0 0x55f178392980 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0bf040 tx=0x7f5bfc035000 comp rx=0 tx=0).stop 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 1 -- 172.21.15.5:0/3576691552 shutdown_connections 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f17839ddb0 0x55f17840b7e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5bf1ffb640 0 client.26516 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f17838e370 0x55f178407b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f178384b40 0x55f178392ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 1 --2- 172.21.15.5:0/3576691552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f1783835d0 0x55f178392980 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 1 -- 172.21.15.5:0/3576691552 >> 172.21.15.5:0/3576691552 conn(0x55f17834c310 msgr2=0x55f17838c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:26.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 1 -- 172.21.15.5:0/3576691552 shutdown_connections 2024-09-15T22:17:26.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 1 -- 172.21.15.5:0/3576691552 wait complete. 2024-09-15T22:17:26.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.377+0000 7f5c12f74740 20 client.26516 trim_cache size 0 max 16384 2024-09-15T22:17:26.383 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:17:26.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 -- 172.21.15.5:0/298970690 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5bf40a35c0 msgr2=0x7f5bf40a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:26.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5bf40a35c0 0x7f5bf40a5a80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5c040169c0 tx=0x7f5c04016930 comp rx=0 tx=0).stop 2024-09-15T22:17:26.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 -- 172.21.15.5:0/298970690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c109450 msgr2=0x7f5c0c118a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:26.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c109450 0x7f5c0c118a20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc0309d0 tx=0x7f5bfc030e90 comp rx=0 tx=0).stop 2024-09-15T22:17:26.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 -- 172.21.15.5:0/298970690 shutdown_connections 2024-09-15T22:17:26.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5bf40a35c0 0x7f5bf40a5a80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5c0c11b140 0x7f5c0c110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5c0c109830 0x7f5c0c1109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 --2- 172.21.15.5:0/298970690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c0c109450 0x7f5c0c118a20 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:26.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 -- 172.21.15.5:0/298970690 >> 172.21.15.5:0/298970690 conn(0x7f5c0c06e140 msgr2=0x7f5c0c11c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:26.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 -- 172.21.15.5:0/298970690 shutdown_connections 2024-09-15T22:17:26.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:26.378+0000 7f5c10b9b640 1 -- 172.21.15.5:0/298970690 wait complete. 2024-09-15T22:17:27.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:26 smithi005 ceph-mon[30944]: pgmap v263: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.6 MiB/s wr, 602 op/s 2024-09-15T22:17:27.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:17:27.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:26 smithi099 ceph-mon[34892]: pgmap v263: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.6 MiB/s wr, 602 op/s 2024-09-15T22:17:27.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:26 smithi129 ceph-mon[35314]: pgmap v263: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.6 MiB/s wr, 602 op/s 2024-09-15T22:17:27.723 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.927+0000 7f154ffff640 1 -- 172.21.15.5:0/2478097373 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554003650 con 0x7f1558109830 2024-09-15T22:17:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.927+0000 7f155f9a6640 1 -- 172.21.15.5:0/2478097373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1558109830 msgr2=0x7f1558109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.927+0000 7f155f9a6640 1 --2- 172.21.15.5:0/2478097373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1558109830 0x7f1558109c90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1554009900 tx=0x7f155402a420 comp rx=0 tx=0).stop 2024-09-15T22:17:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.927+0000 7f155f9a6640 1 -- 172.21.15.5:0/2478097373 shutdown_connections 2024-09-15T22:17:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.927+0000 7f155f9a6640 1 --2- 172.21.15.5:0/2478097373 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f155811b140 0x7f155811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.927+0000 7f155f9a6640 1 --2- 172.21.15.5:0/2478097373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1558109830 0x7f1558109c90 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.927+0000 7f155f9a6640 1 --2- 172.21.15.5:0/2478097373 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1558109450 0x7f1558110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.927+0000 7f155f9a6640 1 -- 172.21.15.5:0/2478097373 >> 172.21.15.5:0/2478097373 conn(0x7f155806e140 msgr2=0x7f155806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.927+0000 7f155f9a6640 1 -- 172.21.15.5:0/2478097373 shutdown_connections 2024-09-15T22:17:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.928+0000 7f155f9a6640 1 -- 172.21.15.5:0/2478097373 wait complete. 2024-09-15T22:17:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.928+0000 7f155f9a6640 1 Processor -- start 2024-09-15T22:17:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.928+0000 7f155f9a6640 1 -- start start 2024-09-15T22:17:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.929+0000 7f155f9a6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558109450 0x7f15581a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.929+0000 7f155f9a6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1558109830 0x7f15581a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.929+0000 7f155f9a6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155811b140 0x7f15581abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.929+0000 7f155f9a6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1558120df0 con 0x7f155811b140 2024-09-15T22:17:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.929+0000 7f155f9a6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1558120c70 con 0x7f1558109450 2024-09-15T22:17:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.929+0000 7f155f9a6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1558120f70 con 0x7f1558109830 2024-09-15T22:17:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.930+0000 7f155df43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155811b140 0x7f15581abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.930+0000 7f155df43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155811b140 0x7f15581abb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56328/0 (socket says 172.21.15.5:56328) 2024-09-15T22:17:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.930+0000 7f155df43640 1 -- 172.21.15.5:0/3007072424 learned_addr learned my addr 172.21.15.5:0/3007072424 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.931+0000 7f155df43640 1 -- 172.21.15.5:0/3007072424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1558109830 msgr2=0x7f15581a77f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:17:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.931+0000 7f155df43640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1558109830 0x7f15581a77f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.931+0000 7f155df43640 1 -- 172.21.15.5:0/3007072424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558109450 msgr2=0x7f15581a72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.931+0000 7f155df43640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558109450 0x7f15581a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.931+0000 7f155df43640 1 -- 172.21.15.5:0/3007072424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15581ac280 con 0x7f155811b140 2024-09-15T22:17:28.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.942+0000 7f155df43640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155811b140 0x7f15581abb80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f154800e970 tx=0x7f154800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.950+0000 7f154e7fc640 1 -- 172.21.15.5:0/3007072424 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154800ccc0 con 0x7f155811b140 2024-09-15T22:17:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.951+0000 7f155f9a6640 1 -- 172.21.15.5:0/3007072424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15581ac570 con 0x7f155811b140 2024-09-15T22:17:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.951+0000 7f155f9a6640 1 -- 172.21.15.5:0/3007072424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1558110920 con 0x7f155811b140 2024-09-15T22:17:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.951+0000 7f154e7fc640 1 -- 172.21.15.5:0/3007072424 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f154800ce60 con 0x7f155811b140 2024-09-15T22:17:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.951+0000 7f154e7fc640 1 -- 172.21.15.5:0/3007072424 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548010690 con 0x7f155811b140 2024-09-15T22:17:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.958+0000 7f1561d7f740 1 Processor -- start 2024-09-15T22:17:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.958+0000 7f1561d7f740 1 -- start start 2024-09-15T22:17:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.959+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb406ab40 0x555eb4069090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:28.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.959+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 0x555eb4069a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:28.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.959+0000 7f155d742640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 0x555eb4069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.959+0000 7f155d742640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 0x555eb4069a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58286/0 (socket says 172.21.15.5:58286) 2024-09-15T22:17:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.964+0000 7f155cf41640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb406ab40 0x555eb4069090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.964+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb4076760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.964+0000 7f1561d7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555eb3fcf2f0 con 0x555eb4074370 2024-09-15T22:17:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.964+0000 7f1561d7f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555eb3f76cf0 con 0x555eb406ab40 2024-09-15T22:17:28.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.964+0000 7f1561d7f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555eb3fcb2b0 con 0x555eb40695d0 2024-09-15T22:17:28.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.964+0000 7f155d742640 1 -- 172.21.15.5:0/4236915799 learned_addr learned my addr 172.21.15.5:0/4236915799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:28.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.968+0000 7f155d742640 1 -- 172.21.15.5:0/4236915799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb406ab40 msgr2=0x555eb4069090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.969+0000 7f155d742640 1 --2- 172.21.15.5:0/4236915799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb406ab40 0x555eb4069090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.969+0000 7f155d742640 1 -- 172.21.15.5:0/4236915799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 msgr2=0x555eb4076760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:17:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.969+0000 7f155d742640 1 --2- 172.21.15.5:0/4236915799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb4076760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.969+0000 7f155d742640 1 -- 172.21.15.5:0/4236915799 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb3f5b3a0 con 0x555eb40695d0 2024-09-15T22:17:28.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.970+0000 7f154e7fc640 1 -- 172.21.15.5:0/3007072424 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f15480046c0 con 0x7f155811b140 2024-09-15T22:17:28.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.971+0000 7f154e7fc640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f15440a3500 0x7f15440a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.971+0000 7f154e7fc640 1 -- 172.21.15.5:0/3007072424 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f15480d0550 con 0x7f155811b140 2024-09-15T22:17:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.971+0000 7f155cf41640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f15440a3500 0x7f15440a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:28.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.972+0000 7f155d742640 1 --2- 172.21.15.5:0/4236915799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 0x555eb4069a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1550009ea0 tx=0x7f1550009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9c754d6940b04d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:28.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.975+0000 7f155cf41640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f15440a3500 0x7f15440a59c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1554009230 tx=0x7f1554000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:28.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.977+0000 7f15437fe640 1 -- 172.21.15.5:0/4236915799 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550010660 con 0x555eb40695d0 2024-09-15T22:17:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.977+0000 7f15437fe640 1 -- 172.21.15.5:0/4236915799 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1550010ce0 con 0x555eb40695d0 2024-09-15T22:17:28.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.977+0000 7f15437fe640 1 -- 172.21.15.5:0/4236915799 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155000fc30 con 0x555eb40695d0 2024-09-15T22:17:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.978+0000 7f1561d7f740 1 -- 172.21.15.5:0/4236915799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 msgr2=0x555eb4069a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:28.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.978+0000 7f1561d7f740 1 --2- 172.21.15.5:0/4236915799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 0x555eb4069a10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1550009ea0 tx=0x7f1550009f90 comp rx=0 tx=0).stop 2024-09-15T22:17:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.980+0000 7f1561d7f740 1 -- 172.21.15.5:0/4236915799 shutdown_connections 2024-09-15T22:17:28.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.980+0000 7f1561d7f740 1 --2- 172.21.15.5:0/4236915799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb4076760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.980+0000 7f1561d7f740 1 --2- 172.21.15.5:0/4236915799 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 0x555eb4069a10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.980+0000 7f1561d7f740 1 --2- 172.21.15.5:0/4236915799 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb406ab40 0x555eb4069090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.980+0000 7f1561d7f740 1 -- 172.21.15.5:0/4236915799 >> 172.21.15.5:0/4236915799 conn(0x555eb4032310 msgr2=0x555eb4065c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.983+0000 7f1561d7f740 1 -- 172.21.15.5:0/4236915799 shutdown_connections 2024-09-15T22:17:28.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.983+0000 7f1561d7f740 1 -- 172.21.15.5:0/4236915799 wait complete. 2024-09-15T22:17:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.983+0000 7f1561d7f740 1 -- start start 2024-09-15T22:17:28.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.983+0000 7f1561d7f740 1 Processor -- start 2024-09-15T22:17:28.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.984+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb40695d0 0x555eb4073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.984+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 0x555eb4070f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.984+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb4071490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.984+0000 7f1561d7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555eb40770c0 con 0x555eb4074370 2024-09-15T22:17:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.984+0000 7f1561d7f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555eb3f8a060 con 0x555eb40695d0 2024-09-15T22:17:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.984+0000 7f1561d7f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555eb4077240 con 0x555eb406ab40 2024-09-15T22:17:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.984+0000 7f1561d7f740 10 client.0 _ll_register_callbacks cb 0x555eb3cdd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.985+0000 7f155d742640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb40695d0 0x555eb4073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:28.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.985+0000 7f155d742640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb40695d0 0x555eb4073ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:46458/0 (socket says 172.21.15.5:46458) 2024-09-15T22:17:28.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.985+0000 7f155d742640 1 -- 172.21.15.5:0/3265815897 learned_addr learned my addr 172.21.15.5:0/3265815897 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:28.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.985+0000 7f155cf41640 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 0x555eb4070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:28.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.986+0000 7f155cf41640 1 -- 172.21.15.5:0/3265815897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb40695d0 msgr2=0x555eb4073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.986+0000 7f155cf41640 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb40695d0 0x555eb4073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.986+0000 7f155cf41640 1 -- 172.21.15.5:0/3265815897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 msgr2=0x555eb4071490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.986+0000 7f155cf41640 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb4071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.986+0000 7f155cf41640 1 -- 172.21.15.5:0/3265815897 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb3d403c0 con 0x555eb406ab40 2024-09-15T22:17:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.986+0000 7f155cf41640 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 0x555eb4070f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1554033040 tx=0x7f1554009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63200a968cafc0ed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.988+0000 7f1540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:17:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.988+0000 7f1540ff9640 1 -- 172.21.15.5:0/3265815897 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155404e070 con 0x555eb406ab40 2024-09-15T22:17:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.988+0000 7f1540ff9640 1 -- 172.21.15.5:0/3265815897 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f155404a450 con 0x555eb406ab40 2024-09-15T22:17:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.988+0000 7f1561d7f740 1 -- 172.21.15.5:0/3265815897 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1550010ce0 con 0x555eb406ab40 2024-09-15T22:17:28.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.992+0000 7f1540ff9640 1 -- 172.21.15.5:0/3265815897 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f155405a050 con 0x555eb406ab40 2024-09-15T22:17:28.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.993+0000 7f1561d7f740 10 client.26470 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:28.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.993+0000 7f1561d7f740 1 -- 172.21.15.5:0/3265815897 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555eb4082b10 con 0x555eb406ab40 2024-09-15T22:17:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.999+0000 7f1540ff9640 1 -- 172.21.15.5:0/3265815897 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1550010ce0 con 0x555eb406ab40 2024-09-15T22:17:29.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:28.999+0000 7f1540ff9640 1 -- 172.21.15.5:0/3265815897 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f15540317e0 con 0x555eb406ab40 2024-09-15T22:17:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.000+0000 7f1561d7f740 10 client.26470 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.000+0000 7f1561d7f740 10 client.26470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.001+0000 7f1561d7f740 20 client.26470 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.001+0000 7f1561d7f740 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555eb4083f70 0x555eb4086430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.001+0000 7f1561d7f740 4 client.26470 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.001+0000 7f1561d7f740 1 -- 172.21.15.5:0/3265815897 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eb3fdc860 con 0x555eb4083f70 2024-09-15T22:17:29.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.004+0000 7f155d742640 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555eb4083f70 0x555eb4086430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.006+0000 7f155d742640 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555eb4083f70 0x555eb4086430 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a480dc997e3ce02 server_cookie=38c0d66f15798699 in_seq=0 out_seq=0 2024-09-15T22:17:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.007+0000 7f1540ff9640 10 client.26470 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.007+0000 7f1540ff9640 1 -- 172.21.15.5:0/3265815897 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555eb3fdc860 con 0x555eb4083f70 2024-09-15T22:17:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.007+0000 7f1540ff9640 10 client.26470 handle_command_reply: tid=0 2024-09-15T22:17:29.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.007+0000 7f1561d7f740 1 client.26470 shutdown 2024-09-15T22:17:29.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.008+0000 7f1561d7f740 1 -- 172.21.15.5:0/3265815897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 msgr2=0x555eb4070f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:29.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.008+0000 7f1561d7f740 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 0x555eb4070f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1554033040 tx=0x7f1554009260 comp rx=0 tx=0).stop 2024-09-15T22:17:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.009+0000 7f1561d7f740 1 -- 172.21.15.5:0/3265815897 shutdown_connections 2024-09-15T22:17:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.010+0000 7f1561d7f740 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb4071490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.010+0000 7f1561d7f740 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 0x555eb4070f50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.010+0000 7f1561d7f740 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555eb4083f70 0x555eb4086430 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.010+0000 7f1561d7f740 1 --2- 172.21.15.5:0/3265815897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb40695d0 0x555eb4073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.010+0000 7f1540ff9640 0 client.26470 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.010+0000 7f1561d7f740 1 -- 172.21.15.5:0/3265815897 >> 172.21.15.5:0/3265815897 conn(0x555eb4032310 msgr2=0x555eb406b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.010+0000 7f1561d7f740 1 -- 172.21.15.5:0/3265815897 shutdown_connections 2024-09-15T22:17:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.010+0000 7f1561d7f740 1 -- 172.21.15.5:0/3265815897 wait complete. 2024-09-15T22:17:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.010+0000 7f1561d7f740 20 client.26470 trim_cache size 0 max 16384 2024-09-15T22:17:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.031+0000 7f1561d7f740 1 Processor -- start 2024-09-15T22:17:29.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.031+0000 7f1561d7f740 1 -- start start 2024-09-15T22:17:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.031+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb40695d0 0x555eb40803c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.031+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 0x555eb4065700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.031+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb4074370 0x555eb4065c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.031+0000 7f1561d7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f153c0129f0 con 0x555eb40695d0 2024-09-15T22:17:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.031+0000 7f1561d7f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f153c0102c0 con 0x555eb4074370 2024-09-15T22:17:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.031+0000 7f1561d7f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f153c00e130 con 0x555eb406ab40 2024-09-15T22:17:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.032+0000 7f155cf41640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 0x555eb4065700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:29.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.032+0000 7f155cf41640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 0x555eb4065700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58298/0 (socket says 172.21.15.5:58298) 2024-09-15T22:17:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.032+0000 7f155cf41640 1 -- 172.21.15.5:0/1751338386 learned_addr learned my addr 172.21.15.5:0/1751338386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:29.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.033+0000 7f155d742640 1 --2- 172.21.15.5:0/1751338386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb40695d0 0x555eb40803c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.034+0000 7f155d742640 1 -- 172.21.15.5:0/1751338386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 msgr2=0x555eb4065700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.034+0000 7f155d742640 1 --2- 172.21.15.5:0/1751338386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 0x555eb4065700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.034+0000 7f155d742640 1 -- 172.21.15.5:0/1751338386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb4074370 msgr2=0x555eb4065c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:17:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.034+0000 7f155d742640 1 --2- 172.21.15.5:0/1751338386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb4074370 0x555eb4065c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.034+0000 7f155d742640 1 -- 172.21.15.5:0/1751338386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eb4066180 con 0x555eb40695d0 2024-09-15T22:17:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.034+0000 7f155d742640 1 --2- 172.21.15.5:0/1751338386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb40695d0 0x555eb40803c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1550004a80 tx=0x7f155000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2cb3bddae6de82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.035+0000 7f1542ffd640 1 -- 172.21.15.5:0/1751338386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550004d20 con 0x555eb40695d0 2024-09-15T22:17:29.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.035+0000 7f1542ffd640 1 -- 172.21.15.5:0/1751338386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1550003e50 con 0x555eb40695d0 2024-09-15T22:17:29.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.035+0000 7f1542ffd640 1 -- 172.21.15.5:0/1751338386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1550004050 con 0x555eb40695d0 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.036+0000 7f1561d7f740 1 -- 172.21.15.5:0/1751338386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb40695d0 msgr2=0x555eb40803c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.036+0000 7f1561d7f740 1 --2- 172.21.15.5:0/1751338386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb40695d0 0x555eb40803c0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1550004a80 tx=0x7f155000f3c0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.036+0000 7f1561d7f740 1 -- 172.21.15.5:0/1751338386 shutdown_connections 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.036+0000 7f1561d7f740 1 --2- 172.21.15.5:0/1751338386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb4074370 0x555eb4065c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.036+0000 7f1561d7f740 1 --2- 172.21.15.5:0/1751338386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb406ab40 0x555eb4065700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.036+0000 7f1561d7f740 1 --2- 172.21.15.5:0/1751338386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb40695d0 0x555eb40803c0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.036+0000 7f1561d7f740 1 -- 172.21.15.5:0/1751338386 >> 172.21.15.5:0/1751338386 conn(0x555eb4032310 msgr2=0x555eb4070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.036+0000 7f1561d7f740 1 -- 172.21.15.5:0/1751338386 shutdown_connections 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.036+0000 7f1561d7f740 1 -- 172.21.15.5:0/1751338386 wait complete. 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.036+0000 7f1561d7f740 1 -- start start 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.037+0000 7f1561d7f740 1 Processor -- start 2024-09-15T22:17:29.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.037+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 0x555eb407c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.037+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb406ab40 0x555eb406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.037+0000 7f1561d7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb406de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.037+0000 7f1561d7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f153c00e780 con 0x555eb4074370 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.037+0000 7f1561d7f740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f153c00fbf0 con 0x555eb406ab40 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.037+0000 7f1561d7f740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f153c00ee50 con 0x555eb40695d0 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.037+0000 7f1561d7f740 10 client.0 _ll_register_callbacks cb 0x7f1554041070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.037+0000 7f155df43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb406de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.037+0000 7f155cf41640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb406ab40 0x555eb406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f155df43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb406de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56372/0 (socket says 172.21.15.5:56372) 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f155df43640 1 -- 172.21.15.5:0/2629305828 learned_addr learned my addr 172.21.15.5:0/2629305828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f155cf41640 1 -- 172.21.15.5:0/2629305828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 msgr2=0x555eb407c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f155d742640 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 0x555eb407c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:29.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f155cf41640 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 0x555eb407c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f155cf41640 1 -- 172.21.15.5:0/2629305828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 msgr2=0x555eb406de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f155cf41640 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb406de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f155cf41640 1 -- 172.21.15.5:0/2629305828 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eb406e370 con 0x555eb406ab40 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f155df43640 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb406de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f155cf41640 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb406ab40 0x555eb406d8f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1554006010 tx=0x7f1554031670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8be5b7937d59e62e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f1540ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f1540ff9640 1 -- 172.21.15.5:0/2629305828 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1554062070 con 0x555eb406ab40 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.038+0000 7f1540ff9640 1 -- 172.21.15.5:0/2629305828 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f1554031970 con 0x555eb406ab40 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.039+0000 7f1561d7f740 1 -- 172.21.15.5:0/2629305828 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1550003e50 con 0x555eb406ab40 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.039+0000 7f1540ff9640 1 -- 172.21.15.5:0/2629305828 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f155406e050 con 0x555eb406ab40 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.039+0000 7f1561d7f740 10 client.26528 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:29.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.039+0000 7f1561d7f740 1 -- 172.21.15.5:0/2629305828 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555eb40ee960 con 0x555eb406ab40 2024-09-15T22:17:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.039+0000 7f1540ff9640 1 -- 172.21.15.5:0/2629305828 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1550003e50 con 0x555eb406ab40 2024-09-15T22:17:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.039+0000 7f1540ff9640 1 -- 172.21.15.5:0/2629305828 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f155404a4e0 con 0x555eb406ab40 2024-09-15T22:17:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.039+0000 7f1561d7f740 10 client.26528 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.039+0000 7f1561d7f740 10 client.26528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.039+0000 7f1561d7f740 20 client.26528 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:29.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.040+0000 7f1561d7f740 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555eb40ef080 0x555eb40f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:29.045 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:28 smithi005 ceph-mon[30944]: pgmap v264: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-09-15T22:17:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.040+0000 7f1561d7f740 4 client.26528 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:17:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.040+0000 7f1561d7f740 1 -- 172.21.15.5:0/2629305828 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555eb3fd6d70 con 0x555eb40ef080 2024-09-15T22:17:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.040+0000 7f155d742640 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555eb40ef080 0x555eb40f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.040+0000 7f1540ff9640 10 client.26528 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.040+0000 7f155d742640 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555eb40ef080 0x555eb40f14a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc0547970d502ed server_cookie=e07b16135fe3d2e3 in_seq=0 out_seq=0 2024-09-15T22:17:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1540ff9640 1 -- 172.21.15.5:0/2629305828 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555eb3fd6d70 con 0x555eb40ef080 2024-09-15T22:17:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1540ff9640 10 client.26528 handle_command_reply: tid=0 2024-09-15T22:17:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1561d7f740 1 client.26528 shutdown 2024-09-15T22:17:29.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1561d7f740 1 -- 172.21.15.5:0/2629305828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb406ab40 msgr2=0x555eb406d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:29.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1561d7f740 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb406ab40 0x555eb406d8f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1554006010 tx=0x7f1554031670 comp rx=0 tx=0).stop 2024-09-15T22:17:29.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1561d7f740 1 -- 172.21.15.5:0/2629305828 shutdown_connections 2024-09-15T22:17:29.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1561d7f740 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eb4074370 0x555eb406de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1561d7f740 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555eb40ef080 0x555eb40f14a0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1561d7f740 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555eb406ab40 0x555eb406d8f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1561d7f740 1 --2- 172.21.15.5:0/2629305828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555eb40695d0 0x555eb407c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1561d7f740 1 -- 172.21.15.5:0/2629305828 >> 172.21.15.5:0/2629305828 conn(0x555eb4032310 msgr2=0x555eb406f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:29.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1540ff9640 0 client.26528 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:29.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1561d7f740 1 -- 172.21.15.5:0/2629305828 shutdown_connections 2024-09-15T22:17:29.048 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:17:29.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.041+0000 7f1561d7f740 1 -- 172.21.15.5:0/2629305828 wait complete. 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.042+0000 7f1561d7f740 20 client.26528 trim_cache size 0 max 16384 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.042+0000 7f155f9a6640 1 -- 172.21.15.5:0/3007072424 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f15440a3500 msgr2=0x7f15440a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.042+0000 7f155f9a6640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f15440a3500 0x7f15440a59c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1554009230 tx=0x7f1554000bc0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.042+0000 7f155f9a6640 1 -- 172.21.15.5:0/3007072424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155811b140 msgr2=0x7f15581abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.042+0000 7f155f9a6640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155811b140 0x7f15581abb80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f154800e970 tx=0x7f154800ee30 comp rx=0 tx=0).stop 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.043+0000 7f155f9a6640 1 -- 172.21.15.5:0/3007072424 shutdown_connections 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.043+0000 7f155f9a6640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f15440a3500 0x7f15440a59c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.043+0000 7f155f9a6640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f155811b140 0x7f15581abb80 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.043+0000 7f155f9a6640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1558109830 0x7f15581a77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.043+0000 7f155f9a6640 1 --2- 172.21.15.5:0/3007072424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1558109450 0x7f15581a72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.043+0000 7f155f9a6640 1 -- 172.21.15.5:0/3007072424 >> 172.21.15.5:0/3007072424 conn(0x7f155806e140 msgr2=0x7f155811bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.043+0000 7f155f9a6640 1 -- 172.21.15.5:0/3007072424 shutdown_connections 2024-09-15T22:17:29.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:29.043+0000 7f155f9a6640 1 -- 172.21.15.5:0/3007072424 wait complete. 2024-09-15T22:17:29.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:28 smithi099 ceph-mon[34892]: pgmap v264: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-09-15T22:17:29.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:28 smithi129 ceph-mon[35314]: pgmap v264: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 5.5 MiB/s wr, 593 op/s 2024-09-15T22:17:29.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 480.0 s, 592.4 tps, lat 166.713 ms stddev 190.828 2024-09-15T22:17:30.868 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:17:30.869 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:17:31.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:30 smithi099 ceph-mon[34892]: pgmap v265: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.3 MiB/s wr, 550 op/s 2024-09-15T22:17:31.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:30 smithi129 ceph-mon[35314]: pgmap v265: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.3 MiB/s wr, 550 op/s 2024-09-15T22:17:31.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:30 smithi005 ceph-mon[30944]: pgmap v265: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 5.3 MiB/s wr, 550 op/s 2024-09-15T22:17:31.386 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.981+0000 7ff3de02b640 1 -- 172.21.15.5:0/602854584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d8100100 msgr2=0x7ff3d8100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.981+0000 7ff3de02b640 1 --2- 172.21.15.5:0/602854584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d8100100 0x7ff3d8100560 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00fb10 tx=0x7ff3cc02e570 comp rx=0 tx=0).stop 2024-09-15T22:17:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.981+0000 7ff3de02b640 1 -- 172.21.15.5:0/602854584 shutdown_connections 2024-09-15T22:17:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.981+0000 7ff3de02b640 1 --2- 172.21.15.5:0/602854584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d8100aa0 0x7ff3d810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.981+0000 7ff3de02b640 1 --2- 172.21.15.5:0/602854584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d8100100 0x7ff3d8100560 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.981+0000 7ff3de02b640 1 --2- 172.21.15.5:0/602854584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d8106b80 0x7ff3d8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.981+0000 7ff3de02b640 1 -- 172.21.15.5:0/602854584 >> 172.21.15.5:0/602854584 conn(0x7ff3d8077230 msgr2=0x7ff3d80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.981+0000 7ff3de02b640 1 -- 172.21.15.5:0/602854584 shutdown_connections 2024-09-15T22:17:31.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.981+0000 7ff3de02b640 1 -- 172.21.15.5:0/602854584 wait complete. 2024-09-15T22:17:31.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.982+0000 7ff3de02b640 1 Processor -- start 2024-09-15T22:17:31.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.982+0000 7ff3de02b640 1 -- start start 2024-09-15T22:17:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.982+0000 7ff3de02b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d8100100 0x7ff3d81962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.982+0000 7ff3de02b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d8100aa0 0x7ff3d8196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.982+0000 7ff3de02b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d8106b80 0x7ff3d819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.982+0000 7ff3de02b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d810fe40 con 0x7ff3d8100100 2024-09-15T22:17:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.982+0000 7ff3de02b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d810fcc0 con 0x7ff3d8106b80 2024-09-15T22:17:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.982+0000 7ff3de02b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d810ffc0 con 0x7ff3d8100aa0 2024-09-15T22:17:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d8100aa0 0x7ff3d8196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:31.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d8100100 0x7ff3d81962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d8100100 0x7ff3d81962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56400/0 (socket says 172.21.15.5:56400) 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d77fe640 1 -- 172.21.15.5:0/4012388910 learned_addr learned my addr 172.21.15.5:0/4012388910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d8100aa0 0x7ff3d8196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58314/0 (socket says 172.21.15.5:58314) 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d7fff640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d8106b80 0x7ff3d819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d6ffd640 1 -- 172.21.15.5:0/4012388910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d8106b80 msgr2=0x7ff3d819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d6ffd640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d8106b80 0x7ff3d819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d6ffd640 1 -- 172.21.15.5:0/4012388910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d8100100 msgr2=0x7ff3d81962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d6ffd640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d8100100 0x7ff3d81962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d6ffd640 1 -- 172.21.15.5:0/4012388910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d819b290 con 0x7ff3d8100aa0 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.983+0000 7ff3d6ffd640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d8100aa0 0x7ff3d8196820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc006010 tx=0x7ff3cc00f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.984+0000 7ff3d4ff9640 1 -- 172.21.15.5:0/4012388910 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc044070 con 0x7ff3d8100aa0 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.984+0000 7ff3d4ff9640 1 -- 172.21.15.5:0/4012388910 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3cc00ee40 con 0x7ff3d8100aa0 2024-09-15T22:17:31.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.984+0000 7ff3de02b640 1 -- 172.21.15.5:0/4012388910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d819b520 con 0x7ff3d8100aa0 2024-09-15T22:17:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.984+0000 7ff3d4ff9640 1 -- 172.21.15.5:0/4012388910 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc003c60 con 0x7ff3d8100aa0 2024-09-15T22:17:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.984+0000 7ff3de02b640 1 -- 172.21.15.5:0/4012388910 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d81a2e60 con 0x7ff3d8100aa0 2024-09-15T22:17:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.987+0000 7ff3e0404740 1 Processor -- start 2024-09-15T22:17:31.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.987+0000 7ff3e0404740 1 -- start start 2024-09-15T22:17:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.987+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedc9b60 0x561aeedc9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.987+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 0x561aeedd4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.987+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedd4d40 0x561aeedd7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.987+0000 7ff3e0404740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561aeed2f280 con 0x561aeedd4d40 2024-09-15T22:17:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.987+0000 7ff3e0404740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561aeecd6b10 con 0x561aeedc9b60 2024-09-15T22:17:31.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.987+0000 7ff3e0404740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561aeed2b240 con 0x561aeedca480 2024-09-15T22:17:31.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.987+0000 7ff3d4ff9640 1 -- 172.21.15.5:0/4012388910 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff3cc047070 con 0x7ff3d8100aa0 2024-09-15T22:17:31.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.987+0000 7ff3d77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 0x561aeedd4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:31.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.988+0000 7ff3d77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 0x561aeedd4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58318/0 (socket says 172.21.15.5:58318) 2024-09-15T22:17:31.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.988+0000 7ff3d77fe640 1 -- 172.21.15.5:0/2393225541 learned_addr learned my addr 172.21.15.5:0/2393225541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:31.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.989+0000 7ff3d77fe640 1 -- 172.21.15.5:0/2393225541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedc9b60 msgr2=0x561aeedc9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:31.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.989+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/2393225541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedc9b60 0x561aeedc9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:31.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.989+0000 7ff3d77fe640 1 -- 172.21.15.5:0/2393225541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedd4d40 msgr2=0x561aeedd7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:31.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.989+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/2393225541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedd4d40 0x561aeedd7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:31.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.989+0000 7ff3d77fe640 1 -- 172.21.15.5:0/2393225541 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aeece31f0 con 0x561aeedca480 2024-09-15T22:17:31.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.989+0000 7ff3d4ff9640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff3c00a3430 0x7ff3c00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:31.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.989+0000 7ff3d4ff9640 1 -- 172.21.15.5:0/4012388910 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7ff3cc0f4300 con 0x7ff3d8100aa0 2024-09-15T22:17:32.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.990+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/2393225541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 0x561aeedd4800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800bda0 tx=0x7ff3c80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1bbf5ac6f0ebace server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:32.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.990+0000 7ff3d7fff640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff3c00a3430 0x7ff3c00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:32.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.990+0000 7ff3bdffb640 1 -- 172.21.15.5:0/2393225541 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8024680 con 0x561aeedca480 2024-09-15T22:17:32.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.990+0000 7ff3bdffb640 1 -- 172.21.15.5:0/2393225541 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c8022860 con 0x561aeedca480 2024-09-15T22:17:32.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.990+0000 7ff3bdffb640 1 -- 172.21.15.5:0/2393225541 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c80237e0 con 0x561aeedca480 2024-09-15T22:17:32.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.991+0000 7ff3e0404740 1 -- 172.21.15.5:0/2393225541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 msgr2=0x561aeedd4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:32.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.991+0000 7ff3d7fff640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff3c00a3430 0x7ff3c00a58f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff3d81077c0 tx=0x7ff3d0009ae0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:32.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.991+0000 7ff3e0404740 1 --2- 172.21.15.5:0/2393225541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 0x561aeedd4800 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800bda0 tx=0x7ff3c80107a0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.991+0000 7ff3e0404740 1 -- 172.21.15.5:0/2393225541 shutdown_connections 2024-09-15T22:17:32.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.991+0000 7ff3e0404740 1 --2- 172.21.15.5:0/2393225541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedd4d40 0x561aeedd7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.991+0000 7ff3e0404740 1 --2- 172.21.15.5:0/2393225541 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 0x561aeedd4800 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.991+0000 7ff3e0404740 1 --2- 172.21.15.5:0/2393225541 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedc9b60 0x561aeedc9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.991+0000 7ff3e0404740 1 -- 172.21.15.5:0/2393225541 >> 172.21.15.5:0/2393225541 conn(0x561aeedc60f0 msgr2=0x561aeedc64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.991+0000 7ff3e0404740 1 -- 172.21.15.5:0/2393225541 shutdown_connections 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.991+0000 7ff3e0404740 1 -- 172.21.15.5:0/2393225541 wait complete. 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3e0404740 1 -- start start 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3e0404740 1 Processor -- start 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 0x561aeedd1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedca480 0x561aeedd2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedd4d40 0x561aeedd2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3e0404740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561aeedd7910 con 0x561aeedca480 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3e0404740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561aeece9f10 con 0x561aeedd4d40 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3e0404740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561aeedd7a90 con 0x561aeedc9b60 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3e0404740 10 client.0 _ll_register_callbacks cb 0x561aeecb2a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3d77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 0x561aeedd1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:32.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3d77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 0x561aeedd1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58322/0 (socket says 172.21.15.5:58322) 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3d77fe640 1 -- 172.21.15.5:0/1033964110 learned_addr learned my addr 172.21.15.5:0/1033964110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.992+0000 7ff3d7fff640 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedd4d40 0x561aeedd2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.993+0000 7ff3d77fe640 1 -- 172.21.15.5:0/1033964110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedd4d40 msgr2=0x561aeedd2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.993+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedd4d40 0x561aeedd2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.993+0000 7ff3d77fe640 1 -- 172.21.15.5:0/1033964110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedca480 msgr2=0x561aeedd2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.993+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedca480 0x561aeedd2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.993+0000 7ff3d77fe640 1 -- 172.21.15.5:0/1033964110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aeeaa06a0 con 0x561aeedc9b60 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.993+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 0x561aeedd1ce0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8022cb0 tx=0x7ff3c8022ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f99987a9b6f21c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.993+0000 7ff3b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.993+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/1033964110 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8033060 con 0x561aeedc9b60 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.993+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/1033964110 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7ff3c8022590 con 0x561aeedc9b60 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.993+0000 7ff3e0404740 1 -- 172.21.15.5:0/1033964110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3c8022860 con 0x561aeedc9b60 2024-09-15T22:17:32.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/1033964110 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff3c8005040 con 0x561aeedc9b60 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3e0404740 10 client.26485 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3e0404740 1 -- 172.21.15.5:0/1033964110 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561aeede35d0 con 0x561aeedc9b60 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/1033964110 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c8022860 con 0x561aeedc9b60 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/1033964110 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff3c8024480 con 0x561aeedc9b60 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3e0404740 10 client.26485 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3e0404740 10 client.26485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3e0404740 20 client.26485 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3e0404740 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561aeede49e0 0x561aeede6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3e0404740 4 client.26485 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3e0404740 1 -- 172.21.15.5:0/1033964110 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aeed3c7f0 con 0x561aeede49e0 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.994+0000 7ff3d6ffd640 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561aeede49e0 0x561aeede6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:32.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.995+0000 7ff3d6ffd640 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561aeede49e0 0x561aeede6dd0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752fd6bd5ab8cbba server_cookie=60502070d7c49d0e in_seq=0 out_seq=0 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.995+0000 7ff3b6ffd640 10 client.26485 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/1033964110 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561aeed3c7f0 con 0x561aeede49e0 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3b6ffd640 10 client.26485 handle_command_reply: tid=0 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 1 client.26485 shutdown 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 1 -- 172.21.15.5:0/1033964110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 msgr2=0x561aeedd1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 0x561aeedd1ce0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8022cb0 tx=0x7ff3c8022ce0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 1 -- 172.21.15.5:0/1033964110 shutdown_connections 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561aeede49e0 0x561aeede6dd0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedd4d40 0x561aeedd2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedca480 0x561aeedd2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 1 --2- 172.21.15.5:0/1033964110 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 0x561aeedd1ce0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 1 -- 172.21.15.5:0/1033964110 >> 172.21.15.5:0/1033964110 conn(0x561aeedc60f0 msgr2=0x561aeedcbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3b6ffd640 0 client.26485 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:32.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 1 -- 172.21.15.5:0/1033964110 shutdown_connections 2024-09-15T22:17:32.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 1 -- 172.21.15.5:0/1033964110 wait complete. 2024-09-15T22:17:32.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:31.996+0000 7ff3e0404740 20 client.26485 trim_cache size 0 max 16384 2024-09-15T22:17:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3e0404740 1 Processor -- start 2024-09-15T22:17:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3e0404740 1 -- start start 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedc9b60 0x561aeedd3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 0x561aeedec510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedd4d40 0x561aeedeca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3e0404740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d81af840 con 0x561aeedc9b60 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3e0404740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d81ad110 con 0x561aeedd4d40 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3e0404740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d81aaf80 con 0x561aeedca480 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3d6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 0x561aeedec510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedc9b60 0x561aeedd3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3d6ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 0x561aeedec510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58326/0 (socket says 172.21.15.5:58326) 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3d6ffd640 1 -- 172.21.15.5:0/4155949980 learned_addr learned my addr 172.21.15.5:0/4155949980 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/4155949980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedc9b60 0x561aeedd3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56434/0 (socket says 172.21.15.5:56434) 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3d7fff640 1 --2- 172.21.15.5:0/4155949980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedd4d40 0x561aeedeca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3d77fe640 1 -- 172.21.15.5:0/4155949980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 msgr2=0x561aeedec510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/4155949980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 0x561aeedec510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3d77fe640 1 -- 172.21.15.5:0/4155949980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedd4d40 msgr2=0x561aeedeca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.019+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/4155949980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedd4d40 0x561aeedeca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.020+0000 7ff3d77fe640 1 -- 172.21.15.5:0/4155949980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aeedc7280 con 0x561aeedc9b60 2024-09-15T22:17:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.020+0000 7ff3d7fff640 1 --2- 172.21.15.5:0/4155949980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedd4d40 0x561aeedeca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.020+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/4155949980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedc9b60 0x561aeedd3b40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800b500 tx=0x7ff3c800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63fb86949cd19b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.020+0000 7ff3bd7fa640 1 -- 172.21.15.5:0/4155949980 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c803d080 con 0x561aeedc9b60 2024-09-15T22:17:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.020+0000 7ff3bd7fa640 1 -- 172.21.15.5:0/4155949980 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c8024bf0 con 0x561aeedc9b60 2024-09-15T22:17:32.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.020+0000 7ff3bd7fa640 1 -- 172.21.15.5:0/4155949980 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8005440 con 0x561aeedc9b60 2024-09-15T22:17:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.021+0000 7ff3e0404740 1 -- 172.21.15.5:0/4155949980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedc9b60 msgr2=0x561aeedd3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.021+0000 7ff3e0404740 1 --2- 172.21.15.5:0/4155949980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedc9b60 0x561aeedd3b40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff3c800b500 tx=0x7ff3c800b6c0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.021+0000 7ff3e0404740 1 -- 172.21.15.5:0/4155949980 shutdown_connections 2024-09-15T22:17:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.021+0000 7ff3e0404740 1 --2- 172.21.15.5:0/4155949980 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedd4d40 0x561aeedeca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.021+0000 7ff3e0404740 1 --2- 172.21.15.5:0/4155949980 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedca480 0x561aeedec510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.021+0000 7ff3e0404740 1 --2- 172.21.15.5:0/4155949980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedc9b60 0x561aeedd3b40 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.021+0000 7ff3e0404740 1 -- 172.21.15.5:0/4155949980 >> 172.21.15.5:0/4155949980 conn(0x561aeedc60f0 msgr2=0x561aeedd3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.021+0000 7ff3e0404740 1 -- 172.21.15.5:0/4155949980 shutdown_connections 2024-09-15T22:17:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.022+0000 7ff3e0404740 1 -- 172.21.15.5:0/4155949980 wait complete. 2024-09-15T22:17:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.022+0000 7ff3e0404740 1 -- start start 2024-09-15T22:17:32.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.023+0000 7ff3e0404740 1 Processor -- start 2024-09-15T22:17:32.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.023+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 0x561aeeddd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:32.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.023+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedca480 0x561aeedd8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.023+0000 7ff3e0404740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedd4d40 0x561aeedd8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.023+0000 7ff3e0404740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d81ab5d0 con 0x561aeedd4d40 2024-09-15T22:17:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.023+0000 7ff3e0404740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d81aca40 con 0x561aeedca480 2024-09-15T22:17:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.023+0000 7ff3e0404740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d81abca0 con 0x561aeedc9b60 2024-09-15T22:17:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.023+0000 7ff3e0404740 10 client.0 _ll_register_callbacks cb 0x7ff3c800bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3d77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 0x561aeeddd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedd4d40 0x561aeedd8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3d77fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 0x561aeeddd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58332/0 (socket says 172.21.15.5:58332) 2024-09-15T22:17:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3d77fe640 1 -- 172.21.15.5:0/730332458 learned_addr learned my addr 172.21.15.5:0/730332458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3d7fff640 1 -- 172.21.15.5:0/730332458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 msgr2=0x561aeeddd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:32.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3d7fff640 1 --2- 172.21.15.5:0/730332458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 0x561aeeddd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3d7fff640 1 -- 172.21.15.5:0/730332458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedca480 msgr2=0x561aeedd8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3d7fff640 1 --2- 172.21.15.5:0/730332458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedca480 0x561aeedd8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3d7fff640 1 -- 172.21.15.5:0/730332458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aeedd8eb0 con 0x561aeedd4d40 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3d7fff640 1 --2- 172.21.15.5:0/730332458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedd4d40 0x561aeedd8970 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009e20 tx=0x7ff3d0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f22c5a26b5a4c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/730332458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d002a420 con 0x561aeedd4d40 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/730332458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7ff3d001f420 con 0x561aeedd4d40 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.024+0000 7ff3e0404740 1 -- 172.21.15.5:0/730332458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3c8024bf0 con 0x561aeedd4d40 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/730332458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff3d001f620 con 0x561aeedd4d40 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3e0404740 10 client.17010 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3e0404740 1 -- 172.21.15.5:0/730332458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561aeee4f250 con 0x561aeedd4d40 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/730332458 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c8024bf0 con 0x561aeedd4d40 2024-09-15T22:17:32.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/730332458 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff3d000f040 con 0x561aeedd4d40 2024-09-15T22:17:32.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3e0404740 10 client.17010 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:32.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3e0404740 10 client.17010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:32.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3e0404740 20 client.17010 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:32.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3e0404740 1 --2- 172.21.15.5:0/730332458 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561aeee4f9f0 0x561aeee51e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:32.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3e0404740 4 client.17010 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:17:32.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3e0404740 1 -- 172.21.15.5:0/730332458 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561aeede7300 con 0x561aeee4f9f0 2024-09-15T22:17:32.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.025+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/730332458 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561aeee4f9f0 0x561aeee51e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:32.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.026+0000 7ff3d77fe640 1 --2- 172.21.15.5:0/730332458 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561aeee4f9f0 0x561aeee51e30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2b56415bad42aa server_cookie=d4b41268a17c3660 in_seq=0 out_seq=0 2024-09-15T22:17:32.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.026+0000 7ff3b6ffd640 10 client.17010 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.027+0000 7ff3b6ffd640 1 -- 172.21.15.5:0/730332458 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561aeede7300 con 0x561aeee4f9f0 2024-09-15T22:17:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.027+0000 7ff3b6ffd640 10 client.17010 handle_command_reply: tid=0 2024-09-15T22:17:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.027+0000 7ff3e0404740 1 client.17010 shutdown 2024-09-15T22:17:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.027+0000 7ff3e0404740 1 -- 172.21.15.5:0/730332458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedd4d40 msgr2=0x561aeedd8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.027+0000 7ff3e0404740 1 --2- 172.21.15.5:0/730332458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedd4d40 0x561aeedd8970 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009e20 tx=0x7ff3d0004ab0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.028+0000 7ff3e0404740 1 -- 172.21.15.5:0/730332458 shutdown_connections 2024-09-15T22:17:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.028+0000 7ff3e0404740 1 --2- 172.21.15.5:0/730332458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561aeedd4d40 0x561aeedd8970 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.028+0000 7ff3e0404740 1 --2- 172.21.15.5:0/730332458 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561aeee4f9f0 0x561aeee51e30 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.028+0000 7ff3e0404740 1 --2- 172.21.15.5:0/730332458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561aeedca480 0x561aeedd8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.028+0000 7ff3e0404740 1 --2- 172.21.15.5:0/730332458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561aeedc9b60 0x561aeeddd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.028+0000 7ff3e0404740 1 -- 172.21.15.5:0/730332458 >> 172.21.15.5:0/730332458 conn(0x561aeedc60f0 msgr2=0x561aeedcbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.028+0000 7ff3b6ffd640 0 client.17010 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.028+0000 7ff3e0404740 1 -- 172.21.15.5:0/730332458 shutdown_connections 2024-09-15T22:17:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.028+0000 7ff3e0404740 1 -- 172.21.15.5:0/730332458 wait complete. 2024-09-15T22:17:32.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.028+0000 7ff3e0404740 20 client.17010 trim_cache size 0 max 16384 2024-09-15T22:17:32.033 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:17:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:17:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "2ea3e27b-3ade-40e3-b51d-4cbd3aba586d", 2024-09-15T22:17:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:17:32.034 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:17:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.028+0000 7ff3de02b640 1 -- 172.21.15.5:0/4012388910 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff3c00a3430 msgr2=0x7ff3c00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.029+0000 7ff3de02b640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff3c00a3430 0x7ff3c00a58f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff3d81077c0 tx=0x7ff3d0009ae0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.029+0000 7ff3de02b640 1 -- 172.21.15.5:0/4012388910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d8100aa0 msgr2=0x7ff3d8196820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.029+0000 7ff3de02b640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d8100aa0 0x7ff3d8196820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc006010 tx=0x7ff3cc00f3c0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.029+0000 7ff3de02b640 1 -- 172.21.15.5:0/4012388910 shutdown_connections 2024-09-15T22:17:32.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.029+0000 7ff3de02b640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff3c00a3430 0x7ff3c00a58f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.029+0000 7ff3de02b640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff3d8106b80 0x7ff3d819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.029+0000 7ff3de02b640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff3d8100aa0 0x7ff3d8196820 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.029+0000 7ff3de02b640 1 --2- 172.21.15.5:0/4012388910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff3d8100100 0x7ff3d81962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:32.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.029+0000 7ff3de02b640 1 -- 172.21.15.5:0/4012388910 >> 172.21.15.5:0/4012388910 conn(0x7ff3d8077230 msgr2=0x7ff3d80fc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:32.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.029+0000 7ff3de02b640 1 -- 172.21.15.5:0/4012388910 shutdown_connections 2024-09-15T22:17:32.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:32.029+0000 7ff3de02b640 1 -- 172.21.15.5:0/4012388910 wait complete. 2024-09-15T22:17:32.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:17:33.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:32 smithi005 ceph-mon[30944]: pgmap v266: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.2 MiB/s wr, 679 op/s 2024-09-15T22:17:33.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:32 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:17:33.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:32 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:33.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:32 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:17:33.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:32 smithi099 ceph-mon[34892]: pgmap v266: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.2 MiB/s wr, 679 op/s 2024-09-15T22:17:33.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:32 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:17:33.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:32 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:33.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:32 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:17:33.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:32 smithi129 ceph-mon[35314]: pgmap v266: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 6.2 MiB/s wr, 679 op/s 2024-09-15T22:17:33.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:32 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:17:33.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:32 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:33.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:32 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:17:33.385 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.951+0000 7f45ef816640 1 -- 172.21.15.5:0/1463222252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e811b140 msgr2=0x7f45e811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.951+0000 7f45ef816640 1 --2- 172.21.15.5:0/1463222252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e811b140 0x7f45e811d530 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f45e0009ca0 tx=0x7f45e002a8f0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.952+0000 7f45ef816640 1 -- 172.21.15.5:0/1463222252 shutdown_connections 2024-09-15T22:17:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.952+0000 7f45ef816640 1 --2- 172.21.15.5:0/1463222252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e811b140 0x7f45e811d530 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.952+0000 7f45ef816640 1 --2- 172.21.15.5:0/1463222252 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45e8109830 0x7f45e8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.952+0000 7f45ef816640 1 --2- 172.21.15.5:0/1463222252 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e8109450 0x7f45e8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.952+0000 7f45ef816640 1 -- 172.21.15.5:0/1463222252 >> 172.21.15.5:0/1463222252 conn(0x7f45e806e140 msgr2=0x7f45e806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.956+0000 7f45ef816640 1 -- 172.21.15.5:0/1463222252 shutdown_connections 2024-09-15T22:17:33.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.957+0000 7f45ef816640 1 -- 172.21.15.5:0/1463222252 wait complete. 2024-09-15T22:17:33.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.957+0000 7f45ef816640 1 Processor -- start 2024-09-15T22:17:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.962+0000 7f45ef816640 1 -- start start 2024-09-15T22:17:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.962+0000 7f45ef816640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e8109450 0x7f45e81a7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.962+0000 7f45ef816640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8109830 0x7f45e81a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.962+0000 7f45ef816640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45e811b140 0x7f45e81abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.962+0000 7f45ef816640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8120e30 con 0x7f45e8109830 2024-09-15T22:17:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.962+0000 7f45ef816640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8120cb0 con 0x7f45e811b140 2024-09-15T22:17:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.962+0000 7f45ef816640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f45e8120fb0 con 0x7f45e8109450 2024-09-15T22:17:33.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.962+0000 7f45ed5b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e8109450 0x7f45e81a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.963+0000 7f45eddb3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45e811b140 0x7f45e81abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.963+0000 7f45ed5b2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e8109450 0x7f45e81a7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:47856/0 (socket says 172.21.15.5:47856) 2024-09-15T22:17:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.963+0000 7f45ed5b2640 1 -- 172.21.15.5:0/3027515536 learned_addr learned my addr 172.21.15.5:0/3027515536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:33.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.964+0000 7f45ecdb1640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8109830 0x7f45e81a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.964+0000 7f45ed5b2640 1 -- 172.21.15.5:0/3027515536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45e811b140 msgr2=0x7f45e81abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.964+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45e811b140 0x7f45e81abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.964+0000 7f45ed5b2640 1 -- 172.21.15.5:0/3027515536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8109830 msgr2=0x7f45e81a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.964+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8109830 0x7f45e81a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.964+0000 7f45ed5b2640 1 -- 172.21.15.5:0/3027515536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e81ac240 con 0x7f45e8109450 2024-09-15T22:17:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.964+0000 7f45ecdb1640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8109830 0x7f45e81a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:17:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.964+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e8109450 0x7f45e81a7290 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f45e400c890 tx=0x7f45e400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.966+0000 7f45de7fc640 1 -- 172.21.15.5:0/3027515536 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e4004220 con 0x7f45e8109450 2024-09-15T22:17:33.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.966+0000 7f45de7fc640 1 -- 172.21.15.5:0/3027515536 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e40043c0 con 0x7f45e8109450 2024-09-15T22:17:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.966+0000 7f45de7fc640 1 -- 172.21.15.5:0/3027515536 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40049e0 con 0x7f45e8109450 2024-09-15T22:17:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.968+0000 7f45ef816640 1 -- 172.21.15.5:0/3027515536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45e81ac530 con 0x7f45e8109450 2024-09-15T22:17:33.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.968+0000 7f45ef816640 1 -- 172.21.15.5:0/3027515536 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45e81b3e10 con 0x7f45e8109450 2024-09-15T22:17:33.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.973+0000 7f45f1bef740 1 Processor -- start 2024-09-15T22:17:33.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.974+0000 7f45de7fc640 1 -- 172.21.15.5:0/3027515536 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f45e4004c40 con 0x7f45e8109450 2024-09-15T22:17:33.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.974+0000 7f45f1bef740 1 -- start start 2024-09-15T22:17:33.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.974+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e14a0 0x556fc78e1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:33.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.974+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e1dc0 0x556fc78ec140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:33.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.974+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78ec680 0x556fc78eea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:33.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.974+0000 7f45f1bef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556fc78472e0 con 0x556fc78e1dc0 2024-09-15T22:17:33.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.974+0000 7f45f1bef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556fc77eece0 con 0x556fc78ec680 2024-09-15T22:17:33.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.974+0000 7f45f1bef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556fc78432a0 con 0x556fc78e14a0 2024-09-15T22:17:33.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.975+0000 7f45de7fc640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f45d40a3540 0x7f45d40a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.975+0000 7f45de7fc640 1 -- 172.21.15.5:0/3027515536 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f45e40d02c0 con 0x7f45e8109450 2024-09-15T22:17:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.976+0000 7f45ed5b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e1dc0 0x556fc78ec140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.976+0000 7f45ed5b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e1dc0 0x556fc78ec140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50278/0 (socket says 172.21.15.5:50278) 2024-09-15T22:17:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.976+0000 7f45ed5b2640 1 -- 172.21.15.5:0/1318342902 learned_addr learned my addr 172.21.15.5:0/1318342902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.976+0000 7f45ed5b2640 1 -- 172.21.15.5:0/1318342902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e14a0 msgr2=0x556fc78e1880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:17:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.976+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/1318342902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e14a0 0x556fc78e1880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.976+0000 7f45ed5b2640 1 -- 172.21.15.5:0/1318342902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78ec680 msgr2=0x556fc78eea70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:17:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.976+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/1318342902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78ec680 0x556fc78eea70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.976+0000 7f45ed5b2640 1 -- 172.21.15.5:0/1318342902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fc77d3440 con 0x556fc78e1dc0 2024-09-15T22:17:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.977+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/1318342902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e1dc0 0x556fc78ec140 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f45e40a5740 tx=0x7f45e400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57615a0413357162 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:33.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.978+0000 7f45eddb3640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f45d40a3540 0x7f45d40a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:33.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.979+0000 7f45eddb3640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f45d40a3540 0x7f45d40a5a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45e0005fe0 tx=0x7f45e002f340 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:33.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.981+0000 7f45d37fe640 1 -- 172.21.15.5:0/1318342902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e401d040 con 0x556fc78e1dc0 2024-09-15T22:17:33.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.981+0000 7f45d37fe640 1 -- 172.21.15.5:0/1318342902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e40bd0c0 con 0x556fc78e1dc0 2024-09-15T22:17:33.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.981+0000 7f45d37fe640 1 -- 172.21.15.5:0/1318342902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e40a5a40 con 0x556fc78e1dc0 2024-09-15T22:17:33.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.982+0000 7f45f1bef740 1 -- 172.21.15.5:0/1318342902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e1dc0 msgr2=0x556fc78ec140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:33.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.982+0000 7f45f1bef740 1 --2- 172.21.15.5:0/1318342902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e1dc0 0x556fc78ec140 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f45e40a5740 tx=0x7f45e400f000 comp rx=0 tx=0).stop 2024-09-15T22:17:33.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.988+0000 7f45f1bef740 1 -- 172.21.15.5:0/1318342902 shutdown_connections 2024-09-15T22:17:33.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.988+0000 7f45f1bef740 1 --2- 172.21.15.5:0/1318342902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78ec680 0x556fc78eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.988+0000 7f45f1bef740 1 --2- 172.21.15.5:0/1318342902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e1dc0 0x556fc78ec140 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.988+0000 7f45f1bef740 1 --2- 172.21.15.5:0/1318342902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e14a0 0x556fc78e1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:33.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.988+0000 7f45f1bef740 1 -- 172.21.15.5:0/1318342902 >> 172.21.15.5:0/1318342902 conn(0x556fc78dda30 msgr2=0x556fc78dde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:34.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.994+0000 7f45f1bef740 1 -- 172.21.15.5:0/1318342902 shutdown_connections 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.994+0000 7f45f1bef740 1 -- 172.21.15.5:0/1318342902 wait complete. 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.994+0000 7f45f1bef740 1 -- start start 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.995+0000 7f45f1bef740 1 Processor -- start 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.995+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 0x556fc78ec020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.995+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78e1dc0 0x556fc78e92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.995+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78ec680 0x556fc78e9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.995+0000 7f45f1bef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556fc78ef250 con 0x556fc78e14a0 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.995+0000 7f45f1bef740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556fc7802050 con 0x556fc78e1dc0 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.995+0000 7f45f1bef740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556fc78ef3d0 con 0x556fc78ec680 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.996+0000 7f45f1bef740 10 client.0 _ll_register_callbacks cb 0x556fc7842160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.996+0000 7f45ed5b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 0x556fc78ec020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.996+0000 7f45ed5b2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 0x556fc78ec020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50288/0 (socket says 172.21.15.5:50288) 2024-09-15T22:17:34.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.996+0000 7f45ed5b2640 1 -- 172.21.15.5:0/1866106552 learned_addr learned my addr 172.21.15.5:0/1866106552 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:34.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.996+0000 7f45ed5b2640 1 -- 172.21.15.5:0/1866106552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78ec680 msgr2=0x556fc78e9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.996+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/1866106552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78ec680 0x556fc78e9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.996+0000 7f45ed5b2640 1 -- 172.21.15.5:0/1866106552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78e1dc0 msgr2=0x556fc78e92d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:17:34.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.996+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/1866106552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78e1dc0 0x556fc78e92d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.996+0000 7f45ed5b2640 1 -- 172.21.15.5:0/1866106552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fc75b83c0 con 0x556fc78e14a0 2024-09-15T22:17:34.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.996+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/1866106552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 0x556fc78ec020 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f45e4000c00 tx=0x7f45e409d180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5714b68cc58fc99 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:34.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.999+0000 7f45d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:17:34.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.999+0000 7f45d0ff9640 1 -- 172.21.15.5:0/1866106552 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e401d070 con 0x556fc78e14a0 2024-09-15T22:17:34.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.999+0000 7f45d0ff9640 1 -- 172.21.15.5:0/1866106552 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f45e40cda00 con 0x556fc78e14a0 2024-09-15T22:17:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:33.999+0000 7f45f1bef740 1 -- 172.21.15.5:0/1866106552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e40bd0c0 con 0x556fc78e14a0 2024-09-15T22:17:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.006+0000 7f45d0ff9640 1 -- 172.21.15.5:0/1866106552 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45e4097d80 con 0x556fc78e14a0 2024-09-15T22:17:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.006+0000 7f45f1bef740 10 client.17022 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:34.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.006+0000 7f45f1bef740 1 -- 172.21.15.5:0/1866106552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556fc78facc0 con 0x556fc78e14a0 2024-09-15T22:17:34.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.010+0000 7f45d0ff9640 1 -- 172.21.15.5:0/1866106552 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e40bd0c0 con 0x556fc78e14a0 2024-09-15T22:17:34.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.010+0000 7f45d0ff9640 1 -- 172.21.15.5:0/1866106552 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f45e40b9060 con 0x556fc78e14a0 2024-09-15T22:17:34.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.010+0000 7f45f1bef740 10 client.17022 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:34.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.010+0000 7f45f1bef740 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:34.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.010+0000 7f45f1bef740 20 client.17022 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:34.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.010+0000 7f45f1bef740 1 --2- 172.21.15.5:0/1866106552 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556fc78fc010 0x556fc78fe400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.010+0000 7f45f1bef740 4 client.17022 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:34.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.010+0000 7f45f1bef740 1 -- 172.21.15.5:0/1866106552 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fc7854850 con 0x556fc78fc010 2024-09-15T22:17:34.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.010+0000 7f45ecdb1640 1 --2- 172.21.15.5:0/1866106552 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556fc78fc010 0x556fc78fe400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.011+0000 7f45ecdb1640 1 --2- 172.21.15.5:0/1866106552 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556fc78fc010 0x556fc78fe400 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f174c9b70efebdb5 server_cookie=a603f74c84b45918 in_seq=0 out_seq=0 2024-09-15T22:17:34.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.014+0000 7f45d0ff9640 10 client.17022 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:34.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.014+0000 7f45d0ff9640 1 -- 172.21.15.5:0/1866106552 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fc7854850 con 0x556fc78fc010 2024-09-15T22:17:34.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.014+0000 7f45d0ff9640 10 client.17022 handle_command_reply: tid=0 2024-09-15T22:17:34.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.014+0000 7f45f1bef740 1 client.17022 shutdown 2024-09-15T22:17:34.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.019+0000 7f45f1bef740 1 -- 172.21.15.5:0/1866106552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 msgr2=0x556fc78ec020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:34.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.019+0000 7f45f1bef740 1 --2- 172.21.15.5:0/1866106552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 0x556fc78ec020 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f45e4000c00 tx=0x7f45e409d180 comp rx=0 tx=0).stop 2024-09-15T22:17:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.025+0000 7f45f1bef740 1 -- 172.21.15.5:0/1866106552 shutdown_connections 2024-09-15T22:17:34.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.025+0000 7f45f1bef740 1 --2- 172.21.15.5:0/1866106552 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78ec680 0x556fc78e9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.025+0000 7f45f1bef740 1 --2- 172.21.15.5:0/1866106552 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556fc78fc010 0x556fc78fe400 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.025+0000 7f45f1bef740 1 --2- 172.21.15.5:0/1866106552 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78e1dc0 0x556fc78e92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.025+0000 7f45f1bef740 1 --2- 172.21.15.5:0/1866106552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 0x556fc78ec020 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.025+0000 7f45f1bef740 1 -- 172.21.15.5:0/1866106552 >> 172.21.15.5:0/1866106552 conn(0x556fc78dda30 msgr2=0x556fc78dcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:34.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.026+0000 7f45d0ff9640 0 client.17022 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:34.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.028+0000 7f45f1bef740 1 -- 172.21.15.5:0/1866106552 shutdown_connections 2024-09-15T22:17:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.031+0000 7f45f1bef740 1 -- 172.21.15.5:0/1866106552 wait complete. 2024-09-15T22:17:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.031+0000 7f45f1bef740 20 client.17022 trim_cache size 0 max 16384 2024-09-15T22:17:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.067+0000 7f45f1bef740 1 Processor -- start 2024-09-15T22:17:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.067+0000 7f45f1bef740 1 -- start start 2024-09-15T22:17:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.067+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 0x556fc7903ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.067+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e1dc0 0x556fc7901040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.067+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78ec680 0x556fc7901580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.068+0000 7f45ecdb1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e1dc0 0x556fc7901040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.068+0000 7f45ecdb1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e1dc0 0x556fc7901040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:47866/0 (socket says 172.21.15.5:47866) 2024-09-15T22:17:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.068+0000 7f45ecdb1640 1 -- 172.21.15.5:0/3690702359 learned_addr learned my addr 172.21.15.5:0/3690702359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.068+0000 7f45eddb3640 1 --2- 172.21.15.5:0/3690702359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78ec680 0x556fc7901580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.068+0000 7f45f1bef740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc0129f0 con 0x556fc78e14a0 2024-09-15T22:17:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.068+0000 7f45f1bef740 1 -- 172.21.15.5:0/3690702359 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc0102c0 con 0x556fc78ec680 2024-09-15T22:17:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.068+0000 7f45f1bef740 1 -- 172.21.15.5:0/3690702359 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc00e130 con 0x556fc78e1dc0 2024-09-15T22:17:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.068+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/3690702359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 0x556fc7903ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.069+0000 7f45ed5b2640 1 -- 172.21.15.5:0/3690702359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e1dc0 msgr2=0x556fc7901040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.069+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/3690702359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e1dc0 0x556fc7901040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.074 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:33 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.069+0000 7f45ed5b2640 1 -- 172.21.15.5:0/3690702359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78ec680 msgr2=0x556fc7901580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.069+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/3690702359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78ec680 0x556fc7901580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.069+0000 7f45ed5b2640 1 -- 172.21.15.5:0/3690702359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fc78e8280 con 0x556fc78e14a0 2024-09-15T22:17:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.070+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/3690702359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 0x556fc7903ed0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f45e4099980 tx=0x7f45e4004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8181f656f0a1c8e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.070+0000 7f45d2ffd640 1 -- 172.21.15.5:0/3690702359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e401d050 con 0x556fc78e14a0 2024-09-15T22:17:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.070+0000 7f45d2ffd640 1 -- 172.21.15.5:0/3690702359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45e40a2770 con 0x556fc78e14a0 2024-09-15T22:17:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.070+0000 7f45d2ffd640 1 -- 172.21.15.5:0/3690702359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e409c450 con 0x556fc78e14a0 2024-09-15T22:17:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.071+0000 7f45f1bef740 1 -- 172.21.15.5:0/3690702359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 msgr2=0x556fc7903ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.071+0000 7f45f1bef740 1 --2- 172.21.15.5:0/3690702359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 0x556fc7903ed0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f45e4099980 tx=0x7f45e4004570 comp rx=0 tx=0).stop 2024-09-15T22:17:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.072+0000 7f45f1bef740 1 -- 172.21.15.5:0/3690702359 shutdown_connections 2024-09-15T22:17:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.072+0000 7f45f1bef740 1 --2- 172.21.15.5:0/3690702359 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78ec680 0x556fc7901580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.072+0000 7f45f1bef740 1 --2- 172.21.15.5:0/3690702359 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e1dc0 0x556fc7901040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.072+0000 7f45f1bef740 1 --2- 172.21.15.5:0/3690702359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78e14a0 0x556fc7903ed0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.073+0000 7f45f1bef740 1 -- 172.21.15.5:0/3690702359 >> 172.21.15.5:0/3690702359 conn(0x556fc78dda30 msgr2=0x556fc78e36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:34.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.073+0000 7f45f1bef740 1 -- 172.21.15.5:0/3690702359 shutdown_connections 2024-09-15T22:17:34.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.073+0000 7f45f1bef740 1 -- 172.21.15.5:0/3690702359 wait complete. 2024-09-15T22:17:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.073+0000 7f45f1bef740 1 -- start start 2024-09-15T22:17:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.074+0000 7f45f1bef740 1 Processor -- start 2024-09-15T22:17:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.075+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e14a0 0x556fc7901670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.075+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78e1dc0 0x556fc78fe9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.075+0000 7f45f1bef740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78ec680 0x556fc78fef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.075+0000 7f45ecdb1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78e1dc0 0x556fc78fe9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.075+0000 7f45ecdb1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78e1dc0 0x556fc78fe9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:45248/0 (socket says 172.21.15.5:45248) 2024-09-15T22:17:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.075+0000 7f45ecdb1640 1 -- 172.21.15.5:0/2165692241 learned_addr learned my addr 172.21.15.5:0/2165692241 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.075+0000 7f45f1bef740 1 -- 172.21.15.5:0/2165692241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc00e780 con 0x556fc78ec680 2024-09-15T22:17:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.075+0000 7f45f1bef740 1 -- 172.21.15.5:0/2165692241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc00fbf0 con 0x556fc78e1dc0 2024-09-15T22:17:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.075+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e14a0 0x556fc7901670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.075+0000 7f45f1bef740 1 -- 172.21.15.5:0/2165692241 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f45cc00ee50 con 0x556fc78e14a0 2024-09-15T22:17:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.076+0000 7f45f1bef740 10 client.0 _ll_register_callbacks cb 0x7f45e409d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.076+0000 7f45ecdb1640 1 -- 172.21.15.5:0/2165692241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e14a0 msgr2=0x556fc7901670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.076+0000 7f45ecdb1640 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e14a0 0x556fc7901670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.076+0000 7f45ecdb1640 1 -- 172.21.15.5:0/2165692241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78ec680 msgr2=0x556fc78fef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.076+0000 7f45ecdb1640 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78ec680 0x556fc78fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.076+0000 7f45ecdb1640 1 -- 172.21.15.5:0/2165692241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fc78ff5f0 con 0x556fc78e1dc0 2024-09-15T22:17:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.077+0000 7f45f1bef740 1 -- 172.21.15.5:0/2165692241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45e40a2770 con 0x556fc78e1dc0 2024-09-15T22:17:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.077+0000 7f45ecdb1640 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78e1dc0 0x556fc78fe9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x556fc78dedb0 tx=0x7f45d8009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=578d1c3d043ec109 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.077+0000 7f45d0ff9640 10 client.26549 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:17:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.078+0000 7f45d0ff9640 1 -- 172.21.15.5:0/2165692241 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d800f040 con 0x556fc78e1dc0 2024-09-15T22:17:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.078+0000 7f45d0ff9640 1 -- 172.21.15.5:0/2165692241 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f45d8016c80 con 0x556fc78e1dc0 2024-09-15T22:17:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.078+0000 7f45d0ff9640 1 -- 172.21.15.5:0/2165692241 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45d8005970 con 0x556fc78e1dc0 2024-09-15T22:17:34.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.078+0000 7f45f1bef740 10 client.26549 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:34.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.078+0000 7f45f1bef740 1 -- 172.21.15.5:0/2165692241 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=25}) -- 0x556fc7966730 con 0x556fc78e1dc0 2024-09-15T22:17:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.079+0000 7f45d0ff9640 1 -- 172.21.15.5:0/2165692241 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f45d801e470 con 0x556fc78e1dc0 2024-09-15T22:17:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.079+0000 7f45f1bef740 10 client.26549 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.079+0000 7f45f1bef740 10 client.26549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:17:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.079+0000 7f45f1bef740 20 client.26549 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.079+0000 7f45f1bef740 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556fc7966d20 0x556fc7969140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.079+0000 7f45f1bef740 4 client.26549 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:17:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.079+0000 7f45f1bef740 1 -- 172.21.15.5:0/2165692241 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556fc784ed60 con 0x556fc7966d20 2024-09-15T22:17:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.079+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556fc7966d20 0x556fc7969140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:34.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.081+0000 7f45ed5b2640 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556fc7966d20 0x556fc7969140 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4308a56dcd282582 server_cookie=1b836c93348d3b03 in_seq=0 out_seq=0 2024-09-15T22:17:34.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.081+0000 7f45d0ff9640 10 client.26549 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:34.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.081+0000 7f45d0ff9640 1 -- 172.21.15.5:0/2165692241 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556fc784ed60 con 0x556fc7966d20 2024-09-15T22:17:34.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.081+0000 7f45d0ff9640 10 client.26549 handle_command_reply: tid=0 2024-09-15T22:17:34.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.081+0000 7f45f1bef740 1 client.26549 shutdown 2024-09-15T22:17:34.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.085+0000 7f45f1bef740 1 -- 172.21.15.5:0/2165692241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78e1dc0 msgr2=0x556fc78fe9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:34.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.085+0000 7f45f1bef740 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78e1dc0 0x556fc78fe9c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x556fc78dedb0 tx=0x7f45d8009f10 comp rx=0 tx=0).stop 2024-09-15T22:17:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.087+0000 7f45f1bef740 1 -- 172.21.15.5:0/2165692241 shutdown_connections 2024-09-15T22:17:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.087+0000 7f45f1bef740 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fc78ec680 0x556fc78fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.087+0000 7f45f1bef740 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556fc7966d20 0x556fc7969140 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.087+0000 7f45d0ff9640 0 client.26549 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.087+0000 7f45f1bef740 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556fc78e1dc0 0x556fc78fe9c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.087+0000 7f45f1bef740 1 --2- 172.21.15.5:0/2165692241 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556fc78e14a0 0x556fc7901670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.087+0000 7f45f1bef740 1 -- 172.21.15.5:0/2165692241 >> 172.21.15.5:0/2165692241 conn(0x556fc78dda30 msgr2=0x556fc78dc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.087+0000 7f45f1bef740 1 -- 172.21.15.5:0/2165692241 shutdown_connections 2024-09-15T22:17:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.087+0000 7f45f1bef740 1 -- 172.21.15.5:0/2165692241 wait complete. 2024-09-15T22:17:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.088+0000 7f45f1bef740 20 client.26549 trim_cache size 0 max 16384 2024-09-15T22:17:34.091 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:17:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:17:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:17:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "2ea3e27b-3ade-40e3-b51d-4cbd3aba586d": { 2024-09-15T22:17:34.091 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:17:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "2ea3e27b-3ade-40e3-b51d-4cbd3aba586d", 2024-09-15T22:17:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:17:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:17:34.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:17:34.092 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:17:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.088+0000 7f45ef816640 1 -- 172.21.15.5:0/3027515536 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f45d40a3540 msgr2=0x7f45d40a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.088+0000 7f45ef816640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f45d40a3540 0x7f45d40a5a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f45e0005fe0 tx=0x7f45e002f340 comp rx=0 tx=0).stop 2024-09-15T22:17:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.088+0000 7f45ef816640 1 -- 172.21.15.5:0/3027515536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e8109450 msgr2=0x7f45e81a7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.088+0000 7f45ef816640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e8109450 0x7f45e81a7290 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f45e400c890 tx=0x7f45e400cd50 comp rx=0 tx=0).stop 2024-09-15T22:17:34.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.089+0000 7f45ef816640 1 -- 172.21.15.5:0/3027515536 shutdown_connections 2024-09-15T22:17:34.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.089+0000 7f45ef816640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f45d40a3540 0x7f45d40a5a00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.089+0000 7f45ef816640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f45e811b140 0x7f45e81abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.089+0000 7f45ef816640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45e8109830 0x7f45e81a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.089+0000 7f45ef816640 1 --2- 172.21.15.5:0/3027515536 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f45e8109450 0x7f45e81a7290 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:34.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.089+0000 7f45ef816640 1 -- 172.21.15.5:0/3027515536 >> 172.21.15.5:0/3027515536 conn(0x7f45e806e140 msgr2=0x7f45e811bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:34.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.089+0000 7f45ef816640 1 -- 172.21.15.5:0/3027515536 shutdown_connections 2024-09-15T22:17:34.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:34.089+0000 7f45ef816640 1 -- 172.21.15.5:0/3027515536 wait complete. 2024-09-15T22:17:34.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:33 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:34.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:33 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:17:34.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 485.0 s, 665.0 tps, lat 151.749 ms stddev 197.683 2024-09-15T22:17:34.819 INFO:tasks.cephfs.filesystem:scrub status for tag:2ea3e27b-3ade-40e3-b51d-4cbd3aba586d - {'path': '/', 'tag': '2ea3e27b-3ade-40e3-b51d-4cbd3aba586d', 'options': 'recursive,force'} 2024-09-15T22:17:35.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:34 smithi099 ceph-mon[34892]: pgmap v267: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-09-15T22:17:35.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:34 smithi129 ceph-mon[35314]: pgmap v267: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-09-15T22:17:35.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:34 smithi005 ceph-mon[30944]: pgmap v267: 129 pgs: 129 active+clean; 3.1 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 4.7 MiB/s wr, 507 op/s 2024-09-15T22:17:37.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:36 smithi099 ceph-mon[34892]: pgmap v268: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 6.1 MiB/s wr, 659 op/s 2024-09-15T22:17:37.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:36 smithi129 ceph-mon[35314]: pgmap v268: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 6.1 MiB/s wr, 659 op/s 2024-09-15T22:17:37.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:36 smithi005 ceph-mon[30944]: pgmap v268: 129 pgs: 129 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 6.1 MiB/s wr, 659 op/s 2024-09-15T22:17:38.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:37 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:17:38.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:37 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:17:38.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:37 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:17:39.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:38 smithi099 ceph-mon[34892]: pgmap v269: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-09-15T22:17:39.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:38 smithi129 ceph-mon[35314]: pgmap v269: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-09-15T22:17:39.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:38 smithi005 ceph-mon[30944]: pgmap v269: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.4 MiB/s wr, 473 op/s 2024-09-15T22:17:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 490.0 s, 689.8 tps, lat 147.948 ms stddev 175.227 2024-09-15T22:17:41.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:40 smithi099 ceph-mon[34892]: pgmap v270: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-09-15T22:17:41.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:40 smithi129 ceph-mon[35314]: pgmap v270: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-09-15T22:17:41.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:40 smithi005 ceph-mon[30944]: pgmap v270: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 4.6 MiB/s wr, 491 op/s 2024-09-15T22:17:42.041 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:17:42.042 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.35 2024-09-15T22:17:42.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:17:42.572 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:43.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:42 smithi099 ceph-mon[34892]: pgmap v271: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.3 MiB/s wr, 718 op/s 2024-09-15T22:17:43.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:42 smithi129 ceph-mon[35314]: pgmap v271: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.3 MiB/s wr, 718 op/s 2024-09-15T22:17:43.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:42 smithi005 ceph-mon[30944]: pgmap v271: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.3 MiB/s wr, 718 op/s 2024-09-15T22:17:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 495.0 s, 619.8 tps, lat 150.298 ms stddev 174.897 2024-09-15T22:17:45.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:44 smithi099 ceph-mon[34892]: pgmap v272: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 588 op/s 2024-09-15T22:17:45.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:44 smithi129 ceph-mon[35314]: pgmap v272: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 588 op/s 2024-09-15T22:17:45.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:44 smithi005 ceph-mon[30944]: pgmap v272: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 5.5 MiB/s wr, 588 op/s 2024-09-15T22:17:47.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:46 smithi099 ceph-mon[34892]: pgmap v273: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 733 op/s 2024-09-15T22:17:47.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:46 smithi129 ceph-mon[35314]: pgmap v273: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 733 op/s 2024-09-15T22:17:47.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:46 smithi005 ceph-mon[30944]: pgmap v273: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 6.7 MiB/s wr, 733 op/s 2024-09-15T22:17:47.841 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:17:47.841 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:17:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.035+0000 7f8f8f7fe640 1 -- 172.21.15.5:0/3094692952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f80031940 con 0x7f8f90100e20 2024-09-15T22:17:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.036+0000 7f8f96a88640 1 -- 172.21.15.5:0/3094692952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f90100e20 msgr2=0x7f8f90101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.036+0000 7f8f96a88640 1 --2- 172.21.15.5:0/3094692952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f90100e20 0x7f8f90101200 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009900 tx=0x7f8f8002a480 comp rx=0 tx=0).stop 2024-09-15T22:17:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.036+0000 7f8f96a88640 1 -- 172.21.15.5:0/3094692952 shutdown_connections 2024-09-15T22:17:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.036+0000 7f8f96a88640 1 --2- 172.21.15.5:0/3094692952 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f90102ef0 0x7f8f9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.036+0000 7f8f96a88640 1 --2- 172.21.15.5:0/3094692952 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f90102570 0x7f8f901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.036+0000 7f8f96a88640 1 --2- 172.21.15.5:0/3094692952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f90100e20 0x7f8f90101200 secure :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8f80009900 tx=0x7f8f8002a480 comp rx=0 tx=0).stop 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.036+0000 7f8f96a88640 1 -- 172.21.15.5:0/3094692952 >> 172.21.15.5:0/3094692952 conn(0x7f8f900fc7d0 msgr2=0x7f8f900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.040+0000 7f8f96a88640 1 -- 172.21.15.5:0/3094692952 shutdown_connections 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.040+0000 7f8f96a88640 1 -- 172.21.15.5:0/3094692952 wait complete. 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.041+0000 7f8f96a88640 1 Processor -- start 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.041+0000 7f8f96a88640 1 -- start start 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.042+0000 7f8f96a88640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f90102570 0x7f8f901164d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.042+0000 7f8f96a88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f90102ef0 0x7f8f90116a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.042+0000 7f8f96a88640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f901115f0 0x7f8f90111aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.042+0000 7f8f8ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f90102ef0 0x7f8f90116a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.042+0000 7f8f8ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f90102ef0 0x7f8f90116a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47096/0 (socket says 172.21.15.5:47096) 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.042+0000 7f8f8ffff640 1 -- 172.21.15.5:0/1721039014 learned_addr learned my addr 172.21.15.5:0/1721039014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:47.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.042+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9010fe70 con 0x7f8f90102ef0 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.042+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9010fcf0 con 0x7f8f901115f0 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.042+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9010fff0 con 0x7f8f90102570 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.044+0000 7f8f95025640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f901115f0 0x7f8f90111aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.044+0000 7f8f8ffff640 1 -- 172.21.15.5:0/1721039014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f90102570 msgr2=0x7f8f901164d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.044+0000 7f8f8ffff640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f90102570 0x7f8f901164d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.044+0000 7f8f8ffff640 1 -- 172.21.15.5:0/1721039014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f901115f0 msgr2=0x7f8f90111aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.044+0000 7f8f8ffff640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f901115f0 0x7f8f90111aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.044+0000 7f8f8ffff640 1 -- 172.21.15.5:0/1721039014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f90112360 con 0x7f8f90102ef0 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.047+0000 7f8f8ffff640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f90102ef0 0x7f8f90116a10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800eab0 tx=0x7f8f7800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.050+0000 7f8f8dffb640 1 -- 172.21.15.5:0/1721039014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7800ce40 con 0x7f8f90102ef0 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.050+0000 7f8f8dffb640 1 -- 172.21.15.5:0/1721039014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f78011950 con 0x7f8f90102ef0 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.050+0000 7f8f8dffb640 1 -- 172.21.15.5:0/1721039014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7801aad0 con 0x7f8f90102ef0 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.050+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9011c1f0 con 0x7f8f90102ef0 2024-09-15T22:17:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.050+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9011c6e0 con 0x7f8f90102ef0 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.052+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f70005290 con 0x7f8f90102ef0 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.055+0000 7f8f8dffb640 1 -- 172.21.15.5:0/1721039014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8f7801acb0 con 0x7f8f90102ef0 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.057+0000 7f8f8dffb640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f7c0a3580 0x7f8f7c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.057+0000 7f8f8dffb640 1 -- 172.21.15.5:0/1721039014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f8f780d09b0 con 0x7f8f90102ef0 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.061+0000 7f8f94824640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f7c0a3580 0x7f8f7c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.066+0000 7f8f94824640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f7c0a3580 0x7f8f7c0a5a40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000c00 tx=0x7f8f800095e0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.067+0000 7f8f8dffb640 1 -- 172.21.15.5:0/1721039014 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f7801f060 con 0x7f8f90102ef0 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.489+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f70005850 con 0x7f8f90102ef0 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.490+0000 7f8f8dffb640 1 -- 172.21.15.5:0/1721039014 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f8f78016030 con 0x7f8f90102ef0 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f7c0a3580 msgr2=0x7f8f7c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f7c0a3580 0x7f8f7c0a5a40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8f80000c00 tx=0x7f8f800095e0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f90102ef0 msgr2=0x7f8f90116a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:47.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f90102ef0 0x7f8f90116a10 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800eab0 tx=0x7f8f7800ef70 comp rx=0 tx=0).stop 2024-09-15T22:17:47.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 shutdown_connections 2024-09-15T22:17:47.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8f7c0a3580 0x7f8f7c0a5a40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8f901115f0 0x7f8f90111aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f90102ef0 0x7f8f90116a10 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 --2- 172.21.15.5:0/1721039014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8f90102570 0x7f8f901164d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:47.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 >> 172.21.15.5:0/1721039014 conn(0x7f8f900fc7d0 msgr2=0x7f8f900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:47.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 shutdown_connections 2024-09-15T22:17:47.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:47.492+0000 7f8f96a88640 1 -- 172.21.15.5:0/1721039014 wait complete. 2024-09-15T22:17:48.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:47 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1721039014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:48.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:47 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1721039014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:48.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:47 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1721039014' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:17:49.141 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0' 2024-09-15T22:17:49.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0 2024-09-15T22:17:49.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:48 smithi099 ceph-mon[34892]: pgmap v274: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 581 op/s 2024-09-15T22:17:49.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:48 smithi129 ceph-mon[35314]: pgmap v274: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 581 op/s 2024-09-15T22:17:49.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:48 smithi005 ceph-mon[30944]: pgmap v274: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 581 op/s 2024-09-15T22:17:49.625 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:17:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 500.0 s, 631.6 tps, lat 168.933 ms stddev 247.214 2024-09-15T22:17:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.279+0000 7fc9a0be2640 1 -- 172.21.15.5:0/1580226482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99c1082a0 msgr2=0x7fc99c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.279+0000 7fc9a0be2640 1 --2- 172.21.15.5:0/1580226482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99c1082a0 0x7fc99c10ec40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc994009cd0 tx=0x7fc99402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.279+0000 7fc9a0be2640 1 -- 172.21.15.5:0/1580226482 shutdown_connections 2024-09-15T22:17:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.279+0000 7fc9a0be2640 1 --2- 172.21.15.5:0/1580226482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99c1082a0 0x7fc99c10ec40 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.279+0000 7fc9a0be2640 1 --2- 172.21.15.5:0/1580226482 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99c1036e0 0x7fc99c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.279+0000 7fc9a0be2640 1 --2- 172.21.15.5:0/1580226482 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c102d30 0x7fc99c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.279+0000 7fc9a0be2640 1 -- 172.21.15.5:0/1580226482 >> 172.21.15.5:0/1580226482 conn(0x7fc99c0fc7d0 msgr2=0x7fc99c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.280+0000 7fc9a0be2640 1 -- 172.21.15.5:0/1580226482 shutdown_connections 2024-09-15T22:17:50.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.280+0000 7fc9a0be2640 1 -- 172.21.15.5:0/1580226482 wait complete. 2024-09-15T22:17:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc9a0be2640 1 Processor -- start 2024-09-15T22:17:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc9a0be2640 1 -- start start 2024-09-15T22:17:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc9a0be2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99c102d30 0x7fc99c1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc9a0be2640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c1036e0 0x7fc99c1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc9a0be2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99c1082a0 0x7fc99c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc9a0be2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c1120a0 con 0x7fc99c1082a0 2024-09-15T22:17:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc9a0be2640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c111f20 con 0x7fc99c1036e0 2024-09-15T22:17:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc9a0be2640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c112220 con 0x7fc99c102d30 2024-09-15T22:17:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc99ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99c1082a0 0x7fc99c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc99ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99c1082a0 0x7fc99c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47118/0 (socket says 172.21.15.5:47118) 2024-09-15T22:17:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc99a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99c102d30 0x7fc99c1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc99ad9d640 1 -- 172.21.15.5:0/536580334 learned_addr learned my addr 172.21.15.5:0/536580334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.281+0000 7fc999d9b640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c1036e0 0x7fc99c1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.282+0000 7fc99a59c640 1 -- 172.21.15.5:0/536580334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c1036e0 msgr2=0x7fc99c1a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.282+0000 7fc99a59c640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c1036e0 0x7fc99c1a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.282+0000 7fc99a59c640 1 -- 172.21.15.5:0/536580334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99c1082a0 msgr2=0x7fc99c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.282+0000 7fc99a59c640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99c1082a0 0x7fc99c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.282+0000 7fc99a59c640 1 -- 172.21.15.5:0/536580334 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c19acf0 con 0x7fc99c102d30 2024-09-15T22:17:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.282+0000 7fc99a59c640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99c102d30 0x7fc99c1a0400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00e9f0 tx=0x7fc98c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.282+0000 7fc98b7fe640 1 -- 172.21.15.5:0/536580334 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00ccc0 con 0x7fc99c102d30 2024-09-15T22:17:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.282+0000 7fc9a0be2640 1 -- 172.21.15.5:0/536580334 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c19afe0 con 0x7fc99c102d30 2024-09-15T22:17:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.282+0000 7fc98b7fe640 1 -- 172.21.15.5:0/536580334 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc98c010910 con 0x7fc99c102d30 2024-09-15T22:17:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.282+0000 7fc98b7fe640 1 -- 172.21.15.5:0/536580334 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0189c0 con 0x7fc99c102d30 2024-09-15T22:17:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.282+0000 7fc9a0be2640 1 -- 172.21.15.5:0/536580334 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c1a72d0 con 0x7fc99c102d30 2024-09-15T22:17:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.285+0000 7fc98b7fe640 1 -- 172.21.15.5:0/536580334 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc98c018b60 con 0x7fc99c102d30 2024-09-15T22:17:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.286+0000 7fc98b7fe640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc9840a33b0 0x7fc9840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.286+0000 7fc98b7fe640 1 -- 172.21.15.5:0/536580334 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7fc98c0d0470 con 0x7fc99c102d30 2024-09-15T22:17:50.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.287+0000 7fc999d9b640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc9840a33b0 0x7fc9840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.287+0000 7fc9a2fbb740 1 Processor -- start 2024-09-15T22:17:50.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc9a2fbb740 1 -- start start 2024-09-15T22:17:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc9a2fbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 0x5592eda41b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc9a2fbb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda42090 0x5592eda54d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc9a2fbb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 0x5592eda57630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc9a2fbb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5592ed9a6160 con 0x5592eda55240 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc9a2fbb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5592ed94d720 con 0x5592eda41770 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc9a2fbb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5592ed9a2120 con 0x5592eda42090 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc99a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 0x5592eda41b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc99a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 0x5592eda41b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:43726/0 (socket says 172.21.15.5:43726) 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc99a59c640 1 -- 172.21.15.5:0/1660157889 learned_addr learned my addr 172.21.15.5:0/1660157889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/1660157889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 0x5592eda57630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc999d9b640 1 --2- 172.21.15.5:0/1660157889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda42090 0x5592eda54d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc99ad9d640 1 -- 172.21.15.5:0/1660157889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda42090 msgr2=0x5592eda54d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/1660157889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda42090 0x5592eda54d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc99ad9d640 1 -- 172.21.15.5:0/1660157889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 msgr2=0x5592eda41b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/1660157889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 0x5592eda41b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.288+0000 7fc99ad9d640 1 -- 172.21.15.5:0/1660157889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592ed717ba0 con 0x5592eda55240 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.289+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/1660157889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 0x5592eda57630 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc994005e30 tx=0x7fc994005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931ef021dbd53248 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.289+0000 7fc999d9b640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc9840a33b0 0x7fc9840a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc99c10f010 tx=0x7fc990005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.289+0000 7fc988ff9640 1 -- 172.21.15.5:0/1660157889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994048070 con 0x5592eda55240 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.289+0000 7fc988ff9640 1 -- 172.21.15.5:0/1660157889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc994036070 con 0x5592eda55240 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.289+0000 7fc988ff9640 1 -- 172.21.15.5:0/1660157889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994030a40 con 0x5592eda55240 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.290+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/1660157889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 msgr2=0x5592eda57630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.290+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/1660157889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 0x5592eda57630 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc994005e30 tx=0x7fc994005f80 comp rx=0 tx=0).stop 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.290+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/1660157889 shutdown_connections 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.290+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/1660157889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 0x5592eda57630 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.290+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/1660157889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda42090 0x5592eda54d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.290+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/1660157889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 0x5592eda41b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.290+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/1660157889 >> 172.21.15.5:0/1660157889 conn(0x5592eda3dde0 msgr2=0x5592eda3e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.290+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/1660157889 shutdown_connections 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.290+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/1660157889 wait complete. 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc9a2fbb740 1 -- start start 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc9a2fbb740 1 Processor -- start 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc9a2fbb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda41770 0x5592eda52220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc9a2fbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda42090 0x5592eda52760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc9a2fbb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 0x5592eda52ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc9a2fbb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5592eda57d50 con 0x5592eda55240 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc9a2fbb740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5592ed92e900 con 0x5592eda42090 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc9a2fbb740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5592eda57ed0 con 0x5592eda41770 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc9a2fbb740 10 client.0 _ll_register_callbacks cb 0x5592ed92c780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc999d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda42090 0x5592eda52760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc99ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 0x5592eda52ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc999d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda42090 0x5592eda52760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:43730/0 (socket says 172.21.15.5:43730) 2024-09-15T22:17:50.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.291+0000 7fc999d9b640 1 -- 172.21.15.5:0/2793473928 learned_addr learned my addr 172.21.15.5:0/2793473928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.292+0000 7fc99a59c640 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda41770 0x5592eda52220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.292+0000 7fc99ad9d640 1 -- 172.21.15.5:0/2793473928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda41770 msgr2=0x5592eda52220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.292+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda41770 0x5592eda52220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.292+0000 7fc99ad9d640 1 -- 172.21.15.5:0/2793473928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda42090 msgr2=0x5592eda52760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.292+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda42090 0x5592eda52760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.292+0000 7fc99ad9d640 1 -- 172.21.15.5:0/2793473928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592ed96ec50 con 0x5592eda55240 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.292+0000 7fc999d9b640 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda42090 0x5592eda52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.292+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 0x5592eda52ca0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc994009ca0 tx=0x7fc994003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c331bb38e0bfa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.292+0000 7fc981ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.292+0000 7fc981ffb640 1 -- 172.21.15.5:0/2793473928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994048040 con 0x5592eda55240 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.292+0000 7fc981ffb640 1 -- 172.21.15.5:0/2793473928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7fc994031980 con 0x5592eda55240 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.293+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2793473928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc994036070 con 0x5592eda55240 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.293+0000 7fc981ffb640 1 -- 172.21.15.5:0/2793473928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc994031b80 con 0x5592eda55240 2024-09-15T22:17:50.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.293+0000 7fc9a2fbb740 10 client.17043 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.293+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2793473928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592eda57d50 con 0x5592eda55240 2024-09-15T22:17:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.293+0000 7fc981ffb640 1 -- 172.21.15.5:0/2793473928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc994053070 con 0x5592eda55240 2024-09-15T22:17:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.293+0000 7fc981ffb640 1 -- 172.21.15.5:0/2793473928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc994042030 con 0x5592eda55240 2024-09-15T22:17:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.293+0000 7fc9a2fbb740 10 client.17043 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.293+0000 7fc9a2fbb740 10 client.17043 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:17:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.293+0000 7fc9a2fbb740 20 client.17043 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.293+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5592eda64d30 0x5592eda67120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.294+0000 7fc9a2fbb740 4 client.17043 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:17:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.294+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2793473928 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc994036070 con 0x5592eda64d30 2024-09-15T22:17:50.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.294+0000 7fc99a59c640 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5592eda64d30 0x5592eda67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.295+0000 7fc99a59c640 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5592eda64d30 0x5592eda67120 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c43c67eda2bc2f3 server_cookie=125770978be466c2 in_seq=0 out_seq=0 2024-09-15T22:17:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.295+0000 7fc981ffb640 10 client.17043 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.295+0000 7fc981ffb640 1 -- 172.21.15.5:0/2793473928 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc994036070 con 0x5592eda64d30 2024-09-15T22:17:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.295+0000 7fc981ffb640 10 client.17043 handle_command_reply: tid=0 2024-09-15T22:17:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.295+0000 7fc9a2fbb740 1 client.17043 shutdown 2024-09-15T22:17:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2793473928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 msgr2=0x5592eda52ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 0x5592eda52ca0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc994009ca0 tx=0x7fc994003970 comp rx=0 tx=0).stop 2024-09-15T22:17:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2793473928 shutdown_connections 2024-09-15T22:17:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda55240 0x5592eda52ca0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5592eda64d30 0x5592eda67120 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda42090 0x5592eda52760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2793473928 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda41770 0x5592eda52220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2793473928 >> 172.21.15.5:0/2793473928 conn(0x5592eda3dde0 msgr2=0x5592eda43a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc981ffb640 0 client.17043 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2793473928 shutdown_connections 2024-09-15T22:17:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2793473928 wait complete. 2024-09-15T22:17:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.296+0000 7fc9a2fbb740 20 client.17043 trim_cache size 0 max 16384 2024-09-15T22:17:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.327+0000 7fc9a2fbb740 1 Processor -- start 2024-09-15T22:17:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.327+0000 7fc9a2fbb740 1 -- start start 2024-09-15T22:17:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.328+0000 7fc9a2fbb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda41770 0x5592eda54190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.328+0000 7fc9a2fbb740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda42090 0x5592eda6c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.328+0000 7fc9a2fbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda55240 0x5592eda6cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.328+0000 7fc999d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda42090 0x5592eda6c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.328+0000 7fc999d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda42090 0x5592eda6c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:45442/0 (socket says 172.21.15.5:45442) 2024-09-15T22:17:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.328+0000 7fc999d9b640 1 -- 172.21.15.5:0/307195579 learned_addr learned my addr 172.21.15.5:0/307195579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.328+0000 7fc9a2fbb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c0128d0 con 0x5592eda41770 2024-09-15T22:17:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.328+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/307195579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c0101a0 con 0x5592eda55240 2024-09-15T22:17:50.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.328+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/307195579 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c00e010 con 0x5592eda42090 2024-09-15T22:17:50.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.328+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/307195579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda55240 0x5592eda6cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.332+0000 7fc99a59c640 1 --2- 172.21.15.5:0/307195579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda41770 0x5592eda54190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.332+0000 7fc99ad9d640 1 -- 172.21.15.5:0/307195579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda42090 msgr2=0x5592eda6c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.332+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/307195579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda42090 0x5592eda6c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.332+0000 7fc99ad9d640 1 -- 172.21.15.5:0/307195579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda41770 msgr2=0x5592eda54190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.332+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/307195579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda41770 0x5592eda54190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.332+0000 7fc99ad9d640 1 -- 172.21.15.5:0/307195579 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592eda511d0 con 0x5592eda55240 2024-09-15T22:17:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.336+0000 7fc99a59c640 1 --2- 172.21.15.5:0/307195579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda41770 0x5592eda54190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.339+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/307195579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda55240 0x5592eda6cd50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc994004570 tx=0x7fc994005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2323459734cf09fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.347+0000 7fc983fff640 1 -- 172.21.15.5:0/307195579 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc994048070 con 0x5592eda55240 2024-09-15T22:17:50.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.347+0000 7fc983fff640 1 -- 172.21.15.5:0/307195579 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc994036070 con 0x5592eda55240 2024-09-15T22:17:50.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.347+0000 7fc983fff640 1 -- 172.21.15.5:0/307195579 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9940309a0 con 0x5592eda55240 2024-09-15T22:17:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.348+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/307195579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda55240 msgr2=0x5592eda6cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.349+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/307195579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda55240 0x5592eda6cd50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc994004570 tx=0x7fc994005d80 comp rx=0 tx=0).stop 2024-09-15T22:17:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.353+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/307195579 shutdown_connections 2024-09-15T22:17:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.353+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/307195579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda55240 0x5592eda6cd50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.353+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/307195579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda42090 0x5592eda6c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.353+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/307195579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda41770 0x5592eda54190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.353+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/307195579 >> 172.21.15.5:0/307195579 conn(0x5592eda3dde0 msgr2=0x5592eda53a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.361+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/307195579 shutdown_connections 2024-09-15T22:17:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.361+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/307195579 wait complete. 2024-09-15T22:17:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.361+0000 7fc9a2fbb740 1 -- start start 2024-09-15T22:17:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.361+0000 7fc9a2fbb740 1 Processor -- start 2024-09-15T22:17:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.362+0000 7fc9a2fbb740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 0x5592eda5da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.362+0000 7fc9a2fbb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda42090 0x5592eda58d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.362+0000 7fc99a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 0x5592eda5da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.362+0000 7fc99a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 0x5592eda5da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:43752/0 (socket says 172.21.15.5:43752) 2024-09-15T22:17:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.362+0000 7fc99a59c640 1 -- 172.21.15.5:0/2521482206 learned_addr learned my addr 172.21.15.5:0/2521482206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:17:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.362+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda55240 0x5592eda59370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.363+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2521482206 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c00e660 con 0x5592eda42090 2024-09-15T22:17:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.363+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2521482206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c00fad0 con 0x5592eda41770 2024-09-15T22:17:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.363+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda55240 0x5592eda59370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.363+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2521482206 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c00ed30 con 0x5592eda55240 2024-09-15T22:17:50.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.363+0000 7fc9a2fbb740 10 client.0 _ll_register_callbacks cb 0x7fc994046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:17:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.363+0000 7fc99a59c640 1 -- 172.21.15.5:0/2521482206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda55240 msgr2=0x5592eda59370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.363+0000 7fc99a59c640 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda55240 0x5592eda59370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.363+0000 7fc99a59c640 1 -- 172.21.15.5:0/2521482206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda42090 msgr2=0x5592eda58d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:17:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.363+0000 7fc99a59c640 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda42090 0x5592eda58d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.363+0000 7fc99a59c640 1 -- 172.21.15.5:0/2521482206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592eda59ba0 con 0x5592eda41770 2024-09-15T22:17:50.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.365+0000 7fc99a59c640 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 0x5592eda5da50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc98c09c0a0 tx=0x7fc98c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2336b3142b73a733 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:17:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.365+0000 7fc981ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:17:50.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.367+0000 7fc99ad9d640 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda55240 0x5592eda59370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:17:50.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.367+0000 7fc981ffb640 1 -- 172.21.15.5:0/2521482206 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c0ba060 con 0x5592eda41770 2024-09-15T22:17:50.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.367+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2521482206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc994036070 con 0x5592eda41770 2024-09-15T22:17:50.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.367+0000 7fc981ffb640 1 -- 172.21.15.5:0/2521482206 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7fc98c010430 con 0x5592eda41770 2024-09-15T22:17:50.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.367+0000 7fc981ffb640 1 -- 172.21.15.5:0/2521482206 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc98c082050 con 0x5592eda41770 2024-09-15T22:17:50.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.368+0000 7fc9a2fbb740 10 client.26573 fetch_fsmap learned FSMap version 25 2024-09-15T22:17:50.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.368+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2521482206 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5592edae08b0 con 0x5592eda41770 2024-09-15T22:17:50.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.373+0000 7fc981ffb640 1 -- 172.21.15.5:0/2521482206 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc994036070 con 0x5592eda41770 2024-09-15T22:17:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.373+0000 7fc981ffb640 1 -- 172.21.15.5:0/2521482206 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc98c010ac0 con 0x5592eda41770 2024-09-15T22:17:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.374+0000 7fc9a2fbb740 10 client.26573 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:17:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.374+0000 7fc9a2fbb740 10 client.26573 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:17:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.374+0000 7fc9a2fbb740 20 client.26573 populate_metadata read hostname 'smithi005' 2024-09-15T22:17:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.374+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5592edae0fa0 0x5592edae3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:17:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.374+0000 7fc9a2fbb740 4 client.26573 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}] 2024-09-15T22:17:50.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.374+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2521482206 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}) -- 0x5592ed9b3890 con 0x5592edae0fa0 2024-09-15T22:17:50.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.377+0000 7fc999d9b640 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5592edae0fa0 0x5592edae3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:17:50.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.393+0000 7fc981ffb640 10 client.26573 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:50.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.393+0000 7fc999d9b640 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5592edae0fa0 0x5592edae3420 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df66f5b84383e67a server_cookie=c8c168bf008562a0 in_seq=0 out_seq=0 2024-09-15T22:17:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc981ffb640 1 -- 172.21.15.5:0/2521482206 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5592ed9b3890 con 0x5592edae0fa0 2024-09-15T22:17:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc981ffb640 10 client.26573 handle_command_reply: tid=0 2024-09-15T22:17:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 1 client.26573 shutdown 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2521482206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 msgr2=0x5592eda5da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 0x5592eda5da50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc98c09c0a0 tx=0x7fc98c01a040 comp rx=0 tx=0).stop 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2521482206 shutdown_connections 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5592eda55240 0x5592eda59370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5592eda42090 0x5592eda58d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5592edae0fa0 0x5592edae3420 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 1 --2- 172.21.15.5:0/2521482206 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5592eda41770 0x5592eda5da50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc981ffb640 0 client.26573 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2521482206 >> 172.21.15.5:0/2521482206 conn(0x5592eda3dde0 msgr2=0x5592eda542a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2521482206 shutdown_connections 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 1 -- 172.21.15.5:0/2521482206 wait complete. 2024-09-15T22:17:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.725+0000 7fc9a2fbb740 20 client.26573 trim_cache size 0 max 16384 2024-09-15T22:17:50.729 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:17:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:17:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:17:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 30, 2024-09-15T22:17:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:17:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "49f84031": { 2024-09-15T22:17:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 30, 2024-09-15T22:17:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:17:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:17:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:17:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.3, 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:17:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:17:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:17:50.731 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:17:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 -- 172.21.15.5:0/536580334 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc9840a33b0 msgr2=0x7fc9840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc9840a33b0 0x7fc9840a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc99c10f010 tx=0x7fc990005f90 comp rx=0 tx=0).stop 2024-09-15T22:17:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 -- 172.21.15.5:0/536580334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99c102d30 msgr2=0x7fc99c1a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:17:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99c102d30 0x7fc99c1a0400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00e9f0 tx=0x7fc98c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 -- 172.21.15.5:0/536580334 shutdown_connections 2024-09-15T22:17:50.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc9840a33b0 0x7fc9840a5870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc99c1082a0 0x7fc99c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc99c1036e0 0x7fc99c1a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 --2- 172.21.15.5:0/536580334 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc99c102d30 0x7fc99c1a0400 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:17:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 -- 172.21.15.5:0/536580334 >> 172.21.15.5:0/536580334 conn(0x7fc99c0fc7d0 msgr2=0x7fc99c1008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:17:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 -- 172.21.15.5:0/536580334 shutdown_connections 2024-09-15T22:17:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:17:50.726+0000 7fc9a0be2640 1 -- 172.21.15.5:0/536580334 wait complete. 2024-09-15T22:17:51.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:50 smithi099 ceph-mon[34892]: pgmap v275: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-09-15T22:17:51.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:50 smithi129 ceph-mon[35314]: pgmap v275: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-09-15T22:17:51.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:50 smithi005 ceph-mon[30944]: pgmap v275: 129 pgs: 129 active+clean; 3.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 5.3 MiB/s wr, 586 op/s 2024-09-15T22:17:51.507 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '49f84031', quiesce took 9.5 seconds. Will release after: 12.850000000000001 2024-09-15T22:17:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:52 smithi099 ceph-mon[34892]: pgmap v276: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 763 op/s 2024-09-15T22:17:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:52 smithi129 ceph-mon[35314]: pgmap v276: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 763 op/s 2024-09-15T22:17:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:52 smithi005 ceph-mon[30944]: pgmap v276: 129 pgs: 129 active+clean; 3.2 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 6.9 MiB/s wr, 763 op/s 2024-09-15T22:17:54.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 505.0 s, 110.5 tps, lat 133.280 ms stddev 148.403 2024-09-15T22:17:55.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:55 smithi099 ceph-mon[34892]: pgmap v277: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.9 MiB/s wr, 518 op/s 2024-09-15T22:17:55.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:55 smithi129 ceph-mon[35314]: pgmap v277: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.9 MiB/s wr, 518 op/s 2024-09-15T22:17:55.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:54 smithi005 ceph-mon[30944]: pgmap v277: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.9 MiB/s wr, 518 op/s 2024-09-15T22:17:57.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:57 smithi099 ceph-mon[34892]: pgmap v278: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-09-15T22:17:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:57 smithi129 ceph-mon[35314]: pgmap v278: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-09-15T22:17:57.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:57 smithi005 ceph-mon[30944]: pgmap v278: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 4.4 MiB/s wr, 488 op/s 2024-09-15T22:17:59.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:17:59 smithi099 ceph-mon[34892]: pgmap v279: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-09-15T22:17:59.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:17:59 smithi129 ceph-mon[35314]: pgmap v279: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-09-15T22:17:59.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:17:59 smithi005 ceph-mon[30944]: pgmap v279: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-09-15T22:17:59.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 510.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:18:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:01 smithi099 ceph-mon[34892]: pgmap v280: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.5 MiB/s wr, 280 op/s 2024-09-15T22:18:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:01 smithi129 ceph-mon[35314]: pgmap v280: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.5 MiB/s wr, 280 op/s 2024-09-15T22:18:01.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:01 smithi005 ceph-mon[30944]: pgmap v280: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 2.5 MiB/s wr, 280 op/s 2024-09-15T22:18:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:03 smithi099 ceph-mon[34892]: pgmap v281: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.3 MiB/s wr, 258 op/s 2024-09-15T22:18:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:03 smithi129 ceph-mon[35314]: pgmap v281: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.3 MiB/s wr, 258 op/s 2024-09-15T22:18:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:03 smithi005 ceph-mon[30944]: pgmap v281: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.3 MiB/s wr, 258 op/s 2024-09-15T22:18:04.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:18:04.634 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 515.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:18:04.820 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:18:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.098+0000 7f2401211640 1 -- 172.21.15.5:0/828566577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23fc102790 msgr2=0x7f23fc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.098+0000 7f2401211640 1 --2- 172.21.15.5:0/828566577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23fc102790 0x7f23fc1067e0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f23f0009900 tx=0x7f23f002a410 comp rx=0 tx=0).stop 2024-09-15T22:18:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.098+0000 7f2401211640 1 -- 172.21.15.5:0/828566577 shutdown_connections 2024-09-15T22:18:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.098+0000 7f2401211640 1 --2- 172.21.15.5:0/828566577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23fc106f10 0x7f23fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.098+0000 7f2401211640 1 --2- 172.21.15.5:0/828566577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23fc102790 0x7f23fc1067e0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.098+0000 7f2401211640 1 --2- 172.21.15.5:0/828566577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc101de0 0x7f23fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:05.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.098+0000 7f2401211640 1 -- 172.21.15.5:0/828566577 >> 172.21.15.5:0/828566577 conn(0x7f23fc0fd8c0 msgr2=0x7f23fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.098+0000 7f2401211640 1 -- 172.21.15.5:0/828566577 shutdown_connections 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.098+0000 7f2401211640 1 -- 172.21.15.5:0/828566577 wait complete. 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.098+0000 7f2401211640 1 Processor -- start 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f2401211640 1 -- start start 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f2401211640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23fc101de0 0x7f23fc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f2401211640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23fc102790 0x7f23fc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f2401211640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc106f10 0x7f23fc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f2401211640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc111f90 con 0x7f23fc102790 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f2401211640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc111e10 con 0x7f23fc101de0 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f2401211640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f23fc112110 con 0x7f23fc106f10 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f23fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23fc102790 0x7f23fc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f23fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23fc102790 0x7f23fc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46034/0 (socket says 172.21.15.5:46034) 2024-09-15T22:18:05.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f23fa59c640 1 -- 172.21.15.5:0/1756786767 learned_addr learned my addr 172.21.15.5:0/1756786767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f23fad9d640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23fc101de0 0x7f23fc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f23fb59e640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc106f10 0x7f23fc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f23fa59c640 1 -- 172.21.15.5:0/1756786767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc106f10 msgr2=0x7f23fc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f23fa59c640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc106f10 0x7f23fc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f23fa59c640 1 -- 172.21.15.5:0/1756786767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23fc101de0 msgr2=0x7f23fc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f23fa59c640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23fc101de0 0x7f23fc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f23fa59c640 1 -- 172.21.15.5:0/1756786767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23fc1a6950 con 0x7f23fc102790 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.099+0000 7f23fb59e640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc106f10 0x7f23fc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.100+0000 7f23fa59c640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23fc102790 0x7f23fc074a30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f23f0006010 tx=0x7f23f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.100+0000 7f23e3fff640 1 -- 172.21.15.5:0/1756786767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f003f030 con 0x7f23fc102790 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.100+0000 7f2401211640 1 -- 172.21.15.5:0/1756786767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23fc1a6be0 con 0x7f23fc102790 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.100+0000 7f23e3fff640 1 -- 172.21.15.5:0/1756786767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23f0003ce0 con 0x7f23fc102790 2024-09-15T22:18:05.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.100+0000 7f23e3fff640 1 -- 172.21.15.5:0/1756786767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f0035070 con 0x7f23fc102790 2024-09-15T22:18:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.100+0000 7f2401211640 1 -- 172.21.15.5:0/1756786767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23fc1a70c0 con 0x7f23fc102790 2024-09-15T22:18:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.101+0000 7f2401211640 1 -- 172.21.15.5:0/1756786767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23fc1021c0 con 0x7f23fc102790 2024-09-15T22:18:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.102+0000 7f23e3fff640 1 -- 172.21.15.5:0/1756786767 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f23f0042030 con 0x7f23fc102790 2024-09-15T22:18:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.103+0000 7f23e3fff640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23e40a33b0 0x7f23e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.103+0000 7f23fad9d640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23e40a33b0 0x7f23e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.103+0000 7f23e3fff640 1 -- 172.21.15.5:0/1756786767 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f23f00f0810 con 0x7f23fc102790 2024-09-15T22:18:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.104+0000 7f23fad9d640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23e40a33b0 0x7f23e40a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009990 tx=0x7f23ec009110 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:05.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.106+0000 7f23e3fff640 1 -- 172.21.15.5:0/1756786767 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23f00bcdf0 con 0x7f23fc102790 2024-09-15T22:18:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:05.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:05 smithi005 ceph-mon[30944]: pgmap v282: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 313 KiB/s wr, 44 op/s 2024-09-15T22:18:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.330+0000 7f2401211640 1 -- 172.21.15.5:0/1756786767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23fc075a20 con 0x7f23fc102790 2024-09-15T22:18:05.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.331+0000 7f23e3fff640 1 -- 172.21.15.5:0/1756786767 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f23f00bc6f0 con 0x7f23fc102790 2024-09-15T22:18:05.333 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:18:05.333 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:18:05.333 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:18:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 -- 172.21.15.5:0/1756786767 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23e40a33b0 msgr2=0x7f23e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:05.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23e40a33b0 0x7f23e40a5870 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f23ec009990 tx=0x7f23ec009110 comp rx=0 tx=0).stop 2024-09-15T22:18:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 -- 172.21.15.5:0/1756786767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23fc102790 msgr2=0x7f23fc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23fc102790 0x7f23fc074a30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f23f0006010 tx=0x7f23f00042f0 comp rx=0 tx=0).stop 2024-09-15T22:18:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 -- 172.21.15.5:0/1756786767 shutdown_connections 2024-09-15T22:18:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23e40a33b0 0x7f23e40a5870 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f23fc106f10 0x7f23fc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23fc102790 0x7f23fc074a30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 --2- 172.21.15.5:0/1756786767 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f23fc101de0 0x7f23fc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 -- 172.21.15.5:0/1756786767 >> 172.21.15.5:0/1756786767 conn(0x7f23fc0fd8c0 msgr2=0x7f23fc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 -- 172.21.15.5:0/1756786767 shutdown_connections 2024-09-15T22:18:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:05.334+0000 7f2401211640 1 -- 172.21.15.5:0/1756786767 wait complete. 2024-09-15T22:18:05.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:05 smithi099 ceph-mon[34892]: pgmap v282: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 313 KiB/s wr, 44 op/s 2024-09-15T22:18:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:05 smithi129 ceph-mon[35314]: pgmap v282: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 313 KiB/s wr, 44 op/s 2024-09-15T22:18:06.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:06 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1756786767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:06.161 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id 49f84031 --release --await' 2024-09-15T22:18:06.161 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id 49f84031 --release --await 2024-09-15T22:18:06.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:06 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1756786767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:06.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:06 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1756786767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.350+0000 7f88e6041640 1 -- 172.21.15.5:0/1545001575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e0108290 msgr2=0x7f88e010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.350+0000 7f88e6041640 1 --2- 172.21.15.5:0/1545001575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e0108290 0x7f88e010ec30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009cd0 tx=0x7f88d802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.350+0000 7f88e6041640 1 -- 172.21.15.5:0/1545001575 shutdown_connections 2024-09-15T22:18:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.350+0000 7f88e6041640 1 --2- 172.21.15.5:0/1545001575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e0108290 0x7f88e010ec30 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.350+0000 7f88e6041640 1 --2- 172.21.15.5:0/1545001575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e0103740 0x7f88e0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.350+0000 7f88e6041640 1 --2- 172.21.15.5:0/1545001575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e0102d90 0x7f88e0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.350+0000 7f88e6041640 1 -- 172.21.15.5:0/1545001575 >> 172.21.15.5:0/1545001575 conn(0x7f88e00fc7d0 msgr2=0x7f88e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.350+0000 7f88e6041640 1 -- 172.21.15.5:0/1545001575 shutdown_connections 2024-09-15T22:18:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.350+0000 7f88e6041640 1 -- 172.21.15.5:0/1545001575 wait complete. 2024-09-15T22:18:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88e6041640 1 Processor -- start 2024-09-15T22:18:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88e6041640 1 -- start start 2024-09-15T22:18:06.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88e6041640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e0102d90 0x7f88e01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88e6041640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e0103740 0x7f88e01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88e6041640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e0108290 0x7f88e019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88e6041640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88e0112090 con 0x7f88e0103740 2024-09-15T22:18:06.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88e6041640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88e0111f10 con 0x7f88e0108290 2024-09-15T22:18:06.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88e6041640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88e0112210 con 0x7f88e0102d90 2024-09-15T22:18:06.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e0103740 0x7f88e01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e0103740 0x7f88e01989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46050/0 (socket says 172.21.15.5:46050) 2024-09-15T22:18:06.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88d7fff640 1 -- 172.21.15.5:0/2917352570 learned_addr learned my addr 172.21.15.5:0/2917352570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:06.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.351+0000 7f88df7fe640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e0102d90 0x7f88e01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88dffff640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e0108290 0x7f88e019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88d7fff640 1 -- 172.21.15.5:0/2917352570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e0102d90 msgr2=0x7f88e01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88d7fff640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e0102d90 0x7f88e01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88d7fff640 1 -- 172.21.15.5:0/2917352570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e0108290 msgr2=0x7f88e019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88d7fff640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e0108290 0x7f88e019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88d7fff640 1 -- 172.21.15.5:0/2917352570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e019d460 con 0x7f88e0103740 2024-09-15T22:18:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88df7fe640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e0102d90 0x7f88e01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88dffff640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e0108290 0x7f88e019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88d7fff640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e0103740 0x7f88e01989f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f88c800c970 tx=0x7f88c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88dd7fa640 1 -- 172.21.15.5:0/2917352570 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c8014070 con 0x7f88e0103740 2024-09-15T22:18:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88dd7fa640 1 -- 172.21.15.5:0/2917352570 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88c8004440 con 0x7f88e0103740 2024-09-15T22:18:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88e6041640 1 -- 172.21.15.5:0/2917352570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e019d750 con 0x7f88e0103740 2024-09-15T22:18:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88dd7fa640 1 -- 172.21.15.5:0/2917352570 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c800f450 con 0x7f88e0103740 2024-09-15T22:18:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.352+0000 7f88e6041640 1 -- 172.21.15.5:0/2917352570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e01a5030 con 0x7f88e0103740 2024-09-15T22:18:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.355+0000 7f88e841a740 1 Processor -- start 2024-09-15T22:18:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.355+0000 7f88e841a740 1 -- start start 2024-09-15T22:18:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.356+0000 7f88e841a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7847880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.356+0000 7f88dd7fa640 1 -- 172.21.15.5:0/2917352570 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f88c8017070 con 0x7f88e0103740 2024-09-15T22:18:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.356+0000 7f88e841a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7847dc0 0x5563b7852140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.356+0000 7f88e841a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7852680 0x5563b7854a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.356+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7852680 0x5563b7854a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.356+0000 7f88e841a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5563b77ad2e0 con 0x5563b7852680 2024-09-15T22:18:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7847880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88dffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7847880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:40612/0 (socket says 172.21.15.5:40612) 2024-09-15T22:18:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88dffff640 1 -- 172.21.15.5:0/1632577588 learned_addr learned my addr 172.21.15.5:0/1632577588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88df7fe640 1 --2- 172.21.15.5:0/1632577588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7847dc0 0x5563b7852140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88e841a740 1 -- 172.21.15.5:0/1632577588 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5563b7754ce0 con 0x5563b78474a0 2024-09-15T22:18:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88e841a740 1 -- 172.21.15.5:0/1632577588 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5563b77a92a0 con 0x5563b7847dc0 2024-09-15T22:18:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88df7fe640 1 -- 172.21.15.5:0/1632577588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 msgr2=0x5563b7847880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88df7fe640 1 --2- 172.21.15.5:0/1632577588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7847880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88df7fe640 1 -- 172.21.15.5:0/1632577588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7852680 msgr2=0x5563b7854a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88df7fe640 1 --2- 172.21.15.5:0/1632577588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7852680 0x5563b7854a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88df7fe640 1 -- 172.21.15.5:0/1632577588 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563b7739440 con 0x5563b7847dc0 2024-09-15T22:18:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88dffff640 1 --2- 172.21.15.5:0/1632577588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7847880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.357+0000 7f88dd7fa640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88cc0a3430 0x7f88cc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.358+0000 7f88df7fe640 1 --2- 172.21.15.5:0/1632577588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7847dc0 0x5563b7852140 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009fd0 tx=0x7f88d00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c494bc836f7922a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.358+0000 7f88dffff640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88cc0a3430 0x7f88cc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.358+0000 7f88dd7fa640 1 -- 172.21.15.5:0/2917352570 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f88c80cff70 con 0x7f88e0103740 2024-09-15T22:18:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.358+0000 7f88d5ffb640 1 -- 172.21.15.5:0/1632577588 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0004310 con 0x5563b7847dc0 2024-09-15T22:18:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.358+0000 7f88d5ffb640 1 -- 172.21.15.5:0/1632577588 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d00044b0 con 0x5563b7847dc0 2024-09-15T22:18:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.358+0000 7f88d5ffb640 1 -- 172.21.15.5:0/1632577588 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0003650 con 0x5563b7847dc0 2024-09-15T22:18:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.359+0000 7f88dffff640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88cc0a3430 0x7f88cc0a58f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009430 tx=0x7f88d8009360 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.359+0000 7f88e841a740 1 -- 172.21.15.5:0/1632577588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7847dc0 msgr2=0x5563b7852140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.359+0000 7f88e841a740 1 --2- 172.21.15.5:0/1632577588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7847dc0 0x5563b7852140 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009fd0 tx=0x7f88d00118d0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.359+0000 7f88e841a740 1 -- 172.21.15.5:0/1632577588 shutdown_connections 2024-09-15T22:18:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.359+0000 7f88e841a740 1 --2- 172.21.15.5:0/1632577588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7852680 0x5563b7854a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.359+0000 7f88e841a740 1 --2- 172.21.15.5:0/1632577588 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7847dc0 0x5563b7852140 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.359+0000 7f88e841a740 1 --2- 172.21.15.5:0/1632577588 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7847880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.359+0000 7f88e841a740 1 -- 172.21.15.5:0/1632577588 >> 172.21.15.5:0/1632577588 conn(0x5563b7843a30 msgr2=0x5563b7843e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.359+0000 7f88e841a740 1 -- 172.21.15.5:0/1632577588 shutdown_connections 2024-09-15T22:18:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.359+0000 7f88e841a740 1 -- 172.21.15.5:0/1632577588 wait complete. 2024-09-15T22:18:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.359+0000 7f88e841a740 1 -- start start 2024-09-15T22:18:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88e841a740 1 Processor -- start 2024-09-15T22:18:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88e841a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7851fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88e841a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b784f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88e841a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 0x5563b784f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88df7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7851fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b784f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88df7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7851fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:40628/0 (socket says 172.21.15.5:40628) 2024-09-15T22:18:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88df7fe640 1 -- 172.21.15.5:0/3936502874 learned_addr learned my addr 172.21.15.5:0/3936502874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88e841a740 1 -- 172.21.15.5:0/3936502874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5563b7859660 con 0x5563b7847dc0 2024-09-15T22:18:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88e841a740 1 -- 172.21.15.5:0/3936502874 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5563b7768050 con 0x5563b78474a0 2024-09-15T22:18:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88e841a740 1 -- 172.21.15.5:0/3936502874 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5563b78597e0 con 0x5563b7852680 2024-09-15T22:18:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.360+0000 7f88e841a740 10 client.0 _ll_register_callbacks cb 0x5563b77a8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88dffff640 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 0x5563b784f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88df7fe640 1 -- 172.21.15.5:0/3936502874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 msgr2=0x5563b784f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88df7fe640 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 0x5563b784f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88df7fe640 1 -- 172.21.15.5:0/3936502874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 msgr2=0x5563b784f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88df7fe640 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b784f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88df7fe640 1 -- 172.21.15.5:0/3936502874 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563b751e3c0 con 0x5563b78474a0 2024-09-15T22:18:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88d7fff640 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b784f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88df7fe640 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7851fd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f88d0004bb0 tx=0x7f88d0009d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc20c0cfc42717f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88beffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:18:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88beffd640 1 -- 172.21.15.5:0/3936502874 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0029070 con 0x5563b78474a0 2024-09-15T22:18:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88beffd640 1 -- 172.21.15.5:0/3936502874 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f88d0005200 con 0x5563b78474a0 2024-09-15T22:18:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88e841a740 1 -- 172.21.15.5:0/3936502874 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88d00044b0 con 0x5563b78474a0 2024-09-15T22:18:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88beffd640 1 -- 172.21.15.5:0/3936502874 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88d0035050 con 0x5563b78474a0 2024-09-15T22:18:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88e841a740 10 client.26588 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.361+0000 7f88e841a740 1 -- 172.21.15.5:0/3936502874 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5563b7865010 con 0x5563b78474a0 2024-09-15T22:18:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.362+0000 7f88beffd640 1 -- 172.21.15.5:0/3936502874 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d00044b0 con 0x5563b78474a0 2024-09-15T22:18:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.362+0000 7f88beffd640 1 -- 172.21.15.5:0/3936502874 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f88d000f4e0 con 0x5563b78474a0 2024-09-15T22:18:06.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.362+0000 7f88e841a740 10 client.26588 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.362+0000 7f88e841a740 10 client.26588 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.362+0000 7f88e841a740 20 client.26588 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.362+0000 7f88e841a740 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5563b7866360 0x5563b7868750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.362+0000 7f88e841a740 4 client.26588 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.362+0000 7f88e841a740 1 -- 172.21.15.5:0/3936502874 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563b77ba850 con 0x5563b7866360 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.362+0000 7f88d7fff640 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5563b7866360 0x5563b7868750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.363+0000 7f88beffd640 10 client.26588 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.363+0000 7f88d7fff640 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5563b7866360 0x5563b7868750 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb5e659d774ccc01 server_cookie=7c940907c36b7c58 in_seq=0 out_seq=0 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.363+0000 7f88beffd640 1 -- 172.21.15.5:0/3936502874 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563b77ba850 con 0x5563b7866360 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.363+0000 7f88beffd640 10 client.26588 handle_command_reply: tid=0 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.363+0000 7f88e841a740 1 client.26588 shutdown 2024-09-15T22:18:06.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88e841a740 1 -- 172.21.15.5:0/3936502874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 msgr2=0x5563b7851fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88e841a740 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7851fd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f88d0004bb0 tx=0x7f88d0009d00 comp rx=0 tx=0).stop 2024-09-15T22:18:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88e841a740 1 -- 172.21.15.5:0/3936502874 shutdown_connections 2024-09-15T22:18:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88e841a740 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 0x5563b784f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88e841a740 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b784f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88e841a740 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5563b7866360 0x5563b7868750 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88e841a740 1 --2- 172.21.15.5:0/3936502874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b7851fd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88beffd640 0 client.26588 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88e841a740 1 -- 172.21.15.5:0/3936502874 >> 172.21.15.5:0/3936502874 conn(0x5563b7843a30 msgr2=0x5563b7849710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88e841a740 1 -- 172.21.15.5:0/3936502874 shutdown_connections 2024-09-15T22:18:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88e841a740 1 -- 172.21.15.5:0/3936502874 wait complete. 2024-09-15T22:18:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.364+0000 7f88e841a740 20 client.26588 trim_cache size 0 max 16384 2024-09-15T22:18:06.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88e841a740 1 Processor -- start 2024-09-15T22:18:06.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88e841a740 1 -- start start 2024-09-15T22:18:06.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88e841a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b78628c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88e841a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b786dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88df7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b78628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b786dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88df7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b78628c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:40636/0 (socket says 172.21.15.5:40636) 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b786dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46080/0 (socket says 172.21.15.5:46080) 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88df7fe640 1 -- 172.21.15.5:0/206941103 learned_addr learned my addr 172.21.15.5:0/206941103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88e841a740 1 --2- 172.21.15.5:0/206941103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 0x5563b786e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88e841a740 1 -- 172.21.15.5:0/206941103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88e01af7b0 con 0x5563b7847dc0 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88e841a740 1 -- 172.21.15.5:0/206941103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88e01ad080 con 0x5563b78474a0 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.383+0000 7f88e841a740 1 -- 172.21.15.5:0/206941103 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88e01aaef0 con 0x5563b7852680 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.384+0000 7f88df7fe640 1 -- 172.21.15.5:0/206941103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 msgr2=0x5563b786e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.384+0000 7f88df7fe640 1 --2- 172.21.15.5:0/206941103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 0x5563b786e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.384+0000 7f88df7fe640 1 -- 172.21.15.5:0/206941103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 msgr2=0x5563b786dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.384+0000 7f88df7fe640 1 --2- 172.21.15.5:0/206941103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b786dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.384+0000 7f88df7fe640 1 -- 172.21.15.5:0/206941103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563b784fb50 con 0x5563b78474a0 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.384+0000 7f88d7fff640 1 --2- 172.21.15.5:0/206941103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b786dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.384+0000 7f88df7fe640 1 --2- 172.21.15.5:0/206941103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b78628c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f88d000a010 tx=0x7f88d0005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baf90b107ca9713f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.384+0000 7f88d57fa640 1 -- 172.21.15.5:0/206941103 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0032020 con 0x5563b78474a0 2024-09-15T22:18:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.384+0000 7f88d57fa640 1 -- 172.21.15.5:0/206941103 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88d000f6e0 con 0x5563b78474a0 2024-09-15T22:18:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.384+0000 7f88d57fa640 1 -- 172.21.15.5:0/206941103 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d0029070 con 0x5563b78474a0 2024-09-15T22:18:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.385+0000 7f88e841a740 1 -- 172.21.15.5:0/206941103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 msgr2=0x5563b78628c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.385+0000 7f88e841a740 1 --2- 172.21.15.5:0/206941103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b78628c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f88d000a010 tx=0x7f88d0005fb0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.385+0000 7f88e841a740 1 -- 172.21.15.5:0/206941103 shutdown_connections 2024-09-15T22:18:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.385+0000 7f88e841a740 1 --2- 172.21.15.5:0/206941103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 0x5563b786e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.385+0000 7f88e841a740 1 --2- 172.21.15.5:0/206941103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b786dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.385+0000 7f88e841a740 1 --2- 172.21.15.5:0/206941103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b78628c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.385+0000 7f88e841a740 1 -- 172.21.15.5:0/206941103 >> 172.21.15.5:0/206941103 conn(0x5563b7843a30 msgr2=0x5563b784e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.385+0000 7f88e841a740 1 -- 172.21.15.5:0/206941103 shutdown_connections 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.386+0000 7f88e841a740 1 -- 172.21.15.5:0/206941103 wait complete. 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.386+0000 7f88e841a740 1 -- start start 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.386+0000 7f88e841a740 1 Processor -- start 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88e841a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b78599f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88e841a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b7859f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88df7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b78599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b7859f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88d7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b7859f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46086/0 (socket says 172.21.15.5:46086) 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88d7fff640 1 -- 172.21.15.5:0/2114930741 learned_addr learned my addr 172.21.15.5:0/2114930741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88e841a740 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 0x5563b785a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88e841a740 1 -- 172.21.15.5:0/2114930741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88e01ab540 con 0x5563b7847dc0 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88e841a740 1 -- 172.21.15.5:0/2114930741 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f88e01ac9b0 con 0x5563b78474a0 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88e841a740 1 -- 172.21.15.5:0/2114930741 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f88e01abc10 con 0x5563b7852680 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88d7fff640 1 -- 172.21.15.5:0/2114930741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 msgr2=0x5563b785a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88d7fff640 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 0x5563b785a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88d7fff640 1 -- 172.21.15.5:0/2114930741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 msgr2=0x5563b78599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88d7fff640 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b78599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88d7fff640 1 -- 172.21.15.5:0/2114930741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563b78cef30 con 0x5563b7847dc0 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.387+0000 7f88df7fe640 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b78599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.388+0000 7f88e841a740 10 client.0 _ll_register_callbacks cb 0x7f88c80d3870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.388+0000 7f88e841a740 1 -- 172.21.15.5:0/2114930741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88d000f6e0 con 0x5563b7847dc0 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.388+0000 7f88d7fff640 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b7859f30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f88c8098250 tx=0x7f88c80d39a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d784f6af57dff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.388+0000 7f88beffd640 10 client.17067 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.388+0000 7f88beffd640 1 -- 172.21.15.5:0/2114930741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c80b9050 con 0x5563b7847dc0 2024-09-15T22:18:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.388+0000 7f88beffd640 1 -- 172.21.15.5:0/2114930741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f88c8098460 con 0x5563b7847dc0 2024-09-15T22:18:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.388+0000 7f88beffd640 1 -- 172.21.15.5:0/2114930741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88c809b750 con 0x5563b7847dc0 2024-09-15T22:18:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.388+0000 7f88e841a740 10 client.17067 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.389+0000 7f88e841a740 1 -- 172.21.15.5:0/2114930741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5563b78d0000 con 0x5563b7847dc0 2024-09-15T22:18:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.389+0000 7f88beffd640 1 -- 172.21.15.5:0/2114930741 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f88c80d3020 con 0x5563b7847dc0 2024-09-15T22:18:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.389+0000 7f88e841a740 10 client.17067 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.389+0000 7f88e841a740 10 client.17067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.389+0000 7f88e841a740 20 client.17067 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.389+0000 7f88e841a740 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5563b7866360 0x5563b78d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.389+0000 7f88e841a740 4 client.17067 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:18:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.389+0000 7f88e841a740 1 -- 172.21.15.5:0/2114930741 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563b77b4d60 con 0x5563b7866360 2024-09-15T22:18:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.389+0000 7f88df7fe640 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5563b7866360 0x5563b78d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.390+0000 7f88df7fe640 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5563b7866360 0x5563b78d27c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=797abf36ed562fc0 server_cookie=5432028b6dc37bc9 in_seq=0 out_seq=0 2024-09-15T22:18:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.390+0000 7f88beffd640 10 client.17067 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.390+0000 7f88beffd640 1 -- 172.21.15.5:0/2114930741 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5563b77b4d60 con 0x5563b7866360 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.390+0000 7f88beffd640 10 client.17067 handle_command_reply: tid=0 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.390+0000 7f88e841a740 1 client.17067 shutdown 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88e841a740 1 -- 172.21.15.5:0/2114930741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 msgr2=0x5563b7859f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88e841a740 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b7859f30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f88c8098250 tx=0x7f88c80d39a0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88e841a740 1 -- 172.21.15.5:0/2114930741 shutdown_connections 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88e841a740 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5563b7852680 0x5563b785a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88e841a740 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5563b7847dc0 0x5563b7859f30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88e841a740 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5563b7866360 0x5563b78d27c0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88e841a740 1 --2- 172.21.15.5:0/2114930741 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5563b78474a0 0x5563b78599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88e841a740 1 -- 172.21.15.5:0/2114930741 >> 172.21.15.5:0/2114930741 conn(0x5563b7843a30 msgr2=0x5563b7849f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88beffd640 0 client.17067 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88e841a740 1 -- 172.21.15.5:0/2114930741 shutdown_connections 2024-09-15T22:18:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88e841a740 1 -- 172.21.15.5:0/2114930741 wait complete. 2024-09-15T22:18:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.391+0000 7f88e841a740 20 client.17067 trim_cache size 0 max 16384 2024-09-15T22:18:06.397 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:18:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:18:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:18:06.397 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:18:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.392+0000 7f88e6041640 1 -- 172.21.15.5:0/2917352570 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88cc0a3430 msgr2=0x7f88cc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.392+0000 7f88e6041640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88cc0a3430 0x7f88cc0a58f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009430 tx=0x7f88d8009360 comp rx=0 tx=0).stop 2024-09-15T22:18:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.392+0000 7f88e6041640 1 -- 172.21.15.5:0/2917352570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e0103740 msgr2=0x7f88e01989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.392+0000 7f88e6041640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e0103740 0x7f88e01989f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f88c800c970 tx=0x7f88c800ce30 comp rx=0 tx=0).stop 2024-09-15T22:18:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.392+0000 7f88e6041640 1 -- 172.21.15.5:0/2917352570 shutdown_connections 2024-09-15T22:18:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.392+0000 7f88e6041640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88cc0a3430 0x7f88cc0a58f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.392+0000 7f88e6041640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f88e0108290 0x7f88e019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.392+0000 7f88e6041640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88e0103740 0x7f88e01989f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.392+0000 7f88e6041640 1 --2- 172.21.15.5:0/2917352570 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f88e0102d90 0x7f88e01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:06.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.392+0000 7f88e6041640 1 -- 172.21.15.5:0/2917352570 >> 172.21.15.5:0/2917352570 conn(0x7f88e00fc7d0 msgr2=0x7f88e00fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:06.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.392+0000 7f88e6041640 1 -- 172.21.15.5:0/2917352570 shutdown_connections 2024-09-15T22:18:06.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:06.393+0000 7f88e6041640 1 -- 172.21.15.5:0/2917352570 wait complete. 2024-09-15T22:18:06.667 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:07.079 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:07 smithi005 ceph-mon[30944]: pgmap v283: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 209 KiB/s wr, 45 op/s 2024-09-15T22:18:07.107 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:18:07.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:18:07.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:07 smithi099 ceph-mon[34892]: pgmap v283: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 209 KiB/s wr, 45 op/s 2024-09-15T22:18:07.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:07 smithi129 ceph-mon[35314]: pgmap v283: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 209 KiB/s wr, 45 op/s 2024-09-15T22:18:07.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.580+0000 7f1881d39640 1 -- 172.21.15.5:0/608503078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c106a90 msgr2=0x7f187c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:07.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.580+0000 7f1881d39640 1 --2- 172.21.15.5:0/608503078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c106a90 0x7f187c106e70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f186c009cd0 tx=0x7f186c02a8e0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.581+0000 7f1881d39640 1 -- 172.21.15.5:0/608503078 shutdown_connections 2024-09-15T22:18:07.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.581+0000 7f1881d39640 1 --2- 172.21.15.5:0/608503078 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f187c101430 0x7f187c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.581+0000 7f1881d39640 1 --2- 172.21.15.5:0/608503078 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f187c100a90 0x7f187c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.581+0000 7f1881d39640 1 --2- 172.21.15.5:0/608503078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c106a90 0x7f187c106e70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.581+0000 7f1881d39640 1 -- 172.21.15.5:0/608503078 >> 172.21.15.5:0/608503078 conn(0x7f187c0fc7d0 msgr2=0x7f187c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:07.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.581+0000 7f1881d39640 1 -- 172.21.15.5:0/608503078 shutdown_connections 2024-09-15T22:18:07.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.581+0000 7f1881d39640 1 -- 172.21.15.5:0/608503078 wait complete. 2024-09-15T22:18:07.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.581+0000 7f1881d39640 1 Processor -- start 2024-09-15T22:18:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f1881d39640 1 -- start start 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f1881d39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c100a90 0x7f187c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f1881d39640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f187c101430 0x7f187c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f1881d39640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f187c106a90 0x7f187c1a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f1881d39640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f187c110e90 con 0x7f187c100a90 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f1881d39640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f187c110d10 con 0x7f187c101430 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f1881d39640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f187c111010 con 0x7f187c106a90 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f187b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c100a90 0x7f187c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f187b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c100a90 0x7f187c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46100/0 (socket says 172.21.15.5:46100) 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f187b7fe640 1 -- 172.21.15.5:0/1664788385 learned_addr learned my addr 172.21.15.5:0/1664788385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f187bfff640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f187c106a90 0x7f187c1a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.582+0000 7f187affd640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f187c101430 0x7f187c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f187b7fe640 1 -- 172.21.15.5:0/1664788385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f187c106a90 msgr2=0x7f187c1a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f187b7fe640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f187c106a90 0x7f187c1a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f187b7fe640 1 -- 172.21.15.5:0/1664788385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f187c101430 msgr2=0x7f187c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f187b7fe640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f187c101430 0x7f187c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f187b7fe640 1 -- 172.21.15.5:0/1664788385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187c1a48c0 con 0x7f187c100a90 2024-09-15T22:18:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f187affd640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f187c101430 0x7f187c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f187b7fe640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c100a90 0x7f187c199300 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f186c02ae00 tx=0x7f186c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f1878ff9640 1 -- 172.21.15.5:0/1664788385 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c004040 con 0x7f187c100a90 2024-09-15T22:18:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f1878ff9640 1 -- 172.21.15.5:0/1664788385 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f186c0046c0 con 0x7f187c100a90 2024-09-15T22:18:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f1881d39640 1 -- 172.21.15.5:0/1664788385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187c1a4a90 con 0x7f187c100a90 2024-09-15T22:18:07.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f1881d39640 1 -- 172.21.15.5:0/1664788385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f187c1a4f70 con 0x7f187c100a90 2024-09-15T22:18:07.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.583+0000 7f1878ff9640 1 -- 172.21.15.5:0/1664788385 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c046480 con 0x7f187c100a90 2024-09-15T22:18:07.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f1884112740 1 Processor -- start 2024-09-15T22:18:07.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f1884112740 1 -- start start 2024-09-15T22:18:07.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f1884112740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5ccb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f1884112740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 0x5648d5cd5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f1884112740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cd8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f1884112740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5648d5c30620 con 0x5648d5ccb060 2024-09-15T22:18:07.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f1884112740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5648d5bd7e50 con 0x5648d5cd6240 2024-09-15T22:18:07.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f1884112740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5648d5c2c5e0 con 0x5648d5ccb980 2024-09-15T22:18:07.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f187bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5ccb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f1878ff9640 1 -- 172.21.15.5:0/1664788385 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f186c0305a0 con 0x7f187c100a90 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f187bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5ccb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46116/0 (socket says 172.21.15.5:46116) 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.586+0000 7f187bfff640 1 -- 172.21.15.5:0/3157370779 learned_addr learned my addr 172.21.15.5:0/3157370779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.587+0000 7f187affd640 1 --2- 172.21.15.5:0/3157370779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cd8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.587+0000 7f187b7fe640 1 --2- 172.21.15.5:0/3157370779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 0x5648d5cd5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.587+0000 7f187bfff640 1 -- 172.21.15.5:0/3157370779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 msgr2=0x5648d5cd5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.587+0000 7f187bfff640 1 --2- 172.21.15.5:0/3157370779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 0x5648d5cd5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.587+0000 7f187bfff640 1 -- 172.21.15.5:0/3157370779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 msgr2=0x5648d5cd8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.587+0000 7f187bfff640 1 --2- 172.21.15.5:0/3157370779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cd8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.587+0000 7f187bfff640 1 -- 172.21.15.5:0/3157370779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648d5bc8d40 con 0x5648d5ccb060 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.587+0000 7f187b7fe640 1 --2- 172.21.15.5:0/3157370779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 0x5648d5cd5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:07.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.587+0000 7f187bfff640 1 --2- 172.21.15.5:0/3157370779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5ccb440 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f187400b6b0 tx=0x7f1874004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c033f9a42b1e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.587+0000 7f1878ff9640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18640a3430 0x7f18640a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.587+0000 7f1861ffb640 1 -- 172.21.15.5:0/3157370779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1874003b20 con 0x5648d5ccb060 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.588+0000 7f1861ffb640 1 -- 172.21.15.5:0/3157370779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1874015070 con 0x5648d5ccb060 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.588+0000 7f1861ffb640 1 -- 172.21.15.5:0/3157370779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18740106a0 con 0x5648d5ccb060 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.588+0000 7f187b7fe640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18640a3430 0x7f18640a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.588+0000 7f1878ff9640 1 -- 172.21.15.5:0/1664788385 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f186c02e030 con 0x7f187c100a90 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.588+0000 7f1884112740 1 -- 172.21.15.5:0/3157370779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 msgr2=0x5648d5ccb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.588+0000 7f187b7fe640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18640a3430 0x7f18640a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f186c0eeac0 tx=0x7f186c0c0040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.588+0000 7f1884112740 1 --2- 172.21.15.5:0/3157370779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5ccb440 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f187400b6b0 tx=0x7f1874004610 comp rx=0 tx=0).stop 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.589+0000 7f1884112740 1 -- 172.21.15.5:0/3157370779 shutdown_connections 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.589+0000 7f1884112740 1 --2- 172.21.15.5:0/3157370779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cd8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.589+0000 7f1884112740 1 --2- 172.21.15.5:0/3157370779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 0x5648d5cd5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.589+0000 7f1884112740 1 --2- 172.21.15.5:0/3157370779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5ccb440 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.589+0000 7f1884112740 1 -- 172.21.15.5:0/3157370779 >> 172.21.15.5:0/3157370779 conn(0x5648d5cc75f0 msgr2=0x5648d5cc79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.589+0000 7f1884112740 1 -- 172.21.15.5:0/3157370779 shutdown_connections 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.589+0000 7f1884112740 1 -- 172.21.15.5:0/3157370779 wait complete. 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.589+0000 7f1884112740 1 -- start start 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.589+0000 7f1884112740 1 Processor -- start 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f1884112740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb060 0x5648d5cd31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f1884112740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cd36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f1884112740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cd3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f1884112740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5648d5cd8e10 con 0x5648d5ccb980 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f1884112740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5648d5bb8f10 con 0x5648d5cd6240 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f1884112740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5648d5cd8f90 con 0x5648d5ccb060 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cd36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cd36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46118/0 (socket says 172.21.15.5:46118) 2024-09-15T22:18:07.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187affd640 1 -- 172.21.15.5:0/3052429442 learned_addr learned my addr 172.21.15.5:0/3052429442 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:07.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f1884112740 10 client.0 _ll_register_callbacks cb 0x5648d5bdcdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:07.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187bfff640 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb060 0x5648d5cd31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187bfff640 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cd3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187affd640 1 -- 172.21.15.5:0/3052429442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb060 msgr2=0x5648d5cd31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:07.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187affd640 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb060 0x5648d5cd31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187affd640 1 -- 172.21.15.5:0/3052429442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 msgr2=0x5648d5cd3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:07.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187affd640 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cd3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187affd640 1 -- 172.21.15.5:0/3052429442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648d593f580 con 0x5648d5ccb980 2024-09-15T22:18:07.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187bfff640 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cd3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:07.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f187affd640 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cd36f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1870011800 tx=0x7f1870011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9baef78709400d42 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:07.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.590+0000 7f185affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:07.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f185affd640 1 -- 172.21.15.5:0/3052429442 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187000fe70 con 0x5648d5ccb980 2024-09-15T22:18:07.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f185affd640 1 -- 172.21.15.5:0/3052429442 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f1870005020 con 0x5648d5ccb980 2024-09-15T22:18:07.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f1884112740 1 -- 172.21.15.5:0/3052429442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1874015070 con 0x5648d5ccb980 2024-09-15T22:18:07.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f185affd640 1 -- 172.21.15.5:0/3052429442 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1870023050 con 0x5648d5ccb980 2024-09-15T22:18:07.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f1884112740 10 client.17079 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:07.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f1884112740 1 -- 172.21.15.5:0/3052429442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5648d5a09960 con 0x5648d5ccb980 2024-09-15T22:18:07.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f185affd640 1 -- 172.21.15.5:0/3052429442 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1874015070 con 0x5648d5ccb980 2024-09-15T22:18:07.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f185affd640 1 -- 172.21.15.5:0/3052429442 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1870018030 con 0x5648d5ccb980 2024-09-15T22:18:07.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f1884112740 10 client.17079 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:07.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f1884112740 10 client.17079 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f1884112740 20 client.17079 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f1884112740 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5648d5cea2e0 0x5648d5cec720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f1884112740 4 client.17079 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.591+0000 7f1884112740 1 -- 172.21.15.5:0/3052429442 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648d5c3db90 con 0x5648d5cea2e0 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.592+0000 7f187bfff640 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5648d5cea2e0 0x5648d5cec720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.592+0000 7f187bfff640 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5648d5cea2e0 0x5648d5cec720 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b712906b70231 server_cookie=9ae00f08ec785e02 in_seq=0 out_seq=0 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.592+0000 7f185affd640 10 client.17079 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f185affd640 1 -- 172.21.15.5:0/3052429442 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648d5c3db90 con 0x5648d5cea2e0 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f185affd640 10 client.17079 handle_command_reply: tid=0 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 1 client.17079 shutdown 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 1 -- 172.21.15.5:0/3052429442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 msgr2=0x5648d5cd36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cd36f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f1870011800 tx=0x7f1870011cc0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 1 -- 172.21.15.5:0/3052429442 shutdown_connections 2024-09-15T22:18:07.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5648d5cea2e0 0x5648d5cec720 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f185affd640 0 client.17079 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:07.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cd3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cd36f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 1 --2- 172.21.15.5:0/3052429442 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb060 0x5648d5cd31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 1 -- 172.21.15.5:0/3052429442 >> 172.21.15.5:0/3052429442 conn(0x5648d5cc75f0 msgr2=0x5648d5ccd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:07.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 1 -- 172.21.15.5:0/3052429442 shutdown_connections 2024-09-15T22:18:07.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 1 -- 172.21.15.5:0/3052429442 wait complete. 2024-09-15T22:18:07.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.593+0000 7f1884112740 20 client.17079 trim_cache size 0 max 16384 2024-09-15T22:18:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.617+0000 7f1884112740 1 Processor -- start 2024-09-15T22:18:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.617+0000 7f1884112740 1 -- start start 2024-09-15T22:18:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.617+0000 7f1884112740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb060 0x5648d5ccdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.617+0000 7f1884112740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cf1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f1884112740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cf2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f1884112740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f187c1af820 con 0x5648d5ccb980 2024-09-15T22:18:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f1884112740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f187c1ad0f0 con 0x5648d5cd6240 2024-09-15T22:18:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f187affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cf1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f187affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cf1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46134/0 (socket says 172.21.15.5:46134) 2024-09-15T22:18:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f187affd640 1 -- 172.21.15.5:0/3052736010 learned_addr learned my addr 172.21.15.5:0/3052736010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f187bfff640 1 --2- 172.21.15.5:0/3052736010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb060 0x5648d5ccdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f1884112740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f187c1aaf60 con 0x5648d5ccb060 2024-09-15T22:18:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f187bfff640 1 --2- 172.21.15.5:0/3052736010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cf2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f187affd640 1 -- 172.21.15.5:0/3052736010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb060 msgr2=0x5648d5ccdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f187affd640 1 --2- 172.21.15.5:0/3052736010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb060 0x5648d5ccdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f187affd640 1 -- 172.21.15.5:0/3052736010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 msgr2=0x5648d5cf2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:07.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.618+0000 7f187affd640 1 --2- 172.21.15.5:0/3052736010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cf2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.619+0000 7f187affd640 1 -- 172.21.15.5:0/3052736010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648d5cced70 con 0x5648d5ccb980 2024-09-15T22:18:07.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.619+0000 7f187bfff640 1 --2- 172.21.15.5:0/3052736010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cf2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:07.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.619+0000 7f187affd640 1 --2- 172.21.15.5:0/3052736010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cf1ff0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1870000c00 tx=0x7f1870005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d2f1a1b6a28580 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:07.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.619+0000 7f18617fa640 1 -- 172.21.15.5:0/3052736010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187002b070 con 0x5648d5ccb980 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.619+0000 7f18617fa640 1 -- 172.21.15.5:0/3052736010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f187000f790 con 0x5648d5ccb980 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.619+0000 7f18617fa640 1 -- 172.21.15.5:0/3052736010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1870027870 con 0x5648d5ccb980 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.620+0000 7f1884112740 1 -- 172.21.15.5:0/3052736010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 msgr2=0x5648d5cf1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.620+0000 7f1884112740 1 --2- 172.21.15.5:0/3052736010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cf1ff0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f1870000c00 tx=0x7f1870005a90 comp rx=0 tx=0).stop 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.620+0000 7f1884112740 1 -- 172.21.15.5:0/3052736010 shutdown_connections 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.620+0000 7f1884112740 1 --2- 172.21.15.5:0/3052736010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cf2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.620+0000 7f1884112740 1 --2- 172.21.15.5:0/3052736010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb980 0x5648d5cf1ff0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.620+0000 7f1884112740 1 --2- 172.21.15.5:0/3052736010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb060 0x5648d5ccdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.620+0000 7f1884112740 1 -- 172.21.15.5:0/3052736010 >> 172.21.15.5:0/3052736010 conn(0x5648d5cc75f0 msgr2=0x5648d5cd1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.620+0000 7f1884112740 1 -- 172.21.15.5:0/3052736010 shutdown_connections 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.620+0000 7f1884112740 1 -- 172.21.15.5:0/3052736010 wait complete. 2024-09-15T22:18:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.620+0000 7f1884112740 1 -- start start 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.621+0000 7f1884112740 1 Processor -- start 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.621+0000 7f1884112740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5cda1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.621+0000 7f1884112740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 0x5648d5cda6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.621+0000 7f1884112740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cdac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.621+0000 7f1884112740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f187c1ab5b0 con 0x5648d5ccb060 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.621+0000 7f1884112740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f187c1aca20 con 0x5648d5cd6240 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.621+0000 7f1884112740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f187c1abc80 con 0x5648d5ccb980 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.621+0000 7f1884112740 10 client.0 _ll_register_callbacks cb 0x7f1870011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.621+0000 7f187bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5cda1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.621+0000 7f187bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5cda1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46144/0 (socket says 172.21.15.5:46144) 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.621+0000 7f187bfff640 1 -- 172.21.15.5:0/4034688504 learned_addr learned my addr 172.21.15.5:0/4034688504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f187affd640 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 0x5648d5cda6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f187bfff640 1 -- 172.21.15.5:0/4034688504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 msgr2=0x5648d5cda6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f187bfff640 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 0x5648d5cda6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f187bfff640 1 -- 172.21.15.5:0/4034688504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 msgr2=0x5648d5cdac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f187bfff640 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cdac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f187bfff640 1 -- 172.21.15.5:0/4034688504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648d5d54210 con 0x5648d5ccb060 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f187affd640 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 0x5648d5cda6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f187bfff640 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5cda1a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f187400b6b0 tx=0x7f1874011680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d9e0b694f879f36 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f185affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f185affd640 1 -- 172.21.15.5:0/4034688504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187402d070 con 0x5648d5ccb060 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f1884112740 1 -- 172.21.15.5:0/4034688504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f187000f790 con 0x5648d5ccb060 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f185affd640 1 -- 172.21.15.5:0/4034688504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9541+0+0 (secure 0 0 0) 0x7f1874005300 con 0x5648d5ccb060 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f185affd640 1 -- 172.21.15.5:0/4034688504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f187403c050 con 0x5648d5ccb060 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.622+0000 7f1884112740 10 client.17085 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.623+0000 7f1884112740 1 -- 172.21.15.5:0/4034688504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5648d5d55600 con 0x5648d5ccb060 2024-09-15T22:18:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.623+0000 7f185affd640 1 -- 172.21.15.5:0/4034688504 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f187000f790 con 0x5648d5ccb060 2024-09-15T22:18:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.623+0000 7f185affd640 1 -- 172.21.15.5:0/4034688504 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f18740048b0 con 0x5648d5ccb060 2024-09-15T22:18:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.623+0000 7f1884112740 10 client.17085 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.623+0000 7f1884112740 10 client.17085 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:18:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.623+0000 7f1884112740 20 client.17085 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.623+0000 7f1884112740 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5648d5cea2e0 0x5648d5d57d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.623+0000 7f1884112740 4 client.17085 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "49f84031", "release": true, "await": true}] 2024-09-15T22:18:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.623+0000 7f1884112740 1 -- 172.21.15.5:0/4034688504 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "49f84031", "release": true, "await": true}) -- 0x5648d5c380a0 con 0x5648d5cea2e0 2024-09-15T22:18:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.623+0000 7f187affd640 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5648d5cea2e0 0x5648d5d57d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.624+0000 7f185affd640 10 client.17085 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.624+0000 7f187affd640 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5648d5cea2e0 0x5648d5d57d10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56f8aca7e43d405f server_cookie=38fef05a6eb0876c in_seq=0 out_seq=0 2024-09-15T22:18:07.681 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:18:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:18:07.681 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 32, 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "49f84031": { 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 32, 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.3, 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:18:07.682 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:18:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:18:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:18:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:18:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:18:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:07.683 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:07.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:07.684 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:18:07.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f185affd640 1 -- 172.21.15.5:0/4034688504 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5648d5c380a0 con 0x5648d5cea2e0 2024-09-15T22:18:07.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f185affd640 10 client.17085 handle_command_reply: tid=0 2024-09-15T22:18:07.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f1884112740 1 client.17085 shutdown 2024-09-15T22:18:07.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f1884112740 1 -- 172.21.15.5:0/4034688504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 msgr2=0x5648d5cda1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:07.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f1884112740 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5cda1a0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f187400b6b0 tx=0x7f1874011680 comp rx=0 tx=0).stop 2024-09-15T22:18:07.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f1884112740 1 -- 172.21.15.5:0/4034688504 shutdown_connections 2024-09-15T22:18:07.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f1884112740 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5648d5cea2e0 0x5648d5d57d10 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f1884112740 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5648d5cd6240 0x5648d5cdac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f185affd640 0 client.17085 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:07.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f1884112740 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5648d5ccb980 0x5648d5cda6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f1884112740 1 --2- 172.21.15.5:0/4034688504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648d5ccb060 0x5648d5cda1a0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f1884112740 1 -- 172.21.15.5:0/4034688504 >> 172.21.15.5:0/4034688504 conn(0x5648d5cc75f0 msgr2=0x5648d5cc5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f1884112740 1 -- 172.21.15.5:0/4034688504 shutdown_connections 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.678+0000 7f1884112740 1 -- 172.21.15.5:0/4034688504 wait complete. 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.679+0000 7f1884112740 20 client.17085 trim_cache size 0 max 16384 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.679+0000 7f1881d39640 1 -- 172.21.15.5:0/1664788385 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18640a3430 msgr2=0x7f18640a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.679+0000 7f1881d39640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18640a3430 0x7f18640a58f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f186c0eeac0 tx=0x7f186c0c0040 comp rx=0 tx=0).stop 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.679+0000 7f1881d39640 1 -- 172.21.15.5:0/1664788385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c100a90 msgr2=0x7f187c199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.679+0000 7f1881d39640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c100a90 0x7f187c199300 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f186c02ae00 tx=0x7f186c003970 comp rx=0 tx=0).stop 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.680+0000 7f1881d39640 1 -- 172.21.15.5:0/1664788385 shutdown_connections 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.680+0000 7f1881d39640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18640a3430 0x7f18640a58f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.680+0000 7f1881d39640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f187c106a90 0x7f187c1a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.680+0000 7f1881d39640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f187c101430 0x7f187c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.680+0000 7f1881d39640 1 --2- 172.21.15.5:0/1664788385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f187c100a90 0x7f187c199300 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:07.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.680+0000 7f1881d39640 1 -- 172.21.15.5:0/1664788385 >> 172.21.15.5:0/1664788385 conn(0x7f187c0fc7d0 msgr2=0x7f187c0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.680+0000 7f1881d39640 1 -- 172.21.15.5:0/1664788385 shutdown_connections 2024-09-15T22:18:07.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:07.680+0000 7f1881d39640 1 -- 172.21.15.5:0/1664788385 wait complete. 2024-09-15T22:18:08.660 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '49f84031', total seconds elapsed: 26.6 2024-09-15T22:18:08.660 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.35 2024-09-15T22:18:08.660 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:18:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.926+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1540575335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c011b140 msgr2=0x7fc0c011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:08.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.926+0000 7fc0bf59e640 1 --2- 172.21.15.5:0/1540575335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c011b140 0x7fc0c011d530 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0009be0 tx=0x7fc0b002a710 comp rx=0 tx=0).stop 2024-09-15T22:18:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.929+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1540575335 shutdown_connections 2024-09-15T22:18:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.929+0000 7fc0bf59e640 1 --2- 172.21.15.5:0/1540575335 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c011b140 0x7fc0c011d530 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.929+0000 7fc0bf59e640 1 --2- 172.21.15.5:0/1540575335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0c006ea30 0x7fc0c006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.929+0000 7fc0bf59e640 1 --2- 172.21.15.5:0/1540575335 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0c0110190 0x7fc0c006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.929+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1540575335 >> 172.21.15.5:0/1540575335 conn(0x7fc0c0073180 msgr2=0x7fc0c0073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:08.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.932+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1540575335 shutdown_connections 2024-09-15T22:18:08.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.932+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1540575335 wait complete. 2024-09-15T22:18:08.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.933+0000 7fc0bf59e640 1 Processor -- start 2024-09-15T22:18:08.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.933+0000 7fc0bf59e640 1 -- start start 2024-09-15T22:18:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.933+0000 7fc0bf59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c006ea30 0x7fc0c0114450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.933+0000 7fc0bf59e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0c0110190 0x7fc0c0114990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.933+0000 7fc0bf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0c011b140 0x7fc0c0118d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.933+0000 7fc0bf59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0110cb0 con 0x7fc0c011b140 2024-09-15T22:18:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.933+0000 7fc0bf59e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0110b30 con 0x7fc0c006ea30 2024-09-15T22:18:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.933+0000 7fc0bf59e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c0110e30 con 0x7fc0c0110190 2024-09-15T22:18:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.935+0000 7fc0bed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0c011b140 0x7fc0c0118d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.935+0000 7fc0bed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0c011b140 0x7fc0c0118d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46168/0 (socket says 172.21.15.5:46168) 2024-09-15T22:18:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.935+0000 7fc0bed9d640 1 -- 172.21.15.5:0/1696344030 learned_addr learned my addr 172.21.15.5:0/1696344030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:08.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.936+0000 7fc0bed9d640 1 -- 172.21.15.5:0/1696344030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0c0110190 msgr2=0x7fc0c0114990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.936+0000 7fc0bed9d640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0c0110190 0x7fc0c0114990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.936+0000 7fc0bed9d640 1 -- 172.21.15.5:0/1696344030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c006ea30 msgr2=0x7fc0c0114450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:18:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.936+0000 7fc0bed9d640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c006ea30 0x7fc0c0114450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:08.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.936+0000 7fc0bed9d640 1 -- 172.21.15.5:0/1696344030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c0119360 con 0x7fc0c011b140 2024-09-15T22:18:08.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.942+0000 7fc0bed9d640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0c011b140 0x7fc0c0118d20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0006010 tx=0x7fc0b0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:08.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.945+0000 7fc0b77fe640 1 -- 172.21.15.5:0/1696344030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0041020 con 0x7fc0c011b140 2024-09-15T22:18:08.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.945+0000 7fc0b77fe640 1 -- 172.21.15.5:0/1696344030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0b0030c80 con 0x7fc0c011b140 2024-09-15T22:18:08.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.945+0000 7fc0b77fe640 1 -- 172.21.15.5:0/1696344030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0041020 con 0x7fc0c011b140 2024-09-15T22:18:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.945+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1696344030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c0119590 con 0x7fc0c011b140 2024-09-15T22:18:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.946+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1696344030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c0119a20 con 0x7fc0c011b140 2024-09-15T22:18:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.947+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1696344030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c01096f0 con 0x7fc0c011b140 2024-09-15T22:18:08.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.954+0000 7fc0b77fe640 1 -- 172.21.15.5:0/1696344030 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc0b0044020 con 0x7fc0c011b140 2024-09-15T22:18:08.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.955+0000 7fc0b77fe640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0ac0a3830 0x7fc0ac0a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:08.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.956+0000 7fc0b77fe640 1 -- 172.21.15.5:0/1696344030 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9629+0+0 (secure 0 0 0) 0x7fc0b0030a80 con 0x7fc0c011b140 2024-09-15T22:18:08.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.956+0000 7fc0be59c640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0ac0a3830 0x7fc0ac0a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.957+0000 7fc0be59c640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0ac0a3830 0x7fc0ac0a5cf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009670 tx=0x7fc0b8006bd0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:08.961+0000 7fc0b77fe640 1 -- 172.21.15.5:0/1696344030 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0b004b070 con 0x7fc0c011b140 2024-09-15T22:18:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:09 smithi099 ceph-mon[34892]: pgmap v284: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 55 KiB/s wr, 25 op/s 2024-09-15T22:18:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:09 smithi099 ceph-mon[34892]: osdmap e94: 12 total, 12 up, 12 in 2024-09-15T22:18:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:09 smithi129 ceph-mon[35314]: pgmap v284: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 55 KiB/s wr, 25 op/s 2024-09-15T22:18:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:09 smithi129 ceph-mon[35314]: osdmap e94: 12 total, 12 up, 12 in 2024-09-15T22:18:09.349 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:09 smithi005 ceph-mon[30944]: pgmap v284: 129 pgs: 129 active+clean; 3.2 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 55 KiB/s wr, 25 op/s 2024-09-15T22:18:09.349 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:09 smithi005 ceph-mon[30944]: osdmap e94: 12 total, 12 up, 12 in 2024-09-15T22:18:09.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.382+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1696344030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0c006ee70 con 0x7fc0c011b140 2024-09-15T22:18:09.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.386+0000 7fc0b77fe640 1 -- 172.21.15.5:0/1696344030 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fc0c006ee70 con 0x7fc0c011b140 2024-09-15T22:18:09.561 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:18:09.561 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:18:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:18:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.392+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1696344030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0ac0a3830 msgr2=0x7fc0ac0a5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.392+0000 7fc0bf59e640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0ac0a3830 0x7fc0ac0a5cf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009670 tx=0x7fc0b8006bd0 comp rx=0 tx=0).stop 2024-09-15T22:18:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.392+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1696344030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0c011b140 msgr2=0x7fc0c0118d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.392+0000 7fc0bf59e640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0c011b140 0x7fc0c0118d20 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0006010 tx=0x7fc0b0031a50 comp rx=0 tx=0).stop 2024-09-15T22:18:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.393+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1696344030 shutdown_connections 2024-09-15T22:18:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.393+0000 7fc0bf59e640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc0ac0a3830 0x7fc0ac0a5cf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.393+0000 7fc0bf59e640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0c011b140 0x7fc0c0118d20 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.393+0000 7fc0bf59e640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc0c0110190 0x7fc0c0114990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.393+0000 7fc0bf59e640 1 --2- 172.21.15.5:0/1696344030 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc0c006ea30 0x7fc0c0114450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.393+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1696344030 >> 172.21.15.5:0/1696344030 conn(0x7fc0c0073180 msgr2=0x7fc0c011b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.393+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1696344030 shutdown_connections 2024-09-15T22:18:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:09.393+0000 7fc0bf59e640 1 -- 172.21.15.5:0/1696344030 wait complete. 2024-09-15T22:18:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 520.0 s, 138.7 tps, lat 2728.533 ms stddev 6178.356 2024-09-15T22:18:10.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:18:10.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:10 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1696344030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:10.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:10 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1696344030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:10.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:10 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1696344030' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:10.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.448+0000 7f2606ede640 1 -- 172.21.15.5:0/4161080623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2600110190 msgr2=0x7f260006e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:10.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.448+0000 7f2606ede640 1 --2- 172.21.15.5:0/4161080623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2600110190 0x7f260006e4f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25fc01a910 tx=0x7f25fc03b470 comp rx=0 tx=0).stop 2024-09-15T22:18:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.450+0000 7f2606ede640 1 -- 172.21.15.5:0/4161080623 shutdown_connections 2024-09-15T22:18:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.450+0000 7f2606ede640 1 --2- 172.21.15.5:0/4161080623 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f260011b140 0x7f260011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.450+0000 7f2606ede640 1 --2- 172.21.15.5:0/4161080623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f260006ea30 0x7f260006ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.450+0000 7f2606ede640 1 --2- 172.21.15.5:0/4161080623 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2600110190 0x7f260006e4f0 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25fc01a910 tx=0x7f25fc03b470 comp rx=0 tx=0).stop 2024-09-15T22:18:10.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.450+0000 7f2606ede640 1 -- 172.21.15.5:0/4161080623 >> 172.21.15.5:0/4161080623 conn(0x7f2600073180 msgr2=0x7f2600073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:10.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.456+0000 7f2606ede640 1 -- 172.21.15.5:0/4161080623 shutdown_connections 2024-09-15T22:18:10.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.457+0000 7f2606ede640 1 -- 172.21.15.5:0/4161080623 wait complete. 2024-09-15T22:18:10.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.458+0000 7f2606ede640 1 Processor -- start 2024-09-15T22:18:10.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.458+0000 7f2606ede640 1 -- start start 2024-09-15T22:18:10.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.458+0000 7f2606ede640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260011b140 0x7f260011abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.459+0000 7f2605edc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260011b140 0x7f260011abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.459+0000 7f2605edc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260011b140 0x7f260011abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:40706/0 (socket says 172.21.15.5:40706) 2024-09-15T22:18:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.459+0000 7f2606ede640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2600116c50 0x7f26001170b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:10.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.459+0000 7f2605edc640 1 -- 172.21.15.5:0/2801560779 learned_addr learned my addr 172.21.15.5:0/2801560779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.459+0000 7f2606ede640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26001176b0 0x7f26001ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.459+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2600113380 con 0x7f2600116c50 2024-09-15T22:18:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.459+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2600113200 con 0x7f260011b140 2024-09-15T22:18:10.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.459+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2600113500 con 0x7f26001176b0 2024-09-15T22:18:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.461+0000 7f26056db640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2600116c50 0x7f26001170b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.461+0000 7f26066dd640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26001176b0 0x7f26001ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.461+0000 7f2605edc640 1 -- 172.21.15.5:0/2801560779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26001176b0 msgr2=0x7f26001ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:10.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.462+0000 7f2605edc640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26001176b0 0x7f26001ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.462+0000 7f2605edc640 1 -- 172.21.15.5:0/2801560779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2600116c50 msgr2=0x7f26001170b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.462+0000 7f2605edc640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2600116c50 0x7f26001170b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:10.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.462+0000 7f2605edc640 1 -- 172.21.15.5:0/2801560779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26001cd640 con 0x7f260011b140 2024-09-15T22:18:10.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.463+0000 7f2605edc640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260011b140 0x7f260011abb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f25fc03b9e0 tx=0x7f25fc004e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.464+0000 7f25f6ffd640 1 -- 172.21.15.5:0/2801560779 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc012070 con 0x7f260011b140 2024-09-15T22:18:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.464+0000 7f25f6ffd640 1 -- 172.21.15.5:0/2801560779 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25fc0035f0 con 0x7f260011b140 2024-09-15T22:18:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.464+0000 7f25f6ffd640 1 -- 172.21.15.5:0/2801560779 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc040770 con 0x7f260011b140 2024-09-15T22:18:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.465+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26001cd8d0 con 0x7f260011b140 2024-09-15T22:18:10.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.465+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26001cddc0 con 0x7f260011b140 2024-09-15T22:18:10.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.466+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e4005290 con 0x7f260011b140 2024-09-15T22:18:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.469+0000 7f25f6ffd640 1 -- 172.21.15.5:0/2801560779 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f25fc049020 con 0x7f260011b140 2024-09-15T22:18:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.470+0000 7f25f6ffd640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25ec0a3540 0x7f25ec0a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.470+0000 7f25f6ffd640 1 -- 172.21.15.5:0/2801560779 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9629+0+0 (secure 0 0 0) 0x7f25fc0ba8f0 con 0x7f260011b140 2024-09-15T22:18:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.471+0000 7f26056db640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25ec0a3540 0x7f25ec0a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.472+0000 7f26056db640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25ec0a3540 0x7f25ec0a5a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f25f0005fd0 tx=0x7f25f0005850 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.472+0000 7f25f6ffd640 1 -- 172.21.15.5:0/2801560779 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25fc04a3a0 con 0x7f260011b140 2024-09-15T22:18:10.741 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.876+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25e4005580 con 0x7f260011b140 2024-09-15T22:18:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.877+0000 7f25f6ffd640 1 -- 172.21.15.5:0/2801560779 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f25fc0bcf50 con 0x7f260011b140 2024-09-15T22:18:10.879 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:18:10.879 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:18:10.880 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:18:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25ec0a3540 msgr2=0x7f25ec0a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25ec0a3540 0x7f25ec0a5a00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f25f0005fd0 tx=0x7f25f0005850 comp rx=0 tx=0).stop 2024-09-15T22:18:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260011b140 msgr2=0x7f260011abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260011b140 0x7f260011abb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f25fc03b9e0 tx=0x7f25fc004e10 comp rx=0 tx=0).stop 2024-09-15T22:18:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 shutdown_connections 2024-09-15T22:18:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f25ec0a3540 0x7f25ec0a5a00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f26001176b0 0x7f26001ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2600116c50 0x7f26001170b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 --2- 172.21.15.5:0/2801560779 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f260011b140 0x7f260011abb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 >> 172.21.15.5:0/2801560779 conn(0x7f2600073180 msgr2=0x7f260011b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 shutdown_connections 2024-09-15T22:18:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:10.880+0000 7f2606ede640 1 -- 172.21.15.5:0/2801560779 wait complete. 2024-09-15T22:18:11.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:11 smithi099 ceph-mon[34892]: pgmap v286: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-09-15T22:18:11.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:11 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2801560779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:11.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:11 smithi129 ceph-mon[35314]: pgmap v286: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-09-15T22:18:11.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:11 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2801560779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:11.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:11 smithi005 ceph-mon[30944]: pgmap v286: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 3.1 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 117 KiB/s wr, 35 op/s 2024-09-15T22:18:11.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:11 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2801560779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:11.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.514+0000 7ffa1359e640 1 -- 172.21.15.5:0/3137921377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa141068d0 msgr2=0x7ffa14106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.514+0000 7ffa1359e640 1 --2- 172.21.15.5:0/3137921377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa141068d0 0x7ffa14106cb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009ca0 tx=0x7ffa0c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.514+0000 7ffa1359e640 1 -- 172.21.15.5:0/3137921377 shutdown_connections 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.514+0000 7ffa1359e640 1 --2- 172.21.15.5:0/3137921377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa14101270 0x7ffa1410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.514+0000 7ffa1359e640 1 --2- 172.21.15.5:0/3137921377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa141008d0 0x7ffa14100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.514+0000 7ffa1359e640 1 --2- 172.21.15.5:0/3137921377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa141068d0 0x7ffa14106cb0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.514+0000 7ffa1359e640 1 -- 172.21.15.5:0/3137921377 >> 172.21.15.5:0/3137921377 conn(0x7ffa140fc630 msgr2=0x7ffa140fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.514+0000 7ffa1359e640 1 -- 172.21.15.5:0/3137921377 shutdown_connections 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.514+0000 7ffa1359e640 1 -- 172.21.15.5:0/3137921377 wait complete. 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.515+0000 7ffa1359e640 1 Processor -- start 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.515+0000 7ffa1359e640 1 -- start start 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.515+0000 7ffa1359e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa141008d0 0x7ffa14198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.515+0000 7ffa1359e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa14101270 0x7ffa141987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.515+0000 7ffa1359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa141068d0 0x7ffa1419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.515+0000 7ffa1259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa141008d0 0x7ffa14198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.515+0000 7ffa1259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa141008d0 0x7ffa14198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:41944/0 (socket says 172.21.15.5:41944) 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa1259c640 1 -- 172.21.15.5:0/1673123372 learned_addr learned my addr 172.21.15.5:0/1673123372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa1359e640 1 -- 172.21.15.5:0/1673123372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14111e30 con 0x7ffa141068d0 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa1359e640 1 -- 172.21.15.5:0/1673123372 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14111cb0 con 0x7ffa14101270 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa1359e640 1 -- 172.21.15.5:0/1673123372 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffa14111fb0 con 0x7ffa141008d0 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa12d9d640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa141068d0 0x7ffa1419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa14101270 0x7ffa141987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa1259c640 1 -- 172.21.15.5:0/1673123372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa14101270 msgr2=0x7ffa141987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa1259c640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa14101270 0x7ffa141987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa1259c640 1 -- 172.21.15.5:0/1673123372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa141068d0 msgr2=0x7ffa1419cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:11.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa1259c640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa141068d0 0x7ffa1419cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa1259c640 1 -- 172.21.15.5:0/1673123372 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1419d220 con 0x7ffa141008d0 2024-09-15T22:18:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.516+0000 7ffa1259c640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa141008d0 0x7ffa14198270 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c000f80 tx=0x7ffa0c003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.517+0000 7ffa037fe640 1 -- 172.21.15.5:0/1673123372 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c031650 con 0x7ffa141008d0 2024-09-15T22:18:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.517+0000 7ffa037fe640 1 -- 172.21.15.5:0/1673123372 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa0c0317f0 con 0x7ffa141008d0 2024-09-15T22:18:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.517+0000 7ffa1359e640 1 -- 172.21.15.5:0/1673123372 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1419d4b0 con 0x7ffa141008d0 2024-09-15T22:18:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.517+0000 7ffa1359e640 1 -- 172.21.15.5:0/1673123372 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa141a4df0 con 0x7ffa141008d0 2024-09-15T22:18:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.517+0000 7ffa037fe640 1 -- 172.21.15.5:0/1673123372 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c02faa0 con 0x7ffa141008d0 2024-09-15T22:18:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.522+0000 7ffa1af59740 1 Processor -- start 2024-09-15T22:18:11.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.522+0000 7ffa1af59740 1 -- start start 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.522+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43dddcf0 0x563e43ddc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.522+0000 7ffa037fe640 1 -- 172.21.15.5:0/1673123372 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ffa0c02fc40 con 0x7ffa141008d0 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.522+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43ddcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.522+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43deb930 0x563e43dedd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.522+0000 7ffa1af59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e43d422f0 con 0x563e43dddcf0 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.522+0000 7ffa1af59740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563e43ce9cf0 con 0x563e43ddc780 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.522+0000 7ffa1af59740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563e43d3e2b0 con 0x563e43deb930 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.523+0000 7ffa12d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43dddcf0 0x563e43ddc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.523+0000 7ffa12d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43dddcf0 0x563e43ddc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46222/0 (socket says 172.21.15.5:46222) 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.523+0000 7ffa12d9d640 1 -- 172.21.15.5:0/1877464849 learned_addr learned my addr 172.21.15.5:0/1877464849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.523+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/1877464849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43deb930 0x563e43dedd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.523+0000 7ffa1259c640 1 --2- 172.21.15.5:0/1877464849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43ddcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.523+0000 7ffa11d9b640 1 -- 172.21.15.5:0/1877464849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 msgr2=0x563e43ddcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.523+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/1877464849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43ddcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.523+0000 7ffa11d9b640 1 -- 172.21.15.5:0/1877464849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43dddcf0 msgr2=0x563e43ddc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.523+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/1877464849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43dddcf0 0x563e43ddc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.523+0000 7ffa11d9b640 1 -- 172.21.15.5:0/1877464849 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e43cce3a0 con 0x563e43deb930 2024-09-15T22:18:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.524+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/1877464849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43deb930 0x563e43dedd20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffa04005fd0 tx=0x7ffa0400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfd79b49c8cc935d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.524+0000 7ffa00ff9640 1 -- 172.21.15.5:0/1877464849 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400dea0 con 0x563e43deb930 2024-09-15T22:18:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.524+0000 7ffa00ff9640 1 -- 172.21.15.5:0/1877464849 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa04005500 con 0x563e43deb930 2024-09-15T22:18:11.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.524+0000 7ffa00ff9640 1 -- 172.21.15.5:0/1877464849 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa040116a0 con 0x563e43deb930 2024-09-15T22:18:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.525+0000 7ffa037fe640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff9fc0a3430 0x7ff9fc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.525+0000 7ffa1af59740 1 -- 172.21.15.5:0/1877464849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43deb930 msgr2=0x563e43dedd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.525+0000 7ffa1af59740 1 --2- 172.21.15.5:0/1877464849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43deb930 0x563e43dedd20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffa04005fd0 tx=0x7ffa0400f730 comp rx=0 tx=0).stop 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.525+0000 7ffa12d9d640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff9fc0a3430 0x7ff9fc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.525+0000 7ffa037fe640 1 -- 172.21.15.5:0/1673123372 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9629+0+0 (secure 0 0 0) 0x7ffa0c0f1440 con 0x7ffa141008d0 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.525+0000 7ffa1af59740 1 -- 172.21.15.5:0/1877464849 shutdown_connections 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.525+0000 7ffa1af59740 1 --2- 172.21.15.5:0/1877464849 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43deb930 0x563e43dedd20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.525+0000 7ffa1af59740 1 --2- 172.21.15.5:0/1877464849 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43ddcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.525+0000 7ffa1af59740 1 --2- 172.21.15.5:0/1877464849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43dddcf0 0x563e43ddc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.525+0000 7ffa1af59740 1 -- 172.21.15.5:0/1877464849 >> 172.21.15.5:0/1877464849 conn(0x563e43da5310 msgr2=0x563e43dd8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.525+0000 7ffa1af59740 1 -- 172.21.15.5:0/1877464849 shutdown_connections 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.526+0000 7ffa1af59740 1 -- 172.21.15.5:0/1877464849 wait complete. 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.526+0000 7ffa12d9d640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff9fc0a3430 0x7ff9fc0a58f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009e20 tx=0x7ffa08005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.526+0000 7ffa1af59740 1 -- start start 2024-09-15T22:18:11.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.526+0000 7ffa1af59740 1 Processor -- start 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.526+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43deb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.526+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43de8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.526+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43deb930 0x563e43de8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.526+0000 7ffa1af59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e43dee630 con 0x563e43deb930 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.526+0000 7ffa1af59740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563e43cfd060 con 0x563e43ddc780 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.526+0000 7ffa1af59740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563e43dee7b0 con 0x563e43dddcf0 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa1af59740 10 client.0 _ll_register_callbacks cb 0x563e43a508c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa12d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43deb930 0x563e43de8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa1259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43deb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa12d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43deb930 0x563e43de8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46230/0 (socket says 172.21.15.5:46230) 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa12d9d640 1 -- 172.21.15.5:0/3494565276 learned_addr learned my addr 172.21.15.5:0/3494565276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:11.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43de8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa1259c640 1 -- 172.21.15.5:0/3494565276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 msgr2=0x563e43de8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa1259c640 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43de8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa1259c640 1 -- 172.21.15.5:0/3494565276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43deb930 msgr2=0x563e43de8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa1259c640 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43deb930 0x563e43de8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa1259c640 1 -- 172.21.15.5:0/3494565276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e43ab33c0 con 0x563e43ddc780 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ffa1259c640 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43deb2b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0ee850 tx=0x7ffa0c02ffe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb102823f738e3f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.527+0000 7ff9f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3494565276 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c0bcd40 con 0x563e43ddc780 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3494565276 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9629+0+0 (secure 0 0 0) 0x7ffa0c0b9a50 con 0x563e43ddc780 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ffa1af59740 1 -- 172.21.15.5:0/3494565276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa04005500 con 0x563e43ddc780 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3494565276 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa0c0319a0 con 0x563e43ddc780 2024-09-15T22:18:11.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ffa1af59740 10 client.26624 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ffa1af59740 1 -- 172.21.15.5:0/3494565276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563e43dee630 con 0x563e43ddc780 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3494565276 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa04005500 con 0x563e43ddc780 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3494565276 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ffa0c0d6230 con 0x563e43ddc780 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ffa1af59740 10 client.26624 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ffa1af59740 10 client.26624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ffa1af59740 20 client.26624 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563e43dfb480 0x563e43dfd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ffa1af59740 4 client.26624 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ffa1af59740 1 -- 172.21.15.5:0/3494565276 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e43d4f860 con 0x563e43dfb480 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.528+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563e43dfb480 0x563e43dfd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.529+0000 7ff9f9ffb640 10 client.26624 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.529+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563e43dfb480 0x563e43dfd8d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8f63f86e9b08ba11 server_cookie=9f26d71fe151fe0b in_seq=0 out_seq=0 2024-09-15T22:18:11.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3494565276 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e43d4f860 con 0x563e43dfb480 2024-09-15T22:18:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ff9f9ffb640 10 client.26624 handle_command_reply: tid=0 2024-09-15T22:18:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 1 client.26624 shutdown 2024-09-15T22:18:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 1 -- 172.21.15.5:0/3494565276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 msgr2=0x563e43deb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43deb2b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c0ee850 tx=0x7ffa0c02ffe0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 1 -- 172.21.15.5:0/3494565276 shutdown_connections 2024-09-15T22:18:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563e43dfb480 0x563e43dfd8d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43deb930 0x563e43de8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43de8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3494565276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43deb2b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 1 -- 172.21.15.5:0/3494565276 >> 172.21.15.5:0/3494565276 conn(0x563e43da5310 msgr2=0x563e43ddeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ff9f9ffb640 0 client.26624 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:18:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 1 -- 172.21.15.5:0/3494565276 shutdown_connections 2024-09-15T22:18:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 1 -- 172.21.15.5:0/3494565276 wait complete. 2024-09-15T22:18:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.530+0000 7ffa1af59740 20 client.26624 trim_cache size 0 max 16384 2024-09-15T22:18:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.547+0000 7ffa1af59740 1 Processor -- start 2024-09-15T22:18:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa1af59740 1 -- start start 2024-09-15T22:18:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43ddc780 0x563e43df78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43e031a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43deb930 0x563e43e036e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa1af59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141af710 con 0x563e43ddc780 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa1af59740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141acfe0 con 0x563e43deb930 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa1af59740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141aae50 con 0x563e43dddcf0 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa1259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43ddc780 0x563e43df78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa11d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43e031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa11d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43e031a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:41974/0 (socket says 172.21.15.5:41974) 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa11d9b640 1 -- 172.21.15.5:0/555581165 learned_addr learned my addr 172.21.15.5:0/555581165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa12d9d640 1 --2- 172.21.15.5:0/555581165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43deb930 0x563e43e036e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa11d9b640 1 -- 172.21.15.5:0/555581165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43deb930 msgr2=0x563e43e036e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/555581165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43deb930 0x563e43e036e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa11d9b640 1 -- 172.21.15.5:0/555581165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43ddc780 msgr2=0x563e43df78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/555581165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43ddc780 0x563e43df78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa11d9b640 1 -- 172.21.15.5:0/555581165 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e43dd9080 con 0x563e43dddcf0 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.548+0000 7ffa12d9d640 1 --2- 172.21.15.5:0/555581165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43deb930 0x563e43e036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.549+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/555581165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43e031a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffa04005fd0 tx=0x7ffa0400d500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1bcb928b350a6d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.549+0000 7ff9fbfff640 1 -- 172.21.15.5:0/555581165 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa0400dea0 con 0x563e43dddcf0 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.549+0000 7ff9fbfff640 1 -- 172.21.15.5:0/555581165 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563e43d4f860 con 0x563e43dddcf0 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.549+0000 7ff9fbfff640 1 -- 172.21.15.5:0/555581165 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04003900 con 0x563e43dddcf0 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.550+0000 7ffa1af59740 1 -- 172.21.15.5:0/555581165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 msgr2=0x563e43e031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.550+0000 7ffa1af59740 1 --2- 172.21.15.5:0/555581165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43e031a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffa04005fd0 tx=0x7ffa0400d500 comp rx=0 tx=0).stop 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.550+0000 7ffa1af59740 1 -- 172.21.15.5:0/555581165 shutdown_connections 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.550+0000 7ffa1af59740 1 --2- 172.21.15.5:0/555581165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43deb930 0x563e43e036e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.550+0000 7ffa1af59740 1 --2- 172.21.15.5:0/555581165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43e031a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.550+0000 7ffa1af59740 1 --2- 172.21.15.5:0/555581165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43ddc780 0x563e43df78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.550+0000 7ffa1af59740 1 -- 172.21.15.5:0/555581165 >> 172.21.15.5:0/555581165 conn(0x563e43da5310 msgr2=0x563e43de38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.550+0000 7ffa1af59740 1 -- 172.21.15.5:0/555581165 shutdown_connections 2024-09-15T22:18:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.550+0000 7ffa1af59740 1 -- 172.21.15.5:0/555581165 wait complete. 2024-09-15T22:18:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.550+0000 7ffa1af59740 1 -- start start 2024-09-15T22:18:11.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.551+0000 7ffa1af59740 1 Processor -- start 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.551+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43df1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.551+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43df1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.551+0000 7ffa1af59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43deb930 0x563e43df22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.551+0000 7ffa11d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43df1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa11d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43df1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:41988/0 (socket says 172.21.15.5:41988) 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa11d9b640 1 -- 172.21.15.5:0/3546966496 learned_addr learned my addr 172.21.15.5:0/3546966496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa12d9d640 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43deb930 0x563e43df22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa1259c640 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43df1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa1af59740 1 -- 172.21.15.5:0/3546966496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141ab4a0 con 0x563e43deb930 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa1af59740 1 -- 172.21.15.5:0/3546966496 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141ac910 con 0x563e43ddc780 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa1af59740 1 -- 172.21.15.5:0/3546966496 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ffa141abb70 con 0x563e43dddcf0 2024-09-15T22:18:11.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa1af59740 10 client.0 _ll_register_callbacks cb 0x7ffa0c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa11d9b640 1 -- 172.21.15.5:0/3546966496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 msgr2=0x563e43df1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43df1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa11d9b640 1 -- 172.21.15.5:0/3546966496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43deb930 msgr2=0x563e43df22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43deb930 0x563e43df22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa11d9b640 1 -- 172.21.15.5:0/3546966496 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e43de3380 con 0x563e43dddcf0 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.552+0000 7ffa1259c640 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43df1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ffa11d9b640 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43df1d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400db30 tx=0x7ffa0400d720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8d7ed0decbb7ad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ff9f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3546966496 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04004880 con 0x563e43dddcf0 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ffa1af59740 1 -- 172.21.15.5:0/3546966496 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563e43d4f860 con 0x563e43dddcf0 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3546966496 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9629+0+0 (secure 0 0 0) 0x7ffa04004ce0 con 0x563e43dddcf0 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3546966496 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffa04044050 con 0x563e43dddcf0 2024-09-15T22:18:11.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ffa1af59740 10 client.26569 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ffa1af59740 1 -- 172.21.15.5:0/3546966496 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563e43e6abd0 con 0x563e43dddcf0 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3546966496 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563e43d4f860 con 0x563e43dddcf0 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3546966496 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ffa0403b700 con 0x563e43dddcf0 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ffa1af59740 10 client.26569 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ffa1af59740 10 client.26569 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ffa1af59740 20 client.26569 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563e43e6b130 0x563e43e6d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ffa1af59740 4 client.26569 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.553+0000 7ffa1af59740 1 -- 172.21.15.5:0/3546966496 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e43d49d70 con 0x563e43e6b130 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.554+0000 7ffa1259c640 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563e43e6b130 0x563e43e6d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ff9f9ffb640 10 client.26569 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ffa1259c640 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563e43e6b130 0x563e43e6d550 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe437d7608b9beb4 server_cookie=131a38e3f0b17327 in_seq=0 out_seq=0 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ff9f9ffb640 1 -- 172.21.15.5:0/3546966496 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e43d49d70 con 0x563e43e6b130 2024-09-15T22:18:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ff9f9ffb640 10 client.26569 handle_command_reply: tid=0 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ffa1af59740 1 client.26569 shutdown 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ffa1af59740 1 -- 172.21.15.5:0/3546966496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 msgr2=0x563e43df1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43df1d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffa0400db30 tx=0x7ffa0400d720 comp rx=0 tx=0).stop 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ffa1af59740 1 -- 172.21.15.5:0/3546966496 shutdown_connections 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563e43e6b130 0x563e43e6d550 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e43deb930 0x563e43df22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563e43dddcf0 0x563e43df1d80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ffa1af59740 1 --2- 172.21.15.5:0/3546966496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563e43ddc780 0x563e43df1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.555+0000 7ffa1af59740 1 -- 172.21.15.5:0/3546966496 >> 172.21.15.5:0/3546966496 conn(0x563e43da5310 msgr2=0x563e43dd7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.556+0000 7ff9f9ffb640 0 client.26569 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.556+0000 7ffa1af59740 1 -- 172.21.15.5:0/3546966496 shutdown_connections 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.556+0000 7ffa1af59740 1 -- 172.21.15.5:0/3546966496 wait complete. 2024-09-15T22:18:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.556+0000 7ffa1af59740 20 client.26569 trim_cache size 0 max 16384 2024-09-15T22:18:11.563 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:18:11.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 -- 172.21.15.5:0/1673123372 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff9fc0a3430 msgr2=0x7ff9fc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff9fc0a3430 0x7ff9fc0a58f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009e20 tx=0x7ffa08005f90 comp rx=0 tx=0).stop 2024-09-15T22:18:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 -- 172.21.15.5:0/1673123372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa141008d0 msgr2=0x7ffa14198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa141008d0 0x7ffa14198270 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c000f80 tx=0x7ffa0c003f10 comp rx=0 tx=0).stop 2024-09-15T22:18:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 -- 172.21.15.5:0/1673123372 shutdown_connections 2024-09-15T22:18:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff9fc0a3430 0x7ff9fc0a58f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa141068d0 0x7ffa1419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ffa14101270 0x7ffa141987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 --2- 172.21.15.5:0/1673123372 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ffa141008d0 0x7ffa14198270 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 -- 172.21.15.5:0/1673123372 >> 172.21.15.5:0/1673123372 conn(0x7ffa140fc630 msgr2=0x7ffa140fcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 -- 172.21.15.5:0/1673123372 shutdown_connections 2024-09-15T22:18:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:11.557+0000 7ffa1359e640 1 -- 172.21.15.5:0/1673123372 wait complete. 2024-09-15T22:18:11.942 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0' 2024-09-15T22:18:11.942 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 142.35 --await-for 45.0 2024-09-15T22:18:12.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:18:12.912 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:13.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:13 smithi099 ceph-mon[34892]: pgmap v287: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-09-15T22:18:13.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:13 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:18:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:13 smithi129 ceph-mon[35314]: pgmap v287: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-09-15T22:18:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:13 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:18:13.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:13 smithi005 ceph-mon[30944]: pgmap v287: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-09-15T22:18:13.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:13 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:18:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.606+0000 7f4744762640 1 -- 172.21.15.5:0/3883971121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c101440 msgr2=0x7f473c105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.606+0000 7f4744762640 1 --2- 172.21.15.5:0/3883971121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c101440 0x7f473c105910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f472c009860 tx=0x7f472c02a3d0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.608+0000 7f4744762640 1 -- 172.21.15.5:0/3883971121 shutdown_connections 2024-09-15T22:18:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.608+0000 7f4744762640 1 --2- 172.21.15.5:0/3883971121 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f473c105f30 0x7f473c1063c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.608+0000 7f4744762640 1 --2- 172.21.15.5:0/3883971121 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c101440 0x7f473c105910 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.608+0000 7f4744762640 1 --2- 172.21.15.5:0/3883971121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f473c100b20 0x7f473c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.608+0000 7f4744762640 1 -- 172.21.15.5:0/3883971121 >> 172.21.15.5:0/3883971121 conn(0x7f473c0fc7d0 msgr2=0x7f473c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.612+0000 7f4744762640 1 -- 172.21.15.5:0/3883971121 shutdown_connections 2024-09-15T22:18:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.612+0000 7f4744762640 1 -- 172.21.15.5:0/3883971121 wait complete. 2024-09-15T22:18:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.613+0000 7f4744762640 1 Processor -- start 2024-09-15T22:18:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.613+0000 7f4744762640 1 -- start start 2024-09-15T22:18:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.613+0000 7f4744762640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f473c100b20 0x7f473c114e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.613+0000 7f4744762640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f473c101440 0x7f473c110ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.613+0000 7f4744762640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c105f30 0x7f473c111420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.613+0000 7f4744762640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f473c109860 con 0x7f473c100b20 2024-09-15T22:18:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.613+0000 7f4744762640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1096e0 con 0x7f473c105f30 2024-09-15T22:18:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.613+0000 7f4744762640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1099e0 con 0x7f473c101440 2024-09-15T22:18:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.614+0000 7f4742cff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c105f30 0x7f473c111420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.614+0000 7f4742cff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c105f30 0x7f473c111420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:49014/0 (socket says 172.21.15.5:49014) 2024-09-15T22:18:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.614+0000 7f4742cff640 1 -- 172.21.15.5:0/2651212036 learned_addr learned my addr 172.21.15.5:0/2651212036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.615+0000 7f4742cff640 1 -- 172.21.15.5:0/2651212036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f473c101440 msgr2=0x7f473c110ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:18:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.615+0000 7f4742cff640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f473c101440 0x7f473c110ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.615+0000 7f4742cff640 1 -- 172.21.15.5:0/2651212036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f473c100b20 msgr2=0x7f473c114e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.615+0000 7f4742cff640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f473c100b20 0x7f473c114e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.615+0000 7f4742cff640 1 -- 172.21.15.5:0/2651212036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c111b50 con 0x7f473c105f30 2024-09-15T22:18:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.615+0000 7f4742cff640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c105f30 0x7f473c111420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f473400eb10 tx=0x7f473400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.615+0000 7f47337fe640 1 -- 172.21.15.5:0/2651212036 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734004b20 con 0x7f473c105f30 2024-09-15T22:18:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.616+0000 7f47337fe640 1 -- 172.21.15.5:0/2651212036 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4734004cc0 con 0x7f473c105f30 2024-09-15T22:18:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.616+0000 7f47337fe640 1 -- 172.21.15.5:0/2651212036 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47340107b0 con 0x7f473c105f30 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.616+0000 7f4744762640 1 -- 172.21.15.5:0/2651212036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c1c4950 con 0x7f473c105f30 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.616+0000 7f4744762640 1 -- 172.21.15.5:0/2651212036 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c1c4d50 con 0x7f473c105f30 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.620+0000 7f4746b3b740 1 Processor -- start 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.621+0000 7f47337fe640 1 -- 172.21.15.5:0/2651212036 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4734005020 con 0x7f473c105f30 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.621+0000 7f4746b3b740 1 -- start start 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.622+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc985c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.622+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc9904f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.622+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc990a30 0x55bbfc992e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.622+0000 7f4746b3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bbfc8ea160 con 0x55bbfc986170 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.622+0000 7f4746b3b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bbfc891720 con 0x55bbfc985850 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.622+0000 7f4746b3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bbfc8e6120 con 0x55bbfc990a30 2024-09-15T22:18:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.622+0000 7f47424fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc9904f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.622+0000 7f4741cfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc985c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.622+0000 7f47424fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc9904f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56220/0 (socket says 172.21.15.5:56220) 2024-09-15T22:18:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.623+0000 7f47424fe640 1 -- 172.21.15.5:0/2769282537 learned_addr learned my addr 172.21.15.5:0/2769282537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.623+0000 7f4741cfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc985c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:49026/0 (socket says 172.21.15.5:49026) 2024-09-15T22:18:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.623+0000 7f4741cfd640 1 --2- 172.21.15.5:0/2769282537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc990a30 0x55bbfc992e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.623+0000 7f47337fe640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47280a3500 0x7f47280a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.627+0000 7f47337fe640 1 -- 172.21.15.5:0/2651212036 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9629+0+0 (secure 0 0 0) 0x7f47340d07f0 con 0x7f473c105f30 2024-09-15T22:18:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.627+0000 7f47424fe640 1 -- 172.21.15.5:0/2769282537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc990a30 msgr2=0x55bbfc992e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.628+0000 7f47424fe640 1 --2- 172.21.15.5:0/2769282537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc990a30 0x55bbfc992e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.628+0000 7f47424fe640 1 -- 172.21.15.5:0/2769282537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 msgr2=0x55bbfc985c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.628+0000 7f47424fe640 1 --2- 172.21.15.5:0/2769282537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc985c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.628+0000 7f47424fe640 1 -- 172.21.15.5:0/2769282537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbfc65bba0 con 0x55bbfc986170 2024-09-15T22:18:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.630+0000 7f4742cff640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47280a3500 0x7f47280a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.632+0000 7f47424fe640 1 --2- 172.21.15.5:0/2769282537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc9904f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f473801ab40 tx=0x7f473801c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd7ecc44a95cc2c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.633+0000 7f4730ff9640 1 -- 172.21.15.5:0/2769282537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4738013030 con 0x55bbfc986170 2024-09-15T22:18:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.634+0000 7f4742cff640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47280a3500 0x7f47280a59c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f47340049d0 tx=0x7f4734005990 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.634+0000 7f4730ff9640 1 -- 172.21.15.5:0/2769282537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4738005020 con 0x55bbfc986170 2024-09-15T22:18:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.635+0000 7f4746b3b740 1 -- 172.21.15.5:0/2769282537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 msgr2=0x55bbfc9904f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.635+0000 7f4746b3b740 1 --2- 172.21.15.5:0/2769282537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc9904f0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f473801ab40 tx=0x7f473801c2d0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.635+0000 7f4746b3b740 1 -- 172.21.15.5:0/2769282537 shutdown_connections 2024-09-15T22:18:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.635+0000 7f4746b3b740 1 --2- 172.21.15.5:0/2769282537 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc990a30 0x55bbfc992e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.635+0000 7f4746b3b740 1 --2- 172.21.15.5:0/2769282537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc9904f0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.635+0000 7f4746b3b740 1 --2- 172.21.15.5:0/2769282537 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc985c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.635+0000 7f4746b3b740 1 -- 172.21.15.5:0/2769282537 >> 172.21.15.5:0/2769282537 conn(0x55bbfc981de0 msgr2=0x55bbfc9821c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.636+0000 7f4746b3b740 1 -- 172.21.15.5:0/2769282537 shutdown_connections 2024-09-15T22:18:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.636+0000 7f4746b3b740 1 -- 172.21.15.5:0/2769282537 wait complete. 2024-09-15T22:18:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.636+0000 7f4746b3b740 1 -- start start 2024-09-15T22:18:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.636+0000 7f4746b3b740 1 Processor -- start 2024-09-15T22:18:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.637+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc985850 0x55bbfc98d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.637+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc98dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.637+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc990a30 0x55bbfc98e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.637+0000 7f4746b3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bbfc9936a0 con 0x55bbfc986170 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.637+0000 7f4746b3b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bbfc872900 con 0x55bbfc990a30 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.637+0000 7f4746b3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bbfc993820 con 0x55bbfc985850 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.637+0000 7f4746b3b740 10 client.0 _ll_register_callbacks cb 0x55bbfc870780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.637+0000 7f4741cfd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc985850 0x55bbfc98d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.638+0000 7f4741cfd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc985850 0x55bbfc98d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:48224/0 (socket says 172.21.15.5:48224) 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.638+0000 7f4741cfd640 1 -- 172.21.15.5:0/659933502 learned_addr learned my addr 172.21.15.5:0/659933502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.638+0000 7f4741cfd640 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc990a30 0x55bbfc98e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.638+0000 7f47424fe640 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc98dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.639+0000 7f4741cfd640 1 -- 172.21.15.5:0/659933502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc990a30 msgr2=0x55bbfc98e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.639+0000 7f4741cfd640 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc990a30 0x55bbfc98e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.639+0000 7f4741cfd640 1 -- 172.21.15.5:0/659933502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 msgr2=0x55bbfc98dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.639+0000 7f4741cfd640 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc98dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.639+0000 7f4741cfd640 1 -- 172.21.15.5:0/659933502 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbfc8b2c50 con 0x55bbfc985850 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.639+0000 7f4741cfd640 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc985850 0x55bbfc98d850 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f472c009860 tx=0x7f472c0056d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db38d511737fe97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.642+0000 7f4725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.642+0000 7f4725ffb640 1 -- 172.21.15.5:0/659933502 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c03d870 con 0x55bbfc985850 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.642+0000 7f4746b3b740 1 -- 172.21.15.5:0/659933502 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4738005020 con 0x55bbfc985850 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.642+0000 7f4725ffb640 1 -- 172.21.15.5:0/659933502 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9629+0+0 (secure 0 0 0) 0x7f472c03da10 con 0x55bbfc985850 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.643+0000 7f4725ffb640 1 -- 172.21.15.5:0/659933502 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f472c054050 con 0x55bbfc985850 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.644+0000 7f4746b3b740 10 client.26578 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.644+0000 7f4746b3b740 1 -- 172.21.15.5:0/659933502 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bbfc99f320 con 0x55bbfc985850 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.644+0000 7f4725ffb640 1 -- 172.21.15.5:0/659933502 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4738005020 con 0x55bbfc985850 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.644+0000 7f4725ffb640 1 -- 172.21.15.5:0/659933502 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f472c04b070 con 0x55bbfc985850 2024-09-15T22:18:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.644+0000 7f4746b3b740 10 client.26578 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.644+0000 7f4746b3b740 10 client.26578 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:18:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.644+0000 7f4746b3b740 20 client.26578 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.644+0000 7f4746b3b740 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bbfc9a07e0 0x55bbfc9a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.644+0000 7f4746b3b740 4 client.26578 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.644+0000 7f4746b3b740 1 -- 172.21.15.5:0/659933502 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbfc8f7890 con 0x55bbfc9a07e0 2024-09-15T22:18:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.645+0000 7f47424fe640 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bbfc9a07e0 0x55bbfc9a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.661+0000 7f47424fe640 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bbfc9a07e0 0x55bbfc9a2bd0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16fad79500fc2c2 server_cookie=6610b5440d97e298 in_seq=0 out_seq=0 2024-09-15T22:18:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.661+0000 7f4725ffb640 10 client.26578 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.665+0000 7f4725ffb640 1 -- 172.21.15.5:0/659933502 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbfc8f7890 con 0x55bbfc9a07e0 2024-09-15T22:18:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.665+0000 7f4725ffb640 10 client.26578 handle_command_reply: tid=0 2024-09-15T22:18:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.665+0000 7f4746b3b740 1 client.26578 shutdown 2024-09-15T22:18:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.666+0000 7f4746b3b740 1 -- 172.21.15.5:0/659933502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc985850 msgr2=0x55bbfc98d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.666+0000 7f4746b3b740 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc985850 0x55bbfc98d850 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f472c009860 tx=0x7f472c0056d0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.671+0000 7f4746b3b740 1 -- 172.21.15.5:0/659933502 shutdown_connections 2024-09-15T22:18:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.671+0000 7f4746b3b740 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bbfc9a07e0 0x55bbfc9a2bd0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.671+0000 7f4746b3b740 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc990a30 0x55bbfc98e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.671+0000 7f4746b3b740 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc98dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.671+0000 7f4746b3b740 1 --2- 172.21.15.5:0/659933502 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc985850 0x55bbfc98d850 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.671+0000 7f4746b3b740 1 -- 172.21.15.5:0/659933502 >> 172.21.15.5:0/659933502 conn(0x55bbfc981de0 msgr2=0x55bbfc987b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.671+0000 7f4725ffb640 0 client.26578 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.672+0000 7f4746b3b740 1 -- 172.21.15.5:0/659933502 shutdown_connections 2024-09-15T22:18:13.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.672+0000 7f4746b3b740 1 -- 172.21.15.5:0/659933502 wait complete. 2024-09-15T22:18:13.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.673+0000 7f4746b3b740 20 client.26578 trim_cache size 0 max 16384 2024-09-15T22:18:13.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.713+0000 7f4746b3b740 1 Processor -- start 2024-09-15T22:18:13.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.713+0000 7f4746b3b740 1 -- start start 2024-09-15T22:18:13.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.713+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc9a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.713+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc986170 0x55bbfc9a8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc990a30 0x55bbfc98aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4746b3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1d1ac0 con 0x55bbfc990a30 2024-09-15T22:18:13.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4746b3b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1cf390 con 0x55bbfc985850 2024-09-15T22:18:13.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4746b3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1cd200 con 0x55bbfc986170 2024-09-15T22:18:13.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4741cfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc9a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4741cfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc9a8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:49042/0 (socket says 172.21.15.5:49042) 2024-09-15T22:18:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4741cfd640 1 -- 172.21.15.5:0/3447911070 learned_addr learned my addr 172.21.15.5:0/3447911070 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f47424fe640 1 --2- 172.21.15.5:0/3447911070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc986170 0x55bbfc9a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4741cfd640 1 -- 172.21.15.5:0/3447911070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc986170 msgr2=0x55bbfc9a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4741cfd640 1 --2- 172.21.15.5:0/3447911070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc986170 0x55bbfc9a8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4741cfd640 1 -- 172.21.15.5:0/3447911070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc990a30 msgr2=0x55bbfc98aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4741cfd640 1 --2- 172.21.15.5:0/3447911070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc990a30 0x55bbfc98aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.714+0000 7f4741cfd640 1 -- 172.21.15.5:0/3447911070 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbfc985fa0 con 0x55bbfc985850 2024-09-15T22:18:13.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.715+0000 7f4741cfd640 1 --2- 172.21.15.5:0/3447911070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc9a8430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f472c03ac90 tx=0x7f472c005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f132148c59019fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:13.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.721+0000 7f4727fff640 1 -- 172.21.15.5:0/3447911070 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c005740 con 0x55bbfc985850 2024-09-15T22:18:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.721+0000 7f4727fff640 1 -- 172.21.15.5:0/3447911070 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f472c03a940 con 0x55bbfc985850 2024-09-15T22:18:13.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.721+0000 7f4727fff640 1 -- 172.21.15.5:0/3447911070 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c03dcb0 con 0x55bbfc985850 2024-09-15T22:18:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.723+0000 7f4746b3b740 1 -- 172.21.15.5:0/3447911070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 msgr2=0x55bbfc9a8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:13.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.723+0000 7f4746b3b740 1 --2- 172.21.15.5:0/3447911070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc9a8430 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f472c03ac90 tx=0x7f472c005a90 comp rx=0 tx=0).stop 2024-09-15T22:18:13.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.725+0000 7f4746b3b740 1 -- 172.21.15.5:0/3447911070 shutdown_connections 2024-09-15T22:18:13.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.725+0000 7f4746b3b740 1 --2- 172.21.15.5:0/3447911070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc990a30 0x55bbfc98aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.725+0000 7f4746b3b740 1 --2- 172.21.15.5:0/3447911070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc986170 0x55bbfc9a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.725+0000 7f4746b3b740 1 --2- 172.21.15.5:0/3447911070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc9a8430 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.725+0000 7f4746b3b740 1 -- 172.21.15.5:0/3447911070 >> 172.21.15.5:0/3447911070 conn(0x55bbfc981de0 msgr2=0x55bbfc98f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.728+0000 7f4746b3b740 1 -- 172.21.15.5:0/3447911070 shutdown_connections 2024-09-15T22:18:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.728+0000 7f4746b3b740 1 -- 172.21.15.5:0/3447911070 wait complete. 2024-09-15T22:18:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.728+0000 7f4746b3b740 1 -- start start 2024-09-15T22:18:13.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.728+0000 7f4746b3b740 1 Processor -- start 2024-09-15T22:18:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.729+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc991db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.729+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc9922f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.729+0000 7f4746b3b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc990a30 0x55bbfc992830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.729+0000 7f4746b3b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1cd850 con 0x55bbfc986170 2024-09-15T22:18:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.729+0000 7f4746b3b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1cecc0 con 0x55bbfc985850 2024-09-15T22:18:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.729+0000 7f4746b3b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f473c1cdf20 con 0x55bbfc990a30 2024-09-15T22:18:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.729+0000 7f4746b3b740 10 client.0 _ll_register_callbacks cb 0x7f47340093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.729+0000 7f4741cfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc991db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.729+0000 7f4741cfd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc991db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:49054/0 (socket says 172.21.15.5:49054) 2024-09-15T22:18:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.729+0000 7f4741cfd640 1 -- 172.21.15.5:0/1412903609 learned_addr learned my addr 172.21.15.5:0/1412903609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:13.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.729+0000 7f4741cfd640 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc990a30 0x55bbfc992830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.730+0000 7f4741cfd640 1 -- 172.21.15.5:0/1412903609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc990a30 msgr2=0x55bbfc992830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:13.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.730+0000 7f4741cfd640 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc990a30 0x55bbfc992830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.730+0000 7f4741cfd640 1 -- 172.21.15.5:0/1412903609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 msgr2=0x55bbfc9922f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:13.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.730+0000 7f4741cfd640 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc9922f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.730+0000 7f4741cfd640 1 -- 172.21.15.5:0/1412903609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbfc997300 con 0x55bbfc985850 2024-09-15T22:18:13.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.730+0000 7f4741cfd640 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc991db0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f472c0094b0 tx=0x7f472c03d910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85014246177de03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:13.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.730+0000 7f4725ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:18:13.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.730+0000 7f4725ffb640 1 -- 172.21.15.5:0/1412903609 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f472c03a660 con 0x55bbfc985850 2024-09-15T22:18:13.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.730+0000 7f4725ffb640 1 -- 172.21.15.5:0/1412903609 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9629+0+0 (secure 0 0 0) 0x7f472c03da50 con 0x55bbfc985850 2024-09-15T22:18:13.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.731+0000 7f4746b3b740 1 -- 172.21.15.5:0/1412903609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f472c03a940 con 0x55bbfc985850 2024-09-15T22:18:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.731+0000 7f4725ffb640 1 -- 172.21.15.5:0/1412903609 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f472c06a050 con 0x55bbfc985850 2024-09-15T22:18:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.733+0000 7f4746b3b740 10 client.26645 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:13.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.733+0000 7f4746b3b740 1 -- 172.21.15.5:0/1412903609 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bbfca1c820 con 0x55bbfc985850 2024-09-15T22:18:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.737+0000 7f4725ffb640 1 -- 172.21.15.5:0/1412903609 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f472c03a940 con 0x55bbfc985850 2024-09-15T22:18:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.737+0000 7f4725ffb640 1 -- 172.21.15.5:0/1412903609 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f472c04b040 con 0x55bbfc985850 2024-09-15T22:18:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.737+0000 7f4746b3b740 10 client.26645 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.737+0000 7f4746b3b740 10 client.26645 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:18:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.737+0000 7f4746b3b740 20 client.26645 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:13.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.737+0000 7f4746b3b740 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bbfca1cf80 0x55bbfca1f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:13.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.737+0000 7f4746b3b740 4 client.26645 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}] 2024-09-15T22:18:13.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.737+0000 7f4746b3b740 1 -- 172.21.15.5:0/1412903609 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.35, "await_for": 45.0}) -- 0x55bbfc8f1be0 con 0x55bbfca1cf80 2024-09-15T22:18:13.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.739+0000 7f47424fe640 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bbfca1cf80 0x55bbfca1f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:13.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.745+0000 7f4725ffb640 10 client.26645 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:13.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.745+0000 7f47424fe640 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bbfca1cf80 0x55bbfca1f3a0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe9074b6017eb5dd server_cookie=90798469f1e4e461 in_seq=0 out_seq=0 2024-09-15T22:18:13.769 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.977+0000 7f4725ffb640 1 -- 172.21.15.5:0/1412903609 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55bbfc8f1be0 con 0x55bbfca1cf80 2024-09-15T22:18:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.977+0000 7f4725ffb640 10 client.26645 handle_command_reply: tid=0 2024-09-15T22:18:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.977+0000 7f4746b3b740 1 client.26645 shutdown 2024-09-15T22:18:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4746b3b740 1 -- 172.21.15.5:0/1412903609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 msgr2=0x55bbfc991db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4746b3b740 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc991db0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f472c0094b0 tx=0x7f472c03d910 comp rx=0 tx=0).stop 2024-09-15T22:18:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4746b3b740 1 -- 172.21.15.5:0/1412903609 shutdown_connections 2024-09-15T22:18:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4746b3b740 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bbfc990a30 0x55bbfc992830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4746b3b740 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bbfc986170 0x55bbfc9922f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4746b3b740 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bbfca1cf80 0x55bbfca1f3a0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4746b3b740 1 --2- 172.21.15.5:0/1412903609 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bbfc985850 0x55bbfc991db0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4725ffb640 0 client.26645 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4746b3b740 1 -- 172.21.15.5:0/1412903609 >> 172.21.15.5:0/1412903609 conn(0x55bbfc981de0 msgr2=0x55bbfc9806c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4746b3b740 1 -- 172.21.15.5:0/1412903609 shutdown_connections 2024-09-15T22:18:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4746b3b740 1 -- 172.21.15.5:0/1412903609 wait complete. 2024-09-15T22:18:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.978+0000 7f4746b3b740 20 client.26645 trim_cache size 0 max 16384 2024-09-15T22:18:13.982 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:18:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:18:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:18:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 34, 2024-09-15T22:18:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:18:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "9589889c": { 2024-09-15T22:18:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 34, 2024-09-15T22:18:13.982 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.3, 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:13.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:13.984 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:13.984 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:18:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 -- 172.21.15.5:0/2651212036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47280a3500 msgr2=0x7f47280a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47280a3500 0x7f47280a59c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f47340049d0 tx=0x7f4734005990 comp rx=0 tx=0).stop 2024-09-15T22:18:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 -- 172.21.15.5:0/2651212036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c105f30 msgr2=0x7f473c111420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c105f30 0x7f473c111420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f473400eb10 tx=0x7f473400c2d0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 -- 172.21.15.5:0/2651212036 shutdown_connections 2024-09-15T22:18:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47280a3500 0x7f47280a59c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f473c105f30 0x7f473c111420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f473c101440 0x7f473c110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 --2- 172.21.15.5:0/2651212036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f473c100b20 0x7f473c114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 -- 172.21.15.5:0/2651212036 >> 172.21.15.5:0/2651212036 conn(0x7f473c0fc7d0 msgr2=0x7f473c109fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 -- 172.21.15.5:0/2651212036 shutdown_connections 2024-09-15T22:18:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:13.979+0000 7f4744762640 1 -- 172.21.15.5:0/2651212036 wait complete. 2024-09-15T22:18:14.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:14 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:18:14.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:14 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:18:14.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:14 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:18:14.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:14 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:18:14.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:14 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:18:14.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:14 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:18:14.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:14 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:18:14.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:14 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:18:14.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:14 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:18:14.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 525.0 s, 393.1 tps, lat 212.922 ms stddev 288.027 2024-09-15T22:18:15.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:15 smithi099 ceph-mon[34892]: pgmap v288: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-09-15T22:18:15.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:15 smithi129 ceph-mon[35314]: pgmap v288: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-09-15T22:18:15.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:15 smithi005 ceph-mon[30944]: pgmap v288: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-09-15T22:18:16.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.873+0000 7f909cd59640 1 -- 172.21.15.5:0/2943257074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f909810e810 msgr2=0x7f9098110c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:16.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.873+0000 7f909cd59640 1 --2- 172.21.15.5:0/2943257074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f909810e810 0x7f9098110c00 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f9090009ca0 tx=0x7f909002a8b0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.873+0000 7f909cd59640 1 -- 172.21.15.5:0/2943257074 shutdown_connections 2024-09-15T22:18:16.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.873+0000 7f909cd59640 1 --2- 172.21.15.5:0/2943257074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f909810e810 0x7f9098110c00 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.873+0000 7f909cd59640 1 --2- 172.21.15.5:0/2943257074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9098102fc0 0x7f90981005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.873+0000 7f909cd59640 1 --2- 172.21.15.5:0/2943257074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9098102b50 0x7f9098100090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.873+0000 7f909cd59640 1 -- 172.21.15.5:0/2943257074 >> 172.21.15.5:0/2943257074 conn(0x7f9098077210 msgr2=0x7f90980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.873+0000 7f909cd59640 1 -- 172.21.15.5:0/2943257074 shutdown_connections 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.873+0000 7f909cd59640 1 -- 172.21.15.5:0/2943257074 wait complete. 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.874+0000 7f909cd59640 1 Processor -- start 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.874+0000 7f909cd59640 1 -- start start 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.874+0000 7f909cd59640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9098102b50 0x7f909819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.874+0000 7f909cd59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f909810e810 0x7f909819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.874+0000 7f909cd59640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f909819ee40 0x7f909819f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f909cd59640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90981141b0 con 0x7f909810e810 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f909cd59640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9098114030 con 0x7f909819ee40 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f909cd59640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9098114330 con 0x7f9098102b50 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f9095d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f909810e810 0x7f909819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f9095d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f909810e810 0x7f909819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56258/0 (socket says 172.21.15.5:56258) 2024-09-15T22:18:16.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f9095d9b640 1 -- 172.21.15.5:0/791349703 learned_addr learned my addr 172.21.15.5:0/791349703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f9096d9d640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f909819ee40 0x7f909819f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f909659c640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9098102b50 0x7f909819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f9095d9b640 1 -- 172.21.15.5:0/791349703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9098102b50 msgr2=0x7f909819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f9095d9b640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9098102b50 0x7f909819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f9095d9b640 1 -- 172.21.15.5:0/791349703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f909819ee40 msgr2=0x7f909819f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f9095d9b640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f909819ee40 0x7f909819f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f9095d9b640 1 -- 172.21.15.5:0/791349703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909819f900 con 0x7f909810e810 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f9095d9b640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f909810e810 0x7f909819aab0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f908c00c8c0 tx=0x7f908c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f90877fe640 1 -- 172.21.15.5:0/791349703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c007bb0 con 0x7f909810e810 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f90877fe640 1 -- 172.21.15.5:0/791349703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c007d50 con 0x7f909810e810 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f90877fe640 1 -- 172.21.15.5:0/791349703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c018830 con 0x7f909810e810 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f909cd59640 1 -- 172.21.15.5:0/791349703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909819fbf0 con 0x7f909810e810 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.875+0000 7f909cd59640 1 -- 172.21.15.5:0/791349703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90981a73d0 con 0x7f909810e810 2024-09-15T22:18:16.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.878+0000 7f90877fe640 1 -- 172.21.15.5:0/791349703 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f908c018a10 con 0x7f909810e810 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.878+0000 7f909f132740 1 Processor -- start 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909f132740 1 -- start start 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909f132740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfbcb40 0x55b8edfbb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909f132740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfbba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909f132740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfc6370 0x55b8edfc8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909f132740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b8edf212f0 con 0x55b8edfbb5d0 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909f132740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b8edec8cf0 con 0x55b8edfbcb40 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909f132740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b8edf1d2b0 con 0x55b8edfc6370 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfbba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfbba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56262/0 (socket says 172.21.15.5:56262) 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909659c640 1 -- 172.21.15.5:0/3345332645 learned_addr learned my addr 172.21.15.5:0/3345332645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f90877fe640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90800a33b0 0x7f90800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f9096d9d640 1 --2- 172.21.15.5:0/3345332645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfbcb40 0x55b8edfbb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f9095d9b640 1 --2- 172.21.15.5:0/3345332645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfc6370 0x55b8edfc8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909659c640 1 -- 172.21.15.5:0/3345332645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfc6370 msgr2=0x55b8edfc8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909659c640 1 --2- 172.21.15.5:0/3345332645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfc6370 0x55b8edfc8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909659c640 1 -- 172.21.15.5:0/3345332645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfbcb40 msgr2=0x55b8edfbb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909659c640 1 --2- 172.21.15.5:0/3345332645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfbcb40 0x55b8edfbb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f909659c640 1 -- 172.21.15.5:0/3345332645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8edead3a0 con 0x55b8edfbb5d0 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f9096d9d640 1 --2- 172.21.15.5:0/3345332645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfbcb40 0x55b8edfbb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.879+0000 7f9095d9b640 1 --2- 172.21.15.5:0/3345332645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfc6370 0x55b8edfc8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.880+0000 7f9096d9d640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90800a33b0 0x7f90800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.880+0000 7f90877fe640 1 -- 172.21.15.5:0/791349703 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9629+0+0 (secure 0 0 0) 0x7f908c0d0250 con 0x7f909810e810 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.880+0000 7f909659c640 1 --2- 172.21.15.5:0/3345332645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfbba10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f908800bd10 tx=0x7f908800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a1f9d179abe62 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.880+0000 7f9084ff9640 1 -- 172.21.15.5:0/3345332645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088015070 con 0x55b8edfbb5d0 2024-09-15T22:18:16.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.880+0000 7f9084ff9640 1 -- 172.21.15.5:0/3345332645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908801a940 con 0x55b8edfbb5d0 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.880+0000 7f9084ff9640 1 -- 172.21.15.5:0/3345332645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088011a10 con 0x55b8edfbb5d0 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f909f132740 1 -- 172.21.15.5:0/3345332645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 msgr2=0x55b8edfbba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f9096d9d640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90800a33b0 0x7f90800a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9090005fe0 tx=0x7f90900095c0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f909f132740 1 --2- 172.21.15.5:0/3345332645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfbba10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f908800bd10 tx=0x7f908800f730 comp rx=0 tx=0).stop 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f909f132740 1 -- 172.21.15.5:0/3345332645 shutdown_connections 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f909f132740 1 --2- 172.21.15.5:0/3345332645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfc6370 0x55b8edfc8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f909f132740 1 --2- 172.21.15.5:0/3345332645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfbba10 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f909f132740 1 --2- 172.21.15.5:0/3345332645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfbcb40 0x55b8edfbb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f909f132740 1 -- 172.21.15.5:0/3345332645 >> 172.21.15.5:0/3345332645 conn(0x55b8edf84310 msgr2=0x55b8edfb7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f909f132740 1 -- 172.21.15.5:0/3345332645 shutdown_connections 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f909f132740 1 -- 172.21.15.5:0/3345332645 wait complete. 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f909f132740 1 -- start start 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.881+0000 7f909f132740 1 Processor -- start 2024-09-15T22:18:16.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f909f132740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfbb5d0 0x55b8edfc5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f909f132740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfc2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f909f132740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfc6370 0x55b8edfc34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f909f132740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b8edfc9070 con 0x55b8edfbcb40 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f909f132740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b8ededc060 con 0x55b8edfbb5d0 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f909f132740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b8edfc91f0 con 0x55b8edfc6370 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f909f132740 10 client.0 _ll_register_callbacks cb 0x55b8edc2f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f9095d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfc2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f9095d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfc2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56278/0 (socket says 172.21.15.5:56278) 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f9095d9b640 1 -- 172.21.15.5:0/3213308132 learned_addr learned my addr 172.21.15.5:0/3213308132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f909659c640 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfbb5d0 0x55b8edfc5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f9096d9d640 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfc6370 0x55b8edfc34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f9095d9b640 1 -- 172.21.15.5:0/3213308132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfc6370 msgr2=0x55b8edfc34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:16.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f9095d9b640 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfc6370 0x55b8edfc34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f9095d9b640 1 -- 172.21.15.5:0/3213308132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfbb5d0 msgr2=0x55b8edfc5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f9095d9b640 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfbb5d0 0x55b8edfc5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f9095d9b640 1 -- 172.21.15.5:0/3213308132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8edc923c0 con 0x55b8edfbcb40 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f9095d9b640 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfc2f70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f908c09c010 tx=0x7f908c09b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a83ff10d5aee1c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.882+0000 7f907dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f907dffb640 1 -- 172.21.15.5:0/3213308132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c09c080 con 0x55b8edfbcb40 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f907dffb640 1 -- 172.21.15.5:0/3213308132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9629+0+0 (secure 0 0 0) 0x7f908c018d00 con 0x55b8edfbcb40 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f909f132740 1 -- 172.21.15.5:0/3213308132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f908801a940 con 0x55b8edfbcb40 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f907dffb640 1 -- 172.21.15.5:0/3213308132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f908c0a0430 con 0x55b8edfbcb40 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f909f132740 10 client.17136 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f909f132740 1 -- 172.21.15.5:0/3213308132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b8edfd4aa0 con 0x55b8edfbcb40 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f907dffb640 1 -- 172.21.15.5:0/3213308132 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908801a940 con 0x55b8edfbcb40 2024-09-15T22:18:16.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f907dffb640 1 -- 172.21.15.5:0/3213308132 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f908c0a4860 con 0x55b8edfbcb40 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f909f132740 10 client.17136 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f909f132740 10 client.17136 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f909f132740 20 client.17136 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f909f132740 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b8edfd5eb0 0x55b8edfd82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f909f132740 4 client.17136 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f909f132740 1 -- 172.21.15.5:0/3213308132 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8edf2e860 con 0x55b8edfd5eb0 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.883+0000 7f909659c640 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b8edfd5eb0 0x55b8edfd82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.884+0000 7f909659c640 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b8edfd5eb0 0x55b8edfd82a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93194dc8782077e3 server_cookie=c4a2967d0a04d152 in_seq=0 out_seq=0 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.884+0000 7f907dffb640 10 client.17136 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f907dffb640 1 -- 172.21.15.5:0/3213308132 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b8edf2e860 con 0x55b8edfd5eb0 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f907dffb640 10 client.17136 handle_command_reply: tid=0 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 1 client.17136 shutdown 2024-09-15T22:18:16.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 1 -- 172.21.15.5:0/3213308132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 msgr2=0x55b8edfc2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfc2f70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f908c09c010 tx=0x7f908c09b9e0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 1 -- 172.21.15.5:0/3213308132 shutdown_connections 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b8edfd5eb0 0x55b8edfd82a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfc6370 0x55b8edfc34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfc2f70 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f907dffb640 0 client.17136 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 1 --2- 172.21.15.5:0/3213308132 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfbb5d0 0x55b8edfc5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 1 -- 172.21.15.5:0/3213308132 >> 172.21.15.5:0/3213308132 conn(0x55b8edf84310 msgr2=0x55b8edfbd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 1 -- 172.21.15.5:0/3213308132 shutdown_connections 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 1 -- 172.21.15.5:0/3213308132 wait complete. 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.885+0000 7f909f132740 20 client.17136 trim_cache size 0 max 16384 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f909f132740 1 Processor -- start 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f909f132740 1 -- start start 2024-09-15T22:18:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f909f132740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfbb5d0 0x55b8edfc4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f909f132740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfbe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f909f132740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfc6370 0x55b8edfbe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f909f132740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90981b3ca0 con 0x55b8edfbcb40 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f909f132740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90981b1570 con 0x55b8edfc6370 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f909f132740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90981af3e0 con 0x55b8edfbb5d0 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f9095d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfbe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f9095d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfbe120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56290/0 (socket says 172.21.15.5:56290) 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f9095d9b640 1 -- 172.21.15.5:0/4038492720 learned_addr learned my addr 172.21.15.5:0/4038492720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f909659c640 1 --2- 172.21.15.5:0/4038492720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfbb5d0 0x55b8edfc4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f9096d9d640 1 --2- 172.21.15.5:0/4038492720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfc6370 0x55b8edfbe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f9095d9b640 1 -- 172.21.15.5:0/4038492720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfbb5d0 msgr2=0x55b8edfc4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f9095d9b640 1 --2- 172.21.15.5:0/4038492720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfbb5d0 0x55b8edfc4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f9095d9b640 1 -- 172.21.15.5:0/4038492720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfc6370 msgr2=0x55b8edfbe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:16.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f9095d9b640 1 --2- 172.21.15.5:0/4038492720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfc6370 0x55b8edfbe660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.898+0000 7f9095d9b640 1 -- 172.21.15.5:0/4038492720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8edfc14c0 con 0x55b8edfbcb40 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f9095d9b640 1 --2- 172.21.15.5:0/4038492720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfbe120 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f908c00cfd0 tx=0x7f908c098320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a08e30213007a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f907ffff640 1 -- 172.21.15.5:0/4038492720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c09c080 con 0x55b8edfbcb40 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f907ffff640 1 -- 172.21.15.5:0/4038492720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c0a4cf0 con 0x55b8edfbcb40 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f907ffff640 1 -- 172.21.15.5:0/4038492720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c09b660 con 0x55b8edfbcb40 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f909f132740 1 -- 172.21.15.5:0/4038492720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 msgr2=0x55b8edfbe120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f909f132740 1 --2- 172.21.15.5:0/4038492720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfbe120 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f908c00cfd0 tx=0x7f908c098320 comp rx=0 tx=0).stop 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f909f132740 1 -- 172.21.15.5:0/4038492720 shutdown_connections 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f909f132740 1 --2- 172.21.15.5:0/4038492720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfc6370 0x55b8edfbe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f909f132740 1 --2- 172.21.15.5:0/4038492720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbcb40 0x55b8edfbe120 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f909f132740 1 --2- 172.21.15.5:0/4038492720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfbb5d0 0x55b8edfc4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f909f132740 1 -- 172.21.15.5:0/4038492720 >> 172.21.15.5:0/4038492720 conn(0x55b8edf84310 msgr2=0x55b8edfc47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f909f132740 1 -- 172.21.15.5:0/4038492720 shutdown_connections 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.899+0000 7f909f132740 1 -- 172.21.15.5:0/4038492720 wait complete. 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909f132740 1 -- start start 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909f132740 1 Processor -- start 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909f132740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909f132740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfbcb40 0x55b8edfca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909f132740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfc6370 0x55b8edfcad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909f132740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90981afa30 con 0x55b8edfbb5d0 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909f132740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90981b0ea0 con 0x55b8edfc6370 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909f132740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90981b0100 con 0x55b8edfbcb40 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909f132740 10 client.0 _ll_register_callbacks cb 0x7f908c0983f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56302/0 (socket says 172.21.15.5:56302) 2024-09-15T22:18:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f909659c640 1 -- 172.21.15.5:0/3363469562 learned_addr learned my addr 172.21.15.5:0/3363469562 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.900+0000 7f9095d9b640 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfbcb40 0x55b8edfca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909659c640 1 -- 172.21.15.5:0/3363469562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfbcb40 msgr2=0x55b8edfca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909659c640 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfbcb40 0x55b8edfca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909659c640 1 -- 172.21.15.5:0/3363469562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfc6370 msgr2=0x55b8edfcad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f9096d9d640 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfc6370 0x55b8edfcad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909659c640 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfc6370 0x55b8edfcad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909659c640 1 -- 172.21.15.5:0/3363469562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ee03fc40 con 0x55b8edfbb5d0 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909659c640 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfca2e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f908800baa0 tx=0x7f908800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb01e8fa08c919df server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f907dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f907dffb640 1 -- 172.21.15.5:0/3363469562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088015040 con 0x55b8edfbb5d0 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f907dffb640 1 -- 172.21.15.5:0/3363469562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9629+0+0 (secure 0 0 0) 0x7f9088005250 con 0x55b8edfbb5d0 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909f132740 1 -- 172.21.15.5:0/3363469562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f908c0a4cf0 con 0x55b8edfbb5d0 2024-09-15T22:18:16.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f907dffb640 1 -- 172.21.15.5:0/3363469562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9088037050 con 0x55b8edfbb5d0 2024-09-15T22:18:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909f132740 10 client.17142 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909f132740 1 -- 172.21.15.5:0/3363469562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b8ee040f20 con 0x55b8edfbb5d0 2024-09-15T22:18:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f907dffb640 1 -- 172.21.15.5:0/3363469562 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c0a4cf0 con 0x55b8edfbb5d0 2024-09-15T22:18:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f907dffb640 1 -- 172.21.15.5:0/3363469562 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9088004b40 con 0x55b8edfbb5d0 2024-09-15T22:18:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909f132740 10 client.17142 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909f132740 10 client.17142 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:18:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909f132740 20 client.17142 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909f132740 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b8ee041370 0x55b8ee043760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909f132740 4 client.17142 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:18:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.901+0000 7f909f132740 1 -- 172.21.15.5:0/3363469562 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b8edf28d70 con 0x55b8ee041370 2024-09-15T22:18:16.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f9095d9b640 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b8ee041370 0x55b8ee043760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:16.976 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:18:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f907dffb640 10 client.17142 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:18:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f9095d9b640 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b8ee041370 0x55b8ee043760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5c44c3dddc54018 server_cookie=93cc1c1b2b6453f3 in_seq=0 out_seq=0 2024-09-15T22:18:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f907dffb640 1 -- 172.21.15.5:0/3363469562 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b8edf28d70 con 0x55b8ee041370 2024-09-15T22:18:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f907dffb640 10 client.17142 handle_command_reply: tid=0 2024-09-15T22:18:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f909f132740 1 client.17142 shutdown 2024-09-15T22:18:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f909f132740 1 -- 172.21.15.5:0/3363469562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 msgr2=0x55b8edfca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f909f132740 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfca2e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f908800baa0 tx=0x7f908800b460 comp rx=0 tx=0).stop 2024-09-15T22:18:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f909f132740 1 -- 172.21.15.5:0/3363469562 shutdown_connections 2024-09-15T22:18:16.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f909f132740 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55b8ee041370 0x55b8ee043760 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f909f132740 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b8edfc6370 0x55b8edfcad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f909f132740 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b8edfbcb40 0x55b8edfca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f907dffb640 0 client.17142 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.902+0000 7f909f132740 1 --2- 172.21.15.5:0/3363469562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8edfbb5d0 0x55b8edfca2e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909f132740 1 -- 172.21.15.5:0/3363469562 >> 172.21.15.5:0/3363469562 conn(0x55b8edf84310 msgr2=0x55b8edfc4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909f132740 1 -- 172.21.15.5:0/3363469562 shutdown_connections 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909f132740 1 -- 172.21.15.5:0/3363469562 wait complete. 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909f132740 20 client.17142 trim_cache size 0 max 16384 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 -- 172.21.15.5:0/791349703 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90800a33b0 msgr2=0x7f90800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90800a33b0 0x7f90800a5870 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9090005fe0 tx=0x7f90900095c0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 -- 172.21.15.5:0/791349703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f909810e810 msgr2=0x7f909819aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f909810e810 0x7f909819aab0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f908c00c8c0 tx=0x7f908c00cd80 comp rx=0 tx=0).stop 2024-09-15T22:18:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 -- 172.21.15.5:0/791349703 shutdown_connections 2024-09-15T22:18:16.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90800a33b0 0x7f90800a5870 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f909819ee40 0x7f909819f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f909810e810 0x7f909819aab0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 --2- 172.21.15.5:0/791349703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9098102b50 0x7f909819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:16.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 -- 172.21.15.5:0/791349703 >> 172.21.15.5:0/791349703 conn(0x7f9098077210 msgr2=0x7f909810fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:16.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 -- 172.21.15.5:0/791349703 shutdown_connections 2024-09-15T22:18:16.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:15.903+0000 7f909cd59640 1 -- 172.21.15.5:0/791349703 wait complete. 2024-09-15T22:18:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:17 smithi099 ceph-mon[34892]: pgmap v289: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-09-15T22:18:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:17 smithi129 ceph-mon[35314]: pgmap v289: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-09-15T22:18:17.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:17 smithi005 ceph-mon[30944]: pgmap v289: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-09-15T22:18:18.010 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:18:18.703 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9589889c', quiesce took 10.0 seconds. Will release after: 12.350000000000001 2024-09-15T22:18:18.718 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:19.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.183+0000 7fd063f83640 1 -- 172.21.15.5:0/3193691979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c10c180 msgr2=0x7fd05c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:19.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.183+0000 7fd063f83640 1 --2- 172.21.15.5:0/3193691979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c10c180 0x7fd05c10e600 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fd054009ca0 tx=0x7fd05402a950 comp rx=0 tx=0).stop 2024-09-15T22:18:19.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 -- 172.21.15.5:0/3193691979 shutdown_connections 2024-09-15T22:18:19.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 --2- 172.21.15.5:0/3193691979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c10c180 0x7fd05c10e600 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 --2- 172.21.15.5:0/3193691979 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd05c1036c0 0x7fd05c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 --2- 172.21.15.5:0/3193691979 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd05c102da0 0x7fd05c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 -- 172.21.15.5:0/3193691979 >> 172.21.15.5:0/3193691979 conn(0x7fd05c0fc7a0 msgr2=0x7fd05c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:19.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 -- 172.21.15.5:0/3193691979 shutdown_connections 2024-09-15T22:18:19.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 -- 172.21.15.5:0/3193691979 wait complete. 2024-09-15T22:18:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 Processor -- start 2024-09-15T22:18:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 -- start start 2024-09-15T22:18:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd05c102da0 0x7fd05c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1036c0 0x7fd05c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd05c10c180 0x7fd05c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c111f60 con 0x7fd05c1036c0 2024-09-15T22:18:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c111de0 con 0x7fd05c10c180 2024-09-15T22:18:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.184+0000 7fd063f83640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c1120e0 con 0x7fd05c102da0 2024-09-15T22:18:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd06151e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1036c0 0x7fd05c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd06151e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1036c0 0x7fd05c1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56310/0 (socket says 172.21.15.5:56310) 2024-09-15T22:18:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd06151e640 1 -- 172.21.15.5:0/2685119842 learned_addr learned my addr 172.21.15.5:0/2685119842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd061d1f640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd05c102da0 0x7fd05c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd062520640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd05c10c180 0x7fd05c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd06151e640 1 -- 172.21.15.5:0/2685119842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd05c102da0 msgr2=0x7fd05c1982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd06151e640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd05c102da0 0x7fd05c1982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd06151e640 1 -- 172.21.15.5:0/2685119842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd05c10c180 msgr2=0x7fd05c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd06151e640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd05c10c180 0x7fd05c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd06151e640 1 -- 172.21.15.5:0/2685119842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c19d250 con 0x7fd05c1036c0 2024-09-15T22:18:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd062520640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd05c10c180 0x7fd05c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd06151e640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1036c0 0x7fd05c1987e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00c920 tx=0x7fd04c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd052ffd640 1 -- 172.21.15.5:0/2685119842 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c007bb0 con 0x7fd05c1036c0 2024-09-15T22:18:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd052ffd640 1 -- 172.21.15.5:0/2685119842 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd04c007d50 con 0x7fd05c1036c0 2024-09-15T22:18:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd063f83640 1 -- 172.21.15.5:0/2685119842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05c19d540 con 0x7fd05c1036c0 2024-09-15T22:18:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd052ffd640 1 -- 172.21.15.5:0/2685119842 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c0187d0 con 0x7fd05c1036c0 2024-09-15T22:18:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.185+0000 7fd063f83640 1 -- 172.21.15.5:0/2685119842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05c1a4e20 con 0x7fd05c1036c0 2024-09-15T22:18:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.188+0000 7fd052ffd640 1 -- 172.21.15.5:0/2685119842 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd04c00f450 con 0x7fd05c1036c0 2024-09-15T22:18:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.188+0000 7fd06635c740 1 Processor -- start 2024-09-15T22:18:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.188+0000 7fd06635c740 1 -- start start 2024-09-15T22:18:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.188+0000 7fd06635c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea2b40 0x557c6eea1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.188+0000 7fd06635c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 0x557c6eea1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.188+0000 7fd06635c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eeb0780 0x557c6eeb2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.188+0000 7fd06635c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c6ee072f0 con 0x557c6eea15d0 2024-09-15T22:18:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.188+0000 7fd06635c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557c6edaecf0 con 0x557c6eea2b40 2024-09-15T22:18:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.188+0000 7fd06635c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c6ee032b0 con 0x557c6eeb0780 2024-09-15T22:18:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.189+0000 7fd061d1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 0x557c6eea1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.189+0000 7fd061d1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 0x557c6eea1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56314/0 (socket says 172.21.15.5:56314) 2024-09-15T22:18:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.189+0000 7fd061d1f640 1 -- 172.21.15.5:0/118750472 learned_addr learned my addr 172.21.15.5:0/118750472 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.189+0000 7fd062520640 1 --2- 172.21.15.5:0/118750472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea2b40 0x557c6eea1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.189+0000 7fd06151e640 1 --2- 172.21.15.5:0/118750472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eeb0780 0x557c6eeb2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.189+0000 7fd052ffd640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd0480a33b0 0x7fd0480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.189+0000 7fd062520640 1 -- 172.21.15.5:0/118750472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eeb0780 msgr2=0x557c6eeb2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.189+0000 7fd062520640 1 --2- 172.21.15.5:0/118750472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eeb0780 0x557c6eeb2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.189+0000 7fd062520640 1 -- 172.21.15.5:0/118750472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 msgr2=0x557c6eea1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.189+0000 7fd062520640 1 --2- 172.21.15.5:0/118750472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 0x557c6eea1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.189+0000 7fd062520640 1 -- 172.21.15.5:0/118750472 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6ed933a0 con 0x557c6eea2b40 2024-09-15T22:18:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.190+0000 7fd062520640 1 --2- 172.21.15.5:0/118750472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea2b40 0x557c6eea1090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd05402ae70 tx=0x7fd05402f9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6ae81143a81e811 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.190+0000 7fd062520640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd0480a33b0 0x7fd0480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.190+0000 7fd052ffd640 1 -- 172.21.15.5:0/2685119842 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7fd04c0d0200 con 0x7fd05c1036c0 2024-09-15T22:18:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.190+0000 7fd047fff640 1 -- 172.21.15.5:0/118750472 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05404a070 con 0x557c6eea2b40 2024-09-15T22:18:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.190+0000 7fd047fff640 1 -- 172.21.15.5:0/118750472 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd054003e50 con 0x557c6eea2b40 2024-09-15T22:18:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.190+0000 7fd047fff640 1 -- 172.21.15.5:0/118750472 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054034040 con 0x557c6eea2b40 2024-09-15T22:18:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd06635c740 1 -- 172.21.15.5:0/118750472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea2b40 msgr2=0x557c6eea1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd062520640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd0480a33b0 0x7fd0480a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd0540049d0 tx=0x7fd054039450 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd06635c740 1 --2- 172.21.15.5:0/118750472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea2b40 0x557c6eea1090 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd05402ae70 tx=0x7fd05402f9c0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd06635c740 1 -- 172.21.15.5:0/118750472 shutdown_connections 2024-09-15T22:18:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd06635c740 1 --2- 172.21.15.5:0/118750472 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eeb0780 0x557c6eeb2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd06635c740 1 --2- 172.21.15.5:0/118750472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 0x557c6eea1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd06635c740 1 --2- 172.21.15.5:0/118750472 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea2b40 0x557c6eea1090 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd06635c740 1 -- 172.21.15.5:0/118750472 >> 172.21.15.5:0/118750472 conn(0x557c6ee6a310 msgr2=0x557c6ee9dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd06635c740 1 -- 172.21.15.5:0/118750472 shutdown_connections 2024-09-15T22:18:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd06635c740 1 -- 172.21.15.5:0/118750472 wait complete. 2024-09-15T22:18:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd06635c740 1 -- start start 2024-09-15T22:18:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.191+0000 7fd06635c740 1 Processor -- start 2024-09-15T22:18:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd06635c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 0x557c6eeb01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd06635c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eea2b40 0x557c6eead460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd06635c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eeb0780 0x557c6eeadbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd06635c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c6eeb3420 con 0x557c6eeb0780 2024-09-15T22:18:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd06635c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x557c6edc2060 con 0x557c6eea15d0 2024-09-15T22:18:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd06635c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x557c6eeb35a0 con 0x557c6eea2b40 2024-09-15T22:18:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd062520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eeb0780 0x557c6eeadbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd06635c740 10 client.0 _ll_register_callbacks cb 0x557c6eb158c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd061d1f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 0x557c6eeb01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd062520640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eeb0780 0x557c6eeadbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56316/0 (socket says 172.21.15.5:56316) 2024-09-15T22:18:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd062520640 1 -- 172.21.15.5:0/4081865957 learned_addr learned my addr 172.21.15.5:0/4081865957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd06151e640 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eea2b40 0x557c6eead460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd062520640 1 -- 172.21.15.5:0/4081865957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eea2b40 msgr2=0x557c6eead460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd062520640 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eea2b40 0x557c6eead460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd062520640 1 -- 172.21.15.5:0/4081865957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 msgr2=0x557c6eeb01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd062520640 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 0x557c6eeb01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd062520640 1 -- 172.21.15.5:0/4081865957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6eb783c0 con 0x557c6eeb0780 2024-09-15T22:18:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.192+0000 7fd061d1f640 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 0x557c6eeb01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.193+0000 7fd062520640 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eeb0780 0x557c6eeadbf0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd054039fd0 tx=0x7fd054043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05af58525f15d3e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.193+0000 7fd0457fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.193+0000 7fd0457fa640 1 -- 172.21.15.5:0/4081865957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05405c060 con 0x557c6eeb0780 2024-09-15T22:18:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.193+0000 7fd06635c740 1 -- 172.21.15.5:0/4081865957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd054003e50 con 0x557c6eeb0780 2024-09-15T22:18:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.193+0000 7fd0457fa640 1 -- 172.21.15.5:0/4081865957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7fd054004040 con 0x557c6eeb0780 2024-09-15T22:18:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.193+0000 7fd0457fa640 1 -- 172.21.15.5:0/4081865957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd054063050 con 0x557c6eeb0780 2024-09-15T22:18:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.193+0000 7fd06635c740 10 client.17154 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.193+0000 7fd06635c740 1 -- 172.21.15.5:0/4081865957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c6eeb3420 con 0x557c6eeb0780 2024-09-15T22:18:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.193+0000 7fd0457fa640 1 -- 172.21.15.5:0/4081865957 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd054003e50 con 0x557c6eeb0780 2024-09-15T22:18:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.194+0000 7fd0457fa640 1 -- 172.21.15.5:0/4081865957 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd05404d040 con 0x557c6eeb0780 2024-09-15T22:18:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.194+0000 7fd06635c740 10 client.17154 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.194+0000 7fd06635c740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.194+0000 7fd06635c740 20 client.17154 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.194+0000 7fd06635c740 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c6eec0080 0x557c6eec2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.194+0000 7fd06635c740 4 client.17154 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.194+0000 7fd06635c740 1 -- 172.21.15.5:0/4081865957 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c6ee14860 con 0x557c6eec0080 2024-09-15T22:18:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.194+0000 7fd061d1f640 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c6eec0080 0x557c6eec2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.194+0000 7fd0457fa640 10 client.17154 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.194+0000 7fd061d1f640 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c6eec0080 0x557c6eec2470 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d1006f17a6a9ee server_cookie=c36bc6f1a3f86f40 in_seq=0 out_seq=0 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd0457fa640 1 -- 172.21.15.5:0/4081865957 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c6ee14860 con 0x557c6eec0080 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd0457fa640 10 client.17154 handle_command_reply: tid=0 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd06635c740 1 client.17154 shutdown 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd06635c740 1 -- 172.21.15.5:0/4081865957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eeb0780 msgr2=0x557c6eeadbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd06635c740 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eeb0780 0x557c6eeadbf0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd054039fd0 tx=0x7fd054043000 comp rx=0 tx=0).stop 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd06635c740 1 -- 172.21.15.5:0/4081865957 shutdown_connections 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd06635c740 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eeb0780 0x557c6eeadbf0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd06635c740 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eea2b40 0x557c6eead460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd06635c740 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c6eec0080 0x557c6eec2470 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd06635c740 1 --2- 172.21.15.5:0/4081865957 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 0x557c6eeb01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd0457fa640 0 client.17154 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.195+0000 7fd06635c740 1 -- 172.21.15.5:0/4081865957 >> 172.21.15.5:0/4081865957 conn(0x557c6ee6a310 msgr2=0x557c6eea3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.196+0000 7fd06635c740 1 -- 172.21.15.5:0/4081865957 shutdown_connections 2024-09-15T22:18:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.196+0000 7fd06635c740 1 -- 172.21.15.5:0/4081865957 wait complete. 2024-09-15T22:18:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.196+0000 7fd06635c740 20 client.17154 trim_cache size 0 max 16384 2024-09-15T22:18:19.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd06635c740 1 Processor -- start 2024-09-15T22:18:19.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd06635c740 1 -- start start 2024-09-15T22:18:19.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:19 smithi005 ceph-mon[30944]: pgmap v290: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.8 MiB/s wr, 325 op/s 2024-09-15T22:18:19.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:19 smithi005 ceph-mon[30944]: osdmap e95: 12 total, 12 up, 12 in 2024-09-15T22:18:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd06635c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 0x557c6ee9cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd06635c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eea2b40 0x557c6eec7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd06635c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eeb0780 0x557c6eec80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd06635c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c1af660 con 0x557c6eea15d0 2024-09-15T22:18:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd06635c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c1acf30 con 0x557c6eeb0780 2024-09-15T22:18:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd06635c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c1aada0 con 0x557c6eea2b40 2024-09-15T22:18:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd061d1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 0x557c6ee9cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd061d1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 0x557c6ee9cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56318/0 (socket says 172.21.15.5:56318) 2024-09-15T22:18:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd061d1f640 1 -- 172.21.15.5:0/3234620388 learned_addr learned my addr 172.21.15.5:0/3234620388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd06151e640 1 --2- 172.21.15.5:0/3234620388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eea2b40 0x557c6eec7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd062520640 1 --2- 172.21.15.5:0/3234620388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eeb0780 0x557c6eec80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd061d1f640 1 -- 172.21.15.5:0/3234620388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eea2b40 msgr2=0x557c6eec7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd061d1f640 1 --2- 172.21.15.5:0/3234620388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eea2b40 0x557c6eec7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd061d1f640 1 -- 172.21.15.5:0/3234620388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eeb0780 msgr2=0x557c6eec80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd061d1f640 1 --2- 172.21.15.5:0/3234620388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eeb0780 0x557c6eec80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.209+0000 7fd061d1f640 1 -- 172.21.15.5:0/3234620388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6eea4120 con 0x557c6eea15d0 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.210+0000 7fd061d1f640 1 --2- 172.21.15.5:0/3234620388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 0x557c6ee9cbf0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd058010580 tx=0x7fd05800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08088fdcb4455a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.210+0000 7fd0477fe640 1 -- 172.21.15.5:0/3234620388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05800fad0 con 0x557c6eea15d0 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.210+0000 7fd0477fe640 1 -- 172.21.15.5:0/3234620388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd058005500 con 0x557c6eea15d0 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.210+0000 7fd0477fe640 1 -- 172.21.15.5:0/3234620388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd058004e40 con 0x557c6eea15d0 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.211+0000 7fd06635c740 1 -- 172.21.15.5:0/3234620388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 msgr2=0x557c6ee9cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.211+0000 7fd06635c740 1 --2- 172.21.15.5:0/3234620388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 0x557c6ee9cbf0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd058010580 tx=0x7fd05800a2d0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.211+0000 7fd06635c740 1 -- 172.21.15.5:0/3234620388 shutdown_connections 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.211+0000 7fd06635c740 1 --2- 172.21.15.5:0/3234620388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eeb0780 0x557c6eec80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.211+0000 7fd06635c740 1 --2- 172.21.15.5:0/3234620388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eea2b40 0x557c6eec7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.211+0000 7fd06635c740 1 --2- 172.21.15.5:0/3234620388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea15d0 0x557c6ee9cbf0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.211+0000 7fd06635c740 1 -- 172.21.15.5:0/3234620388 >> 172.21.15.5:0/3234620388 conn(0x557c6ee6a310 msgr2=0x557c6ee9c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.211+0000 7fd06635c740 1 -- 172.21.15.5:0/3234620388 shutdown_connections 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.211+0000 7fd06635c740 1 -- 172.21.15.5:0/3234620388 wait complete. 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.211+0000 7fd06635c740 1 -- start start 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.211+0000 7fd06635c740 1 Processor -- start 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06635c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 0x557c6eeb4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06635c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea2b40 0x557c6eeb4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06635c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eeb0780 0x557c6eeb4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06635c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c1ab3f0 con 0x557c6eea2b40 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06635c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c1ac860 con 0x557c6eea15d0 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06635c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c1abac0 con 0x557c6eeb0780 2024-09-15T22:18:19.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06635c740 10 client.0 _ll_register_callbacks cb 0x7fd054048070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd061d1f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 0x557c6eeb4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd061d1f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 0x557c6eeb4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:49140/0 (socket says 172.21.15.5:49140) 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd061d1f640 1 -- 172.21.15.5:0/2451792498 learned_addr learned my addr 172.21.15.5:0/2451792498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06151e640 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea2b40 0x557c6eeb4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd062520640 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eeb0780 0x557c6eeb4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06151e640 1 -- 172.21.15.5:0/2451792498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eeb0780 msgr2=0x557c6eeb4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06151e640 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eeb0780 0x557c6eeb4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06151e640 1 -- 172.21.15.5:0/2451792498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 msgr2=0x557c6eeb4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06151e640 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 0x557c6eeb4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06151e640 1 -- 172.21.15.5:0/2451792498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6ef29b30 con 0x557c6eea2b40 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd06151e640 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea2b40 0x557c6eeb4a40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0cd550 tx=0x7fd04c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e14f9218d8de158 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.212+0000 7fd0457fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:19.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd0457fa640 1 -- 172.21.15.5:0/2451792498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c09beb0 con 0x557c6eea2b40 2024-09-15T22:18:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd0457fa640 1 -- 172.21.15.5:0/2451792498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7fd04c0bc0b0 con 0x557c6eea2b40 2024-09-15T22:18:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd06635c740 1 -- 172.21.15.5:0/2451792498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd058005500 con 0x557c6eea2b40 2024-09-15T22:18:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd0457fa640 1 -- 172.21.15.5:0/2451792498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd04c09ba40 con 0x557c6eea2b40 2024-09-15T22:18:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd06635c740 10 client.17160 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd06635c740 1 -- 172.21.15.5:0/2451792498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557c6ef2adf0 con 0x557c6eea2b40 2024-09-15T22:18:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd0457fa640 1 -- 172.21.15.5:0/2451792498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd058005500 con 0x557c6eea2b40 2024-09-15T22:18:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd0457fa640 1 -- 172.21.15.5:0/2451792498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd04c098750 con 0x557c6eea2b40 2024-09-15T22:18:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd06635c740 10 client.17160 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd06635c740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd06635c740 20 client.17160 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:19.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd06635c740 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c6ef2b570 0x557c6ef2d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd06635c740 4 client.17160 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:18:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.213+0000 7fd06635c740 1 -- 172.21.15.5:0/2451792498 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c6ee0ed70 con 0x557c6ef2b570 2024-09-15T22:18:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.214+0000 7fd061d1f640 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c6ef2b570 0x557c6ef2d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.214+0000 7fd0457fa640 10 client.17160 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.214+0000 7fd061d1f640 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c6ef2b570 0x557c6ef2d990 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc84f38bc738f47 server_cookie=f98517273b5743f in_seq=0 out_seq=0 2024-09-15T22:18:19.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd0457fa640 1 -- 172.21.15.5:0/2451792498 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c6ee0ed70 con 0x557c6ef2b570 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd0457fa640 10 client.17160 handle_command_reply: tid=0 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 1 client.17160 shutdown 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 1 -- 172.21.15.5:0/2451792498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea2b40 msgr2=0x557c6eeb4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea2b40 0x557c6eeb4a40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0cd550 tx=0x7fd04c00a620 comp rx=0 tx=0).stop 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 1 -- 172.21.15.5:0/2451792498 shutdown_connections 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x557c6eeb0780 0x557c6eeb4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c6eea2b40 0x557c6eeb4a40 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x557c6ef2b570 0x557c6ef2d990 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 1 --2- 172.21.15.5:0/2451792498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x557c6eea15d0 0x557c6eeb4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd0457fa640 0 client.17160 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 1 -- 172.21.15.5:0/2451792498 >> 172.21.15.5:0/2451792498 conn(0x557c6ee6a310 msgr2=0x557c6eea3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 1 -- 172.21.15.5:0/2451792498 shutdown_connections 2024-09-15T22:18:19.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 1 -- 172.21.15.5:0/2451792498 wait complete. 2024-09-15T22:18:19.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.215+0000 7fd06635c740 20 client.17160 trim_cache size 0 max 16384 2024-09-15T22:18:19.234 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:18:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 -- 172.21.15.5:0/2685119842 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd0480a33b0 msgr2=0x7fd0480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd0480a33b0 0x7fd0480a5870 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd0540049d0 tx=0x7fd054039450 comp rx=0 tx=0).stop 2024-09-15T22:18:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 -- 172.21.15.5:0/2685119842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1036c0 msgr2=0x7fd05c1987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1036c0 0x7fd05c1987e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00c920 tx=0x7fd04c00cd20 comp rx=0 tx=0).stop 2024-09-15T22:18:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 -- 172.21.15.5:0/2685119842 shutdown_connections 2024-09-15T22:18:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd0480a33b0 0x7fd0480a5870 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd05c10c180 0x7fd05c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1036c0 0x7fd05c1987e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 --2- 172.21.15.5:0/2685119842 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd05c102da0 0x7fd05c1982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 -- 172.21.15.5:0/2685119842 >> 172.21.15.5:0/2685119842 conn(0x7fd05c0fc7a0 msgr2=0x7fd05c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 -- 172.21.15.5:0/2685119842 shutdown_connections 2024-09-15T22:18:19.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:19.216+0000 7fd063f83640 1 -- 172.21.15.5:0/2685119842 wait complete. 2024-09-15T22:18:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:19 smithi099 ceph-mon[34892]: pgmap v290: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.8 MiB/s wr, 325 op/s 2024-09-15T22:18:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:19 smithi099 ceph-mon[34892]: osdmap e95: 12 total, 12 up, 12 in 2024-09-15T22:18:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:19 smithi129 ceph-mon[35314]: pgmap v290: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.8 MiB/s wr, 325 op/s 2024-09-15T22:18:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:19 smithi129 ceph-mon[35314]: osdmap e95: 12 total, 12 up, 12 in 2024-09-15T22:18:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 530.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:18:21.020 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:18:21.020 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:18:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:21.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:21 smithi099 ceph-mon[34892]: pgmap v292: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-09-15T22:18:21.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:21 smithi129 ceph-mon[35314]: pgmap v292: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-09-15T22:18:21.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:21 smithi005 ceph-mon[30944]: pgmap v292: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 2.8 MiB/s wr, 318 op/s 2024-09-15T22:18:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.497+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3557434476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c0101de0 msgr2=0x7ff5c01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.497+0000 7ff5c48c7640 1 --2- 172.21.15.5:0/3557434476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c0101de0 0x7ff5c01021c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009900 tx=0x7ff5b002a410 comp rx=0 tx=0).stop 2024-09-15T22:18:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.497+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3557434476 shutdown_connections 2024-09-15T22:18:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.497+0000 7ff5c48c7640 1 --2- 172.21.15.5:0/3557434476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5c0106f10 0x7ff5c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.497+0000 7ff5c48c7640 1 --2- 172.21.15.5:0/3557434476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c0102790 0x7ff5c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.497+0000 7ff5c48c7640 1 --2- 172.21.15.5:0/3557434476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c0101de0 0x7ff5c01021c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.497+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3557434476 >> 172.21.15.5:0/3557434476 conn(0x7ff5c00fd8c0 msgr2=0x7ff5c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.497+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3557434476 shutdown_connections 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.497+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3557434476 wait complete. 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.498+0000 7ff5c48c7640 1 Processor -- start 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.498+0000 7ff5c48c7640 1 -- start start 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.498+0000 7ff5c48c7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5c0101de0 0x7ff5c019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.498+0000 7ff5c48c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c0102790 0x7ff5c019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.498+0000 7ff5c48c7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c0106f10 0x7ff5c010dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.498+0000 7ff5c48c7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c0111fb0 con 0x7ff5c0102790 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.498+0000 7ff5c48c7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c0111e30 con 0x7ff5c0106f10 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.498+0000 7ff5c48c7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c0112130 con 0x7ff5c0101de0 2024-09-15T22:18:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5bdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c0102790 0x7ff5c019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5bdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c0102790 0x7ff5c019f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56334/0 (socket says 172.21.15.5:56334) 2024-09-15T22:18:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5be59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5c0101de0 0x7ff5c019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/3013516378 learned_addr learned my addr 172.21.15.5:0/3013516378 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c0106f10 0x7ff5c010dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/3013516378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5c0101de0 msgr2=0x7ff5c019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5c0101de0 0x7ff5c019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/3013516378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c0106f10 msgr2=0x7ff5c010dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c0106f10 0x7ff5c010dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/3013516378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c010e4e0 con 0x7ff5c0102790 2024-09-15T22:18:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c0106f10 0x7ff5c010dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:22.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c0102790 0x7ff5c019f590 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400b4a0 tx=0x7ff5b400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:22.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5af7fe640 1 -- 172.21.15.5:0/3013516378 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4014070 con 0x7ff5c0102790 2024-09-15T22:18:22.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3013516378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c010e7d0 con 0x7ff5c0102790 2024-09-15T22:18:22.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5af7fe640 1 -- 172.21.15.5:0/3013516378 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b4010af0 con 0x7ff5c0102790 2024-09-15T22:18:22.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.499+0000 7ff5af7fe640 1 -- 172.21.15.5:0/3013516378 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4017d10 con 0x7ff5c0102790 2024-09-15T22:18:22.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.500+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3013516378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c019ad60 con 0x7ff5c0102790 2024-09-15T22:18:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.502+0000 7ff5af7fe640 1 -- 172.21.15.5:0/3013516378 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff5b4017070 con 0x7ff5c0102790 2024-09-15T22:18:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.502+0000 7ff5c6ca0740 1 Processor -- start 2024-09-15T22:18:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.502+0000 7ff5c6ca0740 1 -- start start 2024-09-15T22:18:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8db60 0x561091c8df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8e480 0x561091c98800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5af7fe640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff5a80a33b0 0x7ff5a80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c98d40 0x561091c9b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561091bf3280 con 0x561091c8db60 2024-09-15T22:18:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561091b9ab10 con 0x561091c98d40 2024-09-15T22:18:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561091bef240 con 0x561091c8e480 2024-09-15T22:18:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5bed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8db60 0x561091c8df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5bed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8db60 0x561091c8df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56342/0 (socket says 172.21.15.5:56342) 2024-09-15T22:18:22.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5bed9d640 1 -- 172.21.15.5:0/3558191010 learned_addr learned my addr 172.21.15.5:0/3558191010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5be59c640 1 --2- 172.21.15.5:0/3558191010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8e480 0x561091c98800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff5a80a33b0 0x7ff5a80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5af7fe640 1 -- 172.21.15.5:0/3013516378 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7ff5b40cf3e0 con 0x7ff5c0102790 2024-09-15T22:18:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/3558191010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c98d40 0x561091c9b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5be59c640 1 -- 172.21.15.5:0/3558191010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c98d40 msgr2=0x561091c9b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5be59c640 1 --2- 172.21.15.5:0/3558191010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c98d40 0x561091c9b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5be59c640 1 -- 172.21.15.5:0/3558191010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8db60 msgr2=0x561091c8df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5be59c640 1 --2- 172.21.15.5:0/3558191010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8db60 0x561091c8df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5be59c640 1 -- 172.21.15.5:0/3558191010 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561091ba71f0 con 0x561091c8e480 2024-09-15T22:18:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/3558191010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8db60 0x561091c8df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.503+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff5a80a33b0 0x7ff5a80a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8012cf0 tx=0x7ff5b80094a0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.504+0000 7ff5be59c640 1 --2- 172.21.15.5:0/3558191010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8e480 0x561091c98800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005940 tx=0x7ff5b0031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70ea054fe45f3531 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.504+0000 7ff5acff9640 1 -- 172.21.15.5:0/3558191010 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0040070 con 0x561091c8e480 2024-09-15T22:18:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.504+0000 7ff5acff9640 1 -- 172.21.15.5:0/3558191010 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b0005020 con 0x561091c8e480 2024-09-15T22:18:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.504+0000 7ff5acff9640 1 -- 172.21.15.5:0/3558191010 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0035040 con 0x561091c8e480 2024-09-15T22:18:22.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.505+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/3558191010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8e480 msgr2=0x561091c98800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.505+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/3558191010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8e480 0x561091c98800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005940 tx=0x7ff5b0031630 comp rx=0 tx=0).stop 2024-09-15T22:18:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.505+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/3558191010 shutdown_connections 2024-09-15T22:18:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.505+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/3558191010 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c98d40 0x561091c9b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.505+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/3558191010 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8e480 0x561091c98800 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.505+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/3558191010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8db60 0x561091c8df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.505+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/3558191010 >> 172.21.15.5:0/3558191010 conn(0x561091c8a0f0 msgr2=0x561091c8a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.505+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/3558191010 shutdown_connections 2024-09-15T22:18:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.505+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/3558191010 wait complete. 2024-09-15T22:18:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.505+0000 7ff5c6ca0740 1 -- start start 2024-09-15T22:18:22.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.505+0000 7ff5c6ca0740 1 Processor -- start 2024-09-15T22:18:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8db60 0x561091c95b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091c960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c98d40 0x561091c965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561091c9b910 con 0x561091c8e480 2024-09-15T22:18:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561091badf10 con 0x561091c98d40 2024-09-15T22:18:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561091c9ba90 con 0x561091c8db60 2024-09-15T22:18:22.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5c6ca0740 10 client.0 _ll_register_callbacks cb 0x561091b76a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5bdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091c960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5bdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091c960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56354/0 (socket says 172.21.15.5:56354) 2024-09-15T22:18:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2643256349 learned_addr learned my addr 172.21.15.5:0/2643256349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5be59c640 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8db60 0x561091c95b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c98d40 0x561091c965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2643256349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8db60 msgr2=0x561091c95b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8db60 0x561091c95b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2643256349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c98d40 msgr2=0x561091c965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c98d40 0x561091c965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.506+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/2643256349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610919646a0 con 0x561091c8e480 2024-09-15T22:18:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.507+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091c960a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4097da0 tx=0x7ff5b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736ebd2088cc0b8b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.507+0000 7ff5a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.507+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/2643256349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b401e070 con 0x561091c8e480 2024-09-15T22:18:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.507+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/2643256349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7ff5b4017360 con 0x561091c8e480 2024-09-15T22:18:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.507+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2643256349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5b0005020 con 0x561091c8e480 2024-09-15T22:18:22.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.507+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/2643256349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5b409b3d0 con 0x561091c8e480 2024-09-15T22:18:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.507+0000 7ff5c6ca0740 10 client.17172 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.507+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2643256349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561091ca7460 con 0x561091c8e480 2024-09-15T22:18:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.507+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/2643256349 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b0005020 con 0x561091c8e480 2024-09-15T22:18:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.507+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/2643256349 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff5b4010ca0 con 0x561091c8e480 2024-09-15T22:18:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.508+0000 7ff5c6ca0740 10 client.17172 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:22.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.508+0000 7ff5c6ca0740 10 client.17172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.508+0000 7ff5c6ca0740 20 client.17172 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.508+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561091ca88c0 0x561091caad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.508+0000 7ff5c6ca0740 4 client.17172 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.508+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2643256349 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561091c007f0 con 0x561091ca88c0 2024-09-15T22:18:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.508+0000 7ff5be59c640 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561091ca88c0 0x561091caad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.508+0000 7ff5be59c640 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561091ca88c0 0x561091caad30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88d4f5688725c19b server_cookie=b8e48b50655d419d in_seq=0 out_seq=0 2024-09-15T22:18:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.508+0000 7ff5a5ffb640 10 client.17172 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.509+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/2643256349 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561091c007f0 con 0x561091ca88c0 2024-09-15T22:18:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.509+0000 7ff5a5ffb640 10 client.17172 handle_command_reply: tid=0 2024-09-15T22:18:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.509+0000 7ff5c6ca0740 1 client.17172 shutdown 2024-09-15T22:18:22.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.509+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2643256349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 msgr2=0x561091c960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.509+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091c960a0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4097da0 tx=0x7ff5b40046c0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.509+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2643256349 shutdown_connections 2024-09-15T22:18:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.510+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561091ca88c0 0x561091caad30 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.510+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c98d40 0x561091c965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.510+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091c960a0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.510+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/2643256349 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8db60 0x561091c95b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.510+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2643256349 >> 172.21.15.5:0/2643256349 conn(0x561091c8a0f0 msgr2=0x561091c8fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.510+0000 7ff5a5ffb640 0 client.17172 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.510+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2643256349 shutdown_connections 2024-09-15T22:18:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.510+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2643256349 wait complete. 2024-09-15T22:18:22.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.510+0000 7ff5c6ca0740 20 client.17172 trim_cache size 0 max 16384 2024-09-15T22:18:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.527+0000 7ff5c6ca0740 1 Processor -- start 2024-09-15T22:18:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.527+0000 7ff5c6ca0740 1 -- start start 2024-09-15T22:18:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.527+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c8db60 0x561091cb05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.527+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8e480 0x561091cb0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.527+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c98d40 0x561091c92de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.527+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c01c0880 con 0x561091c98d40 2024-09-15T22:18:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.527+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c01be150 con 0x561091c8db60 2024-09-15T22:18:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.527+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c01bbfc0 con 0x561091c8e480 2024-09-15T22:18:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5bed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c98d40 0x561091c92de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5bed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c98d40 0x561091c92de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56356/0 (socket says 172.21.15.5:56356) 2024-09-15T22:18:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5bed9d640 1 -- 172.21.15.5:0/2020728304 learned_addr learned my addr 172.21.15.5:0/2020728304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:22.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5be59c640 1 --2- 172.21.15.5:0/2020728304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c8db60 0x561091cb05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/2020728304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8e480 0x561091cb0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5bed9d640 1 -- 172.21.15.5:0/2020728304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8e480 msgr2=0x561091cb0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:22.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/2020728304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8e480 0x561091cb0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5bed9d640 1 -- 172.21.15.5:0/2020728304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c8db60 msgr2=0x561091cb05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:22.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/2020728304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c8db60 0x561091cb05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5bed9d640 1 -- 172.21.15.5:0/2020728304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561091c85830 con 0x561091c98d40 2024-09-15T22:18:22.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5be59c640 1 --2- 172.21.15.5:0/2020728304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c8db60 0x561091cb05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:22.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/2020728304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c98d40 0x561091c92de0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009fd0 tx=0x7ff5b80097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f81e2bfd2948ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:22.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5a7fff640 1 -- 172.21.15.5:0/2020728304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b8012ea0 con 0x561091c98d40 2024-09-15T22:18:22.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5a7fff640 1 -- 172.21.15.5:0/2020728304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b80109d0 con 0x561091c98d40 2024-09-15T22:18:22.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.528+0000 7ff5a7fff640 1 -- 172.21.15.5:0/2020728304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80056f0 con 0x561091c98d40 2024-09-15T22:18:22.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.529+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2020728304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c98d40 msgr2=0x561091c92de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:22.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.529+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/2020728304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c98d40 0x561091c92de0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8009fd0 tx=0x7ff5b80097d0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.529+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2020728304 shutdown_connections 2024-09-15T22:18:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.529+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/2020728304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c98d40 0x561091c92de0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.529+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/2020728304 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c8e480 0x561091cb0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.529+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/2020728304 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c8db60 0x561091cb05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.529+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2020728304 >> 172.21.15.5:0/2020728304 conn(0x561091c8a0f0 msgr2=0x561091c97ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:22.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.529+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2020728304 shutdown_connections 2024-09-15T22:18:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.529+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/2020728304 wait complete. 2024-09-15T22:18:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.529+0000 7ff5c6ca0740 1 -- start start 2024-09-15T22:18:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5c6ca0740 1 Processor -- start 2024-09-15T22:18:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c8db60 0x561091ca0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091ca1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5c6ca0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c98d40 0x561091c9dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c01bc610 con 0x561091c8e480 2024-09-15T22:18:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c01bda80 con 0x561091c8db60 2024-09-15T22:18:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5c6ca0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c01bcce0 con 0x561091c98d40 2024-09-15T22:18:22.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5c6ca0740 10 client.0 _ll_register_callbacks cb 0x7ff5b4009230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:22.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5bdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091ca1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5bdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091ca1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56362/0 (socket says 172.21.15.5:56362) 2024-09-15T22:18:22.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/477210489 learned_addr learned my addr 172.21.15.5:0/477210489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:22.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5be59c640 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c8db60 0x561091ca0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5bed9d640 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c98d40 0x561091c9dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/477210489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c98d40 msgr2=0x561091c9dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:22.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c98d40 0x561091c9dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/477210489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c8db60 msgr2=0x561091ca0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:22.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c8db60 0x561091ca0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.530+0000 7ff5bdd9b640 1 -- 172.21.15.5:0/477210489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561091c9e2f0 con 0x561091c8e480 2024-09-15T22:18:22.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5bdd9b640 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091ca1220 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff5b409b0d0 tx=0x7ff5b409bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17abf934d83c8241 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:22.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5a5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:22.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/477210489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b401e040 con 0x561091c8e480 2024-09-15T22:18:22.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/477210489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7ff5b40b42a0 con 0x561091c8e480 2024-09-15T22:18:22.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/477210489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5b80109d0 con 0x561091c8e480 2024-09-15T22:18:22.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/477210489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff5b409b1a0 con 0x561091c8e480 2024-09-15T22:18:22.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5c6ca0740 10 client.17178 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:22.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/477210489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561091d13f30 con 0x561091c8e480 2024-09-15T22:18:22.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/477210489 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5b80109d0 con 0x561091c8e480 2024-09-15T22:18:22.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/477210489 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7ff5b409a820 con 0x561091c8e480 2024-09-15T22:18:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5c6ca0740 10 client.17178 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5c6ca0740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5c6ca0740 20 client.17178 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561091d144d0 0x561091d168f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5c6ca0740 4 client.17178 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:18:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.531+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/477210489 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561091bfad00 con 0x561091d144d0 2024-09-15T22:18:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.532+0000 7ff5be59c640 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561091d144d0 0x561091d168f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.532+0000 7ff5be59c640 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561091d144d0 0x561091d168f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2900a9d9b11c12ba server_cookie=dc7319859f58655a in_seq=0 out_seq=0 2024-09-15T22:18:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.532+0000 7ff5a5ffb640 10 client.17178 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.533+0000 7ff5a5ffb640 1 -- 172.21.15.5:0/477210489 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561091bfad00 con 0x561091d144d0 2024-09-15T22:18:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.533+0000 7ff5a5ffb640 10 client.17178 handle_command_reply: tid=0 2024-09-15T22:18:22.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.533+0000 7ff5c6ca0740 1 client.17178 shutdown 2024-09-15T22:18:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.533+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/477210489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 msgr2=0x561091ca1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.533+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091ca1220 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff5b409b0d0 tx=0x7ff5b409bb50 comp rx=0 tx=0).stop 2024-09-15T22:18:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/477210489 shutdown_connections 2024-09-15T22:18:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561091c98d40 0x561091c9dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561091c8e480 0x561091ca1220 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561091d144d0 0x561091d168f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c6ca0740 1 --2- 172.21.15.5:0/477210489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561091c8db60 0x561091ca0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/477210489 >> 172.21.15.5:0/477210489 conn(0x561091c8a0f0 msgr2=0x561091c971d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5a5ffb640 0 client.17178 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/477210489 shutdown_connections 2024-09-15T22:18:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c6ca0740 1 -- 172.21.15.5:0/477210489 wait complete. 2024-09-15T22:18:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c6ca0740 20 client.17178 trim_cache size 0 max 16384 2024-09-15T22:18:22.556 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:18:22.556 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:18:22.556 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "4d16ecb4-0b34-4ff0-ad56-c017fb7d4d11", 2024-09-15T22:18:22.557 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:18:22.557 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:18:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3013516378 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff5a80a33b0 msgr2=0x7ff5a80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c48c7640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff5a80a33b0 0x7ff5a80a5870 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff5b8012cf0 tx=0x7ff5b80094a0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3013516378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c0102790 msgr2=0x7ff5c019f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.534+0000 7ff5c48c7640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c0102790 0x7ff5c019f590 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400b4a0 tx=0x7ff5b400b960 comp rx=0 tx=0).stop 2024-09-15T22:18:22.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.535+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3013516378 shutdown_connections 2024-09-15T22:18:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.535+0000 7ff5c48c7640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff5a80a33b0 0x7ff5a80a5870 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.535+0000 7ff5c48c7640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff5c0106f10 0x7ff5c010dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.535+0000 7ff5c48c7640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5c0102790 0x7ff5c019f590 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.535+0000 7ff5c48c7640 1 --2- 172.21.15.5:0/3013516378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff5c0101de0 0x7ff5c019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.535+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3013516378 >> 172.21.15.5:0/3013516378 conn(0x7ff5c00fd8c0 msgr2=0x7ff5c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.535+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3013516378 shutdown_connections 2024-09-15T22:18:22.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:22.535+0000 7ff5c48c7640 1 -- 172.21.15.5:0/3013516378 wait complete. 2024-09-15T22:18:23.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:18:23.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:23 smithi099 ceph-mon[34892]: pgmap v293: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-09-15T22:18:23.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:23 smithi129 ceph-mon[35314]: pgmap v293: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-09-15T22:18:23.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:23 smithi005 ceph-mon[30944]: pgmap v293: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.8 MiB/s wr, 209 op/s 2024-09-15T22:18:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:24.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.041+0000 7f0a6b30f640 1 -- 172.21.15.5:0/687857754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a6410c150 msgr2=0x7f0a6410e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:24.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.041+0000 7f0a6b30f640 1 --2- 172.21.15.5:0/687857754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a6410c150 0x7f0a6410e5f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c009ca0 tx=0x7f0a5c02a950 comp rx=0 tx=0).stop 2024-09-15T22:18:24.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.041+0000 7f0a6b30f640 1 -- 172.21.15.5:0/687857754 shutdown_connections 2024-09-15T22:18:24.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.041+0000 7f0a6b30f640 1 --2- 172.21.15.5:0/687857754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a6410c150 0x7f0a6410e5f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.041+0000 7f0a6b30f640 1 --2- 172.21.15.5:0/687857754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a64103640 0x7f0a6410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.041+0000 7f0a6b30f640 1 --2- 172.21.15.5:0/687857754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a64102d20 0x7f0a64103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.041+0000 7f0a6b30f640 1 -- 172.21.15.5:0/687857754 >> 172.21.15.5:0/687857754 conn(0x7f0a640fc780 msgr2=0x7f0a640feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:24.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.041+0000 7f0a6b30f640 1 -- 172.21.15.5:0/687857754 shutdown_connections 2024-09-15T22:18:24.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.041+0000 7f0a6b30f640 1 -- 172.21.15.5:0/687857754 wait complete. 2024-09-15T22:18:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.041+0000 7f0a6b30f640 1 Processor -- start 2024-09-15T22:18:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.041+0000 7f0a6b30f640 1 -- start start 2024-09-15T22:18:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a6b30f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a64102d20 0x7f0a64198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a6b30f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a64103640 0x7f0a64198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a6b30f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a6410c150 0x7f0a6419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a6b30f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a64111f30 con 0x7f0a6410c150 2024-09-15T22:18:24.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a6b30f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a64111db0 con 0x7f0a64103640 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a6b30f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641120b0 con 0x7f0a64102d20 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a698ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a6410c150 0x7f0a6419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a698ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a6410c150 0x7f0a6419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52268/0 (socket says 172.21.15.5:52268) 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a698ac640 1 -- 172.21.15.5:0/3224322309 learned_addr learned my addr 172.21.15.5:0/3224322309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a688aa640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a64103640 0x7f0a64198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a690ab640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a64102d20 0x7f0a64198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a698ac640 1 -- 172.21.15.5:0/3224322309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a64102d20 msgr2=0x7f0a64198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a698ac640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a64102d20 0x7f0a64198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a698ac640 1 -- 172.21.15.5:0/3224322309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a64103640 msgr2=0x7f0a64198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a698ac640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a64103640 0x7f0a64198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a698ac640 1 -- 172.21.15.5:0/3224322309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6419d3e0 con 0x7f0a6410c150 2024-09-15T22:18:24.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.042+0000 7f0a698ac640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a6410c150 0x7f0a6419cd00 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c004af0 tx=0x7f0a5c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:24.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.043+0000 7f0a5a7fc640 1 -- 172.21.15.5:0/3224322309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c034040 con 0x7f0a6410c150 2024-09-15T22:18:24.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.043+0000 7f0a5a7fc640 1 -- 172.21.15.5:0/3224322309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a5c030920 con 0x7f0a6410c150 2024-09-15T22:18:24.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.043+0000 7f0a6b30f640 1 -- 172.21.15.5:0/3224322309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6419d670 con 0x7f0a6410c150 2024-09-15T22:18:24.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.043+0000 7f0a5a7fc640 1 -- 172.21.15.5:0/3224322309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a5c0047b0 con 0x7f0a6410c150 2024-09-15T22:18:24.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.043+0000 7f0a6b30f640 1 -- 172.21.15.5:0/3224322309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a641a4fb0 con 0x7f0a6410c150 2024-09-15T22:18:24.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.045+0000 7f0a5a7fc640 1 -- 172.21.15.5:0/3224322309 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0a5c037020 con 0x7f0a6410c150 2024-09-15T22:18:24.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a6d6e8740 1 Processor -- start 2024-09-15T22:18:24.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a6d6e8740 1 -- start start 2024-09-15T22:18:24.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249ad380 0x5626249ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249bc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249bca90 0x5626249bee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5626249132e0 con 0x5626249adca0 2024-09-15T22:18:24.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a5a7fc640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a500a33b0 0x7f0a500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5626248bace0 con 0x5626249bca90 2024-09-15T22:18:24.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56262490f2a0 con 0x5626249ad380 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a690ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249bc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a690ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249bc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52270/0 (socket says 172.21.15.5:52270) 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a698ac640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a500a33b0 0x7f0a500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a690ab640 1 -- 172.21.15.5:0/1848808144 learned_addr learned my addr 172.21.15.5:0/1848808144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a688aa640 1 --2- 172.21.15.5:0/1848808144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249bca90 0x5626249bee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.046+0000 7f0a5a7fc640 1 -- 172.21.15.5:0/3224322309 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7f0a5c0f0df0 con 0x7f0a6410c150 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.047+0000 7f0a688aa640 1 --2- 172.21.15.5:0/1848808144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249ad380 0x5626249ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.047+0000 7f0a690ab640 1 -- 172.21.15.5:0/1848808144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249ad380 msgr2=0x5626249ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.047+0000 7f0a690ab640 1 --2- 172.21.15.5:0/1848808144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249ad380 0x5626249ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.047+0000 7f0a690ab640 1 -- 172.21.15.5:0/1848808144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249bca90 msgr2=0x5626249bee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.047+0000 7f0a690ab640 1 --2- 172.21.15.5:0/1848808144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249bca90 0x5626249bee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.047+0000 7f0a690ab640 1 -- 172.21.15.5:0/1848808144 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56262489f440 con 0x5626249adca0 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.047+0000 7f0a698ac640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a500a33b0 0x7f0a500a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0c54c0 tx=0x7f0a5c0b9110 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:24.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.047+0000 7f0a690ab640 1 --2- 172.21.15.5:0/1848808144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249bc550 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000bcd0 tx=0x7f0a600107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952b389d3c795a95 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:24.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.047+0000 7f0a4f7fe640 1 -- 172.21.15.5:0/1848808144 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60005220 con 0x5626249adca0 2024-09-15T22:18:24.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.047+0000 7f0a4f7fe640 1 -- 172.21.15.5:0/1848808144 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a60004c80 con 0x5626249adca0 2024-09-15T22:18:24.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.047+0000 7f0a4f7fe640 1 -- 172.21.15.5:0/1848808144 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600117c0 con 0x5626249adca0 2024-09-15T22:18:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.048+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1848808144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 msgr2=0x5626249bc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.048+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/1848808144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249bc550 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000bcd0 tx=0x7f0a600107a0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.048+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1848808144 shutdown_connections 2024-09-15T22:18:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.048+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/1848808144 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249bca90 0x5626249bee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.048+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/1848808144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249bc550 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.048+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/1848808144 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249ad380 0x5626249ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.048+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1848808144 >> 172.21.15.5:0/1848808144 conn(0x5626249a9a30 msgr2=0x5626249a9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.048+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1848808144 shutdown_connections 2024-09-15T22:18:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.048+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1848808144 wait complete. 2024-09-15T22:18:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.048+0000 7f0a6d6e8740 1 -- start start 2024-09-15T22:18:24.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.048+0000 7f0a6d6e8740 1 Processor -- start 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249ad380 0x5626249bc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249bca90 0x5626249b9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5626249bf660 con 0x5626249adca0 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5626248ce050 con 0x5626249ad380 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5626249bf7e0 con 0x5626249bca90 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a6d6e8740 10 client.0 _ll_register_callbacks cb 0x56262490e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a690ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a690ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249b96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52274/0 (socket says 172.21.15.5:52274) 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a690ab640 1 -- 172.21.15.5:0/1745283042 learned_addr learned my addr 172.21.15.5:0/1745283042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a688aa640 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249ad380 0x5626249bc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a688aa640 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249bca90 0x5626249b9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a690ab640 1 -- 172.21.15.5:0/1745283042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249bca90 msgr2=0x5626249b9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a690ab640 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249bca90 0x5626249b9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a690ab640 1 -- 172.21.15.5:0/1745283042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249ad380 msgr2=0x5626249bc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a690ab640 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249ad380 0x5626249bc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a690ab640 1 -- 172.21.15.5:0/1745283042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626246843c0 con 0x5626249adca0 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.049+0000 7f0a688aa640 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249bca90 0x5626249b9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a690ab640 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249b96b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0a60005750 tx=0x7f0a60005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686cc925da6208b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a4cff9640 1 -- 172.21.15.5:0/1745283042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000b8c0 con 0x5626249adca0 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1745283042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a60004c80 con 0x5626249adca0 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a4cff9640 1 -- 172.21.15.5:0/1745283042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7f0a600048d0 con 0x5626249adca0 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a4cff9640 1 -- 172.21.15.5:0/1745283042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0a6002f050 con 0x5626249adca0 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a6d6e8740 10 client.17190 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1745283042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5626249cb070 con 0x5626249adca0 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a4cff9640 1 -- 172.21.15.5:0/1745283042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a60004c80 con 0x5626249adca0 2024-09-15T22:18:24.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a4cff9640 1 -- 172.21.15.5:0/1745283042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0a60003b10 con 0x5626249adca0 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a6d6e8740 10 client.17190 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.050+0000 7f0a6d6e8740 10 client.17190 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.051+0000 7f0a6d6e8740 20 client.17190 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.051+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5626249cc4d0 0x5626249ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.051+0000 7f0a6d6e8740 4 client.17190 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.051+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1745283042 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562624920850 con 0x5626249cc4d0 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.051+0000 7f0a688aa640 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5626249cc4d0 0x5626249ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.051+0000 7f0a688aa640 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5626249cc4d0 0x5626249ce8c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b2c2a48d4a3a15 server_cookie=3ba9b43996488744 in_seq=0 out_seq=0 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.051+0000 7f0a4cff9640 10 client.17190 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a4cff9640 1 -- 172.21.15.5:0/1745283042 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562624920850 con 0x5626249cc4d0 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a4cff9640 10 client.17190 handle_command_reply: tid=0 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a6d6e8740 1 client.17190 shutdown 2024-09-15T22:18:24.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1745283042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 msgr2=0x5626249b96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249b96b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f0a60005750 tx=0x7f0a60005230 comp rx=0 tx=0).stop 2024-09-15T22:18:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1745283042 shutdown_connections 2024-09-15T22:18:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249bca90 0x5626249b9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249adca0 0x5626249b96b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5626249cc4d0 0x5626249ce8c0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/1745283042 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249ad380 0x5626249bc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1745283042 >> 172.21.15.5:0/1745283042 conn(0x5626249a9a30 msgr2=0x5626249af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a4cff9640 0 client.17190 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1745283042 shutdown_connections 2024-09-15T22:18:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.052+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/1745283042 wait complete. 2024-09-15T22:18:24.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.053+0000 7f0a6d6e8740 20 client.17190 trim_cache size 0 max 16384 2024-09-15T22:18:24.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a6d6e8740 1 Processor -- start 2024-09-15T22:18:24.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a6d6e8740 1 -- start start 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249c8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249adca0 0x5626249b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249bca90 0x5626249b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641af820 con 0x5626249ad380 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641ad0f0 con 0x5626249adca0 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641aaf60 con 0x5626249bca90 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a688aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a688aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52282/0 (socket says 172.21.15.5:52282) 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a688aa640 1 -- 172.21.15.5:0/4000059430 learned_addr learned my addr 172.21.15.5:0/4000059430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a690ab640 1 --2- 172.21.15.5:0/4000059430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249adca0 0x5626249b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a688aa640 1 -- 172.21.15.5:0/4000059430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249bca90 msgr2=0x5626249b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a688aa640 1 --2- 172.21.15.5:0/4000059430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249bca90 0x5626249b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a688aa640 1 -- 172.21.15.5:0/4000059430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249adca0 msgr2=0x5626249b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a688aa640 1 --2- 172.21.15.5:0/4000059430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249adca0 0x5626249b8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.070+0000 7f0a688aa640 1 -- 172.21.15.5:0/4000059430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626249b9f80 con 0x5626249ad380 2024-09-15T22:18:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.071+0000 7f0a688aa640 1 --2- 172.21.15.5:0/4000059430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249c8900 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009890 tx=0x7f0a54010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21cd7281e7363a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.071+0000 7f0a4effd640 1 -- 172.21.15.5:0/4000059430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54009ce0 con 0x5626249ad380 2024-09-15T22:18:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.071+0000 7f0a4effd640 1 -- 172.21.15.5:0/4000059430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a54005500 con 0x5626249ad380 2024-09-15T22:18:24.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.071+0000 7f0a4effd640 1 -- 172.21.15.5:0/4000059430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a540216a0 con 0x5626249ad380 2024-09-15T22:18:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.071+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/4000059430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 msgr2=0x5626249c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.071+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/4000059430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249c8900 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009890 tx=0x7f0a54010900 comp rx=0 tx=0).stop 2024-09-15T22:18:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.071+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/4000059430 shutdown_connections 2024-09-15T22:18:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.071+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/4000059430 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249bca90 0x5626249b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.071+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/4000059430 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249adca0 0x5626249b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.071+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/4000059430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249c8900 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.071+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/4000059430 >> 172.21.15.5:0/4000059430 conn(0x5626249a9a30 msgr2=0x5626249a9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.072+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/4000059430 shutdown_connections 2024-09-15T22:18:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.072+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/4000059430 wait complete. 2024-09-15T22:18:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.072+0000 7f0a6d6e8740 1 -- start start 2024-09-15T22:18:24.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.072+0000 7f0a6d6e8740 1 Processor -- start 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.072+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249bffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.072+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249adca0 0x5626249c0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.072+0000 7f0a6d6e8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249bca90 0x562624a34e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.072+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641ab5b0 con 0x5626249ad380 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.072+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641aca20 con 0x5626249bca90 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.072+0000 7f0a6d6e8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0a641abc80 con 0x5626249adca0 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a6d6e8740 10 client.0 _ll_register_callbacks cb 0x7f0a6000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a688aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249bffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a688aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249bffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52294/0 (socket says 172.21.15.5:52294) 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a688aa640 1 -- 172.21.15.5:0/724060821 learned_addr learned my addr 172.21.15.5:0/724060821 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a690ab640 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249adca0 0x5626249c0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a688aa640 1 -- 172.21.15.5:0/724060821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249adca0 msgr2=0x5626249c0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a688aa640 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249adca0 0x5626249c0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a688aa640 1 -- 172.21.15.5:0/724060821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249bca90 msgr2=0x562624a34e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a688aa640 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249bca90 0x562624a34e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a688aa640 1 -- 172.21.15.5:0/724060821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562624a353c0 con 0x5626249ad380 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a688aa640 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249bffd0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009630 tx=0x7f0a54005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a619274ce27fa99 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a4cff9640 1 -- 172.21.15.5:0/724060821 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a54004700 con 0x5626249ad380 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a4cff9640 1 -- 172.21.15.5:0/724060821 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7f0a540048a0 con 0x5626249ad380 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/724060821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a54005500 con 0x5626249ad380 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.073+0000 7f0a4cff9640 1 -- 172.21.15.5:0/724060821 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0a54004aa0 con 0x5626249ad380 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.074+0000 7f0a6d6e8740 10 client.17196 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.074+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/724060821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562624a365e0 con 0x5626249ad380 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.074+0000 7f0a4cff9640 1 -- 172.21.15.5:0/724060821 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a54005500 con 0x5626249ad380 2024-09-15T22:18:24.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.074+0000 7f0a4cff9640 1 -- 172.21.15.5:0/724060821 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0a54021600 con 0x5626249ad380 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.074+0000 7f0a6d6e8740 10 client.17196 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.074+0000 7f0a6d6e8740 10 client.17196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.074+0000 7f0a6d6e8740 20 client.17196 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.074+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562624a36b50 0x562624a38f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.074+0000 7f0a6d6e8740 4 client.17196 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.074+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/724060821 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56262491ad60 con 0x562624a36b50 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.074+0000 7f0a690ab640 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562624a36b50 0x562624a38f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a690ab640 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562624a36b50 0x562624a38f70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5dcd24a117475b8 server_cookie=a718073098a29e1b in_seq=0 out_seq=0 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a4cff9640 10 client.17196 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a4cff9640 1 -- 172.21.15.5:0/724060821 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56262491ad60 con 0x562624a36b50 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a4cff9640 10 client.17196 handle_command_reply: tid=0 2024-09-15T22:18:24.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 1 client.17196 shutdown 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/724060821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 msgr2=0x5626249bffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249bffd0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009630 tx=0x7f0a54005090 comp rx=0 tx=0).stop 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/724060821 shutdown_connections 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562624a36b50 0x562624a38f70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5626249bca90 0x562624a34e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5626249adca0 0x5626249c0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 1 --2- 172.21.15.5:0/724060821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626249ad380 0x5626249bffd0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/724060821 >> 172.21.15.5:0/724060821 conn(0x5626249a9a30 msgr2=0x5626249b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a4cff9640 0 client.17196 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/724060821 shutdown_connections 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 1 -- 172.21.15.5:0/724060821 wait complete. 2024-09-15T22:18:24.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.075+0000 7f0a6d6e8740 20 client.17196 trim_cache size 0 max 16384 2024-09-15T22:18:24.084 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:18:24.084 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:18:24.084 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:18:24.084 INFO:teuthology.orchestra.run.smithi005.stdout: "4d16ecb4-0b34-4ff0-ad56-c017fb7d4d11": { 2024-09-15T22:18:24.084 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:18:24.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "4d16ecb4-0b34-4ff0-ad56-c017fb7d4d11", 2024-09-15T22:18:24.084 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:18:24.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:24.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:24.084 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:18:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 -- 172.21.15.5:0/3224322309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a500a33b0 msgr2=0x7f0a500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a500a33b0 0x7f0a500a5870 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c0c54c0 tx=0x7f0a5c0b9110 comp rx=0 tx=0).stop 2024-09-15T22:18:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 -- 172.21.15.5:0/3224322309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a6410c150 msgr2=0x7f0a6419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a6410c150 0x7f0a6419cd00 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f0a5c004af0 tx=0x7f0a5c02f5b0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 -- 172.21.15.5:0/3224322309 shutdown_connections 2024-09-15T22:18:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f0a500a33b0 0x7f0a500a5870 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a6410c150 0x7f0a6419cd00 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0a64103640 0x7f0a64198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 --2- 172.21.15.5:0/3224322309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0a64102d20 0x7f0a64198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 -- 172.21.15.5:0/3224322309 >> 172.21.15.5:0/3224322309 conn(0x7f0a640fc780 msgr2=0x7f0a640fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 -- 172.21.15.5:0/3224322309 shutdown_connections 2024-09-15T22:18:24.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:24.076+0000 7f0a6b30f640 1 -- 172.21.15.5:0/3224322309 wait complete. 2024-09-15T22:18:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:24 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:18:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:24 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:24 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:18:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:24 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:24 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:18:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:24 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:24 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:18:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:24 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:24.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:24 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:18:24.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:24 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:24.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:24 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:18:24.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:24 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:18:24.664 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 535.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:18:25.018 INFO:tasks.cephfs.filesystem:scrub status for tag:4d16ecb4-0b34-4ff0-ad56-c017fb7d4d11 - {'path': '/', 'tag': '4d16ecb4-0b34-4ff0-ad56-c017fb7d4d11', 'options': 'recursive,force'} 2024-09-15T22:18:25.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:25 smithi099 ceph-mon[34892]: pgmap v294: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-09-15T22:18:25.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:25 smithi129 ceph-mon[35314]: pgmap v294: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-09-15T22:18:25.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:25 smithi005 ceph-mon[30944]: pgmap v294: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-09-15T22:18:27.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:27 smithi129 ceph-mon[35314]: pgmap v295: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 141 KiB/s wr, 23 op/s 2024-09-15T22:18:27.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:27 smithi005 ceph-mon[30944]: pgmap v295: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 141 KiB/s wr, 23 op/s 2024-09-15T22:18:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:27 smithi099 ceph-mon[34892]: pgmap v295: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 141 KiB/s wr, 23 op/s 2024-09-15T22:18:28.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:28 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:18:28.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:28 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:18:28.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:28 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:18:29.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:29 smithi005 ceph-mon[30944]: pgmap v296: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-15T22:18:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:29 smithi099 ceph-mon[34892]: pgmap v296: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-15T22:18:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:29 smithi129 ceph-mon[35314]: pgmap v296: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 KiB/s wr, 7 op/s 2024-09-15T22:18:29.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 540.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:18:31.054 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:18:31.328 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:31.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:31 smithi005 ceph-mon[30944]: pgmap v297: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-15T22:18:31.392 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:31 smithi099 ceph-mon[34892]: pgmap v297: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-15T22:18:31.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:31 smithi129 ceph-mon[35314]: pgmap v297: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-09-15T22:18:31.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.799+0000 7f76e596b640 1 -- 172.21.15.5:0/1456747815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e0102570 msgr2=0x7f76e01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:31.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.799+0000 7f76e596b640 1 --2- 172.21.15.5:0/1456747815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e0102570 0x7f76e01029b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f76d4009900 tx=0x7f76d402a480 comp rx=0 tx=0).stop 2024-09-15T22:18:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.799+0000 7f76e596b640 1 -- 172.21.15.5:0/1456747815 shutdown_connections 2024-09-15T22:18:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.799+0000 7f76e596b640 1 --2- 172.21.15.5:0/1456747815 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e0102ef0 0x7f76e010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.799+0000 7f76e596b640 1 --2- 172.21.15.5:0/1456747815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e0102570 0x7f76e01029b0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.799+0000 7f76e596b640 1 --2- 172.21.15.5:0/1456747815 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76e0100e20 0x7f76e0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.799+0000 7f76e596b640 1 -- 172.21.15.5:0/1456747815 >> 172.21.15.5:0/1456747815 conn(0x7f76e00fc7d0 msgr2=0x7f76e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.799+0000 7f76e596b640 1 -- 172.21.15.5:0/1456747815 shutdown_connections 2024-09-15T22:18:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.799+0000 7f76e596b640 1 -- 172.21.15.5:0/1456747815 wait complete. 2024-09-15T22:18:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.799+0000 7f76e596b640 1 Processor -- start 2024-09-15T22:18:31.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76e596b640 1 -- start start 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76e596b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e0100e20 0x7f76e0199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76e596b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e0102570 0x7f76e0199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76e596b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76e0102ef0 0x7f76e01a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76e596b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76e0110f80 con 0x7f76e0102570 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76e596b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f76e0110e00 con 0x7f76e0100e20 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76e596b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f76e0111100 con 0x7f76e0102ef0 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76de7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e0102570 0x7f76e0199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76de7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e0102570 0x7f76e0199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52324/0 (socket says 172.21.15.5:52324) 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76de7fc640 1 -- 172.21.15.5:0/3018206793 learned_addr learned my addr 172.21.15.5:0/3018206793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76deffd640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e0100e20 0x7f76e0199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76df7fe640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76e0102ef0 0x7f76e01a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:31.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76de7fc640 1 -- 172.21.15.5:0/3018206793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76e0102ef0 msgr2=0x7f76e01a43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76de7fc640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76e0102ef0 0x7f76e01a43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76de7fc640 1 -- 172.21.15.5:0/3018206793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e0100e20 msgr2=0x7f76e0199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76de7fc640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e0100e20 0x7f76e0199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76de7fc640 1 -- 172.21.15.5:0/3018206793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e01a4910 con 0x7f76e0102570 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76df7fe640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76e0102ef0 0x7f76e01a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.800+0000 7f76de7fc640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e0102570 0x7f76e0199890 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f76d4006010 tx=0x7f76d4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.801+0000 7f76e4969640 1 -- 172.21.15.5:0/3018206793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d403f070 con 0x7f76e0102570 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.801+0000 7f76e4969640 1 -- 172.21.15.5:0/3018206793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d4031c30 con 0x7f76e0102570 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.801+0000 7f76e596b640 1 -- 172.21.15.5:0/3018206793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76e01a4ae0 con 0x7f76e0102570 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.801+0000 7f76e4969640 1 -- 172.21.15.5:0/3018206793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d4035030 con 0x7f76e0102570 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.801+0000 7f76e596b640 1 -- 172.21.15.5:0/3018206793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76e01a4f30 con 0x7f76e0102570 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.802+0000 7f76e596b640 1 -- 172.21.15.5:0/3018206793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76e0101200 con 0x7f76e0102570 2024-09-15T22:18:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.803+0000 7f76e4969640 1 -- 172.21.15.5:0/3018206793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f76d4042070 con 0x7f76e0102570 2024-09-15T22:18:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.804+0000 7f76e4969640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f76c80a33b0 0x7f76c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.804+0000 7f76e4969640 1 -- 172.21.15.5:0/3018206793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7f76d40f0da0 con 0x7f76e0102570 2024-09-15T22:18:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.804+0000 7f76deffd640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f76c80a33b0 0x7f76c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.804+0000 7f76deffd640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f76c80a33b0 0x7f76c80a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f76d0005c40 tx=0x7f76d000a570 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:31.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:31.807+0000 7f76e4969640 1 -- 172.21.15.5:0/3018206793 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76d4031de0 con 0x7f76e0102570 2024-09-15T22:18:32.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.018+0000 7f76e596b640 1 -- 172.21.15.5:0/3018206793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76e0109090 con 0x7f76e0102570 2024-09-15T22:18:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.019+0000 7f76e4969640 1 -- 172.21.15.5:0/3018206793 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f76d40bd380 con 0x7f76e0102570 2024-09-15T22:18:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:18:32.022 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:18:32.022 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:18:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.021+0000 7f76e596b640 1 -- 172.21.15.5:0/3018206793 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f76c80a33b0 msgr2=0x7f76c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.021+0000 7f76e596b640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f76c80a33b0 0x7f76c80a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f76d0005c40 tx=0x7f76d000a570 comp rx=0 tx=0).stop 2024-09-15T22:18:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.021+0000 7f76e596b640 1 -- 172.21.15.5:0/3018206793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e0102570 msgr2=0x7f76e0199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.021+0000 7f76e596b640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e0102570 0x7f76e0199890 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f76d4006010 tx=0x7f76d4003c10 comp rx=0 tx=0).stop 2024-09-15T22:18:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.021+0000 7f76e596b640 1 -- 172.21.15.5:0/3018206793 shutdown_connections 2024-09-15T22:18:32.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.021+0000 7f76e596b640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f76c80a33b0 0x7f76c80a5870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.021+0000 7f76e596b640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f76e0102ef0 0x7f76e01a43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.021+0000 7f76e596b640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e0102570 0x7f76e0199890 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.021+0000 7f76e596b640 1 --2- 172.21.15.5:0/3018206793 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f76e0100e20 0x7f76e0199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.021+0000 7f76e596b640 1 -- 172.21.15.5:0/3018206793 >> 172.21.15.5:0/3018206793 conn(0x7f76e00fc7d0 msgr2=0x7f76e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.022+0000 7f76e596b640 1 -- 172.21.15.5:0/3018206793 shutdown_connections 2024-09-15T22:18:32.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:32.022+0000 7f76e596b640 1 -- 172.21.15.5:0/3018206793 wait complete. 2024-09-15T22:18:32.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:32 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3018206793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:32.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:32 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3018206793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:32.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:32 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3018206793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:18:32.841 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id 9589889c --release --await' 2024-09-15T22:18:32.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id 9589889c --release --await 2024-09-15T22:18:33.101 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:33.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:33 smithi005 ceph-mon[30944]: pgmap v298: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-09-15T22:18:33.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:33 smithi099 ceph-mon[34892]: pgmap v298: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-09-15T22:18:33.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:33 smithi129 ceph-mon[35314]: pgmap v298: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-09-15T22:18:33.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.596+0000 7fce135ad640 1 -- 172.21.15.5:0/4133689585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c106f10 msgr2=0x7fce0c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.596+0000 7fce135ad640 1 --2- 172.21.15.5:0/4133689585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c106f10 0x7fce0c10ea90 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fce04009ca0 tx=0x7fce0402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.596+0000 7fce135ad640 1 -- 172.21.15.5:0/4133689585 shutdown_connections 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.596+0000 7fce135ad640 1 --2- 172.21.15.5:0/4133689585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c106f10 0x7fce0c10ea90 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.596+0000 7fce135ad640 1 --2- 172.21.15.5:0/4133689585 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce0c102790 0x7fce0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.596+0000 7fce135ad640 1 --2- 172.21.15.5:0/4133689585 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce0c101de0 0x7fce0c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.596+0000 7fce135ad640 1 -- 172.21.15.5:0/4133689585 >> 172.21.15.5:0/4133689585 conn(0x7fce0c0fd8c0 msgr2=0x7fce0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.597+0000 7fce135ad640 1 -- 172.21.15.5:0/4133689585 shutdown_connections 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.597+0000 7fce135ad640 1 -- 172.21.15.5:0/4133689585 wait complete. 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.597+0000 7fce135ad640 1 Processor -- start 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.597+0000 7fce135ad640 1 -- start start 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.597+0000 7fce135ad640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce0c101de0 0x7fce0c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.597+0000 7fce135ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c102790 0x7fce0c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.597+0000 7fce135ad640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce0c106f10 0x7fce0c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.597+0000 7fce135ad640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c111f70 con 0x7fce0c102790 2024-09-15T22:18:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.597+0000 7fce135ad640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c111df0 con 0x7fce0c101de0 2024-09-15T22:18:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.597+0000 7fce135ad640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1120f0 con 0x7fce0c106f10 2024-09-15T22:18:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce10b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c102790 0x7fce0c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce10b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c102790 0x7fce0c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60924/0 (socket says 172.21.15.5:60924) 2024-09-15T22:18:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce10b48640 1 -- 172.21.15.5:0/3566786360 learned_addr learned my addr 172.21.15.5:0/3566786360 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce11b4a640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce0c106f10 0x7fce0c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce11349640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce0c101de0 0x7fce0c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce10b48640 1 -- 172.21.15.5:0/3566786360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce0c106f10 msgr2=0x7fce0c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce10b48640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce0c106f10 0x7fce0c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce10b48640 1 -- 172.21.15.5:0/3566786360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce0c101de0 msgr2=0x7fce0c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce10b48640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce0c101de0 0x7fce0c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce10b48640 1 -- 172.21.15.5:0/3566786360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c19d360 con 0x7fce0c102790 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce10b48640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c102790 0x7fce0c1988f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00d770 tx=0x7fcdfc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce027fc640 1 -- 172.21.15.5:0/3566786360 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc017070 con 0x7fce0c102790 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce135ad640 1 -- 172.21.15.5:0/3566786360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c19d650 con 0x7fce0c102790 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce027fc640 1 -- 172.21.15.5:0/3566786360 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdfc005500 con 0x7fce0c102790 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce027fc640 1 -- 172.21.15.5:0/3566786360 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc004e00 con 0x7fce0c102790 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.598+0000 7fce135ad640 1 -- 172.21.15.5:0/3566786360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c1a4f30 con 0x7fce0c102790 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.600+0000 7fce027fc640 1 -- 172.21.15.5:0/3566786360 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcdfc004fa0 con 0x7fce0c102790 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce15986740 1 Processor -- start 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce15986740 1 -- start start 2024-09-15T22:18:33.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce15986740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cb9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce15986740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cc8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce15986740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890ccaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce15986740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564890c1e620 con 0x564890cb9880 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce15986740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564890bc5e50 con 0x564890cb8f60 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce15986740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564890c1a5e0 con 0x564890cc8650 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce027fc640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcdf80a33b0 0x7fcdf80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce11349640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cc8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce11349640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cc8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60938/0 (socket says 172.21.15.5:60938) 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce11b4a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cb9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce11349640 1 -- 172.21.15.5:0/2963792467 learned_addr learned my addr 172.21.15.5:0/2963792467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.601+0000 7fce11b4a640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcdf80a33b0 0x7fcdf80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fce10b48640 1 --2- 172.21.15.5:0/2963792467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890ccaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fce027fc640 1 -- 172.21.15.5:0/3566786360 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7fcdfc0cfb40 con 0x7fce0c102790 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fce11b4a640 1 -- 172.21.15.5:0/2963792467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 msgr2=0x564890ccaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fce11b4a640 1 --2- 172.21.15.5:0/2963792467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890ccaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fce11b4a640 1 -- 172.21.15.5:0/2963792467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 msgr2=0x564890cc8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fce11b4a640 1 --2- 172.21.15.5:0/2963792467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cc8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fce11b4a640 1 -- 172.21.15.5:0/2963792467 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564890bb6d40 con 0x564890cb8f60 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fce10b48640 1 --2- 172.21.15.5:0/2963792467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890ccaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fce11b4a640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcdf80a33b0 0x7fcdf80a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce04009a60 tx=0x7fce04036000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fce11b4a640 1 --2- 172.21.15.5:0/2963792467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cb9340 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fce04006010 tx=0x7fce04031760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c5d042f6c083509 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fcdf77fe640 1 -- 172.21.15.5:0/2963792467 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04055070 con 0x564890cb8f60 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fcdf77fe640 1 -- 172.21.15.5:0/2963792467 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce04036070 con 0x564890cb8f60 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.602+0000 7fcdf77fe640 1 -- 172.21.15.5:0/2963792467 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce04049660 con 0x564890cb8f60 2024-09-15T22:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.603+0000 7fce15986740 1 -- 172.21.15.5:0/2963792467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 msgr2=0x564890cb9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.603+0000 7fce15986740 1 --2- 172.21.15.5:0/2963792467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cb9340 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fce04006010 tx=0x7fce04031760 comp rx=0 tx=0).stop 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.603+0000 7fce15986740 1 -- 172.21.15.5:0/2963792467 shutdown_connections 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.603+0000 7fce15986740 1 --2- 172.21.15.5:0/2963792467 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890ccaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.603+0000 7fce15986740 1 --2- 172.21.15.5:0/2963792467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cc8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.603+0000 7fce15986740 1 --2- 172.21.15.5:0/2963792467 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cb9340 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.603+0000 7fce15986740 1 -- 172.21.15.5:0/2963792467 >> 172.21.15.5:0/2963792467 conn(0x564890cb55f0 msgr2=0x564890cb59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.603+0000 7fce15986740 1 -- 172.21.15.5:0/2963792467 shutdown_connections 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.603+0000 7fce15986740 1 -- 172.21.15.5:0/2963792467 wait complete. 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.603+0000 7fce15986740 1 -- start start 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce15986740 1 Processor -- start 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce15986740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cc5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce15986740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cc5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce15986740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890cc60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce15986740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564890ccb0a0 con 0x564890cb9880 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce15986740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564890ba6f10 con 0x564890cb8f60 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce15986740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564890ccb220 con 0x564890cc8650 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce15986740 10 client.0 _ll_register_callbacks cb 0x564890bcadb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce10b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cc5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce10b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cc5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60944/0 (socket says 172.21.15.5:60944) 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce10b48640 1 -- 172.21.15.5:0/3988010774 learned_addr learned my addr 172.21.15.5:0/3988010774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce11b4a640 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890cc60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce11349640 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cc5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce10b48640 1 -- 172.21.15.5:0/3988010774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 msgr2=0x564890cc60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.604+0000 7fce10b48640 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890cc60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce10b48640 1 -- 172.21.15.5:0/3988010774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 msgr2=0x564890cc5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce10b48640 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cc5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce10b48640 1 -- 172.21.15.5:0/3988010774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56489092d580 con 0x564890cb9880 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce10b48640 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cc5bb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009610 tx=0x7fcdfc0209d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=471ed11ad3ea50b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fcdf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/3988010774 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc020040 con 0x564890cb9880 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/3988010774 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7fcdfc018d60 con 0x564890cb9880 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce15986740 1 -- 172.21.15.5:0/3988010774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce04036070 con 0x564890cb9880 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/3988010774 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcdfc09c120 con 0x564890cb9880 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce15986740 10 client.17214 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce15986740 1 -- 172.21.15.5:0/3988010774 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5648909f7960 con 0x564890cb9880 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/3988010774 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce04036070 con 0x564890cb9880 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/3988010774 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcdfc0cd110 con 0x564890cb9880 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce15986740 10 client.17214 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce15986740 10 client.17214 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce15986740 20 client.17214 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce15986740 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564890cd81f0 0x564890cda5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce15986740 4 client.17214 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.605+0000 7fce15986740 1 -- 172.21.15.5:0/3988010774 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564890c2bb90 con 0x564890cd81f0 2024-09-15T22:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.606+0000 7fce11349640 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564890cd81f0 0x564890cda5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.606+0000 7fcdf4ff9640 10 client.17214 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.606+0000 7fce11349640 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564890cd81f0 0x564890cda5e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a45dcf72044802be server_cookie=9cfeace1b4180be9 in_seq=0 out_seq=0 2024-09-15T22:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/3988010774 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564890c2bb90 con 0x564890cd81f0 2024-09-15T22:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fcdf4ff9640 10 client.17214 handle_command_reply: tid=0 2024-09-15T22:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fce15986740 1 client.17214 shutdown 2024-09-15T22:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fce15986740 1 -- 172.21.15.5:0/3988010774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 msgr2=0x564890cc5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fce15986740 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cc5bb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009610 tx=0x7fcdfc0209d0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fce15986740 1 -- 172.21.15.5:0/3988010774 shutdown_connections 2024-09-15T22:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fce15986740 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890cc60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fce15986740 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cc5bb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fce15986740 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564890cd81f0 0x564890cda5e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fce15986740 1 --2- 172.21.15.5:0/3988010774 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cc5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fcdf4ff9640 0 client.17214 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fce15986740 1 -- 172.21.15.5:0/3988010774 >> 172.21.15.5:0/3988010774 conn(0x564890cb55f0 msgr2=0x564890cbb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fce15986740 1 -- 172.21.15.5:0/3988010774 shutdown_connections 2024-09-15T22:18:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.607+0000 7fce15986740 1 -- 172.21.15.5:0/3988010774 wait complete. 2024-09-15T22:18:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.608+0000 7fce15986740 20 client.17214 trim_cache size 0 max 16384 2024-09-15T22:18:33.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce15986740 1 Processor -- start 2024-09-15T22:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce15986740 1 -- start start 2024-09-15T22:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce15986740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cb8f60 0x564890cc3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce15986740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cbb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce15986740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cc8650 0x564890cdfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce15986740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1af750 con 0x564890cb9880 2024-09-15T22:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce15986740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1ad020 con 0x564890cc8650 2024-09-15T22:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce15986740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1aae90 con 0x564890cb8f60 2024-09-15T22:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce10b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cbb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce10b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cbb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60954/0 (socket says 172.21.15.5:60954) 2024-09-15T22:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce10b48640 1 -- 172.21.15.5:0/224575399 learned_addr learned my addr 172.21.15.5:0/224575399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce11349640 1 --2- 172.21.15.5:0/224575399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cb8f60 0x564890cc3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce11b4a640 1 --2- 172.21.15.5:0/224575399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cc8650 0x564890cdfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce11349640 1 -- 172.21.15.5:0/224575399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cc8650 msgr2=0x564890cdfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce11349640 1 --2- 172.21.15.5:0/224575399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cc8650 0x564890cdfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce11349640 1 -- 172.21.15.5:0/224575399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 msgr2=0x564890cbb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce11349640 1 --2- 172.21.15.5:0/224575399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cbb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.625+0000 7fce11349640 1 -- 172.21.15.5:0/224575399 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564890cb96b0 con 0x564890cb8f60 2024-09-15T22:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.626+0000 7fce10b48640 1 --2- 172.21.15.5:0/224575399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cbb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.626+0000 7fce11349640 1 --2- 172.21.15.5:0/224575399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cb8f60 0x564890cc3f40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x564890cdad80 tx=0x7fce08010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4b5e2e0cfb08530 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.626+0000 7fcdf6ffd640 1 -- 172.21.15.5:0/224575399 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce08021030 con 0x564890cb8f60 2024-09-15T22:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.626+0000 7fcdf6ffd640 1 -- 172.21.15.5:0/224575399 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce080035f0 con 0x564890cb8f60 2024-09-15T22:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.626+0000 7fcdf6ffd640 1 -- 172.21.15.5:0/224575399 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce0801c680 con 0x564890cb8f60 2024-09-15T22:18:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.627+0000 7fce15986740 1 -- 172.21.15.5:0/224575399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cb8f60 msgr2=0x564890cc3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.627+0000 7fce15986740 1 --2- 172.21.15.5:0/224575399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cb8f60 0x564890cc3f40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x564890cdad80 tx=0x7fce08010800 comp rx=0 tx=0).stop 2024-09-15T22:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.627+0000 7fce15986740 1 -- 172.21.15.5:0/224575399 shutdown_connections 2024-09-15T22:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.627+0000 7fce15986740 1 --2- 172.21.15.5:0/224575399 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cc8650 0x564890cdfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.627+0000 7fce15986740 1 --2- 172.21.15.5:0/224575399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cbb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.627+0000 7fce15986740 1 --2- 172.21.15.5:0/224575399 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cb8f60 0x564890cc3f40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.627+0000 7fce15986740 1 -- 172.21.15.5:0/224575399 >> 172.21.15.5:0/224575399 conn(0x564890cb55f0 msgr2=0x564890cbaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.627+0000 7fce15986740 1 -- 172.21.15.5:0/224575399 shutdown_connections 2024-09-15T22:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.627+0000 7fce15986740 1 -- 172.21.15.5:0/224575399 wait complete. 2024-09-15T22:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.627+0000 7fce15986740 1 -- start start 2024-09-15T22:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.627+0000 7fce15986740 1 Processor -- start 2024-09-15T22:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce15986740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cd0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce15986740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce15986740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890ccd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce15986740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1ab4e0 con 0x564890cb9880 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce15986740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1ac950 con 0x564890cb8f60 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce15986740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1abbb0 con 0x564890cc8650 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce10b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce15986740 10 client.0 _ll_register_callbacks cb 0x7fcdfc09b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce10b48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cccd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60960/0 (socket says 172.21.15.5:60960) 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce10b48640 1 -- 172.21.15.5:0/2592576488 learned_addr learned my addr 172.21.15.5:0/2592576488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce11349640 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cd0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce11b4a640 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890ccd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce10b48640 1 -- 172.21.15.5:0/2592576488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 msgr2=0x564890ccd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce10b48640 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890ccd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce10b48640 1 -- 172.21.15.5:0/2592576488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 msgr2=0x564890cd0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce10b48640 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cd0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.628+0000 7fce10b48640 1 -- 172.21.15.5:0/2592576488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564890ccd900 con 0x564890cb9880 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fce10b48640 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cccd80 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc099cd0 tx=0x7fcdfc09b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eae6f0b81405321 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fcdf4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/2592576488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc020040 con 0x564890cb9880 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/2592576488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7fcdfc09ae70 con 0x564890cb9880 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fce15986740 1 -- 172.21.15.5:0/2592576488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce080035f0 con 0x564890cb9880 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/2592576488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcdfc09ba20 con 0x564890cb9880 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fce15986740 10 client.17220 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fce15986740 1 -- 172.21.15.5:0/2592576488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564890d43cb0 con 0x564890cb9880 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/2592576488 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce080035f0 con 0x564890cb9880 2024-09-15T22:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/2592576488 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcdfc0ba150 con 0x564890cb9880 2024-09-15T22:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fce15986740 10 client.17220 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fce15986740 10 client.17220 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fce15986740 20 client.17220 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fce15986740 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564890d44270 0x564890d46690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fce15986740 4 client.17220 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9589889c", "release": true, "await": true}] 2024-09-15T22:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.629+0000 7fce15986740 1 -- 172.21.15.5:0/2592576488 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9589889c", "release": true, "await": true}) -- 0x564890c260a0 con 0x564890d44270 2024-09-15T22:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.630+0000 7fce11349640 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564890d44270 0x564890d46690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.630+0000 7fcdf4ff9640 10 client.17220 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.630+0000 7fce11349640 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564890d44270 0x564890d46690 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=982583872d28c347 server_cookie=43a76fe2bca88af7 in_seq=0 out_seq=0 2024-09-15T22:18:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fcdf4ff9640 1 -- 172.21.15.5:0/2592576488 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564890c260a0 con 0x564890d44270 2024-09-15T22:18:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fcdf4ff9640 10 client.17220 handle_command_reply: tid=0 2024-09-15T22:18:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fce15986740 1 client.17220 shutdown 2024-09-15T22:18:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fce15986740 1 -- 172.21.15.5:0/2592576488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 msgr2=0x564890cccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fce15986740 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cccd80 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc099cd0 tx=0x7fcdfc09b7f0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fce15986740 1 -- 172.21.15.5:0/2592576488 shutdown_connections 2024-09-15T22:18:33.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fce15986740 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564890cc8650 0x564890ccd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fce15986740 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564890cb9880 0x564890cccd80 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fce15986740 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564890d44270 0x564890d46690 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fce15986740 1 --2- 172.21.15.5:0/2592576488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564890cb8f60 0x564890cd0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fcdf4ff9640 0 client.17220 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fce15986740 1 -- 172.21.15.5:0/2592576488 >> 172.21.15.5:0/2592576488 conn(0x564890cb55f0 msgr2=0x564890cbb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fce15986740 1 -- 172.21.15.5:0/2592576488 shutdown_connections 2024-09-15T22:18:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.677+0000 7fce15986740 1 -- 172.21.15.5:0/2592576488 wait complete. 2024-09-15T22:18:33.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce15986740 20 client.17220 trim_cache size 0 max 16384 2024-09-15T22:18:33.682 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 36, 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "9589889c": { 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 36, 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.3, 2024-09-15T22:18:33.683 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:18:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:18:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:18:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:18:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:18:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:18:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:33.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:18:33.684 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:18:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 -- 172.21.15.5:0/3566786360 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcdf80a33b0 msgr2=0x7fcdf80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcdf80a33b0 0x7fcdf80a5870 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce04009a60 tx=0x7fce04036000 comp rx=0 tx=0).stop 2024-09-15T22:18:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 -- 172.21.15.5:0/3566786360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c102790 msgr2=0x7fce0c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c102790 0x7fce0c1988f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00d770 tx=0x7fcdfc00dc30 comp rx=0 tx=0).stop 2024-09-15T22:18:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 -- 172.21.15.5:0/3566786360 shutdown_connections 2024-09-15T22:18:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcdf80a33b0 0x7fcdf80a5870 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fce0c106f10 0x7fce0c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c102790 0x7fce0c1988f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 --2- 172.21.15.5:0/3566786360 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fce0c101de0 0x7fce0c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 -- 172.21.15.5:0/3566786360 >> 172.21.15.5:0/3566786360 conn(0x7fce0c0fd8c0 msgr2=0x7fce0c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:33.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 -- 172.21.15.5:0/3566786360 shutdown_connections 2024-09-15T22:18:33.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:33.678+0000 7fce135ad640 1 -- 172.21.15.5:0/3566786360 wait complete. 2024-09-15T22:18:34.388 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9589889c', total seconds elapsed: 25.7 2024-09-15T22:18:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 545.0 s, 128.7 tps, lat 3165.492 ms stddev 7259.052 2024-09-15T22:18:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:35 smithi099 ceph-mon[34892]: pgmap v299: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 6 op/s 2024-09-15T22:18:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:35 smithi129 ceph-mon[35314]: pgmap v299: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 6 op/s 2024-09-15T22:18:35.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:35 smithi005 ceph-mon[30944]: pgmap v299: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 6 op/s 2024-09-15T22:18:37.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:37 smithi099 ceph-mon[34892]: pgmap v300: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 504 KiB/s wr, 59 op/s 2024-09-15T22:18:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:37 smithi129 ceph-mon[35314]: pgmap v300: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 504 KiB/s wr, 59 op/s 2024-09-15T22:18:37.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:37 smithi005 ceph-mon[30944]: pgmap v300: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 504 KiB/s wr, 59 op/s 2024-09-15T22:18:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:39 smithi099 ceph-mon[34892]: pgmap v301: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 625 KiB/s wr, 73 op/s 2024-09-15T22:18:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:39 smithi129 ceph-mon[35314]: pgmap v301: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 625 KiB/s wr, 73 op/s 2024-09-15T22:18:39.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:39 smithi005 ceph-mon[30944]: pgmap v301: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 625 KiB/s wr, 73 op/s 2024-09-15T22:18:39.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 550.0 s, 673.0 tps, lat 168.341 ms stddev 666.657 2024-09-15T22:18:41.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:41 smithi005 ceph-mon[30944]: pgmap v302: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-15T22:18:41.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:41 smithi099 ceph-mon[34892]: pgmap v302: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-15T22:18:41.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:41 smithi129 ceph-mon[35314]: pgmap v302: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.1 MiB/s wr, 111 op/s 2024-09-15T22:18:43.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:43 smithi099 ceph-mon[34892]: pgmap v303: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.0 MiB/s wr, 328 op/s 2024-09-15T22:18:43.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:43 smithi129 ceph-mon[35314]: pgmap v303: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.0 MiB/s wr, 328 op/s 2024-09-15T22:18:43.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:43 smithi005 ceph-mon[30944]: pgmap v303: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 3.0 MiB/s wr, 328 op/s 2024-09-15T22:18:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 555.0 s, 641.6 tps, lat 161.294 ms stddev 191.387 2024-09-15T22:18:45.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:45 smithi099 ceph-mon[34892]: pgmap v304: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 3.2 MiB/s wr, 350 op/s 2024-09-15T22:18:45.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:45 smithi129 ceph-mon[35314]: pgmap v304: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 3.2 MiB/s wr, 350 op/s 2024-09-15T22:18:45.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:45 smithi005 ceph-mon[30944]: pgmap v304: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 3.2 MiB/s wr, 350 op/s 2024-09-15T22:18:47.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:47 smithi099 ceph-mon[34892]: pgmap v305: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 5.1 MiB/s wr, 562 op/s 2024-09-15T22:18:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:47 smithi129 ceph-mon[35314]: pgmap v305: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 5.1 MiB/s wr, 562 op/s 2024-09-15T22:18:47.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:47 smithi005 ceph-mon[30944]: pgmap v305: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 5.1 MiB/s wr, 562 op/s 2024-09-15T22:18:49.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:49 smithi099 ceph-mon[34892]: pgmap v306: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.9 MiB/s wr, 543 op/s 2024-09-15T22:18:49.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:49 smithi129 ceph-mon[35314]: pgmap v306: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.9 MiB/s wr, 543 op/s 2024-09-15T22:18:49.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:49 smithi005 ceph-mon[30944]: pgmap v306: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.9 MiB/s wr, 543 op/s 2024-09-15T22:18:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 560.0 s, 631.4 tps, lat 155.573 ms stddev 170.784 2024-09-15T22:18:51.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:51 smithi099 ceph-mon[34892]: pgmap v307: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-09-15T22:18:51.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:51 smithi129 ceph-mon[35314]: pgmap v307: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-09-15T22:18:51.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:51 smithi005 ceph-mon[30944]: pgmap v307: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 5.0 MiB/s wr, 551 op/s 2024-09-15T22:18:53.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:53 smithi099 ceph-mon[34892]: pgmap v308: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.2 MiB/s wr, 695 op/s 2024-09-15T22:18:53.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:53 smithi129 ceph-mon[35314]: pgmap v308: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.2 MiB/s wr, 695 op/s 2024-09-15T22:18:53.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:53 smithi005 ceph-mon[30944]: pgmap v308: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.2 MiB/s wr, 695 op/s 2024-09-15T22:18:54.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 565.0 s, 657.4 tps, lat 154.693 ms stddev 182.290 2024-09-15T22:18:55.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:18:55.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:55 smithi005 ceph-mon[30944]: pgmap v309: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.4 MiB/s wr, 497 op/s 2024-09-15T22:18:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:55.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:55 smithi099 ceph-mon[34892]: pgmap v309: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.4 MiB/s wr, 497 op/s 2024-09-15T22:18:55.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:55 smithi129 ceph-mon[35314]: pgmap v309: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 4.4 MiB/s wr, 497 op/s 2024-09-15T22:18:56.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.172+0000 7fa80a9e4640 1 -- 172.21.15.5:0/1958959858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa804109b10 msgr2=0x7fa804114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.172+0000 7fa80a9e4640 1 --2- 172.21.15.5:0/1958959858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa804109b10 0x7fa804114570 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc009ca0 tx=0x7fa7fc02a960 comp rx=0 tx=0).stop 2024-09-15T22:18:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.172+0000 7fa80a9e4640 1 -- 172.21.15.5:0/1958959858 shutdown_connections 2024-09-15T22:18:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.172+0000 7fa80a9e4640 1 --2- 172.21.15.5:0/1958959858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa804114ab0 0x7fa804116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.172+0000 7fa80a9e4640 1 --2- 172.21.15.5:0/1958959858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa804109b10 0x7fa804114570 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.172+0000 7fa80a9e4640 1 --2- 172.21.15.5:0/1958959858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa8041091f0 0x7fa8041095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.172+0000 7fa80a9e4640 1 -- 172.21.15.5:0/1958959858 >> 172.21.15.5:0/1958959858 conn(0x7fa80406e140 msgr2=0x7fa80406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:56.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.174+0000 7fa80a9e4640 1 -- 172.21.15.5:0/1958959858 shutdown_connections 2024-09-15T22:18:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.178+0000 7fa80a9e4640 1 -- 172.21.15.5:0/1958959858 wait complete. 2024-09-15T22:18:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.178+0000 7fa80a9e4640 1 Processor -- start 2024-09-15T22:18:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.179+0000 7fa80a9e4640 1 -- start start 2024-09-15T22:18:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.179+0000 7fa80a9e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8041091f0 0x7fa80410a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.179+0000 7fa80a9e4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa804109b10 0x7fa80410a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.179+0000 7fa80a9e4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa804114ab0 0x7fa80410ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.179+0000 7fa80a9e4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa80411a840 con 0x7fa8041091f0 2024-09-15T22:18:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.179+0000 7fa80a9e4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa80411a6c0 con 0x7fa804114ab0 2024-09-15T22:18:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.179+0000 7fa80a9e4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa80411a9c0 con 0x7fa804109b10 2024-09-15T22:18:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.180+0000 7fa808f81640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa804114ab0 0x7fa80410ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.180+0000 7fa808f81640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa804114ab0 0x7fa80410ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:55656/0 (socket says 172.21.15.5:55656) 2024-09-15T22:18:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.180+0000 7fa808f81640 1 -- 172.21.15.5:0/295482179 learned_addr learned my addr 172.21.15.5:0/295482179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:56.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.183+0000 7fa808f81640 1 -- 172.21.15.5:0/295482179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa804109b10 msgr2=0x7fa80410a910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:18:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.183+0000 7fa808f81640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa804109b10 0x7fa80410a910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.183+0000 7fa808f81640 1 -- 172.21.15.5:0/295482179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8041091f0 msgr2=0x7fa80410a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.183+0000 7fa808f81640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8041091f0 0x7fa80410a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.183+0000 7fa808f81640 1 -- 172.21.15.5:0/295482179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8041a4e90 con 0x7fa804114ab0 2024-09-15T22:18:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.185+0000 7fa808f81640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa804114ab0 0x7fa80410ae50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800b390 tx=0x7fa7f800b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.186+0000 7fa8017fa640 1 -- 172.21.15.5:0/295482179 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f8015020 con 0x7fa804114ab0 2024-09-15T22:18:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.188+0000 7fa80a9e4640 1 -- 172.21.15.5:0/295482179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8041a5180 con 0x7fa804114ab0 2024-09-15T22:18:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.188+0000 7fa80a9e4640 1 -- 172.21.15.5:0/295482179 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8041a5710 con 0x7fa804114ab0 2024-09-15T22:18:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.191+0000 7fa8017fa640 1 -- 172.21.15.5:0/295482179 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f8011d10 con 0x7fa804114ab0 2024-09-15T22:18:56.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.191+0000 7fa8017fa640 1 -- 172.21.15.5:0/295482179 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f800f2e0 con 0x7fa804114ab0 2024-09-15T22:18:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.195+0000 7fa80cdbd740 1 Processor -- start 2024-09-15T22:18:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.196+0000 7fa8017fa640 1 -- 172.21.15.5:0/295482179 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa7f8018020 con 0x7fa804114ab0 2024-09-15T22:18:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.196+0000 7fa80cdbd740 1 -- start start 2024-09-15T22:18:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.197+0000 7fa80cdbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df874a0 0x559d9df87880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.197+0000 7fa80cdbd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df87dc0 0x559d9df92140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.197+0000 7fa80cdbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 0x559d9df94a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.197+0000 7fa80cdbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559d9deed2e0 con 0x559d9df92680 2024-09-15T22:18:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.197+0000 7fa80cdbd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559d9de94ce0 con 0x559d9df87dc0 2024-09-15T22:18:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.197+0000 7fa80cdbd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559d9dee92a0 con 0x559d9df874a0 2024-09-15T22:18:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.198+0000 7fa8017fa640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7ec0a3480 0x7fa7ec0a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.198+0000 7fa8017fa640 1 -- 172.21.15.5:0/295482179 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7fa7f80cf770 con 0x7fa804114ab0 2024-09-15T22:18:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.200+0000 7fa8037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 0x559d9df94a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.200+0000 7fa8037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 0x559d9df94a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36924/0 (socket says 172.21.15.5:36924) 2024-09-15T22:18:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.200+0000 7fa8037fe640 1 -- 172.21.15.5:0/3347317930 learned_addr learned my addr 172.21.15.5:0/3347317930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.200+0000 7fa8037fe640 1 --2- 172.21.15.5:0/3347317930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df874a0 0x559d9df87880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.201+0000 7fa808f81640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7ec0a3480 0x7fa7ec0a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.201+0000 7fa803fff640 1 --2- 172.21.15.5:0/3347317930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df87dc0 0x559d9df92140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.202+0000 7fa8037fe640 1 -- 172.21.15.5:0/3347317930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df87dc0 msgr2=0x559d9df92140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.202+0000 7fa8037fe640 1 --2- 172.21.15.5:0/3347317930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df87dc0 0x559d9df92140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.202+0000 7fa8037fe640 1 -- 172.21.15.5:0/3347317930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 msgr2=0x559d9df94a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.202+0000 7fa8037fe640 1 --2- 172.21.15.5:0/3347317930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 0x559d9df94a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.202+0000 7fa8037fe640 1 -- 172.21.15.5:0/3347317930 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d9de79440 con 0x559d9df874a0 2024-09-15T22:18:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.202+0000 7fa8037fe640 1 --2- 172.21.15.5:0/3347317930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df874a0 0x559d9df87880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x559d9df84db0 tx=0x7fa7fc000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb6834085edfe03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.204+0000 7fa7ea7fc640 1 -- 172.21.15.5:0/3347317930 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc034040 con 0x559d9df874a0 2024-09-15T22:18:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.204+0000 7fa7ea7fc640 1 -- 172.21.15.5:0/3347317930 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7fc02f560 con 0x559d9df874a0 2024-09-15T22:18:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.205+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3347317930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df874a0 msgr2=0x559d9df87880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.205+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3347317930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df874a0 0x559d9df87880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x559d9df84db0 tx=0x7fa7fc000f40 comp rx=0 tx=0).stop 2024-09-15T22:18:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.212+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3347317930 shutdown_connections 2024-09-15T22:18:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.212+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3347317930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 0x559d9df94a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.212+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3347317930 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df87dc0 0x559d9df92140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.212+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3347317930 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df874a0 0x559d9df87880 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.212+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3347317930 >> 172.21.15.5:0/3347317930 conn(0x559d9df83a30 msgr2=0x559d9df83e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.217+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3347317930 shutdown_connections 2024-09-15T22:18:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.217+0000 7fa808f81640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7ec0a3480 0x7fa7ec0a5940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa7f809b6f0 tx=0x7fa7f80106e0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.217+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3347317930 wait complete. 2024-09-15T22:18:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.218+0000 7fa80cdbd740 1 -- start start 2024-09-15T22:18:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.219+0000 7fa80cdbd740 1 Processor -- start 2024-09-15T22:18:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.221+0000 7fa80cdbd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df874a0 0x559d9df91f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.221+0000 7fa80cdbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df87dc0 0x559d9df8f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.221+0000 7fa80cdbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 0x559d9df8f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.221+0000 7fa80cdbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559d9df952a0 con 0x559d9df92680 2024-09-15T22:18:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.221+0000 7fa80cdbd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559d9dea8050 con 0x559d9df874a0 2024-09-15T22:18:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.221+0000 7fa80cdbd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559d9df95420 con 0x559d9df87dc0 2024-09-15T22:18:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.221+0000 7fa80cdbd740 10 client.0 _ll_register_callbacks cb 0x559d9dee8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.221+0000 7fa8037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 0x559d9df8f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.221+0000 7fa8037fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 0x559d9df8f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36934/0 (socket says 172.21.15.5:36934) 2024-09-15T22:18:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.221+0000 7fa8037fe640 1 -- 172.21.15.5:0/3168936468 learned_addr learned my addr 172.21.15.5:0/3168936468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.222+0000 7fa8037fe640 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df874a0 0x559d9df91f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.223+0000 7fa803fff640 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df87dc0 0x559d9df8f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.224+0000 7fa803fff640 1 -- 172.21.15.5:0/3168936468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df874a0 msgr2=0x559d9df91f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.224+0000 7fa803fff640 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df874a0 0x559d9df91f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.224+0000 7fa803fff640 1 -- 172.21.15.5:0/3168936468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 msgr2=0x559d9df8f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.224+0000 7fa803fff640 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 0x559d9df8f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.224+0000 7fa803fff640 1 -- 172.21.15.5:0/3168936468 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d9dc5e3c0 con 0x559d9df87dc0 2024-09-15T22:18:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.225+0000 7fa803fff640 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df87dc0 0x559d9df8f1d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40114b0 tx=0x7fa7f4011970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf35027cc5d14a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:56.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.225+0000 7fa7e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:18:56.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.225+0000 7fa7e37fe640 1 -- 172.21.15.5:0/3168936468 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4005020 con 0x559d9df87dc0 2024-09-15T22:18:56.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.226+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3168936468 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7fc02f560 con 0x559d9df87dc0 2024-09-15T22:18:56.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.226+0000 7fa7e37fe640 1 -- 172.21.15.5:0/3168936468 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7fa7f40051c0 con 0x559d9df87dc0 2024-09-15T22:18:56.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.226+0000 7fa7e37fe640 1 -- 172.21.15.5:0/3168936468 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa7f40053c0 con 0x559d9df87dc0 2024-09-15T22:18:56.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.226+0000 7fa80cdbd740 10 client.26668 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:56.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.226+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3168936468 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559d9dfa0d30 con 0x559d9df87dc0 2024-09-15T22:18:56.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.226+0000 7fa7e37fe640 1 -- 172.21.15.5:0/3168936468 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7fc02f560 con 0x559d9df87dc0 2024-09-15T22:18:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.228+0000 7fa7e37fe640 1 -- 172.21.15.5:0/3168936468 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa7f4004cc0 con 0x559d9df87dc0 2024-09-15T22:18:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.228+0000 7fa80cdbd740 10 client.26668 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.228+0000 7fa80cdbd740 10 client.26668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.228+0000 7fa80cdbd740 20 client.26668 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.228+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559d9dfa2660 0x559d9dfa4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.228+0000 7fa80cdbd740 4 client.26668 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:18:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.228+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3168936468 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d9defa850 con 0x559d9dfa2660 2024-09-15T22:18:56.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.230+0000 7fa8037fe640 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559d9dfa2660 0x559d9dfa4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.234+0000 7fa7e37fe640 10 client.26668 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:56.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.234+0000 7fa8037fe640 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559d9dfa2660 0x559d9dfa4a80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48039ca0aa32b85c server_cookie=82fc71108ad6722c in_seq=0 out_seq=0 2024-09-15T22:18:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.236+0000 7fa7e37fe640 1 -- 172.21.15.5:0/3168936468 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d9defa850 con 0x559d9dfa2660 2024-09-15T22:18:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.236+0000 7fa7e37fe640 10 client.26668 handle_command_reply: tid=0 2024-09-15T22:18:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.236+0000 7fa80cdbd740 1 client.26668 shutdown 2024-09-15T22:18:56.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.240+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3168936468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df87dc0 msgr2=0x559d9df8f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:56.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.240+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df87dc0 0x559d9df8f1d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40114b0 tx=0x7fa7f4011970 comp rx=0 tx=0).stop 2024-09-15T22:18:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.243+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3168936468 shutdown_connections 2024-09-15T22:18:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.243+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df92680 0x559d9df8f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.243+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df87dc0 0x559d9df8f1d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.243+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559d9dfa2660 0x559d9dfa4a80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.243+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3168936468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df874a0 0x559d9df91f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.243+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3168936468 >> 172.21.15.5:0/3168936468 conn(0x559d9df83a30 msgr2=0x559d9df82b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.243+0000 7fa7e37fe640 0 client.26668 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:56.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.243+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3168936468 shutdown_connections 2024-09-15T22:18:56.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.243+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3168936468 wait complete. 2024-09-15T22:18:56.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.244+0000 7fa80cdbd740 20 client.26668 trim_cache size 0 max 16384 2024-09-15T22:18:56.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.281+0000 7fa80cdbd740 1 Processor -- start 2024-09-15T22:18:56.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.281+0000 7fa80cdbd740 1 -- start start 2024-09-15T22:18:56.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.281+0000 7fa80cdbd740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df83a30 0x559d9df9e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.282+0000 7fa80cdbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df874a0 0x559d9df8f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.282+0000 7fa80cdbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df87dc0 0x559d9df8a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.282+0000 7fa80cdbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f00128a0 con 0x559d9df87dc0 2024-09-15T22:18:56.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.282+0000 7fa80cdbd740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f0010170 con 0x559d9df83a30 2024-09-15T22:18:56.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.282+0000 7fa80cdbd740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f000e070 con 0x559d9df874a0 2024-09-15T22:18:56.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.282+0000 7fa8037fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df83a30 0x559d9df9e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.282+0000 7fa8037fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df83a30 0x559d9df9e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:55690/0 (socket says 172.21.15.5:55690) 2024-09-15T22:18:56.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.282+0000 7fa8037fe640 1 -- 172.21.15.5:0/1924809603 learned_addr learned my addr 172.21.15.5:0/1924809603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:56.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.282+0000 7fa8037fe640 1 --2- 172.21.15.5:0/1924809603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df87dc0 0x559d9df8a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.282+0000 7fa803fff640 1 --2- 172.21.15.5:0/1924809603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df874a0 0x559d9df8f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.283+0000 7fa8037fe640 1 -- 172.21.15.5:0/1924809603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df874a0 msgr2=0x559d9df8f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:56.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.283+0000 7fa8037fe640 1 --2- 172.21.15.5:0/1924809603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df874a0 0x559d9df8f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.283+0000 7fa8037fe640 1 -- 172.21.15.5:0/1924809603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df87dc0 msgr2=0x559d9df8a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:56.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.283+0000 7fa8037fe640 1 --2- 172.21.15.5:0/1924809603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df87dc0 0x559d9df8a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.283+0000 7fa8037fe640 1 -- 172.21.15.5:0/1924809603 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d9dfa50e0 con 0x559d9df83a30 2024-09-15T22:18:56.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.283+0000 7fa8037fe640 1 --2- 172.21.15.5:0/1924809603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df83a30 0x559d9df9e5e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc003f20 tx=0x7fa7fc02f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=933a94e6892360b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:56.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.283+0000 7fa7e9ffb640 1 -- 172.21.15.5:0/1924809603 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc034070 con 0x559d9df83a30 2024-09-15T22:18:56.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.283+0000 7fa7e9ffb640 1 -- 172.21.15.5:0/1924809603 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7fc02fa80 con 0x559d9df83a30 2024-09-15T22:18:56.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.283+0000 7fa7e9ffb640 1 -- 172.21.15.5:0/1924809603 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc004380 con 0x559d9df83a30 2024-09-15T22:18:56.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.284+0000 7fa80cdbd740 1 -- 172.21.15.5:0/1924809603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df83a30 msgr2=0x559d9df9e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.284+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/1924809603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df83a30 0x559d9df9e5e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc003f20 tx=0x7fa7fc02f460 comp rx=0 tx=0).stop 2024-09-15T22:18:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.286+0000 7fa80cdbd740 1 -- 172.21.15.5:0/1924809603 shutdown_connections 2024-09-15T22:18:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.286+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/1924809603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df87dc0 0x559d9df8a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.286+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/1924809603 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df874a0 0x559d9df8f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.286+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/1924809603 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df83a30 0x559d9df9e5e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.286+0000 7fa80cdbd740 1 -- 172.21.15.5:0/1924809603 >> 172.21.15.5:0/1924809603 conn(0x559d9df8f870 msgr2=0x559d9df8a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.286+0000 7fa80cdbd740 1 -- 172.21.15.5:0/1924809603 shutdown_connections 2024-09-15T22:18:56.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.286+0000 7fa80cdbd740 1 -- 172.21.15.5:0/1924809603 wait complete. 2024-09-15T22:18:56.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.287+0000 7fa80cdbd740 1 -- start start 2024-09-15T22:18:56.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.287+0000 7fa80cdbd740 1 Processor -- start 2024-09-15T22:18:56.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.288+0000 7fa80cdbd740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df83a30 0x559d9df963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.289+0000 7fa8037fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df83a30 0x559d9df963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.289+0000 7fa8037fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df83a30 0x559d9df963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:33148/0 (socket says 172.21.15.5:33148) 2024-09-15T22:18:56.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.289+0000 7fa8037fe640 1 -- 172.21.15.5:0/3613905255 learned_addr learned my addr 172.21.15.5:0/3613905255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:18:56.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.289+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df874a0 0x559d9df96920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.289+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df87dc0 0x559d9df96e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.290+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3613905255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f000e630 con 0x559d9df874a0 2024-09-15T22:18:56.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.290+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3613905255 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f000faa0 con 0x559d9df87dc0 2024-09-15T22:18:56.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.290+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3613905255 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7f000ed00 con 0x559d9df83a30 2024-09-15T22:18:56.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.290+0000 7fa803fff640 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df874a0 0x559d9df96920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.290+0000 7fa8037fe640 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df87dc0 0x559d9df96e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.290+0000 7fa803fff640 1 -- 172.21.15.5:0/3613905255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df83a30 msgr2=0x559d9df963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:56.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.290+0000 7fa803fff640 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df83a30 0x559d9df963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.290+0000 7fa803fff640 1 -- 172.21.15.5:0/3613905255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df87dc0 msgr2=0x559d9df96e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:18:56.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.290+0000 7fa803fff640 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df87dc0 0x559d9df96e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.290+0000 7fa803fff640 1 -- 172.21.15.5:0/3613905255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d9df846c0 con 0x559d9df874a0 2024-09-15T22:18:56.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.290+0000 7fa8037fe640 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df87dc0 0x559d9df96e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:18:56.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.291+0000 7fa80cdbd740 10 client.0 _ll_register_callbacks cb 0x7fa7f4009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:18:56.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.291+0000 7fa803fff640 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df874a0 0x559d9df96920 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400f880 tx=0x7fa7f4009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90fcb58039ef5743 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:18:56.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.291+0000 7fa7e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:18:56.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.291+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3613905255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7fc02fa80 con 0x559d9df874a0 2024-09-15T22:18:56.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.291+0000 7fa7e37fe640 1 -- 172.21.15.5:0/3613905255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4004220 con 0x559d9df874a0 2024-09-15T22:18:56.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.291+0000 7fa7e37fe640 1 -- 172.21.15.5:0/3613905255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7fa7f40043c0 con 0x559d9df874a0 2024-09-15T22:18:56.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.292+0000 7fa7e37fe640 1 -- 172.21.15.5:0/3613905255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa7f40035f0 con 0x559d9df874a0 2024-09-15T22:18:56.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.292+0000 7fa80cdbd740 10 client.17235 fetch_fsmap learned FSMap version 25 2024-09-15T22:18:56.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.292+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3613905255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=25}) -- 0x559d9df84ae0 con 0x559d9df874a0 2024-09-15T22:18:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.292+0000 7fa7e37fe640 1 -- 172.21.15.5:0/3613905255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa7f4003790 con 0x559d9df874a0 2024-09-15T22:18:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.292+0000 7fa80cdbd740 10 client.17235 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:18:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.292+0000 7fa80cdbd740 10 client.17235 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:18:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.292+0000 7fa80cdbd740 20 client.17235 populate_metadata read hostname 'smithi005' 2024-09-15T22:18:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.292+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559d9df92680 0x559d9e00f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:18:56.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.293+0000 7fa80cdbd740 4 client.17235 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:18:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.293+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3613905255 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559d9def4d60 con 0x559d9df92680 2024-09-15T22:18:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.293+0000 7fa8037fe640 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559d9df92680 0x559d9e00f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:18:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.293+0000 7fa8037fe640 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559d9df92680 0x559d9e00f700 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4c4dc34ddbf2ee server_cookie=5c3a948ba2f3f9ac in_seq=0 out_seq=0 2024-09-15T22:18:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.293+0000 7fa7e37fe640 10 client.17235 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.294+0000 7fa7e37fe640 1 -- 172.21.15.5:0/3613905255 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559d9def4d60 con 0x559d9df92680 2024-09-15T22:18:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.294+0000 7fa7e37fe640 10 client.17235 handle_command_reply: tid=0 2024-09-15T22:18:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.294+0000 7fa80cdbd740 1 client.17235 shutdown 2024-09-15T22:18:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.294+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3613905255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df874a0 msgr2=0x559d9df96920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.294+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df874a0 0x559d9df96920 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400f880 tx=0x7fa7f4009b60 comp rx=0 tx=0).stop 2024-09-15T22:18:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.294+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3613905255 shutdown_connections 2024-09-15T22:18:56.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.294+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559d9df92680 0x559d9e00f700 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.294+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559d9df87dc0 0x559d9df96e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.294+0000 7fa7e37fe640 0 client.17235 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:18:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.294+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d9df874a0 0x559d9df96920 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.294+0000 7fa80cdbd740 1 --2- 172.21.15.5:0/3613905255 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559d9df83a30 0x559d9df963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.295+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3613905255 >> 172.21.15.5:0/3613905255 conn(0x559d9df8f870 msgr2=0x559d9df91250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.295+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3613905255 shutdown_connections 2024-09-15T22:18:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.295+0000 7fa80cdbd740 1 -- 172.21.15.5:0/3613905255 wait complete. 2024-09-15T22:18:56.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.295+0000 7fa80cdbd740 20 client.17235 trim_cache size 0 max 16384 2024-09-15T22:18:56.299 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:18:56.300 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:18:56.300 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:18:56.300 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:18:56.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.296+0000 7fa80a9e4640 1 -- 172.21.15.5:0/295482179 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7ec0a3480 msgr2=0x7fa7ec0a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:56.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.296+0000 7fa80a9e4640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7ec0a3480 0x7fa7ec0a5940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa7f809b6f0 tx=0x7fa7f80106e0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.296+0000 7fa80a9e4640 1 -- 172.21.15.5:0/295482179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa804114ab0 msgr2=0x7fa80410ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:18:56.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.296+0000 7fa80a9e4640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa804114ab0 0x7fa80410ae50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7f800b390 tx=0x7fa7f800b850 comp rx=0 tx=0).stop 2024-09-15T22:18:56.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.296+0000 7fa80a9e4640 1 -- 172.21.15.5:0/295482179 shutdown_connections 2024-09-15T22:18:56.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.296+0000 7fa80a9e4640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7ec0a3480 0x7fa7ec0a5940 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.296+0000 7fa80a9e4640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa804114ab0 0x7fa80410ae50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.296+0000 7fa80a9e4640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa804109b10 0x7fa80410a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.296+0000 7fa80a9e4640 1 --2- 172.21.15.5:0/295482179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8041091f0 0x7fa80410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:18:56.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.297+0000 7fa80a9e4640 1 -- 172.21.15.5:0/295482179 >> 172.21.15.5:0/295482179 conn(0x7fa80406e140 msgr2=0x7fa804071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:18:56.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.297+0000 7fa80a9e4640 1 -- 172.21.15.5:0/295482179 shutdown_connections 2024-09-15T22:18:56.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:18:56.297+0000 7fa80a9e4640 1 -- 172.21.15.5:0/295482179 wait complete. 2024-09-15T22:18:57.038 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:18:57.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:18:57.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:57 smithi005 ceph-mon[30944]: pgmap v310: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.1 MiB/s wr, 678 op/s 2024-09-15T22:18:57.523 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:18:57.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:57 smithi099 ceph-mon[34892]: pgmap v310: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.1 MiB/s wr, 678 op/s 2024-09-15T22:18:57.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:57 smithi129 ceph-mon[35314]: pgmap v310: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 6.1 MiB/s wr, 678 op/s 2024-09-15T22:18:59.436 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:18:59 smithi005 ceph-mon[30944]: pgmap v311: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-09-15T22:18:59.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:18:59 smithi099 ceph-mon[34892]: pgmap v311: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-09-15T22:18:59.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:18:59 smithi129 ceph-mon[35314]: pgmap v311: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.8 MiB/s wr, 540 op/s 2024-09-15T22:18:59.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 570.0 s, 415.4 tps, lat 224.978 ms stddev 262.475 2024-09-15T22:19:01.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:01 smithi005 ceph-mon[30944]: pgmap v312: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-09-15T22:19:01.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:01 smithi099 ceph-mon[34892]: pgmap v312: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-09-15T22:19:01.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:01 smithi129 ceph-mon[35314]: pgmap v312: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 4.5 MiB/s wr, 506 op/s 2024-09-15T22:19:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.410+0000 7f90d3f7d640 1 -- 172.21.15.5:0/2448258420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90cc10d530 msgr2=0x7f90cc1150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.410+0000 7f90d3f7d640 1 --2- 172.21.15.5:0/2448258420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90cc10d530 0x7f90cc1150b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f90c401acb0 tx=0x7f90c403b900 comp rx=0 tx=0).stop 2024-09-15T22:19:02.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.412+0000 7f90d3f7d640 1 -- 172.21.15.5:0/2448258420 shutdown_connections 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.412+0000 7f90d3f7d640 1 --2- 172.21.15.5:0/2448258420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90cc10d530 0x7f90cc1150b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.412+0000 7f90d3f7d640 1 --2- 172.21.15.5:0/2448258420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90cc108e00 0x7f90cc10ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.412+0000 7f90d3f7d640 1 --2- 172.21.15.5:0/2448258420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90cc108450 0x7f90cc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.412+0000 7f90d3f7d640 1 -- 172.21.15.5:0/2448258420 >> 172.21.15.5:0/2448258420 conn(0x7f90cc077ba0 msgr2=0x7f90cc079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.415+0000 7f90d3f7d640 1 -- 172.21.15.5:0/2448258420 shutdown_connections 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.415+0000 7f90d3f7d640 1 -- 172.21.15.5:0/2448258420 wait complete. 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d3f7d640 1 Processor -- start 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d3f7d640 1 -- start start 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d3f7d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90cc108450 0x7f90cc114c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d3f7d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90cc108e00 0x7f90cc110d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d3f7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90cc10d530 0x7f90cc082a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:02.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d3f7d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc06e610 con 0x7f90cc10d530 2024-09-15T22:19:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d3f7d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc06e490 con 0x7f90cc108450 2024-09-15T22:19:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d3f7d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f90cc06e790 con 0x7f90cc108e00 2024-09-15T22:19:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d1d19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90cc108450 0x7f90cc114c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d1d19640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90cc108450 0x7f90cc114c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:55702/0 (socket says 172.21.15.5:55702) 2024-09-15T22:19:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d1d19640 1 -- 172.21.15.5:0/3709071591 learned_addr learned my addr 172.21.15.5:0/3709071591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.416+0000 7f90d1518640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90cc108e00 0x7f90cc110d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.417+0000 7f90d251a640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90cc10d530 0x7f90cc082a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.417+0000 7f90d1518640 1 -- 172.21.15.5:0/3709071591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90cc108450 msgr2=0x7f90cc114c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.417+0000 7f90d1518640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90cc108450 0x7f90cc114c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.417+0000 7f90d1518640 1 -- 172.21.15.5:0/3709071591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90cc10d530 msgr2=0x7f90cc082a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:02.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.417+0000 7f90d1518640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90cc10d530 0x7f90cc082a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.417+0000 7f90d1518640 1 -- 172.21.15.5:0/3709071591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90cc082fc0 con 0x7f90cc108e00 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.417+0000 7f90d1518640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90cc108e00 0x7f90cc110d00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f90c800c9a0 tx=0x7f90c800ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.417+0000 7f90c2ffd640 1 -- 172.21.15.5:0/3709071591 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8007bb0 con 0x7f90cc108e00 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.418+0000 7f90c2ffd640 1 -- 172.21.15.5:0/3709071591 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90c8007d50 con 0x7f90cc108e00 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.418+0000 7f90c2ffd640 1 -- 172.21.15.5:0/3709071591 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8018800 con 0x7f90cc108e00 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.418+0000 7f90d3f7d640 1 -- 172.21.15.5:0/3709071591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90cc1112a0 con 0x7f90cc108e00 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.418+0000 7f90d3f7d640 1 -- 172.21.15.5:0/3709071591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90cc111790 con 0x7f90cc108e00 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.420+0000 7f90d3f7d640 1 -- 172.21.15.5:0/3709071591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90cc1098e0 con 0x7f90cc108e00 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.424+0000 7f90c2ffd640 1 -- 172.21.15.5:0/3709071591 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f90c800f450 con 0x7f90cc108e00 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.424+0000 7f90c2ffd640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90b80a35c0 0x7f90b80a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.425+0000 7f90c2ffd640 1 -- 172.21.15.5:0/3709071591 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7f90c80d0400 con 0x7f90cc108e00 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.425+0000 7f90d1d19640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90b80a35c0 0x7f90b80a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.425+0000 7f90d1d19640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90b80a35c0 0x7f90b80a5a80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009a40 tx=0x7f90bc0091c0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:02.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.427+0000 7f90c2ffd640 1 -- 172.21.15.5:0/3709071591 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90c8014030 con 0x7f90cc108e00 2024-09-15T22:19:02.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.805+0000 7f90d3f7d640 1 -- 172.21.15.5:0/3709071591 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90cc108830 con 0x7f90cc108e00 2024-09-15T22:19:02.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.807+0000 7f90c2ffd640 1 -- 172.21.15.5:0/3709071591 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f90cc108830 con 0x7f90cc108e00 2024-09-15T22:19:02.809 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:19:02.809 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:19:02.809 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:19:02.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.814+0000 7f90d3f7d640 1 -- 172.21.15.5:0/3709071591 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90b80a35c0 msgr2=0x7f90b80a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.814+0000 7f90d3f7d640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90b80a35c0 0x7f90b80a5a80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009a40 tx=0x7f90bc0091c0 comp rx=0 tx=0).stop 2024-09-15T22:19:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.814+0000 7f90d3f7d640 1 -- 172.21.15.5:0/3709071591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90cc108e00 msgr2=0x7f90cc110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.814+0000 7f90d3f7d640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90cc108e00 0x7f90cc110d00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f90c800c9a0 tx=0x7f90c800ce60 comp rx=0 tx=0).stop 2024-09-15T22:19:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.814+0000 7f90d3f7d640 1 -- 172.21.15.5:0/3709071591 shutdown_connections 2024-09-15T22:19:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.814+0000 7f90d3f7d640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f90b80a35c0 0x7f90b80a5a80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.814+0000 7f90d3f7d640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90cc10d530 0x7f90cc082a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.814+0000 7f90d3f7d640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f90cc108e00 0x7f90cc110d00 secure :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f90c800c9a0 tx=0x7f90c800ce60 comp rx=0 tx=0).stop 2024-09-15T22:19:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.814+0000 7f90d3f7d640 1 --2- 172.21.15.5:0/3709071591 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f90cc108450 0x7f90cc114c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.814+0000 7f90d3f7d640 1 -- 172.21.15.5:0/3709071591 >> 172.21.15.5:0/3709071591 conn(0x7f90cc077ba0 msgr2=0x7f90cc0786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.815+0000 7f90d3f7d640 1 -- 172.21.15.5:0/3709071591 shutdown_connections 2024-09-15T22:19:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:02.815+0000 7f90d3f7d640 1 -- 172.21.15.5:0/3709071591 wait complete. 2024-09-15T22:19:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:03 smithi099 ceph-mon[34892]: pgmap v313: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.7 MiB/s wr, 644 op/s 2024-09-15T22:19:03.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:03 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3709071591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:19:03.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:03 smithi129 ceph-mon[35314]: pgmap v313: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.7 MiB/s wr, 644 op/s 2024-09-15T22:19:03.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:03 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3709071591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:19:03.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:03 smithi005 ceph-mon[30944]: pgmap v313: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 5.7 MiB/s wr, 644 op/s 2024-09-15T22:19:03.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:03 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3709071591' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:19:03.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:19:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 575.0 s, 468.0 tps, lat 224.138 ms stddev 288.226 2024-09-15T22:19:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.748+0000 7f37ffd5b640 1 -- 172.21.15.5:0/3433879079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f8109830 msgr2=0x7f37f8109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.748+0000 7f37ffd5b640 1 --2- 172.21.15.5:0/3433879079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f8109830 0x7f37f8109c90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009900 tx=0x7f37ec02a440 comp rx=0 tx=0).stop 2024-09-15T22:19:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.748+0000 7f37ffd5b640 1 -- 172.21.15.5:0/3433879079 shutdown_connections 2024-09-15T22:19:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.748+0000 7f37ffd5b640 1 --2- 172.21.15.5:0/3433879079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f811b140 0x7f37f811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.748+0000 7f37ffd5b640 1 --2- 172.21.15.5:0/3433879079 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f8109830 0x7f37f8109c90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.748+0000 7f37ffd5b640 1 --2- 172.21.15.5:0/3433879079 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37f8109450 0x7f37f8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.748+0000 7f37ffd5b640 1 -- 172.21.15.5:0/3433879079 >> 172.21.15.5:0/3433879079 conn(0x7f37f806e140 msgr2=0x7f37f806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:04.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.749+0000 7f37ffd5b640 1 -- 172.21.15.5:0/3433879079 shutdown_connections 2024-09-15T22:19:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.749+0000 7f37ffd5b640 1 -- 172.21.15.5:0/3433879079 wait complete. 2024-09-15T22:19:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.749+0000 7f37ffd5b640 1 Processor -- start 2024-09-15T22:19:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.750+0000 7f37ffd5b640 1 -- start start 2024-09-15T22:19:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.750+0000 7f37ffd5b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f8109450 0x7f37f81186b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.750+0000 7f37ffd5b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f8109830 0x7f37f8118bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.750+0000 7f37ffd5b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37f811b140 0x7f37f8111920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.750+0000 7f37ffd5b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37f8120eb0 con 0x7f37f8109830 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.750+0000 7f37ffd5b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f37f8120d30 con 0x7f37f8109450 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.750+0000 7f37ffd5b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f37f8121030 con 0x7f37f811b140 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.750+0000 7f37fd2f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f8109830 0x7f37f8118bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.750+0000 7f37fd2f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f8109830 0x7f37f8118bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41534/0 (socket says 172.21.15.5:41534) 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.750+0000 7f37fd2f6640 1 -- 172.21.15.5:0/1716516094 learned_addr learned my addr 172.21.15.5:0/1716516094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.750+0000 7f37fe2f8640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37f811b140 0x7f37f8111920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.751+0000 7f37fd2f6640 1 -- 172.21.15.5:0/1716516094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37f811b140 msgr2=0x7f37f8111920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.751+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37f811b140 0x7f37f8111920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.751+0000 7f37fd2f6640 1 -- 172.21.15.5:0/1716516094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f8109450 msgr2=0x7f37f81186b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.751+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f8109450 0x7f37f81186b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.751+0000 7f37fd2f6640 1 -- 172.21.15.5:0/1716516094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f81121e0 con 0x7f37f8109830 2024-09-15T22:19:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.751+0000 7f37fe2f8640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37f811b140 0x7f37f8111920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:19:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.751+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f8109830 0x7f37f8118bf0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f37ec02a9b0 tx=0x7f37ec003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:04.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.751+0000 7f37f6ffd640 1 -- 172.21.15.5:0/1716516094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec03f070 con 0x7f37f8109830 2024-09-15T22:19:04.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.751+0000 7f37ffd5b640 1 -- 172.21.15.5:0/1716516094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f8112470 con 0x7f37f8109830 2024-09-15T22:19:04.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.751+0000 7f37ffd5b640 1 -- 172.21.15.5:0/1716516094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f8116460 con 0x7f37f8109830 2024-09-15T22:19:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.755+0000 7f3802134740 1 Processor -- start 2024-09-15T22:19:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.755+0000 7f37f6ffd640 1 -- 172.21.15.5:0/1716516094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ec031e30 con 0x7f37f8109830 2024-09-15T22:19:04.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.755+0000 7f37f6ffd640 1 -- 172.21.15.5:0/1716516094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec035030 con 0x7f37f8109830 2024-09-15T22:19:04.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.756+0000 7f3802134740 1 -- start start 2024-09-15T22:19:04.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.757+0000 7f3802134740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc98b40 0x555c6dc97090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.757+0000 7f3802134740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc975d0 0x555c6dc97a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.757+0000 7f3802134740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dca2370 0x555c6dca4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.757+0000 7f3802134740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c6dbfd2f0 con 0x555c6dca2370 2024-09-15T22:19:04.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.757+0000 7f3802134740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555c6dba4cf0 con 0x555c6dc98b40 2024-09-15T22:19:04.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.757+0000 7f3802134740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555c6dbf92b0 con 0x555c6dc975d0 2024-09-15T22:19:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.757+0000 7f37fe2f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc98b40 0x555c6dc97090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.757+0000 7f37fdaf7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc975d0 0x555c6dc97a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.757+0000 7f37fe2f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc98b40 0x555c6dc97090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:48660/0 (socket says 172.21.15.5:48660) 2024-09-15T22:19:04.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.757+0000 7f37fe2f8640 1 -- 172.21.15.5:0/2050857667 learned_addr learned my addr 172.21.15.5:0/2050857667 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:04.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.758+0000 7f37fe2f8640 1 -- 172.21.15.5:0/2050857667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc975d0 msgr2=0x555c6dc97a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:04.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.758+0000 7f37fe2f8640 1 --2- 172.21.15.5:0/2050857667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc975d0 0x555c6dc97a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.758+0000 7f37fe2f8640 1 -- 172.21.15.5:0/2050857667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dca2370 msgr2=0x555c6dca4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:04.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.758+0000 7f37fe2f8640 1 --2- 172.21.15.5:0/2050857667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dca2370 0x555c6dca4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.758+0000 7f37fe2f8640 1 -- 172.21.15.5:0/2050857667 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c6db893a0 con 0x555c6dc98b40 2024-09-15T22:19:04.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.759+0000 7f37fe2f8640 1 --2- 172.21.15.5:0/2050857667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc98b40 0x555c6dc97090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f37e8002180 tx=0x7f37e800d5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2542e66a017a7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:04.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.760+0000 7f37dffff640 1 -- 172.21.15.5:0/2050857667 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8016020 con 0x555c6dc98b40 2024-09-15T22:19:04.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.760+0000 7f37dffff640 1 -- 172.21.15.5:0/2050857667 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e8012ad0 con 0x555c6dc98b40 2024-09-15T22:19:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.761+0000 7f3802134740 1 -- 172.21.15.5:0/2050857667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc98b40 msgr2=0x555c6dc97090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.761+0000 7f3802134740 1 --2- 172.21.15.5:0/2050857667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc98b40 0x555c6dc97090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f37e8002180 tx=0x7f37e800d5e0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.761+0000 7f3802134740 1 -- 172.21.15.5:0/2050857667 shutdown_connections 2024-09-15T22:19:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.761+0000 7f3802134740 1 --2- 172.21.15.5:0/2050857667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dca2370 0x555c6dca4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.761+0000 7f3802134740 1 --2- 172.21.15.5:0/2050857667 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc975d0 0x555c6dc97a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.761+0000 7f3802134740 1 --2- 172.21.15.5:0/2050857667 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc98b40 0x555c6dc97090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.761+0000 7f3802134740 1 -- 172.21.15.5:0/2050857667 >> 172.21.15.5:0/2050857667 conn(0x555c6dc60310 msgr2=0x555c6dc93c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.762+0000 7f37f6ffd640 1 -- 172.21.15.5:0/1716516094 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f37ec042070 con 0x7f37f8109830 2024-09-15T22:19:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.764+0000 7f37f6ffd640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f37e00a3500 0x7f37e00a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.764+0000 7f37f6ffd640 1 -- 172.21.15.5:0/1716516094 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7f37ec0f2bc0 con 0x7f37f8109830 2024-09-15T22:19:04.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.767+0000 7f3802134740 1 -- 172.21.15.5:0/2050857667 shutdown_connections 2024-09-15T22:19:04.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.770+0000 7f3802134740 1 -- 172.21.15.5:0/2050857667 wait complete. 2024-09-15T22:19:04.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.770+0000 7f3802134740 1 -- start start 2024-09-15T22:19:04.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.770+0000 7f37fdaf7640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f37e00a3500 0x7f37e00a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.770+0000 7f3802134740 1 Processor -- start 2024-09-15T22:19:04.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.774+0000 7f3802134740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 0x555c6dca1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.774+0000 7f3802134740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc98b40 0x555c6dc9f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.774+0000 7f3802134740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.774+0000 7f3802134740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c6dca50c0 con 0x555c6dc9f5b0 2024-09-15T22:19:04.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.774+0000 7f3802134740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555c6dbb8060 con 0x555c6dc975d0 2024-09-15T22:19:04.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.774+0000 7f3802134740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555c6dca5240 con 0x555c6dc98b40 2024-09-15T22:19:04.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.774+0000 7f3802134740 10 client.0 _ll_register_callbacks cb 0x555c6d90b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:04.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.775+0000 7f37fd2f6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc98b40 0x555c6dc9f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.775+0000 7f37fd2f6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc98b40 0x555c6dc9f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57838/0 (socket says 172.21.15.5:57838) 2024-09-15T22:19:04.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.775+0000 7f37fd2f6640 1 -- 172.21.15.5:0/3685204855 learned_addr learned my addr 172.21.15.5:0/3685204855 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:04.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.775+0000 7f37fdaf7640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f37e00a3500 0x7f37e00a59c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x555c6dc94ac0 tx=0x7f37f0008040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.777+0000 7f37fdaf7640 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 0x555c6dca1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.777+0000 7f37fe2f8640 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.778+0000 7f37fdaf7640 1 -- 172.21.15.5:0/3685204855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc98b40 msgr2=0x555c6dc9f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.778+0000 7f37fdaf7640 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc98b40 0x555c6dc9f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.778+0000 7f37fdaf7640 1 -- 172.21.15.5:0/3685204855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc9f5b0 msgr2=0x555c6dc9f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.778+0000 7f37fdaf7640 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.778+0000 7f37fdaf7640 1 -- 172.21.15.5:0/3685204855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c6d96e3c0 con 0x555c6dc975d0 2024-09-15T22:19:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.780+0000 7f37fdaf7640 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 0x555c6dca1d70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f37f000bfd0 tx=0x7f37f0004770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb4fbd1ff2d7df2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.780+0000 7f37dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:19:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.783+0000 7f37dd7fa640 1 -- 172.21.15.5:0/3685204855 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f0024060 con 0x555c6dc975d0 2024-09-15T22:19:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.783+0000 7f3802134740 1 -- 172.21.15.5:0/3685204855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e8012ad0 con 0x555c6dc975d0 2024-09-15T22:19:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.783+0000 7f37dd7fa640 1 -- 172.21.15.5:0/3685204855 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7f37f0012d30 con 0x555c6dc975d0 2024-09-15T22:19:04.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.784+0000 7f37dd7fa640 1 -- 172.21.15.5:0/3685204855 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37f002b050 con 0x555c6dc975d0 2024-09-15T22:19:04.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.784+0000 7f3802134740 10 client.26744 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:04.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.784+0000 7f3802134740 1 -- 172.21.15.5:0/3685204855 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555c6dcb1090 con 0x555c6dc975d0 2024-09-15T22:19:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.788+0000 7f37dd7fa640 1 -- 172.21.15.5:0/3685204855 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e8012ad0 con 0x555c6dc975d0 2024-09-15T22:19:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.788+0000 7f37dd7fa640 1 -- 172.21.15.5:0/3685204855 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f37f0013050 con 0x555c6dc975d0 2024-09-15T22:19:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.788+0000 7f3802134740 10 client.26744 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.788+0000 7f3802134740 10 client.26744 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:19:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.788+0000 7f3802134740 20 client.26744 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.788+0000 7f3802134740 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555c6dcb29c0 0x555c6dcb4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.788+0000 7f3802134740 4 client.26744 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.788+0000 7f3802134740 1 -- 172.21.15.5:0/3685204855 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c6dc0a860 con 0x555c6dcb29c0 2024-09-15T22:19:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.792+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555c6dcb29c0 0x555c6dcb4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.802+0000 7f37dd7fa640 10 client.26744 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:19:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.802+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555c6dcb29c0 0x555c6dcb4e20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ecfe872ac0e26a1 server_cookie=393a53a363a995c3 in_seq=0 out_seq=0 2024-09-15T22:19:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.804+0000 7f37dd7fa640 1 -- 172.21.15.5:0/3685204855 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c6dc0a860 con 0x555c6dcb29c0 2024-09-15T22:19:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.804+0000 7f37dd7fa640 10 client.26744 handle_command_reply: tid=0 2024-09-15T22:19:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.804+0000 7f3802134740 1 client.26744 shutdown 2024-09-15T22:19:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.806+0000 7f3802134740 1 -- 172.21.15.5:0/3685204855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 msgr2=0x555c6dca1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.806+0000 7f3802134740 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 0x555c6dca1d70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f37f000bfd0 tx=0x7f37f0004770 comp rx=0 tx=0).stop 2024-09-15T22:19:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.810+0000 7f3802134740 1 -- 172.21.15.5:0/3685204855 shutdown_connections 2024-09-15T22:19:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.810+0000 7f3802134740 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555c6dcb29c0 0x555c6dcb4e20 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.810+0000 7f3802134740 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.810+0000 7f3802134740 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc98b40 0x555c6dc9f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.810+0000 7f3802134740 1 --2- 172.21.15.5:0/3685204855 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 0x555c6dca1d70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.810+0000 7f3802134740 1 -- 172.21.15.5:0/3685204855 >> 172.21.15.5:0/3685204855 conn(0x555c6dc60310 msgr2=0x555c6dc99960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.810+0000 7f37dd7fa640 0 client.26744 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:19:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.811+0000 7f3802134740 1 -- 172.21.15.5:0/3685204855 shutdown_connections 2024-09-15T22:19:04.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.811+0000 7f3802134740 1 -- 172.21.15.5:0/3685204855 wait complete. 2024-09-15T22:19:04.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.811+0000 7f3802134740 20 client.26744 trim_cache size 0 max 16384 2024-09-15T22:19:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.842+0000 7f3802134740 1 Processor -- start 2024-09-15T22:19:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.843+0000 7f3802134740 1 -- start start 2024-09-15T22:19:04.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.844+0000 7f3802134740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 0x555c6dc93d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.844+0000 7f3802134740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 0x555c6dca0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.844+0000 7f37fdaf7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 0x555c6dc93d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.844+0000 7f37fdaf7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 0x555c6dc93d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:48678/0 (socket says 172.21.15.5:48678) 2024-09-15T22:19:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.845+0000 7f37fdaf7640 1 -- 172.21.15.5:0/2946321473 learned_addr learned my addr 172.21.15.5:0/2946321473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.845+0000 7f3802134740 1 --2- 172.21.15.5:0/2946321473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.845+0000 7f3802134740 1 -- 172.21.15.5:0/2946321473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37e4012a30 con 0x555c6dc98b40 2024-09-15T22:19:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.845+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/2946321473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 0x555c6dca0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.845+0000 7f37fe2f8640 1 --2- 172.21.15.5:0/2946321473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.846+0000 7f3802134740 1 -- 172.21.15.5:0/2946321473 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f37e4010300 con 0x555c6dc975d0 2024-09-15T22:19:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.846+0000 7f3802134740 1 -- 172.21.15.5:0/2946321473 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f37e400e170 con 0x555c6dc9f5b0 2024-09-15T22:19:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.846+0000 7f37fd2f6640 1 -- 172.21.15.5:0/2946321473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc9f5b0 msgr2=0x555c6dc9a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.846+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/2946321473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.846+0000 7f37fd2f6640 1 -- 172.21.15.5:0/2946321473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 msgr2=0x555c6dc93d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.846+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/2946321473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 0x555c6dc93d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.846+0000 7f37fd2f6640 1 -- 172.21.15.5:0/2946321473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c6dc94730 con 0x555c6dc98b40 2024-09-15T22:19:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.847+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/2946321473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 0x555c6dca0f90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f37ec006010 tx=0x7f37ec04b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186b38e16d1a3062 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:04.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.847+0000 7f37df7fe640 1 -- 172.21.15.5:0/2946321473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec049040 con 0x555c6dc98b40 2024-09-15T22:19:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.849+0000 7f37df7fe640 1 -- 172.21.15.5:0/2946321473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ec0bcd40 con 0x555c6dc98b40 2024-09-15T22:19:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.849+0000 7f37df7fe640 1 -- 172.21.15.5:0/2946321473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec0bb0b0 con 0x555c6dc98b40 2024-09-15T22:19:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.850+0000 7f3802134740 1 -- 172.21.15.5:0/2946321473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 msgr2=0x555c6dca0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.850+0000 7f3802134740 1 --2- 172.21.15.5:0/2946321473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 0x555c6dca0f90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f37ec006010 tx=0x7f37ec04b000 comp rx=0 tx=0).stop 2024-09-15T22:19:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.850+0000 7f3802134740 1 -- 172.21.15.5:0/2946321473 shutdown_connections 2024-09-15T22:19:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.850+0000 7f3802134740 1 --2- 172.21.15.5:0/2946321473 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.850+0000 7f3802134740 1 --2- 172.21.15.5:0/2946321473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 0x555c6dca0f90 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.850+0000 7f3802134740 1 --2- 172.21.15.5:0/2946321473 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc975d0 0x555c6dc93d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.851+0000 7f3802134740 1 -- 172.21.15.5:0/2946321473 >> 172.21.15.5:0/2946321473 conn(0x555c6dc60310 msgr2=0x555c6dca0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.852+0000 7f3802134740 1 -- 172.21.15.5:0/2946321473 shutdown_connections 2024-09-15T22:19:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.852+0000 7f3802134740 1 -- 172.21.15.5:0/2946321473 wait complete. 2024-09-15T22:19:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.852+0000 7f3802134740 1 -- start start 2024-09-15T22:19:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.853+0000 7f3802134740 1 Processor -- start 2024-09-15T22:19:04.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.853+0000 7f3802134740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc975d0 0x555c6dcaaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.854+0000 7f3802134740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 0x555c6dc9bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.854+0000 7f37fdaf7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc975d0 0x555c6dcaaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.854+0000 7f37fdaf7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc975d0 0x555c6dcaaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57856/0 (socket says 172.21.15.5:57856) 2024-09-15T22:19:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.854+0000 7f37fd2f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 0x555c6dc9bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.854+0000 7f37fd2f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 0x555c6dc9bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41574/0 (socket says 172.21.15.5:41574) 2024-09-15T22:19:04.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.854+0000 7f37fdaf7640 1 -- 172.21.15.5:0/3499311559 learned_addr learned my addr 172.21.15.5:0/3499311559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.854+0000 7f3802134740 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.855+0000 7f3802134740 1 -- 172.21.15.5:0/3499311559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37e400e7c0 con 0x555c6dc98b40 2024-09-15T22:19:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.855+0000 7f37fe2f8640 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.855+0000 7f3802134740 1 -- 172.21.15.5:0/3499311559 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f37e400fc30 con 0x555c6dc9f5b0 2024-09-15T22:19:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.855+0000 7f3802134740 1 -- 172.21.15.5:0/3499311559 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f37e400ee90 con 0x555c6dc975d0 2024-09-15T22:19:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.856+0000 7f3802134740 10 client.0 _ll_register_callbacks cb 0x7f37f000b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.856+0000 7f37fd2f6640 1 -- 172.21.15.5:0/3499311559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc975d0 msgr2=0x555c6dcaaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.856+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc975d0 0x555c6dcaaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.856+0000 7f37fd2f6640 1 -- 172.21.15.5:0/3499311559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc9f5b0 msgr2=0x555c6dc9c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.856+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.856+0000 7f37fd2f6640 1 -- 172.21.15.5:0/3499311559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c6dc9c680 con 0x555c6dc98b40 2024-09-15T22:19:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.856+0000 7f37fd2f6640 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 0x555c6dc9bb80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x555c6dbb8100 tx=0x7f37ec0bb2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b15d7bd81b7b94e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.856+0000 7f3802134740 1 -- 172.21.15.5:0/3499311559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37ec0bcd40 con 0x555c6dc98b40 2024-09-15T22:19:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.856+0000 7f37dd7fa640 10 client.17250 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:19:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.857+0000 7f37dd7fa640 1 -- 172.21.15.5:0/3499311559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec0d6050 con 0x555c6dc98b40 2024-09-15T22:19:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.858+0000 7f37dd7fa640 1 -- 172.21.15.5:0/3499311559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9541+0+0 (secure 0 0 0) 0x7f37ec0dd2d0 con 0x555c6dc98b40 2024-09-15T22:19:04.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.858+0000 7f37dd7fa640 1 -- 172.21.15.5:0/3499311559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37ec049070 con 0x555c6dc98b40 2024-09-15T22:19:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.861+0000 7f3802134740 10 client.17250 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:04.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.861+0000 7f3802134740 1 -- 172.21.15.5:0/3499311559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=25}) -- 0x555c6dd1d340 con 0x555c6dc98b40 2024-09-15T22:19:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.862+0000 7f37dd7fa640 1 -- 172.21.15.5:0/3499311559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f37ec0f0130 con 0x555c6dc98b40 2024-09-15T22:19:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.862+0000 7f3802134740 10 client.17250 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.862+0000 7f3802134740 10 client.17250 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:19:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.862+0000 7f3802134740 20 client.17250 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.862+0000 7f3802134740 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555c6dd1da20 0x555c6dd1fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.862+0000 7f3802134740 4 client.17250 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:19:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.862+0000 7f3802134740 1 -- 172.21.15.5:0/3499311559 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f37ec0bcd40 con 0x555c6dd1da20 2024-09-15T22:19:04.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.862+0000 7f37fdaf7640 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555c6dd1da20 0x555c6dd1fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.864+0000 7f37fdaf7640 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555c6dd1da20 0x555c6dd1fe40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9ec075f5e001697e server_cookie=5878c9d462337646 in_seq=0 out_seq=0 2024-09-15T22:19:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.864+0000 7f37dd7fa640 10 client.17250 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:19:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.867+0000 7f37dd7fa640 1 -- 172.21.15.5:0/3499311559 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f37ec0bcd40 con 0x555c6dd1da20 2024-09-15T22:19:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.867+0000 7f37dd7fa640 10 client.17250 handle_command_reply: tid=0 2024-09-15T22:19:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.867+0000 7f3802134740 1 client.17250 shutdown 2024-09-15T22:19:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.869+0000 7f3802134740 1 -- 172.21.15.5:0/3499311559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 msgr2=0x555c6dc9bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.869+0000 7f3802134740 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 0x555c6dc9bb80 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x555c6dbb8100 tx=0x7f37ec0bb2e0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.869+0000 7f3802134740 1 -- 172.21.15.5:0/3499311559 shutdown_connections 2024-09-15T22:19:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.869+0000 7f3802134740 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x555c6dd1da20 0x555c6dd1fe40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.869+0000 7f3802134740 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555c6dc9f5b0 0x555c6dc9c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.869+0000 7f3802134740 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c6dc98b40 0x555c6dc9bb80 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.869+0000 7f3802134740 1 --2- 172.21.15.5:0/3499311559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555c6dc975d0 0x555c6dcaaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.869+0000 7f3802134740 1 -- 172.21.15.5:0/3499311559 >> 172.21.15.5:0/3499311559 conn(0x555c6dc60310 msgr2=0x555c6dc92d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.870+0000 7f37dd7fa640 0 client.17250 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:19:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.870+0000 7f3802134740 1 -- 172.21.15.5:0/3499311559 shutdown_connections 2024-09-15T22:19:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.870+0000 7f3802134740 1 -- 172.21.15.5:0/3499311559 wait complete. 2024-09-15T22:19:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.870+0000 7f3802134740 20 client.17250 trim_cache size 0 max 16384 2024-09-15T22:19:04.875 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:19:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.874+0000 7f37ffd5b640 1 -- 172.21.15.5:0/1716516094 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f37e00a3500 msgr2=0x7f37e00a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.874+0000 7f37ffd5b640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f37e00a3500 0x7f37e00a59c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x555c6dc94ac0 tx=0x7f37f0008040 comp rx=0 tx=0).stop 2024-09-15T22:19:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.874+0000 7f37ffd5b640 1 -- 172.21.15.5:0/1716516094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f8109830 msgr2=0x7f37f8118bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.874+0000 7f37ffd5b640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f8109830 0x7f37f8118bf0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f37ec02a9b0 tx=0x7f37ec003cd0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.874+0000 7f37ffd5b640 1 -- 172.21.15.5:0/1716516094 shutdown_connections 2024-09-15T22:19:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.874+0000 7f37ffd5b640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f37e00a3500 0x7f37e00a59c0 secure :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0x555c6dc94ac0 tx=0x7f37f0008040 comp rx=0 tx=0).stop 2024-09-15T22:19:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.874+0000 7f37ffd5b640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f37f811b140 0x7f37f8111920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.874+0000 7f37ffd5b640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f8109830 0x7f37f8118bf0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.874+0000 7f37ffd5b640 1 --2- 172.21.15.5:0/1716516094 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f37f8109450 0x7f37f81186b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.874+0000 7f37ffd5b640 1 -- 172.21.15.5:0/1716516094 >> 172.21.15.5:0/1716516094 conn(0x7f37f806e140 msgr2=0x7f37f811c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.875+0000 7f37ffd5b640 1 -- 172.21.15.5:0/1716516094 shutdown_connections 2024-09-15T22:19:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:04.875+0000 7f37ffd5b640 1 -- 172.21.15.5:0/1716516094 wait complete. 2024-09-15T22:19:05.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:05 smithi099 ceph-mon[34892]: pgmap v314: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-09-15T22:19:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:05 smithi129 ceph-mon[35314]: pgmap v314: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-09-15T22:19:05.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:05 smithi005 ceph-mon[30944]: pgmap v314: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.2 MiB/s wr, 475 op/s 2024-09-15T22:19:07.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:07 smithi099 ceph-mon[34892]: pgmap v315: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.3 MiB/s wr, 606 op/s 2024-09-15T22:19:07.595 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:07 smithi005 ceph-mon[30944]: pgmap v315: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.3 MiB/s wr, 606 op/s 2024-09-15T22:19:07.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:07 smithi129 ceph-mon[35314]: pgmap v315: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 5.3 MiB/s wr, 606 op/s 2024-09-15T22:19:07.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:19:08.393 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:08.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:08 smithi005 ceph-mon[30944]: osdmap e96: 12 total, 12 up, 12 in 2024-09-15T22:19:08.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:08 smithi099 ceph-mon[34892]: osdmap e96: 12 total, 12 up, 12 in 2024-09-15T22:19:08.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:08 smithi129 ceph-mon[35314]: osdmap e96: 12 total, 12 up, 12 in 2024-09-15T22:19:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.003+0000 7fcfd376b640 1 -- 172.21.15.5:0/2520501972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfcc102ef0 msgr2=0x7fcfcc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.003+0000 7fcfd376b640 1 --2- 172.21.15.5:0/2520501972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfcc102ef0 0x7fcfcc10da40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009c10 tx=0x7fcfc402a7a0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.005+0000 7fcfd376b640 1 -- 172.21.15.5:0/2520501972 shutdown_connections 2024-09-15T22:19:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.005+0000 7fcfd376b640 1 --2- 172.21.15.5:0/2520501972 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfcc102ef0 0x7fcfcc10da40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.005+0000 7fcfd376b640 1 --2- 172.21.15.5:0/2520501972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfcc102570 0x7fcfcc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.005+0000 7fcfd376b640 1 --2- 172.21.15.5:0/2520501972 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfcc100e20 0x7fcfcc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.005+0000 7fcfd376b640 1 -- 172.21.15.5:0/2520501972 >> 172.21.15.5:0/2520501972 conn(0x7fcfcc0fc7d0 msgr2=0x7fcfcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:09.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.010+0000 7fcfd376b640 1 -- 172.21.15.5:0/2520501972 shutdown_connections 2024-09-15T22:19:09.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.011+0000 7fcfd376b640 1 -- 172.21.15.5:0/2520501972 wait complete. 2024-09-15T22:19:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.012+0000 7fcfd376b640 1 Processor -- start 2024-09-15T22:19:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.012+0000 7fcfd376b640 1 -- start start 2024-09-15T22:19:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.012+0000 7fcfd376b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfcc100e20 0x7fcfcc1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.013+0000 7fcfd376b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfcc102570 0x7fcfcc1968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.013+0000 7fcfd376b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfcc102ef0 0x7fcfcc19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.013+0000 7fcfd0d06640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfcc102570 0x7fcfcc1968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.013+0000 7fcfd0d06640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfcc102570 0x7fcfcc1968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57874/0 (socket says 172.21.15.5:57874) 2024-09-15T22:19:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.013+0000 7fcfd0d06640 1 -- 172.21.15.5:0/1339719130 learned_addr learned my addr 172.21.15.5:0/1339719130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.013+0000 7fcfd1d08640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfcc102ef0 0x7fcfcc19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.013+0000 7fcfd376b640 1 -- 172.21.15.5:0/1339719130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc10fec0 con 0x7fcfcc100e20 2024-09-15T22:19:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.013+0000 7fcfd376b640 1 -- 172.21.15.5:0/1339719130 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc10fd40 con 0x7fcfcc102ef0 2024-09-15T22:19:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.013+0000 7fcfd376b640 1 -- 172.21.15.5:0/1339719130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfcc110040 con 0x7fcfcc102570 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.014+0000 7fcfd1507640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfcc100e20 0x7fcfcc1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.014+0000 7fcfd0d06640 1 -- 172.21.15.5:0/1339719130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfcc102ef0 msgr2=0x7fcfcc19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.014+0000 7fcfd0d06640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfcc102ef0 0x7fcfcc19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.014+0000 7fcfd0d06640 1 -- 172.21.15.5:0/1339719130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfcc100e20 msgr2=0x7fcfcc1963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.014+0000 7fcfd0d06640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfcc100e20 0x7fcfcc1963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.014+0000 7fcfd0d06640 1 -- 172.21.15.5:0/1339719130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfcc19b350 con 0x7fcfcc102570 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.015+0000 7fcfd0d06640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfcc102570 0x7fcfcc1968e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00d950 tx=0x7fcfbc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.015+0000 7fcfc27fc640 1 -- 172.21.15.5:0/1339719130 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc017070 con 0x7fcfcc102570 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.015+0000 7fcfc27fc640 1 -- 172.21.15.5:0/1339719130 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfbc009c30 con 0x7fcfcc102570 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.016+0000 7fcfd376b640 1 -- 172.21.15.5:0/1339719130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfcc19b640 con 0x7fcfcc102570 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.016+0000 7fcfd376b640 1 -- 172.21.15.5:0/1339719130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfcc1a2f20 con 0x7fcfcc102570 2024-09-15T22:19:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.016+0000 7fcfc27fc640 1 -- 172.21.15.5:0/1339719130 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc00f620 con 0x7fcfcc102570 2024-09-15T22:19:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.020+0000 7fcfc27fc640 1 -- 172.21.15.5:0/1339719130 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcfbc00f7c0 con 0x7fcfcc102570 2024-09-15T22:19:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.023+0000 7fcfc27fc640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcfb80a79f0 0x7fcfb80a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.023+0000 7fcfd1507640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcfb80a79f0 0x7fcfb80a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.023+0000 7fcfc27fc640 1 -- 172.21.15.5:0/1339719130 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9629+0+0 (secure 0 0 0) 0x7fcfbc0d0ce0 con 0x7fcfcc102570 2024-09-15T22:19:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.024+0000 7fcfd1507640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcfb80a79f0 0x7fcfb80a9eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006fd0 tx=0x7fcfc8008040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.025+0000 7fcfd5b44740 1 Processor -- start 2024-09-15T22:19:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.025+0000 7fcfd5b44740 1 -- start start 2024-09-15T22:19:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.025+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.025+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b75d0 0x5591456b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.025+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.026+0000 7fcfd1d08640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.026+0000 7fcfd1d08640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:48710/0 (socket says 172.21.15.5:48710) 2024-09-15T22:19:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.026+0000 7fcfd1d08640 1 -- 172.21.15.5:0/1090843218 learned_addr learned my addr 172.21.15.5:0/1090843218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.026+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1090843218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55914561d2f0 con 0x5591456b8b40 2024-09-15T22:19:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.026+0000 7fcfd0d06640 1 --2- 172.21.15.5:0/1090843218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b75d0 0x5591456b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.026+0000 7fcfd1507640 1 --2- 172.21.15.5:0/1090843218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.027+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1090843218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5591455c4cf0 con 0x5591456c2370 2024-09-15T22:19:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.027+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1090843218 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5591456192b0 con 0x5591456b75d0 2024-09-15T22:19:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.027+0000 7fcfd1d08640 1 -- 172.21.15.5:0/1090843218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b75d0 msgr2=0x5591456b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.027+0000 7fcfd1d08640 1 --2- 172.21.15.5:0/1090843218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b75d0 0x5591456b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.027+0000 7fcfd1d08640 1 -- 172.21.15.5:0/1090843218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 msgr2=0x5591456b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.027+0000 7fcfd1d08640 1 --2- 172.21.15.5:0/1090843218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.027+0000 7fcfd1d08640 1 -- 172.21.15.5:0/1090843218 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591455a93a0 con 0x5591456c2370 2024-09-15T22:19:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.027+0000 7fcfd0d06640 1 --2- 172.21.15.5:0/1090843218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b75d0 0x5591456b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.028+0000 7fcfd1507640 1 --2- 172.21.15.5:0/1090843218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.028+0000 7fcfd1d08640 1 --2- 172.21.15.5:0/1090843218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456c4760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009be0 tx=0x7fcfc4030450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b72da1514cd1d0f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.028+0000 7fcfb77fe640 1 -- 172.21.15.5:0/1090843218 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4041030 con 0x5591456c2370 2024-09-15T22:19:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.028+0000 7fcfb77fe640 1 -- 172.21.15.5:0/1090843218 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc4003e50 con 0x5591456c2370 2024-09-15T22:19:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.028+0000 7fcfb77fe640 1 -- 172.21.15.5:0/1090843218 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4036040 con 0x5591456c2370 2024-09-15T22:19:09.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.029+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1090843218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 msgr2=0x5591456c4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:09.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.029+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/1090843218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456c4760 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009be0 tx=0x7fcfc4030450 comp rx=0 tx=0).stop 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.029+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1090843218 shutdown_connections 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.029+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/1090843218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456c4760 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.029+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/1090843218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b75d0 0x5591456b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.029+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/1090843218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.029+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1090843218 >> 172.21.15.5:0/1090843218 conn(0x559145680310 msgr2=0x5591456b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.029+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1090843218 shutdown_connections 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.029+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1090843218 wait complete. 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.030+0000 7fcfd5b44740 1 -- start start 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.030+0000 7fcfd5b44740 1 Processor -- start 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456b75d0 0x5591456c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456bef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456c2370 0x5591456bf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd5b44740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5591456c5010 con 0x5591456b8b40 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd1507640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456b75d0 0x5591456c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd0d06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456bef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd1507640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456b75d0 0x5591456c1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:48716/0 (socket says 172.21.15.5:48716) 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd1507640 1 -- 172.21.15.5:0/4153965463 learned_addr learned my addr 172.21.15.5:0/4153965463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd5b44740 1 -- 172.21.15.5:0/4153965463 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5591455d8060 con 0x5591456b75d0 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd5b44740 1 -- 172.21.15.5:0/4153965463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5591456c5190 con 0x5591456c2370 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd1d08640 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456c2370 0x5591456bf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd5b44740 10 client.0 _ll_register_callbacks cb 0x55914532b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd1d08640 1 -- 172.21.15.5:0/4153965463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456b75d0 msgr2=0x5591456c1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd1d08640 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456b75d0 0x5591456c1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd1d08640 1 -- 172.21.15.5:0/4153965463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 msgr2=0x5591456bef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd1d08640 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456bef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.031+0000 7fcfd1d08640 1 -- 172.21.15.5:0/4153965463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914538e3c0 con 0x5591456c2370 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.032+0000 7fcfd1507640 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456b75d0 0x5591456c1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.032+0000 7fcfd0d06640 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456bef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.032+0000 7fcfd1d08640 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456c2370 0x5591456bf480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009c10 tx=0x7fcfc4030e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f08121fd99bc549c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.032+0000 7fcfb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.032+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/4153965463 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4036050 con 0x5591456c2370 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.032+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/4153965463 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9629+0+0 (secure 0 0 0) 0x7fcfc4030740 con 0x5591456c2370 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.032+0000 7fcfd5b44740 1 -- 172.21.15.5:0/4153965463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc4003e50 con 0x5591456c2370 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.032+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/4153965463 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcfc4041040 con 0x5591456c2370 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.032+0000 7fcfd5b44740 10 client.26704 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.032+0000 7fcfd5b44740 1 -- 172.21.15.5:0/4153965463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5591456c5010 con 0x5591456c2370 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.033+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/4153965463 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc40309c0 con 0x5591456c2370 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.033+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/4153965463 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcfc403d460 con 0x5591456c2370 2024-09-15T22:19:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.033+0000 7fcfd5b44740 10 client.26704 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.033+0000 7fcfd5b44740 10 client.26704 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:19:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.033+0000 7fcfd5b44740 20 client.26704 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.033+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5591456d1b70 0x5591456d3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.033+0000 7fcfd5b44740 4 client.26704 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.033+0000 7fcfd5b44740 1 -- 172.21.15.5:0/4153965463 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcfc4003e50 con 0x5591456d1b70 2024-09-15T22:19:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.033+0000 7fcfd1507640 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5591456d1b70 0x5591456d3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.033+0000 7fcfd1507640 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5591456d1b70 0x5591456d3f60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1eac44b504f4dc6f server_cookie=a462fe522cc42f83 in_seq=0 out_seq=0 2024-09-15T22:19:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.034+0000 7fcfb4ff9640 10 client.26704 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:19:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.034+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/4153965463 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fcfc4003e50 con 0x5591456d1b70 2024-09-15T22:19:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.034+0000 7fcfb4ff9640 10 client.26704 handle_command_reply: tid=0 2024-09-15T22:19:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.034+0000 7fcfd5b44740 1 client.26704 shutdown 2024-09-15T22:19:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfd5b44740 1 -- 172.21.15.5:0/4153965463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456c2370 msgr2=0x5591456bf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456c2370 0x5591456bf480 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009c10 tx=0x7fcfc4030e70 comp rx=0 tx=0).stop 2024-09-15T22:19:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfd5b44740 1 -- 172.21.15.5:0/4153965463 shutdown_connections 2024-09-15T22:19:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5591456d1b70 0x5591456d3f60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfb4ff9640 0 client.26704 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:19:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456c2370 0x5591456bf480 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456bef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/4153965463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456b75d0 0x5591456c1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfd5b44740 1 -- 172.21.15.5:0/4153965463 >> 172.21.15.5:0/4153965463 conn(0x559145680310 msgr2=0x5591456b9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfd5b44740 1 -- 172.21.15.5:0/4153965463 shutdown_connections 2024-09-15T22:19:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfd5b44740 1 -- 172.21.15.5:0/4153965463 wait complete. 2024-09-15T22:19:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.035+0000 7fcfd5b44740 20 client.26704 trim_cache size 0 max 16384 2024-09-15T22:19:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.050+0000 7fcfd5b44740 1 Processor -- start 2024-09-15T22:19:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.050+0000 7fcfd5b44740 1 -- start start 2024-09-15T22:19:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b75d0 0x5591456ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b8b40 0x5591456d96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456d9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd5b44740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb0012950 con 0x5591456b75d0 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd5b44740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb0010220 con 0x5591456c2370 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd5b44740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb000e090 con 0x5591456b8b40 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd1507640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b75d0 0x5591456ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd0d06640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b8b40 0x5591456d96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd1507640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b75d0 0x5591456ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41606/0 (socket says 172.21.15.5:41606) 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd0d06640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b8b40 0x5591456d96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57904/0 (socket says 172.21.15.5:57904) 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd1507640 1 -- 172.21.15.5:0/641457182 learned_addr learned my addr 172.21.15.5:0/641457182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd1d08640 1 --2- 172.21.15.5:0/641457182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456d9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd1507640 1 -- 172.21.15.5:0/641457182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b8b40 msgr2=0x5591456d96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd1507640 1 --2- 172.21.15.5:0/641457182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b8b40 0x5591456d96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd1507640 1 -- 172.21.15.5:0/641457182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 msgr2=0x5591456d9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd1507640 1 --2- 172.21.15.5:0/641457182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456d9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd1507640 1 -- 172.21.15.5:0/641457182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591456b4960 con 0x5591456b75d0 2024-09-15T22:19:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd0d06640 1 --2- 172.21.15.5:0/641457182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b8b40 0x5591456d96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.051+0000 7fcfd1507640 1 --2- 172.21.15.5:0/641457182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b75d0 0x5591456ce310 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800f010 tx=0x7fcfc8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184ae56d590ebb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.052+0000 7fcfb6ffd640 1 -- 172.21.15.5:0/641457182 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8030070 con 0x5591456b75d0 2024-09-15T22:19:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.052+0000 7fcfb6ffd640 1 -- 172.21.15.5:0/641457182 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc800d820 con 0x5591456b75d0 2024-09-15T22:19:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.052+0000 7fcfb6ffd640 1 -- 172.21.15.5:0/641457182 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8023550 con 0x5591456b75d0 2024-09-15T22:19:09.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.052+0000 7fcfd5b44740 1 -- 172.21.15.5:0/641457182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b75d0 msgr2=0x5591456ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.052+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/641457182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b75d0 0x5591456ce310 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800f010 tx=0x7fcfc8010000 comp rx=0 tx=0).stop 2024-09-15T22:19:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.052+0000 7fcfd5b44740 1 -- 172.21.15.5:0/641457182 shutdown_connections 2024-09-15T22:19:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.052+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/641457182 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456d9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.052+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/641457182 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b8b40 0x5591456d96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.052+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/641457182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b75d0 0x5591456ce310 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.052+0000 7fcfd5b44740 1 -- 172.21.15.5:0/641457182 >> 172.21.15.5:0/641457182 conn(0x559145680310 msgr2=0x5591456b2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.053+0000 7fcfd5b44740 1 -- 172.21.15.5:0/641457182 shutdown_connections 2024-09-15T22:19:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.053+0000 7fcfd5b44740 1 -- 172.21.15.5:0/641457182 wait complete. 2024-09-15T22:19:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.053+0000 7fcfd5b44740 1 -- start start 2024-09-15T22:19:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.053+0000 7fcfd5b44740 1 Processor -- start 2024-09-15T22:19:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b75d0 0x5591456bbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456bc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd5b44740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456bc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd5b44740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb000e6e0 con 0x5591456b8b40 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd5b44740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb000fb50 con 0x5591456c2370 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd5b44740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb000edb0 con 0x5591456b75d0 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd0d06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456bc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd5b44740 10 client.0 _ll_register_callbacks cb 0x7fcfc8009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd0d06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456bc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41612/0 (socket says 172.21.15.5:41612) 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd0d06640 1 -- 172.21.15.5:0/1517842659 learned_addr learned my addr 172.21.15.5:0/1517842659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd1507640 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b75d0 0x5591456bbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd1d08640 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456bc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd0d06640 1 -- 172.21.15.5:0/1517842659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b75d0 msgr2=0x5591456bbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:09.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd0d06640 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b75d0 0x5591456bbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd0d06640 1 -- 172.21.15.5:0/1517842659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 msgr2=0x5591456bc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd0d06640 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456bc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd0d06640 1 -- 172.21.15.5:0/1517842659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914573b6d0 con 0x5591456b8b40 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd1d08640 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456bc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd0d06640 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456bc490 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009610 tx=0x7fcfbc09cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d258a26228c022e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfb4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/1517842659 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc020040 con 0x5591456b8b40 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/1517842659 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9629+0+0 (secure 0 0 0) 0x7fcfbc0bb4a0 con 0x5591456b8b40 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.054+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1517842659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc800d820 con 0x5591456b8b40 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/1517842659 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcfbc0ce410 con 0x5591456b8b40 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfd5b44740 10 client.17268 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1517842659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55914573c9f0 con 0x5591456b8b40 2024-09-15T22:19:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/1517842659 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc800d820 con 0x5591456b8b40 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/1517842659 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcfbc0a2050 con 0x5591456b8b40 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfd5b44740 10 client.17268 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfd5b44740 10 client.17268 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfd5b44740 20 client.17268 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5591456d1b70 0x55914573f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfd5b44740 4 client.17268 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1517842659 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55914562a860 con 0x5591456d1b70 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.055+0000 7fcfd1507640 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5591456d1b70 0x55914573f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfd1507640 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5591456d1b70 0x55914573f180 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=467d64e90ae3b541 server_cookie=267b8129d2cc3d44 in_seq=0 out_seq=0 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfb4ff9640 10 client.17268 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfb4ff9640 1 -- 172.21.15.5:0/1517842659 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55914562a860 con 0x5591456d1b70 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfb4ff9640 10 client.17268 handle_command_reply: tid=0 2024-09-15T22:19:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfd5b44740 1 client.17268 shutdown 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1517842659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 msgr2=0x5591456bc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456bc490 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc009610 tx=0x7fcfbc09cbd0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1517842659 shutdown_connections 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5591456d1b70 0x55914573f180 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5591456c2370 0x5591456bc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5591456b8b40 0x5591456bc490 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfd5b44740 1 --2- 172.21.15.5:0/1517842659 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5591456b75d0 0x5591456bbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1517842659 >> 172.21.15.5:0/1517842659 conn(0x559145680310 msgr2=0x5591456c07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfb4ff9640 0 client.17268 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.056+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1517842659 shutdown_connections 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.057+0000 7fcfd5b44740 1 -- 172.21.15.5:0/1517842659 wait complete. 2024-09-15T22:19:09.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.057+0000 7fcfd5b44740 20 client.17268 trim_cache size 0 max 16384 2024-09-15T22:19:09.065 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:19:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.058+0000 7fcfd376b640 1 -- 172.21.15.5:0/1339719130 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcfb80a79f0 msgr2=0x7fcfb80a9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.058+0000 7fcfd376b640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcfb80a79f0 0x7fcfb80a9eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006fd0 tx=0x7fcfc8008040 comp rx=0 tx=0).stop 2024-09-15T22:19:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.058+0000 7fcfd376b640 1 -- 172.21.15.5:0/1339719130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfcc102570 msgr2=0x7fcfcc1968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.058+0000 7fcfd376b640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfcc102570 0x7fcfcc1968e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00d950 tx=0x7fcfbc00de10 comp rx=0 tx=0).stop 2024-09-15T22:19:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.058+0000 7fcfd376b640 1 -- 172.21.15.5:0/1339719130 shutdown_connections 2024-09-15T22:19:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.058+0000 7fcfd376b640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcfb80a79f0 0x7fcfb80a9eb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.058+0000 7fcfd376b640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcfcc102ef0 0x7fcfcc19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.058+0000 7fcfd376b640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcfcc102570 0x7fcfcc1968e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.058+0000 7fcfd376b640 1 --2- 172.21.15.5:0/1339719130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfcc100e20 0x7fcfcc1963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.058+0000 7fcfd376b640 1 -- 172.21.15.5:0/1339719130 >> 172.21.15.5:0/1339719130 conn(0x7fcfcc0fc7d0 msgr2=0x7fcfcc1048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:09.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.059+0000 7fcfd376b640 1 -- 172.21.15.5:0/1339719130 shutdown_connections 2024-09-15T22:19:09.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:09.059+0000 7fcfd376b640 1 -- 172.21.15.5:0/1339719130 wait complete. 2024-09-15T22:19:09.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:09 smithi005 ceph-mon[30944]: pgmap v317: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.9 MiB/s wr, 457 op/s 2024-09-15T22:19:09.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:09 smithi099 ceph-mon[34892]: pgmap v317: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.9 MiB/s wr, 457 op/s 2024-09-15T22:19:09.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:09 smithi129 ceph-mon[35314]: pgmap v317: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.9 MiB/s wr, 457 op/s 2024-09-15T22:19:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 580.0 s, 542.0 tps, lat 184.770 ms stddev 226.189 2024-09-15T22:19:09.938 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:19:10.367 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.223+0000 7f7b8359e640 1 -- 172.21.15.5:0/4208309678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b841068d0 msgr2=0x7f7b84106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.223+0000 7f7b8359e640 1 --2- 172.21.15.5:0/4208309678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b841068d0 0x7f7b84106cb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009890 tx=0x7f7b7402a380 comp rx=0 tx=0).stop 2024-09-15T22:19:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.223+0000 7f7b8359e640 1 -- 172.21.15.5:0/4208309678 shutdown_connections 2024-09-15T22:19:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.223+0000 7f7b8359e640 1 --2- 172.21.15.5:0/4208309678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b84101270 0x7f7b8410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.223+0000 7f7b8359e640 1 --2- 172.21.15.5:0/4208309678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b841008d0 0x7f7b84100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.223+0000 7f7b8359e640 1 --2- 172.21.15.5:0/4208309678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b841068d0 0x7f7b84106cb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.223+0000 7f7b8359e640 1 -- 172.21.15.5:0/4208309678 >> 172.21.15.5:0/4208309678 conn(0x7f7b840fc630 msgr2=0x7f7b840fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:11.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.224+0000 7f7b8359e640 1 -- 172.21.15.5:0/4208309678 shutdown_connections 2024-09-15T22:19:11.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.224+0000 7f7b8359e640 1 -- 172.21.15.5:0/4208309678 wait complete. 2024-09-15T22:19:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b8359e640 1 Processor -- start 2024-09-15T22:19:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b8359e640 1 -- start start 2024-09-15T22:19:11.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b8359e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b841008d0 0x7f7b84074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b8359e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84101270 0x7f7b840711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b8359e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b841068d0 0x7f7b84071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b8359e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8410fbc0 con 0x7f7b841068d0 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b8359e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8410fa40 con 0x7f7b84101270 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b8359e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8410fd40 con 0x7f7b841008d0 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84101270 0x7f7b840711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b8259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b841008d0 0x7f7b84074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84101270 0x7f7b840711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:48760/0 (socket says 172.21.15.5:48760) 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.225+0000 7f7b7bfff640 1 -- 172.21.15.5:0/1878727299 learned_addr learned my addr 172.21.15.5:0/1878727299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.226+0000 7f7b82d9d640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b841068d0 0x7f7b84071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.226+0000 7f7b8259c640 1 -- 172.21.15.5:0/1878727299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84101270 msgr2=0x7f7b840711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.226+0000 7f7b8259c640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84101270 0x7f7b840711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.226+0000 7f7b8259c640 1 -- 172.21.15.5:0/1878727299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b841068d0 msgr2=0x7f7b84071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.226+0000 7f7b8259c640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b841068d0 0x7f7b84071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.226+0000 7f7b8259c640 1 -- 172.21.15.5:0/1878727299 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b841af0a0 con 0x7f7b841008d0 2024-09-15T22:19:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.226+0000 7f7b8259c640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b841008d0 0x7f7b84074120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7b740057c0 tx=0x7f7b74033ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.226+0000 7f7b7b7fe640 1 -- 172.21.15.5:0/1878727299 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74037040 con 0x7f7b841008d0 2024-09-15T22:19:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.227+0000 7f7b8359e640 1 -- 172.21.15.5:0/1878727299 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b841af270 con 0x7f7b841008d0 2024-09-15T22:19:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.227+0000 7f7b7b7fe640 1 -- 172.21.15.5:0/1878727299 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b74033d90 con 0x7f7b841008d0 2024-09-15T22:19:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.227+0000 7f7b7b7fe640 1 -- 172.21.15.5:0/1878727299 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b740038d0 con 0x7f7b841008d0 2024-09-15T22:19:11.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.227+0000 7f7b8359e640 1 -- 172.21.15.5:0/1878727299 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b841af770 con 0x7f7b841008d0 2024-09-15T22:19:11.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.232+0000 7f7b7b7fe640 1 -- 172.21.15.5:0/1878727299 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7b74042020 con 0x7f7b841008d0 2024-09-15T22:19:11.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.233+0000 7f7b8b11a740 1 Processor -- start 2024-09-15T22:19:11.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.233+0000 7f7b7b7fe640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7b700a3430 0x7f7b700a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.233+0000 7f7b8b11a740 1 -- start start 2024-09-15T22:19:11.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.233+0000 7f7b7b7fe640 1 -- 172.21.15.5:0/1878727299 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9629+0+0 (secure 0 0 0) 0x7f7b740efa80 con 0x7f7b841008d0 2024-09-15T22:19:11.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 0x55da4f1cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1d9520 0x55da4f1db910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55da4f1342f0 con 0x55da4f1d9520 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55da4f0dbcf0 con 0x55da4f1cfcf0 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55da4f1302b0 con 0x55da4f1ce780 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b82d9d640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7b700a3430 0x7f7b700a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b82d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1d9520 0x55da4f1db910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 0x55da4f1cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b82d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1d9520 0x55da4f1db910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41626/0 (socket says 172.21.15.5:41626) 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b82d9d640 1 -- 172.21.15.5:0/1676689331 learned_addr learned my addr 172.21.15.5:0/1676689331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 0x55da4f1cebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57932/0 (socket says 172.21.15.5:57932) 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8259c640 1 --2- 172.21.15.5:0/1676689331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8259c640 1 -- 172.21.15.5:0/1676689331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 msgr2=0x55da4f1cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8259c640 1 --2- 172.21.15.5:0/1676689331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 0x55da4f1cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8259c640 1 -- 172.21.15.5:0/1676689331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1d9520 msgr2=0x55da4f1db910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8259c640 1 --2- 172.21.15.5:0/1676689331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1d9520 0x55da4f1db910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8259c640 1 -- 172.21.15.5:0/1676689331 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da4f0c03a0 con 0x55da4f1cfcf0 2024-09-15T22:19:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b82d9d640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7b700a3430 0x7f7b700a58f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009aa0 tx=0x7f7b7c009f00 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.234+0000 7f7b8259c640 1 --2- 172.21.15.5:0/1676689331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1ce240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7b740ece90 tx=0x7f7b7402a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=865ffc8ff5ff0701 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.235+0000 7f7b78ff9640 1 -- 172.21.15.5:0/1676689331 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b740bfdd0 con 0x55da4f1cfcf0 2024-09-15T22:19:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.235+0000 7f7b78ff9640 1 -- 172.21.15.5:0/1676689331 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b740c4690 con 0x55da4f1cfcf0 2024-09-15T22:19:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.235+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1676689331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 msgr2=0x55da4f1ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.235+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/1676689331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1ce240 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7b740ece90 tx=0x7f7b7402a380 comp rx=0 tx=0).stop 2024-09-15T22:19:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.235+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1676689331 shutdown_connections 2024-09-15T22:19:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.235+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/1676689331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1d9520 0x55da4f1db910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.235+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/1676689331 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 0x55da4f1cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.235+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/1676689331 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1ce240 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.235+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1676689331 >> 172.21.15.5:0/1676689331 conn(0x55da4f197310 msgr2=0x55da4f1cae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.236+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1676689331 shutdown_connections 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.236+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1676689331 wait complete. 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.236+0000 7f7b8b11a740 1 -- start start 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.236+0000 7f7b8b11a740 1 Processor -- start 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.236+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1ce780 0x55da4f1d8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.236+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.236+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1d9520 0x55da4f1d65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.236+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55da4f0dbcf0 con 0x55da4f1ce780 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.236+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55da4f0ef060 con 0x55da4f1cfcf0 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.236+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55da4f1dbfe0 con 0x55da4f1d9520 2024-09-15T22:19:11.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b82d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1d9520 0x55da4f1d65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1d60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:48772/0 (socket says 172.21.15.5:48772) 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b7bfff640 1 -- 172.21.15.5:0/1321081458 learned_addr learned my addr 172.21.15.5:0/1321081458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b8259c640 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1ce780 0x55da4f1d8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b7bfff640 1 -- 172.21.15.5:0/1321081458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1d9520 msgr2=0x55da4f1d65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b7bfff640 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1d9520 0x55da4f1d65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b7bfff640 1 -- 172.21.15.5:0/1321081458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1ce780 msgr2=0x55da4f1d8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b7bfff640 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1ce780 0x55da4f1d8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b7bfff640 1 -- 172.21.15.5:0/1321081458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da4eea53c0 con 0x55da4f1cfcf0 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b7bfff640 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1d60a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c014690 tx=0x7f7b6c014b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c898a4b3186a92fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.237+0000 7f7b8b11a740 10 client.0 _ll_register_callbacks cb 0x55da4ee428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.238+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1321081458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b740c4690 con 0x55da4f1cfcf0 2024-09-15T22:19:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.238+0000 7f7b69ffb640 1 -- 172.21.15.5:0/1321081458 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c01b030 con 0x55da4f1cfcf0 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.238+0000 7f7b69ffb640 1 -- 172.21.15.5:0/1321081458 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9629+0+0 (secure 0 0 0) 0x7f7b6c004020 con 0x55da4f1cfcf0 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.238+0000 7f7b69ffb640 1 -- 172.21.15.5:0/1321081458 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c026050 con 0x55da4f1cfcf0 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.238+0000 7f7b8b11a740 10 client.26777 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.238+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1321081458 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55da4f1efe60 con 0x55da4f1cfcf0 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.239+0000 7f7b69ffb640 1 -- 172.21.15.5:0/1321081458 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7b6c004260 con 0x55da4f1cfcf0 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.239+0000 7f7b8b11a740 10 client.26777 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.239+0000 7f7b8b11a740 10 client.26777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.239+0000 7f7b8b11a740 20 client.26777 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.239+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55da4f1f15f0 0x55da4f1f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.239+0000 7f7b8b11a740 4 client.26777 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.239+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1321081458 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da4f141860 con 0x55da4f1f15f0 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.239+0000 7f7b8259c640 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55da4f1f15f0 0x55da4f1f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.239+0000 7f7b8259c640 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55da4f1f15f0 0x55da4f1f39e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2f41fa286c50768 server_cookie=ef0dbfc92321d77 in_seq=0 out_seq=0 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.239+0000 7f7b69ffb640 10 client.26777 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:11.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b69ffb640 1 -- 172.21.15.5:0/1321081458 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da4f141860 con 0x55da4f1f15f0 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b69ffb640 10 client.26777 handle_command_reply: tid=0 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b8b11a740 1 client.26777 shutdown 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1321081458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 msgr2=0x55da4f1d60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1d60a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c014690 tx=0x7f7b6c014b50 comp rx=0 tx=0).stop 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1321081458 shutdown_connections 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1d9520 0x55da4f1d65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55da4f1f15f0 0x55da4f1f39e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1d60a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b69ffb640 0 client.26777 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/1321081458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1ce780 0x55da4f1d8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1321081458 >> 172.21.15.5:0/1321081458 conn(0x55da4f197310 msgr2=0x55da4f1d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1321081458 shutdown_connections 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.240+0000 7f7b8b11a740 1 -- 172.21.15.5:0/1321081458 wait complete. 2024-09-15T22:19:11.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.241+0000 7f7b8b11a740 20 client.26777 trim_cache size 0 max 16384 2024-09-15T22:19:11.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.263+0000 7f7b8b11a740 1 Processor -- start 2024-09-15T22:19:11.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.268+0000 7f7b8b11a740 1 -- start start 2024-09-15T22:19:11.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.268+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f197310 0x55da4f1ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.268+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 0x55da4f1d1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.268+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.268+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841b7fa0 con 0x55da4f197310 2024-09-15T22:19:11.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.268+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841b5870 con 0x55da4f1cfcf0 2024-09-15T22:19:11.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.268+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841b36e0 con 0x55da4f1ce780 2024-09-15T22:19:11.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.270+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 0x55da4f1d1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.270+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 0x55da4f1d1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:57958/0 (socket says 172.21.15.5:57958) 2024-09-15T22:19:11.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.270+0000 7f7b7bfff640 1 -- 172.21.15.5:0/968855059 learned_addr learned my addr 172.21.15.5:0/968855059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:11.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.270+0000 7f7b8259c640 1 --2- 172.21.15.5:0/968855059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f197310 0x55da4f1ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.270+0000 7f7b8259c640 1 -- 172.21.15.5:0/968855059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 msgr2=0x55da4f1d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:11.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.270+0000 7f7b8259c640 1 --2- 172.21.15.5:0/968855059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 0x55da4f1d1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.270+0000 7f7b8259c640 1 -- 172.21.15.5:0/968855059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 msgr2=0x55da4f1d18a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:19:11.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.270+0000 7f7b8259c640 1 --2- 172.21.15.5:0/968855059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1d18a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.270+0000 7f7b8259c640 1 -- 172.21.15.5:0/968855059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da4f1ce590 con 0x55da4f197310 2024-09-15T22:19:11.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.271+0000 7f7b8259c640 1 --2- 172.21.15.5:0/968855059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f197310 0x55da4f1ca8b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x55da4f1d0320 tx=0x7f7b74005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14f59827bc0fe00 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.272+0000 7f7b6bfff640 1 -- 172.21.15.5:0/968855059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b740bfdd0 con 0x55da4f197310 2024-09-15T22:19:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.272+0000 7f7b6bfff640 1 -- 172.21.15.5:0/968855059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b740b8110 con 0x55da4f197310 2024-09-15T22:19:11.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.272+0000 7f7b6bfff640 1 -- 172.21.15.5:0/968855059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74032920 con 0x55da4f197310 2024-09-15T22:19:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.279+0000 7f7b8b11a740 1 -- 172.21.15.5:0/968855059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f197310 msgr2=0x55da4f1ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.279+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/968855059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f197310 0x55da4f1ca8b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x55da4f1d0320 tx=0x7f7b74005bd0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.280+0000 7f7b8b11a740 1 -- 172.21.15.5:0/968855059 shutdown_connections 2024-09-15T22:19:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.280+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/968855059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1cfcf0 0x55da4f1d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.280+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/968855059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1ce780 0x55da4f1d1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.280+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/968855059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f197310 0x55da4f1ca8b0 secure :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0x55da4f1d0320 tx=0x7f7b74005bd0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.280+0000 7f7b8b11a740 1 -- 172.21.15.5:0/968855059 >> 172.21.15.5:0/968855059 conn(0x55da4f1d6740 msgr2=0x55da4f1d5190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.286+0000 7f7b8b11a740 1 -- 172.21.15.5:0/968855059 shutdown_connections 2024-09-15T22:19:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.288+0000 7f7b8b11a740 1 -- 172.21.15.5:0/968855059 wait complete. 2024-09-15T22:19:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.289+0000 7f7b8b11a740 1 -- start start 2024-09-15T22:19:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.289+0000 7f7b8b11a740 1 Processor -- start 2024-09-15T22:19:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.289+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1ce780 0x55da4f1ea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.289+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1cfcf0 0x55da4f1e5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.289+0000 7f7b8b11a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1d9520 0x55da4f1e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.290+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841b3d30 con 0x55da4f1d9520 2024-09-15T22:19:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.290+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841b51a0 con 0x55da4f1ce780 2024-09-15T22:19:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.290+0000 7f7b8b11a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841b4400 con 0x55da4f1cfcf0 2024-09-15T22:19:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.290+0000 7f7b8b11a740 10 client.0 _ll_register_callbacks cb 0x7f7b6c00ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.293+0000 7f7b8259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1ce780 0x55da4f1ea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.293+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1cfcf0 0x55da4f1e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.293+0000 7f7b8259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1ce780 0x55da4f1ea0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:48800/0 (socket says 172.21.15.5:48800) 2024-09-15T22:19:11.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.293+0000 7f7b8259c640 1 -- 172.21.15.5:0/987691534 learned_addr learned my addr 172.21.15.5:0/987691534 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.295+0000 7f7b82d9d640 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1d9520 0x55da4f1e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.298+0000 7f7b7bfff640 1 -- 172.21.15.5:0/987691534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1ce780 msgr2=0x55da4f1ea0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.298+0000 7f7b7bfff640 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1ce780 0x55da4f1ea0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.298+0000 7f7b7bfff640 1 -- 172.21.15.5:0/987691534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1d9520 msgr2=0x55da4f1e58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.298+0000 7f7b7bfff640 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1d9520 0x55da4f1e58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.298+0000 7f7b7bfff640 1 -- 172.21.15.5:0/987691534 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da4f1e60e0 con 0x55da4f1cfcf0 2024-09-15T22:19:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.301+0000 7f7b7bfff640 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1cfcf0 0x55da4f1e5370 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c014b90 tx=0x7f7b6c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29b386daec1b042 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.301+0000 7f7b69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:19:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.301+0000 7f7b69ffb640 1 -- 172.21.15.5:0/987691534 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c030060 con 0x55da4f1cfcf0 2024-09-15T22:19:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.302+0000 7f7b69ffb640 1 -- 172.21.15.5:0/987691534 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9629+0+0 (secure 0 0 0) 0x7f7b6c005bf0 con 0x55da4f1cfcf0 2024-09-15T22:19:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.303+0000 7f7b8b11a740 1 -- 172.21.15.5:0/987691534 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b740b8110 con 0x55da4f1cfcf0 2024-09-15T22:19:11.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.306+0000 7f7b69ffb640 1 -- 172.21.15.5:0/987691534 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7b6c01b070 con 0x55da4f1cfcf0 2024-09-15T22:19:11.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.311+0000 7f7b8b11a740 10 client.26728 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:11.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.311+0000 7f7b8b11a740 1 -- 172.21.15.5:0/987691534 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55da4f27de70 con 0x55da4f1cfcf0 2024-09-15T22:19:11.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.312+0000 7f7b69ffb640 1 -- 172.21.15.5:0/987691534 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b740b8110 con 0x55da4f1cfcf0 2024-09-15T22:19:11.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.313+0000 7f7b69ffb640 1 -- 172.21.15.5:0/987691534 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7b6c017420 con 0x55da4f1cfcf0 2024-09-15T22:19:11.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.313+0000 7f7b8b11a740 10 client.26728 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:11.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.313+0000 7f7b8b11a740 10 client.26728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:11.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.313+0000 7f7b8b11a740 20 client.26728 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:11.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.313+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55da4f27e590 0x55da4f280980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:11.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.313+0000 7f7b8b11a740 4 client.26728 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:19:11.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.313+0000 7f7b8b11a740 1 -- 172.21.15.5:0/987691534 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da4f13bd70 con 0x55da4f27e590 2024-09-15T22:19:11.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.318+0000 7f7b8259c640 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55da4f27e590 0x55da4f280980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.322+0000 7f7b8259c640 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55da4f27e590 0x55da4f280980 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbc0a8bf1c47676 server_cookie=ab1d8af514a84688 in_seq=0 out_seq=0 2024-09-15T22:19:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.322+0000 7f7b69ffb640 10 client.26728 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.324+0000 7f7b69ffb640 1 -- 172.21.15.5:0/987691534 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da4f13bd70 con 0x55da4f27e590 2024-09-15T22:19:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.324+0000 7f7b69ffb640 10 client.26728 handle_command_reply: tid=0 2024-09-15T22:19:11.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.325+0000 7f7b8b11a740 1 client.26728 shutdown 2024-09-15T22:19:11.327 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:11 smithi005 ceph-mon[30944]: pgmap v318: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.9 MiB/s wr, 457 op/s 2024-09-15T22:19:11.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.330+0000 7f7b8b11a740 1 -- 172.21.15.5:0/987691534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1cfcf0 msgr2=0x55da4f1e5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:11.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.330+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1cfcf0 0x55da4f1e5370 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c014b90 tx=0x7f7b6c0046c0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.331+0000 7f7b8b11a740 1 -- 172.21.15.5:0/987691534 shutdown_connections 2024-09-15T22:19:11.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.331+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f1d9520 0x55da4f1e58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.331+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55da4f1cfcf0 0x55da4f1e5370 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.331+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55da4f27e590 0x55da4f280980 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.332+0000 7f7b8b11a740 1 --2- 172.21.15.5:0/987691534 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55da4f1ce780 0x55da4f1ea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.332+0000 7f7b8b11a740 1 -- 172.21.15.5:0/987691534 >> 172.21.15.5:0/987691534 conn(0x55da4f1d6740 msgr2=0x55da4f1d7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.332+0000 7f7b69ffb640 0 client.26728 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.332+0000 7f7b8b11a740 1 -- 172.21.15.5:0/987691534 shutdown_connections 2024-09-15T22:19:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.332+0000 7f7b8b11a740 1 -- 172.21.15.5:0/987691534 wait complete. 2024-09-15T22:19:11.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.332+0000 7f7b8b11a740 20 client.26728 trim_cache size 0 max 16384 2024-09-15T22:19:11.335 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:19:11.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.334+0000 7f7b8359e640 1 -- 172.21.15.5:0/1878727299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7b700a3430 msgr2=0x7f7b700a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:11.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.334+0000 7f7b8359e640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7b700a3430 0x7f7b700a58f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009aa0 tx=0x7f7b7c009f00 comp rx=0 tx=0).stop 2024-09-15T22:19:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.334+0000 7f7b8359e640 1 -- 172.21.15.5:0/1878727299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b841008d0 msgr2=0x7f7b84074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.334+0000 7f7b8359e640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b841008d0 0x7f7b84074120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7b740057c0 tx=0x7f7b74033ae0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.334+0000 7f7b8359e640 1 -- 172.21.15.5:0/1878727299 shutdown_connections 2024-09-15T22:19:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.334+0000 7f7b8359e640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7b700a3430 0x7f7b700a58f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.334+0000 7f7b8359e640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b841068d0 0x7f7b84071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.334+0000 7f7b8359e640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7b84101270 0x7f7b840711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.334+0000 7f7b8359e640 1 --2- 172.21.15.5:0/1878727299 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7b841008d0 0x7f7b84074120 secure :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7b740057c0 tx=0x7f7b74033ae0 comp rx=0 tx=0).stop 2024-09-15T22:19:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.334+0000 7f7b8359e640 1 -- 172.21.15.5:0/1878727299 >> 172.21.15.5:0/1878727299 conn(0x7f7b840fc630 msgr2=0x7f7b840fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.335+0000 7f7b8359e640 1 -- 172.21.15.5:0/1878727299 shutdown_connections 2024-09-15T22:19:11.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:11.335+0000 7f7b8359e640 1 -- 172.21.15.5:0/1878727299 wait complete. 2024-09-15T22:19:11.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:11 smithi099 ceph-mon[34892]: pgmap v318: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.9 MiB/s wr, 457 op/s 2024-09-15T22:19:11.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:11 smithi129 ceph-mon[35314]: pgmap v318: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 3.9 MiB/s wr, 457 op/s 2024-09-15T22:19:13.549 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:13 smithi005 ceph-mon[30944]: pgmap v319: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.2 MiB/s wr, 481 op/s 2024-09-15T22:19:13.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:13 smithi099 ceph-mon[34892]: pgmap v319: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.2 MiB/s wr, 481 op/s 2024-09-15T22:19:13.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:13 smithi129 ceph-mon[35314]: pgmap v319: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.2 MiB/s wr, 481 op/s 2024-09-15T22:19:14.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:14 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:19:14.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:14 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:19:14.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:14 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:19:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 585.0 s, 669.8 tps, lat 141.972 ms stddev 168.338 2024-09-15T22:19:15.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:15 smithi099 ceph-mon[34892]: pgmap v320: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.3 MiB/s wr, 494 op/s 2024-09-15T22:19:15.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:15 smithi129 ceph-mon[35314]: pgmap v320: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.3 MiB/s wr, 494 op/s 2024-09-15T22:19:15.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:15 smithi005 ceph-mon[30944]: pgmap v320: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 4.3 MiB/s wr, 494 op/s 2024-09-15T22:19:16.723 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:19:16.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:19:17.109 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: pgmap v321: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.9 MiB/s wr, 553 op/s 2024-09-15T22:19:17.109 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.109 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.110 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:16 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: pgmap v321: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.9 MiB/s wr, 553 op/s 2024-09-15T22:19:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:16 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.343 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: pgmap v321: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 4.9 MiB/s wr, 553 op/s 2024-09-15T22:19:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:17.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:16 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.018+0000 7f1f8939d640 1 -- 172.21.15.5:0/2382020452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f84101dd0 msgr2=0x7f1f841021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.018+0000 7f1f8939d640 1 --2- 172.21.15.5:0/2382020452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f84101dd0 0x7f1f841021b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c01acb0 tx=0x7f1f7c03b900 comp rx=0 tx=0).stop 2024-09-15T22:19:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.019+0000 7f1f8939d640 1 -- 172.21.15.5:0/2382020452 shutdown_connections 2024-09-15T22:19:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.019+0000 7f1f8939d640 1 --2- 172.21.15.5:0/2382020452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84106f00 0x7f1f8410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.019+0000 7f1f8939d640 1 --2- 172.21.15.5:0/2382020452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f84102780 0x7f1f841067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.019+0000 7f1f8939d640 1 --2- 172.21.15.5:0/2382020452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f84101dd0 0x7f1f841021b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.019+0000 7f1f8939d640 1 -- 172.21.15.5:0/2382020452 >> 172.21.15.5:0/2382020452 conn(0x7f1f840fd8b0 msgr2=0x7f1f840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.020+0000 7f1f8939d640 1 -- 172.21.15.5:0/2382020452 shutdown_connections 2024-09-15T22:19:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.020+0000 7f1f8939d640 1 -- 172.21.15.5:0/2382020452 wait complete. 2024-09-15T22:19:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.021+0000 7f1f8939d640 1 Processor -- start 2024-09-15T22:19:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.021+0000 7f1f8939d640 1 -- start start 2024-09-15T22:19:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.021+0000 7f1f8939d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f84101dd0 0x7f1f8410e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.021+0000 7f1f8939d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102780 0x7f1f8410a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.021+0000 7f1f8939d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f84106f00 0x7f1f8410ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.021+0000 7f1f8939d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8406e610 con 0x7f1f84101dd0 2024-09-15T22:19:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.021+0000 7f1f8939d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8406e490 con 0x7f1f84106f00 2024-09-15T22:19:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.021+0000 7f1f8939d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8406e790 con 0x7f1f84102780 2024-09-15T22:19:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.022+0000 7f1f827fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102780 0x7f1f8410a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.022+0000 7f1f827fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102780 0x7f1f8410a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:55828/0 (socket says 172.21.15.5:55828) 2024-09-15T22:19:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.022+0000 7f1f827fc640 1 -- 172.21.15.5:0/3858942509 learned_addr learned my addr 172.21.15.5:0/3858942509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.022+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f84101dd0 0x7f1f8410e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.023+0000 7f1f837fe640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f84106f00 0x7f1f8410ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.023+0000 7f1f82ffd640 1 -- 172.21.15.5:0/3858942509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102780 msgr2=0x7f1f8410a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.023+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102780 0x7f1f8410a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.023+0000 7f1f82ffd640 1 -- 172.21.15.5:0/3858942509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f84106f00 msgr2=0x7f1f8410ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.023+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f84106f00 0x7f1f8410ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.023+0000 7f1f82ffd640 1 -- 172.21.15.5:0/3858942509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8410b410 con 0x7f1f84101dd0 2024-09-15T22:19:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.023+0000 7f1f827fc640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102780 0x7f1f8410a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:19:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.024+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f84101dd0 0x7f1f8410e700 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c03bef0 tx=0x7f1f7c0406a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.033+0000 7f1f6bfff640 1 -- 172.21.15.5:0/3858942509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c011070 con 0x7f1f84101dd0 2024-09-15T22:19:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.033+0000 7f1f8939d640 1 -- 172.21.15.5:0/3858942509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f841be340 con 0x7f1f84101dd0 2024-09-15T22:19:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.033+0000 7f1f8939d640 1 -- 172.21.15.5:0/3858942509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f841be6e0 con 0x7f1f84101dd0 2024-09-15T22:19:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.033+0000 7f1f6bfff640 1 -- 172.21.15.5:0/3858942509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f7c003e40 con 0x7f1f84101dd0 2024-09-15T22:19:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.033+0000 7f1f6bfff640 1 -- 172.21.15.5:0/3858942509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c045070 con 0x7f1f84101dd0 2024-09-15T22:19:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.038+0000 7f1f6bfff640 1 -- 172.21.15.5:0/3858942509 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1f7c014070 con 0x7f1f84101dd0 2024-09-15T22:19:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.040+0000 7f1f6bfff640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f6c0a35c0 0x7f1f6c0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.040+0000 7f1f8b776740 1 Processor -- start 2024-09-15T22:19:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.040+0000 7f1f8b776740 1 -- start start 2024-09-15T22:19:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.040+0000 7f1f6bfff640 1 -- 172.21.15.5:0/3858942509 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9629+0+0 (secure 0 0 0) 0x7f1f7c0f0ae0 con 0x7f1f84101dd0 2024-09-15T22:19:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.040+0000 7f1f827fc640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f6c0a35c0 0x7f1f6c0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.040+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 0x55843af228c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.040+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af22e90 0x55843af1d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.040+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 0x55843af1da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f8b776740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55843ae81280 con 0x55843af1d630 2024-09-15T22:19:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f8b776740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55843ae28b10 con 0x55843af22e90 2024-09-15T22:19:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f8b776740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55843ae7d240 con 0x55843af224e0 2024-09-15T22:19:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f82ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 0x55843af228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f82ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 0x55843af228c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:55840/0 (socket says 172.21.15.5:55840) 2024-09-15T22:19:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f82ffd640 1 -- 172.21.15.5:0/1450957661 learned_addr learned my addr 172.21.15.5:0/1450957661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f837fe640 1 --2- 172.21.15.5:0/1450957661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 0x55843af1da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f82ffd640 1 -- 172.21.15.5:0/1450957661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af22e90 msgr2=0x55843af1d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f827fc640 1 --2- 172.21.15.5:0/1450957661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af22e90 0x55843af1d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/1450957661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af22e90 0x55843af1d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f82ffd640 1 -- 172.21.15.5:0/1450957661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 msgr2=0x55843af1da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/1450957661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 0x55843af1da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f82ffd640 1 -- 172.21.15.5:0/1450957661 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843ae351f0 con 0x55843af224e0 2024-09-15T22:19:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f827fc640 1 --2- 172.21.15.5:0/1450957661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af22e90 0x55843af1d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:19:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.041+0000 7f1f827fc640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f6c0a35c0 0x7f1f6c0a5a80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1f8410bf60 tx=0x7f1f78008040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.042+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/1450957661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 0x55843af228c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0c5050 tx=0x7f1f7c0bc810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98454f067b7b931d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.042+0000 7f1f697fa640 1 -- 172.21.15.5:0/1450957661 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0bc910 con 0x55843af224e0 2024-09-15T22:19:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.042+0000 7f1f697fa640 1 -- 172.21.15.5:0/1450957661 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f7c0c0cc0 con 0x55843af224e0 2024-09-15T22:19:18.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.042+0000 7f1f697fa640 1 -- 172.21.15.5:0/1450957661 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0ee000 con 0x55843af224e0 2024-09-15T22:19:18.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.043+0000 7f1f8b776740 1 -- 172.21.15.5:0/1450957661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 msgr2=0x55843af228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.043+0000 7f1f8b776740 1 --2- 172.21.15.5:0/1450957661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 0x55843af228c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0c5050 tx=0x7f1f7c0bc810 comp rx=0 tx=0).stop 2024-09-15T22:19:18.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.043+0000 7f1f8b776740 1 -- 172.21.15.5:0/1450957661 shutdown_connections 2024-09-15T22:19:18.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.043+0000 7f1f8b776740 1 --2- 172.21.15.5:0/1450957661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 0x55843af1da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.043+0000 7f1f8b776740 1 --2- 172.21.15.5:0/1450957661 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af22e90 0x55843af1d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.043+0000 7f1f8b776740 1 --2- 172.21.15.5:0/1450957661 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 0x55843af228c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.043+0000 7f1f8b776740 1 -- 172.21.15.5:0/1450957661 >> 172.21.15.5:0/1450957661 conn(0x55843af17ab0 msgr2=0x55843af17e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.045+0000 7f1f8b776740 1 -- 172.21.15.5:0/1450957661 shutdown_connections 2024-09-15T22:19:18.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.045+0000 7f1f8b776740 1 -- 172.21.15.5:0/1450957661 wait complete. 2024-09-15T22:19:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.046+0000 7f1f8b776740 1 -- start start 2024-09-15T22:19:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.046+0000 7f1f8b776740 1 Processor -- start 2024-09-15T22:19:18.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.047+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 0x55843af26300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.047+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af224e0 0x55843af26840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.047+0000 7f1f82ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 0x55843af26300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.048+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af22e90 0x55843af26d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.048+0000 7f1f82ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 0x55843af26300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37058/0 (socket says 172.21.15.5:37058) 2024-09-15T22:19:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.048+0000 7f1f827fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af224e0 0x55843af26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.048+0000 7f1f827fc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af224e0 0x55843af26840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33904/0 (socket says 172.21.15.5:33904) 2024-09-15T22:19:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.048+0000 7f1f82ffd640 1 -- 172.21.15.5:0/2524692324 learned_addr learned my addr 172.21.15.5:0/2524692324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.048+0000 7f1f837fe640 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af22e90 0x55843af26d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.048+0000 7f1f8b776740 1 -- 172.21.15.5:0/2524692324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55843af296f0 con 0x55843af1d630 2024-09-15T22:19:18.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.048+0000 7f1f8b776740 1 -- 172.21.15.5:0/2524692324 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55843ae3bf10 con 0x55843af224e0 2024-09-15T22:19:18.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.048+0000 7f1f8b776740 1 -- 172.21.15.5:0/2524692324 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55843af29870 con 0x55843af22e90 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.049+0000 7f1f8b776740 10 client.0 _ll_register_callbacks cb 0x55843ae04a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.049+0000 7f1f82ffd640 1 -- 172.21.15.5:0/2524692324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af22e90 msgr2=0x55843af26d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.049+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af22e90 0x55843af26d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.049+0000 7f1f82ffd640 1 -- 172.21.15.5:0/2524692324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af224e0 msgr2=0x55843af26840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.049+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af224e0 0x55843af26840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.049+0000 7f1f82ffd640 1 -- 172.21.15.5:0/2524692324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843abf26a0 con 0x55843af1d630 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.049+0000 7f1f827fc640 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af224e0 0x55843af26840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.049+0000 7f1f8b776740 1 -- 172.21.15.5:0/2524692324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f7c0c0cc0 con 0x55843af1d630 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.049+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 0x55843af26300 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c000f80 tx=0x7f1f7c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d07b641b6aa1ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.049+0000 7f1f66ff5640 10 client.17295 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.050+0000 7f1f66ff5640 1 -- 172.21.15.5:0/2524692324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0d9070 con 0x55843af1d630 2024-09-15T22:19:18.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.050+0000 7f1f66ff5640 1 -- 172.21.15.5:0/2524692324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9629+0+0 (secure 0 0 0) 0x7f1f7c004730 con 0x55843af1d630 2024-09-15T22:19:18.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.050+0000 7f1f66ff5640 1 -- 172.21.15.5:0/2524692324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f7c0db2f0 con 0x55843af1d630 2024-09-15T22:19:18.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.050+0000 7f1f8b776740 10 client.17295 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:18.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.050+0000 7f1f8b776740 1 -- 172.21.15.5:0/2524692324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55843af35080 con 0x55843af1d630 2024-09-15T22:19:18.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.051+0000 7f1f66ff5640 1 -- 172.21.15.5:0/2524692324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1f7c004b40 con 0x55843af1d630 2024-09-15T22:19:18.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.051+0000 7f1f8b776740 10 client.17295 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.051+0000 7f1f8b776740 10 client.17295 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.051+0000 7f1f8b776740 20 client.17295 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.051+0000 7f1f8b776740 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55843af36200 0x55843af385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.051+0000 7f1f8b776740 4 client.17295 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.051+0000 7f1f8b776740 1 -- 172.21.15.5:0/2524692324 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55843ae8e7f0 con 0x55843af36200 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.051+0000 7f1f827fc640 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55843af36200 0x55843af385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.051+0000 7f1f66ff5640 10 client.17295 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.051+0000 7f1f827fc640 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55843af36200 0x55843af385f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bac03a9c599cda6 server_cookie=9eb4cc9664f69557 in_seq=0 out_seq=0 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f66ff5640 1 -- 172.21.15.5:0/2524692324 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55843ae8e7f0 con 0x55843af36200 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f66ff5640 10 client.17295 handle_command_reply: tid=0 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f8b776740 1 client.17295 shutdown 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f8b776740 1 -- 172.21.15.5:0/2524692324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 msgr2=0x55843af26300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f8b776740 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 0x55843af26300 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c000f80 tx=0x7f1f7c005ef0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f8b776740 1 -- 172.21.15.5:0/2524692324 shutdown_connections 2024-09-15T22:19:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f8b776740 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af22e90 0x55843af26d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f8b776740 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55843af36200 0x55843af385f0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f66ff5640 0 client.17295 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f8b776740 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af224e0 0x55843af26840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f8b776740 1 --2- 172.21.15.5:0/2524692324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af1d630 0x55843af26300 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.052+0000 7f1f8b776740 1 -- 172.21.15.5:0/2524692324 >> 172.21.15.5:0/2524692324 conn(0x55843af17ab0 msgr2=0x55843af17270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.053+0000 7f1f8b776740 1 -- 172.21.15.5:0/2524692324 shutdown_connections 2024-09-15T22:19:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.053+0000 7f1f8b776740 1 -- 172.21.15.5:0/2524692324 wait complete. 2024-09-15T22:19:18.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.053+0000 7f1f8b776740 20 client.17295 trim_cache size 0 max 16384 2024-09-15T22:19:18.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.079+0000 7f1f8b776740 1 Processor -- start 2024-09-15T22:19:18.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.079+0000 7f1f8b776740 1 -- start start 2024-09-15T22:19:18.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.080+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 0x55843af1cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.080+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 0x55843af23330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.080+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af22e90 0x55843af23870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.080+0000 7f1f82ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 0x55843af1cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.080+0000 7f1f827fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 0x55843af23330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.080+0000 7f1f827fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 0x55843af23330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:55854/0 (socket says 172.21.15.5:55854) 2024-09-15T22:19:18.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.080+0000 7f1f82ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 0x55843af1cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33912/0 (socket says 172.21.15.5:33912) 2024-09-15T22:19:18.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.080+0000 7f1f827fc640 1 -- 172.21.15.5:0/1353150846 learned_addr learned my addr 172.21.15.5:0/1353150846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:18.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.080+0000 7f1f837fe640 1 --2- 172.21.15.5:0/1353150846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af22e90 0x55843af23870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.080+0000 7f1f8b776740 1 -- 172.21.15.5:0/1353150846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70012940 con 0x55843af22e90 2024-09-15T22:19:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.080+0000 7f1f8b776740 1 -- 172.21.15.5:0/1353150846 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f70010210 con 0x55843af1d630 2024-09-15T22:19:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.081+0000 7f1f8b776740 1 -- 172.21.15.5:0/1353150846 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7000e080 con 0x55843af224e0 2024-09-15T22:19:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.081+0000 7f1f837fe640 1 -- 172.21.15.5:0/1353150846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 msgr2=0x55843af23330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.081+0000 7f1f837fe640 1 --2- 172.21.15.5:0/1353150846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 0x55843af23330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.081+0000 7f1f837fe640 1 -- 172.21.15.5:0/1353150846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 msgr2=0x55843af1cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.081+0000 7f1f837fe640 1 --2- 172.21.15.5:0/1353150846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 0x55843af1cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.081+0000 7f1f837fe640 1 -- 172.21.15.5:0/1353150846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55843af32950 con 0x55843af22e90 2024-09-15T22:19:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.081+0000 7f1f837fe640 1 --2- 172.21.15.5:0/1353150846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af22e90 0x55843af23870 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009ca0 tx=0x7f1f74005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4683c88bd56691 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.082+0000 7f1f68ff9640 1 -- 172.21.15.5:0/1353150846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74022070 con 0x55843af22e90 2024-09-15T22:19:18.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.082+0000 7f1f68ff9640 1 -- 172.21.15.5:0/1353150846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f74009e50 con 0x55843af22e90 2024-09-15T22:19:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.082+0000 7f1f68ff9640 1 -- 172.21.15.5:0/1353150846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7401d6c0 con 0x55843af22e90 2024-09-15T22:19:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.083+0000 7f1f8b776740 1 -- 172.21.15.5:0/1353150846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af22e90 msgr2=0x55843af23870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.083+0000 7f1f8b776740 1 --2- 172.21.15.5:0/1353150846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af22e90 0x55843af23870 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009ca0 tx=0x7f1f74005f90 comp rx=0 tx=0).stop 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.083+0000 7f1f8b776740 1 -- 172.21.15.5:0/1353150846 shutdown_connections 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.083+0000 7f1f8b776740 1 --2- 172.21.15.5:0/1353150846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af22e90 0x55843af23870 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.083+0000 7f1f8b776740 1 --2- 172.21.15.5:0/1353150846 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af224e0 0x55843af23330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.083+0000 7f1f8b776740 1 --2- 172.21.15.5:0/1353150846 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 0x55843af1cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.083+0000 7f1f8b776740 1 -- 172.21.15.5:0/1353150846 >> 172.21.15.5:0/1353150846 conn(0x55843af17ab0 msgr2=0x55843af17270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.083+0000 7f1f8b776740 1 -- 172.21.15.5:0/1353150846 shutdown_connections 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.083+0000 7f1f8b776740 1 -- 172.21.15.5:0/1353150846 wait complete. 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.083+0000 7f1f8b776740 1 -- start start 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.083+0000 7f1f8b776740 1 Processor -- start 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 0x55843af2eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af224e0 0x55843af2f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f8b776740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af22e90 0x55843af2ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f8b776740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7000e6d0 con 0x55843af224e0 2024-09-15T22:19:18.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f8b776740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7000fb40 con 0x55843af1d630 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f8b776740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7000eda0 con 0x55843af22e90 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f82ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 0x55843af2eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f837fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af22e90 0x55843af2ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f82ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 0x55843af2eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33926/0 (socket says 172.21.15.5:33926) 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f82ffd640 1 -- 172.21.15.5:0/3133395853 learned_addr learned my addr 172.21.15.5:0/3133395853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f827fc640 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af224e0 0x55843af2f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f8b776740 10 client.0 _ll_register_callbacks cb 0x7f1f74009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f82ffd640 1 -- 172.21.15.5:0/3133395853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af22e90 msgr2=0x55843af2ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af22e90 0x55843af2ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f82ffd640 1 -- 172.21.15.5:0/3133395853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af224e0 msgr2=0x55843af2f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af224e0 0x55843af2f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.084+0000 7f1f82ffd640 1 -- 172.21.15.5:0/3133395853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55843af2bfd0 con 0x55843af1d630 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.085+0000 7f1f82ffd640 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 0x55843af2eac0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0b8d70 tx=0x7f1f7c040810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15dd652a9cfe4a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.085+0000 7f1f66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:19:18.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.085+0000 7f1f8b776740 1 -- 172.21.15.5:0/3133395853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f74009e50 con 0x55843af1d630 2024-09-15T22:19:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.085+0000 7f1f66ff5640 1 -- 172.21.15.5:0/3133395853 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0d9070 con 0x55843af1d630 2024-09-15T22:19:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.085+0000 7f1f66ff5640 1 -- 172.21.15.5:0/3133395853 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9629+0+0 (secure 0 0 0) 0x7f1f7c041ae0 con 0x55843af1d630 2024-09-15T22:19:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.085+0000 7f1f66ff5640 1 -- 172.21.15.5:0/3133395853 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1f7c0bc960 con 0x55843af1d630 2024-09-15T22:19:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.085+0000 7f1f8b776740 10 client.26795 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:18.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.085+0000 7f1f8b776740 1 -- 172.21.15.5:0/3133395853 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55843af2bfd0 con 0x55843af1d630 2024-09-15T22:19:18.091 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:18 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-15T22:19:18.091 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:18 smithi005 ceph-mon[30944]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-15T22:19:18.091 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:18 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-15T22:19:18.091 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:18 smithi005 ceph-mon[30944]: Unable to set osd_memory_target on smithi129 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:19:18.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.086+0000 7f1f66ff5640 1 -- 172.21.15.5:0/3133395853 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1f7c041ce0 con 0x55843af1d630 2024-09-15T22:19:18.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.086+0000 7f1f8b776740 10 client.26795 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:18.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.086+0000 7f1f8b776740 10 client.26795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:18.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.086+0000 7f1f8b776740 20 client.26795 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:18.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.086+0000 7f1f8b776740 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55843afa1760 0x55843afa3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:18.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.086+0000 7f1f8b776740 4 client.26795 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.086+0000 7f1f8b776740 1 -- 172.21.15.5:0/3133395853 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55843ae88d00 con 0x55843afa1760 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.087+0000 7f1f827fc640 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55843afa1760 0x55843afa3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.087+0000 7f1f827fc640 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55843afa1760 0x55843afa3b80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d399a32e48ed607b server_cookie=baba3cc546f16e51 in_seq=0 out_seq=0 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.087+0000 7f1f66ff5640 10 client.26795 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.088+0000 7f1f66ff5640 1 -- 172.21.15.5:0/3133395853 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55843ae88d00 con 0x55843afa1760 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.088+0000 7f1f66ff5640 10 client.26795 handle_command_reply: tid=0 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.088+0000 7f1f8b776740 1 client.26795 shutdown 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.089+0000 7f1f8b776740 1 -- 172.21.15.5:0/3133395853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 msgr2=0x55843af2eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.089+0000 7f1f8b776740 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 0x55843af2eac0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0b8d70 tx=0x7f1f7c040810 comp rx=0 tx=0).stop 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.089+0000 7f1f8b776740 1 -- 172.21.15.5:0/3133395853 shutdown_connections 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.089+0000 7f1f8b776740 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55843af22e90 0x55843af2ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.089+0000 7f1f8b776740 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55843af224e0 0x55843af2f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.089+0000 7f1f8b776740 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55843afa1760 0x55843afa3b80 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.089+0000 7f1f8b776740 1 --2- 172.21.15.5:0/3133395853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55843af1d630 0x55843af2eac0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.089+0000 7f1f66ff5640 0 client.26795 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:18.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.089+0000 7f1f8b776740 1 -- 172.21.15.5:0/3133395853 >> 172.21.15.5:0/3133395853 conn(0x55843af17ab0 msgr2=0x55843af31940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.090+0000 7f1f8b776740 1 -- 172.21.15.5:0/3133395853 shutdown_connections 2024-09-15T22:19:18.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.090+0000 7f1f8b776740 1 -- 172.21.15.5:0/3133395853 wait complete. 2024-09-15T22:19:18.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.090+0000 7f1f8b776740 20 client.26795 trim_cache size 0 max 16384 2024-09-15T22:19:18.095 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:19:18.095 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:19:18.095 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "ee568019-8cf6-474c-a2ab-d348261cb4db", 2024-09-15T22:19:18.095 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:19:18.096 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:19:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.092+0000 7f1f8939d640 1 -- 172.21.15.5:0/3858942509 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f6c0a35c0 msgr2=0x7f1f6c0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.092+0000 7f1f8939d640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f6c0a35c0 0x7f1f6c0a5a80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1f8410bf60 tx=0x7f1f78008040 comp rx=0 tx=0).stop 2024-09-15T22:19:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.092+0000 7f1f8939d640 1 -- 172.21.15.5:0/3858942509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f84101dd0 msgr2=0x7f1f8410e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.092+0000 7f1f8939d640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f84101dd0 0x7f1f8410e700 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c03bef0 tx=0x7f1f7c0406a0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.093+0000 7f1f8939d640 1 -- 172.21.15.5:0/3858942509 shutdown_connections 2024-09-15T22:19:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.093+0000 7f1f8939d640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f6c0a35c0 0x7f1f6c0a5a80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.093+0000 7f1f8939d640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f84106f00 0x7f1f8410ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.093+0000 7f1f8939d640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f84102780 0x7f1f8410a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.093+0000 7f1f8939d640 1 --2- 172.21.15.5:0/3858942509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f84101dd0 0x7f1f8410e700 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.093+0000 7f1f8939d640 1 -- 172.21.15.5:0/3858942509 >> 172.21.15.5:0/3858942509 conn(0x7f1f840fd8b0 msgr2=0x7f1f840fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.093+0000 7f1f8939d640 1 -- 172.21.15.5:0/3858942509 shutdown_connections 2024-09-15T22:19:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:18.093+0000 7f1f8939d640 1 -- 172.21.15.5:0/3858942509 wait complete. 2024-09-15T22:19:18.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:18 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-15T22:19:18.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:18 smithi099 ceph-mon[34892]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-15T22:19:18.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:18 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-15T22:19:18.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:18 smithi099 ceph-mon[34892]: Unable to set osd_memory_target on smithi129 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:19:18.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:18 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi099 to 176.0M 2024-09-15T22:19:18.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:18 smithi129 ceph-mon[35314]: Unable to set osd_memory_target on smithi099 to 184568115: error parsing value: Value '184568115' is below minimum 939524096 2024-09-15T22:19:18.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:18 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi129 to 176.0M 2024-09-15T22:19:18.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:18 smithi129 ceph-mon[35314]: Unable to set osd_memory_target on smithi129 to 184588902: error parsing value: Value '184588902' is below minimum 939524096 2024-09-15T22:19:18.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:19:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:19 smithi099 ceph-mon[34892]: pgmap v322: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.0 MiB/s wr, 556 op/s 2024-09-15T22:19:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:19 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:19:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:19 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:19 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:19:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:19 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:19 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:19 smithi099 ceph-mon[34892]: osdmap e97: 12 total, 12 up, 12 in 2024-09-15T22:19:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:19 smithi129 ceph-mon[35314]: pgmap v322: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.0 MiB/s wr, 556 op/s 2024-09-15T22:19:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:19 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:19:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:19 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:19 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:19:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:19 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:19 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:19.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:19 smithi129 ceph-mon[35314]: osdmap e97: 12 total, 12 up, 12 in 2024-09-15T22:19:19.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:19 smithi005 ceph-mon[30944]: pgmap v322: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 5.0 MiB/s wr, 556 op/s 2024-09-15T22:19:19.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:19 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:19:19.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:19 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:19.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:19 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:19:19.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:19 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:19.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:19 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:19.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:19 smithi005 ceph-mon[30944]: osdmap e97: 12 total, 12 up, 12 in 2024-09-15T22:19:19.448 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 590.0 s, 605.4 tps, lat 171.091 ms stddev 219.049 2024-09-15T22:19:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.050+0000 7f353bd82640 1 -- 172.21.15.5:0/3225816439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3534109450 msgr2=0x7f3534110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.050+0000 7f353bd82640 1 --2- 172.21.15.5:0/3225816439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3534109450 0x7f3534110250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3530009860 tx=0x7f353002a360 comp rx=0 tx=0).stop 2024-09-15T22:19:20.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.054+0000 7f353bd82640 1 -- 172.21.15.5:0/3225816439 shutdown_connections 2024-09-15T22:19:20.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.054+0000 7f353bd82640 1 --2- 172.21.15.5:0/3225816439 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f353411b140 0x7f353411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.054+0000 7f353bd82640 1 --2- 172.21.15.5:0/3225816439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534109830 0x7f3534109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.054+0000 7f353bd82640 1 --2- 172.21.15.5:0/3225816439 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3534109450 0x7f3534110250 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.054+0000 7f353bd82640 1 -- 172.21.15.5:0/3225816439 >> 172.21.15.5:0/3225816439 conn(0x7f353406e140 msgr2=0x7f353406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.056+0000 7f353bd82640 1 -- 172.21.15.5:0/3225816439 shutdown_connections 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.056+0000 7f353bd82640 1 -- 172.21.15.5:0/3225816439 wait complete. 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.057+0000 7f353bd82640 1 Processor -- start 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.057+0000 7f353bd82640 1 -- start start 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.057+0000 7f353bd82640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534109450 0x7f3534118730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.057+0000 7f353bd82640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3534109830 0x7f35341117c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.057+0000 7f353bd82640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f353411b140 0x7f3534111d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.057+0000 7f353bd82640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3534120df0 con 0x7f3534109450 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.057+0000 7f353bd82640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3534120c70 con 0x7f3534109830 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.057+0000 7f353bd82640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3534120f70 con 0x7f353411b140 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.058+0000 7f353a31f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f353411b140 0x7f3534111d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.058+0000 7f353a31f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f353411b140 0x7f3534111d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:55878/0 (socket says 172.21.15.5:55878) 2024-09-15T22:19:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.058+0000 7f353a31f640 1 -- 172.21.15.5:0/450974685 learned_addr learned my addr 172.21.15.5:0/450974685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.059+0000 7f353931d640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3534109830 0x7f35341117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.059+0000 7f353a31f640 1 -- 172.21.15.5:0/450974685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3534109830 msgr2=0x7f35341117c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.059+0000 7f353a31f640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3534109830 0x7f35341117c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.059+0000 7f353a31f640 1 -- 172.21.15.5:0/450974685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534109450 msgr2=0x7f3534118730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.059+0000 7f353a31f640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534109450 0x7f3534118730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.059+0000 7f353a31f640 1 -- 172.21.15.5:0/450974685 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35341122f0 con 0x7f353411b140 2024-09-15T22:19:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.059+0000 7f353a31f640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f353411b140 0x7f3534111d00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f352c00cab0 tx=0x7f352c00cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.061+0000 7f352affd640 1 -- 172.21.15.5:0/450974685 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c00adc0 con 0x7f353411b140 2024-09-15T22:19:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.061+0000 7f352affd640 1 -- 172.21.15.5:0/450974685 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c00f930 con 0x7f353411b140 2024-09-15T22:19:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.061+0000 7f352affd640 1 -- 172.21.15.5:0/450974685 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c018ab0 con 0x7f353411b140 2024-09-15T22:19:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.061+0000 7f353bd82640 1 -- 172.21.15.5:0/450974685 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35341125e0 con 0x7f353411b140 2024-09-15T22:19:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.061+0000 7f353bd82640 1 -- 172.21.15.5:0/450974685 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3534116580 con 0x7f353411b140 2024-09-15T22:19:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.070+0000 7f353e15b740 1 Processor -- start 2024-09-15T22:19:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.071+0000 7f353e15b740 1 -- start start 2024-09-15T22:19:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.071+0000 7f353e15b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68e4a0 0x56182e68e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.071+0000 7f353e15b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68edc0 0x56182e699140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.071+0000 7f353e15b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e69ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.071+0000 7f353e15b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56182e5f42e0 con 0x56182e699680 2024-09-15T22:19:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.071+0000 7f353e15b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56182e59bce0 con 0x56182e68e4a0 2024-09-15T22:19:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.071+0000 7f353e15b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56182e5f02a0 con 0x56182e68edc0 2024-09-15T22:19:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.072+0000 7f352affd640 1 -- 172.21.15.5:0/450974685 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f352c00f450 con 0x7f353411b140 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.073+0000 7f352affd640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f35200a35c0 0x7f35200a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.073+0000 7f352affd640 1 -- 172.21.15.5:0/450974685 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9541+0+0 (secure 0 0 0) 0x7f352c0d06d0 con 0x7f353411b140 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.074+0000 7f353931d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68e4a0 0x56182e68e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.074+0000 7f353931d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68e4a0 0x56182e68e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33950/0 (socket says 172.21.15.5:33950) 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.074+0000 7f353931d640 1 -- 172.21.15.5:0/131231046 learned_addr learned my addr 172.21.15.5:0/131231046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.074+0000 7f353931d640 1 --2- 172.21.15.5:0/131231046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e69ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.074+0000 7f3539b1e640 1 --2- 172.21.15.5:0/131231046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68edc0 0x56182e699140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.075+0000 7f353931d640 1 -- 172.21.15.5:0/131231046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68edc0 msgr2=0x56182e699140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.075+0000 7f353931d640 1 --2- 172.21.15.5:0/131231046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68edc0 0x56182e699140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.075+0000 7f353931d640 1 -- 172.21.15.5:0/131231046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 msgr2=0x56182e69ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.075+0000 7f353931d640 1 --2- 172.21.15.5:0/131231046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e69ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.075+0000 7f353931d640 1 -- 172.21.15.5:0/131231046 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182e580440 con 0x56182e68e4a0 2024-09-15T22:19:20.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.077+0000 7f353931d640 1 --2- 172.21.15.5:0/131231046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68e4a0 0x56182e68e880 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3524014810 tx=0x7f3524009600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a339c2bd3d5bb640 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.079+0000 7f353a31f640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f35200a35c0 0x7f35200a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.082+0000 7f351ffff640 1 -- 172.21.15.5:0/131231046 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524014310 con 0x56182e68e4a0 2024-09-15T22:19:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.082+0000 7f351ffff640 1 -- 172.21.15.5:0/131231046 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524014e50 con 0x56182e68e4a0 2024-09-15T22:19:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.082+0000 7f351ffff640 1 -- 172.21.15.5:0/131231046 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524003850 con 0x56182e68e4a0 2024-09-15T22:19:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.083+0000 7f353e15b740 1 -- 172.21.15.5:0/131231046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68e4a0 msgr2=0x56182e68e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.083+0000 7f353e15b740 1 --2- 172.21.15.5:0/131231046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68e4a0 0x56182e68e880 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3524014810 tx=0x7f3524009600 comp rx=0 tx=0).stop 2024-09-15T22:19:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.083+0000 7f353a31f640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f35200a35c0 0x7f35200a5a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f352c09c4e0 tx=0x7f352c0076f0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.086+0000 7f353e15b740 1 -- 172.21.15.5:0/131231046 shutdown_connections 2024-09-15T22:19:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.086+0000 7f353e15b740 1 --2- 172.21.15.5:0/131231046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e69ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.086+0000 7f353e15b740 1 --2- 172.21.15.5:0/131231046 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68edc0 0x56182e699140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.086+0000 7f353e15b740 1 --2- 172.21.15.5:0/131231046 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68e4a0 0x56182e68e880 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.086+0000 7f353e15b740 1 -- 172.21.15.5:0/131231046 >> 172.21.15.5:0/131231046 conn(0x56182e68aa30 msgr2=0x56182e68ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.088+0000 7f353e15b740 1 -- 172.21.15.5:0/131231046 shutdown_connections 2024-09-15T22:19:20.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.090+0000 7f353e15b740 1 -- 172.21.15.5:0/131231046 wait complete. 2024-09-15T22:19:20.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.090+0000 7f353e15b740 1 -- start start 2024-09-15T22:19:20.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.090+0000 7f353e15b740 1 Processor -- start 2024-09-15T22:19:20.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.095+0000 7f353e15b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 0x56182e698f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.095+0000 7f353e15b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e696220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.095+0000 7f353e15b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e696760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.095+0000 7f353e15b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56182e69c250 con 0x56182e699680 2024-09-15T22:19:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.095+0000 7f353e15b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56182e5af050 con 0x56182e68edc0 2024-09-15T22:19:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.095+0000 7f353e15b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56182e69c3d0 con 0x56182e68e4a0 2024-09-15T22:19:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.095+0000 7f353e15b740 10 client.0 _ll_register_callbacks cb 0x56182e5ef160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.097+0000 7f3539b1e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e696220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.099+0000 7f353931d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e696760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.100+0000 7f353931d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e696760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37110/0 (socket says 172.21.15.5:37110) 2024-09-15T22:19:20.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.100+0000 7f353931d640 1 -- 172.21.15.5:0/560899738 learned_addr learned my addr 172.21.15.5:0/560899738 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:20.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.100+0000 7f353931d640 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 0x56182e698f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.102+0000 7f3539b1e640 1 -- 172.21.15.5:0/560899738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 msgr2=0x56182e698f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.102+0000 7f3539b1e640 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 0x56182e698f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.102+0000 7f3539b1e640 1 -- 172.21.15.5:0/560899738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 msgr2=0x56182e696760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.102+0000 7f3539b1e640 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e696760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.102+0000 7f3539b1e640 1 -- 172.21.15.5:0/560899738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182e3653c0 con 0x56182e68edc0 2024-09-15T22:19:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.103+0000 7f3539b1e640 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e696220 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f353002a950 tx=0x7f3530004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e36734db8bfb64 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.103+0000 7f351d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:19:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.103+0000 7f351d7fa640 1 -- 172.21.15.5:0/560899738 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3530040020 con 0x56182e68edc0 2024-09-15T22:19:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.103+0000 7f353e15b740 1 -- 172.21.15.5:0/560899738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3524014e50 con 0x56182e68edc0 2024-09-15T22:19:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.103+0000 7f351d7fa640 1 -- 172.21.15.5:0/560899738 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9541+0+0 (secure 0 0 0) 0x7f3530030720 con 0x56182e68edc0 2024-09-15T22:19:20.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.104+0000 7f351d7fa640 1 -- 172.21.15.5:0/560899738 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f353004b050 con 0x56182e68edc0 2024-09-15T22:19:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.107+0000 7f353e15b740 10 client.26801 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:20.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.107+0000 7f353e15b740 1 -- 172.21.15.5:0/560899738 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56182e6a7c50 con 0x56182e68edc0 2024-09-15T22:19:20.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.110+0000 7f351d7fa640 1 -- 172.21.15.5:0/560899738 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524014e50 con 0x56182e68edc0 2024-09-15T22:19:20.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.110+0000 7f351d7fa640 1 -- 172.21.15.5:0/560899738 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f35300049a0 con 0x56182e68edc0 2024-09-15T22:19:20.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.110+0000 7f353e15b740 10 client.26801 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:20.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.110+0000 7f353e15b740 10 client.26801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:20.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.110+0000 7f353e15b740 20 client.26801 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:20.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.110+0000 7f353e15b740 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182e6a95b0 0x56182e6aba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.110+0000 7f353e15b740 4 client.26801 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:20.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.110+0000 7f353e15b740 1 -- 172.21.15.5:0/560899738 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56182e601850 con 0x56182e6a95b0 2024-09-15T22:19:20.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.115+0000 7f353931d640 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182e6a95b0 0x56182e6aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.119+0000 7f353931d640 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182e6a95b0 0x56182e6aba30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=621d27a747b99828 server_cookie=4013e01bb16e22c6 in_seq=0 out_seq=0 2024-09-15T22:19:20.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.119+0000 7f351d7fa640 10 client.26801 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:20.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.121+0000 7f351d7fa640 1 -- 172.21.15.5:0/560899738 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56182e601850 con 0x56182e6a95b0 2024-09-15T22:19:20.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.121+0000 7f351d7fa640 10 client.26801 handle_command_reply: tid=0 2024-09-15T22:19:20.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.121+0000 7f353e15b740 1 client.26801 shutdown 2024-09-15T22:19:20.124 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:20 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:20.124 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:20 smithi005 ceph-mon[30944]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-15T22:19:20.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:20 smithi005 ceph-mon[30944]: Unable to set osd_memory_target on smithi005 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-15T22:19:20.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:19:20.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:19:20.125 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.123+0000 7f353e15b740 1 -- 172.21.15.5:0/560899738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 msgr2=0x56182e696220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.123+0000 7f353e15b740 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e696220 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f353002a950 tx=0x7f3530004060 comp rx=0 tx=0).stop 2024-09-15T22:19:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.124+0000 7f353e15b740 1 -- 172.21.15.5:0/560899738 shutdown_connections 2024-09-15T22:19:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.124+0000 7f353e15b740 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e696760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.124+0000 7f353e15b740 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182e6a95b0 0x56182e6aba30 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.124+0000 7f353e15b740 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e696220 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.124+0000 7f353e15b740 1 --2- 172.21.15.5:0/560899738 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 0x56182e698f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.124+0000 7f351d7fa640 0 client.26801 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.124+0000 7f353e15b740 1 -- 172.21.15.5:0/560899738 >> 172.21.15.5:0/560899738 conn(0x56182e68aa30 msgr2=0x56182e689b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.124+0000 7f353e15b740 1 -- 172.21.15.5:0/560899738 shutdown_connections 2024-09-15T22:19:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.124+0000 7f353e15b740 1 -- 172.21.15.5:0/560899738 wait complete. 2024-09-15T22:19:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.124+0000 7f353e15b740 20 client.26801 trim_cache size 0 max 16384 2024-09-15T22:19:20.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.157+0000 7f353e15b740 1 Processor -- start 2024-09-15T22:19:20.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.158+0000 7f353e15b740 1 -- start start 2024-09-15T22:19:20.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.158+0000 7f353e15b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 0x56182e6a5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.158+0000 7f353e15b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e691900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.158+0000 7f353e15b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e695200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.158+0000 7f353e15b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35180129a0 con 0x56182e699680 2024-09-15T22:19:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.158+0000 7f353e15b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3518010270 con 0x56182e68edc0 2024-09-15T22:19:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.158+0000 7f353e15b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f351800e0e0 con 0x56182e68e4a0 2024-09-15T22:19:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.159+0000 7f353931d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 0x56182e6a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.159+0000 7f353931d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 0x56182e6a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:55912/0 (socket says 172.21.15.5:55912) 2024-09-15T22:19:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.159+0000 7f353931d640 1 -- 172.21.15.5:0/3735309614 learned_addr learned my addr 172.21.15.5:0/3735309614 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.159+0000 7f353931d640 1 -- 172.21.15.5:0/3735309614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 msgr2=0x56182e691900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:19:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.159+0000 7f353931d640 1 --2- 172.21.15.5:0/3735309614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e691900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.159+0000 7f353931d640 1 -- 172.21.15.5:0/3735309614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 msgr2=0x56182e695200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.159+0000 7f353931d640 1 --2- 172.21.15.5:0/3735309614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e695200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.159+0000 7f353931d640 1 -- 172.21.15.5:0/3735309614 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182e68bae0 con 0x56182e68e4a0 2024-09-15T22:19:20.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.160+0000 7f353931d640 1 --2- 172.21.15.5:0/3735309614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 0x56182e6a5540 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3524009530 tx=0x7f3524009a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=862e677e0861b99a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.160+0000 7f351f7fe640 1 -- 172.21.15.5:0/3735309614 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352400f730 con 0x56182e68e4a0 2024-09-15T22:19:20.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.160+0000 7f351f7fe640 1 -- 172.21.15.5:0/3735309614 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524033070 con 0x56182e68e4a0 2024-09-15T22:19:20.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.160+0000 7f351f7fe640 1 -- 172.21.15.5:0/3735309614 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524005360 con 0x56182e68e4a0 2024-09-15T22:19:20.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.162+0000 7f353e15b740 1 -- 172.21.15.5:0/3735309614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 msgr2=0x56182e6a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.162+0000 7f353e15b740 1 --2- 172.21.15.5:0/3735309614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 0x56182e6a5540 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3524009530 tx=0x7f3524009a60 comp rx=0 tx=0).stop 2024-09-15T22:19:20.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.163+0000 7f353e15b740 1 -- 172.21.15.5:0/3735309614 shutdown_connections 2024-09-15T22:19:20.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.163+0000 7f353e15b740 1 --2- 172.21.15.5:0/3735309614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e699680 0x56182e695200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.163+0000 7f353e15b740 1 --2- 172.21.15.5:0/3735309614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e691900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.163+0000 7f353e15b740 1 --2- 172.21.15.5:0/3735309614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e68e4a0 0x56182e6a5540 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.163+0000 7f353e15b740 1 -- 172.21.15.5:0/3735309614 >> 172.21.15.5:0/3735309614 conn(0x56182e68aa30 msgr2=0x56182e694af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.164+0000 7f353e15b740 1 -- 172.21.15.5:0/3735309614 shutdown_connections 2024-09-15T22:19:20.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.164+0000 7f353e15b740 1 -- 172.21.15.5:0/3735309614 wait complete. 2024-09-15T22:19:20.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.165+0000 7f353e15b740 1 -- start start 2024-09-15T22:19:20.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.165+0000 7f353e15b740 1 Processor -- start 2024-09-15T22:19:20.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.167+0000 7f353e15b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e68e4a0 0x56182e69ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.167+0000 7f353e15b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e69cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.167+0000 7f353e15b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e699680 0x56182e7122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.167+0000 7f353e15b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f351800e730 con 0x56182e68e4a0 2024-09-15T22:19:20.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.167+0000 7f353e15b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f351800fba0 con 0x56182e68edc0 2024-09-15T22:19:20.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.167+0000 7f353e15b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f351800ee00 con 0x56182e699680 2024-09-15T22:19:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.167+0000 7f353e15b740 10 client.0 _ll_register_callbacks cb 0x7f3530035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.168+0000 7f353931d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e699680 0x56182e7122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.168+0000 7f353931d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e699680 0x56182e7122c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:55914/0 (socket says 172.21.15.5:55914) 2024-09-15T22:19:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.168+0000 7f353931d640 1 -- 172.21.15.5:0/1731302800 learned_addr learned my addr 172.21.15.5:0/1731302800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.168+0000 7f3539b1e640 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e69cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.168+0000 7f353931d640 1 -- 172.21.15.5:0/1731302800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 msgr2=0x56182e69cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.168+0000 7f353931d640 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e69cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.168+0000 7f353931d640 1 -- 172.21.15.5:0/1731302800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e68e4a0 msgr2=0x56182e69ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.168+0000 7f353931d640 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e68e4a0 0x56182e69ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.168+0000 7f353931d640 1 -- 172.21.15.5:0/1731302800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182e712800 con 0x56182e699680 2024-09-15T22:19:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.168+0000 7f353931d640 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e699680 0x56182e7122c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3524009660 tx=0x7f3524009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ccf7a535f936c06 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.169+0000 7f351d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:19:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.170+0000 7f351d7fa640 1 -- 172.21.15.5:0/1731302800 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35240140c0 con 0x56182e699680 2024-09-15T22:19:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.170+0000 7f351d7fa640 1 -- 172.21.15.5:0/1731302800 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9541+0+0 (secure 0 0 0) 0x7f3524014260 con 0x56182e699680 2024-09-15T22:19:20.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.170+0000 7f353e15b740 1 -- 172.21.15.5:0/1731302800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3524033070 con 0x56182e699680 2024-09-15T22:19:20.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.171+0000 7f351d7fa640 1 -- 172.21.15.5:0/1731302800 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3524014460 con 0x56182e699680 2024-09-15T22:19:20.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.171+0000 7f353e15b740 10 client.26761 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.171+0000 7f353e15b740 1 -- 172.21.15.5:0/1731302800 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56182e7139b0 con 0x56182e699680 2024-09-15T22:19:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.172+0000 7f351d7fa640 1 -- 172.21.15.5:0/1731302800 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524033070 con 0x56182e699680 2024-09-15T22:19:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.172+0000 7f351d7fa640 1 -- 172.21.15.5:0/1731302800 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3524040570 con 0x56182e699680 2024-09-15T22:19:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.172+0000 7f353e15b740 10 client.26761 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.172+0000 7f353e15b740 10 client.26761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.172+0000 7f353e15b740 20 client.26761 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.172+0000 7f353e15b740 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182e713f10 0x56182e716330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.172+0000 7f353e15b740 4 client.26761 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:19:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.172+0000 7f353e15b740 1 -- 172.21.15.5:0/1731302800 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56182e5fbd60 con 0x56182e713f10 2024-09-15T22:19:20.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.172+0000 7f3539b1e640 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182e713f10 0x56182e716330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.173+0000 7f3539b1e640 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182e713f10 0x56182e716330 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c04be06469c55ea4 server_cookie=20ee2608e24dfd08 in_seq=0 out_seq=0 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.173+0000 7f351d7fa640 10 client.26761 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.173+0000 7f351d7fa640 1 -- 172.21.15.5:0/1731302800 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56182e5fbd60 con 0x56182e713f10 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.173+0000 7f351d7fa640 10 client.26761 handle_command_reply: tid=0 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.173+0000 7f353e15b740 1 client.26761 shutdown 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.174+0000 7f353e15b740 1 -- 172.21.15.5:0/1731302800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e699680 msgr2=0x56182e7122c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.174+0000 7f353e15b740 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e699680 0x56182e7122c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3524009660 tx=0x7f3524009b50 comp rx=0 tx=0).stop 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.174+0000 7f353e15b740 1 -- 172.21.15.5:0/1731302800 shutdown_connections 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.174+0000 7f353e15b740 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182e699680 0x56182e7122c0 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3524009660 tx=0x7f3524009b50 comp rx=0 tx=0).stop 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.174+0000 7f353e15b740 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182e713f10 0x56182e716330 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.174+0000 7f353e15b740 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182e68edc0 0x56182e69cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.174+0000 7f353e15b740 1 --2- 172.21.15.5:0/1731302800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182e68e4a0 0x56182e69ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.174+0000 7f353e15b740 1 -- 172.21.15.5:0/1731302800 >> 172.21.15.5:0/1731302800 conn(0x56182e68aa30 msgr2=0x56182e695310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.174+0000 7f351d7fa640 0 client.26761 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.175+0000 7f353e15b740 1 -- 172.21.15.5:0/1731302800 shutdown_connections 2024-09-15T22:19:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.175+0000 7f353e15b740 1 -- 172.21.15.5:0/1731302800 wait complete. 2024-09-15T22:19:20.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.176+0000 7f353e15b740 20 client.26761 trim_cache size 0 max 16384 2024-09-15T22:19:20.181 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:19:20.181 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:19:20.181 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:19:20.181 INFO:teuthology.orchestra.run.smithi005.stdout: "ee568019-8cf6-474c-a2ab-d348261cb4db": { 2024-09-15T22:19:20.181 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:19:20.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "ee568019-8cf6-474c-a2ab-d348261cb4db", 2024-09-15T22:19:20.181 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:19:20.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:19:20.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:19:20.181 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:19:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 -- 172.21.15.5:0/450974685 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f35200a35c0 msgr2=0x7f35200a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f35200a35c0 0x7f35200a5a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f352c09c4e0 tx=0x7f352c0076f0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 -- 172.21.15.5:0/450974685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f353411b140 msgr2=0x7f3534111d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f353411b140 0x7f3534111d00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f352c00cab0 tx=0x7f352c00cf70 comp rx=0 tx=0).stop 2024-09-15T22:19:20.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 -- 172.21.15.5:0/450974685 shutdown_connections 2024-09-15T22:19:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f35200a35c0 0x7f35200a5a80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f353411b140 0x7f3534111d00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3534109830 0x7f35341117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 --2- 172.21.15.5:0/450974685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534109450 0x7f3534118730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 -- 172.21.15.5:0/450974685 >> 172.21.15.5:0/450974685 conn(0x7f353406e140 msgr2=0x7f353411bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 -- 172.21.15.5:0/450974685 shutdown_connections 2024-09-15T22:19:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:20.178+0000 7f353bd82640 1 -- 172.21.15.5:0/450974685 wait complete. 2024-09-15T22:19:20.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:20 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:20.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:20 smithi099 ceph-mon[34892]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-15T22:19:20.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:20 smithi099 ceph-mon[34892]: Unable to set osd_memory_target on smithi005 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-15T22:19:20.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:19:20.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:19:20.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:20 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:19:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-15T22:19:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:20 smithi129 ceph-mon[35314]: Adjusting osd_memory_target on smithi005 to 177.0M 2024-09-15T22:19:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:20 smithi129 ceph-mon[35314]: Unable to set osd_memory_target on smithi005 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-15T22:19:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:19:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:19:20.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:19:20.975 INFO:tasks.cephfs.filesystem:scrub status for tag:ee568019-8cf6-474c-a2ab-d348261cb4db - {'path': '/', 'tag': 'ee568019-8cf6-474c-a2ab-d348261cb4db', 'options': 'recursive,force'} 2024-09-15T22:19:21.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:21 smithi099 ceph-mon[34892]: pgmap v324: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 585 op/s 2024-09-15T22:19:21.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:21 smithi129 ceph-mon[35314]: pgmap v324: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 585 op/s 2024-09-15T22:19:21.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:21 smithi005 ceph-mon[30944]: pgmap v324: 129 pgs: 129 active+clean; 2.6 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.3 MiB/s wr, 585 op/s 2024-09-15T22:19:23.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:23 smithi099 ceph-mon[34892]: pgmap v325: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 605 op/s 2024-09-15T22:19:23.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:23 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:19:23.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:23 smithi129 ceph-mon[35314]: pgmap v325: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 605 op/s 2024-09-15T22:19:23.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:23 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:19:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:23 smithi005 ceph-mon[30944]: pgmap v325: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.4 MiB/s wr, 605 op/s 2024-09-15T22:19:23.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:23 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:19:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 595.0 s, 698.4 tps, lat 144.274 ms stddev 219.527 2024-09-15T22:19:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:25 smithi099 ceph-mon[34892]: pgmap v326: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.5 MiB/s wr, 606 op/s 2024-09-15T22:19:25.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:25 smithi129 ceph-mon[35314]: pgmap v326: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.5 MiB/s wr, 606 op/s 2024-09-15T22:19:25.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:25 smithi005 ceph-mon[30944]: pgmap v326: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.5 MiB/s wr, 606 op/s 2024-09-15T22:19:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:27 smithi099 ceph-mon[34892]: pgmap v327: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.9 MiB/s wr, 638 op/s 2024-09-15T22:19:27.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:27 smithi129 ceph-mon[35314]: pgmap v327: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.9 MiB/s wr, 638 op/s 2024-09-15T22:19:27.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:27 smithi005 ceph-mon[30944]: pgmap v327: 129 pgs: 129 active+clean; 2.6 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 5.9 MiB/s wr, 638 op/s 2024-09-15T22:19:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:29 smithi099 ceph-mon[34892]: pgmap v328: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.3 MiB/s wr, 651 op/s 2024-09-15T22:19:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:29 smithi129 ceph-mon[35314]: pgmap v328: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.3 MiB/s wr, 651 op/s 2024-09-15T22:19:29.609 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:29 smithi005 ceph-mon[30944]: pgmap v328: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 6.3 MiB/s wr, 651 op/s 2024-09-15T22:19:29.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 600.0 s, 570.6 tps, lat 176.868 ms stddev 199.375 2024-09-15T22:19:31.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:31 smithi099 ceph-mon[34892]: pgmap v329: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.8 MiB/s wr, 599 op/s 2024-09-15T22:19:31.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:31 smithi129 ceph-mon[35314]: pgmap v329: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.8 MiB/s wr, 599 op/s 2024-09-15T22:19:31.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:31 smithi005 ceph-mon[30944]: pgmap v329: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 5.8 MiB/s wr, 599 op/s 2024-09-15T22:19:33.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:33 smithi099 ceph-mon[34892]: pgmap v330: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 9.2 MiB/s wr, 780 op/s 2024-09-15T22:19:33.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:33 smithi129 ceph-mon[35314]: pgmap v330: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 9.2 MiB/s wr, 780 op/s 2024-09-15T22:19:33.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:33 smithi005 ceph-mon[30944]: pgmap v330: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 9.2 MiB/s wr, 780 op/s 2024-09-15T22:19:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 605.0 s, 592.2 tps, lat 164.996 ms stddev 229.948 2024-09-15T22:19:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:35 smithi099 ceph-mon[34892]: pgmap v331: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.6 MiB/s wr, 606 op/s 2024-09-15T22:19:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:35 smithi129 ceph-mon[35314]: pgmap v331: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.6 MiB/s wr, 606 op/s 2024-09-15T22:19:35.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:35 smithi005 ceph-mon[30944]: pgmap v331: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 7.6 MiB/s wr, 606 op/s 2024-09-15T22:19:37.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:37 smithi099 ceph-mon[34892]: pgmap v332: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 764 op/s 2024-09-15T22:19:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:37 smithi129 ceph-mon[35314]: pgmap v332: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 764 op/s 2024-09-15T22:19:37.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:37 smithi005 ceph-mon[30944]: pgmap v332: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 11 MiB/s wr, 764 op/s 2024-09-15T22:19:39.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:39 smithi099 ceph-mon[34892]: pgmap v333: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.3 MiB/s wr, 613 op/s 2024-09-15T22:19:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:39 smithi129 ceph-mon[35314]: pgmap v333: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.3 MiB/s wr, 613 op/s 2024-09-15T22:19:39.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:39 smithi005 ceph-mon[30944]: pgmap v333: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 9.3 MiB/s wr, 613 op/s 2024-09-15T22:19:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 610.0 s, 688.6 tps, lat 149.231 ms stddev 171.546 2024-09-15T22:19:41.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:41 smithi099 ceph-mon[34892]: pgmap v334: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 8.3 MiB/s wr, 520 op/s 2024-09-15T22:19:41.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:41 smithi129 ceph-mon[35314]: pgmap v334: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 8.3 MiB/s wr, 520 op/s 2024-09-15T22:19:41.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:41 smithi005 ceph-mon[30944]: pgmap v334: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 8.3 MiB/s wr, 520 op/s 2024-09-15T22:19:43.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:43 smithi099 ceph-mon[34892]: pgmap v335: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 734 op/s 2024-09-15T22:19:43.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:43 smithi129 ceph-mon[35314]: pgmap v335: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 734 op/s 2024-09-15T22:19:43.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:43 smithi005 ceph-mon[30944]: pgmap v335: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 12 MiB/s wr, 734 op/s 2024-09-15T22:19:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 615.0 s, 654.8 tps, lat 148.911 ms stddev 186.802 2024-09-15T22:19:45.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:45 smithi099 ceph-mon[34892]: pgmap v336: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.1 MiB/s wr, 520 op/s 2024-09-15T22:19:45.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:45 smithi129 ceph-mon[35314]: pgmap v336: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.1 MiB/s wr, 520 op/s 2024-09-15T22:19:45.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:45 smithi005 ceph-mon[30944]: pgmap v336: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 8.1 MiB/s wr, 520 op/s 2024-09-15T22:19:47.592 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:47 smithi099 ceph-mon[34892]: pgmap v337: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 11 MiB/s wr, 658 op/s 2024-09-15T22:19:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:47 smithi129 ceph-mon[35314]: pgmap v337: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 11 MiB/s wr, 658 op/s 2024-09-15T22:19:47.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:47 smithi005 ceph-mon[30944]: pgmap v337: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 11 MiB/s wr, 658 op/s 2024-09-15T22:19:49.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:49 smithi099 ceph-mon[34892]: pgmap v338: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.8 MiB/s wr, 547 op/s 2024-09-15T22:19:49.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:49 smithi129 ceph-mon[35314]: pgmap v338: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.8 MiB/s wr, 547 op/s 2024-09-15T22:19:49.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:49 smithi005 ceph-mon[30944]: pgmap v338: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 8.8 MiB/s wr, 547 op/s 2024-09-15T22:19:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 620.0 s, 694.4 tps, lat 148.139 ms stddev 204.477 2024-09-15T22:19:50.976 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:19:51.334 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:51 smithi005 ceph-mon[30944]: pgmap v339: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.8 MiB/s wr, 472 op/s 2024-09-15T22:19:51.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:51 smithi099 ceph-mon[34892]: pgmap v339: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.8 MiB/s wr, 472 op/s 2024-09-15T22:19:51.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:51 smithi129 ceph-mon[35314]: pgmap v339: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.8 MiB/s wr, 472 op/s 2024-09-15T22:19:51.877 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:52.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.735+0000 7f9b640f3640 1 -- 172.21.15.5:0/2050894752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c1027a0 msgr2=0x7f9b5c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:52.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.735+0000 7f9b640f3640 1 --2- 172.21.15.5:0/2050894752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c1027a0 0x7f9b5c1067f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c013150 tx=0x7f9b4c032780 comp rx=0 tx=0).stop 2024-09-15T22:19:52.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.735+0000 7f9b640f3640 1 -- 172.21.15.5:0/2050894752 shutdown_connections 2024-09-15T22:19:52.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.735+0000 7f9b640f3640 1 --2- 172.21.15.5:0/2050894752 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c106f20 0x7f9b5c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.735+0000 7f9b640f3640 1 --2- 172.21.15.5:0/2050894752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c1027a0 0x7f9b5c1067f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.735+0000 7f9b640f3640 1 --2- 172.21.15.5:0/2050894752 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b5c101df0 0x7f9b5c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.735+0000 7f9b640f3640 1 -- 172.21.15.5:0/2050894752 >> 172.21.15.5:0/2050894752 conn(0x7f9b5c0fd8f0 msgr2=0x7f9b5c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:52.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.736+0000 7f9b640f3640 1 -- 172.21.15.5:0/2050894752 shutdown_connections 2024-09-15T22:19:52.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.737+0000 7f9b640f3640 1 -- 172.21.15.5:0/2050894752 wait complete. 2024-09-15T22:19:52.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.737+0000 7f9b640f3640 1 Processor -- start 2024-09-15T22:19:52.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.737+0000 7f9b640f3640 1 -- start start 2024-09-15T22:19:52.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.738+0000 7f9b640f3640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b5c101df0 0x7f9b5c1a7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.738+0000 7f9b640f3640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c1027a0 0x7f9b5c1a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.738+0000 7f9b640f3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c106f20 0x7f9b5c1abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.738+0000 7f9b640f3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c10fe00 con 0x7f9b5c106f20 2024-09-15T22:19:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.738+0000 7f9b640f3640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c10fc80 con 0x7f9b5c1027a0 2024-09-15T22:19:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.738+0000 7f9b640f3640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5c10ff80 con 0x7f9b5c101df0 2024-09-15T22:19:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.738+0000 7f9b61e8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b5c101df0 0x7f9b5c1a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.738+0000 7f9b61e8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b5c101df0 0x7f9b5c1a7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:33582/0 (socket says 172.21.15.5:33582) 2024-09-15T22:19:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.738+0000 7f9b61e8f640 1 -- 172.21.15.5:0/1301905090 learned_addr learned my addr 172.21.15.5:0/1301905090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.738+0000 7f9b6168e640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c1027a0 0x7f9b5c1a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.738+0000 7f9b62690640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c106f20 0x7f9b5c1abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.739+0000 7f9b6168e640 1 -- 172.21.15.5:0/1301905090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b5c101df0 msgr2=0x7f9b5c1a7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.739+0000 7f9b6168e640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b5c101df0 0x7f9b5c1a7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.739+0000 7f9b6168e640 1 -- 172.21.15.5:0/1301905090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c106f20 msgr2=0x7f9b5c1abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.739+0000 7f9b6168e640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c106f20 0x7f9b5c1abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.739+0000 7f9b6168e640 1 -- 172.21.15.5:0/1301905090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5c1ac300 con 0x7f9b5c1027a0 2024-09-15T22:19:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.739+0000 7f9b6168e640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c1027a0 0x7f9b5c1a7890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c013150 tx=0x7f9b4c0042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.740+0000 7f9b52ffd640 1 -- 172.21.15.5:0/1301905090 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c005d50 con 0x7f9b5c1027a0 2024-09-15T22:19:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.740+0000 7f9b640f3640 1 -- 172.21.15.5:0/1301905090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5c1ac590 con 0x7f9b5c1027a0 2024-09-15T22:19:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.740+0000 7f9b52ffd640 1 -- 172.21.15.5:0/1301905090 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b4c004ba0 con 0x7f9b5c1027a0 2024-09-15T22:19:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.740+0000 7f9b52ffd640 1 -- 172.21.15.5:0/1301905090 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0044e0 con 0x7f9b5c1027a0 2024-09-15T22:19:52.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.740+0000 7f9b640f3640 1 -- 172.21.15.5:0/1301905090 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b5c19b300 con 0x7f9b5c1027a0 2024-09-15T22:19:52.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.744+0000 7f9b52ffd640 1 -- 172.21.15.5:0/1301905090 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9b4c0046c0 con 0x7f9b5c1027a0 2024-09-15T22:19:52.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.745+0000 7f9b52ffd640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b480a35c0 0x7f9b480a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.745+0000 7f9b52ffd640 1 -- 172.21.15.5:0/1301905090 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9541+0+0 (secure 0 0 0) 0x7f9b4c0f0240 con 0x7f9b5c1027a0 2024-09-15T22:19:52.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.745+0000 7f9b61e8f640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b480a35c0 0x7f9b480a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.746+0000 7f9b61e8f640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b480a35c0 0x7f9b480a5a80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009a30 tx=0x7f9b58009190 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.747+0000 7f9b664cc740 1 Processor -- start 2024-09-15T22:19:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.747+0000 7f9b664cc740 1 -- start start 2024-09-15T22:19:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.747+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea234a0 0x55a90ea23880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.747+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea23dc0 0x55a90ea2e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.747+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea30a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b664cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a90e9892e0 con 0x55a90ea2e680 2024-09-15T22:19:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b664cc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a90e930ce0 con 0x55a90ea234a0 2024-09-15T22:19:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b664cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a90e9852a0 con 0x55a90ea23dc0 2024-09-15T22:19:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b62690640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea30a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b62690640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea30a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48900/0 (socket says 172.21.15.5:48900) 2024-09-15T22:19:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b62690640 1 -- 172.21.15.5:0/2355267622 learned_addr learned my addr 172.21.15.5:0/2355267622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b62690640 1 -- 172.21.15.5:0/2355267622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea23dc0 msgr2=0x55a90ea2e140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:19:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b62690640 1 --2- 172.21.15.5:0/2355267622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea23dc0 0x55a90ea2e140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b62690640 1 -- 172.21.15.5:0/2355267622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea234a0 msgr2=0x55a90ea23880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:19:52.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b62690640 1 --2- 172.21.15.5:0/2355267622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea234a0 0x55a90ea23880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b62690640 1 -- 172.21.15.5:0/2355267622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90e915440 con 0x55a90ea2e680 2024-09-15T22:19:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.748+0000 7f9b62690640 1 --2- 172.21.15.5:0/2355267622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea30a70 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009fd0 tx=0x7f9b540118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51d3958aecd93a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.749+0000 7f9b47fff640 1 -- 172.21.15.5:0/2355267622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54004d20 con 0x55a90ea2e680 2024-09-15T22:19:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.749+0000 7f9b47fff640 1 -- 172.21.15.5:0/2355267622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b54004050 con 0x55a90ea2e680 2024-09-15T22:19:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.749+0000 7f9b47fff640 1 -- 172.21.15.5:0/2355267622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54005280 con 0x55a90ea2e680 2024-09-15T22:19:52.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.750+0000 7f9b664cc740 1 -- 172.21.15.5:0/2355267622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 msgr2=0x55a90ea30a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.750+0000 7f9b664cc740 1 --2- 172.21.15.5:0/2355267622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea30a70 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009fd0 tx=0x7f9b540118d0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.750+0000 7f9b664cc740 1 -- 172.21.15.5:0/2355267622 shutdown_connections 2024-09-15T22:19:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.750+0000 7f9b664cc740 1 --2- 172.21.15.5:0/2355267622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea30a70 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.750+0000 7f9b664cc740 1 --2- 172.21.15.5:0/2355267622 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea23dc0 0x55a90ea2e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.750+0000 7f9b664cc740 1 --2- 172.21.15.5:0/2355267622 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea234a0 0x55a90ea23880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.750+0000 7f9b664cc740 1 -- 172.21.15.5:0/2355267622 >> 172.21.15.5:0/2355267622 conn(0x55a90ea1fa30 msgr2=0x55a90ea1fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.750+0000 7f9b664cc740 1 -- 172.21.15.5:0/2355267622 shutdown_connections 2024-09-15T22:19:52.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.750+0000 7f9b664cc740 1 -- 172.21.15.5:0/2355267622 wait complete. 2024-09-15T22:19:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.751+0000 7f9b664cc740 1 -- start start 2024-09-15T22:19:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.751+0000 7f9b664cc740 1 Processor -- start 2024-09-15T22:19:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.752+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea234a0 0x55a90ea2e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.752+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea23dc0 0x55a90ea2b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.752+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 0x55a90ea2bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.752+0000 7f9b664cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a90ea31250 con 0x55a90ea234a0 2024-09-15T22:19:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.752+0000 7f9b664cc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a90e944050 con 0x55a90ea2e680 2024-09-15T22:19:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.752+0000 7f9b664cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a90ea313d0 con 0x55a90ea23dc0 2024-09-15T22:19:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.752+0000 7f9b664cc740 10 client.0 _ll_register_callbacks cb 0x55a90e984160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.752+0000 7f9b6168e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea23dc0 0x55a90ea2b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.752+0000 7f9b6168e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea23dc0 0x55a90ea2b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:33598/0 (socket says 172.21.15.5:33598) 2024-09-15T22:19:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.752+0000 7f9b6168e640 1 -- 172.21.15.5:0/236274244 learned_addr learned my addr 172.21.15.5:0/236274244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.752+0000 7f9b62690640 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 0x55a90ea2bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.753+0000 7f9b61e8f640 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea234a0 0x55a90ea2e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.753+0000 7f9b6168e640 1 -- 172.21.15.5:0/236274244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 msgr2=0x55a90ea2bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.753+0000 7f9b6168e640 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 0x55a90ea2bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.753+0000 7f9b6168e640 1 -- 172.21.15.5:0/236274244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea234a0 msgr2=0x55a90ea2e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.753+0000 7f9b6168e640 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea234a0 0x55a90ea2e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.753+0000 7f9b6168e640 1 -- 172.21.15.5:0/236274244 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90e6fa3c0 con 0x55a90ea23dc0 2024-09-15T22:19:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.753+0000 7f9b61e8f640 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea234a0 0x55a90ea2e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:19:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.753+0000 7f9b6168e640 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea23dc0 0x55a90ea2b3e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c006010 tx=0x7f9b4c0bc060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d060de87cde5e21 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.753+0000 7f9b457fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:19:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.754+0000 7f9b457fa640 1 -- 172.21.15.5:0/236274244 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0ed9b0 con 0x55a90ea23dc0 2024-09-15T22:19:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.754+0000 7f9b457fa640 1 -- 172.21.15.5:0/236274244 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9541+0+0 (secure 0 0 0) 0x7f9b4c0c0130 con 0x55a90ea23dc0 2024-09-15T22:19:52.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.754+0000 7f9b664cc740 1 -- 172.21.15.5:0/236274244 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b54004050 con 0x55a90ea23dc0 2024-09-15T22:19:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.754+0000 7f9b457fa640 1 -- 172.21.15.5:0/236274244 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b4c0bc8d0 con 0x55a90ea23dc0 2024-09-15T22:19:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.755+0000 7f9b664cc740 10 client.26767 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.755+0000 7f9b664cc740 1 -- 172.21.15.5:0/236274244 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a90ea31250 con 0x55a90ea23dc0 2024-09-15T22:19:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.755+0000 7f9b457fa640 1 -- 172.21.15.5:0/236274244 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b4c0bc1d0 con 0x55a90ea23dc0 2024-09-15T22:19:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.756+0000 7f9b457fa640 1 -- 172.21.15.5:0/236274244 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9b4c04b060 con 0x55a90ea23dc0 2024-09-15T22:19:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.757+0000 7f9b664cc740 10 client.26767 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.757+0000 7f9b664cc740 10 client.26767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.757+0000 7f9b664cc740 20 client.26767 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.757+0000 7f9b664cc740 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a90ea3e290 0x55a90ea406b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.757+0000 7f9b664cc740 4 client.26767 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:52.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.757+0000 7f9b664cc740 1 -- 172.21.15.5:0/236274244 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9b54004050 con 0x55a90ea3e290 2024-09-15T22:19:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.758+0000 7f9b61e8f640 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a90ea3e290 0x55a90ea406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.759+0000 7f9b61e8f640 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a90ea3e290 0x55a90ea406b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128e0f6c8ef54f8c server_cookie=8518fd82c2f5634a in_seq=0 out_seq=0 2024-09-15T22:19:52.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.759+0000 7f9b457fa640 10 client.26767 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.759+0000 7f9b457fa640 1 -- 172.21.15.5:0/236274244 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f9b54004050 con 0x55a90ea3e290 2024-09-15T22:19:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.759+0000 7f9b457fa640 10 client.26767 handle_command_reply: tid=0 2024-09-15T22:19:52.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.759+0000 7f9b664cc740 1 client.26767 shutdown 2024-09-15T22:19:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b664cc740 1 -- 172.21.15.5:0/236274244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea23dc0 msgr2=0x55a90ea2b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b664cc740 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea23dc0 0x55a90ea2b3e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c006010 tx=0x7f9b4c0bc060 comp rx=0 tx=0).stop 2024-09-15T22:19:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b664cc740 1 -- 172.21.15.5:0/236274244 shutdown_connections 2024-09-15T22:19:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b664cc740 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a90ea3e290 0x55a90ea406b0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b664cc740 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 0x55a90ea2bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b457fa640 0 client.26767 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b664cc740 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea23dc0 0x55a90ea2b3e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b664cc740 1 --2- 172.21.15.5:0/236274244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea234a0 0x55a90ea2e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b664cc740 1 -- 172.21.15.5:0/236274244 >> 172.21.15.5:0/236274244 conn(0x55a90ea1fa30 msgr2=0x55a90ea25710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b664cc740 1 -- 172.21.15.5:0/236274244 shutdown_connections 2024-09-15T22:19:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b664cc740 1 -- 172.21.15.5:0/236274244 wait complete. 2024-09-15T22:19:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.760+0000 7f9b664cc740 20 client.26767 trim_cache size 0 max 16384 2024-09-15T22:19:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.794+0000 7f9b664cc740 1 Processor -- start 2024-09-15T22:19:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.794+0000 7f9b664cc740 1 -- start start 2024-09-15T22:19:52.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.794+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea234a0 0x55a90ea26760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.794+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea23dc0 0x55a90ea45f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.794+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea46480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.795+0000 7f9b664cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b400127e0 con 0x55a90ea2e680 2024-09-15T22:19:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.795+0000 7f9b664cc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b400100b0 con 0x55a90ea23dc0 2024-09-15T22:19:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.795+0000 7f9b664cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4000dfb0 con 0x55a90ea234a0 2024-09-15T22:19:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.795+0000 7f9b62690640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea46480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.795+0000 7f9b62690640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea46480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48916/0 (socket says 172.21.15.5:48916) 2024-09-15T22:19:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.795+0000 7f9b62690640 1 -- 172.21.15.5:0/947338162 learned_addr learned my addr 172.21.15.5:0/947338162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.796+0000 7f9b62690640 1 -- 172.21.15.5:0/947338162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea234a0 msgr2=0x55a90ea26760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.796+0000 7f9b62690640 1 --2- 172.21.15.5:0/947338162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea234a0 0x55a90ea26760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.796+0000 7f9b62690640 1 -- 172.21.15.5:0/947338162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea23dc0 msgr2=0x55a90ea45f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.796+0000 7f9b62690640 1 --2- 172.21.15.5:0/947338162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea23dc0 0x55a90ea45f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.796+0000 7f9b62690640 1 -- 172.21.15.5:0/947338162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90ea3df90 con 0x55a90ea2e680 2024-09-15T22:19:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.797+0000 7f9b62690640 1 --2- 172.21.15.5:0/947338162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea46480 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009f70 tx=0x7f9b54003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca24d3ac46f9938e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.797+0000 7f9b477fe640 1 -- 172.21.15.5:0/947338162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54027070 con 0x55a90ea2e680 2024-09-15T22:19:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.797+0000 7f9b477fe640 1 -- 172.21.15.5:0/947338162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b54005720 con 0x55a90ea2e680 2024-09-15T22:19:52.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.798+0000 7f9b477fe640 1 -- 172.21.15.5:0/947338162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400fc90 con 0x55a90ea2e680 2024-09-15T22:19:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.801+0000 7f9b664cc740 1 -- 172.21.15.5:0/947338162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 msgr2=0x55a90ea46480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:52.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.801+0000 7f9b664cc740 1 --2- 172.21.15.5:0/947338162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea46480 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009f70 tx=0x7f9b54003ae0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.805+0000 7f9b664cc740 1 -- 172.21.15.5:0/947338162 shutdown_connections 2024-09-15T22:19:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.805+0000 7f9b664cc740 1 --2- 172.21.15.5:0/947338162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea2e680 0x55a90ea46480 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.805+0000 7f9b664cc740 1 --2- 172.21.15.5:0/947338162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea23dc0 0x55a90ea45f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.805+0000 7f9b664cc740 1 --2- 172.21.15.5:0/947338162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea234a0 0x55a90ea26760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.805+0000 7f9b664cc740 1 -- 172.21.15.5:0/947338162 >> 172.21.15.5:0/947338162 conn(0x55a90ea1fa30 msgr2=0x55a90ea26050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:52.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.807+0000 7f9b664cc740 1 -- 172.21.15.5:0/947338162 shutdown_connections 2024-09-15T22:19:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.809+0000 7f9b664cc740 1 -- 172.21.15.5:0/947338162 wait complete. 2024-09-15T22:19:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.810+0000 7f9b664cc740 1 -- start start 2024-09-15T22:19:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.810+0000 7f9b664cc740 1 Processor -- start 2024-09-15T22:19:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.811+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea234a0 0x55a90ea366b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.811+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea23dc0 0x55a90ea36bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.811+0000 7f9b664cc740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 0x55a90ea33680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.811+0000 7f9b664cc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4000e570 con 0x55a90ea23dc0 2024-09-15T22:19:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.811+0000 7f9b664cc740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4000f9e0 con 0x55a90ea2e680 2024-09-15T22:19:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.812+0000 7f9b664cc740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4000ec40 con 0x55a90ea234a0 2024-09-15T22:19:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.812+0000 7f9b664cc740 10 client.0 _ll_register_callbacks cb 0x7f9b4c004ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.812+0000 7f9b62690640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 0x55a90ea33680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.812+0000 7f9b62690640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 0x55a90ea33680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:45890/0 (socket says 172.21.15.5:45890) 2024-09-15T22:19:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.812+0000 7f9b62690640 1 -- 172.21.15.5:0/2942171271 learned_addr learned my addr 172.21.15.5:0/2942171271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.814+0000 7f9b62690640 1 -- 172.21.15.5:0/2942171271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea234a0 msgr2=0x55a90ea366b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:19:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.814+0000 7f9b62690640 1 --2- 172.21.15.5:0/2942171271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea234a0 0x55a90ea366b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.814+0000 7f9b62690640 1 -- 172.21.15.5:0/2942171271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea23dc0 msgr2=0x55a90ea36bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.814+0000 7f9b62690640 1 --2- 172.21.15.5:0/2942171271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea23dc0 0x55a90ea36bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.814+0000 7f9b62690640 1 -- 172.21.15.5:0/2942171271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90ea33bc0 con 0x55a90ea2e680 2024-09-15T22:19:52.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.815+0000 7f9b62690640 1 --2- 172.21.15.5:0/2942171271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 0x55a90ea33680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005b00 tx=0x7f9b54005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c772fdccff20a155 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:52.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.815+0000 7f9b457fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:19:52.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.818+0000 7f9b457fa640 1 -- 172.21.15.5:0/2942171271 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b54027070 con 0x55a90ea2e680 2024-09-15T22:19:52.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.818+0000 7f9b457fa640 1 -- 172.21.15.5:0/2942171271 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9541+0+0 (secure 0 0 0) 0x7f9b540052c0 con 0x55a90ea2e680 2024-09-15T22:19:52.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.818+0000 7f9b664cc740 1 -- 172.21.15.5:0/2942171271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b54005720 con 0x55a90ea2e680 2024-09-15T22:19:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.819+0000 7f9b457fa640 1 -- 172.21.15.5:0/2942171271 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b5403d050 con 0x55a90ea2e680 2024-09-15T22:19:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.821+0000 7f9b664cc740 10 client.26816 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.821+0000 7f9b664cc740 1 -- 172.21.15.5:0/2942171271 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a90eaa9670 con 0x55a90ea2e680 2024-09-15T22:19:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.824+0000 7f9b457fa640 1 -- 172.21.15.5:0/2942171271 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b54005720 con 0x55a90ea2e680 2024-09-15T22:19:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.824+0000 7f9b457fa640 1 -- 172.21.15.5:0/2942171271 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9b54023430 con 0x55a90ea2e680 2024-09-15T22:19:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.824+0000 7f9b664cc740 10 client.26816 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.824+0000 7f9b664cc740 10 client.26816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:19:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.824+0000 7f9b664cc740 20 client.26816 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.824+0000 7f9b664cc740 1 --2- 172.21.15.5:0/2942171271 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a90ea3e290 0x55a90eaabcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.824+0000 7f9b664cc740 4 client.26816 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:19:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.824+0000 7f9b664cc740 1 -- 172.21.15.5:0/2942171271 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a90e996850 con 0x55a90ea3e290 2024-09-15T22:19:52.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.825+0000 7f9b61e8f640 1 --2- 172.21.15.5:0/2942171271 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a90ea3e290 0x55a90eaabcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.827+0000 7f9b61e8f640 1 --2- 172.21.15.5:0/2942171271 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a90ea3e290 0x55a90eaabcd0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1b87b1a26549697 server_cookie=7be0aa2d460840f7 in_seq=0 out_seq=0 2024-09-15T22:19:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.828+0000 7f9b457fa640 10 client.26816 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.828+0000 7f9b457fa640 1 -- 172.21.15.5:0/2942171271 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a90e996850 con 0x55a90ea3e290 2024-09-15T22:19:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.828+0000 7f9b457fa640 10 client.26816 handle_command_reply: tid=0 2024-09-15T22:19:52.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.828+0000 7f9b664cc740 1 client.26816 shutdown 2024-09-15T22:19:52.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.829+0000 7f9b664cc740 1 -- 172.21.15.5:0/2942171271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 msgr2=0x55a90ea33680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:52.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.829+0000 7f9b664cc740 1 --2- 172.21.15.5:0/2942171271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 0x55a90ea33680 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9b54005b00 tx=0x7f9b54005c70 comp rx=0 tx=0).stop 2024-09-15T22:19:52.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.829+0000 7f9b664cc740 1 -- 172.21.15.5:0/2942171271 shutdown_connections 2024-09-15T22:19:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.829+0000 7f9b664cc740 1 --2- 172.21.15.5:0/2942171271 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a90ea3e290 0x55a90eaabcd0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.829+0000 7f9b664cc740 1 --2- 172.21.15.5:0/2942171271 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a90ea2e680 0x55a90ea33680 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.829+0000 7f9b664cc740 1 --2- 172.21.15.5:0/2942171271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a90ea23dc0 0x55a90ea36bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.829+0000 7f9b664cc740 1 --2- 172.21.15.5:0/2942171271 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a90ea234a0 0x55a90ea366b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.829+0000 7f9b664cc740 1 -- 172.21.15.5:0/2942171271 >> 172.21.15.5:0/2942171271 conn(0x55a90ea1fa30 msgr2=0x55a90ea25710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:52.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.829+0000 7f9b457fa640 0 client.26816 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:19:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.835+0000 7f9b664cc740 1 -- 172.21.15.5:0/2942171271 shutdown_connections 2024-09-15T22:19:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.835+0000 7f9b664cc740 1 -- 172.21.15.5:0/2942171271 wait complete. 2024-09-15T22:19:52.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.835+0000 7f9b664cc740 20 client.26816 trim_cache size 0 max 16384 2024-09-15T22:19:52.837 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:19:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:19:52.837 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:19:52.837 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:19:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.839+0000 7f9b640f3640 1 -- 172.21.15.5:0/1301905090 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b480a35c0 msgr2=0x7f9b480a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.839+0000 7f9b640f3640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b480a35c0 0x7f9b480a5a80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009a30 tx=0x7f9b58009190 comp rx=0 tx=0).stop 2024-09-15T22:19:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.839+0000 7f9b640f3640 1 -- 172.21.15.5:0/1301905090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c1027a0 msgr2=0x7f9b5c1a7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.839+0000 7f9b640f3640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c1027a0 0x7f9b5c1a7890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c013150 tx=0x7f9b4c0042f0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.839+0000 7f9b640f3640 1 -- 172.21.15.5:0/1301905090 shutdown_connections 2024-09-15T22:19:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.839+0000 7f9b640f3640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9b480a35c0 0x7f9b480a5a80 secure :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009a30 tx=0x7f9b58009190 comp rx=0 tx=0).stop 2024-09-15T22:19:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.839+0000 7f9b640f3640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b5c106f20 0x7f9b5c1abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.839+0000 7f9b640f3640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9b5c1027a0 0x7f9b5c1a7890 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.839+0000 7f9b640f3640 1 --2- 172.21.15.5:0/1301905090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9b5c101df0 0x7f9b5c1a7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:52.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.839+0000 7f9b640f3640 1 -- 172.21.15.5:0/1301905090 >> 172.21.15.5:0/1301905090 conn(0x7f9b5c0fd8f0 msgr2=0x7f9b5c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.840+0000 7f9b640f3640 1 -- 172.21.15.5:0/1301905090 shutdown_connections 2024-09-15T22:19:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:52.840+0000 7f9b640f3640 1 -- 172.21.15.5:0/1301905090 wait complete. 2024-09-15T22:19:53.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:53 smithi099 ceph-mon[34892]: pgmap v340: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 683 op/s 2024-09-15T22:19:53.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:53 smithi129 ceph-mon[35314]: pgmap v340: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 683 op/s 2024-09-15T22:19:53.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:53 smithi005 ceph-mon[30944]: pgmap v340: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 11 MiB/s wr, 683 op/s 2024-09-15T22:19:54.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 625.0 s, 642.6 tps, lat 154.074 ms stddev 167.528 2024-09-15T22:19:54.812 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:19:54.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:19:55.261 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:55.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:55 smithi099 ceph-mon[34892]: pgmap v341: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 486 op/s 2024-09-15T22:19:55.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:55 smithi129 ceph-mon[35314]: pgmap v341: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 486 op/s 2024-09-15T22:19:55.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:55 smithi005 ceph-mon[30944]: pgmap v341: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 7.7 MiB/s wr, 486 op/s 2024-09-15T22:19:55.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.926+0000 7fa47ba84640 1 -- 172.21.15.5:0/524385615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa474109450 msgr2=0x7fa474110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:55.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.926+0000 7fa47ba84640 1 --2- 172.21.15.5:0/524385615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa474109450 0x7fa474110250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa470009860 tx=0x7fa47002a380 comp rx=0 tx=0).stop 2024-09-15T22:19:55.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.927+0000 7fa47ba84640 1 -- 172.21.15.5:0/524385615 shutdown_connections 2024-09-15T22:19:55.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.927+0000 7fa47ba84640 1 --2- 172.21.15.5:0/524385615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa47411b140 0x7fa47411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:55.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.927+0000 7fa47ba84640 1 --2- 172.21.15.5:0/524385615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa474109830 0x7fa474109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:55.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.927+0000 7fa47ba84640 1 --2- 172.21.15.5:0/524385615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa474109450 0x7fa474110250 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:55.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.927+0000 7fa47ba84640 1 -- 172.21.15.5:0/524385615 >> 172.21.15.5:0/524385615 conn(0x7fa47406e140 msgr2=0x7fa47406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:55.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.927+0000 7fa47ba84640 1 -- 172.21.15.5:0/524385615 shutdown_connections 2024-09-15T22:19:55.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.927+0000 7fa47ba84640 1 -- 172.21.15.5:0/524385615 wait complete. 2024-09-15T22:19:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.928+0000 7fa47ba84640 1 Processor -- start 2024-09-15T22:19:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.928+0000 7fa47ba84640 1 -- start start 2024-09-15T22:19:55.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.928+0000 7fa47ba84640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa474109450 0x7fa474118730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.928+0000 7fa47ba84640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa474109830 0x7fa4741117c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.928+0000 7fa47ba84640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47411b140 0x7fa474111d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.928+0000 7fa47ba84640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa474120df0 con 0x7fa47411b140 2024-09-15T22:19:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.928+0000 7fa47ba84640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa474120c70 con 0x7fa474109830 2024-09-15T22:19:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.928+0000 7fa47ba84640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa474120f70 con 0x7fa474109450 2024-09-15T22:19:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.929+0000 7fa47901f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa474109830 0x7fa4741117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.929+0000 7fa47901f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa474109830 0x7fa4741117c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:56752/0 (socket says 172.21.15.5:56752) 2024-09-15T22:19:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.929+0000 7fa47901f640 1 -- 172.21.15.5:0/140241431 learned_addr learned my addr 172.21.15.5:0/140241431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.929+0000 7fa479820640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa474109450 0x7fa474118730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:55.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.930+0000 7fa47a021640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47411b140 0x7fa474111d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:55.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.933+0000 7fa47a021640 1 -- 172.21.15.5:0/140241431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa474109450 msgr2=0x7fa474118730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:55.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.933+0000 7fa47a021640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa474109450 0x7fa474118730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:55.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.933+0000 7fa47a021640 1 -- 172.21.15.5:0/140241431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa474109830 msgr2=0x7fa4741117c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:55.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.933+0000 7fa47a021640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa474109830 0x7fa4741117c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:55.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.933+0000 7fa47a021640 1 -- 172.21.15.5:0/140241431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4741122f0 con 0x7fa47411b140 2024-09-15T22:19:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.939+0000 7fa47a021640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47411b140 0x7fa474111d00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00eeb0 tx=0x7fa46c00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:55.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.942+0000 7fa46affd640 1 -- 172.21.15.5:0/140241431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c004b20 con 0x7fa47411b140 2024-09-15T22:19:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.942+0000 7fa47ba84640 1 -- 172.21.15.5:0/140241431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4741125e0 con 0x7fa47411b140 2024-09-15T22:19:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.942+0000 7fa47ba84640 1 -- 172.21.15.5:0/140241431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4741165d0 con 0x7fa47411b140 2024-09-15T22:19:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.942+0000 7fa46affd640 1 -- 172.21.15.5:0/140241431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa46c003e50 con 0x7fa47411b140 2024-09-15T22:19:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.942+0000 7fa46affd640 1 -- 172.21.15.5:0/140241431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c00b570 con 0x7fa47411b140 2024-09-15T22:19:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.948+0000 7fa46affd640 1 -- 172.21.15.5:0/140241431 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa46c01d020 con 0x7fa47411b140 2024-09-15T22:19:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.950+0000 7fa46affd640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa4600a35c0 0x7fa4600a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.950+0000 7fa46affd640 1 -- 172.21.15.5:0/140241431 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9541+0+0 (secure 0 0 0) 0x7fa46c0cfe20 con 0x7fa47411b140 2024-09-15T22:19:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.951+0000 7fa479820640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa4600a35c0 0x7fa4600a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:55.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.952+0000 7fa479820640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa4600a35c0 0x7fa4600a5a80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa470000c00 tx=0x7fa4700304b0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:55.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.953+0000 7fa47ba84640 1 -- 172.21.15.5:0/140241431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa458005290 con 0x7fa47411b140 2024-09-15T22:19:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:55.962+0000 7fa46affd640 1 -- 172.21.15.5:0/140241431 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa46c09c3a0 con 0x7fa47411b140 2024-09-15T22:19:56.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.301+0000 7fa47ba84640 1 -- 172.21.15.5:0/140241431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa458005850 con 0x7fa47411b140 2024-09-15T22:19:56.306 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:19:56.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.303+0000 7fa46affd640 1 -- 172.21.15.5:0/140241431 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fa46c09bca0 con 0x7fa47411b140 2024-09-15T22:19:56.306 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:19:56.307 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:19:56.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.308+0000 7fa47ba84640 1 -- 172.21.15.5:0/140241431 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa4600a35c0 msgr2=0x7fa4600a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.308+0000 7fa47ba84640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa4600a35c0 0x7fa4600a5a80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa470000c00 tx=0x7fa4700304b0 comp rx=0 tx=0).stop 2024-09-15T22:19:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.308+0000 7fa47ba84640 1 -- 172.21.15.5:0/140241431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47411b140 msgr2=0x7fa474111d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.308+0000 7fa47ba84640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47411b140 0x7fa474111d00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00eeb0 tx=0x7fa46c00c740 comp rx=0 tx=0).stop 2024-09-15T22:19:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.308+0000 7fa47ba84640 1 -- 172.21.15.5:0/140241431 shutdown_connections 2024-09-15T22:19:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.308+0000 7fa47ba84640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa4600a35c0 0x7fa4600a5a80 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa470000c00 tx=0x7fa4700304b0 comp rx=0 tx=0).stop 2024-09-15T22:19:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.308+0000 7fa47ba84640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa47411b140 0x7fa474111d00 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.308+0000 7fa47ba84640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa474109830 0x7fa4741117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.308+0000 7fa47ba84640 1 --2- 172.21.15.5:0/140241431 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa474109450 0x7fa474118730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:56.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.308+0000 7fa47ba84640 1 -- 172.21.15.5:0/140241431 >> 172.21.15.5:0/140241431 conn(0x7fa47406e140 msgr2=0x7fa47411be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:56.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.316+0000 7fa47ba84640 1 -- 172.21.15.5:0/140241431 shutdown_connections 2024-09-15T22:19:56.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:56.316+0000 7fa47ba84640 1 -- 172.21.15.5:0/140241431 wait complete. 2024-09-15T22:19:56.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:56 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/140241431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:19:56.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:56 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/140241431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:19:56.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:56 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/140241431' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:19:57.165 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:19:57.403 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:19:57.403 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.85 2024-09-15T22:19:57.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:19:57.745 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:57 smithi129 ceph-mon[35314]: pgmap v342: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.7 MiB/s wr, 605 op/s 2024-09-15T22:19:57.770 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:57 smithi099 ceph-mon[34892]: pgmap v342: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.7 MiB/s wr, 605 op/s 2024-09-15T22:19:57.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:57 smithi005 ceph-mon[30944]: pgmap v342: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 8.7 MiB/s wr, 605 op/s 2024-09-15T22:19:59.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:19:59 smithi005 ceph-mon[30944]: pgmap v343: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 525 op/s 2024-09-15T22:19:59.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.546+0000 7f7129c93640 1 -- 172.21.15.5:0/1173756148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71241012f0 msgr2=0x7f712410dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:59.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.546+0000 7f7129c93640 1 --2- 172.21.15.5:0/1173756148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71241012f0 0x7f712410dfb0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7118009900 tx=0x7f711802a410 comp rx=0 tx=0).stop 2024-09-15T22:19:59.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.549+0000 7f7129c93640 1 -- 172.21.15.5:0/1173756148 shutdown_connections 2024-09-15T22:19:59.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.549+0000 7f7129c93640 1 --2- 172.21.15.5:0/1173756148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f712410e4f0 0x7f71241108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.549+0000 7f7129c93640 1 --2- 172.21.15.5:0/1173756148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71241012f0 0x7f712410dfb0 secure :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7118009900 tx=0x7f711802a410 comp rx=0 tx=0).stop 2024-09-15T22:19:59.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.549+0000 7f7129c93640 1 --2- 172.21.15.5:0/1173756148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7124100940 0x7f7124100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.549+0000 7f7129c93640 1 -- 172.21.15.5:0/1173756148 >> 172.21.15.5:0/1173756148 conn(0x7f71240fc7d0 msgr2=0x7f71240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:59.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.551+0000 7f7129c93640 1 -- 172.21.15.5:0/1173756148 shutdown_connections 2024-09-15T22:19:59.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.551+0000 7f7129c93640 1 -- 172.21.15.5:0/1173756148 wait complete. 2024-09-15T22:19:59.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.552+0000 7f7129c93640 1 Processor -- start 2024-09-15T22:19:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f7129c93640 1 -- start start 2024-09-15T22:19:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f7129c93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124100940 0x7f71241a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f7129c93640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f712410e4f0 0x7f71241a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f7129c93640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f712419a5e0 0x7f712419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f71237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124100940 0x7f71241a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f71237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124100940 0x7f71241a0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56016/0 (socket says 172.21.15.5:56016) 2024-09-15T22:19:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f71237fe640 1 -- 172.21.15.5:0/943894548 learned_addr learned my addr 172.21.15.5:0/943894548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f7122ffd640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f712410e4f0 0x7f71241a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f7129c93640 1 -- 172.21.15.5:0/943894548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7124114190 con 0x7f7124100940 2024-09-15T22:19:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f7129c93640 1 -- 172.21.15.5:0/943894548 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7124114010 con 0x7f712419a5e0 2024-09-15T22:19:59.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f7129c93640 1 -- 172.21.15.5:0/943894548 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7124114310 con 0x7f712410e4f0 2024-09-15T22:19:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.553+0000 7f7123fff640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f712419a5e0 0x7f712419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.554+0000 7f7122ffd640 1 -- 172.21.15.5:0/943894548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f712419a5e0 msgr2=0x7f712419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.554+0000 7f7122ffd640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f712419a5e0 0x7f712419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.554+0000 7f7122ffd640 1 -- 172.21.15.5:0/943894548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124100940 msgr2=0x7f71241a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:59.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.554+0000 7f7122ffd640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124100940 0x7f71241a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.554+0000 7f7122ffd640 1 -- 172.21.15.5:0/943894548 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712419b100 con 0x7f712410e4f0 2024-09-15T22:19:59.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.554+0000 7f7123fff640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f712419a5e0 0x7f712419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:59.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.554+0000 7f7122ffd640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f712410e4f0 0x7f71241a0960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7118008000 tx=0x7f7118004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:59.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.555+0000 7f7120ff9640 1 -- 172.21.15.5:0/943894548 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711803f030 con 0x7f712410e4f0 2024-09-15T22:19:59.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.555+0000 7f7129c93640 1 -- 172.21.15.5:0/943894548 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712419b390 con 0x7f712410e4f0 2024-09-15T22:19:59.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.555+0000 7f7120ff9640 1 -- 172.21.15.5:0/943894548 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7118003ad0 con 0x7f712410e4f0 2024-09-15T22:19:59.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.555+0000 7f7120ff9640 1 -- 172.21.15.5:0/943894548 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118049060 con 0x7f712410e4f0 2024-09-15T22:19:59.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.555+0000 7f7129c93640 1 -- 172.21.15.5:0/943894548 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71241a74c0 con 0x7f712410e4f0 2024-09-15T22:19:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.557+0000 7f7120ff9640 1 -- 172.21.15.5:0/943894548 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7118042030 con 0x7f712410e4f0 2024-09-15T22:19:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.558+0000 7f7120ff9640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f710c0a3460 0x7f710c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.558+0000 7f7120ff9640 1 -- 172.21.15.5:0/943894548 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9541+0+0 (secure 0 0 0) 0x7f71180effe0 con 0x7f712410e4f0 2024-09-15T22:19:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.558+0000 7f71237fe640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f710c0a3460 0x7f710c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.559+0000 7f712c06c740 1 Processor -- start 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.559+0000 7f712c06c740 1 -- start start 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.559+0000 7f71237fe640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f710c0a3460 0x7f710c0a5920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f711400ac90 tx=0x7f711400a380 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f712c06c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c897e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f712c06c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c897ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f712c06c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8993f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f712c06c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e0c88e42f0 con 0x55e0c897e5d0 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f712c06c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e0c888bcf0 con 0x55e0c897fb40 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f712c06c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e0c88e02b0 con 0x55e0c8991b80 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f7122ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c897ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f7122ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c897ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56024/0 (socket says 172.21.15.5:56024) 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f7122ffd640 1 -- 172.21.15.5:0/408793135 learned_addr learned my addr 172.21.15.5:0/408793135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f7123fff640 1 --2- 172.21.15.5:0/408793135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8993f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f71237fe640 1 --2- 172.21.15.5:0/408793135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c897e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f7123fff640 1 -- 172.21.15.5:0/408793135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 msgr2=0x55e0c897e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.560+0000 7f7123fff640 1 --2- 172.21.15.5:0/408793135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c897e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.561+0000 7f7123fff640 1 -- 172.21.15.5:0/408793135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 msgr2=0x55e0c897ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.561+0000 7f7123fff640 1 --2- 172.21.15.5:0/408793135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c897ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.561+0000 7f7123fff640 1 -- 172.21.15.5:0/408793135 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c88703a0 con 0x55e0c8991b80 2024-09-15T22:19:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.561+0000 7f7122ffd640 1 --2- 172.21.15.5:0/408793135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c897ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:19:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.561+0000 7f7123fff640 1 --2- 172.21.15.5:0/408793135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8993f70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7124102b20 tx=0x7f711c011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=988d398d5d48736d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.561+0000 7f7109ffb640 1 -- 172.21.15.5:0/408793135 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c004d20 con 0x55e0c8991b80 2024-09-15T22:19:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.562+0000 7f7109ffb640 1 -- 172.21.15.5:0/408793135 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f711c00f970 con 0x55e0c8991b80 2024-09-15T22:19:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.562+0000 7f7109ffb640 1 -- 172.21.15.5:0/408793135 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c0045b0 con 0x55e0c8991b80 2024-09-15T22:19:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.562+0000 7f712c06c740 1 -- 172.21.15.5:0/408793135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 msgr2=0x55e0c8993f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.562+0000 7f712c06c740 1 --2- 172.21.15.5:0/408793135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8993f70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7124102b20 tx=0x7f711c011bf0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.563+0000 7f712c06c740 1 -- 172.21.15.5:0/408793135 shutdown_connections 2024-09-15T22:19:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.563+0000 7f712c06c740 1 --2- 172.21.15.5:0/408793135 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8993f70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.563+0000 7f712c06c740 1 --2- 172.21.15.5:0/408793135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c897ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.563+0000 7f712c06c740 1 --2- 172.21.15.5:0/408793135 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c897e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.563+0000 7f712c06c740 1 -- 172.21.15.5:0/408793135 >> 172.21.15.5:0/408793135 conn(0x55e0c8947310 msgr2=0x55e0c897ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.563+0000 7f712c06c740 1 -- 172.21.15.5:0/408793135 shutdown_connections 2024-09-15T22:19:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.563+0000 7f712c06c740 1 -- 172.21.15.5:0/408793135 wait complete. 2024-09-15T22:19:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.564+0000 7f712c06c740 1 -- start start 2024-09-15T22:19:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.564+0000 7f712c06c740 1 Processor -- start 2024-09-15T22:19:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.564+0000 7f712c06c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c8986d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.565+0000 7f712c06c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c8983fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.565+0000 7f712c06c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8984550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.565+0000 7f71237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c8986d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.565+0000 7f71237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c8986d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56036/0 (socket says 172.21.15.5:56036) 2024-09-15T22:19:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.565+0000 7f71237fe640 1 -- 172.21.15.5:0/2086456610 learned_addr learned my addr 172.21.15.5:0/2086456610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.565+0000 7f7122ffd640 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c8983fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.565+0000 7f712c06c740 1 -- 172.21.15.5:0/2086456610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e0c8994880 con 0x55e0c897e5d0 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.565+0000 7f712c06c740 1 -- 172.21.15.5:0/2086456610 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e0c889f060 con 0x55e0c897fb40 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.565+0000 7f7123fff640 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8984550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.565+0000 7f712c06c740 1 -- 172.21.15.5:0/2086456610 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e0c8994a00 con 0x55e0c8991b80 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.565+0000 7f712c06c740 10 client.0 _ll_register_callbacks cb 0x55e0c85f28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.566+0000 7f7123fff640 1 -- 172.21.15.5:0/2086456610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 msgr2=0x55e0c8983fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.566+0000 7f7123fff640 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c8983fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.566+0000 7f7123fff640 1 -- 172.21.15.5:0/2086456610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 msgr2=0x55e0c8986d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.566+0000 7f7123fff640 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c8986d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.566+0000 7f7123fff640 1 -- 172.21.15.5:0/2086456610 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c86553c0 con 0x55e0c8991b80 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.566+0000 7f712c06c740 1 -- 172.21.15.5:0/2086456610 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f711c00f970 con 0x55e0c8991b80 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.566+0000 7f7123fff640 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8984550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f711c003fd0 tx=0x7f711c003c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=827833d139470a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.566+0000 7f7102ffd640 10 client.26785 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:19:59.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.566+0000 7f7102ffd640 1 -- 172.21.15.5:0/2086456610 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c01e040 con 0x55e0c8991b80 2024-09-15T22:19:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.566+0000 7f7102ffd640 1 -- 172.21.15.5:0/2086456610 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9541+0+0 (secure 0 0 0) 0x7f711c011050 con 0x55e0c8991b80 2024-09-15T22:19:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.567+0000 7f7102ffd640 1 -- 172.21.15.5:0/2086456610 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f711c011250 con 0x55e0c8991b80 2024-09-15T22:19:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.567+0000 7f712c06c740 10 client.26785 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:59.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.567+0000 7f712c06c740 1 -- 172.21.15.5:0/2086456610 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55e0c89a0150 con 0x55e0c8991b80 2024-09-15T22:19:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.569+0000 7f7102ffd640 1 -- 172.21.15.5:0/2086456610 <== mon.1 v2:172.21.15.129:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f711c018070 con 0x55e0c8991b80 2024-09-15T22:19:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.569+0000 7f712c06c740 10 client.26785 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.569+0000 7f712c06c740 10 client.26785 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:19:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.569+0000 7f712c06c740 20 client.26785 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.569+0000 7f712c06c740 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e0c89a14a0 0x55e0c89a3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.569+0000 7f712c06c740 4 client.26785 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:19:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.569+0000 7f712c06c740 1 -- 172.21.15.5:0/2086456610 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0c88f1860 con 0x55e0c89a14a0 2024-09-15T22:19:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.569+0000 7f71237fe640 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e0c89a14a0 0x55e0c89a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.570+0000 7f71237fe640 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e0c89a14a0 0x55e0c89a3890 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=658fad4ce137ee08 server_cookie=612e4059c24e124c in_seq=0 out_seq=0 2024-09-15T22:19:59.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.570+0000 7f7102ffd640 10 client.26785 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:19:59.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.570+0000 7f7102ffd640 1 -- 172.21.15.5:0/2086456610 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0c88f1860 con 0x55e0c89a14a0 2024-09-15T22:19:59.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.570+0000 7f7102ffd640 10 client.26785 handle_command_reply: tid=0 2024-09-15T22:19:59.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.570+0000 7f712c06c740 1 client.26785 shutdown 2024-09-15T22:19:59.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.571+0000 7f712c06c740 1 -- 172.21.15.5:0/2086456610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 msgr2=0x55e0c8984550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:59.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.571+0000 7f712c06c740 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8984550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f711c003fd0 tx=0x7f711c003c60 comp rx=0 tx=0).stop 2024-09-15T22:19:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.571+0000 7f712c06c740 1 -- 172.21.15.5:0/2086456610 shutdown_connections 2024-09-15T22:19:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.571+0000 7f712c06c740 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e0c89a14a0 0x55e0c89a3890 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.571+0000 7f712c06c740 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8984550 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.571+0000 7f712c06c740 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c8983fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.571+0000 7f712c06c740 1 --2- 172.21.15.5:0/2086456610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c8986d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.571+0000 7f712c06c740 1 -- 172.21.15.5:0/2086456610 >> 172.21.15.5:0/2086456610 conn(0x55e0c8947310 msgr2=0x55e0c8980930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.571+0000 7f7102ffd640 0 client.26785 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:19:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.571+0000 7f712c06c740 1 -- 172.21.15.5:0/2086456610 shutdown_connections 2024-09-15T22:19:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.571+0000 7f712c06c740 1 -- 172.21.15.5:0/2086456610 wait complete. 2024-09-15T22:19:59.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.572+0000 7f712c06c740 20 client.26785 trim_cache size 0 max 16384 2024-09-15T22:19:59.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.596+0000 7f712c06c740 1 Processor -- start 2024-09-15T22:19:59.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.596+0000 7f712c06c740 1 -- start start 2024-09-15T22:19:59.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.597+0000 7f712c06c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c89862e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.597+0000 7f712c06c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c897fb40 0x55e0c899db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.598+0000 7f712c06c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c8991b80 0x55e0c89a9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.598+0000 7f712c06c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71241afce0 con 0x55e0c897e5d0 2024-09-15T22:19:59.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.598+0000 7f7123fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c8991b80 0x55e0c89a9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.598+0000 7f712c06c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71241ad5b0 con 0x55e0c8991b80 2024-09-15T22:19:59.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.598+0000 7f712c06c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71241ab420 con 0x55e0c897fb40 2024-09-15T22:19:59.604 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:19:59.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.602+0000 7f7122ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c897fb40 0x55e0c899db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.602+0000 7f7122ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c897fb40 0x55e0c899db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:49752/0 (socket says 172.21.15.5:49752) 2024-09-15T22:19:59.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.602+0000 7f7122ffd640 1 -- 172.21.15.5:0/1671152074 learned_addr learned my addr 172.21.15.5:0/1671152074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:59.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.602+0000 7f71237fe640 1 --2- 172.21.15.5:0/1671152074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c89862e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.606+0000 7f7122ffd640 1 -- 172.21.15.5:0/1671152074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c8991b80 msgr2=0x55e0c89a9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:59.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.606+0000 7f7122ffd640 1 --2- 172.21.15.5:0/1671152074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c8991b80 0x55e0c89a9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.606+0000 7f7122ffd640 1 -- 172.21.15.5:0/1671152074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 msgr2=0x55e0c89862e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:59.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.606+0000 7f7122ffd640 1 --2- 172.21.15.5:0/1671152074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c89862e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.606+0000 7f7122ffd640 1 -- 172.21.15.5:0/1671152074 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c897b960 con 0x55e0c897fb40 2024-09-15T22:19:59.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.608+0000 7f7122ffd640 1 --2- 172.21.15.5:0/1671152074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c897fb40 0x55e0c899db20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7118000c00 tx=0x7f71180dc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5024db881a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:59.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.613+0000 7f71097fa640 1 -- 172.21.15.5:0/1671152074 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118035040 con 0x55e0c897fb40 2024-09-15T22:19:59.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.613+0000 7f71097fa640 1 -- 172.21.15.5:0/1671152074 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71180c0210 con 0x55e0c897fb40 2024-09-15T22:19:59.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.613+0000 7f71097fa640 1 -- 172.21.15.5:0/1671152074 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118000e60 con 0x55e0c897fb40 2024-09-15T22:19:59.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.614+0000 7f712c06c740 1 -- 172.21.15.5:0/1671152074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c897fb40 msgr2=0x55e0c899db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:59.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.614+0000 7f712c06c740 1 --2- 172.21.15.5:0/1671152074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c897fb40 0x55e0c899db20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7118000c00 tx=0x7f71180dc040 comp rx=0 tx=0).stop 2024-09-15T22:19:59.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.621+0000 7f712c06c740 1 -- 172.21.15.5:0/1671152074 shutdown_connections 2024-09-15T22:19:59.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.621+0000 7f712c06c740 1 --2- 172.21.15.5:0/1671152074 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c8991b80 0x55e0c89a9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.621+0000 7f712c06c740 1 --2- 172.21.15.5:0/1671152074 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c897fb40 0x55e0c899db20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.621+0000 7f712c06c740 1 --2- 172.21.15.5:0/1671152074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c89862e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.621+0000 7f712c06c740 1 -- 172.21.15.5:0/1671152074 >> 172.21.15.5:0/1671152074 conn(0x55e0c8947310 msgr2=0x55e0c8993bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.626+0000 7f712c06c740 1 -- 172.21.15.5:0/1671152074 shutdown_connections 2024-09-15T22:19:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.626+0000 7f712c06c740 1 -- 172.21.15.5:0/1671152074 wait complete. 2024-09-15T22:19:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.626+0000 7f712c06c740 1 -- start start 2024-09-15T22:19:59.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.627+0000 7f712c06c740 1 Processor -- start 2024-09-15T22:19:59.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.631+0000 7f712c06c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c899a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.631+0000 7f712c06c740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c89978b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.631+0000 7f712c06c740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8998040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.631+0000 7f712c06c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71241aba70 con 0x55e0c897e5d0 2024-09-15T22:19:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.631+0000 7f712c06c740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f71241acee0 con 0x55e0c897fb40 2024-09-15T22:19:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.631+0000 7f712c06c740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f71241ac140 con 0x55e0c8991b80 2024-09-15T22:19:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.631+0000 7f712c06c740 10 client.0 _ll_register_callbacks cb 0x7f7118005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:19:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.635+0000 7f7123fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8998040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.635+0000 7f7123fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8998040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:49758/0 (socket says 172.21.15.5:49758) 2024-09-15T22:19:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.635+0000 7f7123fff640 1 -- 172.21.15.5:0/4246838275 learned_addr learned my addr 172.21.15.5:0/4246838275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:19:59.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.635+0000 7f7122ffd640 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c89978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.636+0000 7f71237fe640 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c899a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.638+0000 7f7123fff640 1 -- 172.21.15.5:0/4246838275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 msgr2=0x55e0c89978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:59.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.638+0000 7f7123fff640 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c89978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.638+0000 7f7123fff640 1 -- 172.21.15.5:0/4246838275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 msgr2=0x55e0c899a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:19:59.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.638+0000 7f7123fff640 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c899a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.638+0000 7f7123fff640 1 -- 172.21.15.5:0/4246838275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c8987360 con 0x55e0c8991b80 2024-09-15T22:19:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.641+0000 7f7123fff640 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8998040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f711c0053a0 tx=0x7f711c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93dafb86cc052d3b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:19:59.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.641+0000 7f7102ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:19:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.643+0000 7f7102ffd640 1 -- 172.21.15.5:0/4246838275 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c03a060 con 0x55e0c8991b80 2024-09-15T22:19:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.643+0000 7f712c06c740 1 -- 172.21.15.5:0/4246838275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71180c0210 con 0x55e0c8991b80 2024-09-15T22:19:59.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.644+0000 7f7102ffd640 1 -- 172.21.15.5:0/4246838275 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9541+0+0 (secure 0 0 0) 0x7f711c019a00 con 0x55e0c8991b80 2024-09-15T22:19:59.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.647+0000 7f7102ffd640 1 -- 172.21.15.5:0/4246838275 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f711c041050 con 0x55e0c8991b80 2024-09-15T22:19:59.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.647+0000 7f712c06c740 10 client.26791 fetch_fsmap learned FSMap version 25 2024-09-15T22:19:59.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.647+0000 7f712c06c740 1 -- 172.21.15.5:0/4246838275 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0c8988640 con 0x55e0c8991b80 2024-09-15T22:19:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.652+0000 7f7102ffd640 1 -- 172.21.15.5:0/4246838275 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71180c0210 con 0x55e0c8991b80 2024-09-15T22:19:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.652+0000 7f7102ffd640 1 -- 172.21.15.5:0/4246838275 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f711c034040 con 0x55e0c8991b80 2024-09-15T22:19:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.652+0000 7f712c06c740 10 client.26791 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:19:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.652+0000 7f712c06c740 10 client.26791 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:19:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.652+0000 7f712c06c740 20 client.26791 populate_metadata read hostname 'smithi005' 2024-09-15T22:19:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.652+0000 7f712c06c740 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e0c8988db0 0x55e0c898b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:19:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.652+0000 7f712c06c740 4 client.26791 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:19:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.652+0000 7f712c06c740 1 -- 172.21.15.5:0/4246838275 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0c88ebd70 con 0x55e0c8988db0 2024-09-15T22:19:59.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.653+0000 7f71237fe640 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e0c8988db0 0x55e0c898b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:19:59.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.653+0000 7f7102ffd640 10 client.26791 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:19:59.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.654+0000 7f71237fe640 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e0c8988db0 0x55e0c898b1a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bbeb89ef59ae3bae server_cookie=7ea48198c9e08b60 in_seq=0 out_seq=0 2024-09-15T22:19:59.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.656+0000 7f7102ffd640 1 -- 172.21.15.5:0/4246838275 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0c88ebd70 con 0x55e0c8988db0 2024-09-15T22:19:59.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.656+0000 7f7102ffd640 10 client.26791 handle_command_reply: tid=0 2024-09-15T22:19:59.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.656+0000 7f712c06c740 1 client.26791 shutdown 2024-09-15T22:19:59.663 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 630.0 s, 515.5 tps, lat 191.184 ms stddev 272.441 2024-09-15T22:19:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.666+0000 7f712c06c740 1 -- 172.21.15.5:0/4246838275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 msgr2=0x55e0c8998040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.666+0000 7f712c06c740 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8998040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f711c0053a0 tx=0x7f711c009500 comp rx=0 tx=0).stop 2024-09-15T22:19:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.666+0000 7f712c06c740 1 -- 172.21.15.5:0/4246838275 shutdown_connections 2024-09-15T22:19:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.666+0000 7f712c06c740 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55e0c8988db0 0x55e0c898b1a0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.666+0000 7f712c06c740 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e0c8991b80 0x55e0c8998040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.666+0000 7f712c06c740 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e0c897fb40 0x55e0c89978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.666+0000 7f712c06c740 1 --2- 172.21.15.5:0/4246838275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0c897e5d0 0x55e0c899a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.666+0000 7f712c06c740 1 -- 172.21.15.5:0/4246838275 >> 172.21.15.5:0/4246838275 conn(0x55e0c8947310 msgr2=0x55e0c89794e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:59.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.666+0000 7f7102ffd640 0 client.26791 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:19:59.669 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:19:59.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.667+0000 7f712c06c740 1 -- 172.21.15.5:0/4246838275 shutdown_connections 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.667+0000 7f712c06c740 1 -- 172.21.15.5:0/4246838275 wait complete. 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.667+0000 7f712c06c740 20 client.26791 trim_cache size 0 max 16384 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.668+0000 7f7129c93640 1 -- 172.21.15.5:0/943894548 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f710c0a3460 msgr2=0x7f710c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.668+0000 7f7129c93640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f710c0a3460 0x7f710c0a5920 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f711400ac90 tx=0x7f711400a380 comp rx=0 tx=0).stop 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.668+0000 7f7129c93640 1 -- 172.21.15.5:0/943894548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f712410e4f0 msgr2=0x7f71241a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.668+0000 7f7129c93640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f712410e4f0 0x7f71241a0960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7118008000 tx=0x7f7118004270 comp rx=0 tx=0).stop 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.668+0000 7f7129c93640 1 -- 172.21.15.5:0/943894548 shutdown_connections 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.668+0000 7f7129c93640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f710c0a3460 0x7f710c0a5920 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.668+0000 7f7129c93640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f712419a5e0 0x7f712419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.668+0000 7f7129c93640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f712410e4f0 0x7f71241a0960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.668+0000 7f7129c93640 1 --2- 172.21.15.5:0/943894548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7124100940 0x7f71241a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.668+0000 7f7129c93640 1 -- 172.21.15.5:0/943894548 >> 172.21.15.5:0/943894548 conn(0x7f71240fc7d0 msgr2=0x7f71240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:19:59.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.668+0000 7f7129c93640 1 -- 172.21.15.5:0/943894548 shutdown_connections 2024-09-15T22:19:59.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:19:59.669+0000 7f7129c93640 1 -- 172.21.15.5:0/943894548 wait complete. 2024-09-15T22:19:59.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:19:59 smithi099 ceph-mon[34892]: pgmap v343: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 525 op/s 2024-09-15T22:19:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:19:59 smithi129 ceph-mon[35314]: pgmap v343: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.3 MiB/s wr, 525 op/s 2024-09-15T22:20:00.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:20:00.634 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:00 smithi005 ceph-mon[30944]: overall HEALTH_OK 2024-09-15T22:20:00.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:00 smithi099 ceph-mon[34892]: overall HEALTH_OK 2024-09-15T22:20:00.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:00 smithi129 ceph-mon[35314]: overall HEALTH_OK 2024-09-15T22:20:01.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.251+0000 7f4d78b98640 1 -- 172.21.15.5:0/3067855391 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70003610 con 0x7f4d741091f0 2024-09-15T22:20:01.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.252+0000 7f4d7bdfe640 1 -- 172.21.15.5:0/3067855391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d741091f0 msgr2=0x7f4d741095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:01.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.252+0000 7f4d7bdfe640 1 --2- 172.21.15.5:0/3067855391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d741091f0 0x7f4d741095d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4d70009900 tx=0x7f4d7002a440 comp rx=0 tx=0).stop 2024-09-15T22:20:01.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.252+0000 7f4d7bdfe640 1 -- 172.21.15.5:0/3067855391 shutdown_connections 2024-09-15T22:20:01.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.252+0000 7f4d7bdfe640 1 --2- 172.21.15.5:0/3067855391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d74116ea0 0x7f4d74119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.252+0000 7f4d7bdfe640 1 --2- 172.21.15.5:0/3067855391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d74109b10 0x7f4d74116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.252+0000 7f4d7bdfe640 1 --2- 172.21.15.5:0/3067855391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d741091f0 0x7f4d741095d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.252+0000 7f4d7bdfe640 1 -- 172.21.15.5:0/3067855391 >> 172.21.15.5:0/3067855391 conn(0x7f4d7406e140 msgr2=0x7f4d7406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.254+0000 7f4d7bdfe640 1 -- 172.21.15.5:0/3067855391 shutdown_connections 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.254+0000 7f4d7bdfe640 1 -- 172.21.15.5:0/3067855391 wait complete. 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.255+0000 7f4d7bdfe640 1 Processor -- start 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.255+0000 7f4d7bdfe640 1 -- start start 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.256+0000 7f4d7bdfe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d741091f0 0x7f4d7410c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.256+0000 7f4d7bdfe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d74109b10 0x7f4d7410cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.256+0000 7f4d7bdfe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d74116ea0 0x7f4d7410d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.256+0000 7f4d7bdfe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7411ca00 con 0x7f4d74109b10 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.256+0000 7f4d7bdfe640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7411c880 con 0x7f4d74116ea0 2024-09-15T22:20:01.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.256+0000 7f4d7bdfe640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4d7411cb80 con 0x7f4d741091f0 2024-09-15T22:20:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.256+0000 7f4d79b9a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d741091f0 0x7f4d7410c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.256+0000 7f4d79b9a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d741091f0 0x7f4d7410c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:49776/0 (socket says 172.21.15.5:49776) 2024-09-15T22:20:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.256+0000 7f4d79b9a640 1 -- 172.21.15.5:0/160287086 learned_addr learned my addr 172.21.15.5:0/160287086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.256+0000 7f4d79399640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d74109b10 0x7f4d7410cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.256+0000 7f4d7a39b640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d74116ea0 0x7f4d7410d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.257+0000 7f4d79b9a640 1 -- 172.21.15.5:0/160287086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d74116ea0 msgr2=0x7f4d7410d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.257+0000 7f4d79b9a640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d74116ea0 0x7f4d7410d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.257+0000 7f4d79b9a640 1 -- 172.21.15.5:0/160287086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d74109b10 msgr2=0x7f4d7410cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.257+0000 7f4d79b9a640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d74109b10 0x7f4d7410cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:01.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.257+0000 7f4d79b9a640 1 -- 172.21.15.5:0/160287086 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d741a2c60 con 0x7f4d741091f0 2024-09-15T22:20:01.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.257+0000 7f4d79b9a640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d741091f0 0x7f4d7410c5e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4d700058c0 tx=0x7f4d70005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:01.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.257+0000 7f4d6affd640 1 -- 172.21.15.5:0/160287086 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d70040030 con 0x7f4d741091f0 2024-09-15T22:20:01.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.258+0000 7f4d6affd640 1 -- 172.21.15.5:0/160287086 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d70004a30 con 0x7f4d741091f0 2024-09-15T22:20:01.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.259+0000 7f4d6affd640 1 -- 172.21.15.5:0/160287086 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7003c7c0 con 0x7f4d741091f0 2024-09-15T22:20:01.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.259+0000 7f4d7bdfe640 1 -- 172.21.15.5:0/160287086 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d741a2ef0 con 0x7f4d741091f0 2024-09-15T22:20:01.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.259+0000 7f4d7bdfe640 1 -- 172.21.15.5:0/160287086 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d741a3350 con 0x7f4d741091f0 2024-09-15T22:20:01.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.260+0000 7f4d68ff9640 1 -- 172.21.15.5:0/160287086 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d74073b00 con 0x7f4d741091f0 2024-09-15T22:20:01.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.265+0000 7f4d6affd640 1 -- 172.21.15.5:0/160287086 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4d70043030 con 0x7f4d741091f0 2024-09-15T22:20:01.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.266+0000 7f4d6affd640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d600a3420 0x7f4d600a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:01.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.266+0000 7f4d6affd640 1 -- 172.21.15.5:0/160287086 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9541+0+0 (secure 0 0 0) 0x7f4d700f1210 con 0x7f4d741091f0 2024-09-15T22:20:01.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.267+0000 7f4d79399640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d600a3420 0x7f4d600a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:01.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.270+0000 7f4d79399640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d600a3420 0x7f4d600a58e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4d7410dda0 tx=0x7f4d64005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:01.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.270+0000 7f4d6affd640 1 -- 172.21.15.5:0/160287086 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d700bd930 con 0x7f4d741091f0 2024-09-15T22:20:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:01.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:01 smithi005 ceph-mon[30944]: pgmap v344: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 455 op/s 2024-09-15T22:20:01.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.636+0000 7f4d68ff9640 1 -- 172.21.15.5:0/160287086 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d7410db10 con 0x7f4d741091f0 2024-09-15T22:20:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.637+0000 7f4d6affd640 1 -- 172.21.15.5:0/160287086 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f4d700bd230 con 0x7f4d741091f0 2024-09-15T22:20:01.639 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:20:01.639 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:20:01.639 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:20:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 -- 172.21.15.5:0/160287086 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d600a3420 msgr2=0x7f4d600a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d600a3420 0x7f4d600a58e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4d7410dda0 tx=0x7f4d64005f90 comp rx=0 tx=0).stop 2024-09-15T22:20:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 -- 172.21.15.5:0/160287086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d741091f0 msgr2=0x7f4d7410c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d741091f0 0x7f4d7410c5e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4d700058c0 tx=0x7f4d70005230 comp rx=0 tx=0).stop 2024-09-15T22:20:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 -- 172.21.15.5:0/160287086 shutdown_connections 2024-09-15T22:20:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4d600a3420 0x7f4d600a58e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4d74116ea0 0x7f4d7410d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d74109b10 0x7f4d7410cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 --2- 172.21.15.5:0/160287086 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4d741091f0 0x7f4d7410c5e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 -- 172.21.15.5:0/160287086 >> 172.21.15.5:0/160287086 conn(0x7f4d7406e140 msgr2=0x7f4d74117380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:01.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 -- 172.21.15.5:0/160287086 shutdown_connections 2024-09-15T22:20:01.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:01.639+0000 7f4d68ff9640 1 -- 172.21.15.5:0/160287086 wait complete. 2024-09-15T22:20:01.741 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:01 smithi099 ceph-mon[34892]: pgmap v344: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 455 op/s 2024-09-15T22:20:01.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:01 smithi129 ceph-mon[35314]: pgmap v344: 129 pgs: 129 active+clean; 2.9 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 7.1 MiB/s wr, 455 op/s 2024-09-15T22:20:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:02 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/160287086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:02 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/160287086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:02.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:02 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/160287086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:03.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:03 smithi099 ceph-mon[34892]: pgmap v345: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 735 op/s 2024-09-15T22:20:03.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:03 smithi099 ceph-mon[34892]: osdmap e98: 12 total, 12 up, 12 in 2024-09-15T22:20:03.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:03 smithi129 ceph-mon[35314]: pgmap v345: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 735 op/s 2024-09-15T22:20:03.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:03 smithi129 ceph-mon[35314]: osdmap e98: 12 total, 12 up, 12 in 2024-09-15T22:20:03.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:03 smithi005 ceph-mon[30944]: pgmap v345: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 10 MiB/s wr, 735 op/s 2024-09-15T22:20:03.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:03 smithi005 ceph-mon[30944]: osdmap e98: 12 total, 12 up, 12 in 2024-09-15T22:20:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 635.0 s, 450.7 tps, lat 218.490 ms stddev 332.637 2024-09-15T22:20:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.733+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/1950210894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d24101430 msgr2=0x7f5d2410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.733+0000 7f5d2c5cf640 1 --2- 172.21.15.5:0/1950210894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d24101430 0x7f5d2410d960 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009ca0 tx=0x7f5d1c02a8f0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.733+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/1950210894 shutdown_connections 2024-09-15T22:20:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.733+0000 7f5d2c5cf640 1 --2- 172.21.15.5:0/1950210894 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d24101430 0x7f5d2410d960 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.733+0000 7f5d2c5cf640 1 --2- 172.21.15.5:0/1950210894 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d24100a90 0x7f5d24100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.733+0000 7f5d2c5cf640 1 --2- 172.21.15.5:0/1950210894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d24106a90 0x7f5d24106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.733+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/1950210894 >> 172.21.15.5:0/1950210894 conn(0x7f5d240fc7d0 msgr2=0x7f5d240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.735+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/1950210894 shutdown_connections 2024-09-15T22:20:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.735+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/1950210894 wait complete. 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.736+0000 7f5d2c5cf640 1 Processor -- start 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.736+0000 7f5d2c5cf640 1 -- start start 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.736+0000 7f5d2c5cf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d24100a90 0x7f5d24073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.736+0000 7f5d2c5cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d24101430 0x7f5d24073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.736+0000 7f5d2c5cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d24106a90 0x7f5d2406dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.736+0000 7f5d2c5cf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24110fa0 con 0x7f5d24106a90 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.736+0000 7f5d2c5cf640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24110e20 con 0x7f5d24100a90 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.737+0000 7f5d2c5cf640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5d24111120 con 0x7f5d24101430 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.737+0000 7f5d2ab6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d24106a90 0x7f5d2406dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.737+0000 7f5d2ab6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d24106a90 0x7f5d2406dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33356/0 (socket says 172.21.15.5:33356) 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.737+0000 7f5d2ab6c640 1 -- 172.21.15.5:0/648459122 learned_addr learned my addr 172.21.15.5:0/648459122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.737+0000 7f5d2a36b640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d24100a90 0x7f5d24073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.738+0000 7f5d2a36b640 1 -- 172.21.15.5:0/648459122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d24101430 msgr2=0x7f5d24073ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.738+0000 7f5d2a36b640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d24101430 0x7f5d24073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.738+0000 7f5d2a36b640 1 -- 172.21.15.5:0/648459122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d24106a90 msgr2=0x7f5d2406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.738+0000 7f5d2a36b640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d24106a90 0x7f5d2406dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.738+0000 7f5d2a36b640 1 -- 172.21.15.5:0/648459122 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d2406e520 con 0x7f5d24100a90 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.738+0000 7f5d2ab6c640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d24106a90 0x7f5d2406dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.738+0000 7f5d2a36b640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d24100a90 0x7f5d24073980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000b3f0 tx=0x7f5d2000b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.738+0000 7f5d1b7fe640 1 -- 172.21.15.5:0/648459122 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20015020 con 0x7f5d24100a90 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.738+0000 7f5d1b7fe640 1 -- 172.21.15.5:0/648459122 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d20011d10 con 0x7f5d24100a90 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.738+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/648459122 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d2406e810 con 0x7f5d24100a90 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.739+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/648459122 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d2419c730 con 0x7f5d24100a90 2024-09-15T22:20:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.739+0000 7f5d1b7fe640 1 -- 172.21.15.5:0/648459122 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d20010e40 con 0x7f5d24100a90 2024-09-15T22:20:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.742+0000 7f5d2e9a8740 1 Processor -- start 2024-09-15T22:20:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.742+0000 7f5d2e9a8740 1 -- start start 2024-09-15T22:20:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.743+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880440 0x5644be880820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.743+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880df0 0x5644be885470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.743+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 0x5644be885ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.743+0000 7f5d2ab6c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880440 0x5644be880820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.743+0000 7f5d2ab6c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880440 0x5644be880820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:41900/0 (socket says 172.21.15.5:41900) 2024-09-15T22:20:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.743+0000 7f5d2ab6c640 1 -- 172.21.15.5:0/1602246508 learned_addr learned my addr 172.21.15.5:0/1602246508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.743+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/1602246508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5644be7dc2f0 con 0x5644be885a40 2024-09-15T22:20:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.743+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/1602246508 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5644be783cf0 con 0x5644be880440 2024-09-15T22:20:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.743+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/1602246508 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5644be7d82b0 con 0x5644be880df0 2024-09-15T22:20:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.743+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/1602246508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 0x5644be885ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.744+0000 7f5d29b6a640 1 -- 172.21.15.5:0/1602246508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880df0 msgr2=0x5644be885470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:20:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.744+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/1602246508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880df0 0x5644be885470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.744+0000 7f5d29b6a640 1 -- 172.21.15.5:0/1602246508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880440 msgr2=0x5644be880820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.744+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/1602246508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880440 0x5644be880820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.744+0000 7f5d29b6a640 1 -- 172.21.15.5:0/1602246508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644be7683a0 con 0x5644be885a40 2024-09-15T22:20:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.744+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/1602246508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 0x5644be885ef0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5d140096f0 tx=0x7f5d14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd27ceb58c4cb86c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.745+0000 7f5d18ff9640 1 -- 172.21.15.5:0/1602246508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14015020 con 0x5644be885a40 2024-09-15T22:20:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.745+0000 7f5d18ff9640 1 -- 172.21.15.5:0/1602246508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1400bdb0 con 0x5644be885a40 2024-09-15T22:20:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.745+0000 7f5d18ff9640 1 -- 172.21.15.5:0/1602246508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14010380 con 0x5644be885a40 2024-09-15T22:20:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.746+0000 7f5d1b7fe640 1 -- 172.21.15.5:0/648459122 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5d20018020 con 0x7f5d24100a90 2024-09-15T22:20:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.746+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/1602246508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 msgr2=0x5644be885ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.746+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/1602246508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 0x5644be885ef0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f5d140096f0 tx=0x7f5d14009bb0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.747+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/1602246508 shutdown_connections 2024-09-15T22:20:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.747+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/1602246508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 0x5644be885ef0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.747+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/1602246508 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880df0 0x5644be885470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.747+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/1602246508 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880440 0x5644be880820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.747+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/1602246508 >> 172.21.15.5:0/1602246508 conn(0x5644be83f310 msgr2=0x5644be8726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:05.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.747+0000 7f5d1b7fe640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d100a35c0 0x7f5d100a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.748+0000 7f5d1b7fe640 1 -- 172.21.15.5:0/648459122 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9629+0+0 (secure 0 0 0) 0x7f5d200cf6c0 con 0x7f5d24100a90 2024-09-15T22:20:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.749+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d100a35c0 0x7f5d100a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.750+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/1602246508 shutdown_connections 2024-09-15T22:20:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.750+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/1602246508 wait complete. 2024-09-15T22:20:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.750+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d100a35c0 0x7f5d100a5a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400b4b0 tx=0x7f5d14008040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.750+0000 7f5d2e9a8740 1 -- start start 2024-09-15T22:20:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.751+0000 7f5d2e9a8740 1 Processor -- start 2024-09-15T22:20:05.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.751+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880440 0x5644be88bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.751+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be880df0 0x5644be888ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.751+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be885a40 0x5644be889530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.752+0000 7f5d2e9a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5644be88c8d0 con 0x5644be880df0 2024-09-15T22:20:05.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.752+0000 7f5d2ab6c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be885a40 0x5644be889530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.752+0000 7f5d2ab6c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be885a40 0x5644be889530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:41914/0 (socket says 172.21.15.5:41914) 2024-09-15T22:20:05.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.752+0000 7f5d2ab6c640 1 -- 172.21.15.5:0/2914465309 learned_addr learned my addr 172.21.15.5:0/2914465309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:05.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.752+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2914465309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5644be797060 con 0x5644be885a40 2024-09-15T22:20:05.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.752+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2914465309 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5644be88ca50 con 0x5644be880440 2024-09-15T22:20:05.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.752+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be880df0 0x5644be888ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.752+0000 7f5d2e9a8740 10 client.0 _ll_register_callbacks cb 0x5644be4ea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:05.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.753+0000 7f5d2a36b640 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880440 0x5644be88bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.754+0000 7f5d2ab6c640 1 -- 172.21.15.5:0/2914465309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880440 msgr2=0x5644be88bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:05.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.754+0000 7f5d2ab6c640 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880440 0x5644be88bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.754+0000 7f5d2ab6c640 1 -- 172.21.15.5:0/2914465309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be880df0 msgr2=0x5644be888ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:05.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.754+0000 7f5d2ab6c640 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be880df0 0x5644be888ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.754+0000 7f5d2ab6c640 1 -- 172.21.15.5:0/2914465309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644be54d3c0 con 0x5644be885a40 2024-09-15T22:20:05.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.754+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be880df0 0x5644be888ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:05.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.754+0000 7f5d2ab6c640 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be885a40 0x5644be889530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c02f8e0 tx=0x7f5d1c02f910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc5c01c7078426ec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:05.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:05 smithi005 ceph-mon[30944]: pgmap v347: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 647 op/s 2024-09-15T22:20:05.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.755+0000 7f5d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:20:05.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.755+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2914465309 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c02fc40 con 0x5644be885a40 2024-09-15T22:20:05.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.756+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2914465309 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9629+0+0 (secure 0 0 0) 0x7f5d1c02fde0 con 0x5644be885a40 2024-09-15T22:20:05.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.756+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2914465309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d1400bdb0 con 0x5644be885a40 2024-09-15T22:20:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.760+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2914465309 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d1c0305f0 con 0x5644be885a40 2024-09-15T22:20:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.760+0000 7f5d2e9a8740 10 client.26852 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.760+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2914465309 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644be898280 con 0x5644be885a40 2024-09-15T22:20:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.761+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2914465309 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1400bdb0 con 0x5644be885a40 2024-09-15T22:20:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.761+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2914465309 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5d1c04c030 con 0x5644be885a40 2024-09-15T22:20:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.761+0000 7f5d2e9a8740 10 client.26852 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.761+0000 7f5d2e9a8740 10 client.26852 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:20:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.761+0000 7f5d2e9a8740 20 client.26852 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.761+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5644be899800 0x5644be89bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.761+0000 7f5d2e9a8740 4 client.26852 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.761+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2914465309 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644be7e9860 con 0x5644be899800 2024-09-15T22:20:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.761+0000 7f5d2a36b640 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5644be899800 0x5644be89bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.762+0000 7f5d0dffb640 10 client.26852 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:20:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.763+0000 7f5d2a36b640 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5644be899800 0x5644be89bc60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a17927722c376fc6 server_cookie=bbee646ad04e95e4 in_seq=0 out_seq=0 2024-09-15T22:20:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.768+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2914465309 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644be7e9860 con 0x5644be899800 2024-09-15T22:20:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.768+0000 7f5d0dffb640 10 client.26852 handle_command_reply: tid=0 2024-09-15T22:20:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.768+0000 7f5d2e9a8740 1 client.26852 shutdown 2024-09-15T22:20:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.769+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2914465309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be885a40 msgr2=0x5644be889530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.769+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be885a40 0x5644be889530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c02f8e0 tx=0x7f5d1c02f910 comp rx=0 tx=0).stop 2024-09-15T22:20:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.773+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2914465309 shutdown_connections 2024-09-15T22:20:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.773+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5644be899800 0x5644be89bc60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.773+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be885a40 0x5644be889530 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.773+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be880df0 0x5644be888ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.773+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2914465309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880440 0x5644be88bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.773+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2914465309 >> 172.21.15.5:0/2914465309 conn(0x5644be83f310 msgr2=0x5644be87fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.773+0000 7f5d0dffb640 0 client.26852 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:20:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.777+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2914465309 shutdown_connections 2024-09-15T22:20:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.778+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2914465309 wait complete. 2024-09-15T22:20:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.778+0000 7f5d2e9a8740 20 client.26852 trim_cache size 0 max 16384 2024-09-15T22:20:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.808+0000 7f5d2e9a8740 1 Processor -- start 2024-09-15T22:20:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.808+0000 7f5d2e9a8740 1 -- start start 2024-09-15T22:20:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.808+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880440 0x5644be87fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.808+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 0x5644be8a1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.808+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 0x5644be8a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.809+0000 7f5d2e9a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08012a40 con 0x5644be885a40 2024-09-15T22:20:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.809+0000 7f5d2e9a8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08010310 con 0x5644be880df0 2024-09-15T22:20:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.809+0000 7f5d2e9a8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0800e180 con 0x5644be880440 2024-09-15T22:20:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.809+0000 7f5d2ab6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 0x5644be8a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.809+0000 7f5d2ab6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 0x5644be8a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33384/0 (socket says 172.21.15.5:33384) 2024-09-15T22:20:05.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.809+0000 7f5d2ab6c640 1 -- 172.21.15.5:0/3219319028 learned_addr learned my addr 172.21.15.5:0/3219319028 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.813+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/3219319028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 0x5644be8a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.813+0000 7f5d2a36b640 1 --2- 172.21.15.5:0/3219319028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880440 0x5644be87fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.814+0000 7f5d2ab6c640 1 -- 172.21.15.5:0/3219319028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880440 msgr2=0x5644be87fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:05.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.814+0000 7f5d2ab6c640 1 --2- 172.21.15.5:0/3219319028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880440 0x5644be87fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.814+0000 7f5d2ab6c640 1 -- 172.21.15.5:0/3219319028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 msgr2=0x5644be8a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.814+0000 7f5d2ab6c640 1 --2- 172.21.15.5:0/3219319028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 0x5644be8a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.814+0000 7f5d2ab6c640 1 -- 172.21.15.5:0/3219319028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644be876240 con 0x5644be885a40 2024-09-15T22:20:05.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.820+0000 7f5d2ab6c640 1 --2- 172.21.15.5:0/3219319028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 0x5644be8a1a70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c02a930 tx=0x7f5d1c009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcfc6ed7cdbb964 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.823+0000 7f5d0ffff640 1 -- 172.21.15.5:0/3219319028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c02fc40 con 0x5644be885a40 2024-09-15T22:20:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.823+0000 7f5d0ffff640 1 -- 172.21.15.5:0/3219319028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c036040 con 0x5644be885a40 2024-09-15T22:20:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.824+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/3219319028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 msgr2=0x5644be8a1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.824+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/3219319028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 0x5644be8a1a70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c02a930 tx=0x7f5d1c009990 comp rx=0 tx=0).stop 2024-09-15T22:20:05.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.827+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/3219319028 shutdown_connections 2024-09-15T22:20:05.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.827+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/3219319028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be885a40 0x5644be8a1a70 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.827+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/3219319028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 0x5644be8a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.827+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/3219319028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be880440 0x5644be87fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.827+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/3219319028 >> 172.21.15.5:0/3219319028 conn(0x5644be83f310 msgr2=0x5644be88b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:05.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.829+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/3219319028 shutdown_connections 2024-09-15T22:20:05.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.833+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/3219319028 wait complete. 2024-09-15T22:20:05.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.833+0000 7f5d2e9a8740 1 -- start start 2024-09-15T22:20:05.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.833+0000 7f5d2e9a8740 1 Processor -- start 2024-09-15T22:20:05.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.834+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be880440 0x5644be8a1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.834+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 0x5644be8a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.834+0000 7f5d2e9a8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be885a40 0x5644be8a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.834+0000 7f5d2e9a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0800e7d0 con 0x5644be880440 2024-09-15T22:20:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.834+0000 7f5d2e9a8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0800fc40 con 0x5644be880df0 2024-09-15T22:20:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.834+0000 7f5d2e9a8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0800eea0 con 0x5644be885a40 2024-09-15T22:20:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.834+0000 7f5d2e9a8740 10 client.0 _ll_register_callbacks cb 0x7f5d1c02f340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:05.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.834+0000 7f5d29b6a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 0x5644be8a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.834+0000 7f5d29b6a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 0x5644be8a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:41928/0 (socket says 172.21.15.5:41928) 2024-09-15T22:20:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.834+0000 7f5d29b6a640 1 -- 172.21.15.5:0/2911340615 learned_addr learned my addr 172.21.15.5:0/2911340615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.835+0000 7f5d29b6a640 1 -- 172.21.15.5:0/2911340615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be885a40 msgr2=0x5644be8a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.835+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/2911340615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be885a40 0x5644be8a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.835+0000 7f5d29b6a640 1 -- 172.21.15.5:0/2911340615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be880440 msgr2=0x5644be8a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.835+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/2911340615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be880440 0x5644be8a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.835+0000 7f5d29b6a640 1 -- 172.21.15.5:0/2911340615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644be903660 con 0x5644be880df0 2024-09-15T22:20:05.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:05 smithi099 ceph-mon[34892]: pgmap v347: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 647 op/s 2024-09-15T22:20:05.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.840+0000 7f5d29b6a640 1 --2- 172.21.15.5:0/2911340615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 0x5644be8a1760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d14011b60 tx=0x7f5d14011b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fef8713ed5c7d0e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:05.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.840+0000 7f5d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:20:05.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:05 smithi129 ceph-mon[35314]: pgmap v347: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 8.6 MiB/s wr, 647 op/s 2024-09-15T22:20:05.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.845+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2911340615 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d14020070 con 0x5644be880df0 2024-09-15T22:20:05.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.845+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2911340615 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9629+0+0 (secure 0 0 0) 0x7f5d1401a830 con 0x5644be880df0 2024-09-15T22:20:05.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.846+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2911340615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d1c036040 con 0x5644be880df0 2024-09-15T22:20:05.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.846+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2911340615 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d14015040 con 0x5644be880df0 2024-09-15T22:20:05.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.846+0000 7f5d2e9a8740 10 client.26858 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:05.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.846+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2911340615 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5644be9048e0 con 0x5644be880df0 2024-09-15T22:20:05.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.853+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2911340615 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d1c036040 con 0x5644be880df0 2024-09-15T22:20:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.853+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2911340615 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5d14039cc0 con 0x5644be880df0 2024-09-15T22:20:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.853+0000 7f5d2e9a8740 10 client.26858 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.853+0000 7f5d2e9a8740 10 client.26858 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:20:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.853+0000 7f5d2e9a8740 20 client.26858 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.853+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2911340615 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5644be904e70 0x5644be9072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.853+0000 7f5d2e9a8740 4 client.26858 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.853+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2911340615 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644be7e3d70 con 0x5644be904e70 2024-09-15T22:20:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.853+0000 7f5d2a36b640 1 --2- 172.21.15.5:0/2911340615 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5644be904e70 0x5644be9072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.854+0000 7f5d0dffb640 10 client.26858 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:20:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.854+0000 7f5d2a36b640 1 --2- 172.21.15.5:0/2911340615 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5644be904e70 0x5644be9072d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5924ddada256056 server_cookie=f1e72dfe39605156 in_seq=0 out_seq=0 2024-09-15T22:20:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.858+0000 7f5d0dffb640 1 -- 172.21.15.5:0/2911340615 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644be7e3d70 con 0x5644be904e70 2024-09-15T22:20:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.858+0000 7f5d0dffb640 10 client.26858 handle_command_reply: tid=0 2024-09-15T22:20:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.858+0000 7f5d2e9a8740 1 client.26858 shutdown 2024-09-15T22:20:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.862+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2911340615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 msgr2=0x5644be8a1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.862+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2911340615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 0x5644be8a1760 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5d14011b60 tx=0x7f5d14011b90 comp rx=0 tx=0).stop 2024-09-15T22:20:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.863+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2911340615 shutdown_connections 2024-09-15T22:20:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.863+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2911340615 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5644be904e70 0x5644be9072d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.863+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2911340615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5644be885a40 0x5644be8a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.863+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2911340615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5644be880df0 0x5644be8a1760 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.863+0000 7f5d0dffb640 0 client.26858 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:20:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.863+0000 7f5d2e9a8740 1 --2- 172.21.15.5:0/2911340615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644be880440 0x5644be8a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.863+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2911340615 >> 172.21.15.5:0/2911340615 conn(0x5644be83f310 msgr2=0x5644be88a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.864+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2911340615 shutdown_connections 2024-09-15T22:20:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.868+0000 7f5d2e9a8740 1 -- 172.21.15.5:0/2911340615 wait complete. 2024-09-15T22:20:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.868+0000 7f5d2e9a8740 20 client.26858 trim_cache size 0 max 16384 2024-09-15T22:20:05.870 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:20:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/648459122 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d100a35c0 msgr2=0x7f5d100a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d100a35c0 0x7f5d100a5a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400b4b0 tx=0x7f5d14008040 comp rx=0 tx=0).stop 2024-09-15T22:20:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/648459122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d24100a90 msgr2=0x7f5d24073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d24100a90 0x7f5d24073980 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000b3f0 tx=0x7f5d2000b8b0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/648459122 shutdown_connections 2024-09-15T22:20:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d100a35c0 0x7f5d100a5a80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d24106a90 0x7f5d2406dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d24101430 0x7f5d24073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 --2- 172.21.15.5:0/648459122 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d24100a90 0x7f5d24073980 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/648459122 >> 172.21.15.5:0/648459122 conn(0x7f5d240fc7d0 msgr2=0x7f5d24102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:05.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/648459122 shutdown_connections 2024-09-15T22:20:05.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:05.871+0000 7f5d2c5cf640 1 -- 172.21.15.5:0/648459122 wait complete. 2024-09-15T22:20:06.273 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0' 2024-09-15T22:20:06.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0 2024-09-15T22:20:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:07 smithi099 ceph-mon[34892]: pgmap v348: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.3 MiB/s wr, 647 op/s 2024-09-15T22:20:07.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:07 smithi129 ceph-mon[35314]: pgmap v348: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.3 MiB/s wr, 647 op/s 2024-09-15T22:20:07.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:07 smithi005 ceph-mon[30944]: pgmap v348: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 9.3 MiB/s wr, 647 op/s 2024-09-15T22:20:08.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:20:08.827 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:09.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.542+0000 7f07c638b640 1 -- 172.21.15.5:0/278571264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0077040 msgr2=0x7f07c0112320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.542+0000 7f07c638b640 1 --2- 172.21.15.5:0/278571264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0077040 0x7f07c0112320 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009900 tx=0x7f07b002a410 comp rx=0 tx=0).stop 2024-09-15T22:20:09.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.542+0000 7f07c638b640 1 -- 172.21.15.5:0/278571264 shutdown_connections 2024-09-15T22:20:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.542+0000 7f07c638b640 1 --2- 172.21.15.5:0/278571264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0112860 0x7f07c0114c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.542+0000 7f07c638b640 1 --2- 172.21.15.5:0/278571264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0077040 0x7f07c0112320 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.542+0000 7f07c638b640 1 --2- 172.21.15.5:0/278571264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07c0076720 0x7f07c0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.542+0000 7f07c638b640 1 -- 172.21.15.5:0/278571264 >> 172.21.15.5:0/278571264 conn(0x7f07c00ff8a0 msgr2=0x7f07c0101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.543+0000 7f07c638b640 1 -- 172.21.15.5:0/278571264 shutdown_connections 2024-09-15T22:20:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.543+0000 7f07c638b640 1 -- 172.21.15.5:0/278571264 wait complete. 2024-09-15T22:20:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.543+0000 7f07c638b640 1 Processor -- start 2024-09-15T22:20:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.544+0000 7f07c638b640 1 -- start start 2024-09-15T22:20:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.544+0000 7f07c638b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0076720 0x7f07c01a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.544+0000 7f07c638b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07c0077040 0x7f07c01a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.544+0000 7f07c638b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0112860 0x7f07c019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.544+0000 7f07c638b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07c0118430 con 0x7f07c0076720 2024-09-15T22:20:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.544+0000 7f07c638b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01182b0 con 0x7f07c0112860 2024-09-15T22:20:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.544+0000 7f07c638b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07c01185b0 con 0x7f07c0077040 2024-09-15T22:20:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.544+0000 7f07c4b88640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07c0077040 0x7f07c01a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.544+0000 7f07c5b8a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0112860 0x7f07c019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.544+0000 7f07c5389640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0076720 0x7f07c01a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.545+0000 7f07c5389640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0076720 0x7f07c01a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33410/0 (socket says 172.21.15.5:33410) 2024-09-15T22:20:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.545+0000 7f07c5389640 1 -- 172.21.15.5:0/1956295434 learned_addr learned my addr 172.21.15.5:0/1956295434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.545+0000 7f07c5b8a640 1 -- 172.21.15.5:0/1956295434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07c0077040 msgr2=0x7f07c01a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.545+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07c0077040 0x7f07c01a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.545+0000 7f07c5b8a640 1 -- 172.21.15.5:0/1956295434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0076720 msgr2=0x7f07c01a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.545+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0076720 0x7f07c01a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.545+0000 7f07c5b8a640 1 -- 172.21.15.5:0/1956295434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c019efd0 con 0x7f07c0112860 2024-09-15T22:20:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.545+0000 7f07c4b88640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07c0077040 0x7f07c01a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.545+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0112860 0x7f07c019e760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f07c007aa10 tx=0x7f07b800edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.546+0000 7f07b67fc640 1 -- 172.21.15.5:0/1956295434 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8004d20 con 0x7f07c0112860 2024-09-15T22:20:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.546+0000 7f07b67fc640 1 -- 172.21.15.5:0/1956295434 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b8005020 con 0x7f07c0112860 2024-09-15T22:20:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.546+0000 7f07c638b640 1 -- 172.21.15.5:0/1956295434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c019f2c0 con 0x7f07c0112860 2024-09-15T22:20:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.546+0000 7f07c638b640 1 -- 172.21.15.5:0/1956295434 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c01ab610 con 0x7f07c0112860 2024-09-15T22:20:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.546+0000 7f07b67fc640 1 -- 172.21.15.5:0/1956295434 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b80116d0 con 0x7f07c0112860 2024-09-15T22:20:09.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.549+0000 7f07b67fc640 1 -- 172.21.15.5:0/1956295434 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f07b800caf0 con 0x7f07c0112860 2024-09-15T22:20:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.549+0000 7f07b67fc640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07ac0a3430 0x7f07ac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.549+0000 7f07b67fc640 1 -- 172.21.15.5:0/1956295434 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f07b80cffe0 con 0x7f07c0112860 2024-09-15T22:20:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.550+0000 7f07c5389640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07ac0a3430 0x7f07ac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.551+0000 7f07c5389640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07ac0a3430 0x7f07ac0a58f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009c90 tx=0x7f07bc005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.557+0000 7f07c99c6740 1 Processor -- start 2024-09-15T22:20:09.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.557+0000 7f07c99c6740 1 -- start start 2024-09-15T22:20:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.557+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729e7a20 0x5561729e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.557+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 0x5561729f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.557+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729f2c00 0x5561729f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.557+0000 7f07c99c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55617294c160 con 0x5561729e7a20 2024-09-15T22:20:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.557+0000 7f07c99c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5561728f3720 con 0x5561729f2c00 2024-09-15T22:20:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.557+0000 7f07c99c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556172948120 con 0x5561729e8340 2024-09-15T22:20:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.557+0000 7f07c4b88640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 0x5561729f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.557+0000 7f07c4b88640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 0x5561729f26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:37908/0 (socket says 172.21.15.5:37908) 2024-09-15T22:20:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.557+0000 7f07c4b88640 1 -- 172.21.15.5:0/1456067868 learned_addr learned my addr 172.21.15.5:0/1456067868 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.558+0000 7f07c5389640 1 --2- 172.21.15.5:0/1456067868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729e7a20 0x5561729e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.558+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/1456067868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729f2c00 0x5561729f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.558+0000 7f07c4b88640 1 -- 172.21.15.5:0/1456067868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729f2c00 msgr2=0x5561729f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.558+0000 7f07c4b88640 1 --2- 172.21.15.5:0/1456067868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729f2c00 0x5561729f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.558+0000 7f07c4b88640 1 -- 172.21.15.5:0/1456067868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729e7a20 msgr2=0x5561729e7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.558+0000 7f07c4b88640 1 --2- 172.21.15.5:0/1456067868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729e7a20 0x5561729e7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.558+0000 7f07c4b88640 1 -- 172.21.15.5:0/1456067868 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561726bdba0 con 0x5561729e8340 2024-09-15T22:20:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.559+0000 7f07c4b88640 1 --2- 172.21.15.5:0/1456067868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 0x5561729f26c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009900 tx=0x7f07b00043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2bbac3fb3670d61 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.559+0000 7f07ab7fe640 1 -- 172.21.15.5:0/1456067868 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0035040 con 0x5561729e8340 2024-09-15T22:20:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.559+0000 7f07ab7fe640 1 -- 172.21.15.5:0/1456067868 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b0004760 con 0x5561729e8340 2024-09-15T22:20:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.559+0000 7f07ab7fe640 1 -- 172.21.15.5:0/1456067868 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0003ab0 con 0x5561729e8340 2024-09-15T22:20:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.560+0000 7f07c99c6740 1 -- 172.21.15.5:0/1456067868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 msgr2=0x5561729f26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.560+0000 7f07c99c6740 1 --2- 172.21.15.5:0/1456067868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 0x5561729f26c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009900 tx=0x7f07b00043f0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.560+0000 7f07c99c6740 1 -- 172.21.15.5:0/1456067868 shutdown_connections 2024-09-15T22:20:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.560+0000 7f07c99c6740 1 --2- 172.21.15.5:0/1456067868 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729f2c00 0x5561729f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.560+0000 7f07c99c6740 1 --2- 172.21.15.5:0/1456067868 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 0x5561729f26c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.560+0000 7f07c99c6740 1 --2- 172.21.15.5:0/1456067868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729e7a20 0x5561729e7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.560+0000 7f07c99c6740 1 -- 172.21.15.5:0/1456067868 >> 172.21.15.5:0/1456067868 conn(0x5561729e3fb0 msgr2=0x5561729e4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.560+0000 7f07c99c6740 1 -- 172.21.15.5:0/1456067868 shutdown_connections 2024-09-15T22:20:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.560+0000 7f07c99c6740 1 -- 172.21.15.5:0/1456067868 wait complete. 2024-09-15T22:20:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.561+0000 7f07c99c6740 1 -- start start 2024-09-15T22:20:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.561+0000 7f07c99c6740 1 Processor -- start 2024-09-15T22:20:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.561+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 0x5561729efa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.561+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 0x5561729effa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.561+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 0x5561729f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.561+0000 7f07c99c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5561729f5820 con 0x5561729f2c00 2024-09-15T22:20:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.561+0000 7f07c99c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5561728d4900 con 0x5561729e7a20 2024-09-15T22:20:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.561+0000 7f07c99c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5561729f59a0 con 0x5561729e8340 2024-09-15T22:20:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.561+0000 7f07c99c6740 10 client.0 _ll_register_callbacks cb 0x5561728d2780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c5389640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 0x5561729efa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c4b88640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 0x5561729effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c5389640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 0x5561729efa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:41962/0 (socket says 172.21.15.5:41962) 2024-09-15T22:20:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c5389640 1 -- 172.21.15.5:0/9326917 learned_addr learned my addr 172.21.15.5:0/9326917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 0x5561729f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c5b8a640 1 -- 172.21.15.5:0/9326917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 msgr2=0x5561729effa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 0x5561729effa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c5b8a640 1 -- 172.21.15.5:0/9326917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 msgr2=0x5561729efa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 0x5561729efa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c5b8a640 1 -- 172.21.15.5:0/9326917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556172914c50 con 0x5561729f2c00 2024-09-15T22:20:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c5389640 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 0x5561729efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 0x5561729f04e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f07b80cd2d0 tx=0x7f07b809bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367a96e87aaca96 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.562+0000 7f07a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.563+0000 7f07a8ff9640 1 -- 172.21.15.5:0/9326917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b801f040 con 0x5561729f2c00 2024-09-15T22:20:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.563+0000 7f07c99c6740 1 -- 172.21.15.5:0/9326917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b0004760 con 0x5561729f2c00 2024-09-15T22:20:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.563+0000 7f07a8ff9640 1 -- 172.21.15.5:0/9326917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f07b809b360 con 0x5561729f2c00 2024-09-15T22:20:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.563+0000 7f07a8ff9640 1 -- 172.21.15.5:0/9326917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f07b809b560 con 0x5561729f2c00 2024-09-15T22:20:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.563+0000 7f07c99c6740 10 client.17376 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.563+0000 7f07c99c6740 1 -- 172.21.15.5:0/9326917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556172a014b0 con 0x5561729f2c00 2024-09-15T22:20:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.564+0000 7f07a8ff9640 1 -- 172.21.15.5:0/9326917 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b0004760 con 0x5561729f2c00 2024-09-15T22:20:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.564+0000 7f07a8ff9640 1 -- 172.21.15.5:0/9326917 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f07b800f040 con 0x5561729f2c00 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.564+0000 7f07c99c6740 10 client.17376 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.564+0000 7f07c99c6740 10 client.17376 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.564+0000 7f07c99c6740 20 client.17376 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.564+0000 7f07c99c6740 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556172a02f00 0x556172a05380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.564+0000 7f07c99c6740 4 client.17376 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.564+0000 7f07c99c6740 1 -- 172.21.15.5:0/9326917 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556172959890 con 0x556172a02f00 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.565+0000 7f07c5389640 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556172a02f00 0x556172a05380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.565+0000 7f07c5389640 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556172a02f00 0x556172a05380 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e5c3c48398349b9 server_cookie=79b2c4cc28634db1 in_seq=0 out_seq=0 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.565+0000 7f07a8ff9640 10 client.17376 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.566+0000 7f07a8ff9640 1 -- 172.21.15.5:0/9326917 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556172959890 con 0x556172a02f00 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.566+0000 7f07a8ff9640 10 client.17376 handle_command_reply: tid=0 2024-09-15T22:20:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.566+0000 7f07c99c6740 1 client.17376 shutdown 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.566+0000 7f07c99c6740 1 -- 172.21.15.5:0/9326917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 msgr2=0x5561729f04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.566+0000 7f07c99c6740 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 0x5561729f04e0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f07b80cd2d0 tx=0x7f07b809bfd0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.567+0000 7f07c99c6740 1 -- 172.21.15.5:0/9326917 shutdown_connections 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.567+0000 7f07c99c6740 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 0x5561729f04e0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.567+0000 7f07c99c6740 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e8340 0x5561729effa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.567+0000 7f07c99c6740 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556172a02f00 0x556172a05380 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.567+0000 7f07c99c6740 1 --2- 172.21.15.5:0/9326917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 0x5561729efa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.567+0000 7f07c99c6740 1 -- 172.21.15.5:0/9326917 >> 172.21.15.5:0/9326917 conn(0x5561729e3fb0 msgr2=0x5561729e9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.567+0000 7f07a8ff9640 0 client.17376 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.567+0000 7f07c99c6740 1 -- 172.21.15.5:0/9326917 shutdown_connections 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.567+0000 7f07c99c6740 1 -- 172.21.15.5:0/9326917 wait complete. 2024-09-15T22:20:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.568+0000 7f07c99c6740 20 client.17376 trim_cache size 0 max 16384 2024-09-15T22:20:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.593+0000 7f07c99c6740 1 Processor -- start 2024-09-15T22:20:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.593+0000 7f07c99c6740 1 -- start start 2024-09-15T22:20:09.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:09 smithi005 ceph-mon[30944]: pgmap v349: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.3 MiB/s wr, 616 op/s 2024-09-15T22:20:09.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:09 smithi005 ceph-mon[30944]: osdmap e99: 12 total, 12 up, 12 in 2024-09-15T22:20:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.593+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e7a20 0x556172a0ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.593+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e8340 0x556172a0b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.593+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 0x5561729eccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.593+0000 7f07c99c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4012990 con 0x5561729f2c00 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.593+0000 7f07c99c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4010260 con 0x5561729e8340 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.593+0000 7f07c99c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07a400e0d0 con 0x5561729e7a20 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.594+0000 7f07c5389640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e7a20 0x556172a0ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.594+0000 7f07c5389640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e7a20 0x556172a0ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:37936/0 (socket says 172.21.15.5:37936) 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.594+0000 7f07c5389640 1 -- 172.21.15.5:0/648544675 learned_addr learned my addr 172.21.15.5:0/648544675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.594+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/648544675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 0x5561729eccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.594+0000 7f07c5b8a640 1 -- 172.21.15.5:0/648544675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e7a20 msgr2=0x556172a0ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.594+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/648544675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e7a20 0x556172a0ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.594+0000 7f07c5b8a640 1 -- 172.21.15.5:0/648544675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e8340 msgr2=0x556172a0b1d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.594+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/648544675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e8340 0x556172a0b1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.594+0000 7f07c5b8a640 1 -- 172.21.15.5:0/648544675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561729e3860 con 0x5561729f2c00 2024-09-15T22:20:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.594+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/648544675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 0x5561729eccc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f07b809a1c0 tx=0x7f07b80a4c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9dc0755f16110f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.595+0000 7f07aaffd640 1 -- 172.21.15.5:0/648544675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b801f040 con 0x5561729f2c00 2024-09-15T22:20:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.595+0000 7f07aaffd640 1 -- 172.21.15.5:0/648544675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b8098580 con 0x5561729f2c00 2024-09-15T22:20:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.595+0000 7f07aaffd640 1 -- 172.21.15.5:0/648544675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8097c60 con 0x5561729f2c00 2024-09-15T22:20:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.596+0000 7f07c99c6740 1 -- 172.21.15.5:0/648544675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 msgr2=0x5561729eccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.596+0000 7f07c99c6740 1 --2- 172.21.15.5:0/648544675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 0x5561729eccc0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f07b809a1c0 tx=0x7f07b80a4c50 comp rx=0 tx=0).stop 2024-09-15T22:20:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.597+0000 7f07c99c6740 1 -- 172.21.15.5:0/648544675 shutdown_connections 2024-09-15T22:20:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.597+0000 7f07c99c6740 1 --2- 172.21.15.5:0/648544675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729f2c00 0x5561729eccc0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.597+0000 7f07c99c6740 1 --2- 172.21.15.5:0/648544675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e8340 0x556172a0b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.597+0000 7f07c99c6740 1 --2- 172.21.15.5:0/648544675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729e7a20 0x556172a0ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.597+0000 7f07c99c6740 1 -- 172.21.15.5:0/648544675 >> 172.21.15.5:0/648544675 conn(0x5561729e3fb0 msgr2=0x5561729eada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.597+0000 7f07c99c6740 1 -- 172.21.15.5:0/648544675 shutdown_connections 2024-09-15T22:20:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.597+0000 7f07c99c6740 1 -- 172.21.15.5:0/648544675 wait complete. 2024-09-15T22:20:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.597+0000 7f07c99c6740 1 -- start start 2024-09-15T22:20:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.597+0000 7f07c99c6740 1 Processor -- start 2024-09-15T22:20:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.598+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 0x5561729fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.598+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729e8340 0x5561729f6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.598+0000 7f07c99c6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729f2c00 0x5561729f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.598+0000 7f07c99c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07a400e720 con 0x5561729e8340 2024-09-15T22:20:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.598+0000 7f07c99c6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f07a400fb90 con 0x5561729e7a20 2024-09-15T22:20:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.598+0000 7f07c99c6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f07a400edf0 con 0x5561729f2c00 2024-09-15T22:20:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.599+0000 7f07c5389640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 0x5561729fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.599+0000 7f07c5389640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 0x5561729fb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:41984/0 (socket says 172.21.15.5:41984) 2024-09-15T22:20:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.599+0000 7f07c5389640 1 -- 172.21.15.5:0/2814922025 learned_addr learned my addr 172.21.15.5:0/2814922025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.599+0000 7f07c99c6740 10 client.0 _ll_register_callbacks cb 0x7f07b8004ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.599+0000 7f07c4b88640 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729e8340 0x5561729f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.599+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729f2c00 0x5561729f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07c5b8a640 1 -- 172.21.15.5:0/2814922025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 msgr2=0x5561729fb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 0x5561729fb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07c5b8a640 1 -- 172.21.15.5:0/2814922025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729e8340 msgr2=0x5561729f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729e8340 0x5561729f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07c5b8a640 1 -- 172.21.15.5:0/2814922025 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561729f73d0 con 0x5561729f2c00 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07c5b8a640 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729f2c00 0x5561729f6ba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f07b809a1c0 tx=0x7f07b809bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eaa073a5edb74b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07a8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07a8ff9640 1 -- 172.21.15.5:0/2814922025 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b801f040 con 0x5561729f2c00 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07a8ff9640 1 -- 172.21.15.5:0/2814922025 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f07b8097e70 con 0x5561729f2c00 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07c99c6740 1 -- 172.21.15.5:0/2814922025 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8098580 con 0x5561729f2c00 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07a8ff9640 1 -- 172.21.15.5:0/2814922025 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f07b80baaf0 con 0x5561729f2c00 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07c99c6740 10 client.26818 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.600+0000 7f07c99c6740 1 -- 172.21.15.5:0/2814922025 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5561729f73d0 con 0x5561729f2c00 2024-09-15T22:20:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.601+0000 7f07a8ff9640 1 -- 172.21.15.5:0/2814922025 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b8098580 con 0x5561729f2c00 2024-09-15T22:20:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.601+0000 7f07a8ff9640 1 -- 172.21.15.5:0/2814922025 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f07b8096a80 con 0x5561729f2c00 2024-09-15T22:20:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.601+0000 7f07c99c6740 10 client.26818 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.601+0000 7f07c99c6740 10 client.26818 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:20:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.601+0000 7f07c99c6740 20 client.26818 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.601+0000 7f07c99c6740 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556172a7ef30 0x556172a81320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.601+0000 7f07c99c6740 4 client.26818 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}] 2024-09-15T22:20:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.601+0000 7f07c99c6740 1 -- 172.21.15.5:0/2814922025 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}) -- 0x556172953be0 con 0x556172a7ef30 2024-09-15T22:20:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.601+0000 7f07c5389640 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556172a7ef30 0x556172a81320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.602+0000 7f07a8ff9640 10 client.26818 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.602+0000 7f07c5389640 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556172a7ef30 0x556172a81320 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d656385893c4ff6d server_cookie=70b19cfb76adcf0a in_seq=0 out_seq=0 2024-09-15T22:20:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 640.0 s, 544.4 tps, lat 187.530 ms stddev 223.741 2024-09-15T22:20:09.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:09 smithi099 ceph-mon[34892]: pgmap v349: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.3 MiB/s wr, 616 op/s 2024-09-15T22:20:09.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:09 smithi099 ceph-mon[34892]: osdmap e99: 12 total, 12 up, 12 in 2024-09-15T22:20:09.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:09 smithi129 ceph-mon[35314]: pgmap v349: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 7.3 MiB/s wr, 616 op/s 2024-09-15T22:20:09.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:09 smithi129 ceph-mon[35314]: osdmap e99: 12 total, 12 up, 12 in 2024-09-15T22:20:09.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.965+0000 7f07a8ff9640 1 -- 172.21.15.5:0/2814922025 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556172953be0 con 0x556172a7ef30 2024-09-15T22:20:09.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.965+0000 7f07a8ff9640 10 client.26818 handle_command_reply: tid=0 2024-09-15T22:20:09.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.965+0000 7f07c99c6740 1 client.26818 shutdown 2024-09-15T22:20:09.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c99c6740 1 -- 172.21.15.5:0/2814922025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729f2c00 msgr2=0x5561729f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c99c6740 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729f2c00 0x5561729f6ba0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f07b809a1c0 tx=0x7f07b809bfd0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c99c6740 1 -- 172.21.15.5:0/2814922025 shutdown_connections 2024-09-15T22:20:09.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c99c6740 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5561729f2c00 0x5561729f6ba0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c99c6740 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5561729e8340 0x5561729f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c99c6740 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x556172a7ef30 0x556172a81320 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c99c6740 1 --2- 172.21.15.5:0/2814922025 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5561729e7a20 0x5561729fb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07a8ff9640 0 client.26818 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:09.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c99c6740 1 -- 172.21.15.5:0/2814922025 >> 172.21.15.5:0/2814922025 conn(0x5561729e3fb0 msgr2=0x5561729e9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c99c6740 1 -- 172.21.15.5:0/2814922025 shutdown_connections 2024-09-15T22:20:09.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c99c6740 1 -- 172.21.15.5:0/2814922025 wait complete. 2024-09-15T22:20:09.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c99c6740 20 client.26818 trim_cache size 0 max 16384 2024-09-15T22:20:09.971 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:20:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:20:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:20:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 38, 2024-09-15T22:20:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:20:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "6c8391fc": { 2024-09-15T22:20:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 38, 2024-09-15T22:20:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:20:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:20:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:20:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:20:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:20:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:20:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 141.8, 2024-09-15T22:20:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:20:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:20:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:20:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:20:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:20:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:20:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:09.973 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:09.973 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:09.973 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:09.973 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:20:09.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c638b640 1 -- 172.21.15.5:0/1956295434 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07ac0a3430 msgr2=0x7f07ac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c638b640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07ac0a3430 0x7f07ac0a58f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009c90 tx=0x7f07bc005f90 comp rx=0 tx=0).stop 2024-09-15T22:20:09.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c638b640 1 -- 172.21.15.5:0/1956295434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0112860 msgr2=0x7f07c019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:09.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.966+0000 7f07c638b640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0112860 0x7f07c019e760 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f07c007aa10 tx=0x7f07b800edd0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.967+0000 7f07c638b640 1 -- 172.21.15.5:0/1956295434 shutdown_connections 2024-09-15T22:20:09.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.967+0000 7f07c638b640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f07ac0a3430 0x7f07ac0a58f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.967+0000 7f07c638b640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f07c0112860 0x7f07c019e760 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.967+0000 7f07c638b640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f07c0077040 0x7f07c01a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.967+0000 7f07c638b640 1 --2- 172.21.15.5:0/1956295434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07c0076720 0x7f07c01a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:09.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.967+0000 7f07c638b640 1 -- 172.21.15.5:0/1956295434 >> 172.21.15.5:0/1956295434 conn(0x7f07c00ff8a0 msgr2=0x7f07c0101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:09.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.967+0000 7f07c638b640 1 -- 172.21.15.5:0/1956295434 shutdown_connections 2024-09-15T22:20:09.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:09.967+0000 7f07c638b640 1 -- 172.21.15.5:0/1956295434 wait complete. 2024-09-15T22:20:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.096+0000 7f894328b640 1 -- 172.21.15.5:0/4277450476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893c106f50 msgr2=0x7f893c10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.096+0000 7f894328b640 1 --2- 172.21.15.5:0/4277450476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893c106f50 0x7f893c10eaf0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8934009cd0 tx=0x7f893402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.096+0000 7f894328b640 1 -- 172.21.15.5:0/4277450476 shutdown_connections 2024-09-15T22:20:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.096+0000 7f894328b640 1 --2- 172.21.15.5:0/4277450476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893c106f50 0x7f893c10eaf0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.096+0000 7f894328b640 1 --2- 172.21.15.5:0/4277450476 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f893c100ca0 0x7f893c106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.096+0000 7f894328b640 1 --2- 172.21.15.5:0/4277450476 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f893c1002f0 0x7f893c1006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.096+0000 7f894328b640 1 -- 172.21.15.5:0/4277450476 >> 172.21.15.5:0/4277450476 conn(0x7f893c077210 msgr2=0x7f893c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.096+0000 7f894328b640 1 -- 172.21.15.5:0/4277450476 shutdown_connections 2024-09-15T22:20:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.096+0000 7f894328b640 1 -- 172.21.15.5:0/4277450476 wait complete. 2024-09-15T22:20:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f894328b640 1 Processor -- start 2024-09-15T22:20:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f894328b640 1 -- start start 2024-09-15T22:20:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f894328b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f893c1002f0 0x7f893c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f894328b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893c100ca0 0x7f893c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f894328b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f893c106f50 0x7f893c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f894328b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f893c111fa0 con 0x7f893c100ca0 2024-09-15T22:20:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f894328b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f893c111e20 con 0x7f893c106f50 2024-09-15T22:20:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f894328b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f893c112120 con 0x7f893c1002f0 2024-09-15T22:20:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f8940826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893c100ca0 0x7f893c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f8940826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893c100ca0 0x7f893c1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33466/0 (socket says 172.21.15.5:33466) 2024-09-15T22:20:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f8940826640 1 -- 172.21.15.5:0/1592600602 learned_addr learned my addr 172.21.15.5:0/1592600602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f8941027640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f893c1002f0 0x7f893c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.097+0000 7f8941828640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f893c106f50 0x7f893c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f8941027640 1 -- 172.21.15.5:0/1592600602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f893c106f50 msgr2=0x7f893c101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f8941027640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f893c106f50 0x7f893c101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f8941027640 1 -- 172.21.15.5:0/1592600602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893c100ca0 msgr2=0x7f893c1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f8941027640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893c100ca0 0x7f893c1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f8941027640 1 -- 172.21.15.5:0/1592600602 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c1a6960 con 0x7f893c1002f0 2024-09-15T22:20:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f8940826640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893c100ca0 0x7f893c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f8941828640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f893c106f50 0x7f893c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f8941027640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f893c1002f0 0x7f893c101270 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f893800e9f0 tx=0x7f893800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f89327fc640 1 -- 172.21.15.5:0/1592600602 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893800ccc0 con 0x7f893c1002f0 2024-09-15T22:20:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f89327fc640 1 -- 172.21.15.5:0/1592600602 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8938010910 con 0x7f893c1002f0 2024-09-15T22:20:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f89327fc640 1 -- 172.21.15.5:0/1592600602 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8938018960 con 0x7f893c1002f0 2024-09-15T22:20:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f894328b640 1 -- 172.21.15.5:0/1592600602 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893c1a6c50 con 0x7f893c1002f0 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.098+0000 7f894328b640 1 -- 172.21.15.5:0/1592600602 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893c1a7190 con 0x7f893c1002f0 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.100+0000 7f89327fc640 1 -- 172.21.15.5:0/1592600602 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8938010ac0 con 0x7f893c1002f0 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8945664740 1 Processor -- start 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8945664740 1 -- start start 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8945664740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 0x56104d54e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8945664740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54e5d0 0x56104d54ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8945664740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d55b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8945664740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56104d4b42f0 con 0x56104d54e5d0 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8945664740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56104d45bcf0 con 0x56104d54fb40 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8945664740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56104d4b02b0 con 0x56104d559370 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8941027640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54e5d0 0x56104d54ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8941828640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 0x56104d54e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f89327fc640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89280a33b0 0x7f89280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8941828640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 0x56104d54e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:42018/0 (socket says 172.21.15.5:42018) 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8941027640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54e5d0 0x56104d54ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33472/0 (socket says 172.21.15.5:33472) 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8940826640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d55b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8941828640 1 -- 172.21.15.5:0/3988639546 learned_addr learned my addr 172.21.15.5:0/3988639546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8941828640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89280a33b0 0x7f89280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f89327fc640 1 -- 172.21.15.5:0/1592600602 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f89380d0490 con 0x7f893c1002f0 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8941027640 1 -- 172.21.15.5:0/3988639546 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 msgr2=0x56104d55b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8941027640 1 --2- 172.21.15.5:0/3988639546 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d55b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8941027640 1 -- 172.21.15.5:0/3988639546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 msgr2=0x56104d54e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8941027640 1 --2- 172.21.15.5:0/3988639546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 0x56104d54e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.101+0000 7f8941027640 1 -- 172.21.15.5:0/3988639546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104d4403a0 con 0x56104d54e5d0 2024-09-15T22:20:10.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.102+0000 7f8941027640 1 --2- 172.21.15.5:0/3988639546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54e5d0 0x56104d54ea10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f89380cd7e0 tx=0x7f8938009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5726d81491ce536b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.102+0000 7f8941828640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89280a33b0 0x7f89280a5870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8934009a60 tx=0x7f893402a8e0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.102+0000 7f89277fe640 1 -- 172.21.15.5:0/3988639546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89380a4fe0 con 0x56104d54e5d0 2024-09-15T22:20:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.102+0000 7f89277fe640 1 -- 172.21.15.5:0/3988639546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89380cda60 con 0x56104d54e5d0 2024-09-15T22:20:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.102+0000 7f89277fe640 1 -- 172.21.15.5:0/3988639546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893809b870 con 0x56104d54e5d0 2024-09-15T22:20:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.102+0000 7f8945664740 1 -- 172.21.15.5:0/3988639546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54e5d0 msgr2=0x56104d54ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.102+0000 7f8945664740 1 --2- 172.21.15.5:0/3988639546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54e5d0 0x56104d54ea10 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f89380cd7e0 tx=0x7f8938009700 comp rx=0 tx=0).stop 2024-09-15T22:20:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.102+0000 7f8945664740 1 -- 172.21.15.5:0/3988639546 shutdown_connections 2024-09-15T22:20:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.102+0000 7f8945664740 1 --2- 172.21.15.5:0/3988639546 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d55b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.102+0000 7f8945664740 1 --2- 172.21.15.5:0/3988639546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54e5d0 0x56104d54ea10 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 --2- 172.21.15.5:0/3988639546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 0x56104d54e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 -- 172.21.15.5:0/3988639546 >> 172.21.15.5:0/3988639546 conn(0x56104d517310 msgr2=0x56104d54ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 -- 172.21.15.5:0/3988639546 shutdown_connections 2024-09-15T22:20:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 -- 172.21.15.5:0/3988639546 wait complete. 2024-09-15T22:20:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 -- start start 2024-09-15T22:20:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 Processor -- start 2024-09-15T22:20:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d54e5d0 0x56104d558d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 0x56104d556020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d559370 0x56104d556610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56104d55c010 con 0x56104d559370 2024-09-15T22:20:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56104d46f060 con 0x56104d54fb40 2024-09-15T22:20:10.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56104d55c190 con 0x56104d54e5d0 2024-09-15T22:20:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8945664740 10 client.0 _ll_register_callbacks cb 0x56104d1c28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8941828640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d559370 0x56104d556610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8941828640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d559370 0x56104d556610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33484/0 (socket says 172.21.15.5:33484) 2024-09-15T22:20:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8940826640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 0x56104d556020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8941027640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d54e5d0 0x56104d558d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.103+0000 7f8941828640 1 -- 172.21.15.5:0/4216632498 learned_addr learned my addr 172.21.15.5:0/4216632498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8941828640 1 -- 172.21.15.5:0/4216632498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d54e5d0 msgr2=0x56104d558d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8941828640 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d54e5d0 0x56104d558d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8941828640 1 -- 172.21.15.5:0/4216632498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 msgr2=0x56104d556020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8941828640 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 0x56104d556020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8941828640 1 -- 172.21.15.5:0/4216632498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104d2253c0 con 0x56104d559370 2024-09-15T22:20:10.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8940826640 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 0x56104d556020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:10.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8941828640 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d559370 0x56104d556610 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8934009ca0 tx=0x7f89340095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d7d76c17a45994 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8924ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:10.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8924ff9640 1 -- 172.21.15.5:0/4216632498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893404e070 con 0x56104d559370 2024-09-15T22:20:10.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8924ff9640 1 -- 172.21.15.5:0/4216632498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f89340307a0 con 0x56104d559370 2024-09-15T22:20:10.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8945664740 1 -- 172.21.15.5:0/4216632498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89380cda60 con 0x56104d559370 2024-09-15T22:20:10.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8924ff9640 1 -- 172.21.15.5:0/4216632498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f893405a050 con 0x56104d559370 2024-09-15T22:20:10.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8945664740 10 client.17394 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:10.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.104+0000 7f8945664740 1 -- 172.21.15.5:0/4216632498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56104d567b00 con 0x56104d559370 2024-09-15T22:20:10.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.105+0000 7f8924ff9640 1 -- 172.21.15.5:0/4216632498 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f89380cda60 con 0x56104d559370 2024-09-15T22:20:10.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.105+0000 7f8924ff9640 1 -- 172.21.15.5:0/4216632498 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8934030070 con 0x56104d559370 2024-09-15T22:20:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.105+0000 7f8945664740 10 client.17394 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.105+0000 7f8945664740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.105+0000 7f8945664740 20 client.17394 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.105+0000 7f8945664740 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56104d568fd0 0x56104d56b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.105+0000 7f8945664740 4 client.17394 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.105+0000 7f8945664740 1 -- 172.21.15.5:0/4216632498 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56104d4c1860 con 0x56104d568fd0 2024-09-15T22:20:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.105+0000 7f8941027640 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56104d568fd0 0x56104d56b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.105+0000 7f8924ff9640 10 client.17394 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.105+0000 7f8941027640 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56104d568fd0 0x56104d56b450 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f824a120793f43 server_cookie=72136bee69d9fbb3 in_seq=0 out_seq=0 2024-09-15T22:20:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8924ff9640 1 -- 172.21.15.5:0/4216632498 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56104d4c1860 con 0x56104d568fd0 2024-09-15T22:20:10.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8924ff9640 10 client.17394 handle_command_reply: tid=0 2024-09-15T22:20:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 1 client.17394 shutdown 2024-09-15T22:20:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 1 -- 172.21.15.5:0/4216632498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d559370 msgr2=0x56104d556610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d559370 0x56104d556610 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8934009ca0 tx=0x7f89340095c0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 1 -- 172.21.15.5:0/4216632498 shutdown_connections 2024-09-15T22:20:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d559370 0x56104d556610 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56104d568fd0 0x56104d56b450 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54fb40 0x56104d556020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 1 --2- 172.21.15.5:0/4216632498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d54e5d0 0x56104d558d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8924ff9640 0 client.17394 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 1 -- 172.21.15.5:0/4216632498 >> 172.21.15.5:0/4216632498 conn(0x56104d517310 msgr2=0x56104d550990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 1 -- 172.21.15.5:0/4216632498 shutdown_connections 2024-09-15T22:20:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 1 -- 172.21.15.5:0/4216632498 wait complete. 2024-09-15T22:20:10.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.106+0000 7f8945664740 20 client.17394 trim_cache size 0 max 16384 2024-09-15T22:20:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.119+0000 7f8945664740 1 Processor -- start 2024-09-15T22:20:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.119+0000 7f8945664740 1 -- start start 2024-09-15T22:20:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8945664740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54e5d0 0x56104d54ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8945664740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d5652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8945664740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d570d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8945664740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1b3cb0 con 0x56104d54fb40 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8945664740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1b1580 con 0x56104d54e5d0 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8945664740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1af3f0 con 0x56104d559370 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8940826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d5652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8941027640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54e5d0 0x56104d54ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8940826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d5652b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33492/0 (socket says 172.21.15.5:33492) 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8940826640 1 -- 172.21.15.5:0/299901777 learned_addr learned my addr 172.21.15.5:0/299901777 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8941828640 1 --2- 172.21.15.5:0/299901777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d570d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8940826640 1 -- 172.21.15.5:0/299901777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 msgr2=0x56104d570d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8940826640 1 --2- 172.21.15.5:0/299901777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d570d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8940826640 1 -- 172.21.15.5:0/299901777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54e5d0 msgr2=0x56104d54ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8940826640 1 --2- 172.21.15.5:0/299901777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54e5d0 0x56104d54ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8940826640 1 -- 172.21.15.5:0/299901777 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104d558990 con 0x56104d54fb40 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8940826640 1 --2- 172.21.15.5:0/299901777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d5652b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f892c004360 tx=0x7f892c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4629c307a71b3ec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.120+0000 7f8926ffd640 1 -- 172.21.15.5:0/299901777 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c011640 con 0x56104d54fb40 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.121+0000 7f8926ffd640 1 -- 172.21.15.5:0/299901777 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f892c020070 con 0x56104d54fb40 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.121+0000 7f8926ffd640 1 -- 172.21.15.5:0/299901777 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c01bc20 con 0x56104d54fb40 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.121+0000 7f8945664740 1 -- 172.21.15.5:0/299901777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 msgr2=0x56104d5652b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.121+0000 7f8945664740 1 --2- 172.21.15.5:0/299901777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d5652b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f892c004360 tx=0x7f892c0039c0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.121+0000 7f8945664740 1 -- 172.21.15.5:0/299901777 shutdown_connections 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.121+0000 7f8945664740 1 --2- 172.21.15.5:0/299901777 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d570d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.121+0000 7f8945664740 1 --2- 172.21.15.5:0/299901777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d5652b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.121+0000 7f8945664740 1 --2- 172.21.15.5:0/299901777 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54e5d0 0x56104d54ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.121+0000 7f8945664740 1 -- 172.21.15.5:0/299901777 >> 172.21.15.5:0/299901777 conn(0x56104d517310 msgr2=0x56104d54a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.121+0000 7f8945664740 1 -- 172.21.15.5:0/299901777 shutdown_connections 2024-09-15T22:20:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.121+0000 7f8945664740 1 -- 172.21.15.5:0/299901777 wait complete. 2024-09-15T22:20:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8945664740 1 -- start start 2024-09-15T22:20:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8945664740 1 Processor -- start 2024-09-15T22:20:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8945664740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54e5d0 0x56104d55cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8945664740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d55d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8945664740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d5d44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8945664740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1afa40 con 0x56104d54fb40 2024-09-15T22:20:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8945664740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1b0eb0 con 0x56104d54e5d0 2024-09-15T22:20:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8945664740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1b0110 con 0x56104d559370 2024-09-15T22:20:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8945664740 10 client.0 _ll_register_callbacks cb 0x7f8934009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8940826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d55d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8940826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d55d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33496/0 (socket says 172.21.15.5:33496) 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8940826640 1 -- 172.21.15.5:0/651366310 learned_addr learned my addr 172.21.15.5:0/651366310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8941828640 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d5d44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8941027640 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54e5d0 0x56104d55cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8940826640 1 -- 172.21.15.5:0/651366310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 msgr2=0x56104d5d44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8940826640 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d5d44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8940826640 1 -- 172.21.15.5:0/651366310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54e5d0 msgr2=0x56104d55cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8940826640 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54e5d0 0x56104d55cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.122+0000 7f8940826640 1 -- 172.21.15.5:0/651366310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104d5d4b00 con 0x56104d54fb40 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.123+0000 7f8941828640 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d5d44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.123+0000 7f8940826640 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d55d0a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f892c004650 tx=0x7f892c01bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e955d350ddf549e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.123+0000 7f8924ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:10.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.123+0000 7f8924ff9640 1 -- 172.21.15.5:0/651366310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c02e470 con 0x56104d54fb40 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.123+0000 7f8924ff9640 1 -- 172.21.15.5:0/651366310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f892c02e610 con 0x56104d54fb40 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.123+0000 7f8945664740 1 -- 172.21.15.5:0/651366310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f892c020070 con 0x56104d54fb40 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.123+0000 7f8924ff9640 1 -- 172.21.15.5:0/651366310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f892c037050 con 0x56104d54fb40 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.123+0000 7f8945664740 10 client.17400 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.123+0000 7f8945664740 1 -- 172.21.15.5:0/651366310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56104d5dbe50 con 0x56104d54fb40 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.123+0000 7f8924ff9640 1 -- 172.21.15.5:0/651366310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f892c020070 con 0x56104d54fb40 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.123+0000 7f8924ff9640 1 -- 172.21.15.5:0/651366310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f892c0118c0 con 0x56104d54fb40 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.124+0000 7f8945664740 10 client.17400 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.124+0000 7f8945664740 10 client.17400 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.124+0000 7f8945664740 20 client.17400 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.124+0000 7f8945664740 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56104d5dc5a0 0x56104d5de9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:10.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.124+0000 7f8945664740 4 client.17400 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.124+0000 7f8945664740 1 -- 172.21.15.5:0/651366310 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56104d4bbd70 con 0x56104d5dc5a0 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.124+0000 7f8941027640 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56104d5dc5a0 0x56104d5de9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.124+0000 7f8924ff9640 10 client.17400 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.124+0000 7f8941027640 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56104d5dc5a0 0x56104d5de9c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1a9ffe6b91aa5bd server_cookie=9e75ae0986159954 in_seq=0 out_seq=0 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.124+0000 7f8924ff9640 1 -- 172.21.15.5:0/651366310 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56104d4bbd70 con 0x56104d5dc5a0 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.124+0000 7f8924ff9640 10 client.17400 handle_command_reply: tid=0 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 1 client.17400 shutdown 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 1 -- 172.21.15.5:0/651366310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 msgr2=0x56104d55d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d55d0a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f892c004650 tx=0x7f892c01bf30 comp rx=0 tx=0).stop 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 1 -- 172.21.15.5:0/651366310 shutdown_connections 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56104d559370 0x56104d5d44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56104d54fb40 0x56104d55d0a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56104d5dc5a0 0x56104d5de9c0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 1 --2- 172.21.15.5:0/651366310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56104d54e5d0 0x56104d55cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8924ff9640 0 client.17400 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 1 -- 172.21.15.5:0/651366310 >> 172.21.15.5:0/651366310 conn(0x56104d517310 msgr2=0x56104d5548f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 1 -- 172.21.15.5:0/651366310 shutdown_connections 2024-09-15T22:20:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 1 -- 172.21.15.5:0/651366310 wait complete. 2024-09-15T22:20:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f8945664740 20 client.17400 trim_cache size 0 max 16384 2024-09-15T22:20:10.138 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:20:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f894328b640 1 -- 172.21.15.5:0/1592600602 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89280a33b0 msgr2=0x7f89280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f894328b640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89280a33b0 0x7f89280a5870 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8934009a60 tx=0x7f893402a8e0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f894328b640 1 -- 172.21.15.5:0/1592600602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f893c1002f0 msgr2=0x7f893c101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.125+0000 7f894328b640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f893c1002f0 0x7f893c101270 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f893800e9f0 tx=0x7f893800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.126+0000 7f894328b640 1 -- 172.21.15.5:0/1592600602 shutdown_connections 2024-09-15T22:20:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.126+0000 7f894328b640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89280a33b0 0x7f89280a5870 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.126+0000 7f894328b640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f893c106f50 0x7f893c101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.126+0000 7f894328b640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f893c100ca0 0x7f893c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.126+0000 7f894328b640 1 --2- 172.21.15.5:0/1592600602 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f893c1002f0 0x7f893c101270 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:10.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.126+0000 7f894328b640 1 -- 172.21.15.5:0/1592600602 >> 172.21.15.5:0/1592600602 conn(0x7f893c077210 msgr2=0x7f893c105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:10.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.126+0000 7f894328b640 1 -- 172.21.15.5:0/1592600602 shutdown_connections 2024-09-15T22:20:10.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:10.126+0000 7f894328b640 1 -- 172.21.15.5:0/1592600602 wait complete. 2024-09-15T22:20:11.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:11 smithi099 ceph-mon[34892]: pgmap v351: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.8 MiB/s wr, 349 op/s 2024-09-15T22:20:11.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:11 smithi129 ceph-mon[35314]: pgmap v351: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.8 MiB/s wr, 349 op/s 2024-09-15T22:20:11.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:11 smithi005 ceph-mon[30944]: pgmap v351: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 4.8 MiB/s wr, 349 op/s 2024-09-15T22:20:11.919 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6c8391fc', quiesce took 14.5 seconds. Will release after: 7.350000000000001 2024-09-15T22:20:12.181 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:20:12.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:20:12.440 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.928+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/1134425345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8102570 msgr2=0x7fc3b81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.928+0000 7fc3bcd3f640 1 --2- 172.21.15.5:0/1134425345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8102570 0x7fc3b81029b0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009900 tx=0x7fc3ac02a480 comp rx=0 tx=0).stop 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.928+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/1134425345 shutdown_connections 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.928+0000 7fc3bcd3f640 1 --2- 172.21.15.5:0/1134425345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3b8102ef0 0x7fc3b810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.928+0000 7fc3bcd3f640 1 --2- 172.21.15.5:0/1134425345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8102570 0x7fc3b81029b0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.928+0000 7fc3bcd3f640 1 --2- 172.21.15.5:0/1134425345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3b8100e20 0x7fc3b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.928+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/1134425345 >> 172.21.15.5:0/1134425345 conn(0x7fc3b80fc7d0 msgr2=0x7fc3b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.928+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/1134425345 shutdown_connections 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.928+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/1134425345 wait complete. 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.928+0000 7fc3bcd3f640 1 Processor -- start 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3bcd3f640 1 -- start start 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3bcd3f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3b8100e20 0x7fc3b80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3bcd3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8102570 0x7fc3b80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3bcd3f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3b8102ef0 0x7fc3b80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3bcd3f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8110f50 con 0x7fc3b8102570 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3bcd3f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8110dd0 con 0x7fc3b8100e20 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3bcd3f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81110d0 con 0x7fc3b8102ef0 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8102570 0x7fc3b80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8102570 0x7fc3b80fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33522/0 (socket says 172.21.15.5:33522) 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3b5d9b640 1 -- 172.21.15.5:0/3777173573 learned_addr learned my addr 172.21.15.5:0/3777173573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3b659c640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3b8100e20 0x7fc3b80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3b6d9d640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3b8102ef0 0x7fc3b80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3b5d9b640 1 -- 172.21.15.5:0/3777173573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3b8102ef0 msgr2=0x7fc3b80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3b8102ef0 0x7fc3b80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3b5d9b640 1 -- 172.21.15.5:0/3777173573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3b8100e20 msgr2=0x7fc3b80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3b8100e20 0x7fc3b80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3b5d9b640 1 -- 172.21.15.5:0/3777173573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b81a69c0 con 0x7fc3b8102570 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.929+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8102570 0x7fc3b80fd1c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac006010 tx=0x7fc3ac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.930+0000 7fc3a77fe640 1 -- 172.21.15.5:0/3777173573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac03f070 con 0x7fc3b8102570 2024-09-15T22:20:12.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.930+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/3777173573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b81a6c50 con 0x7fc3b8102570 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.930+0000 7fc3a77fe640 1 -- 172.21.15.5:0/3777173573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3ac031c30 con 0x7fc3b8102570 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.930+0000 7fc3a77fe640 1 -- 172.21.15.5:0/3777173573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac035030 con 0x7fc3b8102570 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.930+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/3777173573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b81a7130 con 0x7fc3b8102570 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.932+0000 7fc3a77fe640 1 -- 172.21.15.5:0/3777173573 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc3ac042070 con 0x7fc3b8102570 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.932+0000 7fc3bf118740 1 Processor -- start 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.932+0000 7fc3bf118740 1 -- start start 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b510f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b511480 0x56500b51b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b51bd40 0x56500b51e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3bf118740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56500b476280 con 0x56500b510b60 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3bf118740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56500b41db10 con 0x56500b511480 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3bf118740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56500b472240 con 0x56500b51bd40 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b510f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b510f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33532/0 (socket says 172.21.15.5:33532) 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b6d9d640 1 -- 172.21.15.5:0/1458169212 learned_addr learned my addr 172.21.15.5:0/1458169212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b659c640 1 --2- 172.21.15.5:0/1458169212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b511480 0x56500b51b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3a77fe640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3a00a33b0 0x7fc3a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/1458169212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b51bd40 0x56500b51e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b6d9d640 1 -- 172.21.15.5:0/1458169212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b51bd40 msgr2=0x56500b51e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b6d9d640 1 --2- 172.21.15.5:0/1458169212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b51bd40 0x56500b51e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b6d9d640 1 -- 172.21.15.5:0/1458169212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b511480 msgr2=0x56500b51b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b6d9d640 1 --2- 172.21.15.5:0/1458169212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b511480 0x56500b51b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b6d9d640 1 -- 172.21.15.5:0/1458169212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56500b42a1f0 con 0x56500b510b60 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b659c640 1 --2- 172.21.15.5:0/1458169212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b511480 0x56500b51b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3a77fe640 1 -- 172.21.15.5:0/3777173573 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7fc3ac0f08d0 con 0x7fc3b8102570 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b6d9d640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3a00a33b0 0x7fc3a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.933+0000 7fc3b6d9d640 1 --2- 172.21.15.5:0/1458169212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b510f40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0010440 tx=0x7fc3b0010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed3f44eff4bcc5b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.934+0000 7fc3a4ff9640 1 -- 172.21.15.5:0/1458169212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b001d400 con 0x56500b510b60 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.934+0000 7fc3a4ff9640 1 -- 172.21.15.5:0/1458169212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b001da80 con 0x56500b510b60 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.934+0000 7fc3a4ff9640 1 -- 172.21.15.5:0/1458169212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b001c940 con 0x56500b510b60 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.934+0000 7fc3bf118740 1 -- 172.21.15.5:0/1458169212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 msgr2=0x56500b510f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.934+0000 7fc3b6d9d640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3a00a33b0 0x7fc3a00a5870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0021000 tx=0x7fc3b001e000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.934+0000 7fc3bf118740 1 --2- 172.21.15.5:0/1458169212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b510f40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0010440 tx=0x7fc3b0010900 comp rx=0 tx=0).stop 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 -- 172.21.15.5:0/1458169212 shutdown_connections 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 --2- 172.21.15.5:0/1458169212 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b51bd40 0x56500b51e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 --2- 172.21.15.5:0/1458169212 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b511480 0x56500b51b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 --2- 172.21.15.5:0/1458169212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b510f40 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 -- 172.21.15.5:0/1458169212 >> 172.21.15.5:0/1458169212 conn(0x56500b50d0f0 msgr2=0x56500b50d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 -- 172.21.15.5:0/1458169212 shutdown_connections 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 -- 172.21.15.5:0/1458169212 wait complete. 2024-09-15T22:20:12.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 -- start start 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 Processor -- start 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b518c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b511480 0x56500b519150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b51bd40 0x56500b519690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56500b51e910 con 0x56500b510b60 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56500b430f10 con 0x56500b51bd40 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56500b51ea90 con 0x56500b511480 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3bf118740 10 client.0 _ll_register_callbacks cb 0x56500b3f9a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3b659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b518c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3b659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b518c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33546/0 (socket says 172.21.15.5:33546) 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.935+0000 7fc3b659c640 1 -- 172.21.15.5:0/3585574429 learned_addr learned my addr 172.21.15.5:0/3585574429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b511480 0x56500b519150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc3b6d9d640 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b51bd40 0x56500b519690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc3b659c640 1 -- 172.21.15.5:0/3585574429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b511480 msgr2=0x56500b519150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc3b659c640 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b511480 0x56500b519150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc3b659c640 1 -- 172.21.15.5:0/3585574429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b51bd40 msgr2=0x56500b519690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc3b659c640 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b51bd40 0x56500b519690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc3b659c640 1 -- 172.21.15.5:0/3585574429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56500b1e76a0 con 0x56500b510b60 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc3b659c640 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b518c10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80127b0 tx=0x7fc3a800af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2120192480259754 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc39dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc39dffb640 1 -- 172.21.15.5:0/3585574429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8010dc0 con 0x56500b510b60 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc39dffb640 1 -- 172.21.15.5:0/3585574429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7fc3a8005020 con 0x56500b510b60 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc3bf118740 1 -- 172.21.15.5:0/3585574429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3b001da80 con 0x56500b510b60 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc39dffb640 1 -- 172.21.15.5:0/3585574429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc3a8025050 con 0x56500b510b60 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc3bf118740 10 client.17412 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc3bf118740 1 -- 172.21.15.5:0/3585574429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56500b52a570 con 0x56500b510b60 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.936+0000 7fc39dffb640 1 -- 172.21.15.5:0/3585574429 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3b001da80 con 0x56500b510b60 2024-09-15T22:20:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.937+0000 7fc39dffb640 1 -- 172.21.15.5:0/3585574429 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc3a801a430 con 0x56500b510b60 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.937+0000 7fc3bf118740 10 client.17412 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.937+0000 7fc3bf118740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.937+0000 7fc3bf118740 20 client.17412 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.937+0000 7fc3bf118740 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56500b52ba30 0x56500b52deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.937+0000 7fc3bf118740 4 client.17412 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.937+0000 7fc3bf118740 1 -- 172.21.15.5:0/3585574429 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56500b4837f0 con 0x56500b52ba30 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.937+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56500b52ba30 0x56500b52deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.937+0000 7fc39dffb640 10 client.17412 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.937+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56500b52ba30 0x56500b52deb0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a4d5ab5efc9204e server_cookie=7016be4809f39408 in_seq=0 out_seq=0 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc39dffb640 1 -- 172.21.15.5:0/3585574429 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56500b4837f0 con 0x56500b52ba30 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc39dffb640 10 client.17412 handle_command_reply: tid=0 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 1 client.17412 shutdown 2024-09-15T22:20:12.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 1 -- 172.21.15.5:0/3585574429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 msgr2=0x56500b518c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b518c10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80127b0 tx=0x7fc3a800af90 comp rx=0 tx=0).stop 2024-09-15T22:20:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 1 -- 172.21.15.5:0/3585574429 shutdown_connections 2024-09-15T22:20:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56500b52ba30 0x56500b52deb0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b51bd40 0x56500b519690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b511480 0x56500b519150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc39dffb640 0 client.17412 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 1 --2- 172.21.15.5:0/3585574429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b518c10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 1 -- 172.21.15.5:0/3585574429 >> 172.21.15.5:0/3585574429 conn(0x56500b50d0f0 msgr2=0x56500b50c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 1 -- 172.21.15.5:0/3585574429 shutdown_connections 2024-09-15T22:20:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 1 -- 172.21.15.5:0/3585574429 wait complete. 2024-09-15T22:20:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.938+0000 7fc3bf118740 20 client.17412 trim_cache size 0 max 16384 2024-09-15T22:20:12.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3bf118740 1 Processor -- start 2024-09-15T22:20:12.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3bf118740 1 -- start start 2024-09-15T22:20:12.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b510b60 0x56500b513610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b511480 0x56500b5337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b51bd40 0x56500b533d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3bf118740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81b3bf0 con 0x56500b511480 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3bf118740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81b14c0 con 0x56500b510b60 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3bf118740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81af330 con 0x56500b51bd40 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b511480 0x56500b5337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3b5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b511480 0x56500b5337d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33552/0 (socket says 172.21.15.5:33552) 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3b5d9b640 1 -- 172.21.15.5:0/967085059 learned_addr learned my addr 172.21.15.5:0/967085059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3b5d9b640 1 -- 172.21.15.5:0/967085059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b51bd40 msgr2=0x56500b533d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3b659c640 1 --2- 172.21.15.5:0/967085059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b510b60 0x56500b513610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3b6d9d640 1 --2- 172.21.15.5:0/967085059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b51bd40 0x56500b533d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/967085059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b51bd40 0x56500b533d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3b5d9b640 1 -- 172.21.15.5:0/967085059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b510b60 msgr2=0x56500b513610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/967085059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b510b60 0x56500b513610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3b5d9b640 1 -- 172.21.15.5:0/967085059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56500b50cf20 con 0x56500b511480 2024-09-15T22:20:12.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.956+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/967085059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b511480 0x56500b5337d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0b8c70 tx=0x7fc3ac0db040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ca75a71cbace40 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc39ffff640 1 -- 172.21.15.5:0/967085059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac049040 con 0x56500b511480 2024-09-15T22:20:12.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc39ffff640 1 -- 172.21.15.5:0/967085059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3ac0bc860 con 0x56500b511480 2024-09-15T22:20:12.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc39ffff640 1 -- 172.21.15.5:0/967085059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0b8ed0 con 0x56500b511480 2024-09-15T22:20:12.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc3bf118740 1 -- 172.21.15.5:0/967085059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b511480 msgr2=0x56500b5337d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc3bf118740 1 --2- 172.21.15.5:0/967085059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b511480 0x56500b5337d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac0b8c70 tx=0x7fc3ac0db040 comp rx=0 tx=0).stop 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc3bf118740 1 -- 172.21.15.5:0/967085059 shutdown_connections 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc3bf118740 1 --2- 172.21.15.5:0/967085059 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b51bd40 0x56500b533d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc3bf118740 1 --2- 172.21.15.5:0/967085059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b511480 0x56500b5337d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc3bf118740 1 --2- 172.21.15.5:0/967085059 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b510b60 0x56500b513610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc3bf118740 1 -- 172.21.15.5:0/967085059 >> 172.21.15.5:0/967085059 conn(0x56500b50d0f0 msgr2=0x56500b51ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc3bf118740 1 -- 172.21.15.5:0/967085059 shutdown_connections 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.957+0000 7fc3bf118740 1 -- 172.21.15.5:0/967085059 wait complete. 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3bf118740 1 -- start start 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3bf118740 1 Processor -- start 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b520530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b511480 0x56500b520a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3bf118740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b51bd40 0x56500b523b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3bf118740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81af980 con 0x56500b510b60 2024-09-15T22:20:12.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3bf118740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81b0df0 con 0x56500b51bd40 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3bf118740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b81b0050 con 0x56500b511480 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3bf118740 10 client.0 _ll_register_callbacks cb 0x7fc3a800a670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3b659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b520530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3b659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b520530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33562/0 (socket says 172.21.15.5:33562) 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3b659c640 1 -- 172.21.15.5:0/2805602490 learned_addr learned my addr 172.21.15.5:0/2805602490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3b6d9d640 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b51bd40 0x56500b523b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b511480 0x56500b520a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3b659c640 1 -- 172.21.15.5:0/2805602490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b511480 msgr2=0x56500b520a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3b659c640 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b511480 0x56500b520a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3b659c640 1 -- 172.21.15.5:0/2805602490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b51bd40 msgr2=0x56500b523b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3b659c640 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b51bd40 0x56500b523b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3b659c640 1 -- 172.21.15.5:0/2805602490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56500b507680 con 0x56500b510b60 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.958+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b511480 0x56500b520a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:12.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc3b659c640 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b520530 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8000c00 tx=0x7fc3a801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9f5fe688276c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc39dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc39dffb640 1 -- 172.21.15.5:0/2805602490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a802d070 con 0x56500b510b60 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc39dffb640 1 -- 172.21.15.5:0/2805602490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7fc3a8010dc0 con 0x56500b510b60 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc3bf118740 1 -- 172.21.15.5:0/2805602490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3ac0bc860 con 0x56500b510b60 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc39dffb640 1 -- 172.21.15.5:0/2805602490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc3a8038050 con 0x56500b510b60 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc3bf118740 10 client.17418 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc3bf118740 1 -- 172.21.15.5:0/2805602490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56500b5971f0 con 0x56500b510b60 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc39dffb640 1 -- 172.21.15.5:0/2805602490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3ac0bc860 con 0x56500b510b60 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc39dffb640 1 -- 172.21.15.5:0/2805602490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc3a8003960 con 0x56500b510b60 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc3bf118740 10 client.17418 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc3bf118740 10 client.17418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc3bf118740 20 client.17418 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc3bf118740 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56500b597910 0x56500b599d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc3bf118740 4 client.17418 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:20:12.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.959+0000 7fc3bf118740 1 -- 172.21.15.5:0/2805602490 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56500b47dd00 con 0x56500b597910 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.960+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56500b597910 0x56500b599d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.960+0000 7fc3b5d9b640 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56500b597910 0x56500b599d00 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=211c72bfa0f1cf1e server_cookie=5f8e01d149fc23ac in_seq=0 out_seq=0 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.960+0000 7fc39dffb640 10 client.17418 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.961+0000 7fc39dffb640 1 -- 172.21.15.5:0/2805602490 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56500b47dd00 con 0x56500b597910 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.961+0000 7fc39dffb640 10 client.17418 handle_command_reply: tid=0 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.961+0000 7fc3bf118740 1 client.17418 shutdown 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.961+0000 7fc3bf118740 1 -- 172.21.15.5:0/2805602490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 msgr2=0x56500b520530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc3bf118740 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b520530 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8000c00 tx=0x7fc3a801ad70 comp rx=0 tx=0).stop 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc3bf118740 1 -- 172.21.15.5:0/2805602490 shutdown_connections 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc3bf118740 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56500b597910 0x56500b599d00 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc3bf118740 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56500b51bd40 0x56500b523b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc3bf118740 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56500b511480 0x56500b520a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc39dffb640 0 client.17418 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:12.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc3bf118740 1 --2- 172.21.15.5:0/2805602490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56500b510b60 0x56500b520530 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc3bf118740 1 -- 172.21.15.5:0/2805602490 >> 172.21.15.5:0/2805602490 conn(0x56500b50d0f0 msgr2=0x56500b526b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc3bf118740 1 -- 172.21.15.5:0/2805602490 shutdown_connections 2024-09-15T22:20:12.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc3bf118740 1 -- 172.21.15.5:0/2805602490 wait complete. 2024-09-15T22:20:12.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc3bf118740 20 client.17418 trim_cache size 0 max 16384 2024-09-15T22:20:12.969 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:20:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:20:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "747d64f3-e523-4d82-9ad1-e7ed30b11f59", 2024-09-15T22:20:12.969 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:20:12.969 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:20:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.962+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/3777173573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3a00a33b0 msgr2=0x7fc3a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.963+0000 7fc3bcd3f640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3a00a33b0 0x7fc3a00a5870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0021000 tx=0x7fc3b001e000 comp rx=0 tx=0).stop 2024-09-15T22:20:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.963+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/3777173573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8102570 msgr2=0x7fc3b80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.963+0000 7fc3bcd3f640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8102570 0x7fc3b80fd1c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac006010 tx=0x7fc3ac003c10 comp rx=0 tx=0).stop 2024-09-15T22:20:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.963+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/3777173573 shutdown_connections 2024-09-15T22:20:12.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.963+0000 7fc3bcd3f640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc3a00a33b0 0x7fc3a00a5870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.963+0000 7fc3bcd3f640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc3b8102ef0 0x7fc3b80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.963+0000 7fc3bcd3f640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3b8102570 0x7fc3b80fd1c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.963+0000 7fc3bcd3f640 1 --2- 172.21.15.5:0/3777173573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc3b8100e20 0x7fc3b80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.963+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/3777173573 >> 172.21.15.5:0/3777173573 conn(0x7fc3b80fc7d0 msgr2=0x7fc3b806a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.963+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/3777173573 shutdown_connections 2024-09-15T22:20:12.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:12.963+0000 7fc3bcd3f640 1 -- 172.21.15.5:0/3777173573 wait complete. 2024-09-15T22:20:13.755 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:20:13.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:13 smithi005 ceph-mon[30944]: pgmap v352: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.7 MiB/s wr, 516 op/s 2024-09-15T22:20:13.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:13 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:20:13.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:13 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:13.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:13 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:20:13.779 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:13 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:13.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:13 smithi099 ceph-mon[34892]: pgmap v352: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.7 MiB/s wr, 516 op/s 2024-09-15T22:20:13.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:13 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:20:13.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:13 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:13.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:13 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:20:13.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:13 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:13 smithi129 ceph-mon[35314]: pgmap v352: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 6.7 MiB/s wr, 516 op/s 2024-09-15T22:20:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:13 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:20:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:13 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:13 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:20:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:13 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:20:14.014 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.451+0000 7f4362991640 1 -- 172.21.15.5:0/4242065523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c106f10 msgr2=0x7f435c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.451+0000 7f4362991640 1 --2- 172.21.15.5:0/4242065523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c106f10 0x7f435c10ea90 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4354009ca0 tx=0x7f435402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.451+0000 7f4362991640 1 -- 172.21.15.5:0/4242065523 shutdown_connections 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.451+0000 7f4362991640 1 --2- 172.21.15.5:0/4242065523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c106f10 0x7f435c10ea90 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.451+0000 7f4362991640 1 --2- 172.21.15.5:0/4242065523 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f435c102790 0x7f435c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.451+0000 7f4362991640 1 --2- 172.21.15.5:0/4242065523 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f435c101de0 0x7f435c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.451+0000 7f4362991640 1 -- 172.21.15.5:0/4242065523 >> 172.21.15.5:0/4242065523 conn(0x7f435c0fd8c0 msgr2=0x7f435c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.451+0000 7f4362991640 1 -- 172.21.15.5:0/4242065523 shutdown_connections 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.451+0000 7f4362991640 1 -- 172.21.15.5:0/4242065523 wait complete. 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.451+0000 7f4362991640 1 Processor -- start 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.451+0000 7f4362991640 1 -- start start 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4362991640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f435c101de0 0x7f435c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4362991640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f435c102790 0x7f435c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4362991640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c106f10 0x7f435c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4362991640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f435c111f70 con 0x7f435c106f10 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4362991640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f435c111df0 con 0x7f435c102790 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4362991640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1120f0 con 0x7f435c101de0 2024-09-15T22:20:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4360f2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c106f10 0x7f435c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4360f2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c106f10 0x7f435c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41426/0 (socket says 172.21.15.5:41426) 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4360f2e640 1 -- 172.21.15.5:0/2125192511 learned_addr learned my addr 172.21.15.5:0/2125192511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f435bfff640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f435c101de0 0x7f435c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f435b7fe640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f435c102790 0x7f435c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4360f2e640 1 -- 172.21.15.5:0/2125192511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f435c101de0 msgr2=0x7f435c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4360f2e640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f435c101de0 0x7f435c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4360f2e640 1 -- 172.21.15.5:0/2125192511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f435c102790 msgr2=0x7f435c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4360f2e640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f435c102790 0x7f435c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.452+0000 7f4360f2e640 1 -- 172.21.15.5:0/2125192511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c1a68b0 con 0x7f435c106f10 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.453+0000 7f4360f2e640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c106f10 0x7f435c074f70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4354031a20 tx=0x7f4354031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.453+0000 7f43597fa640 1 -- 172.21.15.5:0/2125192511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354003c30 con 0x7f435c106f10 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.453+0000 7f43597fa640 1 -- 172.21.15.5:0/2125192511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4354003dd0 con 0x7f435c106f10 2024-09-15T22:20:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.453+0000 7f4362991640 1 -- 172.21.15.5:0/2125192511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c1a6b40 con 0x7f435c106f10 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.453+0000 7f43597fa640 1 -- 172.21.15.5:0/2125192511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435402fcf0 con 0x7f435c106f10 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.453+0000 7f4362991640 1 -- 172.21.15.5:0/2125192511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c1a7080 con 0x7f435c106f10 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.455+0000 7f4364d6a740 1 Processor -- start 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.455+0000 7f43597fa640 1 -- 172.21.15.5:0/2125192511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4354004050 con 0x7f435c106f10 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f4364d6a740 1 -- start start 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x56171746d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746ddc0 0x561717478140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717478680 0x56171747aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f4364d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5617173d32e0 con 0x56171746d4a0 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f4364d6a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56171737ace0 con 0x561717478680 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f4364d6a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5617173cf2a0 con 0x56171746ddc0 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x56171746d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x56171746d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41434/0 (socket says 172.21.15.5:41434) 2024-09-15T22:20:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f435b7fe640 1 -- 172.21.15.5:0/3106903747 learned_addr learned my addr 172.21.15.5:0/3106903747 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f435bfff640 1 --2- 172.21.15.5:0/3106903747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746ddc0 0x561717478140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f43597fa640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43440a33b0 0x7f43440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f435b7fe640 1 -- 172.21.15.5:0/3106903747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746ddc0 msgr2=0x561717478140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f435b7fe640 1 --2- 172.21.15.5:0/3106903747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746ddc0 0x561717478140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f435b7fe640 1 -- 172.21.15.5:0/3106903747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717478680 msgr2=0x56171747aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f435b7fe640 1 --2- 172.21.15.5:0/3106903747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717478680 0x56171747aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f435b7fe640 1 -- 172.21.15.5:0/3106903747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171735f440 con 0x56171746d4a0 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f4360f2e640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43440a33b0 0x7f43440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.456+0000 7f435b7fe640 1 --2- 172.21.15.5:0/3106903747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x56171746d880 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f43500058a0 tx=0x7f43500127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb60aa3c2111fefa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.457+0000 7f43597fa640 1 -- 172.21.15.5:0/2125192511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f43540f12b0 con 0x7f435c106f10 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.457+0000 7f43427fc640 1 -- 172.21.15.5:0/3106903747 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43500120d0 con 0x56171746d4a0 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.457+0000 7f43427fc640 1 -- 172.21.15.5:0/3106903747 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4350016070 con 0x56171746d4a0 2024-09-15T22:20:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.457+0000 7f43427fc640 1 -- 172.21.15.5:0/3106903747 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43500107b0 con 0x56171746d4a0 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.457+0000 7f4364d6a740 1 -- 172.21.15.5:0/3106903747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 msgr2=0x56171746d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.457+0000 7f4364d6a740 1 --2- 172.21.15.5:0/3106903747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x56171746d880 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f43500058a0 tx=0x7f43500127c0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.457+0000 7f4360f2e640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43440a33b0 0x7f43440a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f43540bd030 tx=0x7f43540dd040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 -- 172.21.15.5:0/3106903747 shutdown_connections 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 --2- 172.21.15.5:0/3106903747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717478680 0x56171747aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 --2- 172.21.15.5:0/3106903747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746ddc0 0x561717478140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 --2- 172.21.15.5:0/3106903747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x56171746d880 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 -- 172.21.15.5:0/3106903747 >> 172.21.15.5:0/3106903747 conn(0x561717469a30 msgr2=0x561717469e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 -- 172.21.15.5:0/3106903747 shutdown_connections 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 -- 172.21.15.5:0/3106903747 wait complete. 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 -- start start 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 Processor -- start 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x561717477f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746ddc0 0x5617174751f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717478680 0x561717475730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56171747b250 con 0x56171746d4a0 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56171738e050 con 0x561717478680 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56171747b3d0 con 0x56171746ddc0 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f4364d6a740 10 client.0 _ll_register_callbacks cb 0x5617173ce160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x561717477f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x561717477f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41450/0 (socket says 172.21.15.5:41450) 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.458+0000 7f435b7fe640 1 -- 172.21.15.5:0/3155152675 learned_addr learned my addr 172.21.15.5:0/3155152675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f435b7fe640 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717478680 0x561717475730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f435bfff640 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746ddc0 0x5617174751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f435b7fe640 1 -- 172.21.15.5:0/3155152675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746ddc0 msgr2=0x5617174751f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f435b7fe640 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746ddc0 0x5617174751f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f435b7fe640 1 -- 172.21.15.5:0/3155152675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717478680 msgr2=0x561717475730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f435b7fe640 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717478680 0x561717475730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f435b7fe640 1 -- 172.21.15.5:0/3155152675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617171443c0 con 0x56171746d4a0 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f435b7fe640 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x561717477f40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4350010490 tx=0x7f435001f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a117d2fb41dc1ffc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f433b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f433b7fe640 1 -- 172.21.15.5:0/3155152675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43500035f0 con 0x56171746d4a0 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f4364d6a740 1 -- 172.21.15.5:0/3155152675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4350016070 con 0x56171746d4a0 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f433b7fe640 1 -- 172.21.15.5:0/3155152675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f4350003790 con 0x56171746d4a0 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f433b7fe640 1 -- 172.21.15.5:0/3155152675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4350003990 con 0x56171746d4a0 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f4364d6a740 10 client.17430 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f4364d6a740 1 -- 172.21.15.5:0/3155152675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561717486c60 con 0x56171746d4a0 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.459+0000 7f433b7fe640 1 -- 172.21.15.5:0/3155152675 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4350016070 con 0x56171746d4a0 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.460+0000 7f433b7fe640 1 -- 172.21.15.5:0/3155152675 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f435001bb10 con 0x56171746d4a0 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.460+0000 7f4364d6a740 10 client.17430 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.460+0000 7f4364d6a740 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.460+0000 7f4364d6a740 20 client.17430 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.460+0000 7f4364d6a740 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561717488070 0x56171748a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.460+0000 7f4364d6a740 4 client.17430 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.460+0000 7f4364d6a740 1 -- 172.21.15.5:0/3155152675 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617173e0850 con 0x561717488070 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.460+0000 7f435bfff640 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561717488070 0x56171748a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.460+0000 7f433b7fe640 10 client.17430 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.460+0000 7f435bfff640 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561717488070 0x56171748a460 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cfad82cbd232bb server_cookie=cefdc5afb187580a in_seq=0 out_seq=0 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.461+0000 7f433b7fe640 1 -- 172.21.15.5:0/3155152675 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617173e0850 con 0x561717488070 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.461+0000 7f433b7fe640 10 client.17430 handle_command_reply: tid=0 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.461+0000 7f4364d6a740 1 client.17430 shutdown 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f4364d6a740 1 -- 172.21.15.5:0/3155152675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 msgr2=0x561717477f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f4364d6a740 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x561717477f40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f4350010490 tx=0x7f435001f040 comp rx=0 tx=0).stop 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f4364d6a740 1 -- 172.21.15.5:0/3155152675 shutdown_connections 2024-09-15T22:20:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f4364d6a740 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561717488070 0x56171748a460 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f4364d6a740 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717478680 0x561717475730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f4364d6a740 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746ddc0 0x5617174751f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f4364d6a740 1 --2- 172.21.15.5:0/3155152675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746d4a0 0x561717477f40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f4364d6a740 1 -- 172.21.15.5:0/3155152675 >> 172.21.15.5:0/3155152675 conn(0x561717469a30 msgr2=0x56171746f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f433b7fe640 0 client.17430 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f4364d6a740 1 -- 172.21.15.5:0/3155152675 shutdown_connections 2024-09-15T22:20:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f4364d6a740 1 -- 172.21.15.5:0/3155152675 wait complete. 2024-09-15T22:20:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.462+0000 7f4364d6a740 20 client.17430 trim_cache size 0 max 16384 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f4364d6a740 1 Processor -- start 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f4364d6a740 1 -- start start 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717469a30 0x561717473ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746d4a0 0x56171748fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x5617174900c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f4364d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1b3be0 con 0x56171746ddc0 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f4364d6a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1b14b0 con 0x561717469a30 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f4364d6a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1af320 con 0x56171746d4a0 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x5617174900c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x5617174900c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41460/0 (socket says 172.21.15.5:41460) 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f435b7fe640 1 -- 172.21.15.5:0/73581443 learned_addr learned my addr 172.21.15.5:0/73581443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f435b7fe640 1 --2- 172.21.15.5:0/73581443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717469a30 0x561717473ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.479+0000 7f435bfff640 1 --2- 172.21.15.5:0/73581443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746d4a0 0x56171748fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.480+0000 7f435b7fe640 1 -- 172.21.15.5:0/73581443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746d4a0 msgr2=0x56171748fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.480+0000 7f435b7fe640 1 --2- 172.21.15.5:0/73581443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746d4a0 0x56171748fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.480+0000 7f435b7fe640 1 -- 172.21.15.5:0/73581443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717469a30 msgr2=0x561717473ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.480+0000 7f435b7fe640 1 --2- 172.21.15.5:0/73581443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717469a30 0x561717473ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.480+0000 7f435b7fe640 1 -- 172.21.15.5:0/73581443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617174741a0 con 0x56171746ddc0 2024-09-15T22:20:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.480+0000 7f435bfff640 1 --2- 172.21.15.5:0/73581443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746d4a0 0x56171748fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.480+0000 7f435b7fe640 1 --2- 172.21.15.5:0/73581443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x5617174900c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f435000b560 tx=0x7f435001bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1e3e4c46a1f3b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.480+0000 7f4341ffb640 1 -- 172.21.15.5:0/73581443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435000b690 con 0x56171746ddc0 2024-09-15T22:20:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.480+0000 7f4341ffb640 1 -- 172.21.15.5:0/73581443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4350016040 con 0x56171746ddc0 2024-09-15T22:20:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.480+0000 7f4341ffb640 1 -- 172.21.15.5:0/73581443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350003d90 con 0x56171746ddc0 2024-09-15T22:20:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 -- 172.21.15.5:0/73581443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 msgr2=0x5617174900c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 --2- 172.21.15.5:0/73581443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x5617174900c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f435000b560 tx=0x7f435001bf90 comp rx=0 tx=0).stop 2024-09-15T22:20:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 -- 172.21.15.5:0/73581443 shutdown_connections 2024-09-15T22:20:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 --2- 172.21.15.5:0/73581443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x5617174900c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 --2- 172.21.15.5:0/73581443 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56171746d4a0 0x56171748fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 --2- 172.21.15.5:0/73581443 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561717469a30 0x561717473ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 -- 172.21.15.5:0/73581443 >> 172.21.15.5:0/73581443 conn(0x561717475890 msgr2=0x561717468b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 -- 172.21.15.5:0/73581443 shutdown_connections 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 -- 172.21.15.5:0/73581443 wait complete. 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 -- start start 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 Processor -- start 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561717469a30 0x561717480af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56171746d4a0 0x56171747de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x56171747e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1af970 con 0x56171746ddc0 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1b0de0 con 0x56171746d4a0 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1b0040 con 0x561717469a30 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.481+0000 7f4364d6a740 10 client.0 _ll_register_callbacks cb 0x7f435000bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x56171747e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x56171747e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41466/0 (socket says 172.21.15.5:41466) 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f435b7fe640 1 -- 172.21.15.5:0/311691315 learned_addr learned my addr 172.21.15.5:0/311691315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f435b7fe640 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561717469a30 0x561717480af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f435bfff640 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56171746d4a0 0x56171747de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f435b7fe640 1 -- 172.21.15.5:0/311691315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561717469a30 msgr2=0x561717480af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f435b7fe640 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561717469a30 0x561717480af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f435b7fe640 1 -- 172.21.15.5:0/311691315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56171746d4a0 msgr2=0x56171747de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f435b7fe640 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56171746d4a0 0x56171747de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f435b7fe640 1 -- 172.21.15.5:0/311691315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171747e8c0 con 0x56171746ddc0 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f435b7fe640 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x56171747e380 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f435000c010 tx=0x7f435000bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33328292256b2c04 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f433b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f433b7fe640 1 -- 172.21.15.5:0/311691315 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435001b470 con 0x56171746ddc0 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f433b7fe640 1 -- 172.21.15.5:0/311691315 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f435001b610 con 0x56171746ddc0 2024-09-15T22:20:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f4364d6a740 1 -- 172.21.15.5:0/311691315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4350016040 con 0x56171746ddc0 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f433b7fe640 1 -- 172.21.15.5:0/311691315 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f435001b810 con 0x56171746ddc0 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f4364d6a740 10 client.17436 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.482+0000 7f4364d6a740 1 -- 172.21.15.5:0/311691315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617174f2880 con 0x56171746ddc0 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.483+0000 7f433b7fe640 1 -- 172.21.15.5:0/311691315 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4350016040 con 0x56171746ddc0 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.483+0000 7f433b7fe640 1 -- 172.21.15.5:0/311691315 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4350037ac0 con 0x56171746ddc0 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.483+0000 7f4364d6a740 10 client.17436 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.483+0000 7f4364d6a740 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.483+0000 7f4364d6a740 20 client.17436 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.483+0000 7f4364d6a740 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561717478680 0x5617174f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.483+0000 7f4364d6a740 4 client.17436 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.483+0000 7f4364d6a740 1 -- 172.21.15.5:0/311691315 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5617173dad60 con 0x561717478680 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.483+0000 7f435bfff640 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561717478680 0x5617174f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.483+0000 7f433b7fe640 10 client.17436 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.483+0000 7f435bfff640 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561717478680 0x5617174f4d60 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2918937d74cbaab server_cookie=1660b69cf56778fd in_seq=0 out_seq=0 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f433b7fe640 1 -- 172.21.15.5:0/311691315 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5617173dad60 con 0x561717478680 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f433b7fe640 10 client.17436 handle_command_reply: tid=0 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 1 client.17436 shutdown 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 1 -- 172.21.15.5:0/311691315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 msgr2=0x56171747e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x56171747e380 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f435000c010 tx=0x7f435000bd90 comp rx=0 tx=0).stop 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 1 -- 172.21.15.5:0/311691315 shutdown_connections 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56171746ddc0 0x56171747e380 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x561717478680 0x5617174f4d60 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56171746d4a0 0x56171747de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 1 --2- 172.21.15.5:0/311691315 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561717469a30 0x561717480af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 1 -- 172.21.15.5:0/311691315 >> 172.21.15.5:0/311691315 conn(0x561717475890 msgr2=0x561717468b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f433b7fe640 0 client.17436 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:14.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 1 -- 172.21.15.5:0/311691315 shutdown_connections 2024-09-15T22:20:14.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 1 -- 172.21.15.5:0/311691315 wait complete. 2024-09-15T22:20:14.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.484+0000 7f4364d6a740 20 client.17436 trim_cache size 0 max 16384 2024-09-15T22:20:14.493 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:20:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:20:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:20:14.493 INFO:teuthology.orchestra.run.smithi005.stdout: "747d64f3-e523-4d82-9ad1-e7ed30b11f59": { 2024-09-15T22:20:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:20:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "747d64f3-e523-4d82-9ad1-e7ed30b11f59", 2024-09-15T22:20:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:20:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:14.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:14.494 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:20:14.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 -- 172.21.15.5:0/2125192511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43440a33b0 msgr2=0x7f43440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43440a33b0 0x7f43440a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f43540bd030 tx=0x7f43540dd040 comp rx=0 tx=0).stop 2024-09-15T22:20:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 -- 172.21.15.5:0/2125192511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c106f10 msgr2=0x7f435c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c106f10 0x7f435c074f70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4354031a20 tx=0x7f4354031a50 comp rx=0 tx=0).stop 2024-09-15T22:20:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 -- 172.21.15.5:0/2125192511 shutdown_connections 2024-09-15T22:20:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43440a33b0 0x7f43440a5870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f435c106f10 0x7f435c074f70 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f435c102790 0x7f435c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 --2- 172.21.15.5:0/2125192511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f435c101de0 0x7f435c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 -- 172.21.15.5:0/2125192511 >> 172.21.15.5:0/2125192511 conn(0x7f435c0fd8c0 msgr2=0x7f435c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 -- 172.21.15.5:0/2125192511 shutdown_connections 2024-09-15T22:20:14.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:14.485+0000 7f4362991640 1 -- 172.21.15.5:0/2125192511 wait complete. 2024-09-15T22:20:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 645.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:20:14.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:14 smithi005 ceph-mon[30944]: pgmap v353: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.7 MiB/s wr, 449 op/s 2024-09-15T22:20:14.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:14 smithi099 ceph-mon[34892]: pgmap v353: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.7 MiB/s wr, 449 op/s 2024-09-15T22:20:14.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:14 smithi129 ceph-mon[35314]: pgmap v353: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 5.7 MiB/s wr, 449 op/s 2024-09-15T22:20:15.145 INFO:tasks.cephfs.filesystem:scrub status for tag:747d64f3-e523-4d82-9ad1-e7ed30b11f59 - {'path': '/', 'tag': '747d64f3-e523-4d82-9ad1-e7ed30b11f59', 'options': 'recursive,force'} 2024-09-15T22:20:17.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:16 smithi099 ceph-mon[34892]: pgmap v354: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-09-15T22:20:17.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:16 smithi129 ceph-mon[35314]: pgmap v354: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-09-15T22:20:17.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:16 smithi005 ceph-mon[30944]: pgmap v354: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 3.6 MiB/s wr, 288 op/s 2024-09-15T22:20:18.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:17 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:20:18.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:17 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:20:18.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:17 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:20:19.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:18 smithi099 ceph-mon[34892]: pgmap v355: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.7 MiB/s wr, 220 op/s 2024-09-15T22:20:19.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:18 smithi129 ceph-mon[35314]: pgmap v355: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.7 MiB/s wr, 220 op/s 2024-09-15T22:20:19.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:18 smithi005 ceph-mon[30944]: pgmap v355: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 2.7 MiB/s wr, 220 op/s 2024-09-15T22:20:19.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:20:19.556 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 650.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:20:19.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:19 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:20:19.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:19 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:19.961 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:19 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:19.961 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:19 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:19.961 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:19 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.042+0000 7f2ee9967640 1 -- 172.21.15.5:0/2750750577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4106b80 msgr2=0x7f2ee4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.042+0000 7f2ee9967640 1 --2- 172.21.15.5:0/2750750577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4106b80 0x7f2ee4106f60 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009900 tx=0x7f2ed402a410 comp rx=0 tx=0).stop 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.042+0000 7f2ee9967640 1 -- 172.21.15.5:0/2750750577 shutdown_connections 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.042+0000 7f2ee9967640 1 --2- 172.21.15.5:0/2750750577 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee4100aa0 0x7f2ee410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.042+0000 7f2ee9967640 1 --2- 172.21.15.5:0/2750750577 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ee4100100 0x7f2ee4100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.042+0000 7f2ee9967640 1 --2- 172.21.15.5:0/2750750577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4106b80 0x7f2ee4106f60 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.042+0000 7f2ee9967640 1 -- 172.21.15.5:0/2750750577 >> 172.21.15.5:0/2750750577 conn(0x7f2ee4077230 msgr2=0x7f2ee40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.042+0000 7f2ee9967640 1 -- 172.21.15.5:0/2750750577 shutdown_connections 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.042+0000 7f2ee9967640 1 -- 172.21.15.5:0/2750750577 wait complete. 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.043+0000 7f2ee9967640 1 Processor -- start 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.043+0000 7f2ee9967640 1 -- start start 2024-09-15T22:20:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.043+0000 7f2ee9967640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee4100100 0x7f2ee419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.043+0000 7f2ee9967640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4100aa0 0x7f2ee419ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.043+0000 7f2ee9967640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ee4106b80 0x7f2ee419dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.043+0000 7f2ee9967640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4112050 con 0x7f2ee4100aa0 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.043+0000 7f2ee9967640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4111ed0 con 0x7f2ee4106b80 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.043+0000 7f2ee9967640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee41121d0 con 0x7f2ee4100100 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.043+0000 7f2ee27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4100aa0 0x7f2ee419ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.043+0000 7f2ee27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4100aa0 0x7f2ee419ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41490/0 (socket says 172.21.15.5:41490) 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.043+0000 7f2ee27fc640 1 -- 172.21.15.5:0/4271022452 learned_addr learned my addr 172.21.15.5:0/4271022452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee2ffd640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee4100100 0x7f2ee419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee37fe640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ee4106b80 0x7f2ee419dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee27fc640 1 -- 172.21.15.5:0/4271022452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee4100100 msgr2=0x7f2ee419a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee27fc640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee4100100 0x7f2ee419a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee27fc640 1 -- 172.21.15.5:0/4271022452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ee4106b80 msgr2=0x7f2ee419dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee27fc640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ee4106b80 0x7f2ee419dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee27fc640 1 -- 172.21.15.5:0/4271022452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee419e6f0 con 0x7f2ee4100aa0 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee27fc640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4100aa0 0x7f2ee419ace0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800e9f0 tx=0x7f2ed800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee8965640 1 -- 172.21.15.5:0/4271022452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed800cd80 con 0x7f2ee4100aa0 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee9967640 1 -- 172.21.15.5:0/4271022452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee419e9e0 con 0x7f2ee4100aa0 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee8965640 1 -- 172.21.15.5:0/4271022452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed8010910 con 0x7f2ee4100aa0 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee8965640 1 -- 172.21.15.5:0/4271022452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8018990 con 0x7f2ee4100aa0 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.044+0000 7f2ee9967640 1 -- 172.21.15.5:0/4271022452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee41a7350 con 0x7f2ee4100aa0 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.046+0000 7f2ee9967640 1 -- 172.21.15.5:0/4271022452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee4100560 con 0x7f2ee4100aa0 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.047+0000 7f2ee8965640 1 -- 172.21.15.5:0/4271022452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2ed8010ac0 con 0x7f2ee4100aa0 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.048+0000 7f2ee8965640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ecc0a33b0 0x7f2ecc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.048+0000 7f2ee2ffd640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ecc0a33b0 0x7f2ecc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.048+0000 7f2ee8965640 1 -- 172.21.15.5:0/4271022452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f2ed80d0450 con 0x7f2ee4100aa0 2024-09-15T22:20:20.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.048+0000 7f2ee2ffd640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ecc0a33b0 0x7f2ecc0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009230 tx=0x7f2ed40058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:20.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.052+0000 7f2ee8965640 1 -- 172.21.15.5:0/4271022452 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ed809c3c0 con 0x7f2ee4100aa0 2024-09-15T22:20:20.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:19 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:20:20.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:19 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:20.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:19 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:20.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:19 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:20.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:19 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:20.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:19 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:20:20.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:19 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:20.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:19 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:20.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:19 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:20.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:19 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.265+0000 7f2ee9967640 1 -- 172.21.15.5:0/4271022452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ee41a7520 con 0x7f2ee4100aa0 2024-09-15T22:20:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.266+0000 7f2ee8965640 1 -- 172.21.15.5:0/4271022452 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f2ed80a06c0 con 0x7f2ee4100aa0 2024-09-15T22:20:20.269 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:20:20.269 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:20:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 -- 172.21.15.5:0/4271022452 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ecc0a33b0 msgr2=0x7f2ecc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ecc0a33b0 0x7f2ecc0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009230 tx=0x7f2ed40058d0 comp rx=0 tx=0).stop 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 -- 172.21.15.5:0/4271022452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4100aa0 msgr2=0x7f2ee419ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4100aa0 0x7f2ee419ace0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800e9f0 tx=0x7f2ed800eeb0 comp rx=0 tx=0).stop 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 -- 172.21.15.5:0/4271022452 shutdown_connections 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ecc0a33b0 0x7f2ecc0a5870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ee4106b80 0x7f2ee419dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4100aa0 0x7f2ee419ace0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 --2- 172.21.15.5:0/4271022452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ee4100100 0x7f2ee419a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 -- 172.21.15.5:0/4271022452 >> 172.21.15.5:0/4271022452 conn(0x7f2ee4077230 msgr2=0x7f2ee4105010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 -- 172.21.15.5:0/4271022452 shutdown_connections 2024-09-15T22:20:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:20.268+0000 7f2ee9967640 1 -- 172.21.15.5:0/4271022452 wait complete. 2024-09-15T22:20:20.935 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id 6c8391fc --release --await' 2024-09-15T22:20:20.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id 6c8391fc --release --await 2024-09-15T22:20:21.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:20 smithi099 ceph-mon[34892]: pgmap v356: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.5 MiB/s wr, 202 op/s 2024-09-15T22:20:21.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:20:21.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:20:21.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:21.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:20 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4271022452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:21.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:20 smithi129 ceph-mon[35314]: pgmap v356: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.5 MiB/s wr, 202 op/s 2024-09-15T22:20:21.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:20:21.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:20:21.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:21.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:20 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4271022452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:21.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:20 smithi005 ceph-mon[30944]: pgmap v356: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 2.5 MiB/s wr, 202 op/s 2024-09-15T22:20:21.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:20:21.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:20:21.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:20:21.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:20 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4271022452' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:21.193 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.631+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/2526543691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98100a80 msgr2=0x7f7f98100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.631+0000 7f7f9ee2d640 1 --2- 172.21.15.5:0/2526543691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98100a80 0x7f7f98100ee0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009cd0 tx=0x7f7f8802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.631+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/2526543691 shutdown_connections 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.631+0000 7f7f9ee2d640 1 --2- 172.21.15.5:0/2526543691 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f98101420 0x7f7f9810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.631+0000 7f7f9ee2d640 1 --2- 172.21.15.5:0/2526543691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98100a80 0x7f7f98100ee0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.631+0000 7f7f9ee2d640 1 --2- 172.21.15.5:0/2526543691 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f98106a80 0x7f7f98106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.631+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/2526543691 >> 172.21.15.5:0/2526543691 conn(0x7f7f980fc7a0 msgr2=0x7f7f980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.631+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/2526543691 shutdown_connections 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.631+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/2526543691 wait complete. 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9ee2d640 1 Processor -- start 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9ee2d640 1 -- start start 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9ee2d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f98100a80 0x7f7f98198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9ee2d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f98101420 0x7f7f98198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9ee2d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106a80 0x7f7f9819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9ee2d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98110e80 con 0x7f7f98106a80 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9ee2d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98110d00 con 0x7f7f98100a80 2024-09-15T22:20:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9ee2d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f98111000 con 0x7f7f98101420 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9d3ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106a80 0x7f7f9819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9d3ca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106a80 0x7f7f9819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41510/0 (socket says 172.21.15.5:41510) 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9d3ca640 1 -- 172.21.15.5:0/786462057 learned_addr learned my addr 172.21.15.5:0/786462057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.632+0000 7f7f9cbc9640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f98100a80 0x7f7f98198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f97fff640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f98101420 0x7f7f98198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f9d3ca640 1 -- 172.21.15.5:0/786462057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f98101420 msgr2=0x7f7f98198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f9d3ca640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f98101420 0x7f7f98198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f9d3ca640 1 -- 172.21.15.5:0/786462057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f98100a80 msgr2=0x7f7f98198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f9d3ca640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f98100a80 0x7f7f98198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f9d3ca640 1 -- 172.21.15.5:0/786462057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f9819d3d0 con 0x7f7f98106a80 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f9d3ca640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106a80 0x7f7f9819ccf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000d750 tx=0x7f7f8000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f95ffb640 1 -- 172.21.15.5:0/786462057 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80017070 con 0x7f7f98106a80 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f95ffb640 1 -- 172.21.15.5:0/786462057 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f80005500 con 0x7f7f98106a80 2024-09-15T22:20:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/786462057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f9819d6c0 con 0x7f7f98106a80 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f95ffb640 1 -- 172.21.15.5:0/786462057 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f80004d90 con 0x7f7f98106a80 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.633+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/786462057 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f981a4fa0 con 0x7f7f98106a80 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.635+0000 7f7f95ffb640 1 -- 172.21.15.5:0/786462057 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f7f80004f30 con 0x7f7f98106a80 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7fa1206740 1 Processor -- start 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7fa1206740 1 -- start start 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47060 0x56182bb47440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb51d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb54630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7fa1206740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56182baac620 con 0x56182bb47980 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7fa1206740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56182ba53e50 con 0x56182bb47060 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7fa1206740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56182baa85e0 con 0x56182bb52240 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f9cbc9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb51d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f9cbc9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb51d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41522/0 (socket says 172.21.15.5:41522) 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f9cbc9640 1 -- 172.21.15.5:0/1908339784 learned_addr learned my addr 172.21.15.5:0/1908339784 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f95ffb640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7f840a3430 0x7f7f840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1908339784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47060 0x56182bb47440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1908339784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb54630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f9d3ca640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7f840a3430 0x7f7f840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f9cbc9640 1 -- 172.21.15.5:0/1908339784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 msgr2=0x56182bb54630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f9cbc9640 1 --2- 172.21.15.5:0/1908339784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb54630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f9cbc9640 1 -- 172.21.15.5:0/1908339784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47060 msgr2=0x56182bb47440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f9cbc9640 1 --2- 172.21.15.5:0/1908339784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47060 0x56182bb47440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f9cbc9640 1 -- 172.21.15.5:0/1908339784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182ba44d40 con 0x56182bb47980 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.636+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1908339784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb54630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.637+0000 7f7f95ffb640 1 -- 172.21.15.5:0/786462057 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f7f800cf9f0 con 0x7f7f98106a80 2024-09-15T22:20:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.637+0000 7f7f9cbc9640 1 --2- 172.21.15.5:0/1908339784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb51d00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000bc90 tx=0x7f7f900107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e929c671c6a3329c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.637+0000 7f7f9d3ca640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7f840a3430 0x7f7f840a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f800a3fb0 tx=0x7f7f80066040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.637+0000 7f7f8effd640 1 -- 172.21.15.5:0/1908339784 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9001c480 con 0x56182bb47980 2024-09-15T22:20:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.637+0000 7f7f8effd640 1 -- 172.21.15.5:0/1908339784 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f9001cb00 con 0x56182bb47980 2024-09-15T22:20:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.637+0000 7f7f8effd640 1 -- 172.21.15.5:0/1908339784 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f90011c00 con 0x56182bb47980 2024-09-15T22:20:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.638+0000 7f7fa1206740 1 -- 172.21.15.5:0/1908339784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 msgr2=0x56182bb51d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.638+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1908339784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb51d00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000bc90 tx=0x7f7f900107a0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.638+0000 7f7fa1206740 1 -- 172.21.15.5:0/1908339784 shutdown_connections 2024-09-15T22:20:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.638+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1908339784 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb54630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.638+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1908339784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb51d00 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.638+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1908339784 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47060 0x56182bb47440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.638+0000 7f7fa1206740 1 -- 172.21.15.5:0/1908339784 >> 172.21.15.5:0/1908339784 conn(0x56182bb435f0 msgr2=0x56182bb439d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.638+0000 7f7fa1206740 1 -- 172.21.15.5:0/1908339784 shutdown_connections 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.638+0000 7f7fa1206740 1 -- 172.21.15.5:0/1908339784 wait complete. 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.638+0000 7f7fa1206740 1 -- start start 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.638+0000 7f7fa1206740 1 Processor -- start 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47060 0x56182bb4f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb4f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb4fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7fa1206740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56182bb54e10 con 0x56182bb47980 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7fa1206740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56182ba34f10 con 0x56182bb47060 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7fa1206740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56182bb54f90 con 0x56182bb52240 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7fa1206740 10 client.0 _ll_register_callbacks cb 0x56182ba58db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f9cbc9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb4f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f9cbc9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb4f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41532/0 (socket says 172.21.15.5:41532) 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f9cbc9640 1 -- 172.21.15.5:0/1650224370 learned_addr learned my addr 172.21.15.5:0/1650224370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47060 0x56182bb4f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb4fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f9cbc9640 1 -- 172.21.15.5:0/1650224370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 msgr2=0x56182bb4fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f9cbc9640 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb4fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f9cbc9640 1 -- 172.21.15.5:0/1650224370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47060 msgr2=0x56182bb4f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f9cbc9640 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47060 0x56182bb4f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f9cbc9640 1 -- 172.21.15.5:0/1650224370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182b7bb580 con 0x56182bb47980 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb4fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f9cbc9640 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb4f660 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7f90012c60 tx=0x7f7f90012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597c02dc53510ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f77fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.639+0000 7f7f77fff640 1 -- 172.21.15.5:0/1650224370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9001c690 con 0x56182bb47980 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7f77fff640 1 -- 172.21.15.5:0/1650224370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f7f9001c830 con 0x56182bb47980 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7fa1206740 1 -- 172.21.15.5:0/1650224370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f9001cb00 con 0x56182bb47980 2024-09-15T22:20:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7f77fff640 1 -- 172.21.15.5:0/1650224370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f90004020 con 0x56182bb47980 2024-09-15T22:20:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7fa1206740 10 client.17454 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7fa1206740 1 -- 172.21.15.5:0/1650224370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56182b885960 con 0x56182bb47980 2024-09-15T22:20:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7f77fff640 1 -- 172.21.15.5:0/1650224370 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f9001cb00 con 0x56182bb47980 2024-09-15T22:20:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7f77fff640 1 -- 172.21.15.5:0/1650224370 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7f900046c0 con 0x56182bb47980 2024-09-15T22:20:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7fa1206740 10 client.17454 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7fa1206740 10 client.17454 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:20:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7fa1206740 20 client.17454 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182bb61cd0 0x56182bb640c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7fa1206740 4 client.17454 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7fa1206740 1 -- 172.21.15.5:0/1650224370 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56182bab9b90 con 0x56182bb61cd0 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.640+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182bb61cd0 0x56182bb640c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.641+0000 7f7f77fff640 10 client.17454 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.641+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182bb61cd0 0x56182bb640c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cff627f8939f1d23 server_cookie=15b8e498e46cb729 in_seq=0 out_seq=0 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7f77fff640 1 -- 172.21.15.5:0/1650224370 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56182bab9b90 con 0x56182bb61cd0 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7f77fff640 10 client.17454 handle_command_reply: tid=0 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 1 client.17454 shutdown 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 1 -- 172.21.15.5:0/1650224370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 msgr2=0x56182bb4f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb4f660 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f7f90012c60 tx=0x7f7f90012c90 comp rx=0 tx=0).stop 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 1 -- 172.21.15.5:0/1650224370 shutdown_connections 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb4fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47980 0x56182bb4f660 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182bb61cd0 0x56182bb640c0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1650224370 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47060 0x56182bb4f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7f77fff640 0 client.17454 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 1 -- 172.21.15.5:0/1650224370 >> 172.21.15.5:0/1650224370 conn(0x56182bb435f0 msgr2=0x56182bb42750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 1 -- 172.21.15.5:0/1650224370 shutdown_connections 2024-09-15T22:20:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 1 -- 172.21.15.5:0/1650224370 wait complete. 2024-09-15T22:20:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.642+0000 7f7fa1206740 20 client.17454 trim_cache size 0 max 16384 2024-09-15T22:20:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7fa1206740 1 Processor -- start 2024-09-15T22:20:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7fa1206740 1 -- start start 2024-09-15T22:20:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb50ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47980 0x56182bb49a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb69830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7fa1206740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981b1a50 con 0x56182bb47060 2024-09-15T22:20:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7fa1206740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981af320 con 0x56182bb47980 2024-09-15T22:20:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7fa1206740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981ad190 con 0x56182bb52240 2024-09-15T22:20:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb50ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb50ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41540/0 (socket says 172.21.15.5:41540) 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7f97fff640 1 -- 172.21.15.5:0/2741767716 learned_addr learned my addr 172.21.15.5:0/2741767716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.659+0000 7f7f9cbc9640 1 --2- 172.21.15.5:0/2741767716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47980 0x56182bb49a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.660+0000 7f7f97fff640 1 -- 172.21.15.5:0/2741767716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 msgr2=0x56182bb69830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.660+0000 7f7f97fff640 1 --2- 172.21.15.5:0/2741767716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb69830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.660+0000 7f7f97fff640 1 -- 172.21.15.5:0/2741767716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47980 msgr2=0x56182bb49a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.660+0000 7f7f97fff640 1 --2- 172.21.15.5:0/2741767716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47980 0x56182bb49a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.660+0000 7f7f97fff640 1 -- 172.21.15.5:0/2741767716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56182bb43090 con 0x56182bb47060 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.660+0000 7f7f97fff640 1 --2- 172.21.15.5:0/2741767716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb50ff0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009cd0 tx=0x7f7f88032fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399498822eff9b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.660+0000 7f7f8e7fc640 1 -- 172.21.15.5:0/2741767716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88003970 con 0x56182bb47060 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.660+0000 7f7f8e7fc640 1 -- 172.21.15.5:0/2741767716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f88036040 con 0x56182bb47060 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.660+0000 7f7f8e7fc640 1 -- 172.21.15.5:0/2741767716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f880042a0 con 0x56182bb47060 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.660+0000 7f7fa1206740 1 -- 172.21.15.5:0/2741767716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 msgr2=0x56182bb50ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.660+0000 7f7fa1206740 1 --2- 172.21.15.5:0/2741767716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb50ff0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009cd0 tx=0x7f7f88032fd0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 -- 172.21.15.5:0/2741767716 shutdown_connections 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 --2- 172.21.15.5:0/2741767716 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb52240 0x56182bb69830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 --2- 172.21.15.5:0/2741767716 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb47980 0x56182bb49a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 --2- 172.21.15.5:0/2741767716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb50ff0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 -- 172.21.15.5:0/2741767716 >> 172.21.15.5:0/2741767716 conn(0x56182bb435f0 msgr2=0x56182bb42750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 -- 172.21.15.5:0/2741767716 shutdown_connections 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 -- 172.21.15.5:0/2741767716 wait complete. 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 -- start start 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 Processor -- start 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb5a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb47980 0x56182bb5a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb52240 0x56182bb56730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981ad7e0 con 0x56182bb47060 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981aec50 con 0x56182bb52240 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f7f981adeb0 con 0x56182bb47980 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7fa1206740 10 client.0 _ll_register_callbacks cb 0x7f7f90011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb5a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7f97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb5a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41550/0 (socket says 172.21.15.5:41550) 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.661+0000 7f7f97fff640 1 -- 172.21.15.5:0/1881339854 learned_addr learned my addr 172.21.15.5:0/1881339854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f9cbc9640 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb47980 0x56182bb5a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb52240 0x56182bb56730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f97fff640 1 -- 172.21.15.5:0/1881339854 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb47980 msgr2=0x56182bb5a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb47980 0x56182bb5a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f97fff640 1 -- 172.21.15.5:0/1881339854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb52240 msgr2=0x56182bb56730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb52240 0x56182bb56730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f97fff640 1 -- 172.21.15.5:0/1881339854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56182bb56f60 con 0x56182bb47060 2024-09-15T22:20:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f77fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f97fff640 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb5a2a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009620 tx=0x7f7f88009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8450c673965260 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f77fff640 1 -- 172.21.15.5:0/1881339854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f88031940 con 0x56182bb47060 2024-09-15T22:20:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f77fff640 1 -- 172.21.15.5:0/1881339854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f7f88031ae0 con 0x56182bb47060 2024-09-15T22:20:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7fa1206740 1 -- 172.21.15.5:0/1881339854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f88036040 con 0x56182bb47060 2024-09-15T22:20:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7f77fff640 1 -- 172.21.15.5:0/1881339854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7f88031ce0 con 0x56182bb47060 2024-09-15T22:20:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7fa1206740 10 client.17460 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.662+0000 7f7fa1206740 1 -- 172.21.15.5:0/1881339854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56182bbcd2b0 con 0x56182bb47060 2024-09-15T22:20:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.663+0000 7f7f77fff640 1 -- 172.21.15.5:0/1881339854 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f88036040 con 0x56182bb47060 2024-09-15T22:20:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.663+0000 7f7f77fff640 1 -- 172.21.15.5:0/1881339854 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7f88030070 con 0x56182bb47060 2024-09-15T22:20:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.663+0000 7f7fa1206740 10 client.17460 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.663+0000 7f7fa1206740 10 client.17460 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:20:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.663+0000 7f7fa1206740 20 client.17460 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.663+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182bb61cd0 0x56182bbcf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.663+0000 7f7fa1206740 4 client.17460 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6c8391fc", "release": true, "await": true}] 2024-09-15T22:20:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.663+0000 7f7fa1206740 1 -- 172.21.15.5:0/1881339854 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6c8391fc", "release": true, "await": true}) -- 0x56182bb3e950 con 0x56182bb61cd0 2024-09-15T22:20:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.663+0000 7f7f9cbc9640 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182bb61cd0 0x56182bbcf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.663+0000 7f7f77fff640 10 client.17460 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.663+0000 7f7f9cbc9640 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182bb61cd0 0x56182bbcf910 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe22a2302f058a56 server_cookie=2d753f3ba3406ba0 in_seq=0 out_seq=0 2024-09-15T22:20:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7f77fff640 1 -- 172.21.15.5:0/1881339854 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56182bb3e950 con 0x56182bb61cd0 2024-09-15T22:20:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7f77fff640 10 client.17460 handle_command_reply: tid=0 2024-09-15T22:20:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 1 client.17460 shutdown 2024-09-15T22:20:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 1 -- 172.21.15.5:0/1881339854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 msgr2=0x56182bb5a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb5a2a0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009620 tx=0x7f7f88009c20 comp rx=0 tx=0).stop 2024-09-15T22:20:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 1 -- 172.21.15.5:0/1881339854 shutdown_connections 2024-09-15T22:20:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56182bb61cd0 0x56182bbcf910 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56182bb52240 0x56182bb56730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56182bb47980 0x56182bb5a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 1 --2- 172.21.15.5:0/1881339854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56182bb47060 0x56182bb5a2a0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 1 -- 172.21.15.5:0/1881339854 >> 172.21.15.5:0/1881339854 conn(0x56182bb435f0 msgr2=0x56182bb41f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7f77fff640 0 client.17460 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:21.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 1 -- 172.21.15.5:0/1881339854 shutdown_connections 2024-09-15T22:20:21.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 1 -- 172.21.15.5:0/1881339854 wait complete. 2024-09-15T22:20:21.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.707+0000 7f7fa1206740 20 client.17460 trim_cache size 0 max 16384 2024-09-15T22:20:21.711 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:20:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:20:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:20:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 40, 2024-09-15T22:20:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:20:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "6c8391fc": { 2024-09-15T22:20:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 40, 2024-09-15T22:20:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:20:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:20:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:20:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:20:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:20:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:20:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 141.8, 2024-09-15T22:20:21.712 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:20:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:20:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:20:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:20:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:20:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:20:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:21.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:21.713 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:20:21.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.708+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/786462057 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7f840a3430 msgr2=0x7f7f840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:21.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.708+0000 7f7f9ee2d640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7f840a3430 0x7f7f840a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f800a3fb0 tx=0x7f7f80066040 comp rx=0 tx=0).stop 2024-09-15T22:20:21.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.708+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/786462057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106a80 msgr2=0x7f7f9819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:21.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.708+0000 7f7f9ee2d640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106a80 0x7f7f9819ccf0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000d750 tx=0x7f7f8000dc10 comp rx=0 tx=0).stop 2024-09-15T22:20:21.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.709+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/786462057 shutdown_connections 2024-09-15T22:20:21.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.709+0000 7f7f9ee2d640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f7f840a3430 0x7f7f840a58f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.709+0000 7f7f9ee2d640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f98106a80 0x7f7f9819ccf0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.709+0000 7f7f9ee2d640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f7f98101420 0x7f7f98198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.709+0000 7f7f9ee2d640 1 --2- 172.21.15.5:0/786462057 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f7f98100a80 0x7f7f98198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:21.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.709+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/786462057 >> 172.21.15.5:0/786462057 conn(0x7f7f980fc7a0 msgr2=0x7f7f9806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:21.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.709+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/786462057 shutdown_connections 2024-09-15T22:20:21.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:21.709+0000 7f7f9ee2d640 1 -- 172.21.15.5:0/786462057 wait complete. 2024-09-15T22:20:22.478 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6c8391fc', total seconds elapsed: 25.1 2024-09-15T22:20:22.478 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.85 2024-09-15T22:20:22.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:20:22.982 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:23.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:22 smithi099 ceph-mon[34892]: pgmap v357: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.3 MiB/s wr, 186 op/s 2024-09-15T22:20:23.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:22 smithi129 ceph-mon[35314]: pgmap v357: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.3 MiB/s wr, 186 op/s 2024-09-15T22:20:23.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:22 smithi005 ceph-mon[30944]: pgmap v357: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 2.3 MiB/s wr, 186 op/s 2024-09-15T22:20:23.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.578+0000 7f55f7a4d640 1 -- 172.21.15.5:0/299582559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55f0109e00 msgr2=0x7f55f00742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.578+0000 7f55f7a4d640 1 --2- 172.21.15.5:0/299582559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55f0109e00 0x7f55f00742d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f55e800afd0 tx=0x7f55e80085b0 comp rx=0 tx=0).stop 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.578+0000 7f55f7a4d640 1 -- 172.21.15.5:0/299582559 shutdown_connections 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.578+0000 7f55f7a4d640 1 --2- 172.21.15.5:0/299582559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f0074990 0x7f55f007b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.578+0000 7f55f7a4d640 1 --2- 172.21.15.5:0/299582559 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55f0109e00 0x7f55f00742d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.578+0000 7f55f7a4d640 1 --2- 172.21.15.5:0/299582559 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55f0109450 0x7f55f0109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.578+0000 7f55f7a4d640 1 -- 172.21.15.5:0/299582559 >> 172.21.15.5:0/299582559 conn(0x7f55f006e140 msgr2=0x7f55f006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.579+0000 7f55f7a4d640 1 -- 172.21.15.5:0/299582559 shutdown_connections 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.579+0000 7f55f7a4d640 1 -- 172.21.15.5:0/299582559 wait complete. 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.580+0000 7f55f7a4d640 1 Processor -- start 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.580+0000 7f55f7a4d640 1 -- start start 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.580+0000 7f55f7a4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f0074990 0x7f55f007aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.580+0000 7f55f7a4d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55f0109450 0x7f55f0075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:23.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.580+0000 7f55f7a4d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55f0076470 0x7f55f0076920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.580+0000 7f55f7a4d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0082b70 con 0x7f55f0074990 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.580+0000 7f55f7a4d640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f55f00829f0 con 0x7f55f0109450 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.580+0000 7f55f7a4d640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f55f0082cf0 con 0x7f55f0076470 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.581+0000 7f55f4fe8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55f0109450 0x7f55f0075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.581+0000 7f55f4fe8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55f0109450 0x7f55f0075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:44636/0 (socket says 172.21.15.5:44636) 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.581+0000 7f55f4fe8640 1 -- 172.21.15.5:0/990321392 learned_addr learned my addr 172.21.15.5:0/990321392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.581+0000 7f55f5fea640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55f0076470 0x7f55f0076920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.581+0000 7f55f4fe8640 1 -- 172.21.15.5:0/990321392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55f0076470 msgr2=0x7f55f0076920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.581+0000 7f55f4fe8640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55f0076470 0x7f55f0076920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.581+0000 7f55f4fe8640 1 -- 172.21.15.5:0/990321392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f0074990 msgr2=0x7f55f007aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.581+0000 7f55f4fe8640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f0074990 0x7f55f007aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.581+0000 7f55f4fe8640 1 -- 172.21.15.5:0/990321392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f0137290 con 0x7f55f0109450 2024-09-15T22:20:23.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.582+0000 7f55f4fe8640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55f0109450 0x7f55f0075f30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55e8034040 tx=0x7f55e8032620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.584+0000 7f55e67fc640 1 -- 172.21.15.5:0/990321392 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8040030 con 0x7f55f0109450 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.584+0000 7f55e67fc640 1 -- 172.21.15.5:0/990321392 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55e803c400 con 0x7f55f0109450 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.584+0000 7f55f7a4d640 1 -- 172.21.15.5:0/990321392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f0137520 con 0x7f55f0109450 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.584+0000 7f55f7a4d640 1 -- 172.21.15.5:0/990321392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f0137a60 con 0x7f55f0109450 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.584+0000 7f55e67fc640 1 -- 172.21.15.5:0/990321392 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e802ae60 con 0x7f55f0109450 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.585+0000 7f55dbfff640 1 -- 172.21.15.5:0/990321392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f0073b00 con 0x7f55f0109450 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.590+0000 7f55e67fc640 1 -- 172.21.15.5:0/990321392 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f55e8043070 con 0x7f55f0109450 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.591+0000 7f55e67fc640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f55dc0a35c0 0x7f55dc0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.592+0000 7f55f57e9640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f55dc0a35c0 0x7f55dc0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.592+0000 7f55e67fc640 1 -- 172.21.15.5:0/990321392 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f55e80f13e0 con 0x7f55f0109450 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.592+0000 7f55f57e9640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f55dc0a35c0 0x7f55dc0a5a80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b3c0 tx=0x7f55ec00d040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:23.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.594+0000 7f55e67fc640 1 -- 172.21.15.5:0/990321392 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55e804a070 con 0x7f55f0109450 2024-09-15T22:20:23.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.991+0000 7f55dbfff640 1 -- 172.21.15.5:0/990321392 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55f0109e00 con 0x7f55f0109450 2024-09-15T22:20:23.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:23.993+0000 7f55e67fc640 1 -- 172.21.15.5:0/990321392 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f55e80bdaf0 con 0x7f55f0109450 2024-09-15T22:20:23.995 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:20:23.995 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:20:23.995 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 -- 172.21.15.5:0/990321392 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f55dc0a35c0 msgr2=0x7f55dc0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f55dc0a35c0 0x7f55dc0a5a80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b3c0 tx=0x7f55ec00d040 comp rx=0 tx=0).stop 2024-09-15T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 -- 172.21.15.5:0/990321392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55f0109450 msgr2=0x7f55f0075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55f0109450 0x7f55f0075f30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f55e8034040 tx=0x7f55e8032620 comp rx=0 tx=0).stop 2024-09-15T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 -- 172.21.15.5:0/990321392 shutdown_connections 2024-09-15T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f55dc0a35c0 0x7f55dc0a5a80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f55f0076470 0x7f55f0076920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f55f0109450 0x7f55f0075f30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 --2- 172.21.15.5:0/990321392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f0074990 0x7f55f007aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 -- 172.21.15.5:0/990321392 >> 172.21.15.5:0/990321392 conn(0x7f55f006e140 msgr2=0x7f55f00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:24.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 -- 172.21.15.5:0/990321392 shutdown_connections 2024-09-15T22:20:24.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:24.000+0000 7f55dbfff640 1 -- 172.21.15.5:0/990321392 wait complete. 2024-09-15T22:20:24.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 655.0 s, 300.1 tps, lat 997.143 ms stddev 3038.195 2024-09-15T22:20:25.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:24 smithi099 ceph-mon[34892]: pgmap v358: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 523 KiB/s wr, 27 op/s 2024-09-15T22:20:25.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:24 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/990321392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:25.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:24 smithi129 ceph-mon[35314]: pgmap v358: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 523 KiB/s wr, 27 op/s 2024-09-15T22:20:25.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:24 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/990321392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:25.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:24 smithi005 ceph-mon[30944]: pgmap v358: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 523 KiB/s wr, 27 op/s 2024-09-15T22:20:25.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:24 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/990321392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:26.090 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0' 2024-09-15T22:20:26.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 141.85 --await-for 45.0 2024-09-15T22:20:26.581 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:27.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:26 smithi099 ceph-mon[34892]: pgmap v359: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-09-15T22:20:27.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:26 smithi129 ceph-mon[35314]: pgmap v359: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-09-15T22:20:27.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:26 smithi005 ceph-mon[30944]: pgmap v359: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-09-15T22:20:28.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.221+0000 7f2ec32b7640 1 -- 172.21.15.5:0/2364593651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ebc102ef0 msgr2=0x7f2ebc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:28.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.221+0000 7f2ec32b7640 1 --2- 172.21.15.5:0/2364593651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ebc102ef0 0x7f2ebc10da40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009c10 tx=0x7f2eb402a770 comp rx=0 tx=0).stop 2024-09-15T22:20:28.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.221+0000 7f2ec32b7640 1 -- 172.21.15.5:0/2364593651 shutdown_connections 2024-09-15T22:20:28.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.221+0000 7f2ec32b7640 1 --2- 172.21.15.5:0/2364593651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ebc102ef0 0x7f2ebc10da40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.221+0000 7f2ec32b7640 1 --2- 172.21.15.5:0/2364593651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ebc102570 0x7f2ebc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.221+0000 7f2ec32b7640 1 --2- 172.21.15.5:0/2364593651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc100e20 0x7f2ebc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.221+0000 7f2ec32b7640 1 -- 172.21.15.5:0/2364593651 >> 172.21.15.5:0/2364593651 conn(0x7f2ebc0fc7d0 msgr2=0x7f2ebc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:28.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.221+0000 7f2ec32b7640 1 -- 172.21.15.5:0/2364593651 shutdown_connections 2024-09-15T22:20:28.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.221+0000 7f2ec32b7640 1 -- 172.21.15.5:0/2364593651 wait complete. 2024-09-15T22:20:28.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.222+0000 7f2ec32b7640 1 Processor -- start 2024-09-15T22:20:28.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.222+0000 7f2ec32b7640 1 -- start start 2024-09-15T22:20:28.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec32b7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc100e20 0x7f2ebc196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec32b7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ebc102570 0x7f2ebc1967c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec32b7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ebc102ef0 0x7f2ebc19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec32b7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc10fe20 con 0x7f2ebc100e20 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec32b7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc10fca0 con 0x7f2ebc102ef0 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec32b7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc10ffa0 con 0x7f2ebc102570 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec1053640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc100e20 0x7f2ebc196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec1053640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc100e20 0x7f2ebc196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36840/0 (socket says 172.21.15.5:36840) 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec1053640 1 -- 172.21.15.5:0/233724186 learned_addr learned my addr 172.21.15.5:0/233724186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec1854640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ebc102ef0 0x7f2ebc19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec1053640 1 -- 172.21.15.5:0/233724186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ebc102570 msgr2=0x7f2ebc1967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec1053640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ebc102570 0x7f2ebc1967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec1053640 1 -- 172.21.15.5:0/233724186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ebc102ef0 msgr2=0x7f2ebc19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec1053640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ebc102ef0 0x7f2ebc19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.223+0000 7f2ec1053640 1 -- 172.21.15.5:0/233724186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc19b230 con 0x7f2ebc100e20 2024-09-15T22:20:28.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.224+0000 7f2ec1053640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc100e20 0x7f2ebc196280 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800e9c0 tx=0x7f2eb800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:28.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.224+0000 7f2eb27fc640 1 -- 172.21.15.5:0/233724186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800cde0 con 0x7f2ebc100e20 2024-09-15T22:20:28.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.224+0000 7f2eb27fc640 1 -- 172.21.15.5:0/233724186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb80035f0 con 0x7f2ebc100e20 2024-09-15T22:20:28.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.224+0000 7f2eb27fc640 1 -- 172.21.15.5:0/233724186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8010690 con 0x7f2ebc100e20 2024-09-15T22:20:28.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.225+0000 7f2ec32b7640 1 -- 172.21.15.5:0/233724186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ebc19b520 con 0x7f2ebc100e20 2024-09-15T22:20:28.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.225+0000 7f2ec32b7640 1 -- 172.21.15.5:0/233724186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ebc1a2e00 con 0x7f2ebc100e20 2024-09-15T22:20:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.229+0000 7f2ec5690740 1 Processor -- start 2024-09-15T22:20:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.229+0000 7f2ec5690740 1 -- start start 2024-09-15T22:20:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.229+0000 7f2ec5690740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f0fc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.229+0000 7f2ec5690740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f106d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.229+0000 7f2ec5690740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f107240 0x558d6f109630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.229+0000 7f2ec5690740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558d6f058160 con 0x558d6f107240 2024-09-15T22:20:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.229+0000 7f2ec5690740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558d6efff720 con 0x558d6f0fc060 2024-09-15T22:20:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.229+0000 7f2ec5690740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558d6f054120 con 0x558d6f0fc980 2024-09-15T22:20:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.229+0000 7f2ec1854640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f0fc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2ec1854640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f0fc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:44656/0 (socket says 172.21.15.5:44656) 2024-09-15T22:20:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2ec1854640 1 -- 172.21.15.5:0/1876630148 learned_addr learned my addr 172.21.15.5:0/1876630148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2eb27fc640 1 -- 172.21.15.5:0/233724186 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2eb80037a0 con 0x7f2ebc100e20 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2ec1053640 1 --2- 172.21.15.5:0/1876630148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f106d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2ec1854640 1 -- 172.21.15.5:0/1876630148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 msgr2=0x558d6f106d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2ec0852640 1 --2- 172.21.15.5:0/1876630148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f107240 0x558d6f109630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2ec1854640 1 --2- 172.21.15.5:0/1876630148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f106d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2ec1854640 1 -- 172.21.15.5:0/1876630148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f107240 msgr2=0x558d6f109630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2ec1854640 1 --2- 172.21.15.5:0/1876630148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f107240 0x558d6f109630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2ec1854640 1 -- 172.21.15.5:0/1876630148 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6edc9ba0 con 0x558d6f0fc060 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2ec0852640 1 --2- 172.21.15.5:0/1876630148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f107240 0x558d6f109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.230+0000 7f2ec1854640 1 --2- 172.21.15.5:0/1876630148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f0fc440 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009c10 tx=0x7f2eb4005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6440ecf5b407ce8b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.231+0000 7f2eb27fc640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ea80a3580 0x7f2ea80a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.231+0000 7f2ea77fe640 1 -- 172.21.15.5:0/1876630148 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4036070 con 0x558d6f0fc060 2024-09-15T22:20:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.231+0000 7f2ea77fe640 1 -- 172.21.15.5:0/1876630148 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eb4037040 con 0x558d6f0fc060 2024-09-15T22:20:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.231+0000 7f2ea77fe640 1 -- 172.21.15.5:0/1876630148 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4003e60 con 0x558d6f0fc060 2024-09-15T22:20:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.231+0000 7f2eb27fc640 1 -- 172.21.15.5:0/233724186 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f2eb80d04b0 con 0x7f2ebc100e20 2024-09-15T22:20:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.231+0000 7f2ec1053640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ea80a3580 0x7f2ea80a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.232+0000 7f2ec5690740 1 -- 172.21.15.5:0/1876630148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 msgr2=0x558d6f0fc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.232+0000 7f2ec5690740 1 --2- 172.21.15.5:0/1876630148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f0fc440 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2eb4009c10 tx=0x7f2eb4005ec0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.232+0000 7f2ec1053640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ea80a3580 0x7f2ea80a5a40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x558d6f0f9450 tx=0x7f2eb80d0700 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.232+0000 7f2ec5690740 1 -- 172.21.15.5:0/1876630148 shutdown_connections 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.232+0000 7f2ec5690740 1 --2- 172.21.15.5:0/1876630148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f107240 0x558d6f109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.232+0000 7f2ec5690740 1 --2- 172.21.15.5:0/1876630148 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f106d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.232+0000 7f2ec5690740 1 --2- 172.21.15.5:0/1876630148 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f0fc440 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.232+0000 7f2ec5690740 1 -- 172.21.15.5:0/1876630148 >> 172.21.15.5:0/1876630148 conn(0x558d6f0f85f0 msgr2=0x558d6f0f89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.232+0000 7f2ec5690740 1 -- 172.21.15.5:0/1876630148 shutdown_connections 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.232+0000 7f2ec5690740 1 -- 172.21.15.5:0/1876630148 wait complete. 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.232+0000 7f2ec5690740 1 -- start start 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.233+0000 7f2ec5690740 1 Processor -- start 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.233+0000 7f2ec5690740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f104140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.233+0000 7f2ec5690740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f104680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.233+0000 7f2ec1854640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f104140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.233+0000 7f2ec1854640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f104140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:44660/0 (socket says 172.21.15.5:44660) 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.234+0000 7f2ec1854640 1 -- 172.21.15.5:0/238112749 learned_addr learned my addr 172.21.15.5:0/238112749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.234+0000 7f2ec5690740 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f107240 0x558d6f104bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.234+0000 7f2ec0852640 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f104680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.234+0000 7f2ec5690740 1 -- 172.21.15.5:0/238112749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558d6f109db0 con 0x558d6f107240 2024-09-15T22:20:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.234+0000 7f2ec5690740 1 -- 172.21.15.5:0/238112749 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558d6efe0900 con 0x558d6f0fc060 2024-09-15T22:20:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.234+0000 7f2ec5690740 1 -- 172.21.15.5:0/238112749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558d6f109f30 con 0x558d6f0fc980 2024-09-15T22:20:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.234+0000 7f2ec5690740 10 client.0 _ll_register_callbacks cb 0x558d6efde780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.234+0000 7f2ec1854640 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f107240 0x558d6f104bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.235+0000 7f2ec0852640 1 -- 172.21.15.5:0/238112749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 msgr2=0x558d6f104140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.235+0000 7f2ec0852640 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f104140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.235+0000 7f2ec0852640 1 -- 172.21.15.5:0/238112749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f107240 msgr2=0x558d6f104bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.235+0000 7f2ec0852640 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f107240 0x558d6f104bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.235+0000 7f2ec0852640 1 -- 172.21.15.5:0/238112749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d6f020c50 con 0x558d6f0fc980 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.235+0000 7f2ec1854640 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.235+0000 7f2ec0852640 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f104680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009fa0 tx=0x7f2eac010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b79d4c9891b5f0d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.235+0000 7f2ea4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.235+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/238112749 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac012860 con 0x558d6f0fc980 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.235+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/238112749 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f2eac012a00 con 0x558d6f0fc980 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.235+0000 7f2ec5690740 1 -- 172.21.15.5:0/238112749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eb4037040 con 0x558d6f0fc980 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.236+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/238112749 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2eac024050 con 0x558d6f0fc980 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.236+0000 7f2ec5690740 10 client.26896 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.236+0000 7f2ec5690740 1 -- 172.21.15.5:0/238112749 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d6f115a40 con 0x558d6f0fc980 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.236+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/238112749 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eac012c00 con 0x558d6f0fc980 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.236+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/238112749 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2eac00f040 con 0x558d6f0fc980 2024-09-15T22:20:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.236+0000 7f2ec5690740 10 client.26896 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:28.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.236+0000 7f2ec5690740 10 client.26896 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:20:28.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.236+0000 7f2ec5690740 20 client.26896 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:28.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.236+0000 7f2ec5690740 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558d6f116f50 0x558d6f119370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.236+0000 7f2ec5690740 4 client.26896 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:28.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.236+0000 7f2ec5690740 1 -- 172.21.15.5:0/238112749 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2eb4037040 con 0x558d6f116f50 2024-09-15T22:20:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.237+0000 7f2ec1854640 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558d6f116f50 0x558d6f119370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.238+0000 7f2ec1854640 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558d6f116f50 0x558d6f119370 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f02809f9e8d2ad7 server_cookie=1577eab891c69a81 in_seq=0 out_seq=0 2024-09-15T22:20:28.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.238+0000 7f2ea4ff9640 10 client.26896 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/238112749 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2eb4037040 con 0x558d6f116f50 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ea4ff9640 10 client.26896 handle_command_reply: tid=0 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ec5690740 1 client.26896 shutdown 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ec5690740 1 -- 172.21.15.5:0/238112749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 msgr2=0x558d6f104680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ec5690740 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f104680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2eac009fa0 tx=0x7f2eac010800 comp rx=0 tx=0).stop 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ec5690740 1 -- 172.21.15.5:0/238112749 shutdown_connections 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ec5690740 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f107240 0x558d6f104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ec5690740 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f104680 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ec5690740 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558d6f116f50 0x558d6f119370 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ea4ff9640 0 client.26896 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ec5690740 1 --2- 172.21.15.5:0/238112749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc060 0x558d6f104140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ec5690740 1 -- 172.21.15.5:0/238112749 >> 172.21.15.5:0/238112749 conn(0x558d6f0f85f0 msgr2=0x558d6f0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:28.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ec5690740 1 -- 172.21.15.5:0/238112749 shutdown_connections 2024-09-15T22:20:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.239+0000 7f2ec5690740 1 -- 172.21.15.5:0/238112749 wait complete. 2024-09-15T22:20:28.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.240+0000 7f2ec5690740 20 client.26896 trim_cache size 0 max 16384 2024-09-15T22:20:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.260+0000 7f2ec5690740 1 Processor -- start 2024-09-15T22:20:28.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.260+0000 7f2ec5690740 1 -- start start 2024-09-15T22:20:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.260+0000 7f2ec5690740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 0x558d6f0eed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.260+0000 7f2ec5690740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f102a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.260+0000 7f2ec5690740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f107240 0x558d6f104a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec5690740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1af810 con 0x558d6f0fc060 2024-09-15T22:20:28.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec5690740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1ad0e0 con 0x558d6f107240 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec5690740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1aaf50 con 0x558d6f0fc980 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec1854640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 0x558d6f0eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec0852640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec1854640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 0x558d6f0eed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36884/0 (socket says 172.21.15.5:36884) 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec1854640 1 -- 172.21.15.5:0/1966406619 learned_addr learned my addr 172.21.15.5:0/1966406619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec0852640 1 --2- 172.21.15.5:0/1966406619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f102a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:34626/0 (socket says 172.21.15.5:34626) 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec1854640 1 --2- 172.21.15.5:0/1966406619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f107240 0x558d6f104a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec0852640 1 -- 172.21.15.5:0/1966406619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f107240 msgr2=0x558d6f104a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec0852640 1 --2- 172.21.15.5:0/1966406619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f107240 0x558d6f104a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec0852640 1 -- 172.21.15.5:0/1966406619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 msgr2=0x558d6f0eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec0852640 1 --2- 172.21.15.5:0/1966406619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 0x558d6f0eed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec0852640 1 -- 172.21.15.5:0/1966406619 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d6f0eb3f0 con 0x558d6f0fc980 2024-09-15T22:20:28.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.261+0000 7f2ec1854640 1 --2- 172.21.15.5:0/1966406619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 0x558d6f0eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:28.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.262+0000 7f2ec0852640 1 --2- 172.21.15.5:0/1966406619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f102a10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2eac004700 tx=0x7f2eac009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6278517c08b3d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.262+0000 7f2ea6ffd640 1 -- 172.21.15.5:0/1966406619 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac02c070 con 0x558d6f0fc980 2024-09-15T22:20:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.262+0000 7f2ea6ffd640 1 -- 172.21.15.5:0/1966406619 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eac011240 con 0x558d6f0fc980 2024-09-15T22:20:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.262+0000 7f2ea6ffd640 1 -- 172.21.15.5:0/1966406619 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac021040 con 0x558d6f0fc980 2024-09-15T22:20:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.263+0000 7f2ec5690740 1 -- 172.21.15.5:0/1966406619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 msgr2=0x558d6f102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.263+0000 7f2ec5690740 1 --2- 172.21.15.5:0/1966406619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f102a10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2eac004700 tx=0x7f2eac009fd0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.263+0000 7f2ec5690740 1 -- 172.21.15.5:0/1966406619 shutdown_connections 2024-09-15T22:20:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.263+0000 7f2ec5690740 1 --2- 172.21.15.5:0/1966406619 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f107240 0x558d6f104a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.263+0000 7f2ec5690740 1 --2- 172.21.15.5:0/1966406619 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f0fc980 0x558d6f102a10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.263+0000 7f2ec5690740 1 --2- 172.21.15.5:0/1966406619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 0x558d6f0eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.263+0000 7f2ec5690740 1 -- 172.21.15.5:0/1966406619 >> 172.21.15.5:0/1966406619 conn(0x558d6f0f85f0 msgr2=0x558d6f0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.263+0000 7f2ec5690740 1 -- 172.21.15.5:0/1966406619 shutdown_connections 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.263+0000 7f2ec5690740 1 -- 172.21.15.5:0/1966406619 wait complete. 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.263+0000 7f2ec5690740 1 -- start start 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.263+0000 7f2ec5690740 1 Processor -- start 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec5690740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 0x558d6f10bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec5690740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc980 0x558d6f10c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec5690740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f107240 0x558d6f179b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec5690740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1ab5a0 con 0x558d6f0fc060 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec5690740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1aca10 con 0x558d6f0fc980 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec5690740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1abc70 con 0x558d6f107240 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec5690740 10 client.0 _ll_register_callbacks cb 0x7f2eac009390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec0852640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc980 0x558d6f10c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec1854640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 0x558d6f10bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec0852640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc980 0x558d6f10c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:44682/0 (socket says 172.21.15.5:44682) 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec0852640 1 -- 172.21.15.5:0/724703658 learned_addr learned my addr 172.21.15.5:0/724703658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec1854640 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f107240 0x558d6f179b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec0852640 1 -- 172.21.15.5:0/724703658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f107240 msgr2=0x558d6f179b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec0852640 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f107240 0x558d6f179b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec0852640 1 -- 172.21.15.5:0/724703658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 msgr2=0x558d6f10bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec0852640 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 0x558d6f10bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec0852640 1 -- 172.21.15.5:0/724703658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d6f17a3c0 con 0x558d6f0fc980 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.264+0000 7f2ec1854640 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 0x558d6f10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.265+0000 7f2ec0852640 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc980 0x558d6f10c040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2eac001eb0 tx=0x7f2eac005ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=334da90291efda25 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.265+0000 7f2ea4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.265+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/724703658 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac035070 con 0x558d6f0fc980 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.265+0000 7f2ec5690740 1 -- 172.21.15.5:0/724703658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2eac011240 con 0x558d6f0fc980 2024-09-15T22:20:28.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.265+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/724703658 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f2eac0055f0 con 0x558d6f0fc980 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.265+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/724703658 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2eac041050 con 0x558d6f0fc980 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.265+0000 7f2ec5690740 10 client.26966 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.265+0000 7f2ec5690740 1 -- 172.21.15.5:0/724703658 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d6f1826f0 con 0x558d6f0fc980 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.265+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/724703658 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2eac011240 con 0x558d6f0fc980 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.265+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/724703658 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2eac0047a0 con 0x558d6f0fc980 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.266+0000 7f2ec5690740 10 client.26966 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.266+0000 7f2ec5690740 10 client.26966 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.266+0000 7f2ec5690740 20 client.26966 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.266+0000 7f2ec5690740 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558d6f182e80 0x558d6f1852a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.266+0000 7f2ec5690740 4 client.26966 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}] 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.266+0000 7f2ec5690740 1 -- 172.21.15.5:0/724703658 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.85, "await_for": 45.0}) -- 0x558d6f065890 con 0x558d6f182e80 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.266+0000 7f2ec1854640 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558d6f182e80 0x558d6f1852a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:28.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.267+0000 7f2ec1854640 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558d6f182e80 0x558d6f1852a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f79d1396f47490f server_cookie=fed201431f5dbf88 in_seq=0 out_seq=0 2024-09-15T22:20:28.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.267+0000 7f2ea4ff9640 10 client.26966 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ea4ff9640 1 -- 172.21.15.5:0/724703658 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558d6f065890 con 0x558d6f182e80 2024-09-15T22:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ea4ff9640 10 client.26966 handle_command_reply: tid=0 2024-09-15T22:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 1 client.26966 shutdown 2024-09-15T22:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 1 -- 172.21.15.5:0/724703658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc980 msgr2=0x558d6f10c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc980 0x558d6f10c040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2eac001eb0 tx=0x7f2eac005ae0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 1 -- 172.21.15.5:0/724703658 shutdown_connections 2024-09-15T22:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558d6f107240 0x558d6f179b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558d6f182e80 0x558d6f1852a0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558d6f0fc980 0x558d6f10c040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ea4ff9640 0 client.26966 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 1 --2- 172.21.15.5:0/724703658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558d6f0fc060 0x558d6f10bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 1 -- 172.21.15.5:0/724703658 >> 172.21.15.5:0/724703658 conn(0x558d6f0f85f0 msgr2=0x558d6f112000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 1 -- 172.21.15.5:0/724703658 shutdown_connections 2024-09-15T22:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 1 -- 172.21.15.5:0/724703658 wait complete. 2024-09-15T22:20:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.642+0000 7f2ec5690740 20 client.26966 trim_cache size 0 max 16384 2024-09-15T22:20:28.646 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:20:28.646 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:20:28.646 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:20:28.646 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 42, 2024-09-15T22:20:28.646 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:20:28.646 INFO:teuthology.orchestra.run.smithi005.stdout: "3e6abb11": { 2024-09-15T22:20:28.646 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 42, 2024-09-15T22:20:28.646 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:20:28.646 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 141.8, 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:28.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:28.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:28.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:28.648 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:20:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 -- 172.21.15.5:0/233724186 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ea80a3580 msgr2=0x7f2ea80a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ea80a3580 0x7f2ea80a5a40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x558d6f0f9450 tx=0x7f2eb80d0700 comp rx=0 tx=0).stop 2024-09-15T22:20:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 -- 172.21.15.5:0/233724186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc100e20 msgr2=0x7f2ebc196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc100e20 0x7f2ebc196280 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800e9c0 tx=0x7f2eb800ee80 comp rx=0 tx=0).stop 2024-09-15T22:20:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 -- 172.21.15.5:0/233724186 shutdown_connections 2024-09-15T22:20:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2ea80a3580 0x7f2ea80a5a40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2ebc102ef0 0x7f2ebc19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2ebc102570 0x7f2ebc1967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 --2- 172.21.15.5:0/233724186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ebc100e20 0x7f2ebc196280 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 -- 172.21.15.5:0/233724186 >> 172.21.15.5:0/233724186 conn(0x7f2ebc0fc7d0 msgr2=0x7f2ebc104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 -- 172.21.15.5:0/233724186 shutdown_connections 2024-09-15T22:20:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:28.643+0000 7f2ec32b7640 1 -- 172.21.15.5:0/233724186 wait complete. 2024-09-15T22:20:28.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:28 smithi005 ceph-mon[30944]: pgmap v360: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-15T22:20:29.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:28 smithi099 ceph-mon[34892]: pgmap v360: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-15T22:20:29.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:28 smithi129 ceph-mon[35314]: pgmap v360: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-15T22:20:29.307 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3e6abb11', quiesce took 6.8 seconds. Will release after: 15.05 2024-09-15T22:20:29.662 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 660.0 s, 355.9 tps, lat 207.764 ms stddev 267.304 2024-09-15T22:20:31.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:30 smithi099 ceph-mon[34892]: pgmap v361: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.3 MiB/s wr, 164 op/s 2024-09-15T22:20:31.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:30 smithi129 ceph-mon[35314]: pgmap v361: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.3 MiB/s wr, 164 op/s 2024-09-15T22:20:31.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:30 smithi005 ceph-mon[30944]: pgmap v361: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.3 MiB/s wr, 164 op/s 2024-09-15T22:20:33.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:32 smithi099 ceph-mon[34892]: pgmap v362: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-09-15T22:20:33.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:32 smithi129 ceph-mon[35314]: pgmap v362: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-09-15T22:20:33.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:32 smithi005 ceph-mon[30944]: pgmap v362: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 3.5 MiB/s wr, 281 op/s 2024-09-15T22:20:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 665.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:20:35.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:34 smithi129 ceph-mon[35314]: pgmap v363: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 3.7 MiB/s wr, 286 op/s 2024-09-15T22:20:35.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:34 smithi005 ceph-mon[30944]: pgmap v363: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 3.7 MiB/s wr, 286 op/s 2024-09-15T22:20:35.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:34 smithi099 ceph-mon[34892]: pgmap v363: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 3.7 MiB/s wr, 286 op/s 2024-09-15T22:20:37.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:36 smithi129 ceph-mon[35314]: pgmap v364: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 3.5 MiB/s wr, 282 op/s 2024-09-15T22:20:37.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:36 smithi005 ceph-mon[30944]: pgmap v364: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 3.5 MiB/s wr, 282 op/s 2024-09-15T22:20:37.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:36 smithi099 ceph-mon[34892]: pgmap v364: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 3.5 MiB/s wr, 282 op/s 2024-09-15T22:20:39.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:38 smithi005 ceph-mon[30944]: pgmap v365: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 182 op/s 2024-09-15T22:20:39.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:38 smithi099 ceph-mon[34892]: pgmap v365: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 182 op/s 2024-09-15T22:20:39.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:38 smithi129 ceph-mon[35314]: pgmap v365: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 2.0 MiB/s wr, 182 op/s 2024-09-15T22:20:39.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 670.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:20:41.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:40 smithi129 ceph-mon[35314]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-15T22:20:41.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:40 smithi005 ceph-mon[30944]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-15T22:20:41.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:40 smithi099 ceph-mon[34892]: pgmap v366: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 1.4 MiB/s wr, 126 op/s 2024-09-15T22:20:43.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:42 smithi005 ceph-mon[30944]: pgmap v367: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-15T22:20:43.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:42 smithi099 ceph-mon[34892]: pgmap v367: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-15T22:20:43.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:42 smithi129 ceph-mon[35314]: pgmap v367: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-15T22:20:44.358 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:20:44.631 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:44.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 675.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:20:45.054 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:44 smithi005 ceph-mon[30944]: pgmap v368: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 223 KiB/s wr, 12 op/s 2024-09-15T22:20:45.146 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:20:45.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.136+0000 7fe048f0b640 1 -- 172.21.15.5:0/3868273151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe044106a80 msgr2=0x7fe044106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:45.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.136+0000 7fe048f0b640 1 --2- 172.21.15.5:0/3868273151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe044106a80 0x7fe044106e60 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fe034009930 tx=0x7fe03402a420 comp rx=0 tx=0).stop 2024-09-15T22:20:45.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.136+0000 7fe048f0b640 1 -- 172.21.15.5:0/3868273151 shutdown_connections 2024-09-15T22:20:45.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.136+0000 7fe048f0b640 1 --2- 172.21.15.5:0/3868273151 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044101420 0x7fe04410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:45.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.136+0000 7fe048f0b640 1 --2- 172.21.15.5:0/3868273151 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe044100a80 0x7fe044100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:45.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.136+0000 7fe048f0b640 1 --2- 172.21.15.5:0/3868273151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe044106a80 0x7fe044106e60 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.136+0000 7fe048f0b640 1 -- 172.21.15.5:0/3868273151 >> 172.21.15.5:0/3868273151 conn(0x7fe0440fc7a0 msgr2=0x7fe0440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.136+0000 7fe048f0b640 1 -- 172.21.15.5:0/3868273151 shutdown_connections 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.136+0000 7fe048f0b640 1 -- 172.21.15.5:0/3868273151 wait complete. 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.137+0000 7fe048f0b640 1 Processor -- start 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.137+0000 7fe048f0b640 1 -- start start 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.137+0000 7fe048f0b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044100a80 0x7fe04410e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.137+0000 7fe048f0b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe044101420 0x7fe04410e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.137+0000 7fe048f0b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe044106a80 0x7fe04410ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.137+0000 7fe048f0b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe044111fb0 con 0x7fe044101420 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.137+0000 7fe048f0b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe044111e30 con 0x7fe044106a80 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.137+0000 7fe048f0b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe044112130 con 0x7fe044100a80 2024-09-15T22:20:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe044101420 0x7fe04410e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe044101420 0x7fe04410e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49036/0 (socket says 172.21.15.5:49036) 2024-09-15T22:20:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03bfff640 1 -- 172.21.15.5:0/618674747 learned_addr learned my addr 172.21.15.5:0/618674747 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe04259c640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044100a80 0x7fe04410e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe042d9d640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe044106a80 0x7fe04410ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03bfff640 1 -- 172.21.15.5:0/618674747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044100a80 msgr2=0x7fe04410e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03bfff640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044100a80 0x7fe04410e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03bfff640 1 -- 172.21.15.5:0/618674747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe044106a80 msgr2=0x7fe04410ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03bfff640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe044106a80 0x7fe04410ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03bfff640 1 -- 172.21.15.5:0/618674747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0441a6930 con 0x7fe044101420 2024-09-15T22:20:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03bfff640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe044101420 0x7fe04410e7a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c890 tx=0x7fe02c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03b7fe640 1 -- 172.21.15.5:0/618674747 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c004220 con 0x7fe044101420 2024-09-15T22:20:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03b7fe640 1 -- 172.21.15.5:0/618674747 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe02c0043c0 con 0x7fe044101420 2024-09-15T22:20:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe048f0b640 1 -- 172.21.15.5:0/618674747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0441a6c20 con 0x7fe044101420 2024-09-15T22:20:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe03b7fe640 1 -- 172.21.15.5:0/618674747 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c0049e0 con 0x7fe044101420 2024-09-15T22:20:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.138+0000 7fe048f0b640 1 -- 172.21.15.5:0/618674747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0441a71b0 con 0x7fe044101420 2024-09-15T22:20:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.139+0000 7fe048f0b640 1 -- 172.21.15.5:0/618674747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe044100ee0 con 0x7fe044101420 2024-09-15T22:20:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.141+0000 7fe03b7fe640 1 -- 172.21.15.5:0/618674747 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe02c0079b0 con 0x7fe044101420 2024-09-15T22:20:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.141+0000 7fe03b7fe640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0300a3430 0x7fe0300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.141+0000 7fe04259c640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0300a3430 0x7fe0300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.141+0000 7fe03b7fe640 1 -- 172.21.15.5:0/618674747 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7fe02c0d00e0 con 0x7fe044101420 2024-09-15T22:20:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.142+0000 7fe04259c640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0300a3430 0x7fe0300a58f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe034009900 tx=0x7fe0340058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.145+0000 7fe03b7fe640 1 -- 172.21.15.5:0/618674747 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe02c014030 con 0x7fe044101420 2024-09-15T22:20:45.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:44 smithi099 ceph-mon[34892]: pgmap v368: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 223 KiB/s wr, 12 op/s 2024-09-15T22:20:45.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:44 smithi129 ceph-mon[35314]: pgmap v368: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 223 KiB/s wr, 12 op/s 2024-09-15T22:20:45.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.363+0000 7fe048f0b640 1 -- 172.21.15.5:0/618674747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe04410f790 con 0x7fe044101420 2024-09-15T22:20:45.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.364+0000 7fe03b7fe640 1 -- 172.21.15.5:0/618674747 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fe02c09c740 con 0x7fe044101420 2024-09-15T22:20:45.366 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:20:45.366 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:20:45.367 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:20:45.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 -- 172.21.15.5:0/618674747 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0300a3430 msgr2=0x7fe0300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:45.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0300a3430 0x7fe0300a58f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe034009900 tx=0x7fe0340058d0 comp rx=0 tx=0).stop 2024-09-15T22:20:45.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 -- 172.21.15.5:0/618674747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe044101420 msgr2=0x7fe04410e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:45.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe044101420 0x7fe04410e7a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c890 tx=0x7fe02c00cd50 comp rx=0 tx=0).stop 2024-09-15T22:20:45.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 -- 172.21.15.5:0/618674747 shutdown_connections 2024-09-15T22:20:45.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe0300a3430 0x7fe0300a58f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:45.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe044106a80 0x7fe04410ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:45.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe044101420 0x7fe04410e7a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:45.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 --2- 172.21.15.5:0/618674747 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe044100a80 0x7fe04410e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:45.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 -- 172.21.15.5:0/618674747 >> 172.21.15.5:0/618674747 conn(0x7fe0440fc7a0 msgr2=0x7fe0440fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:45.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 -- 172.21.15.5:0/618674747 shutdown_connections 2024-09-15T22:20:45.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:45.367+0000 7fe048f0b640 1 -- 172.21.15.5:0/618674747 wait complete. 2024-09-15T22:20:46.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:45 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/618674747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:46.177 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id 3e6abb11 --release --await' 2024-09-15T22:20:46.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id 3e6abb11 --release --await 2024-09-15T22:20:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:46.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:45 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/618674747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:46.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:45 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/618674747' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.721+0000 7f0668089640 1 -- 172.21.15.5:0/1569291658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0660101de0 msgr2=0x7f06601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.721+0000 7f0668089640 1 --2- 172.21.15.5:0/1569291658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0660101de0 0x7f06601021c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f0650009ca0 tx=0x7f065002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 -- 172.21.15.5:0/1569291658 shutdown_connections 2024-09-15T22:20:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 --2- 172.21.15.5:0/1569291658 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660106f10 0x7f066010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 --2- 172.21.15.5:0/1569291658 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0660102790 0x7f06601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 --2- 172.21.15.5:0/1569291658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0660101de0 0x7f06601021c0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 -- 172.21.15.5:0/1569291658 >> 172.21.15.5:0/1569291658 conn(0x7f06600fd8c0 msgr2=0x7f06600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 -- 172.21.15.5:0/1569291658 shutdown_connections 2024-09-15T22:20:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 -- 172.21.15.5:0/1569291658 wait complete. 2024-09-15T22:20:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 Processor -- start 2024-09-15T22:20:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 -- start start 2024-09-15T22:20:46.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0660101de0 0x7f06601984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660102790 0x7f06601989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0660106f10 0x7f066019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0660111f70 con 0x7f0660106f10 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0660111df0 con 0x7f0660102790 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.722+0000 7f0668089640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06601120f0 con 0x7f0660101de0 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0666626640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0660106f10 0x7f066019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0666626640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0660106f10 0x7f066019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49064/0 (socket says 172.21.15.5:49064) 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0666626640 1 -- 172.21.15.5:0/279214208 learned_addr learned my addr 172.21.15.5:0/279214208 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0665e25640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0660101de0 0x7f06601984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0665624640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660102790 0x7f06601989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0666626640 1 -- 172.21.15.5:0/279214208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0660101de0 msgr2=0x7f06601984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0666626640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0660101de0 0x7f06601984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0666626640 1 -- 172.21.15.5:0/279214208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660102790 msgr2=0x7f06601989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0666626640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660102790 0x7f06601989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0666626640 1 -- 172.21.15.5:0/279214208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066019d460 con 0x7f0660106f10 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0665624640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660102790 0x7f06601989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0666626640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0660106f10 0x7f066019cd80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f065800d750 tx=0x7f065800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f065effd640 1 -- 172.21.15.5:0/279214208 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658017070 con 0x7f0660106f10 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f065effd640 1 -- 172.21.15.5:0/279214208 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0658005500 con 0x7f0660106f10 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0668089640 1 -- 172.21.15.5:0/279214208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066019d750 con 0x7f0660106f10 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f065effd640 1 -- 172.21.15.5:0/279214208 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0658004d90 con 0x7f0660106f10 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.723+0000 7f0668089640 1 -- 172.21.15.5:0/279214208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06601a5030 con 0x7f0660106f10 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.726+0000 7f065effd640 1 -- 172.21.15.5:0/279214208 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0658004f30 con 0x7f0660106f10 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.726+0000 7f066a462740 1 Processor -- start 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.726+0000 7f066a462740 1 -- start start 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.726+0000 7f066a462740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed1a360 0x55dfaed1a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.726+0000 7f066a462740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed29550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.726+0000 7f066a462740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed29a90 0x55dfaed2be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.726+0000 7f066a462740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dfaec802e0 con 0x55dfaed1ac80 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.726+0000 7f066a462740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dfaec27ce0 con 0x55dfaed29a90 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.726+0000 7f066a462740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dfaec7c2a0 con 0x55dfaed1a360 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.726+0000 7f065effd640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06480a3430 0x7f06480a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665e25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed29550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665e25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed29550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49074/0 (socket says 172.21.15.5:49074) 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665e25640 1 -- 172.21.15.5:0/2124293816 learned_addr learned my addr 172.21.15.5:0/2124293816 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665624640 1 --2- 172.21.15.5:0/2124293816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed1a360 0x55dfaed1a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0666626640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06480a3430 0x7f06480a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f065effd640 1 -- 172.21.15.5:0/279214208 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f06580cfa30 con 0x7f0660106f10 2024-09-15T22:20:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665624640 1 --2- 172.21.15.5:0/2124293816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed29a90 0x55dfaed2be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665e25640 1 -- 172.21.15.5:0/2124293816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed1a360 msgr2=0x55dfaed1a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665e25640 1 --2- 172.21.15.5:0/2124293816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed1a360 0x55dfaed1a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665e25640 1 -- 172.21.15.5:0/2124293816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed29a90 msgr2=0x55dfaed2be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665e25640 1 --2- 172.21.15.5:0/2124293816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed29a90 0x55dfaed2be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665e25640 1 -- 172.21.15.5:0/2124293816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfaec0c440 con 0x55dfaed1ac80 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665624640 1 --2- 172.21.15.5:0/2124293816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed29a90 0x55dfaed2be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0665e25640 1 --2- 172.21.15.5:0/2124293816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed29550 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0650009bf0 tx=0x7f0650005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9932838b2223b25 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0666626640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06480a3430 0x7f06480a58f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f065801d000 tx=0x7f0658004830 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0647fff640 1 -- 172.21.15.5:0/2124293816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650049070 con 0x55dfaed1ac80 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0647fff640 1 -- 172.21.15.5:0/2124293816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0650036040 con 0x55dfaed1ac80 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.727+0000 7f0647fff640 1 -- 172.21.15.5:0/2124293816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065002fb50 con 0x55dfaed1ac80 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.728+0000 7f066a462740 1 -- 172.21.15.5:0/2124293816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 msgr2=0x55dfaed29550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.728+0000 7f066a462740 1 --2- 172.21.15.5:0/2124293816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed29550 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f0650009bf0 tx=0x7f0650005fa0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.728+0000 7f066a462740 1 -- 172.21.15.5:0/2124293816 shutdown_connections 2024-09-15T22:20:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.728+0000 7f066a462740 1 --2- 172.21.15.5:0/2124293816 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed29a90 0x55dfaed2be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.728+0000 7f066a462740 1 --2- 172.21.15.5:0/2124293816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed29550 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.728+0000 7f066a462740 1 --2- 172.21.15.5:0/2124293816 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed1a360 0x55dfaed1a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.728+0000 7f066a462740 1 -- 172.21.15.5:0/2124293816 >> 172.21.15.5:0/2124293816 conn(0x55dfaed16a30 msgr2=0x55dfaed16e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.728+0000 7f066a462740 1 -- 172.21.15.5:0/2124293816 shutdown_connections 2024-09-15T22:20:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.728+0000 7f066a462740 1 -- 172.21.15.5:0/2124293816 wait complete. 2024-09-15T22:20:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.729+0000 7f066a462740 1 -- start start 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.729+0000 7f066a462740 1 Processor -- start 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f066a462740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1a360 0x55dfaed29420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f066a462740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f066a462740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed26c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f066a462740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dfaed2c480 con 0x55dfaed1ac80 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f066a462740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55dfaec3b050 con 0x55dfaed1a360 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f066a462740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55dfaed2c600 con 0x55dfaed29a90 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f066a462740 10 client.0 _ll_register_callbacks cb 0x55dfaec7b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f0665e25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f0665e25640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed266d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49088/0 (socket says 172.21.15.5:49088) 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f0665e25640 1 -- 172.21.15.5:0/18856341 learned_addr learned my addr 172.21.15.5:0/18856341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f0665624640 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1a360 0x55dfaed29420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f0665624640 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed26c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f0665e25640 1 -- 172.21.15.5:0/18856341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 msgr2=0x55dfaed26c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f0665e25640 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed26c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f0665e25640 1 -- 172.21.15.5:0/18856341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1a360 msgr2=0x55dfaed29420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f0665e25640 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1a360 0x55dfaed29420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f0665e25640 1 -- 172.21.15.5:0/18856341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfae9f13c0 con 0x55dfaed1ac80 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f0665e25640 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed266d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0650009620 tx=0x7f065002f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74541403268d116f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.730+0000 7f06457fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f06457fa640 1 -- 172.21.15.5:0/18856341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650053060 con 0x55dfaed1ac80 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f06457fa640 1 -- 172.21.15.5:0/18856341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f065002f8d0 con 0x55dfaed1ac80 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f066a462740 1 -- 172.21.15.5:0/18856341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0650036040 con 0x55dfaed1ac80 2024-09-15T22:20:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f06457fa640 1 -- 172.21.15.5:0/18856341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0650049070 con 0x55dfaed1ac80 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f066a462740 10 client.17493 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f066a462740 1 -- 172.21.15.5:0/18856341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfaed2c480 con 0x55dfaed1ac80 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f06457fa640 1 -- 172.21.15.5:0/18856341 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0650036040 con 0x55dfaed1ac80 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f06457fa640 1 -- 172.21.15.5:0/18856341 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0650030c00 con 0x55dfaed1ac80 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f066a462740 10 client.17493 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f066a462740 10 client.17493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f066a462740 20 client.17493 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f066a462740 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55dfaed38fa0 0x55dfaed3b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f066a462740 4 client.17493 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.731+0000 7f066a462740 1 -- 172.21.15.5:0/18856341 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfaec8d850 con 0x55dfaed38fa0 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.732+0000 7f0665624640 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55dfaed38fa0 0x55dfaed3b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.732+0000 7f06457fa640 10 client.17493 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.732+0000 7f0665624640 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55dfaed38fa0 0x55dfaed3b3c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc78de3954ab7e3b server_cookie=704b60760827ff8a in_seq=0 out_seq=0 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f06457fa640 1 -- 172.21.15.5:0/18856341 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfaec8d850 con 0x55dfaed38fa0 2024-09-15T22:20:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f06457fa640 10 client.17493 handle_command_reply: tid=0 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 1 client.17493 shutdown 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 1 -- 172.21.15.5:0/18856341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 msgr2=0x55dfaed266d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed266d0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f0650009620 tx=0x7f065002f6b0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 1 -- 172.21.15.5:0/18856341 shutdown_connections 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed26c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1ac80 0x55dfaed266d0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55dfaed38fa0 0x55dfaed3b3c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 1 --2- 172.21.15.5:0/18856341 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1a360 0x55dfaed29420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f06457fa640 0 client.17493 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 1 -- 172.21.15.5:0/18856341 >> 172.21.15.5:0/18856341 conn(0x55dfaed16a30 msgr2=0x55dfaed1c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 1 -- 172.21.15.5:0/18856341 shutdown_connections 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 1 -- 172.21.15.5:0/18856341 wait complete. 2024-09-15T22:20:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.733+0000 7f066a462740 20 client.17493 trim_cache size 0 max 16384 2024-09-15T22:20:46.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.751+0000 7f066a462740 1 Processor -- start 2024-09-15T22:20:46.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.751+0000 7f066a462740 1 -- start start 2024-09-15T22:20:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f066a462740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed35720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f066a462740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1ac80 0x55dfaed40c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f066a462740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed41190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f066a462740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06601af700 con 0x55dfaed1a360 2024-09-15T22:20:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f066a462740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06601acfd0 con 0x55dfaed1ac80 2024-09-15T22:20:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f066a462740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06601aae40 con 0x55dfaed29a90 2024-09-15T22:20:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f0665624640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed35720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f0665624640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed35720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49100/0 (socket says 172.21.15.5:49100) 2024-09-15T22:20:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f0665624640 1 -- 172.21.15.5:0/3900424123 learned_addr learned my addr 172.21.15.5:0/3900424123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f0665624640 1 --2- 172.21.15.5:0/3900424123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed41190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f0665e25640 1 --2- 172.21.15.5:0/3900424123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1ac80 0x55dfaed40c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f0665624640 1 -- 172.21.15.5:0/3900424123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 msgr2=0x55dfaed41190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f0665624640 1 --2- 172.21.15.5:0/3900424123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed41190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f0665624640 1 -- 172.21.15.5:0/3900424123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1ac80 msgr2=0x55dfaed40c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f0665624640 1 --2- 172.21.15.5:0/3900424123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1ac80 0x55dfaed40c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f0665624640 1 -- 172.21.15.5:0/3900424123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfaed1aa70 con 0x55dfaed1a360 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.752+0000 7f0665e25640 1 --2- 172.21.15.5:0/3900424123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1ac80 0x55dfaed40c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.753+0000 7f0665624640 1 --2- 172.21.15.5:0/3900424123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed35720 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x55dfaed3bbf0 tx=0x7f065400b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc713d016af73c03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.753+0000 7f06477fe640 1 -- 172.21.15.5:0/3900424123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654025070 con 0x55dfaed1a360 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.753+0000 7f06477fe640 1 -- 172.21.15.5:0/3900424123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0654012aa0 con 0x55dfaed1a360 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.753+0000 7f06477fe640 1 -- 172.21.15.5:0/3900424123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654010b10 con 0x55dfaed1a360 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.754+0000 7f066a462740 1 -- 172.21.15.5:0/3900424123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 msgr2=0x55dfaed35720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.754+0000 7f066a462740 1 --2- 172.21.15.5:0/3900424123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed35720 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x55dfaed3bbf0 tx=0x7f065400b690 comp rx=0 tx=0).stop 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.754+0000 7f066a462740 1 -- 172.21.15.5:0/3900424123 shutdown_connections 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.754+0000 7f066a462740 1 --2- 172.21.15.5:0/3900424123 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed41190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.754+0000 7f066a462740 1 --2- 172.21.15.5:0/3900424123 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1ac80 0x55dfaed40c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.754+0000 7f066a462740 1 --2- 172.21.15.5:0/3900424123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed35720 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.754+0000 7f066a462740 1 -- 172.21.15.5:0/3900424123 >> 172.21.15.5:0/3900424123 conn(0x55dfaed16a30 msgr2=0x55dfaed21d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.754+0000 7f066a462740 1 -- 172.21.15.5:0/3900424123 shutdown_connections 2024-09-15T22:20:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.754+0000 7f066a462740 1 -- 172.21.15.5:0/3900424123 wait complete. 2024-09-15T22:20:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.755+0000 7f066a462740 1 -- start start 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.755+0000 7f066a462740 1 Processor -- start 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.755+0000 7f066a462740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed31c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.755+0000 7f066a462740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1ac80 0x55dfaed32150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.755+0000 7f066a462740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed2d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.755+0000 7f066a462740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06601ab490 con 0x55dfaed1a360 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.755+0000 7f066a462740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06601ac900 con 0x55dfaed1ac80 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.755+0000 7f066a462740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06601abb60 con 0x55dfaed29a90 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f066a462740 10 client.0 _ll_register_callbacks cb 0x7f0650047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f0665624640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed31c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f0665624640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed31c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49114/0 (socket says 172.21.15.5:49114) 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f0665624640 1 -- 172.21.15.5:0/2387684708 learned_addr learned my addr 172.21.15.5:0/2387684708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f0665624640 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed2d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f0665e25640 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1ac80 0x55dfaed32150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f0665624640 1 -- 172.21.15.5:0/2387684708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 msgr2=0x55dfaed2d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f0665624640 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed2d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f0665624640 1 -- 172.21.15.5:0/2387684708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1ac80 msgr2=0x55dfaed32150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f0665624640 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1ac80 0x55dfaed32150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f0665624640 1 -- 172.21.15.5:0/2387684708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfaed2da10 con 0x55dfaed1a360 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f0665624640 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed31c10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f065400bcf0 tx=0x7f06540056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8217706e19511ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f06457fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.756+0000 7f06457fa640 1 -- 172.21.15.5:0/2387684708 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654025040 con 0x55dfaed1a360 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f06457fa640 1 -- 172.21.15.5:0/2387684708 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f06540125c0 con 0x55dfaed1a360 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f066a462740 1 -- 172.21.15.5:0/2387684708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0654012aa0 con 0x55dfaed1a360 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f06457fa640 1 -- 172.21.15.5:0/2387684708 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f06540127c0 con 0x55dfaed1a360 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f066a462740 10 client.17499 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f066a462740 1 -- 172.21.15.5:0/2387684708 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dfaeda3890 con 0x55dfaed1a360 2024-09-15T22:20:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f06457fa640 1 -- 172.21.15.5:0/2387684708 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0654014500 con 0x55dfaed1a360 2024-09-15T22:20:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f06457fa640 1 -- 172.21.15.5:0/2387684708 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f0654010a30 con 0x55dfaed1a360 2024-09-15T22:20:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f066a462740 10 client.17499 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f066a462740 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f066a462740 20 client.17499 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f066a462740 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55dfaeda3f50 0x55dfaeda6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f066a462740 4 client.17499 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:20:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.757+0000 7f066a462740 1 -- 172.21.15.5:0/2387684708 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0654012aa0 con 0x55dfaeda3f50 2024-09-15T22:20:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.758+0000 7f0665e25640 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55dfaeda3f50 0x55dfaeda6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:46.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f06457fa640 10 client.17499 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:46.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f0665e25640 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55dfaeda3f50 0x55dfaeda6370 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=710433e651c7b04d server_cookie=fb2e7e696624de2 in_seq=0 out_seq=0 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f06457fa640 1 -- 172.21.15.5:0/2387684708 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f0654012aa0 con 0x55dfaeda3f50 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f06457fa640 10 client.17499 handle_command_reply: tid=0 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f066a462740 1 client.17499 shutdown 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f066a462740 1 -- 172.21.15.5:0/2387684708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 msgr2=0x55dfaed31c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f066a462740 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed31c10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f065400bcf0 tx=0x7f06540056d0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f066a462740 1 -- 172.21.15.5:0/2387684708 shutdown_connections 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f066a462740 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55dfaed29a90 0x55dfaed2d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f066a462740 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55dfaeda3f50 0x55dfaeda6370 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f066a462740 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55dfaed1ac80 0x55dfaed32150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f066a462740 1 --2- 172.21.15.5:0/2387684708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfaed1a360 0x55dfaed31c10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f066a462740 1 -- 172.21.15.5:0/2387684708 >> 172.21.15.5:0/2387684708 conn(0x55dfaed16a30 msgr2=0x55dfaed24fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f06457fa640 0 client.17499 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f066a462740 1 -- 172.21.15.5:0/2387684708 shutdown_connections 2024-09-15T22:20:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.759+0000 7f066a462740 1 -- 172.21.15.5:0/2387684708 wait complete. 2024-09-15T22:20:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.760+0000 7f066a462740 20 client.17499 trim_cache size 0 max 16384 2024-09-15T22:20:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:20:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:20:46.765 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:20:46.765 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:20:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.760+0000 7f0668089640 1 -- 172.21.15.5:0/279214208 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06480a3430 msgr2=0x7f06480a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.760+0000 7f0668089640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06480a3430 0x7f06480a58f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f065801d000 tx=0x7f0658004830 comp rx=0 tx=0).stop 2024-09-15T22:20:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.760+0000 7f0668089640 1 -- 172.21.15.5:0/279214208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0660106f10 msgr2=0x7f066019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.760+0000 7f0668089640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0660106f10 0x7f066019cd80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f065800d750 tx=0x7f065800dc10 comp rx=0 tx=0).stop 2024-09-15T22:20:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.760+0000 7f0668089640 1 -- 172.21.15.5:0/279214208 shutdown_connections 2024-09-15T22:20:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.760+0000 7f0668089640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06480a3430 0x7f06480a58f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.760+0000 7f0668089640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0660106f10 0x7f066019cd80 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.760+0000 7f0668089640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0660102790 0x7f06601989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.760+0000 7f0668089640 1 --2- 172.21.15.5:0/279214208 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0660101de0 0x7f06601984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.760+0000 7f0668089640 1 -- 172.21.15.5:0/279214208 >> 172.21.15.5:0/279214208 conn(0x7f06600fd8c0 msgr2=0x7f06600fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.761+0000 7f0668089640 1 -- 172.21.15.5:0/279214208 shutdown_connections 2024-09-15T22:20:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:46.761+0000 7f0668089640 1 -- 172.21.15.5:0/279214208 wait complete. 2024-09-15T22:20:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:47.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:46 smithi005 ceph-mon[30944]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-15T22:20:47.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:46 smithi099 ceph-mon[34892]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-15T22:20:47.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:46 smithi129 ceph-mon[35314]: pgmap v369: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-15T22:20:47.563 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:20:47.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:20:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.102+0000 7f5005a81640 1 -- 172.21.15.5:0/3610748551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5000102ef0 msgr2=0x7f500010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.102+0000 7f5005a81640 1 --2- 172.21.15.5:0/3610748551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5000102ef0 0x7f500010da40 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009ca0 tx=0x7f4ff802a950 comp rx=0 tx=0).stop 2024-09-15T22:20:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.102+0000 7f5005a81640 1 -- 172.21.15.5:0/3610748551 shutdown_connections 2024-09-15T22:20:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.102+0000 7f5005a81640 1 --2- 172.21.15.5:0/3610748551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5000102ef0 0x7f500010da40 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.102+0000 7f5005a81640 1 --2- 172.21.15.5:0/3610748551 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000102570 0x7f50001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.102+0000 7f5005a81640 1 --2- 172.21.15.5:0/3610748551 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5000100e20 0x7f5000101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.102+0000 7f5005a81640 1 -- 172.21.15.5:0/3610748551 >> 172.21.15.5:0/3610748551 conn(0x7f50000fc7d0 msgr2=0x7f50000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:48.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.102+0000 7f5005a81640 1 -- 172.21.15.5:0/3610748551 shutdown_connections 2024-09-15T22:20:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.102+0000 7f5005a81640 1 -- 172.21.15.5:0/3610748551 wait complete. 2024-09-15T22:20:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f5005a81640 1 Processor -- start 2024-09-15T22:20:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f5005a81640 1 -- start start 2024-09-15T22:20:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f5005a81640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000100e20 0x7f50000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f5005a81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5000102570 0x7f50000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f5005a81640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5000102ef0 0x7f50000fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f5005a81640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5000110f80 con 0x7f5000102570 2024-09-15T22:20:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f5005a81640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5000110e00 con 0x7f5000102ef0 2024-09-15T22:20:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f5005a81640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5000111100 con 0x7f5000100e20 2024-09-15T22:20:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5000102570 0x7f50000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5000102570 0x7f50000fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49132/0 (socket says 172.21.15.5:49132) 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f4fff7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000100e20 0x7f50000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.103+0000 7f4ff7fff640 1 -- 172.21.15.5:0/3575839141 learned_addr learned my addr 172.21.15.5:0/3575839141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f4ffffff640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5000102ef0 0x7f50000fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f4ff7fff640 1 -- 172.21.15.5:0/3575839141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000100e20 msgr2=0x7f50000fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f4ff7fff640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000100e20 0x7f50000fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f4ff7fff640 1 -- 172.21.15.5:0/3575839141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5000102ef0 msgr2=0x7f50000fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f4ff7fff640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5000102ef0 0x7f50000fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f4ff7fff640 1 -- 172.21.15.5:0/3575839141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50001a68f0 con 0x7f5000102570 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f4fff7fe640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000100e20 0x7f50000fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f4ff7fff640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5000102570 0x7f50000fd1c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800c920 tx=0x7f4fe800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f4ffd7fa640 1 -- 172.21.15.5:0/3575839141 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8007c70 con 0x7f5000102570 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f4ffd7fa640 1 -- 172.21.15.5:0/3575839141 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe8007e10 con 0x7f5000102570 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f4ffd7fa640 1 -- 172.21.15.5:0/3575839141 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe8018890 con 0x7f5000102570 2024-09-15T22:20:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f5005a81640 1 -- 172.21.15.5:0/3575839141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50001a6be0 con 0x7f5000102570 2024-09-15T22:20:48.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.104+0000 7f5005a81640 1 -- 172.21.15.5:0/3575839141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50001a7120 con 0x7f5000102570 2024-09-15T22:20:48.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.107+0000 7f5007e5a740 1 Processor -- start 2024-09-15T22:20:48.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.107+0000 7f5007e5a740 1 -- start start 2024-09-15T22:20:48.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.107+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2a9060 0x5609ba2a9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.107+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9980 0x5609ba2b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2b4240 0x5609ba2b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f5007e5a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609ba20e620 con 0x5609ba2a9060 2024-09-15T22:20:48.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f5007e5a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5609ba1b5e50 con 0x5609ba2b4240 2024-09-15T22:20:48.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f5007e5a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5609ba20a5e0 con 0x5609ba2a9980 2024-09-15T22:20:48.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4ffffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2a9060 0x5609ba2a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4ffffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2a9060 0x5609ba2a9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49136/0 (socket says 172.21.15.5:49136) 2024-09-15T22:20:48.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4fff7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9980 0x5609ba2b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4ffffff640 1 -- 172.21.15.5:0/2233895498 learned_addr learned my addr 172.21.15.5:0/2233895498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:48.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4fff7fe640 1 -- 172.21.15.5:0/2233895498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2b4240 msgr2=0x5609ba2b6630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:20:48.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4fff7fe640 1 --2- 172.21.15.5:0/2233895498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2b4240 0x5609ba2b6630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4fff7fe640 1 -- 172.21.15.5:0/2233895498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2a9060 msgr2=0x5609ba2a9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:48.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4fff7fe640 1 --2- 172.21.15.5:0/2233895498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2a9060 0x5609ba2a9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4fff7fe640 1 -- 172.21.15.5:0/2233895498 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609ba1a6d40 con 0x5609ba2a9980 2024-09-15T22:20:48.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4ffd7fa640 1 -- 172.21.15.5:0/3575839141 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4fe8018a70 con 0x7f5000102570 2024-09-15T22:20:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4ffffff640 1 --2- 172.21.15.5:0/2233895498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2a9060 0x5609ba2a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.108+0000 7f4fff7fe640 1 --2- 172.21.15.5:0/2233895498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9980 0x5609ba2b3d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0005fd0 tx=0x7f4ff000bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5951472f68ed4472 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.109+0000 7f4ff5ffb640 1 -- 172.21.15.5:0/2233895498 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff0011700 con 0x5609ba2a9980 2024-09-15T22:20:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.109+0000 7f4ff5ffb640 1 -- 172.21.15.5:0/2233895498 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff0011d80 con 0x5609ba2a9980 2024-09-15T22:20:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.109+0000 7f4ff5ffb640 1 -- 172.21.15.5:0/2233895498 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff001a5e0 con 0x5609ba2a9980 2024-09-15T22:20:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.109+0000 7f4ffd7fa640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4fec0a33b0 0x7f4fec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.109+0000 7f4ffffff640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4fec0a33b0 0x7f4fec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.109+0000 7f4ffd7fa640 1 -- 172.21.15.5:0/3575839141 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f4fe80d0140 con 0x7f5000102570 2024-09-15T22:20:48.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.110+0000 7f5007e5a740 1 -- 172.21.15.5:0/2233895498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9980 msgr2=0x5609ba2b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:48.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.110+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2233895498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9980 0x5609ba2b3d00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ff0005fd0 tx=0x7f4ff000bf90 comp rx=0 tx=0).stop 2024-09-15T22:20:48.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.110+0000 7f4ffffff640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4fec0a33b0 0x7f4fec0a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8005fe0 tx=0x7f4ff8005c50 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.110+0000 7f5007e5a740 1 -- 172.21.15.5:0/2233895498 shutdown_connections 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.110+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2233895498 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2b4240 0x5609ba2b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.110+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2233895498 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9980 0x5609ba2b3d00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.110+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2233895498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2a9060 0x5609ba2a9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.110+0000 7f5007e5a740 1 -- 172.21.15.5:0/2233895498 >> 172.21.15.5:0/2233895498 conn(0x5609ba2a55f0 msgr2=0x5609ba2a59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.110+0000 7f5007e5a740 1 -- 172.21.15.5:0/2233895498 shutdown_connections 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.110+0000 7f5007e5a740 1 -- 172.21.15.5:0/2233895498 wait complete. 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.110+0000 7f5007e5a740 1 -- start start 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f5007e5a740 1 Processor -- start 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9060 0x5609ba2b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 0x5609ba2b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f5007e5a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609ba2b6e10 con 0x5609ba2b4240 2024-09-15T22:20:48.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f5007e5a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5609ba196f10 con 0x5609ba2a9980 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f5007e5a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5609ba2b6f90 con 0x5609ba2a9060 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f5007e5a740 10 client.0 _ll_register_callbacks cb 0x5609ba1badb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 0x5609ba2b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f4ff7fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 0x5609ba2b16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:48944/0 (socket says 172.21.15.5:48944) 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f4ff7fff640 1 -- 172.21.15.5:0/2300830129 learned_addr learned my addr 172.21.15.5:0/2300830129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.111+0000 7f4ffffff640 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f4ff7fff640 1 -- 172.21.15.5:0/2300830129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9060 msgr2=0x5609ba2b1170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f4ff7fff640 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9060 0x5609ba2b1170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f4ff7fff640 1 -- 172.21.15.5:0/2300830129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 msgr2=0x5609ba2b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f4ff7fff640 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2b1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f4ff7fff640 1 -- 172.21.15.5:0/2300830129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609b9f1d580 con 0x5609ba2a9980 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f4ff7fff640 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 0x5609ba2b16b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80cda90 tx=0x7f4fe809c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12c10da335c8f0a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:48.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f4fdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2300830129 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe80a4c90 con 0x5609ba2a9980 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2300830129 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f4fe8018d60 con 0x5609ba2a9980 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f5007e5a740 1 -- 172.21.15.5:0/2300830129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff0011d80 con 0x5609ba2a9980 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2300830129 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4fe800fb30 con 0x5609ba2a9980 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f5007e5a740 10 client.26993 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.112+0000 7f5007e5a740 1 -- 172.21.15.5:0/2300830129 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5609b9fe7960 con 0x5609ba2a9980 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.113+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2300830129 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fe8098690 con 0x5609ba2a9980 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.113+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2300830129 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4fe8096b90 con 0x5609ba2a9980 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.113+0000 7f5007e5a740 10 client.26993 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.113+0000 7f5007e5a740 10 client.26993 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.113+0000 7f5007e5a740 20 client.26993 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.113+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5609ba2c3de0 0x5609ba2c6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.113+0000 7f5007e5a740 4 client.26993 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:48.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.113+0000 7f5007e5a740 1 -- 172.21.15.5:0/2300830129 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4ff0011d80 con 0x5609ba2c3de0 2024-09-15T22:20:48.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.113+0000 7f4fff7fe640 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5609ba2c3de0 0x5609ba2c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.114+0000 7f4fdeffd640 10 client.26993 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:48.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.114+0000 7f4fff7fe640 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5609ba2c3de0 0x5609ba2c6200 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c99a3b2532dc66fa server_cookie=b4737bb141b05c19 in_seq=0 out_seq=0 2024-09-15T22:20:48.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2300830129 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4ff0011d80 con 0x5609ba2c3de0 2024-09-15T22:20:48.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f4fdeffd640 10 client.26993 handle_command_reply: tid=0 2024-09-15T22:20:48.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 1 client.26993 shutdown 2024-09-15T22:20:48.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 1 -- 172.21.15.5:0/2300830129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 msgr2=0x5609ba2b16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:48.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 0x5609ba2b16b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4fe80cda90 tx=0x7f4fe809c7e0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 1 -- 172.21.15.5:0/2300830129 shutdown_connections 2024-09-15T22:20:48.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5609ba2c3de0 0x5609ba2c6200 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 0x5609ba2b16b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2300830129 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9060 0x5609ba2b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f4fdeffd640 0 client.26993 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:48.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 1 -- 172.21.15.5:0/2300830129 >> 172.21.15.5:0/2300830129 conn(0x5609ba2a55f0 msgr2=0x5609ba2ab320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:48.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 1 -- 172.21.15.5:0/2300830129 shutdown_connections 2024-09-15T22:20:48.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 1 -- 172.21.15.5:0/2300830129 wait complete. 2024-09-15T22:20:48.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.115+0000 7f5007e5a740 20 client.26993 trim_cache size 0 max 16384 2024-09-15T22:20:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.137+0000 7f5007e5a740 1 Processor -- start 2024-09-15T22:20:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.137+0000 7f5007e5a740 1 -- start start 2024-09-15T22:20:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.137+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9060 0x5609ba2ab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.137+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 0x5609ba2cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.137+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2cc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.137+0000 7f5007e5a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50001b3b80 con 0x5609ba2b4240 2024-09-15T22:20:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.137+0000 7f5007e5a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50001b1450 con 0x5609ba2a9980 2024-09-15T22:20:48.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.137+0000 7f5007e5a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50001af2c0 con 0x5609ba2a9060 2024-09-15T22:20:48.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.137+0000 7f4ffffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.137+0000 7f4ffffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49160/0 (socket says 172.21.15.5:49160) 2024-09-15T22:20:48.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4ffffff640 1 -- 172.21.15.5:0/827664917 learned_addr learned my addr 172.21.15.5:0/827664917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:48.140 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:48.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4ff7fff640 1 --2- 172.21.15.5:0/827664917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 0x5609ba2cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4fff7fe640 1 --2- 172.21.15.5:0/827664917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9060 0x5609ba2ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4ffffff640 1 -- 172.21.15.5:0/827664917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9060 msgr2=0x5609ba2ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4ffffff640 1 --2- 172.21.15.5:0/827664917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9060 0x5609ba2ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4ffffff640 1 -- 172.21.15.5:0/827664917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 msgr2=0x5609ba2cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4ffffff640 1 --2- 172.21.15.5:0/827664917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 0x5609ba2cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4ffffff640 1 -- 172.21.15.5:0/827664917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609ba2a6760 con 0x5609ba2b4240 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4ffffff640 1 --2- 172.21.15.5:0/827664917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2cc010 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009ca0 tx=0x7f4ff8042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc32fc8bbe27af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4ff57fa640 1 -- 172.21.15.5:0/827664917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8056070 con 0x5609ba2b4240 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4ff57fa640 1 -- 172.21.15.5:0/827664917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff8003d70 con 0x5609ba2b4240 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.138+0000 7f4ff57fa640 1 -- 172.21.15.5:0/827664917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8034040 con 0x5609ba2b4240 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.139+0000 7f5007e5a740 1 -- 172.21.15.5:0/827664917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 msgr2=0x5609ba2cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.139+0000 7f5007e5a740 1 --2- 172.21.15.5:0/827664917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2cc010 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8009ca0 tx=0x7f4ff8042000 comp rx=0 tx=0).stop 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.139+0000 7f5007e5a740 1 -- 172.21.15.5:0/827664917 shutdown_connections 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.139+0000 7f5007e5a740 1 --2- 172.21.15.5:0/827664917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2cc010 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.139+0000 7f5007e5a740 1 --2- 172.21.15.5:0/827664917 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9980 0x5609ba2cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.139+0000 7f5007e5a740 1 --2- 172.21.15.5:0/827664917 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9060 0x5609ba2ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.139+0000 7f5007e5a740 1 -- 172.21.15.5:0/827664917 >> 172.21.15.5:0/827664917 conn(0x5609ba2a55f0 msgr2=0x5609ba2aab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.140+0000 7f5007e5a740 1 -- 172.21.15.5:0/827664917 shutdown_connections 2024-09-15T22:20:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.140+0000 7f5007e5a740 1 -- 172.21.15.5:0/827664917 wait complete. 2024-09-15T22:20:48.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.140+0000 7f5007e5a740 1 -- start start 2024-09-15T22:20:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.140+0000 7f5007e5a740 1 Processor -- start 2024-09-15T22:20:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.140+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9060 0x5609ba2bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.140+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9980 0x5609ba2b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f5007e5a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2b8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f5007e5a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50001af910 con 0x5609ba2b4240 2024-09-15T22:20:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f5007e5a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f50001b0d80 con 0x5609ba2a9060 2024-09-15T22:20:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f5007e5a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f50001affe0 con 0x5609ba2a9980 2024-09-15T22:20:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f5007e5a740 10 client.0 _ll_register_callbacks cb 0x7f4fe809bc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4ffffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2b8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4fff7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9060 0x5609ba2bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4fff7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9060 0x5609ba2bc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:48960/0 (socket says 172.21.15.5:48960) 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4fff7fe640 1 -- 172.21.15.5:0/2174315326 learned_addr learned my addr 172.21.15.5:0/2174315326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4ffffff640 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2b8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49164/0 (socket says 172.21.15.5:49164) 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4ff7fff640 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9980 0x5609ba2b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4fff7fe640 1 -- 172.21.15.5:0/2174315326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9980 msgr2=0x5609ba2b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4fff7fe640 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9980 0x5609ba2b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4fff7fe640 1 -- 172.21.15.5:0/2174315326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 msgr2=0x5609ba2b8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4fff7fe640 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2b8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4fff7fe640 1 -- 172.21.15.5:0/2174315326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609ba2a6280 con 0x5609ba2a9060 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4ffffff640 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2b8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4fff7fe640 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9060 0x5609ba2bc6d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4ff001cbb0 tx=0x7f4ff001cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=220ad6fae324fa20 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.141+0000 7f4fdeffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2174315326 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff001c690 con 0x5609ba2a9060 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2174315326 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f4ff0005040 con 0x5609ba2a9060 2024-09-15T22:20:48.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f5007e5a740 1 -- 172.21.15.5:0/2174315326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ff8003d70 con 0x5609ba2a9060 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2174315326 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ff001c830 con 0x5609ba2a9060 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f5007e5a740 10 client.26999 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f5007e5a740 1 -- 172.21.15.5:0/2174315326 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5609ba32f690 con 0x5609ba2a9060 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2174315326 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ff8003d70 con 0x5609ba2a9060 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2174315326 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4ff0025050 con 0x5609ba2a9060 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f5007e5a740 10 client.26999 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f5007e5a740 10 client.26999 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f5007e5a740 20 client.26999 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5609ba32fa80 0x5609ba331e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f5007e5a740 4 client.26999 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3e6abb11", "release": true, "await": true}] 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.142+0000 7f5007e5a740 1 -- 172.21.15.5:0/2174315326 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3e6abb11", "release": true, "await": true}) -- 0x5609ba2acd90 con 0x5609ba32fa80 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.143+0000 7f4ff7fff640 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5609ba32fa80 0x5609ba331e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.143+0000 7f4ff7fff640 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5609ba32fa80 0x5609ba331e70 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da02b01be4e8ac35 server_cookie=9419dc660b444af1 in_seq=0 out_seq=0 2024-09-15T22:20:48.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.143+0000 7f4fdeffd640 10 client.26999 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.193+0000 7f4fdeffd640 1 -- 172.21.15.5:0/2174315326 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5609ba2acd90 con 0x5609ba32fa80 2024-09-15T22:20:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.193+0000 7f4fdeffd640 10 client.26999 handle_command_reply: tid=0 2024-09-15T22:20:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.193+0000 7f5007e5a740 1 client.26999 shutdown 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.193+0000 7f5007e5a740 1 -- 172.21.15.5:0/2174315326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9060 msgr2=0x5609ba2bc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.193+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9060 0x5609ba2bc6d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4ff001cbb0 tx=0x7f4ff001cb40 comp rx=0 tx=0).stop 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.193+0000 7f5007e5a740 1 -- 172.21.15.5:0/2174315326 shutdown_connections 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.193+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609ba2b4240 0x5609ba2b8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.193+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5609ba2a9980 0x5609ba2b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.193+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5609ba32fa80 0x5609ba331e70 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.193+0000 7f5007e5a740 1 --2- 172.21.15.5:0/2174315326 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5609ba2a9060 0x5609ba2bc6d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.193+0000 7f5007e5a740 1 -- 172.21.15.5:0/2174315326 >> 172.21.15.5:0/2174315326 conn(0x5609ba2a55f0 msgr2=0x5609ba2ab320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.194+0000 7f4fdeffd640 0 client.26999 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.194+0000 7f5007e5a740 1 -- 172.21.15.5:0/2174315326 shutdown_connections 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.194+0000 7f5007e5a740 1 -- 172.21.15.5:0/2174315326 wait complete. 2024-09-15T22:20:48.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.194+0000 7f5007e5a740 20 client.26999 trim_cache size 0 max 16384 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 44, 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "3e6abb11": { 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 44, 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:20:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 141.8, 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:20:48.198 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:20:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.194+0000 7f5005a81640 1 -- 172.21.15.5:0/3575839141 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4fec0a33b0 msgr2=0x7f4fec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.194+0000 7f5005a81640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4fec0a33b0 0x7f4fec0a5870 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8005fe0 tx=0x7f4ff8005c50 comp rx=0 tx=0).stop 2024-09-15T22:20:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.194+0000 7f5005a81640 1 -- 172.21.15.5:0/3575839141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5000102570 msgr2=0x7f50000fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.194+0000 7f5005a81640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5000102570 0x7f50000fd1c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800c920 tx=0x7f4fe800cde0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.195+0000 7f5005a81640 1 -- 172.21.15.5:0/3575839141 shutdown_connections 2024-09-15T22:20:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.195+0000 7f5005a81640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4fec0a33b0 0x7f4fec0a5870 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.195+0000 7f5005a81640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5000102ef0 0x7f50000fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.195+0000 7f5005a81640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5000102570 0x7f50000fd1c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.195+0000 7f5005a81640 1 --2- 172.21.15.5:0/3575839141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5000100e20 0x7f50000fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.195+0000 7f5005a81640 1 -- 172.21.15.5:0/3575839141 >> 172.21.15.5:0/3575839141 conn(0x7f50000fc7d0 msgr2=0x7f500006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:48.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.195+0000 7f5005a81640 1 -- 172.21.15.5:0/3575839141 shutdown_connections 2024-09-15T22:20:48.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:48.195+0000 7f5005a81640 1 -- 172.21.15.5:0/3575839141 wait complete. 2024-09-15T22:20:49.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:48 smithi099 ceph-mon[34892]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-15T22:20:49.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:48 smithi129 ceph-mon[35314]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-15T22:20:49.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:48 smithi005 ceph-mon[30944]: pgmap v370: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-09-15T22:20:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 680.0 s, 137.9 tps, lat 3084.446 ms stddev 7071.991 2024-09-15T22:20:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.702+0000 7f8595595640 1 -- 172.21.15.5:0/78737581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8590103670 msgr2=0x7f859010bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.702+0000 7f8595595640 1 --2- 172.21.15.5:0/78737581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8590103670 0x7f859010bb60 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f8584009900 tx=0x7f858402a480 comp rx=0 tx=0).stop 2024-09-15T22:20:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.702+0000 7f8595595640 1 -- 172.21.15.5:0/78737581 shutdown_connections 2024-09-15T22:20:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.702+0000 7f8595595640 1 --2- 172.21.15.5:0/78737581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f859010c180 0x7f859010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.702+0000 7f8595595640 1 --2- 172.21.15.5:0/78737581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8590103670 0x7f859010bb60 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.702+0000 7f8595595640 1 --2- 172.21.15.5:0/78737581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8590102d50 0x7f8590103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.702+0000 7f8595595640 1 -- 172.21.15.5:0/78737581 >> 172.21.15.5:0/78737581 conn(0x7f85900fc7d0 msgr2=0x7f85900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.702+0000 7f8595595640 1 -- 172.21.15.5:0/78737581 shutdown_connections 2024-09-15T22:20:49.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.702+0000 7f8595595640 1 -- 172.21.15.5:0/78737581 wait complete. 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.702+0000 7f8595595640 1 Processor -- start 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.703+0000 7f8595595640 1 -- start start 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.703+0000 7f8595595640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8590102d50 0x7f85901984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.703+0000 7f8595595640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8590103670 0x7f8590198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.704+0000 7f8595595640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859010c180 0x7f859019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.704+0000 7f858e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8590103670 0x7f8590198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.704+0000 7f858e7fc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8590103670 0x7f8590198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:45902/0 (socket says 172.21.15.5:45902) 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.704+0000 7f858e7fc640 1 -- 172.21.15.5:0/1512492715 learned_addr learned my addr 172.21.15.5:0/1512492715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.704+0000 7f858f7fe640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859010c180 0x7f859019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.704+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8590111fd0 con 0x7f859010c180 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.704+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8590111e50 con 0x7f8590102d50 2024-09-15T22:20:49.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.704+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8590112150 con 0x7f8590103670 2024-09-15T22:20:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.705+0000 7f858effd640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8590102d50 0x7f85901984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.705+0000 7f858f7fe640 1 -- 172.21.15.5:0/1512492715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8590103670 msgr2=0x7f8590198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:49.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.705+0000 7f858f7fe640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8590103670 0x7f8590198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.705+0000 7f858f7fe640 1 -- 172.21.15.5:0/1512492715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8590102d50 msgr2=0x7f85901984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.705+0000 7f858f7fe640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8590102d50 0x7f85901984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.705+0000 7f858f7fe640 1 -- 172.21.15.5:0/1512492715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859019d480 con 0x7f859010c180 2024-09-15T22:20:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.705+0000 7f858effd640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8590102d50 0x7f85901984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.705+0000 7f858f7fe640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859010c180 0x7f859019cda0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8588009fd0 tx=0x7f858800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.706+0000 7f8577fff640 1 -- 172.21.15.5:0/1512492715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f858800ce20 con 0x7f859010c180 2024-09-15T22:20:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.706+0000 7f8577fff640 1 -- 172.21.15.5:0/1512492715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8588005020 con 0x7f859010c180 2024-09-15T22:20:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.706+0000 7f8577fff640 1 -- 172.21.15.5:0/1512492715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85880106b0 con 0x7f859010c180 2024-09-15T22:20:49.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.706+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859019d770 con 0x7f859010c180 2024-09-15T22:20:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.706+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85901a5050 con 0x7f859010c180 2024-09-15T22:20:49.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.708+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8590103130 con 0x7f859010c180 2024-09-15T22:20:49.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.711+0000 7f8577fff640 1 -- 172.21.15.5:0/1512492715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f85880051d0 con 0x7f859010c180 2024-09-15T22:20:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.711+0000 7f8577fff640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85780a3580 0x7f85780a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.712+0000 7f8577fff640 1 -- 172.21.15.5:0/1512492715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7f85880d0890 con 0x7f859010c180 2024-09-15T22:20:49.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.712+0000 7f858effd640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85780a3580 0x7f85780a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:49.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.712+0000 7f858effd640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85780a3580 0x7f85780a5a40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8580009b50 tx=0x7f8580005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:49.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:49.715+0000 7f8577fff640 1 -- 172.21.15.5:0/1512492715 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85880531b0 con 0x7f859010c180 2024-09-15T22:20:50.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.095+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8590199830 con 0x7f859010c180 2024-09-15T22:20:50.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.097+0000 7f8577fff640 1 -- 172.21.15.5:0/1512492715 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f8590199830 con 0x7f859010c180 2024-09-15T22:20:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:20:50.099 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:20:50.099 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:20:50.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.105+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85780a3580 msgr2=0x7f85780a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:50.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.105+0000 7f8595595640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85780a3580 0x7f85780a5a40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8580009b50 tx=0x7f8580005f90 comp rx=0 tx=0).stop 2024-09-15T22:20:50.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.105+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859010c180 msgr2=0x7f859019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:50.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.105+0000 7f8595595640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859010c180 0x7f859019cda0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8588009fd0 tx=0x7f858800ef10 comp rx=0 tx=0).stop 2024-09-15T22:20:50.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.106+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 shutdown_connections 2024-09-15T22:20:50.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.106+0000 7f8595595640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f85780a3580 0x7f85780a5a40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:50.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.106+0000 7f8595595640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f859010c180 0x7f859019cda0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:50.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.106+0000 7f8595595640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8590103670 0x7f8590198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:50.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.106+0000 7f8595595640 1 --2- 172.21.15.5:0/1512492715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8590102d50 0x7f85901984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:50.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.106+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 >> 172.21.15.5:0/1512492715 conn(0x7f85900fc7d0 msgr2=0x7f85900fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:50.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.110+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 shutdown_connections 2024-09-15T22:20:50.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:50.111+0000 7f8595595640 1 -- 172.21.15.5:0/1512492715 wait complete. 2024-09-15T22:20:50.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:20:51.012 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3e6abb11', total seconds elapsed: 28.5 2024-09-15T22:20:51.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:50 smithi099 ceph-mon[34892]: pgmap v371: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 5 op/s 2024-09-15T22:20:51.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:50 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1512492715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:51.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:50 smithi129 ceph-mon[35314]: pgmap v371: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 5 op/s 2024-09-15T22:20:51.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:50 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1512492715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:51.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:50 smithi005 ceph-mon[30944]: pgmap v371: 129 pgs: 129 active+clean; 2.9 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 12 KiB/s wr, 5 op/s 2024-09-15T22:20:51.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:50 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1512492715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:20:51.444 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:53.234 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:52 smithi005 ceph-mon[30944]: pgmap v372: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 792 KiB/s wr, 88 op/s 2024-09-15T22:20:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.312+0000 7efdb9550640 1 -- 172.21.15.5:0/3373955822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb4106ac0 msgr2=0x7efdb4106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.312+0000 7efdb9550640 1 --2- 172.21.15.5:0/3373955822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb4106ac0 0x7efdb4106ea0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efdac009be0 tx=0x7efdac02a6c0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.312+0000 7efdb9550640 1 -- 172.21.15.5:0/3373955822 shutdown_connections 2024-09-15T22:20:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.312+0000 7efdb9550640 1 --2- 172.21.15.5:0/3373955822 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdb4100970 0x7efdb4105730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.312+0000 7efdb9550640 1 --2- 172.21.15.5:0/3373955822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb40ffed0 0x7efdb4100350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.312+0000 7efdb9550640 1 --2- 172.21.15.5:0/3373955822 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb4106ac0 0x7efdb4106ea0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.312+0000 7efdb9550640 1 -- 172.21.15.5:0/3373955822 >> 172.21.15.5:0/3373955822 conn(0x7efdb4077010 msgr2=0x7efdb40fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.312+0000 7efdb9550640 1 -- 172.21.15.5:0/3373955822 shutdown_connections 2024-09-15T22:20:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.313+0000 7efdb9550640 1 -- 172.21.15.5:0/3373955822 wait complete. 2024-09-15T22:20:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.313+0000 7efdb9550640 1 Processor -- start 2024-09-15T22:20:53.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.315+0000 7efdb9550640 1 -- start start 2024-09-15T22:20:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.315+0000 7efdb9550640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb40ffed0 0x7efdb41a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.315+0000 7efdb9550640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdb4100970 0x7efdb41a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.315+0000 7efdb9550640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb4106ac0 0x7efdb419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.315+0000 7efdb9550640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdb4118420 con 0x7efdb4106ac0 2024-09-15T22:20:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.315+0000 7efdb9550640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41182a0 con 0x7efdb40ffed0 2024-09-15T22:20:53.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.315+0000 7efdb9550640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41185a0 con 0x7efdb4100970 2024-09-15T22:20:53.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb8d4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb4106ac0 0x7efdb419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb8d4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb4106ac0 0x7efdb419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49206/0 (socket says 172.21.15.5:49206) 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb8d4f640 1 -- 172.21.15.5:0/3479005853 learned_addr learned my addr 172.21.15.5:0/3479005853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb37fe640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdb4100970 0x7efdb41a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb3fff640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb40ffed0 0x7efdb41a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb37fe640 1 -- 172.21.15.5:0/3479005853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb40ffed0 msgr2=0x7efdb41a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb37fe640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb40ffed0 0x7efdb41a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb37fe640 1 -- 172.21.15.5:0/3479005853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb4106ac0 msgr2=0x7efdb419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb37fe640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb4106ac0 0x7efdb419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb37fe640 1 -- 172.21.15.5:0/3479005853 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb419efa0 con 0x7efdb4100970 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb8d4f640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb4106ac0 0x7efdb419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.316+0000 7efdb3fff640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb40ffed0 0x7efdb41a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.317+0000 7efdb37fe640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdb4100970 0x7efdb41a4ba0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efda8005fd0 tx=0x7efda800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:53.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.317+0000 7efdb17fa640 1 -- 172.21.15.5:0/3479005853 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda800cd00 con 0x7efdb4100970 2024-09-15T22:20:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.317+0000 7efdb9550640 1 -- 172.21.15.5:0/3479005853 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb419f290 con 0x7efdb4100970 2024-09-15T22:20:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.317+0000 7efdb17fa640 1 -- 172.21.15.5:0/3479005853 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda80035f0 con 0x7efdb4100970 2024-09-15T22:20:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.317+0000 7efdb17fa640 1 -- 172.21.15.5:0/3479005853 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda8010670 con 0x7efdb4100970 2024-09-15T22:20:53.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.317+0000 7efdb9550640 1 -- 172.21.15.5:0/3479005853 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb41ab5e0 con 0x7efdb4100970 2024-09-15T22:20:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.320+0000 7efdb17fa640 1 -- 172.21.15.5:0/3479005853 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7efda80037a0 con 0x7efdb4100970 2024-09-15T22:20:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdbcb8b740 1 Processor -- start 2024-09-15T22:20:53.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdbcb8b740 1 -- start start 2024-09-15T22:20:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fae780 0x564d70faebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 0x564d70fbb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d70f142f0 con 0x564d70fae780 2024-09-15T22:20:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564d70ebbcf0 con 0x564d70fb9520 2024-09-15T22:20:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564d70f102b0 con 0x564d70fafcf0 2024-09-15T22:20:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdb8d4f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdb17fa640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efd9c0a3430 0x7efd9c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdb8d4f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:45926/0 (socket says 172.21.15.5:45926) 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdb8d4f640 1 -- 172.21.15.5:0/113989016 learned_addr learned my addr 172.21.15.5:0/113989016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdb3fff640 1 --2- 172.21.15.5:0/113989016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fae780 0x564d70faebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.321+0000 7efdb37fe640 1 --2- 172.21.15.5:0/113989016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 0x564d70fbb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.322+0000 7efdb8d4f640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efd9c0a3430 0x7efd9c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.322+0000 7efdb8d4f640 1 -- 172.21.15.5:0/113989016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 msgr2=0x564d70fbb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.322+0000 7efdb8d4f640 1 --2- 172.21.15.5:0/113989016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 0x564d70fbb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.322+0000 7efdb8d4f640 1 -- 172.21.15.5:0/113989016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fae780 msgr2=0x564d70faebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.322+0000 7efdb8d4f640 1 --2- 172.21.15.5:0/113989016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fae780 0x564d70faebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.322+0000 7efdb8d4f640 1 -- 172.21.15.5:0/113989016 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d70ea03a0 con 0x564d70fafcf0 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.322+0000 7efdb17fa640 1 -- 172.21.15.5:0/3479005853 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7efda80d0310 con 0x7efdb4100970 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.322+0000 7efdb8d4f640 1 --2- 172.21.15.5:0/113989016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fae240 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efda4013cb0 tx=0x7efda400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22e8d8bd3e3804bd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:53.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.322+0000 7efdb8d4f640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efd9c0a3430 0x7efd9c0a58f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efda4024000 tx=0x7efda4022000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.323+0000 7efd9a7fc640 1 -- 172.21.15.5:0/113989016 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4023020 con 0x564d70fafcf0 2024-09-15T22:20:53.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.323+0000 7efd9a7fc640 1 -- 172.21.15.5:0/113989016 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda4013170 con 0x564d70fafcf0 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.323+0000 7efd9a7fc640 1 -- 172.21.15.5:0/113989016 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda401e3e0 con 0x564d70fafcf0 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.324+0000 7efdbcb8b740 1 -- 172.21.15.5:0/113989016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 msgr2=0x564d70fae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.324+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/113989016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fae240 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efda4013cb0 tx=0x7efda400f790 comp rx=0 tx=0).stop 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.324+0000 7efdbcb8b740 1 -- 172.21.15.5:0/113989016 shutdown_connections 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.324+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/113989016 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 0x564d70fbb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.324+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/113989016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fae780 0x564d70faebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.324+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/113989016 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fae240 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.324+0000 7efdbcb8b740 1 -- 172.21.15.5:0/113989016 >> 172.21.15.5:0/113989016 conn(0x564d70f77310 msgr2=0x564d70faae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.324+0000 7efdbcb8b740 1 -- 172.21.15.5:0/113989016 shutdown_connections 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.324+0000 7efdbcb8b740 1 -- 172.21.15.5:0/113989016 wait complete. 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.324+0000 7efdbcb8b740 1 -- start start 2024-09-15T22:20:53.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.325+0000 7efdbcb8b740 1 Processor -- start 2024-09-15T22:20:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.326+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fae780 0x564d70fb8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.326+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fafcf0 0x564d70fb61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.326+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 0x564d70fb6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.326+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d70fbc220 con 0x564d70fafcf0 2024-09-15T22:20:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.326+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564d70ecf060 con 0x564d70fb9520 2024-09-15T22:20:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.326+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564d70fbc3a0 con 0x564d70fae780 2024-09-15T22:20:53.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.326+0000 7efdbcb8b740 10 client.0 _ll_register_callbacks cb 0x564d70c228c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.328+0000 7efdb3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fae780 0x564d70fb8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.328+0000 7efdb3fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fae780 0x564d70fb8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:45930/0 (socket says 172.21.15.5:45930) 2024-09-15T22:20:53.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.328+0000 7efdb3fff640 1 -- 172.21.15.5:0/186799674 learned_addr learned my addr 172.21.15.5:0/186799674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.328+0000 7efdb8d4f640 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 0x564d70fb6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.328+0000 7efdb37fe640 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fafcf0 0x564d70fb61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.328+0000 7efdb3fff640 1 -- 172.21.15.5:0/186799674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 msgr2=0x564d70fb6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:53.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.328+0000 7efdb3fff640 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 0x564d70fb6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.328+0000 7efdb3fff640 1 -- 172.21.15.5:0/186799674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fafcf0 msgr2=0x564d70fb61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:53.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.328+0000 7efdb3fff640 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fafcf0 0x564d70fb61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.328+0000 7efdb3fff640 1 -- 172.21.15.5:0/186799674 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d70c853c0 con 0x564d70fae780 2024-09-15T22:20:53.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.329+0000 7efdb3fff640 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fae780 0x564d70fb8f40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efdac009be0 tx=0x7efdac030c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d0c9645d68a66b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:53.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.329+0000 7efd937fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:20:53.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.329+0000 7efd937fe640 1 -- 172.21.15.5:0/186799674 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdac042070 con 0x564d70fae780 2024-09-15T22:20:53.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.329+0000 7efdbcb8b740 1 -- 172.21.15.5:0/186799674 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda4013170 con 0x564d70fae780 2024-09-15T22:20:53.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.329+0000 7efd937fe640 1 -- 172.21.15.5:0/186799674 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7efdac0303a0 con 0x564d70fae780 2024-09-15T22:20:53.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.329+0000 7efd937fe640 1 -- 172.21.15.5:0/186799674 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efdac036040 con 0x564d70fae780 2024-09-15T22:20:53.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.329+0000 7efdbcb8b740 10 client.26953 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:53.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.329+0000 7efdbcb8b740 1 -- 172.21.15.5:0/186799674 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564d70fc7cf0 con 0x564d70fae780 2024-09-15T22:20:53.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.330+0000 7efd937fe640 1 -- 172.21.15.5:0/186799674 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda4013170 con 0x564d70fae780 2024-09-15T22:20:53.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.330+0000 7efd937fe640 1 -- 172.21.15.5:0/186799674 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efdac0042e0 con 0x564d70fae780 2024-09-15T22:20:53.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.330+0000 7efdbcb8b740 10 client.26953 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:53.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.330+0000 7efdbcb8b740 10 client.26953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:20:53.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.330+0000 7efdbcb8b740 20 client.26953 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:53.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.330+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x564d70fc91b0 0x564d70fcb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.330+0000 7efdbcb8b740 4 client.26953 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:53.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.330+0000 7efdbcb8b740 1 -- 172.21.15.5:0/186799674 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d70f21860 con 0x564d70fc91b0 2024-09-15T22:20:53.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.331+0000 7efdb37fe640 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x564d70fc91b0 0x564d70fcb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.331+0000 7efdb37fe640 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x564d70fc91b0 0x564d70fcb5a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6695f80e6580ab6 server_cookie=b65ff23450afa8f0 in_seq=0 out_seq=0 2024-09-15T22:20:53.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.331+0000 7efd937fe640 10 client.26953 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:20:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.336+0000 7efd937fe640 1 -- 172.21.15.5:0/186799674 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d70f21860 con 0x564d70fc91b0 2024-09-15T22:20:53.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.336+0000 7efd937fe640 10 client.26953 handle_command_reply: tid=0 2024-09-15T22:20:53.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:52 smithi099 ceph-mon[34892]: pgmap v372: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 792 KiB/s wr, 88 op/s 2024-09-15T22:20:53.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.338+0000 7efdbcb8b740 1 client.26953 shutdown 2024-09-15T22:20:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:52 smithi129 ceph-mon[35314]: pgmap v372: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 792 KiB/s wr, 88 op/s 2024-09-15T22:20:53.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.348+0000 7efdbcb8b740 1 -- 172.21.15.5:0/186799674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fae780 msgr2=0x564d70fb8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.348+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fae780 0x564d70fb8f40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efdac009be0 tx=0x7efdac030c60 comp rx=0 tx=0).stop 2024-09-15T22:20:53.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.352+0000 7efdbcb8b740 1 -- 172.21.15.5:0/186799674 shutdown_connections 2024-09-15T22:20:53.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.352+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x564d70fc91b0 0x564d70fcb5a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.352+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 0x564d70fb6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.352+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fafcf0 0x564d70fb61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.352+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/186799674 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fae780 0x564d70fb8f40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.352+0000 7efdbcb8b740 1 -- 172.21.15.5:0/186799674 >> 172.21.15.5:0/186799674 conn(0x564d70f77310 msgr2=0x564d70fb0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:53.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.353+0000 7efd937fe640 0 client.26953 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:20:53.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.355+0000 7efdbcb8b740 1 -- 172.21.15.5:0/186799674 shutdown_connections 2024-09-15T22:20:53.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.355+0000 7efdbcb8b740 1 -- 172.21.15.5:0/186799674 wait complete. 2024-09-15T22:20:53.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.357+0000 7efdbcb8b740 20 client.26953 trim_cache size 0 max 16384 2024-09-15T22:20:53.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.389+0000 7efdbcb8b740 1 Processor -- start 2024-09-15T22:20:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.393+0000 7efdbcb8b740 1 -- start start 2024-09-15T22:20:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.394+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fae780 0x564d70fb61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.394+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fc54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.394+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 0x564d70fb12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.394+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b7fe0 con 0x564d70fae780 2024-09-15T22:20:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.394+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b58b0 con 0x564d70fb9520 2024-09-15T22:20:53.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.394+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b3720 con 0x564d70fafcf0 2024-09-15T22:20:53.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.397+0000 7efdb37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fc54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.397+0000 7efdb37fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fc54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:48922/0 (socket says 172.21.15.5:48922) 2024-09-15T22:20:53.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.397+0000 7efdb37fe640 1 -- 172.21.15.5:0/125089364 learned_addr learned my addr 172.21.15.5:0/125089364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:53.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.398+0000 7efdb37fe640 1 -- 172.21.15.5:0/125089364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 msgr2=0x564d70fb12d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:20:53.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.398+0000 7efdb37fe640 1 --2- 172.21.15.5:0/125089364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 0x564d70fb12d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.398+0000 7efdb37fe640 1 -- 172.21.15.5:0/125089364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fae780 msgr2=0x564d70fb61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:53.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.398+0000 7efdb37fe640 1 --2- 172.21.15.5:0/125089364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fae780 0x564d70fb61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.398+0000 7efdb37fe640 1 -- 172.21.15.5:0/125089364 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d70fabb10 con 0x564d70fafcf0 2024-09-15T22:20:53.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.402+0000 7efdb37fe640 1 --2- 172.21.15.5:0/125089364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fc54c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efda80cd6c0 tx=0x7efda80a0040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11c14500082d45dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:53.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.407+0000 7efd99ffb640 1 -- 172.21.15.5:0/125089364 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda809c1f0 con 0x564d70fafcf0 2024-09-15T22:20:53.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.407+0000 7efd99ffb640 1 -- 172.21.15.5:0/125089364 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda80a05a0 con 0x564d70fafcf0 2024-09-15T22:20:53.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.408+0000 7efdbcb8b740 1 -- 172.21.15.5:0/125089364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 msgr2=0x564d70fc54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.408+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/125089364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fc54c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efda80cd6c0 tx=0x7efda80a0040 comp rx=0 tx=0).stop 2024-09-15T22:20:53.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.410+0000 7efdbcb8b740 1 -- 172.21.15.5:0/125089364 shutdown_connections 2024-09-15T22:20:53.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.410+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/125089364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fb9520 0x564d70fb12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.410+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/125089364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fafcf0 0x564d70fc54c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.410+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/125089364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fae780 0x564d70fb61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.410+0000 7efdbcb8b740 1 -- 172.21.15.5:0/125089364 >> 172.21.15.5:0/125089364 conn(0x564d70f77310 msgr2=0x564d70fb5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:53.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.414+0000 7efdbcb8b740 1 -- 172.21.15.5:0/125089364 shutdown_connections 2024-09-15T22:20:53.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.416+0000 7efdbcb8b740 1 -- 172.21.15.5:0/125089364 wait complete. 2024-09-15T22:20:53.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.416+0000 7efdbcb8b740 1 -- start start 2024-09-15T22:20:53.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.417+0000 7efdbcb8b740 1 Processor -- start 2024-09-15T22:20:53.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.420+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fae780 0x564d70fc1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.420+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fafcf0 0x564d70fc1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.420+0000 7efdbcb8b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fb9520 0x564d70fbcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.420+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b3d70 con 0x564d70fafcf0 2024-09-15T22:20:53.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.420+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b51e0 con 0x564d70fae780 2024-09-15T22:20:53.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.420+0000 7efdbcb8b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efdb41b4440 con 0x564d70fb9520 2024-09-15T22:20:53.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.420+0000 7efdbcb8b740 10 client.0 _ll_register_callbacks cb 0x7efda8005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.423+0000 7efdb8d4f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fb9520 0x564d70fbcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.423+0000 7efdb8d4f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fb9520 0x564d70fbcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:48926/0 (socket says 172.21.15.5:48926) 2024-09-15T22:20:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.423+0000 7efdb8d4f640 1 -- 172.21.15.5:0/3553569736 learned_addr learned my addr 172.21.15.5:0/3553569736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.423+0000 7efdb3fff640 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fae780 0x564d70fc1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.424+0000 7efdb37fe640 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fafcf0 0x564d70fc1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.426+0000 7efdb8d4f640 1 -- 172.21.15.5:0/3553569736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fae780 msgr2=0x564d70fc1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:53.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.426+0000 7efdb8d4f640 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fae780 0x564d70fc1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.426+0000 7efdb8d4f640 1 -- 172.21.15.5:0/3553569736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fafcf0 msgr2=0x564d70fc1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:53.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.426+0000 7efdb8d4f640 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fafcf0 0x564d70fc1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.426+0000 7efdb8d4f640 1 -- 172.21.15.5:0/3553569736 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d70fbd370 con 0x564d70fb9520 2024-09-15T22:20:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.430+0000 7efdb8d4f640 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fb9520 0x564d70fbcb90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efda4010ba0 tx=0x7efda4010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6a8cd65cde9ac1a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.430+0000 7efd937fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:20:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.433+0000 7efd937fe640 1 -- 172.21.15.5:0/3553569736 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda4036060 con 0x564d70fb9520 2024-09-15T22:20:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.433+0000 7efd937fe640 1 -- 172.21.15.5:0/3553569736 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7efda4010430 con 0x564d70fb9520 2024-09-15T22:20:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.434+0000 7efdbcb8b740 1 -- 172.21.15.5:0/3553569736 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda80a05a0 con 0x564d70fb9520 2024-09-15T22:20:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.439+0000 7efd937fe640 1 -- 172.21.15.5:0/3553569736 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efda4023040 con 0x564d70fb9520 2024-09-15T22:20:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.440+0000 7efdbcb8b740 10 client.26959 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:53.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.440+0000 7efdbcb8b740 1 -- 172.21.15.5:0/3553569736 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564d710448d0 con 0x564d70fb9520 2024-09-15T22:20:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.443+0000 7efd937fe640 1 -- 172.21.15.5:0/3553569736 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda80a05a0 con 0x564d70fb9520 2024-09-15T22:20:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.443+0000 7efd937fe640 1 -- 172.21.15.5:0/3553569736 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efda401fce0 con 0x564d70fb9520 2024-09-15T22:20:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.443+0000 7efdbcb8b740 10 client.26959 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.443+0000 7efdbcb8b740 10 client.26959 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:20:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.443+0000 7efdbcb8b740 20 client.26959 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.443+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x564d71044dc0 0x564d710471b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.443+0000 7efdbcb8b740 4 client.26959 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:53.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.443+0000 7efdbcb8b740 1 -- 172.21.15.5:0/3553569736 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d70f1bd70 con 0x564d71044dc0 2024-09-15T22:20:53.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.450+0000 7efdb3fff640 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x564d71044dc0 0x564d710471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.458+0000 7efd937fe640 10 client.26959 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:20:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.458+0000 7efdb3fff640 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x564d71044dc0 0x564d710471b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed5db06e8e61510e server_cookie=2e437100e252e00b in_seq=0 out_seq=0 2024-09-15T22:20:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.459+0000 7efd937fe640 1 -- 172.21.15.5:0/3553569736 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d70f1bd70 con 0x564d71044dc0 2024-09-15T22:20:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.459+0000 7efd937fe640 10 client.26959 handle_command_reply: tid=0 2024-09-15T22:20:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.459+0000 7efdbcb8b740 1 client.26959 shutdown 2024-09-15T22:20:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.461+0000 7efdbcb8b740 1 -- 172.21.15.5:0/3553569736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fb9520 msgr2=0x564d70fbcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.461+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fb9520 0x564d70fbcb90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7efda4010ba0 tx=0x7efda4010bd0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.461+0000 7efdbcb8b740 1 -- 172.21.15.5:0/3553569736 shutdown_connections 2024-09-15T22:20:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.461+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x564d71044dc0 0x564d710471b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.461+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564d70fb9520 0x564d70fbcb90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.461+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d70fafcf0 0x564d70fc1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.461+0000 7efdbcb8b740 1 --2- 172.21.15.5:0/3553569736 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564d70fae780 0x564d70fc1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.461+0000 7efd937fe640 0 client.26959 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:20:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.461+0000 7efdbcb8b740 1 -- 172.21.15.5:0/3553569736 >> 172.21.15.5:0/3553569736 conn(0x564d70f77310 msgr2=0x564d70fa9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.461+0000 7efdbcb8b740 1 -- 172.21.15.5:0/3553569736 shutdown_connections 2024-09-15T22:20:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.461+0000 7efdbcb8b740 1 -- 172.21.15.5:0/3553569736 wait complete. 2024-09-15T22:20:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.462+0000 7efdbcb8b740 20 client.26959 trim_cache size 0 max 16384 2024-09-15T22:20:53.467 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:20:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.466+0000 7efdb9550640 1 -- 172.21.15.5:0/3479005853 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efd9c0a3430 msgr2=0x7efd9c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.466+0000 7efdb9550640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efd9c0a3430 0x7efd9c0a58f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efda4024000 tx=0x7efda4022000 comp rx=0 tx=0).stop 2024-09-15T22:20:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.466+0000 7efdb9550640 1 -- 172.21.15.5:0/3479005853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdb4100970 msgr2=0x7efdb41a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.466+0000 7efdb9550640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdb4100970 0x7efdb41a4ba0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efda8005fd0 tx=0x7efda800ee50 comp rx=0 tx=0).stop 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.466+0000 7efdb9550640 1 -- 172.21.15.5:0/3479005853 shutdown_connections 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.466+0000 7efdb9550640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efd9c0a3430 0x7efd9c0a58f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.466+0000 7efdb9550640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efdb4106ac0 0x7efdb419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.466+0000 7efdb9550640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efdb4100970 0x7efdb41a4ba0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.466+0000 7efdb9550640 1 --2- 172.21.15.5:0/3479005853 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efdb40ffed0 0x7efdb41a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.466+0000 7efdb9550640 1 -- 172.21.15.5:0/3479005853 >> 172.21.15.5:0/3479005853 conn(0x7efdb4077010 msgr2=0x7efdb4103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.467+0000 7efdb9550640 1 -- 172.21.15.5:0/3479005853 shutdown_connections 2024-09-15T22:20:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:53.467+0000 7efdb9550640 1 -- 172.21.15.5:0/3479005853 wait complete. 2024-09-15T22:20:54.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:20:54.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 685.0 s, 538.2 tps, lat 188.284 ms stddev 238.934 2024-09-15T22:20:54.871 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:55.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:54 smithi099 ceph-mon[34892]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-09-15T22:20:55.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:54 smithi129 ceph-mon[35314]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-09-15T22:20:55.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:54 smithi005 ceph-mon[30944]: pgmap v373: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-09-15T22:20:57.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.012+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/4247715925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5ac103740 msgr2=0x7fc5ac107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:57.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.012+0000 7fc5b1fd4640 1 --2- 172.21.15.5:0/4247715925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5ac103740 0x7fc5ac107b80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0009860 tx=0x7fc5a002a360 comp rx=0 tx=0).stop 2024-09-15T22:20:57.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.012+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/4247715925 shutdown_connections 2024-09-15T22:20:57.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.012+0000 7fc5b1fd4640 1 --2- 172.21.15.5:0/4247715925 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5ac108290 0x7fc5ac10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.012+0000 7fc5b1fd4640 1 --2- 172.21.15.5:0/4247715925 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5ac103740 0x7fc5ac107b80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.012+0000 7fc5b1fd4640 1 --2- 172.21.15.5:0/4247715925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5ac102d90 0x7fc5ac103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.012+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/4247715925 >> 172.21.15.5:0/4247715925 conn(0x7fc5ac0fc7d0 msgr2=0x7fc5ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:57.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.012+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/4247715925 shutdown_connections 2024-09-15T22:20:57.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.012+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/4247715925 wait complete. 2024-09-15T22:20:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.013+0000 7fc5b1fd4640 1 Processor -- start 2024-09-15T22:20:57.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.013+0000 7fc5b1fd4640 1 -- start start 2024-09-15T22:20:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.014+0000 7fc5b1fd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5ac102d90 0x7fc5ac1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.014+0000 7fc5b1fd4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5ac103740 0x7fc5ac198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.014+0000 7fc5b1fd4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5ac108290 0x7fc5ac19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.014+0000 7fc5b1fd4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac112090 con 0x7fc5ac102d90 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.014+0000 7fc5b1fd4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac111f10 con 0x7fc5ac108290 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.014+0000 7fc5b1fd4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac112210 con 0x7fc5ac103740 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.014+0000 7fc5ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5ac102d90 0x7fc5ac1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.014+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5ac103740 0x7fc5ac198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.014+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5ac103740 0x7fc5ac198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:48946/0 (socket says 172.21.15.5:48946) 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.014+0000 7fc5aaffd640 1 -- 172.21.15.5:0/172179463 learned_addr learned my addr 172.21.15.5:0/172179463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.014+0000 7fc5abfff640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5ac108290 0x7fc5ac19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.015+0000 7fc5aaffd640 1 -- 172.21.15.5:0/172179463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5ac108290 msgr2=0x7fc5ac19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.015+0000 7fc5aaffd640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5ac108290 0x7fc5ac19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.015+0000 7fc5aaffd640 1 -- 172.21.15.5:0/172179463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5ac102d90 msgr2=0x7fc5ac1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.015+0000 7fc5aaffd640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5ac102d90 0x7fc5ac1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.015+0000 7fc5aaffd640 1 -- 172.21.15.5:0/172179463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac19d560 con 0x7fc5ac103740 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.015+0000 7fc5aaffd640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5ac103740 0x7fc5ac198af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0035040 tx=0x7fc5a0030d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.015+0000 7fc5a8ff9640 1 -- 172.21.15.5:0/172179463 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a003f020 con 0x7fc5ac103740 2024-09-15T22:20:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.015+0000 7fc5a8ff9640 1 -- 172.21.15.5:0/172179463 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a0032cd0 con 0x7fc5ac103740 2024-09-15T22:20:57.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.015+0000 7fc5a8ff9640 1 -- 172.21.15.5:0/172179463 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0037040 con 0x7fc5ac103740 2024-09-15T22:20:57.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.015+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/172179463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac19d7f0 con 0x7fc5ac103740 2024-09-15T22:20:57.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.015+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/172179463 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac19dc10 con 0x7fc5ac103740 2024-09-15T22:20:57.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.018+0000 7fc5a8ff9640 1 -- 172.21.15.5:0/172179463 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc5a0042020 con 0x7fc5ac103740 2024-09-15T22:20:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.019+0000 7fc5b43ad740 1 Processor -- start 2024-09-15T22:20:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.020+0000 7fc5b43ad740 1 -- start start 2024-09-15T22:20:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.020+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 0x563027129090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.020+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 0x563027129a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.020+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 0x563027136760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.020+0000 7fc5b43ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56302708f2f0 con 0x56302712ab40 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.020+0000 7fc5a8ff9640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5940a3430 0x7fc5940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.020+0000 7fc5ab7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 0x563027129a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.020+0000 7fc5ab7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 0x563027129a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:48956/0 (socket says 172.21.15.5:48956) 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.021+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/1470638950 learned_addr learned my addr 172.21.15.5:0/1470638950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.021+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1470638950 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563027036cf0 con 0x563027134370 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.021+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1470638950 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56302708b2b0 con 0x5630271295d0 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.021+0000 7fc5abfff640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5940a3430 0x7fc5940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.021+0000 7fc5abfff640 1 --2- 172.21.15.5:0/1470638950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 0x563027129090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.021+0000 7fc5a8ff9640 1 -- 172.21.15.5:0/172179463 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7fc5a00efe70 con 0x7fc5ac103740 2024-09-15T22:20:57.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.021+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/1470638950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 msgr2=0x563027136760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.021+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/1470638950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 0x563027136760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.021+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/1470638950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 msgr2=0x563027129090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.021+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/1470638950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 0x563027129090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.021+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/1470638950 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302701b3a0 con 0x5630271295d0 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.022+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/1470638950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 0x563027129a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009cd0 tx=0x7fc59c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84d78cf8b3743d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.022+0000 7fc591ffb640 1 -- 172.21.15.5:0/1470638950 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c01e070 con 0x5630271295d0 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.022+0000 7fc591ffb640 1 -- 172.21.15.5:0/1470638950 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c019b10 con 0x5630271295d0 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.022+0000 7fc591ffb640 1 -- 172.21.15.5:0/1470638950 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c00eab0 con 0x5630271295d0 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.023+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1470638950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 msgr2=0x563027129a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.023+0000 7fc5abfff640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5940a3430 0x7fc5940a58f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009270 tx=0x7fc5a40091e0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.023+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/1470638950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 0x563027129a10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009cd0 tx=0x7fc59c011810 comp rx=0 tx=0).stop 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.023+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1470638950 shutdown_connections 2024-09-15T22:20:57.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.023+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/1470638950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 0x563027136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.023+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/1470638950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 0x563027129a10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.023+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/1470638950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 0x563027129090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.023+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1470638950 >> 172.21.15.5:0/1470638950 conn(0x5630270f2310 msgr2=0x563027125c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.023+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1470638950 shutdown_connections 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.023+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1470638950 wait complete. 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5b43ad740 1 -- start start 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5b43ad740 1 Processor -- start 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027133e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56302712ab40 0x5630271310c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563027134370 0x563027131850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5b43ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56302713b480 con 0x5630271295d0 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5b43ad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56302704a060 con 0x56302712ab40 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5b43ad740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56302713b600 con 0x563027134370 2024-09-15T22:20:57.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5b43ad740 10 client.0 _ll_register_callbacks cb 0x563026d9d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027133e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027133e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58890/0 (socket says 172.21.15.5:58890) 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/2295318257 learned_addr learned my addr 172.21.15.5:0/2295318257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5aaffd640 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56302712ab40 0x5630271310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.024+0000 7fc5abfff640 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563027134370 0x563027131850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.025+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/2295318257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563027134370 msgr2=0x563027131850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.025+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563027134370 0x563027131850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.025+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/2295318257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56302712ab40 msgr2=0x5630271310c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.025+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56302712ab40 0x5630271310c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.025+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/2295318257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563026e003c0 con 0x5630271295d0 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.025+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027133e10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc59c01c040 tx=0x7fc59c009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416d44c0d3c5aada server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.025+0000 7fc58affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.025+0000 7fc58affd640 1 -- 172.21.15.5:0/2295318257 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c01e050 con 0x5630271295d0 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.025+0000 7fc58affd640 1 -- 172.21.15.5:0/2295318257 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7fc59c005240 con 0x5630271295d0 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.025+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2295318257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc59c019b10 con 0x5630271295d0 2024-09-15T22:20:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.026+0000 7fc58affd640 1 -- 172.21.15.5:0/2295318257 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc59c033050 con 0x5630271295d0 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.026+0000 7fc5b43ad740 10 client.17550 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.026+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2295318257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563027146fd0 con 0x5630271295d0 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.026+0000 7fc58affd640 1 -- 172.21.15.5:0/2295318257 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c019b10 con 0x5630271295d0 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.026+0000 7fc58affd640 1 -- 172.21.15.5:0/2295318257 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc59c004880 con 0x5630271295d0 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.026+0000 7fc5b43ad740 10 client.17550 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.026+0000 7fc5b43ad740 10 client.17550 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.026+0000 7fc5b43ad740 20 client.17550 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.026+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5630271487f0 0x56302714abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.026+0000 7fc5b43ad740 4 client.17550 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.026+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2295318257 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56302709c860 con 0x5630271487f0 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.027+0000 7fc5aaffd640 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5630271487f0 0x56302714abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.027+0000 7fc58affd640 10 client.17550 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:20:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.027+0000 7fc5aaffd640 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5630271487f0 0x56302714abe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1cfba9122843b63b server_cookie=fb85330d5da47e89 in_seq=0 out_seq=0 2024-09-15T22:20:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc58affd640 1 -- 172.21.15.5:0/2295318257 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56302709c860 con 0x5630271487f0 2024-09-15T22:20:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc58affd640 10 client.17550 handle_command_reply: tid=0 2024-09-15T22:20:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 1 client.17550 shutdown 2024-09-15T22:20:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2295318257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 msgr2=0x563027133e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027133e10 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc59c01c040 tx=0x7fc59c009fc0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2295318257 shutdown_connections 2024-09-15T22:20:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x5630271487f0 0x56302714abe0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563027134370 0x563027131850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56302712ab40 0x5630271310c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2295318257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027133e10 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2295318257 >> 172.21.15.5:0/2295318257 conn(0x5630270f2310 msgr2=0x56302712b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc58affd640 0 client.17550 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:20:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2295318257 shutdown_connections 2024-09-15T22:20:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2295318257 wait complete. 2024-09-15T22:20:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.028+0000 7fc5b43ad740 20 client.17550 trim_cache size 0 max 16384 2024-09-15T22:20:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.044+0000 7fc5b43ad740 1 Processor -- start 2024-09-15T22:20:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.044+0000 7fc5b43ad740 1 -- start start 2024-09-15T22:20:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.044+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027124bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.044+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56302712ab40 0x5630271309a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.044+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 0x563027130ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.044+0000 7fc5ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027124bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.044+0000 7fc5ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027124bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58892/0 (socket says 172.21.15.5:58892) 2024-09-15T22:20:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.044+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/1737246715 learned_addr learned my addr 172.21.15.5:0/1737246715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.044+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1737246715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1af890 con 0x5630271295d0 2024-09-15T22:20:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.044+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1737246715 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1ad160 con 0x563027134370 2024-09-15T22:20:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.044+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1737246715 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1aafd0 con 0x56302712ab40 2024-09-15T22:20:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5aaffd640 1 --2- 172.21.15.5:0/1737246715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56302712ab40 0x5630271309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5abfff640 1 --2- 172.21.15.5:0/1737246715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 0x563027130ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/1737246715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56302712ab40 msgr2=0x5630271309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/1737246715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56302712ab40 0x5630271309a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/1737246715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 msgr2=0x563027130ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/1737246715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 0x563027130ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5ab7fe640 1 -- 172.21.15.5:0/1737246715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56302714af50 con 0x5630271295d0 2024-09-15T22:20:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5aaffd640 1 --2- 172.21.15.5:0/1737246715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56302712ab40 0x5630271309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/1737246715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027124bf0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e5e0 tx=0x7fc59c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1acdc0f18935b774 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5917fa640 1 -- 172.21.15.5:0/1737246715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c01e070 con 0x5630271295d0 2024-09-15T22:20:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5917fa640 1 -- 172.21.15.5:0/1737246715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc59c02a070 con 0x5630271295d0 2024-09-15T22:20:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.045+0000 7fc5917fa640 1 -- 172.21.15.5:0/1737246715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c025930 con 0x5630271295d0 2024-09-15T22:20:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.046+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1737246715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 msgr2=0x563027124bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.046+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/1737246715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027124bf0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00e5e0 tx=0x7fc59c009260 comp rx=0 tx=0).stop 2024-09-15T22:20:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.046+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1737246715 shutdown_connections 2024-09-15T22:20:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.046+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/1737246715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 0x563027130ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.046+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/1737246715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56302712ab40 0x5630271309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.046+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/1737246715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630271295d0 0x563027124bf0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.046+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1737246715 >> 172.21.15.5:0/1737246715 conn(0x5630270f2310 msgr2=0x5630271244e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.046+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1737246715 shutdown_connections 2024-09-15T22:20:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.046+0000 7fc5b43ad740 1 -- 172.21.15.5:0/1737246715 wait complete. 2024-09-15T22:20:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.047+0000 7fc5b43ad740 1 -- start start 2024-09-15T22:20:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.047+0000 7fc5b43ad740 1 Processor -- start 2024-09-15T22:20:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.047+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 0x56302713a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.047+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 0x563027136e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.047+0000 7fc5b43ad740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 0x563027137370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5b43ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1ab620 con 0x56302712ab40 2024-09-15T22:20:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5b43ad740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1aca90 con 0x563027134370 2024-09-15T22:20:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5b43ad740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac1abcf0 con 0x5630271295d0 2024-09-15T22:20:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5b43ad740 10 client.0 _ll_register_callbacks cb 0x7fc59c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 0x563027136e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5aaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 0x563027136e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58896/0 (socket says 172.21.15.5:58896) 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5aaffd640 1 -- 172.21.15.5:0/2562971200 learned_addr learned my addr 172.21.15.5:0/2562971200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 0x56302713a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5abfff640 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 0x563027137370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5aaffd640 1 -- 172.21.15.5:0/2562971200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 msgr2=0x56302713a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5aaffd640 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 0x56302713a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5aaffd640 1 -- 172.21.15.5:0/2562971200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 msgr2=0x563027137370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5aaffd640 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 0x563027137370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.048+0000 7fc5aaffd640 1 -- 172.21.15.5:0/2562971200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563027136ad0 con 0x56302712ab40 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.049+0000 7fc5aaffd640 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 0x563027136e30 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0000c00 tx=0x7fc5a002a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca0e304daaef687 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:57.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.049+0000 7fc58affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.049+0000 7fc58affd640 1 -- 172.21.15.5:0/2562971200 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00bc500 con 0x56302712ab40 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.049+0000 7fc58affd640 1 -- 172.21.15.5:0/2562971200 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7fc5a004bd30 con 0x56302712ab40 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.049+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2562971200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc59c02a070 con 0x56302712ab40 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.049+0000 7fc58affd640 1 -- 172.21.15.5:0/2562971200 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5a00bbe00 con 0x56302712ab40 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.049+0000 7fc5b43ad740 10 client.17556 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.049+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2562971200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630271405a0 con 0x56302712ab40 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.050+0000 7fc58affd640 1 -- 172.21.15.5:0/2562971200 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5a00c01b0 con 0x56302712ab40 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.050+0000 7fc58affd640 1 -- 172.21.15.5:0/2562971200 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc5a003a030 con 0x56302712ab40 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.050+0000 7fc5b43ad740 10 client.17556 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.050+0000 7fc5b43ad740 10 client.17556 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.050+0000 7fc5b43ad740 20 client.17556 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.050+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563027140af0 0x563027140f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:57.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.050+0000 7fc5b43ad740 4 client.17556 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.050+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2562971200 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fc59c02a070 con 0x563027140af0 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.050+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563027140af0 0x563027140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc5ab7fe640 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563027140af0 0x563027140f00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca9a25eaf1f9973d server_cookie=89c9f95a2efd7ae5 in_seq=0 out_seq=0 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc58affd640 10 client.17556 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc58affd640 1 -- 172.21.15.5:0/2562971200 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fc59c02a070 con 0x563027140af0 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc58affd640 10 client.17556 handle_command_reply: tid=0 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc5b43ad740 1 client.17556 shutdown 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2562971200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 msgr2=0x563027136e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 0x563027136e30 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0000c00 tx=0x7fc5a002a360 comp rx=0 tx=0).stop 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2562971200 shutdown_connections 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563027140af0 0x563027140f00 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563027134370 0x563027137370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56302712ab40 0x563027136e30 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc5b43ad740 1 --2- 172.21.15.5:0/2562971200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630271295d0 0x56302713a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2562971200 >> 172.21.15.5:0/2562971200 conn(0x5630270f2310 msgr2=0x563027124d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:57.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc58affd640 0 client.17556 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:20:57.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.051+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2562971200 shutdown_connections 2024-09-15T22:20:57.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.052+0000 7fc5b43ad740 1 -- 172.21.15.5:0/2562971200 wait complete. 2024-09-15T22:20:57.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.052+0000 7fc5b43ad740 20 client.17556 trim_cache size 0 max 16384 2024-09-15T22:20:57.061 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:20:57.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.052+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/172179463 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5940a3430 msgr2=0x7fc5940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:57.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.052+0000 7fc5b1fd4640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5940a3430 0x7fc5940a58f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4009270 tx=0x7fc5a40091e0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.053+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/172179463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5ac103740 msgr2=0x7fc5ac198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:57.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.053+0000 7fc5b1fd4640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5ac103740 0x7fc5ac198af0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0035040 tx=0x7fc5a0030d30 comp rx=0 tx=0).stop 2024-09-15T22:20:57.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.053+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/172179463 shutdown_connections 2024-09-15T22:20:57.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.053+0000 7fc5b1fd4640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5940a3430 0x7fc5940a58f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.053+0000 7fc5b1fd4640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5ac108290 0x7fc5ac19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.053+0000 7fc5b1fd4640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc5ac103740 0x7fc5ac198af0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.053+0000 7fc5b1fd4640 1 --2- 172.21.15.5:0/172179463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5ac102d90 0x7fc5ac1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:57.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.053+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/172179463 >> 172.21.15.5:0/172179463 conn(0x7fc5ac0fc7d0 msgr2=0x7fc5ac105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:57.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.053+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/172179463 shutdown_connections 2024-09-15T22:20:57.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:57.053+0000 7fc5b1fd4640 1 -- 172.21.15.5:0/172179463 wait complete. 2024-09-15T22:20:57.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:56 smithi099 ceph-mon[34892]: pgmap v374: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-09-15T22:20:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:56 smithi129 ceph-mon[35314]: pgmap v374: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-09-15T22:20:57.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:56 smithi005 ceph-mon[30944]: pgmap v374: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-09-15T22:20:57.760 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:20:58.278 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:20:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.916+0000 7fd048dc8640 1 -- 172.21.15.5:0/1819096652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044102ef0 msgr2=0x7fd04410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.916+0000 7fd048dc8640 1 --2- 172.21.15.5:0/1819096652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044102ef0 0x7fd04410da40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009cd0 tx=0x7fd03c02a980 comp rx=0 tx=0).stop 2024-09-15T22:20:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.918+0000 7fd048dc8640 1 -- 172.21.15.5:0/1819096652 shutdown_connections 2024-09-15T22:20:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.918+0000 7fd048dc8640 1 --2- 172.21.15.5:0/1819096652 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044102ef0 0x7fd04410da40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.918+0000 7fd048dc8640 1 --2- 172.21.15.5:0/1819096652 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd044102570 0x7fd0441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.918+0000 7fd048dc8640 1 --2- 172.21.15.5:0/1819096652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044100e20 0x7fd044101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.918+0000 7fd048dc8640 1 -- 172.21.15.5:0/1819096652 >> 172.21.15.5:0/1819096652 conn(0x7fd0440fc7d0 msgr2=0x7fd0440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:58.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.924+0000 7fd048dc8640 1 -- 172.21.15.5:0/1819096652 shutdown_connections 2024-09-15T22:20:58.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.924+0000 7fd048dc8640 1 -- 172.21.15.5:0/1819096652 wait complete. 2024-09-15T22:20:58.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.925+0000 7fd048dc8640 1 Processor -- start 2024-09-15T22:20:58.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.925+0000 7fd048dc8640 1 -- start start 2024-09-15T22:20:58.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.925+0000 7fd048dc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044100e20 0x7fd044073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.925+0000 7fd048dc8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044102570 0x7fd044073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.925+0000 7fd048dc8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd044102ef0 0x7fd04406cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.925+0000 7fd048dc8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd04410fea0 con 0x7fd044100e20 2024-09-15T22:20:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.925+0000 7fd048dc8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd04410fd20 con 0x7fd044102570 2024-09-15T22:20:58.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.925+0000 7fd048dc8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd044110020 con 0x7fd044102ef0 2024-09-15T22:20:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.927+0000 7fd04259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044100e20 0x7fd044073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.927+0000 7fd04259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044100e20 0x7fd044073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58926/0 (socket says 172.21.15.5:58926) 2024-09-15T22:20:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.927+0000 7fd04259c640 1 -- 172.21.15.5:0/1131096162 learned_addr learned my addr 172.21.15.5:0/1131096162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.927+0000 7fd04259c640 1 -- 172.21.15.5:0/1131096162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd044102ef0 msgr2=0x7fd04406cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.927+0000 7fd04259c640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd044102ef0 0x7fd04406cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.927+0000 7fd04259c640 1 -- 172.21.15.5:0/1131096162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044102570 msgr2=0x7fd044073ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.927+0000 7fd04259c640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044102570 0x7fd044073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.927+0000 7fd04259c640 1 -- 172.21.15.5:0/1131096162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04406d490 con 0x7fd044100e20 2024-09-15T22:20:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.928+0000 7fd04259c640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044100e20 0x7fd044073980 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd03400b4a0 tx=0x7fd03400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.929+0000 7fd0337fe640 1 -- 172.21.15.5:0/1131096162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034021070 con 0x7fd044100e20 2024-09-15T22:20:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.929+0000 7fd048dc8640 1 -- 172.21.15.5:0/1131096162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04406d780 con 0x7fd044100e20 2024-09-15T22:20:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.929+0000 7fd048dc8640 1 -- 172.21.15.5:0/1131096162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd044071790 con 0x7fd044100e20 2024-09-15T22:20:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.929+0000 7fd0337fe640 1 -- 172.21.15.5:0/1131096162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd03401c920 con 0x7fd044100e20 2024-09-15T22:20:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.929+0000 7fd0337fe640 1 -- 172.21.15.5:0/1131096162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03401aa20 con 0x7fd044100e20 2024-09-15T22:20:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.933+0000 7fd04b1a1740 1 Processor -- start 2024-09-15T22:20:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.934+0000 7fd04b1a1740 1 -- start start 2024-09-15T22:20:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.934+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 0x5630a1944090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.934+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 0x5630a1944a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.934+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 0x5630a1973770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.934+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630a18aa2f0 con 0x5630a1971380 2024-09-15T22:20:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.934+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5630a1851cf0 con 0x5630a1945b40 2024-09-15T22:20:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.934+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5630a18a62b0 con 0x5630a19445d0 2024-09-15T22:20:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.935+0000 7fd042d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 0x5630a1944090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.935+0000 7fd042d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 0x5630a1944090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:44558/0 (socket says 172.21.15.5:44558) 2024-09-15T22:20:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.935+0000 7fd042d9d640 1 -- 172.21.15.5:0/3710156077 learned_addr learned my addr 172.21.15.5:0/3710156077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.937+0000 7fd041d9b640 1 --2- 172.21.15.5:0/3710156077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 0x5630a1973770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.937+0000 7fd04259c640 1 --2- 172.21.15.5:0/3710156077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 0x5630a1944a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.938+0000 7fd0337fe640 1 -- 172.21.15.5:0/1131096162 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd03401abc0 con 0x7fd044100e20 2024-09-15T22:20:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.940+0000 7fd0337fe640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd02c0a35e0 0x7fd02c0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.940+0000 7fd041d9b640 1 -- 172.21.15.5:0/3710156077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 msgr2=0x5630a1944a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.940+0000 7fd041d9b640 1 --2- 172.21.15.5:0/3710156077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 0x5630a1944a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.940+0000 7fd041d9b640 1 -- 172.21.15.5:0/3710156077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 msgr2=0x5630a1944090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.940+0000 7fd041d9b640 1 --2- 172.21.15.5:0/3710156077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 0x5630a1944090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.940+0000 7fd041d9b640 1 -- 172.21.15.5:0/3710156077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630a18363a0 con 0x5630a1971380 2024-09-15T22:20:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.942+0000 7fd041d9b640 1 --2- 172.21.15.5:0/3710156077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 0x5630a1973770 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd04406e260 tx=0x7fd03800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6adb49cf1bf9df server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.943+0000 7fd0337fe640 1 -- 172.21.15.5:0/1131096162 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7fd0340cf730 con 0x7fd044100e20 2024-09-15T22:20:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.945+0000 7fd030ff9640 1 -- 172.21.15.5:0/3710156077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03800ab60 con 0x5630a1971380 2024-09-15T22:20:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.945+0000 7fd030ff9640 1 -- 172.21.15.5:0/3710156077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038005500 con 0x5630a1971380 2024-09-15T22:20:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.945+0000 7fd030ff9640 1 -- 172.21.15.5:0/3710156077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038014410 con 0x5630a1971380 2024-09-15T22:20:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.945+0000 7fd042d9d640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd02c0a35e0 0x7fd02c0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.946+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3710156077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 msgr2=0x5630a1973770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.946+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/3710156077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 0x5630a1973770 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd04406e260 tx=0x7fd03800be80 comp rx=0 tx=0).stop 2024-09-15T22:20:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.949+0000 7fd042d9d640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd02c0a35e0 0x7fd02c0a5aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006010 tx=0x7fd03c009920 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.949+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3710156077 shutdown_connections 2024-09-15T22:20:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.949+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/3710156077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 0x5630a1973770 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.949+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/3710156077 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 0x5630a1944a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.949+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/3710156077 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 0x5630a1944090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.949+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3710156077 >> 172.21.15.5:0/3710156077 conn(0x5630a190d310 msgr2=0x5630a1940c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:58.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.954+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3710156077 shutdown_connections 2024-09-15T22:20:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.956+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3710156077 wait complete. 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.956+0000 7fd04b1a1740 1 -- start start 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.956+0000 7fd04b1a1740 1 Processor -- start 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.957+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 0x5630a1970d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.957+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1945b40 0x5630a196dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.957+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1971380 0x5630a196e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.957+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630a1974080 con 0x5630a1945b40 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.957+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5630a1865060 con 0x5630a1971380 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.957+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5630a1974200 con 0x5630a19445d0 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.957+0000 7fd04b1a1740 10 client.0 _ll_register_callbacks cb 0x5630a15b88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.957+0000 7fd041d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1945b40 0x5630a196dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.957+0000 7fd041d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1945b40 0x5630a196dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58948/0 (socket says 172.21.15.5:58948) 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.957+0000 7fd041d9b640 1 -- 172.21.15.5:0/3101470764 learned_addr learned my addr 172.21.15.5:0/3101470764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.958+0000 7fd042d9d640 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1971380 0x5630a196e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.958+0000 7fd04259c640 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 0x5630a1970d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.958+0000 7fd042d9d640 1 -- 172.21.15.5:0/3101470764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 msgr2=0x5630a1970d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.958+0000 7fd042d9d640 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 0x5630a1970d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.958+0000 7fd042d9d640 1 -- 172.21.15.5:0/3101470764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1945b40 msgr2=0x5630a196dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.958+0000 7fd042d9d640 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1945b40 0x5630a196dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.958+0000 7fd042d9d640 1 -- 172.21.15.5:0/3101470764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630a161b3c0 con 0x5630a1971380 2024-09-15T22:20:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.958+0000 7fd041d9b640 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1945b40 0x5630a196dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:20:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.959+0000 7fd042d9d640 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1971380 0x5630a196e5e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd03c004af0 tx=0x7fd03c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fd151324f0eb263 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.960+0000 7fd029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:20:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.960+0000 7fd029ffb640 1 -- 172.21.15.5:0/3101470764 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03c04c070 con 0x5630a1971380 2024-09-15T22:20:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.960+0000 7fd029ffb640 1 -- 172.21.15.5:0/3101470764 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7fd03c030d90 con 0x5630a1971380 2024-09-15T22:20:58.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.960+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3101470764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd038005500 con 0x5630a1971380 2024-09-15T22:20:58.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.961+0000 7fd029ffb640 1 -- 172.21.15.5:0/3101470764 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd03c034040 con 0x5630a1971380 2024-09-15T22:20:58.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.962+0000 7fd04b1a1740 10 client.27038 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:58.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.962+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3101470764 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630a197fb70 con 0x5630a1971380 2024-09-15T22:20:58.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.963+0000 7fd029ffb640 1 -- 172.21.15.5:0/3101470764 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038005500 con 0x5630a1971380 2024-09-15T22:20:58.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.963+0000 7fd029ffb640 1 -- 172.21.15.5:0/3101470764 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd03c02f070 con 0x5630a1971380 2024-09-15T22:20:58.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.963+0000 7fd04b1a1740 10 client.27038 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.963+0000 7fd04b1a1740 10 client.27038 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.963+0000 7fd04b1a1740 20 client.27038 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.964+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630a1981020 0x5630a1983450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.964+0000 7fd04b1a1740 4 client.27038 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:20:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.964+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3101470764 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630a18b7860 con 0x5630a1981020 2024-09-15T22:20:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.965+0000 7fd04259c640 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630a1981020 0x5630a1983450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.967+0000 7fd04259c640 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630a1981020 0x5630a1983450 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78164cfc9fd97b8f server_cookie=36ca1b63c270d268 in_seq=0 out_seq=0 2024-09-15T22:20:58.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.967+0000 7fd029ffb640 10 client.27038 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.974+0000 7fd029ffb640 1 -- 172.21.15.5:0/3101470764 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630a18b7860 con 0x5630a1981020 2024-09-15T22:20:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.974+0000 7fd029ffb640 10 client.27038 handle_command_reply: tid=0 2024-09-15T22:20:58.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.974+0000 7fd04b1a1740 1 client.27038 shutdown 2024-09-15T22:20:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.976+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3101470764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1971380 msgr2=0x5630a196e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.976+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1971380 0x5630a196e5e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd03c004af0 tx=0x7fd03c0095c0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.976+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3101470764 shutdown_connections 2024-09-15T22:20:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.976+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630a1981020 0x5630a1983450 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.976+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1971380 0x5630a196e5e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.976+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1945b40 0x5630a196dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.976+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/3101470764 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 0x5630a1970d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:58.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.976+0000 7fd029ffb640 0 client.27038 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:58.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.976+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3101470764 >> 172.21.15.5:0/3101470764 conn(0x5630a190d310 msgr2=0x5630a1946930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:58.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.977+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3101470764 shutdown_connections 2024-09-15T22:20:58.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.977+0000 7fd04b1a1740 1 -- 172.21.15.5:0/3101470764 wait complete. 2024-09-15T22:20:58.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:58.977+0000 7fd04b1a1740 20 client.27038 trim_cache size 0 max 16384 2024-09-15T22:20:59.015 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:20:58 smithi005 ceph-mon[30944]: pgmap v375: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 3.5 MiB/s wr, 291 op/s 2024-09-15T22:20:59.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.012+0000 7fd04b1a1740 1 Processor -- start 2024-09-15T22:20:59.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.012+0000 7fd04b1a1740 1 -- start start 2024-09-15T22:20:59.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.012+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 0x5630a197d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.012+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 0x5630a196d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.012+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 0x5630a1988d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.012+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0240129f0 con 0x5630a1971380 2024-09-15T22:20:59.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.012+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd0240102c0 con 0x5630a1945b40 2024-09-15T22:20:59.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.012+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd02400e130 con 0x5630a19445d0 2024-09-15T22:20:59.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.013+0000 7fd041d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 0x5630a196d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.013+0000 7fd041d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 0x5630a196d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:44572/0 (socket says 172.21.15.5:44572) 2024-09-15T22:20:59.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.013+0000 7fd041d9b640 1 -- 172.21.15.5:0/2860923573 learned_addr learned my addr 172.21.15.5:0/2860923573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.013+0000 7fd041d9b640 1 -- 172.21.15.5:0/2860923573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 msgr2=0x5630a197d3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:20:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.013+0000 7fd041d9b640 1 --2- 172.21.15.5:0/2860923573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 0x5630a197d3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.013+0000 7fd041d9b640 1 -- 172.21.15.5:0/2860923573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 msgr2=0x5630a1988d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.013+0000 7fd041d9b640 1 --2- 172.21.15.5:0/2860923573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 0x5630a1988d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.013+0000 7fd041d9b640 1 -- 172.21.15.5:0/2860923573 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630a19710b0 con 0x5630a1945b40 2024-09-15T22:20:59.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.014+0000 7fd041d9b640 1 --2- 172.21.15.5:0/2860923573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 0x5630a196d8b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd038004b80 tx=0x7fd038004c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b0da2d52c4ac6d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:59.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.015+0000 7fd02bfff640 1 -- 172.21.15.5:0/2860923573 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038023030 con 0x5630a1945b40 2024-09-15T22:20:59.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.015+0000 7fd02bfff640 1 -- 172.21.15.5:0/2860923573 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038004860 con 0x5630a1945b40 2024-09-15T22:20:59.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.015+0000 7fd02bfff640 1 -- 172.21.15.5:0/2860923573 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0380051a0 con 0x5630a1945b40 2024-09-15T22:20:59.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.016+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2860923573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 msgr2=0x5630a196d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.016+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/2860923573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 0x5630a196d8b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd038004b80 tx=0x7fd038004c20 comp rx=0 tx=0).stop 2024-09-15T22:20:59.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.019+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2860923573 shutdown_connections 2024-09-15T22:20:59.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.019+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/2860923573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 0x5630a1988d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.019+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/2860923573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a1945b40 0x5630a196d8b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.019+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/2860923573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a19445d0 0x5630a197d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.019+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2860923573 >> 172.21.15.5:0/2860923573 conn(0x5630a190d310 msgr2=0x5630a196c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.024+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2860923573 shutdown_connections 2024-09-15T22:20:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.024+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2860923573 wait complete. 2024-09-15T22:20:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.024+0000 7fd04b1a1740 1 -- start start 2024-09-15T22:20:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.024+0000 7fd04b1a1740 1 Processor -- start 2024-09-15T22:20:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.025+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a19445d0 0x5630a1975510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.025+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a1945b40 0x5630a1975a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.025+0000 7fd04b1a1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 0x5630a1975f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.025+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd02400e780 con 0x5630a1971380 2024-09-15T22:20:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.025+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd02400fbf0 con 0x5630a19445d0 2024-09-15T22:20:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.025+0000 7fd04b1a1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd02400ee50 con 0x5630a1945b40 2024-09-15T22:20:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.025+0000 7fd04b1a1740 10 client.0 _ll_register_callbacks cb 0x7fd03c009990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:20:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.025+0000 7fd041d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a1945b40 0x5630a1975a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.025+0000 7fd041d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a1945b40 0x5630a1975a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:49042/0 (socket says 172.21.15.5:49042) 2024-09-15T22:20:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.025+0000 7fd041d9b640 1 -- 172.21.15.5:0/2133032388 learned_addr learned my addr 172.21.15.5:0/2133032388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:20:59.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.025+0000 7fd04259c640 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a19445d0 0x5630a1975510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.026+0000 7fd041d9b640 1 -- 172.21.15.5:0/2133032388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a19445d0 msgr2=0x5630a1975510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.026+0000 7fd041d9b640 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a19445d0 0x5630a1975510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.026+0000 7fd041d9b640 1 -- 172.21.15.5:0/2133032388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 msgr2=0x5630a1975f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:20:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.026+0000 7fd041d9b640 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 0x5630a1975f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.026+0000 7fd041d9b640 1 -- 172.21.15.5:0/2133032388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630a19eafd0 con 0x5630a1945b40 2024-09-15T22:20:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.026+0000 7fd041d9b640 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a1945b40 0x5630a1975a50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd03800b830 tx=0x7fd0380052d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb13474d9146b46 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:20:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.026+0000 7fd029ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:20:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.027+0000 7fd029ffb640 1 -- 172.21.15.5:0/2133032388 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03802d030 con 0x5630a1945b40 2024-09-15T22:20:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.027+0000 7fd029ffb640 1 -- 172.21.15.5:0/2133032388 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9541+0+0 (secure 0 0 0) 0x7fd038011040 con 0x5630a1945b40 2024-09-15T22:20:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.028+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2133032388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd038004860 con 0x5630a1945b40 2024-09-15T22:20:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.029+0000 7fd029ffb640 1 -- 172.21.15.5:0/2133032388 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd038023070 con 0x5630a1945b40 2024-09-15T22:20:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.029+0000 7fd04b1a1740 10 client.26989 fetch_fsmap learned FSMap version 25 2024-09-15T22:20:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.029+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2133032388 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630a19ec2b0 con 0x5630a1945b40 2024-09-15T22:20:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.030+0000 7fd029ffb640 1 -- 172.21.15.5:0/2133032388 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd038004860 con 0x5630a1945b40 2024-09-15T22:20:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.030+0000 7fd029ffb640 1 -- 172.21.15.5:0/2133032388 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fd03801f410 con 0x5630a1945b40 2024-09-15T22:20:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.031+0000 7fd04b1a1740 10 client.26989 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:20:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.031+0000 7fd04b1a1740 10 client.26989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:20:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.031+0000 7fd04b1a1740 20 client.26989 populate_metadata read hostname 'smithi005' 2024-09-15T22:20:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.031+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630a19ec7a0 0x5630a19eebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:20:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.031+0000 7fd04b1a1740 4 client.26989 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:20:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.031+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2133032388 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630a18b1d70 con 0x5630a19ec7a0 2024-09-15T22:20:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.032+0000 7fd04259c640 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630a19ec7a0 0x5630a19eebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:20:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.032+0000 7fd04259c640 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630a19ec7a0 0x5630a19eebc0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73fbbda358d241a server_cookie=9a524813b6ad7b9f in_seq=0 out_seq=0 2024-09-15T22:20:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.033+0000 7fd029ffb640 10 client.26989 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.033+0000 7fd029ffb640 1 -- 172.21.15.5:0/2133032388 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630a18b1d70 con 0x5630a19ec7a0 2024-09-15T22:20:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.033+0000 7fd029ffb640 10 client.26989 handle_command_reply: tid=0 2024-09-15T22:20:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.033+0000 7fd04b1a1740 1 client.26989 shutdown 2024-09-15T22:20:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.033+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2133032388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a1945b40 msgr2=0x5630a1975a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.033+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a1945b40 0x5630a1975a50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd03800b830 tx=0x7fd0380052d0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.034+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2133032388 shutdown_connections 2024-09-15T22:20:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.034+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630a1971380 0x5630a1975f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.034+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5630a1945b40 0x5630a1975a50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.034+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5630a19ec7a0 0x5630a19eebc0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.034+0000 7fd04b1a1740 1 --2- 172.21.15.5:0/2133032388 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5630a19445d0 0x5630a1975510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.034+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2133032388 >> 172.21.15.5:0/2133032388 conn(0x5630a190d310 msgr2=0x5630a196c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.035+0000 7fd029ffb640 0 client.26989 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:20:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.035+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2133032388 shutdown_connections 2024-09-15T22:20:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.035+0000 7fd04b1a1740 1 -- 172.21.15.5:0/2133032388 wait complete. 2024-09-15T22:20:59.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.036+0000 7fd04b1a1740 20 client.26989 trim_cache size 0 max 16384 2024-09-15T22:20:59.039 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:20:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 -- 172.21.15.5:0/1131096162 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd02c0a35e0 msgr2=0x7fd02c0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd02c0a35e0 0x7fd02c0a5aa0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006010 tx=0x7fd03c009920 comp rx=0 tx=0).stop 2024-09-15T22:20:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 -- 172.21.15.5:0/1131096162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044100e20 msgr2=0x7fd044073980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:20:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044100e20 0x7fd044073980 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd03400b4a0 tx=0x7fd03400b960 comp rx=0 tx=0).stop 2024-09-15T22:20:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 -- 172.21.15.5:0/1131096162 shutdown_connections 2024-09-15T22:20:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd02c0a35e0 0x7fd02c0a5aa0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd044102ef0 0x7fd04406cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd044102570 0x7fd044073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 --2- 172.21.15.5:0/1131096162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044100e20 0x7fd044073980 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:20:59.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 -- 172.21.15.5:0/1131096162 >> 172.21.15.5:0/1131096162 conn(0x7fd0440fc7d0 msgr2=0x7fd0441048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:20:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 -- 172.21.15.5:0/1131096162 shutdown_connections 2024-09-15T22:20:59.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:20:59.038+0000 7fd048dc8640 1 -- 172.21.15.5:0/1131096162 wait complete. 2024-09-15T22:20:59.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:20:58 smithi099 ceph-mon[34892]: pgmap v375: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 3.5 MiB/s wr, 291 op/s 2024-09-15T22:20:59.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:20:58 smithi129 ceph-mon[35314]: pgmap v375: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 3.5 MiB/s wr, 291 op/s 2024-09-15T22:20:59.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 690.0 s, 618.0 tps, lat 161.388 ms stddev 185.817 2024-09-15T22:21:01.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:00 smithi099 ceph-mon[34892]: pgmap v376: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.0 MiB/s wr, 315 op/s 2024-09-15T22:21:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:00 smithi129 ceph-mon[35314]: pgmap v376: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.0 MiB/s wr, 315 op/s 2024-09-15T22:21:01.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:00 smithi005 ceph-mon[30944]: pgmap v376: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 4.0 MiB/s wr, 315 op/s 2024-09-15T22:21:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:02 smithi099 ceph-mon[34892]: pgmap v377: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.7 MiB/s wr, 534 op/s 2024-09-15T22:21:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:02 smithi129 ceph-mon[35314]: pgmap v377: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.7 MiB/s wr, 534 op/s 2024-09-15T22:21:03.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:02 smithi005 ceph-mon[30944]: pgmap v377: 129 pgs: 129 active+clean; 3.0 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 6.7 MiB/s wr, 534 op/s 2024-09-15T22:21:04.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:03 smithi005 ceph-mon[30944]: osdmap e100: 12 total, 12 up, 12 in 2024-09-15T22:21:04.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:03 smithi099 ceph-mon[34892]: osdmap e100: 12 total, 12 up, 12 in 2024-09-15T22:21:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:03 smithi129 ceph-mon[35314]: osdmap e100: 12 total, 12 up, 12 in 2024-09-15T22:21:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 695.0 s, 411.4 tps, lat 250.054 ms stddev 387.223 2024-09-15T22:21:05.315 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:21:05.316 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:21:05.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:04 smithi099 ceph-mon[34892]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.8 MiB/s wr, 546 op/s 2024-09-15T22:21:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:04 smithi129 ceph-mon[35314]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.8 MiB/s wr, 546 op/s 2024-09-15T22:21:05.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:04 smithi005 ceph-mon[30944]: pgmap v379: 129 pgs: 129 active+clean; 3.0 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 6.8 MiB/s wr, 546 op/s 2024-09-15T22:21:05.893 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:07.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:06 smithi099 ceph-mon[34892]: pgmap v380: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.3 MiB/s wr, 490 op/s 2024-09-15T22:21:07.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:06 smithi129 ceph-mon[35314]: pgmap v380: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.3 MiB/s wr, 490 op/s 2024-09-15T22:21:07.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:06 smithi005 ceph-mon[30944]: pgmap v380: 129 pgs: 129 active+clean; 2.8 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 6.3 MiB/s wr, 490 op/s 2024-09-15T22:21:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:07.992+0000 7faa9295a640 1 -- 172.21.15.5:0/4156959007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa8c101430 msgr2=0x7faa8c10d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:07.992+0000 7faa9295a640 1 --2- 172.21.15.5:0/4156959007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa8c101430 0x7faa8c10d960 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7faa84009ca0 tx=0x7faa8402a8f0 comp rx=0 tx=0).stop 2024-09-15T22:21:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:07.994+0000 7faa9295a640 1 -- 172.21.15.5:0/4156959007 shutdown_connections 2024-09-15T22:21:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:07.994+0000 7faa9295a640 1 --2- 172.21.15.5:0/4156959007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa8c101430 0x7faa8c10d960 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:07.994+0000 7faa9295a640 1 --2- 172.21.15.5:0/4156959007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa8c100a90 0x7faa8c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:07.994+0000 7faa9295a640 1 --2- 172.21.15.5:0/4156959007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa8c106a90 0x7faa8c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:07.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:07.994+0000 7faa9295a640 1 -- 172.21.15.5:0/4156959007 >> 172.21.15.5:0/4156959007 conn(0x7faa8c0fc7d0 msgr2=0x7faa8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:08.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.010+0000 7faa9295a640 1 -- 172.21.15.5:0/4156959007 shutdown_connections 2024-09-15T22:21:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.011+0000 7faa9295a640 1 -- 172.21.15.5:0/4156959007 wait complete. 2024-09-15T22:21:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.012+0000 7faa9295a640 1 Processor -- start 2024-09-15T22:21:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.012+0000 7faa9295a640 1 -- start start 2024-09-15T22:21:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.012+0000 7faa9295a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa8c100a90 0x7faa8c0738c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.012+0000 7faa9295a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa8c101430 0x7faa8c073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.012+0000 7faa9295a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa8c106a90 0x7faa8c06ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.012+0000 7faa9295a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c10feb0 con 0x7faa8c101430 2024-09-15T22:21:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.012+0000 7faa9295a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c10fd30 con 0x7faa8c106a90 2024-09-15T22:21:08.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.012+0000 7faa9295a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c110030 con 0x7faa8c100a90 2024-09-15T22:21:08.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.013+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa8c101430 0x7faa8c073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.013+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa8c101430 0x7faa8c073e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41074/0 (socket says 172.21.15.5:41074) 2024-09-15T22:21:08.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.013+0000 7faa8b7fe640 1 -- 172.21.15.5:0/3147467049 learned_addr learned my addr 172.21.15.5:0/3147467049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:08.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.014+0000 7faa90ef7640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa8c106a90 0x7faa8c06ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.014+0000 7faa8bfff640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa8c100a90 0x7faa8c0738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.015+0000 7faa8b7fe640 1 -- 172.21.15.5:0/3147467049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa8c100a90 msgr2=0x7faa8c0738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.015+0000 7faa8b7fe640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa8c100a90 0x7faa8c0738c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.015+0000 7faa8b7fe640 1 -- 172.21.15.5:0/3147467049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa8c106a90 msgr2=0x7faa8c06ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.015+0000 7faa8b7fe640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa8c106a90 0x7faa8c06ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.015+0000 7faa8b7fe640 1 -- 172.21.15.5:0/3147467049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c06d310 con 0x7faa8c101430 2024-09-15T22:21:08.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.015+0000 7faa90ef7640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa8c106a90 0x7faa8c06ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:08.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.018+0000 7faa8b7fe640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa8c101430 0x7faa8c073e00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faa8000b450 tx=0x7faa8000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.021+0000 7faa897fa640 1 -- 172.21.15.5:0/3147467049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80015020 con 0x7faa8c101430 2024-09-15T22:21:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.021+0000 7faa897fa640 1 -- 172.21.15.5:0/3147467049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa80011d70 con 0x7faa8c101430 2024-09-15T22:21:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.021+0000 7faa897fa640 1 -- 172.21.15.5:0/3147467049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8000f340 con 0x7faa8c101430 2024-09-15T22:21:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.022+0000 7faa9295a640 1 -- 172.21.15.5:0/3147467049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8c06d600 con 0x7faa8c101430 2024-09-15T22:21:08.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.022+0000 7faa9295a640 1 -- 172.21.15.5:0/3147467049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa8c0715c0 con 0x7faa8c101430 2024-09-15T22:21:08.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.027+0000 7faa94d33740 1 Processor -- start 2024-09-15T22:21:08.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.030+0000 7faa94d33740 1 -- start start 2024-09-15T22:21:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.031+0000 7faa94d33740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f949b60 0x55ea1f949f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.031+0000 7faa94d33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f954800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.031+0000 7faa94d33740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f954d40 0x55ea1f957130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.031+0000 7faa94d33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea1f8af280 con 0x55ea1f94a480 2024-09-15T22:21:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.031+0000 7faa94d33740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ea1f856b10 con 0x55ea1f954d40 2024-09-15T22:21:08.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.031+0000 7faa94d33740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ea1f8ab240 con 0x55ea1f949b60 2024-09-15T22:21:08.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.032+0000 7faa8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f954800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.032+0000 7faa8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f954800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41086/0 (socket says 172.21.15.5:41086) 2024-09-15T22:21:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.032+0000 7faa8bfff640 1 -- 172.21.15.5:0/596877007 learned_addr learned my addr 172.21.15.5:0/596877007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.033+0000 7faa8b7fe640 1 --2- 172.21.15.5:0/596877007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f954d40 0x55ea1f957130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.033+0000 7faa8bfff640 1 -- 172.21.15.5:0/596877007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f949b60 msgr2=0x55ea1f949f40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:21:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.033+0000 7faa8bfff640 1 --2- 172.21.15.5:0/596877007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f949b60 0x55ea1f949f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.033+0000 7faa8bfff640 1 -- 172.21.15.5:0/596877007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f954d40 msgr2=0x55ea1f957130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.033+0000 7faa8bfff640 1 --2- 172.21.15.5:0/596877007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f954d40 0x55ea1f957130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.033+0000 7faa8bfff640 1 -- 172.21.15.5:0/596877007 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1f8631f0 con 0x55ea1f94a480 2024-09-15T22:21:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.033+0000 7faa8bfff640 1 --2- 172.21.15.5:0/596877007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f954800 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00b820 tx=0x7faa7c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1154aa3f55bf02e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.036+0000 7faa727fc640 1 -- 172.21.15.5:0/596877007 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c00d950 con 0x55ea1f94a480 2024-09-15T22:21:08.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.036+0000 7faa727fc640 1 -- 172.21.15.5:0/596877007 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa7c019070 con 0x55ea1f94a480 2024-09-15T22:21:08.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.037+0000 7faa94d33740 1 -- 172.21.15.5:0/596877007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 msgr2=0x55ea1f954800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:08.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.037+0000 7faa94d33740 1 --2- 172.21.15.5:0/596877007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f954800 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00b820 tx=0x7faa7c00bce0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.038+0000 7faa897fa640 1 -- 172.21.15.5:0/3147467049 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7faa80018020 con 0x7faa8c101430 2024-09-15T22:21:08.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.039+0000 7faa897fa640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faa740a35c0 0x7faa740a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.039+0000 7faa897fa640 1 -- 172.21.15.5:0/3147467049 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9629+0+0 (secure 0 0 0) 0x7faa800cf7d0 con 0x7faa8c101430 2024-09-15T22:21:08.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.041+0000 7faa94d33740 1 -- 172.21.15.5:0/596877007 shutdown_connections 2024-09-15T22:21:08.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.041+0000 7faa94d33740 1 --2- 172.21.15.5:0/596877007 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f954d40 0x55ea1f957130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.041+0000 7faa94d33740 1 --2- 172.21.15.5:0/596877007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f954800 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.041+0000 7faa94d33740 1 --2- 172.21.15.5:0/596877007 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f949b60 0x55ea1f949f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.041+0000 7faa94d33740 1 -- 172.21.15.5:0/596877007 >> 172.21.15.5:0/596877007 conn(0x55ea1f9460f0 msgr2=0x55ea1f9464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.042+0000 7faa90ef7640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faa740a35c0 0x7faa740a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.042+0000 7faa94d33740 1 -- 172.21.15.5:0/596877007 shutdown_connections 2024-09-15T22:21:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.043+0000 7faa94d33740 1 -- 172.21.15.5:0/596877007 wait complete. 2024-09-15T22:21:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.044+0000 7faa94d33740 1 -- start start 2024-09-15T22:21:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.044+0000 7faa94d33740 1 Processor -- start 2024-09-15T22:21:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.049+0000 7faa94d33740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f949b60 0x55ea1f9547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.049+0000 7faa94d33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f951b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.049+0000 7faa94d33740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f9522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.049+0000 7faa94d33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ea1f957960 con 0x55ea1f94a480 2024-09-15T22:21:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.049+0000 7faa94d33740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55ea1f869f10 con 0x55ea1f949b60 2024-09-15T22:21:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.049+0000 7faa94d33740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55ea1f957ae0 con 0x55ea1f954d40 2024-09-15T22:21:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.049+0000 7faa94d33740 10 client.0 _ll_register_callbacks cb 0x55ea1f832a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:08.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.051+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f951b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.051+0000 7faa8b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f951b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41100/0 (socket says 172.21.15.5:41100) 2024-09-15T22:21:08.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.051+0000 7faa8b7fe640 1 -- 172.21.15.5:0/1302593987 learned_addr learned my addr 172.21.15.5:0/1302593987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:08.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.052+0000 7faa8bfff640 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f949b60 0x55ea1f9547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.053+0000 7faa90ef7640 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f9522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.053+0000 7faa90ef7640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faa740a35c0 0x7faa740a5a80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faa840098d0 tx=0x7faa84033000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.053+0000 7faa8b7fe640 1 -- 172.21.15.5:0/1302593987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 msgr2=0x55ea1f9522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.053+0000 7faa8b7fe640 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f9522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.053+0000 7faa8b7fe640 1 -- 172.21.15.5:0/1302593987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f949b60 msgr2=0x55ea1f9547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.053+0000 7faa8b7fe640 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f949b60 0x55ea1f9547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.053+0000 7faa8b7fe640 1 -- 172.21.15.5:0/1302593987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1f6206a0 con 0x55ea1f94a480 2024-09-15T22:21:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.055+0000 7faa8b7fe640 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f951b40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7faa8000be10 tx=0x7faa8009b310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3445150d75beb751 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:08.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.059+0000 7faa6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:21:08.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.059+0000 7faa6b7fe640 1 -- 172.21.15.5:0/1302593987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8009b710 con 0x55ea1f94a480 2024-09-15T22:21:08.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.059+0000 7faa6b7fe640 1 -- 172.21.15.5:0/1302593987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9629+0+0 (secure 0 0 0) 0x7faa800103e0 con 0x55ea1f94a480 2024-09-15T22:21:08.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.060+0000 7faa94d33740 1 -- 172.21.15.5:0/1302593987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa7c019070 con 0x55ea1f94a480 2024-09-15T22:21:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.063+0000 7faa6b7fe640 1 -- 172.21.15.5:0/1302593987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faa8009fac0 con 0x55ea1f94a480 2024-09-15T22:21:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.064+0000 7faa94d33740 10 client.17574 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:08.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.064+0000 7faa94d33740 1 -- 172.21.15.5:0/1302593987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ea1f963570 con 0x55ea1f94a480 2024-09-15T22:21:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.067+0000 7faa6b7fe640 1 -- 172.21.15.5:0/1302593987 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa7c019070 con 0x55ea1f94a480 2024-09-15T22:21:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.067+0000 7faa6b7fe640 1 -- 172.21.15.5:0/1302593987 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7faa80022060 con 0x55ea1f94a480 2024-09-15T22:21:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.067+0000 7faa94d33740 10 client.17574 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.067+0000 7faa94d33740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.067+0000 7faa94d33740 20 client.17574 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.067+0000 7faa94d33740 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ea1f964e70 0x55ea1f967290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.067+0000 7faa94d33740 4 client.17574 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.067+0000 7faa94d33740 1 -- 172.21.15.5:0/1302593987 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea1f8bc7f0 con 0x55ea1f964e70 2024-09-15T22:21:08.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.071+0000 7faa8bfff640 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ea1f964e70 0x55ea1f967290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.075+0000 7faa8bfff640 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ea1f964e70 0x55ea1f967290 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af673e7f08d5ced2 server_cookie=1621eba2c162de79 in_seq=0 out_seq=0 2024-09-15T22:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.075+0000 7faa6b7fe640 10 client.17574 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.076+0000 7faa6b7fe640 1 -- 172.21.15.5:0/1302593987 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea1f8bc7f0 con 0x55ea1f964e70 2024-09-15T22:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.076+0000 7faa6b7fe640 10 client.17574 handle_command_reply: tid=0 2024-09-15T22:21:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.076+0000 7faa94d33740 1 client.17574 shutdown 2024-09-15T22:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.078+0000 7faa94d33740 1 -- 172.21.15.5:0/1302593987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 msgr2=0x55ea1f951b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.078+0000 7faa94d33740 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f951b40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7faa8000be10 tx=0x7faa8009b310 comp rx=0 tx=0).stop 2024-09-15T22:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.078+0000 7faa94d33740 1 -- 172.21.15.5:0/1302593987 shutdown_connections 2024-09-15T22:21:08.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.078+0000 7faa94d33740 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f9522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.078+0000 7faa94d33740 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f94a480 0x55ea1f951b40 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.078+0000 7faa94d33740 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ea1f964e70 0x55ea1f967290 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.078+0000 7faa94d33740 1 --2- 172.21.15.5:0/1302593987 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f949b60 0x55ea1f9547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.078+0000 7faa94d33740 1 -- 172.21.15.5:0/1302593987 >> 172.21.15.5:0/1302593987 conn(0x55ea1f9460f0 msgr2=0x55ea1f945270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.078+0000 7faa6b7fe640 0 client.17574 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.078+0000 7faa94d33740 1 -- 172.21.15.5:0/1302593987 shutdown_connections 2024-09-15T22:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.078+0000 7faa94d33740 1 -- 172.21.15.5:0/1302593987 wait complete. 2024-09-15T22:21:08.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.079+0000 7faa94d33740 20 client.17574 trim_cache size 0 max 16384 2024-09-15T22:21:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.117+0000 7faa94d33740 1 Processor -- start 2024-09-15T22:21:08.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.117+0000 7faa94d33740 1 -- start start 2024-09-15T22:21:08.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.117+0000 7faa94d33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f946930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.118+0000 7faa94d33740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f94a480 0x55ea1f96cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.118+0000 7faa8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f946930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.118+0000 7faa8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f946930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41108/0 (socket says 172.21.15.5:41108) 2024-09-15T22:21:08.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.118+0000 7faa8bfff640 1 -- 172.21.15.5:0/3704918850 learned_addr learned my addr 172.21.15.5:0/3704918850 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:08.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.118+0000 7faa94d33740 1 --2- 172.21.15.5:0/3704918850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f96d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.118+0000 7faa94d33740 1 -- 172.21.15.5:0/3704918850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa78012b50 con 0x55ea1f949b60 2024-09-15T22:21:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.119+0000 7faa90ef7640 1 --2- 172.21.15.5:0/3704918850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f96d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.119+0000 7faa94d33740 1 -- 172.21.15.5:0/3704918850 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa78010420 con 0x55ea1f94a480 2024-09-15T22:21:08.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.119+0000 7faa94d33740 1 -- 172.21.15.5:0/3704918850 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800e290 con 0x55ea1f954d40 2024-09-15T22:21:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.119+0000 7faa8bfff640 1 -- 172.21.15.5:0/3704918850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 msgr2=0x55ea1f96d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.119+0000 7faa8bfff640 1 --2- 172.21.15.5:0/3704918850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f96d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.119+0000 7faa8bfff640 1 -- 172.21.15.5:0/3704918850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f94a480 msgr2=0x55ea1f96cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.119+0000 7faa8bfff640 1 --2- 172.21.15.5:0/3704918850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f94a480 0x55ea1f96cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.119+0000 7faa8bfff640 1 -- 172.21.15.5:0/3704918850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1f941830 con 0x55ea1f949b60 2024-09-15T22:21:08.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.120+0000 7faa8bfff640 1 --2- 172.21.15.5:0/3704918850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f946930 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00b820 tx=0x7faa7c00f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255e5ea897b83fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.120+0000 7faa71ffb640 1 -- 172.21.15.5:0/3704918850 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c02a070 con 0x55ea1f949b60 2024-09-15T22:21:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.120+0000 7faa71ffb640 1 -- 172.21.15.5:0/3704918850 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa7c019070 con 0x55ea1f949b60 2024-09-15T22:21:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.120+0000 7faa71ffb640 1 -- 172.21.15.5:0/3704918850 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c003910 con 0x55ea1f949b60 2024-09-15T22:21:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.121+0000 7faa94d33740 1 -- 172.21.15.5:0/3704918850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 msgr2=0x55ea1f946930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:08.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.121+0000 7faa94d33740 1 --2- 172.21.15.5:0/3704918850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f946930 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00b820 tx=0x7faa7c00f040 comp rx=0 tx=0).stop 2024-09-15T22:21:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.121+0000 7faa94d33740 1 -- 172.21.15.5:0/3704918850 shutdown_connections 2024-09-15T22:21:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.121+0000 7faa94d33740 1 --2- 172.21.15.5:0/3704918850 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f96d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.121+0000 7faa94d33740 1 --2- 172.21.15.5:0/3704918850 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f94a480 0x55ea1f96cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.121+0000 7faa94d33740 1 --2- 172.21.15.5:0/3704918850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f946930 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.121+0000 7faa94d33740 1 -- 172.21.15.5:0/3704918850 >> 172.21.15.5:0/3704918850 conn(0x55ea1f9460f0 msgr2=0x55ea1f950410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.122+0000 7faa94d33740 1 -- 172.21.15.5:0/3704918850 shutdown_connections 2024-09-15T22:21:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.122+0000 7faa94d33740 1 -- 172.21.15.5:0/3704918850 wait complete. 2024-09-15T22:21:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.122+0000 7faa94d33740 1 -- start start 2024-09-15T22:21:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.122+0000 7faa94d33740 1 Processor -- start 2024-09-15T22:21:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.122+0000 7faa94d33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f95d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.122+0000 7faa94d33740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f94a480 0x55ea1f958550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa94d33740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f958a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa94d33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800e8e0 con 0x55ea1f949b60 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa94d33740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800fd50 con 0x55ea1f94a480 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa94d33740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7faa7800efb0 con 0x55ea1f954d40 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa94d33740 10 client.0 _ll_register_callbacks cb 0x7faa80097a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f95d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f95d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41110/0 (socket says 172.21.15.5:41110) 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa8bfff640 1 -- 172.21.15.5:0/153469332 learned_addr learned my addr 172.21.15.5:0/153469332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa90ef7640 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f958a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa8bfff640 1 -- 172.21.15.5:0/153469332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 msgr2=0x55ea1f958a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa8bfff640 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f958a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa8bfff640 1 -- 172.21.15.5:0/153469332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f94a480 msgr2=0x55ea1f958550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa8bfff640 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f94a480 0x55ea1f958550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa8bfff640 1 -- 172.21.15.5:0/153469332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1f958fd0 con 0x55ea1f949b60 2024-09-15T22:21:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa8bfff640 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f95d220 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00bd20 tx=0x7faa7c00b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87df066066f1e0a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.123+0000 7faa6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa6b7fe640 1 -- 172.21.15.5:0/153469332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7c02a050 con 0x55ea1f949b60 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa6b7fe640 1 -- 172.21.15.5:0/153469332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9629+0+0 (secure 0 0 0) 0x7faa7c00d950 con 0x55ea1f949b60 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa94d33740 1 -- 172.21.15.5:0/153469332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa7c019070 con 0x55ea1f949b60 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa6b7fe640 1 -- 172.21.15.5:0/153469332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faa7c012580 con 0x55ea1f949b60 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa94d33740 10 client.17580 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa94d33740 1 -- 172.21.15.5:0/153469332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ea1f9cf880 con 0x55ea1f949b60 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa6b7fe640 1 -- 172.21.15.5:0/153469332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa7c019070 con 0x55ea1f949b60 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa6b7fe640 1 -- 172.21.15.5:0/153469332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7faa7c013070 con 0x55ea1f949b60 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa94d33740 10 client.17580 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa94d33740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa94d33740 20 client.17580 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.124+0000 7faa94d33740 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ea1f9cfcf0 0x55ea1f9d20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.125+0000 7faa94d33740 4 client.17580 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:21:08.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.125+0000 7faa94d33740 1 -- 172.21.15.5:0/153469332 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ea1f8b6d00 con 0x55ea1f9cfcf0 2024-09-15T22:21:08.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.125+0000 7faa8b7fe640 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ea1f9cfcf0 0x55ea1f9d20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:08.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.125+0000 7faa6b7fe640 10 client.17580 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:08.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.125+0000 7faa8b7fe640 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ea1f9cfcf0 0x55ea1f9d20e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d90e86a32d41b74 server_cookie=d0c1a76670a73dc0 in_seq=0 out_seq=0 2024-09-15T22:21:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.126+0000 7faa6b7fe640 1 -- 172.21.15.5:0/153469332 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ea1f8b6d00 con 0x55ea1f9cfcf0 2024-09-15T22:21:08.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.126+0000 7faa6b7fe640 10 client.17580 handle_command_reply: tid=0 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.126+0000 7faa94d33740 1 client.17580 shutdown 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa94d33740 1 -- 172.21.15.5:0/153469332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 msgr2=0x55ea1f95d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa94d33740 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f95d220 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7faa7c00bd20 tx=0x7faa7c00b200 comp rx=0 tx=0).stop 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa94d33740 1 -- 172.21.15.5:0/153469332 shutdown_connections 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa94d33740 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55ea1f954d40 0x55ea1f958a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa94d33740 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55ea1f9cfcf0 0x55ea1f9d20e0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa94d33740 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55ea1f94a480 0x55ea1f958550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa94d33740 1 --2- 172.21.15.5:0/153469332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea1f949b60 0x55ea1f95d220 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa6b7fe640 0 client.17580 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa94d33740 1 -- 172.21.15.5:0/153469332 >> 172.21.15.5:0/153469332 conn(0x55ea1f9460f0 msgr2=0x55ea1f94b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa94d33740 1 -- 172.21.15.5:0/153469332 shutdown_connections 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa94d33740 1 -- 172.21.15.5:0/153469332 wait complete. 2024-09-15T22:21:08.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.127+0000 7faa94d33740 20 client.17580 trim_cache size 0 max 16384 2024-09-15T22:21:08.134 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:21:08.134 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:21:08.134 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "ac9fe404-bddf-4568-9669-6885a493bb69", 2024-09-15T22:21:08.134 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:21:08.134 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:21:08.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 -- 172.21.15.5:0/3147467049 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faa740a35c0 msgr2=0x7faa740a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faa740a35c0 0x7faa740a5a80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faa840098d0 tx=0x7faa84033000 comp rx=0 tx=0).stop 2024-09-15T22:21:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 -- 172.21.15.5:0/3147467049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa8c101430 msgr2=0x7faa8c073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa8c101430 0x7faa8c073e00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7faa8000b450 tx=0x7faa8000b910 comp rx=0 tx=0).stop 2024-09-15T22:21:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 -- 172.21.15.5:0/3147467049 shutdown_connections 2024-09-15T22:21:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7faa740a35c0 0x7faa740a5a80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7faa8c106a90 0x7faa8c06ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa8c101430 0x7faa8c073e00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 --2- 172.21.15.5:0/3147467049 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7faa8c100a90 0x7faa8c0738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 -- 172.21.15.5:0/3147467049 >> 172.21.15.5:0/3147467049 conn(0x7faa8c0fc7d0 msgr2=0x7faa8c0fcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 -- 172.21.15.5:0/3147467049 shutdown_connections 2024-09-15T22:21:08.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:08.128+0000 7faa9295a640 1 -- 172.21.15.5:0/3147467049 wait complete. 2024-09-15T22:21:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:08 smithi099 ceph-mon[34892]: pgmap v381: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.5 MiB/s wr, 527 op/s 2024-09-15T22:21:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:08 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:21:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:08 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:08 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:21:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:08 smithi129 ceph-mon[35314]: pgmap v381: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.5 MiB/s wr, 527 op/s 2024-09-15T22:21:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:08 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:21:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:08 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:08 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:21:09.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:08 smithi005 ceph-mon[30944]: pgmap v381: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.5 MiB/s wr, 527 op/s 2024-09-15T22:21:09.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:08 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:21:09.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:08 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:09.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:08 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:21:09.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:21:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 700.0 s, 666.6 tps, lat 151.048 ms stddev 196.422 2024-09-15T22:21:10.173 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:10.317 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:09 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:10.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:09 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:10.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:09 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:21:10.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.803+0000 7f94eac56640 1 -- 172.21.15.5:0/1787835858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94e411b140 msgr2=0x7f94e411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.803+0000 7f94eac56640 1 --2- 172.21.15.5:0/1787835858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94e411b140 0x7f94e411d530 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009be0 tx=0x7f94dc02a710 comp rx=0 tx=0).stop 2024-09-15T22:21:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.803+0000 7f94eac56640 1 -- 172.21.15.5:0/1787835858 shutdown_connections 2024-09-15T22:21:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.803+0000 7f94eac56640 1 --2- 172.21.15.5:0/1787835858 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94e411b140 0x7f94e411d530 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.803+0000 7f94eac56640 1 --2- 172.21.15.5:0/1787835858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e406ea30 0x7f94e406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.803+0000 7f94eac56640 1 --2- 172.21.15.5:0/1787835858 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94e4110190 0x7f94e406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.803+0000 7f94eac56640 1 -- 172.21.15.5:0/1787835858 >> 172.21.15.5:0/1787835858 conn(0x7f94e4073180 msgr2=0x7f94e4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.803+0000 7f94eac56640 1 -- 172.21.15.5:0/1787835858 shutdown_connections 2024-09-15T22:21:10.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.804+0000 7f94eac56640 1 -- 172.21.15.5:0/1787835858 wait complete. 2024-09-15T22:21:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.804+0000 7f94eac56640 1 Processor -- start 2024-09-15T22:21:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.804+0000 7f94eac56640 1 -- start start 2024-09-15T22:21:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.805+0000 7f94eac56640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94e4110190 0x7f94e411d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.805+0000 7f94eac56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e411d620 0x7f94e4117160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.805+0000 7f94eac56640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94e4117840 0x7f94e41bbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.805+0000 7f94eac56640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41114b0 con 0x7f94e411d620 2024-09-15T22:21:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.805+0000 7f94eac56640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4111330 con 0x7f94e4117840 2024-09-15T22:21:10.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.805+0000 7f94eac56640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4111630 con 0x7f94e4110190 2024-09-15T22:21:10.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.810+0000 7f94ea455640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94e4117840 0x7f94e41bbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.810+0000 7f94ea455640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94e4117840 0x7f94e41bbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:36744/0 (socket says 172.21.15.5:36744) 2024-09-15T22:21:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.810+0000 7f94ea455640 1 -- 172.21.15.5:0/290945149 learned_addr learned my addr 172.21.15.5:0/290945149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.810+0000 7f94e9c54640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94e4110190 0x7f94e411d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.810+0000 7f94ea455640 1 -- 172.21.15.5:0/290945149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94e4110190 msgr2=0x7f94e411d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.810+0000 7f94ea455640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94e4110190 0x7f94e411d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.810+0000 7f94ea455640 1 -- 172.21.15.5:0/290945149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e411d620 msgr2=0x7f94e4117160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.810+0000 7f94ea455640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e411d620 0x7f94e4117160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.810+0000 7f94ea455640 1 -- 172.21.15.5:0/290945149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e41bc340 con 0x7f94e4117840 2024-09-15T22:21:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.811+0000 7f94ea455640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94e4117840 0x7f94e41bbe00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f94dc034040 tx=0x7f94dc030890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:10.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.811+0000 7f94daffd640 1 -- 172.21.15.5:0/290945149 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc040020 con 0x7f94e4117840 2024-09-15T22:21:10.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.811+0000 7f94eac56640 1 -- 172.21.15.5:0/290945149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e41bc510 con 0x7f94e4117840 2024-09-15T22:21:10.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.811+0000 7f94eac56640 1 -- 172.21.15.5:0/290945149 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e41bc9c0 con 0x7f94e4117840 2024-09-15T22:21:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.811+0000 7f94daffd640 1 -- 172.21.15.5:0/290945149 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc0309b0 con 0x7f94e4117840 2024-09-15T22:21:10.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.811+0000 7f94daffd640 1 -- 172.21.15.5:0/290945149 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc036040 con 0x7f94e4117840 2024-09-15T22:21:10.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.816+0000 7f94ee291740 1 Processor -- start 2024-09-15T22:21:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.819+0000 7f94ee291740 1 -- start start 2024-09-15T22:21:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.819+0000 7f94ee291740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7e9da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.819+0000 7f94ee291740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 0x5557d7ea8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.819+0000 7f94ee291740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7ea8850 0x5557d7eaac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.819+0000 7f94ee291740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5557d7e032e0 con 0x5557d7ea8850 2024-09-15T22:21:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.819+0000 7f94ee291740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5557d7daace0 con 0x5557d7e9d670 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.819+0000 7f94ee291740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5557d7dff2a0 con 0x5557d7e9df90 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.819+0000 7f94daffd640 1 -- 172.21.15.5:0/290945149 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f94dc043020 con 0x7f94e4117840 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.820+0000 7f94e9c54640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 0x5557d7ea8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.820+0000 7f94e9c54640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 0x5557d7ea8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58594/0 (socket says 172.21.15.5:58594) 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.820+0000 7f94e9c54640 1 -- 172.21.15.5:0/42217386 learned_addr learned my addr 172.21.15.5:0/42217386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.820+0000 7f94e9c54640 1 -- 172.21.15.5:0/42217386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 msgr2=0x5557d7e9da50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.820+0000 7f94e9c54640 1 --2- 172.21.15.5:0/42217386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7e9da50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.820+0000 7f94e9c54640 1 -- 172.21.15.5:0/42217386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7ea8850 msgr2=0x5557d7eaac40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.820+0000 7f94e9c54640 1 --2- 172.21.15.5:0/42217386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7ea8850 0x5557d7eaac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.820+0000 7f94e9c54640 1 -- 172.21.15.5:0/42217386 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557d7d8f440 con 0x5557d7e9df90 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.820+0000 7f94e9c54640 1 --2- 172.21.15.5:0/42217386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 0x5557d7ea8310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f94e000cbf0 tx=0x7f94e000e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=933fdb033d8a7c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.821+0000 7f94daffd640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f94d00a35c0 0x7f94d00a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.822+0000 7f94daffd640 1 -- 172.21.15.5:0/290945149 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9629+0+0 (secure 0 0 0) 0x7f94dc0f0bc0 con 0x7f94e4117840 2024-09-15T22:21:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.823+0000 7f94e9453640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f94d00a35c0 0x7f94d00a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.823+0000 7f94cffff640 1 -- 172.21.15.5:0/42217386 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0010630 con 0x5557d7e9df90 2024-09-15T22:21:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.823+0000 7f94cffff640 1 -- 172.21.15.5:0/42217386 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e0010cb0 con 0x5557d7e9df90 2024-09-15T22:21:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.823+0000 7f94cffff640 1 -- 172.21.15.5:0/42217386 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0019cc0 con 0x5557d7e9df90 2024-09-15T22:21:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.826+0000 7f94ee291740 1 -- 172.21.15.5:0/42217386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 msgr2=0x5557d7ea8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.826+0000 7f94ee291740 1 --2- 172.21.15.5:0/42217386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 0x5557d7ea8310 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f94e000cbf0 tx=0x7f94e000e720 comp rx=0 tx=0).stop 2024-09-15T22:21:10.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.830+0000 7f94ee291740 1 -- 172.21.15.5:0/42217386 shutdown_connections 2024-09-15T22:21:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.830+0000 7f94ee291740 1 --2- 172.21.15.5:0/42217386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7ea8850 0x5557d7eaac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.830+0000 7f94ee291740 1 --2- 172.21.15.5:0/42217386 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 0x5557d7ea8310 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.830+0000 7f94ee291740 1 --2- 172.21.15.5:0/42217386 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7e9da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.830+0000 7f94ee291740 1 -- 172.21.15.5:0/42217386 >> 172.21.15.5:0/42217386 conn(0x5557d7e99c00 msgr2=0x5557d7e99fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.832+0000 7f94e9453640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f94d00a35c0 0x7f94d00a5a80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f94d40058c0 tx=0x7f94d4005850 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:10.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.834+0000 7f94ee291740 1 -- 172.21.15.5:0/42217386 shutdown_connections 2024-09-15T22:21:10.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.836+0000 7f94ee291740 1 -- 172.21.15.5:0/42217386 wait complete. 2024-09-15T22:21:10.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.836+0000 7f94ee291740 1 -- start start 2024-09-15T22:21:10.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.836+0000 7f94ee291740 1 Processor -- start 2024-09-15T22:21:10.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.837+0000 7f94ee291740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7ea80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.837+0000 7f94ee291740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7e9df90 0x5557d7ea53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.837+0000 7f94ee291740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7ea8850 0x5557d7ea58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.837+0000 7f94ee291740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5557d7eab420 con 0x5557d7e9df90 2024-09-15T22:21:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.837+0000 7f94ee291740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5557d7dbe050 con 0x5557d7e9d670 2024-09-15T22:21:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.837+0000 7f94ee291740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5557d7eab5a0 con 0x5557d7ea8850 2024-09-15T22:21:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.837+0000 7f94ee291740 10 client.0 _ll_register_callbacks cb 0x5557d7dfe160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.838+0000 7f94ea455640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7ea8850 0x5557d7ea58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.838+0000 7f94e9453640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7e9df90 0x5557d7ea53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.838+0000 7f94e9453640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7e9df90 0x5557d7ea53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41140/0 (socket says 172.21.15.5:41140) 2024-09-15T22:21:10.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.838+0000 7f94e9453640 1 -- 172.21.15.5:0/3387045471 learned_addr learned my addr 172.21.15.5:0/3387045471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:10.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.841+0000 7f94e9c54640 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7ea80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.843+0000 7f94e9c54640 1 -- 172.21.15.5:0/3387045471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7ea8850 msgr2=0x5557d7ea58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.844+0000 7f94e9c54640 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7ea8850 0x5557d7ea58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.844+0000 7f94e9c54640 1 -- 172.21.15.5:0/3387045471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7e9df90 msgr2=0x5557d7ea53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.844+0000 7f94e9c54640 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7e9df90 0x5557d7ea53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.844+0000 7f94e9c54640 1 -- 172.21.15.5:0/3387045471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557d7b743c0 con 0x5557d7e9d670 2024-09-15T22:21:10.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.844+0000 7f94e9453640 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7e9df90 0x5557d7ea53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:10.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.846+0000 7f94e9c54640 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7ea80f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f94e0000c00 tx=0x7f94e0019900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30ceee0d7fa26dc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.847+0000 7f94cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:21:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.847+0000 7f94cd7fa640 1 -- 172.21.15.5:0/3387045471 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0029060 con 0x5557d7e9d670 2024-09-15T22:21:10.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.847+0000 7f94cd7fa640 1 -- 172.21.15.5:0/3387045471 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9629+0+0 (secure 0 0 0) 0x7f94e0005040 con 0x5557d7e9d670 2024-09-15T22:21:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.848+0000 7f94ee291740 1 -- 172.21.15.5:0/3387045471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94e0010cb0 con 0x5557d7e9d670 2024-09-15T22:21:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.849+0000 7f94cd7fa640 1 -- 172.21.15.5:0/3387045471 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f94e0010e60 con 0x5557d7e9d670 2024-09-15T22:21:10.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.851+0000 7f94ee291740 10 client.27062 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:10.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.851+0000 7f94ee291740 1 -- 172.21.15.5:0/3387045471 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5557d7ed8de0 con 0x5557d7e9d670 2024-09-15T22:21:10.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.851+0000 7f94cd7fa640 1 -- 172.21.15.5:0/3387045471 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94e0010cb0 con 0x5557d7e9d670 2024-09-15T22:21:10.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.851+0000 7f94cd7fa640 1 -- 172.21.15.5:0/3387045471 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f94e00046c0 con 0x5557d7e9d670 2024-09-15T22:21:10.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.852+0000 7f94ee291740 10 client.27062 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:10.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.852+0000 7f94ee291740 10 client.27062 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:10.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.852+0000 7f94ee291740 20 client.27062 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:10.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.852+0000 7f94ee291740 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5557d7eda2a0 0x5557d7edc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.852+0000 7f94ee291740 4 client.27062 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:10.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.852+0000 7f94ee291740 1 -- 172.21.15.5:0/3387045471 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557d7e10850 con 0x5557d7eda2a0 2024-09-15T22:21:10.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.853+0000 7f94e9453640 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5557d7eda2a0 0x5557d7edc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.855+0000 7f94e9453640 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5557d7eda2a0 0x5557d7edc690 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa83c0b355b5617d server_cookie=ec38706da9dc4c97 in_seq=0 out_seq=0 2024-09-15T22:21:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.855+0000 7f94cd7fa640 10 client.27062 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.856+0000 7f94cd7fa640 1 -- 172.21.15.5:0/3387045471 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557d7e10850 con 0x5557d7eda2a0 2024-09-15T22:21:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.857+0000 7f94cd7fa640 10 client.27062 handle_command_reply: tid=0 2024-09-15T22:21:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.857+0000 7f94ee291740 1 client.27062 shutdown 2024-09-15T22:21:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.860+0000 7f94ee291740 1 -- 172.21.15.5:0/3387045471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 msgr2=0x5557d7ea80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.860+0000 7f94ee291740 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7ea80f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f94e0000c00 tx=0x7f94e0019900 comp rx=0 tx=0).stop 2024-09-15T22:21:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.860+0000 7f94ee291740 1 -- 172.21.15.5:0/3387045471 shutdown_connections 2024-09-15T22:21:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.860+0000 7f94ee291740 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7ea8850 0x5557d7ea58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.860+0000 7f94ee291740 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7e9df90 0x5557d7ea53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.860+0000 7f94ee291740 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5557d7eda2a0 0x5557d7edc690 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.860+0000 7f94ee291740 1 --2- 172.21.15.5:0/3387045471 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7ea80f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.860+0000 7f94ee291740 1 -- 172.21.15.5:0/3387045471 >> 172.21.15.5:0/3387045471 conn(0x5557d7e99c00 msgr2=0x5557d7e9f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:10.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.861+0000 7f94cd7fa640 0 client.27062 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:10.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.864+0000 7f94ee291740 1 -- 172.21.15.5:0/3387045471 shutdown_connections 2024-09-15T22:21:10.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.864+0000 7f94ee291740 1 -- 172.21.15.5:0/3387045471 wait complete. 2024-09-15T22:21:10.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.864+0000 7f94ee291740 20 client.27062 trim_cache size 0 max 16384 2024-09-15T22:21:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.898+0000 7f94ee291740 1 Processor -- start 2024-09-15T22:21:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.898+0000 7f94ee291740 1 -- start start 2024-09-15T22:21:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.899+0000 7f94ee291740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7eb4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.899+0000 7f94ee291740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 0x5557d7e94b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.899+0000 7f94e9c54640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7eb4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.899+0000 7f94e9c54640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7eb4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:36762/0 (socket says 172.21.15.5:36762) 2024-09-15T22:21:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.900+0000 7f94e9453640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 0x5557d7e94b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.900+0000 7f94e9453640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 0x5557d7e94b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58608/0 (socket says 172.21.15.5:58608) 2024-09-15T22:21:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.900+0000 7f94e9c54640 1 -- 172.21.15.5:0/3961460636 learned_addr learned my addr 172.21.15.5:0/3961460636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.900+0000 7f94ee291740 1 --2- 172.21.15.5:0/3961460636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7ea8850 0x5557d7e9f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.900+0000 7f94ee291740 1 -- 172.21.15.5:0/3961460636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94c8012b10 con 0x5557d7ea8850 2024-09-15T22:21:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.900+0000 7f94ee291740 1 -- 172.21.15.5:0/3961460636 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f94c80103e0 con 0x5557d7e9d670 2024-09-15T22:21:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.900+0000 7f94ee291740 1 -- 172.21.15.5:0/3961460636 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94c800e250 con 0x5557d7e9df90 2024-09-15T22:21:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.900+0000 7f94ea455640 1 --2- 172.21.15.5:0/3961460636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7ea8850 0x5557d7e9f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.901+0000 7f94ea455640 1 -- 172.21.15.5:0/3961460636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 msgr2=0x5557d7e94b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.901+0000 7f94ea455640 1 --2- 172.21.15.5:0/3961460636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 0x5557d7e94b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.901+0000 7f94ea455640 1 -- 172.21.15.5:0/3961460636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 msgr2=0x5557d7eb4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.901+0000 7f94ea455640 1 --2- 172.21.15.5:0/3961460636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7eb4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.901+0000 7f94ea455640 1 -- 172.21.15.5:0/3961460636 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557d7ea4350 con 0x5557d7ea8850 2024-09-15T22:21:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.902+0000 7f94ea455640 1 --2- 172.21.15.5:0/3961460636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7ea8850 0x5557d7e9f7d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0ede60 tx=0x7f94dc0bc510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98acfa52261afb80 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.902+0000 7f94cf7fe640 1 -- 172.21.15.5:0/3961460636 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0baeb0 con 0x5557d7ea8850 2024-09-15T22:21:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.902+0000 7f94cf7fe640 1 -- 172.21.15.5:0/3961460636 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc04ddd0 con 0x5557d7ea8850 2024-09-15T22:21:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.902+0000 7f94cf7fe640 1 -- 172.21.15.5:0/3961460636 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0c22d0 con 0x5557d7ea8850 2024-09-15T22:21:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.903+0000 7f94ee291740 1 -- 172.21.15.5:0/3961460636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7ea8850 msgr2=0x5557d7e9f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.903+0000 7f94ee291740 1 --2- 172.21.15.5:0/3961460636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7ea8850 0x5557d7e9f7d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0ede60 tx=0x7f94dc0bc510 comp rx=0 tx=0).stop 2024-09-15T22:21:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.903+0000 7f94ee291740 1 -- 172.21.15.5:0/3961460636 shutdown_connections 2024-09-15T22:21:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.903+0000 7f94ee291740 1 --2- 172.21.15.5:0/3961460636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7ea8850 0x5557d7e9f7d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.904+0000 7f94ee291740 1 --2- 172.21.15.5:0/3961460636 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9df90 0x5557d7e94b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.904+0000 7f94ee291740 1 --2- 172.21.15.5:0/3961460636 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7e9d670 0x5557d7eb4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.904+0000 7f94ee291740 1 -- 172.21.15.5:0/3961460636 >> 172.21.15.5:0/3961460636 conn(0x5557d7e99c00 msgr2=0x5557d7e9f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.904+0000 7f94ee291740 1 -- 172.21.15.5:0/3961460636 shutdown_connections 2024-09-15T22:21:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.904+0000 7f94ee291740 1 -- 172.21.15.5:0/3961460636 wait complete. 2024-09-15T22:21:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.904+0000 7f94ee291740 1 -- start start 2024-09-15T22:21:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.904+0000 7f94ee291740 1 Processor -- start 2024-09-15T22:21:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.905+0000 7f94ee291740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9d670 0x5557d7eb0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.905+0000 7f94ee291740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7e9df90 0x5557d7eb0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.905+0000 7f94ee291740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7ea8850 0x5557d7ead850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.905+0000 7f94e9c54640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9d670 0x5557d7eb0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.905+0000 7f94e9c54640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9d670 0x5557d7eb0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:58612/0 (socket says 172.21.15.5:58612) 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.905+0000 7f94e9c54640 1 -- 172.21.15.5:0/922138527 learned_addr learned my addr 172.21.15.5:0/922138527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.905+0000 7f94ee291740 1 -- 172.21.15.5:0/922138527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94c800e8a0 con 0x5557d7e9df90 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.905+0000 7f94ee291740 1 -- 172.21.15.5:0/922138527 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f94c800fd10 con 0x5557d7ea8850 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.905+0000 7f94ee291740 1 -- 172.21.15.5:0/922138527 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f94c800ef70 con 0x5557d7e9d670 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.905+0000 7f94ea455640 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7ea8850 0x5557d7ead850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.905+0000 7f94ee291740 10 client.0 _ll_register_callbacks cb 0x7f94e0014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.906+0000 7f94e9c54640 1 -- 172.21.15.5:0/922138527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7ea8850 msgr2=0x5557d7ead850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.906+0000 7f94e9c54640 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7ea8850 0x5557d7ead850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.906+0000 7f94e9c54640 1 -- 172.21.15.5:0/922138527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7e9df90 msgr2=0x5557d7eb0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.906+0000 7f94e9c54640 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7e9df90 0x5557d7eb0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.906+0000 7f94e9c54640 1 -- 172.21.15.5:0/922138527 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557d7eade10 con 0x5557d7e9d670 2024-09-15T22:21:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.907+0000 7f94e9c54640 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9d670 0x5557d7eb0880 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94e000c9b0 tx=0x7f94e0005f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9e083015df312e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.907+0000 7f94cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:21:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.908+0000 7f94cd7fa640 1 -- 172.21.15.5:0/922138527 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0033080 con 0x5557d7e9d670 2024-09-15T22:21:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.908+0000 7f94cd7fa640 1 -- 172.21.15.5:0/922138527 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9629+0+0 (secure 0 0 0) 0x7f94e0004660 con 0x5557d7e9d670 2024-09-15T22:21:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.910+0000 7f94ee291740 1 -- 172.21.15.5:0/922138527 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc04ddd0 con 0x5557d7e9d670 2024-09-15T22:21:10.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.911+0000 7f94cd7fa640 1 -- 172.21.15.5:0/922138527 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f94e000ca70 con 0x5557d7e9d670 2024-09-15T22:21:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.912+0000 7f94ee291740 10 client.27016 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.912+0000 7f94ee291740 1 -- 172.21.15.5:0/922138527 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5557d7f45540 con 0x5557d7e9d670 2024-09-15T22:21:10.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.914+0000 7f94cd7fa640 1 -- 172.21.15.5:0/922138527 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc04ddd0 con 0x5557d7e9d670 2024-09-15T22:21:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.914+0000 7f94cd7fa640 1 -- 172.21.15.5:0/922138527 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f94e0005070 con 0x5557d7e9d670 2024-09-15T22:21:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.915+0000 7f94ee291740 10 client.27016 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.915+0000 7f94ee291740 10 client.27016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.915+0000 7f94ee291740 20 client.27016 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.915+0000 7f94ee291740 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5557d7f45c00 0x5557d7f48040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:10.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.915+0000 7f94ee291740 4 client.27016 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:21:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.917+0000 7f94e9453640 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5557d7f45c00 0x5557d7f48040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:10.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.917+0000 7f94ee291740 1 -- 172.21.15.5:0/922138527 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5557d7e0ad60 con 0x5557d7f45c00 2024-09-15T22:21:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.919+0000 7f94e9453640 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5557d7f45c00 0x5557d7f48040 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b28c68e676470a77 server_cookie=9ce80b6298e9601c in_seq=0 out_seq=0 2024-09-15T22:21:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.919+0000 7f94cd7fa640 10 client.27016 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.920+0000 7f94cd7fa640 1 -- 172.21.15.5:0/922138527 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5557d7e0ad60 con 0x5557d7f45c00 2024-09-15T22:21:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.920+0000 7f94cd7fa640 10 client.27016 handle_command_reply: tid=0 2024-09-15T22:21:10.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.920+0000 7f94ee291740 1 client.27016 shutdown 2024-09-15T22:21:10.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.920+0000 7f94ee291740 1 -- 172.21.15.5:0/922138527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9d670 msgr2=0x5557d7eb0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.920+0000 7f94ee291740 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9d670 0x5557d7eb0880 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f94e000c9b0 tx=0x7f94e0005f20 comp rx=0 tx=0).stop 2024-09-15T22:21:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.920+0000 7f94ee291740 1 -- 172.21.15.5:0/922138527 shutdown_connections 2024-09-15T22:21:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.920+0000 7f94ee291740 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5557d7f45c00 0x5557d7f48040 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.920+0000 7f94ee291740 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5557d7ea8850 0x5557d7ead850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.921+0000 7f94ee291740 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5557d7e9df90 0x5557d7eb0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.921+0000 7f94ee291740 1 --2- 172.21.15.5:0/922138527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5557d7e9d670 0x5557d7eb0880 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.921+0000 7f94ee291740 1 -- 172.21.15.5:0/922138527 >> 172.21.15.5:0/922138527 conn(0x5557d7e99c00 msgr2=0x5557d7e98d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:10.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.921+0000 7f94cd7fa640 0 client.27016 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.921+0000 7f94ee291740 1 -- 172.21.15.5:0/922138527 shutdown_connections 2024-09-15T22:21:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.921+0000 7f94ee291740 1 -- 172.21.15.5:0/922138527 wait complete. 2024-09-15T22:21:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.921+0000 7f94ee291740 20 client.27016 trim_cache size 0 max 16384 2024-09-15T22:21:10.925 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:21:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:21:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:21:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "ac9fe404-bddf-4568-9669-6885a493bb69": { 2024-09-15T22:21:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:21:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "ac9fe404-bddf-4568-9669-6885a493bb69", 2024-09-15T22:21:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:21:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:21:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:21:10.926 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:21:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.923+0000 7f94eac56640 1 -- 172.21.15.5:0/290945149 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f94d00a35c0 msgr2=0x7f94d00a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.923+0000 7f94eac56640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f94d00a35c0 0x7f94d00a5a80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f94d40058c0 tx=0x7f94d4005850 comp rx=0 tx=0).stop 2024-09-15T22:21:10.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.923+0000 7f94eac56640 1 -- 172.21.15.5:0/290945149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94e4117840 msgr2=0x7f94e41bbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.924+0000 7f94eac56640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94e4117840 0x7f94e41bbe00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f94dc034040 tx=0x7f94dc030890 comp rx=0 tx=0).stop 2024-09-15T22:21:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.924+0000 7f94eac56640 1 -- 172.21.15.5:0/290945149 shutdown_connections 2024-09-15T22:21:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.924+0000 7f94eac56640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f94d00a35c0 0x7f94d00a5a80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.924+0000 7f94eac56640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f94e4117840 0x7f94e41bbe00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.924+0000 7f94eac56640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94e411d620 0x7f94e4117160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.924+0000 7f94eac56640 1 --2- 172.21.15.5:0/290945149 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f94e4110190 0x7f94e411d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.924+0000 7f94eac56640 1 -- 172.21.15.5:0/290945149 >> 172.21.15.5:0/290945149 conn(0x7f94e4073180 msgr2=0x7f94e406e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.924+0000 7f94eac56640 1 -- 172.21.15.5:0/290945149 shutdown_connections 2024-09-15T22:21:10.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:10.924+0000 7f94eac56640 1 -- 172.21.15.5:0/290945149 wait complete. 2024-09-15T22:21:11.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:10 smithi099 ceph-mon[34892]: pgmap v382: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.2 MiB/s wr, 519 op/s 2024-09-15T22:21:11.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:10 smithi129 ceph-mon[35314]: pgmap v382: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.2 MiB/s wr, 519 op/s 2024-09-15T22:21:11.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:10 smithi005 ceph-mon[30944]: pgmap v382: 129 pgs: 129 active+clean; 2.7 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 6.2 MiB/s wr, 519 op/s 2024-09-15T22:21:11.666 INFO:tasks.cephfs.filesystem:scrub status for tag:ac9fe404-bddf-4568-9669-6885a493bb69 - {'path': '/', 'tag': 'ac9fe404-bddf-4568-9669-6885a493bb69', 'options': 'recursive,force'} 2024-09-15T22:21:13.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:12 smithi099 ceph-mon[34892]: pgmap v383: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.5 MiB/s wr, 509 op/s 2024-09-15T22:21:13.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:12 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:21:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:12 smithi129 ceph-mon[35314]: pgmap v383: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.5 MiB/s wr, 509 op/s 2024-09-15T22:21:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:12 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:21:13.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:12 smithi005 ceph-mon[30944]: pgmap v383: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 6.5 MiB/s wr, 509 op/s 2024-09-15T22:21:13.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:12 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:21:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 705.0 s, 705.4 tps, lat 132.111 ms stddev 153.138 2024-09-15T22:21:15.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:14 smithi099 ceph-mon[34892]: pgmap v384: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.7 MiB/s wr, 461 op/s 2024-09-15T22:21:15.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:14 smithi099 ceph-mon[34892]: osdmap e101: 12 total, 12 up, 12 in 2024-09-15T22:21:15.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:14 smithi129 ceph-mon[35314]: pgmap v384: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.7 MiB/s wr, 461 op/s 2024-09-15T22:21:15.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:14 smithi129 ceph-mon[35314]: osdmap e101: 12 total, 12 up, 12 in 2024-09-15T22:21:15.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:14 smithi005 ceph-mon[30944]: pgmap v384: 129 pgs: 129 active+clean; 2.7 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 5.7 MiB/s wr, 461 op/s 2024-09-15T22:21:15.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:14 smithi005 ceph-mon[30944]: osdmap e101: 12 total, 12 up, 12 in 2024-09-15T22:21:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:16 smithi099 ceph-mon[34892]: pgmap v386: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.3 MiB/s wr, 608 op/s 2024-09-15T22:21:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:16 smithi129 ceph-mon[35314]: pgmap v386: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.3 MiB/s wr, 608 op/s 2024-09-15T22:21:17.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:16 smithi005 ceph-mon[30944]: pgmap v386: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 7.3 MiB/s wr, 608 op/s 2024-09-15T22:21:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:18 smithi099 ceph-mon[34892]: pgmap v387: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.1 MiB/s wr, 651 op/s 2024-09-15T22:21:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:18 smithi129 ceph-mon[35314]: pgmap v387: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.1 MiB/s wr, 651 op/s 2024-09-15T22:21:19.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:18 smithi005 ceph-mon[30944]: pgmap v387: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.1 MiB/s wr, 651 op/s 2024-09-15T22:21:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 710.0 s, 655.0 tps, lat 158.979 ms stddev 215.705 2024-09-15T22:21:21.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:20 smithi099 ceph-mon[34892]: pgmap v388: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.0 MiB/s wr, 663 op/s 2024-09-15T22:21:21.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:21:21.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:21:21.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:20 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:21:21.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:20 smithi129 ceph-mon[35314]: pgmap v388: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.0 MiB/s wr, 663 op/s 2024-09-15T22:21:21.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:21:21.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:21:21.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:20 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:21:21.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:20 smithi005 ceph-mon[30944]: pgmap v388: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 8.0 MiB/s wr, 663 op/s 2024-09-15T22:21:21.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:21:21.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:21:21.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:20 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:21:22.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:21 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:21:22.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:21 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:21:22.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:21 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:21:22.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:21 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:21:22.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:21 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:21:22.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:21 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:21:22.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:21 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:21:22.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:21 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:21:22.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:21 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:21:23.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:23 smithi099 ceph-mon[34892]: pgmap v389: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.3 MiB/s wr, 709 op/s 2024-09-15T22:21:23.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:23 smithi129 ceph-mon[35314]: pgmap v389: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.3 MiB/s wr, 709 op/s 2024-09-15T22:21:23.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:23 smithi005 ceph-mon[30944]: pgmap v389: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.3 MiB/s wr, 709 op/s 2024-09-15T22:21:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 715.0 s, 597.8 tps, lat 165.152 ms stddev 207.277 2024-09-15T22:21:25.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:25 smithi099 ceph-mon[34892]: pgmap v390: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.4 MiB/s wr, 719 op/s 2024-09-15T22:21:25.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:25 smithi129 ceph-mon[35314]: pgmap v390: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.4 MiB/s wr, 719 op/s 2024-09-15T22:21:25.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:25 smithi005 ceph-mon[30944]: pgmap v390: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 8.4 MiB/s wr, 719 op/s 2024-09-15T22:21:27.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:27 smithi099 ceph-mon[34892]: pgmap v391: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.0 MiB/s wr, 719 op/s 2024-09-15T22:21:27.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:27 smithi129 ceph-mon[35314]: pgmap v391: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.0 MiB/s wr, 719 op/s 2024-09-15T22:21:27.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:27 smithi005 ceph-mon[30944]: pgmap v391: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 8.0 MiB/s wr, 719 op/s 2024-09-15T22:21:29.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:29 smithi099 ceph-mon[34892]: pgmap v392: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-09-15T22:21:29.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:29 smithi129 ceph-mon[35314]: pgmap v392: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-09-15T22:21:29.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:29 smithi005 ceph-mon[30944]: pgmap v392: 129 pgs: 129 active+clean; 2.8 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 7.1 MiB/s wr, 610 op/s 2024-09-15T22:21:29.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 720.0 s, 650.6 tps, lat 155.195 ms stddev 213.602 2024-09-15T22:21:31.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:31 smithi099 ceph-mon[34892]: pgmap v393: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2024-09-15T22:21:31.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:31 smithi129 ceph-mon[35314]: pgmap v393: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2024-09-15T22:21:31.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:31 smithi005 ceph-mon[30944]: pgmap v393: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 6.1 MiB/s wr, 526 op/s 2024-09-15T22:21:33.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:33 smithi099 ceph-mon[34892]: pgmap v394: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 8.7 MiB/s wr, 738 op/s 2024-09-15T22:21:33.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:33 smithi129 ceph-mon[35314]: pgmap v394: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 8.7 MiB/s wr, 738 op/s 2024-09-15T22:21:33.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:33 smithi005 ceph-mon[30944]: pgmap v394: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 8.7 MiB/s wr, 738 op/s 2024-09-15T22:21:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 725.0 s, 538.4 tps, lat 184.076 ms stddev 205.680 2024-09-15T22:21:35.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:35 smithi099 ceph-mon[34892]: pgmap v395: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.9 MiB/s wr, 527 op/s 2024-09-15T22:21:35.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:35 smithi129 ceph-mon[35314]: pgmap v395: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.9 MiB/s wr, 527 op/s 2024-09-15T22:21:35.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:35 smithi005 ceph-mon[30944]: pgmap v395: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.9 MiB/s wr, 527 op/s 2024-09-15T22:21:37.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:37 smithi099 ceph-mon[34892]: pgmap v396: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.8 MiB/s wr, 631 op/s 2024-09-15T22:21:37.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:37 smithi129 ceph-mon[35314]: pgmap v396: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.8 MiB/s wr, 631 op/s 2024-09-15T22:21:37.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:37 smithi005 ceph-mon[30944]: pgmap v396: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 6.8 MiB/s wr, 631 op/s 2024-09-15T22:21:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:39 smithi099 ceph-mon[34892]: pgmap v397: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.0 MiB/s wr, 595 op/s 2024-09-15T22:21:39.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:39 smithi129 ceph-mon[35314]: pgmap v397: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.0 MiB/s wr, 595 op/s 2024-09-15T22:21:39.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:39 smithi005 ceph-mon[30944]: pgmap v397: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 7.0 MiB/s wr, 595 op/s 2024-09-15T22:21:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 730.0 s, 679.0 tps, lat 149.497 ms stddev 210.368 2024-09-15T22:21:41.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:41 smithi129 ceph-mon[35314]: pgmap v398: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.0 MiB/s wr, 533 op/s 2024-09-15T22:21:41.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:41 smithi005 ceph-mon[30944]: pgmap v398: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.0 MiB/s wr, 533 op/s 2024-09-15T22:21:41.668 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:21:41.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:41 smithi099 ceph-mon[34892]: pgmap v398: 129 pgs: 129 active+clean; 2.9 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 6.0 MiB/s wr, 533 op/s 2024-09-15T22:21:42.230 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:43.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.084+0000 7fcf38af0640 1 -- 172.21.15.5:0/1885466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf34109e00 msgr2=0x7fcf3410d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.084+0000 7fcf38af0640 1 --2- 172.21.15.5:0/1885466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf34109e00 0x7fcf3410d690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009900 tx=0x7fcf2802c790 comp rx=0 tx=0).stop 2024-09-15T22:21:43.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.085+0000 7fcf38af0640 1 -- 172.21.15.5:0/1885466 shutdown_connections 2024-09-15T22:21:43.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.085+0000 7fcf38af0640 1 --2- 172.21.15.5:0/1885466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf3410ddc0 0x7fcf34115940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.085+0000 7fcf38af0640 1 --2- 172.21.15.5:0/1885466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf34109e00 0x7fcf3410d690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.085+0000 7fcf38af0640 1 --2- 172.21.15.5:0/1885466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf34109450 0x7fcf34109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.085+0000 7fcf38af0640 1 -- 172.21.15.5:0/1885466 >> 172.21.15.5:0/1885466 conn(0x7fcf3406e140 msgr2=0x7fcf3406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.085+0000 7fcf38af0640 1 -- 172.21.15.5:0/1885466 shutdown_connections 2024-09-15T22:21:43.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.085+0000 7fcf38af0640 1 -- 172.21.15.5:0/1885466 wait complete. 2024-09-15T22:21:43.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.086+0000 7fcf38af0640 1 Processor -- start 2024-09-15T22:21:43.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.089+0000 7fcf38af0640 1 -- start start 2024-09-15T22:21:43.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.089+0000 7fcf38af0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf34109450 0x7fcf34114450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.089+0000 7fcf38af0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf34109e00 0x7fcf34114990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.089+0000 7fcf38af0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf3410ddc0 0x7fcf34114ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.089+0000 7fcf38af0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34117c90 con 0x7fcf3410ddc0 2024-09-15T22:21:43.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.089+0000 7fcf38af0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34117b10 con 0x7fcf34109450 2024-09-15T22:21:43.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.089+0000 7fcf38af0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34117e10 con 0x7fcf34109e00 2024-09-15T22:21:43.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.090+0000 7fcf32d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf3410ddc0 0x7fcf34114ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.090+0000 7fcf32d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf3410ddc0 0x7fcf34114ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39122/0 (socket says 172.21.15.5:39122) 2024-09-15T22:21:43.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.090+0000 7fcf32d9d640 1 -- 172.21.15.5:0/2838004006 learned_addr learned my addr 172.21.15.5:0/2838004006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:43.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.090+0000 7fcf31d9b640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf34109e00 0x7fcf34114990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.091+0000 7fcf3259c640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf34109450 0x7fcf34114450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.096+0000 7fcf32d9d640 1 -- 172.21.15.5:0/2838004006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf34109e00 msgr2=0x7fcf34114990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.096+0000 7fcf32d9d640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf34109e00 0x7fcf34114990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.096+0000 7fcf32d9d640 1 -- 172.21.15.5:0/2838004006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf34109450 msgr2=0x7fcf34114450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.096+0000 7fcf32d9d640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf34109450 0x7fcf34114450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.096+0000 7fcf32d9d640 1 -- 172.21.15.5:0/2838004006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf341bbe30 con 0x7fcf3410ddc0 2024-09-15T22:21:43.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.097+0000 7fcf32d9d640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf3410ddc0 0x7fcf34114ed0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00b970 tx=0x7fcf2c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.098+0000 7fcf237fe640 1 -- 172.21.15.5:0/2838004006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c019070 con 0x7fcf3410ddc0 2024-09-15T22:21:43.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.098+0000 7fcf237fe640 1 -- 172.21.15.5:0/2838004006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf2c00cb40 con 0x7fcf3410ddc0 2024-09-15T22:21:43.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.098+0000 7fcf237fe640 1 -- 172.21.15.5:0/2838004006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c007650 con 0x7fcf3410ddc0 2024-09-15T22:21:43.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.098+0000 7fcf38af0640 1 -- 172.21.15.5:0/2838004006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf341bc120 con 0x7fcf3410ddc0 2024-09-15T22:21:43.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.099+0000 7fcf38af0640 1 -- 172.21.15.5:0/2838004006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf341bc5e0 con 0x7fcf3410ddc0 2024-09-15T22:21:43.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.103+0000 7fcf237fe640 1 -- 172.21.15.5:0/2838004006 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcf2c01c070 con 0x7fcf3410ddc0 2024-09-15T22:21:43.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.104+0000 7fcf237fe640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcf1c0a35c0 0x7fcf1c0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.104+0000 7fcf237fe640 1 -- 172.21.15.5:0/2838004006 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7fcf2c0cf4e0 con 0x7fcf3410ddc0 2024-09-15T22:21:43.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.108+0000 7fcf3259c640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcf1c0a35c0 0x7fcf1c0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.109+0000 7fcf3aec9740 1 Processor -- start 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.109+0000 7fcf3aec9740 1 -- start start 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.109+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e245594a0 0x555e24559880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.109+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24559dc0 0x555e24564140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.109+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e24566a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.109+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e244bf2e0 con 0x555e24564680 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.109+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555e24466ce0 con 0x555e24559dc0 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.109+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555e244bb2a0 con 0x555e245594a0 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.110+0000 7fcf31d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24559dc0 0x555e24564140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.110+0000 7fcf31d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24559dc0 0x555e24564140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:44762/0 (socket says 172.21.15.5:44762) 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.110+0000 7fcf31d9b640 1 -- 172.21.15.5:0/3938923310 learned_addr learned my addr 172.21.15.5:0/3938923310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:43.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.111+0000 7fcf32d9d640 1 --2- 172.21.15.5:0/3938923310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e24566a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.111+0000 7fcf31d9b640 1 -- 172.21.15.5:0/3938923310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e245594a0 msgr2=0x555e24559880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.111+0000 7fcf31d9b640 1 --2- 172.21.15.5:0/3938923310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e245594a0 0x555e24559880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.111+0000 7fcf31d9b640 1 -- 172.21.15.5:0/3938923310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 msgr2=0x555e24566a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.111+0000 7fcf31d9b640 1 --2- 172.21.15.5:0/3938923310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e24566a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.111+0000 7fcf31d9b640 1 -- 172.21.15.5:0/3938923310 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e2444b440 con 0x555e24559dc0 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.111+0000 7fcf31d9b640 1 --2- 172.21.15.5:0/3938923310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24559dc0 0x555e24564140 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcf3406f0d0 tx=0x7fcf28009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97aa4831fa2cdb91 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.112+0000 7fcf20ff9640 1 -- 172.21.15.5:0/3938923310 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28063070 con 0x555e24559dc0 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.112+0000 7fcf20ff9640 1 -- 172.21.15.5:0/3938923310 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf280035f0 con 0x555e24559dc0 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.113+0000 7fcf3aec9740 1 -- 172.21.15.5:0/3938923310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24559dc0 msgr2=0x555e24564140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.113+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/3938923310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24559dc0 0x555e24564140 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcf3406f0d0 tx=0x7fcf28009680 comp rx=0 tx=0).stop 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.113+0000 7fcf3aec9740 1 -- 172.21.15.5:0/3938923310 shutdown_connections 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.113+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/3938923310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e24566a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.113+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/3938923310 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24559dc0 0x555e24564140 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.113+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/3938923310 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e245594a0 0x555e24559880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.113+0000 7fcf3aec9740 1 -- 172.21.15.5:0/3938923310 >> 172.21.15.5:0/3938923310 conn(0x555e24555a30 msgr2=0x555e24555e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.117+0000 7fcf3aec9740 1 -- 172.21.15.5:0/3938923310 shutdown_connections 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.117+0000 7fcf3259c640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcf1c0a35c0 0x7fcf1c0a5a80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcf24006fd0 tx=0x7fcf240090e0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.117+0000 7fcf3aec9740 1 -- 172.21.15.5:0/3938923310 wait complete. 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.117+0000 7fcf3aec9740 1 -- start start 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.117+0000 7fcf3aec9740 1 Processor -- start 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.118+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e245594a0 0x555e24563e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.118+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e24561100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.118+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e24561640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.118+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e245672f0 con 0x555e24564680 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.118+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555e2447a050 con 0x555e245594a0 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.118+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555e24567470 con 0x555e24559dc0 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.118+0000 7fcf3aec9740 10 client.0 _ll_register_callbacks cb 0x555e244ba160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:43.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.118+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e245594a0 0x555e24563e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.118+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e245594a0 0x555e24563e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:44770/0 (socket says 172.21.15.5:44770) 2024-09-15T22:21:43.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.118+0000 7fcf3259c640 1 -- 172.21.15.5:0/1046384558 learned_addr learned my addr 172.21.15.5:0/1046384558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:43.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.118+0000 7fcf31d9b640 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e24561100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.119+0000 7fcf3259c640 1 -- 172.21.15.5:0/1046384558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 msgr2=0x555e24561100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.119+0000 7fcf3259c640 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e24561100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.119+0000 7fcf3259c640 1 -- 172.21.15.5:0/1046384558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 msgr2=0x555e24561640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.119+0000 7fcf3259c640 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e24561640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.119+0000 7fcf3259c640 1 -- 172.21.15.5:0/1046384558 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e242303c0 con 0x555e245594a0 2024-09-15T22:21:43.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.120+0000 7fcf3259c640 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e245594a0 0x555e24563e50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf24015800 tx=0x7fcf24004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dcbf21979d86eae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.120+0000 7fcf19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:21:43.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.121+0000 7fcf19ffb640 1 -- 172.21.15.5:0/1046384558 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24023020 con 0x555e245594a0 2024-09-15T22:21:43.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.121+0000 7fcf19ffb640 1 -- 172.21.15.5:0/1046384558 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7fcf24015d80 con 0x555e245594a0 2024-09-15T22:21:43.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.121+0000 7fcf3aec9740 1 -- 172.21.15.5:0/1046384558 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf280035f0 con 0x555e245594a0 2024-09-15T22:21:43.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.122+0000 7fcf19ffb640 1 -- 172.21.15.5:0/1046384558 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf2401b040 con 0x555e245594a0 2024-09-15T22:21:43.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.122+0000 7fcf3aec9740 10 client.27077 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:43.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.122+0000 7fcf3aec9740 1 -- 172.21.15.5:0/1046384558 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e24594c30 con 0x555e245594a0 2024-09-15T22:21:43.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.125+0000 7fcf19ffb640 1 -- 172.21.15.5:0/1046384558 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf280035f0 con 0x555e245594a0 2024-09-15T22:21:43.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.125+0000 7fcf19ffb640 1 -- 172.21.15.5:0/1046384558 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcf24030020 con 0x555e245594a0 2024-09-15T22:21:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.127+0000 7fcf3aec9740 10 client.27077 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.127+0000 7fcf3aec9740 10 client.27077 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.127+0000 7fcf3aec9740 20 client.27077 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.127+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e245964d0 0x555e24598960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.127+0000 7fcf3aec9740 4 client.27077 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.127+0000 7fcf3aec9740 1 -- 172.21.15.5:0/1046384558 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e244cc850 con 0x555e245964d0 2024-09-15T22:21:43.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.127+0000 7fcf31d9b640 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e245964d0 0x555e24598960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.131+0000 7fcf31d9b640 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e245964d0 0x555e24598960 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6afc57cb0ddff7aa server_cookie=752f380f0957a1c8 in_seq=0 out_seq=0 2024-09-15T22:21:43.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.132+0000 7fcf19ffb640 10 client.27077 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:43.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.132+0000 7fcf19ffb640 1 -- 172.21.15.5:0/1046384558 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e244cc850 con 0x555e245964d0 2024-09-15T22:21:43.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.132+0000 7fcf19ffb640 10 client.27077 handle_command_reply: tid=0 2024-09-15T22:21:43.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.132+0000 7fcf3aec9740 1 client.27077 shutdown 2024-09-15T22:21:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.135+0000 7fcf3aec9740 1 -- 172.21.15.5:0/1046384558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e245594a0 msgr2=0x555e24563e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.135+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e245594a0 0x555e24563e50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcf24015800 tx=0x7fcf24004750 comp rx=0 tx=0).stop 2024-09-15T22:21:43.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.137+0000 7fcf3aec9740 1 -- 172.21.15.5:0/1046384558 shutdown_connections 2024-09-15T22:21:43.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.137+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e24561640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.137+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e24561100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.137+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e245964d0 0x555e24598960 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.137+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/1046384558 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e245594a0 0x555e24563e50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.137+0000 7fcf3aec9740 1 -- 172.21.15.5:0/1046384558 >> 172.21.15.5:0/1046384558 conn(0x555e24555a30 msgr2=0x555e2455b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.137+0000 7fcf19ffb640 0 client.27077 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:43.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.138+0000 7fcf3aec9740 1 -- 172.21.15.5:0/1046384558 shutdown_connections 2024-09-15T22:21:43.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.138+0000 7fcf3aec9740 1 -- 172.21.15.5:0/1046384558 wait complete. 2024-09-15T22:21:43.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.138+0000 7fcf3aec9740 20 client.27077 trim_cache size 0 max 16384 2024-09-15T22:21:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.173+0000 7fcf3aec9740 1 Processor -- start 2024-09-15T22:21:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.173+0000 7fcf3aec9740 1 -- start start 2024-09-15T22:21:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.174+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e245594a0 0x555e2455c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.174+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e24555e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.174+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24564680 0x555e2459e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.174+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34130860 con 0x555e245594a0 2024-09-15T22:21:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.174+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3412e170 con 0x555e24564680 2024-09-15T22:21:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.174+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3410a1e0 con 0x555e24559dc0 2024-09-15T22:21:43.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.175+0000 7fcf31d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e24555e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.175+0000 7fcf32d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24564680 0x555e2459e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.175+0000 7fcf31d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e24555e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:55144/0 (socket says 172.21.15.5:55144) 2024-09-15T22:21:43.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.175+0000 7fcf31d9b640 1 -- 172.21.15.5:0/3377084614 learned_addr learned my addr 172.21.15.5:0/3377084614 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:43.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.175+0000 7fcf3259c640 1 --2- 172.21.15.5:0/3377084614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e245594a0 0x555e2455c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.177+0000 7fcf3259c640 1 -- 172.21.15.5:0/3377084614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 msgr2=0x555e24555e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.177+0000 7fcf3259c640 1 --2- 172.21.15.5:0/3377084614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e24555e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.177+0000 7fcf3259c640 1 -- 172.21.15.5:0/3377084614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24564680 msgr2=0x555e2459e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.177+0000 7fcf3259c640 1 --2- 172.21.15.5:0/3377084614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24564680 0x555e2459e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.177+0000 7fcf3259c640 1 -- 172.21.15.5:0/3377084614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e245600b0 con 0x555e245594a0 2024-09-15T22:21:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.177+0000 7fcf3259c640 1 --2- 172.21.15.5:0/3377084614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e245594a0 0x555e2455c7f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcf24014020 tx=0x7fcf24006d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36af3351f5e88718 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.179+0000 7fcf1bfff640 1 -- 172.21.15.5:0/3377084614 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24023070 con 0x555e245594a0 2024-09-15T22:21:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.179+0000 7fcf1bfff640 1 -- 172.21.15.5:0/3377084614 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf24009770 con 0x555e245594a0 2024-09-15T22:21:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.179+0000 7fcf1bfff640 1 -- 172.21.15.5:0/3377084614 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2401f710 con 0x555e245594a0 2024-09-15T22:21:43.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.180+0000 7fcf3aec9740 1 -- 172.21.15.5:0/3377084614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e245594a0 msgr2=0x555e2455c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.180+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/3377084614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e245594a0 0x555e2455c7f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcf24014020 tx=0x7fcf24006d70 comp rx=0 tx=0).stop 2024-09-15T22:21:43.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.181+0000 7fcf3aec9740 1 -- 172.21.15.5:0/3377084614 shutdown_connections 2024-09-15T22:21:43.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.181+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/3377084614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24564680 0x555e2459e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.181+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/3377084614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e24555e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.181+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/3377084614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e245594a0 0x555e2455c7f0 secure :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcf24014020 tx=0x7fcf24006d70 comp rx=0 tx=0).stop 2024-09-15T22:21:43.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.181+0000 7fcf3aec9740 1 -- 172.21.15.5:0/3377084614 >> 172.21.15.5:0/3377084614 conn(0x555e24555a30 msgr2=0x555e2455c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.188+0000 7fcf3aec9740 1 -- 172.21.15.5:0/3377084614 shutdown_connections 2024-09-15T22:21:43.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.189+0000 7fcf3aec9740 1 -- 172.21.15.5:0/3377084614 wait complete. 2024-09-15T22:21:43.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.190+0000 7fcf3aec9740 1 -- start start 2024-09-15T22:21:43.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.190+0000 7fcf3aec9740 1 Processor -- start 2024-09-15T22:21:43.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.191+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e2456cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.191+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e2456cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.191+0000 7fcf3aec9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24565230 0x555e245656b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.191+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3412c630 con 0x555e24564680 2024-09-15T22:21:43.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.191+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3412daa0 con 0x555e24565230 2024-09-15T22:21:43.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.191+0000 7fcf3aec9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3412cd00 con 0x555e24559dc0 2024-09-15T22:21:43.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.191+0000 7fcf3aec9740 10 client.0 _ll_register_callbacks cb 0x7fcf2400e7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:43.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.191+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e2456cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.192+0000 7fcf31d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e2456cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.192+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e2456cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:55158/0 (socket says 172.21.15.5:55158) 2024-09-15T22:21:43.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.192+0000 7fcf3259c640 1 -- 172.21.15.5:0/660839753 learned_addr learned my addr 172.21.15.5:0/660839753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:43.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.195+0000 7fcf32d9d640 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24565230 0x555e245656b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.195+0000 7fcf31d9b640 1 -- 172.21.15.5:0/660839753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 msgr2=0x555e2456cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.195+0000 7fcf31d9b640 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e2456cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.195+0000 7fcf31d9b640 1 -- 172.21.15.5:0/660839753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24565230 msgr2=0x555e245656b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:43.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.195+0000 7fcf31d9b640 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24565230 0x555e245656b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.195+0000 7fcf31d9b640 1 -- 172.21.15.5:0/660839753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e24565ee0 con 0x555e24564680 2024-09-15T22:21:43.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.195+0000 7fcf31d9b640 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e2456cff0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009850 tx=0x7fcf2802c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5827856f34b601c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:43.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.196+0000 7fcf19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:21:43.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.196+0000 7fcf19ffb640 1 -- 172.21.15.5:0/660839753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28063040 con 0x555e24564680 2024-09-15T22:21:43.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.196+0000 7fcf19ffb640 1 -- 172.21.15.5:0/660839753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7fcf28033740 con 0x555e24564680 2024-09-15T22:21:43.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.197+0000 7fcf3aec9740 1 -- 172.21.15.5:0/660839753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e244c6d60 con 0x555e24564680 2024-09-15T22:21:43.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.201+0000 7fcf19ffb640 1 -- 172.21.15.5:0/660839753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf28033960 con 0x555e24564680 2024-09-15T22:21:43.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.202+0000 7fcf3aec9740 10 client.17601 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:43.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.202+0000 7fcf3aec9740 1 -- 172.21.15.5:0/660839753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e246012d0 con 0x555e24564680 2024-09-15T22:21:43.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.207+0000 7fcf19ffb640 1 -- 172.21.15.5:0/660839753 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555e244c6d60 con 0x555e24564680 2024-09-15T22:21:43.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.207+0000 7fcf19ffb640 1 -- 172.21.15.5:0/660839753 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcf2805d040 con 0x555e24564680 2024-09-15T22:21:43.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.207+0000 7fcf3aec9740 10 client.17601 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:43.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.207+0000 7fcf3aec9740 10 client.17601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:43.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.207+0000 7fcf3aec9740 20 client.17601 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:43.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.207+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e24601560 0x555e24603950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:43.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.207+0000 7fcf3aec9740 4 client.17601 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:21:43.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.207+0000 7fcf3aec9740 1 -- 172.21.15.5:0/660839753 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e24443210 con 0x555e24601560 2024-09-15T22:21:43.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.208+0000 7fcf3259c640 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e24601560 0x555e24603950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:43.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.210+0000 7fcf3259c640 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e24601560 0x555e24603950 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b01678c0664efd2 server_cookie=bc05aa2d3fbe9d3 in_seq=0 out_seq=0 2024-09-15T22:21:43.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.212+0000 7fcf19ffb640 10 client.17601 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:43.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.214+0000 7fcf19ffb640 1 -- 172.21.15.5:0/660839753 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555e24443210 con 0x555e24601560 2024-09-15T22:21:43.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.214+0000 7fcf19ffb640 10 client.17601 handle_command_reply: tid=0 2024-09-15T22:21:43.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.215+0000 7fcf3aec9740 1 client.17601 shutdown 2024-09-15T22:21:43.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.215+0000 7fcf3aec9740 1 -- 172.21.15.5:0/660839753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 msgr2=0x555e2456cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.215+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e2456cff0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009850 tx=0x7fcf2802c790 comp rx=0 tx=0).stop 2024-09-15T22:21:43.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.217+0000 7fcf3aec9740 1 -- 172.21.15.5:0/660839753 shutdown_connections 2024-09-15T22:21:43.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.218+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e24601560 0x555e24603950 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.218+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e24565230 0x555e245656b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.218+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e24564680 0x555e2456cff0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.218+0000 7fcf3aec9740 1 --2- 172.21.15.5:0/660839753 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e24559dc0 0x555e2456cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.218+0000 7fcf3aec9740 1 -- 172.21.15.5:0/660839753 >> 172.21.15.5:0/660839753 conn(0x555e24555a30 msgr2=0x555e24554b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.218+0000 7fcf19ffb640 0 client.17601 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:43.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.221+0000 7fcf3aec9740 1 -- 172.21.15.5:0/660839753 shutdown_connections 2024-09-15T22:21:43.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.222+0000 7fcf3aec9740 1 -- 172.21.15.5:0/660839753 wait complete. 2024-09-15T22:21:43.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.222+0000 7fcf3aec9740 20 client.17601 trim_cache size 0 max 16384 2024-09-15T22:21:43.227 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:21:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:21:43.227 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:21:43.227 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:21:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.226+0000 7fcf38af0640 1 -- 172.21.15.5:0/2838004006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcf1c0a35c0 msgr2=0x7fcf1c0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.226+0000 7fcf38af0640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcf1c0a35c0 0x7fcf1c0a5a80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcf24006fd0 tx=0x7fcf240090e0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.226+0000 7fcf38af0640 1 -- 172.21.15.5:0/2838004006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf3410ddc0 msgr2=0x7fcf34114ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.226+0000 7fcf38af0640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf3410ddc0 0x7fcf34114ed0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00b970 tx=0x7fcf2c00be30 comp rx=0 tx=0).stop 2024-09-15T22:21:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.226+0000 7fcf38af0640 1 -- 172.21.15.5:0/2838004006 shutdown_connections 2024-09-15T22:21:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.226+0000 7fcf38af0640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcf1c0a35c0 0x7fcf1c0a5a80 secure :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcf24006fd0 tx=0x7fcf240090e0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.226+0000 7fcf38af0640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf3410ddc0 0x7fcf34114ed0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.226+0000 7fcf38af0640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcf34109e00 0x7fcf34114990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.226+0000 7fcf38af0640 1 --2- 172.21.15.5:0/2838004006 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcf34109450 0x7fcf34114450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:43.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.226+0000 7fcf38af0640 1 -- 172.21.15.5:0/2838004006 >> 172.21.15.5:0/2838004006 conn(0x7fcf3406e140 msgr2=0x7fcf34119520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:43.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.227+0000 7fcf38af0640 1 -- 172.21.15.5:0/2838004006 shutdown_connections 2024-09-15T22:21:43.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:43.227+0000 7fcf38af0640 1 -- 172.21.15.5:0/2838004006 wait complete. 2024-09-15T22:21:43.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:43 smithi005 ceph-mon[30944]: pgmap v399: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.1 MiB/s wr, 698 op/s 2024-09-15T22:21:43.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:43 smithi099 ceph-mon[34892]: pgmap v399: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.1 MiB/s wr, 698 op/s 2024-09-15T22:21:43.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:43 smithi129 ceph-mon[35314]: pgmap v399: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 8.1 MiB/s wr, 698 op/s 2024-09-15T22:21:44.356 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:21:44.357 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:21:44.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 735.0 s, 735.1 tps, lat 133.539 ms stddev 149.661 2024-09-15T22:21:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.488+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1232697670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4114ab0 msgr2=0x7fdcb4116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:45.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.488+0000 7fdcb8a17640 1 --2- 172.21.15.5:0/1232697670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4114ab0 0x7fdcb4116ea0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fdca4009900 tx=0x7fdca402a4a0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.495+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1232697670 shutdown_connections 2024-09-15T22:21:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.495+0000 7fdcb8a17640 1 --2- 172.21.15.5:0/1232697670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb4114ab0 0x7fdcb4116ea0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.495+0000 7fdcb8a17640 1 --2- 172.21.15.5:0/1232697670 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcb4109b10 0x7fdcb4114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.495+0000 7fdcb8a17640 1 --2- 172.21.15.5:0/1232697670 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcb41091f0 0x7fdcb41095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.495+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1232697670 >> 172.21.15.5:0/1232697670 conn(0x7fdcb406e140 msgr2=0x7fdcb406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:45.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.501+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1232697670 shutdown_connections 2024-09-15T22:21:45.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.501+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1232697670 wait complete. 2024-09-15T22:21:45.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.502+0000 7fdcb8a17640 1 Processor -- start 2024-09-15T22:21:45.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.506+0000 7fdcb8a17640 1 -- start start 2024-09-15T22:21:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.506+0000 7fdcb8a17640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb41091f0 0x7fdcb411ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.507+0000 7fdcb8a17640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcb4109b10 0x7fdcb411f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.507+0000 7fdcb8a17640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcb4114ab0 0x7fdcb41247f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.507+0000 7fdcb8a17640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb410c060 con 0x7fdcb41091f0 2024-09-15T22:21:45.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.507+0000 7fdcb8a17640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb410bee0 con 0x7fdcb4109b10 2024-09-15T22:21:45.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.507+0000 7fdcb8a17640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb410c1e0 con 0x7fdcb4114ab0 2024-09-15T22:21:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.508+0000 7fdcb1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcb4109b10 0x7fdcb411f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.508+0000 7fdcb1d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcb4109b10 0x7fdcb411f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:51550/0 (socket says 172.21.15.5:51550) 2024-09-15T22:21:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.508+0000 7fdcb1d9b640 1 -- 172.21.15.5:0/1140189776 learned_addr learned my addr 172.21.15.5:0/1140189776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.508+0000 7fdcb2d9d640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcb4114ab0 0x7fdcb41247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.509+0000 7fdcb1d9b640 1 -- 172.21.15.5:0/1140189776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcb4114ab0 msgr2=0x7fdcb41247f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.509+0000 7fdcb1d9b640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcb4114ab0 0x7fdcb41247f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.509+0000 7fdcb1d9b640 1 -- 172.21.15.5:0/1140189776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb41091f0 msgr2=0x7fdcb411ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.509+0000 7fdcb1d9b640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb41091f0 0x7fdcb411ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.509+0000 7fdcb1d9b640 1 -- 172.21.15.5:0/1140189776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb41250b0 con 0x7fdcb4109b10 2024-09-15T22:21:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.510+0000 7fdcb1d9b640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcb4109b10 0x7fdcb411f010 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e9f0 tx=0x7fdc9c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.511+0000 7fdcab7fe640 1 -- 172.21.15.5:0/1140189776 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c00cd80 con 0x7fdcb4109b10 2024-09-15T22:21:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.511+0000 7fdcab7fe640 1 -- 172.21.15.5:0/1140189776 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdc9c005500 con 0x7fdcb4109b10 2024-09-15T22:21:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.511+0000 7fdcab7fe640 1 -- 172.21.15.5:0/1140189776 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c010690 con 0x7fdcb4109b10 2024-09-15T22:21:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.511+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1140189776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb41253a0 con 0x7fdcb4109b10 2024-09-15T22:21:45.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.512+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1140189776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb4119e10 con 0x7fdcb4109b10 2024-09-15T22:21:45.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.513+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1140189776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc94005290 con 0x7fdcb4109b10 2024-09-15T22:21:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.519+0000 7fdcab7fe640 1 -- 172.21.15.5:0/1140189776 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fdc9c0046c0 con 0x7fdcb4109b10 2024-09-15T22:21:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.520+0000 7fdcab7fe640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdca00a3540 0x7fdca00a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.520+0000 7fdcab7fe640 1 -- 172.21.15.5:0/1140189776 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7fdc9c0d0620 con 0x7fdcb4109b10 2024-09-15T22:21:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.525+0000 7fdcab7fe640 1 -- 172.21.15.5:0/1140189776 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdc9c09cc80 con 0x7fdcb4109b10 2024-09-15T22:21:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.526+0000 7fdcb259c640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdca00a3540 0x7fdca00a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.538+0000 7fdcb259c640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdca00a3540 0x7fdca00a5a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00ab90 tx=0x7fdcac009250 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:45.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:45 smithi005 ceph-mon[30944]: pgmap v400: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.5 MiB/s wr, 486 op/s 2024-09-15T22:21:45.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:45 smithi099 ceph-mon[34892]: pgmap v400: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.5 MiB/s wr, 486 op/s 2024-09-15T22:21:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:45 smithi129 ceph-mon[35314]: pgmap v400: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 5.5 MiB/s wr, 486 op/s 2024-09-15T22:21:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.886+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1140189776 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc94005580 con 0x7fdcb4109b10 2024-09-15T22:21:45.890 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:21:45.891 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:21:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:21:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.888+0000 7fdcab7fe640 1 -- 172.21.15.5:0/1140189776 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fdc9c09c580 con 0x7fdcb4109b10 2024-09-15T22:21:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.898+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1140189776 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdca00a3540 msgr2=0x7fdca00a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.898+0000 7fdcb8a17640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdca00a3540 0x7fdca00a5a00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00ab90 tx=0x7fdcac009250 comp rx=0 tx=0).stop 2024-09-15T22:21:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.898+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1140189776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcb4109b10 msgr2=0x7fdcb411f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.898+0000 7fdcb8a17640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcb4109b10 0x7fdcb411f010 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e9f0 tx=0x7fdc9c00edf0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.898+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1140189776 shutdown_connections 2024-09-15T22:21:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.898+0000 7fdcb8a17640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdca00a3540 0x7fdca00a5a00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.898+0000 7fdcb8a17640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdcb4114ab0 0x7fdcb41247f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.898+0000 7fdcb8a17640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdcb4109b10 0x7fdcb411f010 secure :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00e9f0 tx=0x7fdc9c00edf0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.898+0000 7fdcb8a17640 1 --2- 172.21.15.5:0/1140189776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb41091f0 0x7fdcb411ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.898+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1140189776 >> 172.21.15.5:0/1140189776 conn(0x7fdcb406e140 msgr2=0x7fdcb4070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.898+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1140189776 shutdown_connections 2024-09-15T22:21:45.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:45.903+0000 7fdcb8a17640 1 -- 172.21.15.5:0/1140189776 wait complete. 2024-09-15T22:21:46.608 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:46 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1140189776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:46.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:21:46.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:46 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1140189776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:46.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:46 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1140189776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:21:47.140 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:47.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:47 smithi005 ceph-mon[30944]: pgmap v401: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.2 MiB/s wr, 565 op/s 2024-09-15T22:21:47.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:47 smithi099 ceph-mon[34892]: pgmap v401: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.2 MiB/s wr, 565 op/s 2024-09-15T22:21:47.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:47 smithi129 ceph-mon[35314]: pgmap v401: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 6.2 MiB/s wr, 565 op/s 2024-09-15T22:21:49.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 740.0 s, 485.3 tps, lat 202.912 ms stddev 249.569 2024-09-15T22:21:49.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:49 smithi099 ceph-mon[34892]: pgmap v402: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 521 op/s 2024-09-15T22:21:49.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:49 smithi129 ceph-mon[35314]: pgmap v402: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 521 op/s 2024-09-15T22:21:49.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:49 smithi005 ceph-mon[30944]: pgmap v402: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 6.3 MiB/s wr, 521 op/s 2024-09-15T22:21:50.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.212+0000 7fbd68f13640 1 -- 172.21.15.5:0/2258886600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd5c09b650 msgr2=0x7fbd5c09ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.212+0000 7fbd68f13640 1 --2- 172.21.15.5:0/2258886600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd5c09b650 0x7fbd5c09ba30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009890 tx=0x7fbd5402a380 comp rx=0 tx=0).stop 2024-09-15T22:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.213+0000 7fbd68f13640 1 -- 172.21.15.5:0/2258886600 shutdown_connections 2024-09-15T22:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.213+0000 7fbd68f13640 1 --2- 172.21.15.5:0/2258886600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd5c0963d0 0x7fbd5c0a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.213+0000 7fbd68f13640 1 --2- 172.21.15.5:0/2258886600 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd5c095a30 0x7fbd5c095e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.213+0000 7fbd68f13640 1 --2- 172.21.15.5:0/2258886600 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd5c09b650 0x7fbd5c09ba30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.213+0000 7fbd68f13640 1 -- 172.21.15.5:0/2258886600 >> 172.21.15.5:0/2258886600 conn(0x7fbd5c00b2d0 msgr2=0x7fbd5c00b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.215+0000 7fbd68f13640 1 -- 172.21.15.5:0/2258886600 shutdown_connections 2024-09-15T22:21:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.216+0000 7fbd68f13640 1 -- 172.21.15.5:0/2258886600 wait complete. 2024-09-15T22:21:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.216+0000 7fbd68f13640 1 Processor -- start 2024-09-15T22:21:50.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd68f13640 1 -- start start 2024-09-15T22:21:50.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd68f13640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd5c095a30 0x7fbd5c13cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd68f13640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd5c0963d0 0x7fbd5c13d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd68f13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd5c09b650 0x7fbd5c141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd68f13640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c0a5a20 con 0x7fbd5c09b650 2024-09-15T22:21:50.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd68f13640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c0a58a0 con 0x7fbd5c0963d0 2024-09-15T22:21:50.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd68f13640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c0a5ba0 con 0x7fbd5c095a30 2024-09-15T22:21:50.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd6259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd5c095a30 0x7fbd5c13cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd6259c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd5c095a30 0x7fbd5c13cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:51646/0 (socket says 172.21.15.5:51646) 2024-09-15T22:21:50.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd6259c640 1 -- 172.21.15.5:0/4044537635 learned_addr learned my addr 172.21.15.5:0/4044537635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:50.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd61d9b640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd5c0963d0 0x7fbd5c13d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.217+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd5c09b650 0x7fbd5c141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.218+0000 7fbd6259c640 1 -- 172.21.15.5:0/4044537635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd5c0963d0 msgr2=0x7fbd5c13d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.218+0000 7fbd6259c640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd5c0963d0 0x7fbd5c13d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.218+0000 7fbd6259c640 1 -- 172.21.15.5:0/4044537635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd5c09b650 msgr2=0x7fbd5c141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.218+0000 7fbd6259c640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd5c09b650 0x7fbd5c141820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.218+0000 7fbd6259c640 1 -- 172.21.15.5:0/4044537635 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c141f00 con 0x7fbd5c095a30 2024-09-15T22:21:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.218+0000 7fbd6259c640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd5c095a30 0x7fbd5c13cf50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbd540306c0 tx=0x7fbd54033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.218+0000 7fbd537fe640 1 -- 172.21.15.5:0/4044537635 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54037040 con 0x7fbd5c095a30 2024-09-15T22:21:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.218+0000 7fbd68f13640 1 -- 172.21.15.5:0/4044537635 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c142190 con 0x7fbd5c095a30 2024-09-15T22:21:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.218+0000 7fbd68f13640 1 -- 172.21.15.5:0/4044537635 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c0071a0 con 0x7fbd5c095a30 2024-09-15T22:21:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.218+0000 7fbd537fe640 1 -- 172.21.15.5:0/4044537635 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd54005510 con 0x7fbd5c095a30 2024-09-15T22:21:50.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.218+0000 7fbd537fe640 1 -- 172.21.15.5:0/4044537635 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd54004c10 con 0x7fbd5c095a30 2024-09-15T22:21:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.223+0000 7fbd6b2ec740 1 Processor -- start 2024-09-15T22:21:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.223+0000 7fbd6b2ec740 1 -- start start 2024-09-15T22:21:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.223+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e6a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.224+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6a780 0x563f36e6abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.224+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e75520 0x563f36e77910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.224+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563f36dd02f0 con 0x563f36e6a780 2024-09-15T22:21:50.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.224+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563f36d77cf0 con 0x563f36e6bcf0 2024-09-15T22:21:50.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.224+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563f36dcc2b0 con 0x563f36e75520 2024-09-15T22:21:50.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.224+0000 7fbd62d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e6a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.224+0000 7fbd62d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e6a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:51566/0 (socket says 172.21.15.5:51566) 2024-09-15T22:21:50.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.224+0000 7fbd62d9d640 1 -- 172.21.15.5:0/622926047 learned_addr learned my addr 172.21.15.5:0/622926047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:50.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.225+0000 7fbd62d9d640 1 -- 172.21.15.5:0/622926047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e75520 msgr2=0x563f36e77910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:21:50.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.225+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/622926047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e75520 0x563f36e77910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.225+0000 7fbd62d9d640 1 -- 172.21.15.5:0/622926047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6a780 msgr2=0x563f36e6abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.225+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/622926047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6a780 0x563f36e6abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.225+0000 7fbd62d9d640 1 -- 172.21.15.5:0/622926047 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f36d5c3a0 con 0x563f36e6bcf0 2024-09-15T22:21:50.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.225+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/622926047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e6a240 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbd640680a0 tx=0x7fbd64050650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ee65e84d11e05a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.226+0000 7fbd537fe640 1 -- 172.21.15.5:0/4044537635 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbd54004250 con 0x7fbd5c095a30 2024-09-15T22:21:50.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.226+0000 7fbd50ff9640 1 -- 172.21.15.5:0/622926047 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64068eb0 con 0x563f36e6bcf0 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.226+0000 7fbd50ff9640 1 -- 172.21.15.5:0/622926047 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd64069050 con 0x563f36e6bcf0 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.226+0000 7fbd50ff9640 1 -- 172.21.15.5:0/622926047 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6408e640 con 0x563f36e6bcf0 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.227+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/622926047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 msgr2=0x563f36e6a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.227+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/622926047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e6a240 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbd640680a0 tx=0x7fbd64050650 comp rx=0 tx=0).stop 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.227+0000 7fbd537fe640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbd4c0a3500 0x7fbd4c0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.227+0000 7fbd537fe640 1 -- 172.21.15.5:0/4044537635 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7fbd540effc0 con 0x7fbd5c095a30 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.227+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/622926047 shutdown_connections 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.227+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/622926047 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e75520 0x563f36e77910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.227+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/622926047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6a780 0x563f36e6abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.227+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/622926047 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e6a240 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.227+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/622926047 >> 172.21.15.5:0/622926047 conn(0x563f36e33310 msgr2=0x563f36e66e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.228+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/622926047 shutdown_connections 2024-09-15T22:21:50.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.228+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbd4c0a3500 0x7fbd4c0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.228+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/622926047 wait complete. 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.228+0000 7fbd6b2ec740 1 -- start start 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.228+0000 7fbd6b2ec740 1 Processor -- start 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.228+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbd4c0a3500 0x7fbd4c0a59c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd64065a40 tx=0x7fbd64051030 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.229+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e6a780 0x563f36e74f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.229+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6bcf0 0x563f36e72200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.229+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e75520 0x563f36e72990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.229+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563f36e780e0 con 0x563f36e6bcf0 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.229+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563f36d8b060 con 0x563f36e75520 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.229+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563f36e78260 con 0x563f36e6a780 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.229+0000 7fbd62d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e75520 0x563f36e72990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.229+0000 7fbd62d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e75520 0x563f36e72990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:51574/0 (socket says 172.21.15.5:51574) 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.229+0000 7fbd62d9d640 1 -- 172.21.15.5:0/4012871493 learned_addr learned my addr 172.21.15.5:0/4012871493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.229+0000 7fbd6b2ec740 10 client.0 _ll_register_callbacks cb 0x563f36ade8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.230+0000 7fbd62d9d640 1 -- 172.21.15.5:0/4012871493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e6a780 msgr2=0x563f36e74f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.230+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/4012871493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e6a780 0x563f36e74f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.230+0000 7fbd62d9d640 1 -- 172.21.15.5:0/4012871493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6bcf0 msgr2=0x563f36e72200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.230+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/4012871493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6bcf0 0x563f36e72200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.230+0000 7fbd62d9d640 1 -- 172.21.15.5:0/4012871493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f36b413c0 con 0x563f36e75520 2024-09-15T22:21:50.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.230+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/4012871493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e75520 0x563f36e72990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd640637b0 tx=0x7fbd64067ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee2859c27d9c4026 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.230+0000 7fbd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:21:50.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.234+0000 7fbd49ffb640 1 -- 172.21.15.5:0/4012871493 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6408fb00 con 0x563f36e75520 2024-09-15T22:21:50.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.234+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/4012871493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd64069050 con 0x563f36e75520 2024-09-15T22:21:50.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.234+0000 7fbd49ffb640 1 -- 172.21.15.5:0/4012871493 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7fbd6408fca0 con 0x563f36e75520 2024-09-15T22:21:50.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.235+0000 7fbd49ffb640 1 -- 172.21.15.5:0/4012871493 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd640af450 con 0x563f36e75520 2024-09-15T22:21:50.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.235+0000 7fbd6b2ec740 10 client.27095 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:50.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.235+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/4012871493 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563f36e83e50 con 0x563f36e75520 2024-09-15T22:21:50.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.239+0000 7fbd49ffb640 1 -- 172.21.15.5:0/4012871493 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd64069050 con 0x563f36e75520 2024-09-15T22:21:50.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.240+0000 7fbd49ffb640 1 -- 172.21.15.5:0/4012871493 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbd640a59c0 con 0x563f36e75520 2024-09-15T22:21:50.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.240+0000 7fbd6b2ec740 10 client.27095 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:50.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.240+0000 7fbd6b2ec740 10 client.27095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:21:50.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.240+0000 7fbd6b2ec740 20 client.27095 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:50.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.240+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/4012871493 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563f36e856c0 0x563f36e87b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.240+0000 7fbd6b2ec740 4 client.27095 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:50.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.240+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/4012871493 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f36ddd860 con 0x563f36e856c0 2024-09-15T22:21:50.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.240+0000 7fbd6259c640 1 --2- 172.21.15.5:0/4012871493 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563f36e856c0 0x563f36e87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.250+0000 7fbd6259c640 1 --2- 172.21.15.5:0/4012871493 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563f36e856c0 0x563f36e87b80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1e48caee01201d8 server_cookie=f27b1a511610d946 in_seq=0 out_seq=0 2024-09-15T22:21:50.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.251+0000 7fbd49ffb640 10 client.27095 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:21:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.255+0000 7fbd49ffb640 1 -- 172.21.15.5:0/4012871493 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563f36ddd860 con 0x563f36e856c0 2024-09-15T22:21:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.255+0000 7fbd49ffb640 10 client.27095 handle_command_reply: tid=0 2024-09-15T22:21:50.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.255+0000 7fbd6b2ec740 1 client.27095 shutdown 2024-09-15T22:21:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.259+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/4012871493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e75520 msgr2=0x563f36e72990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.259+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/4012871493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e75520 0x563f36e72990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbd640637b0 tx=0x7fbd64067ca0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.261+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/4012871493 shutdown_connections 2024-09-15T22:21:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.261+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/4012871493 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563f36e856c0 0x563f36e87b80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.261+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/4012871493 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e75520 0x563f36e72990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.261+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/4012871493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6bcf0 0x563f36e72200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.261+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/4012871493 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e6a780 0x563f36e74f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.261+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/4012871493 >> 172.21.15.5:0/4012871493 conn(0x563f36e33310 msgr2=0x563f36e6cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.262+0000 7fbd49ffb640 0 client.27095 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:21:50.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.267+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/4012871493 shutdown_connections 2024-09-15T22:21:50.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.268+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/4012871493 wait complete. 2024-09-15T22:21:50.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.271+0000 7fbd6b2ec740 20 client.27095 trim_cache size 0 max 16384 2024-09-15T22:21:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.299+0000 7fbd6b2ec740 1 Processor -- start 2024-09-15T22:21:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.299+0000 7fbd6b2ec740 1 -- start start 2024-09-15T22:21:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.299+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e6a780 0x563f36e721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.299+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e70af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.299+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e75520 0x563f36e6d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.299+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd440127b0 con 0x563f36e75520 2024-09-15T22:21:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.299+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44010080 con 0x563f36e6bcf0 2024-09-15T22:21:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.299+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4400df80 con 0x563f36e6a780 2024-09-15T22:21:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.300+0000 7fbd62d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e75520 0x563f36e6d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.301+0000 7fbd62d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e75520 0x563f36e6d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48882/0 (socket says 172.21.15.5:48882) 2024-09-15T22:21:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.301+0000 7fbd62d9d640 1 -- 172.21.15.5:0/704944070 learned_addr learned my addr 172.21.15.5:0/704944070 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.301+0000 7fbd6259c640 1 --2- 172.21.15.5:0/704944070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e6a780 0x563f36e721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.301+0000 7fbd62d9d640 1 -- 172.21.15.5:0/704944070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e6a780 msgr2=0x563f36e721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.301+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/704944070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e6a780 0x563f36e721d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.301+0000 7fbd62d9d640 1 -- 172.21.15.5:0/704944070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 msgr2=0x563f36e70af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.301+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/704944070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e70af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.302+0000 7fbd62d9d640 1 -- 172.21.15.5:0/704944070 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f36e711d0 con 0x563f36e75520 2024-09-15T22:21:50.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.302+0000 7fbd6259c640 1 --2- 172.21.15.5:0/704944070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e6a780 0x563f36e721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:21:50.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.303+0000 7fbd62d9d640 1 --2- 172.21.15.5:0/704944070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e75520 0x563f36e6d2d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbd64051220 tx=0x7fbd64068410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad008d634a9b541 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.303+0000 7fbd4bfff640 1 -- 172.21.15.5:0/704944070 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd640bc070 con 0x563f36e75520 2024-09-15T22:21:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.304+0000 7fbd4bfff640 1 -- 172.21.15.5:0/704944070 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd640aa070 con 0x563f36e75520 2024-09-15T22:21:50.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.304+0000 7fbd4bfff640 1 -- 172.21.15.5:0/704944070 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6408fd10 con 0x563f36e75520 2024-09-15T22:21:50.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.305+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/704944070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e75520 msgr2=0x563f36e6d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.305+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/704944070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e75520 0x563f36e6d2d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fbd64051220 tx=0x7fbd64068410 comp rx=0 tx=0).stop 2024-09-15T22:21:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.310+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/704944070 shutdown_connections 2024-09-15T22:21:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.310+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/704944070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e75520 0x563f36e6d2d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.310+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/704944070 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e70af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.310+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/704944070 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e6a780 0x563f36e721d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.310+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/704944070 >> 172.21.15.5:0/704944070 conn(0x563f36e33310 msgr2=0x563f36e71ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.314+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/704944070 shutdown_connections 2024-09-15T22:21:50.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.315+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/704944070 wait complete. 2024-09-15T22:21:50.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd6b2ec740 1 -- start start 2024-09-15T22:21:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd6b2ec740 1 Processor -- start 2024-09-15T22:21:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6a780 0x563f36e7e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e7eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd6b2ec740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e75520 0x563f36e7a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4400e540 con 0x563f36e6a780 2024-09-15T22:21:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4400f9b0 con 0x563f36e6bcf0 2024-09-15T22:21:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd6b2ec740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4400ec10 con 0x563f36e75520 2024-09-15T22:21:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd6b2ec740 10 client.0 _ll_register_callbacks cb 0x7fbd640688f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:50.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd62d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e75520 0x563f36e7a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd62d9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e75520 0x563f36e7a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:51664/0 (socket says 172.21.15.5:51664) 2024-09-15T22:21:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.316+0000 7fbd62d9d640 1 -- 172.21.15.5:0/684958965 learned_addr learned my addr 172.21.15.5:0/684958965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.317+0000 7fbd61d9b640 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e7eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.317+0000 7fbd6259c640 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6a780 0x563f36e7e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.318+0000 7fbd61d9b640 1 -- 172.21.15.5:0/684958965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e75520 msgr2=0x563f36e7a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.318+0000 7fbd61d9b640 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e75520 0x563f36e7a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.318+0000 7fbd61d9b640 1 -- 172.21.15.5:0/684958965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6a780 msgr2=0x563f36e7e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.318+0000 7fbd61d9b640 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6a780 0x563f36e7e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.318+0000 7fbd61d9b640 1 -- 172.21.15.5:0/684958965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f36e752c0 con 0x563f36e6bcf0 2024-09-15T22:21:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.319+0000 7fbd61d9b640 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e7eaa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x563f36e73840 tx=0x7fbd5800e8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7837c2fb490ed3f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:50.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.319+0000 7fbd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:21:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.319+0000 7fbd49ffb640 1 -- 172.21.15.5:0/684958965 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58017070 con 0x563f36e6bcf0 2024-09-15T22:21:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.319+0000 7fbd49ffb640 1 -- 172.21.15.5:0/684958965 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7fbd58013680 con 0x563f36e6bcf0 2024-09-15T22:21:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.319+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/684958965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd640aa070 con 0x563f36e6bcf0 2024-09-15T22:21:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.319+0000 7fbd49ffb640 1 -- 172.21.15.5:0/684958965 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbd58023050 con 0x563f36e6bcf0 2024-09-15T22:21:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.319+0000 7fbd6b2ec740 10 client.27098 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.319+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/684958965 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563f36eef9a0 con 0x563f36e6bcf0 2024-09-15T22:21:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.320+0000 7fbd49ffb640 1 -- 172.21.15.5:0/684958965 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd640aa070 con 0x563f36e6bcf0 2024-09-15T22:21:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.320+0000 7fbd49ffb640 1 -- 172.21.15.5:0/684958965 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fbd58012970 con 0x563f36e6bcf0 2024-09-15T22:21:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.320+0000 7fbd6b2ec740 10 client.27098 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.320+0000 7fbd6b2ec740 10 client.27098 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:21:50.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.320+0000 7fbd6b2ec740 20 client.27098 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.320+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563f36ef0120 0x563f36ef2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.320+0000 7fbd6b2ec740 4 client.27098 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:21:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.320+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/684958965 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563f36dd7d70 con 0x563f36ef0120 2024-09-15T22:21:50.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.321+0000 7fbd6259c640 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563f36ef0120 0x563f36ef2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.324+0000 7fbd6259c640 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563f36ef0120 0x563f36ef2540 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=599194f2e34ec9b7 server_cookie=7693009d7fd087e6 in_seq=0 out_seq=0 2024-09-15T22:21:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.324+0000 7fbd49ffb640 10 client.27098 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:21:50.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.324+0000 7fbd49ffb640 1 -- 172.21.15.5:0/684958965 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563f36dd7d70 con 0x563f36ef0120 2024-09-15T22:21:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.324+0000 7fbd49ffb640 10 client.27098 handle_command_reply: tid=0 2024-09-15T22:21:50.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.324+0000 7fbd6b2ec740 1 client.27098 shutdown 2024-09-15T22:21:50.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.327+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/684958965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 msgr2=0x563f36e7eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.327+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e7eaa0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x563f36e73840 tx=0x7fbd5800e8e0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.329+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/684958965 shutdown_connections 2024-09-15T22:21:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.329+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x563f36ef0120 0x563f36ef2540 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.329+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563f36e75520 0x563f36e7a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.329+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563f36e6bcf0 0x563f36e7eaa0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.329+0000 7fbd6b2ec740 1 --2- 172.21.15.5:0/684958965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f36e6a780 0x563f36e7e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.329+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/684958965 >> 172.21.15.5:0/684958965 conn(0x563f36e33310 msgr2=0x563f36e71ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.330+0000 7fbd49ffb640 0 client.27098 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:21:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.330+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/684958965 shutdown_connections 2024-09-15T22:21:50.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.333+0000 7fbd6b2ec740 1 -- 172.21.15.5:0/684958965 wait complete. 2024-09-15T22:21:50.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.334+0000 7fbd6b2ec740 20 client.27098 trim_cache size 0 max 16384 2024-09-15T22:21:50.341 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:21:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.338+0000 7fbd68f13640 1 -- 172.21.15.5:0/4044537635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbd4c0a3500 msgr2=0x7fbd4c0a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.338+0000 7fbd68f13640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbd4c0a3500 0x7fbd4c0a59c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbd64065a40 tx=0x7fbd64051030 comp rx=0 tx=0).stop 2024-09-15T22:21:50.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.338+0000 7fbd68f13640 1 -- 172.21.15.5:0/4044537635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd5c095a30 msgr2=0x7fbd5c13cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.338+0000 7fbd68f13640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd5c095a30 0x7fbd5c13cf50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbd540306c0 tx=0x7fbd54033d10 comp rx=0 tx=0).stop 2024-09-15T22:21:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.338+0000 7fbd68f13640 1 -- 172.21.15.5:0/4044537635 shutdown_connections 2024-09-15T22:21:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.338+0000 7fbd68f13640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbd4c0a3500 0x7fbd4c0a59c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.338+0000 7fbd68f13640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd5c09b650 0x7fbd5c141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.338+0000 7fbd68f13640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbd5c0963d0 0x7fbd5c13d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.338+0000 7fbd68f13640 1 --2- 172.21.15.5:0/4044537635 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbd5c095a30 0x7fbd5c13cf50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.338+0000 7fbd68f13640 1 -- 172.21.15.5:0/4044537635 >> 172.21.15.5:0/4044537635 conn(0x7fbd5c00b2d0 msgr2=0x7fbd5c0946d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.339+0000 7fbd68f13640 1 -- 172.21.15.5:0/4044537635 shutdown_connections 2024-09-15T22:21:50.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:50.339+0000 7fbd68f13640 1 -- 172.21.15.5:0/4044537635 wait complete. 2024-09-15T22:21:51.203 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:21:51.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:51 smithi005 ceph-mon[30944]: pgmap v403: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.3 MiB/s wr, 465 op/s 2024-09-15T22:21:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:51.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:51 smithi099 ceph-mon[34892]: pgmap v403: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.3 MiB/s wr, 465 op/s 2024-09-15T22:21:51.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:51 smithi129 ceph-mon[35314]: pgmap v403: 129 pgs: 129 active+clean; 3.0 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 5.3 MiB/s wr, 465 op/s 2024-09-15T22:21:52.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.415+0000 7fa75ca44640 1 -- 172.21.15.5:0/305959575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7581019d0 msgr2=0x7fa758101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.415+0000 7fa75ca44640 1 --2- 172.21.15.5:0/305959575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7581019d0 0x7fa758101e30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa750009930 tx=0x7fa75002a480 comp rx=0 tx=0).stop 2024-09-15T22:21:52.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.416+0000 7fa75ca44640 1 -- 172.21.15.5:0/305959575 shutdown_connections 2024-09-15T22:21:52.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.416+0000 7fa75ca44640 1 --2- 172.21.15.5:0/305959575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa758102370 0x7fa75810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.416+0000 7fa75ca44640 1 --2- 172.21.15.5:0/305959575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7581019d0 0x7fa758101e30 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.416+0000 7fa75ca44640 1 --2- 172.21.15.5:0/305959575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa7581079d0 0x7fa758107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.416+0000 7fa75ca44640 1 -- 172.21.15.5:0/305959575 >> 172.21.15.5:0/305959575 conn(0x7fa7580fd710 msgr2=0x7fa7580ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.416+0000 7fa75ca44640 1 -- 172.21.15.5:0/305959575 shutdown_connections 2024-09-15T22:21:52.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.417+0000 7fa75ca44640 1 -- 172.21.15.5:0/305959575 wait complete. 2024-09-15T22:21:52.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.418+0000 7fa75ca44640 1 Processor -- start 2024-09-15T22:21:52.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.418+0000 7fa75ca44640 1 -- start start 2024-09-15T22:21:52.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.419+0000 7fa75ca44640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7581019d0 0x7fa758198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.419+0000 7fa75ca44640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa758102370 0x7fa7581988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.419+0000 7fa75ca44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7581079d0 0x7fa75819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.419+0000 7fa75ca44640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa758111e60 con 0x7fa7581079d0 2024-09-15T22:21:52.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.419+0000 7fa75ca44640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa758111ce0 con 0x7fa7581019d0 2024-09-15T22:21:52.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.419+0000 7fa75ca44640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa758111fe0 con 0x7fa758102370 2024-09-15T22:21:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.419+0000 7fa756d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7581019d0 0x7fa758198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.420+0000 7fa756d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7581019d0 0x7fa758198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:51618/0 (socket says 172.21.15.5:51618) 2024-09-15T22:21:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.420+0000 7fa756d9d640 1 -- 172.21.15.5:0/1434639170 learned_addr learned my addr 172.21.15.5:0/1434639170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.420+0000 7fa75659c640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa758102370 0x7fa7581988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.420+0000 7fa756d9d640 1 -- 172.21.15.5:0/1434639170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa758102370 msgr2=0x7fa7581988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.420+0000 7fa756d9d640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa758102370 0x7fa7581988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.420+0000 7fa756d9d640 1 -- 172.21.15.5:0/1434639170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7581079d0 msgr2=0x7fa75819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.420+0000 7fa756d9d640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7581079d0 0x7fa75819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.420+0000 7fa756d9d640 1 -- 172.21.15.5:0/1434639170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75819d320 con 0x7fa7581019d0 2024-09-15T22:21:52.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.420+0000 7fa75659c640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa758102370 0x7fa7581988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:52.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.420+0000 7fa756d9d640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7581019d0 0x7fa758198370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa74800ba50 tx=0x7fa74800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.421+0000 7fa73ffff640 1 -- 172.21.15.5:0/1434639170 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748019070 con 0x7fa7581019d0 2024-09-15T22:21:52.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.421+0000 7fa73ffff640 1 -- 172.21.15.5:0/1434639170 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74800cc10 con 0x7fa7581019d0 2024-09-15T22:21:52.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.421+0000 7fa73ffff640 1 -- 172.21.15.5:0/1434639170 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748007730 con 0x7fa7581019d0 2024-09-15T22:21:52.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.421+0000 7fa75ca44640 1 -- 172.21.15.5:0/1434639170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75819d610 con 0x7fa7581019d0 2024-09-15T22:21:52.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.421+0000 7fa75ca44640 1 -- 172.21.15.5:0/1434639170 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7581a4ef0 con 0x7fa7581019d0 2024-09-15T22:21:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.425+0000 7fa73ffff640 1 -- 172.21.15.5:0/1434639170 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa74801c070 con 0x7fa7581019d0 2024-09-15T22:21:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.425+0000 7fa75f61e740 1 Processor -- start 2024-09-15T22:21:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.425+0000 7fa75f61e740 1 -- start start 2024-09-15T22:21:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.426+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349ccfccf0 0x56349ccfb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.426+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfb780 0x56349ccfbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.426+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 0x56349cd11120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.426+0000 7fa75f61e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56349cc612f0 con 0x56349cd0ed30 2024-09-15T22:21:52.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.426+0000 7fa75f61e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56349cc08cf0 con 0x56349ccfccf0 2024-09-15T22:21:52.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.426+0000 7fa75f61e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56349cc5d2b0 con 0x56349ccfb780 2024-09-15T22:21:52.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.426+0000 7fa75659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 0x56349cd11120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.426+0000 7fa75659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 0x56349cd11120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48918/0 (socket says 172.21.15.5:48918) 2024-09-15T22:21:52.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.426+0000 7fa75659c640 1 -- 172.21.15.5:0/510931156 learned_addr learned my addr 172.21.15.5:0/510931156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.426+0000 7fa75759e640 1 --2- 172.21.15.5:0/510931156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349ccfccf0 0x56349ccfb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.427+0000 7fa75759e640 1 -- 172.21.15.5:0/510931156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfb780 msgr2=0x56349ccfbbc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:21:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.427+0000 7fa75759e640 1 --2- 172.21.15.5:0/510931156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfb780 0x56349ccfbbc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.427+0000 7fa75759e640 1 -- 172.21.15.5:0/510931156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 msgr2=0x56349cd11120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.427+0000 7fa75759e640 1 --2- 172.21.15.5:0/510931156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 0x56349cd11120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.427+0000 7fa75759e640 1 -- 172.21.15.5:0/510931156 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349cbed3a0 con 0x56349ccfccf0 2024-09-15T22:21:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.427+0000 7fa75759e640 1 --2- 172.21.15.5:0/510931156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349ccfccf0 0x56349ccfb240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00b9c0 tx=0x7fa74c00be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25ee50c76a3a1a60 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.427+0000 7fa73ffff640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7400a35c0 0x7fa7400a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.427+0000 7fa73d7fa640 1 -- 172.21.15.5:0/510931156 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00dbe0 con 0x56349ccfccf0 2024-09-15T22:21:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.427+0000 7fa73d7fa640 1 -- 172.21.15.5:0/510931156 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c00dd80 con 0x56349ccfccf0 2024-09-15T22:21:52.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.427+0000 7fa73d7fa640 1 -- 172.21.15.5:0/510931156 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0049e0 con 0x56349ccfccf0 2024-09-15T22:21:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.428+0000 7fa756d9d640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7400a35c0 0x7fa7400a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.428+0000 7fa75f61e740 1 -- 172.21.15.5:0/510931156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349ccfccf0 msgr2=0x56349ccfb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.428+0000 7fa75f61e740 1 --2- 172.21.15.5:0/510931156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349ccfccf0 0x56349ccfb240 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa74c00b9c0 tx=0x7fa74c00be80 comp rx=0 tx=0).stop 2024-09-15T22:21:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.428+0000 7fa73ffff640 1 -- 172.21.15.5:0/1434639170 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7fa7480cf6a0 con 0x7fa7581019d0 2024-09-15T22:21:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.429+0000 7fa75f61e740 1 -- 172.21.15.5:0/510931156 shutdown_connections 2024-09-15T22:21:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.429+0000 7fa75f61e740 1 --2- 172.21.15.5:0/510931156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 0x56349cd11120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.429+0000 7fa75f61e740 1 --2- 172.21.15.5:0/510931156 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfb780 0x56349ccfbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.429+0000 7fa75f61e740 1 --2- 172.21.15.5:0/510931156 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349ccfccf0 0x56349ccfb240 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.429+0000 7fa75f61e740 1 -- 172.21.15.5:0/510931156 >> 172.21.15.5:0/510931156 conn(0x56349ccc4310 msgr2=0x56349ccf7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.429+0000 7fa75f61e740 1 -- 172.21.15.5:0/510931156 shutdown_connections 2024-09-15T22:21:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.429+0000 7fa756d9d640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7400a35c0 0x7fa7400a5a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x56349ccf8c70 tx=0x7fa748097a00 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.429+0000 7fa75f61e740 1 -- 172.21.15.5:0/510931156 wait complete. 2024-09-15T22:21:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75f61e740 1 -- start start 2024-09-15T22:21:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75f61e740 1 Processor -- start 2024-09-15T22:21:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349ccfb780 0x56349cd0e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfccf0 0x56349cd0b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 0x56349cd0be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75f61e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56349cd11a30 con 0x56349ccfb780 2024-09-15T22:21:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75f61e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56349cc1c060 con 0x56349cd0ed30 2024-09-15T22:21:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75f61e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56349cd11bb0 con 0x56349ccfccf0 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75f61e740 10 client.0 _ll_register_callbacks cb 0x56349c96f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349ccfb780 0x56349cd0e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349ccfb780 0x56349cd0e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48930/0 (socket says 172.21.15.5:48930) 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.430+0000 7fa75759e640 1 -- 172.21.15.5:0/2513385391 learned_addr learned my addr 172.21.15.5:0/2513385391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.431+0000 7fa75759e640 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 0x56349cd0be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.431+0000 7fa75759e640 1 -- 172.21.15.5:0/2513385391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfccf0 msgr2=0x56349cd0b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.431+0000 7fa75759e640 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfccf0 0x56349cd0b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.431+0000 7fa75759e640 1 -- 172.21.15.5:0/2513385391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 msgr2=0x56349cd0be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.431+0000 7fa75759e640 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 0x56349cd0be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.431+0000 7fa75759e640 1 -- 172.21.15.5:0/2513385391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349c9d23c0 con 0x56349ccfb780 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.431+0000 7fa75759e640 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349ccfb780 0x56349cd0e630 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005750 tx=0x7fa74c0055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5db62e31523dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.431+0000 7fa73aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.432+0000 7fa73aff5640 1 -- 172.21.15.5:0/2513385391 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0049b0 con 0x56349ccfb780 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.432+0000 7fa73aff5640 1 -- 172.21.15.5:0/2513385391 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7fa74c004b50 con 0x56349ccfb780 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.432+0000 7fa75f61e740 1 -- 172.21.15.5:0/2513385391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa74c00dd80 con 0x56349ccfb780 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.432+0000 7fa73aff5640 1 -- 172.21.15.5:0/2513385391 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa74c004d50 con 0x56349ccfb780 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.432+0000 7fa75f61e740 10 client.17625 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.432+0000 7fa75f61e740 1 -- 172.21.15.5:0/2513385391 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56349cd1d470 con 0x56349ccfb780 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.433+0000 7fa73aff5640 1 -- 172.21.15.5:0/2513385391 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c00dd80 con 0x56349ccfb780 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.433+0000 7fa73aff5640 1 -- 172.21.15.5:0/2513385391 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa74c01c430 con 0x56349ccfb780 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.433+0000 7fa75f61e740 10 client.17625 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.433+0000 7fa75f61e740 10 client.17625 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.433+0000 7fa75f61e740 20 client.17625 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.433+0000 7fa75f61e740 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56349cd1e880 0x56349cd20c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.433+0000 7fa75f61e740 4 client.17625 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.433+0000 7fa75f61e740 1 -- 172.21.15.5:0/2513385391 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56349cc6e860 con 0x56349cd1e880 2024-09-15T22:21:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.433+0000 7fa75659c640 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56349cd1e880 0x56349cd20c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.434+0000 7fa73aff5640 10 client.17625 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:21:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.434+0000 7fa75659c640 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56349cd1e880 0x56349cd20c70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bccd216d8b6ddd1 server_cookie=79b2e6183976d1a9 in_seq=0 out_seq=0 2024-09-15T22:21:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.435+0000 7fa73aff5640 1 -- 172.21.15.5:0/2513385391 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56349cc6e860 con 0x56349cd1e880 2024-09-15T22:21:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.435+0000 7fa73aff5640 10 client.17625 handle_command_reply: tid=0 2024-09-15T22:21:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.435+0000 7fa75f61e740 1 client.17625 shutdown 2024-09-15T22:21:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.436+0000 7fa75f61e740 1 -- 172.21.15.5:0/2513385391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349ccfb780 msgr2=0x56349cd0e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.436+0000 7fa75f61e740 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349ccfb780 0x56349cd0e630 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005750 tx=0x7fa74c0055b0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.437+0000 7fa75f61e740 1 -- 172.21.15.5:0/2513385391 shutdown_connections 2024-09-15T22:21:52.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.437+0000 7fa75f61e740 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56349cd1e880 0x56349cd20c70 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.437+0000 7fa75f61e740 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 0x56349cd0be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.437+0000 7fa75f61e740 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfccf0 0x56349cd0b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.437+0000 7fa75f61e740 1 --2- 172.21.15.5:0/2513385391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349ccfb780 0x56349cd0e630 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.437+0000 7fa75f61e740 1 -- 172.21.15.5:0/2513385391 >> 172.21.15.5:0/2513385391 conn(0x56349ccc4310 msgr2=0x56349ccfdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.437+0000 7fa73aff5640 0 client.17625 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:21:52.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.437+0000 7fa75f61e740 1 -- 172.21.15.5:0/2513385391 shutdown_connections 2024-09-15T22:21:52.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.437+0000 7fa75f61e740 1 -- 172.21.15.5:0/2513385391 wait complete. 2024-09-15T22:21:52.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.438+0000 7fa75f61e740 20 client.17625 trim_cache size 0 max 16384 2024-09-15T22:21:52.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.461+0000 7fa75f61e740 1 Processor -- start 2024-09-15T22:21:52.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.461+0000 7fa75f61e740 1 -- start start 2024-09-15T22:21:52.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.461+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfb780 0x56349cd1ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.461+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349ccfccf0 0x56349cd0b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.461+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 0x56349cd10120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75f61e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7581af730 con 0x56349ccfccf0 2024-09-15T22:21:52.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 0x56349cd10120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 0x56349cd10120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:51650/0 (socket says 172.21.15.5:51650) 2024-09-15T22:21:52.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75759e640 1 -- 172.21.15.5:0/261997276 learned_addr learned my addr 172.21.15.5:0/261997276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:52.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75f61e740 1 -- 172.21.15.5:0/261997276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7581ad000 con 0x56349cd0ed30 2024-09-15T22:21:52.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75f61e740 1 -- 172.21.15.5:0/261997276 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7581aae70 con 0x56349ccfb780 2024-09-15T22:21:52.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75759e640 1 --2- 172.21.15.5:0/261997276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfb780 0x56349cd1ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75759e640 1 -- 172.21.15.5:0/261997276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfb780 msgr2=0x56349cd1ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75759e640 1 --2- 172.21.15.5:0/261997276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfb780 0x56349cd1ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75759e640 1 -- 172.21.15.5:0/261997276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349ccfccf0 msgr2=0x56349cd0b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75759e640 1 --2- 172.21.15.5:0/261997276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349ccfccf0 0x56349cd0b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.462+0000 7fa75759e640 1 -- 172.21.15.5:0/261997276 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56349cd0e9a0 con 0x56349cd0ed30 2024-09-15T22:21:52.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.463+0000 7fa75759e640 1 --2- 172.21.15.5:0/261997276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 0x56349cd10120 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005480 tx=0x7fa74c004ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1b46623ee8da9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.463+0000 7fa73cff9640 1 -- 172.21.15.5:0/261997276 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c00d6b0 con 0x56349cd0ed30 2024-09-15T22:21:52.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.463+0000 7fa73cff9640 1 -- 172.21.15.5:0/261997276 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c00d9e0 con 0x56349cd0ed30 2024-09-15T22:21:52.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.463+0000 7fa73cff9640 1 -- 172.21.15.5:0/261997276 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74c0049b0 con 0x56349cd0ed30 2024-09-15T22:21:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.464+0000 7fa75f61e740 1 -- 172.21.15.5:0/261997276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 msgr2=0x56349cd10120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.464+0000 7fa75f61e740 1 --2- 172.21.15.5:0/261997276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 0x56349cd10120 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa74c005480 tx=0x7fa74c004ef0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.464+0000 7fa75f61e740 1 -- 172.21.15.5:0/261997276 shutdown_connections 2024-09-15T22:21:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.464+0000 7fa75f61e740 1 --2- 172.21.15.5:0/261997276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349cd0ed30 0x56349cd10120 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.464+0000 7fa75f61e740 1 --2- 172.21.15.5:0/261997276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349ccfccf0 0x56349cd0b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.464+0000 7fa75f61e740 1 --2- 172.21.15.5:0/261997276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfb780 0x56349cd1ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.464+0000 7fa75f61e740 1 -- 172.21.15.5:0/261997276 >> 172.21.15.5:0/261997276 conn(0x56349ccc4310 msgr2=0x56349ccf6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.464+0000 7fa75f61e740 1 -- 172.21.15.5:0/261997276 shutdown_connections 2024-09-15T22:21:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.464+0000 7fa75f61e740 1 -- 172.21.15.5:0/261997276 wait complete. 2024-09-15T22:21:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.465+0000 7fa75f61e740 1 -- start start 2024-09-15T22:21:52.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.465+0000 7fa75f61e740 1 Processor -- start 2024-09-15T22:21:52.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.465+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349ccfb780 0x56349cd0f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.465+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfccf0 0x56349cd0fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.465+0000 7fa75f61e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 0x56349cd15e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.465+0000 7fa75f61e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7581ab4c0 con 0x56349cd0ed30 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.465+0000 7fa75f61e740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa7581ac930 con 0x56349ccfb780 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.465+0000 7fa75f61e740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa7581abb90 con 0x56349ccfccf0 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.465+0000 7fa75759e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349ccfb780 0x56349cd0f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.465+0000 7fa75659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfccf0 0x56349cd0fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfccf0 0x56349cd0fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:51716/0 (socket says 172.21.15.5:51716) 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75659c640 1 -- 172.21.15.5:0/3030671500 learned_addr learned my addr 172.21.15.5:0/3030671500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75f61e740 10 client.0 _ll_register_callbacks cb 0x7fa750009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75759e640 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 0x56349cd15e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75659c640 1 -- 172.21.15.5:0/3030671500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349ccfb780 msgr2=0x56349cd0f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75659c640 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349ccfb780 0x56349cd0f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75659c640 1 -- 172.21.15.5:0/3030671500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 msgr2=0x56349cd15e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75659c640 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 0x56349cd15e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75659c640 1 -- 172.21.15.5:0/3030671500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56349cd163b0 con 0x56349ccfccf0 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75759e640 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 0x56349cd15e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75659c640 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfccf0 0x56349cd0fa30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa750009930 tx=0x7fa7500094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278f706e55ff75ac server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa73aff5640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa73aff5640 1 -- 172.21.15.5:0/3030671500 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa750035040 con 0x56349ccfccf0 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa73aff5640 1 -- 172.21.15.5:0/3030671500 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7fa750004480 con 0x56349ccfccf0 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.466+0000 7fa75f61e740 1 -- 172.21.15.5:0/3030671500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa74c00d9e0 con 0x56349ccfccf0 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa73aff5640 1 -- 172.21.15.5:0/3030671500 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa750052050 con 0x56349ccfccf0 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa75f61e740 10 client.27052 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa75f61e740 1 -- 172.21.15.5:0/3030671500 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56349cd17690 con 0x56349ccfccf0 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa73aff5640 1 -- 172.21.15.5:0/3030671500 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa74c00d9e0 con 0x56349ccfccf0 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa73aff5640 1 -- 172.21.15.5:0/3030671500 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa7500035f0 con 0x56349ccfccf0 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa75f61e740 10 client.27052 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa75f61e740 10 client.27052 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa75f61e740 20 client.27052 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa75f61e740 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56349cd894e0 0x56349cd8b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa75f61e740 4 client.27052 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa75f61e740 1 -- 172.21.15.5:0/3030671500 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56349cc68d70 con 0x56349cd894e0 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.467+0000 7fa75759e640 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56349cd894e0 0x56349cd8b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa73aff5640 10 client.27052 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa75759e640 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56349cd894e0 0x56349cd8b900 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9687f424153d645b server_cookie=a75775e5209efc09 in_seq=0 out_seq=0 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa73aff5640 1 -- 172.21.15.5:0/3030671500 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56349cc68d70 con 0x56349cd894e0 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa73aff5640 10 client.27052 handle_command_reply: tid=0 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa75f61e740 1 client.27052 shutdown 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa75f61e740 1 -- 172.21.15.5:0/3030671500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfccf0 msgr2=0x56349cd0fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa75f61e740 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfccf0 0x56349cd0fa30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa750009930 tx=0x7fa7500094b0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa75f61e740 1 -- 172.21.15.5:0/3030671500 shutdown_connections 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa75f61e740 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x56349cd894e0 0x56349cd8b900 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa75f61e740 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349cd0ed30 0x56349cd15e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa75f61e740 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56349ccfccf0 0x56349cd0fa30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa75f61e740 1 --2- 172.21.15.5:0/3030671500 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56349ccfb780 0x56349cd0f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa75f61e740 1 -- 172.21.15.5:0/3030671500 >> 172.21.15.5:0/3030671500 conn(0x56349ccc4310 msgr2=0x56349ccfd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.468+0000 7fa73aff5640 0 client.27052 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:21:52.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.469+0000 7fa75f61e740 1 -- 172.21.15.5:0/3030671500 shutdown_connections 2024-09-15T22:21:52.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.469+0000 7fa75f61e740 1 -- 172.21.15.5:0/3030671500 wait complete. 2024-09-15T22:21:52.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.469+0000 7fa75f61e740 20 client.27052 trim_cache size 0 max 16384 2024-09-15T22:21:52.475 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.470+0000 7fa75ca44640 1 -- 172.21.15.5:0/1434639170 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7400a35c0 msgr2=0x7fa7400a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.470+0000 7fa75ca44640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7400a35c0 0x7fa7400a5a80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x56349ccf8c70 tx=0x7fa748097a00 comp rx=0 tx=0).stop 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.470+0000 7fa75ca44640 1 -- 172.21.15.5:0/1434639170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7581019d0 msgr2=0x7fa758198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.470+0000 7fa75ca44640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7581019d0 0x7fa758198370 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa74800ba50 tx=0x7fa74800bf10 comp rx=0 tx=0).stop 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.470+0000 7fa75ca44640 1 -- 172.21.15.5:0/1434639170 shutdown_connections 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.470+0000 7fa75ca44640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7400a35c0 0x7fa7400a5a80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.470+0000 7fa75ca44640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7581079d0 0x7fa75819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.470+0000 7fa75ca44640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa758102370 0x7fa7581988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.470+0000 7fa75ca44640 1 --2- 172.21.15.5:0/1434639170 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa7581019d0 0x7fa758198370 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.470+0000 7fa75ca44640 1 -- 172.21.15.5:0/1434639170 >> 172.21.15.5:0/1434639170 conn(0x7fa7580fd710 msgr2=0x7fa7580fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.470+0000 7fa75ca44640 1 -- 172.21.15.5:0/1434639170 shutdown_connections 2024-09-15T22:21:52.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:52.471+0000 7fa75ca44640 1 -- 172.21.15.5:0/1434639170 wait complete. 2024-09-15T22:21:53.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:21:53.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:53 smithi099 ceph-mon[34892]: pgmap v404: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-09-15T22:21:53.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:53 smithi129 ceph-mon[35314]: pgmap v404: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-09-15T22:21:53.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:53 smithi005 ceph-mon[30944]: pgmap v404: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 7.2 MiB/s wr, 623 op/s 2024-09-15T22:21:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:21:54.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 745.0 s, 510.8 tps, lat 201.212 ms stddev 262.013 2024-09-15T22:21:55.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:55 smithi005 ceph-mon[30944]: pgmap v405: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.0 MiB/s wr, 468 op/s 2024-09-15T22:21:55.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:55 smithi099 ceph-mon[34892]: pgmap v405: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.0 MiB/s wr, 468 op/s 2024-09-15T22:21:55.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:55 smithi129 ceph-mon[35314]: pgmap v405: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 5.0 MiB/s wr, 468 op/s 2024-09-15T22:21:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.845+0000 7f3d2ca27640 1 -- 172.21.15.5:0/4287584045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d28108450 msgr2=0x7f3d28108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.845+0000 7f3d2ca27640 1 --2- 172.21.15.5:0/4287584045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d28108450 0x7f3d28108830 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009be0 tx=0x7f3d1802a6e0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.849+0000 7f3d2ca27640 1 -- 172.21.15.5:0/4287584045 shutdown_connections 2024-09-15T22:21:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.849+0000 7f3d2ca27640 1 --2- 172.21.15.5:0/4287584045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d2810d530 0x7f3d281150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.849+0000 7f3d2ca27640 1 --2- 172.21.15.5:0/4287584045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d28108e00 0x7f3d2810ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.849+0000 7f3d2ca27640 1 --2- 172.21.15.5:0/4287584045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d28108450 0x7f3d28108830 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.849+0000 7f3d2ca27640 1 -- 172.21.15.5:0/4287584045 >> 172.21.15.5:0/4287584045 conn(0x7f3d28077ba0 msgr2=0x7f3d28079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:55.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.856+0000 7f3d2ca27640 1 -- 172.21.15.5:0/4287584045 shutdown_connections 2024-09-15T22:21:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.856+0000 7f3d2ca27640 1 -- 172.21.15.5:0/4287584045 wait complete. 2024-09-15T22:21:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.857+0000 7f3d2ca27640 1 Processor -- start 2024-09-15T22:21:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.857+0000 7f3d2ca27640 1 -- start start 2024-09-15T22:21:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.857+0000 7f3d2ca27640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d28108450 0x7f3d28073bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.857+0000 7f3d2ca27640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d28108e00 0x7f3d2806dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.857+0000 7f3d2ca27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d2810d530 0x7f3d2806e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.857+0000 7f3d2ca27640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2806c660 con 0x7f3d2810d530 2024-09-15T22:21:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.857+0000 7f3d2ca27640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2806c4e0 con 0x7f3d28108e00 2024-09-15T22:21:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.857+0000 7f3d2ca27640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2806c7e0 con 0x7f3d28108450 2024-09-15T22:21:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.857+0000 7f3d25d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d28108e00 0x7f3d2806dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.857+0000 7f3d25d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d28108e00 0x7f3d2806dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:54746/0 (socket says 172.21.15.5:54746) 2024-09-15T22:21:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.858+0000 7f3d25d9b640 1 -- 172.21.15.5:0/3743590487 learned_addr learned my addr 172.21.15.5:0/3743590487 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.858+0000 7f3d26d9d640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d2810d530 0x7f3d2806e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.858+0000 7f3d25d9b640 1 -- 172.21.15.5:0/3743590487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d28108450 msgr2=0x7f3d28073bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.858+0000 7f3d25d9b640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d28108450 0x7f3d28073bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.858+0000 7f3d25d9b640 1 -- 172.21.15.5:0/3743590487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d2810d530 msgr2=0x7f3d2806e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.858+0000 7f3d25d9b640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d2810d530 0x7f3d2806e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.858+0000 7f3d25d9b640 1 -- 172.21.15.5:0/3743590487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2806e980 con 0x7f3d28108e00 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.858+0000 7f3d26d9d640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d2810d530 0x7f3d2806e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.858+0000 7f3d25d9b640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d28108e00 0x7f3d2806dc70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000d950 tx=0x7f3d1000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.859+0000 7f3d1f7fe640 1 -- 172.21.15.5:0/3743590487 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10017070 con 0x7f3d28108e00 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.859+0000 7f3d2ca27640 1 -- 172.21.15.5:0/3743590487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d281b3900 con 0x7f3d28108e00 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.859+0000 7f3d1f7fe640 1 -- 172.21.15.5:0/3743590487 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d10009c30 con 0x7f3d28108e00 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.859+0000 7f3d1f7fe640 1 -- 172.21.15.5:0/3743590487 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d10020060 con 0x7f3d28108e00 2024-09-15T22:21:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.860+0000 7f3d2ca27640 1 -- 172.21.15.5:0/3743590487 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281b3d50 con 0x7f3d28108e00 2024-09-15T22:21:55.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.863+0000 7f3d1f7fe640 1 -- 172.21.15.5:0/3743590487 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3d100109d0 con 0x7f3d28108e00 2024-09-15T22:21:55.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.864+0000 7f3d1f7fe640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d140a3500 0x7f3d140a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.864+0000 7f3d1f7fe640 1 -- 172.21.15.5:0/3743590487 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7f3d100cf790 con 0x7f3d28108e00 2024-09-15T22:21:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.870+0000 7f3d2659c640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d140a3500 0x7f3d140a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.874+0000 7f3d2ee00740 1 Processor -- start 2024-09-15T22:21:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.875+0000 7f3d2ee00740 1 -- start start 2024-09-15T22:21:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.875+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873cfb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.875+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873cfb5d0 0x55d873cfba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.875+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d17380 0x55d873d19770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.875+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d873c612f0 con 0x55d873cfb5d0 2024-09-15T22:21:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.875+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d873c08cf0 con 0x55d873d17380 2024-09-15T22:21:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.875+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d873c5d2b0 con 0x55d873cfcb40 2024-09-15T22:21:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.875+0000 7f3d2659c640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d140a3500 0x7f3d140a59c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d180099a0 tx=0x7f3d18000f40 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.876+0000 7f3d26d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d17380 0x55d873d19770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.876+0000 7f3d26d9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d17380 0x55d873d19770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:54754/0 (socket says 172.21.15.5:54754) 2024-09-15T22:21:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.876+0000 7f3d26d9d640 1 -- 172.21.15.5:0/4206778717 learned_addr learned my addr 172.21.15.5:0/4206778717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.876+0000 7f3d26d9d640 1 -- 172.21.15.5:0/4206778717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 msgr2=0x55d873cfb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.876+0000 7f3d26d9d640 1 --2- 172.21.15.5:0/4206778717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873cfb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.876+0000 7f3d26d9d640 1 -- 172.21.15.5:0/4206778717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873cfb5d0 msgr2=0x55d873cfba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.876+0000 7f3d26d9d640 1 --2- 172.21.15.5:0/4206778717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873cfb5d0 0x55d873cfba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.876+0000 7f3d26d9d640 1 -- 172.21.15.5:0/4206778717 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d873bed3a0 con 0x55d873d17380 2024-09-15T22:21:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.877+0000 7f3d26d9d640 1 --2- 172.21.15.5:0/4206778717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d17380 0x55d873d19770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3d200079b0 tx=0x7f3d20007e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=676c554054f216d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:55.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.879+0000 7f3d1cff9640 1 -- 172.21.15.5:0/4206778717 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20015020 con 0x55d873d17380 2024-09-15T22:21:55.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.879+0000 7f3d1cff9640 1 -- 172.21.15.5:0/4206778717 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2000cc00 con 0x55d873d17380 2024-09-15T22:21:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.880+0000 7f3d2ee00740 1 -- 172.21.15.5:0/4206778717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d17380 msgr2=0x55d873d19770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.880+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/4206778717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d17380 0x55d873d19770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3d200079b0 tx=0x7f3d20007e70 comp rx=0 tx=0).stop 2024-09-15T22:21:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.879+0000 7f3d1cff9640 1 -- 172.21.15.5:0/4206778717 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20010630 con 0x55d873d17380 2024-09-15T22:21:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.880+0000 7f3d2ee00740 1 -- 172.21.15.5:0/4206778717 shutdown_connections 2024-09-15T22:21:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.880+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/4206778717 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d17380 0x55d873d19770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.880+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/4206778717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873cfb5d0 0x55d873cfba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.880+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/4206778717 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873cfb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.880+0000 7f3d2ee00740 1 -- 172.21.15.5:0/4206778717 >> 172.21.15.5:0/4206778717 conn(0x55d873cc4310 msgr2=0x55d873cf7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:55.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.894+0000 7f3d2ee00740 1 -- 172.21.15.5:0/4206778717 shutdown_connections 2024-09-15T22:21:55.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.894+0000 7f3d2ee00740 1 -- 172.21.15.5:0/4206778717 wait complete. 2024-09-15T22:21:55.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.895+0000 7f3d2ee00740 1 -- start start 2024-09-15T22:21:55.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.895+0000 7f3d2ee00740 1 Processor -- start 2024-09-15T22:21:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.895+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873d19540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.895+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d167f0 0x55d873d16c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.896+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d171b0 0x55d873d23650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.896+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d873d1a120 con 0x55d873d171b0 2024-09-15T22:21:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.896+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d873c1c060 con 0x55d873d167f0 2024-09-15T22:21:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.896+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d873d1a2a0 con 0x55d873cfcb40 2024-09-15T22:21:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.896+0000 7f3d2ee00740 10 client.0 _ll_register_callbacks cb 0x55d87396f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.896+0000 7f3d2659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873d19540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.896+0000 7f3d2659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873d19540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:38518/0 (socket says 172.21.15.5:38518) 2024-09-15T22:21:55.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.896+0000 7f3d2659c640 1 -- 172.21.15.5:0/3950451259 learned_addr learned my addr 172.21.15.5:0/3950451259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.898+0000 7f3d2659c640 1 -- 172.21.15.5:0/3950451259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d167f0 msgr2=0x55d873d16c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:21:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.898+0000 7f3d2659c640 1 --2- 172.21.15.5:0/3950451259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d167f0 0x55d873d16c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.898+0000 7f3d2659c640 1 -- 172.21.15.5:0/3950451259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d171b0 msgr2=0x55d873d23650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.898+0000 7f3d2659c640 1 --2- 172.21.15.5:0/3950451259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d171b0 0x55d873d23650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.898+0000 7f3d2659c640 1 -- 172.21.15.5:0/3950451259 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8739d23c0 con 0x55d873cfcb40 2024-09-15T22:21:55.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.898+0000 7f3d2659c640 1 --2- 172.21.15.5:0/3950451259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873d19540 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3d18003c20 tx=0x7f3d18003c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18fe0909b80f162e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.899+0000 7f3d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:21:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.899+0000 7f3d0dffb640 1 -- 172.21.15.5:0/3950451259 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18036040 con 0x55d873cfcb40 2024-09-15T22:21:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.899+0000 7f3d2ee00740 1 -- 172.21.15.5:0/3950451259 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d2000cc00 con 0x55d873cfcb40 2024-09-15T22:21:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.899+0000 7f3d0dffb640 1 -- 172.21.15.5:0/3950451259 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7f3d18004040 con 0x55d873cfcb40 2024-09-15T22:21:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.899+0000 7f3d0dffb640 1 -- 172.21.15.5:0/3950451259 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d18004240 con 0x55d873cfcb40 2024-09-15T22:21:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.900+0000 7f3d2ee00740 10 client.27055 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:55.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.900+0000 7f3d2ee00740 1 -- 172.21.15.5:0/3950451259 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d873d26280 con 0x55d873cfcb40 2024-09-15T22:21:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.905+0000 7f3d0dffb640 1 -- 172.21.15.5:0/3950451259 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d2000cc00 con 0x55d873cfcb40 2024-09-15T22:21:55.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.905+0000 7f3d0dffb640 1 -- 172.21.15.5:0/3950451259 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3d1802fcf0 con 0x55d873cfcb40 2024-09-15T22:21:55.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.905+0000 7f3d2ee00740 10 client.27055 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:55.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.905+0000 7f3d2ee00740 10 client.27055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:55.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.905+0000 7f3d2ee00740 20 client.27055 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:55.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.905+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/3950451259 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d873d27b50 0x55d873d29fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.905+0000 7f3d2ee00740 4 client.27055 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:21:55.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.905+0000 7f3d2ee00740 1 -- 172.21.15.5:0/3950451259 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d873c6e860 con 0x55d873d27b50 2024-09-15T22:21:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.907+0000 7f3d25d9b640 1 --2- 172.21.15.5:0/3950451259 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d873d27b50 0x55d873d29fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.908+0000 7f3d25d9b640 1 --2- 172.21.15.5:0/3950451259 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d873d27b50 0x55d873d29fd0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f408ba8a81fe97fd server_cookie=96b44f6d54c66448 in_seq=0 out_seq=0 2024-09-15T22:21:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.908+0000 7f3d0dffb640 10 client.27055 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.909+0000 7f3d0dffb640 1 -- 172.21.15.5:0/3950451259 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d873c6e860 con 0x55d873d27b50 2024-09-15T22:21:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.909+0000 7f3d0dffb640 10 client.27055 handle_command_reply: tid=0 2024-09-15T22:21:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.909+0000 7f3d2ee00740 1 client.27055 shutdown 2024-09-15T22:21:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.910+0000 7f3d2ee00740 1 -- 172.21.15.5:0/3950451259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 msgr2=0x55d873d19540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.910+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/3950451259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873d19540 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3d18003c20 tx=0x7f3d18003c50 comp rx=0 tx=0).stop 2024-09-15T22:21:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.913+0000 7f3d2ee00740 1 -- 172.21.15.5:0/3950451259 shutdown_connections 2024-09-15T22:21:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.913+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/3950451259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d171b0 0x55d873d23650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.913+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/3950451259 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d873d27b50 0x55d873d29fd0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.913+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/3950451259 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d167f0 0x55d873d16c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.913+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/3950451259 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873d19540 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.913+0000 7f3d2ee00740 1 -- 172.21.15.5:0/3950451259 >> 172.21.15.5:0/3950451259 conn(0x55d873cc4310 msgr2=0x55d873cfd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.915+0000 7f3d0dffb640 0 client.27055 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:55.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.916+0000 7f3d2ee00740 1 -- 172.21.15.5:0/3950451259 shutdown_connections 2024-09-15T22:21:55.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.917+0000 7f3d2ee00740 1 -- 172.21.15.5:0/3950451259 wait complete. 2024-09-15T22:21:55.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.918+0000 7f3d2ee00740 20 client.27055 trim_cache size 0 max 16384 2024-09-15T22:21:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.943+0000 7f3d2ee00740 1 Processor -- start 2024-09-15T22:21:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.943+0000 7f3d2ee00740 1 -- start start 2024-09-15T22:21:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.943+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cc4310 0x55d873d11a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.943+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873cfcb40 0x55d873d11f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.943+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d167f0 0x55d873d23440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.943+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281b8220 con 0x55d873d167f0 2024-09-15T22:21:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.943+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281b5af0 con 0x55d873cfcb40 2024-09-15T22:21:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.943+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3d28082bc0 con 0x55d873cc4310 2024-09-15T22:21:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.944+0000 7f3d26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d167f0 0x55d873d23440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.944+0000 7f3d26d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d167f0 0x55d873d23440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38504/0 (socket says 172.21.15.5:38504) 2024-09-15T22:21:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.944+0000 7f3d26d9d640 1 -- 172.21.15.5:0/2278074518 learned_addr learned my addr 172.21.15.5:0/2278074518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.944+0000 7f3d25d9b640 1 --2- 172.21.15.5:0/2278074518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873cfcb40 0x55d873d11f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.944+0000 7f3d2659c640 1 --2- 172.21.15.5:0/2278074518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cc4310 0x55d873d11a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.946+0000 7f3d2659c640 1 -- 172.21.15.5:0/2278074518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873cfcb40 msgr2=0x55d873d11f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.946+0000 7f3d2659c640 1 --2- 172.21.15.5:0/2278074518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873cfcb40 0x55d873d11f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.946+0000 7f3d2659c640 1 -- 172.21.15.5:0/2278074518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d167f0 msgr2=0x55d873d23440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.946+0000 7f3d2659c640 1 --2- 172.21.15.5:0/2278074518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d167f0 0x55d873d23440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.946+0000 7f3d2659c640 1 -- 172.21.15.5:0/2278074518 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d873d14d40 con 0x55d873cc4310 2024-09-15T22:21:55.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.948+0000 7f3d2659c640 1 --2- 172.21.15.5:0/2278074518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cc4310 0x55d873d11a40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009be0 tx=0x7f3d18005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbe1e9c8e9f23d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:55.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.949+0000 7f3d0ffff640 1 -- 172.21.15.5:0/2278074518 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18003f90 con 0x55d873cc4310 2024-09-15T22:21:55.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.949+0000 7f3d0ffff640 1 -- 172.21.15.5:0/2278074518 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d1803c7b0 con 0x55d873cc4310 2024-09-15T22:21:55.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.949+0000 7f3d0ffff640 1 -- 172.21.15.5:0/2278074518 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1802f860 con 0x55d873cc4310 2024-09-15T22:21:55.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.951+0000 7f3d2ee00740 1 -- 172.21.15.5:0/2278074518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cc4310 msgr2=0x55d873d11a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:55.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.951+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/2278074518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cc4310 0x55d873d11a40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009be0 tx=0x7f3d18005c90 comp rx=0 tx=0).stop 2024-09-15T22:21:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.951+0000 7f3d2ee00740 1 -- 172.21.15.5:0/2278074518 shutdown_connections 2024-09-15T22:21:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.951+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/2278074518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d167f0 0x55d873d23440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.951+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/2278074518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873cfcb40 0x55d873d11f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.951+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/2278074518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cc4310 0x55d873d11a40 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3d18009be0 tx=0x7f3d18005c90 comp rx=0 tx=0).stop 2024-09-15T22:21:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.951+0000 7f3d2ee00740 1 -- 172.21.15.5:0/2278074518 >> 172.21.15.5:0/2278074518 conn(0x55d873d16410 msgr2=0x55d873d11330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:55.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.954+0000 7f3d2ee00740 1 -- 172.21.15.5:0/2278074518 shutdown_connections 2024-09-15T22:21:55.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.956+0000 7f3d2ee00740 1 -- 172.21.15.5:0/2278074518 wait complete. 2024-09-15T22:21:55.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.956+0000 7f3d2ee00740 1 -- start start 2024-09-15T22:21:55.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.957+0000 7f3d2ee00740 1 Processor -- start 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.957+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873d1f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.957+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d167f0 0x55d873d1ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.957+0000 7f3d2ee00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d171b0 0x55d873d1b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.957+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281b3fe0 con 0x55d873d171b0 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.957+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281b5420 con 0x55d873d167f0 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.957+0000 7f3d2ee00740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3d281b4680 con 0x55d873cfcb40 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.957+0000 7f3d2ee00740 10 client.0 _ll_register_callbacks cb 0x7f3d18034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.958+0000 7f3d25d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d167f0 0x55d873d1ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.958+0000 7f3d25d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d167f0 0x55d873d1ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:54782/0 (socket says 172.21.15.5:54782) 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.958+0000 7f3d25d9b640 1 -- 172.21.15.5:0/1913126732 learned_addr learned my addr 172.21.15.5:0/1913126732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.958+0000 7f3d2659c640 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873d1f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.958+0000 7f3d26d9d640 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d171b0 0x55d873d1b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.958+0000 7f3d25d9b640 1 -- 172.21.15.5:0/1913126732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 msgr2=0x55d873d1f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.958+0000 7f3d25d9b640 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873d1f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.958+0000 7f3d25d9b640 1 -- 172.21.15.5:0/1913126732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d171b0 msgr2=0x55d873d1b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:21:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.958+0000 7f3d25d9b640 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d171b0 0x55d873d1b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.958+0000 7f3d25d9b640 1 -- 172.21.15.5:0/1913126732 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d873d1b6f0 con 0x55d873d167f0 2024-09-15T22:21:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.958+0000 7f3d25d9b640 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d167f0 0x55d873d1ac70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3d10097fe0 tx=0x7f3d1000bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fec3b46fcb4d75 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:21:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.959+0000 7f3d0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:21:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.959+0000 7f3d0dffb640 1 -- 172.21.15.5:0/1913126732 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d100a4280 con 0x55d873d167f0 2024-09-15T22:21:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.959+0000 7f3d0dffb640 1 -- 172.21.15.5:0/1913126732 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7f3d10099910 con 0x55d873d167f0 2024-09-15T22:21:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.959+0000 7f3d2ee00740 1 -- 172.21.15.5:0/1913126732 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d1803c7b0 con 0x55d873d167f0 2024-09-15T22:21:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.960+0000 7f3d0dffb640 1 -- 172.21.15.5:0/1913126732 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3d100ad050 con 0x55d873d167f0 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.960+0000 7f3d2ee00740 10 client.27131 fetch_fsmap learned FSMap version 25 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.960+0000 7f3d2ee00740 1 -- 172.21.15.5:0/1913126732 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d873d92630 con 0x55d873d167f0 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.960+0000 7f3d0dffb640 1 -- 172.21.15.5:0/1913126732 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d1803c7b0 con 0x55d873d167f0 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.960+0000 7f3d0dffb640 1 -- 172.21.15.5:0/1913126732 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f3d100afb30 con 0x55d873d167f0 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.960+0000 7f3d2ee00740 10 client.27131 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.960+0000 7f3d2ee00740 10 client.27131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.960+0000 7f3d2ee00740 20 client.27131 populate_metadata read hostname 'smithi005' 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.960+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d873d92d60 0x55d873d95180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.960+0000 7f3d2ee00740 4 client.27131 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.960+0000 7f3d2ee00740 1 -- 172.21.15.5:0/1913126732 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d873c68d70 con 0x55d873d92d60 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.961+0000 7f3d2659c640 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d873d92d60 0x55d873d95180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:21:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.961+0000 7f3d0dffb640 10 client.27131 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.961+0000 7f3d2659c640 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d873d92d60 0x55d873d95180 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db16f4cb13ea568c server_cookie=3fddbfa86f2d6504 in_seq=0 out_seq=0 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d0dffb640 1 -- 172.21.15.5:0/1913126732 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d873c68d70 con 0x55d873d92d60 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d0dffb640 10 client.27131 handle_command_reply: tid=0 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d2ee00740 1 client.27131 shutdown 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d2ee00740 1 -- 172.21.15.5:0/1913126732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d167f0 msgr2=0x55d873d1ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d167f0 0x55d873d1ac70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3d10097fe0 tx=0x7f3d1000bf60 comp rx=0 tx=0).stop 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d2ee00740 1 -- 172.21.15.5:0/1913126732 shutdown_connections 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d873d171b0 0x55d873d1b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d873d92d60 0x55d873d95180 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d873d167f0 0x55d873d1ac70 secure :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3d10097fe0 tx=0x7f3d1000bf60 comp rx=0 tx=0).stop 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d2ee00740 1 --2- 172.21.15.5:0/1913126732 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d873cfcb40 0x55d873d1f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d2ee00740 1 -- 172.21.15.5:0/1913126732 >> 172.21.15.5:0/1913126732 conn(0x55d873d16410 msgr2=0x55d873cf6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.962+0000 7f3d0dffb640 0 client.27131 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:21:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.963+0000 7f3d2ee00740 1 -- 172.21.15.5:0/1913126732 shutdown_connections 2024-09-15T22:21:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.963+0000 7f3d2ee00740 1 -- 172.21.15.5:0/1913126732 wait complete. 2024-09-15T22:21:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.963+0000 7f3d2ee00740 20 client.27131 trim_cache size 0 max 16384 2024-09-15T22:21:55.968 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:21:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.964+0000 7f3d2ca27640 1 -- 172.21.15.5:0/3743590487 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d140a3500 msgr2=0x7f3d140a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.964+0000 7f3d2ca27640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d140a3500 0x7f3d140a59c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3d180099a0 tx=0x7f3d18000f40 comp rx=0 tx=0).stop 2024-09-15T22:21:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.964+0000 7f3d2ca27640 1 -- 172.21.15.5:0/3743590487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d28108e00 msgr2=0x7f3d2806dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:21:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.965+0000 7f3d2ca27640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d28108e00 0x7f3d2806dc70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000d950 tx=0x7f3d1000de10 comp rx=0 tx=0).stop 2024-09-15T22:21:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.965+0000 7f3d2ca27640 1 -- 172.21.15.5:0/3743590487 shutdown_connections 2024-09-15T22:21:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.965+0000 7f3d2ca27640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d140a3500 0x7f3d140a59c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.965+0000 7f3d2ca27640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d2810d530 0x7f3d2806e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.965+0000 7f3d2ca27640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d28108e00 0x7f3d2806dc70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.965+0000 7f3d2ca27640 1 --2- 172.21.15.5:0/3743590487 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d28108450 0x7f3d28073bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:21:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.966+0000 7f3d2ca27640 1 -- 172.21.15.5:0/3743590487 >> 172.21.15.5:0/3743590487 conn(0x7f3d28077ba0 msgr2=0x7f3d28078ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:21:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.966+0000 7f3d2ca27640 1 -- 172.21.15.5:0/3743590487 shutdown_connections 2024-09-15T22:21:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:21:55.966+0000 7f3d2ca27640 1 -- 172.21.15.5:0/3743590487 wait complete. 2024-09-15T22:21:57.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:57 smithi099 ceph-mon[34892]: pgmap v406: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 6.9 MiB/s wr, 633 op/s 2024-09-15T22:21:57.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:57 smithi129 ceph-mon[35314]: pgmap v406: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 6.9 MiB/s wr, 633 op/s 2024-09-15T22:21:57.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:57 smithi005 ceph-mon[30944]: pgmap v406: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 6.9 MiB/s wr, 633 op/s 2024-09-15T22:21:59.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:21:59 smithi005 ceph-mon[30944]: pgmap v407: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.2 MiB/s wr, 616 op/s 2024-09-15T22:21:59.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 750.0 s, 484.4 tps, lat 199.787 ms stddev 249.359 2024-09-15T22:21:59.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:21:59 smithi099 ceph-mon[34892]: pgmap v407: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.2 MiB/s wr, 616 op/s 2024-09-15T22:21:59.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:21:59 smithi129 ceph-mon[35314]: pgmap v407: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 7.2 MiB/s wr, 616 op/s 2024-09-15T22:22:00.692 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:22:00.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:22:01.113 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:01.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:01 smithi005 ceph-mon[30944]: pgmap v408: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.1 MiB/s wr, 549 op/s 2024-09-15T22:22:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.735+0000 7f667b59e640 1 -- 172.21.15.5:0/4281552650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667c1079a0 msgr2=0x7f667c107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.735+0000 7f667b59e640 1 --2- 172.21.15.5:0/4281552650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667c1079a0 0x7f667c107d80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f666c009860 tx=0x7f666c02a360 comp rx=0 tx=0).stop 2024-09-15T22:22:01.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.738+0000 7f667b59e640 1 -- 172.21.15.5:0/4281552650 shutdown_connections 2024-09-15T22:22:01.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.738+0000 7f667b59e640 1 --2- 172.21.15.5:0/4281552650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f667c102340 0x7f667c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.738+0000 7f667b59e640 1 --2- 172.21.15.5:0/4281552650 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f667c1019a0 0x7f667c101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.738+0000 7f667b59e640 1 --2- 172.21.15.5:0/4281552650 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667c1079a0 0x7f667c107d80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.738+0000 7f667b59e640 1 -- 172.21.15.5:0/4281552650 >> 172.21.15.5:0/4281552650 conn(0x7f667c0fd680 msgr2=0x7f667c0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.743+0000 7f667b59e640 1 -- 172.21.15.5:0/4281552650 shutdown_connections 2024-09-15T22:22:01.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.743+0000 7f667b59e640 1 -- 172.21.15.5:0/4281552650 wait complete. 2024-09-15T22:22:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.744+0000 7f667b59e640 1 Processor -- start 2024-09-15T22:22:01.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.744+0000 7f667b59e640 1 -- start start 2024-09-15T22:22:01.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.744+0000 7f667b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f667c1019a0 0x7f667c1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.745+0000 7f667b59e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667c102340 0x7f667c116600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.745+0000 7f667a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f667c1019a0 0x7f667c1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.745+0000 7f667a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f667c1019a0 0x7f667c1160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38528/0 (socket says 172.21.15.5:38528) 2024-09-15T22:22:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.745+0000 7f667a59c640 1 -- 172.21.15.5:0/3287764116 learned_addr learned my addr 172.21.15.5:0/3287764116 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.745+0000 7f667b59e640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f667c1079a0 0x7f667c11bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.745+0000 7f667b59e640 1 -- 172.21.15.5:0/3287764116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0760e0 con 0x7f667c1019a0 2024-09-15T22:22:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.745+0000 7f667b59e640 1 -- 172.21.15.5:0/3287764116 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f667c075f60 con 0x7f667c102340 2024-09-15T22:22:01.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.745+0000 7f667b59e640 1 -- 172.21.15.5:0/3287764116 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f667c076260 con 0x7f667c1079a0 2024-09-15T22:22:01.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.746+0000 7f667ad9d640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f667c1079a0 0x7f667c11bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.747+0000 7f6673fff640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667c102340 0x7f667c116600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.747+0000 7f667a59c640 1 -- 172.21.15.5:0/3287764116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f667c1079a0 msgr2=0x7f667c11bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.747+0000 7f667a59c640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f667c1079a0 0x7f667c11bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.747+0000 7f667a59c640 1 -- 172.21.15.5:0/3287764116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667c102340 msgr2=0x7f667c116600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.747+0000 7f667a59c640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667c102340 0x7f667c116600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.747+0000 7f667a59c640 1 -- 172.21.15.5:0/3287764116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c11c6a0 con 0x7f667c1019a0 2024-09-15T22:22:01.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.748+0000 7f667a59c640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f667c1019a0 0x7f667c1160c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f666c035040 tx=0x7f666c030f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:01.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.749+0000 7f66737fe640 1 -- 172.21.15.5:0/3287764116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c03f020 con 0x7f667c1019a0 2024-09-15T22:22:01.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.749+0000 7f66737fe640 1 -- 172.21.15.5:0/3287764116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f666c0305b0 con 0x7f667c1019a0 2024-09-15T22:22:01.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.749+0000 7f66737fe640 1 -- 172.21.15.5:0/3287764116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c037040 con 0x7f667c1019a0 2024-09-15T22:22:01.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.749+0000 7f667b59e640 1 -- 172.21.15.5:0/3287764116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c11c930 con 0x7f667c1019a0 2024-09-15T22:22:01.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.749+0000 7f667b59e640 1 -- 172.21.15.5:0/3287764116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c1113b0 con 0x7f667c1019a0 2024-09-15T22:22:01.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.753+0000 7f6682cfe740 1 Processor -- start 2024-09-15T22:22:01.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.757+0000 7f66737fe640 1 -- 172.21.15.5:0/3287764116 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f666c042020 con 0x7f667c1019a0 2024-09-15T22:22:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.758+0000 7f66737fe640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f66680a3480 0x7f66680a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.759+0000 7f6682cfe740 1 -- start start 2024-09-15T22:22:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.759+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 0x5615ca31d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.759+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca327a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.759+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca327f50 0x5615ca32a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.759+0000 7f6682cfe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5615ca282280 con 0x5615ca31cd70 2024-09-15T22:22:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.759+0000 7f6682cfe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5615ca229b10 con 0x5615ca31d690 2024-09-15T22:22:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.759+0000 7f6682cfe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5615ca27e240 con 0x5615ca327f50 2024-09-15T22:22:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.760+0000 7f6673fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca327a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.760+0000 7f6673fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca327a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:54800/0 (socket says 172.21.15.5:54800) 2024-09-15T22:22:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.760+0000 7f6673fff640 1 -- 172.21.15.5:0/3415437072 learned_addr learned my addr 172.21.15.5:0/3415437072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.760+0000 7f667ad9d640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f66680a3480 0x7f66680a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.760+0000 7f66737fe640 1 -- 172.21.15.5:0/3287764116 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7f666c0b9480 con 0x7f667c1019a0 2024-09-15T22:22:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.761+0000 7f6673fff640 1 -- 172.21.15.5:0/3415437072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca327f50 msgr2=0x5615ca32a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.761+0000 7f667ad9d640 1 --2- 172.21.15.5:0/3415437072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca327f50 0x5615ca32a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.761+0000 7f6673fff640 1 --2- 172.21.15.5:0/3415437072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca327f50 0x5615ca32a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.761+0000 7f6673fff640 1 -- 172.21.15.5:0/3415437072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 msgr2=0x5615ca31d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.761+0000 7f6673fff640 1 --2- 172.21.15.5:0/3415437072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 0x5615ca31d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.761+0000 7f6673fff640 1 -- 172.21.15.5:0/3415437072 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ca2361f0 con 0x5615ca31d690 2024-09-15T22:22:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.761+0000 7f667ad9d640 1 --2- 172.21.15.5:0/3415437072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca327f50 0x5615ca32a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.762+0000 7f6673fff640 1 --2- 172.21.15.5:0/3415437072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca327a10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f666400cb90 tx=0x7f666400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4137c68664b73e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.762+0000 7f6670ff9640 1 -- 172.21.15.5:0/3415437072 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664010630 con 0x5615ca31d690 2024-09-15T22:22:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.762+0000 7f6670ff9640 1 -- 172.21.15.5:0/3415437072 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6664010cb0 con 0x5615ca31d690 2024-09-15T22:22:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.762+0000 7f6670ff9640 1 -- 172.21.15.5:0/3415437072 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666401ac60 con 0x5615ca31d690 2024-09-15T22:22:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.763+0000 7f6682cfe740 1 -- 172.21.15.5:0/3415437072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 msgr2=0x5615ca327a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.763+0000 7f6682cfe740 1 --2- 172.21.15.5:0/3415437072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca327a10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f666400cb90 tx=0x7f666400e720 comp rx=0 tx=0).stop 2024-09-15T22:22:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.763+0000 7f667ad9d640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f66680a3480 0x7f66680a5940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x5615ca31a680 tx=0x7f6674009a00 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.764+0000 7f6682cfe740 1 -- 172.21.15.5:0/3415437072 shutdown_connections 2024-09-15T22:22:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.764+0000 7f6682cfe740 1 --2- 172.21.15.5:0/3415437072 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca327f50 0x5615ca32a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.764+0000 7f6682cfe740 1 --2- 172.21.15.5:0/3415437072 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca327a10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.764+0000 7f6682cfe740 1 --2- 172.21.15.5:0/3415437072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 0x5615ca31d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.764+0000 7f6682cfe740 1 -- 172.21.15.5:0/3415437072 >> 172.21.15.5:0/3415437072 conn(0x5615ca319300 msgr2=0x5615ca3196e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.767+0000 7f6682cfe740 1 -- 172.21.15.5:0/3415437072 shutdown_connections 2024-09-15T22:22:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.767+0000 7f6682cfe740 1 -- 172.21.15.5:0/3415437072 wait complete. 2024-09-15T22:22:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.767+0000 7f6682cfe740 1 -- start start 2024-09-15T22:22:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.767+0000 7f6682cfe740 1 Processor -- start 2024-09-15T22:22:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.768+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 0x5615ca324dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.768+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca31d690 0x5615ca325310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.768+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca327f50 0x5615ca325850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.768+0000 7f6682cfe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5615ca32abc0 con 0x5615ca31cd70 2024-09-15T22:22:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.768+0000 7f6682cfe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5615ca23cf10 con 0x5615ca327f50 2024-09-15T22:22:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.768+0000 7f6682cfe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5615ca32ad40 con 0x5615ca31d690 2024-09-15T22:22:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.768+0000 7f6682cfe740 10 client.0 _ll_register_callbacks cb 0x5615ca205a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.769+0000 7f6673fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca31d690 0x5615ca325310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.769+0000 7f6673fff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca31d690 0x5615ca325310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:38576/0 (socket says 172.21.15.5:38576) 2024-09-15T22:22:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.769+0000 7f6673fff640 1 -- 172.21.15.5:0/339639642 learned_addr learned my addr 172.21.15.5:0/339639642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.769+0000 7f667ad9d640 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca327f50 0x5615ca325850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.770+0000 7f6673fff640 1 -- 172.21.15.5:0/339639642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca327f50 msgr2=0x5615ca325850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.770+0000 7f6673fff640 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca327f50 0x5615ca325850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.770+0000 7f6673fff640 1 -- 172.21.15.5:0/339639642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 msgr2=0x5615ca324dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.770+0000 7f6673fff640 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 0x5615ca324dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.770+0000 7f6673fff640 1 -- 172.21.15.5:0/339639642 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615c9ff36a0 con 0x5615ca31d690 2024-09-15T22:22:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.779+0000 7f6673fff640 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca31d690 0x5615ca325310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f666400cab0 tx=0x7f666400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a9bc375665afa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.783+0000 7f6661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:22:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.783+0000 7f6661ffb640 1 -- 172.21.15.5:0/339639642 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664010e60 con 0x5615ca31d690 2024-09-15T22:22:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.783+0000 7f6682cfe740 1 -- 172.21.15.5:0/339639642 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6664010cb0 con 0x5615ca31d690 2024-09-15T22:22:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.783+0000 7f6661ffb640 1 -- 172.21.15.5:0/339639642 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7f6664004020 con 0x5615ca31d690 2024-09-15T22:22:01.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.790+0000 7f6661ffb640 1 -- 172.21.15.5:0/339639642 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f666400ce90 con 0x5615ca31d690 2024-09-15T22:22:01.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.791+0000 7f6682cfe740 10 client.27064 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:01.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.791+0000 7f6682cfe740 1 -- 172.21.15.5:0/339639642 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5615ca347710 con 0x5615ca31d690 2024-09-15T22:22:01.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.795+0000 7f6661ffb640 1 -- 172.21.15.5:0/339639642 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6664010cb0 con 0x5615ca31d690 2024-09-15T22:22:01.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.795+0000 7f6661ffb640 1 -- 172.21.15.5:0/339639642 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6664003650 con 0x5615ca31d690 2024-09-15T22:22:01.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.795+0000 7f6682cfe740 10 client.27064 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:01.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.795+0000 7f6682cfe740 10 client.27064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:01.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.795+0000 7f6682cfe740 20 client.27064 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:01.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.795+0000 7f6682cfe740 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5615ca348f40 0x5615ca34b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.795+0000 7f6682cfe740 4 client.27064 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:01.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.795+0000 7f6682cfe740 1 -- 172.21.15.5:0/339639642 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615ca28f7f0 con 0x5615ca348f40 2024-09-15T22:22:01.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.800+0000 7f667a59c640 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5615ca348f40 0x5615ca34b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.811+0000 7f667a59c640 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5615ca348f40 0x5615ca34b3e0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3076e1f3d7fde18b server_cookie=ca159ad1f87b967b in_seq=0 out_seq=0 2024-09-15T22:22:01.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.812+0000 7f6661ffb640 10 client.27064 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:01.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.813+0000 7f6661ffb640 1 -- 172.21.15.5:0/339639642 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615ca28f7f0 con 0x5615ca348f40 2024-09-15T22:22:01.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.813+0000 7f6661ffb640 10 client.27064 handle_command_reply: tid=0 2024-09-15T22:22:01.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.813+0000 7f6682cfe740 1 client.27064 shutdown 2024-09-15T22:22:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.818+0000 7f6682cfe740 1 -- 172.21.15.5:0/339639642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca31d690 msgr2=0x5615ca325310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.818+0000 7f6682cfe740 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca31d690 0x5615ca325310 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f666400cab0 tx=0x7f666400c8d0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.822+0000 7f6682cfe740 1 -- 172.21.15.5:0/339639642 shutdown_connections 2024-09-15T22:22:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.822+0000 7f6682cfe740 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5615ca348f40 0x5615ca34b3e0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.822+0000 7f6682cfe740 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca327f50 0x5615ca325850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.822+0000 7f6682cfe740 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca31d690 0x5615ca325310 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.822+0000 7f6682cfe740 1 --2- 172.21.15.5:0/339639642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 0x5615ca324dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.822+0000 7f6661ffb640 0 client.27064 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.822+0000 7f6682cfe740 1 -- 172.21.15.5:0/339639642 >> 172.21.15.5:0/339639642 conn(0x5615ca319300 msgr2=0x5615ca31f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.826+0000 7f6682cfe740 1 -- 172.21.15.5:0/339639642 shutdown_connections 2024-09-15T22:22:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.827+0000 7f6682cfe740 1 -- 172.21.15.5:0/339639642 wait complete. 2024-09-15T22:22:01.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.829+0000 7f6682cfe740 20 client.27064 trim_cache size 0 max 16384 2024-09-15T22:22:01.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:01 smithi099 ceph-mon[34892]: pgmap v408: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.1 MiB/s wr, 549 op/s 2024-09-15T22:22:01.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:01 smithi129 ceph-mon[35314]: pgmap v408: 129 pgs: 129 active+clean; 3.0 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 6.1 MiB/s wr, 549 op/s 2024-09-15T22:22:01.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.867+0000 7f6682cfe740 1 Processor -- start 2024-09-15T22:22:01.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.867+0000 7f6682cfe740 1 -- start start 2024-09-15T22:22:01.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.868+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 0x5615ca31f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.868+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca350cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.868+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca327f50 0x5615ca351230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.868+0000 7f6673fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca350cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.868+0000 7f6673fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca350cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:54824/0 (socket says 172.21.15.5:54824) 2024-09-15T22:22:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.868+0000 7f6673fff640 1 -- 172.21.15.5:0/3690372669 learned_addr learned my addr 172.21.15.5:0/3690372669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:01.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.869+0000 7f6682cfe740 1 -- 172.21.15.5:0/3690372669 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f667c1b80b0 con 0x5615ca31cd70 2024-09-15T22:22:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.869+0000 7f6682cfe740 1 -- 172.21.15.5:0/3690372669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f667c1b5980 con 0x5615ca31d690 2024-09-15T22:22:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.869+0000 7f6682cfe740 1 -- 172.21.15.5:0/3690372669 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f667c112710 con 0x5615ca327f50 2024-09-15T22:22:01.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.870+0000 7f6673fff640 1 -- 172.21.15.5:0/3690372669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca327f50 msgr2=0x5615ca351230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:22:01.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.870+0000 7f6673fff640 1 --2- 172.21.15.5:0/3690372669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca327f50 0x5615ca351230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.870+0000 7f6673fff640 1 -- 172.21.15.5:0/3690372669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 msgr2=0x5615ca31f820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:22:01.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.870+0000 7f6673fff640 1 --2- 172.21.15.5:0/3690372669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 0x5615ca31f820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.870+0000 7f6673fff640 1 -- 172.21.15.5:0/3690372669 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ca31e280 con 0x5615ca31d690 2024-09-15T22:22:01.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.882+0000 7f6673fff640 1 --2- 172.21.15.5:0/3690372669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca350cf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6664013040 tx=0x7f6664001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15ccda4ba0ec1456 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:01.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.883+0000 7f6663fff640 1 -- 172.21.15.5:0/3690372669 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6664005c20 con 0x5615ca31d690 2024-09-15T22:22:01.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.883+0000 7f6663fff640 1 -- 172.21.15.5:0/3690372669 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f666401acf0 con 0x5615ca31d690 2024-09-15T22:22:01.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.883+0000 7f6663fff640 1 -- 172.21.15.5:0/3690372669 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666401a670 con 0x5615ca31d690 2024-09-15T22:22:01.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.884+0000 7f6682cfe740 1 -- 172.21.15.5:0/3690372669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 msgr2=0x5615ca350cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:01.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.884+0000 7f6682cfe740 1 --2- 172.21.15.5:0/3690372669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca350cf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6664013040 tx=0x7f6664001e70 comp rx=0 tx=0).stop 2024-09-15T22:22:01.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.884+0000 7f6682cfe740 1 -- 172.21.15.5:0/3690372669 shutdown_connections 2024-09-15T22:22:01.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.884+0000 7f6682cfe740 1 --2- 172.21.15.5:0/3690372669 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca327f50 0x5615ca351230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.884+0000 7f6682cfe740 1 --2- 172.21.15.5:0/3690372669 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31d690 0x5615ca350cf0 secure :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6664013040 tx=0x7f6664001e70 comp rx=0 tx=0).stop 2024-09-15T22:22:01.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.884+0000 7f6682cfe740 1 --2- 172.21.15.5:0/3690372669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca31cd70 0x5615ca31f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.884+0000 7f6682cfe740 1 -- 172.21.15.5:0/3690372669 >> 172.21.15.5:0/3690372669 conn(0x5615ca319300 msgr2=0x5615ca323ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:01.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.885+0000 7f6682cfe740 1 -- 172.21.15.5:0/3690372669 shutdown_connections 2024-09-15T22:22:01.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.886+0000 7f6682cfe740 1 -- 172.21.15.5:0/3690372669 wait complete. 2024-09-15T22:22:01.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.887+0000 7f6682cfe740 1 -- start start 2024-09-15T22:22:01.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.887+0000 7f6682cfe740 1 Processor -- start 2024-09-15T22:22:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.888+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31cd70 0x5615ca330620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.888+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca327f50 0x5615ca3219b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.888+0000 7f6682cfe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca321ef0 0x5615ca322370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.888+0000 7f6682cfe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f667c073900 con 0x5615ca327f50 2024-09-15T22:22:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.888+0000 7f6682cfe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f667c1b52b0 con 0x5615ca31cd70 2024-09-15T22:22:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.888+0000 7f6682cfe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f667c073fd0 con 0x5615ca321ef0 2024-09-15T22:22:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.888+0000 7f6682cfe740 10 client.0 _ll_register_callbacks cb 0x7f6664014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.888+0000 7f667a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31cd70 0x5615ca330620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.888+0000 7f667a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31cd70 0x5615ca330620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:54840/0 (socket says 172.21.15.5:54840) 2024-09-15T22:22:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.888+0000 7f667a59c640 1 -- 172.21.15.5:0/98478826 learned_addr learned my addr 172.21.15.5:0/98478826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.889+0000 7f6673fff640 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca327f50 0x5615ca3219b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.889+0000 7f667ad9d640 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca321ef0 0x5615ca322370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.890+0000 7f667a59c640 1 -- 172.21.15.5:0/98478826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca321ef0 msgr2=0x5615ca322370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.890+0000 7f667a59c640 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca321ef0 0x5615ca322370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.890+0000 7f667a59c640 1 -- 172.21.15.5:0/98478826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca327f50 msgr2=0x5615ca3219b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.890+0000 7f667a59c640 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca327f50 0x5615ca3219b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.890+0000 7f667a59c640 1 -- 172.21.15.5:0/98478826 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ca3b2d10 con 0x5615ca31cd70 2024-09-15T22:22:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.890+0000 7f667a59c640 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31cd70 0x5615ca330620 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f666c0b7840 tx=0x7f666c0b7040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ba5c33d10d3671d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.890+0000 7f6661ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:22:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.891+0000 7f6661ffb640 1 -- 172.21.15.5:0/98478826 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c0b7e10 con 0x5615ca31cd70 2024-09-15T22:22:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.891+0000 7f6682cfe740 1 -- 172.21.15.5:0/98478826 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f666401acf0 con 0x5615ca31cd70 2024-09-15T22:22:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.891+0000 7f6661ffb640 1 -- 172.21.15.5:0/98478826 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7f666c0b74b0 con 0x5615ca31cd70 2024-09-15T22:22:01.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.891+0000 7f6661ffb640 1 -- 172.21.15.5:0/98478826 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f666c0b7fb0 con 0x5615ca31cd70 2024-09-15T22:22:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.892+0000 7f6682cfe740 10 client.27149 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:01.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.892+0000 7f6682cfe740 1 -- 172.21.15.5:0/98478826 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5615ca3b3ff0 con 0x5615ca31cd70 2024-09-15T22:22:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.894+0000 7f6661ffb640 1 -- 172.21.15.5:0/98478826 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f666401acf0 con 0x5615ca31cd70 2024-09-15T22:22:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.894+0000 7f6661ffb640 1 -- 172.21.15.5:0/98478826 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f666c03a030 con 0x5615ca31cd70 2024-09-15T22:22:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.894+0000 7f6682cfe740 10 client.27149 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.894+0000 7f6682cfe740 10 client.27149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.894+0000 7f6682cfe740 20 client.27149 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.894+0000 7f6682cfe740 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5615ca3b4760 0x5615ca3b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.894+0000 7f6682cfe740 4 client.27149 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:22:01.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.894+0000 7f6682cfe740 1 -- 172.21.15.5:0/98478826 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5615ca289d00 con 0x5615ca3b4760 2024-09-15T22:22:01.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.894+0000 7f6673fff640 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5615ca3b4760 0x5615ca3b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:01.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.895+0000 7f6661ffb640 10 client.27149 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:01.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.896+0000 7f6673fff640 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5615ca3b4760 0x5615ca3b6b80 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa1d066da6b1f87d server_cookie=b8bbe4230c10a5cc in_seq=0 out_seq=0 2024-09-15T22:22:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.898+0000 7f6661ffb640 1 -- 172.21.15.5:0/98478826 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5615ca289d00 con 0x5615ca3b4760 2024-09-15T22:22:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.898+0000 7f6661ffb640 10 client.27149 handle_command_reply: tid=0 2024-09-15T22:22:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.898+0000 7f6682cfe740 1 client.27149 shutdown 2024-09-15T22:22:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.899+0000 7f6682cfe740 1 -- 172.21.15.5:0/98478826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31cd70 msgr2=0x5615ca330620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:01.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.899+0000 7f6682cfe740 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31cd70 0x5615ca330620 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f666c0b7840 tx=0x7f666c0b7040 comp rx=0 tx=0).stop 2024-09-15T22:22:01.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.905+0000 7f6682cfe740 1 -- 172.21.15.5:0/98478826 shutdown_connections 2024-09-15T22:22:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.905+0000 7f6682cfe740 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5615ca321ef0 0x5615ca322370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.905+0000 7f6682cfe740 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615ca327f50 0x5615ca3219b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.905+0000 7f6682cfe740 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5615ca3b4760 0x5615ca3b6b80 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.905+0000 7f6682cfe740 1 --2- 172.21.15.5:0/98478826 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5615ca31cd70 0x5615ca330620 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.905+0000 7f6682cfe740 1 -- 172.21.15.5:0/98478826 >> 172.21.15.5:0/98478826 conn(0x5615ca319300 msgr2=0x5615ca317bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:01.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.905+0000 7f6661ffb640 0 client.27149 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.906+0000 7f6682cfe740 1 -- 172.21.15.5:0/98478826 shutdown_connections 2024-09-15T22:22:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.906+0000 7f6682cfe740 1 -- 172.21.15.5:0/98478826 wait complete. 2024-09-15T22:22:01.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.906+0000 7f6682cfe740 20 client.27149 trim_cache size 0 max 16384 2024-09-15T22:22:01.909 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:22:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:22:01.909 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "a56324ee-60da-4c6e-8844-b9baa700aeb4", 2024-09-15T22:22:01.910 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:22:01.910 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:22:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.907+0000 7f667b59e640 1 -- 172.21.15.5:0/3287764116 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f66680a3480 msgr2=0x7f66680a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.907+0000 7f667b59e640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f66680a3480 0x7f66680a5940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x5615ca31a680 tx=0x7f6674009a00 comp rx=0 tx=0).stop 2024-09-15T22:22:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.907+0000 7f667b59e640 1 -- 172.21.15.5:0/3287764116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f667c1019a0 msgr2=0x7f667c1160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:01.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.907+0000 7f667b59e640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f667c1019a0 0x7f667c1160c0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f666c035040 tx=0x7f666c030f50 comp rx=0 tx=0).stop 2024-09-15T22:22:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.908+0000 7f667b59e640 1 -- 172.21.15.5:0/3287764116 shutdown_connections 2024-09-15T22:22:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.908+0000 7f667b59e640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f66680a3480 0x7f66680a5940 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.908+0000 7f667b59e640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f667c1079a0 0x7f667c11bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.908+0000 7f667b59e640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f667c102340 0x7f667c116600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.908+0000 7f667b59e640 1 --2- 172.21.15.5:0/3287764116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f667c1019a0 0x7f667c1160c0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:01.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.908+0000 7f667b59e640 1 -- 172.21.15.5:0/3287764116 >> 172.21.15.5:0/3287764116 conn(0x7f667c0fd680 msgr2=0x7f667c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.911+0000 7f667b59e640 1 -- 172.21.15.5:0/3287764116 shutdown_connections 2024-09-15T22:22:01.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:01.911+0000 7f667b59e640 1 -- 172.21.15.5:0/3287764116 wait complete. 2024-09-15T22:22:02.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:02 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:22:02.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:02 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:02.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:02 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:22:02.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:22:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:02 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:22:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:02 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:02.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:02 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:22:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:02 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:22:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:02 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:02.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:02 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:22:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:03.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:03 smithi005 ceph-mon[30944]: pgmap v409: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 7.7 MiB/s wr, 683 op/s 2024-09-15T22:22:03.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:03 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:03.741 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:03 smithi099 ceph-mon[34892]: pgmap v409: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 7.7 MiB/s wr, 683 op/s 2024-09-15T22:22:03.741 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:03 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:03.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:03 smithi129 ceph-mon[35314]: pgmap v409: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 7.7 MiB/s wr, 683 op/s 2024-09-15T22:22:03.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:03 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:22:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.931+0000 7f5fb6f9a640 1 -- 172.21.15.5:0/174999451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb0109830 msgr2=0x7f5fb0109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.931+0000 7f5fb6f9a640 1 --2- 172.21.15.5:0/174999451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb0109830 0x7f5fb0109c90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4009860 tx=0x7f5fa402a380 comp rx=0 tx=0).stop 2024-09-15T22:22:03.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.941+0000 7f5fb6f9a640 1 -- 172.21.15.5:0/174999451 shutdown_connections 2024-09-15T22:22:03.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.941+0000 7f5fb6f9a640 1 --2- 172.21.15.5:0/174999451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fb011b140 0x7f5fb011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.941+0000 7f5fb6f9a640 1 --2- 172.21.15.5:0/174999451 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb0109830 0x7f5fb0109c90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.941+0000 7f5fb6f9a640 1 --2- 172.21.15.5:0/174999451 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fb0109450 0x7f5fb0110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.941+0000 7f5fb6f9a640 1 -- 172.21.15.5:0/174999451 >> 172.21.15.5:0/174999451 conn(0x7f5fb006e140 msgr2=0x7f5fb006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.947+0000 7f5fb6f9a640 1 -- 172.21.15.5:0/174999451 shutdown_connections 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.947+0000 7f5fb6f9a640 1 -- 172.21.15.5:0/174999451 wait complete. 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.948+0000 7f5fb6f9a640 1 Processor -- start 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.948+0000 7f5fb6f9a640 1 -- start start 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.948+0000 7f5fb6f9a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb0109450 0x7f5fb0118700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.948+0000 7f5fb6f9a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fb0109830 0x7f5fb0111790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.948+0000 7f5fb6f9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fb011b140 0x7f5fb0111cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.948+0000 7f5fb6f9a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0120e10 con 0x7f5fb011b140 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.948+0000 7f5fb6f9a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0120c90 con 0x7f5fb0109450 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.948+0000 7f5fb6f9a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0120f90 con 0x7f5fb0109830 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.949+0000 7f5fb4d36640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb0109450 0x7f5fb0118700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:03.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.949+0000 7f5fb4d36640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb0109450 0x7f5fb0118700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:43510/0 (socket says 172.21.15.5:43510) 2024-09-15T22:22:03.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.949+0000 7f5fb4d36640 1 -- 172.21.15.5:0/2993764197 learned_addr learned my addr 172.21.15.5:0/2993764197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.953+0000 7f5fb4d36640 1 -- 172.21.15.5:0/2993764197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fb0109830 msgr2=0x7f5fb0111790 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:22:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.953+0000 7f5fb4d36640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fb0109830 0x7f5fb0111790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.953+0000 7f5fb4d36640 1 -- 172.21.15.5:0/2993764197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fb011b140 msgr2=0x7f5fb0111cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.953+0000 7f5fb4d36640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fb011b140 0x7f5fb0111cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.953+0000 7f5fb4d36640 1 -- 172.21.15.5:0/2993764197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb0112290 con 0x7f5fb0109450 2024-09-15T22:22:03.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.954+0000 7f5fb4d36640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb0109450 0x7f5fb0118700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000e9c0 tx=0x7f5fa000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:03.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.955+0000 7f5fadffb640 1 -- 172.21.15.5:0/2993764197 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa000ccc0 con 0x7f5fb0109450 2024-09-15T22:22:03.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.955+0000 7f5fadffb640 1 -- 172.21.15.5:0/2993764197 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa0005500 con 0x7f5fb0109450 2024-09-15T22:22:03.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.955+0000 7f5fadffb640 1 -- 172.21.15.5:0/2993764197 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0010690 con 0x7f5fb0109450 2024-09-15T22:22:03.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.957+0000 7f5fb6f9a640 1 -- 172.21.15.5:0/2993764197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb0112580 con 0x7f5fb0109450 2024-09-15T22:22:03.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.957+0000 7f5fb6f9a640 1 -- 172.21.15.5:0/2993764197 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb0116550 con 0x7f5fb0109450 2024-09-15T22:22:03.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.961+0000 7f5fb9373740 1 Processor -- start 2024-09-15T22:22:03.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.961+0000 7f5fb9373740 1 -- start start 2024-09-15T22:22:03.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.961+0000 7f5fb9373740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed255790 0x5573ed255b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:03.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.961+0000 7f5fb9373740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed2560b0 0x5573ed260430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:03.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.961+0000 7f5fb9373740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed262d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:03.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.961+0000 7f5fb9373740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573ed1bb2e0 con 0x5573ed260970 2024-09-15T22:22:03.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.961+0000 7f5fb9373740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5573ed162ce0 con 0x5573ed255790 2024-09-15T22:22:03.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.961+0000 7f5fb9373740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5573ed1b72a0 con 0x5573ed2560b0 2024-09-15T22:22:03.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.962+0000 7f5fb5537640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed255790 0x5573ed255b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:03.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.962+0000 7f5fb5537640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed255790 0x5573ed255b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:43524/0 (socket says 172.21.15.5:43524) 2024-09-15T22:22:03.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.962+0000 7f5fb5537640 1 -- 172.21.15.5:0/1919460165 learned_addr learned my addr 172.21.15.5:0/1919460165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:03.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.962+0000 7f5fb5537640 1 -- 172.21.15.5:0/1919460165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed2560b0 msgr2=0x5573ed260430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:22:03.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.962+0000 7f5fb5537640 1 --2- 172.21.15.5:0/1919460165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed2560b0 0x5573ed260430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.962+0000 7f5fb5537640 1 -- 172.21.15.5:0/1919460165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 msgr2=0x5573ed262d60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:22:03.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.962+0000 7f5fb5537640 1 --2- 172.21.15.5:0/1919460165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed262d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.962+0000 7f5fb5537640 1 -- 172.21.15.5:0/1919460165 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ed147440 con 0x5573ed255790 2024-09-15T22:22:03.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.963+0000 7f5fb5537640 1 --2- 172.21.15.5:0/1919460165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed255790 0x5573ed255b70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8002000 tx=0x7f5fa800c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bbcf2b454d23c6f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:03.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.965+0000 7f5fadffb640 1 -- 172.21.15.5:0/2993764197 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5fa00046c0 con 0x7f5fb0109450 2024-09-15T22:22:03.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.966+0000 7f5fadffb640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5f980a3540 0x7f5f980a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:03.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.967+0000 7f5f967fc640 1 -- 172.21.15.5:0/1919460165 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa800cdd0 con 0x5573ed255790 2024-09-15T22:22:03.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.967+0000 7f5f967fc640 1 -- 172.21.15.5:0/1919460165 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa8003e50 con 0x5573ed255790 2024-09-15T22:22:03.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.967+0000 7f5f967fc640 1 -- 172.21.15.5:0/1919460165 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8012680 con 0x5573ed255790 2024-09-15T22:22:03.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.968+0000 7f5fadffb640 1 -- 172.21.15.5:0/2993764197 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7f5fa009a3b0 con 0x7f5fb0109450 2024-09-15T22:22:03.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.968+0000 7f5fb9373740 1 -- 172.21.15.5:0/1919460165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed255790 msgr2=0x5573ed255b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:03.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.968+0000 7f5fb9373740 1 --2- 172.21.15.5:0/1919460165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed255790 0x5573ed255b70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8002000 tx=0x7f5fa800c4b0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.970+0000 7f5fb4d36640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5f980a3540 0x7f5f980a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:03.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.974+0000 7f5fb9373740 1 -- 172.21.15.5:0/1919460165 shutdown_connections 2024-09-15T22:22:03.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.974+0000 7f5fb9373740 1 --2- 172.21.15.5:0/1919460165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed262d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.974+0000 7f5fb9373740 1 --2- 172.21.15.5:0/1919460165 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed2560b0 0x5573ed260430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.974+0000 7f5fb9373740 1 --2- 172.21.15.5:0/1919460165 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed255790 0x5573ed255b70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.974+0000 7f5fb9373740 1 -- 172.21.15.5:0/1919460165 >> 172.21.15.5:0/1919460165 conn(0x5573ed251d20 msgr2=0x5573ed252100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:03.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.977+0000 7f5fb4d36640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5f980a3540 0x7f5f980a5a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0096de0 tx=0x7f5fa00a08b0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:03.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.984+0000 7f5fb9373740 1 -- 172.21.15.5:0/1919460165 shutdown_connections 2024-09-15T22:22:03.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.984+0000 7f5fb9373740 1 -- 172.21.15.5:0/1919460165 wait complete. 2024-09-15T22:22:03.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.984+0000 7f5fb9373740 1 -- start start 2024-09-15T22:22:03.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.984+0000 7f5fb9373740 1 Processor -- start 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.985+0000 7f5fb9373740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed260070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.985+0000 7f5fb9373740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed25d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.985+0000 7f5fb9373740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed25d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.985+0000 7f5fb9373740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573ed2634b0 con 0x5573ed260970 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.985+0000 7f5fb9373740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5573ed176050 con 0x5573ed2560b0 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.985+0000 7f5fb9373740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5573ed263630 con 0x5573ed255790 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.985+0000 7f5fb9373740 10 client.0 _ll_register_callbacks cb 0x5573ed1b6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.986+0000 7f5fb5537640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed260070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.986+0000 7f5fb5537640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed260070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:48460/0 (socket says 172.21.15.5:48460) 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.986+0000 7f5fb5537640 1 -- 172.21.15.5:0/3992823701 learned_addr learned my addr 172.21.15.5:0/3992823701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.986+0000 7f5faffff640 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed25d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:03.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.986+0000 7f5fb5537640 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed25d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:03.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.990+0000 7f5fb5537640 1 -- 172.21.15.5:0/3992823701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 msgr2=0x5573ed25d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.990+0000 7f5fb5537640 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed25d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.990+0000 7f5fb5537640 1 -- 172.21.15.5:0/3992823701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 msgr2=0x5573ed25d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.990+0000 7f5fb5537640 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed25d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.990+0000 7f5fb5537640 1 -- 172.21.15.5:0/3992823701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573ecf2c3c0 con 0x5573ed255790 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.990+0000 7f5fb5537640 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed260070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8001d90 tx=0x7f5fa8009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2e8f0e8cab6e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.990+0000 7f5f8ffff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.990+0000 7f5f8ffff640 1 -- 172.21.15.5:0/3992823701 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8025070 con 0x5573ed255790 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.991+0000 7f5fb9373740 1 -- 172.21.15.5:0/3992823701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa8003e50 con 0x5573ed255790 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.991+0000 7f5f8ffff640 1 -- 172.21.15.5:0/3992823701 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7f5fa800ce40 con 0x5573ed255790 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.991+0000 7f5f8ffff640 1 -- 172.21.15.5:0/3992823701 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5fa802e050 con 0x5573ed255790 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.991+0000 7f5fb9373740 10 client.27073 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:03.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.991+0000 7f5fb9373740 1 -- 172.21.15.5:0/3992823701 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5573ed26ec10 con 0x5573ed255790 2024-09-15T22:22:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.992+0000 7f5f8ffff640 1 -- 172.21.15.5:0/3992823701 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa8003e50 con 0x5573ed255790 2024-09-15T22:22:03.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.992+0000 7f5f8ffff640 1 -- 172.21.15.5:0/3992823701 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5fa8028020 con 0x5573ed255790 2024-09-15T22:22:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.992+0000 7f5fb9373740 10 client.27073 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.992+0000 7f5fb9373740 10 client.27073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.992+0000 7f5fb9373740 20 client.27073 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.992+0000 7f5fb9373740 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573ed2704a0 0x5573ed2728d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.992+0000 7f5fb9373740 4 client.27073 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.992+0000 7f5fb9373740 1 -- 172.21.15.5:0/3992823701 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573ed1c8850 con 0x5573ed2704a0 2024-09-15T22:22:03.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.993+0000 7f5faffff640 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573ed2704a0 0x5573ed2728d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.998+0000 7f5f8ffff640 10 client.27073 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:04.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:03.998+0000 7f5faffff640 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573ed2704a0 0x5573ed2728d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f9baca0cc61b28 server_cookie=49644ec68b4faac2 in_seq=0 out_seq=0 2024-09-15T22:22:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.003+0000 7f5f8ffff640 1 -- 172.21.15.5:0/3992823701 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573ed1c8850 con 0x5573ed2704a0 2024-09-15T22:22:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.003+0000 7f5f8ffff640 10 client.27073 handle_command_reply: tid=0 2024-09-15T22:22:04.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.003+0000 7f5fb9373740 1 client.27073 shutdown 2024-09-15T22:22:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.003+0000 7f5fb9373740 1 -- 172.21.15.5:0/3992823701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 msgr2=0x5573ed260070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.003+0000 7f5fb9373740 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed260070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8001d90 tx=0x7f5fa8009040 comp rx=0 tx=0).stop 2024-09-15T22:22:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.003+0000 7f5fb9373740 1 -- 172.21.15.5:0/3992823701 shutdown_connections 2024-09-15T22:22:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.003+0000 7f5fb9373740 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed25d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.003+0000 7f5fb9373740 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573ed2704a0 0x5573ed2728d0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.003+0000 7f5fb9373740 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed25d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.003+0000 7f5fb9373740 1 --2- 172.21.15.5:0/3992823701 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed260070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.003+0000 7f5fb9373740 1 -- 172.21.15.5:0/3992823701 >> 172.21.15.5:0/3992823701 conn(0x5573ed251d20 msgr2=0x5573ed250ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:04.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.004+0000 7f5f8ffff640 0 client.27073 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:04.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.005+0000 7f5fb9373740 1 -- 172.21.15.5:0/3992823701 shutdown_connections 2024-09-15T22:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.005+0000 7f5fb9373740 1 -- 172.21.15.5:0/3992823701 wait complete. 2024-09-15T22:22:04.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.006+0000 7f5fb9373740 20 client.27073 trim_cache size 0 max 16384 2024-09-15T22:22:04.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.040+0000 7f5fb9373740 1 Processor -- start 2024-09-15T22:22:04.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.040+0000 7f5fb9373740 1 -- start start 2024-09-15T22:22:04.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.040+0000 7f5fb9373740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed2781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.040+0000 7f5fb9373740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed2786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:04.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.041+0000 7f5fb5537640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed2781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.041+0000 7f5fb5537640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed2781a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:48470/0 (socket says 172.21.15.5:48470) 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.041+0000 7f5fb5537640 1 -- 172.21.15.5:0/1563347024 learned_addr learned my addr 172.21.15.5:0/1563347024 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.041+0000 7f5fb9373740 1 --2- 172.21.15.5:0/1563347024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed268870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.041+0000 7f5fb9373740 1 -- 172.21.15.5:0/1563347024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01c91e0 con 0x5573ed260970 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.041+0000 7f5faffff640 1 --2- 172.21.15.5:0/1563347024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed2786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.041+0000 7f5fb9373740 1 -- 172.21.15.5:0/1563347024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01c6ab0 con 0x5573ed2560b0 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.041+0000 7f5fb9373740 1 -- 172.21.15.5:0/1563347024 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01c4920 con 0x5573ed255790 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.041+0000 7f5fb5537640 1 --2- 172.21.15.5:0/1563347024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed268870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.042+0000 7f5faffff640 1 -- 172.21.15.5:0/1563347024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 msgr2=0x5573ed2781a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.042+0000 7f5faffff640 1 --2- 172.21.15.5:0/1563347024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed2781a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.042+0000 7f5faffff640 1 -- 172.21.15.5:0/1563347024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 msgr2=0x5573ed268870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.042+0000 7f5faffff640 1 --2- 172.21.15.5:0/1563347024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed268870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.042+0000 7f5faffff640 1 -- 172.21.15.5:0/1563347024 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ed25d9c0 con 0x5573ed2560b0 2024-09-15T22:22:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.042+0000 7f5faffff640 1 --2- 172.21.15.5:0/1563347024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed2786e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5573ed2530a0 tx=0x7f5fa4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=159e88405f8cbbf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.043+0000 7f5f95ffb640 1 -- 172.21.15.5:0/1563347024 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa4037040 con 0x5573ed2560b0 2024-09-15T22:22:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.043+0000 7f5f95ffb640 1 -- 172.21.15.5:0/1563347024 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa4042030 con 0x5573ed2560b0 2024-09-15T22:22:04.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.043+0000 7f5f95ffb640 1 -- 172.21.15.5:0/1563347024 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa40052a0 con 0x5573ed2560b0 2024-09-15T22:22:04.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.047+0000 7f5fb9373740 1 -- 172.21.15.5:0/1563347024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 msgr2=0x5573ed2786e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:04.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.047+0000 7f5fb9373740 1 --2- 172.21.15.5:0/1563347024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed2786e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5573ed2530a0 tx=0x7f5fa4005f90 comp rx=0 tx=0).stop 2024-09-15T22:22:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.051+0000 7f5fb9373740 1 -- 172.21.15.5:0/1563347024 shutdown_connections 2024-09-15T22:22:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.051+0000 7f5fb9373740 1 --2- 172.21.15.5:0/1563347024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed268870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.051+0000 7f5fb9373740 1 --2- 172.21.15.5:0/1563347024 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed2786e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.051+0000 7f5fb9373740 1 --2- 172.21.15.5:0/1563347024 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed2781a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.051+0000 7f5fb9373740 1 -- 172.21.15.5:0/1563347024 >> 172.21.15.5:0/1563347024 conn(0x5573ed251d20 msgr2=0x5573ed25c410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:04.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.055+0000 7f5fb9373740 1 -- 172.21.15.5:0/1563347024 shutdown_connections 2024-09-15T22:22:04.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.059+0000 7f5fb9373740 1 -- 172.21.15.5:0/1563347024 wait complete. 2024-09-15T22:22:04.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.059+0000 7f5fb9373740 1 -- start start 2024-09-15T22:22:04.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.059+0000 7f5fb9373740 1 Processor -- start 2024-09-15T22:22:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.060+0000 7f5fb9373740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed265680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.060+0000 7f5fb9373740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed265bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.060+0000 7f5fb9373740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed266100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.060+0000 7f5fb9373740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01c4f70 con 0x5573ed260970 2024-09-15T22:22:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.060+0000 7f5fb9373740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01c63e0 con 0x5573ed2560b0 2024-09-15T22:22:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.060+0000 7f5fb9373740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb01c5640 con 0x5573ed255790 2024-09-15T22:22:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.060+0000 7f5fb9373740 10 client.0 _ll_register_callbacks cb 0x7f5fa8001e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.060+0000 7f5faffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed265bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.060+0000 7f5faffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed265bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:43540/0 (socket says 172.21.15.5:43540) 2024-09-15T22:22:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.060+0000 7f5faffff640 1 -- 172.21.15.5:0/940783628 learned_addr learned my addr 172.21.15.5:0/940783628 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.062+0000 7f5fb5537640 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed265680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.062+0000 7f5faffff640 1 -- 172.21.15.5:0/940783628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 msgr2=0x5573ed265680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.062+0000 7f5faffff640 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed265680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.062+0000 7f5faffff640 1 -- 172.21.15.5:0/940783628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 msgr2=0x5573ed266100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.062+0000 7f5faffff640 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed266100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.062+0000 7f5faffff640 1 -- 172.21.15.5:0/940783628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573ed223de0 con 0x5573ed2560b0 2024-09-15T22:22:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.062+0000 7f5fb5537640 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed265680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.063+0000 7f5faffff640 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed265bc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4035040 tx=0x7f5fa4005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ce7ac554829aba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.063+0000 7f5f8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:22:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.066+0000 7f5f8f7fe640 1 -- 172.21.15.5:0/940783628 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa404e020 con 0x5573ed2560b0 2024-09-15T22:22:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.066+0000 7f5fb9373740 1 -- 172.21.15.5:0/940783628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fa4042030 con 0x5573ed2560b0 2024-09-15T22:22:04.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.066+0000 7f5f8f7fe640 1 -- 172.21.15.5:0/940783628 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9541+0+0 (secure 0 0 0) 0x7f5fa4030cb0 con 0x5573ed2560b0 2024-09-15T22:22:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.067+0000 7f5f8f7fe640 1 -- 172.21.15.5:0/940783628 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5fa4059050 con 0x5573ed2560b0 2024-09-15T22:22:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.067+0000 7f5fb9373740 10 client.27167 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:04.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.067+0000 7f5fb9373740 1 -- 172.21.15.5:0/940783628 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5573ed2da320 con 0x5573ed2560b0 2024-09-15T22:22:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.069+0000 7f5f8f7fe640 1 -- 172.21.15.5:0/940783628 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fa4042030 con 0x5573ed2560b0 2024-09-15T22:22:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.069+0000 7f5f8f7fe640 1 -- 172.21.15.5:0/940783628 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5fa40046c0 con 0x5573ed2560b0 2024-09-15T22:22:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.070+0000 7f5fb9373740 10 client.27167 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.070+0000 7f5fb9373740 10 client.27167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.070+0000 7f5fb9373740 20 client.27167 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.070+0000 7f5fb9373740 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573ed2da9f0 0x5573ed2dce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.070+0000 7f5fb9373740 4 client.27167 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:22:04.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.070+0000 7f5fb9373740 1 -- 172.21.15.5:0/940783628 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5573ed1c2d60 con 0x5573ed2da9f0 2024-09-15T22:22:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.070+0000 7f5fb5537640 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573ed2da9f0 0x5573ed2dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.088+0000 7f5fb5537640 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573ed2da9f0 0x5573ed2dce10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e5aa3d5a824eb server_cookie=36bba0c85583cee6 in_seq=0 out_seq=0 2024-09-15T22:22:04.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.088+0000 7f5f8f7fe640 10 client.27167 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.091+0000 7f5f8f7fe640 1 -- 172.21.15.5:0/940783628 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f5fa800c9b0 con 0x5573ed2da9f0 2024-09-15T22:22:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.091+0000 7f5f8f7fe640 10 client.27167 handle_command_reply: tid=0 2024-09-15T22:22:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.091+0000 7f5fb9373740 1 client.27167 shutdown 2024-09-15T22:22:04.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.093+0000 7f5fb9373740 1 -- 172.21.15.5:0/940783628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 msgr2=0x5573ed265bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:04.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.093+0000 7f5fb9373740 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed265bc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5fa4035040 tx=0x7f5fa4005ab0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.097+0000 7f5fb9373740 1 -- 172.21.15.5:0/940783628 shutdown_connections 2024-09-15T22:22:04.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.097+0000 7f5fb9373740 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573ed260970 0x5573ed266100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.097+0000 7f5fb9373740 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573ed2da9f0 0x5573ed2dce10 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.097+0000 7f5fb9373740 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573ed2560b0 0x5573ed265bc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.097+0000 7f5fb9373740 1 --2- 172.21.15.5:0/940783628 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573ed255790 0x5573ed265680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.097+0000 7f5f8f7fe640 0 client.27167 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:04.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.097+0000 7f5fb9373740 1 -- 172.21.15.5:0/940783628 >> 172.21.15.5:0/940783628 conn(0x5573ed251d20 msgr2=0x5573ed2506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:04.105 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:22:04.105 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:22:04.105 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:22:04.105 INFO:teuthology.orchestra.run.smithi005.stdout: "a56324ee-60da-4c6e-8844-b9baa700aeb4": { 2024-09-15T22:22:04.105 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:22:04.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "a56324ee-60da-4c6e-8844-b9baa700aeb4", 2024-09-15T22:22:04.105 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:22:04.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:04.105 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:04.105 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:22:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.102+0000 7f5fb9373740 1 -- 172.21.15.5:0/940783628 shutdown_connections 2024-09-15T22:22:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.102+0000 7f5fb9373740 1 -- 172.21.15.5:0/940783628 wait complete. 2024-09-15T22:22:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.102+0000 7f5fb9373740 20 client.27167 trim_cache size 0 max 16384 2024-09-15T22:22:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.103+0000 7f5f977fe640 1 -- 172.21.15.5:0/2993764197 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5f980a3540 msgr2=0x7f5f980a5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.103+0000 7f5f977fe640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5f980a3540 0x7f5f980a5a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5fa0096de0 tx=0x7f5fa00a08b0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.103+0000 7f5f977fe640 1 -- 172.21.15.5:0/2993764197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb0109450 msgr2=0x7f5fb0118700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:04.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.103+0000 7f5f977fe640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb0109450 0x7f5fb0118700 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5fa000e9c0 tx=0x7f5fa000ee80 comp rx=0 tx=0).stop 2024-09-15T22:22:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.105+0000 7f5f977fe640 1 -- 172.21.15.5:0/2993764197 shutdown_connections 2024-09-15T22:22:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.105+0000 7f5f977fe640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5f980a3540 0x7f5f980a5a00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.105+0000 7f5f977fe640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fb011b140 0x7f5fb0111cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.105+0000 7f5f977fe640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5fb0109830 0x7f5fb0111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.105+0000 7f5f977fe640 1 --2- 172.21.15.5:0/2993764197 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5fb0109450 0x7f5fb0118700 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:04.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.105+0000 7f5f977fe640 1 -- 172.21.15.5:0/2993764197 >> 172.21.15.5:0/2993764197 conn(0x7f5fb006e140 msgr2=0x7f5fb011c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:04.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.109+0000 7f5f977fe640 1 -- 172.21.15.5:0/2993764197 shutdown_connections 2024-09-15T22:22:04.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:04.111+0000 7f5f977fe640 1 -- 172.21.15.5:0/2993764197 wait complete. 2024-09-15T22:22:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 755.0 s, 452.0 tps, lat 214.203 ms stddev 326.156 2024-09-15T22:22:05.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:05 smithi099 ceph-mon[34892]: pgmap v410: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-09-15T22:22:05.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:05 smithi129 ceph-mon[35314]: pgmap v410: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-09-15T22:22:05.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:05 smithi005 ceph-mon[30944]: pgmap v410: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 6.0 MiB/s wr, 564 op/s 2024-09-15T22:22:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:07 smithi099 ceph-mon[34892]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-09-15T22:22:07.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:07 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:22:07.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:07 smithi129 ceph-mon[35314]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-09-15T22:22:07.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:07 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:22:07.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:07 smithi005 ceph-mon[30944]: pgmap v411: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 6.6 MiB/s wr, 606 op/s 2024-09-15T22:22:07.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:07 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:22:08.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:08 smithi005 ceph-mon[30944]: osdmap e102: 12 total, 12 up, 12 in 2024-09-15T22:22:08.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:08 smithi099 ceph-mon[34892]: osdmap e102: 12 total, 12 up, 12 in 2024-09-15T22:22:08.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:08 smithi129 ceph-mon[35314]: osdmap e102: 12 total, 12 up, 12 in 2024-09-15T22:22:09.219 INFO:tasks.cephfs.filesystem:scrub status for tag:a56324ee-60da-4c6e-8844-b9baa700aeb4 - {'path': '/', 'tag': 'a56324ee-60da-4c6e-8844-b9baa700aeb4', 'options': 'recursive,force'} 2024-09-15T22:22:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 760.0 s, 368.2 tps, lat 282.194 ms stddev 426.323 2024-09-15T22:22:09.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:09 smithi099 ceph-mon[34892]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.0 MiB/s wr, 490 op/s 2024-09-15T22:22:09.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:09 smithi129 ceph-mon[35314]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.0 MiB/s wr, 490 op/s 2024-09-15T22:22:09.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:09 smithi005 ceph-mon[30944]: pgmap v413: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.0 MiB/s wr, 490 op/s 2024-09-15T22:22:11.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:11 smithi099 ceph-mon[34892]: pgmap v414: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.1 MiB/s wr, 489 op/s 2024-09-15T22:22:11.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:11 smithi129 ceph-mon[35314]: pgmap v414: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.1 MiB/s wr, 489 op/s 2024-09-15T22:22:11.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:11 smithi005 ceph-mon[30944]: pgmap v414: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 5.1 MiB/s wr, 489 op/s 2024-09-15T22:22:12.170 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.4 2024-09-15T22:22:12.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:22:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:13.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.350+0000 7feb58da8640 1 -- 172.21.15.5:0/1504761157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb5411b140 msgr2=0x7feb5411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:13.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.350+0000 7feb58da8640 1 --2- 172.21.15.5:0/1504761157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb5411b140 0x7feb5411d530 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009ca0 tx=0x7feb4c02a8f0 comp rx=0 tx=0).stop 2024-09-15T22:22:13.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.350+0000 7feb58da8640 1 -- 172.21.15.5:0/1504761157 shutdown_connections 2024-09-15T22:22:13.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.350+0000 7feb58da8640 1 --2- 172.21.15.5:0/1504761157 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb5411b140 0x7feb5411d530 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:13.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.350+0000 7feb58da8640 1 --2- 172.21.15.5:0/1504761157 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb5406ea30 0x7feb5406ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:13.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.350+0000 7feb58da8640 1 --2- 172.21.15.5:0/1504761157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb54110190 0x7feb5406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:13.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.350+0000 7feb58da8640 1 -- 172.21.15.5:0/1504761157 >> 172.21.15.5:0/1504761157 conn(0x7feb54073180 msgr2=0x7feb54073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:13.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.350+0000 7feb58da8640 1 -- 172.21.15.5:0/1504761157 shutdown_connections 2024-09-15T22:22:13.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.350+0000 7feb58da8640 1 -- 172.21.15.5:0/1504761157 wait complete. 2024-09-15T22:22:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.351+0000 7feb58da8640 1 Processor -- start 2024-09-15T22:22:13.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.355+0000 7feb58da8640 1 -- start start 2024-09-15T22:22:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.356+0000 7feb58da8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb5406ea30 0x7feb541a7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.356+0000 7feb58da8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb54110190 0x7feb541a7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.356+0000 7feb58da8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb5411b140 0x7feb541aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:13.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.356+0000 7feb58da8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb54120bf0 con 0x7feb54110190 2024-09-15T22:22:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.356+0000 7feb58da8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7feb54120a70 con 0x7feb5411b140 2024-09-15T22:22:13.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.356+0000 7feb58da8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7feb54120d70 con 0x7feb5406ea30 2024-09-15T22:22:13.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.363+0000 7feb52ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb54110190 0x7feb541a7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.363+0000 7feb52ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb54110190 0x7feb541a7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33488/0 (socket says 172.21.15.5:33488) 2024-09-15T22:22:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.363+0000 7feb52ffd640 1 -- 172.21.15.5:0/704211530 learned_addr learned my addr 172.21.15.5:0/704211530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.363+0000 7feb53fff640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb5411b140 0x7feb541aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:13.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.363+0000 7feb537fe640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb5406ea30 0x7feb541a7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.365+0000 7feb53fff640 1 -- 172.21.15.5:0/704211530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb5406ea30 msgr2=0x7feb541a7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.365+0000 7feb53fff640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb5406ea30 0x7feb541a7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.365+0000 7feb53fff640 1 -- 172.21.15.5:0/704211530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb54110190 msgr2=0x7feb541a7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.365+0000 7feb53fff640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb54110190 0x7feb541a7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.365+0000 7feb53fff640 1 -- 172.21.15.5:0/704211530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb541ac100 con 0x7feb5411b140 2024-09-15T22:22:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.365+0000 7feb52ffd640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb54110190 0x7feb541a7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:22:13.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.366+0000 7feb53fff640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb5411b140 0x7feb541aba20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009bf0 tx=0x7feb4c031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.368+0000 7feb50ff9640 1 -- 172.21.15.5:0/704211530 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c02fa50 con 0x7feb5411b140 2024-09-15T22:22:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.368+0000 7feb50ff9640 1 -- 172.21.15.5:0/704211530 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb4c02d070 con 0x7feb5411b140 2024-09-15T22:22:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.368+0000 7feb50ff9640 1 -- 172.21.15.5:0/704211530 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c0466e0 con 0x7feb5411b140 2024-09-15T22:22:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.368+0000 7feb58da8640 1 -- 172.21.15.5:0/704211530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb541ac390 con 0x7feb5411b140 2024-09-15T22:22:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.368+0000 7feb58da8640 1 -- 172.21.15.5:0/704211530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb541ac790 con 0x7feb5411b140 2024-09-15T22:22:13.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.375+0000 7feb50ff9640 1 -- 172.21.15.5:0/704211530 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7feb4c02fbf0 con 0x7feb5411b140 2024-09-15T22:22:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.377+0000 7feb50ff9640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feb400a35c0 0x7feb400a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.377+0000 7feb58da8640 1 -- 172.21.15.5:0/704211530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb34005290 con 0x7feb5411b140 2024-09-15T22:22:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.377+0000 7feb50ff9640 1 -- 172.21.15.5:0/704211530 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9629+0+0 (secure 0 0 0) 0x7feb4c0f11e0 con 0x7feb5411b140 2024-09-15T22:22:13.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.382+0000 7feb537fe640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feb400a35c0 0x7feb400a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.387+0000 7feb50ff9640 1 -- 172.21.15.5:0/704211530 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb4c0bd820 con 0x7feb5411b140 2024-09-15T22:22:13.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.387+0000 7feb537fe640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feb400a35c0 0x7feb400a5a80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009aa0 tx=0x7feb3c005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.802+0000 7feb58da8640 1 -- 172.21.15.5:0/704211530 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb34005580 con 0x7feb5411b140 2024-09-15T22:22:13.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:13 smithi005 ceph-mon[30944]: pgmap v415: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 4.0 MiB/s wr, 419 op/s 2024-09-15T22:22:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.803+0000 7feb50ff9640 1 -- 172.21.15.5:0/704211530 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7feb4c0418c0 con 0x7feb5411b140 2024-09-15T22:22:13.805 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:22:13.805 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:22:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:22:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.807+0000 7feb58da8640 1 -- 172.21.15.5:0/704211530 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feb400a35c0 msgr2=0x7feb400a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.807+0000 7feb58da8640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feb400a35c0 0x7feb400a5a80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7feb3c009aa0 tx=0x7feb3c005f90 comp rx=0 tx=0).stop 2024-09-15T22:22:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.807+0000 7feb58da8640 1 -- 172.21.15.5:0/704211530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb5411b140 msgr2=0x7feb541aba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.807+0000 7feb58da8640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb5411b140 0x7feb541aba20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009bf0 tx=0x7feb4c031a20 comp rx=0 tx=0).stop 2024-09-15T22:22:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.807+0000 7feb58da8640 1 -- 172.21.15.5:0/704211530 shutdown_connections 2024-09-15T22:22:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.807+0000 7feb58da8640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7feb400a35c0 0x7feb400a5a80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.807+0000 7feb58da8640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7feb5411b140 0x7feb541aba20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.807+0000 7feb58da8640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb54110190 0x7feb541a7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.807+0000 7feb58da8640 1 --2- 172.21.15.5:0/704211530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7feb5406ea30 0x7feb541a7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.807+0000 7feb58da8640 1 -- 172.21.15.5:0/704211530 >> 172.21.15.5:0/704211530 conn(0x7feb54073180 msgr2=0x7feb54119370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.808+0000 7feb58da8640 1 -- 172.21.15.5:0/704211530 shutdown_connections 2024-09-15T22:22:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:13.808+0000 7feb58da8640 1 -- 172.21.15.5:0/704211530 wait complete. 2024-09-15T22:22:13.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:13 smithi099 ceph-mon[34892]: pgmap v415: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 4.0 MiB/s wr, 419 op/s 2024-09-15T22:22:13.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:13 smithi129 ceph-mon[35314]: pgmap v415: 129 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 122 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 4.0 MiB/s wr, 419 op/s 2024-09-15T22:22:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 765.0 s, 503.0 tps, lat 202.946 ms stddev 282.000 2024-09-15T22:22:14.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:14 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/704211530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:14.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:14 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/704211530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:14.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:14 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/704211530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:15.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:15 smithi099 ceph-mon[34892]: pgmap v416: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.4 MiB/s wr, 407 op/s 2024-09-15T22:22:15.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:15 smithi129 ceph-mon[35314]: pgmap v416: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.4 MiB/s wr, 407 op/s 2024-09-15T22:22:15.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:15 smithi005 ceph-mon[30944]: pgmap v416: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 4.4 MiB/s wr, 407 op/s 2024-09-15T22:22:17.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:17 smithi099 ceph-mon[34892]: pgmap v417: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 4.8 MiB/s wr, 422 op/s 2024-09-15T22:22:17.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:17 smithi129 ceph-mon[35314]: pgmap v417: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 4.8 MiB/s wr, 422 op/s 2024-09-15T22:22:17.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:17 smithi005 ceph-mon[30944]: pgmap v417: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 4.8 MiB/s wr, 422 op/s 2024-09-15T22:22:17.972 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 131.4 --await-for 45.0' 2024-09-15T22:22:17.972 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 131.4 --await-for 45.0 2024-09-15T22:22:18.524 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.133+0000 7f892befa640 1 -- 172.21.15.5:0/3574083631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8924101df0 msgr2=0x7f89241021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.133+0000 7f892befa640 1 --2- 172.21.15.5:0/3574083631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8924101df0 0x7f89241021d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f891c009ca0 tx=0x7f891c02ed00 comp rx=0 tx=0).stop 2024-09-15T22:22:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.135+0000 7f892befa640 1 -- 172.21.15.5:0/3574083631 shutdown_connections 2024-09-15T22:22:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.135+0000 7f892befa640 1 --2- 172.21.15.5:0/3574083631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8924106f20 0x7f892410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.135+0000 7f892befa640 1 --2- 172.21.15.5:0/3574083631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89241027a0 0x7f89241067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.135+0000 7f892befa640 1 --2- 172.21.15.5:0/3574083631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8924101df0 0x7f89241021d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.135+0000 7f892befa640 1 -- 172.21.15.5:0/3574083631 >> 172.21.15.5:0/3574083631 conn(0x7f89240fd8b0 msgr2=0x7f89240ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.136+0000 7f892befa640 1 -- 172.21.15.5:0/3574083631 shutdown_connections 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.137+0000 7f892befa640 1 -- 172.21.15.5:0/3574083631 wait complete. 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.138+0000 7f892befa640 1 Processor -- start 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.138+0000 7f892befa640 1 -- start start 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.138+0000 7f892befa640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8924101df0 0x7f8924073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.138+0000 7f892befa640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89241027a0 0x7f8924073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.138+0000 7f892befa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924106f20 0x7f892406d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.138+0000 7f892befa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f892406c220 con 0x7f8924106f20 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.138+0000 7f892befa640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f892406c0a0 con 0x7f89241027a0 2024-09-15T22:22:19.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.138+0000 7f892befa640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f892406c3a0 con 0x7f8924101df0 2024-09-15T22:22:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.143+0000 7f892a497640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924106f20 0x7f892406d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.143+0000 7f892a497640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924106f20 0x7f892406d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50492/0 (socket says 172.21.15.5:50492) 2024-09-15T22:22:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.143+0000 7f892a497640 1 -- 172.21.15.5:0/519885838 learned_addr learned my addr 172.21.15.5:0/519885838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.143+0000 7f8929c96640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8924101df0 0x7f8924073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.145+0000 7f8929495640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89241027a0 0x7f8924073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.145+0000 7f8929c96640 1 -- 172.21.15.5:0/519885838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89241027a0 msgr2=0x7f8924073c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.149+0000 7f8929c96640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89241027a0 0x7f8924073c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.149+0000 7f8929c96640 1 -- 172.21.15.5:0/519885838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924106f20 msgr2=0x7f892406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.149+0000 7f8929c96640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924106f20 0x7f892406d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.149+0000 7f8929c96640 1 -- 172.21.15.5:0/519885838 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892406e010 con 0x7f8924101df0 2024-09-15T22:22:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.149+0000 7f8929c96640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8924101df0 0x7f8924073720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f891c009bf0 tx=0x7f891c035ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.150+0000 7f892a497640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924106f20 0x7f892406d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:22:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.151+0000 7f8929495640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89241027a0 0x7f8924073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.153+0000 7f891affd640 1 -- 172.21.15.5:0/519885838 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c045070 con 0x7f8924101df0 2024-09-15T22:22:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.153+0000 7f891affd640 1 -- 172.21.15.5:0/519885838 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f891c02fe30 con 0x7f8924101df0 2024-09-15T22:22:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.153+0000 7f891affd640 1 -- 172.21.15.5:0/519885838 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c03c030 con 0x7f8924101df0 2024-09-15T22:22:19.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.155+0000 7f892befa640 1 -- 172.21.15.5:0/519885838 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892406e2a0 con 0x7f8924101df0 2024-09-15T22:22:19.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.155+0000 7f892befa640 1 -- 172.21.15.5:0/519885838 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89241ab5e0 con 0x7f8924101df0 2024-09-15T22:22:19.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.161+0000 7f892e2d3740 1 Processor -- start 2024-09-15T22:22:19.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.162+0000 7f892e2d3740 1 -- start start 2024-09-15T22:22:19.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.162+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808d760 0x55938808db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.162+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808e080 0x559388098400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.162+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 0x55938809ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.162+0000 7f892e2d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559387ff32a0 con 0x559388098940 2024-09-15T22:22:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.162+0000 7f892e2d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559387f9ab10 con 0x55938808d760 2024-09-15T22:22:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.162+0000 7f892e2d3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559387fef260 con 0x55938808e080 2024-09-15T22:22:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.166+0000 7f892a497640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808d760 0x55938808db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.166+0000 7f8929495640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 0x55938809ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.166+0000 7f8929495640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 0x55938809ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50494/0 (socket says 172.21.15.5:50494) 2024-09-15T22:22:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.166+0000 7f8929495640 1 -- 172.21.15.5:0/1693745506 learned_addr learned my addr 172.21.15.5:0/1693745506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.168+0000 7f8929c96640 1 --2- 172.21.15.5:0/1693745506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808e080 0x559388098400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.169+0000 7f8929495640 1 -- 172.21.15.5:0/1693745506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808e080 msgr2=0x559388098400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.169+0000 7f8929495640 1 --2- 172.21.15.5:0/1693745506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808e080 0x559388098400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.169+0000 7f8929495640 1 -- 172.21.15.5:0/1693745506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808d760 msgr2=0x55938808db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.169+0000 7f8929495640 1 --2- 172.21.15.5:0/1693745506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808d760 0x55938808db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.169+0000 7f8929495640 1 -- 172.21.15.5:0/1693745506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559387d64b90 con 0x559388098940 2024-09-15T22:22:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.169+0000 7f891affd640 1 -- 172.21.15.5:0/519885838 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f891c0103c0 con 0x7f8924101df0 2024-09-15T22:22:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.170+0000 7f8929495640 1 --2- 172.21.15.5:0/1693745506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 0x55938809ad30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f891400a950 tx=0x7f891400ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d722a8ab6c57f13d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:19.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.171+0000 7f891affd640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89100a35c0 0x7f89100a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.171+0000 7f891affd640 1 -- 172.21.15.5:0/519885838 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f891c0f0140 con 0x7f8924101df0 2024-09-15T22:22:19.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.173+0000 7f890ffff640 1 -- 172.21.15.5:0/1693745506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914015070 con 0x559388098940 2024-09-15T22:22:19.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.173+0000 7f890ffff640 1 -- 172.21.15.5:0/1693745506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f891400cc00 con 0x559388098940 2024-09-15T22:22:19.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.173+0000 7f892a497640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89100a35c0 0x7f89100a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.174+0000 7f892e2d3740 1 -- 172.21.15.5:0/1693745506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 msgr2=0x55938809ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:19.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.174+0000 7f892e2d3740 1 --2- 172.21.15.5:0/1693745506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 0x55938809ad30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f891400a950 tx=0x7f891400ae10 comp rx=0 tx=0).stop 2024-09-15T22:22:19.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.174+0000 7f892a497640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89100a35c0 0x7f89100a5a80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8920006fd0 tx=0x7f8920008040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:19.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.178+0000 7f892e2d3740 1 -- 172.21.15.5:0/1693745506 shutdown_connections 2024-09-15T22:22:19.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.178+0000 7f892e2d3740 1 --2- 172.21.15.5:0/1693745506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 0x55938809ad30 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.178+0000 7f892e2d3740 1 --2- 172.21.15.5:0/1693745506 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808e080 0x559388098400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.178+0000 7f892e2d3740 1 --2- 172.21.15.5:0/1693745506 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808d760 0x55938808db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.178+0000 7f892e2d3740 1 -- 172.21.15.5:0/1693745506 >> 172.21.15.5:0/1693745506 conn(0x559388089cf0 msgr2=0x55938808a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:19.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.182+0000 7f892e2d3740 1 -- 172.21.15.5:0/1693745506 shutdown_connections 2024-09-15T22:22:19.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.185+0000 7f892e2d3740 1 -- 172.21.15.5:0/1693745506 wait complete. 2024-09-15T22:22:19.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.185+0000 7f892e2d3740 1 -- start start 2024-09-15T22:22:19.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.185+0000 7f892e2d3740 1 Processor -- start 2024-09-15T22:22:19.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.187+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808d760 0x5593880983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.187+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808e080 0x559388095690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.187+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 0x559388095e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.187+0000 7f892e2d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55938809b560 con 0x559388098940 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.187+0000 7f892e2d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559387f7b6e0 con 0x55938808e080 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.187+0000 7f892e2d3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55938809b6e0 con 0x55938808d760 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.187+0000 7f892e2d3740 10 client.0 _ll_register_callbacks cb 0x559387f9fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.188+0000 7f892a497640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 0x559388095e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.188+0000 7f892a497640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 0x559388095e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50500/0 (socket says 172.21.15.5:50500) 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.188+0000 7f892a497640 1 -- 172.21.15.5:0/2623079040 learned_addr learned my addr 172.21.15.5:0/2623079040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.188+0000 7f8929c96640 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808d760 0x5593880983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.191+0000 7f8929495640 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808e080 0x559388095690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.191+0000 7f8929495640 1 -- 172.21.15.5:0/2623079040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808d760 msgr2=0x5593880983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.191+0000 7f8929495640 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808d760 0x5593880983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.191+0000 7f8929495640 1 -- 172.21.15.5:0/2623079040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 msgr2=0x559388095e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.191+0000 7f8929495640 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 0x559388095e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.191+0000 7f8929495640 1 -- 172.21.15.5:0/2623079040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593880a4a80 con 0x55938808e080 2024-09-15T22:22:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.192+0000 7f8929495640 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808e080 0x559388095690 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8914008010 tx=0x7f8914006c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87f8311dcee2a20a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.192+0000 7f890d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:22:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.192+0000 7f890d7fa640 1 -- 172.21.15.5:0/2623079040 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914020070 con 0x55938808e080 2024-09-15T22:22:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.192+0000 7f892e2d3740 1 -- 172.21.15.5:0/2623079040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f891400cc00 con 0x55938808e080 2024-09-15T22:22:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.192+0000 7f890d7fa640 1 -- 172.21.15.5:0/2623079040 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f8914004250 con 0x55938808e080 2024-09-15T22:22:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.193+0000 7f890d7fa640 1 -- 172.21.15.5:0/2623079040 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8914015040 con 0x55938808e080 2024-09-15T22:22:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.193+0000 7f892e2d3740 10 client.27176 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.193+0000 7f892e2d3740 1 -- 172.21.15.5:0/2623079040 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5593880a72f0 con 0x55938808e080 2024-09-15T22:22:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.197+0000 7f890d7fa640 1 -- 172.21.15.5:0/2623079040 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f891400cc00 con 0x55938808e080 2024-09-15T22:22:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.197+0000 7f890d7fa640 1 -- 172.21.15.5:0/2623079040 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f89140037c0 con 0x55938808e080 2024-09-15T22:22:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.197+0000 7f892e2d3740 10 client.27176 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.197+0000 7f892e2d3740 10 client.27176 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:22:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.197+0000 7f892e2d3740 20 client.27176 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.197+0000 7f892e2d3740 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5593880a8b40 0x5593880aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.197+0000 7f892e2d3740 4 client.27176 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.197+0000 7f892e2d3740 1 -- 172.21.15.5:0/2623079040 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593880009d0 con 0x5593880a8b40 2024-09-15T22:22:19.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.202+0000 7f8929c96640 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5593880a8b40 0x5593880aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.203+0000 7f8929c96640 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5593880a8b40 0x5593880aafa0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9577b693f70ae7a4 server_cookie=5e786f8aefe5c6ae in_seq=0 out_seq=0 2024-09-15T22:22:19.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.203+0000 7f890d7fa640 10 client.27176 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.204+0000 7f890d7fa640 1 -- 172.21.15.5:0/2623079040 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593880009d0 con 0x5593880a8b40 2024-09-15T22:22:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.204+0000 7f890d7fa640 10 client.27176 handle_command_reply: tid=0 2024-09-15T22:22:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.205+0000 7f892e2d3740 1 client.27176 shutdown 2024-09-15T22:22:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.209+0000 7f892e2d3740 1 -- 172.21.15.5:0/2623079040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808e080 msgr2=0x559388095690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.209+0000 7f892e2d3740 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808e080 0x559388095690 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8914008010 tx=0x7f8914006c20 comp rx=0 tx=0).stop 2024-09-15T22:22:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.209+0000 7f892e2d3740 1 -- 172.21.15.5:0/2623079040 shutdown_connections 2024-09-15T22:22:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.209+0000 7f892e2d3740 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559388098940 0x559388095e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.209+0000 7f892e2d3740 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5593880a8b40 0x5593880aafa0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.209+0000 7f892e2d3740 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808e080 0x559388095690 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.209+0000 7f892e2d3740 1 --2- 172.21.15.5:0/2623079040 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808d760 0x5593880983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.209+0000 7f892e2d3740 1 -- 172.21.15.5:0/2623079040 >> 172.21.15.5:0/2623079040 conn(0x559388089cf0 msgr2=0x559388088e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.209+0000 7f890d7fa640 0 client.27176 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.210+0000 7f892e2d3740 1 -- 172.21.15.5:0/2623079040 shutdown_connections 2024-09-15T22:22:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.210+0000 7f892e2d3740 1 -- 172.21.15.5:0/2623079040 wait complete. 2024-09-15T22:22:19.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.211+0000 7f892e2d3740 20 client.27176 trim_cache size 0 max 16384 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f892e2d3740 1 Processor -- start 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f892e2d3740 1 -- start start 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808d760 0x5593880b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55938808e080 0x5593880b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559388098940 0x559388099d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f892e2d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89241b8020 con 0x55938808e080 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f892e2d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f89241b5970 con 0x559388098940 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f892e2d3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f89241032d0 con 0x55938808d760 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f8929495640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55938808e080 0x5593880b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f8929495640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55938808e080 0x5593880b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50502/0 (socket says 172.21.15.5:50502) 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f8929495640 1 -- 172.21.15.5:0/3545555218 learned_addr learned my addr 172.21.15.5:0/3545555218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:19.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.239+0000 7f8929c96640 1 --2- 172.21.15.5:0/3545555218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808d760 0x5593880b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.240+0000 7f8929c96640 1 -- 172.21.15.5:0/3545555218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559388098940 msgr2=0x559388099d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:19.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.240+0000 7f8929c96640 1 --2- 172.21.15.5:0/3545555218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559388098940 0x559388099d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.240+0000 7f8929c96640 1 -- 172.21.15.5:0/3545555218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55938808e080 msgr2=0x5593880b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:19.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.240+0000 7f8929c96640 1 --2- 172.21.15.5:0/3545555218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55938808e080 0x5593880b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.240+0000 7f8929c96640 1 -- 172.21.15.5:0/3545555218 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559388096190 con 0x55938808d760 2024-09-15T22:22:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.250+0000 7f8929c96640 1 --2- 172.21.15.5:0/3545555218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808d760 0x5593880b0870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f891c0ed540 tx=0x7f891c0f2520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e22fe05061a38145 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.250+0000 7f890f7fe640 1 -- 172.21.15.5:0/3545555218 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c035580 con 0x55938808d760 2024-09-15T22:22:19.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.250+0000 7f890f7fe640 1 -- 172.21.15.5:0/3545555218 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f891c0bc7d0 con 0x55938808d760 2024-09-15T22:22:19.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.250+0000 7f890f7fe640 1 -- 172.21.15.5:0/3545555218 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c0dc0b0 con 0x55938808d760 2024-09-15T22:22:19.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.251+0000 7f892e2d3740 1 -- 172.21.15.5:0/3545555218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808d760 msgr2=0x5593880b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:19.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.251+0000 7f892e2d3740 1 --2- 172.21.15.5:0/3545555218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808d760 0x5593880b0870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f891c0ed540 tx=0x7f891c0f2520 comp rx=0 tx=0).stop 2024-09-15T22:22:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.253+0000 7f892e2d3740 1 -- 172.21.15.5:0/3545555218 shutdown_connections 2024-09-15T22:22:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.253+0000 7f892e2d3740 1 --2- 172.21.15.5:0/3545555218 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559388098940 0x559388099d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.253+0000 7f892e2d3740 1 --2- 172.21.15.5:0/3545555218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55938808e080 0x5593880b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.253+0000 7f892e2d3740 1 --2- 172.21.15.5:0/3545555218 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55938808d760 0x5593880b0870 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.253+0000 7f892e2d3740 1 -- 172.21.15.5:0/3545555218 >> 172.21.15.5:0/3545555218 conn(0x559388089cf0 msgr2=0x559388090940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.253+0000 7f892e2d3740 1 -- 172.21.15.5:0/3545555218 shutdown_connections 2024-09-15T22:22:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.255+0000 7f892e2d3740 1 -- 172.21.15.5:0/3545555218 wait complete. 2024-09-15T22:22:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.256+0000 7f892e2d3740 1 -- start start 2024-09-15T22:22:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.256+0000 7f892e2d3740 1 Processor -- start 2024-09-15T22:22:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.257+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808d760 0x5593880a1270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.257+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55938808e080 0x559388099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.257+0000 7f892e2d3740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559388098940 0x559388099e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.257+0000 7f892e2d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f89241b3e70 con 0x55938808e080 2024-09-15T22:22:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.257+0000 7f892e2d3740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f89241b52a0 con 0x55938808d760 2024-09-15T22:22:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.257+0000 7f892e2d3740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f89241b4500 con 0x559388098940 2024-09-15T22:22:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.257+0000 7f892e2d3740 10 client.0 _ll_register_callbacks cb 0x7f8914018d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:19.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.257+0000 7f892a497640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559388098940 0x559388099e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.257+0000 7f892a497640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559388098940 0x559388099e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:46536/0 (socket says 172.21.15.5:46536) 2024-09-15T22:22:19.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.257+0000 7f892a497640 1 -- 172.21.15.5:0/1792611950 learned_addr learned my addr 172.21.15.5:0/1792611950 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:19.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.258+0000 7f8929495640 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55938808e080 0x559388099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.258+0000 7f892a497640 1 -- 172.21.15.5:0/1792611950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808d760 msgr2=0x5593880a1270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:22:19.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.258+0000 7f892a497640 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808d760 0x5593880a1270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.258+0000 7f892a497640 1 -- 172.21.15.5:0/1792611950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55938808e080 msgr2=0x559388099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:19.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.258+0000 7f892a497640 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55938808e080 0x559388099800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.258+0000 7f892a497640 1 -- 172.21.15.5:0/1792611950 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55938809ed90 con 0x559388098940 2024-09-15T22:22:19.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.262+0000 7f892a497640 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559388098940 0x559388099e80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f892000e540 tx=0x7f8920004700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a4ebf9f46fe6a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:19.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.262+0000 7f890d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:22:19.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.264+0000 7f890d7fa640 1 -- 172.21.15.5:0/1792611950 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892000f040 con 0x559388098940 2024-09-15T22:22:19.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.264+0000 7f892e2d3740 1 -- 172.21.15.5:0/1792611950 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f891c0bc7d0 con 0x559388098940 2024-09-15T22:22:19.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.264+0000 7f890d7fa640 1 -- 172.21.15.5:0/1792611950 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f8920015cd0 con 0x559388098940 2024-09-15T22:22:19.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.268+0000 7f890d7fa640 1 -- 172.21.15.5:0/1792611950 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8920005990 con 0x559388098940 2024-09-15T22:22:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.271+0000 7f892e2d3740 10 client.27091 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.271+0000 7f892e2d3740 1 -- 172.21.15.5:0/1792611950 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5593881139b0 con 0x559388098940 2024-09-15T22:22:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.272+0000 7f890d7fa640 1 -- 172.21.15.5:0/1792611950 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f891c0bc7d0 con 0x559388098940 2024-09-15T22:22:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.272+0000 7f890d7fa640 1 -- 172.21.15.5:0/1792611950 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8920012030 con 0x559388098940 2024-09-15T22:22:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.272+0000 7f892e2d3740 10 client.27091 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.273+0000 7f892e2d3740 10 client.27091 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:22:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.273+0000 7f892e2d3740 20 client.27091 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.273+0000 7f892e2d3740 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5593881141a0 0x5593881165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.273+0000 7f892e2d3740 4 client.27091 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.4, "await_for": 45.0}] 2024-09-15T22:22:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.273+0000 7f892e2d3740 1 -- 172.21.15.5:0/1792611950 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.4, "await_for": 45.0}) -- 0x559387ffad20 con 0x5593881141a0 2024-09-15T22:22:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.273+0000 7f8929c96640 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5593881141a0 0x5593881165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.274+0000 7f890d7fa640 10 client.27091 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.279+0000 7f8929c96640 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5593881141a0 0x5593881165c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d55949c05c98b4e server_cookie=64362dedb12b7e30 in_seq=0 out_seq=0 2024-09-15T22:22:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f890d7fa640 1 -- 172.21.15.5:0/1792611950 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559387ffad20 con 0x5593881141a0 2024-09-15T22:22:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f890d7fa640 10 client.27091 handle_command_reply: tid=0 2024-09-15T22:22:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f892e2d3740 1 client.27091 shutdown 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f892e2d3740 1 -- 172.21.15.5:0/1792611950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559388098940 msgr2=0x559388099e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f892e2d3740 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559388098940 0x559388099e80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f892000e540 tx=0x7f8920004700 comp rx=0 tx=0).stop 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f892e2d3740 1 -- 172.21.15.5:0/1792611950 shutdown_connections 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f892e2d3740 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559388098940 0x559388099e80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f892e2d3740 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55938808e080 0x559388099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f892e2d3740 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5593881141a0 0x5593881165c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f892e2d3740 1 --2- 172.21.15.5:0/1792611950 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55938808d760 0x5593880a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f892e2d3740 1 -- 172.21.15.5:0/1792611950 >> 172.21.15.5:0/1792611950 conn(0x559388089cf0 msgr2=0x559388093f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f890d7fa640 0 client.27091 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.644+0000 7f892e2d3740 1 -- 172.21.15.5:0/1792611950 shutdown_connections 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892e2d3740 1 -- 172.21.15.5:0/1792611950 wait complete. 2024-09-15T22:22:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892e2d3740 20 client.27091 trim_cache size 0 max 16384 2024-09-15T22:22:19.648 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:22:19.648 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:22:19.648 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:22:19.648 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 46, 2024-09-15T22:22:19.648 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:22:19.648 INFO:teuthology.orchestra.run.smithi005.stdout: "843c654b": { 2024-09-15T22:22:19.648 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 46, 2024-09-15T22:22:19.648 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:22:19.648 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 131.4, 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:19.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:19.650 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:19.650 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:22:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 -- 172.21.15.5:0/519885838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89100a35c0 msgr2=0x7f89100a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89100a35c0 0x7f89100a5a80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8920006fd0 tx=0x7f8920008040 comp rx=0 tx=0).stop 2024-09-15T22:22:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 -- 172.21.15.5:0/519885838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8924101df0 msgr2=0x7f8924073720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8924101df0 0x7f8924073720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f891c009bf0 tx=0x7f891c035ed0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 -- 172.21.15.5:0/519885838 shutdown_connections 2024-09-15T22:22:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f89100a35c0 0x7f89100a5a80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8924106f20 0x7f892406d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f89241027a0 0x7f8924073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 --2- 172.21.15.5:0/519885838 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8924101df0 0x7f8924073720 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 -- 172.21.15.5:0/519885838 >> 172.21.15.5:0/519885838 conn(0x7f89240fd8b0 msgr2=0x7f89240fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 -- 172.21.15.5:0/519885838 shutdown_connections 2024-09-15T22:22:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:19.645+0000 7f892befa640 1 -- 172.21.15.5:0/519885838 wait complete. 2024-09-15T22:22:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 770.0 s, 465.8 tps, lat 199.704 ms stddev 249.009 2024-09-15T22:22:19.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:19 smithi099 ceph-mon[34892]: pgmap v418: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.5 MiB/s wr, 412 op/s 2024-09-15T22:22:19.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:19 smithi099 ceph-mon[34892]: osdmap e103: 12 total, 12 up, 12 in 2024-09-15T22:22:19.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:19 smithi129 ceph-mon[35314]: pgmap v418: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.5 MiB/s wr, 412 op/s 2024-09-15T22:22:19.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:19 smithi129 ceph-mon[35314]: osdmap e103: 12 total, 12 up, 12 in 2024-09-15T22:22:19.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:19 smithi005 ceph-mon[30944]: pgmap v418: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 4.5 MiB/s wr, 412 op/s 2024-09-15T22:22:19.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:19 smithi005 ceph-mon[30944]: osdmap e103: 12 total, 12 up, 12 in 2024-09-15T22:22:20.281 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '843c654b', quiesce took 8.1 seconds. Will release after: 3.3000000000000007 2024-09-15T22:22:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:21 smithi099 ceph-mon[34892]: pgmap v420: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 4.6 MiB/s wr, 428 op/s 2024-09-15T22:22:21.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:21 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:22:21.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:21 smithi129 ceph-mon[35314]: pgmap v420: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 4.6 MiB/s wr, 428 op/s 2024-09-15T22:22:21.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:21 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:22:21.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:21 smithi005 ceph-mon[30944]: pgmap v420: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 4.6 MiB/s wr, 428 op/s 2024-09-15T22:22:21.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:21 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:22:22.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:22 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:22:22.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:22 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:22:22.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:22 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:22:22.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:22 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:22:22.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:22 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:22:22.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:22 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:22:22.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:22 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:22:22.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:22 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:22:22.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:22 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:22:23.581 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:22:23.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:23 smithi099 ceph-mon[34892]: pgmap v421: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 4.9 MiB/s wr, 452 op/s 2024-09-15T22:22:23.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:23 smithi129 ceph-mon[35314]: pgmap v421: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 4.9 MiB/s wr, 452 op/s 2024-09-15T22:22:23.856 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:23.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:23 smithi005 ceph-mon[30944]: pgmap v421: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 4.9 MiB/s wr, 452 op/s 2024-09-15T22:22:24.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.343+0000 7fbe22fdb640 1 -- 172.21.15.5:0/4240087225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe1c1036e0 msgr2=0x7fbe1c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.343+0000 7fbe22fdb640 1 --2- 172.21.15.5:0/4240087225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe1c1036e0 0x7fbe1c107b90 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fbe04009930 tx=0x7fbe0402a420 comp rx=0 tx=0).stop 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.343+0000 7fbe22fdb640 1 -- 172.21.15.5:0/4240087225 shutdown_connections 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.343+0000 7fbe22fdb640 1 --2- 172.21.15.5:0/4240087225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe1c1082a0 0x7fbe1c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.343+0000 7fbe22fdb640 1 --2- 172.21.15.5:0/4240087225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe1c1036e0 0x7fbe1c107b90 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.343+0000 7fbe22fdb640 1 --2- 172.21.15.5:0/4240087225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c102d30 0x7fbe1c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.343+0000 7fbe22fdb640 1 -- 172.21.15.5:0/4240087225 >> 172.21.15.5:0/4240087225 conn(0x7fbe1c0fc7d0 msgr2=0x7fbe1c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.343+0000 7fbe22fdb640 1 -- 172.21.15.5:0/4240087225 shutdown_connections 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.343+0000 7fbe22fdb640 1 -- 172.21.15.5:0/4240087225 wait complete. 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.343+0000 7fbe22fdb640 1 Processor -- start 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe22fdb640 1 -- start start 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe22fdb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe1c102d30 0x7fbe1c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:24.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe22fdb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c1036e0 0x7fbe1c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe22fdb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe1c1082a0 0x7fbe1c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe22fdb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c1120a0 con 0x7fbe1c102d30 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe22fdb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c111f20 con 0x7fbe1c1036e0 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe22fdb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fbe1c112220 con 0x7fbe1c1082a0 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe20d77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe1c102d30 0x7fbe1c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe20d77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe1c102d30 0x7fbe1c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59468/0 (socket says 172.21.15.5:59468) 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe20d77640 1 -- 172.21.15.5:0/1399255381 learned_addr learned my addr 172.21.15.5:0/1399255381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe21578640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe1c1082a0 0x7fbe1c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe1bfff640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c1036e0 0x7fbe1c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe20d77640 1 -- 172.21.15.5:0/1399255381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe1c1082a0 msgr2=0x7fbe1c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe20d77640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe1c1082a0 0x7fbe1c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe20d77640 1 -- 172.21.15.5:0/1399255381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c1036e0 msgr2=0x7fbe1c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe20d77640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c1036e0 0x7fbe1c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe20d77640 1 -- 172.21.15.5:0/1399255381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1c1a6b00 con 0x7fbe1c102d30 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe1bfff640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c1036e0 0x7fbe1c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.344+0000 7fbe20d77640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe1c102d30 0x7fbe1c100990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00d950 tx=0x7fbe0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.345+0000 7fbe19ffb640 1 -- 172.21.15.5:0/1399255381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c017030 con 0x7fbe1c102d30 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.345+0000 7fbe19ffb640 1 -- 172.21.15.5:0/1399255381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe0c004020 con 0x7fbe1c102d30 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.345+0000 7fbe22fdb640 1 -- 172.21.15.5:0/1399255381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe1c1a6df0 con 0x7fbe1c102d30 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.345+0000 7fbe19ffb640 1 -- 172.21.15.5:0/1399255381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c0047a0 con 0x7fbe1c102d30 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.345+0000 7fbe22fdb640 1 -- 172.21.15.5:0/1399255381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe1c1a72f0 con 0x7fbe1c102d30 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.346+0000 7fbe22fdb640 1 -- 172.21.15.5:0/1399255381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe1c103110 con 0x7fbe1c102d30 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.347+0000 7fbe19ffb640 1 -- 172.21.15.5:0/1399255381 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fbe0c0041d0 con 0x7fbe1c102d30 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.348+0000 7fbe19ffb640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe080a3430 0x7fbe080a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.348+0000 7fbe1bfff640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe080a3430 0x7fbe080a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.348+0000 7fbe19ffb640 1 -- 172.21.15.5:0/1399255381 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7fbe0c0cfba0 con 0x7fbe1c102d30 2024-09-15T22:22:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.348+0000 7fbe1bfff640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe080a3430 0x7fbe080a58f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbe0402a940 tx=0x7fbe04032c70 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.352+0000 7fbe19ffb640 1 -- 172.21.15.5:0/1399255381 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe0c09c120 con 0x7fbe1c102d30 2024-09-15T22:22:24.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.566+0000 7fbe22fdb640 1 -- 172.21.15.5:0/1399255381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe1c10a2b0 con 0x7fbe1c102d30 2024-09-15T22:22:24.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.567+0000 7fbe19ffb640 1 -- 172.21.15.5:0/1399255381 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fbe0c09ba20 con 0x7fbe1c102d30 2024-09-15T22:22:24.569 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:22:24.569 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:22:24.569 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:22:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 -- 172.21.15.5:0/1399255381 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe080a3430 msgr2=0x7fbe080a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe080a3430 0x7fbe080a58f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbe0402a940 tx=0x7fbe04032c70 comp rx=0 tx=0).stop 2024-09-15T22:22:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 -- 172.21.15.5:0/1399255381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe1c102d30 msgr2=0x7fbe1c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe1c102d30 0x7fbe1c100990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00d950 tx=0x7fbe0c00de10 comp rx=0 tx=0).stop 2024-09-15T22:22:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 -- 172.21.15.5:0/1399255381 shutdown_connections 2024-09-15T22:22:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fbe080a3430 0x7fbe080a58f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fbe1c1082a0 0x7fbe1c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:24.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fbe1c1036e0 0x7fbe1c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 --2- 172.21.15.5:0/1399255381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe1c102d30 0x7fbe1c100990 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 -- 172.21.15.5:0/1399255381 >> 172.21.15.5:0/1399255381 conn(0x7fbe1c0fc7d0 msgr2=0x7fbe1c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 -- 172.21.15.5:0/1399255381 shutdown_connections 2024-09-15T22:22:24.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:24.570+0000 7fbe22fdb640 1 -- 172.21.15.5:0/1399255381 wait complete. 2024-09-15T22:22:24.662 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 775.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:22:25.273 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id 843c654b --release --await' 2024-09-15T22:22:25.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id 843c654b --release --await 2024-09-15T22:22:25.535 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:25.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:25 smithi005 ceph-mon[30944]: pgmap v422: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 4.4 MiB/s wr, 419 op/s 2024-09-15T22:22:25.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:25 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1399255381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:25.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:25 smithi099 ceph-mon[34892]: pgmap v422: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 4.4 MiB/s wr, 419 op/s 2024-09-15T22:22:25.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:25 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1399255381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:25.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:25 smithi129 ceph-mon[35314]: pgmap v422: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 4.4 MiB/s wr, 419 op/s 2024-09-15T22:22:25.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:25 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1399255381' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:26.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.000+0000 7f2686394640 1 -- 172.21.15.5:0/4199738864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26801055b0 msgr2=0x7f268010d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.000+0000 7f2686394640 1 --2- 172.21.15.5:0/4199738864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26801055b0 0x7f268010d130 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2678009ca0 tx=0x7f267802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.001+0000 7f2686394640 1 -- 172.21.15.5:0/4199738864 shutdown_connections 2024-09-15T22:22:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.001+0000 7f2686394640 1 --2- 172.21.15.5:0/4199738864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26801055b0 0x7f268010d130 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.001+0000 7f2686394640 1 --2- 172.21.15.5:0/4199738864 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2680100e30 0x7f2680104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.001+0000 7f2686394640 1 --2- 172.21.15.5:0/4199738864 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2680100480 0x7f2680100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.001+0000 7f2686394640 1 -- 172.21.15.5:0/4199738864 >> 172.21.15.5:0/4199738864 conn(0x7f2680077210 msgr2=0x7f26800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.001+0000 7f2686394640 1 -- 172.21.15.5:0/4199738864 shutdown_connections 2024-09-15T22:22:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.001+0000 7f2686394640 1 -- 172.21.15.5:0/4199738864 wait complete. 2024-09-15T22:22:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.001+0000 7f2686394640 1 Processor -- start 2024-09-15T22:22:26.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.001+0000 7f2686394640 1 -- start start 2024-09-15T22:22:26.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f2686394640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2680100480 0x7f2680198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f2686394640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2680100e30 0x7f2680198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f2686394640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801055b0 0x7f268019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f2686394640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f268010fdb0 con 0x7f2680100480 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f2686394640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f268010fc30 con 0x7f26801055b0 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f2686394640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f268010ff30 con 0x7f2680100e30 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2680100480 0x7f2680198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2680100480 0x7f2680198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59488/0 (socket says 172.21.15.5:59488) 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267ffff640 1 -- 172.21.15.5:0/2530778511 learned_addr learned my addr 172.21.15.5:0/2530778511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267f7fe640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2680100e30 0x7f2680198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f2684931640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801055b0 0x7f268019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267ffff640 1 -- 172.21.15.5:0/2530778511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2680100e30 msgr2=0x7f2680198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267ffff640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2680100e30 0x7f2680198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267ffff640 1 -- 172.21.15.5:0/2530778511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801055b0 msgr2=0x7f268019ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267ffff640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801055b0 0x7f268019ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267ffff640 1 -- 172.21.15.5:0/2530778511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268019d3d0 con 0x7f2680100480 2024-09-15T22:22:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f2684931640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801055b0 0x7f268019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267f7fe640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2680100e30 0x7f2680198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:22:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267ffff640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2680100480 0x7f2680198420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f267000c920 tx=0x7f267000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267d7fa640 1 -- 172.21.15.5:0/2530778511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670007c70 con 0x7f2680100480 2024-09-15T22:22:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267d7fa640 1 -- 172.21.15.5:0/2530778511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2670007e10 con 0x7f2680100480 2024-09-15T22:22:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f2686394640 1 -- 172.21.15.5:0/2530778511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268019d6c0 con 0x7f2680100480 2024-09-15T22:22:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.002+0000 7f267d7fa640 1 -- 172.21.15.5:0/2530778511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670018890 con 0x7f2680100480 2024-09-15T22:22:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.003+0000 7f2686394640 1 -- 172.21.15.5:0/2530778511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26801a4fa0 con 0x7f2680100480 2024-09-15T22:22:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.005+0000 7f267d7fa640 1 -- 172.21.15.5:0/2530778511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2670018a50 con 0x7f2680100480 2024-09-15T22:22:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.005+0000 7f268876d740 1 Processor -- start 2024-09-15T22:22:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.005+0000 7f268876d740 1 -- start start 2024-09-15T22:22:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f268876d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caae5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f268876d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae5860 0x5573caaf4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f268876d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caaf4650 0x5573caaf6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f268876d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573caa4a620 con 0x5573caae4f40 2024-09-15T22:22:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f268876d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5573ca9f1e50 con 0x5573caaf4650 2024-09-15T22:22:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f268876d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5573caa465e0 con 0x5573caae5860 2024-09-15T22:22:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f2684931640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caae5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f267d7fa640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f26680a33a0 0x7f26680a5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f2684931640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caae5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59504/0 (socket says 172.21.15.5:59504) 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f2684931640 1 -- 172.21.15.5:0/1174308889 learned_addr learned my addr 172.21.15.5:0/1174308889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f267f7fe640 1 --2- 172.21.15.5:0/1174308889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caaf4650 0x5573caaf6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f267ffff640 1 --2- 172.21.15.5:0/1174308889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae5860 0x5573caaf4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f2684931640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f26680a33a0 0x7f26680a5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f267d7fa640 1 -- 172.21.15.5:0/2530778511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f26700d03a0 con 0x7f2680100480 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f2684931640 1 -- 172.21.15.5:0/1174308889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae5860 msgr2=0x5573caaf4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f2684931640 1 --2- 172.21.15.5:0/1174308889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae5860 0x5573caaf4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f2684931640 1 -- 172.21.15.5:0/1174308889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caaf4650 msgr2=0x5573caaf6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f2684931640 1 --2- 172.21.15.5:0/1174308889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caaf4650 0x5573caaf6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f2684931640 1 -- 172.21.15.5:0/1174308889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ca9e2d40 con 0x5573caae4f40 2024-09-15T22:22:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.006+0000 7f2684931640 1 --2- 172.21.15.5:0/1174308889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caae5320 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2678009dc0 tx=0x7f2678003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac4bfb16af8a183 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.007+0000 7f26667fc640 1 -- 172.21.15.5:0/1174308889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678004b40 con 0x5573caae4f40 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.007+0000 7f26667fc640 1 -- 172.21.15.5:0/1174308889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267804d070 con 0x5573caae4f40 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.007+0000 7f26667fc640 1 -- 172.21.15.5:0/1174308889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f267802f610 con 0x5573caae4f40 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.007+0000 7f2684931640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f26680a33a0 0x7f26680a5860 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f267804c000 tx=0x7f267804a000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.007+0000 7f268876d740 1 -- 172.21.15.5:0/1174308889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 msgr2=0x5573caae5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.007+0000 7f268876d740 1 --2- 172.21.15.5:0/1174308889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caae5320 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2678009dc0 tx=0x7f2678003ac0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.007+0000 7f268876d740 1 -- 172.21.15.5:0/1174308889 shutdown_connections 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.007+0000 7f268876d740 1 --2- 172.21.15.5:0/1174308889 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caaf4650 0x5573caaf6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.007+0000 7f268876d740 1 --2- 172.21.15.5:0/1174308889 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae5860 0x5573caaf4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.007+0000 7f268876d740 1 --2- 172.21.15.5:0/1174308889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caae5320 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 1 -- 172.21.15.5:0/1174308889 >> 172.21.15.5:0/1174308889 conn(0x5573caae15f0 msgr2=0x5573caae19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 1 -- 172.21.15.5:0/1174308889 shutdown_connections 2024-09-15T22:22:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 1 -- 172.21.15.5:0/1174308889 wait complete. 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 1 -- start start 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 1 Processor -- start 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae4f40 0x5573caaf15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae5860 0x5573caaf1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caaf4650 0x5573caaf2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5573caaf7220 con 0x5573caaf4650 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5573ca9d2f10 con 0x5573caae5860 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5573caaf73a0 con 0x5573caae4f40 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.008+0000 7f268876d740 10 client.0 _ll_register_callbacks cb 0x5573ca9f6db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f2684931640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caaf4650 0x5573caaf2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f267ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae4f40 0x5573caaf15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f2684931640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caaf4650 0x5573caaf2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59510/0 (socket says 172.21.15.5:59510) 2024-09-15T22:22:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f267ffff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae4f40 0x5573caaf15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:46436/0 (socket says 172.21.15.5:46436) 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f2684931640 1 -- 172.21.15.5:0/1875361584 learned_addr learned my addr 172.21.15.5:0/1875361584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f267f7fe640 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae5860 0x5573caaf1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f2684931640 1 -- 172.21.15.5:0/1875361584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae4f40 msgr2=0x5573caaf15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f2684931640 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae4f40 0x5573caaf15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f2684931640 1 -- 172.21.15.5:0/1875361584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae5860 msgr2=0x5573caaf1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f2684931640 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae5860 0x5573caaf1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f2684931640 1 -- 172.21.15.5:0/1875361584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573ca759580 con 0x5573caaf4650 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f267ffff640 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae4f40 0x5573caaf15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f2684931640 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caaf4650 0x5573caaf2040 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f267802a900 tx=0x7f2678004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290985883c15deb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f265f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f265f7fe640 1 -- 172.21.15.5:0/1875361584 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678060060 con 0x5573caaf4650 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f265f7fe640 1 -- 172.21.15.5:0/1875361584 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f2678031820 con 0x5573caaf4650 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.009+0000 7f268876d740 1 -- 172.21.15.5:0/1875361584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f267804d070 con 0x5573caaf4650 2024-09-15T22:22:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f265f7fe640 1 -- 172.21.15.5:0/1875361584 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2678067050 con 0x5573caaf4650 2024-09-15T22:22:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f268876d740 10 client.17691 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f268876d740 1 -- 172.21.15.5:0/1875361584 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5573ca823960 con 0x5573caaf4650 2024-09-15T22:22:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f265f7fe640 1 -- 172.21.15.5:0/1875361584 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f267804d070 con 0x5573caaf4650 2024-09-15T22:22:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f265f7fe640 1 -- 172.21.15.5:0/1875361584 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f267802fac0 con 0x5573caaf4650 2024-09-15T22:22:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f268876d740 10 client.17691 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f268876d740 10 client.17691 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:22:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f268876d740 20 client.17691 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f268876d740 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573cab04110 0x5573cab06500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f268876d740 4 client.17691 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f268876d740 1 -- 172.21.15.5:0/1875361584 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573caa57b90 con 0x5573cab04110 2024-09-15T22:22:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.010+0000 7f267ffff640 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573cab04110 0x5573cab06500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.011+0000 7f267ffff640 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573cab04110 0x5573cab06500 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=841e3d0329f87eee server_cookie=7b129fa91b17a9fc in_seq=0 out_seq=0 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.011+0000 7f265f7fe640 10 client.17691 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.011+0000 7f265f7fe640 1 -- 172.21.15.5:0/1875361584 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573caa57b90 con 0x5573cab04110 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.011+0000 7f265f7fe640 10 client.17691 handle_command_reply: tid=0 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 1 client.17691 shutdown 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 1 -- 172.21.15.5:0/1875361584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caaf4650 msgr2=0x5573caaf2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caaf4650 0x5573caaf2040 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f267802a900 tx=0x7f2678004f00 comp rx=0 tx=0).stop 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 1 -- 172.21.15.5:0/1875361584 shutdown_connections 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caaf4650 0x5573caaf2040 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573cab04110 0x5573cab06500 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae5860 0x5573caaf1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 1 --2- 172.21.15.5:0/1875361584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caae4f40 0x5573caaf15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 1 -- 172.21.15.5:0/1875361584 >> 172.21.15.5:0/1875361584 conn(0x5573caae15f0 msgr2=0x5573caae7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:26.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f265f7fe640 0 client.17691 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:26.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 1 -- 172.21.15.5:0/1875361584 shutdown_connections 2024-09-15T22:22:26.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 1 -- 172.21.15.5:0/1875361584 wait complete. 2024-09-15T22:22:26.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.012+0000 7f268876d740 20 client.17691 trim_cache size 0 max 16384 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.029+0000 7f268876d740 1 Processor -- start 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.029+0000 7f268876d740 1 -- start start 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.029+0000 7f268876d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae4f40 0x5573caaf3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.029+0000 7f268876d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae5860 0x5573cab0baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.029+0000 7f268876d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caaf4650 0x5573cab0bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.029+0000 7f268876d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26801af830 con 0x5573caae5860 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.029+0000 7f268876d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26801ad100 con 0x5573caae4f40 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.029+0000 7f268876d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f26801aaf70 con 0x5573caaf4650 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.029+0000 7f267f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae5860 0x5573cab0baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.029+0000 7f267f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae5860 0x5573cab0baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59526/0 (socket says 172.21.15.5:59526) 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.029+0000 7f267f7fe640 1 -- 172.21.15.5:0/3529801499 learned_addr learned my addr 172.21.15.5:0/3529801499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.030+0000 7f267f7fe640 1 -- 172.21.15.5:0/3529801499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caaf4650 msgr2=0x5573cab0bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.030+0000 7f267f7fe640 1 --2- 172.21.15.5:0/3529801499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caaf4650 0x5573cab0bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.030+0000 7f267f7fe640 1 -- 172.21.15.5:0/3529801499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae4f40 msgr2=0x5573caaf3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.030+0000 7f267ffff640 1 --2- 172.21.15.5:0/3529801499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae4f40 0x5573caaf3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.030+0000 7f267f7fe640 1 --2- 172.21.15.5:0/3529801499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae4f40 0x5573caaf3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.030+0000 7f267f7fe640 1 -- 172.21.15.5:0/3529801499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573caae20b0 con 0x5573caae5860 2024-09-15T22:22:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.030+0000 7f267ffff640 1 --2- 172.21.15.5:0/3529801499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae4f40 0x5573caaf3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.030+0000 7f267f7fe640 1 --2- 172.21.15.5:0/3529801499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae5860 0x5573cab0baa0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f26800ff630 tx=0x7f2674007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d244a113e8ff2e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.030+0000 7f2665ffb640 1 -- 172.21.15.5:0/3529801499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674019070 con 0x5573caae5860 2024-09-15T22:22:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.030+0000 7f2665ffb640 1 -- 172.21.15.5:0/3529801499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2674015420 con 0x5573caae5860 2024-09-15T22:22:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.030+0000 7f2665ffb640 1 -- 172.21.15.5:0/3529801499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2674014610 con 0x5573caae5860 2024-09-15T22:22:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.031+0000 7f268876d740 1 -- 172.21.15.5:0/3529801499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae5860 msgr2=0x5573cab0baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.031+0000 7f268876d740 1 --2- 172.21.15.5:0/3529801499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae5860 0x5573cab0baa0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f26800ff630 tx=0x7f2674007ca0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.031+0000 7f268876d740 1 -- 172.21.15.5:0/3529801499 shutdown_connections 2024-09-15T22:22:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.031+0000 7f268876d740 1 --2- 172.21.15.5:0/3529801499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caaf4650 0x5573cab0bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.031+0000 7f268876d740 1 --2- 172.21.15.5:0/3529801499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae5860 0x5573cab0baa0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.031+0000 7f268876d740 1 --2- 172.21.15.5:0/3529801499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae4f40 0x5573caaf3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.031+0000 7f268876d740 1 -- 172.21.15.5:0/3529801499 >> 172.21.15.5:0/3529801499 conn(0x5573caae15f0 msgr2=0x5573caaf2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.031+0000 7f268876d740 1 -- 172.21.15.5:0/3529801499 shutdown_connections 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.031+0000 7f268876d740 1 -- 172.21.15.5:0/3529801499 wait complete. 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.031+0000 7f268876d740 1 -- start start 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.031+0000 7f268876d740 1 Processor -- start 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f268876d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caaf83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f268876d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae5860 0x5573caaf8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f268876d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caaf4650 0x5573caaf8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f268876d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26801ab5c0 con 0x5573caae4f40 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f268876d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f26801aca30 con 0x5573caae5860 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f268876d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f26801abc90 con 0x5573caaf4650 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f268876d740 10 client.0 _ll_register_callbacks cb 0x7f2678005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f267ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caaf83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f267ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caaf83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59530/0 (socket says 172.21.15.5:59530) 2024-09-15T22:22:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f267ffff640 1 -- 172.21.15.5:0/3965803375 learned_addr learned my addr 172.21.15.5:0/3965803375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f267f7fe640 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae5860 0x5573caaf8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f2684931640 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caaf4650 0x5573caaf8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f267ffff640 1 -- 172.21.15.5:0/3965803375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caaf4650 msgr2=0x5573caaf8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f267ffff640 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caaf4650 0x5573caaf8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f267ffff640 1 -- 172.21.15.5:0/3965803375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae5860 msgr2=0x5573caaf8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f267ffff640 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae5860 0x5573caaf8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f267ffff640 1 -- 172.21.15.5:0/3965803375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573caaf0570 con 0x5573caae4f40 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f2684931640 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caaf4650 0x5573caaf8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f267ffff640 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caaf83d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2670007590 tx=0x7f267006af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a700591775f95d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f265f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f265f7fe640 1 -- 172.21.15.5:0/3965803375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26700bae60 con 0x5573caae4f40 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.032+0000 7f265f7fe640 1 -- 172.21.15.5:0/3965803375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f267009a520 con 0x5573caae4f40 2024-09-15T22:22:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f268876d740 1 -- 172.21.15.5:0/3965803375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2674015420 con 0x5573caae4f40 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f265f7fe640 1 -- 172.21.15.5:0/3965803375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f26700d0160 con 0x5573caae4f40 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f268876d740 10 client.17697 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f268876d740 1 -- 172.21.15.5:0/3965803375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5573caae5690 con 0x5573caae4f40 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f265f7fe640 1 -- 172.21.15.5:0/3965803375 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2674015420 con 0x5573caae4f40 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f265f7fe640 1 -- 172.21.15.5:0/3965803375 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f26700cd9a0 con 0x5573caae4f40 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f268876d740 10 client.17697 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f268876d740 10 client.17697 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f268876d740 20 client.17697 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f268876d740 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573cab6f110 0x5573cab71530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f268876d740 4 client.17697 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "843c654b", "release": true, "await": true}] 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f268876d740 1 -- 172.21.15.5:0/3965803375 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "843c654b", "release": true, "await": true}) -- 0x5573caa520a0 con 0x5573cab6f110 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.033+0000 7f267f7fe640 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573cab6f110 0x5573cab71530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.034+0000 7f267f7fe640 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573cab6f110 0x5573cab71530 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eb5f30b7bf88b92 server_cookie=97037b5aef239fc0 in_seq=0 out_seq=0 2024-09-15T22:22:26.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.034+0000 7f265f7fe640 10 client.17697 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:26.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f265f7fe640 1 -- 172.21.15.5:0/3965803375 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5573caa520a0 con 0x5573cab6f110 2024-09-15T22:22:26.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f265f7fe640 10 client.17697 handle_command_reply: tid=0 2024-09-15T22:22:26.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 1 client.17697 shutdown 2024-09-15T22:22:26.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 1 -- 172.21.15.5:0/3965803375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 msgr2=0x5573caaf83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caaf83d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2670007590 tx=0x7f267006af10 comp rx=0 tx=0).stop 2024-09-15T22:22:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 1 -- 172.21.15.5:0/3965803375 shutdown_connections 2024-09-15T22:22:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5573caaf4650 0x5573caaf8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5573cab6f110 0x5573cab71530 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5573caae5860 0x5573caaf8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 1 --2- 172.21.15.5:0/3965803375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5573caae4f40 0x5573caaf83d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 1 -- 172.21.15.5:0/3965803375 >> 172.21.15.5:0/3965803375 conn(0x5573caae15f0 msgr2=0x5573caaf3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f265f7fe640 0 client.17697 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 1 -- 172.21.15.5:0/3965803375 shutdown_connections 2024-09-15T22:22:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 1 -- 172.21.15.5:0/3965803375 wait complete. 2024-09-15T22:22:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.083+0000 7f268876d740 20 client.17697 trim_cache size 0 max 16384 2024-09-15T22:22:26.087 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:22:26.087 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:22:26.087 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:22:26.087 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 48, 2024-09-15T22:22:26.087 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:22:26.087 INFO:teuthology.orchestra.run.smithi005.stdout: "843c654b": { 2024-09-15T22:22:26.087 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 48, 2024-09-15T22:22:26.087 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:22:26.087 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:22:26.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:22:26.087 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 131.4, 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:26.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:26.089 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:22:26.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 -- 172.21.15.5:0/2530778511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f26680a33a0 msgr2=0x7f26680a5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:26.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f26680a33a0 0x7f26680a5860 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f267804c000 tx=0x7f267804a000 comp rx=0 tx=0).stop 2024-09-15T22:22:26.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 -- 172.21.15.5:0/2530778511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2680100480 msgr2=0x7f2680198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:26.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2680100480 0x7f2680198420 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f267000c920 tx=0x7f267000cde0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 -- 172.21.15.5:0/2530778511 shutdown_connections 2024-09-15T22:22:26.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f26680a33a0 0x7f26680a5860 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f26801055b0 0x7f268019ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2680100e30 0x7f2680198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 --2- 172.21.15.5:0/2530778511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2680100480 0x7f2680198420 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:26.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 -- 172.21.15.5:0/2530778511 >> 172.21.15.5:0/2530778511 conn(0x7f2680077210 msgr2=0x7f2680103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:26.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 -- 172.21.15.5:0/2530778511 shutdown_connections 2024-09-15T22:22:26.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:26.084+0000 7f2686394640 1 -- 172.21.15.5:0/2530778511 wait complete. 2024-09-15T22:22:27.035 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '843c654b', total seconds elapsed: 14.9 2024-09-15T22:22:27.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:27 smithi099 ceph-mon[34892]: pgmap v423: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-09-15T22:22:27.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:27 smithi129 ceph-mon[35314]: pgmap v423: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-09-15T22:22:27.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:27 smithi005 ceph-mon[30944]: pgmap v423: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 3.0 MiB/s wr, 311 op/s 2024-09-15T22:22:29.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 780.0 s, 411.1 tps, lat 497.165 ms stddev 1508.574 2024-09-15T22:22:29.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:29 smithi099 ceph-mon[34892]: pgmap v424: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-09-15T22:22:29.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:29 smithi129 ceph-mon[35314]: pgmap v424: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-09-15T22:22:29.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:29 smithi005 ceph-mon[30944]: pgmap v424: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-09-15T22:22:31.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:31 smithi099 ceph-mon[34892]: pgmap v425: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-15T22:22:31.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:31 smithi129 ceph-mon[35314]: pgmap v425: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-15T22:22:31.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:31 smithi005 ceph-mon[30944]: pgmap v425: 129 pgs: 129 active+clean; 2.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-09-15T22:22:33.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:33 smithi099 ceph-mon[34892]: pgmap v426: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-15T22:22:33.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:33 smithi129 ceph-mon[35314]: pgmap v426: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-15T22:22:33.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:33 smithi005 ceph-mon[30944]: pgmap v426: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 3.3 MiB/s wr, 323 op/s 2024-09-15T22:22:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 785.0 s, 494.8 tps, lat 201.221 ms stddev 243.365 2024-09-15T22:22:35.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:34 smithi099 ceph-mon[34892]: pgmap v427: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-09-15T22:22:35.095 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:34 smithi129 ceph-mon[35314]: pgmap v427: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-09-15T22:22:35.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:34 smithi005 ceph-mon[30944]: pgmap v427: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-09-15T22:22:37.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:36 smithi099 ceph-mon[34892]: pgmap v428: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-09-15T22:22:37.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:36 smithi129 ceph-mon[35314]: pgmap v428: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-09-15T22:22:37.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:36 smithi005 ceph-mon[30944]: pgmap v428: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 3.0 MiB/s wr, 316 op/s 2024-09-15T22:22:39.158 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:38 smithi005 ceph-mon[30944]: pgmap v429: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.2 MiB/s wr, 418 op/s 2024-09-15T22:22:39.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:22:39.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:38 smithi099 ceph-mon[34892]: pgmap v429: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.2 MiB/s wr, 418 op/s 2024-09-15T22:22:39.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:38 smithi129 ceph-mon[35314]: pgmap v429: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 4.2 MiB/s wr, 418 op/s 2024-09-15T22:22:39.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 790.0 s, 512.3 tps, lat 198.613 ms stddev 267.379 2024-09-15T22:22:39.781 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:40.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.459+0000 7f281c6d6640 1 -- 172.21.15.5:0/2100838707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f281410d530 msgr2=0x7f28141150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.459+0000 7f281c6d6640 1 --2- 172.21.15.5:0/2100838707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f281410d530 0x7f28141150b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2810009900 tx=0x7f281002a420 comp rx=0 tx=0).stop 2024-09-15T22:22:40.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.459+0000 7f281c6d6640 1 -- 172.21.15.5:0/2100838707 shutdown_connections 2024-09-15T22:22:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.459+0000 7f281c6d6640 1 --2- 172.21.15.5:0/2100838707 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f281410d530 0x7f28141150b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.459+0000 7f281c6d6640 1 --2- 172.21.15.5:0/2100838707 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2814108e00 0x7f281410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.459+0000 7f281c6d6640 1 --2- 172.21.15.5:0/2100838707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2814108450 0x7f2814108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.459+0000 7f281c6d6640 1 -- 172.21.15.5:0/2100838707 >> 172.21.15.5:0/2100838707 conn(0x7f2814077ba0 msgr2=0x7f2814079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.460+0000 7f281c6d6640 1 -- 172.21.15.5:0/2100838707 shutdown_connections 2024-09-15T22:22:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.460+0000 7f281c6d6640 1 -- 172.21.15.5:0/2100838707 wait complete. 2024-09-15T22:22:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281c6d6640 1 Processor -- start 2024-09-15T22:22:40.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281c6d6640 1 -- start start 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281c6d6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2814108450 0x7f2814073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281c6d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2814108e00 0x7f281406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281c6d6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281410d530 0x7f281406e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281c6d6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f281406c660 con 0x7f2814108e00 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281c6d6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f281406c4e0 con 0x7f281410d530 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281c6d6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f281406c7e0 con 0x7f2814108450 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281a472640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2814108450 0x7f2814073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281ac73640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281410d530 0x7f281406e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281a472640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2814108450 0x7f2814073c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:48644/0 (socket says 172.21.15.5:48644) 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.461+0000 7f281a472640 1 -- 172.21.15.5:0/1082455236 learned_addr learned my addr 172.21.15.5:0/1082455236 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.462+0000 7f281a472640 1 -- 172.21.15.5:0/1082455236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281410d530 msgr2=0x7f281406e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.462+0000 7f281a472640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281410d530 0x7f281406e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.462+0000 7f281a472640 1 -- 172.21.15.5:0/1082455236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2814108e00 msgr2=0x7f281406dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.462+0000 7f281a472640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2814108e00 0x7f281406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.462+0000 7f281a472640 1 -- 172.21.15.5:0/1082455236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281406eab0 con 0x7f2814108450 2024-09-15T22:22:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.462+0000 7f281a472640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2814108450 0x7f2814073c30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f280400b4a0 tx=0x7f280400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.462+0000 7f280b7fe640 1 -- 172.21.15.5:0/1082455236 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804014070 con 0x7f2814108450 2024-09-15T22:22:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.462+0000 7f281c6d6640 1 -- 172.21.15.5:0/1082455236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2814080ad0 con 0x7f2814108450 2024-09-15T22:22:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.462+0000 7f280b7fe640 1 -- 172.21.15.5:0/1082455236 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2804010af0 con 0x7f2814108450 2024-09-15T22:22:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.462+0000 7f281c6d6640 1 -- 172.21.15.5:0/1082455236 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2814081060 con 0x7f2814108450 2024-09-15T22:22:40.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.462+0000 7f280b7fe640 1 -- 172.21.15.5:0/1082455236 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804017d10 con 0x7f2814108450 2024-09-15T22:22:40.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.467+0000 7f281eaaf740 1 Processor -- start 2024-09-15T22:22:40.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.467+0000 7f281eaaf740 1 -- start start 2024-09-15T22:22:40.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.467+0000 7f280b7fe640 1 -- 172.21.15.5:0/1082455236 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2804017070 con 0x7f2814108450 2024-09-15T22:22:40.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.468+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e14a0 0x55a9555e1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.468+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e1dc0 0x55a9555ec140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.468+0000 7f280b7fe640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f28000a3480 0x7f28000a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.468+0000 7f280b7fe640 1 -- 172.21.15.5:0/1082455236 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f28040cf310 con 0x7f2814108450 2024-09-15T22:22:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.468+0000 7f281ac73640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e14a0 0x55a9555e1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.468+0000 7f281ac73640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e14a0 0x55a9555e1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:48646/0 (socket says 172.21.15.5:48646) 2024-09-15T22:22:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.468+0000 7f2819c71640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f28000a3480 0x7f28000a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555ec680 0x55a9555eea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281eaaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a9555472e0 con 0x55a9555ec680 2024-09-15T22:22:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281eaaf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a9554eece0 con 0x55a9555e1dc0 2024-09-15T22:22:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281eaaf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a9555432a0 con 0x55a9555e14a0 2024-09-15T22:22:40.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281ac73640 1 -- 172.21.15.5:0/518371444 learned_addr learned my addr 172.21.15.5:0/518371444 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281a472640 1 --2- 172.21.15.5:0/518371444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e1dc0 0x55a9555ec140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281ac73640 1 --2- 172.21.15.5:0/518371444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555ec680 0x55a9555eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281a472640 1 -- 172.21.15.5:0/518371444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e14a0 msgr2=0x55a9555e1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281a472640 1 --2- 172.21.15.5:0/518371444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e14a0 0x55a9555e1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281a472640 1 -- 172.21.15.5:0/518371444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555ec680 msgr2=0x55a9555eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281a472640 1 --2- 172.21.15.5:0/518371444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555ec680 0x55a9555eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281a472640 1 -- 172.21.15.5:0/518371444 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9554d3440 con 0x55a9555e1dc0 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281ac73640 1 --2- 172.21.15.5:0/518371444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555ec680 0x55a9555eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f2819c71640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f28000a3480 0x7f28000a5940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f281406f470 tx=0x7f280c009250 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f281a472640 1 --2- 172.21.15.5:0/518371444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e1dc0 0x55a9555ec140 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f28040cc770 tx=0x7f280409ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da0babf30081507 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.469+0000 7f2808ff9640 1 -- 172.21.15.5:0/518371444 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280401e050 con 0x55a9555e1dc0 2024-09-15T22:22:40.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.470+0000 7f2808ff9640 1 -- 172.21.15.5:0/518371444 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280409f6b0 con 0x55a9555e1dc0 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.470+0000 7f2808ff9640 1 -- 172.21.15.5:0/518371444 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280401e050 con 0x55a9555e1dc0 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.470+0000 7f281eaaf740 1 -- 172.21.15.5:0/518371444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e1dc0 msgr2=0x55a9555ec140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.470+0000 7f281eaaf740 1 --2- 172.21.15.5:0/518371444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e1dc0 0x55a9555ec140 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f28040cc770 tx=0x7f280409ace0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.470+0000 7f281eaaf740 1 -- 172.21.15.5:0/518371444 shutdown_connections 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.470+0000 7f281eaaf740 1 --2- 172.21.15.5:0/518371444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555ec680 0x55a9555eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.470+0000 7f281eaaf740 1 --2- 172.21.15.5:0/518371444 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e1dc0 0x55a9555ec140 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.470+0000 7f281eaaf740 1 --2- 172.21.15.5:0/518371444 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e14a0 0x55a9555e1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.470+0000 7f281eaaf740 1 -- 172.21.15.5:0/518371444 >> 172.21.15.5:0/518371444 conn(0x55a9555dda30 msgr2=0x55a9555dde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.470+0000 7f281eaaf740 1 -- 172.21.15.5:0/518371444 shutdown_connections 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.470+0000 7f281eaaf740 1 -- 172.21.15.5:0/518371444 wait complete. 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281eaaf740 1 -- start start 2024-09-15T22:22:40.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281eaaf740 1 Processor -- start 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e14a0 0x55a9555ebf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a9555e9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555ec680 0x55a9555e9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281eaaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a9555ef250 con 0x55a9555ec680 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281eaaf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55a955502050 con 0x55a9555e14a0 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281eaaf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55a9555ef3d0 con 0x55a9555e1dc0 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281eaaf740 10 client.0 _ll_register_callbacks cb 0x55a955542160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281a472640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e14a0 0x55a9555ebf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281a472640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e14a0 0x55a9555ebf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:56058/0 (socket says 172.21.15.5:56058) 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281a472640 1 -- 172.21.15.5:0/868166230 learned_addr learned my addr 172.21.15.5:0/868166230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f281ac73640 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555ec680 0x55a9555e9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.471+0000 7f2819c71640 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a9555e9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.472+0000 7f281a472640 1 -- 172.21.15.5:0/868166230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 msgr2=0x55a9555e9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.472+0000 7f281a472640 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a9555e9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.472+0000 7f281a472640 1 -- 172.21.15.5:0/868166230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555ec680 msgr2=0x55a9555e9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.472+0000 7f281a472640 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555ec680 0x55a9555e9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.472+0000 7f281a472640 1 -- 172.21.15.5:0/868166230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9552b83c0 con 0x55a9555e14a0 2024-09-15T22:22:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.472+0000 7f281a472640 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e14a0 0x55a9555ebf60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f28040a3a40 tx=0x7f2804010ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2d476a1ca22ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.472+0000 7f27fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:22:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f27fdffb640 1 -- 172.21.15.5:0/868166230 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280401e050 con 0x55a9555e14a0 2024-09-15T22:22:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f27fdffb640 1 -- 172.21.15.5:0/868166230 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f2804017360 con 0x55a9555e14a0 2024-09-15T22:22:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f281eaaf740 1 -- 172.21.15.5:0/868166230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280409f6b0 con 0x55a9555e14a0 2024-09-15T22:22:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f27fdffb640 1 -- 172.21.15.5:0/868166230 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28040a3f70 con 0x55a9555e14a0 2024-09-15T22:22:40.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f281eaaf740 10 client.27197 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:40.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f281eaaf740 1 -- 172.21.15.5:0/868166230 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a9555fab60 con 0x55a9555e14a0 2024-09-15T22:22:40.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f27fdffb640 1 -- 172.21.15.5:0/868166230 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2804017560 con 0x55a9555e14a0 2024-09-15T22:22:40.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f27fdffb640 1 -- 172.21.15.5:0/868166230 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f280409a750 con 0x55a9555e14a0 2024-09-15T22:22:40.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f281eaaf740 10 client.27197 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:40.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f281eaaf740 10 client.27197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:40.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f281eaaf740 20 client.27197 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:40.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f281eaaf740 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a9555fbeb0 0x55a9555fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f281eaaf740 4 client.27197 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.473+0000 7f281eaaf740 1 -- 172.21.15.5:0/868166230 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f280409f6b0 con 0x55a9555fbeb0 2024-09-15T22:22:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.474+0000 7f2819c71640 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a9555fbeb0 0x55a9555fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.474+0000 7f27fdffb640 10 client.27197 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:40.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.474+0000 7f2819c71640 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a9555fbeb0 0x55a9555fe2a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e29601bac242a9f7 server_cookie=6b7fe08602dd2f3f in_seq=0 out_seq=0 2024-09-15T22:22:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f27fdffb640 1 -- 172.21.15.5:0/868166230 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f280409f6b0 con 0x55a9555fbeb0 2024-09-15T22:22:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f27fdffb640 10 client.27197 handle_command_reply: tid=0 2024-09-15T22:22:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f281eaaf740 1 client.27197 shutdown 2024-09-15T22:22:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f281eaaf740 1 -- 172.21.15.5:0/868166230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e14a0 msgr2=0x55a9555ebf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f281eaaf740 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e14a0 0x55a9555ebf60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f28040a3a40 tx=0x7f2804010ed0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f281eaaf740 1 -- 172.21.15.5:0/868166230 shutdown_connections 2024-09-15T22:22:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f281eaaf740 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555ec680 0x55a9555e9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f281eaaf740 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a9555e9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f281eaaf740 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a9555fbeb0 0x55a9555fe2a0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f27fdffb640 0 client.27197 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f281eaaf740 1 --2- 172.21.15.5:0/868166230 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555e14a0 0x55a9555ebf60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.475+0000 7f281eaaf740 1 -- 172.21.15.5:0/868166230 >> 172.21.15.5:0/868166230 conn(0x55a9555dda30 msgr2=0x55a9555e3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.476+0000 7f281eaaf740 1 -- 172.21.15.5:0/868166230 shutdown_connections 2024-09-15T22:22:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.476+0000 7f281eaaf740 1 -- 172.21.15.5:0/868166230 wait complete. 2024-09-15T22:22:40.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.476+0000 7f281eaaf740 20 client.27197 trim_cache size 0 max 16384 2024-09-15T22:22:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.508+0000 7f281eaaf740 1 Processor -- start 2024-09-15T22:22:40.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.508+0000 7f281eaaf740 1 -- start start 2024-09-15T22:22:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.508+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555e14a0 0x55a955603870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.508+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a955603db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.508+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555ec680 0x55a9555e6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.508+0000 7f281eaaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8012900 con 0x55a9555e14a0 2024-09-15T22:22:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.508+0000 7f281eaaf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27f80101d0 con 0x55a9555ec680 2024-09-15T22:22:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.508+0000 7f281eaaf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f27f800e040 con 0x55a9555e1dc0 2024-09-15T22:22:40.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.509+0000 7f2819c71640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a955603db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.509+0000 7f2819c71640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a955603db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:48660/0 (socket says 172.21.15.5:48660) 2024-09-15T22:22:40.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.509+0000 7f2819c71640 1 -- 172.21.15.5:0/3811857530 learned_addr learned my addr 172.21.15.5:0/3811857530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.509+0000 7f2819c71640 1 -- 172.21.15.5:0/3811857530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555ec680 msgr2=0x55a9555e6470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:22:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.509+0000 7f2819c71640 1 --2- 172.21.15.5:0/3811857530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555ec680 0x55a9555e6470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.509+0000 7f2819c71640 1 -- 172.21.15.5:0/3811857530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555e14a0 msgr2=0x55a955603870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:22:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.509+0000 7f2819c71640 1 --2- 172.21.15.5:0/3811857530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555e14a0 0x55a955603870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.510+0000 7f2819c71640 1 -- 172.21.15.5:0/3811857530 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9555e81c0 con 0x55a9555e1dc0 2024-09-15T22:22:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.510+0000 7f2819c71640 1 --2- 172.21.15.5:0/3811857530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a955603db0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f280c000f80 tx=0x7f280c00f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=becbd0ae72b66ed7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.511+0000 7f27fffff640 1 -- 172.21.15.5:0/3811857530 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c027030 con 0x55a9555e1dc0 2024-09-15T22:22:40.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.511+0000 7f27fffff640 1 -- 172.21.15.5:0/3811857530 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f280c022330 con 0x55a9555e1dc0 2024-09-15T22:22:40.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.512+0000 7f281eaaf740 1 -- 172.21.15.5:0/3811857530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 msgr2=0x55a955603db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.512+0000 7f281eaaf740 1 --2- 172.21.15.5:0/3811857530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a955603db0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f280c000f80 tx=0x7f280c00f000 comp rx=0 tx=0).stop 2024-09-15T22:22:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.523+0000 7f281eaaf740 1 -- 172.21.15.5:0/3811857530 shutdown_connections 2024-09-15T22:22:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.523+0000 7f281eaaf740 1 --2- 172.21.15.5:0/3811857530 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555ec680 0x55a9555e6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.523+0000 7f281eaaf740 1 --2- 172.21.15.5:0/3811857530 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a955603db0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.523+0000 7f281eaaf740 1 --2- 172.21.15.5:0/3811857530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555e14a0 0x55a955603870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.523+0000 7f281eaaf740 1 -- 172.21.15.5:0/3811857530 >> 172.21.15.5:0/3811857530 conn(0x55a9555dda30 msgr2=0x55a9555dcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.523+0000 7f281eaaf740 1 -- 172.21.15.5:0/3811857530 shutdown_connections 2024-09-15T22:22:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.523+0000 7f281eaaf740 1 -- 172.21.15.5:0/3811857530 wait complete. 2024-09-15T22:22:40.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.524+0000 7f281eaaf740 1 -- start start 2024-09-15T22:22:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.524+0000 7f281eaaf740 1 Processor -- start 2024-09-15T22:22:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.524+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555e14a0 0x55a9555f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.524+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a9555f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.524+0000 7f281eaaf740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555ec680 0x55a9555f3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.524+0000 7f281eaaf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27f800e690 con 0x55a9555e14a0 2024-09-15T22:22:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.524+0000 7f281eaaf740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f27f800fb00 con 0x55a9555ec680 2024-09-15T22:22:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.524+0000 7f281eaaf740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f27f800ed60 con 0x55a9555e1dc0 2024-09-15T22:22:40.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.524+0000 7f281eaaf740 10 client.0 _ll_register_callbacks cb 0x7f280409f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.525+0000 7f281ac73640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555ec680 0x55a9555f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.525+0000 7f2819c71640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a9555f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.527+0000 7f281ac73640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555ec680 0x55a9555f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:56072/0 (socket says 172.21.15.5:56072) 2024-09-15T22:22:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.527+0000 7f281ac73640 1 -- 172.21.15.5:0/1826214884 learned_addr learned my addr 172.21.15.5:0/1826214884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.528+0000 7f281a472640 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555e14a0 0x55a9555f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.528+0000 7f281ac73640 1 -- 172.21.15.5:0/1826214884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 msgr2=0x55a9555f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.528+0000 7f281ac73640 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a9555f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.528+0000 7f281ac73640 1 -- 172.21.15.5:0/1826214884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555e14a0 msgr2=0x55a9555f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.528+0000 7f281ac73640 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555e14a0 0x55a9555f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.528+0000 7f281ac73640 1 -- 172.21.15.5:0/1826214884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a955665aa0 con 0x55a9555ec680 2024-09-15T22:22:40.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.528+0000 7f281ac73640 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555ec680 0x55a9555f3100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2810009900 tx=0x7f2810004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e55279057e15fc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:40.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.530+0000 7f27fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:22:40.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.530+0000 7f27fdffb640 1 -- 172.21.15.5:0/1826214884 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281004b060 con 0x55a9555ec680 2024-09-15T22:22:40.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.530+0000 7f281eaaf740 1 -- 172.21.15.5:0/1826214884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a955554850 con 0x55a9555ec680 2024-09-15T22:22:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.531+0000 7f27fdffb640 1 -- 172.21.15.5:0/1826214884 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f28100038c0 con 0x55a9555ec680 2024-09-15T22:22:40.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.532+0000 7f27fdffb640 1 -- 172.21.15.5:0/1826214884 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2810052050 con 0x55a9555ec680 2024-09-15T22:22:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.532+0000 7f281eaaf740 10 client.27200 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.532+0000 7f281eaaf740 1 -- 172.21.15.5:0/1826214884 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a955666e90 con 0x55a9555ec680 2024-09-15T22:22:40.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.533+0000 7f27fdffb640 1 -- 172.21.15.5:0/1826214884 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a955554850 con 0x55a9555ec680 2024-09-15T22:22:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.533+0000 7f27fdffb640 1 -- 172.21.15.5:0/1826214884 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2810046410 con 0x55a9555ec680 2024-09-15T22:22:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.533+0000 7f281eaaf740 10 client.27200 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.533+0000 7f281eaaf740 10 client.27200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.533+0000 7f281eaaf740 20 client.27200 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.533+0000 7f281eaaf740 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a955667590 0x55a9556699b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:40.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.533+0000 7f281eaaf740 4 client.27200 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:22:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.533+0000 7f281eaaf740 1 -- 172.21.15.5:0/1826214884 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a95554ed60 con 0x55a955667590 2024-09-15T22:22:40.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.534+0000 7f281a472640 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a955667590 0x55a9556699b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.537+0000 7f281a472640 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a955667590 0x55a9556699b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12f274a18bfa073 server_cookie=35d7c92a7c51a050 in_seq=0 out_seq=0 2024-09-15T22:22:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.537+0000 7f27fdffb640 10 client.27200 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.537+0000 7f27fdffb640 1 -- 172.21.15.5:0/1826214884 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a95554ed60 con 0x55a955667590 2024-09-15T22:22:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.537+0000 7f27fdffb640 10 client.27200 handle_command_reply: tid=0 2024-09-15T22:22:40.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.537+0000 7f281eaaf740 1 client.27200 shutdown 2024-09-15T22:22:40.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.543+0000 7f281eaaf740 1 -- 172.21.15.5:0/1826214884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555ec680 msgr2=0x55a9555f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.543+0000 7f281eaaf740 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555ec680 0x55a9555f3100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2810009900 tx=0x7f2810004560 comp rx=0 tx=0).stop 2024-09-15T22:22:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.545+0000 7f281eaaf740 1 -- 172.21.15.5:0/1826214884 shutdown_connections 2024-09-15T22:22:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.545+0000 7f281eaaf740 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55a955667590 0x55a9556699b0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.545+0000 7f281eaaf740 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55a9555ec680 0x55a9555f3100 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.545+0000 7f281eaaf740 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55a9555e1dc0 0x55a9555f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.545+0000 7f281eaaf740 1 --2- 172.21.15.5:0/1826214884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9555e14a0 0x55a9555f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.545+0000 7f281eaaf740 1 -- 172.21.15.5:0/1826214884 >> 172.21.15.5:0/1826214884 conn(0x55a9555dda30 msgr2=0x55a9555eaa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.545+0000 7f27fdffb640 0 client.27200 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:40.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.549+0000 7f281eaaf740 1 -- 172.21.15.5:0/1826214884 shutdown_connections 2024-09-15T22:22:40.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.549+0000 7f281eaaf740 1 -- 172.21.15.5:0/1826214884 wait complete. 2024-09-15T22:22:40.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.550+0000 7f281eaaf740 20 client.27200 trim_cache size 0 max 16384 2024-09-15T22:22:40.555 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:22:40.555 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:22:40.555 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:22:40.555 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:22:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.560+0000 7f281c6d6640 1 -- 172.21.15.5:0/1082455236 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f28000a3480 msgr2=0x7f28000a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.560+0000 7f281c6d6640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f28000a3480 0x7f28000a5940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f281406f470 tx=0x7f280c009250 comp rx=0 tx=0).stop 2024-09-15T22:22:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.560+0000 7f281c6d6640 1 -- 172.21.15.5:0/1082455236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2814108450 msgr2=0x7f2814073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.560+0000 7f281c6d6640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2814108450 0x7f2814073c30 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f280400b4a0 tx=0x7f280400b960 comp rx=0 tx=0).stop 2024-09-15T22:22:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.561+0000 7f281c6d6640 1 -- 172.21.15.5:0/1082455236 shutdown_connections 2024-09-15T22:22:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.561+0000 7f281c6d6640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f28000a3480 0x7f28000a5940 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.561+0000 7f281c6d6640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f281410d530 0x7f281406e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.561+0000 7f281c6d6640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2814108e00 0x7f281406dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.561+0000 7f281c6d6640 1 --2- 172.21.15.5:0/1082455236 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2814108450 0x7f2814073c30 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.561+0000 7f281c6d6640 1 -- 172.21.15.5:0/1082455236 >> 172.21.15.5:0/1082455236 conn(0x7f2814077ba0 msgr2=0x7f2814078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.565+0000 7f281c6d6640 1 -- 172.21.15.5:0/1082455236 shutdown_connections 2024-09-15T22:22:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:40.565+0000 7f281c6d6640 1 -- 172.21.15.5:0/1082455236 wait complete. 2024-09-15T22:22:41.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:40 smithi099 ceph-mon[34892]: pgmap v430: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-09-15T22:22:41.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:40 smithi129 ceph-mon[35314]: pgmap v430: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-09-15T22:22:41.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:40 smithi005 ceph-mon[30944]: pgmap v430: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 4.6 MiB/s wr, 437 op/s 2024-09-15T22:22:41.444 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:22:41.444 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:22:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:42.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.667+0000 7f606550a640 1 -- 172.21.15.5:0/929032488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6060109450 msgr2=0x7f6060110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:42.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.667+0000 7f606550a640 1 --2- 172.21.15.5:0/929032488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6060109450 0x7f6060110250 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f6050009900 tx=0x7f605002a440 comp rx=0 tx=0).stop 2024-09-15T22:22:42.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.668+0000 7f606550a640 1 -- 172.21.15.5:0/929032488 shutdown_connections 2024-09-15T22:22:42.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.668+0000 7f606550a640 1 --2- 172.21.15.5:0/929032488 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f606011b140 0x7f606011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.668+0000 7f606550a640 1 --2- 172.21.15.5:0/929032488 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6060109830 0x7f6060109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.668+0000 7f606550a640 1 --2- 172.21.15.5:0/929032488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6060109450 0x7f6060110250 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.668+0000 7f606550a640 1 -- 172.21.15.5:0/929032488 >> 172.21.15.5:0/929032488 conn(0x7f606006e140 msgr2=0x7f606006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:42.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.669+0000 7f606550a640 1 -- 172.21.15.5:0/929032488 shutdown_connections 2024-09-15T22:22:42.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.669+0000 7f606550a640 1 -- 172.21.15.5:0/929032488 wait complete. 2024-09-15T22:22:42.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.669+0000 7f606550a640 1 Processor -- start 2024-09-15T22:22:42.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.670+0000 7f606550a640 1 -- start start 2024-09-15T22:22:42.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.670+0000 7f606550a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6060109450 0x7f6060118720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.670+0000 7f606550a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6060109830 0x7f60601117b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.670+0000 7f606550a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606011b140 0x7f6060111cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.670+0000 7f606550a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6060120ed0 con 0x7f6060109450 2024-09-15T22:22:42.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.670+0000 7f606550a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6060120d50 con 0x7f6060109830 2024-09-15T22:22:42.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.670+0000 7f606550a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6060121050 con 0x7f606011b140 2024-09-15T22:22:42.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.670+0000 7f605f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606011b140 0x7f6060111cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.670+0000 7f605f7fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606011b140 0x7f6060111cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:48692/0 (socket says 172.21.15.5:48692) 2024-09-15T22:22:42.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.670+0000 7f605f7fe640 1 -- 172.21.15.5:0/4109602303 learned_addr learned my addr 172.21.15.5:0/4109602303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:42.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.672+0000 7f605effd640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6060109450 0x7f6060118720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.672+0000 7f605f7fe640 1 -- 172.21.15.5:0/4109602303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6060109830 msgr2=0x7f60601117b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T22:22:42.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.672+0000 7f605f7fe640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6060109830 0x7f60601117b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.672+0000 7f605f7fe640 1 -- 172.21.15.5:0/4109602303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6060109450 msgr2=0x7f6060118720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:42.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.672+0000 7f605f7fe640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6060109450 0x7f6060118720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:42.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.672+0000 7f605f7fe640 1 -- 172.21.15.5:0/4109602303 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60601122e0 con 0x7f606011b140 2024-09-15T22:22:42.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.675+0000 7f605f7fe640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606011b140 0x7f6060111cf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f605800b9e0 tx=0x7f605800bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:42.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.676+0000 7f6047fff640 1 -- 172.21.15.5:0/4109602303 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6058019070 con 0x7f606011b140 2024-09-15T22:22:42.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.676+0000 7f606550a640 1 -- 172.21.15.5:0/4109602303 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60601125d0 con 0x7f606011b140 2024-09-15T22:22:42.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.676+0000 7f606550a640 1 -- 172.21.15.5:0/4109602303 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60601165c0 con 0x7f606011b140 2024-09-15T22:22:42.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.680+0000 7f6047fff640 1 -- 172.21.15.5:0/4109602303 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6058014440 con 0x7f606011b140 2024-09-15T22:22:42.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.680+0000 7f606550a640 1 -- 172.21.15.5:0/4109602303 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6060073b00 con 0x7f606011b140 2024-09-15T22:22:42.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.681+0000 7f6047fff640 1 -- 172.21.15.5:0/4109602303 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60580075b0 con 0x7f606011b140 2024-09-15T22:22:42.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.682+0000 7f6047fff640 1 -- 172.21.15.5:0/4109602303 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f605801c070 con 0x7f606011b140 2024-09-15T22:22:42.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.683+0000 7f6047fff640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f60480a3530 0x7f60480a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:42.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.683+0000 7f6047fff640 1 -- 172.21.15.5:0/4109602303 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f60580d0640 con 0x7f606011b140 2024-09-15T22:22:42.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.683+0000 7f605effd640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f60480a3530 0x7f60480a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:42.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.691+0000 7f605effd640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f60480a3530 0x7f60480a59f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6050000c00 tx=0x7f6050031490 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:42.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:42.691+0000 7f6047fff640 1 -- 172.21.15.5:0/4109602303 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6058006e20 con 0x7f606011b140 2024-09-15T22:22:43.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.048+0000 7f606550a640 1 -- 172.21.15.5:0/4109602303 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60601168e0 con 0x7f606011b140 2024-09-15T22:22:43.052 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:22:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.050+0000 7f6047fff640 1 -- 172.21.15.5:0/4109602303 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f605809cd50 con 0x7f606011b140 2024-09-15T22:22:43.053 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:22:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:22:43.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.054+0000 7f606550a640 1 -- 172.21.15.5:0/4109602303 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f60480a3530 msgr2=0x7f60480a59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:43.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.054+0000 7f606550a640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f60480a3530 0x7f60480a59f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6050000c00 tx=0x7f6050031490 comp rx=0 tx=0).stop 2024-09-15T22:22:43.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.054+0000 7f606550a640 1 -- 172.21.15.5:0/4109602303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606011b140 msgr2=0x7f6060111cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:43.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.054+0000 7f606550a640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606011b140 0x7f6060111cf0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f605800b9e0 tx=0x7f605800bea0 comp rx=0 tx=0).stop 2024-09-15T22:22:43.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.055+0000 7f606550a640 1 -- 172.21.15.5:0/4109602303 shutdown_connections 2024-09-15T22:22:43.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.055+0000 7f606550a640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f60480a3530 0x7f60480a59f0 secure :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f6050000c00 tx=0x7f6050031490 comp rx=0 tx=0).stop 2024-09-15T22:22:43.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.055+0000 7f606550a640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f606011b140 0x7f6060111cf0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:43.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.055+0000 7f606550a640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6060109830 0x7f60601117b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:43.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.055+0000 7f606550a640 1 --2- 172.21.15.5:0/4109602303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6060109450 0x7f6060118720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:43.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.055+0000 7f606550a640 1 -- 172.21.15.5:0/4109602303 >> 172.21.15.5:0/4109602303 conn(0x7f606006e140 msgr2=0x7f606011be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:43.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.057+0000 7f606550a640 1 -- 172.21.15.5:0/4109602303 shutdown_connections 2024-09-15T22:22:43.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:43.057+0000 7f606550a640 1 -- 172.21.15.5:0/4109602303 wait complete. 2024-09-15T22:22:43.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:42 smithi005 ceph-mon[30944]: pgmap v431: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 5.9 MiB/s wr, 580 op/s 2024-09-15T22:22:43.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:42 smithi099 ceph-mon[34892]: pgmap v431: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 5.9 MiB/s wr, 580 op/s 2024-09-15T22:22:43.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:42 smithi129 ceph-mon[35314]: pgmap v431: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 5.9 MiB/s wr, 580 op/s 2024-09-15T22:22:44.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:43 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4109602303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:44.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:43 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4109602303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:44.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:43 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4109602303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 795.0 s, 479.3 tps, lat 211.997 ms stddev 258.108 2024-09-15T22:22:45.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:44 smithi099 ceph-mon[34892]: pgmap v432: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.5 MiB/s wr, 461 op/s 2024-09-15T22:22:45.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:44 smithi129 ceph-mon[35314]: pgmap v432: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.5 MiB/s wr, 461 op/s 2024-09-15T22:22:45.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:44 smithi005 ceph-mon[30944]: pgmap v432: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 4.5 MiB/s wr, 461 op/s 2024-09-15T22:22:45.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:22:45.997 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:47.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.064+0000 7f5d5938b640 1 -- 172.21.15.5:0/3097875377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d541091f0 msgr2=0x7f5d541095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:47.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.064+0000 7f5d5938b640 1 --2- 172.21.15.5:0/3097875377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d541091f0 0x7f5d541095d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009900 tx=0x7f5d4402a4d0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.070+0000 7f5d5938b640 1 -- 172.21.15.5:0/3097875377 shutdown_connections 2024-09-15T22:22:47.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.070+0000 7f5d5938b640 1 --2- 172.21.15.5:0/3097875377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d54114ab0 0x7f5d54116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.070+0000 7f5d5938b640 1 --2- 172.21.15.5:0/3097875377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d54109b10 0x7f5d54114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.070+0000 7f5d5938b640 1 --2- 172.21.15.5:0/3097875377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d541091f0 0x7f5d541095d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.070+0000 7f5d5938b640 1 -- 172.21.15.5:0/3097875377 >> 172.21.15.5:0/3097875377 conn(0x7f5d5406e140 msgr2=0x7f5d5406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:47.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.077+0000 7f5d5938b640 1 -- 172.21.15.5:0/3097875377 shutdown_connections 2024-09-15T22:22:47.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.078+0000 7f5d5938b640 1 -- 172.21.15.5:0/3097875377 wait complete. 2024-09-15T22:22:47.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.078+0000 7f5d5938b640 1 Processor -- start 2024-09-15T22:22:47.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.078+0000 7f5d5938b640 1 -- start start 2024-09-15T22:22:47.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.079+0000 7f5d5938b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d541091f0 0x7f5d541a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.079+0000 7f5d5938b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d54109b10 0x7f5d541a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.079+0000 7f5d5938b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d54114ab0 0x7f5d541a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.079+0000 7f5d5938b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5411a850 con 0x7f5d54114ab0 2024-09-15T22:22:47.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.079+0000 7f5d5938b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5411a6d0 con 0x7f5d541091f0 2024-09-15T22:22:47.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.079+0000 7f5d5938b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5411a9d0 con 0x7f5d54109b10 2024-09-15T22:22:47.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.081+0000 7f5d4bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d54109b10 0x7f5d541a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.081+0000 7f5d4bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d54109b10 0x7f5d541a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:51082/0 (socket says 172.21.15.5:51082) 2024-09-15T22:22:47.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.081+0000 7f5d4bfff640 1 -- 172.21.15.5:0/147978320 learned_addr learned my addr 172.21.15.5:0/147978320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:47.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.081+0000 7f5d52ffd640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d541091f0 0x7f5d541a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.082+0000 7f5d4bfff640 1 -- 172.21.15.5:0/147978320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d541091f0 msgr2=0x7f5d541a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.082+0000 7f5d4bfff640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d541091f0 0x7f5d541a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.082+0000 7f5d4bfff640 1 -- 172.21.15.5:0/147978320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d54114ab0 msgr2=0x7f5d541a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.082+0000 7f5d4bfff640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d54114ab0 0x7f5d541a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.082+0000 7f5d4bfff640 1 -- 172.21.15.5:0/147978320 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d541a5d40 con 0x7f5d54109b10 2024-09-15T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.085+0000 7f5d4bfff640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d54109b10 0x7f5d541a12d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00ad30 tx=0x7f5d3c007590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.085+0000 7f5d50ff9640 1 -- 172.21.15.5:0/147978320 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c004220 con 0x7f5d54109b10 2024-09-15T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.085+0000 7f5d5938b640 1 -- 172.21.15.5:0/147978320 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d541a6030 con 0x7f5d54109b10 2024-09-15T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.085+0000 7f5d5938b640 1 -- 172.21.15.5:0/147978320 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d5410a300 con 0x7f5d54109b10 2024-09-15T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.086+0000 7f5d50ff9640 1 -- 172.21.15.5:0/147978320 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3c0043c0 con 0x7f5d54109b10 2024-09-15T22:22:47.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.086+0000 7f5d50ff9640 1 -- 172.21.15.5:0/147978320 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0035f0 con 0x7f5d54109b10 2024-09-15T22:22:47.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.091+0000 7f5d5b764740 1 Processor -- start 2024-09-15T22:22:47.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.091+0000 7f5d50ff9640 1 -- 172.21.15.5:0/147978320 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5d3c01f020 con 0x7f5d54109b10 2024-09-15T22:22:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.092+0000 7f5d5b764740 1 -- start start 2024-09-15T22:22:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.092+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e3e350 0x55b545e3c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.092+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e3d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.092+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e47b80 0x55b545e49f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.092+0000 7f5d5b764740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b545d9a2f0 con 0x55b545e47b80 2024-09-15T22:22:47.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.092+0000 7f5d5b764740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b545d41cf0 con 0x55b545e3cde0 2024-09-15T22:22:47.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.092+0000 7f5d5b764740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b545d962b0 con 0x55b545e3e350 2024-09-15T22:22:47.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.093+0000 7f5d52ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e3d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.093+0000 7f5d52ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e3d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:57856/0 (socket says 172.21.15.5:57856) 2024-09-15T22:22:47.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.093+0000 7f5d52ffd640 1 -- 172.21.15.5:0/3260053097 learned_addr learned my addr 172.21.15.5:0/3260053097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:47.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.093+0000 7f5d537fe640 1 --2- 172.21.15.5:0/3260053097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e3e350 0x55b545e3c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.094+0000 7f5d52ffd640 1 -- 172.21.15.5:0/3260053097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e3e350 msgr2=0x55b545e3c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:47.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.094+0000 7f5d52ffd640 1 --2- 172.21.15.5:0/3260053097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e3e350 0x55b545e3c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.094+0000 7f5d52ffd640 1 -- 172.21.15.5:0/3260053097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e47b80 msgr2=0x55b545e49f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:47.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.094+0000 7f5d52ffd640 1 --2- 172.21.15.5:0/3260053097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e47b80 0x55b545e49f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.094+0000 7f5d52ffd640 1 -- 172.21.15.5:0/3260053097 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b545d263a0 con 0x55b545e3cde0 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.094+0000 7f5d52ffd640 1 --2- 172.21.15.5:0/3260053097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e3d220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5d44005c40 tx=0x7f5d44031b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=259c73bd6e741728 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.095+0000 7f5d497fa640 1 -- 172.21.15.5:0/3260053097 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4403f070 con 0x55b545e3cde0 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.095+0000 7f5d50ff9640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d340a35c0 0x7f5d340a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.096+0000 7f5d50ff9640 1 -- 172.21.15.5:0/147978320 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f5d3c0d3e70 con 0x7f5d54109b10 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.096+0000 7f5d497fa640 1 -- 172.21.15.5:0/3260053097 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d440035f0 con 0x55b545e3cde0 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.097+0000 7f5d5b764740 1 -- 172.21.15.5:0/3260053097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 msgr2=0x55b545e3d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.097+0000 7f5d5b764740 1 --2- 172.21.15.5:0/3260053097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e3d220 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5d44005c40 tx=0x7f5d44031b00 comp rx=0 tx=0).stop 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.101+0000 7f5d537fe640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d340a35c0 0x7f5d340a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.103+0000 7f5d5b764740 1 -- 172.21.15.5:0/3260053097 shutdown_connections 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.103+0000 7f5d5b764740 1 --2- 172.21.15.5:0/3260053097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e47b80 0x55b545e49f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.103+0000 7f5d5b764740 1 --2- 172.21.15.5:0/3260053097 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e3d220 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.103+0000 7f5d5b764740 1 --2- 172.21.15.5:0/3260053097 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e3e350 0x55b545e3c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.103+0000 7f5d5b764740 1 -- 172.21.15.5:0/3260053097 >> 172.21.15.5:0/3260053097 conn(0x55b545dfd310 msgr2=0x55b545e39470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.104+0000 7f5d5b764740 1 -- 172.21.15.5:0/3260053097 shutdown_connections 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.106+0000 7f5d5b764740 1 -- 172.21.15.5:0/3260053097 wait complete. 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.106+0000 7f5d5b764740 1 -- start start 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.106+0000 7f5d537fe640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d340a35c0 0x7f5d340a5a80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5d5406b430 tx=0x7f5d4c00d040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.106+0000 7f5d5b764740 1 Processor -- start 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.111+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e389a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.111+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e46b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.111+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 0x55b545e47300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.111+0000 7f5d5b764740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b545e309f0 con 0x55b545e3e350 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.111+0000 7f5d5b764740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55b545d55060 con 0x55b545e3cde0 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.111+0000 7f5d5b764740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55b545e30b70 con 0x55b545e47b80 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.111+0000 7f5d5b764740 10 client.0 _ll_register_callbacks cb 0x55b545aa88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.113+0000 7f5d4bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e46b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.113+0000 7f5d4bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e46b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59770/0 (socket says 172.21.15.5:59770) 2024-09-15T22:22:47.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.113+0000 7f5d4bfff640 1 -- 172.21.15.5:0/949968720 learned_addr learned my addr 172.21.15.5:0/949968720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.115+0000 7f5d537fe640 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 0x55b545e47300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.116+0000 7f5d52ffd640 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e389a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.116+0000 7f5d537fe640 1 -- 172.21.15.5:0/949968720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 msgr2=0x55b545e389a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.116+0000 7f5d537fe640 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e389a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.116+0000 7f5d537fe640 1 -- 172.21.15.5:0/949968720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 msgr2=0x55b545e46b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.116+0000 7f5d537fe640 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e46b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.116+0000 7f5d537fe640 1 -- 172.21.15.5:0/949968720 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b545b0b3c0 con 0x55b545e47b80 2024-09-15T22:22:47.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.116+0000 7f5d537fe640 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 0x55b545e47300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5d540743f0 tx=0x7f5d4c004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e84ef5830e78c14b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:47.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.117+0000 7f5d427fc640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:22:47.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.120+0000 7f5d427fc640 1 -- 172.21.15.5:0/949968720 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c025060 con 0x55b545e47b80 2024-09-15T22:22:47.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.120+0000 7f5d427fc640 1 -- 172.21.15.5:0/949968720 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f5d4c0203f0 con 0x55b545e47b80 2024-09-15T22:22:47.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.122+0000 7f5d5b764740 1 -- 172.21.15.5:0/949968720 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d440035f0 con 0x55b545e47b80 2024-09-15T22:22:47.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.122+0000 7f5d427fc640 1 -- 172.21.15.5:0/949968720 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d4c0205f0 con 0x55b545e47b80 2024-09-15T22:22:47.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.125+0000 7f5d5b764740 10 client.27136 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:47.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.125+0000 7f5d5b764740 1 -- 172.21.15.5:0/949968720 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b545e5ec40 con 0x55b545e47b80 2024-09-15T22:22:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.125+0000 7f5d427fc640 1 -- 172.21.15.5:0/949968720 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d440035f0 con 0x55b545e47b80 2024-09-15T22:22:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.125+0000 7f5d427fc640 1 -- 172.21.15.5:0/949968720 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5d4c013500 con 0x55b545e47b80 2024-09-15T22:22:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.125+0000 7f5d5b764740 10 client.27136 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.125+0000 7f5d5b764740 10 client.27136 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:22:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.126+0000 7f5d5b764740 20 client.27136 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.126+0000 7f5d5b764740 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55b545e60350 0x55b545e62770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.126+0000 7f5d5b764740 4 client.27136 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.126+0000 7f5d5b764740 1 -- 172.21.15.5:0/949968720 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b545da7860 con 0x55b545e60350 2024-09-15T22:22:47.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.129+0000 7f5d52ffd640 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55b545e60350 0x55b545e62770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.129+0000 7f5d52ffd640 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55b545e60350 0x55b545e62770 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fae9c07e255e02 server_cookie=12ff6e61b35221d2 in_seq=0 out_seq=0 2024-09-15T22:22:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.129+0000 7f5d427fc640 10 client.27136 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:22:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.130+0000 7f5d427fc640 1 -- 172.21.15.5:0/949968720 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b545da7860 con 0x55b545e60350 2024-09-15T22:22:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.130+0000 7f5d427fc640 10 client.27136 handle_command_reply: tid=0 2024-09-15T22:22:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.131+0000 7f5d5b764740 1 client.27136 shutdown 2024-09-15T22:22:47.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.136+0000 7f5d5b764740 1 -- 172.21.15.5:0/949968720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 msgr2=0x55b545e47300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:47.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.136+0000 7f5d5b764740 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 0x55b545e47300 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5d540743f0 tx=0x7f5d4c004b40 comp rx=0 tx=0).stop 2024-09-15T22:22:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.142+0000 7f5d5b764740 1 -- 172.21.15.5:0/949968720 shutdown_connections 2024-09-15T22:22:47.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:46 smithi005 ceph-mon[30944]: pgmap v433: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.8 MiB/s wr, 485 op/s 2024-09-15T22:22:47.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.142+0000 7f5d5b764740 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55b545e60350 0x55b545e62770 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.142+0000 7f5d5b764740 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 0x55b545e47300 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.142+0000 7f5d5b764740 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e46b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.142+0000 7f5d5b764740 1 --2- 172.21.15.5:0/949968720 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e389a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.142+0000 7f5d5b764740 1 -- 172.21.15.5:0/949968720 >> 172.21.15.5:0/949968720 conn(0x55b545dfd310 msgr2=0x55b545e3f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:47.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.142+0000 7f5d427fc640 0 client.27136 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:22:47.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.143+0000 7f5d5b764740 1 -- 172.21.15.5:0/949968720 shutdown_connections 2024-09-15T22:22:47.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.143+0000 7f5d5b764740 1 -- 172.21.15.5:0/949968720 wait complete. 2024-09-15T22:22:47.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.146+0000 7f5d5b764740 20 client.27136 trim_cache size 0 max 16384 2024-09-15T22:22:47.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.172+0000 7f5d5b764740 1 Processor -- start 2024-09-15T22:22:47.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.172+0000 7f5d5b764740 1 -- start start 2024-09-15T22:22:47.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.173+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e39620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.173+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e46b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.173+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 0x55b545e68040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.173+0000 7f5d5b764740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d380129e0 con 0x55b545e3e350 2024-09-15T22:22:47.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.173+0000 7f5d5b764740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d380102b0 con 0x55b545e3cde0 2024-09-15T22:22:47.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.173+0000 7f5d5b764740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3800e120 con 0x55b545e47b80 2024-09-15T22:22:47.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.173+0000 7f5d537fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 0x55b545e68040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.173+0000 7f5d537fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 0x55b545e68040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:51114/0 (socket says 172.21.15.5:51114) 2024-09-15T22:22:47.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.173+0000 7f5d537fe640 1 -- 172.21.15.5:0/4236505694 learned_addr learned my addr 172.21.15.5:0/4236505694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:47.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.174+0000 7f5d52ffd640 1 --2- 172.21.15.5:0/4236505694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e39620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.174+0000 7f5d4bfff640 1 --2- 172.21.15.5:0/4236505694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e46b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.174+0000 7f5d4bfff640 1 -- 172.21.15.5:0/4236505694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 msgr2=0x55b545e68040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:47.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.174+0000 7f5d4bfff640 1 --2- 172.21.15.5:0/4236505694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 0x55b545e68040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.174+0000 7f5d4bfff640 1 -- 172.21.15.5:0/4236505694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 msgr2=0x55b545e39620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:47.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.174+0000 7f5d4bfff640 1 --2- 172.21.15.5:0/4236505694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e39620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.174+0000 7f5d4bfff640 1 -- 172.21.15.5:0/4236505694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b545e4a2e0 con 0x55b545e3e350 2024-09-15T22:22:47.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.180+0000 7f5d4bfff640 1 --2- 172.21.15.5:0/4236505694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e46b60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005fd0 tx=0x7f5d3c09f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddd09dbddf11ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:47.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.182+0000 7f5d48ff9640 1 -- 172.21.15.5:0/4236505694 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0a40a0 con 0x55b545e3e350 2024-09-15T22:22:47.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.182+0000 7f5d48ff9640 1 -- 172.21.15.5:0/4236505694 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3c0a8960 con 0x55b545e3e350 2024-09-15T22:22:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.182+0000 7f5d48ff9640 1 -- 172.21.15.5:0/4236505694 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c01fbc0 con 0x55b545e3e350 2024-09-15T22:22:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.183+0000 7f5d5b764740 1 -- 172.21.15.5:0/4236505694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 msgr2=0x55b545e46b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:47.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.183+0000 7f5d5b764740 1 --2- 172.21.15.5:0/4236505694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e46b60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005fd0 tx=0x7f5d3c09f650 comp rx=0 tx=0).stop 2024-09-15T22:22:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.188+0000 7f5d5b764740 1 -- 172.21.15.5:0/4236505694 shutdown_connections 2024-09-15T22:22:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.188+0000 7f5d5b764740 1 --2- 172.21.15.5:0/4236505694 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e47b80 0x55b545e68040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.188+0000 7f5d5b764740 1 --2- 172.21.15.5:0/4236505694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e46b60 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.188+0000 7f5d5b764740 1 --2- 172.21.15.5:0/4236505694 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e3cde0 0x55b545e39620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.188+0000 7f5d5b764740 1 -- 172.21.15.5:0/4236505694 >> 172.21.15.5:0/4236505694 conn(0x55b545dfd310 msgr2=0x55b545e38f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.188+0000 7f5d5b764740 1 -- 172.21.15.5:0/4236505694 shutdown_connections 2024-09-15T22:22:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.188+0000 7f5d5b764740 1 -- 172.21.15.5:0/4236505694 wait complete. 2024-09-15T22:22:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.188+0000 7f5d5b764740 1 -- start start 2024-09-15T22:22:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.188+0000 7f5d5b764740 1 Processor -- start 2024-09-15T22:22:47.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.189+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e3cde0 0x55b545e42060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.189+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.189+0000 7f5d5b764740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e47b80 0x55b545e42ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.189+0000 7f5d5b764740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3800e770 con 0x55b545e3e350 2024-09-15T22:22:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.189+0000 7f5d5b764740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3800fbe0 con 0x55b545e47b80 2024-09-15T22:22:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.189+0000 7f5d5b764740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3800ee40 con 0x55b545e3cde0 2024-09-15T22:22:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.189+0000 7f5d5b764740 10 client.0 _ll_register_callbacks cb 0x7f5d4c0098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.190+0000 7f5d4bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.190+0000 7f5d4bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59790/0 (socket says 172.21.15.5:59790) 2024-09-15T22:22:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.190+0000 7f5d4bfff640 1 -- 172.21.15.5:0/4023399870 learned_addr learned my addr 172.21.15.5:0/4023399870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.190+0000 7f5d4bfff640 1 -- 172.21.15.5:0/4023399870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e3cde0 msgr2=0x55b545e42060 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:22:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.190+0000 7f5d4bfff640 1 --2- 172.21.15.5:0/4023399870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e3cde0 0x55b545e42060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.190+0000 7f5d4bfff640 1 -- 172.21.15.5:0/4023399870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e47b80 msgr2=0x55b545e42ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:22:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.190+0000 7f5d4bfff640 1 --2- 172.21.15.5:0/4023399870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e47b80 0x55b545e42ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.190+0000 7f5d4bfff640 1 -- 172.21.15.5:0/4023399870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b545eca090 con 0x55b545e3e350 2024-09-15T22:22:47.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.191+0000 7f5d4bfff640 1 --2- 172.21.15.5:0/4023399870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e425a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0a8430 tx=0x7f5d3c0a41e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67c21e7453f1263 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.192+0000 7f5d427fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:22:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.192+0000 7f5d427fc640 1 -- 172.21.15.5:0/4023399870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0bc020 con 0x55b545e3e350 2024-09-15T22:22:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.192+0000 7f5d5b764740 1 -- 172.21.15.5:0/4023399870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d3c0a8960 con 0x55b545e3e350 2024-09-15T22:22:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.192+0000 7f5d427fc640 1 -- 172.21.15.5:0/4023399870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f5d3c01fbc0 con 0x55b545e3e350 2024-09-15T22:22:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.192+0000 7f5d427fc640 1 -- 172.21.15.5:0/4023399870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d3c01fdc0 con 0x55b545e3e350 2024-09-15T22:22:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.193+0000 7f5d5b764740 10 client.17721 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.193+0000 7f5d5b764740 1 -- 172.21.15.5:0/4023399870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b545ecb370 con 0x55b545e3e350 2024-09-15T22:22:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.193+0000 7f5d427fc640 1 -- 172.21.15.5:0/4023399870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3c0a8960 con 0x55b545e3e350 2024-09-15T22:22:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.194+0000 7f5d427fc640 1 -- 172.21.15.5:0/4023399870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5d3c09ebb0 con 0x55b545e3e350 2024-09-15T22:22:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.194+0000 7f5d5b764740 10 client.17721 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.194+0000 7f5d5b764740 10 client.17721 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:22:47.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.194+0000 7f5d5b764740 20 client.17721 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.194+0000 7f5d5b764740 1 --2- 172.21.15.5:0/4023399870 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55b545ecb9e0 0x55b545ecde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.194+0000 7f5d5b764740 4 client.17721 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:22:47.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.194+0000 7f5d5b764740 1 -- 172.21.15.5:0/4023399870 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b545da1d70 con 0x55b545ecb9e0 2024-09-15T22:22:47.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.195+0000 7f5d52ffd640 1 --2- 172.21.15.5:0/4023399870 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55b545ecb9e0 0x55b545ecde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:47.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.204+0000 7f5d52ffd640 1 --2- 172.21.15.5:0/4023399870 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55b545ecb9e0 0x55b545ecde00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=317de2c3396e0b6e server_cookie=ef173eab81cda775 in_seq=0 out_seq=0 2024-09-15T22:22:47.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.204+0000 7f5d427fc640 10 client.17721 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:22:47.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.208+0000 7f5d427fc640 1 -- 172.21.15.5:0/4023399870 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b545da1d70 con 0x55b545ecb9e0 2024-09-15T22:22:47.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.208+0000 7f5d427fc640 10 client.17721 handle_command_reply: tid=0 2024-09-15T22:22:47.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.208+0000 7f5d5b764740 1 client.17721 shutdown 2024-09-15T22:22:47.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.215+0000 7f5d5b764740 1 -- 172.21.15.5:0/4023399870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 msgr2=0x55b545e425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:47.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.215+0000 7f5d5b764740 1 --2- 172.21.15.5:0/4023399870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e425a0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c0a8430 tx=0x7f5d3c0a41e0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.219+0000 7f5d5b764740 1 -- 172.21.15.5:0/4023399870 shutdown_connections 2024-09-15T22:22:47.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.219+0000 7f5d5b764740 1 --2- 172.21.15.5:0/4023399870 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55b545ecb9e0 0x55b545ecde00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.219+0000 7f5d5b764740 1 --2- 172.21.15.5:0/4023399870 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55b545e47b80 0x55b545e42ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.219+0000 7f5d5b764740 1 --2- 172.21.15.5:0/4023399870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b545e3e350 0x55b545e425a0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.219+0000 7f5d5b764740 1 --2- 172.21.15.5:0/4023399870 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55b545e3cde0 0x55b545e42060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.219+0000 7f5d5b764740 1 -- 172.21.15.5:0/4023399870 >> 172.21.15.5:0/4023399870 conn(0x55b545dfd310 msgr2=0x55b545e374b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:47.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.219+0000 7f5d427fc640 0 client.17721 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:22:47.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.221+0000 7f5d5b764740 1 -- 172.21.15.5:0/4023399870 shutdown_connections 2024-09-15T22:22:47.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.221+0000 7f5d5b764740 1 -- 172.21.15.5:0/4023399870 wait complete. 2024-09-15T22:22:47.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.221+0000 7f5d5b764740 20 client.17721 trim_cache size 0 max 16384 2024-09-15T22:22:47.224 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:22:47.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.222+0000 7f5d5938b640 1 -- 172.21.15.5:0/147978320 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d340a35c0 msgr2=0x7f5d340a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:47.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.222+0000 7f5d5938b640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d340a35c0 0x7f5d340a5a80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5d5406b430 tx=0x7f5d4c00d040 comp rx=0 tx=0).stop 2024-09-15T22:22:47.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.222+0000 7f5d5938b640 1 -- 172.21.15.5:0/147978320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d54109b10 msgr2=0x7f5d541a12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:47.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.222+0000 7f5d5938b640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d54109b10 0x7f5d541a12d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00ad30 tx=0x7f5d3c007590 comp rx=0 tx=0).stop 2024-09-15T22:22:47.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.222+0000 7f5d5938b640 1 -- 172.21.15.5:0/147978320 shutdown_connections 2024-09-15T22:22:47.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.222+0000 7f5d5938b640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5d340a35c0 0x7f5d340a5a80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.222+0000 7f5d5938b640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d54114ab0 0x7f5d541a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.222+0000 7f5d5938b640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f5d54109b10 0x7f5d541a12d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.222+0000 7f5d5938b640 1 --2- 172.21.15.5:0/147978320 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f5d541091f0 0x7f5d541a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:47.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.222+0000 7f5d5938b640 1 -- 172.21.15.5:0/147978320 >> 172.21.15.5:0/147978320 conn(0x7f5d5406e140 msgr2=0x7f5d54071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:47.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.227+0000 7f5d5938b640 1 -- 172.21.15.5:0/147978320 shutdown_connections 2024-09-15T22:22:47.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:47.228+0000 7f5d5938b640 1 -- 172.21.15.5:0/147978320 wait complete. 2024-09-15T22:22:47.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:46 smithi099 ceph-mon[34892]: pgmap v433: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.8 MiB/s wr, 485 op/s 2024-09-15T22:22:47.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:46 smithi129 ceph-mon[35314]: pgmap v433: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.8 MiB/s wr, 485 op/s 2024-09-15T22:22:48.207 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.5 2024-09-15T22:22:48.208 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:22:49.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:48 smithi099 ceph-mon[34892]: pgmap v434: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 505 op/s 2024-09-15T22:22:49.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:48 smithi129 ceph-mon[35314]: pgmap v434: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 505 op/s 2024-09-15T22:22:49.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:48 smithi005 ceph-mon[30944]: pgmap v434: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 5.1 MiB/s wr, 505 op/s 2024-09-15T22:22:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 800.0 s, 525.0 tps, lat 189.007 ms stddev 264.944 2024-09-15T22:22:49.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:22:50.154 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.817+0000 7fc51e97a640 1 -- 172.21.15.5:0/3433950477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc510095b10 msgr2=0x7fc510095f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.817+0000 7fc51e97a640 1 --2- 172.21.15.5:0/3433950477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc510095b10 0x7fc510095f70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc514009890 tx=0x7fc51402a3b0 comp rx=0 tx=0).stop 2024-09-15T22:22:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.822+0000 7fc51e97a640 1 -- 172.21.15.5:0/3433950477 shutdown_connections 2024-09-15T22:22:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.822+0000 7fc51e97a640 1 --2- 172.21.15.5:0/3433950477 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc5100964b0 0x7fc5100a2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.822+0000 7fc51e97a640 1 --2- 172.21.15.5:0/3433950477 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc510095b10 0x7fc510095f70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.822+0000 7fc51e97a640 1 --2- 172.21.15.5:0/3433950477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc51009b640 0x7fc51009ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:50.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.822+0000 7fc51e97a640 1 -- 172.21.15.5:0/3433950477 >> 172.21.15.5:0/3433950477 conn(0x7fc51000b3b0 msgr2=0x7fc51000b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:50.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.824+0000 7fc51e97a640 1 -- 172.21.15.5:0/3433950477 shutdown_connections 2024-09-15T22:22:50.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.824+0000 7fc51e97a640 1 -- 172.21.15.5:0/3433950477 wait complete. 2024-09-15T22:22:50.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51e97a640 1 Processor -- start 2024-09-15T22:22:50.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51e97a640 1 -- start start 2024-09-15T22:22:50.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51e97a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc510095b10 0x7fc5100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:50.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51e97a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5100964b0 0x7fc5100a5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:50.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51e97a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51009b640 0x7fc5100aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:50.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51d177640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5100964b0 0x7fc5100a5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:50.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51d978640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc510095b10 0x7fc5100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:50.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51d978640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc510095b10 0x7fc5100a50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:51150/0 (socket says 172.21.15.5:51150) 2024-09-15T22:22:50.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51d177640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5100964b0 0x7fc5100a5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59800/0 (socket says 172.21.15.5:59800) 2024-09-15T22:22:50.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51d978640 1 -- 172.21.15.5:0/662621267 learned_addr learned my addr 172.21.15.5:0/662621267 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:50.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5100a3850 con 0x7fc5100964b0 2024-09-15T22:22:50.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc5100a36d0 con 0x7fc51009b640 2024-09-15T22:22:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.825+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5100a39d0 con 0x7fc510095b10 2024-09-15T22:22:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.826+0000 7fc51e179640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51009b640 0x7fc5100aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.826+0000 7fc51d978640 1 -- 172.21.15.5:0/662621267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51009b640 msgr2=0x7fc5100aa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.826+0000 7fc51d978640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51009b640 0x7fc5100aa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.826+0000 7fc51d978640 1 -- 172.21.15.5:0/662621267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5100964b0 msgr2=0x7fc5100a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:50.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.826+0000 7fc51d978640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5100964b0 0x7fc5100a5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.826+0000 7fc51d978640 1 -- 172.21.15.5:0/662621267 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5100ab130 con 0x7fc510095b10 2024-09-15T22:22:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.827+0000 7fc51d978640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc510095b10 0x7fc5100a50f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc50800ba60 tx=0x7fc50800bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.828+0000 7fc50effd640 1 -- 172.21.15.5:0/662621267 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508009a80 con 0x7fc510095b10 2024-09-15T22:22:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.828+0000 7fc50effd640 1 -- 172.21.15.5:0/662621267 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc50800d400 con 0x7fc510095b10 2024-09-15T22:22:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.828+0000 7fc50effd640 1 -- 172.21.15.5:0/662621267 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508004d00 con 0x7fc510095b10 2024-09-15T22:22:50.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.828+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5100ab3c0 con 0x7fc510095b10 2024-09-15T22:22:50.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.828+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc510147660 con 0x7fc510095b10 2024-09-15T22:22:50.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.831+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4fc005290 con 0x7fc510095b10 2024-09-15T22:22:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.832+0000 7fc50effd640 1 -- 172.21.15.5:0/662621267 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc508005020 con 0x7fc510095b10 2024-09-15T22:22:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.833+0000 7fc50effd640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5040a35c0 0x7fc5040a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.833+0000 7fc50effd640 1 -- 172.21.15.5:0/662621267 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7fc5080cf650 con 0x7fc510095b10 2024-09-15T22:22:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.836+0000 7fc51d177640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5040a35c0 0x7fc5040a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.841+0000 7fc50effd640 1 -- 172.21.15.5:0/662621267 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc50809bbd0 con 0x7fc510095b10 2024-09-15T22:22:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:50.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:50.851+0000 7fc51d177640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5040a35c0 0x7fc5040a5a80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc514005d00 tx=0x7fc51400b040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:50.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:50 smithi005 ceph-mon[30944]: pgmap v435: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 426 op/s 2024-09-15T22:22:51.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.203+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4fc005850 con 0x7fc510095b10 2024-09-15T22:22:51.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.204+0000 7fc50effd640 1 -- 172.21.15.5:0/662621267 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fc50809b4d0 con 0x7fc510095b10 2024-09-15T22:22:51.207 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:22:51.207 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:22:51.208 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5040a35c0 msgr2=0x7fc5040a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5040a35c0 0x7fc5040a5a80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc514005d00 tx=0x7fc51400b040 comp rx=0 tx=0).stop 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc510095b10 msgr2=0x7fc5100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc510095b10 0x7fc5100a50f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc50800ba60 tx=0x7fc50800bf20 comp rx=0 tx=0).stop 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 shutdown_connections 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5040a35c0 0x7fc5040a5a80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc51009b640 0x7fc5100aa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5100964b0 0x7fc5100a5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 --2- 172.21.15.5:0/662621267 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc510095b10 0x7fc5100a50f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 >> 172.21.15.5:0/662621267 conn(0x7fc51000b3b0 msgr2=0x7fc5100934a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 shutdown_connections 2024-09-15T22:22:51.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:51.207+0000 7fc51e97a640 1 -- 172.21.15.5:0/662621267 wait complete. 2024-09-15T22:22:51.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:50 smithi099 ceph-mon[34892]: pgmap v435: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 426 op/s 2024-09-15T22:22:51.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:50 smithi129 ceph-mon[35314]: pgmap v435: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 4.2 MiB/s wr, 426 op/s 2024-09-15T22:22:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.054+0000 7f79d08d5640 1 -- 172.21.15.5:0/3380891521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79cc102780 msgr2=0x7f79cc1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.054+0000 7f79d08d5640 1 --2- 172.21.15.5:0/3380891521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79cc102780 0x7f79cc1067d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f79c0009860 tx=0x7f79c002a340 comp rx=0 tx=0).stop 2024-09-15T22:22:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.054+0000 7f79d08d5640 1 -- 172.21.15.5:0/3380891521 shutdown_connections 2024-09-15T22:22:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.054+0000 7f79d08d5640 1 --2- 172.21.15.5:0/3380891521 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79cc106f00 0x7f79cc10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.054+0000 7f79d08d5640 1 --2- 172.21.15.5:0/3380891521 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79cc102780 0x7f79cc1067d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.054+0000 7f79d08d5640 1 --2- 172.21.15.5:0/3380891521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79cc101dd0 0x7f79cc1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.054+0000 7f79d08d5640 1 -- 172.21.15.5:0/3380891521 >> 172.21.15.5:0/3380891521 conn(0x7f79cc0fd890 msgr2=0x7f79cc0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.055+0000 7f79d08d5640 1 -- 172.21.15.5:0/3380891521 shutdown_connections 2024-09-15T22:22:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.055+0000 7f79d08d5640 1 -- 172.21.15.5:0/3380891521 wait complete. 2024-09-15T22:22:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.055+0000 7f79d08d5640 1 Processor -- start 2024-09-15T22:22:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.055+0000 7f79d08d5640 1 -- start start 2024-09-15T22:22:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.056+0000 7f79d08d5640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79cc101dd0 0x7f79cc196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.056+0000 7f79d08d5640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79cc102780 0x7f79cc1965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.056+0000 7f79d08d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79cc106f00 0x7f79cc19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.056+0000 7f79d08d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc10fd50 con 0x7f79cc106f00 2024-09-15T22:22:52.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.056+0000 7f79d08d5640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc10fbd0 con 0x7f79cc101dd0 2024-09-15T22:22:52.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.056+0000 7f79d08d5640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc10fed0 con 0x7f79cc102780 2024-09-15T22:22:52.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.056+0000 7f79ca59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79cc101dd0 0x7f79cc196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.056+0000 7f79ca59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79cc101dd0 0x7f79cc196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:57904/0 (socket says 172.21.15.5:57904) 2024-09-15T22:22:52.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.056+0000 7f79ca59c640 1 -- 172.21.15.5:0/3741826384 learned_addr learned my addr 172.21.15.5:0/3741826384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:52.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.056+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79cc102780 0x7f79cc1965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.057+0000 7f79cad9d640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79cc106f00 0x7f79cc19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.057+0000 7f79c9d9b640 1 -- 172.21.15.5:0/3741826384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79cc101dd0 msgr2=0x7f79cc196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.057+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79cc101dd0 0x7f79cc196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.057+0000 7f79c9d9b640 1 -- 172.21.15.5:0/3741826384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79cc106f00 msgr2=0x7f79cc19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.057+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79cc106f00 0x7f79cc19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.057+0000 7f79c9d9b640 1 -- 172.21.15.5:0/3741826384 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79cc19b020 con 0x7f79cc102780 2024-09-15T22:22:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.058+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79cc102780 0x7f79cc1965b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f79c002a860 tx=0x7f79c0033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.063+0000 7f79bb7fe640 1 -- 172.21.15.5:0/3741826384 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0037040 con 0x7f79cc102780 2024-09-15T22:22:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.063+0000 7f79bb7fe640 1 -- 172.21.15.5:0/3741826384 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c0040070 con 0x7f79cc102780 2024-09-15T22:22:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.064+0000 7f79bb7fe640 1 -- 172.21.15.5:0/3741826384 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00046c0 con 0x7f79cc102780 2024-09-15T22:22:52.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.066+0000 7f79d08d5640 1 -- 172.21.15.5:0/3741826384 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79cc19b2b0 con 0x7f79cc102780 2024-09-15T22:22:52.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.066+0000 7f79d08d5640 1 -- 172.21.15.5:0/3741826384 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79cc19b650 con 0x7f79cc102780 2024-09-15T22:22:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.072+0000 7f79bb7fe640 1 -- 172.21.15.5:0/3741826384 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f79c0030550 con 0x7f79cc102780 2024-09-15T22:22:52.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.080+0000 7f79d2cae740 1 Processor -- start 2024-09-15T22:22:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.080+0000 7f79bb7fe640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f79b40a3580 0x7f79b40a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.081+0000 7f79bb7fe640 1 -- 172.21.15.5:0/3741826384 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f79c00eff20 con 0x7f79cc102780 2024-09-15T22:22:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.081+0000 7f79d2cae740 1 -- start start 2024-09-15T22:22:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.082+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4e350 0x563ae3e4c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.082+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e4cde0 0x563ae3e4d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.082+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e57b80 0x563ae3e59f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.082+0000 7f79d2cae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ae3daa2f0 con 0x563ae3e4e350 2024-09-15T22:22:52.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.082+0000 7f79d2cae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563ae3d51cf0 con 0x563ae3e4cde0 2024-09-15T22:22:52.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.082+0000 7f79d2cae740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563ae3da62b0 con 0x563ae3e57b80 2024-09-15T22:22:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.084+0000 7f79cad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e57b80 0x563ae3e59f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.084+0000 7f79cad9d640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f79b40a3580 0x7f79b40a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.084+0000 7f79cad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e57b80 0x563ae3e59f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:51164/0 (socket says 172.21.15.5:51164) 2024-09-15T22:22:52.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.084+0000 7f79cad9d640 1 -- 172.21.15.5:0/1139156733 learned_addr learned my addr 172.21.15.5:0/1139156733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.085+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/1139156733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e4cde0 0x563ae3e4d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.085+0000 7f79cad9d640 1 -- 172.21.15.5:0/1139156733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e4cde0 msgr2=0x563ae3e4d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:52.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.085+0000 7f79cad9d640 1 --2- 172.21.15.5:0/1139156733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e4cde0 0x563ae3e4d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.085+0000 7f79cad9d640 1 -- 172.21.15.5:0/1139156733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4e350 msgr2=0x563ae3e4c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.085+0000 7f79cad9d640 1 --2- 172.21.15.5:0/1139156733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4e350 0x563ae3e4c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.085+0000 7f79cad9d640 1 -- 172.21.15.5:0/1139156733 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae3d363a0 con 0x563ae3e57b80 2024-09-15T22:22:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.085+0000 7f79cad9d640 1 --2- 172.21.15.5:0/1139156733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e57b80 0x563ae3e59f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f79c400b890 tx=0x7f79c400bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb346af86fe6e28c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:52.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.087+0000 7f79b8ff9640 1 -- 172.21.15.5:0/1139156733 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4017830 con 0x563ae3e57b80 2024-09-15T22:22:52.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.091+0000 7f79b8ff9640 1 -- 172.21.15.5:0/1139156733 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c401b470 con 0x563ae3e57b80 2024-09-15T22:22:52.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.091+0000 7f79b8ff9640 1 -- 172.21.15.5:0/1139156733 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c401a630 con 0x563ae3e57b80 2024-09-15T22:22:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.092+0000 7f79d2cae740 1 -- 172.21.15.5:0/1139156733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e57b80 msgr2=0x563ae3e59f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.092+0000 7f79d2cae740 1 --2- 172.21.15.5:0/1139156733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e57b80 0x563ae3e59f70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f79c400b890 tx=0x7f79c400bd50 comp rx=0 tx=0).stop 2024-09-15T22:22:52.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.093+0000 7f79cad9d640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f79b40a3580 0x7f79b40a5a40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f79c4017fd0 tx=0x7f79c400d530 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:52.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.095+0000 7f79d2cae740 1 -- 172.21.15.5:0/1139156733 shutdown_connections 2024-09-15T22:22:52.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.095+0000 7f79d2cae740 1 --2- 172.21.15.5:0/1139156733 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e57b80 0x563ae3e59f70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.095+0000 7f79d2cae740 1 --2- 172.21.15.5:0/1139156733 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e4cde0 0x563ae3e4d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.095+0000 7f79d2cae740 1 --2- 172.21.15.5:0/1139156733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4e350 0x563ae3e4c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.095+0000 7f79d2cae740 1 -- 172.21.15.5:0/1139156733 >> 172.21.15.5:0/1139156733 conn(0x563ae3e0d310 msgr2=0x563ae3e49470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:52.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.100+0000 7f79d2cae740 1 -- 172.21.15.5:0/1139156733 shutdown_connections 2024-09-15T22:22:52.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.104+0000 7f79d2cae740 1 -- 172.21.15.5:0/1139156733 wait complete. 2024-09-15T22:22:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.104+0000 7f79d2cae740 1 -- start start 2024-09-15T22:22:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.104+0000 7f79d2cae740 1 Processor -- start 2024-09-15T22:22:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.105+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4cde0 0x563ae3e54940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.105+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e4e350 0x563ae3e54e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.105+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e57b80 0x563ae3e553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.105+0000 7f79ca59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4cde0 0x563ae3e54940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.105+0000 7f79ca59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4cde0 0x563ae3e54940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:51166/0 (socket says 172.21.15.5:51166) 2024-09-15T22:22:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.105+0000 7f79ca59c640 1 -- 172.21.15.5:0/3153920200 learned_addr learned my addr 172.21.15.5:0/3153920200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.105+0000 7f79d2cae740 1 -- 172.21.15.5:0/3153920200 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ae3e40a00 con 0x563ae3e57b80 2024-09-15T22:22:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.106+0000 7f79d2cae740 1 -- 172.21.15.5:0/3153920200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563ae3d65060 con 0x563ae3e4e350 2024-09-15T22:22:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.106+0000 7f79d2cae740 1 -- 172.21.15.5:0/3153920200 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563ae3e40b80 con 0x563ae3e4cde0 2024-09-15T22:22:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.106+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e4e350 0x563ae3e54e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.106+0000 7f79d2cae740 10 client.0 _ll_register_callbacks cb 0x563ae3ab88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.106+0000 7f79c9d9b640 1 -- 172.21.15.5:0/3153920200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4cde0 msgr2=0x563ae3e54940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.106+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4cde0 0x563ae3e54940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.106+0000 7f79c9d9b640 1 -- 172.21.15.5:0/3153920200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e57b80 msgr2=0x563ae3e553c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.106+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e57b80 0x563ae3e553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.106+0000 7f79c9d9b640 1 -- 172.21.15.5:0/3153920200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae3b1b3c0 con 0x563ae3e4e350 2024-09-15T22:22:52.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.125+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e4e350 0x563ae3e54e80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f79c00ed320 tx=0x7f79c0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc4e61b7cad55e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:52.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.126+0000 7f79b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:22:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.130+0000 7f79b1ffb640 1 -- 172.21.15.5:0/3153920200 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00bbeb0 con 0x563ae3e4e350 2024-09-15T22:22:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.130+0000 7f79d2cae740 1 -- 172.21.15.5:0/3153920200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79c401b470 con 0x563ae3e4e350 2024-09-15T22:22:52.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.130+0000 7f79b1ffb640 1 -- 172.21.15.5:0/3153920200 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f79c0004360 con 0x563ae3e4e350 2024-09-15T22:22:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.130+0000 7f79b1ffb640 1 -- 172.21.15.5:0/3153920200 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79c00c0260 con 0x563ae3e4e350 2024-09-15T22:22:52.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.130+0000 7f79d2cae740 10 client.27227 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:52.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.130+0000 7f79d2cae740 1 -- 172.21.15.5:0/3153920200 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563ae3e6eb80 con 0x563ae3e4e350 2024-09-15T22:22:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.131+0000 7f79b1ffb640 1 -- 172.21.15.5:0/3153920200 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c401b470 con 0x563ae3e4e350 2024-09-15T22:22:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.131+0000 7f79b1ffb640 1 -- 172.21.15.5:0/3153920200 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f79c003a070 con 0x563ae3e4e350 2024-09-15T22:22:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.131+0000 7f79d2cae740 10 client.27227 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.131+0000 7f79d2cae740 10 client.27227 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:22:52.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.131+0000 7f79d2cae740 20 client.27227 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.132+0000 7f79d2cae740 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563ae3e70210 0x563ae3e726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.132+0000 7f79d2cae740 4 client.27227 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:52.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.132+0000 7f79d2cae740 1 -- 172.21.15.5:0/3153920200 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ae3db7860 con 0x563ae3e70210 2024-09-15T22:22:52.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.139+0000 7f79ca59c640 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563ae3e70210 0x563ae3e726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.150+0000 7f79b1ffb640 10 client.27227 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:22:52.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.150+0000 7f79ca59c640 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563ae3e70210 0x563ae3e726d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6596dcbf90be934 server_cookie=909938beb456426e in_seq=0 out_seq=0 2024-09-15T22:22:52.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.154+0000 7f79b1ffb640 1 -- 172.21.15.5:0/3153920200 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ae3db7860 con 0x563ae3e70210 2024-09-15T22:22:52.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.154+0000 7f79b1ffb640 10 client.27227 handle_command_reply: tid=0 2024-09-15T22:22:52.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.154+0000 7f79d2cae740 1 client.27227 shutdown 2024-09-15T22:22:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.156+0000 7f79d2cae740 1 -- 172.21.15.5:0/3153920200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e4e350 msgr2=0x563ae3e54e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.156+0000 7f79d2cae740 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e4e350 0x563ae3e54e80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f79c00ed320 tx=0x7f79c0000bc0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.157+0000 7f79d2cae740 1 -- 172.21.15.5:0/3153920200 shutdown_connections 2024-09-15T22:22:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.157+0000 7f79d2cae740 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563ae3e70210 0x563ae3e726d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.157+0000 7f79d2cae740 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e57b80 0x563ae3e553c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.157+0000 7f79d2cae740 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e4e350 0x563ae3e54e80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.157+0000 7f79d2cae740 1 --2- 172.21.15.5:0/3153920200 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4cde0 0x563ae3e54940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.157+0000 7f79d2cae740 1 -- 172.21.15.5:0/3153920200 >> 172.21.15.5:0/3153920200 conn(0x563ae3e0d310 msgr2=0x563ae3e4f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.157+0000 7f79b1ffb640 0 client.27227 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:22:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.157+0000 7f79d2cae740 1 -- 172.21.15.5:0/3153920200 shutdown_connections 2024-09-15T22:22:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.157+0000 7f79d2cae740 1 -- 172.21.15.5:0/3153920200 wait complete. 2024-09-15T22:22:52.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.157+0000 7f79d2cae740 20 client.27227 trim_cache size 0 max 16384 2024-09-15T22:22:52.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:51 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/662621267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.184+0000 7f79d2cae740 1 Processor -- start 2024-09-15T22:22:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.184+0000 7f79d2cae740 1 -- start start 2024-09-15T22:22:52.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.185+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4cde0 0x563ae3e4f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.185+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4e350 0x563ae3e4fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.185+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 0x563ae3e77fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.185+0000 7f79c9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4e350 0x563ae3e4fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.186+0000 7f79c9d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4e350 0x563ae3e4fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:51174/0 (socket says 172.21.15.5:51174) 2024-09-15T22:22:52.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.186+0000 7f79ca59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4cde0 0x563ae3e4f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.186+0000 7f79ca59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4cde0 0x563ae3e4f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59854/0 (socket says 172.21.15.5:59854) 2024-09-15T22:22:52.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.186+0000 7f79c9d9b640 1 -- 172.21.15.5:0/2979017223 learned_addr learned my addr 172.21.15.5:0/2979017223 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:52.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.186+0000 7f79cad9d640 1 --2- 172.21.15.5:0/2979017223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 0x563ae3e77fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.186+0000 7f79d2cae740 1 -- 172.21.15.5:0/2979017223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1af7b0 con 0x563ae3e4cde0 2024-09-15T22:22:52.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.189+0000 7f79d2cae740 1 -- 172.21.15.5:0/2979017223 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1ad080 con 0x563ae3e57b80 2024-09-15T22:22:52.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.189+0000 7f79d2cae740 1 -- 172.21.15.5:0/2979017223 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1aaef0 con 0x563ae3e4e350 2024-09-15T22:22:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.189+0000 7f79c9d9b640 1 -- 172.21.15.5:0/2979017223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 msgr2=0x563ae3e77fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.189+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/2979017223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 0x563ae3e77fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.189+0000 7f79c9d9b640 1 -- 172.21.15.5:0/2979017223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4cde0 msgr2=0x563ae3e4f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.189+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/2979017223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4cde0 0x563ae3e4f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.189+0000 7f79c9d9b640 1 -- 172.21.15.5:0/2979017223 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ae3e3bd00 con 0x563ae3e4e350 2024-09-15T22:22:52.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.190+0000 7f79c9d9b640 1 --2- 172.21.15.5:0/2979017223 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4e350 0x563ae3e4fe70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f79c00bbbb0 tx=0x7f79c0005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a93fc2f88d3e334 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:52.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.191+0000 7f79b3fff640 1 -- 172.21.15.5:0/2979017223 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00bb9f0 con 0x563ae3e4e350 2024-09-15T22:22:52.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.191+0000 7f79b3fff640 1 -- 172.21.15.5:0/2979017223 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c00bbeb0 con 0x563ae3e4e350 2024-09-15T22:22:52.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.191+0000 7f79b3fff640 1 -- 172.21.15.5:0/2979017223 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c00c4b20 con 0x563ae3e4e350 2024-09-15T22:22:52.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.192+0000 7f79d2cae740 1 -- 172.21.15.5:0/2979017223 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4e350 msgr2=0x563ae3e4fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:52.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.192+0000 7f79d2cae740 1 --2- 172.21.15.5:0/2979017223 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4e350 0x563ae3e4fe70 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f79c00bbbb0 tx=0x7f79c0005fa0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.192+0000 7f79d2cae740 1 -- 172.21.15.5:0/2979017223 shutdown_connections 2024-09-15T22:22:52.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.192+0000 7f79d2cae740 1 --2- 172.21.15.5:0/2979017223 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 0x563ae3e77fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.192+0000 7f79d2cae740 1 --2- 172.21.15.5:0/2979017223 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4e350 0x563ae3e4fe70 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.192+0000 7f79d2cae740 1 --2- 172.21.15.5:0/2979017223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4cde0 0x563ae3e4f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.192+0000 7f79d2cae740 1 -- 172.21.15.5:0/2979017223 >> 172.21.15.5:0/2979017223 conn(0x563ae3e0d310 msgr2=0x563ae3e569c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:52.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.194+0000 7f79d2cae740 1 -- 172.21.15.5:0/2979017223 shutdown_connections 2024-09-15T22:22:52.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.194+0000 7f79d2cae740 1 -- 172.21.15.5:0/2979017223 wait complete. 2024-09-15T22:22:52.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.194+0000 7f79d2cae740 1 -- start start 2024-09-15T22:22:52.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.194+0000 7f79d2cae740 1 Processor -- start 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.195+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4cde0 0x563ae3e42fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.195+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4e350 0x563ae3e43500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.195+0000 7f79d2cae740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 0x563ae3ed8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.195+0000 7f79d2cae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1ab540 con 0x563ae3e4cde0 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.195+0000 7f79d2cae740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1ac9b0 con 0x563ae3e57b80 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.195+0000 7f79d2cae740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1abc10 con 0x563ae3e4e350 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.195+0000 7f79d2cae740 10 client.0 _ll_register_callbacks cb 0x7f79c0009490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.195+0000 7f79ca59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4cde0 0x563ae3e42fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.195+0000 7f79cad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 0x563ae3ed8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.195+0000 7f79cad9d640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 0x563ae3ed8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:57944/0 (socket says 172.21.15.5:57944) 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.195+0000 7f79cad9d640 1 -- 172.21.15.5:0/639541270 learned_addr learned my addr 172.21.15.5:0/639541270 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.196+0000 7f79cad9d640 1 -- 172.21.15.5:0/639541270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4e350 msgr2=0x563ae3e43500 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:22:52.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.196+0000 7f79cad9d640 1 --2- 172.21.15.5:0/639541270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4e350 0x563ae3e43500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.196+0000 7f79cad9d640 1 -- 172.21.15.5:0/639541270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4cde0 msgr2=0x563ae3e42fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:52.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.196+0000 7f79cad9d640 1 --2- 172.21.15.5:0/639541270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4cde0 0x563ae3e42fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.196+0000 7f79cad9d640 1 -- 172.21.15.5:0/639541270 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ae3ed94e0 con 0x563ae3e57b80 2024-09-15T22:22:52.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.197+0000 7f79cad9d640 1 --2- 172.21.15.5:0/639541270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 0x563ae3ed8fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x563ae3e4d880 tx=0x7f79c400d910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468046ba85473053 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:52.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.200+0000 7f79b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:22:52.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.201+0000 7f79b1ffb640 1 -- 172.21.15.5:0/639541270 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4028040 con 0x563ae3e57b80 2024-09-15T22:22:52.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.201+0000 7f79b1ffb640 1 -- 172.21.15.5:0/639541270 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f79c400da00 con 0x563ae3e57b80 2024-09-15T22:22:52.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.202+0000 7f79d2cae740 1 -- 172.21.15.5:0/639541270 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79c00bbeb0 con 0x563ae3e57b80 2024-09-15T22:22:52.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.204+0000 7f79b1ffb640 1 -- 172.21.15.5:0/639541270 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79c400b4b0 con 0x563ae3e57b80 2024-09-15T22:22:52.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.208+0000 7f79d2cae740 10 client.27233 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:52.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.208+0000 7f79d2cae740 1 -- 172.21.15.5:0/639541270 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563ae3eda780 con 0x563ae3e57b80 2024-09-15T22:22:52.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.212+0000 7f79b1ffb640 1 -- 172.21.15.5:0/639541270 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79c00bbeb0 con 0x563ae3e57b80 2024-09-15T22:22:52.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.212+0000 7f79b1ffb640 1 -- 172.21.15.5:0/639541270 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f79c4019820 con 0x563ae3e57b80 2024-09-15T22:22:52.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.212+0000 7f79d2cae740 10 client.27233 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:52.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.212+0000 7f79d2cae740 10 client.27233 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:22:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.212+0000 7f79d2cae740 20 client.27233 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.212+0000 7f79d2cae740 1 --2- 172.21.15.5:0/639541270 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563ae3edaed0 0x563ae3edd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.212+0000 7f79d2cae740 4 client.27233 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:22:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.212+0000 7f79d2cae740 1 -- 172.21.15.5:0/639541270 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ae3db1d70 con 0x563ae3edaed0 2024-09-15T22:22:52.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.212+0000 7f79ca59c640 1 --2- 172.21.15.5:0/639541270 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563ae3edaed0 0x563ae3edd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.213+0000 7f79ca59c640 1 --2- 172.21.15.5:0/639541270 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563ae3edaed0 0x563ae3edd310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d82e3e5d274b4dc4 server_cookie=ad5af5ae2805cd4e in_seq=0 out_seq=0 2024-09-15T22:22:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.213+0000 7f79b1ffb640 10 client.27233 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:22:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.214+0000 7f79b1ffb640 1 -- 172.21.15.5:0/639541270 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ae3db1d70 con 0x563ae3edaed0 2024-09-15T22:22:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.214+0000 7f79b1ffb640 10 client.27233 handle_command_reply: tid=0 2024-09-15T22:22:52.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.214+0000 7f79d2cae740 1 client.27233 shutdown 2024-09-15T22:22:52.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.216+0000 7f79d2cae740 1 -- 172.21.15.5:0/639541270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 msgr2=0x563ae3ed8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:52.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.216+0000 7f79d2cae740 1 --2- 172.21.15.5:0/639541270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 0x563ae3ed8fa0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x563ae3e4d880 tx=0x7f79c400d910 comp rx=0 tx=0).stop 2024-09-15T22:22:52.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.216+0000 7f79d2cae740 1 -- 172.21.15.5:0/639541270 shutdown_connections 2024-09-15T22:22:52.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.216+0000 7f79d2cae740 1 --2- 172.21.15.5:0/639541270 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x563ae3edaed0 0x563ae3edd310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.216+0000 7f79d2cae740 1 --2- 172.21.15.5:0/639541270 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563ae3e57b80 0x563ae3ed8fa0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.216+0000 7f79d2cae740 1 --2- 172.21.15.5:0/639541270 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563ae3e4e350 0x563ae3e43500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.216+0000 7f79d2cae740 1 --2- 172.21.15.5:0/639541270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ae3e4cde0 0x563ae3e42fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.216+0000 7f79d2cae740 1 -- 172.21.15.5:0/639541270 >> 172.21.15.5:0/639541270 conn(0x563ae3e0d310 msgr2=0x563ae3e3f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:52.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.216+0000 7f79b1ffb640 0 client.27233 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:22:52.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.220+0000 7f79d2cae740 1 -- 172.21.15.5:0/639541270 shutdown_connections 2024-09-15T22:22:52.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.220+0000 7f79d2cae740 1 -- 172.21.15.5:0/639541270 wait complete. 2024-09-15T22:22:52.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.220+0000 7f79d2cae740 20 client.27233 trim_cache size 0 max 16384 2024-09-15T22:22:52.225 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:22:52.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.226+0000 7f79d08d5640 1 -- 172.21.15.5:0/3741826384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f79b40a3580 msgr2=0x7f79b40a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:52.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.226+0000 7f79d08d5640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f79b40a3580 0x7f79b40a5a40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f79c4017fd0 tx=0x7f79c400d530 comp rx=0 tx=0).stop 2024-09-15T22:22:52.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.226+0000 7f79d08d5640 1 -- 172.21.15.5:0/3741826384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79cc102780 msgr2=0x7f79cc1965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:52.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.226+0000 7f79d08d5640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79cc102780 0x7f79cc1965b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f79c002a860 tx=0x7f79c0033d90 comp rx=0 tx=0).stop 2024-09-15T22:22:52.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.229+0000 7f79d08d5640 1 -- 172.21.15.5:0/3741826384 shutdown_connections 2024-09-15T22:22:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.229+0000 7f79d08d5640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f79b40a3580 0x7f79b40a5a40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.229+0000 7f79d08d5640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79cc106f00 0x7f79cc19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.229+0000 7f79d08d5640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f79cc102780 0x7f79cc1965b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.229+0000 7f79d08d5640 1 --2- 172.21.15.5:0/3741826384 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f79cc101dd0 0x7f79cc196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.229+0000 7f79d08d5640 1 -- 172.21.15.5:0/3741826384 >> 172.21.15.5:0/3741826384 conn(0x7f79cc0fd890 msgr2=0x7f79cc0fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.231+0000 7f79d08d5640 1 -- 172.21.15.5:0/3741826384 shutdown_connections 2024-09-15T22:22:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:52.231+0000 7f79d08d5640 1 -- 172.21.15.5:0/3741826384 wait complete. 2024-09-15T22:22:52.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:51 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/662621267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:52.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:51 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/662621267' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:22:52.887 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0' 2024-09-15T22:22:52.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 146.5 --await-for 45.0 2024-09-15T22:22:53.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:52 smithi099 ceph-mon[34892]: pgmap v436: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.2 MiB/s wr, 553 op/s 2024-09-15T22:22:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:52 smithi129 ceph-mon[35314]: pgmap v436: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.2 MiB/s wr, 553 op/s 2024-09-15T22:22:53.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:52 smithi005 ceph-mon[30944]: pgmap v436: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 5.2 MiB/s wr, 553 op/s 2024-09-15T22:22:53.529 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:22:53.708 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.392+0000 7f8d8069e640 1 -- 172.21.15.5:0/2120910832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78106840 msgr2=0x7f8d78106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.392+0000 7f8d8069e640 1 --2- 172.21.15.5:0/2120910832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78106840 0x7f8d78106c20 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009900 tx=0x7f8d6c02a420 comp rx=0 tx=0).stop 2024-09-15T22:22:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.392+0000 7f8d7d438640 1 -- 172.21.15.5:0/2120910832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 0 0 0) 0x7f8d6c0319b0 con 0x7f8d78106840 2024-09-15T22:22:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.392+0000 7f8d8069e640 1 -- 172.21.15.5:0/2120910832 shutdown_connections 2024-09-15T22:22:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.392+0000 7f8d8069e640 1 --2- 172.21.15.5:0/2120910832 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d78112b50 0x7f8d78114f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.392+0000 7f8d8069e640 1 --2- 172.21.15.5:0/2120910832 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d78107160 0x7f8d78112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.392+0000 7f8d8069e640 1 --2- 172.21.15.5:0/2120910832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78106840 0x7f8d78106c20 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.392+0000 7f8d8069e640 1 -- 172.21.15.5:0/2120910832 >> 172.21.15.5:0/2120910832 conn(0x7f8d78075f70 msgr2=0x7f8d78076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:54.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.394+0000 7f8d8069e640 1 -- 172.21.15.5:0/2120910832 shutdown_connections 2024-09-15T22:22:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.398+0000 7f8d8069e640 1 -- 172.21.15.5:0/2120910832 wait complete. 2024-09-15T22:22:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.401+0000 7f8d8069e640 1 Processor -- start 2024-09-15T22:22:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.401+0000 7f8d8069e640 1 -- start start 2024-09-15T22:22:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.401+0000 7f8d8069e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d78106840 0x7f8d78100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.401+0000 7f8d8069e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78107160 0x7f8d78101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.401+0000 7f8d8069e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d78112b50 0x7f8d781acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.401+0000 7f8d8069e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7806d0a0 con 0x7f8d78107160 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.401+0000 7f8d8069e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7806cf20 con 0x7f8d78112b50 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.401+0000 7f8d8069e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7806d220 con 0x7f8d78106840 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.402+0000 7f8d7e43a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d78106840 0x7f8d78100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.402+0000 7f8d7dc39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78107160 0x7f8d78101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.402+0000 7f8d7dc39640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78107160 0x7f8d78101130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45018/0 (socket says 172.21.15.5:45018) 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.402+0000 7f8d7dc39640 1 -- 172.21.15.5:0/2204744276 learned_addr learned my addr 172.21.15.5:0/2204744276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.403+0000 7f8d7dc39640 1 -- 172.21.15.5:0/2204744276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d78106840 msgr2=0x7f8d78100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.403+0000 7f8d7dc39640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d78106840 0x7f8d78100bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.403+0000 7f8d7dc39640 1 -- 172.21.15.5:0/2204744276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d78112b50 msgr2=0x7f8d781acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.403+0000 7f8d7dc39640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d78112b50 0x7f8d781acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.403+0000 7f8d7dc39640 1 -- 172.21.15.5:0/2204744276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d781ad680 con 0x7f8d78107160 2024-09-15T22:22:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.404+0000 7f8d7dc39640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78107160 0x7f8d78101130 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400b450 tx=0x7f8d7400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.404+0000 7f8d6b7fe640 1 -- 172.21.15.5:0/2204744276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d74015020 con 0x7f8d78107160 2024-09-15T22:22:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.404+0000 7f8d8069e640 1 -- 172.21.15.5:0/2204744276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d781ad970 con 0x7f8d78107160 2024-09-15T22:22:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.404+0000 7f8d8069e640 1 -- 172.21.15.5:0/2204744276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d781b4030 con 0x7f8d78107160 2024-09-15T22:22:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.405+0000 7f8d6b7fe640 1 -- 172.21.15.5:0/2204744276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d74011d70 con 0x7f8d78107160 2024-09-15T22:22:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.405+0000 7f8d6b7fe640 1 -- 172.21.15.5:0/2204744276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400f340 con 0x7f8d78107160 2024-09-15T22:22:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.413+0000 7f8d6b7fe640 1 -- 172.21.15.5:0/2204744276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8d74018020 con 0x7f8d78107160 2024-09-15T22:22:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.414+0000 7f8d6b7fe640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8d640a35c0 0x7f8d640a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.414+0000 7f8d6b7fe640 1 -- 172.21.15.5:0/2204744276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f8d740cfac0 con 0x7f8d78107160 2024-09-15T22:22:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.414+0000 7f8d82a77740 1 Processor -- start 2024-09-15T22:22:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.416+0000 7f8d82a77740 1 -- start start 2024-09-15T22:22:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.417+0000 7f8d82a77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6cd7760 0x55cac6cd7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.417+0000 7f8d82a77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6cd8080 0x55cac6ce2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.417+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8d640a35c0 0x7f8d640a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.417+0000 7f8d82a77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6ce2940 0x55cac6ce4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.417+0000 7f8d82a77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cac6c3d2a0 con 0x55cac6cd7760 2024-09-15T22:22:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.418+0000 7f8d82a77740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cac6be4b10 con 0x55cac6ce2940 2024-09-15T22:22:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.418+0000 7f8d7ec3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6ce2940 0x55cac6ce4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.418+0000 7f8d7ec3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6ce2940 0x55cac6ce4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:34050/0 (socket says 172.21.15.5:34050) 2024-09-15T22:22:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.418+0000 7f8d7ec3b640 1 -- 172.21.15.5:0/265121841 learned_addr learned my addr 172.21.15.5:0/265121841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.418+0000 7f8d82a77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cac6c39260 con 0x55cac6cd8080 2024-09-15T22:22:54.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.420+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/265121841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6cd7760 0x55cac6cd7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.424+0000 7f8d7e43a640 1 -- 172.21.15.5:0/265121841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6cd8080 msgr2=0x55cac6ce2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.424+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/265121841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6cd8080 0x55cac6ce2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.424+0000 7f8d7e43a640 1 -- 172.21.15.5:0/265121841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6ce2940 msgr2=0x55cac6ce4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.424+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/265121841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6ce2940 0x55cac6ce4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.424+0000 7f8d7e43a640 1 -- 172.21.15.5:0/265121841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac69aeb90 con 0x55cac6cd7760 2024-09-15T22:22:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.425+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8d640a35c0 0x7f8d640a5a80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009850 tx=0x7f8d6c004400 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.429+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/265121841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6cd7760 0x55cac6cd7b40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009230 tx=0x7f8d6c042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01318e22b449fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.431+0000 7f8d68ff9640 1 -- 172.21.15.5:0/265121841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c035050 con 0x55cac6cd7760 2024-09-15T22:22:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.431+0000 7f8d68ff9640 1 -- 172.21.15.5:0/265121841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c04e070 con 0x55cac6cd7760 2024-09-15T22:22:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.431+0000 7f8d68ff9640 1 -- 172.21.15.5:0/265121841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c032c30 con 0x55cac6cd7760 2024-09-15T22:22:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.432+0000 7f8d82a77740 1 -- 172.21.15.5:0/265121841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6cd7760 msgr2=0x55cac6cd7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.432+0000 7f8d82a77740 1 --2- 172.21.15.5:0/265121841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6cd7760 0x55cac6cd7b40 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009230 tx=0x7f8d6c042000 comp rx=0 tx=0).stop 2024-09-15T22:22:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.439+0000 7f8d82a77740 1 -- 172.21.15.5:0/265121841 shutdown_connections 2024-09-15T22:22:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.439+0000 7f8d82a77740 1 --2- 172.21.15.5:0/265121841 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6ce2940 0x55cac6ce4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.439+0000 7f8d82a77740 1 --2- 172.21.15.5:0/265121841 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6cd8080 0x55cac6ce2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.439+0000 7f8d82a77740 1 --2- 172.21.15.5:0/265121841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6cd7760 0x55cac6cd7b40 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.439+0000 7f8d82a77740 1 -- 172.21.15.5:0/265121841 >> 172.21.15.5:0/265121841 conn(0x55cac6cd3cf0 msgr2=0x55cac6cd40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.445+0000 7f8d82a77740 1 -- 172.21.15.5:0/265121841 shutdown_connections 2024-09-15T22:22:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.445+0000 7f8d82a77740 1 -- 172.21.15.5:0/265121841 wait complete. 2024-09-15T22:22:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.445+0000 7f8d82a77740 1 -- start start 2024-09-15T22:22:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.446+0000 7f8d82a77740 1 Processor -- start 2024-09-15T22:22:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.446+0000 7f8d82a77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6cd7760 0x55cac6cdf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.446+0000 7f8d82a77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 0x55cac6cdfde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.447+0000 7f8d82a77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6ce2940 0x55cac6ce0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.447+0000 7f8d82a77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cac6ce5510 con 0x55cac6ce2940 2024-09-15T22:22:54.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.447+0000 7f8d82a77740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55cac6bc56e0 con 0x55cac6cd8080 2024-09-15T22:22:54.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.447+0000 7f8d82a77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55cac6ce5690 con 0x55cac6cd7760 2024-09-15T22:22:54.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.447+0000 7f8d82a77740 10 client.0 _ll_register_callbacks cb 0x55cac6be9a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:54.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.450+0000 7f8d7ec3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6ce2940 0x55cac6ce0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.450+0000 7f8d7ec3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6ce2940 0x55cac6ce0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45036/0 (socket says 172.21.15.5:45036) 2024-09-15T22:22:54.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.450+0000 7f8d7ec3b640 1 -- 172.21.15.5:0/1146095163 learned_addr learned my addr 172.21.15.5:0/1146095163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:54.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.452+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6cd7760 0x55cac6cdf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.452+0000 7f8d7dc39640 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 0x55cac6cdfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.457+0000 7f8d7ec3b640 1 -- 172.21.15.5:0/1146095163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6cd7760 msgr2=0x55cac6cdf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:54.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.457+0000 7f8d7ec3b640 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6cd7760 0x55cac6cdf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.457+0000 7f8d7ec3b640 1 -- 172.21.15.5:0/1146095163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 msgr2=0x55cac6cdfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:54.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.457+0000 7f8d7ec3b640 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 0x55cac6cdfde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.457+0000 7f8d7ec3b640 1 -- 172.21.15.5:0/1146095163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac6ceeb10 con 0x55cac6ce2940 2024-09-15T22:22:54.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.459+0000 7f8d7dc39640 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 0x55cac6cdfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:22:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.460+0000 7f8d7ec3b640 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6ce2940 0x55cac6ce0320 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000e7e0 tx=0x7f8d7000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afcac3cd92dc070 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.461+0000 7f8d61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:22:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.461+0000 7f8d61ffb640 1 -- 172.21.15.5:0/1146095163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70018020 con 0x55cac6ce2940 2024-09-15T22:22:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.461+0000 7f8d82a77740 1 -- 172.21.15.5:0/1146095163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d6c04e070 con 0x55cac6ce2940 2024-09-15T22:22:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.461+0000 7f8d61ffb640 1 -- 172.21.15.5:0/1146095163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f8d7000c820 con 0x55cac6ce2940 2024-09-15T22:22:54.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.461+0000 7f8d61ffb640 1 -- 172.21.15.5:0/1146095163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8d70023050 con 0x55cac6ce2940 2024-09-15T22:22:54.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.463+0000 7f8d82a77740 10 client.17751 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:54.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.463+0000 7f8d82a77740 1 -- 172.21.15.5:0/1146095163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cac6cf1500 con 0x55cac6ce2940 2024-09-15T22:22:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.463+0000 7f8d61ffb640 1 -- 172.21.15.5:0/1146095163 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c04e070 con 0x55cac6ce2940 2024-09-15T22:22:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.464+0000 7f8d61ffb640 1 -- 172.21.15.5:0/1146095163 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8d700133e0 con 0x55cac6ce2940 2024-09-15T22:22:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.465+0000 7f8d82a77740 10 client.17751 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.465+0000 7f8d82a77740 10 client.17751 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:22:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.465+0000 7f8d82a77740 20 client.17751 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.465+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cac6cf2e10 0x55cac6cf5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.465+0000 7f8d82a77740 4 client.17751 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.465+0000 7f8d82a77740 1 -- 172.21.15.5:0/1146095163 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cac6c4a9d0 con 0x55cac6cf2e10 2024-09-15T22:22:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.474+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cac6cf2e10 0x55cac6cf5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.474+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cac6cf2e10 0x55cac6cf5230 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8de2c2d18d3fe224 server_cookie=84654d3e1bd16145 in_seq=0 out_seq=0 2024-09-15T22:22:54.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.474+0000 7f8d61ffb640 10 client.17751 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:54.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.475+0000 7f8d61ffb640 1 -- 172.21.15.5:0/1146095163 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cac6c4a9d0 con 0x55cac6cf2e10 2024-09-15T22:22:54.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.475+0000 7f8d61ffb640 10 client.17751 handle_command_reply: tid=0 2024-09-15T22:22:54.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.475+0000 7f8d82a77740 1 client.17751 shutdown 2024-09-15T22:22:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.478+0000 7f8d82a77740 1 -- 172.21.15.5:0/1146095163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6ce2940 msgr2=0x55cac6ce0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.478+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6ce2940 0x55cac6ce0320 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000e7e0 tx=0x7f8d7000ebe0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.481+0000 7f8d82a77740 1 -- 172.21.15.5:0/1146095163 shutdown_connections 2024-09-15T22:22:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.481+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6ce2940 0x55cac6ce0320 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.481+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cac6cf2e10 0x55cac6cf5230 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.481+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 0x55cac6cdfde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.481+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1146095163 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6cd7760 0x55cac6cdf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.481+0000 7f8d82a77740 1 -- 172.21.15.5:0/1146095163 >> 172.21.15.5:0/1146095163 conn(0x55cac6cd3cf0 msgr2=0x55cac6cd9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.482+0000 7f8d61ffb640 0 client.17751 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:54.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.484+0000 7f8d82a77740 1 -- 172.21.15.5:0/1146095163 shutdown_connections 2024-09-15T22:22:54.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.484+0000 7f8d82a77740 1 -- 172.21.15.5:0/1146095163 wait complete. 2024-09-15T22:22:54.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.485+0000 7f8d82a77740 20 client.17751 trim_cache size 0 max 16384 2024-09-15T22:22:54.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.518+0000 7f8d82a77740 1 Processor -- start 2024-09-15T22:22:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.523+0000 7f8d82a77740 1 -- start start 2024-09-15T22:22:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.523+0000 7f8d82a77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd7760 0x55cac6cde170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.523+0000 7f8d82a77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6cd8080 0x55cac6cfab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.523+0000 7f8d82a77740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ce2940 0x55cac6cfb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.524+0000 7f8d82a77740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c012880 con 0x55cac6cd8080 2024-09-15T22:22:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.524+0000 7f8d82a77740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c010150 con 0x55cac6cd7760 2024-09-15T22:22:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.524+0000 7f8d82a77740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c00e050 con 0x55cac6ce2940 2024-09-15T22:22:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.524+0000 7f8d7ec3b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ce2940 0x55cac6cfb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.524+0000 7f8d7ec3b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ce2940 0x55cac6cfb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:41526/0 (socket says 172.21.15.5:41526) 2024-09-15T22:22:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.524+0000 7f8d7ec3b640 1 -- 172.21.15.5:0/3731092436 learned_addr learned my addr 172.21.15.5:0/3731092436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.526+0000 7f8d7dc39640 1 --2- 172.21.15.5:0/3731092436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6cd8080 0x55cac6cfab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.526+0000 7f8d7ec3b640 1 -- 172.21.15.5:0/3731092436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd7760 msgr2=0x55cac6cde170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:22:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.526+0000 7f8d7ec3b640 1 --2- 172.21.15.5:0/3731092436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd7760 0x55cac6cde170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.526+0000 7f8d7ec3b640 1 -- 172.21.15.5:0/3731092436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6cd8080 msgr2=0x55cac6cfab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.526+0000 7f8d7ec3b640 1 --2- 172.21.15.5:0/3731092436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6cd8080 0x55cac6cfab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.526+0000 7f8d7ec3b640 1 -- 172.21.15.5:0/3731092436 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac6ccf260 con 0x55cac6ce2940 2024-09-15T22:22:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.526+0000 7f8d7ec3b640 1 --2- 172.21.15.5:0/3731092436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ce2940 0x55cac6cfb060 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000e530 tx=0x7f8d70004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e71cac307a91515 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.528+0000 7f8d63fff640 1 -- 172.21.15.5:0/3731092436 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7002b020 con 0x55cac6ce2940 2024-09-15T22:22:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.528+0000 7f8d63fff640 1 -- 172.21.15.5:0/3731092436 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d700139f0 con 0x55cac6ce2940 2024-09-15T22:22:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.528+0000 7f8d63fff640 1 -- 172.21.15.5:0/3731092436 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70018040 con 0x55cac6ce2940 2024-09-15T22:22:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.529+0000 7f8d82a77740 1 -- 172.21.15.5:0/3731092436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ce2940 msgr2=0x55cac6cfb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.529+0000 7f8d82a77740 1 --2- 172.21.15.5:0/3731092436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ce2940 0x55cac6cfb060 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000e530 tx=0x7f8d70004b50 comp rx=0 tx=0).stop 2024-09-15T22:22:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.529+0000 7f8d82a77740 1 -- 172.21.15.5:0/3731092436 shutdown_connections 2024-09-15T22:22:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.529+0000 7f8d82a77740 1 --2- 172.21.15.5:0/3731092436 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ce2940 0x55cac6cfb060 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.529+0000 7f8d82a77740 1 --2- 172.21.15.5:0/3731092436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6cd8080 0x55cac6cfab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.529+0000 7f8d82a77740 1 --2- 172.21.15.5:0/3731092436 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd7760 0x55cac6cde170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.529+0000 7f8d82a77740 1 -- 172.21.15.5:0/3731092436 >> 172.21.15.5:0/3731092436 conn(0x55cac6cd3cf0 msgr2=0x55cac6ce1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.533+0000 7f8d82a77740 1 -- 172.21.15.5:0/3731092436 shutdown_connections 2024-09-15T22:22:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.533+0000 7f8d82a77740 1 -- 172.21.15.5:0/3731092436 wait complete. 2024-09-15T22:22:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.534+0000 7f8d82a77740 1 -- start start 2024-09-15T22:22:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.534+0000 7f8d82a77740 1 Processor -- start 2024-09-15T22:22:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.535+0000 7f8d82a77740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 0x55cac6cea970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.535+0000 7f8d82a77740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6ce2940 0x55cac6ceaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.535+0000 7f8d7e43a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 0x55cac6cea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.535+0000 7f8d7e43a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 0x55cac6cea970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:34060/0 (socket says 172.21.15.5:34060) 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.536+0000 7f8d7e43a640 1 -- 172.21.15.5:0/1226767845 learned_addr learned my addr 172.21.15.5:0/1226767845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.536+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ceb3f0 0x55cac6ce7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.536+0000 7f8d82a77740 1 -- 172.21.15.5:0/1226767845 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c00e610 con 0x55cac6ce2940 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.536+0000 7f8d7ec3b640 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ceb3f0 0x55cac6ce7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.536+0000 7f8d82a77740 1 -- 172.21.15.5:0/1226767845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c00fa80 con 0x55cac6cd8080 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.536+0000 7f8d82a77740 1 -- 172.21.15.5:0/1226767845 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8d5c00ece0 con 0x55cac6ceb3f0 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.536+0000 7f8d82a77740 10 client.0 _ll_register_callbacks cb 0x7f8d7000e420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.537+0000 7f8d7e43a640 1 -- 172.21.15.5:0/1226767845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ceb3f0 msgr2=0x55cac6ce7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.537+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ceb3f0 0x55cac6ce7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.537+0000 7f8d7e43a640 1 -- 172.21.15.5:0/1226767845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6ce2940 msgr2=0x55cac6ceaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.537+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6ce2940 0x55cac6ceaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.537+0000 7f8d7e43a640 1 -- 172.21.15.5:0/1226767845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cac6ce7f50 con 0x55cac6cd8080 2024-09-15T22:22:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.537+0000 7f8d7e43a640 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 0x55cac6cea970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c032760 tx=0x7f8d6c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b209fe6a95e110 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.537+0000 7f8d61ffb640 10 client.27242 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:22:54.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.537+0000 7f8d82a77740 1 -- 172.21.15.5:0/1226767845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d700139f0 con 0x55cac6cd8080 2024-09-15T22:22:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.541+0000 7f8d61ffb640 1 -- 172.21.15.5:0/1226767845 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c05a070 con 0x55cac6cd8080 2024-09-15T22:22:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.541+0000 7f8d61ffb640 1 -- 172.21.15.5:0/1226767845 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f8d6c0046c0 con 0x55cac6cd8080 2024-09-15T22:22:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.541+0000 7f8d61ffb640 1 -- 172.21.15.5:0/1226767845 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8d6c035040 con 0x55cac6cd8080 2024-09-15T22:22:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.545+0000 7f8d82a77740 10 client.27242 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.545+0000 7f8d82a77740 1 -- 172.21.15.5:0/1226767845 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55cac6d5e2f0 con 0x55cac6cd8080 2024-09-15T22:22:54.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.548+0000 7f8d61ffb640 1 -- 172.21.15.5:0/1226767845 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8d6c0048e0 con 0x55cac6cd8080 2024-09-15T22:22:54.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.549+0000 7f8d82a77740 10 client.27242 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:54.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.549+0000 7f8d82a77740 10 client.27242 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:22:54.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.549+0000 7f8d82a77740 20 client.27242 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:54.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.549+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cac6d5e520 0x55cac6d60910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:54.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.549+0000 7f8d82a77740 4 client.27242 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}] 2024-09-15T22:22:54.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.549+0000 7f8d82a77740 1 -- 172.21.15.5:0/1226767845 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.5, "await_for": 45.0}) -- 0x7f8d700139f0 con 0x55cac6d5e520 2024-09-15T22:22:54.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.549+0000 7f8d7dc39640 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cac6d5e520 0x55cac6d60910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:54.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.554+0000 7f8d7dc39640 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cac6d5e520 0x55cac6d60910 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43b2b8bf06347040 server_cookie=77a2f410f80102e0 in_seq=0 out_seq=0 2024-09-15T22:22:54.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.554+0000 7f8d61ffb640 10 client.27242 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:54.661 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 805.0 s, 485.7 tps, lat 204.621 ms stddev 233.748 2024-09-15T22:22:54.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d61ffb640 1 -- 172.21.15.5:0/1226767845 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f8d700139f0 con 0x55cac6d5e520 2024-09-15T22:22:54.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d61ffb640 10 client.27242 handle_command_reply: tid=0 2024-09-15T22:22:54.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 1 client.27242 shutdown 2024-09-15T22:22:54.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 1 -- 172.21.15.5:0/1226767845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 msgr2=0x55cac6cea970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:54.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 0x55cac6cea970 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c032760 tx=0x7f8d6c009260 comp rx=0 tx=0).stop 2024-09-15T22:22:54.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 1 -- 172.21.15.5:0/1226767845 shutdown_connections 2024-09-15T22:22:54.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55cac6ceb3f0 0x55cac6ce7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cac6ce2940 0x55cac6ceaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55cac6d5e520 0x55cac6d60910 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 1 --2- 172.21.15.5:0/1226767845 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55cac6cd8080 0x55cac6cea970 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d61ffb640 0 client.27242 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 1 -- 172.21.15.5:0/1226767845 >> 172.21.15.5:0/1226767845 conn(0x55cac6cd3cf0 msgr2=0x55cac6cd2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 1 -- 172.21.15.5:0/1226767845 shutdown_connections 2024-09-15T22:22:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 1 -- 172.21.15.5:0/1226767845 wait complete. 2024-09-15T22:22:54.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.854+0000 7f8d82a77740 20 client.27242 trim_cache size 0 max 16384 2024-09-15T22:22:54.858 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 50, 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "fedc988e": { 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 50, 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:22:54.859 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 146.5, 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:54.860 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:22:54.861 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:22:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 -- 172.21.15.5:0/2204744276 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8d640a35c0 msgr2=0x7f8d640a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8d640a35c0 0x7f8d640a5a80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009850 tx=0x7f8d6c004400 comp rx=0 tx=0).stop 2024-09-15T22:22:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 -- 172.21.15.5:0/2204744276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78107160 msgr2=0x7f8d78101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78107160 0x7f8d78101130 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400b450 tx=0x7f8d7400b910 comp rx=0 tx=0).stop 2024-09-15T22:22:54.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 -- 172.21.15.5:0/2204744276 shutdown_connections 2024-09-15T22:22:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8d640a35c0 0x7f8d640a5a80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8d78112b50 0x7f8d781acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78107160 0x7f8d78101130 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 --2- 172.21.15.5:0/2204744276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8d78106840 0x7f8d78100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 -- 172.21.15.5:0/2204744276 >> 172.21.15.5:0/2204744276 conn(0x7f8d78075f70 msgr2=0x7f8d781134f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 -- 172.21.15.5:0/2204744276 shutdown_connections 2024-09-15T22:22:54.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:54.855+0000 7f8d8069e640 1 -- 172.21.15.5:0/2204744276 wait complete. 2024-09-15T22:22:55.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.010+0000 7fc840ea0640 1 -- 172.21.15.5:0/3010656008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c106a80 msgr2=0x7fc83c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.010+0000 7fc840ea0640 1 --2- 172.21.15.5:0/3010656008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c106a80 0x7fc83c106e60 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009930 tx=0x7fc82c02a420 comp rx=0 tx=0).stop 2024-09-15T22:22:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.010+0000 7fc840ea0640 1 -- 172.21.15.5:0/3010656008 shutdown_connections 2024-09-15T22:22:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.010+0000 7fc840ea0640 1 --2- 172.21.15.5:0/3010656008 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c101420 0x7fc83c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.010+0000 7fc840ea0640 1 --2- 172.21.15.5:0/3010656008 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83c100a80 0x7fc83c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.010+0000 7fc840ea0640 1 --2- 172.21.15.5:0/3010656008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c106a80 0x7fc83c106e60 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.010+0000 7fc840ea0640 1 -- 172.21.15.5:0/3010656008 >> 172.21.15.5:0/3010656008 conn(0x7fc83c0fc7a0 msgr2=0x7fc83c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.010+0000 7fc840ea0640 1 -- 172.21.15.5:0/3010656008 shutdown_connections 2024-09-15T22:22:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.010+0000 7fc840ea0640 1 -- 172.21.15.5:0/3010656008 wait complete. 2024-09-15T22:22:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.011+0000 7fc840ea0640 1 Processor -- start 2024-09-15T22:22:55.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.011+0000 7fc840ea0640 1 -- start start 2024-09-15T22:22:55.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.011+0000 7fc840ea0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83c100a80 0x7fc83c0fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.011+0000 7fc840ea0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c101420 0x7fc83c0fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.011+0000 7fc840ea0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c106a80 0x7fc83c0fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.011+0000 7fc840ea0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c110e80 con 0x7fc83c101420 2024-09-15T22:22:55.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.011+0000 7fc840ea0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c110d00 con 0x7fc83c100a80 2024-09-15T22:22:55.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.011+0000 7fc840ea0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c111000 con 0x7fc83c106a80 2024-09-15T22:22:55.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc839d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c101420 0x7fc83c0fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc839d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c101420 0x7fc83c0fd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45074/0 (socket says 172.21.15.5:45074) 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc83ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c106a80 0x7fc83c0fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc839d9b640 1 -- 172.21.15.5:0/1929320698 learned_addr learned my addr 172.21.15.5:0/1929320698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc83a59c640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83c100a80 0x7fc83c0fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc839d9b640 1 -- 172.21.15.5:0/1929320698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c106a80 msgr2=0x7fc83c0fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc839d9b640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c106a80 0x7fc83c0fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc839d9b640 1 -- 172.21.15.5:0/1929320698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83c100a80 msgr2=0x7fc83c0fcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc839d9b640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83c100a80 0x7fc83c0fcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc839d9b640 1 -- 172.21.15.5:0/1929320698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83c1a6920 con 0x7fc83c101420 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc839d9b640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c101420 0x7fc83c0fd2d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc83000d950 tx=0x7fc83000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc82b7fe640 1 -- 172.21.15.5:0/1929320698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830017030 con 0x7fc83c101420 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc82b7fe640 1 -- 172.21.15.5:0/1929320698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc830004020 con 0x7fc83c101420 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc840ea0640 1 -- 172.21.15.5:0/1929320698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83c1a6c10 con 0x7fc83c101420 2024-09-15T22:22:55.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc82b7fe640 1 -- 172.21.15.5:0/1929320698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8300047a0 con 0x7fc83c101420 2024-09-15T22:22:55.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.012+0000 7fc840ea0640 1 -- 172.21.15.5:0/1929320698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83c1a71a0 con 0x7fc83c101420 2024-09-15T22:22:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.015+0000 7fc82b7fe640 1 -- 172.21.15.5:0/1929320698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc8300041d0 con 0x7fc83c101420 2024-09-15T22:22:55.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.015+0000 7fc843279740 1 Processor -- start 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.015+0000 7fc843279740 1 -- start start 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.015+0000 7fc843279740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d427071b40 0x55d427070090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.015+0000 7fc843279740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4270705d0 0x55d427070a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.015+0000 7fc843279740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42707d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc843279740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d426fd62f0 con 0x55d42707b370 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc843279740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d426f7dcf0 con 0x55d4270705d0 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc843279740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d426fd22b0 con 0x55d427071b40 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc82b7fe640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc8240a3430 0x7fc8240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc839d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42707d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc839d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42707d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45076/0 (socket says 172.21.15.5:45076) 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc839d9b640 1 -- 172.21.15.5:0/93797172 learned_addr learned my addr 172.21.15.5:0/93797172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc83a59c640 1 --2- 172.21.15.5:0/93797172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4270705d0 0x55d427070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc83ad9d640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc8240a3430 0x7fc8240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc82b7fe640 1 -- 172.21.15.5:0/1929320698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7fc8300cf9f0 con 0x7fc83c101420 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc839d9b640 1 -- 172.21.15.5:0/93797172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d427071b40 msgr2=0x55d427070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc839d9b640 1 --2- 172.21.15.5:0/93797172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d427071b40 0x55d427070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc839d9b640 1 -- 172.21.15.5:0/93797172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4270705d0 msgr2=0x55d427070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc839d9b640 1 --2- 172.21.15.5:0/93797172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4270705d0 0x55d427070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc839d9b640 1 -- 172.21.15.5:0/93797172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d426f623a0 con 0x55d42707b370 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.016+0000 7fc839d9b640 1 --2- 172.21.15.5:0/93797172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42707d760 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc8300982a0 tx=0x7fc830005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3f080bcb5997d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.017+0000 7fc828ff9640 1 -- 172.21.15.5:0/93797172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83009b230 con 0x55d42707b370 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.017+0000 7fc83ad9d640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc8240a3430 0x7fc8240a58f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc834005fd0 tx=0x7fc834009b70 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.017+0000 7fc828ff9640 1 -- 172.21.15.5:0/93797172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc83009bfd0 con 0x55d42707b370 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.017+0000 7fc828ff9640 1 -- 172.21.15.5:0/93797172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8300cd120 con 0x55d42707b370 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.017+0000 7fc843279740 1 -- 172.21.15.5:0/93797172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 msgr2=0x55d42707d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:55.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.017+0000 7fc843279740 1 --2- 172.21.15.5:0/93797172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42707d760 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc8300982a0 tx=0x7fc830005bf0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 -- 172.21.15.5:0/93797172 shutdown_connections 2024-09-15T22:22:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 --2- 172.21.15.5:0/93797172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42707d760 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 --2- 172.21.15.5:0/93797172 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d4270705d0 0x55d427070a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 --2- 172.21.15.5:0/93797172 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d427071b40 0x55d427070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 -- 172.21.15.5:0/93797172 >> 172.21.15.5:0/93797172 conn(0x55d427039310 msgr2=0x55d42706cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 -- 172.21.15.5:0/93797172 shutdown_connections 2024-09-15T22:22:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 -- 172.21.15.5:0/93797172 wait complete. 2024-09-15T22:22:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 -- start start 2024-09-15T22:22:55.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 Processor -- start 2024-09-15T22:22:55.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d42707add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d427071b40 0x55d427078080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d427078810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d42707dfb0 con 0x55d42707b370 2024-09-15T22:22:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d426f91060 con 0x55d427071b40 2024-09-15T22:22:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d42707e130 con 0x55d4270705d0 2024-09-15T22:22:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.018+0000 7fc843279740 10 client.0 _ll_register_callbacks cb 0x55d426ce48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc83ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d427078810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc83a59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d42707add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc83ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d427078810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45084/0 (socket says 172.21.15.5:45084) 2024-09-15T22:22:55.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc83ad9d640 1 -- 172.21.15.5:0/1699880288 learned_addr learned my addr 172.21.15.5:0/1699880288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc839d9b640 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d427071b40 0x55d427078080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc83a59c640 1 -- 172.21.15.5:0/1699880288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d427071b40 msgr2=0x55d427078080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc83a59c640 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d427071b40 0x55d427078080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc83a59c640 1 -- 172.21.15.5:0/1699880288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 msgr2=0x55d427078810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc83a59c640 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d427078810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc83a59c640 1 -- 172.21.15.5:0/1699880288 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d426d473c0 con 0x55d4270705d0 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc839d9b640 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d427071b40 0x55d427078080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc83ad9d640 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d427078810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc83a59c640 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d42707add0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc82c005940 tx=0x7fc82c031920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e5a9b97a3d22fea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc821ffb640 1 -- 172.21.15.5:0/1699880288 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c035040 con 0x55d4270705d0 2024-09-15T22:22:55.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc821ffb640 1 -- 172.21.15.5:0/1699880288 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7fc82c004240 con 0x55d4270705d0 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.019+0000 7fc843279740 1 -- 172.21.15.5:0/1699880288 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc83009bfd0 con 0x55d4270705d0 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc821ffb640 1 -- 172.21.15.5:0/1699880288 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc82c04f050 con 0x55d4270705d0 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc843279740 10 client.27178 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc843279740 1 -- 172.21.15.5:0/1699880288 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d427089ad0 con 0x55d4270705d0 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc821ffb640 1 -- 172.21.15.5:0/1699880288 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc83009bfd0 con 0x55d4270705d0 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc821ffb640 1 -- 172.21.15.5:0/1699880288 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc82c004ba0 con 0x55d4270705d0 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc843279740 10 client.27178 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc843279740 10 client.27178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc843279740 20 client.27178 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc843279740 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d42708af90 0x55d42708d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc843279740 4 client.27178 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc843279740 1 -- 172.21.15.5:0/1699880288 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d426fe3860 con 0x55d42708af90 2024-09-15T22:22:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.020+0000 7fc839d9b640 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d42708af90 0x55d42708d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.021+0000 7fc821ffb640 10 client.27178 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.021+0000 7fc839d9b640 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d42708af90 0x55d42708d380 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a93813b19c4c4093 server_cookie=8974621949d97ec3 in_seq=0 out_seq=0 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.021+0000 7fc821ffb640 1 -- 172.21.15.5:0/1699880288 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d426fe3860 con 0x55d42708af90 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.021+0000 7fc821ffb640 10 client.27178 handle_command_reply: tid=0 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.021+0000 7fc843279740 1 client.27178 shutdown 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc843279740 1 -- 172.21.15.5:0/1699880288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 msgr2=0x55d42707add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc843279740 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d42707add0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc82c005940 tx=0x7fc82c031920 comp rx=0 tx=0).stop 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc843279740 1 -- 172.21.15.5:0/1699880288 shutdown_connections 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc843279740 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d427078810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc843279740 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d42708af90 0x55d42708d380 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc843279740 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d427071b40 0x55d427078080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc843279740 1 --2- 172.21.15.5:0/1699880288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d42707add0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc821ffb640 0 client.27178 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc843279740 1 -- 172.21.15.5:0/1699880288 >> 172.21.15.5:0/1699880288 conn(0x55d427039310 msgr2=0x55d427072990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc843279740 1 -- 172.21.15.5:0/1699880288 shutdown_connections 2024-09-15T22:22:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc843279740 1 -- 172.21.15.5:0/1699880288 wait complete. 2024-09-15T22:22:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.022+0000 7fc843279740 20 client.27178 trim_cache size 0 max 16384 2024-09-15T22:22:55.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc843279740 1 Processor -- start 2024-09-15T22:22:55.037 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:54 smithi005 ceph-mon[30944]: pgmap v437: 129 pgs: 129 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.5 MiB/s wr, 459 op/s 2024-09-15T22:22:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc843279740 1 -- start start 2024-09-15T22:22:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc843279740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d427071300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc843279740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d427071b40 0x55d42706c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc843279740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42706cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc843279740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1b3cc0 con 0x55d42707b370 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc843279740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1b1590 con 0x55d427071b40 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc843279740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1af400 con 0x55d4270705d0 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc83ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42706cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc83ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42706cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45088/0 (socket says 172.21.15.5:45088) 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc83ad9d640 1 -- 172.21.15.5:0/849474378 learned_addr learned my addr 172.21.15.5:0/849474378 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc83a59c640 1 --2- 172.21.15.5:0/849474378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d427071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc839d9b640 1 --2- 172.21.15.5:0/849474378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d427071b40 0x55d42706c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc83ad9d640 1 -- 172.21.15.5:0/849474378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 msgr2=0x55d427071300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc83ad9d640 1 --2- 172.21.15.5:0/849474378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d427071300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.035+0000 7fc83ad9d640 1 -- 172.21.15.5:0/849474378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d427071b40 msgr2=0x55d42706c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.036+0000 7fc83ad9d640 1 --2- 172.21.15.5:0/849474378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d427071b40 0x55d42706c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.036+0000 7fc83ad9d640 1 -- 172.21.15.5:0/849474378 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d42708d9b0 con 0x55d42707b370 2024-09-15T22:22:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.036+0000 7fc83ad9d640 1 --2- 172.21.15.5:0/849474378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42706cc40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc834004340 tx=0x7fc834016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0e66178e682b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.036+0000 7fc823fff640 1 -- 172.21.15.5:0/849474378 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834004b20 con 0x55d42707b370 2024-09-15T22:22:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.036+0000 7fc823fff640 1 -- 172.21.15.5:0/849474378 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc834004cc0 con 0x55d42707b370 2024-09-15T22:22:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.036+0000 7fc823fff640 1 -- 172.21.15.5:0/849474378 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834027630 con 0x55d42707b370 2024-09-15T22:22:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.037+0000 7fc843279740 1 -- 172.21.15.5:0/849474378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 msgr2=0x55d42706cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.037+0000 7fc843279740 1 --2- 172.21.15.5:0/849474378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42706cc40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc834004340 tx=0x7fc834016000 comp rx=0 tx=0).stop 2024-09-15T22:22:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.037+0000 7fc843279740 1 -- 172.21.15.5:0/849474378 shutdown_connections 2024-09-15T22:22:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.037+0000 7fc843279740 1 --2- 172.21.15.5:0/849474378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d42707b370 0x55d42706cc40 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.037+0000 7fc843279740 1 --2- 172.21.15.5:0/849474378 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d427071b40 0x55d42706c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.037+0000 7fc843279740 1 --2- 172.21.15.5:0/849474378 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d427071300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.037+0000 7fc843279740 1 -- 172.21.15.5:0/849474378 >> 172.21.15.5:0/849474378 conn(0x55d427039310 msgr2=0x55d427067a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.037+0000 7fc843279740 1 -- 172.21.15.5:0/849474378 shutdown_connections 2024-09-15T22:22:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.037+0000 7fc843279740 1 -- 172.21.15.5:0/849474378 wait complete. 2024-09-15T22:22:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.037+0000 7fc843279740 1 -- start start 2024-09-15T22:22:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.037+0000 7fc843279740 1 Processor -- start 2024-09-15T22:22:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc843279740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d42707e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc843279740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d427071b40 0x55d42707ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc843279740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d42707b370 0x55d4270fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc843279740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1afa50 con 0x55d427071b40 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc843279740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1b0ec0 con 0x55d42707b370 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc843279740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc83c1b0120 con 0x55d4270705d0 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc843279740 10 client.0 _ll_register_callbacks cb 0x7fc82c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc839d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d427071b40 0x55d42707ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc839d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d427071b40 0x55d42707ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45092/0 (socket says 172.21.15.5:45092) 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc839d9b640 1 -- 172.21.15.5:0/659468264 learned_addr learned my addr 172.21.15.5:0/659468264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc83a59c640 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d42707e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc83ad9d640 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d42707b370 0x55d4270fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc839d9b640 1 -- 172.21.15.5:0/659468264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 msgr2=0x55d42707e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc839d9b640 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d42707e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc839d9b640 1 -- 172.21.15.5:0/659468264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d42707b370 msgr2=0x55d4270fc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:22:55.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc839d9b640 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d42707b370 0x55d4270fc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc839d9b640 1 -- 172.21.15.5:0/659468264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4270fc910 con 0x55d427071b40 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc839d9b640 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d427071b40 0x55d42707ec80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc830097d30 tx=0x7fc83009b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e648f373e86c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.038+0000 7fc821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc821ffb640 1 -- 172.21.15.5:0/659468264 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8300b3070 con 0x55d427071b40 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc821ffb640 1 -- 172.21.15.5:0/659468264 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7fc8300ccfc0 con 0x55d427071b40 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc843279740 1 -- 172.21.15.5:0/659468264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc834004cc0 con 0x55d427071b40 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc821ffb640 1 -- 172.21.15.5:0/659468264 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc830014070 con 0x55d427071b40 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc843279740 10 client.17772 fetch_fsmap learned FSMap version 25 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc843279740 1 -- 172.21.15.5:0/659468264 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d4270fda80 con 0x55d427071b40 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc821ffb640 1 -- 172.21.15.5:0/659468264 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc834004cc0 con 0x55d427071b40 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc821ffb640 1 -- 172.21.15.5:0/659468264 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fc830097f40 con 0x55d427071b40 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc843279740 10 client.17772 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc843279740 10 client.17772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:22:55.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc843279740 20 client.17772 populate_metadata read hostname 'smithi005' 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc843279740 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d4270fe250 0x55d427100670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc843279740 4 client.17772 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc843279740 1 -- 172.21.15.5:0/659468264 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d426fddd70 con 0x55d4270fe250 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.039+0000 7fc83a59c640 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d4270fe250 0x55d427100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc83a59c640 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d4270fe250 0x55d427100670 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c85c3ceb1f9d97 server_cookie=afce57bae88cce94 in_seq=0 out_seq=0 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc821ffb640 10 client.17772 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc821ffb640 1 -- 172.21.15.5:0/659468264 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d426fddd70 con 0x55d4270fe250 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc821ffb640 10 client.17772 handle_command_reply: tid=0 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc843279740 1 client.17772 shutdown 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc843279740 1 -- 172.21.15.5:0/659468264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d427071b40 msgr2=0x55d42707ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc843279740 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d427071b40 0x55d42707ec80 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc830097d30 tx=0x7fc83009b8d0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc843279740 1 -- 172.21.15.5:0/659468264 shutdown_connections 2024-09-15T22:22:55.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc843279740 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d4270fe250 0x55d427100670 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc843279740 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d42707b370 0x55d4270fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc843279740 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d427071b40 0x55d42707ec80 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc843279740 1 --2- 172.21.15.5:0/659468264 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d4270705d0 0x55d42707e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc843279740 1 -- 172.21.15.5:0/659468264 >> 172.21.15.5:0/659468264 conn(0x55d427039310 msgr2=0x55d42706c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:55.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc821ffb640 0 client.17772 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:22:55.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc843279740 1 -- 172.21.15.5:0/659468264 shutdown_connections 2024-09-15T22:22:55.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.040+0000 7fc843279740 1 -- 172.21.15.5:0/659468264 wait complete. 2024-09-15T22:22:55.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc843279740 20 client.17772 trim_cache size 0 max 16384 2024-09-15T22:22:55.049 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:22:55.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 -- 172.21.15.5:0/1929320698 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc8240a3430 msgr2=0x7fc8240a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:55.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc8240a3430 0x7fc8240a58f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc834005fd0 tx=0x7fc834009b70 comp rx=0 tx=0).stop 2024-09-15T22:22:55.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 -- 172.21.15.5:0/1929320698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c101420 msgr2=0x7fc83c0fd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:22:55.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c101420 0x7fc83c0fd2d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc83000d950 tx=0x7fc83000de10 comp rx=0 tx=0).stop 2024-09-15T22:22:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 -- 172.21.15.5:0/1929320698 shutdown_connections 2024-09-15T22:22:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc8240a3430 0x7fc8240a58f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc83c106a80 0x7fc83c0fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc83c101420 0x7fc83c0fd2d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 --2- 172.21.15.5:0/1929320698 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc83c100a80 0x7fc83c0fcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:22:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 -- 172.21.15.5:0/1929320698 >> 172.21.15.5:0/1929320698 conn(0x7fc83c0fc7a0 msgr2=0x7fc83c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:22:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 -- 172.21.15.5:0/1929320698 shutdown_connections 2024-09-15T22:22:55.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:22:55.041+0000 7fc840ea0640 1 -- 172.21.15.5:0/1929320698 wait complete. 2024-09-15T22:22:55.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:54 smithi099 ceph-mon[34892]: pgmap v437: 129 pgs: 129 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.5 MiB/s wr, 459 op/s 2024-09-15T22:22:55.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:54 smithi129 ceph-mon[35314]: pgmap v437: 129 pgs: 129 active+clean; 2.9 GiB data, 24 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 4.5 MiB/s wr, 459 op/s 2024-09-15T22:22:55.745 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fedc988e', quiesce took 7.5 seconds. Will release after: 19.0 2024-09-15T22:22:57.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:56 smithi005 ceph-mon[30944]: pgmap v438: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.5 MiB/s wr, 455 op/s 2024-09-15T22:22:57.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:56 smithi099 ceph-mon[34892]: pgmap v438: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.5 MiB/s wr, 455 op/s 2024-09-15T22:22:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:56 smithi129 ceph-mon[35314]: pgmap v438: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 4.5 MiB/s wr, 455 op/s 2024-09-15T22:22:59.269 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:22:59.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:22:59.299 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:22:58 smithi005 ceph-mon[30944]: pgmap v439: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.6 MiB/s wr, 470 op/s 2024-09-15T22:22:59.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:22:58 smithi099 ceph-mon[34892]: pgmap v439: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.6 MiB/s wr, 470 op/s 2024-09-15T22:22:59.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:22:58 smithi129 ceph-mon[35314]: pgmap v439: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 4.6 MiB/s wr, 470 op/s 2024-09-15T22:22:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:22:59.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 810.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:23:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.013+0000 7f6f07970640 1 -- 172.21.15.5:0/3449987827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f00101de0 msgr2=0x7f6f001021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.013+0000 7f6f07970640 1 --2- 172.21.15.5:0/3449987827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f00101de0 0x7f6f001021c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009900 tx=0x7f6efc02a410 comp rx=0 tx=0).stop 2024-09-15T22:23:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.013+0000 7f6f07970640 1 -- 172.21.15.5:0/3449987827 shutdown_connections 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.013+0000 7f6f07970640 1 --2- 172.21.15.5:0/3449987827 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00106f10 0x7f6f0010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.013+0000 7f6f07970640 1 --2- 172.21.15.5:0/3449987827 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f00102790 0x7f6f001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.013+0000 7f6f07970640 1 --2- 172.21.15.5:0/3449987827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f00101de0 0x7f6f001021c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.013+0000 7f6f07970640 1 -- 172.21.15.5:0/3449987827 >> 172.21.15.5:0/3449987827 conn(0x7f6f000fd8c0 msgr2=0x7f6f000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.013+0000 7f6f07970640 1 -- 172.21.15.5:0/3449987827 shutdown_connections 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.013+0000 7f6f07970640 1 -- 172.21.15.5:0/3449987827 wait complete. 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.014+0000 7f6f07970640 1 Processor -- start 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.014+0000 7f6f07970640 1 -- start start 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.014+0000 7f6f07970640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f00101de0 0x7f6f001984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.014+0000 7f6f07970640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00102790 0x7f6f00198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.014+0000 7f6f07970640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f00106f10 0x7f6f0019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.014+0000 7f6f07970640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f00111f90 con 0x7f6f00101de0 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.014+0000 7f6f07970640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f00111e10 con 0x7f6f00102790 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.014+0000 7f6f07970640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6f00112110 con 0x7f6f00106f10 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.014+0000 7f6f0570c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f00101de0 0x7f6f001984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.014+0000 7f6f0570c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f00101de0 0x7f6f001984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45112/0 (socket says 172.21.15.5:45112) 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.014+0000 7f6f0570c640 1 -- 172.21.15.5:0/644331645 learned_addr learned my addr 172.21.15.5:0/644331645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f00106f10 0x7f6f0019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f04f0b640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00102790 0x7f6f00198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f0570c640 1 -- 172.21.15.5:0/644331645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f00106f10 msgr2=0x7f6f0019cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f0570c640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f00106f10 0x7f6f0019cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f0570c640 1 -- 172.21.15.5:0/644331645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00102790 msgr2=0x7f6f00198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f0570c640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00102790 0x7f6f00198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f0570c640 1 -- 172.21.15.5:0/644331645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f0019d480 con 0x7f6f00101de0 2024-09-15T22:23:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f04f0b640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00102790 0x7f6f00198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f00106f10 0x7f6f0019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f0570c640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f00101de0 0x7f6f001984d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6efc005ab0 tx=0x7f6efc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6ef67fc640 1 -- 172.21.15.5:0/644331645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc03f030 con 0x7f6f00101de0 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6ef67fc640 1 -- 172.21.15.5:0/644331645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc003ce0 con 0x7f6f00101de0 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f07970640 1 -- 172.21.15.5:0/644331645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f0019d710 con 0x7f6f00101de0 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6ef67fc640 1 -- 172.21.15.5:0/644331645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc035070 con 0x7f6f00101de0 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.015+0000 7f6f07970640 1 -- 172.21.15.5:0/644331645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f001a5050 con 0x7f6f00101de0 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6ef67fc640 1 -- 172.21.15.5:0/644331645 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f6efc042030 con 0x7f6f00101de0 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f09d49740 1 Processor -- start 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f09d49740 1 -- start start 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85eb60 0x563efb85ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb869800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb869d40 0x563efb86c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f09d49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563efb7c4280 con 0x563efb85f480 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f09d49740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563efb76bb10 con 0x563efb85eb60 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f09d49740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563efb7c0240 con 0x563efb869d40 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f0570c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb869800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f0570c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb869800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45124/0 (socket says 172.21.15.5:45124) 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f0570c640 1 -- 172.21.15.5:0/1640976697 learned_addr learned my addr 172.21.15.5:0/1640976697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.018+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/1640976697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85eb60 0x563efb85ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6f04f0b640 1 --2- 172.21.15.5:0/1640976697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb869d40 0x563efb86c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6f0570c640 1 -- 172.21.15.5:0/1640976697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb869d40 msgr2=0x563efb86c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6f0570c640 1 --2- 172.21.15.5:0/1640976697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb869d40 0x563efb86c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6f0570c640 1 -- 172.21.15.5:0/1640976697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85eb60 msgr2=0x563efb85ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6f0570c640 1 --2- 172.21.15.5:0/1640976697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85eb60 0x563efb85ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6f0570c640 1 -- 172.21.15.5:0/1640976697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563efb7781f0 con 0x563efb85f480 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6ef67fc640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6eec0a33b0 0x7f6eec0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6eec0a33b0 0x7f6eec0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6f0570c640 1 --2- 172.21.15.5:0/1640976697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb869800 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0ed980 tx=0x7f6efc0f6cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6310cdaa3c544ea server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6ef67fc640 1 -- 172.21.15.5:0/644331645 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f6efc0f06c0 con 0x7f6f00101de0 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6eeb7fe640 1 -- 172.21.15.5:0/1640976697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc049070 con 0x563efb85f480 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6eeb7fe640 1 -- 172.21.15.5:0/1640976697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc0bcca0 con 0x563efb85f480 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.019+0000 7f6eeb7fe640 1 -- 172.21.15.5:0/1640976697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0bc370 con 0x563efb85f480 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.020+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6eec0a33b0 0x7f6eec0a5870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800aee0 tx=0x7f6ef800a4b0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.020+0000 7f6f09d49740 1 -- 172.21.15.5:0/1640976697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 msgr2=0x563efb869800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.020+0000 7f6f09d49740 1 --2- 172.21.15.5:0/1640976697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb869800 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0ed980 tx=0x7f6efc0f6cd0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.020+0000 7f6f09d49740 1 -- 172.21.15.5:0/1640976697 shutdown_connections 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.020+0000 7f6f09d49740 1 --2- 172.21.15.5:0/1640976697 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb869d40 0x563efb86c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.020+0000 7f6f09d49740 1 --2- 172.21.15.5:0/1640976697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb869800 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.020+0000 7f6f09d49740 1 --2- 172.21.15.5:0/1640976697 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85eb60 0x563efb85ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.020+0000 7f6f09d49740 1 -- 172.21.15.5:0/1640976697 >> 172.21.15.5:0/1640976697 conn(0x563efb85b0f0 msgr2=0x563efb85b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.020+0000 7f6f09d49740 1 -- 172.21.15.5:0/1640976697 shutdown_connections 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.020+0000 7f6f09d49740 1 -- 172.21.15.5:0/1640976697 wait complete. 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.020+0000 7f6f09d49740 1 -- start start 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f09d49740 1 Processor -- start 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb866bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb867130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb869d40 0x563efb867670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f09d49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563efb86c6d0 con 0x563efb85f480 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f09d49740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563efb77ef10 con 0x563efb869d40 2024-09-15T22:23:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f09d49740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563efb86c850 con 0x563efb85eb60 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f09d49740 10 client.0 _ll_register_callbacks cb 0x563efb747a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f04f0b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb867130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f04f0b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb867130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45132/0 (socket says 172.21.15.5:45132) 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f04f0b640 1 -- 172.21.15.5:0/682028966 learned_addr learned my addr 172.21.15.5:0/682028966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb869d40 0x563efb867670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f0570c640 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb866bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f04f0b640 1 -- 172.21.15.5:0/682028966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 msgr2=0x563efb866bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f04f0b640 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb866bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f04f0b640 1 -- 172.21.15.5:0/682028966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb869d40 msgr2=0x563efb867670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f04f0b640 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb869d40 0x563efb867670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.021+0000 7f6f04f0b640 1 -- 172.21.15.5:0/682028966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563efb5356a0 con 0x563efb85f480 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6f04f0b640 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb867130 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000e4d0 tx=0x7f6ef000e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e9f946fa320ebd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6ee8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:23:00.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/682028966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0017020 con 0x563efb85f480 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/682028966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f6ef0013660 con 0x563efb85f480 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6f09d49740 1 -- 172.21.15.5:0/682028966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6efc0bcca0 con 0x563efb85f480 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/682028966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ef0022050 con 0x563efb85f480 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6f09d49740 10 client.17784 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6f09d49740 1 -- 172.21.15.5:0/682028966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563efb878270 con 0x563efb85f480 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/682028966 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc0bcca0 con 0x563efb85f480 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/682028966 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6ef0012a80 con 0x563efb85f480 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6f09d49740 10 client.17784 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6f09d49740 10 client.17784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6f09d49740 20 client.17784 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.022+0000 7f6f09d49740 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563efb8797d0 0x563efb87bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.023+0000 7f6f09d49740 4 client.17784 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.023+0000 7f6f09d49740 1 -- 172.21.15.5:0/682028966 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563efb7d17f0 con 0x563efb8797d0 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.023+0000 7f6f0570c640 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563efb8797d0 0x563efb87bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.023+0000 7f6f0570c640 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563efb8797d0 0x563efb87bc10 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f649847a500ec9ad server_cookie=873cbd1bac5612b6 in_seq=0 out_seq=0 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.023+0000 7f6ee8ff9640 10 client.17784 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/682028966 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6efc0a1e80 con 0x563efb8797d0 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6ee8ff9640 10 client.17784 handle_command_reply: tid=0 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 1 client.17784 shutdown 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 1 -- 172.21.15.5:0/682028966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 msgr2=0x563efb867130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb867130 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f6ef000e4d0 tx=0x7f6ef000e8d0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 1 -- 172.21.15.5:0/682028966 shutdown_connections 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563efb8797d0 0x563efb87bc10 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb869d40 0x563efb867670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb85f480 0x563efb867130 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6ee8ff9640 0 client.17784 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 1 --2- 172.21.15.5:0/682028966 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb866bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 1 -- 172.21.15.5:0/682028966 >> 172.21.15.5:0/682028966 conn(0x563efb85b0f0 msgr2=0x563efb860e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 1 -- 172.21.15.5:0/682028966 shutdown_connections 2024-09-15T22:23:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 1 -- 172.21.15.5:0/682028966 wait complete. 2024-09-15T22:23:00.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.024+0000 7f6f09d49740 20 client.17784 trim_cache size 0 max 16384 2024-09-15T22:23:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.042+0000 7f6f09d49740 1 Processor -- start 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.042+0000 7f6f09d49740 1 -- start start 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.042+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb865aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.042+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85f480 0x563efb8814e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.042+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb881a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.042+0000 7f6f09d49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f001b1af0 con 0x563efb869d40 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.042+0000 7f6f09d49740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f001af3c0 con 0x563efb85f480 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.042+0000 7f6f09d49740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6f001ad230 con 0x563efb85eb60 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.042+0000 7f6f05f0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb881a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.042+0000 7f6f05f0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb881a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45148/0 (socket says 172.21.15.5:45148) 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.042+0000 7f6f05f0d640 1 -- 172.21.15.5:0/96040071 learned_addr learned my addr 172.21.15.5:0/96040071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6f04f0b640 1 --2- 172.21.15.5:0/96040071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85f480 0x563efb8814e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6f0570c640 1 --2- 172.21.15.5:0/96040071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb865aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6f05f0d640 1 -- 172.21.15.5:0/96040071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 msgr2=0x563efb865aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/96040071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb865aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6f05f0d640 1 -- 172.21.15.5:0/96040071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85f480 msgr2=0x563efb8814e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/96040071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85f480 0x563efb8814e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6f05f0d640 1 -- 172.21.15.5:0/96040071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563efb8691d0 con 0x563efb869d40 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6f04f0b640 1 --2- 172.21.15.5:0/96040071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85f480 0x563efb8814e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6f0570c640 1 --2- 172.21.15.5:0/96040071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb865aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/96040071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb881a20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80118a0 tx=0x7f6ef8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104c4e5042222724 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6eeaffd640 1 -- 172.21.15.5:0/96040071 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef801fdc0 con 0x563efb869d40 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6eeaffd640 1 -- 172.21.15.5:0/96040071 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef80278c0 con 0x563efb869d40 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.043+0000 7f6eeaffd640 1 -- 172.21.15.5:0/96040071 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef801a790 con 0x563efb869d40 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.044+0000 7f6f09d49740 1 -- 172.21.15.5:0/96040071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 msgr2=0x563efb881a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.044+0000 7f6f09d49740 1 --2- 172.21.15.5:0/96040071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb881a20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80118a0 tx=0x7f6ef8013000 comp rx=0 tx=0).stop 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.044+0000 7f6f09d49740 1 -- 172.21.15.5:0/96040071 shutdown_connections 2024-09-15T22:23:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.044+0000 7f6f09d49740 1 --2- 172.21.15.5:0/96040071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb881a20 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.044+0000 7f6f09d49740 1 --2- 172.21.15.5:0/96040071 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85f480 0x563efb8814e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.044+0000 7f6f09d49740 1 --2- 172.21.15.5:0/96040071 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb865aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.044+0000 7f6f09d49740 1 -- 172.21.15.5:0/96040071 >> 172.21.15.5:0/96040071 conn(0x563efb85b0f0 msgr2=0x563efb865390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.044+0000 7f6f09d49740 1 -- 172.21.15.5:0/96040071 shutdown_connections 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.044+0000 7f6f09d49740 1 -- 172.21.15.5:0/96040071 wait complete. 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.044+0000 7f6f09d49740 1 -- start start 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.044+0000 7f6f09d49740 1 Processor -- start 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb86dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85f480 0x563efb86e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f09d49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb8e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f09d49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f001ad880 con 0x563efb869d40 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f09d49740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6f001aecf0 con 0x563efb85f480 2024-09-15T22:23:00.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f09d49740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6f001adf50 con 0x563efb85eb60 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f09d49740 10 client.0 _ll_register_callbacks cb 0x7f6ef000c1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f05f0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb8e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f05f0d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb8e34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45156/0 (socket says 172.21.15.5:45156) 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f0570c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb86dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f05f0d640 1 -- 172.21.15.5:0/2701441951 learned_addr learned my addr 172.21.15.5:0/2701441951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f04f0b640 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85f480 0x563efb86e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f05f0d640 1 -- 172.21.15.5:0/2701441951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 msgr2=0x563efb86dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb86dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f05f0d640 1 -- 172.21.15.5:0/2701441951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85f480 msgr2=0x563efb86e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85f480 0x563efb86e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.045+0000 7f6f05f0d640 1 -- 172.21.15.5:0/2701441951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563efb8e39f0 con 0x563efb869d40 2024-09-15T22:23:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6f05f0d640 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb8e34b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8027540 tx=0x7f6ef8027570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc5b9d35541af06 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6ee8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:23:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/2701441951 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8036020 con 0x563efb869d40 2024-09-15T22:23:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/2701441951 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f6ef8027a70 con 0x563efb869d40 2024-09-15T22:23:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6f09d49740 1 -- 172.21.15.5:0/2701441951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ef80278c0 con 0x563efb869d40 2024-09-15T22:23:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/2701441951 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ef8041050 con 0x563efb869d40 2024-09-15T22:23:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6f09d49740 10 client.17790 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6f09d49740 1 -- 172.21.15.5:0/2701441951 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563efb8e4b80 con 0x563efb869d40 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/2701441951 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef80278c0 con 0x563efb869d40 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/2701441951 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f6ef8026070 con 0x563efb869d40 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6f09d49740 10 client.17790 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6f09d49740 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6f09d49740 20 client.17790 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6f09d49740 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563efb8e5230 0x563efb8e7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6f09d49740 4 client.17790 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.046+0000 7f6f09d49740 1 -- 172.21.15.5:0/2701441951 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563efb7cbd00 con 0x563efb8e5230 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.047+0000 7f6f0570c640 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563efb8e5230 0x563efb8e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.047+0000 7f6f0570c640 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563efb8e5230 0x563efb8e7650 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8262ca24be016d56 server_cookie=e8e5c08ce3f590ca in_seq=0 out_seq=0 2024-09-15T22:23:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.047+0000 7f6ee8ff9640 10 client.17790 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6ee8ff9640 1 -- 172.21.15.5:0/2701441951 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563efb7cbd00 con 0x563efb8e5230 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6ee8ff9640 10 client.17790 handle_command_reply: tid=0 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 1 client.17790 shutdown 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 1 -- 172.21.15.5:0/2701441951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 msgr2=0x563efb8e34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb8e34b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8027540 tx=0x7f6ef8027570 comp rx=0 tx=0).stop 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 1 -- 172.21.15.5:0/2701441951 shutdown_connections 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563efb869d40 0x563efb8e34b0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563efb8e5230 0x563efb8e7650 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563efb85f480 0x563efb86e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6ee8ff9640 0 client.17790 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 1 --2- 172.21.15.5:0/2701441951 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563efb85eb60 0x563efb86dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 1 -- 172.21.15.5:0/2701441951 >> 172.21.15.5:0/2701441951 conn(0x563efb85b0f0 msgr2=0x563efb860e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 1 -- 172.21.15.5:0/2701441951 shutdown_connections 2024-09-15T22:23:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 1 -- 172.21.15.5:0/2701441951 wait complete. 2024-09-15T22:23:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.048+0000 7f6f09d49740 20 client.17790 trim_cache size 0 max 16384 2024-09-15T22:23:00.057 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:23:00.057 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:23:00.057 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "99aaaa09-a2e0-4ba7-a3c9-d4fa1e35d15c", 2024-09-15T22:23:00.057 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:23:00.057 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:23:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 -- 172.21.15.5:0/644331645 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6eec0a33b0 msgr2=0x7f6eec0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6eec0a33b0 0x7f6eec0a5870 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6ef800aee0 tx=0x7f6ef800a4b0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 -- 172.21.15.5:0/644331645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f00101de0 msgr2=0x7f6f001984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f00101de0 0x7f6f001984d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6efc005ab0 tx=0x7f6efc0042f0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 -- 172.21.15.5:0/644331645 shutdown_connections 2024-09-15T22:23:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f6eec0a33b0 0x7f6eec0a5870 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6f00106f10 0x7f6f0019cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6f00102790 0x7f6f00198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 --2- 172.21.15.5:0/644331645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f00101de0 0x7f6f001984d0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 -- 172.21.15.5:0/644331645 >> 172.21.15.5:0/644331645 conn(0x7f6f000fd8c0 msgr2=0x7f6f000fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 -- 172.21.15.5:0/644331645 shutdown_connections 2024-09-15T22:23:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:00.049+0000 7f6f07970640 1 -- 172.21.15.5:0/644331645 wait complete. 2024-09-15T22:23:00.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:23:01.154 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:01.275 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:00 smithi005 ceph-mon[30944]: pgmap v440: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.7 MiB/s wr, 372 op/s 2024-09-15T22:23:01.275 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:00 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:23:01.275 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:00 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:01.275 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:00 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:23:01.275 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:00 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:00 smithi099 ceph-mon[34892]: pgmap v440: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.7 MiB/s wr, 372 op/s 2024-09-15T22:23:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:00 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:23:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:00 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:00 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:23:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:00 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:00 smithi129 ceph-mon[35314]: pgmap v440: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 3.7 MiB/s wr, 372 op/s 2024-09-15T22:23:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:00 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:23:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:00 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:00 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:23:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:00 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:01.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.637+0000 7f061e2cd640 1 -- 172.21.15.5:0/1410398678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618102ef0 msgr2=0x7f061810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:01.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.637+0000 7f061e2cd640 1 --2- 172.21.15.5:0/1410398678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618102ef0 0x7f061810da40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0610009ca0 tx=0x7f061002a950 comp rx=0 tx=0).stop 2024-09-15T22:23:01.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.637+0000 7f061e2cd640 1 -- 172.21.15.5:0/1410398678 shutdown_connections 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.637+0000 7f061e2cd640 1 --2- 172.21.15.5:0/1410398678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618102ef0 0x7f061810da40 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.637+0000 7f061e2cd640 1 --2- 172.21.15.5:0/1410398678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0618102570 0x7f06181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.637+0000 7f061e2cd640 1 --2- 172.21.15.5:0/1410398678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0618100e20 0x7f0618101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.637+0000 7f061e2cd640 1 -- 172.21.15.5:0/1410398678 >> 172.21.15.5:0/1410398678 conn(0x7f06180fc7d0 msgr2=0x7f06180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.637+0000 7f061e2cd640 1 -- 172.21.15.5:0/1410398678 shutdown_connections 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.637+0000 7f061e2cd640 1 -- 172.21.15.5:0/1410398678 wait complete. 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061e2cd640 1 Processor -- start 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061e2cd640 1 -- start start 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061e2cd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0618100e20 0x7f0618197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061e2cd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0618102570 0x7f06181979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061e2cd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618102ef0 0x7f061819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061e2cd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0618110f80 con 0x7f0618102ef0 2024-09-15T22:23:01.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061e2cd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0618110e00 con 0x7f0618102570 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061e2cd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0618111100 con 0x7f0618100e20 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061c86a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618102ef0 0x7f061819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061c86a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618102ef0 0x7f061819bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45170/0 (socket says 172.21.15.5:45170) 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061c86a640 1 -- 172.21.15.5:0/2533083860 learned_addr learned my addr 172.21.15.5:0/2533083860 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f0617fff640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0618100e20 0x7f0618197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f06177fe640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0618102570 0x7f06181979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061c86a640 1 -- 172.21.15.5:0/2533083860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0618100e20 msgr2=0x7f0618197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061c86a640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0618100e20 0x7f0618197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061c86a640 1 -- 172.21.15.5:0/2533083860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0618102570 msgr2=0x7f06181979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061c86a640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0618102570 0x7f06181979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f061c86a640 1 -- 172.21.15.5:0/2533083860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061819c410 con 0x7f0618102ef0 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.638+0000 7f06177fe640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0618102570 0x7f06181979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:01.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.639+0000 7f061c86a640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618102ef0 0x7f061819bd30 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f061002ae70 tx=0x7f061002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.639+0000 7f06157fa640 1 -- 172.21.15.5:0/2533083860 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610034040 con 0x7f0618102ef0 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.639+0000 7f061e2cd640 1 -- 172.21.15.5:0/2533083860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f061819c6a0 con 0x7f0618102ef0 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.639+0000 7f06157fa640 1 -- 172.21.15.5:0/2533083860 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0610030920 con 0x7f0618102ef0 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.639+0000 7f06157fa640 1 -- 172.21.15.5:0/2533083860 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610045480 con 0x7f0618102ef0 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.639+0000 7f061e2cd640 1 -- 172.21.15.5:0/2533083860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06181a3fe0 con 0x7f0618102ef0 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.641+0000 7f06157fa640 1 -- 172.21.15.5:0/2533083860 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f0610037020 con 0x7f0618102ef0 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.641+0000 7f06206a6740 1 Processor -- start 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.641+0000 7f06206a6740 1 -- start start 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06206a6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd44a0 0x555e36fd4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06206a6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd4dc0 0x555e36fdf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06206a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fe1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06206a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e36f3a2e0 con 0x555e36fdf680 2024-09-15T22:23:01.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06206a6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555e36ee1ce0 con 0x555e36fd44a0 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06206a6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555e36f362a0 con 0x555e36fd4dc0 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fe1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fe1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45176/0 (socket says 172.21.15.5:45176) 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06177fe640 1 -- 172.21.15.5:0/4020966614 learned_addr learned my addr 172.21.15.5:0/4020966614 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06157fa640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06000a33b0 0x7f06000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f0617fff640 1 --2- 172.21.15.5:0/4020966614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd4dc0 0x555e36fdf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06177fe640 1 --2- 172.21.15.5:0/4020966614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd44a0 0x555e36fd4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f061c86a640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06000a33b0 0x7f06000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06177fe640 1 -- 172.21.15.5:0/4020966614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd4dc0 msgr2=0x555e36fdf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06177fe640 1 --2- 172.21.15.5:0/4020966614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd4dc0 0x555e36fdf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06177fe640 1 -- 172.21.15.5:0/4020966614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd44a0 msgr2=0x555e36fd4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06177fe640 1 --2- 172.21.15.5:0/4020966614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd44a0 0x555e36fd4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06177fe640 1 -- 172.21.15.5:0/4020966614 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e36ec6440 con 0x555e36fdf680 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f0617fff640 1 --2- 172.21.15.5:0/4020966614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd4dc0 0x555e36fdf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06157fa640 1 -- 172.21.15.5:0/2533083860 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f06100f0ef0 con 0x7f0618102ef0 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.642+0000 7f06177fe640 1 --2- 172.21.15.5:0/4020966614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fe1a70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f060c00cf70 tx=0x7f060c00c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a0a209e75c4b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f05fe7fc640 1 -- 172.21.15.5:0/4020966614 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c018030 con 0x555e36fdf680 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f05fe7fc640 1 -- 172.21.15.5:0/4020966614 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f060c005500 con 0x555e36fdf680 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f05fe7fc640 1 -- 172.21.15.5:0/4020966614 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c010860 con 0x555e36fdf680 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f061c86a640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06000a33b0 0x7f06000a5870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f06100ee400 tx=0x7f061008cb00 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f06206a6740 1 -- 172.21.15.5:0/4020966614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 msgr2=0x555e36fe1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f06206a6740 1 --2- 172.21.15.5:0/4020966614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fe1a70 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f060c00cf70 tx=0x7f060c00c600 comp rx=0 tx=0).stop 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f06206a6740 1 -- 172.21.15.5:0/4020966614 shutdown_connections 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f06206a6740 1 --2- 172.21.15.5:0/4020966614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fe1a70 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f06206a6740 1 --2- 172.21.15.5:0/4020966614 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd4dc0 0x555e36fdf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f06206a6740 1 --2- 172.21.15.5:0/4020966614 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd44a0 0x555e36fd4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f06206a6740 1 -- 172.21.15.5:0/4020966614 >> 172.21.15.5:0/4020966614 conn(0x555e36fd0a30 msgr2=0x555e36fd0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:01.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f06206a6740 1 -- 172.21.15.5:0/4020966614 shutdown_connections 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.643+0000 7f06206a6740 1 -- 172.21.15.5:0/4020966614 wait complete. 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.644+0000 7f06206a6740 1 -- start start 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.644+0000 7f06206a6740 1 Processor -- start 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.644+0000 7f06206a6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd44a0 0x555e36fdf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.644+0000 7f06206a6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd4dc0 0x555e36fdc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.644+0000 7f06206a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fdc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.644+0000 7f06206a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e36fe2250 con 0x555e36fdf680 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.644+0000 7f06206a6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555e36ef5050 con 0x555e36fd4dc0 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.644+0000 7f06206a6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555e36fe23d0 con 0x555e36fd44a0 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.644+0000 7f06206a6740 10 client.0 _ll_register_callbacks cb 0x555e36f35160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.644+0000 7f06177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fdc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.644+0000 7f06177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fdc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45180/0 (socket says 172.21.15.5:45180) 2024-09-15T22:23:01.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f06177fe640 1 -- 172.21.15.5:0/3522143416 learned_addr learned my addr 172.21.15.5:0/3522143416 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f06177fe640 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd44a0 0x555e36fdf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f0617fff640 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd4dc0 0x555e36fdc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f06177fe640 1 -- 172.21.15.5:0/3522143416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd44a0 msgr2=0x555e36fdf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f06177fe640 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd44a0 0x555e36fdf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f06177fe640 1 -- 172.21.15.5:0/3522143416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd4dc0 msgr2=0x555e36fdc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f06177fe640 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd4dc0 0x555e36fdc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f06177fe640 1 -- 172.21.15.5:0/3522143416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e36cab3c0 con 0x555e36fdf680 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f06177fe640 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fdc840 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f060c001eb0 tx=0x7f060c004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e44ab54638831d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f05f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f05f77fe640 1 -- 172.21.15.5:0/3522143416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c018050 con 0x555e36fdf680 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f06206a6740 1 -- 172.21.15.5:0/3522143416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f060c005500 con 0x555e36fdf680 2024-09-15T22:23:01.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f05f77fe640 1 -- 172.21.15.5:0/3522143416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f060c0048d0 con 0x555e36fdf680 2024-09-15T22:23:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f05f77fe640 1 -- 172.21.15.5:0/3522143416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f060c034050 con 0x555e36fdf680 2024-09-15T22:23:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.645+0000 7f06206a6740 10 client.17802 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.646+0000 7f06206a6740 1 -- 172.21.15.5:0/3522143416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e36fedc40 con 0x555e36fdf680 2024-09-15T22:23:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.646+0000 7f05f77fe640 1 -- 172.21.15.5:0/3522143416 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f060c005500 con 0x555e36fdf680 2024-09-15T22:23:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.646+0000 7f05f77fe640 1 -- 172.21.15.5:0/3522143416 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f060c01d5f0 con 0x555e36fdf680 2024-09-15T22:23:01.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.646+0000 7f06206a6740 10 client.17802 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.646+0000 7f06206a6740 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.646+0000 7f06206a6740 20 client.17802 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.646+0000 7f06206a6740 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e36feeff0 0x555e36ff13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.646+0000 7f06206a6740 4 client.17802 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.646+0000 7f06206a6740 1 -- 172.21.15.5:0/3522143416 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e36f47850 con 0x555e36feeff0 2024-09-15T22:23:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.646+0000 7f0617fff640 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e36feeff0 0x555e36ff13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.647+0000 7f05f77fe640 10 client.17802 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.647+0000 7f0617fff640 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e36feeff0 0x555e36ff13e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7df0eba2b062926 server_cookie=50d6ad9438e2424c in_seq=0 out_seq=0 2024-09-15T22:23:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.647+0000 7f05f77fe640 1 -- 172.21.15.5:0/3522143416 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e36f47850 con 0x555e36feeff0 2024-09-15T22:23:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.647+0000 7f05f77fe640 10 client.17802 handle_command_reply: tid=0 2024-09-15T22:23:01.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.647+0000 7f06206a6740 1 client.17802 shutdown 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f06206a6740 1 -- 172.21.15.5:0/3522143416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 msgr2=0x555e36fdc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f06206a6740 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fdc840 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f060c001eb0 tx=0x7f060c004de0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f06206a6740 1 -- 172.21.15.5:0/3522143416 shutdown_connections 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f06206a6740 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fdf680 0x555e36fdc840 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f06206a6740 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e36feeff0 0x555e36ff13e0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f06206a6740 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd4dc0 0x555e36fdc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f06206a6740 1 --2- 172.21.15.5:0/3522143416 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd44a0 0x555e36fdf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f05f77fe640 0 client.17802 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f06206a6740 1 -- 172.21.15.5:0/3522143416 >> 172.21.15.5:0/3522143416 conn(0x555e36fd0a30 msgr2=0x555e36fd6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f06206a6740 1 -- 172.21.15.5:0/3522143416 shutdown_connections 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f06206a6740 1 -- 172.21.15.5:0/3522143416 wait complete. 2024-09-15T22:23:01.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.648+0000 7f06206a6740 20 client.17802 trim_cache size 0 max 16384 2024-09-15T22:23:01.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f06206a6740 1 Processor -- start 2024-09-15T22:23:01.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f06206a6740 1 -- start start 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f06206a6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd44a0 0x555e36feb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f06206a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fd7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f06206a6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fdf680 0x555e36ff6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f06206a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06181b07e0 con 0x555e36fd4dc0 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f06206a6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06181ae0b0 con 0x555e36fd44a0 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f06206a6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06181abf20 con 0x555e36fdf680 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f0617fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fd7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f0617fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fd7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45186/0 (socket says 172.21.15.5:45186) 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f0617fff640 1 -- 172.21.15.5:0/2999989527 learned_addr learned my addr 172.21.15.5:0/2999989527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f06177fe640 1 --2- 172.21.15.5:0/2999989527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd44a0 0x555e36feb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f06177fe640 1 --2- 172.21.15.5:0/2999989527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fdf680 0x555e36ff6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f0617fff640 1 -- 172.21.15.5:0/2999989527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fdf680 msgr2=0x555e36ff6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:01.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f0617fff640 1 --2- 172.21.15.5:0/2999989527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fdf680 0x555e36ff6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f0617fff640 1 -- 172.21.15.5:0/2999989527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd44a0 msgr2=0x555e36feb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f0617fff640 1 --2- 172.21.15.5:0/2999989527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd44a0 0x555e36feb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f0617fff640 1 -- 172.21.15.5:0/2999989527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e36fdf380 con 0x555e36fd4dc0 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.666+0000 7f06177fe640 1 --2- 172.21.15.5:0/2999989527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fdf680 0x555e36ff6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.667+0000 7f0617fff640 1 --2- 172.21.15.5:0/2999989527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fd7700 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x555e36fdbb50 tx=0x7f060800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e071909cfdec1f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.667+0000 7f05fdffb640 1 -- 172.21.15.5:0/2999989527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0608003ea0 con 0x555e36fd4dc0 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.667+0000 7f05fdffb640 1 -- 172.21.15.5:0/2999989527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0608005500 con 0x555e36fd4dc0 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.667+0000 7f05fdffb640 1 -- 172.21.15.5:0/2999989527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060801d6d0 con 0x555e36fd4dc0 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.667+0000 7f06206a6740 1 -- 172.21.15.5:0/2999989527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 msgr2=0x555e36fd7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.667+0000 7f06206a6740 1 --2- 172.21.15.5:0/2999989527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fd7700 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x555e36fdbb50 tx=0x7f060800be20 comp rx=0 tx=0).stop 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 -- 172.21.15.5:0/2999989527 shutdown_connections 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 --2- 172.21.15.5:0/2999989527 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fdf680 0x555e36ff6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 --2- 172.21.15.5:0/2999989527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fd7700 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 --2- 172.21.15.5:0/2999989527 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fd44a0 0x555e36feb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 -- 172.21.15.5:0/2999989527 >> 172.21.15.5:0/2999989527 conn(0x555e36fd0a30 msgr2=0x555e36fde340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 -- 172.21.15.5:0/2999989527 shutdown_connections 2024-09-15T22:23:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 -- 172.21.15.5:0/2999989527 wait complete. 2024-09-15T22:23:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 -- start start 2024-09-15T22:23:01.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 Processor -- start 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd44a0 0x555e370552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fe7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fdf680 0x555e36fe7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06181ac570 con 0x555e36fd4dc0 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f06181ad9e0 con 0x555e36fdf680 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f06181acc40 con 0x555e36fd44a0 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.668+0000 7f06206a6740 10 client.0 _ll_register_callbacks cb 0x7f060c0123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f0617fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fe7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f06177fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd44a0 0x555e370552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f0617fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fe7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45196/0 (socket says 172.21.15.5:45196) 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f0617fff640 1 -- 172.21.15.5:0/1794994791 learned_addr learned my addr 172.21.15.5:0/1794994791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f06177fe640 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fdf680 0x555e36fe7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f0617fff640 1 -- 172.21.15.5:0/1794994791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd44a0 msgr2=0x555e370552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f0617fff640 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd44a0 0x555e370552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f0617fff640 1 -- 172.21.15.5:0/1794994791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fdf680 msgr2=0x555e36fe7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f0617fff640 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fdf680 0x555e36fe7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f0617fff640 1 -- 172.21.15.5:0/1794994791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e36fe8200 con 0x555e36fd4dc0 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f0617fff640 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fe7620 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0608000c00 tx=0x7f060800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7d4ac3057b2b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f05f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f05f77fe640 1 -- 172.21.15.5:0/1794994791 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06080051b0 con 0x555e36fd4dc0 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f05f77fe640 1 -- 172.21.15.5:0/1794994791 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9541+0+0 (secure 0 0 0) 0x7f0608004700 con 0x555e36fd4dc0 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.669+0000 7f06206a6740 1 -- 172.21.15.5:0/1794994791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0608005500 con 0x555e36fd4dc0 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f05f77fe640 1 -- 172.21.15.5:0/1794994791 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f060803a050 con 0x555e36fd4dc0 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f06206a6740 10 client.17808 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f06206a6740 1 -- 172.21.15.5:0/1794994791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e3705be30 con 0x555e36fd4dc0 2024-09-15T22:23:01.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f05f77fe640 1 -- 172.21.15.5:0/1794994791 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0608005500 con 0x555e36fd4dc0 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f05f77fe640 1 -- 172.21.15.5:0/1794994791 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f060801d8c0 con 0x555e36fd4dc0 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f06206a6740 10 client.17808 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f06206a6740 10 client.17808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f06206a6740 20 client.17808 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f06206a6740 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e3705c4f0 0x555e3705e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f06206a6740 4 client.17808 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f06206a6740 1 -- 172.21.15.5:0/1794994791 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e36f41d60 con 0x555e3705c4f0 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.670+0000 7f06177fe640 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e3705c4f0 0x555e3705e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f05f77fe640 10 client.17808 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06177fe640 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e3705c4f0 0x555e3705e910 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65cc4ad8f75c5e64 server_cookie=15e8ee6c07568758 in_seq=0 out_seq=0 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f05f77fe640 1 -- 172.21.15.5:0/1794994791 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555e36f41d60 con 0x555e3705c4f0 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f05f77fe640 10 client.17808 handle_command_reply: tid=0 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 1 client.17808 shutdown 2024-09-15T22:23:01.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 1 -- 172.21.15.5:0/1794994791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 msgr2=0x555e36fe7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:01.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fe7620 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0608000c00 tx=0x7f060800b700 comp rx=0 tx=0).stop 2024-09-15T22:23:01.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 1 -- 172.21.15.5:0/1794994791 shutdown_connections 2024-09-15T22:23:01.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555e3705c4f0 0x555e3705e910 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555e36fdf680 0x555e36fe7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e36fd4dc0 0x555e36fe7620 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 1 --2- 172.21.15.5:0/1794994791 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555e36fd44a0 0x555e370552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f05f77fe640 0 client.17808 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:01.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 1 -- 172.21.15.5:0/1794994791 >> 172.21.15.5:0/1794994791 conn(0x555e36fd0a30 msgr2=0x555e36fcf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:01.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 1 -- 172.21.15.5:0/1794994791 shutdown_connections 2024-09-15T22:23:01.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 1 -- 172.21.15.5:0/1794994791 wait complete. 2024-09-15T22:23:01.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.671+0000 7f06206a6740 20 client.17808 trim_cache size 0 max 16384 2024-09-15T22:23:01.679 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:23:01.679 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:23:01.679 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:23:01.679 INFO:teuthology.orchestra.run.smithi005.stdout: "99aaaa09-a2e0-4ba7-a3c9-d4fa1e35d15c": { 2024-09-15T22:23:01.679 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:23:01.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "99aaaa09-a2e0-4ba7-a3c9-d4fa1e35d15c", 2024-09-15T22:23:01.679 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:23:01.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:23:01.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:23:01.679 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:23:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 -- 172.21.15.5:0/2533083860 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06000a33b0 msgr2=0x7f06000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06000a33b0 0x7f06000a5870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f06100ee400 tx=0x7f061008cb00 comp rx=0 tx=0).stop 2024-09-15T22:23:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 -- 172.21.15.5:0/2533083860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618102ef0 msgr2=0x7f061819bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618102ef0 0x7f061819bd30 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f061002ae70 tx=0x7f061002f5d0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 -- 172.21.15.5:0/2533083860 shutdown_connections 2024-09-15T22:23:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f06000a33b0 0x7f06000a5870 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0618102ef0 0x7f061819bd30 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0618102570 0x7f06181979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 --2- 172.21.15.5:0/2533083860 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f0618100e20 0x7f0618197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:01.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 -- 172.21.15.5:0/2533083860 >> 172.21.15.5:0/2533083860 conn(0x7f06180fc7d0 msgr2=0x7f06181068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:01.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 -- 172.21.15.5:0/2533083860 shutdown_connections 2024-09-15T22:23:01.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:01.672+0000 7f061e2cd640 1 -- 172.21.15.5:0/2533083860 wait complete. 2024-09-15T22:23:02.563 INFO:tasks.cephfs.filesystem:scrub status for tag:99aaaa09-a2e0-4ba7-a3c9-d4fa1e35d15c - {'path': '/', 'tag': '99aaaa09-a2e0-4ba7-a3c9-d4fa1e35d15c', 'options': 'recursive,force'} 2024-09-15T22:23:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:02 smithi099 ceph-mon[34892]: pgmap v441: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 3.5 MiB/s wr, 352 op/s 2024-09-15T22:23:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:02 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:23:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:02 smithi129 ceph-mon[35314]: pgmap v441: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 3.5 MiB/s wr, 352 op/s 2024-09-15T22:23:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:02 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:23:03.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:02 smithi005 ceph-mon[30944]: pgmap v441: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 3.5 MiB/s wr, 352 op/s 2024-09-15T22:23:03.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:02 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:23:04.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 815.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:23:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:04 smithi099 ceph-mon[34892]: pgmap v442: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-09-15T22:23:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:04 smithi129 ceph-mon[35314]: pgmap v442: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-09-15T22:23:05.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:04 smithi005 ceph-mon[30944]: pgmap v442: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.1 MiB/s wr, 212 op/s 2024-09-15T22:23:07.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:06 smithi099 ceph-mon[34892]: pgmap v443: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-09-15T22:23:07.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:06 smithi129 ceph-mon[35314]: pgmap v443: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-09-15T22:23:07.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:06 smithi005 ceph-mon[30944]: pgmap v443: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-09-15T22:23:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:09 smithi099 ceph-mon[34892]: pgmap v444: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 994 KiB/s wr, 117 op/s 2024-09-15T22:23:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:09 smithi099 ceph-mon[34892]: osdmap e104: 12 total, 12 up, 12 in 2024-09-15T22:23:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:09 smithi129 ceph-mon[35314]: pgmap v444: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 994 KiB/s wr, 117 op/s 2024-09-15T22:23:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:09 smithi129 ceph-mon[35314]: osdmap e104: 12 total, 12 up, 12 in 2024-09-15T22:23:09.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:08 smithi005 ceph-mon[30944]: pgmap v444: 129 pgs: 129 active+clean; 3.0 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 994 KiB/s wr, 117 op/s 2024-09-15T22:23:09.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:09 smithi005 ceph-mon[30944]: osdmap e104: 12 total, 12 up, 12 in 2024-09-15T22:23:09.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 820.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:23:11.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:11 smithi005 ceph-mon[30944]: pgmap v446: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-09-15T22:23:11.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:11 smithi099 ceph-mon[34892]: pgmap v446: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-09-15T22:23:11.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:11 smithi129 ceph-mon[35314]: pgmap v446: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 64 KiB/s wr, 27 op/s 2024-09-15T22:23:13.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:13 smithi099 ceph-mon[34892]: pgmap v447: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-09-15T22:23:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:13 smithi129 ceph-mon[35314]: pgmap v447: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-09-15T22:23:13.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:13 smithi005 ceph-mon[30944]: pgmap v447: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-09-15T22:23:14.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 825.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:23:14.747 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:23:15.026 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:15.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:15 smithi005 ceph-mon[30944]: pgmap v448: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-15T22:23:15.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:15 smithi099 ceph-mon[34892]: pgmap v448: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-15T22:23:15.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:15 smithi129 ceph-mon[35314]: pgmap v448: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 7 op/s 2024-09-15T22:23:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.653+0000 7f9145ca7640 1 -- 172.21.15.5:0/365874028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9140106a80 msgr2=0x7f9140106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.653+0000 7f9145ca7640 1 --2- 172.21.15.5:0/365874028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9140106a80 0x7f9140106e60 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f9130009930 tx=0x7f913002a440 comp rx=0 tx=0).stop 2024-09-15T22:23:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.653+0000 7f9145ca7640 1 -- 172.21.15.5:0/365874028 shutdown_connections 2024-09-15T22:23:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.653+0000 7f9145ca7640 1 --2- 172.21.15.5:0/365874028 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9140101420 0x7f914010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.653+0000 7f9145ca7640 1 --2- 172.21.15.5:0/365874028 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9140100a80 0x7f9140100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.653+0000 7f9145ca7640 1 --2- 172.21.15.5:0/365874028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9140106a80 0x7f9140106e60 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.653+0000 7f9145ca7640 1 -- 172.21.15.5:0/365874028 >> 172.21.15.5:0/365874028 conn(0x7f91400fc7a0 msgr2=0x7f91400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.653+0000 7f9145ca7640 1 -- 172.21.15.5:0/365874028 shutdown_connections 2024-09-15T22:23:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.653+0000 7f9145ca7640 1 -- 172.21.15.5:0/365874028 wait complete. 2024-09-15T22:23:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.653+0000 7f9145ca7640 1 Processor -- start 2024-09-15T22:23:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f9145ca7640 1 -- start start 2024-09-15T22:23:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f9145ca7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9140100a80 0x7f914019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f9145ca7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9140101420 0x7f914019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f9145ca7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9140106a80 0x7f914019ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f9145ca7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9140110e80 con 0x7f9140106a80 2024-09-15T22:23:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f9145ca7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9140110d00 con 0x7f9140100a80 2024-09-15T22:23:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f9145ca7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9140111000 con 0x7f9140101420 2024-09-15T22:23:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f913ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9140106a80 0x7f914019ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:15.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f913ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9140106a80 0x7f914019ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51150/0 (socket says 172.21.15.5:51150) 2024-09-15T22:23:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f913ffff640 1 -- 172.21.15.5:0/1019166680 learned_addr learned my addr 172.21.15.5:0/1019166680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f913f7fe640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9140100a80 0x7f914019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f913effd640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9140101420 0x7f914019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f913ffff640 1 -- 172.21.15.5:0/1019166680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9140101420 msgr2=0x7f914019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f913ffff640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9140101420 0x7f914019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f913ffff640 1 -- 172.21.15.5:0/1019166680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9140100a80 msgr2=0x7f914019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f913ffff640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9140100a80 0x7f914019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.654+0000 7f913ffff640 1 -- 172.21.15.5:0/1019166680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914019f510 con 0x7f9140106a80 2024-09-15T22:23:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.655+0000 7f913ffff640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9140106a80 0x7f914019ee30 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f913800cce0 tx=0x7f913800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:15.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.655+0000 7f913cff9640 1 -- 172.21.15.5:0/1019166680 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913800f450 con 0x7f9140106a80 2024-09-15T22:23:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.655+0000 7f913cff9640 1 -- 172.21.15.5:0/1019166680 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f913800fad0 con 0x7f9140106a80 2024-09-15T22:23:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.655+0000 7f9145ca7640 1 -- 172.21.15.5:0/1019166680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f914019f800 con 0x7f9140106a80 2024-09-15T22:23:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.655+0000 7f913cff9640 1 -- 172.21.15.5:0/1019166680 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9138018c10 con 0x7f9140106a80 2024-09-15T22:23:15.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.655+0000 7f9145ca7640 1 -- 172.21.15.5:0/1019166680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91401a70e0 con 0x7f9140106a80 2024-09-15T22:23:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.656+0000 7f9145ca7640 1 -- 172.21.15.5:0/1019166680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9140100ee0 con 0x7f9140106a80 2024-09-15T22:23:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.657+0000 7f913cff9640 1 -- 172.21.15.5:0/1019166680 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f913800f5f0 con 0x7f9140106a80 2024-09-15T22:23:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.658+0000 7f913cff9640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f91280a33b0 0x7f91280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.658+0000 7f913f7fe640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f91280a33b0 0x7f91280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.658+0000 7f913cff9640 1 -- 172.21.15.5:0/1019166680 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9629+0+0 (secure 0 0 0) 0x7f91380d0880 con 0x7f9140106a80 2024-09-15T22:23:15.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.659+0000 7f913f7fe640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f91280a33b0 0x7f91280a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9130009900 tx=0x7f91300058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:15.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.661+0000 7f913cff9640 1 -- 172.21.15.5:0/1019166680 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9138014030 con 0x7f9140106a80 2024-09-15T22:23:15.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.879+0000 7f9145ca7640 1 -- 172.21.15.5:0/1019166680 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f914019b8c0 con 0x7f9140106a80 2024-09-15T22:23:15.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.879+0000 7f913cff9640 1 -- 172.21.15.5:0/1019166680 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f914019b8c0 con 0x7f9140106a80 2024-09-15T22:23:15.882 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:23:15.882 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:23:15.882 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:23:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 -- 172.21.15.5:0/1019166680 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f91280a33b0 msgr2=0x7f91280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f91280a33b0 0x7f91280a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9130009900 tx=0x7f91300058d0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 -- 172.21.15.5:0/1019166680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9140106a80 msgr2=0x7f914019ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9140106a80 0x7f914019ee30 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f913800cce0 tx=0x7f913800a620 comp rx=0 tx=0).stop 2024-09-15T22:23:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 -- 172.21.15.5:0/1019166680 shutdown_connections 2024-09-15T22:23:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f91280a33b0 0x7f91280a5870 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9140106a80 0x7f914019ee30 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9140101420 0x7f914019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 --2- 172.21.15.5:0/1019166680 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9140100a80 0x7f914019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 -- 172.21.15.5:0/1019166680 >> 172.21.15.5:0/1019166680 conn(0x7f91400fc7a0 msgr2=0x7f914006a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 -- 172.21.15.5:0/1019166680 shutdown_connections 2024-09-15T22:23:15.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:15.882+0000 7f9145ca7640 1 -- 172.21.15.5:0/1019166680 wait complete. 2024-09-15T22:23:16.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:16 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1019166680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:16.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:16 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1019166680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:16.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:16 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1019166680' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:16.636 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id fedc988e --release --await' 2024-09-15T22:23:16.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id fedc988e --release --await 2024-09-15T22:23:16.901 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:17.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:17 smithi005 ceph-mon[30944]: pgmap v449: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-15T22:23:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:17 smithi099 ceph-mon[34892]: pgmap v449: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-15T22:23:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:17 smithi129 ceph-mon[35314]: pgmap v449: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-15T22:23:17.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.400+0000 7fcd13ffc640 1 -- 172.21.15.5:0/3195049867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd0c106a80 msgr2=0x7fcd0c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.400+0000 7fcd13ffc640 1 --2- 172.21.15.5:0/3195049867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd0c106a80 0x7fcd0c106e60 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009930 tx=0x7fcd0802a420 comp rx=0 tx=0).stop 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.400+0000 7fcd13ffc640 1 -- 172.21.15.5:0/3195049867 shutdown_connections 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.400+0000 7fcd13ffc640 1 --2- 172.21.15.5:0/3195049867 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd0c101420 0x7fcd0c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.400+0000 7fcd13ffc640 1 --2- 172.21.15.5:0/3195049867 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd0c100a80 0x7fcd0c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.400+0000 7fcd13ffc640 1 --2- 172.21.15.5:0/3195049867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd0c106a80 0x7fcd0c106e60 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.400+0000 7fcd13ffc640 1 -- 172.21.15.5:0/3195049867 >> 172.21.15.5:0/3195049867 conn(0x7fcd0c0fc7a0 msgr2=0x7fcd0c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.401+0000 7fcd13ffc640 1 -- 172.21.15.5:0/3195049867 shutdown_connections 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.401+0000 7fcd13ffc640 1 -- 172.21.15.5:0/3195049867 wait complete. 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.401+0000 7fcd13ffc640 1 Processor -- start 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.401+0000 7fcd13ffc640 1 -- start start 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.401+0000 7fcd13ffc640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd0c100a80 0x7fcd0c199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.401+0000 7fcd13ffc640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd0c101420 0x7fcd0c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.401+0000 7fcd13ffc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd0c106a80 0x7fcd0c1a43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.401+0000 7fcd13ffc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c110e80 con 0x7fcd0c106a80 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.401+0000 7fcd13ffc640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c110d00 con 0x7fcd0c100a80 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.401+0000 7fcd13ffc640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c111000 con 0x7fcd0c101420 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd12599640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd0c106a80 0x7fcd0c1a43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd12599640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd0c106a80 0x7fcd0c1a43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51168/0 (socket says 172.21.15.5:51168) 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd12599640 1 -- 172.21.15.5:0/4020249413 learned_addr learned my addr 172.21.15.5:0/4020249413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd11597640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd0c101420 0x7fcd0c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd11d98640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd0c100a80 0x7fcd0c199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd12599640 1 -- 172.21.15.5:0/4020249413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd0c101420 msgr2=0x7fcd0c199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd12599640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd0c101420 0x7fcd0c199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd12599640 1 -- 172.21.15.5:0/4020249413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd0c100a80 msgr2=0x7fcd0c199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd12599640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd0c100a80 0x7fcd0c199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd12599640 1 -- 172.21.15.5:0/4020249413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd0c1a4910 con 0x7fcd0c106a80 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd12599640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd0c106a80 0x7fcd0c1a43d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009fd0 tx=0x7fcd0400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd02ffd640 1 -- 172.21.15.5:0/4020249413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd04017070 con 0x7fcd0c106a80 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd13ffc640 1 -- 172.21.15.5:0/4020249413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd0c1a4ae0 con 0x7fcd0c106a80 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd02ffd640 1 -- 172.21.15.5:0/4020249413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd04003e50 con 0x7fcd0c106a80 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd02ffd640 1 -- 172.21.15.5:0/4020249413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd040106c0 con 0x7fcd0c106a80 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.402+0000 7fcd13ffc640 1 -- 172.21.15.5:0/4020249413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd0c1a4fa0 con 0x7fcd0c106a80 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd02ffd640 1 -- 172.21.15.5:0/4020249413 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcd04003970 con 0x7fcd0c106a80 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd163d5740 1 Processor -- start 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd163d5740 1 -- start start 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e060 0x55c901f0e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f18d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f19240 0x55c901f1b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd163d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c901e73620 con 0x55c901f0e980 2024-09-15T22:23:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd163d5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c901e1ae50 con 0x55c901f19240 2024-09-15T22:23:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd163d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c901e6f5e0 con 0x55c901f0e060 2024-09-15T22:23:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd11d98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f18d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd11d98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f18d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51184/0 (socket says 172.21.15.5:51184) 2024-09-15T22:23:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.405+0000 7fcd11d98640 1 -- 172.21.15.5:0/4006103457 learned_addr learned my addr 172.21.15.5:0/4006103457 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd02ffd640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fccf80a33b0 0x7fccf80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd11597640 1 --2- 172.21.15.5:0/4006103457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e060 0x55c901f0e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd11597640 1 --2- 172.21.15.5:0/4006103457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f19240 0x55c901f1b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd12599640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fccf80a33b0 0x7fccf80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd11d98640 1 -- 172.21.15.5:0/4006103457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e060 msgr2=0x55c901f0e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd11d98640 1 --2- 172.21.15.5:0/4006103457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e060 0x55c901f0e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd11d98640 1 -- 172.21.15.5:0/4006103457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f19240 msgr2=0x55c901f1b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd11d98640 1 --2- 172.21.15.5:0/4006103457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f19240 0x55c901f1b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd11d98640 1 -- 172.21.15.5:0/4006103457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c901e0bd40 con 0x55c901f0e980 2024-09-15T22:23:17.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd02ffd640 1 -- 172.21.15.5:0/4020249413 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9629+0+0 (secure 0 0 0) 0x7fcd040cfdf0 con 0x7fcd0c106a80 2024-09-15T22:23:17.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd11d98640 1 --2- 172.21.15.5:0/4006103457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f18d00 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcd0802a940 tx=0x7fcd08005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee4afae9403d487 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fcd12599640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fccf80a33b0 0x7fccf80a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd04098700 tx=0x7fcd040c6a50 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fccf7fff640 1 -- 172.21.15.5:0/4006103457 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08040070 con 0x55c901f0e980 2024-09-15T22:23:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fccf7fff640 1 -- 172.21.15.5:0/4006103457 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd08031930 con 0x55c901f0e980 2024-09-15T22:23:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.406+0000 7fccf7fff640 1 -- 172.21.15.5:0/4006103457 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08035040 con 0x55c901f0e980 2024-09-15T22:23:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.407+0000 7fcd163d5740 1 -- 172.21.15.5:0/4006103457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 msgr2=0x55c901f18d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.407+0000 7fcd163d5740 1 --2- 172.21.15.5:0/4006103457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f18d00 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcd0802a940 tx=0x7fcd08005c00 comp rx=0 tx=0).stop 2024-09-15T22:23:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.407+0000 7fcd163d5740 1 -- 172.21.15.5:0/4006103457 shutdown_connections 2024-09-15T22:23:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.407+0000 7fcd163d5740 1 --2- 172.21.15.5:0/4006103457 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f19240 0x55c901f1b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.407+0000 7fcd163d5740 1 --2- 172.21.15.5:0/4006103457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f18d00 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.407+0000 7fcd163d5740 1 --2- 172.21.15.5:0/4006103457 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e060 0x55c901f0e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.407+0000 7fcd163d5740 1 -- 172.21.15.5:0/4006103457 >> 172.21.15.5:0/4006103457 conn(0x55c901f0a5f0 msgr2=0x55c901f0a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.407+0000 7fcd163d5740 1 -- 172.21.15.5:0/4006103457 shutdown_connections 2024-09-15T22:23:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.407+0000 7fcd163d5740 1 -- 172.21.15.5:0/4006103457 wait complete. 2024-09-15T22:23:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd163d5740 1 -- start start 2024-09-15T22:23:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd163d5740 1 Processor -- start 2024-09-15T22:23:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e060 0x55c901f16060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f19240 0x55c901f16ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd163d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c901f1be10 con 0x55c901f0e980 2024-09-15T22:23:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd163d5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55c901dfbf10 con 0x55c901f19240 2024-09-15T22:23:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd163d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55c901f1bf90 con 0x55c901f0e060 2024-09-15T22:23:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd163d5740 10 client.0 _ll_register_callbacks cb 0x55c901e1fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd11d98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd11d98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f165a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51188/0 (socket says 172.21.15.5:51188) 2024-09-15T22:23:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd11d98640 1 -- 172.21.15.5:0/2900606497 learned_addr learned my addr 172.21.15.5:0/2900606497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd11597640 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e060 0x55c901f16060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.408+0000 7fcd11597640 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f19240 0x55c901f16ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fcd11d98640 1 -- 172.21.15.5:0/2900606497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e060 msgr2=0x55c901f16060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fcd11d98640 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e060 0x55c901f16060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fcd11d98640 1 -- 172.21.15.5:0/2900606497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f19240 msgr2=0x55c901f16ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fcd11d98640 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f19240 0x55c901f16ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fcd11d98640 1 -- 172.21.15.5:0/2900606497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c901b82580 con 0x55c901f0e980 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fcd11d98640 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f165a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcd08005ec0 tx=0x7fcd08031ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac1a882f53981ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fccf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fccf57fa640 1 -- 172.21.15.5:0/2900606497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08040030 con 0x55c901f0e980 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fccf57fa640 1 -- 172.21.15.5:0/2900606497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9629+0+0 (secure 0 0 0) 0x7fcd08004260 con 0x55c901f0e980 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fcd163d5740 1 -- 172.21.15.5:0/2900606497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd08031930 con 0x55c901f0e980 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fccf57fa640 1 -- 172.21.15.5:0/2900606497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd08056050 con 0x55c901f0e980 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fcd163d5740 10 client.17826 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fcd163d5740 1 -- 172.21.15.5:0/2900606497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c901c4c960 con 0x55c901f0e980 2024-09-15T22:23:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.409+0000 7fccf57fa640 1 -- 172.21.15.5:0/2900606497 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd08031930 con 0x55c901f0e980 2024-09-15T22:23:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.410+0000 7fccf57fa640 1 -- 172.21.15.5:0/2900606497 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcd080038a0 con 0x55c901f0e980 2024-09-15T22:23:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.410+0000 7fcd163d5740 10 client.17826 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.410+0000 7fcd163d5740 10 client.17826 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:23:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.410+0000 7fcd163d5740 20 client.17826 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.410+0000 7fcd163d5740 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c901f28b70 0x55c901f2af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.410+0000 7fcd163d5740 4 client.17826 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.410+0000 7fcd163d5740 1 -- 172.21.15.5:0/2900606497 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c901e80b90 con 0x55c901f28b70 2024-09-15T22:23:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.410+0000 7fcd11597640 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c901f28b70 0x55c901f2af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.410+0000 7fcd11597640 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c901f28b70 0x55c901f2af60 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c706136da3befc8 server_cookie=e4efb93ed91a4488 in_seq=0 out_seq=0 2024-09-15T22:23:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.410+0000 7fccf57fa640 10 client.17826 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.411+0000 7fccf57fa640 1 -- 172.21.15.5:0/2900606497 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c901e80b90 con 0x55c901f28b70 2024-09-15T22:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.411+0000 7fccf57fa640 10 client.17826 handle_command_reply: tid=0 2024-09-15T22:23:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.411+0000 7fcd163d5740 1 client.17826 shutdown 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.411+0000 7fcd163d5740 1 -- 172.21.15.5:0/2900606497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 msgr2=0x55c901f165a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.411+0000 7fcd163d5740 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f165a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcd08005ec0 tx=0x7fcd08031ae0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.411+0000 7fcd163d5740 1 -- 172.21.15.5:0/2900606497 shutdown_connections 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.412+0000 7fcd163d5740 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c901f28b70 0x55c901f2af60 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.412+0000 7fcd163d5740 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f19240 0x55c901f16ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.412+0000 7fcd163d5740 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f165a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.412+0000 7fcd163d5740 1 --2- 172.21.15.5:0/2900606497 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e060 0x55c901f16060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.412+0000 7fccf57fa640 0 client.17826 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.412+0000 7fcd163d5740 1 -- 172.21.15.5:0/2900606497 >> 172.21.15.5:0/2900606497 conn(0x55c901f0a5f0 msgr2=0x55c901f102d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.412+0000 7fcd163d5740 1 -- 172.21.15.5:0/2900606497 shutdown_connections 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.412+0000 7fcd163d5740 1 -- 172.21.15.5:0/2900606497 wait complete. 2024-09-15T22:23:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.412+0000 7fcd163d5740 20 client.17826 trim_cache size 0 max 16384 2024-09-15T22:23:17.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd163d5740 1 Processor -- start 2024-09-15T22:23:17.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd163d5740 1 -- start start 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0a5f0 0x55c901f10ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f0e060 0x55c901f09640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f30500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd163d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1b1b70 con 0x55c901f0e980 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd163d5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1af440 con 0x55c901f0e060 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd163d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1ad2b0 con 0x55c901f0a5f0 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd11597640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f30500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd11597640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f30500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51204/0 (socket says 172.21.15.5:51204) 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd11597640 1 -- 172.21.15.5:0/3038760525 learned_addr learned my addr 172.21.15.5:0/3038760525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd11d98640 1 --2- 172.21.15.5:0/3038760525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f0e060 0x55c901f09640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd11597640 1 -- 172.21.15.5:0/3038760525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0a5f0 msgr2=0x55c901f10ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd11597640 1 --2- 172.21.15.5:0/3038760525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0a5f0 0x55c901f10ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd11597640 1 -- 172.21.15.5:0/3038760525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f0e060 msgr2=0x55c901f09640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd11597640 1 --2- 172.21.15.5:0/3038760525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f0e060 0x55c901f09640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd11597640 1 -- 172.21.15.5:0/3038760525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c901f0a090 con 0x55c901f0e980 2024-09-15T22:23:17.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.429+0000 7fcd11d98640 1 --2- 172.21.15.5:0/3038760525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f0e060 0x55c901f09640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:17.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.430+0000 7fcd11597640 1 --2- 172.21.15.5:0/3038760525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f30500 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fccfc005d00 tx=0x7fccfc005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702e7364b27b83dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.430+0000 7fccf77fe640 1 -- 172.21.15.5:0/3038760525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc010e70 con 0x55c901f0e980 2024-09-15T22:23:17.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.430+0000 7fccf77fe640 1 -- 172.21.15.5:0/3038760525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccfc004030 con 0x55c901f0e980 2024-09-15T22:23:17.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.430+0000 7fccf77fe640 1 -- 172.21.15.5:0/3038760525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc005480 con 0x55c901f0e980 2024-09-15T22:23:17.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.430+0000 7fcd163d5740 1 -- 172.21.15.5:0/3038760525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 msgr2=0x55c901f30500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.430+0000 7fcd163d5740 1 --2- 172.21.15.5:0/3038760525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f30500 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fccfc005d00 tx=0x7fccfc005c90 comp rx=0 tx=0).stop 2024-09-15T22:23:17.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.430+0000 7fcd163d5740 1 -- 172.21.15.5:0/3038760525 shutdown_connections 2024-09-15T22:23:17.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.430+0000 7fcd163d5740 1 --2- 172.21.15.5:0/3038760525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e980 0x55c901f30500 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.430+0000 7fcd163d5740 1 --2- 172.21.15.5:0/3038760525 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f0e060 0x55c901f09640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.430+0000 7fcd163d5740 1 --2- 172.21.15.5:0/3038760525 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0a5f0 0x55c901f10ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.430+0000 7fcd163d5740 1 -- 172.21.15.5:0/3038760525 >> 172.21.15.5:0/3038760525 conn(0x55c901f184f0 msgr2=0x55c901f08f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.431+0000 7fcd163d5740 1 -- 172.21.15.5:0/3038760525 shutdown_connections 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.431+0000 7fcd163d5740 1 -- 172.21.15.5:0/3038760525 wait complete. 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.431+0000 7fcd163d5740 1 -- start start 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.431+0000 7fcd163d5740 1 Processor -- start 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.431+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f0a5f0 0x55c901f216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.431+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e060 0x55c901f1e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.431+0000 7fcd163d5740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e980 0x55c901f1e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.431+0000 7fcd163d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1ad900 con 0x55c901f0e060 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.431+0000 7fcd163d5740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1aed70 con 0x55c901f0a5f0 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.431+0000 7fcd163d5740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd0c1adfd0 con 0x55c901f0e980 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.431+0000 7fcd163d5740 10 client.0 _ll_register_callbacks cb 0x7fccfc00ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11d98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e060 0x55c901f1e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11d98640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e060 0x55c901f1e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51208/0 (socket says 172.21.15.5:51208) 2024-09-15T22:23:17.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11d98640 1 -- 172.21.15.5:0/4004626565 learned_addr learned my addr 172.21.15.5:0/4004626565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11597640 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f0a5f0 0x55c901f216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11597640 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e980 0x55c901f1e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11d98640 1 -- 172.21.15.5:0/4004626565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e980 msgr2=0x55c901f1e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11d98640 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e980 0x55c901f1e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11d98640 1 -- 172.21.15.5:0/4004626565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f0a5f0 msgr2=0x55c901f216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11d98640 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f0a5f0 0x55c901f216d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11d98640 1 -- 172.21.15.5:0/4004626565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c901f1ecb0 con 0x55c901f0e060 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11597640 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e980 0x55c901f1e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd11d98640 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e060 0x55c901f1e1b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x55c901f176a0 tx=0x7fcd08004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e569d6cf798803f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fccf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fccf57fa640 1 -- 172.21.15.5:0/4004626565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08040040 con 0x55c901f0e060 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd163d5740 1 -- 172.21.15.5:0/4004626565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccfc004030 con 0x55c901f0e060 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fccf57fa640 1 -- 172.21.15.5:0/4004626565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9629+0+0 (secure 0 0 0) 0x7fcd08005230 con 0x55c901f0e060 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fccf57fa640 1 -- 172.21.15.5:0/4004626565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd08068050 con 0x55c901f0e060 2024-09-15T22:23:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.432+0000 7fcd163d5740 10 client.17832 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.433+0000 7fcd163d5740 1 -- 172.21.15.5:0/4004626565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c901f942f0 con 0x55c901f0e060 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.433+0000 7fccf57fa640 1 -- 172.21.15.5:0/4004626565 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccfc004030 con 0x55c901f0e060 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.433+0000 7fccf57fa640 1 -- 172.21.15.5:0/4004626565 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fcd08004850 con 0x55c901f0e060 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.433+0000 7fcd163d5740 10 client.17832 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.433+0000 7fcd163d5740 10 client.17832 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.433+0000 7fcd163d5740 20 client.17832 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.433+0000 7fcd163d5740 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c901f19240 0x55c901f96950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.433+0000 7fcd163d5740 4 client.17832 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fedc988e", "release": true, "await": true}] 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.433+0000 7fcd163d5740 1 -- 172.21.15.5:0/4004626565 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fedc988e", "release": true, "await": true}) -- 0x55c901e7b0a0 con 0x55c901f19240 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.433+0000 7fcd11597640 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c901f19240 0x55c901f96950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.434+0000 7fcd11597640 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c901f19240 0x55c901f96950 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=404dd6c06b659c0e server_cookie=7c460975c317e5d4 in_seq=0 out_seq=0 2024-09-15T22:23:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.434+0000 7fccf57fa640 10 client.17832 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fccf57fa640 1 -- 172.21.15.5:0/4004626565 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c901e7b0a0 con 0x55c901f19240 2024-09-15T22:23:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fccf57fa640 10 client.17832 handle_command_reply: tid=0 2024-09-15T22:23:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fcd163d5740 1 client.17832 shutdown 2024-09-15T22:23:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fcd163d5740 1 -- 172.21.15.5:0/4004626565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e060 msgr2=0x55c901f1e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fcd163d5740 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e060 0x55c901f1e1b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x55c901f176a0 tx=0x7fcd08004590 comp rx=0 tx=0).stop 2024-09-15T22:23:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fcd163d5740 1 -- 172.21.15.5:0/4004626565 shutdown_connections 2024-09-15T22:23:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fcd163d5740 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55c901f0e980 0x55c901f1e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fcd163d5740 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c901f0e060 0x55c901f1e1b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fcd163d5740 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55c901f19240 0x55c901f96950 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fcd163d5740 1 --2- 172.21.15.5:0/4004626565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55c901f0a5f0 0x55c901f216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fccf57fa640 0 client.17832 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.480+0000 7fcd163d5740 1 -- 172.21.15.5:0/4004626565 >> 172.21.15.5:0/4004626565 conn(0x55c901f184f0 msgr2=0x55c901f09750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd163d5740 1 -- 172.21.15.5:0/4004626565 shutdown_connections 2024-09-15T22:23:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd163d5740 1 -- 172.21.15.5:0/4004626565 wait complete. 2024-09-15T22:23:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd163d5740 20 client.17832 trim_cache size 0 max 16384 2024-09-15T22:23:17.484 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:23:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:23:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:23:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 52, 2024-09-15T22:23:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:23:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "fedc988e": { 2024-09-15T22:23:17.484 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 52, 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 146.5, 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:23:17.485 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:23:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:23:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:23:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:23:17.486 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:23:17.486 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:23:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd13ffc640 1 -- 172.21.15.5:0/4020249413 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fccf80a33b0 msgr2=0x7fccf80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd13ffc640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fccf80a33b0 0x7fccf80a5870 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcd04098700 tx=0x7fcd040c6a50 comp rx=0 tx=0).stop 2024-09-15T22:23:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd13ffc640 1 -- 172.21.15.5:0/4020249413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd0c106a80 msgr2=0x7fcd0c1a43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd13ffc640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd0c106a80 0x7fcd0c1a43d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcd04009fd0 tx=0x7fcd0400b260 comp rx=0 tx=0).stop 2024-09-15T22:23:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd13ffc640 1 -- 172.21.15.5:0/4020249413 shutdown_connections 2024-09-15T22:23:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd13ffc640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fccf80a33b0 0x7fccf80a5870 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd13ffc640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd0c106a80 0x7fcd0c1a43d0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd13ffc640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd0c101420 0x7fcd0c199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd13ffc640 1 --2- 172.21.15.5:0/4020249413 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd0c100a80 0x7fcd0c199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.481+0000 7fcd13ffc640 1 -- 172.21.15.5:0/4020249413 >> 172.21.15.5:0/4020249413 conn(0x7fcd0c0fc7a0 msgr2=0x7fcd0c0fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.482+0000 7fcd13ffc640 1 -- 172.21.15.5:0/4020249413 shutdown_connections 2024-09-15T22:23:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:17.482+0000 7fcd13ffc640 1 -- 172.21.15.5:0/4020249413 wait complete. 2024-09-15T22:23:18.187 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fedc988e', total seconds elapsed: 30.0 2024-09-15T22:23:19.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:19 smithi099 ceph-mon[34892]: pgmap v450: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-15T22:23:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:19 smithi099 ceph-mon[34892]: osdmap e105: 12 total, 12 up, 12 in 2024-09-15T22:23:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:19 smithi129 ceph-mon[35314]: pgmap v450: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-15T22:23:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:19 smithi129 ceph-mon[35314]: osdmap e105: 12 total, 12 up, 12 in 2024-09-15T22:23:19.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:19 smithi005 ceph-mon[30944]: pgmap v450: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2024-09-15T22:23:19.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:19 smithi005 ceph-mon[30944]: osdmap e105: 12 total, 12 up, 12 in 2024-09-15T22:23:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 830.0 s, 251.7 tps, lat 1981.607 ms stddev 6237.260 2024-09-15T22:23:21.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:21 smithi099 ceph-mon[34892]: pgmap v452: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 11 op/s 2024-09-15T22:23:21.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:21 smithi129 ceph-mon[35314]: pgmap v452: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 11 op/s 2024-09-15T22:23:21.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:21 smithi005 ceph-mon[30944]: pgmap v452: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 11 op/s 2024-09-15T22:23:23.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:23 smithi099 ceph-mon[34892]: pgmap v453: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-15T22:23:23.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:23 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:23:23.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:23 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:23:23.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:23 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:23:23.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:23 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:23:23.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:23 smithi129 ceph-mon[35314]: pgmap v453: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-15T22:23:23.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:23 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:23:23.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:23 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:23:23.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:23 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:23:23.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:23 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:23:23.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:23 smithi005 ceph-mon[30944]: pgmap v453: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-09-15T22:23:23.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:23 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:23:23.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:23 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:23:23.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:23 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:23:23.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:23 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:23:24.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 835.0 s, 565.2 tps, lat 186.694 ms stddev 225.406 2024-09-15T22:23:25.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:25 smithi099 ceph-mon[34892]: pgmap v454: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-09-15T22:23:25.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:25 smithi129 ceph-mon[35314]: pgmap v454: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-09-15T22:23:25.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:25 smithi005 ceph-mon[30944]: pgmap v454: 129 pgs: 129 active+clean; 2.7 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 1.8 MiB/s wr, 192 op/s 2024-09-15T22:23:27.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:27 smithi099 ceph-mon[34892]: pgmap v455: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.2 MiB/s wr, 230 op/s 2024-09-15T22:23:27.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:27 smithi129 ceph-mon[35314]: pgmap v455: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.2 MiB/s wr, 230 op/s 2024-09-15T22:23:27.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:27 smithi005 ceph-mon[30944]: pgmap v455: 129 pgs: 129 active+clean; 2.7 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.2 MiB/s wr, 230 op/s 2024-09-15T22:23:29.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:29 smithi099 ceph-mon[34892]: pgmap v456: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 4.3 MiB/s wr, 434 op/s 2024-09-15T22:23:29.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:29 smithi129 ceph-mon[35314]: pgmap v456: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 4.3 MiB/s wr, 434 op/s 2024-09-15T22:23:29.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:29 smithi005 ceph-mon[30944]: pgmap v456: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 4.3 MiB/s wr, 434 op/s 2024-09-15T22:23:29.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 840.0 s, 582.8 tps, lat 164.510 ms stddev 190.925 2024-09-15T22:23:31.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:31 smithi099 ceph-mon[34892]: pgmap v457: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.3 MiB/s wr, 422 op/s 2024-09-15T22:23:31.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:31 smithi129 ceph-mon[35314]: pgmap v457: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.3 MiB/s wr, 422 op/s 2024-09-15T22:23:31.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:31 smithi005 ceph-mon[30944]: pgmap v457: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 4.3 MiB/s wr, 422 op/s 2024-09-15T22:23:32.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:23:33.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:33 smithi099 ceph-mon[34892]: pgmap v458: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.1 MiB/s wr, 523 op/s 2024-09-15T22:23:33.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:33 smithi129 ceph-mon[35314]: pgmap v458: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.1 MiB/s wr, 523 op/s 2024-09-15T22:23:33.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:33 smithi005 ceph-mon[30944]: pgmap v458: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 5.1 MiB/s wr, 523 op/s 2024-09-15T22:23:33.402 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.088+0000 7f2370cfe640 1 -- 172.21.15.5:0/916590586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f236c11b140 msgr2=0x7f236c11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.088+0000 7f2370cfe640 1 --2- 172.21.15.5:0/916590586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f236c11b140 0x7f236c11d530 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2364009ca0 tx=0x7f236402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.092+0000 7f2370cfe640 1 -- 172.21.15.5:0/916590586 shutdown_connections 2024-09-15T22:23:34.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.092+0000 7f2370cfe640 1 --2- 172.21.15.5:0/916590586 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f236c11b140 0x7f236c11d530 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.092+0000 7f2370cfe640 1 --2- 172.21.15.5:0/916590586 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c109830 0x7f236c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.092+0000 7f2370cfe640 1 --2- 172.21.15.5:0/916590586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f236c109450 0x7f236c110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.092+0000 7f2370cfe640 1 -- 172.21.15.5:0/916590586 >> 172.21.15.5:0/916590586 conn(0x7f236c06e140 msgr2=0x7f236c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.093+0000 7f2370cfe640 1 -- 172.21.15.5:0/916590586 shutdown_connections 2024-09-15T22:23:34.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.097+0000 7f2370cfe640 1 -- 172.21.15.5:0/916590586 wait complete. 2024-09-15T22:23:34.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.098+0000 7f2370cfe640 1 Processor -- start 2024-09-15T22:23:34.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.102+0000 7f2370cfe640 1 -- start start 2024-09-15T22:23:34.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.102+0000 7f2370cfe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f236c109450 0x7f236c1a7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.102+0000 7f2370cfe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f236c109830 0x7f236c1a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.102+0000 7f2370cfe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c11b140 0x7f236c1aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.102+0000 7f2370cfe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f236c120df0 con 0x7f236c109450 2024-09-15T22:23:34.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.102+0000 7f2370cfe640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f236c120c70 con 0x7f236c11b140 2024-09-15T22:23:34.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.102+0000 7f2370cfe640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f236c120f70 con 0x7f236c109830 2024-09-15T22:23:34.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.103+0000 7f236a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f236c109450 0x7f236c1a7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.103+0000 7f236a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f236c109450 0x7f236c1a7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52162/0 (socket says 172.21.15.5:52162) 2024-09-15T22:23:34.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.103+0000 7f236a59c640 1 -- 172.21.15.5:0/3171811456 learned_addr learned my addr 172.21.15.5:0/3171811456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:34.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.106+0000 7f2369d9b640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f236c109830 0x7f236c1a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.107+0000 7f2369d9b640 1 -- 172.21.15.5:0/3171811456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c11b140 msgr2=0x7f236c1aba30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:23:34.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.107+0000 7f2369d9b640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c11b140 0x7f236c1aba30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.107+0000 7f2369d9b640 1 -- 172.21.15.5:0/3171811456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f236c109450 msgr2=0x7f236c1a7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.107+0000 7f2369d9b640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f236c109450 0x7f236c1a7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.107+0000 7f2369d9b640 1 -- 172.21.15.5:0/3171811456 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c1ac130 con 0x7f236c109830 2024-09-15T22:23:34.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.107+0000 7f2369d9b640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f236c109830 0x7f236c1a76a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f236000d6d0 tx=0x7f236000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.112+0000 7f235b7fe640 1 -- 172.21.15.5:0/3171811456 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360017070 con 0x7f236c109830 2024-09-15T22:23:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.112+0000 7f2370cfe640 1 -- 172.21.15.5:0/3171811456 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236c1ac420 con 0x7f236c109830 2024-09-15T22:23:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.112+0000 7f2370cfe640 1 -- 172.21.15.5:0/3171811456 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236c110920 con 0x7f236c109830 2024-09-15T22:23:34.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.115+0000 7f23730d7740 1 Processor -- start 2024-09-15T22:23:34.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.116+0000 7f235b7fe640 1 -- 172.21.15.5:0/3171811456 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2360009820 con 0x7f236c109830 2024-09-15T22:23:34.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.116+0000 7f235b7fe640 1 -- 172.21.15.5:0/3171811456 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360004d00 con 0x7f236c109830 2024-09-15T22:23:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.117+0000 7f23730d7740 1 -- start start 2024-09-15T22:23:34.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.118+0000 7f23730d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc84a0 0x55d063fc8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.118+0000 7f23730d7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc8dc0 0x55d063fd3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.118+0000 7f23730d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fd3680 0x55d063fd5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.118+0000 7f23730d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d063f2e2e0 con 0x55d063fd3680 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.118+0000 7f23730d7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d063ed5ce0 con 0x55d063fc8dc0 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.118+0000 7f23730d7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d063f2a2a0 con 0x55d063fc84a0 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.118+0000 7f236ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc84a0 0x55d063fc8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.119+0000 7f236ad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc84a0 0x55d063fc8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43862/0 (socket says 172.21.15.5:43862) 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.119+0000 7f236ad9d640 1 -- 172.21.15.5:0/4085790532 learned_addr learned my addr 172.21.15.5:0/4085790532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.119+0000 7f236ad9d640 1 -- 172.21.15.5:0/4085790532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc8dc0 msgr2=0x55d063fd3140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.119+0000 7f236ad9d640 1 --2- 172.21.15.5:0/4085790532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc8dc0 0x55d063fd3140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.119+0000 7f236ad9d640 1 -- 172.21.15.5:0/4085790532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fd3680 msgr2=0x55d063fd5a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.119+0000 7f236ad9d640 1 --2- 172.21.15.5:0/4085790532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fd3680 0x55d063fd5a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.120+0000 7f236ad9d640 1 -- 172.21.15.5:0/4085790532 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d063eba440 con 0x55d063fc84a0 2024-09-15T22:23:34.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.124+0000 7f235b7fe640 1 -- 172.21.15.5:0/3171811456 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2360010460 con 0x7f236c109830 2024-09-15T22:23:34.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.125+0000 7f235b7fe640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23540a3580 0x7f23540a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.126+0000 7f236ad9d640 1 --2- 172.21.15.5:0/4085790532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc84a0 0x55d063fc8880 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2364005e30 tx=0x7f2364005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21ae631df92229e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:34.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.126+0000 7f235b7fe640 1 -- 172.21.15.5:0/3171811456 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f23600cf900 con 0x7f236c109830 2024-09-15T22:23:34.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.130+0000 7f2358ff9640 1 -- 172.21.15.5:0/4085790532 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23640305c0 con 0x55d063fc84a0 2024-09-15T22:23:34.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.130+0000 7f2358ff9640 1 -- 172.21.15.5:0/4085790532 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f236403f070 con 0x55d063fc84a0 2024-09-15T22:23:34.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.130+0000 7f2358ff9640 1 -- 172.21.15.5:0/4085790532 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364003da0 con 0x55d063fc84a0 2024-09-15T22:23:34.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.134+0000 7f236a59c640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23540a3580 0x7f23540a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.136+0000 7f23730d7740 1 -- 172.21.15.5:0/4085790532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc84a0 msgr2=0x55d063fc8880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.136+0000 7f23730d7740 1 --2- 172.21.15.5:0/4085790532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc84a0 0x55d063fc8880 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2364005e30 tx=0x7f2364005ef0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.138+0000 7f236a59c640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23540a3580 0x7f23540a5a40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55d063fc5890 tx=0x7f235c00a380 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.139+0000 7f23730d7740 1 -- 172.21.15.5:0/4085790532 shutdown_connections 2024-09-15T22:23:34.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.139+0000 7f23730d7740 1 --2- 172.21.15.5:0/4085790532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fd3680 0x55d063fd5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.139+0000 7f23730d7740 1 --2- 172.21.15.5:0/4085790532 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc8dc0 0x55d063fd3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.139+0000 7f23730d7740 1 --2- 172.21.15.5:0/4085790532 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc84a0 0x55d063fc8880 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.139+0000 7f23730d7740 1 -- 172.21.15.5:0/4085790532 >> 172.21.15.5:0/4085790532 conn(0x55d063fc4a30 msgr2=0x55d063fc4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.144+0000 7f23730d7740 1 -- 172.21.15.5:0/4085790532 shutdown_connections 2024-09-15T22:23:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.145+0000 7f23730d7740 1 -- 172.21.15.5:0/4085790532 wait complete. 2024-09-15T22:23:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.145+0000 7f23730d7740 1 -- start start 2024-09-15T22:23:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.146+0000 7f23730d7740 1 Processor -- start 2024-09-15T22:23:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.147+0000 7f23730d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fc84a0 0x55d063fd3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.147+0000 7f23730d7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc8dc0 0x55d063fd0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.147+0000 7f23730d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fd3680 0x55d063fd0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.147+0000 7f23730d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d063fd6250 con 0x55d063fc84a0 2024-09-15T22:23:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.147+0000 7f23730d7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d063ee9050 con 0x55d063fc8dc0 2024-09-15T22:23:34.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.147+0000 7f23730d7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d063fd63d0 con 0x55d063fd3680 2024-09-15T22:23:34.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.147+0000 7f23730d7740 10 client.0 _ll_register_callbacks cb 0x55d063f29160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:34.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.148+0000 7f236a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fc84a0 0x55d063fd3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.148+0000 7f236a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fc84a0 0x55d063fd3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52170/0 (socket says 172.21.15.5:52170) 2024-09-15T22:23:34.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.148+0000 7f236a59c640 1 -- 172.21.15.5:0/2809839852 learned_addr learned my addr 172.21.15.5:0/2809839852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:34.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.149+0000 7f2369d9b640 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc8dc0 0x55d063fd0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.150+0000 7f236ad9d640 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fd3680 0x55d063fd0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.150+0000 7f2369d9b640 1 -- 172.21.15.5:0/2809839852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fd3680 msgr2=0x55d063fd0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:34.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.150+0000 7f2369d9b640 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fd3680 0x55d063fd0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.150+0000 7f2369d9b640 1 -- 172.21.15.5:0/2809839852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fc84a0 msgr2=0x55d063fd3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:34.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.151+0000 7f2369d9b640 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fc84a0 0x55d063fd3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.151+0000 7f2369d9b640 1 -- 172.21.15.5:0/2809839852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d063c9f3c0 con 0x55d063fc8dc0 2024-09-15T22:23:34.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.154+0000 7f2369d9b640 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc8dc0 0x55d063fd0320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f23600ccbf0 tx=0x7f236009bfc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8bcf046b594e646 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:34.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.155+0000 7f2351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:23:34.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.155+0000 7f2351ffb640 1 -- 172.21.15.5:0/2809839852 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360020070 con 0x55d063fc8dc0 2024-09-15T22:23:34.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.155+0000 7f2351ffb640 1 -- 172.21.15.5:0/2809839852 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f23600b46b0 con 0x55d063fc8dc0 2024-09-15T22:23:34.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.155+0000 7f23730d7740 1 -- 172.21.15.5:0/2809839852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f236403f070 con 0x55d063fc8dc0 2024-09-15T22:23:34.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.156+0000 7f2351ffb640 1 -- 172.21.15.5:0/2809839852 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f23600a4470 con 0x55d063fc8dc0 2024-09-15T22:23:34.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.157+0000 7f23730d7740 10 client.27314 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:34.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.157+0000 7f23730d7740 1 -- 172.21.15.5:0/2809839852 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d063fe1d10 con 0x55d063fc8dc0 2024-09-15T22:23:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.158+0000 7f2351ffb640 1 -- 172.21.15.5:0/2809839852 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f236403f070 con 0x55d063fc8dc0 2024-09-15T22:23:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.158+0000 7f2351ffb640 1 -- 172.21.15.5:0/2809839852 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f23600ccf00 con 0x55d063fc8dc0 2024-09-15T22:23:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.161+0000 7f23730d7740 10 client.27314 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.161+0000 7f23730d7740 10 client.27314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.161+0000 7f23730d7740 20 client.27314 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.161+0000 7f23730d7740 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d063fe32c0 0x55d063fe56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.161+0000 7f23730d7740 4 client.27314 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.161+0000 7f23730d7740 1 -- 172.21.15.5:0/2809839852 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d063f3b850 con 0x55d063fe32c0 2024-09-15T22:23:34.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.161+0000 7f236a59c640 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d063fe32c0 0x55d063fe56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.167+0000 7f236a59c640 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d063fe32c0 0x55d063fe56e0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b18fdd5171d97162 server_cookie=b8c5582c652d0d15 in_seq=0 out_seq=0 2024-09-15T22:23:34.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.167+0000 7f2351ffb640 10 client.27314 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:34.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.172+0000 7f2351ffb640 1 -- 172.21.15.5:0/2809839852 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d063f3b850 con 0x55d063fe32c0 2024-09-15T22:23:34.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.172+0000 7f2351ffb640 10 client.27314 handle_command_reply: tid=0 2024-09-15T22:23:34.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.172+0000 7f23730d7740 1 client.27314 shutdown 2024-09-15T22:23:34.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.172+0000 7f23730d7740 1 -- 172.21.15.5:0/2809839852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc8dc0 msgr2=0x55d063fd0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.172+0000 7f23730d7740 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc8dc0 0x55d063fd0320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f23600ccbf0 tx=0x7f236009bfc0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.173+0000 7f23730d7740 1 -- 172.21.15.5:0/2809839852 shutdown_connections 2024-09-15T22:23:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.173+0000 7f23730d7740 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fd3680 0x55d063fd0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.173+0000 7f23730d7740 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d063fe32c0 0x55d063fe56e0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.173+0000 7f23730d7740 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc8dc0 0x55d063fd0320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.173+0000 7f23730d7740 1 --2- 172.21.15.5:0/2809839852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fc84a0 0x55d063fd3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.173+0000 7f23730d7740 1 -- 172.21.15.5:0/2809839852 >> 172.21.15.5:0/2809839852 conn(0x55d063fc4a30 msgr2=0x55d063fca710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.173+0000 7f2351ffb640 0 client.27314 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.173+0000 7f23730d7740 1 -- 172.21.15.5:0/2809839852 shutdown_connections 2024-09-15T22:23:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.177+0000 7f23730d7740 1 -- 172.21.15.5:0/2809839852 wait complete. 2024-09-15T22:23:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.177+0000 7f23730d7740 20 client.27314 trim_cache size 0 max 16384 2024-09-15T22:23:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.217+0000 7f23730d7740 1 Processor -- start 2024-09-15T22:23:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.217+0000 7f23730d7740 1 -- start start 2024-09-15T22:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.218+0000 7f23730d7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc84a0 0x55d063fdf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.218+0000 7f23730d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063feafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.218+0000 7f23730d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fd3680 0x55d063feb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.218+0000 7f23730d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f234c012920 con 0x55d063fd3680 2024-09-15T22:23:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.218+0000 7f23730d7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0101f0 con 0x55d063fc84a0 2024-09-15T22:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.218+0000 7f23730d7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f234c00e060 con 0x55d063fc8dc0 2024-09-15T22:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.218+0000 7f2369d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063feafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.218+0000 7f2369d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063feafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43872/0 (socket says 172.21.15.5:43872) 2024-09-15T22:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.219+0000 7f2369d9b640 1 -- 172.21.15.5:0/1378817434 learned_addr learned my addr 172.21.15.5:0/1378817434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.219+0000 7f2369d9b640 1 -- 172.21.15.5:0/1378817434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc84a0 msgr2=0x55d063fdf510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.219+0000 7f2369d9b640 1 --2- 172.21.15.5:0/1378817434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc84a0 0x55d063fdf510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.219+0000 7f2369d9b640 1 -- 172.21.15.5:0/1378817434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fd3680 msgr2=0x55d063feb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.219+0000 7f2369d9b640 1 --2- 172.21.15.5:0/1378817434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fd3680 0x55d063feb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.219+0000 7f2369d9b640 1 -- 172.21.15.5:0/1378817434 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d063fc8bf0 con 0x55d063fc8dc0 2024-09-15T22:23:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.221+0000 7f2369d9b640 1 --2- 172.21.15.5:0/1378817434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063feafd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2360099ab0 tx=0x7f2360097420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b8b5874a4c4edf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.221+0000 7f2353fff640 1 -- 172.21.15.5:0/1378817434 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360020070 con 0x55d063fc8dc0 2024-09-15T22:23:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.221+0000 7f2353fff640 1 -- 172.21.15.5:0/1378817434 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23600cf670 con 0x55d063fc8dc0 2024-09-15T22:23:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.222+0000 7f23730d7740 1 -- 172.21.15.5:0/1378817434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 msgr2=0x55d063feafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.222+0000 7f23730d7740 1 --2- 172.21.15.5:0/1378817434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063feafd0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2360099ab0 tx=0x7f2360097420 comp rx=0 tx=0).stop 2024-09-15T22:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.224+0000 7f23730d7740 1 -- 172.21.15.5:0/1378817434 shutdown_connections 2024-09-15T22:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.224+0000 7f23730d7740 1 --2- 172.21.15.5:0/1378817434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fd3680 0x55d063feb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.224+0000 7f23730d7740 1 --2- 172.21.15.5:0/1378817434 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063feafd0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.224+0000 7f23730d7740 1 --2- 172.21.15.5:0/1378817434 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fc84a0 0x55d063fdf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.224+0000 7f23730d7740 1 -- 172.21.15.5:0/1378817434 >> 172.21.15.5:0/1378817434 conn(0x55d063fc4a30 msgr2=0x55d063fcf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.224+0000 7f23730d7740 1 -- 172.21.15.5:0/1378817434 shutdown_connections 2024-09-15T22:23:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.224+0000 7f23730d7740 1 -- 172.21.15.5:0/1378817434 wait complete. 2024-09-15T22:23:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.224+0000 7f23730d7740 1 -- start start 2024-09-15T22:23:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.225+0000 7f23730d7740 1 Processor -- start 2024-09-15T22:23:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.225+0000 7f23730d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fc84a0 0x55d063fd6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.225+0000 7f23730d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063fd7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.225+0000 7f23730d7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fd3680 0x55d064045f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.225+0000 7f23730d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f234c00e6b0 con 0x55d063fc84a0 2024-09-15T22:23:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.225+0000 7f23730d7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f234c00fb20 con 0x55d063fd3680 2024-09-15T22:23:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.225+0000 7f23730d7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f234c00ed80 con 0x55d063fc8dc0 2024-09-15T22:23:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.225+0000 7f23730d7740 10 client.0 _ll_register_callbacks cb 0x7f2360004ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.225+0000 7f2369d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063fd7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.226+0000 7f2369d9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063fd7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43878/0 (socket says 172.21.15.5:43878) 2024-09-15T22:23:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.226+0000 7f2369d9b640 1 -- 172.21.15.5:0/2049748063 learned_addr learned my addr 172.21.15.5:0/2049748063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.226+0000 7f236ad9d640 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fd3680 0x55d064045f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.226+0000 7f236a59c640 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fc84a0 0x55d063fd6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.226+0000 7f2369d9b640 1 -- 172.21.15.5:0/2049748063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fd3680 msgr2=0x55d064045f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.226+0000 7f2369d9b640 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fd3680 0x55d064045f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.226+0000 7f2369d9b640 1 -- 172.21.15.5:0/2049748063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fc84a0 msgr2=0x55d063fd6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.226+0000 7f2369d9b640 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fc84a0 0x55d063fd6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.226+0000 7f2369d9b640 1 -- 172.21.15.5:0/2049748063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d064046600 con 0x55d063fc8dc0 2024-09-15T22:23:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.234+0000 7f2369d9b640 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063fd7320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2360009610 tx=0x7f23600980d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2758e8afdf5e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.243+0000 7f2351ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:23:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.243+0000 7f2351ffb640 1 -- 172.21.15.5:0/2049748063 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360020040 con 0x55d063fc8dc0 2024-09-15T22:23:34.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.243+0000 7f2351ffb640 1 -- 172.21.15.5:0/2049748063 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f236009cd30 con 0x55d063fc8dc0 2024-09-15T22:23:34.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.244+0000 7f23730d7740 1 -- 172.21.15.5:0/2049748063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23600cf670 con 0x55d063fc8dc0 2024-09-15T22:23:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.250+0000 7f2351ffb640 1 -- 172.21.15.5:0/2049748063 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f23600b4250 con 0x55d063fc8dc0 2024-09-15T22:23:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.251+0000 7f23730d7740 10 client.27247 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:34.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.251+0000 7f23730d7740 1 -- 172.21.15.5:0/2049748063 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d064046600 con 0x55d063fc8dc0 2024-09-15T22:23:34.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.256+0000 7f2351ffb640 1 -- 172.21.15.5:0/2049748063 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23600cf670 con 0x55d063fc8dc0 2024-09-15T22:23:34.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.256+0000 7f2351ffb640 1 -- 172.21.15.5:0/2049748063 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2360097620 con 0x55d063fc8dc0 2024-09-15T22:23:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.257+0000 7f23730d7740 10 client.27247 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.257+0000 7f23730d7740 10 client.27247 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.257+0000 7f23730d7740 20 client.27247 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.257+0000 7f23730d7740 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d06404dbe0 0x55d06404ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.257+0000 7f23730d7740 4 client.27247 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:23:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.257+0000 7f23730d7740 1 -- 172.21.15.5:0/2049748063 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d063f35d60 con 0x55d06404dbe0 2024-09-15T22:23:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.257+0000 7f236a59c640 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d06404dbe0 0x55d06404ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.268+0000 7f236a59c640 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d06404dbe0 0x55d06404ffd0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=743d6920868dc45f server_cookie=3cd0fce705d41174 in_seq=0 out_seq=0 2024-09-15T22:23:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.269+0000 7f2351ffb640 10 client.27247 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.273+0000 7f2351ffb640 1 -- 172.21.15.5:0/2049748063 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d063f35d60 con 0x55d06404dbe0 2024-09-15T22:23:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.273+0000 7f2351ffb640 10 client.27247 handle_command_reply: tid=0 2024-09-15T22:23:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.276+0000 7f23730d7740 1 client.27247 shutdown 2024-09-15T22:23:34.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.277+0000 7f23730d7740 1 -- 172.21.15.5:0/2049748063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 msgr2=0x55d063fd7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.277+0000 7f23730d7740 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063fd7320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2360009610 tx=0x7f23600980d0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.281+0000 7f23730d7740 1 -- 172.21.15.5:0/2049748063 shutdown_connections 2024-09-15T22:23:34.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.281+0000 7f23730d7740 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55d06404dbe0 0x55d06404ffd0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.281+0000 7f23730d7740 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d063fd3680 0x55d064045f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.281+0000 7f23730d7740 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d063fc8dc0 0x55d063fd7320 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.281+0000 7f23730d7740 1 --2- 172.21.15.5:0/2049748063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d063fc84a0 0x55d063fd6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.281+0000 7f23730d7740 1 -- 172.21.15.5:0/2049748063 >> 172.21.15.5:0/2049748063 conn(0x55d063fc4a30 msgr2=0x55d063fc3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.281+0000 7f2351ffb640 0 client.27247 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:34.284 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:23:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:23:34.284 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:23:34.284 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:23:34.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.281+0000 7f23730d7740 1 -- 172.21.15.5:0/2049748063 shutdown_connections 2024-09-15T22:23:34.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.281+0000 7f23730d7740 1 -- 172.21.15.5:0/2049748063 wait complete. 2024-09-15T22:23:34.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.281+0000 7f23730d7740 20 client.27247 trim_cache size 0 max 16384 2024-09-15T22:23:34.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 -- 172.21.15.5:0/3171811456 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23540a3580 msgr2=0x7f23540a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23540a3580 0x7f23540a5a40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55d063fc5890 tx=0x7f235c00a380 comp rx=0 tx=0).stop 2024-09-15T22:23:34.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 -- 172.21.15.5:0/3171811456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f236c109830 msgr2=0x7f236c1a76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:34.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f236c109830 0x7f236c1a76a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f236000d6d0 tx=0x7f236000db90 comp rx=0 tx=0).stop 2024-09-15T22:23:34.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 -- 172.21.15.5:0/3171811456 shutdown_connections 2024-09-15T22:23:34.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f23540a3580 0x7f23540a5a40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f236c11b140 0x7f236c1aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f236c109830 0x7f236c1a76a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 --2- 172.21.15.5:0/3171811456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f236c109450 0x7f236c1a7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:34.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 -- 172.21.15.5:0/3171811456 >> 172.21.15.5:0/3171811456 conn(0x7f236c06e140 msgr2=0x7f236c11bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:34.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 -- 172.21.15.5:0/3171811456 shutdown_connections 2024-09-15T22:23:34.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:34.284+0000 7f2370cfe640 1 -- 172.21.15.5:0/3171811456 wait complete. 2024-09-15T22:23:34.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 845.0 s, 490.2 tps, lat 209.367 ms stddev 277.986 2024-09-15T22:23:35.179 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:23:35.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:23:35.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:35 smithi099 ceph-mon[34892]: pgmap v459: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 4.9 MiB/s wr, 504 op/s 2024-09-15T22:23:35.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:35 smithi129 ceph-mon[35314]: pgmap v459: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 4.9 MiB/s wr, 504 op/s 2024-09-15T22:23:35.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:35 smithi005 ceph-mon[30944]: pgmap v459: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 4.9 MiB/s wr, 504 op/s 2024-09-15T22:23:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:36.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.367+0000 7f45296cf640 1 -- 172.21.15.5:0/3315480215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4524109450 msgr2=0x7f4524110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.367+0000 7f45296cf640 1 --2- 172.21.15.5:0/3315480215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4524109450 0x7f4524110250 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f451c009ca0 tx=0x7f451c02a8f0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.367+0000 7f45296cf640 1 -- 172.21.15.5:0/3315480215 shutdown_connections 2024-09-15T22:23:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.367+0000 7f45296cf640 1 --2- 172.21.15.5:0/3315480215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452411b140 0x7f452411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.367+0000 7f45296cf640 1 --2- 172.21.15.5:0/3315480215 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4524109830 0x7f4524109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.367+0000 7f45296cf640 1 --2- 172.21.15.5:0/3315480215 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4524109450 0x7f4524110250 secure :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f451c009ca0 tx=0x7f451c02a8f0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.367+0000 7f45296cf640 1 -- 172.21.15.5:0/3315480215 >> 172.21.15.5:0/3315480215 conn(0x7f452406e140 msgr2=0x7f452406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.370+0000 7f45296cf640 1 -- 172.21.15.5:0/3315480215 shutdown_connections 2024-09-15T22:23:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.370+0000 7f45296cf640 1 -- 172.21.15.5:0/3315480215 wait complete. 2024-09-15T22:23:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.371+0000 7f45296cf640 1 Processor -- start 2024-09-15T22:23:36.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.371+0000 7f45296cf640 1 -- start start 2024-09-15T22:23:36.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.372+0000 7f45296cf640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4524109830 0x7f452411a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:36.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.372+0000 7f45296cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452411b140 0x7f4524113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.372+0000 7f45296cf640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4524113ec0 0x7f4524114370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:36.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.372+0000 7f4522ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4524109830 0x7f452411a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:36.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.372+0000 7f4522ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4524109830 0x7f452411a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43908/0 (socket says 172.21.15.5:43908) 2024-09-15T22:23:36.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.372+0000 7f4522ffd640 1 -- 172.21.15.5:0/3334400235 learned_addr learned my addr 172.21.15.5:0/3334400235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:36.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.372+0000 7f45296cf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4524120df0 con 0x7f452411b140 2024-09-15T22:23:36.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.373+0000 7f45296cf640 1 -- 172.21.15.5:0/3334400235 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4524120c70 con 0x7f4524113ec0 2024-09-15T22:23:36.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.373+0000 7f45296cf640 1 -- 172.21.15.5:0/3334400235 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4524120f70 con 0x7f4524109830 2024-09-15T22:23:36.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.373+0000 7f45237fe640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4524113ec0 0x7f4524114370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.374+0000 7f4522ffd640 1 -- 172.21.15.5:0/3334400235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4524113ec0 msgr2=0x7f4524114370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.374+0000 7f4522ffd640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4524113ec0 0x7f4524114370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.374+0000 7f4522ffd640 1 -- 172.21.15.5:0/3334400235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452411b140 msgr2=0x7f4524113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.374+0000 7f4522ffd640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452411b140 0x7f4524113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.374+0000 7f4522ffd640 1 -- 172.21.15.5:0/3334400235 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4524118520 con 0x7f4524109830 2024-09-15T22:23:36.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.375+0000 7f45237fe640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4524113ec0 0x7f4524114370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:36.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.375+0000 7f4522ffd640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4524109830 0x7f452411a8f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f451c000f80 tx=0x7f451c030820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.377+0000 7f450bfff640 1 -- 172.21.15.5:0/3334400235 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c030da0 con 0x7f4524109830 2024-09-15T22:23:36.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.378+0000 7f450bfff640 1 -- 172.21.15.5:0/3334400235 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f451c02d070 con 0x7f4524109830 2024-09-15T22:23:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.378+0000 7f450bfff640 1 -- 172.21.15.5:0/3334400235 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c03b660 con 0x7f4524109830 2024-09-15T22:23:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.378+0000 7f45296cf640 1 -- 172.21.15.5:0/3334400235 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45241187b0 con 0x7f4524109830 2024-09-15T22:23:36.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.378+0000 7f45296cf640 1 -- 172.21.15.5:0/3334400235 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4524118ca0 con 0x7f4524109830 2024-09-15T22:23:36.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.384+0000 7f450bfff640 1 -- 172.21.15.5:0/3334400235 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f451c03b800 con 0x7f4524109830 2024-09-15T22:23:36.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.386+0000 7f450bfff640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f450c0a35c0 0x7f450c0a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:36.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.386+0000 7f4509ffb640 1 -- 172.21.15.5:0/3334400235 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4524073b00 con 0x7f4524109830 2024-09-15T22:23:36.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.386+0000 7f450bfff640 1 -- 172.21.15.5:0/3334400235 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f451c0f14a0 con 0x7f4524109830 2024-09-15T22:23:36.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.392+0000 7f450bfff640 1 -- 172.21.15.5:0/3334400235 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f451c04b060 con 0x7f4524109830 2024-09-15T22:23:36.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.398+0000 7f45227fc640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f450c0a35c0 0x7f450c0a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:36.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.402+0000 7f45227fc640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f450c0a35c0 0x7f450c0a5a80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f45140058c0 tx=0x7f4514005850 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:36.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.784+0000 7f4509ffb640 1 -- 172.21.15.5:0/3334400235 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4524114eb0 con 0x7f4524109830 2024-09-15T22:23:36.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.784+0000 7f450bfff640 1 -- 172.21.15.5:0/3334400235 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f451c0bdb30 con 0x7f4524109830 2024-09-15T22:23:36.787 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:23:36.788 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:23:36.788 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:23:36.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.787+0000 7f4509ffb640 1 -- 172.21.15.5:0/3334400235 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f450c0a35c0 msgr2=0x7f450c0a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:36.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.787+0000 7f4509ffb640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f450c0a35c0 0x7f450c0a5a80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f45140058c0 tx=0x7f4514005850 comp rx=0 tx=0).stop 2024-09-15T22:23:36.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.787+0000 7f4509ffb640 1 -- 172.21.15.5:0/3334400235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4524109830 msgr2=0x7f452411a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:36.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.787+0000 7f4509ffb640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4524109830 0x7f452411a8f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f451c000f80 tx=0x7f451c030820 comp rx=0 tx=0).stop 2024-09-15T22:23:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.788+0000 7f4509ffb640 1 -- 172.21.15.5:0/3334400235 shutdown_connections 2024-09-15T22:23:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.788+0000 7f4509ffb640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f450c0a35c0 0x7f450c0a5a80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.788+0000 7f4509ffb640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4524113ec0 0x7f4524114370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.788+0000 7f4509ffb640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f452411b140 0x7f4524113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.788+0000 7f4509ffb640 1 --2- 172.21.15.5:0/3334400235 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4524109830 0x7f452411a8f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.788+0000 7f4509ffb640 1 -- 172.21.15.5:0/3334400235 >> 172.21.15.5:0/3334400235 conn(0x7f452406e140 msgr2=0x7f452410ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.788+0000 7f4509ffb640 1 -- 172.21.15.5:0/3334400235 shutdown_connections 2024-09-15T22:23:36.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:36.788+0000 7f4509ffb640 1 -- 172.21.15.5:0/3334400235 wait complete. 2024-09-15T22:23:37.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:37 smithi099 ceph-mon[34892]: pgmap v460: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 4.8 MiB/s wr, 490 op/s 2024-09-15T22:23:37.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:37 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3334400235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:37.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:37 smithi129 ceph-mon[35314]: pgmap v460: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 4.8 MiB/s wr, 490 op/s 2024-09-15T22:23:37.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:37 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3334400235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:37.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:37 smithi005 ceph-mon[30944]: pgmap v460: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 4.8 MiB/s wr, 490 op/s 2024-09-15T22:23:37.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:37 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3334400235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:23:37.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:23:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:39.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:39 smithi129 ceph-mon[35314]: pgmap v461: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-09-15T22:23:39.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:39 smithi005 ceph-mon[30944]: pgmap v461: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-09-15T22:23:39.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:39 smithi099 ceph-mon[34892]: pgmap v461: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 6.1 MiB/s wr, 631 op/s 2024-09-15T22:23:39.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 850.0 s, 646.6 tps, lat 155.963 ms stddev 186.570 2024-09-15T22:23:41.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:41 smithi005 ceph-mon[30944]: pgmap v462: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.6 MiB/s wr, 481 op/s 2024-09-15T22:23:41.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:41 smithi099 ceph-mon[34892]: pgmap v462: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.6 MiB/s wr, 481 op/s 2024-09-15T22:23:41.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:41 smithi129 ceph-mon[35314]: pgmap v462: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 4.6 MiB/s wr, 481 op/s 2024-09-15T22:23:43.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:43 smithi005 ceph-mon[30944]: pgmap v463: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.9 MiB/s wr, 623 op/s 2024-09-15T22:23:43.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:43 smithi099 ceph-mon[34892]: pgmap v463: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.9 MiB/s wr, 623 op/s 2024-09-15T22:23:43.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:43 smithi129 ceph-mon[35314]: pgmap v463: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 5.9 MiB/s wr, 623 op/s 2024-09-15T22:23:44.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.071+0000 7fb35a339640 1 -- 172.21.15.5:0/2047411977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb354102d30 msgr2=0x7fb354103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:44.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.071+0000 7fb35a339640 1 --2- 172.21.15.5:0/2047411977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb354102d30 0x7fb354103110 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fb344009930 tx=0x7fb34402a420 comp rx=0 tx=0).stop 2024-09-15T22:23:44.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.071+0000 7fb35a339640 1 -- 172.21.15.5:0/2047411977 shutdown_connections 2024-09-15T22:23:44.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.071+0000 7fb35a339640 1 --2- 172.21.15.5:0/2047411977 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb3541082a0 0x7fb35410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.071+0000 7fb35a339640 1 --2- 172.21.15.5:0/2047411977 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3541036e0 0x7fb354107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.071+0000 7fb35a339640 1 --2- 172.21.15.5:0/2047411977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb354102d30 0x7fb354103110 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.071+0000 7fb35a339640 1 -- 172.21.15.5:0/2047411977 >> 172.21.15.5:0/2047411977 conn(0x7fb3540fc7d0 msgr2=0x7fb3540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.071+0000 7fb35a339640 1 -- 172.21.15.5:0/2047411977 shutdown_connections 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.071+0000 7fb35a339640 1 -- 172.21.15.5:0/2047411977 wait complete. 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb35a339640 1 Processor -- start 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb35a339640 1 -- start start 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb35a339640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb354102d30 0x7fb3541985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb35a339640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3541036e0 0x7fb354198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb35a339640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3541082a0 0x7fb35419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb35a339640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3541120a0 con 0x7fb3541082a0 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb35a339640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb354111f20 con 0x7fb354102d30 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb35a339640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb354112220 con 0x7fb3541036e0 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb3588d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3541082a0 0x7fb35419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb3588d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3541082a0 0x7fb35419ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52228/0 (socket says 172.21.15.5:52228) 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb3588d6640 1 -- 172.21.15.5:0/553659309 learned_addr learned my addr 172.21.15.5:0/553659309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb353fff640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb354102d30 0x7fb3541985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb34bfff640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3541036e0 0x7fb354198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb3588d6640 1 -- 172.21.15.5:0/553659309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3541036e0 msgr2=0x7fb354198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb3588d6640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3541036e0 0x7fb354198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb3588d6640 1 -- 172.21.15.5:0/553659309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb354102d30 msgr2=0x7fb3541985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb3588d6640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb354102d30 0x7fb3541985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb3588d6640 1 -- 172.21.15.5:0/553659309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35419d590 con 0x7fb3541082a0 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.072+0000 7fb353fff640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb354102d30 0x7fb3541985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.073+0000 7fb3588d6640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3541082a0 0x7fb35419ceb0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00eb10 tx=0x7fb34c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.073+0000 7fb351ffb640 1 -- 172.21.15.5:0/553659309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c004b20 con 0x7fb3541082a0 2024-09-15T22:23:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.073+0000 7fb351ffb640 1 -- 172.21.15.5:0/553659309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb34c004cc0 con 0x7fb3541082a0 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.073+0000 7fb35a339640 1 -- 172.21.15.5:0/553659309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb35419d880 con 0x7fb3541082a0 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.073+0000 7fb351ffb640 1 -- 172.21.15.5:0/553659309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c010760 con 0x7fb3541082a0 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.073+0000 7fb35a339640 1 -- 172.21.15.5:0/553659309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3541a5160 con 0x7fb3541082a0 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.076+0000 7fb35c712740 1 Processor -- start 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.076+0000 7fb351ffb640 1 -- 172.21.15.5:0/553659309 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb34c005020 con 0x7fb3541082a0 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.076+0000 7fb35c712740 1 -- start start 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.076+0000 7fb35c712740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0fb40 0x55d835c0e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.076+0000 7fb35c712740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0e5d0 0x55d835c0ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.076+0000 7fb35c712740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c1fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.076+0000 7fb35c712740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d835b742f0 con 0x55d835c1d780 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.076+0000 7fb35c712740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d835b1bcf0 con 0x55d835c0e5d0 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.076+0000 7fb35c712740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d835b702b0 con 0x55d835c0fb40 2024-09-15T22:23:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c1fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c1fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52230/0 (socket says 172.21.15.5:52230) 2024-09-15T22:23:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34bfff640 1 -- 172.21.15.5:0/894661934 learned_addr learned my addr 172.21.15.5:0/894661934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb353fff640 1 --2- 172.21.15.5:0/894661934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0e5d0 0x55d835c0ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34bfff640 1 --2- 172.21.15.5:0/894661934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0fb40 0x55d835c0e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb351ffb640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb3400a3430 0x7fb3400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34bfff640 1 -- 172.21.15.5:0/894661934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0fb40 msgr2=0x55d835c0e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34bfff640 1 --2- 172.21.15.5:0/894661934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0fb40 0x55d835c0e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34bfff640 1 -- 172.21.15.5:0/894661934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0e5d0 msgr2=0x55d835c0ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34bfff640 1 --2- 172.21.15.5:0/894661934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0e5d0 0x55d835c0ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34bfff640 1 -- 172.21.15.5:0/894661934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d835b003a0 con 0x55d835c1d780 2024-09-15T22:23:44.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb351ffb640 1 -- 172.21.15.5:0/553659309 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7fb34c0d0660 con 0x7fb3541082a0 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb3588d6640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb3400a3430 0x7fb3400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34bfff640 1 --2- 172.21.15.5:0/894661934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c1fb70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00cfa0 tx=0x7fb33c00c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27be9c6f8726f2cc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34a7fc640 1 -- 172.21.15.5:0/894661934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c019030 con 0x55d835c1d780 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34a7fc640 1 -- 172.21.15.5:0/894661934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb33c005500 con 0x55d835c1d780 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.077+0000 7fb34a7fc640 1 -- 172.21.15.5:0/894661934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c011780 con 0x55d835c1d780 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.078+0000 7fb35c712740 1 -- 172.21.15.5:0/894661934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 msgr2=0x55d835c1fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.078+0000 7fb3588d6640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb3400a3430 0x7fb3400a58f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0cda60 tx=0x7fb34c00efd0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.078+0000 7fb35c712740 1 --2- 172.21.15.5:0/894661934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c1fb70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00cfa0 tx=0x7fb33c00c670 comp rx=0 tx=0).stop 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.078+0000 7fb35c712740 1 -- 172.21.15.5:0/894661934 shutdown_connections 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.078+0000 7fb35c712740 1 --2- 172.21.15.5:0/894661934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c1fb70 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.078+0000 7fb35c712740 1 --2- 172.21.15.5:0/894661934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0e5d0 0x55d835c0ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.078+0000 7fb35c712740 1 --2- 172.21.15.5:0/894661934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0fb40 0x55d835c0e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.078+0000 7fb35c712740 1 -- 172.21.15.5:0/894661934 >> 172.21.15.5:0/894661934 conn(0x55d835bd7310 msgr2=0x55d835c0ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:44.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.078+0000 7fb35c712740 1 -- 172.21.15.5:0/894661934 shutdown_connections 2024-09-15T22:23:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.078+0000 7fb35c712740 1 -- 172.21.15.5:0/894661934 wait complete. 2024-09-15T22:23:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.078+0000 7fb35c712740 1 -- start start 2024-09-15T22:23:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb35c712740 1 Processor -- start 2024-09-15T22:23:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb35c712740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0e5d0 0x55d835c1d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb35c712740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0fb40 0x55d835c1a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb35c712740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c1d780 0x55d835c1aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb35c712740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d835c20480 con 0x55d835c0e5d0 2024-09-15T22:23:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb35c712740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55d835b2f060 con 0x55d835c1d780 2024-09-15T22:23:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb35c712740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55d835c20600 con 0x55d835c0fb40 2024-09-15T22:23:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb35c712740 10 client.0 _ll_register_callbacks cb 0x55d8358828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:44.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0e5d0 0x55d835c1d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.035 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:23:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0e5d0 0x55d835c1d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52238/0 (socket says 172.21.15.5:52238) 2024-09-15T22:23:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb34bfff640 1 -- 172.21.15.5:0/2858670715 learned_addr learned my addr 172.21.15.5:0/2858670715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb34bfff640 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c1d780 0x55d835c1aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb353fff640 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0fb40 0x55d835c1a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb34bfff640 1 -- 172.21.15.5:0/2858670715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0fb40 msgr2=0x55d835c1a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb34bfff640 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0fb40 0x55d835c1a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb34bfff640 1 -- 172.21.15.5:0/2858670715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c1d780 msgr2=0x55d835c1aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb34bfff640 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c1d780 0x55d835c1aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb34bfff640 1 -- 172.21.15.5:0/2858670715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8358e53c0 con 0x55d835c0e5d0 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.079+0000 7fb353fff640 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0fb40 0x55d835c1a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb34bfff640 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0e5d0 0x55d835c1d190 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb33c004e40 tx=0x7fb33c004d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f78d7f7ef73be37 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb3337fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb3337fe640 1 -- 172.21.15.5:0/2858670715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c019040 con 0x55d835c0e5d0 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb3337fe640 1 -- 172.21.15.5:0/2858670715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7fb33c011520 con 0x55d835c0e5d0 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb35c712740 1 -- 172.21.15.5:0/2858670715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb33c005500 con 0x55d835c0e5d0 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb3337fe640 1 -- 172.21.15.5:0/2858670715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb33c031050 con 0x55d835c0e5d0 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb35c712740 10 client.17856 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:44.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb35c712740 1 -- 172.21.15.5:0/2858670715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d835c2bf50 con 0x55d835c0e5d0 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb3337fe640 1 -- 172.21.15.5:0/2858670715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb33c005500 con 0x55d835c0e5d0 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb3337fe640 1 -- 172.21.15.5:0/2858670715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb33c01e6c0 con 0x55d835c0e5d0 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb35c712740 10 client.17856 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb35c712740 10 client.17856 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb35c712740 20 client.17856 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb35c712740 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55d835c2d2a0 0x55d835c2f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb35c712740 4 client.17856 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.080+0000 7fb35c712740 1 -- 172.21.15.5:0/2858670715 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d835b81860 con 0x55d835c2d2a0 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.081+0000 7fb353fff640 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55d835c2d2a0 0x55d835c2f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.081+0000 7fb353fff640 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55d835c2d2a0 0x55d835c2f690 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73163a9f246e0d41 server_cookie=ee0295201f124186 in_seq=0 out_seq=0 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.081+0000 7fb3337fe640 10 client.17856 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb3337fe640 1 -- 172.21.15.5:0/2858670715 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d835b81860 con 0x55d835c2d2a0 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb3337fe640 10 client.17856 handle_command_reply: tid=0 2024-09-15T22:23:44.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 1 client.17856 shutdown 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 1 -- 172.21.15.5:0/2858670715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0e5d0 msgr2=0x55d835c1d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0e5d0 0x55d835c1d190 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb33c004e40 tx=0x7fb33c004d40 comp rx=0 tx=0).stop 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 1 -- 172.21.15.5:0/2858670715 shutdown_connections 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55d835c2d2a0 0x55d835c2f690 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c1d780 0x55d835c1aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0fb40 0x55d835c1a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 1 --2- 172.21.15.5:0/2858670715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0e5d0 0x55d835c1d190 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb3337fe640 0 client.17856 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 1 -- 172.21.15.5:0/2858670715 >> 172.21.15.5:0/2858670715 conn(0x55d835bd7310 msgr2=0x55d835c10960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 1 -- 172.21.15.5:0/2858670715 shutdown_connections 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 1 -- 172.21.15.5:0/2858670715 wait complete. 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.082+0000 7fb35c712740 20 client.17856 trim_cache size 0 max 16384 2024-09-15T22:23:44.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.095+0000 7fb35c712740 1 Processor -- start 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.095+0000 7fb35c712740 1 -- start start 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.095+0000 7fb35c712740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0e5d0 0x55d835c29720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.095+0000 7fb35c712740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0fb40 0x55d835c19310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.095+0000 7fb35c712740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c34dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.095+0000 7fb35c712740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3541af7e0 con 0x55d835c1d780 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.095+0000 7fb35c712740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb3541ad0b0 con 0x55d835c0fb40 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.095+0000 7fb35c712740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb3541aaf20 con 0x55d835c0e5d0 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.095+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c34dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c34dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52252/0 (socket says 172.21.15.5:52252) 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb34bfff640 1 -- 172.21.15.5:0/2412588520 learned_addr learned my addr 172.21.15.5:0/2412588520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb353fff640 1 --2- 172.21.15.5:0/2412588520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0fb40 0x55d835c19310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb34bfff640 1 --2- 172.21.15.5:0/2412588520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0e5d0 0x55d835c29720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb34bfff640 1 -- 172.21.15.5:0/2412588520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0e5d0 msgr2=0x55d835c29720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:44.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb34bfff640 1 --2- 172.21.15.5:0/2412588520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0e5d0 0x55d835c29720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb34bfff640 1 -- 172.21.15.5:0/2412588520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0fb40 msgr2=0x55d835c19310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb34bfff640 1 --2- 172.21.15.5:0/2412588520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0fb40 0x55d835c19310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb34bfff640 1 -- 172.21.15.5:0/2412588520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d835c0b1a0 con 0x55d835c1d780 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb34bfff640 1 --2- 172.21.15.5:0/2412588520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c34dd0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb33c004e10 tx=0x7fb33c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f01ac7b31f953d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb349ffb640 1 -- 172.21.15.5:0/2412588520 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c00cd00 con 0x55d835c1d780 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb349ffb640 1 -- 172.21.15.5:0/2412588520 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb33c005230 con 0x55d835c1d780 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.096+0000 7fb349ffb640 1 -- 172.21.15.5:0/2412588520 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c011520 con 0x55d835c1d780 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.097+0000 7fb35c712740 1 -- 172.21.15.5:0/2412588520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 msgr2=0x55d835c34dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.097+0000 7fb35c712740 1 --2- 172.21.15.5:0/2412588520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c34dd0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb33c004e10 tx=0x7fb33c005f30 comp rx=0 tx=0).stop 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.097+0000 7fb35c712740 1 -- 172.21.15.5:0/2412588520 shutdown_connections 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.097+0000 7fb35c712740 1 --2- 172.21.15.5:0/2412588520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c1d780 0x55d835c34dd0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.097+0000 7fb35c712740 1 --2- 172.21.15.5:0/2412588520 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0fb40 0x55d835c19310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.097+0000 7fb35c712740 1 --2- 172.21.15.5:0/2412588520 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c0e5d0 0x55d835c29720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.097+0000 7fb35c712740 1 -- 172.21.15.5:0/2412588520 >> 172.21.15.5:0/2412588520 conn(0x55d835bd7310 msgr2=0x55d835c18c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.097+0000 7fb35c712740 1 -- 172.21.15.5:0/2412588520 shutdown_connections 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.097+0000 7fb35c712740 1 -- 172.21.15.5:0/2412588520 wait complete. 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.097+0000 7fb35c712740 1 -- start start 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.097+0000 7fb35c712740 1 Processor -- start 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb35c712740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0e5d0 0x55d835c25a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb35c712740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0fb40 0x55d835c25f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb35c712740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c1d780 0x55d835c16c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb35c712740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3541ab570 con 0x55d835c0fb40 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb35c712740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb3541ac9e0 con 0x55d835c0e5d0 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb35c712740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb3541abc40 con 0x55d835c1d780 2024-09-15T22:23:44.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb35c712740 10 client.0 _ll_register_callbacks cb 0x7fb344009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb353fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0fb40 0x55d835c25f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb353fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0fb40 0x55d835c25f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52262/0 (socket says 172.21.15.5:52262) 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb353fff640 1 -- 172.21.15.5:0/3054978339 learned_addr learned my addr 172.21.15.5:0/3054978339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb34bfff640 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0e5d0 0x55d835c25a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb34bfff640 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c1d780 0x55d835c16c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb353fff640 1 -- 172.21.15.5:0/3054978339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c1d780 msgr2=0x55d835c16c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb353fff640 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c1d780 0x55d835c16c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb353fff640 1 -- 172.21.15.5:0/3054978339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0e5d0 msgr2=0x55d835c25a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb353fff640 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0e5d0 0x55d835c25a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb353fff640 1 -- 172.21.15.5:0/3054978339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d835c174c0 con 0x55d835c0fb40 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb34bfff640 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0e5d0 0x55d835c25a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb353fff640 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0fb40 0x55d835c25f80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb344009900 tx=0x7fb3440041c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa475bb85c93bfb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:44.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.098+0000 7fb3337fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.119+0000 7fb3337fe640 1 -- 172.21.15.5:0/3054978339 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34404a070 con 0x55d835c0fb40 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.119+0000 7fb35c712740 1 -- 172.21.15.5:0/3054978339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb33c005230 con 0x55d835c0fb40 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.119+0000 7fb3337fe640 1 -- 172.21.15.5:0/3054978339 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7fb344005220 con 0x55d835c0fb40 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.119+0000 7fb3337fe640 1 -- 172.21.15.5:0/3054978339 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb344035040 con 0x55d835c0fb40 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.120+0000 7fb35c712740 10 client.17862 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.120+0000 7fb35c712740 1 -- 172.21.15.5:0/3054978339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d835c977e0 con 0x55d835c0fb40 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.121+0000 7fb3337fe640 1 -- 172.21.15.5:0/3054978339 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb344004c80 con 0x55d835c0fb40 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.121+0000 7fb3337fe640 1 -- 172.21.15.5:0/3054978339 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb344031b00 con 0x55d835c0fb40 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.121+0000 7fb35c712740 10 client.17862 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.121+0000 7fb35c712740 10 client.17862 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.121+0000 7fb35c712740 20 client.17862 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.121+0000 7fb35c712740 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55d835c97f00 0x55d835c9a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:44.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.121+0000 7fb35c712740 4 client.17862 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.121+0000 7fb35c712740 1 -- 172.21.15.5:0/3054978339 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb33c005230 con 0x55d835c97f00 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.122+0000 7fb34bfff640 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55d835c97f00 0x55d835c9a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.125+0000 7fb34bfff640 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55d835c97f00 0x55d835c9a320 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e15c9b218836815 server_cookie=1e0b97be1e00b90a in_seq=0 out_seq=0 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.125+0000 7fb3337fe640 10 client.17862 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.125+0000 7fb3337fe640 1 -- 172.21.15.5:0/3054978339 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb33c005230 con 0x55d835c97f00 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.125+0000 7fb3337fe640 10 client.17862 handle_command_reply: tid=0 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.125+0000 7fb35c712740 1 client.17862 shutdown 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.131+0000 7fb35c712740 1 -- 172.21.15.5:0/3054978339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0fb40 msgr2=0x55d835c25f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.131+0000 7fb35c712740 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0fb40 0x55d835c25f80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb344009900 tx=0x7fb3440041c0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.133+0000 7fb35c712740 1 -- 172.21.15.5:0/3054978339 shutdown_connections 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.133+0000 7fb35c712740 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x55d835c97f00 0x55d835c9a320 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.133+0000 7fb35c712740 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55d835c1d780 0x55d835c16c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.133+0000 7fb35c712740 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d835c0fb40 0x55d835c25f80 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.133+0000 7fb35c712740 1 --2- 172.21.15.5:0/3054978339 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55d835c0e5d0 0x55d835c25a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.133+0000 7fb35c712740 1 -- 172.21.15.5:0/3054978339 >> 172.21.15.5:0/3054978339 conn(0x55d835bd7310 msgr2=0x55d835c19420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:44.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.133+0000 7fb3337fe640 0 client.17862 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:23:44.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.134+0000 7fb35c712740 1 -- 172.21.15.5:0/3054978339 shutdown_connections 2024-09-15T22:23:44.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.135+0000 7fb35c712740 1 -- 172.21.15.5:0/3054978339 wait complete. 2024-09-15T22:23:44.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.135+0000 7fb35c712740 20 client.17862 trim_cache size 0 max 16384 2024-09-15T22:23:44.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.136+0000 7fb35a339640 1 -- 172.21.15.5:0/553659309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb3400a3430 msgr2=0x7fb3400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:44.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.136+0000 7fb35a339640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb3400a3430 0x7fb3400a58f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0cda60 tx=0x7fb34c00efd0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.136+0000 7fb35a339640 1 -- 172.21.15.5:0/553659309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3541082a0 msgr2=0x7fb35419ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.136+0000 7fb35a339640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3541082a0 0x7fb35419ceb0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00eb10 tx=0x7fb34c00c2d0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.138+0000 7fb35a339640 1 -- 172.21.15.5:0/553659309 shutdown_connections 2024-09-15T22:23:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.138+0000 7fb35a339640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb3400a3430 0x7fb3400a58f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.138+0000 7fb35a339640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3541082a0 0x7fb35419ceb0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.138+0000 7fb35a339640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb3541036e0 0x7fb354198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.138+0000 7fb35a339640 1 --2- 172.21.15.5:0/553659309 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb354102d30 0x7fb3541985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.138+0000 7fb35a339640 1 -- 172.21.15.5:0/553659309 >> 172.21.15.5:0/553659309 conn(0x7fb3540fc7d0 msgr2=0x7fb354105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.138+0000 7fb35a339640 1 -- 172.21.15.5:0/553659309 shutdown_connections 2024-09-15T22:23:44.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:43.138+0000 7fb35a339640 1 -- 172.21.15.5:0/553659309 wait complete. 2024-09-15T22:23:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 855.0 s, 528.2 tps, lat 188.754 ms stddev 228.891 2024-09-15T22:23:45.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:45 smithi005 ceph-mon[30944]: pgmap v464: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.1 MiB/s wr, 539 op/s 2024-09-15T22:23:45.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:45 smithi099 ceph-mon[34892]: pgmap v464: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.1 MiB/s wr, 539 op/s 2024-09-15T22:23:45.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:45 smithi129 ceph-mon[35314]: pgmap v464: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 5.1 MiB/s wr, 539 op/s 2024-09-15T22:23:46.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:23:47.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:47 smithi005 ceph-mon[30944]: pgmap v465: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.0 MiB/s wr, 515 op/s 2024-09-15T22:23:47.166 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:47.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:47 smithi099 ceph-mon[34892]: pgmap v465: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.0 MiB/s wr, 515 op/s 2024-09-15T22:23:47.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:47 smithi129 ceph-mon[35314]: pgmap v465: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.0 MiB/s wr, 515 op/s 2024-09-15T22:23:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.770+0000 7f2127725640 1 -- 172.21.15.5:0/1481852546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120109450 msgr2=0x7f2120110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:47.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.770+0000 7f2127725640 1 --2- 172.21.15.5:0/1481852546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120109450 0x7f2120110250 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2110009900 tx=0x7f211002a440 comp rx=0 tx=0).stop 2024-09-15T22:23:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.773+0000 7f2127725640 1 -- 172.21.15.5:0/1481852546 shutdown_connections 2024-09-15T22:23:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.773+0000 7f2127725640 1 --2- 172.21.15.5:0/1481852546 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f212011b140 0x7f212011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.773+0000 7f2127725640 1 --2- 172.21.15.5:0/1481852546 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2120109830 0x7f2120109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.773+0000 7f2127725640 1 --2- 172.21.15.5:0/1481852546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120109450 0x7f2120110250 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.773+0000 7f2127725640 1 -- 172.21.15.5:0/1481852546 >> 172.21.15.5:0/1481852546 conn(0x7f212006e140 msgr2=0x7f212006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.778+0000 7f2127725640 1 -- 172.21.15.5:0/1481852546 shutdown_connections 2024-09-15T22:23:47.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.779+0000 7f2127725640 1 -- 172.21.15.5:0/1481852546 wait complete. 2024-09-15T22:23:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.780+0000 7f2127725640 1 Processor -- start 2024-09-15T22:23:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.780+0000 7f2127725640 1 -- start start 2024-09-15T22:23:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.780+0000 7f2127725640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2120109450 0x7f212011a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.780+0000 7f2127725640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120109830 0x7f21201149d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.780+0000 7f2127725640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f212011b140 0x7f2120114f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.781+0000 7f2127725640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2120111370 con 0x7f2120109830 2024-09-15T22:23:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.781+0000 7f2127725640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f21201111f0 con 0x7f2120109450 2024-09-15T22:23:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.781+0000 7f2127725640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f21201114f0 con 0x7f212011b140 2024-09-15T22:23:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.781+0000 7f2124cc0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120109830 0x7f21201149d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.781+0000 7f2124cc0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120109830 0x7f21201149d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40824/0 (socket says 172.21.15.5:40824) 2024-09-15T22:23:47.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.781+0000 7f2124cc0640 1 -- 172.21.15.5:0/331145754 learned_addr learned my addr 172.21.15.5:0/331145754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.782+0000 7f2124cc0640 1 -- 172.21.15.5:0/331145754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f212011b140 msgr2=0x7f2120114f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.782+0000 7f2124cc0640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f212011b140 0x7f2120114f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.782+0000 7f2124cc0640 1 -- 172.21.15.5:0/331145754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2120109450 msgr2=0x7f212011a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.782+0000 7f2124cc0640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2120109450 0x7f212011a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.782+0000 7f2124cc0640 1 -- 172.21.15.5:0/331145754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21201157d0 con 0x7f2120109830 2024-09-15T22:23:47.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.782+0000 7f2124cc0640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120109830 0x7f21201149d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f211c00b450 tx=0x7f211c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:47.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.783+0000 7f21167fc640 1 -- 172.21.15.5:0/331145754 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c015020 con 0x7f2120109830 2024-09-15T22:23:47.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.783+0000 7f21167fc640 1 -- 172.21.15.5:0/331145754 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c011d70 con 0x7f2120109830 2024-09-15T22:23:47.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.784+0000 7f21167fc640 1 -- 172.21.15.5:0/331145754 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c00f340 con 0x7f2120109830 2024-09-15T22:23:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.784+0000 7f2127725640 1 -- 172.21.15.5:0/331145754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f212007a4b0 con 0x7f2120109830 2024-09-15T22:23:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.784+0000 7f2127725640 1 -- 172.21.15.5:0/331145754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f212007a970 con 0x7f2120109830 2024-09-15T22:23:47.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.793+0000 7f2129afe740 1 Processor -- start 2024-09-15T22:23:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.793+0000 7f21167fc640 1 -- 172.21.15.5:0/331145754 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f211c018020 con 0x7f2120109830 2024-09-15T22:23:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.794+0000 7f2129afe740 1 -- start start 2024-09-15T22:23:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.794+0000 7f2129afe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0107b40 0x561ec0106090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.794+0000 7f2129afe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 0x561ec0106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.794+0000 7f2129afe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0111370 0x561ec0113760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.794+0000 7f2129afe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561ec006c2f0 con 0x561ec01065d0 2024-09-15T22:23:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.794+0000 7f2129afe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561ec0013cf0 con 0x561ec0107b40 2024-09-15T22:23:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.794+0000 7f2129afe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561ec00682b0 con 0x561ec0111370 2024-09-15T22:23:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.794+0000 7f2124cc0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0111370 0x561ec0113760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.794+0000 7f2124cc0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0111370 0x561ec0113760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53810/0 (socket says 172.21.15.5:53810) 2024-09-15T22:23:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.794+0000 7f2124cc0640 1 -- 172.21.15.5:0/2997722280 learned_addr learned my addr 172.21.15.5:0/2997722280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.795+0000 7f2125cc2640 1 --2- 172.21.15.5:0/2997722280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0107b40 0x561ec0106090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.796+0000 7f21167fc640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f210c0a34a0 0x7f210c0a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.796+0000 7f21167fc640 1 -- 172.21.15.5:0/331145754 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f211c0cf940 con 0x7f2120109830 2024-09-15T22:23:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.796+0000 7f21254c1640 1 --2- 172.21.15.5:0/2997722280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 0x561ec0106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.796+0000 7f21254c1640 1 -- 172.21.15.5:0/2997722280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0111370 msgr2=0x561ec0113760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.796+0000 7f21254c1640 1 --2- 172.21.15.5:0/2997722280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0111370 0x561ec0113760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.796+0000 7f21254c1640 1 -- 172.21.15.5:0/2997722280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0107b40 msgr2=0x561ec0106090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.796+0000 7f21254c1640 1 --2- 172.21.15.5:0/2997722280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0107b40 0x561ec0106090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.797+0000 7f21254c1640 1 -- 172.21.15.5:0/2997722280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ebfff83a0 con 0x561ec01065d0 2024-09-15T22:23:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.797+0000 7f21254c1640 1 --2- 172.21.15.5:0/2997722280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 0x561ec0106a10 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2110009850 tx=0x7f2110004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42028c8062ce7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.797+0000 7f210b7fe640 1 -- 172.21.15.5:0/2997722280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211003e070 con 0x561ec01065d0 2024-09-15T22:23:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.797+0000 7f210b7fe640 1 -- 172.21.15.5:0/2997722280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2110004470 con 0x561ec01065d0 2024-09-15T22:23:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.797+0000 7f210b7fe640 1 -- 172.21.15.5:0/2997722280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110035040 con 0x561ec01065d0 2024-09-15T22:23:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.799+0000 7f2125cc2640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f210c0a34a0 0x7f210c0a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.800+0000 7f2129afe740 1 -- 172.21.15.5:0/2997722280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 msgr2=0x561ec0106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:47.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.800+0000 7f2129afe740 1 --2- 172.21.15.5:0/2997722280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 0x561ec0106a10 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2110009850 tx=0x7f2110004040 comp rx=0 tx=0).stop 2024-09-15T22:23:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.803+0000 7f2125cc2640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f210c0a34a0 0x7f210c0a5960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2118009ef0 tx=0x7f2118004b90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.803+0000 7f2129afe740 1 -- 172.21.15.5:0/2997722280 shutdown_connections 2024-09-15T22:23:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.803+0000 7f2129afe740 1 --2- 172.21.15.5:0/2997722280 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0111370 0x561ec0113760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.803+0000 7f2129afe740 1 --2- 172.21.15.5:0/2997722280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 0x561ec0106a10 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.803+0000 7f2129afe740 1 --2- 172.21.15.5:0/2997722280 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0107b40 0x561ec0106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.803+0000 7f2129afe740 1 -- 172.21.15.5:0/2997722280 >> 172.21.15.5:0/2997722280 conn(0x561ec00cf310 msgr2=0x561ec0102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:47.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.810+0000 7f2129afe740 1 -- 172.21.15.5:0/2997722280 shutdown_connections 2024-09-15T22:23:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.814+0000 7f2129afe740 1 -- 172.21.15.5:0/2997722280 wait complete. 2024-09-15T22:23:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.814+0000 7f2129afe740 1 -- start start 2024-09-15T22:23:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.814+0000 7f2129afe740 1 Processor -- start 2024-09-15T22:23:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.814+0000 7f2129afe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 0x561ec0110d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.815+0000 7f2129afe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 0x561ec010e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.815+0000 7f2129afe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0111370 0x561ec010e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.815+0000 7f2129afe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561ec0114010 con 0x561ec01065d0 2024-09-15T22:23:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.815+0000 7f2129afe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x561ec0027060 con 0x561ec0111370 2024-09-15T22:23:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.815+0000 7f2129afe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x561ec0114190 con 0x561ec0107b40 2024-09-15T22:23:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.815+0000 7f2129afe740 10 client.0 _ll_register_callbacks cb 0x561ebfd7a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.817+0000 7f21254c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 0x561ec0110d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.817+0000 7f21254c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 0x561ec0110d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40836/0 (socket says 172.21.15.5:40836) 2024-09-15T22:23:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.817+0000 7f21254c1640 1 -- 172.21.15.5:0/4120958996 learned_addr learned my addr 172.21.15.5:0/4120958996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:47.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.817+0000 7f2125cc2640 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0111370 0x561ec010e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.820+0000 7f2124cc0640 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 0x561ec010e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.820+0000 7f21254c1640 1 -- 172.21.15.5:0/4120958996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 msgr2=0x561ec010e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:47.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.820+0000 7f21254c1640 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 0x561ec010e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.820+0000 7f21254c1640 1 -- 172.21.15.5:0/4120958996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0111370 msgr2=0x561ec010e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.820+0000 7f21254c1640 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0111370 0x561ec010e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.820+0000 7f21254c1640 1 -- 172.21.15.5:0/4120958996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ebfddd3c0 con 0x561ec01065d0 2024-09-15T22:23:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.824+0000 7f21254c1640 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 0x561ec0110d80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2110004fd0 tx=0x7f2110009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481645acbc18b9e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.824+0000 7f2108ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:23:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.831+0000 7f2108ff9640 1 -- 172.21.15.5:0/4120958996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211003e030 con 0x561ec01065d0 2024-09-15T22:23:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.831+0000 7f2108ff9640 1 -- 172.21.15.5:0/4120958996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f2110005050 con 0x561ec01065d0 2024-09-15T22:23:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.831+0000 7f2129afe740 1 -- 172.21.15.5:0/4120958996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2110004470 con 0x561ec01065d0 2024-09-15T22:23:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.839+0000 7f2108ff9640 1 -- 172.21.15.5:0/4120958996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2110035040 con 0x561ec01065d0 2024-09-15T22:23:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.840+0000 7f2129afe740 10 client.17874 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:47.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.840+0000 7f2129afe740 1 -- 172.21.15.5:0/4120958996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561ec011fae0 con 0x561ec01065d0 2024-09-15T22:23:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.845+0000 7f2108ff9640 1 -- 172.21.15.5:0/4120958996 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2110004470 con 0x561ec01065d0 2024-09-15T22:23:47.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.845+0000 7f2108ff9640 1 -- 172.21.15.5:0/4120958996 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f21100035f0 con 0x561ec01065d0 2024-09-15T22:23:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.845+0000 7f2129afe740 10 client.17874 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.845+0000 7f2129afe740 10 client.17874 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:23:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.845+0000 7f2129afe740 20 client.17874 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.845+0000 7f2129afe740 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561ec0121030 0x561ec0123480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.845+0000 7f2129afe740 4 client.17874 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:47.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.845+0000 7f2129afe740 1 -- 172.21.15.5:0/4120958996 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ec0079860 con 0x561ec0121030 2024-09-15T22:23:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.849+0000 7f2124cc0640 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561ec0121030 0x561ec0123480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.852+0000 7f2124cc0640 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561ec0121030 0x561ec0123480 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98a3004b0846deca server_cookie=f468d6444bf6ca24 in_seq=0 out_seq=0 2024-09-15T22:23:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.853+0000 7f2108ff9640 10 client.17874 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:23:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.854+0000 7f2108ff9640 1 -- 172.21.15.5:0/4120958996 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ec0079860 con 0x561ec0121030 2024-09-15T22:23:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.854+0000 7f2108ff9640 10 client.17874 handle_command_reply: tid=0 2024-09-15T22:23:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.855+0000 7f2129afe740 1 client.17874 shutdown 2024-09-15T22:23:47.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2129afe740 1 -- 172.21.15.5:0/4120958996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 msgr2=0x561ec0110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2129afe740 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 0x561ec0110d80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2110004fd0 tx=0x7f2110009590 comp rx=0 tx=0).stop 2024-09-15T22:23:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2129afe740 1 -- 172.21.15.5:0/4120958996 shutdown_connections 2024-09-15T22:23:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2129afe740 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561ec0121030 0x561ec0123480 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2129afe740 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0111370 0x561ec010e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2129afe740 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 0x561ec010e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2129afe740 1 --2- 172.21.15.5:0/4120958996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec01065d0 0x561ec0110d80 secure :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f2110004fd0 tx=0x7f2110009590 comp rx=0 tx=0).stop 2024-09-15T22:23:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2129afe740 1 -- 172.21.15.5:0/4120958996 >> 172.21.15.5:0/4120958996 conn(0x561ec00cf310 msgr2=0x561ec0108930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2108ff9640 0 client.17874 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:23:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2129afe740 1 -- 172.21.15.5:0/4120958996 shutdown_connections 2024-09-15T22:23:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2129afe740 1 -- 172.21.15.5:0/4120958996 wait complete. 2024-09-15T22:23:47.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.867+0000 7f2129afe740 20 client.17874 trim_cache size 0 max 16384 2024-09-15T22:23:47.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.893+0000 7f2129afe740 1 Processor -- start 2024-09-15T22:23:47.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.893+0000 7f2129afe740 1 -- start start 2024-09-15T22:23:47.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2129afe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0107b40 0x561ec0101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2129afe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec0111370 0x561ec0109120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2129afe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec010e110 0x561ec0109660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2129afe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f21201d9fc0 con 0x561ec0111370 2024-09-15T22:23:47.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2129afe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2120084b00 con 0x561ec0107b40 2024-09-15T22:23:47.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2129afe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2120082970 con 0x561ec010e110 2024-09-15T22:23:47.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2124cc0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec0111370 0x561ec0109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2125cc2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec010e110 0x561ec0109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2125cc2640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec010e110 0x561ec0109660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53824/0 (socket says 172.21.15.5:53824) 2024-09-15T22:23:47.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2125cc2640 1 -- 172.21.15.5:0/1842190458 learned_addr learned my addr 172.21.15.5:0/1842190458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:47.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f21254c1640 1 --2- 172.21.15.5:0/1842190458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0107b40 0x561ec0101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2124cc0640 1 -- 172.21.15.5:0/1842190458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec010e110 msgr2=0x561ec0109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:47.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2124cc0640 1 --2- 172.21.15.5:0/1842190458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec010e110 0x561ec0109660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2124cc0640 1 -- 172.21.15.5:0/1842190458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0107b40 msgr2=0x561ec0101bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2124cc0640 1 --2- 172.21.15.5:0/1842190458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0107b40 0x561ec0101bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.894+0000 7f2124cc0640 1 -- 172.21.15.5:0/1842190458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ec010e9b0 con 0x561ec0111370 2024-09-15T22:23:47.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.895+0000 7f2124cc0640 1 --2- 172.21.15.5:0/1842190458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec0111370 0x561ec0109120 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f211c009fd0 tx=0x7f211c0d5440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddeaed0b5d7834 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.896+0000 7f210affd640 1 -- 172.21.15.5:0/1842190458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0b4070 con 0x561ec0111370 2024-09-15T22:23:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.896+0000 7f210affd640 1 -- 172.21.15.5:0/1842190458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c09bfd0 con 0x561ec0111370 2024-09-15T22:23:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.896+0000 7f210affd640 1 -- 172.21.15.5:0/1842190458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0a40b0 con 0x561ec0111370 2024-09-15T22:23:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.897+0000 7f2129afe740 1 -- 172.21.15.5:0/1842190458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec0111370 msgr2=0x561ec0109120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.897+0000 7f2129afe740 1 --2- 172.21.15.5:0/1842190458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec0111370 0x561ec0109120 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f211c009fd0 tx=0x7f211c0d5440 comp rx=0 tx=0).stop 2024-09-15T22:23:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.897+0000 7f2129afe740 1 -- 172.21.15.5:0/1842190458 shutdown_connections 2024-09-15T22:23:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.897+0000 7f2129afe740 1 --2- 172.21.15.5:0/1842190458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec010e110 0x561ec0109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.897+0000 7f2129afe740 1 --2- 172.21.15.5:0/1842190458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec0111370 0x561ec0109120 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.897+0000 7f2129afe740 1 --2- 172.21.15.5:0/1842190458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec0107b40 0x561ec0101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.897+0000 7f2129afe740 1 -- 172.21.15.5:0/1842190458 >> 172.21.15.5:0/1842190458 conn(0x561ec00cf310 msgr2=0x561ec01014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.897+0000 7f2129afe740 1 -- 172.21.15.5:0/1842190458 shutdown_connections 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.897+0000 7f2129afe740 1 -- 172.21.15.5:0/1842190458 wait complete. 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.898+0000 7f2129afe740 1 -- start start 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.898+0000 7f2129afe740 1 Processor -- start 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.898+0000 7f2129afe740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 0x561ec0119860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.898+0000 7f2129afe740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec010e110 0x561ec010aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.898+0000 7f2129afe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec0111370 0x561ec010af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.898+0000 7f2129afe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2120082fc0 con 0x561ec0111370 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.898+0000 7f2129afe740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2120084430 con 0x561ec010e110 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.898+0000 7f2129afe740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2120083690 con 0x561ec0107b40 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.898+0000 7f2129afe740 10 client.0 _ll_register_callbacks cb 0x7f2120110870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.899+0000 7f21254c1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 0x561ec0119860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.899+0000 7f21254c1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 0x561ec0119860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53828/0 (socket says 172.21.15.5:53828) 2024-09-15T22:23:47.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.899+0000 7f21254c1640 1 -- 172.21.15.5:0/2207771883 learned_addr learned my addr 172.21.15.5:0/2207771883 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.899+0000 7f21254c1640 1 -- 172.21.15.5:0/2207771883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec010e110 msgr2=0x561ec010aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.899+0000 7f21254c1640 1 --2- 172.21.15.5:0/2207771883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec010e110 0x561ec010aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.899+0000 7f21254c1640 1 -- 172.21.15.5:0/2207771883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec0111370 msgr2=0x561ec010af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.899+0000 7f21254c1640 1 --2- 172.21.15.5:0/2207771883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec0111370 0x561ec010af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.899+0000 7f21254c1640 1 -- 172.21.15.5:0/2207771883 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ec010b4c0 con 0x561ec0107b40 2024-09-15T22:23:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.904+0000 7f21254c1640 1 --2- 172.21.15.5:0/2207771883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 0x561ec0119860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2110005c40 tx=0x7f2110004ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd3c0c7e6e08a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.905+0000 7f2108ff9640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:23:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.905+0000 7f2108ff9640 1 -- 172.21.15.5:0/2207771883 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2110059040 con 0x561ec0107b40 2024-09-15T22:23:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.905+0000 7f2108ff9640 1 -- 172.21.15.5:0/2207771883 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f2110003b60 con 0x561ec0107b40 2024-09-15T22:23:47.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.906+0000 7f2129afe740 1 -- 172.21.15.5:0/2207771883 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f211c09bfd0 con 0x561ec0107b40 2024-09-15T22:23:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.908+0000 7f2108ff9640 1 -- 172.21.15.5:0/2207771883 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2110065050 con 0x561ec0107b40 2024-09-15T22:23:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.908+0000 7f2129afe740 10 client.27277 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.908+0000 7f2129afe740 1 -- 172.21.15.5:0/2207771883 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561ec018ba00 con 0x561ec0107b40 2024-09-15T22:23:47.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.914+0000 7f2108ff9640 1 -- 172.21.15.5:0/2207771883 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f211c09bfd0 con 0x561ec0107b40 2024-09-15T22:23:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.914+0000 7f2108ff9640 1 -- 172.21.15.5:0/2207771883 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f2110068020 con 0x561ec0107b40 2024-09-15T22:23:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.914+0000 7f2129afe740 10 client.27277 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:47.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.914+0000 7f2129afe740 10 client.27277 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:23:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.914+0000 7f2129afe740 20 client.27277 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.914+0000 7f2129afe740 1 --2- 172.21.15.5:0/2207771883 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561ec018c000 0x561ec018e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.914+0000 7f2129afe740 4 client.27277 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:23:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.914+0000 7f2129afe740 1 -- 172.21.15.5:0/2207771883 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ec0073d70 con 0x561ec018c000 2024-09-15T22:23:47.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.915+0000 7f2124cc0640 1 --2- 172.21.15.5:0/2207771883 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561ec018c000 0x561ec018e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:47.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.923+0000 7f2124cc0640 1 --2- 172.21.15.5:0/2207771883 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561ec018c000 0x561ec018e480 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b8428b9750c062e server_cookie=16436153ea4876c5 in_seq=0 out_seq=0 2024-09-15T22:23:47.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.923+0000 7f2108ff9640 10 client.27277 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:23:47.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.928+0000 7f2108ff9640 1 -- 172.21.15.5:0/2207771883 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ec0073d70 con 0x561ec018c000 2024-09-15T22:23:47.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.928+0000 7f2108ff9640 10 client.27277 handle_command_reply: tid=0 2024-09-15T22:23:47.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.928+0000 7f2129afe740 1 client.27277 shutdown 2024-09-15T22:23:47.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.928+0000 7f2129afe740 1 -- 172.21.15.5:0/2207771883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 msgr2=0x561ec0119860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:47.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.928+0000 7f2129afe740 1 --2- 172.21.15.5:0/2207771883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 0x561ec0119860 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2110005c40 tx=0x7f2110004ae0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.928+0000 7f2129afe740 1 -- 172.21.15.5:0/2207771883 shutdown_connections 2024-09-15T22:23:47.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.928+0000 7f2129afe740 1 --2- 172.21.15.5:0/2207771883 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x561ec018c000 0x561ec018e480 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.928+0000 7f2129afe740 1 --2- 172.21.15.5:0/2207771883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561ec0111370 0x561ec010af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.928+0000 7f2129afe740 1 --2- 172.21.15.5:0/2207771883 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x561ec010e110 0x561ec010aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.928+0000 7f2129afe740 1 --2- 172.21.15.5:0/2207771883 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x561ec0107b40 0x561ec0119860 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.928+0000 7f2129afe740 1 -- 172.21.15.5:0/2207771883 >> 172.21.15.5:0/2207771883 conn(0x561ec00cf310 msgr2=0x561ec0101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:47.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.930+0000 7f2108ff9640 0 client.27277 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:23:47.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.930+0000 7f2129afe740 1 -- 172.21.15.5:0/2207771883 shutdown_connections 2024-09-15T22:23:47.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.930+0000 7f2129afe740 1 -- 172.21.15.5:0/2207771883 wait complete. 2024-09-15T22:23:47.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.930+0000 7f2129afe740 20 client.27277 trim_cache size 0 max 16384 2024-09-15T22:23:47.932 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:23:47.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.931+0000 7f2127725640 1 -- 172.21.15.5:0/331145754 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f210c0a34a0 msgr2=0x7f210c0a5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:47.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.931+0000 7f2127725640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f210c0a34a0 0x7f210c0a5960 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2118009ef0 tx=0x7f2118004b90 comp rx=0 tx=0).stop 2024-09-15T22:23:47.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.931+0000 7f2127725640 1 -- 172.21.15.5:0/331145754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120109830 msgr2=0x7f21201149d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:47.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.931+0000 7f2127725640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120109830 0x7f21201149d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f211c00b450 tx=0x7f211c00b910 comp rx=0 tx=0).stop 2024-09-15T22:23:47.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.931+0000 7f2127725640 1 -- 172.21.15.5:0/331145754 shutdown_connections 2024-09-15T22:23:47.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.931+0000 7f2127725640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f210c0a34a0 0x7f210c0a5960 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.931+0000 7f2127725640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f212011b140 0x7f2120114f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.931+0000 7f2127725640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2120109830 0x7f21201149d0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.931+0000 7f2127725640 1 --2- 172.21.15.5:0/331145754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2120109450 0x7f212011a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.931+0000 7f2127725640 1 -- 172.21.15.5:0/331145754 >> 172.21.15.5:0/331145754 conn(0x7f212006e140 msgr2=0x7f2120070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.931+0000 7f2127725640 1 -- 172.21.15.5:0/331145754 shutdown_connections 2024-09-15T22:23:47.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:47.935+0000 7f2127725640 1 -- 172.21.15.5:0/331145754 wait complete. 2024-09-15T22:23:48.891 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:23:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:49.581 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:49 smithi005 ceph-mon[30944]: pgmap v466: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.9 MiB/s wr, 637 op/s 2024-09-15T22:23:49.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:49 smithi099 ceph-mon[34892]: pgmap v466: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.9 MiB/s wr, 637 op/s 2024-09-15T22:23:49.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:49 smithi129 ceph-mon[35314]: pgmap v466: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 5.9 MiB/s wr, 637 op/s 2024-09-15T22:23:49.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 860.0 s, 489.0 tps, lat 202.478 ms stddev 233.094 2024-09-15T22:23:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.026+0000 7f1a0458e640 1 -- 172.21.15.5:0/2072349174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc106a80 msgr2=0x7f19fc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.026+0000 7f1a0458e640 1 --2- 172.21.15.5:0/2072349174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc106a80 0x7f19fc106e60 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009930 tx=0x7f19f802a420 comp rx=0 tx=0).stop 2024-09-15T22:23:50.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.027+0000 7f1a0458e640 1 -- 172.21.15.5:0/2072349174 shutdown_connections 2024-09-15T22:23:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.027+0000 7f1a0458e640 1 --2- 172.21.15.5:0/2072349174 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19fc101420 0x7f19fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.027+0000 7f1a0458e640 1 --2- 172.21.15.5:0/2072349174 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19fc100a80 0x7f19fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.027+0000 7f1a0458e640 1 --2- 172.21.15.5:0/2072349174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc106a80 0x7f19fc106e60 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.027+0000 7f1a0458e640 1 -- 172.21.15.5:0/2072349174 >> 172.21.15.5:0/2072349174 conn(0x7f19fc0fc7a0 msgr2=0x7f19fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.027+0000 7f1a0458e640 1 -- 172.21.15.5:0/2072349174 shutdown_connections 2024-09-15T22:23:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.027+0000 7f1a0458e640 1 -- 172.21.15.5:0/2072349174 wait complete. 2024-09-15T22:23:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.027+0000 7f1a0458e640 1 Processor -- start 2024-09-15T22:23:50.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.027+0000 7f1a0458e640 1 -- start start 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.028+0000 7f1a0458e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19fc100a80 0x7f19fc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.028+0000 7f1a0458e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc101420 0x7f19fc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.028+0000 7f1a0458e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19fc106a80 0x7f19fc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.028+0000 7f1a0458e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc110e80 con 0x7f19fc101420 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.028+0000 7f1a0458e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc110d00 con 0x7f19fc100a80 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.028+0000 7f1a0458e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19fc111000 con 0x7f19fc106a80 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.029+0000 7f1a02b2b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19fc106a80 0x7f19fc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.029+0000 7f1a0232a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19fc100a80 0x7f19fc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.029+0000 7f1a0232a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19fc100a80 0x7f19fc198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:40870/0 (socket says 172.21.15.5:40870) 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.029+0000 7f1a0232a640 1 -- 172.21.15.5:0/2263587379 learned_addr learned my addr 172.21.15.5:0/2263587379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.029+0000 7f1a01b29640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc101420 0x7f19fc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.029+0000 7f1a0232a640 1 -- 172.21.15.5:0/2263587379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19fc106a80 msgr2=0x7f19fc19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.029+0000 7f1a0232a640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19fc106a80 0x7f19fc19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.029+0000 7f1a0232a640 1 -- 172.21.15.5:0/2263587379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc101420 msgr2=0x7f19fc1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.029+0000 7f1a0232a640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc101420 0x7f19fc1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.029+0000 7f1a0232a640 1 -- 172.21.15.5:0/2263587379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19fc19d430 con 0x7f19fc100a80 2024-09-15T22:23:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.030+0000 7f1a0232a640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19fc100a80 0x7f19fc198480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009230 tx=0x7f19f8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.030+0000 7f19f37fe640 1 -- 172.21.15.5:0/2263587379 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f803f030 con 0x7f19fc100a80 2024-09-15T22:23:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.030+0000 7f1a0458e640 1 -- 172.21.15.5:0/2263587379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19fc19d6c0 con 0x7f19fc100a80 2024-09-15T22:23:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.030+0000 7f19f37fe640 1 -- 172.21.15.5:0/2263587379 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f8003ad0 con 0x7f19fc100a80 2024-09-15T22:23:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.030+0000 7f19f37fe640 1 -- 172.21.15.5:0/2263587379 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f8035070 con 0x7f19fc100a80 2024-09-15T22:23:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.030+0000 7f1a0458e640 1 -- 172.21.15.5:0/2263587379 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19fc102580 con 0x7f19fc100a80 2024-09-15T22:23:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.033+0000 7f1a06967740 1 Processor -- start 2024-09-15T22:23:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.033+0000 7f1a06967740 1 -- start start 2024-09-15T22:23:50.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.033+0000 7f19f37fe640 1 -- 172.21.15.5:0/2263587379 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f19f8042030 con 0x7f19fc100a80 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a06967740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x55909785a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a06967740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 0x55909785aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a06967740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x559097867760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a06967740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5590977c02f0 con 0x559097865370 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a06967740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x559097767cf0 con 0x55909785a5d0 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a06967740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5590977bc2b0 con 0x55909785bb40 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a01b29640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x559097867760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a0232a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 0x55909785aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a02b2b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x55909785a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a02b2b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x55909785a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53858/0 (socket says 172.21.15.5:53858) 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a02b2b640 1 -- 172.21.15.5:0/2764757288 learned_addr learned my addr 172.21.15.5:0/2764757288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a02b2b640 1 -- 172.21.15.5:0/2764757288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 msgr2=0x55909785aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/2764757288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 0x55909785aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a02b2b640 1 -- 172.21.15.5:0/2764757288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 msgr2=0x559097867760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/2764757288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x559097867760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a02b2b640 1 -- 172.21.15.5:0/2764757288 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909774c3a0 con 0x55909785bb40 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f1a0232a640 1 --2- 172.21.15.5:0/2764757288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 0x55909785aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.034+0000 7f19f37fe640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19e80a33b0 0x7f19e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.035+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/2764757288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x55909785a090 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009f50 tx=0x7f19f40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a692dd4f4dca1e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.035+0000 7f1a0232a640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19e80a33b0 0x7f19e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.035+0000 7f19f37fe640 1 -- 172.21.15.5:0/2263587379 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f19f80f0090 con 0x7f19fc100a80 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.035+0000 7f19f0ff9640 1 -- 172.21.15.5:0/2764757288 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4004b20 con 0x55909785bb40 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.035+0000 7f19f0ff9640 1 -- 172.21.15.5:0/2764757288 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f4004cc0 con 0x55909785bb40 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.035+0000 7f19f0ff9640 1 -- 172.21.15.5:0/2764757288 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4005020 con 0x55909785bb40 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a06967740 1 -- 172.21.15.5:0/2764757288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 msgr2=0x55909785a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a0232a640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19e80a33b0 0x7f19e80a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f19f80ed490 tx=0x7f19f802a420 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a06967740 1 --2- 172.21.15.5:0/2764757288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x55909785a090 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009f50 tx=0x7f19f40118d0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a06967740 1 -- 172.21.15.5:0/2764757288 shutdown_connections 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a06967740 1 --2- 172.21.15.5:0/2764757288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x559097867760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a06967740 1 --2- 172.21.15.5:0/2764757288 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 0x55909785aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a06967740 1 --2- 172.21.15.5:0/2764757288 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x55909785a090 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a06967740 1 -- 172.21.15.5:0/2764757288 >> 172.21.15.5:0/2764757288 conn(0x559097823310 msgr2=0x559097856c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a06967740 1 -- 172.21.15.5:0/2764757288 shutdown_connections 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a06967740 1 -- 172.21.15.5:0/2764757288 wait complete. 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a06967740 1 -- start start 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.036+0000 7f1a06967740 1 Processor -- start 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a06967740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 0x559097864cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a06967740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097861f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a06967740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x5590978624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a06967740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559097868070 con 0x559097865370 2024-09-15T22:23:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a06967740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55909777b060 con 0x55909785a5d0 2024-09-15T22:23:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a06967740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5590978681f0 con 0x55909785bb40 2024-09-15T22:23:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a01b29640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097861f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a02b2b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 0x559097864cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a01b29640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097861f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53864/0 (socket says 172.21.15.5:53864) 2024-09-15T22:23:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a01b29640 1 -- 172.21.15.5:0/3939546395 learned_addr learned my addr 172.21.15.5:0/3939546395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a02b2b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 0x559097864cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:40886/0 (socket says 172.21.15.5:40886) 2024-09-15T22:23:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a06967740 10 client.0 _ll_register_callbacks cb 0x5590974ce8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x5590978624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a01b29640 1 -- 172.21.15.5:0/3939546395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 msgr2=0x559097864cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a01b29640 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 0x559097864cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a01b29640 1 -- 172.21.15.5:0/3939546395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 msgr2=0x5590978624a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a01b29640 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x5590978624a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a01b29640 1 -- 172.21.15.5:0/3939546395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590975313c0 con 0x55909785bb40 2024-09-15T22:23:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.037+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x5590978624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.038+0000 7f1a01b29640 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097861f60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b990 tx=0x7f19ec00be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d083326d6a20ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.038+0000 7f19e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:23:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.038+0000 7f19e5ffb640 1 -- 172.21.15.5:0/3939546395 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec005020 con 0x55909785bb40 2024-09-15T22:23:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.038+0000 7f19e5ffb640 1 -- 172.21.15.5:0/3939546395 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f19ec0051c0 con 0x55909785bb40 2024-09-15T22:23:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.038+0000 7f1a06967740 1 -- 172.21.15.5:0/3939546395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f4004cc0 con 0x55909785bb40 2024-09-15T22:23:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.038+0000 7f19e5ffb640 1 -- 172.21.15.5:0/3939546395 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f19ec024050 con 0x55909785bb40 2024-09-15T22:23:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.038+0000 7f1a06967740 10 client.27289 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.038+0000 7f1a06967740 1 -- 172.21.15.5:0/3939546395 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559097873b00 con 0x55909785bb40 2024-09-15T22:23:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.038+0000 7f19e5ffb640 1 -- 172.21.15.5:0/3939546395 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f4004cc0 con 0x55909785bb40 2024-09-15T22:23:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.039+0000 7f19e5ffb640 1 -- 172.21.15.5:0/3939546395 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f19ec01b470 con 0x55909785bb40 2024-09-15T22:23:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.039+0000 7f1a06967740 10 client.27289 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.039+0000 7f1a06967740 10 client.27289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.039+0000 7f1a06967740 20 client.27289 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.040+0000 7f1a06967740 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559097875010 0x559097877430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.040+0000 7f1a06967740 4 client.27289 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.040+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559097875010 0x559097877430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.040+0000 7f1a06967740 1 -- 172.21.15.5:0/3939546395 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590977cd860 con 0x559097875010 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.041+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559097875010 0x559097877430 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fab7c56e833dee1 server_cookie=2ede433e48a91ef1 in_seq=0 out_seq=0 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.041+0000 7f19e5ffb640 10 client.27289 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.041+0000 7f19e5ffb640 1 -- 172.21.15.5:0/3939546395 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590977cd860 con 0x559097875010 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.041+0000 7f19e5ffb640 10 client.27289 handle_command_reply: tid=0 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.041+0000 7f1a06967740 1 client.27289 shutdown 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f1a06967740 1 -- 172.21.15.5:0/3939546395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 msgr2=0x559097861f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f1a06967740 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097861f60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b990 tx=0x7f19ec00be50 comp rx=0 tx=0).stop 2024-09-15T22:23:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f1a06967740 1 -- 172.21.15.5:0/3939546395 shutdown_connections 2024-09-15T22:23:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f1a06967740 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x5590978624a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f1a06967740 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097861f60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f1a06967740 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559097875010 0x559097877430 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f1a06967740 1 --2- 172.21.15.5:0/3939546395 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785a5d0 0x559097864cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f1a06967740 1 -- 172.21.15.5:0/3939546395 >> 172.21.15.5:0/3939546395 conn(0x559097823310 msgr2=0x55909785c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f19e5ffb640 0 client.27289 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f1a06967740 1 -- 172.21.15.5:0/3939546395 shutdown_connections 2024-09-15T22:23:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f1a06967740 1 -- 172.21.15.5:0/3939546395 wait complete. 2024-09-15T22:23:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.042+0000 7f1a06967740 20 client.27289 trim_cache size 0 max 16384 2024-09-15T22:23:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.058+0000 7f1a06967740 1 Processor -- start 2024-09-15T22:23:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.058+0000 7f1a06967740 1 -- start start 2024-09-15T22:23:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.058+0000 7f1a06967740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55909785a5d0 0x559097871370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.058+0000 7f1a06967740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097855bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.058+0000 7f1a06967740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559097865370 0x55909787cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.058+0000 7f1a06967740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19e00128d0 con 0x55909785a5d0 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.058+0000 7f1a06967740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19e00101a0 con 0x559097865370 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.058+0000 7f1a06967740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19e000e010 con 0x55909785bb40 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a01b29640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097855bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a02b2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55909785a5d0 0x559097871370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a01b29640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097855bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53872/0 (socket says 172.21.15.5:53872) 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a01b29640 1 -- 172.21.15.5:0/2985327470 learned_addr learned my addr 172.21.15.5:0/2985327470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/2985327470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559097865370 0x55909787cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a02b2b640 1 -- 172.21.15.5:0/2985327470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 msgr2=0x559097855bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/2985327470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097855bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a02b2b640 1 -- 172.21.15.5:0/2985327470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559097865370 msgr2=0x55909787cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/2985327470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559097865370 0x55909787cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a02b2b640 1 -- 172.21.15.5:0/2985327470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559097862fa0 con 0x55909785a5d0 2024-09-15T22:23:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a01b29640 1 --2- 172.21.15.5:0/2985327470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097855bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:23:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/2985327470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55909785a5d0 0x559097871370 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f19f4012fd0 tx=0x7f19f4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d30b14cdae51b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.059+0000 7f19e7fff640 1 -- 172.21.15.5:0/2985327470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4004080 con 0x55909785a5d0 2024-09-15T22:23:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.060+0000 7f19e7fff640 1 -- 172.21.15.5:0/2985327470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f40046c0 con 0x55909785a5d0 2024-09-15T22:23:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.060+0000 7f19e7fff640 1 -- 172.21.15.5:0/2985327470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4027890 con 0x55909785a5d0 2024-09-15T22:23:50.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.060+0000 7f1a06967740 1 -- 172.21.15.5:0/2985327470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55909785a5d0 msgr2=0x559097871370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.060+0000 7f1a06967740 1 --2- 172.21.15.5:0/2985327470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55909785a5d0 0x559097871370 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f19f4012fd0 tx=0x7f19f4009500 comp rx=0 tx=0).stop 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.060+0000 7f1a06967740 1 -- 172.21.15.5:0/2985327470 shutdown_connections 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.060+0000 7f1a06967740 1 --2- 172.21.15.5:0/2985327470 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559097865370 0x55909787cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.060+0000 7f1a06967740 1 --2- 172.21.15.5:0/2985327470 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785bb40 0x559097855bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.060+0000 7f1a06967740 1 --2- 172.21.15.5:0/2985327470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55909785a5d0 0x559097871370 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.060+0000 7f1a06967740 1 -- 172.21.15.5:0/2985327470 >> 172.21.15.5:0/2985327470 conn(0x559097823310 msgr2=0x5590978554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.060+0000 7f1a06967740 1 -- 172.21.15.5:0/2985327470 shutdown_connections 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.060+0000 7f1a06967740 1 -- 172.21.15.5:0/2985327470 wait complete. 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a06967740 1 -- start start 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a06967740 1 Processor -- start 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a06967740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785a5d0 0x559097869640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a06967740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785bb40 0x559097869b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a06967740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x5590978e74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a06967740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19e000e660 con 0x559097865370 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a06967740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f19e000fad0 con 0x55909785bb40 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a06967740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f19e000ed30 con 0x55909785a5d0 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a06967740 10 client.0 _ll_register_callbacks cb 0x7f19ec0042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a01b29640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785bb40 0x559097869b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a02b2b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785a5d0 0x559097869640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a01b29640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785bb40 0x559097869b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:40908/0 (socket says 172.21.15.5:40908) 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a01b29640 1 -- 172.21.15.5:0/2161707754 learned_addr learned my addr 172.21.15.5:0/2161707754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.061+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x5590978e74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f1a01b29640 1 -- 172.21.15.5:0/2161707754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785a5d0 msgr2=0x559097869640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f1a01b29640 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785a5d0 0x559097869640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f1a01b29640 1 -- 172.21.15.5:0/2161707754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 msgr2=0x5590978e74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f1a01b29640 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x5590978e74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f1a01b29640 1 -- 172.21.15.5:0/2161707754 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590978e7a20 con 0x55909785bb40 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f1a01b29640 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785bb40 0x559097869b80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b960 tx=0x7f19ec01b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7805d15b2bd21a4f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f19e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f19e5ffb640 1 -- 172.21.15.5:0/2161707754 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec01bcc0 con 0x55909785bb40 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f1a06967740 1 -- 172.21.15.5:0/2161707754 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f40046c0 con 0x55909785bb40 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f19e5ffb640 1 -- 172.21.15.5:0/2161707754 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f19ec017040 con 0x55909785bb40 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f19e5ffb640 1 -- 172.21.15.5:0/2161707754 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f19ec039050 con 0x55909785bb40 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f1a06967740 10 client.27368 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.062+0000 7f1a06967740 1 -- 172.21.15.5:0/2161707754 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590978e8bc0 con 0x55909785bb40 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.063+0000 7f19e5ffb640 1 -- 172.21.15.5:0/2161707754 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f40046c0 con 0x55909785bb40 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.063+0000 7f19e5ffb640 1 -- 172.21.15.5:0/2161707754 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f19ec0117b0 con 0x55909785bb40 2024-09-15T22:23:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.063+0000 7f1a06967740 10 client.27368 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.063+0000 7f1a06967740 10 client.27368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.063+0000 7f1a06967740 20 client.27368 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.063+0000 7f1a06967740 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5590978e9310 0x5590978eb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.063+0000 7f1a06967740 4 client.27368 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.063+0000 7f1a06967740 1 -- 172.21.15.5:0/2161707754 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590977c7d70 con 0x5590978e9310 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.063+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5590978e9310 0x5590978eb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.064+0000 7f1a02b2b640 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5590978e9310 0x5590978eb730 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f066f9c92f99d7d4 server_cookie=d5641e4aea748e0e in_seq=0 out_seq=0 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.064+0000 7f19e5ffb640 10 client.27368 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.064+0000 7f19e5ffb640 1 -- 172.21.15.5:0/2161707754 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590977c7d70 con 0x5590978e9310 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.064+0000 7f19e5ffb640 10 client.27368 handle_command_reply: tid=0 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.064+0000 7f1a06967740 1 client.27368 shutdown 2024-09-15T22:23:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.064+0000 7f1a06967740 1 -- 172.21.15.5:0/2161707754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785bb40 msgr2=0x559097869b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.064+0000 7f1a06967740 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785bb40 0x559097869b80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b960 tx=0x7f19ec01b720 comp rx=0 tx=0).stop 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.065+0000 7f1a06967740 1 -- 172.21.15.5:0/2161707754 shutdown_connections 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.065+0000 7f1a06967740 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559097865370 0x5590978e74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.065+0000 7f1a06967740 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5590978e9310 0x5590978eb730 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.065+0000 7f1a06967740 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55909785bb40 0x559097869b80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.065+0000 7f1a06967740 1 --2- 172.21.15.5:0/2161707754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55909785a5d0 0x559097869640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.065+0000 7f1a06967740 1 -- 172.21.15.5:0/2161707754 >> 172.21.15.5:0/2161707754 conn(0x559097823310 msgr2=0x559097863fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.065+0000 7f19e5ffb640 0 client.27368 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.065+0000 7f1a06967740 1 -- 172.21.15.5:0/2161707754 shutdown_connections 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.065+0000 7f1a06967740 1 -- 172.21.15.5:0/2161707754 wait complete. 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.065+0000 7f1a06967740 20 client.27368 trim_cache size 0 max 16384 2024-09-15T22:23:50.073 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:23:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 -- 172.21.15.5:0/2263587379 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19e80a33b0 msgr2=0x7f19e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19e80a33b0 0x7f19e80a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f19f80ed490 tx=0x7f19f802a420 comp rx=0 tx=0).stop 2024-09-15T22:23:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 -- 172.21.15.5:0/2263587379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19fc100a80 msgr2=0x7f19fc198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19fc100a80 0x7f19fc198480 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009230 tx=0x7f19f8004270 comp rx=0 tx=0).stop 2024-09-15T22:23:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 -- 172.21.15.5:0/2263587379 shutdown_connections 2024-09-15T22:23:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f19e80a33b0 0x7f19e80a5870 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f19fc106a80 0x7f19fc19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19fc101420 0x7f19fc1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 --2- 172.21.15.5:0/2263587379 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f19fc100a80 0x7f19fc198480 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 -- 172.21.15.5:0/2263587379 >> 172.21.15.5:0/2263587379 conn(0x7f19fc0fc7a0 msgr2=0x7f19fc1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 -- 172.21.15.5:0/2263587379 shutdown_connections 2024-09-15T22:23:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:50.066+0000 7f1a0458e640 1 -- 172.21.15.5:0/2263587379 wait complete. 2024-09-15T22:23:51.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:51 smithi099 ceph-mon[34892]: pgmap v467: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-09-15T22:23:51.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:51 smithi129 ceph-mon[35314]: pgmap v467: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-09-15T22:23:51.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:51 smithi005 ceph-mon[30944]: pgmap v467: 129 pgs: 129 active+clean; 2.8 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 4.4 MiB/s wr, 483 op/s 2024-09-15T22:23:51.739 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:23:51.739 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:23:52.289 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:53.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.105+0000 7fe598e8f640 1 -- 172.21.15.5:0/4226972993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe594100980 msgr2=0x7fe594100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:53.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.105+0000 7fe598e8f640 1 --2- 172.21.15.5:0/4226972993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe594100980 0x7fe594100d60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009ca0 tx=0x7fe58c02a980 comp rx=0 tx=0).stop 2024-09-15T22:23:53.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.105+0000 7fe598e8f640 1 -- 172.21.15.5:0/4226972993 shutdown_connections 2024-09-15T22:23:53.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.105+0000 7fe598e8f640 1 --2- 172.21.15.5:0/4226972993 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe594105d90 0x7fe594106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.105+0000 7fe598e8f640 1 --2- 172.21.15.5:0/4226972993 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe5941012a0 0x7fe594105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.105+0000 7fe598e8f640 1 --2- 172.21.15.5:0/4226972993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe594100980 0x7fe594100d60 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.105+0000 7fe598e8f640 1 -- 172.21.15.5:0/4226972993 >> 172.21.15.5:0/4226972993 conn(0x7fe5940fc610 msgr2=0x7fe5940fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:53.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.106+0000 7fe598e8f640 1 -- 172.21.15.5:0/4226972993 shutdown_connections 2024-09-15T22:23:53.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.106+0000 7fe598e8f640 1 -- 172.21.15.5:0/4226972993 wait complete. 2024-09-15T22:23:53.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.108+0000 7fe598e8f640 1 Processor -- start 2024-09-15T22:23:53.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.108+0000 7fe598e8f640 1 -- start start 2024-09-15T22:23:53.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.108+0000 7fe598e8f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe594100980 0x7fe5941a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe598e8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5941012a0 0x7fe5941a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe598e8f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe594105d90 0x7fe59419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe598e8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5941184d0 con 0x7fe5941012a0 2024-09-15T22:23:53.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe5937fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe594100980 0x7fe5941a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe598e8f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe594118350 con 0x7fe594105d90 2024-09-15T22:23:53.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe598e8f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe594118650 con 0x7fe594100980 2024-09-15T22:23:53.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe593fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe594105d90 0x7fe59419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe593fff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe594105d90 0x7fe59419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:40938/0 (socket says 172.21.15.5:40938) 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe593fff640 1 -- 172.21.15.5:0/2187540584 learned_addr learned my addr 172.21.15.5:0/2187540584 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe593fff640 1 -- 172.21.15.5:0/2187540584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe594100980 msgr2=0x7fe5941a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe593fff640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe594100980 0x7fe5941a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe593fff640 1 -- 172.21.15.5:0/2187540584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5941012a0 msgr2=0x7fe5941a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.109+0000 7fe593fff640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5941012a0 0x7fe5941a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.110+0000 7fe593fff640 1 -- 172.21.15.5:0/2187540584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59419f050 con 0x7fe594105d90 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.110+0000 7fe593fff640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe594105d90 0x7fe59419e820 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe58400b3f0 tx=0x7fe58400b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.110+0000 7fe590ff9640 1 -- 172.21.15.5:0/2187540584 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584015020 con 0x7fe594105d90 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.110+0000 7fe590ff9640 1 -- 172.21.15.5:0/2187540584 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe584011d10 con 0x7fe594105d90 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.110+0000 7fe590ff9640 1 -- 172.21.15.5:0/2187540584 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58400f2e0 con 0x7fe594105d90 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.110+0000 7fe598e8f640 1 -- 172.21.15.5:0/2187540584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59419f340 con 0x7fe594105d90 2024-09-15T22:23:53.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.111+0000 7fe598e8f640 1 -- 172.21.15.5:0/2187540584 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5941ab4e0 con 0x7fe594105d90 2024-09-15T22:23:53.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.114+0000 7fe590ff9640 1 -- 172.21.15.5:0/2187540584 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fe584018020 con 0x7fe594105d90 2024-09-15T22:23:53.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.115+0000 7fe59c4ca740 1 Processor -- start 2024-09-15T22:23:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.115+0000 7fe59c4ca740 1 -- start start 2024-09-15T22:23:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.116+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c557150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.116+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 0x56504c561a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.116+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c561f50 0x56504c564340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.116+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56504c4bc280 con 0x56504c561f50 2024-09-15T22:23:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.116+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56504c463b10 con 0x56504c556d70 2024-09-15T22:23:53.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.116+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56504c4b8240 con 0x56504c557690 2024-09-15T22:23:53.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.117+0000 7fe5937fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 0x56504c561a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.117+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c557150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.117+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c557150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:40942/0 (socket says 172.21.15.5:40942) 2024-09-15T22:23:53.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.117+0000 7fe592ffd640 1 -- 172.21.15.5:0/3938797160 learned_addr learned my addr 172.21.15.5:0/3938797160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:53.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.117+0000 7fe592ffd640 1 --2- 172.21.15.5:0/3938797160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c561f50 0x56504c564340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.117+0000 7fe590ff9640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe5800a3430 0x7fe5800a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.118+0000 7fe592ffd640 1 -- 172.21.15.5:0/3938797160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 msgr2=0x56504c561a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:53.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.118+0000 7fe590ff9640 1 -- 172.21.15.5:0/2187540584 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7fe5840d09b0 con 0x7fe594105d90 2024-09-15T22:23:53.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.118+0000 7fe592ffd640 1 --2- 172.21.15.5:0/3938797160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 0x56504c561a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.118+0000 7fe592ffd640 1 -- 172.21.15.5:0/3938797160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c561f50 msgr2=0x56504c564340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:53.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.118+0000 7fe592ffd640 1 --2- 172.21.15.5:0/3938797160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c561f50 0x56504c564340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.118+0000 7fe593fff640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe5800a3430 0x7fe5800a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.118+0000 7fe592ffd640 1 -- 172.21.15.5:0/3938797160 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504c4701f0 con 0x56504c556d70 2024-09-15T22:23:53.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.119+0000 7fe5937fe640 1 --2- 172.21.15.5:0/3938797160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 0x56504c561a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:23:53.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.119+0000 7fe593fff640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe5800a3430 0x7fe5800a58f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5840cde10 tx=0x7fe584006e10 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:53.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.119+0000 7fe592ffd640 1 --2- 172.21.15.5:0/3938797160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c557150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009fd0 tx=0x7fe57c009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f915b9b19e519517 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:53.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.119+0000 7fe589ffb640 1 -- 172.21.15.5:0/3938797160 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c012040 con 0x56504c556d70 2024-09-15T22:23:53.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.119+0000 7fe589ffb640 1 -- 172.21.15.5:0/3938797160 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57c004020 con 0x56504c556d70 2024-09-15T22:23:53.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.119+0000 7fe589ffb640 1 -- 172.21.15.5:0/3938797160 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c007550 con 0x56504c556d70 2024-09-15T22:23:53.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.120+0000 7fe59c4ca740 1 -- 172.21.15.5:0/3938797160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 msgr2=0x56504c557150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:53.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.120+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/3938797160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c557150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009fd0 tx=0x7fe57c009e20 comp rx=0 tx=0).stop 2024-09-15T22:23:53.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.120+0000 7fe59c4ca740 1 -- 172.21.15.5:0/3938797160 shutdown_connections 2024-09-15T22:23:53.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.120+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/3938797160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c561f50 0x56504c564340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.120+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/3938797160 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 0x56504c561a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.120+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/3938797160 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c557150 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.120+0000 7fe59c4ca740 1 -- 172.21.15.5:0/3938797160 >> 172.21.15.5:0/3938797160 conn(0x56504c553300 msgr2=0x56504c5536e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:53.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.121+0000 7fe59c4ca740 1 -- 172.21.15.5:0/3938797160 shutdown_connections 2024-09-15T22:23:53.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.121+0000 7fe59c4ca740 1 -- 172.21.15.5:0/3938797160 wait complete. 2024-09-15T22:23:53.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.121+0000 7fe59c4ca740 1 -- start start 2024-09-15T22:23:53.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.121+0000 7fe59c4ca740 1 Processor -- start 2024-09-15T22:23:53.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 0x56504c55ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c557690 0x56504c55f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c561f50 0x56504c55f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56504c575b30 con 0x56504c556d70 2024-09-15T22:23:53.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x56504c476f10 con 0x56504c557690 2024-09-15T22:23:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x56504c575cb0 con 0x56504c561f50 2024-09-15T22:23:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe59c4ca740 10 client.0 _ll_register_callbacks cb 0x56504c43fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c561f50 0x56504c55f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c561f50 0x56504c55f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:53916/0 (socket says 172.21.15.5:53916) 2024-09-15T22:23:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe592ffd640 1 -- 172.21.15.5:0/4146205130 learned_addr learned my addr 172.21.15.5:0/4146205130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe592ffd640 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 0x56504c55ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe5937fe640 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c557690 0x56504c55f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe592ffd640 1 -- 172.21.15.5:0/4146205130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c557690 msgr2=0x56504c55f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe592ffd640 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c557690 0x56504c55f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe592ffd640 1 -- 172.21.15.5:0/4146205130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 msgr2=0x56504c55ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe592ffd640 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 0x56504c55ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.122+0000 7fe592ffd640 1 -- 172.21.15.5:0/4146205130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504c22d6a0 con 0x56504c561f50 2024-09-15T22:23:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.123+0000 7fe592ffd640 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c561f50 0x56504c55f910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe57c007230 tx=0x7fe57c007130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=452a9f83cdf3b40e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.123+0000 7fe572ffd640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:23:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.127+0000 7fe572ffd640 1 -- 172.21.15.5:0/4146205130 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c021040 con 0x56504c561f50 2024-09-15T22:23:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.127+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4146205130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57c004020 con 0x56504c561f50 2024-09-15T22:23:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.127+0000 7fe572ffd640 1 -- 172.21.15.5:0/4146205130 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7fe57c0089a0 con 0x56504c561f50 2024-09-15T22:23:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.128+0000 7fe572ffd640 1 -- 172.21.15.5:0/4146205130 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe57c008ba0 con 0x56504c561f50 2024-09-15T22:23:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.128+0000 7fe59c4ca740 10 client.27298 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.128+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4146205130 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56504c581730 con 0x56504c561f50 2024-09-15T22:23:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.128+0000 7fe572ffd640 1 -- 172.21.15.5:0/4146205130 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57c008d40 con 0x56504c561f50 2024-09-15T22:23:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.129+0000 7fe572ffd640 1 -- 172.21.15.5:0/4146205130 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x56504c561f50 2024-09-15T22:23:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.129+0000 7fe59c4ca740 10 client.27298 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.129+0000 7fe59c4ca740 10 client.27298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.130+0000 7fe59c4ca740 20 client.27298 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.130+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56504c582a80 0x56504c584e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.130+0000 7fe59c4ca740 4 client.27298 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.130+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4146205130 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe57c004020 con 0x56504c582a80 2024-09-15T22:23:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.130+0000 7fe5937fe640 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56504c582a80 0x56504c584e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.130+0000 7fe5937fe640 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56504c582a80 0x56504c584e70 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69f8dd08fb0d348c server_cookie=32a8bdd77decde0e in_seq=0 out_seq=0 2024-09-15T22:23:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.130+0000 7fe572ffd640 10 client.27298 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe572ffd640 1 -- 172.21.15.5:0/4146205130 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe57c004020 con 0x56504c582a80 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe572ffd640 10 client.27298 handle_command_reply: tid=0 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe59c4ca740 1 client.27298 shutdown 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4146205130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c561f50 msgr2=0x56504c55f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c561f50 0x56504c55f910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe57c007230 tx=0x7fe57c007130 comp rx=0 tx=0).stop 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4146205130 shutdown_connections 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c561f50 0x56504c55f910 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56504c582a80 0x56504c584e70 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c557690 0x56504c55f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4146205130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 0x56504c55ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe572ffd640 0 client.27298 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.131+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4146205130 >> 172.21.15.5:0/4146205130 conn(0x56504c553300 msgr2=0x56504c558fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.132+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4146205130 shutdown_connections 2024-09-15T22:23:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.132+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4146205130 wait complete. 2024-09-15T22:23:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.132+0000 7fe59c4ca740 20 client.27298 trim_cache size 0 max 16384 2024-09-15T22:23:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.153+0000 7fe59c4ca740 1 Processor -- start 2024-09-15T22:23:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.153+0000 7fe59c4ca740 1 -- start start 2024-09-15T22:23:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.153+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 0x56504c560cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.153+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c557690 0x56504c55f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.153+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c561f50 0x56504c55f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.153+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe5941af000 con 0x56504c556d70 2024-09-15T22:23:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.153+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5941af6d0 con 0x56504c557690 2024-09-15T22:23:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.153+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5941acfa0 con 0x56504c561f50 2024-09-15T22:23:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.154+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 0x56504c560cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.154+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 0x56504c560cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40934/0 (socket says 172.21.15.5:40934) 2024-09-15T22:23:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.154+0000 7fe592ffd640 1 -- 172.21.15.5:0/156950478 learned_addr learned my addr 172.21.15.5:0/156950478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.154+0000 7fe5937fe640 1 --2- 172.21.15.5:0/156950478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c557690 0x56504c55f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.154+0000 7fe592ffd640 1 -- 172.21.15.5:0/156950478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c561f50 msgr2=0x56504c55f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.154+0000 7fe592ffd640 1 --2- 172.21.15.5:0/156950478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c561f50 0x56504c55f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.154+0000 7fe592ffd640 1 -- 172.21.15.5:0/156950478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c557690 msgr2=0x56504c55f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.154+0000 7fe592ffd640 1 --2- 172.21.15.5:0/156950478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c557690 0x56504c55f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.154+0000 7fe592ffd640 1 -- 172.21.15.5:0/156950478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504c55de80 con 0x56504c556d70 2024-09-15T22:23:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.154+0000 7fe592ffd640 1 --2- 172.21.15.5:0/156950478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 0x56504c560cf0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe57c003c00 tx=0x7fe57c015e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a45dec9d93420fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.154+0000 7fe5897fa640 1 -- 172.21.15.5:0/156950478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c004470 con 0x56504c556d70 2024-09-15T22:23:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.155+0000 7fe5897fa640 1 -- 172.21.15.5:0/156950478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57c001b80 con 0x56504c556d70 2024-09-15T22:23:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.155+0000 7fe5897fa640 1 -- 172.21.15.5:0/156950478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0040b0 con 0x56504c556d70 2024-09-15T22:23:53.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.156+0000 7fe59c4ca740 1 -- 172.21.15.5:0/156950478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 msgr2=0x56504c560cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:53.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.156+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/156950478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 0x56504c560cf0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe57c003c00 tx=0x7fe57c015e00 comp rx=0 tx=0).stop 2024-09-15T22:23:53.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.156+0000 7fe59c4ca740 1 -- 172.21.15.5:0/156950478 shutdown_connections 2024-09-15T22:23:53.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.156+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/156950478 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c561f50 0x56504c55f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.156+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/156950478 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c557690 0x56504c55f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.156+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/156950478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c556d70 0x56504c560cf0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.156+0000 7fe59c4ca740 1 -- 172.21.15.5:0/156950478 >> 172.21.15.5:0/156950478 conn(0x56504c553300 msgr2=0x56504c5605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:53.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.156+0000 7fe59c4ca740 1 -- 172.21.15.5:0/156950478 shutdown_connections 2024-09-15T22:23:53.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.156+0000 7fe59c4ca740 1 -- 172.21.15.5:0/156950478 wait complete. 2024-09-15T22:23:53.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.156+0000 7fe59c4ca740 1 -- start start 2024-09-15T22:23:53.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe59c4ca740 1 Processor -- start 2024-09-15T22:23:53.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c57b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 0x56504c577f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe59c4ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c561f50 0x56504c578440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe594074240 con 0x56504c561f50 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fe5941ac8d0 con 0x56504c556d70 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe59c4ca740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fe5941abb30 con 0x56504c557690 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c57b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe592ffd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c57b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:40974/0 (socket says 172.21.15.5:40974) 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe592ffd640 1 -- 172.21.15.5:0/4266285141 learned_addr learned my addr 172.21.15.5:0/4266285141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe59c4ca740 10 client.0 _ll_register_callbacks cb 0x7fe57c0019d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe5937fe640 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 0x56504c577f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.157+0000 7fe592ffd640 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c561f50 0x56504c578440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe592ffd640 1 -- 172.21.15.5:0/4266285141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 msgr2=0x56504c577f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe592ffd640 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 0x56504c577f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe592ffd640 1 -- 172.21.15.5:0/4266285141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c561f50 msgr2=0x56504c578440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe592ffd640 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c561f50 0x56504c578440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe592ffd640 1 -- 172.21.15.5:0/4266285141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504c578980 con 0x56504c556d70 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe5937fe640 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 0x56504c577f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe592ffd640 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c57b330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00b040 tx=0x7fe57c0038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a22a02765f73070 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe572ffd640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe572ffd640 1 -- 172.21.15.5:0/4266285141 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c003690 con 0x56504c556d70 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4266285141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57c001b80 con 0x56504c556d70 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe572ffd640 1 -- 172.21.15.5:0/4266285141 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7fe57c003e50 con 0x56504c556d70 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe572ffd640 1 -- 172.21.15.5:0/4266285141 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe57c047050 con 0x56504c556d70 2024-09-15T22:23:53.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe59c4ca740 10 client.27383 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:53.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.158+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4266285141 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56504c5ee240 con 0x56504c556d70 2024-09-15T22:23:53.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.159+0000 7fe572ffd640 1 -- 172.21.15.5:0/4266285141 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57c001b80 con 0x56504c556d70 2024-09-15T22:23:53.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.159+0000 7fe572ffd640 1 -- 172.21.15.5:0/4266285141 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fe57c03f020 con 0x56504c556d70 2024-09-15T22:23:53.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.159+0000 7fe59c4ca740 10 client.27383 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:53.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.159+0000 7fe59c4ca740 10 client.27383 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:53.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.159+0000 7fe59c4ca740 20 client.27383 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:53.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.159+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56504c5ee8a0 0x56504c5f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:53.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.159+0000 7fe59c4ca740 4 client.27383 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:23:53.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.159+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4266285141 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56504c4c97f0 con 0x56504c5ee8a0 2024-09-15T22:23:53.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.159+0000 7fe5937fe640 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56504c5ee8a0 0x56504c5f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:53.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.159+0000 7fe572ffd640 10 client.27383 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:53.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.159+0000 7fe5937fe640 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56504c5ee8a0 0x56504c5f0c90 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e219920a712e992 server_cookie=dca122883b4347c2 in_seq=0 out_seq=0 2024-09-15T22:23:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.160+0000 7fe572ffd640 1 -- 172.21.15.5:0/4266285141 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56504c4c97f0 con 0x56504c5ee8a0 2024-09-15T22:23:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.160+0000 7fe572ffd640 10 client.27383 handle_command_reply: tid=0 2024-09-15T22:23:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.161+0000 7fe59c4ca740 1 client.27383 shutdown 2024-09-15T22:23:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.161+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4266285141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 msgr2=0x56504c57b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.161+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c57b330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00b040 tx=0x7fe57c0038a0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.161+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4266285141 shutdown_connections 2024-09-15T22:23:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.161+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56504c561f50 0x56504c578440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.161+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x56504c557690 0x56504c577f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.161+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56504c5ee8a0 0x56504c5f0c90 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.161+0000 7fe59c4ca740 1 --2- 172.21.15.5:0/4266285141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x56504c556d70 0x56504c57b330 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.161+0000 7fe572ffd640 0 client.27383 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.161+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4266285141 >> 172.21.15.5:0/4266285141 conn(0x56504c553300 msgr2=0x56504c552410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.162+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4266285141 shutdown_connections 2024-09-15T22:23:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.162+0000 7fe59c4ca740 1 -- 172.21.15.5:0/4266285141 wait complete. 2024-09-15T22:23:53.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.162+0000 7fe59c4ca740 20 client.27383 trim_cache size 0 max 16384 2024-09-15T22:23:53.173 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:23:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:23:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d4303284-aedf-4ffc-a8d9-80ec1a3ef710", 2024-09-15T22:23:53.173 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:23:53.173 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.162+0000 7fe598e8f640 1 -- 172.21.15.5:0/2187540584 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe5800a3430 msgr2=0x7fe5800a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.162+0000 7fe598e8f640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe5800a3430 0x7fe5800a58f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5840cde10 tx=0x7fe584006e10 comp rx=0 tx=0).stop 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.163+0000 7fe598e8f640 1 -- 172.21.15.5:0/2187540584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe594105d90 msgr2=0x7fe59419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.163+0000 7fe598e8f640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe594105d90 0x7fe59419e820 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe58400b3f0 tx=0x7fe58400b8b0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.163+0000 7fe598e8f640 1 -- 172.21.15.5:0/2187540584 shutdown_connections 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.163+0000 7fe598e8f640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fe5800a3430 0x7fe5800a58f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.163+0000 7fe598e8f640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fe594105d90 0x7fe59419e820 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.163+0000 7fe598e8f640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5941012a0 0x7fe5941a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.163+0000 7fe598e8f640 1 --2- 172.21.15.5:0/2187540584 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fe594100980 0x7fe5941a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.163+0000 7fe598e8f640 1 -- 172.21.15.5:0/2187540584 >> 172.21.15.5:0/2187540584 conn(0x7fe5940fc610 msgr2=0x7fe59410f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.163+0000 7fe598e8f640 1 -- 172.21.15.5:0/2187540584 shutdown_connections 2024-09-15T22:23:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:53.163+0000 7fe598e8f640 1 -- 172.21.15.5:0/2187540584 wait complete. 2024-09-15T22:23:53.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:53 smithi005 ceph-mon[30944]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.8 MiB/s wr, 608 op/s 2024-09-15T22:23:53.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:53 smithi099 ceph-mon[34892]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.8 MiB/s wr, 608 op/s 2024-09-15T22:23:53.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:53 smithi129 ceph-mon[35314]: pgmap v468: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 5.8 MiB/s wr, 608 op/s 2024-09-15T22:23:53.899 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:23:54.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:54 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:23:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:23:54.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:54 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:54.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:54 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:23:54.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:54 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:23:54.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:54 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:54.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:54 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:23:54.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:54 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:23:54.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:54 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:54.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:54 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:23:54.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 865.0 s, 508.6 tps, lat 187.804 ms stddev 220.737 2024-09-15T22:23:55.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:55 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:55.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:55 smithi005 ceph-mon[30944]: pgmap v469: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-09-15T22:23:55.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:55 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:55.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:55 smithi099 ceph-mon[34892]: pgmap v469: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-09-15T22:23:55.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:55 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:23:55.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:55 smithi129 ceph-mon[35314]: pgmap v469: 129 pgs: 129 active+clean; 2.9 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.5 MiB/s wr, 487 op/s 2024-09-15T22:23:55.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.081+0000 7f8361102640 1 -- 172.21.15.5:0/1326552220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83540952d0 msgr2=0x7f83540a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.082+0000 7f8361102640 1 --2- 172.21.15.5:0/1326552220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83540952d0 0x7f83540a1320 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8350009900 tx=0x7f835002a410 comp rx=0 tx=0).stop 2024-09-15T22:23:55.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.082+0000 7f8361102640 1 -- 172.21.15.5:0/1326552220 shutdown_connections 2024-09-15T22:23:55.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.082+0000 7f8361102640 1 --2- 172.21.15.5:0/1326552220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83540952d0 0x7f83540a1320 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.082+0000 7f8361102640 1 --2- 172.21.15.5:0/1326552220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8354094930 0x7f8354094d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.082+0000 7f8361102640 1 --2- 172.21.15.5:0/1326552220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f835409a550 0x7f835409a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.082+0000 7f8361102640 1 -- 172.21.15.5:0/1326552220 >> 172.21.15.5:0/1326552220 conn(0x7f835400a1f0 msgr2=0x7f835400a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.082+0000 7f8361102640 1 -- 172.21.15.5:0/1326552220 shutdown_connections 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.082+0000 7f8361102640 1 -- 172.21.15.5:0/1326552220 wait complete. 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.083+0000 7f8361102640 1 Processor -- start 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.083+0000 7f8361102640 1 -- start start 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.084+0000 7f8361102640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8354094930 0x7f8354131a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.084+0000 7f8361102640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83540952d0 0x7f8354131f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.084+0000 7f8361102640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f835409a550 0x7f835412bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.084+0000 7f8361102640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83540a4880 con 0x7f83540952d0 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.084+0000 7f8361102640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f83540a4700 con 0x7f835409a550 2024-09-15T22:23:55.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.084+0000 7f8361102640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f83540a4a00 con 0x7f8354094930 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.084+0000 7f835b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83540952d0 0x7f8354131f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.084+0000 7f835bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8354094930 0x7f8354131a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.084+0000 7f835bfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8354094930 0x7f8354131a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43952/0 (socket says 172.21.15.5:43952) 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.084+0000 7f835bfff640 1 -- 172.21.15.5:0/2293522573 learned_addr learned my addr 172.21.15.5:0/2293522573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.085+0000 7f8360901640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f835409a550 0x7f835412bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.085+0000 7f835bfff640 1 -- 172.21.15.5:0/2293522573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f835409a550 msgr2=0x7f835412bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.085+0000 7f835bfff640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f835409a550 0x7f835412bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.085+0000 7f835bfff640 1 -- 172.21.15.5:0/2293522573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83540952d0 msgr2=0x7f8354131f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.085+0000 7f835bfff640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83540952d0 0x7f8354131f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.085+0000 7f835bfff640 1 -- 172.21.15.5:0/2293522573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835412c390 con 0x7f8354094930 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.085+0000 7f835bfff640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8354094930 0x7f8354131a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f834800d950 tx=0x7f834800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.085+0000 7f83597fa640 1 -- 172.21.15.5:0/2293522573 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348017030 con 0x7f8354094930 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.085+0000 7f83597fa640 1 -- 172.21.15.5:0/2293522573 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8348004020 con 0x7f8354094930 2024-09-15T22:23:55.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.086+0000 7f8361102640 1 -- 172.21.15.5:0/2293522573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835412c620 con 0x7f8354094930 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.086+0000 7f8361102640 1 -- 172.21.15.5:0/2293522573 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8354096040 con 0x7f8354094930 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.086+0000 7f83597fa640 1 -- 172.21.15.5:0/2293522573 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348017030 con 0x7f8354094930 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.090+0000 7f836473d740 1 Processor -- start 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.090+0000 7f836473d740 1 -- start start 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.090+0000 7f836473d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 0x559667adf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.090+0000 7f836473d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667ada630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.090+0000 7f836473d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667af2ad0 0x559667af4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.090+0000 7f836473d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559667a3c2e0 con 0x559667af2ad0 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.090+0000 7f836473d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5596679e3ce0 con 0x559667adf660 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.090+0000 7f836473d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559667a382a0 con 0x559667aded40 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.091+0000 7f8360901640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 0x559667adf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.091+0000 7f8360901640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 0x559667adf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:43954/0 (socket says 172.21.15.5:43954) 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.091+0000 7f8360901640 1 -- 172.21.15.5:0/2456361093 learned_addr learned my addr 172.21.15.5:0/2456361093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:55.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.092+0000 7f8360901640 1 -- 172.21.15.5:0/2456361093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 msgr2=0x559667ada630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T22:23:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.092+0000 7f8360901640 1 --2- 172.21.15.5:0/2456361093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667ada630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.092+0000 7f8360901640 1 -- 172.21.15.5:0/2456361093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667af2ad0 msgr2=0x559667af4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.092+0000 7f8360901640 1 --2- 172.21.15.5:0/2456361093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667af2ad0 0x559667af4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.092+0000 7f8360901640 1 -- 172.21.15.5:0/2456361093 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596679c8440 con 0x559667aded40 2024-09-15T22:23:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.092+0000 7f83597fa640 1 -- 172.21.15.5:0/2293522573 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f83480054d0 con 0x7f8354094930 2024-09-15T22:23:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.093+0000 7f83597fa640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f834c0a3640 0x7f834c0a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.094+0000 7f83597fa640 1 -- 172.21.15.5:0/2293522573 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f83480cf830 con 0x7f8354094930 2024-09-15T22:23:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.094+0000 7f8360901640 1 --2- 172.21.15.5:0/2456361093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 0x559667adf120 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8350009900 tx=0x7f83500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7637e934c8b8ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.094+0000 7f835bfff640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f834c0a3640 0x7f834c0a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.094+0000 7f83467fc640 1 -- 172.21.15.5:0/2456361093 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350035070 con 0x559667aded40 2024-09-15T22:23:55.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.094+0000 7f83467fc640 1 -- 172.21.15.5:0/2456361093 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83500044b0 con 0x559667aded40 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.094+0000 7f83467fc640 1 -- 172.21.15.5:0/2456361093 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83500035f0 con 0x559667aded40 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.095+0000 7f836473d740 1 -- 172.21.15.5:0/2456361093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 msgr2=0x559667adf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.095+0000 7f835bfff640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f834c0a3640 0x7f834c0a5b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8348098080 tx=0x7f83480979e0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.095+0000 7f836473d740 1 --2- 172.21.15.5:0/2456361093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 0x559667adf120 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8350009900 tx=0x7f83500046c0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.095+0000 7f836473d740 1 -- 172.21.15.5:0/2456361093 shutdown_connections 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.095+0000 7f836473d740 1 --2- 172.21.15.5:0/2456361093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667af2ad0 0x559667af4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.095+0000 7f836473d740 1 --2- 172.21.15.5:0/2456361093 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667ada630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.095+0000 7f836473d740 1 --2- 172.21.15.5:0/2456361093 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 0x559667adf120 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.095+0000 7f836473d740 1 -- 172.21.15.5:0/2456361093 >> 172.21.15.5:0/2456361093 conn(0x559667adb410 msgr2=0x559667adb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.095+0000 7f836473d740 1 -- 172.21.15.5:0/2456361093 shutdown_connections 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.095+0000 7f836473d740 1 -- 172.21.15.5:0/2456361093 wait complete. 2024-09-15T22:23:55.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.096+0000 7f836473d740 1 -- start start 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.096+0000 7f836473d740 1 Processor -- start 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.096+0000 7f836473d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667ada5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f836473d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667ad7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f836473d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667af2ad0 0x559667ad8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f8360901640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667ada5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f836473d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559667af5400 con 0x559667aded40 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f8360901640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667ada5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56016/0 (socket says 172.21.15.5:56016) 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f8360901640 1 -- 172.21.15.5:0/3419649381 learned_addr learned my addr 172.21.15.5:0/3419649381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f836473d740 1 -- 172.21.15.5:0/3419649381 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5596679f7050 con 0x559667adf660 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f836473d740 1 -- 172.21.15.5:0/3419649381 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x559667af5580 con 0x559667af2ad0 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f8360901640 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667af2ad0 0x559667ad8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f835b7fe640 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667ad7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f836473d740 10 client.0 _ll_register_callbacks cb 0x559667a37160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f8360901640 1 -- 172.21.15.5:0/3419649381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667af2ad0 msgr2=0x559667ad8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f8360901640 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667af2ad0 0x559667ad8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f8360901640 1 -- 172.21.15.5:0/3419649381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 msgr2=0x559667ad7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f8360901640 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667ad7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.097+0000 7f8360901640 1 -- 172.21.15.5:0/3419649381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596677ad3c0 con 0x559667aded40 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.098+0000 7f835b7fe640 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667ad7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.098+0000 7f8360901640 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667ada5d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8350004ad0 tx=0x7f8350004b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63816c49fe08e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.098+0000 7f833b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.098+0000 7f833b7fe640 1 -- 172.21.15.5:0/3419649381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350035070 con 0x559667aded40 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.098+0000 7f836473d740 1 -- 172.21.15.5:0/3419649381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83500044b0 con 0x559667aded40 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.098+0000 7f833b7fe640 1 -- 172.21.15.5:0/3419649381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f8350005230 con 0x559667aded40 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.098+0000 7f833b7fe640 1 -- 172.21.15.5:0/3419649381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f835005a050 con 0x559667aded40 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.098+0000 7f836473d740 10 client.17919 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:55.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.098+0000 7f836473d740 1 -- 172.21.15.5:0/3419649381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559667b00f20 con 0x559667aded40 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.099+0000 7f833b7fe640 1 -- 172.21.15.5:0/3419649381 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83500044b0 con 0x559667aded40 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.099+0000 7f833b7fe640 1 -- 172.21.15.5:0/3419649381 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f8350031540 con 0x559667aded40 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.099+0000 7f836473d740 10 client.17919 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.099+0000 7f836473d740 10 client.17919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.099+0000 7f836473d740 20 client.17919 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.099+0000 7f836473d740 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559667b02270 0x559667b04660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.099+0000 7f836473d740 4 client.17919 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.099+0000 7f836473d740 1 -- 172.21.15.5:0/3419649381 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559667a49850 con 0x559667b02270 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.100+0000 7f835b7fe640 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559667b02270 0x559667b04660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.101+0000 7f835b7fe640 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559667b02270 0x559667b04660 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90a38402fbd411c6 server_cookie=71a94e277df08ac0 in_seq=0 out_seq=0 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.101+0000 7f833b7fe640 10 client.17919 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.101+0000 7f833b7fe640 1 -- 172.21.15.5:0/3419649381 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559667a49850 con 0x559667b02270 2024-09-15T22:23:55.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.101+0000 7f833b7fe640 10 client.17919 handle_command_reply: tid=0 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.101+0000 7f836473d740 1 client.17919 shutdown 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.102+0000 7f836473d740 1 -- 172.21.15.5:0/3419649381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 msgr2=0x559667ada5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.102+0000 7f836473d740 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667ada5d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8350004ad0 tx=0x7f8350004b30 comp rx=0 tx=0).stop 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.102+0000 7f836473d740 1 -- 172.21.15.5:0/3419649381 shutdown_connections 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.102+0000 7f836473d740 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667af2ad0 0x559667ad8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.102+0000 7f836473d740 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559667b02270 0x559667b04660 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.102+0000 7f836473d740 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667ad7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.102+0000 7f836473d740 1 --2- 172.21.15.5:0/3419649381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667ada5d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.102+0000 7f836473d740 1 -- 172.21.15.5:0/3419649381 >> 172.21.15.5:0/3419649381 conn(0x559667adb410 msgr2=0x559667ae1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.102+0000 7f833b7fe640 0 client.17919 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.103+0000 7f836473d740 1 -- 172.21.15.5:0/3419649381 shutdown_connections 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.103+0000 7f836473d740 1 -- 172.21.15.5:0/3419649381 wait complete. 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.103+0000 7f836473d740 20 client.17919 trim_cache size 0 max 16384 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.124+0000 7f836473d740 1 Processor -- start 2024-09-15T22:23:55.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.124+0000 7f836473d740 1 -- start start 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.124+0000 7f836473d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 0x559667ad1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.124+0000 7f836473d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667adf660 0x559667ad6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.124+0000 7f836473d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667af2ad0 0x559667b072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.124+0000 7f836473d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8354143190 con 0x559667adf660 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.124+0000 7f836473d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8354140a60 con 0x559667af2ad0 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.124+0000 7f836473d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f835413e8d0 con 0x559667aded40 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.124+0000 7f835b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667adf660 0x559667ad6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f835b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667adf660 0x559667ad6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56030/0 (socket says 172.21.15.5:56030) 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f835b7fe640 1 -- 172.21.15.5:0/3894118631 learned_addr learned my addr 172.21.15.5:0/3894118631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f8360901640 1 --2- 172.21.15.5:0/3894118631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667af2ad0 0x559667b072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f8360901640 1 --2- 172.21.15.5:0/3894118631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 0x559667ad1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f8360901640 1 -- 172.21.15.5:0/3894118631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667af2ad0 msgr2=0x559667b072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f8360901640 1 --2- 172.21.15.5:0/3894118631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667af2ad0 0x559667b072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f8360901640 1 -- 172.21.15.5:0/3894118631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667adf660 msgr2=0x559667ad6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f8360901640 1 --2- 172.21.15.5:0/3894118631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667adf660 0x559667ad6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f8360901640 1 -- 172.21.15.5:0/3894118631 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559667adfee0 con 0x559667aded40 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f8360901640 1 --2- 172.21.15.5:0/3894118631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 0x559667ad1d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8350004740 tx=0x7f8350000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97cd755e04756e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f8345ffb640 1 -- 172.21.15.5:0/3894118631 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350063020 con 0x559667aded40 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f8345ffb640 1 -- 172.21.15.5:0/3894118631 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8350004ba0 con 0x559667aded40 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.125+0000 7f8345ffb640 1 -- 172.21.15.5:0/3894118631 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350035070 con 0x559667aded40 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.126+0000 7f836473d740 1 -- 172.21.15.5:0/3894118631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 msgr2=0x559667ad1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.126+0000 7f836473d740 1 --2- 172.21.15.5:0/3894118631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 0x559667ad1d40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8350004740 tx=0x7f8350000bc0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.126+0000 7f836473d740 1 -- 172.21.15.5:0/3894118631 shutdown_connections 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.126+0000 7f836473d740 1 --2- 172.21.15.5:0/3894118631 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667af2ad0 0x559667b072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.126+0000 7f836473d740 1 --2- 172.21.15.5:0/3894118631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667adf660 0x559667ad6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.126+0000 7f836473d740 1 --2- 172.21.15.5:0/3894118631 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667aded40 0x559667ad1d40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.126+0000 7f836473d740 1 -- 172.21.15.5:0/3894118631 >> 172.21.15.5:0/3894118631 conn(0x559667adb410 msgr2=0x559667ad6170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.126+0000 7f836473d740 1 -- 172.21.15.5:0/3894118631 shutdown_connections 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.126+0000 7f836473d740 1 -- 172.21.15.5:0/3894118631 wait complete. 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.127+0000 7f836473d740 1 -- start start 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.127+0000 7f836473d740 1 Processor -- start 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.127+0000 7f836473d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667b07970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.127+0000 7f836473d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667b04d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.127+0000 7f836473d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667af2ad0 0x559667b05250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.127+0000 7f836473d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f835413ef20 con 0x559667aded40 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.127+0000 7f836473d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8354140390 con 0x559667adf660 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.127+0000 7f836473d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f835413f5f0 con 0x559667af2ad0 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.127+0000 7f836473d740 10 client.0 _ll_register_callbacks cb 0x7f8350004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.128+0000 7f835b7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667b04d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.128+0000 7f8360901640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667b07970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.128+0000 7f8360901640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667b07970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56036/0 (socket says 172.21.15.5:56036) 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.128+0000 7f8360901640 1 -- 172.21.15.5:0/1187986775 learned_addr learned my addr 172.21.15.5:0/1187986775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.128+0000 7f8360901640 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667af2ad0 0x559667b05250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.128+0000 7f8360901640 1 -- 172.21.15.5:0/1187986775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667af2ad0 msgr2=0x559667b05250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.128+0000 7f8360901640 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667af2ad0 0x559667b05250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.128+0000 7f8360901640 1 -- 172.21.15.5:0/1187986775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 msgr2=0x559667b04d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.128+0000 7f8360901640 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667b04d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.128+0000 7f8360901640 1 -- 172.21.15.5:0/1187986775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559667b05a60 con 0x559667aded40 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.128+0000 7f835b7fe640 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667b04d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.129+0000 7f8360901640 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667b07970 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8350031990 tx=0x7f835002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c80915507c1098 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.129+0000 7f833b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.129+0000 7f833b7fe640 1 -- 172.21.15.5:0/1187986775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8350063030 con 0x559667aded40 2024-09-15T22:23:55.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.129+0000 7f833b7fe640 1 -- 172.21.15.5:0/1187986775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9541+0+0 (secure 0 0 0) 0x7f8350031540 con 0x559667aded40 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.129+0000 7f836473d740 1 -- 172.21.15.5:0/1187986775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8350004ba0 con 0x559667aded40 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.129+0000 7f833b7fe640 1 -- 172.21.15.5:0/1187986775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8350072050 con 0x559667aded40 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.129+0000 7f836473d740 10 client.17925 fetch_fsmap learned FSMap version 25 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.129+0000 7f836473d740 1 -- 172.21.15.5:0/1187986775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559667b6cb30 con 0x559667aded40 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.130+0000 7f833b7fe640 1 -- 172.21.15.5:0/1187986775 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8350004ba0 con 0x559667aded40 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.130+0000 7f833b7fe640 1 -- 172.21.15.5:0/1187986775 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f83500037f0 con 0x559667aded40 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.130+0000 7f836473d740 10 client.17925 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.130+0000 7f836473d740 10 client.17925 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.130+0000 7f836473d740 20 client.17925 populate_metadata read hostname 'smithi005' 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.130+0000 7f836473d740 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559667b6d1f0 0x559667b6f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.130+0000 7f836473d740 4 client.17925 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:23:55.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.130+0000 7f836473d740 1 -- 172.21.15.5:0/1187986775 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559667a43d60 con 0x559667b6d1f0 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.130+0000 7f835b7fe640 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559667b6d1f0 0x559667b6f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f835b7fe640 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559667b6d1f0 0x559667b6f5e0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abff31d34e461d9b server_cookie=8eef8dbf66ca5604 in_seq=0 out_seq=0 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f833b7fe640 10 client.17925 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f833b7fe640 1 -- 172.21.15.5:0/1187986775 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559667a43d60 con 0x559667b6d1f0 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f833b7fe640 10 client.17925 handle_command_reply: tid=0 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 1 client.17925 shutdown 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 1 -- 172.21.15.5:0/1187986775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 msgr2=0x559667b07970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667b07970 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8350031990 tx=0x7f835002a410 comp rx=0 tx=0).stop 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 1 -- 172.21.15.5:0/1187986775 shutdown_connections 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x559667af2ad0 0x559667b05250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x559667b6d1f0 0x559667b6f5e0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x559667adf660 0x559667b04d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 1 --2- 172.21.15.5:0/1187986775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559667aded40 0x559667b07970 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 1 -- 172.21.15.5:0/1187986775 >> 172.21.15.5:0/1187986775 conn(0x559667adb410 msgr2=0x559667afd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:55.977 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:23:55.977 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:23:55.977 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:23:55.977 INFO:teuthology.orchestra.run.smithi005.stdout: "d4303284-aedf-4ffc-a8d9-80ec1a3ef710": { 2024-09-15T22:23:55.977 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:23:55.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d4303284-aedf-4ffc-a8d9-80ec1a3ef710", 2024-09-15T22:23:55.977 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:23:55.977 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:23:55.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:23:55.978 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:23:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f833b7fe640 0 client.17925 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:23:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 1 -- 172.21.15.5:0/1187986775 shutdown_connections 2024-09-15T22:23:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 1 -- 172.21.15.5:0/1187986775 wait complete. 2024-09-15T22:23:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.131+0000 7f836473d740 20 client.17925 trim_cache size 0 max 16384 2024-09-15T22:23:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.132+0000 7f8361102640 1 -- 172.21.15.5:0/2293522573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f834c0a3640 msgr2=0x7f834c0a5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.132+0000 7f8361102640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f834c0a3640 0x7f834c0a5b00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8348098080 tx=0x7f83480979e0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.132+0000 7f8361102640 1 -- 172.21.15.5:0/2293522573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8354094930 msgr2=0x7f8354131a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:23:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.132+0000 7f8361102640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8354094930 0x7f8354131a50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f834800d950 tx=0x7f834800de10 comp rx=0 tx=0).stop 2024-09-15T22:23:55.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.132+0000 7f8361102640 1 -- 172.21.15.5:0/2293522573 shutdown_connections 2024-09-15T22:23:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.132+0000 7f8361102640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f834c0a3640 0x7f834c0a5b00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.132+0000 7f8361102640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f835409a550 0x7f835412bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.132+0000 7f8361102640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83540952d0 0x7f8354131f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.132+0000 7f8361102640 1 --2- 172.21.15.5:0/2293522573 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8354094930 0x7f8354131a50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:23:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.132+0000 7f8361102640 1 -- 172.21.15.5:0/2293522573 >> 172.21.15.5:0/2293522573 conn(0x7f835400a1f0 msgr2=0x7f8354004cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:23:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.133+0000 7f8361102640 1 -- 172.21.15.5:0/2293522573 shutdown_connections 2024-09-15T22:23:55.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:23:55.133+0000 7f8361102640 1 -- 172.21.15.5:0/2293522573 wait complete. 2024-09-15T22:23:56.834 INFO:tasks.cephfs.filesystem:scrub status for tag:d4303284-aedf-4ffc-a8d9-80ec1a3ef710 - {'path': '/', 'tag': 'd4303284-aedf-4ffc-a8d9-80ec1a3ef710', 'options': 'recursive,force'} 2024-09-15T22:23:57.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:57 smithi099 ceph-mon[34892]: pgmap v470: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 478 op/s 2024-09-15T22:23:57.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:57 smithi129 ceph-mon[35314]: pgmap v470: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 478 op/s 2024-09-15T22:23:57.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:57 smithi005 ceph-mon[30944]: pgmap v470: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.4 MiB/s wr, 478 op/s 2024-09-15T22:23:58.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:58 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:23:58.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:58 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:23:58.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:58 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:23:59.495 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:23:59 smithi129 ceph-mon[35314]: pgmap v471: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.2 MiB/s wr, 570 op/s 2024-09-15T22:23:59.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:23:59 smithi099 ceph-mon[34892]: pgmap v471: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.2 MiB/s wr, 570 op/s 2024-09-15T22:23:59.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:23:59 smithi005 ceph-mon[30944]: pgmap v471: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 5.2 MiB/s wr, 570 op/s 2024-09-15T22:23:59.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 870.0 s, 503.8 tps, lat 209.223 ms stddev 270.587 2024-09-15T22:24:01.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:01 smithi099 ceph-mon[34892]: pgmap v472: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-09-15T22:24:01.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:01 smithi129 ceph-mon[35314]: pgmap v472: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-09-15T22:24:01.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:01 smithi005 ceph-mon[30944]: pgmap v472: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 4.1 MiB/s wr, 432 op/s 2024-09-15T22:24:03.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:03 smithi099 ceph-mon[34892]: pgmap v473: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-09-15T22:24:03.594 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:03 smithi129 ceph-mon[35314]: pgmap v473: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-09-15T22:24:03.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:03 smithi005 ceph-mon[30944]: pgmap v473: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 5.2 MiB/s wr, 536 op/s 2024-09-15T22:24:04.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 875.0 s, 540.8 tps, lat 185.518 ms stddev 234.974 2024-09-15T22:24:05.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:05 smithi099 ceph-mon[34892]: pgmap v474: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-09-15T22:24:05.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:05 smithi129 ceph-mon[35314]: pgmap v474: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-09-15T22:24:05.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:05 smithi005 ceph-mon[30944]: pgmap v474: 129 pgs: 129 active+clean; 2.9 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 4.1 MiB/s wr, 460 op/s 2024-09-15T22:24:07.402 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-15T22:24:07.402 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-15T22:24:07.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:24:07.474 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:07 smithi005 ceph-mon[30944]: pgmap v475: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-09-15T22:24:07.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:07 smithi099 ceph-mon[34892]: pgmap v475: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-09-15T22:24:07.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:07 smithi129 ceph-mon[35314]: pgmap v475: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 4.0 MiB/s wr, 453 op/s 2024-09-15T22:24:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:08.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:08 smithi099 ceph-mon[34892]: osdmap e106: 12 total, 12 up, 12 in 2024-09-15T22:24:08.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:08 smithi129 ceph-mon[35314]: osdmap e106: 12 total, 12 up, 12 in 2024-09-15T22:24:08.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:08 smithi005 ceph-mon[30944]: osdmap e106: 12 total, 12 up, 12 in 2024-09-15T22:24:09.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.117+0000 7f133afca640 1 -- 172.21.15.5:0/3356157692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1334100940 msgr2=0x7f1334100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.117+0000 7f133afca640 1 --2- 172.21.15.5:0/3356157692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1334100940 0x7f1334100d20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1324009900 tx=0x7f132402a410 comp rx=0 tx=0).stop 2024-09-15T22:24:09.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.117+0000 7f133afca640 1 -- 172.21.15.5:0/3356157692 shutdown_connections 2024-09-15T22:24:09.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.117+0000 7f133afca640 1 --2- 172.21.15.5:0/3356157692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f133410e4f0 0x7f13341108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.117+0000 7f133afca640 1 --2- 172.21.15.5:0/3356157692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13341012f0 0x7f133410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.117+0000 7f133afca640 1 --2- 172.21.15.5:0/3356157692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1334100940 0x7f1334100d20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.117+0000 7f133afca640 1 -- 172.21.15.5:0/3356157692 >> 172.21.15.5:0/3356157692 conn(0x7f13340fc7d0 msgr2=0x7f13340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.117+0000 7f133afca640 1 -- 172.21.15.5:0/3356157692 shutdown_connections 2024-09-15T22:24:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.117+0000 7f133afca640 1 -- 172.21.15.5:0/3356157692 wait complete. 2024-09-15T22:24:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f133afca640 1 Processor -- start 2024-09-15T22:24:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f133afca640 1 -- start start 2024-09-15T22:24:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f133afca640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1334100940 0x7f133419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f133afca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13341012f0 0x7f133419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f133afca640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f133410e4f0 0x7f133419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f133afca640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13341141e0 con 0x7f13341012f0 2024-09-15T22:24:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f133afca640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1334114060 con 0x7f1334100940 2024-09-15T22:24:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f133afca640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1334114360 con 0x7f133410e4f0 2024-09-15T22:24:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f1333fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13341012f0 0x7f133419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f1333fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13341012f0 0x7f133419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54250/0 (socket says 172.21.15.5:54250) 2024-09-15T22:24:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f1333fff640 1 -- 172.21.15.5:0/3580643709 learned_addr learned my addr 172.21.15.5:0/3580643709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.118+0000 7f1338d66640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1334100940 0x7f133419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.119+0000 7f1339567640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f133410e4f0 0x7f133419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.119+0000 7f1333fff640 1 -- 172.21.15.5:0/3580643709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f133410e4f0 msgr2=0x7f133419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.119+0000 7f1333fff640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f133410e4f0 0x7f133419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.119+0000 7f1333fff640 1 -- 172.21.15.5:0/3580643709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1334100940 msgr2=0x7f133419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.119+0000 7f1333fff640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1334100940 0x7f133419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.119+0000 7f1333fff640 1 -- 172.21.15.5:0/3580643709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133419f610 con 0x7f13341012f0 2024-09-15T22:24:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.119+0000 7f1339567640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f133410e4f0 0x7f133419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.119+0000 7f1333fff640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13341012f0 0x7f133419aba0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f131c00e9f0 tx=0x7f131c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.119+0000 7f1331ffb640 1 -- 172.21.15.5:0/3580643709 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c00cd80 con 0x7f13341012f0 2024-09-15T22:24:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.119+0000 7f1331ffb640 1 -- 172.21.15.5:0/3580643709 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c010910 con 0x7f13341012f0 2024-09-15T22:24:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.119+0000 7f133afca640 1 -- 172.21.15.5:0/3580643709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133419f900 con 0x7f13341012f0 2024-09-15T22:24:09.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.120+0000 7f1331ffb640 1 -- 172.21.15.5:0/3580643709 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c018990 con 0x7f13341012f0 2024-09-15T22:24:09.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.120+0000 7f133afca640 1 -- 172.21.15.5:0/3580643709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13341a71e0 con 0x7f13341012f0 2024-09-15T22:24:09.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.121+0000 7f133afca640 1 -- 172.21.15.5:0/3580643709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1334100d20 con 0x7f13341012f0 2024-09-15T22:24:09.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.123+0000 7f1331ffb640 1 -- 172.21.15.5:0/3580643709 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f131c010ac0 con 0x7f13341012f0 2024-09-15T22:24:09.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.124+0000 7f1331ffb640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f13200a3430 0x7f13200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:09.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.125+0000 7f1338d66640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f13200a3430 0x7f13200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:09.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.125+0000 7f1331ffb640 1 -- 172.21.15.5:0/3580643709 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9629+0+0 (secure 0 0 0) 0x7f131c0d0490 con 0x7f13341012f0 2024-09-15T22:24:09.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.125+0000 7f1338d66640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f13200a3430 0x7f13200a58f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1324000c00 tx=0x7f1324005b50 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:09.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.130+0000 7f1331ffb640 1 -- 172.21.15.5:0/3580643709 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f131c09ca20 con 0x7f13341012f0 2024-09-15T22:24:09.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.500+0000 7f133afca640 1 -- 172.21.15.5:0/3580643709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13341096e0 con 0x7f13341012f0 2024-09-15T22:24:09.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:09 smithi005 ceph-mon[30944]: pgmap v477: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.2 MiB/s wr, 470 op/s 2024-09-15T22:24:09.506 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:09.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.504+0000 7f1331ffb640 1 -- 172.21.15.5:0/3580643709 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f131c09c320 con 0x7f13341012f0 2024-09-15T22:24:09.507 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:24:09.507 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:24:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.515+0000 7f133afca640 1 -- 172.21.15.5:0/3580643709 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f13200a3430 msgr2=0x7f13200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.515+0000 7f133afca640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f13200a3430 0x7f13200a58f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1324000c00 tx=0x7f1324005b50 comp rx=0 tx=0).stop 2024-09-15T22:24:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.515+0000 7f133afca640 1 -- 172.21.15.5:0/3580643709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13341012f0 msgr2=0x7f133419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.515+0000 7f133afca640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13341012f0 0x7f133419aba0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f131c00e9f0 tx=0x7f131c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.516+0000 7f133afca640 1 -- 172.21.15.5:0/3580643709 shutdown_connections 2024-09-15T22:24:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.516+0000 7f133afca640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f13200a3430 0x7f13200a58f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.516+0000 7f133afca640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f133410e4f0 0x7f133419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.516+0000 7f133afca640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13341012f0 0x7f133419aba0 secure :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f131c00e9f0 tx=0x7f131c00eeb0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.516+0000 7f133afca640 1 --2- 172.21.15.5:0/3580643709 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1334100940 0x7f133419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:09.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.516+0000 7f133afca640 1 -- 172.21.15.5:0/3580643709 >> 172.21.15.5:0/3580643709 conn(0x7f13340fc7d0 msgr2=0x7f13340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:09.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.517+0000 7f133afca640 1 -- 172.21.15.5:0/3580643709 shutdown_connections 2024-09-15T22:24:09.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:09.519+0000 7f133afca640 1 -- 172.21.15.5:0/3580643709 wait complete. 2024-09-15T22:24:09.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:09 smithi099 ceph-mon[34892]: pgmap v477: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.2 MiB/s wr, 470 op/s 2024-09-15T22:24:09.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:09 smithi129 ceph-mon[35314]: pgmap v477: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.2 MiB/s wr, 470 op/s 2024-09-15T22:24:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 880.0 s, 402.2 tps, lat 245.450 ms stddev 297.993 2024-09-15T22:24:10.433 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-15T22:24:10.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-15T22:24:10.488 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:10 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3580643709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:10.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:10 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3580643709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:10.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:10 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3580643709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:10.881 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:11.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:11 smithi005 ceph-mon[30944]: pgmap v478: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.4 MiB/s wr, 495 op/s 2024-09-15T22:24:11.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:11 smithi099 ceph-mon[34892]: pgmap v478: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.4 MiB/s wr, 495 op/s 2024-09-15T22:24:11.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:11 smithi129 ceph-mon[35314]: pgmap v478: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 4.4 MiB/s wr, 495 op/s 2024-09-15T22:24:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.612+0000 7fda1f5fe640 1 -- 172.21.15.5:0/4066934949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18105560 msgr2=0x7fda1810d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.612+0000 7fda1f5fe640 1 --2- 172.21.15.5:0/4066934949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18105560 0x7fda1810d0e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fda0c00a910 tx=0x7fda0c02a620 comp rx=0 tx=0).stop 2024-09-15T22:24:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.614+0000 7fda1f5fe640 1 -- 172.21.15.5:0/4066934949 shutdown_connections 2024-09-15T22:24:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.614+0000 7fda1f5fe640 1 --2- 172.21.15.5:0/4066934949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18105560 0x7fda1810d0e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.614+0000 7fda1f5fe640 1 --2- 172.21.15.5:0/4066934949 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda18100de0 0x7fda18104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.614+0000 7fda1f5fe640 1 --2- 172.21.15.5:0/4066934949 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda18100430 0x7fda18100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.614+0000 7fda1f5fe640 1 -- 172.21.15.5:0/4066934949 >> 172.21.15.5:0/4066934949 conn(0x7fda18077230 msgr2=0x7fda180fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.614+0000 7fda1f5fe640 1 -- 172.21.15.5:0/4066934949 shutdown_connections 2024-09-15T22:24:11.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.618+0000 7fda1f5fe640 1 -- 172.21.15.5:0/4066934949 wait complete. 2024-09-15T22:24:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.619+0000 7fda1f5fe640 1 Processor -- start 2024-09-15T22:24:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.619+0000 7fda1f5fe640 1 -- start start 2024-09-15T22:24:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.619+0000 7fda1f5fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18100430 0x7fda1810bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.619+0000 7fda1f5fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda18100de0 0x7fda1810c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.619+0000 7fda1f5fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda18105560 0x7fda18117040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.619+0000 7fda1f5fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda1810f3d0 con 0x7fda18100430 2024-09-15T22:24:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.619+0000 7fda1f5fe640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fda1810f250 con 0x7fda18100de0 2024-09-15T22:24:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.619+0000 7fda1f5fe640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fda1810f550 con 0x7fda18105560 2024-09-15T22:24:11.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.620+0000 7fda1cb99640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda18100de0 0x7fda1810c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.620+0000 7fda1cb99640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda18100de0 0x7fda1810c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:52726/0 (socket says 172.21.15.5:52726) 2024-09-15T22:24:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.620+0000 7fda1cb99640 1 -- 172.21.15.5:0/2472411225 learned_addr learned my addr 172.21.15.5:0/2472411225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.620+0000 7fda1db9b640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda18105560 0x7fda18117040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.622+0000 7fda1cb99640 1 -- 172.21.15.5:0/2472411225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda18105560 msgr2=0x7fda18117040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.622+0000 7fda1cb99640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda18105560 0x7fda18117040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.622+0000 7fda1cb99640 1 -- 172.21.15.5:0/2472411225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18100430 msgr2=0x7fda1810bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.622+0000 7fda1cb99640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18100430 0x7fda1810bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.622+0000 7fda1cb99640 1 -- 172.21.15.5:0/2472411225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda181177c0 con 0x7fda18100de0 2024-09-15T22:24:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.623+0000 7fda1cb99640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda18100de0 0x7fda1810c080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fda140097d0 tx=0x7fda14009c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.623+0000 7fda0a7fc640 1 -- 172.21.15.5:0/2472411225 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1400eab0 con 0x7fda18100de0 2024-09-15T22:24:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.623+0000 7fda1f5fe640 1 -- 172.21.15.5:0/2472411225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda18117ab0 con 0x7fda18100de0 2024-09-15T22:24:11.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.623+0000 7fda1f5fe640 1 -- 172.21.15.5:0/2472411225 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda1811a080 con 0x7fda18100de0 2024-09-15T22:24:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.627+0000 7fda219d7740 1 Processor -- start 2024-09-15T22:24:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.627+0000 7fda219d7740 1 -- start start 2024-09-15T22:24:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.627+0000 7fda219d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc951ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.627+0000 7fda219d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc951b080 0x562dc9525400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.627+0000 7fda219d7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc9525940 0x562dc9527d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.627+0000 7fda219d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562dc94802a0 con 0x562dc951a760 2024-09-15T22:24:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.627+0000 7fda219d7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562dc9427b10 con 0x562dc9525940 2024-09-15T22:24:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.627+0000 7fda219d7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562dc947c260 con 0x562dc951b080 2024-09-15T22:24:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.628+0000 7fda0a7fc640 1 -- 172.21.15.5:0/2472411225 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda14019070 con 0x7fda18100de0 2024-09-15T22:24:11.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.628+0000 7fda0a7fc640 1 -- 172.21.15.5:0/2472411225 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda14013360 con 0x7fda18100de0 2024-09-15T22:24:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.630+0000 7fda1db9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc951ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.630+0000 7fda1db9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc951ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54266/0 (socket says 172.21.15.5:54266) 2024-09-15T22:24:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.630+0000 7fda1db9b640 1 -- 172.21.15.5:0/3676504592 learned_addr learned my addr 172.21.15.5:0/3676504592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.631+0000 7fda1d39a640 1 --2- 172.21.15.5:0/3676504592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc951b080 0x562dc9525400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.631+0000 7fda1d39a640 1 -- 172.21.15.5:0/3676504592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc9525940 msgr2=0x562dc9527d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.631+0000 7fda1d39a640 1 --2- 172.21.15.5:0/3676504592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc9525940 0x562dc9527d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.631+0000 7fda1d39a640 1 -- 172.21.15.5:0/3676504592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 msgr2=0x562dc951ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.631+0000 7fda1d39a640 1 --2- 172.21.15.5:0/3676504592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc951ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.633+0000 7fda1d39a640 1 -- 172.21.15.5:0/3676504592 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc91f1b90 con 0x562dc951b080 2024-09-15T22:24:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.635+0000 7fda1d39a640 1 --2- 172.21.15.5:0/3676504592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc951b080 0x562dc9525400 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fda10009d40 tx=0x7fda1000b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dcd4451074c88d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.637+0000 7fda037fe640 1 -- 172.21.15.5:0/3676504592 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10018070 con 0x562dc951b080 2024-09-15T22:24:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.638+0000 7fda037fe640 1 -- 172.21.15.5:0/3676504592 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda100138f0 con 0x562dc951b080 2024-09-15T22:24:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.638+0000 7fda037fe640 1 -- 172.21.15.5:0/3676504592 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda10012a00 con 0x562dc951b080 2024-09-15T22:24:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.639+0000 7fda219d7740 1 -- 172.21.15.5:0/3676504592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc951b080 msgr2=0x562dc9525400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.639+0000 7fda219d7740 1 --2- 172.21.15.5:0/3676504592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc951b080 0x562dc9525400 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fda10009d40 tx=0x7fda1000b7c0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.640+0000 7fda0a7fc640 1 -- 172.21.15.5:0/2472411225 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fda14013070 con 0x7fda18100de0 2024-09-15T22:24:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.641+0000 7fda219d7740 1 -- 172.21.15.5:0/3676504592 shutdown_connections 2024-09-15T22:24:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.641+0000 7fda219d7740 1 --2- 172.21.15.5:0/3676504592 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc9525940 0x562dc9527d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.641+0000 7fda219d7740 1 --2- 172.21.15.5:0/3676504592 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc951b080 0x562dc9525400 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.641+0000 7fda219d7740 1 --2- 172.21.15.5:0/3676504592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc951ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.641+0000 7fda219d7740 1 -- 172.21.15.5:0/3676504592 >> 172.21.15.5:0/3676504592 conn(0x562dc9516cf0 msgr2=0x562dc95170d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.641+0000 7fda0a7fc640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fda040a35c0 0x7fda040a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.641+0000 7fda0a7fc640 1 -- 172.21.15.5:0/2472411225 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9629+0+0 (secure 0 0 0) 0x7fda140cfa00 con 0x7fda18100de0 2024-09-15T22:24:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.644+0000 7fda1d39a640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fda040a35c0 0x7fda040a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.645+0000 7fda219d7740 1 -- 172.21.15.5:0/3676504592 shutdown_connections 2024-09-15T22:24:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.649+0000 7fda219d7740 1 -- 172.21.15.5:0/3676504592 wait complete. 2024-09-15T22:24:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.649+0000 7fda219d7740 1 -- start start 2024-09-15T22:24:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.649+0000 7fda219d7740 1 Processor -- start 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.650+0000 7fda1d39a640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fda040a35c0 0x7fda040a5a80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fda10009ad0 tx=0x7fda10009f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.651+0000 7fda219d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc95227d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.651+0000 7fda219d7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951b080 0x562dc9522d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.651+0000 7fda219d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc9525940 0x562dc9523250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.651+0000 7fda219d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562dc9528510 con 0x562dc951a760 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.651+0000 7fda219d7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x562dc94086e0 con 0x562dc951b080 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.651+0000 7fda219d7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x562dc9528690 con 0x562dc9525940 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.651+0000 7fda219d7740 10 client.0 _ll_register_callbacks cb 0x562dc942ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.654+0000 7fda1db9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc9525940 0x562dc9523250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.654+0000 7fda1cb99640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951b080 0x562dc9522d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.655+0000 7fda1db9b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc9525940 0x562dc9523250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:59638/0 (socket says 172.21.15.5:59638) 2024-09-15T22:24:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.655+0000 7fda1db9b640 1 -- 172.21.15.5:0/3698858729 learned_addr learned my addr 172.21.15.5:0/3698858729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:11.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.656+0000 7fda1d39a640 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc95227d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.656+0000 7fda1cb99640 1 -- 172.21.15.5:0/3698858729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc9525940 msgr2=0x562dc9523250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:11.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.656+0000 7fda1cb99640 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc9525940 0x562dc9523250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.656+0000 7fda1cb99640 1 -- 172.21.15.5:0/3698858729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 msgr2=0x562dc95227d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:11.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.656+0000 7fda1cb99640 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc95227d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.656+0000 7fda1cb99640 1 -- 172.21.15.5:0/3698858729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc9531a40 con 0x562dc951b080 2024-09-15T22:24:11.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.662+0000 7fda1cb99640 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951b080 0x562dc9522d10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fda140ccca0 tx=0x7fda1409b300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bae1aca9809fdba server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:11.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.663+0000 7fda00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:24:11.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.665+0000 7fda00ff9640 1 -- 172.21.15.5:0/3698858729 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda140cd130 con 0x562dc951b080 2024-09-15T22:24:11.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.665+0000 7fda219d7740 1 -- 172.21.15.5:0/3698858729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda100138f0 con 0x562dc951b080 2024-09-15T22:24:11.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.665+0000 7fda00ff9640 1 -- 172.21.15.5:0/3698858729 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9629+0+0 (secure 0 0 0) 0x7fda1409cde0 con 0x562dc951b080 2024-09-15T22:24:11.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.669+0000 7fda00ff9640 1 -- 172.21.15.5:0/3698858729 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fda140a4520 con 0x562dc951b080 2024-09-15T22:24:11.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.672+0000 7fda219d7740 10 client.27410 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:11.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.672+0000 7fda219d7740 1 -- 172.21.15.5:0/3698858729 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562dc9534410 con 0x562dc951b080 2024-09-15T22:24:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.677+0000 7fda00ff9640 1 -- 172.21.15.5:0/3698858729 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda100138f0 con 0x562dc951b080 2024-09-15T22:24:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.677+0000 7fda00ff9640 1 -- 172.21.15.5:0/3698858729 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fda140b4ab0 con 0x562dc951b080 2024-09-15T22:24:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.677+0000 7fda219d7740 10 client.27410 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.677+0000 7fda219d7740 10 client.27410 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:24:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.677+0000 7fda219d7740 20 client.27410 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.677+0000 7fda219d7740 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562dc9535ca0 0x562dc95380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.677+0000 7fda219d7740 4 client.27410 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.677+0000 7fda219d7740 1 -- 172.21.15.5:0/3698858729 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dc948d9d0 con 0x562dc9535ca0 2024-09-15T22:24:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.680+0000 7fda1d39a640 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562dc9535ca0 0x562dc95380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.692+0000 7fda1d39a640 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562dc9535ca0 0x562dc95380e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f05461e8a6937f8 server_cookie=7eb3f41b35f06ca9 in_seq=0 out_seq=0 2024-09-15T22:24:11.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.693+0000 7fda00ff9640 10 client.27410 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:11.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.697+0000 7fda00ff9640 1 -- 172.21.15.5:0/3698858729 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562dc948d9d0 con 0x562dc9535ca0 2024-09-15T22:24:11.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.697+0000 7fda00ff9640 10 client.27410 handle_command_reply: tid=0 2024-09-15T22:24:11.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.697+0000 7fda219d7740 1 client.27410 shutdown 2024-09-15T22:24:11.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.698+0000 7fda219d7740 1 -- 172.21.15.5:0/3698858729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951b080 msgr2=0x562dc9522d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:11.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.698+0000 7fda219d7740 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951b080 0x562dc9522d10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fda140ccca0 tx=0x7fda1409b300 comp rx=0 tx=0).stop 2024-09-15T22:24:11.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.698+0000 7fda219d7740 1 -- 172.21.15.5:0/3698858729 shutdown_connections 2024-09-15T22:24:11.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.698+0000 7fda219d7740 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc9525940 0x562dc9523250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.698+0000 7fda219d7740 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562dc9535ca0 0x562dc95380e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.698+0000 7fda219d7740 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951b080 0x562dc9522d10 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fda140ccca0 tx=0x7fda1409b300 comp rx=0 tx=0).stop 2024-09-15T22:24:11.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.698+0000 7fda219d7740 1 --2- 172.21.15.5:0/3698858729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc95227d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.698+0000 7fda219d7740 1 -- 172.21.15.5:0/3698858729 >> 172.21.15.5:0/3698858729 conn(0x562dc9516cf0 msgr2=0x562dc951ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:11.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.698+0000 7fda00ff9640 0 client.27410 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:11.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.699+0000 7fda219d7740 1 -- 172.21.15.5:0/3698858729 shutdown_connections 2024-09-15T22:24:11.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.699+0000 7fda219d7740 1 -- 172.21.15.5:0/3698858729 wait complete. 2024-09-15T22:24:11.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.699+0000 7fda219d7740 20 client.27410 trim_cache size 0 max 16384 2024-09-15T22:24:11.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.732+0000 7fda219d7740 1 Processor -- start 2024-09-15T22:24:11.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.733+0000 7fda219d7740 1 -- start start 2024-09-15T22:24:11.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.733+0000 7fda219d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc951a760 0x562dc95230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.733+0000 7fda219d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc9525940 0x562dc951c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.733+0000 7fda219d7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 0x562dc953d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.733+0000 7fda219d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc012b40 con 0x562dc9525940 2024-09-15T22:24:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.733+0000 7fda219d7740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc010410 con 0x562dc951ce50 2024-09-15T22:24:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.733+0000 7fda219d7740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc00e280 con 0x562dc951a760 2024-09-15T22:24:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.735+0000 7fda1cb99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc9525940 0x562dc951c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.735+0000 7fda1cb99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc9525940 0x562dc951c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54270/0 (socket says 172.21.15.5:54270) 2024-09-15T22:24:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.735+0000 7fda1cb99640 1 -- 172.21.15.5:0/790912965 learned_addr learned my addr 172.21.15.5:0/790912965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.737+0000 7fda1d39a640 1 --2- 172.21.15.5:0/790912965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc951a760 0x562dc95230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.738+0000 7fda1db9b640 1 --2- 172.21.15.5:0/790912965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 0x562dc953d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.739+0000 7fda1cb99640 1 -- 172.21.15.5:0/790912965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc951a760 msgr2=0x562dc95230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.739+0000 7fda1cb99640 1 --2- 172.21.15.5:0/790912965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc951a760 0x562dc95230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.739+0000 7fda1cb99640 1 -- 172.21.15.5:0/790912965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 msgr2=0x562dc953d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.739+0000 7fda1cb99640 1 --2- 172.21.15.5:0/790912965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 0x562dc953d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.739+0000 7fda1cb99640 1 -- 172.21.15.5:0/790912965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dc9520d40 con 0x562dc9525940 2024-09-15T22:24:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.742+0000 7fda1cb99640 1 --2- 172.21.15.5:0/790912965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc9525940 0x562dc951c910 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fda14014c00 tx=0x7fda1400b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baacb5145698608 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.743+0000 7fda02ffd640 1 -- 172.21.15.5:0/790912965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda140a9080 con 0x562dc9525940 2024-09-15T22:24:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.743+0000 7fda02ffd640 1 -- 172.21.15.5:0/790912965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda140b4070 con 0x562dc9525940 2024-09-15T22:24:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.743+0000 7fda02ffd640 1 -- 172.21.15.5:0/790912965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1409cbe0 con 0x562dc9525940 2024-09-15T22:24:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.749+0000 7fda219d7740 1 -- 172.21.15.5:0/790912965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc9525940 msgr2=0x562dc951c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.749+0000 7fda219d7740 1 --2- 172.21.15.5:0/790912965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc9525940 0x562dc951c910 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fda14014c00 tx=0x7fda1400b7a0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.754+0000 7fda219d7740 1 -- 172.21.15.5:0/790912965 shutdown_connections 2024-09-15T22:24:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.754+0000 7fda219d7740 1 --2- 172.21.15.5:0/790912965 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 0x562dc953d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.754+0000 7fda219d7740 1 --2- 172.21.15.5:0/790912965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc9525940 0x562dc951c910 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.754+0000 7fda219d7740 1 --2- 172.21.15.5:0/790912965 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc951a760 0x562dc95230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.754+0000 7fda219d7740 1 -- 172.21.15.5:0/790912965 >> 172.21.15.5:0/790912965 conn(0x562dc9516cf0 msgr2=0x562dc951c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.754+0000 7fda219d7740 1 -- 172.21.15.5:0/790912965 shutdown_connections 2024-09-15T22:24:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.754+0000 7fda219d7740 1 -- 172.21.15.5:0/790912965 wait complete. 2024-09-15T22:24:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.755+0000 7fda219d7740 1 -- start start 2024-09-15T22:24:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.755+0000 7fda219d7740 1 Processor -- start 2024-09-15T22:24:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.756+0000 7fda219d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc952da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.756+0000 7fda219d7740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 0x562dc952df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.756+0000 7fda219d7740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc9525940 0x562dc9529e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.756+0000 7fda1cb99640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 0x562dc952df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.756+0000 7fda1cb99640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 0x562dc952df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:52756/0 (socket says 172.21.15.5:52756) 2024-09-15T22:24:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.757+0000 7fda1cb99640 1 -- 172.21.15.5:0/2653953489 learned_addr learned my addr 172.21.15.5:0/2653953489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.757+0000 7fda219d7740 1 -- 172.21.15.5:0/2653953489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc00e8d0 con 0x562dc951a760 2024-09-15T22:24:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.757+0000 7fda219d7740 1 -- 172.21.15.5:0/2653953489 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc00fd40 con 0x562dc951ce50 2024-09-15T22:24:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.757+0000 7fda219d7740 1 -- 172.21.15.5:0/2653953489 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc00efa0 con 0x562dc9525940 2024-09-15T22:24:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.757+0000 7fda219d7740 10 client.0 _ll_register_callbacks cb 0x7fda1409b4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.757+0000 7fda1cb99640 1 -- 172.21.15.5:0/2653953489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc9525940 msgr2=0x562dc9529e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.757+0000 7fda1cb99640 1 --2- 172.21.15.5:0/2653953489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc9525940 0x562dc9529e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.757+0000 7fda1cb99640 1 -- 172.21.15.5:0/2653953489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 msgr2=0x562dc952da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.757+0000 7fda1cb99640 1 --2- 172.21.15.5:0/2653953489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc952da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.757+0000 7fda1cb99640 1 -- 172.21.15.5:0/2653953489 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dc952a6a0 con 0x562dc951ce50 2024-09-15T22:24:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.758+0000 7fda1cb99640 1 --2- 172.21.15.5:0/2653953489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 0x562dc952df70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fda14014b60 tx=0x7fda1409b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2a29bf6d8e73f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.758+0000 7fda00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:24:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.758+0000 7fda219d7740 1 -- 172.21.15.5:0/2653953489 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda140b4070 con 0x562dc951ce50 2024-09-15T22:24:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.758+0000 7fda00ff9640 1 -- 172.21.15.5:0/2653953489 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1409b210 con 0x562dc951ce50 2024-09-15T22:24:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.758+0000 7fda00ff9640 1 -- 172.21.15.5:0/2653953489 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9629+0+0 (secure 0 0 0) 0x7fda1409cde0 con 0x562dc951ce50 2024-09-15T22:24:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.758+0000 7fda00ff9640 1 -- 172.21.15.5:0/2653953489 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fda140cd130 con 0x562dc951ce50 2024-09-15T22:24:11.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.760+0000 7fda219d7740 10 client.27416 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:11.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.760+0000 7fda219d7740 1 -- 172.21.15.5:0/2653953489 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({fsmap=25}) -- 0x562dc952a6a0 con 0x562dc951ce50 2024-09-15T22:24:11.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.762+0000 7fda00ff9640 1 -- 172.21.15.5:0/2653953489 <== mon.2 v2:172.21.15.99:3300/0 4 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fda1409ad00 con 0x562dc951ce50 2024-09-15T22:24:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.762+0000 7fda219d7740 10 client.27416 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.762+0000 7fda219d7740 10 client.27416 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:24:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.762+0000 7fda219d7740 20 client.27416 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.762+0000 7fda219d7740 1 --2- 172.21.15.5:0/2653953489 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562dc95a15b0 0x562dc95a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.762+0000 7fda219d7740 4 client.27416 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-15T22:24:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.762+0000 7fda219d7740 1 -- 172.21.15.5:0/2653953489 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x7fda140b4070 con 0x562dc95a15b0 2024-09-15T22:24:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.762+0000 7fda1d39a640 1 --2- 172.21.15.5:0/2653953489 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562dc95a15b0 0x562dc95a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.768+0000 7fda00ff9640 10 client.27416 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:11.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:11.768+0000 7fda1d39a640 1 --2- 172.21.15.5:0/2653953489 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562dc95a15b0 0x562dc95a39f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde6d90585cb5c12 server_cookie=6fbcdce171a0f88b in_seq=0 out_seq=0 2024-09-15T22:24:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.106+0000 7fda00ff9640 1 -- 172.21.15.5:0/2653953489 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fda140b4070 con 0x562dc95a15b0 2024-09-15T22:24:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda00ff9640 10 client.27416 handle_command_reply: tid=0 2024-09-15T22:24:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 1 client.27416 shutdown 2024-09-15T22:24:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 1 -- 172.21.15.5:0/2653953489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 msgr2=0x562dc952df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 1 --2- 172.21.15.5:0/2653953489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 0x562dc952df70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fda14014b60 tx=0x7fda1409b920 comp rx=0 tx=0).stop 2024-09-15T22:24:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 1 -- 172.21.15.5:0/2653953489 shutdown_connections 2024-09-15T22:24:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 1 --2- 172.21.15.5:0/2653953489 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x562dc9525940 0x562dc9529e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 1 --2- 172.21.15.5:0/2653953489 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x562dc95a15b0 0x562dc95a39f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 1 --2- 172.21.15.5:0/2653953489 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x562dc951ce50 0x562dc952df70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 1 --2- 172.21.15.5:0/2653953489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562dc951a760 0x562dc952da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda00ff9640 0 client.27416 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 1 -- 172.21.15.5:0/2653953489 >> 172.21.15.5:0/2653953489 conn(0x562dc9516cf0 msgr2=0x562dc95307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 1 -- 172.21.15.5:0/2653953489 shutdown_connections 2024-09-15T22:24:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 1 -- 172.21.15.5:0/2653953489 wait complete. 2024-09-15T22:24:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.107+0000 7fda219d7740 20 client.27416 trim_cache size 0 max 16384 2024-09-15T22:24:12.110 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:24:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:24:12.110 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 54, 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "17d4158a": { 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 54, 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 137.7, 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:24:12.111 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:24:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:24:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:24:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:24:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:12.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:12.112 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:24:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 -- 172.21.15.5:0/2472411225 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fda040a35c0 msgr2=0x7fda040a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fda040a35c0 0x7fda040a5a80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fda10009ad0 tx=0x7fda10009f90 comp rx=0 tx=0).stop 2024-09-15T22:24:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 -- 172.21.15.5:0/2472411225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda18100de0 msgr2=0x7fda1810c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda18100de0 0x7fda1810c080 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fda140097d0 tx=0x7fda14009c90 comp rx=0 tx=0).stop 2024-09-15T22:24:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 -- 172.21.15.5:0/2472411225 shutdown_connections 2024-09-15T22:24:12.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fda040a35c0 0x7fda040a5a80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fda18105560 0x7fda18117040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fda18100de0 0x7fda1810c080 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 --2- 172.21.15.5:0/2472411225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda18100430 0x7fda1810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:12.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 -- 172.21.15.5:0/2472411225 >> 172.21.15.5:0/2472411225 conn(0x7fda18077230 msgr2=0x7fda18104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:12.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 -- 172.21.15.5:0/2472411225 shutdown_connections 2024-09-15T22:24:12.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:12.108+0000 7fda1f5fe640 1 -- 172.21.15.5:0/2472411225 wait complete. 2024-09-15T22:24:13.513 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:13 smithi005 ceph-mon[30944]: pgmap v479: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.2 MiB/s wr, 476 op/s 2024-09-15T22:24:13.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:13 smithi099 ceph-mon[34892]: pgmap v479: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.2 MiB/s wr, 476 op/s 2024-09-15T22:24:13.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:13 smithi129 ceph-mon[35314]: pgmap v479: 129 pgs: 2 active+clean+snaptrim, 3 active+clean+snaptrim_wait, 124 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 4.2 MiB/s wr, 476 op/s 2024-09-15T22:24:14.663 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 885.0 s, 232.3 tps, lat 185.876 ms stddev 242.333 2024-09-15T22:24:14.907 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '17d4158a', quiesce took 7.5 seconds. Will release after: 10.2 2024-09-15T22:24:15.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:15 smithi099 ceph-mon[34892]: pgmap v480: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-09-15T22:24:15.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:15 smithi129 ceph-mon[35314]: pgmap v480: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-09-15T22:24:15.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:15 smithi005 ceph-mon[30944]: pgmap v480: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.1 MiB/s wr, 448 op/s 2024-09-15T22:24:17.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:17 smithi099 ceph-mon[34892]: pgmap v481: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-09-15T22:24:17.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:17 smithi129 ceph-mon[35314]: pgmap v481: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-09-15T22:24:17.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:17 smithi005 ceph-mon[30944]: pgmap v481: 129 pgs: 129 active+clean; 2.8 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 4.1 MiB/s wr, 431 op/s 2024-09-15T22:24:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:19 smithi099 ceph-mon[34892]: pgmap v482: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-09-15T22:24:19.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:19 smithi099 ceph-mon[34892]: osdmap e107: 12 total, 12 up, 12 in 2024-09-15T22:24:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:19 smithi129 ceph-mon[35314]: pgmap v482: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-09-15T22:24:19.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:19 smithi129 ceph-mon[35314]: osdmap e107: 12 total, 12 up, 12 in 2024-09-15T22:24:19.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:19 smithi005 ceph-mon[30944]: pgmap v482: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.5 MiB/s wr, 275 op/s 2024-09-15T22:24:19.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:19 smithi005 ceph-mon[30944]: osdmap e107: 12 total, 12 up, 12 in 2024-09-15T22:24:19.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 890.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:24:21.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:21 smithi099 ceph-mon[34892]: pgmap v484: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.2 MiB/s wr, 236 op/s 2024-09-15T22:24:21.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:21 smithi129 ceph-mon[35314]: pgmap v484: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.2 MiB/s wr, 236 op/s 2024-09-15T22:24:21.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:21 smithi005 ceph-mon[30944]: pgmap v484: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 2.2 MiB/s wr, 236 op/s 2024-09-15T22:24:23.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:23 smithi099 ceph-mon[34892]: pgmap v485: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1017 KiB/s wr, 111 op/s 2024-09-15T22:24:23.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:23 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:24:23.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:23 smithi129 ceph-mon[35314]: pgmap v485: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1017 KiB/s wr, 111 op/s 2024-09-15T22:24:23.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:23 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:24:23.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:23 smithi005 ceph-mon[30944]: pgmap v485: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1017 KiB/s wr, 111 op/s 2024-09-15T22:24:23.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:23 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:24:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:24 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:24:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:24 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:24:24.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:24 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:24:24.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:24 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:24:24.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:24 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:24:24.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:24 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:24:24.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:24 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:24:24.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:24 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:24:24.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:24 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:24:24.665 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 895.0 s, 0.0 tps, lat 0.000 ms stddev 0.000 2024-09-15T22:24:25.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:24:25.381 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:25.664 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:25 smithi005 ceph-mon[30944]: pgmap v486: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 585 KiB/s wr, 57 op/s 2024-09-15T22:24:25.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:25 smithi099 ceph-mon[34892]: pgmap v486: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 585 KiB/s wr, 57 op/s 2024-09-15T22:24:25.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:25 smithi129 ceph-mon[35314]: pgmap v486: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 585 KiB/s wr, 57 op/s 2024-09-15T22:24:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.880+0000 7f9a8bc13640 1 -- 172.21.15.5:0/3901687648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a84102790 msgr2=0x7f9a841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.880+0000 7f9a8bc13640 1 --2- 172.21.15.5:0/3901687648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a84102790 0x7f9a841067e0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f9a74009900 tx=0x7f9a7402a410 comp rx=0 tx=0).stop 2024-09-15T22:24:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.880+0000 7f9a8bc13640 1 -- 172.21.15.5:0/3901687648 shutdown_connections 2024-09-15T22:24:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.880+0000 7f9a8bc13640 1 --2- 172.21.15.5:0/3901687648 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a84106f10 0x7f9a8410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:25.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.880+0000 7f9a8bc13640 1 --2- 172.21.15.5:0/3901687648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a84102790 0x7f9a841067e0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.880+0000 7f9a8bc13640 1 --2- 172.21.15.5:0/3901687648 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84101de0 0x7f9a841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.880+0000 7f9a8bc13640 1 -- 172.21.15.5:0/3901687648 >> 172.21.15.5:0/3901687648 conn(0x7f9a840fd8c0 msgr2=0x7f9a840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.880+0000 7f9a8bc13640 1 -- 172.21.15.5:0/3901687648 shutdown_connections 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.880+0000 7f9a8bc13640 1 -- 172.21.15.5:0/3901687648 wait complete. 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.881+0000 7f9a8bc13640 1 Processor -- start 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.881+0000 7f9a8bc13640 1 -- start start 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.881+0000 7f9a8bc13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a84101de0 0x7f9a84198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.881+0000 7f9a8bc13640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84102790 0x7f9a84198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.881+0000 7f9a8bc13640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a84106f10 0x7f9a8419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.881+0000 7f9a8bc13640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84111f30 con 0x7f9a84101de0 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.881+0000 7f9a8bc13640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f9a84111db0 con 0x7f9a84106f10 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.881+0000 7f9a8bc13640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f9a841120b0 con 0x7f9a84102790 2024-09-15T22:24:25.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.881+0000 7f9a899af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a84101de0 0x7f9a84198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.881+0000 7f9a899af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a84101de0 0x7f9a84198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52180/0 (socket says 172.21.15.5:52180) 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.881+0000 7f9a899af640 1 -- 172.21.15.5:0/1847288522 learned_addr learned my addr 172.21.15.5:0/1847288522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a891ae640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84102790 0x7f9a84198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a8a1b0640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a84106f10 0x7f9a8419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a899af640 1 -- 172.21.15.5:0/1847288522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84102790 msgr2=0x7f9a84198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a899af640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84102790 0x7f9a84198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a899af640 1 -- 172.21.15.5:0/1847288522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a84106f10 msgr2=0x7f9a8419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a899af640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a84106f10 0x7f9a8419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a899af640 1 -- 172.21.15.5:0/1847288522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8419d3e0 con 0x7f9a84101de0 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a8a1b0640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a84106f10 0x7f9a8419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a899af640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a84101de0 0x7f9a84198430 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000db80 tx=0x7f9a8000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a7affd640 1 -- 172.21.15.5:0/1847288522 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a80017070 con 0x7f9a84101de0 2024-09-15T22:24:25.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a8bc13640 1 -- 172.21.15.5:0/1847288522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8419d6d0 con 0x7f9a84101de0 2024-09-15T22:24:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a7affd640 1 -- 172.21.15.5:0/1847288522 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a80003b70 con 0x7f9a84101de0 2024-09-15T22:24:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a7affd640 1 -- 172.21.15.5:0/1847288522 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a800106c0 con 0x7f9a84101de0 2024-09-15T22:24:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.882+0000 7f9a8bc13640 1 -- 172.21.15.5:0/1847288522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a841a4fb0 con 0x7f9a84101de0 2024-09-15T22:24:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.883+0000 7f9a8bc13640 1 -- 172.21.15.5:0/1847288522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a841021c0 con 0x7f9a84101de0 2024-09-15T22:24:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.884+0000 7f9a7affd640 1 -- 172.21.15.5:0/1847288522 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f9a80004020 con 0x7f9a84101de0 2024-09-15T22:24:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.885+0000 7f9a7affd640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9a700a33b0 0x7f9a700a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.885+0000 7f9a891ae640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9a700a33b0 0x7f9a700a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.885+0000 7f9a7affd640 1 -- 172.21.15.5:0/1847288522 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f9a800d0300 con 0x7f9a84101de0 2024-09-15T22:24:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.886+0000 7f9a891ae640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9a700a33b0 0x7f9a700a5870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9a840fdce0 tx=0x7f9a74005a20 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:25.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:25.888+0000 7f9a7affd640 1 -- 172.21.15.5:0/1847288522 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a80020070 con 0x7f9a84101de0 2024-09-15T22:24:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.102+0000 7f9a8bc13640 1 -- 172.21.15.5:0/1847288522 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a84199790 con 0x7f9a84101de0 2024-09-15T22:24:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.103+0000 7f9a7affd640 1 -- 172.21.15.5:0/1847288522 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f9a800bae20 con 0x7f9a84101de0 2024-09-15T22:24:26.106 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:26.106 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:24:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:24:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 -- 172.21.15.5:0/1847288522 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9a700a33b0 msgr2=0x7f9a700a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9a700a33b0 0x7f9a700a5870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9a840fdce0 tx=0x7f9a74005a20 comp rx=0 tx=0).stop 2024-09-15T22:24:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 -- 172.21.15.5:0/1847288522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a84101de0 msgr2=0x7f9a84198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a84101de0 0x7f9a84198430 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000db80 tx=0x7f9a8000b630 comp rx=0 tx=0).stop 2024-09-15T22:24:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 -- 172.21.15.5:0/1847288522 shutdown_connections 2024-09-15T22:24:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f9a700a33b0 0x7f9a700a5870 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f9a84106f10 0x7f9a8419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f9a84102790 0x7f9a84198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 --2- 172.21.15.5:0/1847288522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9a84101de0 0x7f9a84198430 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 -- 172.21.15.5:0/1847288522 >> 172.21.15.5:0/1847288522 conn(0x7f9a840fd8c0 msgr2=0x7f9a840fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 -- 172.21.15.5:0/1847288522 shutdown_connections 2024-09-15T22:24:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:26.105+0000 7f9a8bc13640 1 -- 172.21.15.5:0/1847288522 wait complete. 2024-09-15T22:24:26.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:26 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1847288522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:26.835 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:24:26.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:26 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1847288522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:26.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:26 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1847288522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:26.920 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id 17d4158a --release --await' 2024-09-15T22:24:26.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id 17d4158a --release --await 2024-09-15T22:24:27.117 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:27.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:27 smithi005 ceph-mon[30944]: pgmap v487: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 262 KiB/s wr, 29 op/s 2024-09-15T22:24:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.607+0000 7f6155182640 1 -- 172.21.15.5:0/2282131692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6150100d00 msgr2=0x7f6150106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.607+0000 7f6155182640 1 --2- 172.21.15.5:0/2282131692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6150100d00 0x7f6150106840 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6144009930 tx=0x7f614402a420 comp rx=0 tx=0).stop 2024-09-15T22:24:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 -- 172.21.15.5:0/2282131692 shutdown_connections 2024-09-15T22:24:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 --2- 172.21.15.5:0/2282131692 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150106f50 0x7f615010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 --2- 172.21.15.5:0/2282131692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6150100d00 0x7f6150106840 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 --2- 172.21.15.5:0/2282131692 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6150100350 0x7f6150100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 -- 172.21.15.5:0/2282131692 >> 172.21.15.5:0/2282131692 conn(0x7f6150077210 msgr2=0x7f61500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 -- 172.21.15.5:0/2282131692 shutdown_connections 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 -- 172.21.15.5:0/2282131692 wait complete. 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 Processor -- start 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 -- start start 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6150100350 0x7f6150198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6150100d00 0x7f61501989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150106f50 0x7f615019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6150111fa0 con 0x7f6150100350 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6150111e20 con 0x7f6150106f50 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.608+0000 7f6155182640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f6150112120 con 0x7f6150100d00 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f614ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6150100350 0x7f6150198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f614ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6150100350 0x7f6150198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52206/0 (socket says 172.21.15.5:52206) 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f614ed9d640 1 -- 172.21.15.5:0/4013868377 learned_addr learned my addr 172.21.15.5:0/4013868377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f614f59e640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150106f50 0x7f615019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f614e59c640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6150100d00 0x7f61501989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f614ed9d640 1 -- 172.21.15.5:0/4013868377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6150100d00 msgr2=0x7f61501989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f614ed9d640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6150100d00 0x7f61501989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f614ed9d640 1 -- 172.21.15.5:0/4013868377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150106f50 msgr2=0x7f615019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f614ed9d640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150106f50 0x7f615019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f614ed9d640 1 -- 172.21.15.5:0/4013868377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615019d430 con 0x7f6150100350 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f614ed9d640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6150100350 0x7f6150198480 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f614000d950 tx=0x7f614000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f6137fff640 1 -- 172.21.15.5:0/4013868377 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140017070 con 0x7f6150100350 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f6137fff640 1 -- 172.21.15.5:0/4013868377 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6140009c30 con 0x7f6150100350 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f6155182640 1 -- 172.21.15.5:0/4013868377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615019d720 con 0x7f6150100350 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f6137fff640 1 -- 172.21.15.5:0/4013868377 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614000f6b0 con 0x7f6150100350 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.609+0000 7f6155182640 1 -- 172.21.15.5:0/4013868377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61501a5000 con 0x7f6150100350 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.612+0000 7f6137fff640 1 -- 172.21.15.5:0/4013868377 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f61400109d0 con 0x7f6150100350 2024-09-15T22:24:27.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.612+0000 7f615755b740 1 Processor -- start 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.612+0000 7f615755b740 1 -- start start 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.612+0000 7f615755b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f93543a0 0x5574f9354780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.612+0000 7f615755b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f9354cc0 0x5574f9363550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.612+0000 7f615755b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9365e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.612+0000 7f615755b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574f92ba2e0 con 0x5574f9363a90 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.612+0000 7f615755b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5574f9261ce0 con 0x5574f9354cc0 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.612+0000 7f615755b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5574f92b62a0 con 0x5574f93543a0 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.612+0000 7f614e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9365e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f614e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9365e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52208/0 (socket says 172.21.15.5:52208) 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f614e59c640 1 -- 172.21.15.5:0/1461887090 learned_addr learned my addr 172.21.15.5:0/1461887090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f6137fff640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f61380a3430 0x7f61380a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f614f59e640 1 --2- 172.21.15.5:0/1461887090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f93543a0 0x5574f9354780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f614ed9d640 1 --2- 172.21.15.5:0/1461887090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f9354cc0 0x5574f9363550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f614e59c640 1 -- 172.21.15.5:0/1461887090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f93543a0 msgr2=0x5574f9354780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f614e59c640 1 --2- 172.21.15.5:0/1461887090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f93543a0 0x5574f9354780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f614e59c640 1 -- 172.21.15.5:0/1461887090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f9354cc0 msgr2=0x5574f9363550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f614e59c640 1 --2- 172.21.15.5:0/1461887090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f9354cc0 0x5574f9363550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f614e59c640 1 -- 172.21.15.5:0/1461887090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f9246440 con 0x5574f9363a90 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f614f59e640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f61380a3430 0x7f61380a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f6137fff640 1 -- 172.21.15.5:0/4013868377 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f61400cfc60 con 0x7f6150100350 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f614e59c640 1 --2- 172.21.15.5:0/1461887090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9365e80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f615006a0a0 tx=0x7f6144031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fe5ab111f23107 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f61357fa640 1 -- 172.21.15.5:0/1461887090 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144040070 con 0x5574f9363a90 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f61357fa640 1 -- 172.21.15.5:0/1461887090 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6144005020 con 0x5574f9363a90 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.613+0000 7f61357fa640 1 -- 172.21.15.5:0/1461887090 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144035040 con 0x5574f9363a90 2024-09-15T22:24:27.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f615755b740 1 -- 172.21.15.5:0/1461887090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 msgr2=0x5574f9365e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f614f59e640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f61380a3430 0x7f61380a58f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6148014610 tx=0x7f6148004ab0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f615755b740 1 --2- 172.21.15.5:0/1461887090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9365e80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f615006a0a0 tx=0x7f6144031630 comp rx=0 tx=0).stop 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f615755b740 1 -- 172.21.15.5:0/1461887090 shutdown_connections 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f615755b740 1 --2- 172.21.15.5:0/1461887090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9365e80 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f615755b740 1 --2- 172.21.15.5:0/1461887090 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f9354cc0 0x5574f9363550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f615755b740 1 --2- 172.21.15.5:0/1461887090 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f93543a0 0x5574f9354780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f615755b740 1 -- 172.21.15.5:0/1461887090 >> 172.21.15.5:0/1461887090 conn(0x5574f9350a30 msgr2=0x5574f9350e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f615755b740 1 -- 172.21.15.5:0/1461887090 shutdown_connections 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f615755b740 1 -- 172.21.15.5:0/1461887090 wait complete. 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f615755b740 1 -- start start 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.614+0000 7f615755b740 1 Processor -- start 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f615755b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f93543a0 0x5574f93632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f615755b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f9354cc0 0x5574f9360560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f615755b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9360aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f615755b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5574f9366660 con 0x5574f9363a90 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f615755b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5574f9275050 con 0x5574f93543a0 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f615755b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5574f93667e0 con 0x5574f9354cc0 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f615755b740 10 client.0 _ll_register_callbacks cb 0x5574f92b5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f614f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9360aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f614f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9360aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52212/0 (socket says 172.21.15.5:52212) 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f614f59e640 1 -- 172.21.15.5:0/779477041 learned_addr learned my addr 172.21.15.5:0/779477041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f614ed9d640 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f93543a0 0x5574f93632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f614e59c640 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f9354cc0 0x5574f9360560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f614f59e640 1 -- 172.21.15.5:0/779477041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f9354cc0 msgr2=0x5574f9360560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f614f59e640 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f9354cc0 0x5574f9360560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f614f59e640 1 -- 172.21.15.5:0/779477041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f93543a0 msgr2=0x5574f93632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:27.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f614f59e640 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f93543a0 0x5574f93632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f614f59e640 1 -- 172.21.15.5:0/779477041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f902b3c0 con 0x5574f9363a90 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f614f59e640 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9360aa0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f6148011ee0 tx=0x7f6148004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba098cdfa9dc24c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f6132ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.615+0000 7f6132ff5640 1 -- 172.21.15.5:0/779477041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148022070 con 0x5574f9363a90 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f6132ff5640 1 -- 172.21.15.5:0/779477041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f6148003b70 con 0x5574f9363a90 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f615755b740 1 -- 172.21.15.5:0/779477041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6144005020 con 0x5574f9363a90 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f6132ff5640 1 -- 172.21.15.5:0/779477041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f614802e050 con 0x5574f9363a90 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f615755b740 10 client.17961 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f615755b740 1 -- 172.21.15.5:0/779477041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5574f9372000 con 0x5574f9363a90 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f6132ff5640 1 -- 172.21.15.5:0/779477041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6144005020 con 0x5574f9363a90 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f6132ff5640 1 -- 172.21.15.5:0/779477041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f614801e410 con 0x5574f9363a90 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f615755b740 10 client.17961 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:27.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f615755b740 10 client.17961 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f615755b740 20 client.17961 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f615755b740 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574f93734b0 0x5574f9375950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f615755b740 4 client.17961 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f615755b740 1 -- 172.21.15.5:0/779477041 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574f92c7850 con 0x5574f93734b0 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.616+0000 7f614ed9d640 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574f93734b0 0x5574f9375950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.617+0000 7f614ed9d640 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574f93734b0 0x5574f9375950 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f45c5e9be75279 server_cookie=d6ea69d013eb7310 in_seq=0 out_seq=0 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.617+0000 7f6132ff5640 10 client.17961 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f6132ff5640 1 -- 172.21.15.5:0/779477041 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574f92c7850 con 0x5574f93734b0 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f6132ff5640 10 client.17961 handle_command_reply: tid=0 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 1 client.17961 shutdown 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 1 -- 172.21.15.5:0/779477041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 msgr2=0x5574f9360aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9360aa0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f6148011ee0 tx=0x7f6148004a40 comp rx=0 tx=0).stop 2024-09-15T22:24:27.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 1 -- 172.21.15.5:0/779477041 shutdown_connections 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9363a90 0x5574f9360aa0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f9354cc0 0x5574f9360560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574f93734b0 0x5574f9375950 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 1 --2- 172.21.15.5:0/779477041 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f93543a0 0x5574f93632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f6132ff5640 0 client.17961 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 1 -- 172.21.15.5:0/779477041 >> 172.21.15.5:0/779477041 conn(0x5574f9350a30 msgr2=0x5574f9356790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 1 -- 172.21.15.5:0/779477041 shutdown_connections 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 1 -- 172.21.15.5:0/779477041 wait complete. 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.618+0000 7f615755b740 20 client.17961 trim_cache size 0 max 16384 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.635+0000 7f615755b740 1 Processor -- start 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.635+0000 7f615755b740 1 -- start start 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.635+0000 7f615755b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f93543a0 0x5574f937b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.635+0000 7f615755b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f937b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.635+0000 7f615755b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f9363a90 0x5574f935d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.635+0000 7f615755b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61501af7a0 con 0x5574f9354cc0 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.635+0000 7f615755b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f61501ad070 con 0x5574f93543a0 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.635+0000 7f615755b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f61501aaee0 con 0x5574f9363a90 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f937b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f937b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52214/0 (socket says 172.21.15.5:52214) 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614e59c640 1 -- 172.21.15.5:0/423538499 learned_addr learned my addr 172.21.15.5:0/423538499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614f59e640 1 --2- 172.21.15.5:0/423538499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f9363a90 0x5574f935d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614e59c640 1 -- 172.21.15.5:0/423538499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f9363a90 msgr2=0x5574f935d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614ed9d640 1 --2- 172.21.15.5:0/423538499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f93543a0 0x5574f937b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614e59c640 1 --2- 172.21.15.5:0/423538499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f9363a90 0x5574f935d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614e59c640 1 -- 172.21.15.5:0/423538499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f93543a0 msgr2=0x5574f937b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614e59c640 1 --2- 172.21.15.5:0/423538499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f93543a0 0x5574f937b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614e59c640 1 -- 172.21.15.5:0/423538499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574f9350860 con 0x5574f9354cc0 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614ed9d640 1 --2- 172.21.15.5:0/423538499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f93543a0 0x5574f937b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f614e59c640 1 --2- 172.21.15.5:0/423538499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f937b760 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6144005940 tx=0x7f6144005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fcaf2928ae3d4c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f6134ff9640 1 -- 172.21.15.5:0/423538499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144040070 con 0x5574f9354cc0 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f6134ff9640 1 -- 172.21.15.5:0/423538499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61440319a0 con 0x5574f9354cc0 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.636+0000 7f6134ff9640 1 -- 172.21.15.5:0/423538499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144035040 con 0x5574f9354cc0 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 -- 172.21.15.5:0/423538499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 msgr2=0x5574f937b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 --2- 172.21.15.5:0/423538499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f937b760 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f6144005940 tx=0x7f6144005fd0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 -- 172.21.15.5:0/423538499 shutdown_connections 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 --2- 172.21.15.5:0/423538499 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f9363a90 0x5574f935d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 --2- 172.21.15.5:0/423538499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f937b760 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 --2- 172.21.15.5:0/423538499 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f93543a0 0x5574f937b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 -- 172.21.15.5:0/423538499 >> 172.21.15.5:0/423538499 conn(0x5574f9350a30 msgr2=0x5574f935ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:27.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 -- 172.21.15.5:0/423538499 shutdown_connections 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 -- 172.21.15.5:0/423538499 wait complete. 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 -- start start 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 Processor -- start 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f93543a0 0x5574f936c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.637+0000 7f615755b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f93675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f615755b740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f9363a90 0x5574f9367b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f615755b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61501ab530 con 0x5574f9354cc0 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f615755b740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f61501ac9a0 con 0x5574f9363a90 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f615755b740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f61501abc00 con 0x5574f93543a0 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f615755b740 10 client.0 _ll_register_callbacks cb 0x7f6148011b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f93675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f93675a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52224/0 (socket says 172.21.15.5:52224) 2024-09-15T22:24:27.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614e59c640 1 -- 172.21.15.5:0/2621796533 learned_addr learned my addr 172.21.15.5:0/2621796533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614ed9d640 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f93543a0 0x5574f936c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614f59e640 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f9363a90 0x5574f9367b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614e59c640 1 -- 172.21.15.5:0/2621796533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f93543a0 msgr2=0x5574f936c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614e59c640 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f93543a0 0x5574f936c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614e59c640 1 -- 172.21.15.5:0/2621796533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f9363a90 msgr2=0x5574f9367b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614e59c640 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f9363a90 0x5574f9367b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614e59c640 1 -- 172.21.15.5:0/2621796533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574f9351bc0 con 0x5574f9354cc0 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614f59e640 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f9363a90 0x5574f9367b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f614e59c640 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f93675a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f614402a940 tx=0x7f614403c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6cc61c2cd2eb08 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f6132ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f6132ff5640 1 -- 172.21.15.5:0/2621796533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144053070 con 0x5574f9354cc0 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f6132ff5640 1 -- 172.21.15.5:0/2621796533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f614403ccf0 con 0x5574f9354cc0 2024-09-15T22:24:27.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.638+0000 7f615755b740 1 -- 172.21.15.5:0/2621796533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61440319a0 con 0x5574f9354cc0 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f6132ff5640 1 -- 172.21.15.5:0/2621796533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f614405f050 con 0x5574f9354cc0 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f615755b740 10 client.17967 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f615755b740 1 -- 172.21.15.5:0/2621796533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5574f93de040 con 0x5574f9354cc0 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f6132ff5640 1 -- 172.21.15.5:0/2621796533 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61440319a0 con 0x5574f9354cc0 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f6132ff5640 1 -- 172.21.15.5:0/2621796533 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f614404f510 con 0x5574f9354cc0 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f615755b740 10 client.17967 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f615755b740 10 client.17967 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f615755b740 20 client.17967 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f615755b740 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574f93de310 0x5574f93e0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f615755b740 4 client.17967 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f615755b740 1 -- 172.21.15.5:0/2621796533 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574f92c1d60 con 0x5574f93de310 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.639+0000 7f614ed9d640 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574f93de310 0x5574f93e0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f614ed9d640 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574f93de310 0x5574f93e0700 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c389f3541d7db92 server_cookie=557606461955007e in_seq=0 out_seq=0 2024-09-15T22:24:27.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f6132ff5640 10 client.17967 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f6132ff5640 1 -- 172.21.15.5:0/2621796533 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5574f92c1d60 con 0x5574f93de310 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f6132ff5640 10 client.17967 handle_command_reply: tid=0 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 1 client.17967 shutdown 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 1 -- 172.21.15.5:0/2621796533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 msgr2=0x5574f93675a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f93675a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f614402a940 tx=0x7f614403c6f0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 1 -- 172.21.15.5:0/2621796533 shutdown_connections 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5574f93de310 0x5574f93e0700 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5574f9363a90 0x5574f9367b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5574f9354cc0 0x5574f93675a0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 1 --2- 172.21.15.5:0/2621796533 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5574f93543a0 0x5574f936c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f6132ff5640 0 client.17967 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 1 -- 172.21.15.5:0/2621796533 >> 172.21.15.5:0/2621796533 conn(0x5574f9350a30 msgr2=0x5574f93625e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 1 -- 172.21.15.5:0/2621796533 shutdown_connections 2024-09-15T22:24:27.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 1 -- 172.21.15.5:0/2621796533 wait complete. 2024-09-15T22:24:27.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.640+0000 7f615755b740 20 client.17967 trim_cache size 0 max 16384 2024-09-15T22:24:27.660 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:24:27.660 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:24:27.660 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:24:27.660 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:24:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 -- 172.21.15.5:0/4013868377 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f61380a3430 msgr2=0x7f61380a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f61380a3430 0x7f61380a58f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6148014610 tx=0x7f6148004ab0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 -- 172.21.15.5:0/4013868377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6150100350 msgr2=0x7f6150198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6150100350 0x7f6150198480 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f614000d950 tx=0x7f614000de10 comp rx=0 tx=0).stop 2024-09-15T22:24:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 -- 172.21.15.5:0/4013868377 shutdown_connections 2024-09-15T22:24:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f61380a3430 0x7f61380a58f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6150106f50 0x7f615019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6150100d00 0x7f61501989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 --2- 172.21.15.5:0/4013868377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6150100350 0x7f6150198480 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 -- 172.21.15.5:0/4013868377 >> 172.21.15.5:0/4013868377 conn(0x7f6150077210 msgr2=0x7f6150105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 -- 172.21.15.5:0/4013868377 shutdown_connections 2024-09-15T22:24:27.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:27.641+0000 7f6155182640 1 -- 172.21.15.5:0/4013868377 wait complete. 2024-09-15T22:24:27.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:27 smithi099 ceph-mon[34892]: pgmap v487: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 262 KiB/s wr, 29 op/s 2024-09-15T22:24:27.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:27 smithi129 ceph-mon[35314]: pgmap v487: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 262 KiB/s wr, 29 op/s 2024-09-15T22:24:28.360 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:24:28.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:24:28.379 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.896+0000 7f1c2f111640 1 -- 172.21.15.5:0/2436344398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c28100a70 msgr2=0x7f1c2810ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.896+0000 7f1c2f111640 1 --2- 172.21.15.5:0/2436344398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c28100a70 0x7f1c2810ea60 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1c20009ca0 tx=0x7f1c2002a8c0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.896+0000 7f1c2f111640 1 -- 172.21.15.5:0/2436344398 shutdown_connections 2024-09-15T22:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.896+0000 7f1c2f111640 1 --2- 172.21.15.5:0/2436344398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c28100a70 0x7f1c2810ea60 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.896+0000 7f1c2f111640 1 --2- 172.21.15.5:0/2436344398 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c281000d0 0x7f1c28100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.896+0000 7f1c2f111640 1 --2- 172.21.15.5:0/2436344398 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c28106ba0 0x7f1c28106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.896+0000 7f1c2f111640 1 -- 172.21.15.5:0/2436344398 >> 172.21.15.5:0/2436344398 conn(0x7f1c28077210 msgr2=0x7f1c280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.896+0000 7f1c2f111640 1 -- 172.21.15.5:0/2436344398 shutdown_connections 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.896+0000 7f1c2f111640 1 -- 172.21.15.5:0/2436344398 wait complete. 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2f111640 1 Processor -- start 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2f111640 1 -- start start 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2f111640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c281000d0 0x7f1c281012b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2f111640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c28100a70 0x7f1c281017f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2f111640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c28106ba0 0x7f1c28101d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2f111640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c28112090 con 0x7f1c28106ba0 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2f111640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c28111f10 con 0x7f1c281000d0 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2f111640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1c28112210 con 0x7f1c28100a70 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2d6ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c28106ba0 0x7f1c28101d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2d6ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c28106ba0 0x7f1c28101d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52240/0 (socket says 172.21.15.5:52240) 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2d6ae640 1 -- 172.21.15.5:0/537686441 learned_addr learned my addr 172.21.15.5:0/537686441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c27fff640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c28100a70 0x7f1c281017f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.897+0000 7f1c2cead640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c281000d0 0x7f1c281012b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.898+0000 7f1c2d6ae640 1 -- 172.21.15.5:0/537686441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c28100a70 msgr2=0x7f1c281017f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.898+0000 7f1c2d6ae640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c28100a70 0x7f1c281017f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.898+0000 7f1c2d6ae640 1 -- 172.21.15.5:0/537686441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c281000d0 msgr2=0x7f1c281012b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.898+0000 7f1c2d6ae640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c281000d0 0x7f1c281012b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.898+0000 7f1c2d6ae640 1 -- 172.21.15.5:0/537686441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c281a6910 con 0x7f1c28106ba0 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.898+0000 7f1c2d6ae640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c28106ba0 0x7f1c28101d30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006010 tx=0x7f1c20003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.898+0000 7f1c25ffb640 1 -- 172.21.15.5:0/537686441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c20031650 con 0x7f1c28106ba0 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.898+0000 7f1c2f111640 1 -- 172.21.15.5:0/537686441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c281a6ba0 con 0x7f1c28106ba0 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.898+0000 7f1c2f111640 1 -- 172.21.15.5:0/537686441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c281a7130 con 0x7f1c28106ba0 2024-09-15T22:24:28.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.898+0000 7f1c25ffb640 1 -- 172.21.15.5:0/537686441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c200317f0 con 0x7f1c28106ba0 2024-09-15T22:24:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.898+0000 7f1c25ffb640 1 -- 172.21.15.5:0/537686441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c200309b0 con 0x7f1c28106ba0 2024-09-15T22:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.901+0000 7f1c25ffb640 1 -- 172.21.15.5:0/537686441 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1c20030c10 con 0x7f1c28106ba0 2024-09-15T22:24:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.901+0000 7f1c25ffb640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c180a3430 0x7f1c180a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c2cead640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c180a3430 0x7f1c180a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c25ffb640 1 -- 172.21.15.5:0/537686441 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f1c200f1450 con 0x7f1c28106ba0 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c314ea740 1 Processor -- start 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c314ea740 1 -- start start 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c2cead640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c180a3430 0x7f1c180a58f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009b50 tx=0x7f1c1c005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36060 0x564423d36440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d36980 0x564423d40d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d41240 0x564423d43630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c314ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564423c9b620 con 0x564423d36060 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c314ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564423c42e50 con 0x564423d36980 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c314ea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564423c975e0 con 0x564423d41240 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.902+0000 7f1c2cead640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36060 0x564423d36440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c2cead640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36060 0x564423d36440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52256/0 (socket says 172.21.15.5:52256) 2024-09-15T22:24:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c2cead640 1 -- 172.21.15.5:0/2188570999 learned_addr learned my addr 172.21.15.5:0/2188570999 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c27fff640 1 --2- 172.21.15.5:0/2188570999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d36980 0x564423d40d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c2d6ae640 1 --2- 172.21.15.5:0/2188570999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d41240 0x564423d43630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c2cead640 1 -- 172.21.15.5:0/2188570999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d41240 msgr2=0x564423d43630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c2cead640 1 --2- 172.21.15.5:0/2188570999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d41240 0x564423d43630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c2cead640 1 -- 172.21.15.5:0/2188570999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d36980 msgr2=0x564423d40d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c2cead640 1 --2- 172.21.15.5:0/2188570999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d36980 0x564423d40d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c2cead640 1 -- 172.21.15.5:0/2188570999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564423c33d40 con 0x564423d36060 2024-09-15T22:24:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c2cead640 1 --2- 172.21.15.5:0/2188570999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36060 0x564423d36440 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c005bc0 tx=0x7f1c1c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbc699e6283f9a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c12ffd640 1 -- 172.21.15.5:0/2188570999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c01e420 con 0x564423d36060 2024-09-15T22:24:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.903+0000 7f1c12ffd640 1 -- 172.21.15.5:0/2188570999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c01eaa0 con 0x564423d36060 2024-09-15T22:24:28.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.904+0000 7f1c12ffd640 1 -- 172.21.15.5:0/2188570999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c026c80 con 0x564423d36060 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.904+0000 7f1c314ea740 1 -- 172.21.15.5:0/2188570999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36060 msgr2=0x564423d36440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.904+0000 7f1c314ea740 1 --2- 172.21.15.5:0/2188570999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36060 0x564423d36440 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c005bc0 tx=0x7f1c1c014000 comp rx=0 tx=0).stop 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.904+0000 7f1c314ea740 1 -- 172.21.15.5:0/2188570999 shutdown_connections 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.904+0000 7f1c314ea740 1 --2- 172.21.15.5:0/2188570999 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d41240 0x564423d43630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.904+0000 7f1c314ea740 1 --2- 172.21.15.5:0/2188570999 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d36980 0x564423d40d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.904+0000 7f1c314ea740 1 --2- 172.21.15.5:0/2188570999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36060 0x564423d36440 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.904+0000 7f1c314ea740 1 -- 172.21.15.5:0/2188570999 >> 172.21.15.5:0/2188570999 conn(0x564423d325f0 msgr2=0x564423d329d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c314ea740 1 -- 172.21.15.5:0/2188570999 shutdown_connections 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c314ea740 1 -- 172.21.15.5:0/2188570999 wait complete. 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c314ea740 1 -- start start 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c314ea740 1 Processor -- start 2024-09-15T22:24:28.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 0x564423d3e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d3e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d41240 0x564423d3ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c314ea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564423d43e10 con 0x564423d36980 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c314ea740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x564423c23f10 con 0x564423d41240 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c314ea740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x564423d43f90 con 0x564423d36060 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c314ea740 10 client.0 _ll_register_callbacks cb 0x564423c47db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.905+0000 7f1c27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d3e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d3e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52274/0 (socket says 172.21.15.5:52274) 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c27fff640 1 -- 172.21.15.5:0/2866654664 learned_addr learned my addr 172.21.15.5:0/2866654664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c2cead640 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 0x564423d3e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c2d6ae640 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d41240 0x564423d3ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c27fff640 1 -- 172.21.15.5:0/2866654664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 msgr2=0x564423d3e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c27fff640 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 0x564423d3e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c27fff640 1 -- 172.21.15.5:0/2866654664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d41240 msgr2=0x564423d3ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c27fff640 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d41240 0x564423d3ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c27fff640 1 -- 172.21.15.5:0/2866654664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644239aa580 con 0x564423d36980 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c27fff640 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d3e680 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400bfb0 tx=0x7f1c140118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70e035975c78383 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c07fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c07fff640 1 -- 172.21.15.5:0/2866654664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14004020 con 0x564423d36980 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c07fff640 1 -- 172.21.15.5:0/2866654664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f1c140041c0 con 0x564423d36980 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.906+0000 7f1c314ea740 1 -- 172.21.15.5:0/2866654664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c1c01eaa0 con 0x564423d36980 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c07fff640 1 -- 172.21.15.5:0/2866654664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c14024050 con 0x564423d36980 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c314ea740 10 client.17979 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c314ea740 1 -- 172.21.15.5:0/2866654664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564423a74960 con 0x564423d36980 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c07fff640 1 -- 172.21.15.5:0/2866654664 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c01eaa0 con 0x564423d36980 2024-09-15T22:24:28.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c07fff640 1 -- 172.21.15.5:0/2866654664 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1c14003950 con 0x564423d36980 2024-09-15T22:24:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c314ea740 10 client.17979 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c314ea740 10 client.17979 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:24:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c314ea740 20 client.17979 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c314ea740 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564423d50f00 0x564423d53310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c314ea740 4 client.17979 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c314ea740 1 -- 172.21.15.5:0/2866654664 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564423ca8b90 con 0x564423d50f00 2024-09-15T22:24:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.907+0000 7f1c2cead640 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564423d50f00 0x564423d53310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.908+0000 7f1c2cead640 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564423d50f00 0x564423d53310 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcffa1d6b1b94054 server_cookie=a67e7b01d9840ae7 in_seq=0 out_seq=0 2024-09-15T22:24:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.908+0000 7f1c07fff640 10 client.17979 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c07fff640 1 -- 172.21.15.5:0/2866654664 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564423ca8b90 con 0x564423d50f00 2024-09-15T22:24:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c07fff640 10 client.17979 handle_command_reply: tid=0 2024-09-15T22:24:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 1 client.17979 shutdown 2024-09-15T22:24:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 1 -- 172.21.15.5:0/2866654664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 msgr2=0x564423d3e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d3e680 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400bfb0 tx=0x7f1c140118d0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 1 -- 172.21.15.5:0/2866654664 shutdown_connections 2024-09-15T22:24:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564423d50f00 0x564423d53310 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d41240 0x564423d3ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c07fff640 0 client.17979 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d3e680 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 1 --2- 172.21.15.5:0/2866654664 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 0x564423d3e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 1 -- 172.21.15.5:0/2866654664 >> 172.21.15.5:0/2866654664 conn(0x564423d325f0 msgr2=0x564423d31750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 1 -- 172.21.15.5:0/2866654664 shutdown_connections 2024-09-15T22:24:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 1 -- 172.21.15.5:0/2866654664 wait complete. 2024-09-15T22:24:28.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.909+0000 7f1c314ea740 20 client.17979 trim_cache size 0 max 16384 2024-09-15T22:24:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.927+0000 7f1c314ea740 1 Processor -- start 2024-09-15T22:24:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.927+0000 7f1c314ea740 1 -- start start 2024-09-15T22:24:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 0x564423d58c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d59160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d41240 0x564423d3b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d59160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c2cead640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 0x564423d58c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d59160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52280/0 (socket says 172.21.15.5:52280) 2024-09-15T22:24:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c2cead640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 0x564423d58c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:42142/0 (socket says 172.21.15.5:42142) 2024-09-15T22:24:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c27fff640 1 -- 172.21.15.5:0/1555588765 learned_addr learned my addr 172.21.15.5:0/1555588765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c314ea740 1 -- 172.21.15.5:0/1555588765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c0128d0 con 0x564423d36980 2024-09-15T22:24:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c314ea740 1 -- 172.21.15.5:0/1555588765 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c0101a0 con 0x564423d41240 2024-09-15T22:24:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c314ea740 1 -- 172.21.15.5:0/1555588765 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c00e010 con 0x564423d36060 2024-09-15T22:24:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c2d6ae640 1 --2- 172.21.15.5:0/1555588765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d41240 0x564423d3b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c2cead640 1 -- 172.21.15.5:0/1555588765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d41240 msgr2=0x564423d3b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c2cead640 1 --2- 172.21.15.5:0/1555588765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d41240 0x564423d3b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c2cead640 1 -- 172.21.15.5:0/1555588765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 msgr2=0x564423d59160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.928+0000 7f1c2cead640 1 --2- 172.21.15.5:0/1555588765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d59160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.929+0000 7f1c2cead640 1 -- 172.21.15.5:0/1555588765 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564423d336a0 con 0x564423d36060 2024-09-15T22:24:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.929+0000 7f1c27fff640 1 --2- 172.21.15.5:0/1555588765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d59160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.929+0000 7f1c2cead640 1 --2- 172.21.15.5:0/1555588765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 0x564423d58c20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009a10 tx=0x7f1c1c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63178cfdb1f627aa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.929+0000 7f1c127fc640 1 -- 172.21.15.5:0/1555588765 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c005240 con 0x564423d36060 2024-09-15T22:24:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.929+0000 7f1c127fc640 1 -- 172.21.15.5:0/1555588765 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c026d10 con 0x564423d36060 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.929+0000 7f1c127fc640 1 -- 172.21.15.5:0/1555588765 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c01e810 con 0x564423d36060 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.930+0000 7f1c314ea740 1 -- 172.21.15.5:0/1555588765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 msgr2=0x564423d58c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.930+0000 7f1c314ea740 1 --2- 172.21.15.5:0/1555588765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 0x564423d58c20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009a10 tx=0x7f1c1c017000 comp rx=0 tx=0).stop 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.930+0000 7f1c314ea740 1 -- 172.21.15.5:0/1555588765 shutdown_connections 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.930+0000 7f1c314ea740 1 --2- 172.21.15.5:0/1555588765 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d41240 0x564423d3b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.930+0000 7f1c314ea740 1 --2- 172.21.15.5:0/1555588765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d59160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.930+0000 7f1c314ea740 1 --2- 172.21.15.5:0/1555588765 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d36060 0x564423d58c20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.930+0000 7f1c314ea740 1 -- 172.21.15.5:0/1555588765 >> 172.21.15.5:0/1555588765 conn(0x564423d325f0 msgr2=0x564423d3ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.930+0000 7f1c314ea740 1 -- 172.21.15.5:0/1555588765 shutdown_connections 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.930+0000 7f1c314ea740 1 -- 172.21.15.5:0/1555588765 wait complete. 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.931+0000 7f1c314ea740 1 -- start start 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.931+0000 7f1c314ea740 1 Processor -- start 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d36060 0x564423d49850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d45b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c314ea740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d41240 0x564423d460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c2cead640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d36060 0x564423d49850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c2cead640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d36060 0x564423d49850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:34462/0 (socket says 172.21.15.5:34462) 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d45b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d45b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52282/0 (socket says 172.21.15.5:52282) 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c2cead640 1 -- 172.21.15.5:0/387300673 learned_addr learned my addr 172.21.15.5:0/387300673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c314ea740 1 -- 172.21.15.5:0/387300673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c00e660 con 0x564423d36980 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c314ea740 1 -- 172.21.15.5:0/387300673 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c00fad0 con 0x564423d36060 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c314ea740 1 -- 172.21.15.5:0/387300673 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c00ed30 con 0x564423d41240 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c2d6ae640 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d41240 0x564423d460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.932+0000 7f1c314ea740 10 client.0 _ll_register_callbacks cb 0x7f1c1400ba50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:28.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.933+0000 7f1c2d6ae640 1 -- 172.21.15.5:0/387300673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d36060 msgr2=0x564423d49850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.933+0000 7f1c2d6ae640 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d36060 0x564423d49850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.933+0000 7f1c2d6ae640 1 -- 172.21.15.5:0/387300673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 msgr2=0x564423d45b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.933+0000 7f1c2d6ae640 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d45b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.933+0000 7f1c2d6ae640 1 -- 172.21.15.5:0/387300673 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564423d467c0 con 0x564423d41240 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.933+0000 7f1c27fff640 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d45b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.933+0000 7f1c2d6ae640 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d41240 0x564423d460d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1c200ee8b0 tx=0x7f1c20030f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38571dec5d7f68f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.933+0000 7f1c07fff640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.933+0000 7f1c07fff640 1 -- 172.21.15.5:0/387300673 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c200d5070 con 0x564423d41240 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.933+0000 7f1c314ea740 1 -- 172.21.15.5:0/387300673 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c1c026d10 con 0x564423d41240 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.933+0000 7f1c07fff640 1 -- 172.21.15.5:0/387300673 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f1c200d6270 con 0x564423d41240 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c07fff640 1 -- 172.21.15.5:0/387300673 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c200bdb40 con 0x564423d41240 2024-09-15T22:24:28.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c314ea740 10 client.27376 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c314ea740 1 -- 172.21.15.5:0/387300673 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564423dbcb90 con 0x564423d41240 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c07fff640 1 -- 172.21.15.5:0/387300673 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c1c026d10 con 0x564423d41240 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c07fff640 1 -- 172.21.15.5:0/387300673 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f1c200f10b0 con 0x564423d41240 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c314ea740 10 client.27376 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c314ea740 10 client.27376 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c314ea740 20 client.27376 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c314ea740 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564423d50f00 0x564423dbf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c314ea740 4 client.27376 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "17d4158a", "release": true, "await": true}] 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c314ea740 1 -- 172.21.15.5:0/387300673 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "17d4158a", "release": true, "await": true}) -- 0x564423ca30a0 con 0x564423d50f00 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.934+0000 7f1c2cead640 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564423d50f00 0x564423dbf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.935+0000 7f1c2cead640 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564423d50f00 0x564423dbf220 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c2a9d4ef2d24af9 server_cookie=da9a3f8f9f208059 in_seq=0 out_seq=0 2024-09-15T22:24:28.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.935+0000 7f1c07fff640 10 client.27376 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c07fff640 1 -- 172.21.15.5:0/387300673 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564423ca30a0 con 0x564423d50f00 2024-09-15T22:24:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c07fff640 10 client.27376 handle_command_reply: tid=0 2024-09-15T22:24:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 1 client.27376 shutdown 2024-09-15T22:24:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 1 -- 172.21.15.5:0/387300673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d41240 msgr2=0x564423d460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d41240 0x564423d460d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1c200ee8b0 tx=0x7f1c20030f00 comp rx=0 tx=0).stop 2024-09-15T22:24:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 1 -- 172.21.15.5:0/387300673 shutdown_connections 2024-09-15T22:24:28.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x564423d41240 0x564423d460d0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564423d36980 0x564423d45b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x564423d50f00 0x564423dbf220 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 1 --2- 172.21.15.5:0/387300673 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x564423d36060 0x564423d49850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c07fff640 0 client.27376 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 1 -- 172.21.15.5:0/387300673 >> 172.21.15.5:0/387300673 conn(0x564423d325f0 msgr2=0x564423d30f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 1 -- 172.21.15.5:0/387300673 shutdown_connections 2024-09-15T22:24:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 1 -- 172.21.15.5:0/387300673 wait complete. 2024-09-15T22:24:28.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.986+0000 7f1c314ea740 20 client.27376 trim_cache size 0 max 16384 2024-09-15T22:24:28.995 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 56, 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "17d4158a": { 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 56, 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:24:28.996 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 137.7, 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:28.997 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:28.998 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:24:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.988+0000 7f1c2f111640 1 -- 172.21.15.5:0/537686441 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c180a3430 msgr2=0x7f1c180a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.988+0000 7f1c2f111640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c180a3430 0x7f1c180a58f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009b50 tx=0x7f1c1c005f90 comp rx=0 tx=0).stop 2024-09-15T22:24:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.988+0000 7f1c2f111640 1 -- 172.21.15.5:0/537686441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c28106ba0 msgr2=0x7f1c28101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.988+0000 7f1c2f111640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c28106ba0 0x7f1c28101d30 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1c20006010 tx=0x7f1c20003ce0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.993+0000 7f1c2f111640 1 -- 172.21.15.5:0/537686441 shutdown_connections 2024-09-15T22:24:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.993+0000 7f1c2f111640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1c180a3430 0x7f1c180a58f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.993+0000 7f1c2f111640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c28106ba0 0x7f1c28101d30 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.993+0000 7f1c2f111640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1c28100a70 0x7f1c281017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.993+0000 7f1c2f111640 1 --2- 172.21.15.5:0/537686441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1c281000d0 0x7f1c281012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.993+0000 7f1c2f111640 1 -- 172.21.15.5:0/537686441 >> 172.21.15.5:0/537686441 conn(0x7f1c28077210 msgr2=0x7f1c280fc720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.993+0000 7f1c2f111640 1 -- 172.21.15.5:0/537686441 shutdown_connections 2024-09-15T22:24:28.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:28.993+0000 7f1c2f111640 1 -- 172.21.15.5:0/537686441 wait complete. 2024-09-15T22:24:29.612 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:29 smithi005 ceph-mon[30944]: pgmap v488: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-09-15T22:24:29.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.610+0000 7f8a5d92e640 1 -- 172.21.15.5:0/2224539575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a5811b140 msgr2=0x7f8a5811d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:29.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.610+0000 7f8a5d92e640 1 --2- 172.21.15.5:0/2224539575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a5811b140 0x7f8a5811d530 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8a5001acb0 tx=0x7f8a5003b920 comp rx=0 tx=0).stop 2024-09-15T22:24:29.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.614+0000 7f8a5d92e640 1 -- 172.21.15.5:0/2224539575 shutdown_connections 2024-09-15T22:24:29.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.614+0000 7f8a5d92e640 1 --2- 172.21.15.5:0/2224539575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a5811b140 0x7f8a5811d530 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:29.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.614+0000 7f8a5d92e640 1 --2- 172.21.15.5:0/2224539575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a58109830 0x7f8a58109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:29.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.614+0000 7f8a5d92e640 1 --2- 172.21.15.5:0/2224539575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a58109450 0x7f8a58110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:29.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.614+0000 7f8a5d92e640 1 -- 172.21.15.5:0/2224539575 >> 172.21.15.5:0/2224539575 conn(0x7f8a5806e140 msgr2=0x7f8a5806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:29.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.615+0000 7f8a5d92e640 1 -- 172.21.15.5:0/2224539575 shutdown_connections 2024-09-15T22:24:29.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.615+0000 7f8a5d92e640 1 -- 172.21.15.5:0/2224539575 wait complete. 2024-09-15T22:24:29.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.615+0000 7f8a5d92e640 1 Processor -- start 2024-09-15T22:24:29.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.616+0000 7f8a5d92e640 1 -- start start 2024-09-15T22:24:29.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.616+0000 7f8a5d92e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a58109450 0x7f8a5811a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:29.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.616+0000 7f8a5d92e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a58109830 0x7f8a58116860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.616+0000 7f8a5d92e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a5811b140 0x7f8a58116da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.616+0000 7f8a5d92e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a58113340 con 0x7f8a58109450 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.616+0000 7f8a5d92e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581131c0 con 0x7f8a58109830 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.616+0000 7f8a5d92e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8a581134c0 con 0x7f8a5811b140 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.616+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a5811b140 0x7f8a58116da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.616+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a5811b140 0x7f8a58116da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:42162/0 (socket says 172.21.15.5:42162) 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.616+0000 7f8a577fe640 1 -- 172.21.15.5:0/3800713961 learned_addr learned my addr 172.21.15.5:0/3800713961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.617+0000 7f8a577fe640 1 -- 172.21.15.5:0/3800713961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a58109830 msgr2=0x7f8a58116860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.617+0000 7f8a577fe640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a58109830 0x7f8a58116860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.617+0000 7f8a577fe640 1 -- 172.21.15.5:0/3800713961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a58109450 msgr2=0x7f8a5811a7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.617+0000 7f8a577fe640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a58109450 0x7f8a5811a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.617+0000 7f8a577fe640 1 -- 172.21.15.5:0/3800713961 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a581174d0 con 0x7f8a5811b140 2024-09-15T22:24:29.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.617+0000 7f8a577fe640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a5811b140 0x7f8a58116da0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8a5003be90 tx=0x7f8a500405a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.620+0000 7f8a5c92c640 1 -- 172.21.15.5:0/3800713961 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50045070 con 0x7f8a5811b140 2024-09-15T22:24:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.620+0000 7f8a5c92c640 1 -- 172.21.15.5:0/3800713961 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a50040740 con 0x7f8a5811b140 2024-09-15T22:24:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.620+0000 7f8a5c92c640 1 -- 172.21.15.5:0/3800713961 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5004b430 con 0x7f8a5811b140 2024-09-15T22:24:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.622+0000 7f8a5d92e640 1 -- 172.21.15.5:0/3800713961 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5807a450 con 0x7f8a5811b140 2024-09-15T22:24:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.622+0000 7f8a5d92e640 1 -- 172.21.15.5:0/3800713961 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5807a8b0 con 0x7f8a5811b140 2024-09-15T22:24:29.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.624+0000 7f8a5d92e640 1 -- 172.21.15.5:0/3800713961 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a58073b00 con 0x7f8a5811b140 2024-09-15T22:24:29.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.631+0000 7f8a5c92c640 1 -- 172.21.15.5:0/3800713961 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8a50048020 con 0x7f8a5811b140 2024-09-15T22:24:29.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.632+0000 7f8a5c92c640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8a400a34f0 0x7f8a400a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:29.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.632+0000 7f8a5c92c640 1 -- 172.21.15.5:0/3800713961 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f8a50101db0 con 0x7f8a5811b140 2024-09-15T22:24:29.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.633+0000 7f8a56ffd640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8a400a34f0 0x7f8a400a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:29.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.633+0000 7f8a56ffd640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8a400a34f0 0x7f8a400a59b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009670 tx=0x7f8a48006bd0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:29.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:29.633+0000 7f8a5c92c640 1 -- 172.21.15.5:0/3800713961 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a50059060 con 0x7f8a5811b140 2024-09-15T22:24:29.660 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 900.0 s, 41.0 tps, lat 6860.352 ms stddev 8457.034 2024-09-15T22:24:29.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:29 smithi099 ceph-mon[34892]: pgmap v488: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-09-15T22:24:29.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:29 smithi129 ceph-mon[35314]: pgmap v488: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 44 KiB/s wr, 9 op/s 2024-09-15T22:24:29.987 INFO:teuthology.orchestra.run.smithi005.stdout:transaction type: 2024-09-15T22:24:29.987 INFO:teuthology.orchestra.run.smithi005.stdout:scaling factor: 32 2024-09-15T22:24:29.987 INFO:teuthology.orchestra.run.smithi005.stdout:query mode: simple 2024-09-15T22:24:29.987 INFO:teuthology.orchestra.run.smithi005.stdout:number of clients: 100 2024-09-15T22:24:29.987 INFO:teuthology.orchestra.run.smithi005.stdout:number of threads: 4 2024-09-15T22:24:29.987 INFO:teuthology.orchestra.run.smithi005.stdout:duration: 900 s 2024-09-15T22:24:29.987 INFO:teuthology.orchestra.run.smithi005.stdout:number of transactions actually processed: 744109 2024-09-15T22:24:29.987 INFO:teuthology.orchestra.run.smithi005.stdout:latency average = 120.953 ms 2024-09-15T22:24:29.988 INFO:teuthology.orchestra.run.smithi005.stdout:latency stddev = 839.116 ms 2024-09-15T22:24:29.988 INFO:teuthology.orchestra.run.smithi005.stdout:tps = 826.487862 (including connections establishing) 2024-09-15T22:24:29.988 INFO:teuthology.orchestra.run.smithi005.stdout:tps = 826.490270 (excluding connections establishing) 2024-09-15T22:24:29.992 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2024-09-15T22:24:30.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.033+0000 7f8a5d92e640 1 -- 172.21.15.5:0/3800713961 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a58109c90 con 0x7f8a5811b140 2024-09-15T22:24:30.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.039+0000 7f8a5c92c640 1 -- 172.21.15.5:0/3800713961 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f8a58109c90 con 0x7f8a5811b140 2024-09-15T22:24:30.044 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:30.044 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:24:30.044 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:24:30.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.044+0000 7f8a5d92e640 1 -- 172.21.15.5:0/3800713961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8a400a34f0 msgr2=0x7f8a400a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:30.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.044+0000 7f8a5d92e640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8a400a34f0 0x7f8a400a59b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009670 tx=0x7f8a48006bd0 comp rx=0 tx=0).stop 2024-09-15T22:24:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.045+0000 7f8a5d92e640 1 -- 172.21.15.5:0/3800713961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a5811b140 msgr2=0x7f8a58116da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.045+0000 7f8a5d92e640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a5811b140 0x7f8a58116da0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8a5003be90 tx=0x7f8a500405a0 comp rx=0 tx=0).stop 2024-09-15T22:24:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.045+0000 7f8a5d92e640 1 -- 172.21.15.5:0/3800713961 shutdown_connections 2024-09-15T22:24:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.045+0000 7f8a5d92e640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8a400a34f0 0x7f8a400a59b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.045+0000 7f8a5d92e640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8a5811b140 0x7f8a58116da0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.045+0000 7f8a5d92e640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8a58109830 0x7f8a58116860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.045+0000 7f8a5d92e640 1 --2- 172.21.15.5:0/3800713961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a58109450 0x7f8a5811a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:30.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.045+0000 7f8a5d92e640 1 -- 172.21.15.5:0/3800713961 >> 172.21.15.5:0/3800713961 conn(0x7f8a5806e140 msgr2=0x7f8a58070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:30.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.050+0000 7f8a5d92e640 1 -- 172.21.15.5:0/3800713961 shutdown_connections 2024-09-15T22:24:30.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:30.053+0000 7f8a5d92e640 1 -- 172.21.15.5:0/3800713961 wait complete. 2024-09-15T22:24:30.381 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '17d4158a', total seconds elapsed: 23.0 2024-09-15T22:24:30.381 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-15T22:24:30.381 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:24:30.725 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:30 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3800713961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:30.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:30 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3800713961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:30.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:30 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3800713961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:31.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:24:31.530 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:31.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:31 smithi005 ceph-mon[30944]: pgmap v489: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 56 KiB/s wr, 11 op/s 2024-09-15T22:24:31.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:31 smithi099 ceph-mon[34892]: pgmap v489: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 56 KiB/s wr, 11 op/s 2024-09-15T22:24:31.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:31 smithi129 ceph-mon[35314]: pgmap v489: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 56 KiB/s wr, 11 op/s 2024-09-15T22:24:32.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.250+0000 7fdb02c04640 1 -- 172.21.15.5:0/1344192778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdafc074990 msgr2=0x7fdafc07b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:32.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.250+0000 7fdb02c04640 1 --2- 172.21.15.5:0/1344192778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdafc074990 0x7fdafc07b3c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400b880 tx=0x7fdaf4005f90 comp rx=0 tx=0).stop 2024-09-15T22:24:32.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.250+0000 7fdb02c04640 1 -- 172.21.15.5:0/1344192778 shutdown_connections 2024-09-15T22:24:32.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.250+0000 7fdb02c04640 1 --2- 172.21.15.5:0/1344192778 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdafc074990 0x7fdafc07b3c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:32.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.250+0000 7fdb02c04640 1 --2- 172.21.15.5:0/1344192778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc109e00 0x7fdafc0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.250+0000 7fdb02c04640 1 --2- 172.21.15.5:0/1344192778 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdafc109450 0x7fdafc109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.250+0000 7fdb02c04640 1 -- 172.21.15.5:0/1344192778 >> 172.21.15.5:0/1344192778 conn(0x7fdafc06e140 msgr2=0x7fdafc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.251+0000 7fdb02c04640 1 -- 172.21.15.5:0/1344192778 shutdown_connections 2024-09-15T22:24:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.251+0000 7fdb02c04640 1 -- 172.21.15.5:0/1344192778 wait complete. 2024-09-15T22:24:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.252+0000 7fdb02c04640 1 Processor -- start 2024-09-15T22:24:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.252+0000 7fdb02c04640 1 -- start start 2024-09-15T22:24:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.253+0000 7fdb02c04640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdafc074990 0x7fdafc07b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.253+0000 7fdb02c04640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc109450 0x7fdafc0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.253+0000 7fdb02c04640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdafc109e00 0x7fdafc0766e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:32.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.253+0000 7fdb02c04640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc07e730 con 0x7fdafc109450 2024-09-15T22:24:32.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.253+0000 7fdb02c04640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc07e5b0 con 0x7fdafc109e00 2024-09-15T22:24:32.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.253+0000 7fdb02c04640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdafc07e8b0 con 0x7fdafc074990 2024-09-15T22:24:32.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.255+0000 7fdb011a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdafc109e00 0x7fdafc0766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:32.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.255+0000 7fdb011a1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdafc109e00 0x7fdafc0766e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:34492/0 (socket says 172.21.15.5:34492) 2024-09-15T22:24:32.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.255+0000 7fdb011a1640 1 -- 172.21.15.5:0/3343360504 learned_addr learned my addr 172.21.15.5:0/3343360504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:32.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.255+0000 7fdb009a0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdafc074990 0x7fdafc07b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.255+0000 7fdafbfff640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc109450 0x7fdafc0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.255+0000 7fdb011a1640 1 -- 172.21.15.5:0/3343360504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdafc074990 msgr2=0x7fdafc07b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.255+0000 7fdb011a1640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdafc074990 0x7fdafc07b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.255+0000 7fdb011a1640 1 -- 172.21.15.5:0/3343360504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc109450 msgr2=0x7fdafc0761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.255+0000 7fdb011a1640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc109450 0x7fdafc0761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:32.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.255+0000 7fdb011a1640 1 -- 172.21.15.5:0/3343360504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdafc076fa0 con 0x7fdafc109e00 2024-09-15T22:24:32.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.255+0000 7fdb011a1640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdafc109e00 0x7fdafc0766e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4034040 tx=0x7fdaf402acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.260+0000 7fdaf9ffb640 1 -- 172.21.15.5:0/3343360504 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf403f020 con 0x7fdafc109e00 2024-09-15T22:24:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.260+0000 7fdb02c04640 1 -- 172.21.15.5:0/3343360504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdafc1ccfb0 con 0x7fdafc109e00 2024-09-15T22:24:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.261+0000 7fdb02c04640 1 -- 172.21.15.5:0/3343360504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdafc1cd400 con 0x7fdafc109e00 2024-09-15T22:24:32.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.261+0000 7fdaf9ffb640 1 -- 172.21.15.5:0/3343360504 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf4005550 con 0x7fdafc109e00 2024-09-15T22:24:32.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.261+0000 7fdaf9ffb640 1 -- 172.21.15.5:0/3343360504 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf40035f0 con 0x7fdafc109e00 2024-09-15T22:24:32.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.265+0000 7fdb02c04640 1 -- 172.21.15.5:0/3343360504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae8005290 con 0x7fdafc109e00 2024-09-15T22:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.272+0000 7fdaf9ffb640 1 -- 172.21.15.5:0/3343360504 <== mon.2 v2:172.21.15.99:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fdaf4042020 con 0x7fdafc109e00 2024-09-15T22:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.273+0000 7fdaf9ffb640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdae40a3580 0x7fdae40a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:32.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.273+0000 7fdaf9ffb640 1 -- 172.21.15.5:0/3343360504 <== mon.2 v2:172.21.15.99:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7fdaf40f0180 con 0x7fdafc109e00 2024-09-15T22:24:32.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.281+0000 7fdaf9ffb640 1 -- 172.21.15.5:0/3343360504 <== mon.2 v2:172.21.15.99:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdaf40bc810 con 0x7fdafc109e00 2024-09-15T22:24:32.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.283+0000 7fdb009a0640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdae40a3580 0x7fdae40a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:32.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.287+0000 7fdb009a0640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdae40a3580 0x7fdae40a5a40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdaec005c40 tx=0x7fdaec005bd0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.712+0000 7fdb02c04640 1 -- 172.21.15.5:0/3343360504 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdae8005850 con 0x7fdafc109e00 2024-09-15T22:24:32.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.716+0000 7fdaf9ffb640 1 -- 172.21.15.5:0/3343360504 <== mon.2 v2:172.21.15.99:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fdaf40bc110 con 0x7fdafc109e00 2024-09-15T22:24:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:32.719 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:24:32.720 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:24:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.723+0000 7fdb02c04640 1 -- 172.21.15.5:0/3343360504 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdae40a3580 msgr2=0x7fdae40a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.723+0000 7fdb02c04640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdae40a3580 0x7fdae40a5a40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdaec005c40 tx=0x7fdaec005bd0 comp rx=0 tx=0).stop 2024-09-15T22:24:32.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.723+0000 7fdb02c04640 1 -- 172.21.15.5:0/3343360504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdafc109e00 msgr2=0x7fdafc0766e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.723+0000 7fdb02c04640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdafc109e00 0x7fdafc0766e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4034040 tx=0x7fdaf402acc0 comp rx=0 tx=0).stop 2024-09-15T22:24:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.723+0000 7fdb02c04640 1 -- 172.21.15.5:0/3343360504 shutdown_connections 2024-09-15T22:24:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.723+0000 7fdb02c04640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdae40a3580 0x7fdae40a5a40 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdaec005c40 tx=0x7fdaec005bd0 comp rx=0 tx=0).stop 2024-09-15T22:24:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.724+0000 7fdb02c04640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdafc109e00 0x7fdafc0766e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.724+0000 7fdb02c04640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc109450 0x7fdafc0761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.724+0000 7fdb02c04640 1 --2- 172.21.15.5:0/3343360504 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdafc074990 0x7fdafc07b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:32.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.724+0000 7fdb02c04640 1 -- 172.21.15.5:0/3343360504 >> 172.21.15.5:0/3343360504 conn(0x7fdafc06e140 msgr2=0x7fdafc0723b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.736+0000 7fdb02c04640 1 -- 172.21.15.5:0/3343360504 shutdown_connections 2024-09-15T22:24:32.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:32.737+0000 7fdb02c04640 1 -- 172.21.15.5:0/3343360504 wait complete. 2024-09-15T22:24:33.651 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-15T22:24:33.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-15T22:24:33.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:33 smithi005 ceph-mon[30944]: pgmap v490: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-15T22:24:33.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:33 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3343360504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:33.740 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:24:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.065+0000 7f3200d40640 1 -- 172.21.15.5:0/4171203700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31fc11b140 msgr2=0x7f31fc11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.065+0000 7f3200d40640 1 --2- 172.21.15.5:0/4171203700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31fc11b140 0x7f31fc11d530 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009860 tx=0x7f31ec02a380 comp rx=0 tx=0).stop 2024-09-15T22:24:33.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.069+0000 7f3200d40640 1 -- 172.21.15.5:0/4171203700 shutdown_connections 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.069+0000 7f3200d40640 1 --2- 172.21.15.5:0/4171203700 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31fc11b140 0x7f31fc11d530 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.069+0000 7f3200d40640 1 --2- 172.21.15.5:0/4171203700 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31fc109830 0x7f31fc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.069+0000 7f3200d40640 1 --2- 172.21.15.5:0/4171203700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31fc109450 0x7f31fc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.069+0000 7f3200d40640 1 -- 172.21.15.5:0/4171203700 >> 172.21.15.5:0/4171203700 conn(0x7f31fc06e140 msgr2=0x7f31fc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.078+0000 7f3200d40640 1 -- 172.21.15.5:0/4171203700 shutdown_connections 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.079+0000 7f3200d40640 1 -- 172.21.15.5:0/4171203700 wait complete. 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.080+0000 7f3200d40640 1 Processor -- start 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.084+0000 7f3200d40640 1 -- start start 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.084+0000 7f3200d40640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31fc109450 0x7f31fc1186b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.084+0000 7f3200d40640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31fc109830 0x7f31fc118bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.084+0000 7f3200d40640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31fc11b140 0x7f31fc111920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.084+0000 7f3200d40640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc120de0 con 0x7f31fc109830 2024-09-15T22:24:33.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.084+0000 7f3200d40640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc120c60 con 0x7f31fc11b140 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.084+0000 7f3200d40640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc120f60 con 0x7f31fc109450 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.084+0000 7f31fa59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31fc109450 0x7f31fc1186b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.084+0000 7f31fa59c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31fc109450 0x7f31fc1186b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:42198/0 (socket says 172.21.15.5:42198) 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.084+0000 7f31fa59c640 1 -- 172.21.15.5:0/838329220 learned_addr learned my addr 172.21.15.5:0/838329220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.085+0000 7f31fad9d640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31fc11b140 0x7f31fc111920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.085+0000 7f31f9d9b640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31fc109830 0x7f31fc118bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.085+0000 7f31fa59c640 1 -- 172.21.15.5:0/838329220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31fc11b140 msgr2=0x7f31fc111920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.085+0000 7f31fa59c640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31fc11b140 0x7f31fc111920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.085+0000 7f31fa59c640 1 -- 172.21.15.5:0/838329220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31fc109830 msgr2=0x7f31fc118bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.085+0000 7f31fa59c640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31fc109830 0x7f31fc118bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.085+0000 7f31fa59c640 1 -- 172.21.15.5:0/838329220 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc1121e0 con 0x7f31fc109450 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.085+0000 7f31f9d9b640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31fc109830 0x7f31fc118bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.087+0000 7f31fa59c640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31fc109450 0x7f31fc1186b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f31f000e9c0 tx=0x7f31f000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:33.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.094+0000 7f31eb7fe640 1 -- 172.21.15.5:0/838329220 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f000ccc0 con 0x7f31fc109450 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.094+0000 7f31eb7fe640 1 -- 172.21.15.5:0/838329220 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31f0005500 con 0x7f31fc109450 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.094+0000 7f3200d40640 1 -- 172.21.15.5:0/838329220 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31fc1124d0 con 0x7f31fc109450 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.094+0000 7f3200d40640 1 -- 172.21.15.5:0/838329220 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31fc1164c0 con 0x7f31fc109450 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.094+0000 7f31eb7fe640 1 -- 172.21.15.5:0/838329220 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0010430 con 0x7f31fc109450 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.098+0000 7f3203119740 1 Processor -- start 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.098+0000 7f3203119740 1 -- start start 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.098+0000 7f3203119740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11bdb40 0x5604e11bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.098+0000 7f3203119740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bc5d0 0x5604e11bca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.098+0000 7f3203119740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11c7370 0x5604e11c9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.098+0000 7f3203119740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5604e11222f0 con 0x5604e11c7370 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.098+0000 7f3203119740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5604e10c9cf0 con 0x5604e11bc5d0 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.098+0000 7f3203119740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5604e111e2b0 con 0x5604e11bdb40 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.100+0000 7f31fa59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bc5d0 0x5604e11bca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.100+0000 7f31fa59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bc5d0 0x5604e11bca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:34528/0 (socket says 172.21.15.5:34528) 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.100+0000 7f31fa59c640 1 -- 172.21.15.5:0/1341583103 learned_addr learned my addr 172.21.15.5:0/1341583103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.101+0000 7f31f9d9b640 1 --2- 172.21.15.5:0/1341583103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11c7370 0x5604e11c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.103+0000 7f31fa59c640 1 -- 172.21.15.5:0/1341583103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11bdb40 msgr2=0x5604e11bc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.103+0000 7f31eb7fe640 1 -- 172.21.15.5:0/838329220 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f31f00046c0 con 0x7f31fc109450 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.103+0000 7f31fa59c640 1 --2- 172.21.15.5:0/1341583103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11bdb40 0x5604e11bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.103+0000 7f31fa59c640 1 -- 172.21.15.5:0/1341583103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11c7370 msgr2=0x5604e11c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.103+0000 7f31fa59c640 1 --2- 172.21.15.5:0/1341583103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11c7370 0x5604e11c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.103+0000 7f31fa59c640 1 -- 172.21.15.5:0/1341583103 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e10ae3a0 con 0x5604e11bc5d0 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.106+0000 7f31fa59c640 1 --2- 172.21.15.5:0/1341583103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bc5d0 0x5604e11bca10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f31f00d43a0 tx=0x7f31f00d77a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93cf4262c8e43cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.107+0000 7f31eb7fe640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e40a35c0 0x7f31e40a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.108+0000 7f31fad9d640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e40a35c0 0x7f31e40a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.108+0000 7f31eb7fe640 1 -- 172.21.15.5:0/838329220 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f31f00d70b0 con 0x7f31fc109450 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.108+0000 7f31e8ff9640 1 -- 172.21.15.5:0/1341583103 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00dd070 con 0x5604e11bc5d0 2024-09-15T22:24:33.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.109+0000 7f31e8ff9640 1 -- 172.21.15.5:0/1341583103 <== mon.2 v2:172.21.15.99:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31f0047e50 con 0x5604e11bc5d0 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.110+0000 7f3203119740 1 -- 172.21.15.5:0/1341583103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bc5d0 msgr2=0x5604e11bca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.110+0000 7f3203119740 1 --2- 172.21.15.5:0/1341583103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bc5d0 0x5604e11bca10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f31f00d43a0 tx=0x7f31f00d77a0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.111+0000 7f31fad9d640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e40a35c0 0x7f31e40a5a80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009460 tx=0x7f31ec009490 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.116+0000 7f3203119740 1 -- 172.21.15.5:0/1341583103 shutdown_connections 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.116+0000 7f3203119740 1 --2- 172.21.15.5:0/1341583103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11c7370 0x5604e11c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.116+0000 7f3203119740 1 --2- 172.21.15.5:0/1341583103 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bc5d0 0x5604e11bca10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.116+0000 7f3203119740 1 --2- 172.21.15.5:0/1341583103 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11bdb40 0x5604e11bc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.116+0000 7f3203119740 1 -- 172.21.15.5:0/1341583103 >> 172.21.15.5:0/1341583103 conn(0x5604e1185310 msgr2=0x5604e11b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.116+0000 7f3203119740 1 -- 172.21.15.5:0/1341583103 shutdown_connections 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.117+0000 7f3203119740 1 -- 172.21.15.5:0/1341583103 wait complete. 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.117+0000 7f3203119740 1 -- start start 2024-09-15T22:24:33.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.118+0000 7f3203119740 1 Processor -- start 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.120+0000 7f3203119740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11bc5d0 0x5604e11c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.120+0000 7f3203119740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bdb40 0x5604e11c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.120+0000 7f3203119740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 0x5604e11c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.120+0000 7f3203119740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5604e11ca0c0 con 0x5604e11bc5d0 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.120+0000 7f3203119740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x5604e10dd060 con 0x5604e11bdb40 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.120+0000 7f3203119740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5604e11ca240 con 0x5604e11c7370 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.120+0000 7f3203119740 10 client.0 _ll_register_callbacks cb 0x5604e0e308c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.120+0000 7f31fad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 0x5604e11c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.120+0000 7f31fad9d640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 0x5604e11c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:42220/0 (socket says 172.21.15.5:42220) 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.120+0000 7f31fad9d640 1 -- 172.21.15.5:0/2657885167 learned_addr learned my addr 172.21.15.5:0/2657885167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.120+0000 7f31f9d9b640 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bdb40 0x5604e11c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.125+0000 7f31fad9d640 1 -- 172.21.15.5:0/2657885167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bdb40 msgr2=0x5604e11c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.125+0000 7f31fad9d640 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bdb40 0x5604e11c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.125+0000 7f31fad9d640 1 -- 172.21.15.5:0/2657885167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11bc5d0 msgr2=0x5604e11c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:33.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.125+0000 7f31fad9d640 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11bc5d0 0x5604e11c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.125+0000 7f31fad9d640 1 -- 172.21.15.5:0/2657885167 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e0e933c0 con 0x5604e11c7370 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.128+0000 7f31fad9d640 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 0x5604e11c44d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f31ec034000 tx=0x7f31ec02a880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51bf197f6b4af7ce server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.128+0000 7f31e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.133+0000 7f31e1ffb640 1 -- 172.21.15.5:0/2657885167 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec042070 con 0x5604e11c7370 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.133+0000 7f31e1ffb640 1 -- 172.21.15.5:0/2657885167 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f31ec0037f0 con 0x5604e11c7370 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.136+0000 7f3203119740 1 -- 172.21.15.5:0/2657885167 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31f0047e50 con 0x5604e11c7370 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.142+0000 7f31e1ffb640 1 -- 172.21.15.5:0/2657885167 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f31ec037040 con 0x5604e11c7370 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.144+0000 7f3203119740 10 client.27391 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.144+0000 7f3203119740 1 -- 172.21.15.5:0/2657885167 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5604e11d5be0 con 0x5604e11c7370 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.145+0000 7f31e1ffb640 1 -- 172.21.15.5:0/2657885167 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31f0047e50 con 0x5604e11c7370 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.145+0000 7f31e1ffb640 1 -- 172.21.15.5:0/2657885167 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f31ec032440 con 0x5604e11c7370 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.145+0000 7f3203119740 10 client.27391 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.145+0000 7f3203119740 10 client.27391 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.145+0000 7f3203119740 20 client.27391 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.145+0000 7f3203119740 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x5604e11d7570 0x5604e11d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.145+0000 7f3203119740 4 client.27391 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.145+0000 7f3203119740 1 -- 172.21.15.5:0/2657885167 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604e112f860 con 0x5604e11d7570 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.146+0000 7f31fa59c640 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x5604e11d7570 0x5604e11d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.157+0000 7f31fa59c640 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x5604e11d7570 0x5604e11d9a10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6241329743bd91ad server_cookie=62ed6c1b96f5002b in_seq=0 out_seq=0 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.157+0000 7f31e1ffb640 10 client.27391 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.161+0000 7f31e1ffb640 1 -- 172.21.15.5:0/2657885167 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604e112f860 con 0x5604e11d7570 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.161+0000 7f31e1ffb640 10 client.27391 handle_command_reply: tid=0 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.162+0000 7f3203119740 1 client.27391 shutdown 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.163+0000 7f3203119740 1 -- 172.21.15.5:0/2657885167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 msgr2=0x5604e11c44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.163+0000 7f3203119740 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 0x5604e11c44d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f31ec034000 tx=0x7f31ec02a880 comp rx=0 tx=0).stop 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.163+0000 7f3203119740 1 -- 172.21.15.5:0/2657885167 shutdown_connections 2024-09-15T22:24:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.163+0000 7f3203119740 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x5604e11d7570 0x5604e11d9a10 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.163+0000 7f3203119740 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 0x5604e11c44d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.163+0000 7f3203119740 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bdb40 0x5604e11c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.163+0000 7f3203119740 1 --2- 172.21.15.5:0/2657885167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11bc5d0 0x5604e11c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.163+0000 7f3203119740 1 -- 172.21.15.5:0/2657885167 >> 172.21.15.5:0/2657885167 conn(0x5604e1185310 msgr2=0x5604e11be9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.164+0000 7f31e1ffb640 0 client.27391 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.164+0000 7f3203119740 1 -- 172.21.15.5:0/2657885167 shutdown_connections 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.164+0000 7f3203119740 1 -- 172.21.15.5:0/2657885167 wait complete. 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.165+0000 7f3203119740 20 client.27391 trim_cache size 0 max 16384 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.195+0000 7f3203119740 1 Processor -- start 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.195+0000 7f3203119740 1 -- start start 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.195+0000 7f3203119740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bc5d0 0x5604e11b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.195+0000 7f3203119740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11bdb40 0x5604e11d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.195+0000 7f3203119740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 0x5604e11bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.195+0000 7f3203119740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc012a30 con 0x5604e11bdb40 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.195+0000 7f3203119740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc010300 con 0x5604e11bc5d0 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.195+0000 7f3203119740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc00e170 con 0x5604e11c7370 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.196+0000 7f31f9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11bdb40 0x5604e11d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.196+0000 7f31f9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11bdb40 0x5604e11d3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52342/0 (socket says 172.21.15.5:52342) 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.196+0000 7f31f9d9b640 1 -- 172.21.15.5:0/933731828 learned_addr learned my addr 172.21.15.5:0/933731828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.196+0000 7f31fad9d640 1 --2- 172.21.15.5:0/933731828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 0x5604e11bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.197+0000 7f31fad9d640 1 -- 172.21.15.5:0/933731828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bc5d0 msgr2=0x5604e11b8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.197+0000 7f31fad9d640 1 --2- 172.21.15.5:0/933731828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bc5d0 0x5604e11b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.197+0000 7f31fad9d640 1 -- 172.21.15.5:0/933731828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11bdb40 msgr2=0x5604e11d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.197+0000 7f31fad9d640 1 --2- 172.21.15.5:0/933731828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11bdb40 0x5604e11d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.197+0000 7f31fad9d640 1 -- 172.21.15.5:0/933731828 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e11bd760 con 0x5604e11c7370 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.197+0000 7f31fad9d640 1 --2- 172.21.15.5:0/933731828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 0x5604e11bf1b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f31ec02a970 tx=0x7f31ec0096b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd70764def1316e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.197+0000 7f31e3fff640 1 -- 172.21.15.5:0/933731828 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec042040 con 0x5604e11c7370 2024-09-15T22:24:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.197+0000 7f31e3fff640 1 -- 172.21.15.5:0/933731828 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31ec032e50 con 0x5604e11c7370 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.198+0000 7f3203119740 1 -- 172.21.15.5:0/933731828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 msgr2=0x5604e11bf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.198+0000 7f3203119740 1 --2- 172.21.15.5:0/933731828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 0x5604e11bf1b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f31ec02a970 tx=0x7f31ec0096b0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.199+0000 7f3203119740 1 -- 172.21.15.5:0/933731828 shutdown_connections 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.199+0000 7f3203119740 1 --2- 172.21.15.5:0/933731828 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11c7370 0x5604e11bf1b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.199+0000 7f3203119740 1 --2- 172.21.15.5:0/933731828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e11bdb40 0x5604e11d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.199+0000 7f3203119740 1 --2- 172.21.15.5:0/933731828 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11bc5d0 0x5604e11b8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.199+0000 7f3203119740 1 -- 172.21.15.5:0/933731828 >> 172.21.15.5:0/933731828 conn(0x5604e1185310 msgr2=0x5604e11b8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.202+0000 7f3203119740 1 -- 172.21.15.5:0/933731828 shutdown_connections 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.202+0000 7f3203119740 1 -- 172.21.15.5:0/933731828 wait complete. 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.203+0000 7f3203119740 1 -- start start 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.203+0000 7f3203119740 1 Processor -- start 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.203+0000 7f3203119740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11cad10 0x5604e11cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.203+0000 7f3203119740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11cb630 0x5604e123a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.203+0000 7f3203119740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e123a920 0x5604e123ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.203+0000 7f3203119740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc00e7c0 con 0x5604e123a920 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.203+0000 7f3203119740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc00fc30 con 0x5604e11cb630 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.203+0000 7f3203119740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc00ee90 con 0x5604e11cad10 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.203+0000 7f3203119740 10 client.0 _ll_register_callbacks cb 0x7f31ec035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.204+0000 7f31f9d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11cb630 0x5604e123a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.204+0000 7f31f9d9b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11cb630 0x5604e123a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:34554/0 (socket says 172.21.15.5:34554) 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.204+0000 7f31f9d9b640 1 -- 172.21.15.5:0/442522113 learned_addr learned my addr 172.21.15.5:0/442522113 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.204+0000 7f31fad9d640 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e123a920 0x5604e123ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.205+0000 7f31f9d9b640 1 -- 172.21.15.5:0/442522113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11cad10 msgr2=0x5604e11cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.205+0000 7f31f9d9b640 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11cad10 0x5604e11cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.205+0000 7f31f9d9b640 1 -- 172.21.15.5:0/442522113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e123a920 msgr2=0x5604e123ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.205+0000 7f31f9d9b640 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e123a920 0x5604e123ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.205+0000 7f31f9d9b640 1 -- 172.21.15.5:0/442522113 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e1241540 con 0x5604e11cb630 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.207+0000 7f31f9d9b640 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11cb630 0x5604e123a320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f31fc112f90 tx=0x7f31f400f7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94e5059c3c46559 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.207+0000 7f31e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.209+0000 7f31e1ffb640 1 -- 172.21.15.5:0/442522113 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f401f060 con 0x5604e11cb630 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.209+0000 7f3203119740 1 -- 172.21.15.5:0/442522113 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5604e1129d70 con 0x5604e11cb630 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.209+0000 7f31e1ffb640 1 -- 172.21.15.5:0/442522113 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f31f401a5f0 con 0x5604e11cb630 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.209+0000 7f31e1ffb640 1 -- 172.21.15.5:0/442522113 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f31f4026050 con 0x5604e11cb630 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.211+0000 7f3203119740 10 client.27470 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.211+0000 7f3203119740 1 -- 172.21.15.5:0/442522113 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5604e12428d0 con 0x5604e11cb630 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.215+0000 7f31e1ffb640 1 -- 172.21.15.5:0/442522113 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5604e1129d70 con 0x5604e11cb630 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.215+0000 7f31e1ffb640 1 -- 172.21.15.5:0/442522113 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f31f4019070 con 0x5604e11cb630 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.215+0000 7f3203119740 10 client.27470 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.215+0000 7f3203119740 10 client.27470 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.215+0000 7f3203119740 20 client.27470 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.215+0000 7f3203119740 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x5604e12430b0 0x5604e12454d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.215+0000 7f3203119740 4 client.27470 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.215+0000 7f3203119740 1 -- 172.21.15.5:0/442522113 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604e10a6170 con 0x5604e12430b0 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.215+0000 7f31fa59c640 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x5604e12430b0 0x5604e12454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.227+0000 7f31fa59c640 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x5604e12430b0 0x5604e12454d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea01b920063ea56b server_cookie=134ce762eea2e111 in_seq=0 out_seq=0 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.228+0000 7f31e1ffb640 10 client.27470 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.232+0000 7f31e1ffb640 1 -- 172.21.15.5:0/442522113 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5604e10a6170 con 0x5604e12430b0 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.232+0000 7f31e1ffb640 10 client.27470 handle_command_reply: tid=0 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.232+0000 7f3203119740 1 client.27470 shutdown 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.233+0000 7f3203119740 1 -- 172.21.15.5:0/442522113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11cb630 msgr2=0x5604e123a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.233+0000 7f3203119740 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11cb630 0x5604e123a320 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f31fc112f90 tx=0x7f31f400f7c0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.238+0000 7f3203119740 1 -- 172.21.15.5:0/442522113 shutdown_connections 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.238+0000 7f3203119740 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x5604e12430b0 0x5604e12454d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.238+0000 7f3203119740 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604e123a920 0x5604e123ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.238+0000 7f3203119740 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x5604e11cb630 0x5604e123a320 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.238+0000 7f3203119740 1 --2- 172.21.15.5:0/442522113 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x5604e11cad10 0x5604e11cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.238+0000 7f31e1ffb640 0 client.27470 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.238+0000 7f3203119740 1 -- 172.21.15.5:0/442522113 >> 172.21.15.5:0/442522113 conn(0x5604e1185310 msgr2=0x5604e11c57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.241+0000 7f3203119740 1 -- 172.21.15.5:0/442522113 shutdown_connections 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.241+0000 7f3203119740 1 -- 172.21.15.5:0/442522113 wait complete. 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.246+0000 7f3203119740 20 client.27470 trim_cache size 0 max 16384 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.250+0000 7f3200d40640 1 -- 172.21.15.5:0/838329220 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e40a35c0 msgr2=0x7f31e40a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.250+0000 7f3200d40640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e40a35c0 0x7f31e40a5a80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009460 tx=0x7f31ec009490 comp rx=0 tx=0).stop 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.250+0000 7f3200d40640 1 -- 172.21.15.5:0/838329220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31fc109450 msgr2=0x7f31fc1186b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.250+0000 7f3200d40640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31fc109450 0x7f31fc1186b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f31f000e9c0 tx=0x7f31f000ee80 comp rx=0 tx=0).stop 2024-09-15T22:24:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.250+0000 7f3200d40640 1 -- 172.21.15.5:0/838329220 shutdown_connections 2024-09-15T22:24:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.250+0000 7f3200d40640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e40a35c0 0x7f31e40a5a80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.250+0000 7f3200d40640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31fc11b140 0x7f31fc111920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.250+0000 7f3200d40640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31fc109830 0x7f31fc118bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.250+0000 7f3200d40640 1 --2- 172.21.15.5:0/838329220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31fc109450 0x7f31fc1186b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.250+0000 7f3200d40640 1 -- 172.21.15.5:0/838329220 >> 172.21.15.5:0/838329220 conn(0x7f31fc06e140 msgr2=0x7f31fc11c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.251+0000 7f3200d40640 1 -- 172.21.15.5:0/838329220 shutdown_connections 2024-09-15T22:24:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:33.251+0000 7f3200d40640 1 -- 172.21.15.5:0/838329220 wait complete. 2024-09-15T22:24:33.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:33 smithi099 ceph-mon[34892]: pgmap v490: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-15T22:24:33.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:33 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3343360504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:33.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:33 smithi129 ceph-mon[35314]: pgmap v490: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.2 MiB/s wr, 110 op/s 2024-09-15T22:24:33.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:33 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3343360504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:34.803 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:24:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:35.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:35 smithi099 ceph-mon[34892]: pgmap v491: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-15T22:24:35.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:35 smithi129 ceph-mon[35314]: pgmap v491: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-15T22:24:35.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:35 smithi005 ceph-mon[30944]: pgmap v491: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.7 MiB/s wr, 163 op/s 2024-09-15T22:24:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.594+0000 7fb7baffd640 1 -- 172.21.15.5:0/735073985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac031920 con 0x7fb7bc100b40 2024-09-15T22:24:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.594+0000 7fb7c28bb640 1 -- 172.21.15.5:0/735073985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc100b40 msgr2=0x7fb7bc100f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:36.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.594+0000 7fb7c28bb640 1 --2- 172.21.15.5:0/735073985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc100b40 0x7fb7bc100f20 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009900 tx=0x7fb7ac02a4b0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.598+0000 7fb7c28bb640 1 -- 172.21.15.5:0/735073985 shutdown_connections 2024-09-15T22:24:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.598+0000 7fb7c28bb640 1 --2- 172.21.15.5:0/735073985 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc105f50 0x7fb7bc1063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.598+0000 7fb7c28bb640 1 --2- 172.21.15.5:0/735073985 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7bc101460 0x7fb7bc105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.598+0000 7fb7c28bb640 1 --2- 172.21.15.5:0/735073985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc100b40 0x7fb7bc100f20 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.598+0000 7fb7c28bb640 1 -- 172.21.15.5:0/735073985 >> 172.21.15.5:0/735073985 conn(0x7fb7bc0fc7d0 msgr2=0x7fb7bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.600+0000 7fb7c28bb640 1 -- 172.21.15.5:0/735073985 shutdown_connections 2024-09-15T22:24:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.601+0000 7fb7c28bb640 1 -- 172.21.15.5:0/735073985 wait complete. 2024-09-15T22:24:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.603+0000 7fb7c28bb640 1 Processor -- start 2024-09-15T22:24:36.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.604+0000 7fb7c28bb640 1 -- start start 2024-09-15T22:24:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.605+0000 7fb7c28bb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc100b40 0x7fb7bc1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.605+0000 7fb7c28bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc101460 0x7fb7bc1a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.605+0000 7fb7c28bb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7bc105f50 0x7fb7bc19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.605+0000 7fb7c28bb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc1185b0 con 0x7fb7bc101460 2024-09-15T22:24:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.605+0000 7fb7c28bb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc118430 con 0x7fb7bc100b40 2024-09-15T22:24:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.605+0000 7fb7c28bb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7bc118730 con 0x7fb7bc105f50 2024-09-15T22:24:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.606+0000 7fb7c0e58640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7bc105f50 0x7fb7bc19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.606+0000 7fb7c0e58640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7bc105f50 0x7fb7bc19e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:40118/0 (socket says 172.21.15.5:40118) 2024-09-15T22:24:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.606+0000 7fb7c0e58640 1 -- 172.21.15.5:0/1539872052 learned_addr learned my addr 172.21.15.5:0/1539872052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.607+0000 7fb7c0e58640 1 -- 172.21.15.5:0/1539872052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc100b40 msgr2=0x7fb7bc1a4720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:24:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.607+0000 7fb7c0e58640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc100b40 0x7fb7bc1a4720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.607+0000 7fb7c0e58640 1 -- 172.21.15.5:0/1539872052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc101460 msgr2=0x7fb7bc1a4c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T22:24:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.607+0000 7fb7c0e58640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc101460 0x7fb7bc1a4c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.607+0000 7fb7c0e58640 1 -- 172.21.15.5:0/1539872052 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7bc19f010 con 0x7fb7bc105f50 2024-09-15T22:24:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.607+0000 7fb7c0e58640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7bc105f50 0x7fb7bc19e8e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400eaa0 tx=0x7fb7b400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:36.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.607+0000 7fb7b97fa640 1 -- 172.21.15.5:0/1539872052 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b400cdb0 con 0x7fb7bc105f50 2024-09-15T22:24:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.608+0000 7fb7c28bb640 1 -- 172.21.15.5:0/1539872052 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7bc19f300 con 0x7fb7bc105f50 2024-09-15T22:24:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.608+0000 7fb7c28bb640 1 -- 172.21.15.5:0/1539872052 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7bc1ab5f0 con 0x7fb7bc105f50 2024-09-15T22:24:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.608+0000 7fb7b97fa640 1 -- 172.21.15.5:0/1539872052 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b4005020 con 0x7fb7bc105f50 2024-09-15T22:24:36.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.608+0000 7fb7b97fa640 1 -- 172.21.15.5:0/1539872052 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40116b0 con 0x7fb7bc105f50 2024-09-15T22:24:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.622+0000 7fb7b97fa640 1 -- 172.21.15.5:0/1539872052 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fb7b4011850 con 0x7fb7bc105f50 2024-09-15T22:24:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.623+0000 7fb7b97fa640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb7a40a34b0 0x7fb7a40a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.623+0000 7fb7b97fa640 1 -- 172.21.15.5:0/1539872052 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7fb7b40d1360 con 0x7fb7bc105f50 2024-09-15T22:24:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.625+0000 7fb7c4c94740 1 Processor -- start 2024-09-15T22:24:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.626+0000 7fb7c4c94740 1 -- start start 2024-09-15T22:24:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.626+0000 7fb7c4c94740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f06b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.626+0000 7fb7c4c94740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 0x55f688f11400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.626+0000 7fb7c4c94740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 0x55f688f13d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.626+0000 7fb7c4c94740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f688e6c2a0 con 0x55f688f07080 2024-09-15T22:24:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.626+0000 7fb7c4c94740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f688e13b10 con 0x55f688f11940 2024-09-15T22:24:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.626+0000 7fb7c4c94740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f688e68260 con 0x55f688f06760 2024-09-15T22:24:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.627+0000 7fb7c0e58640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 0x55f688f13d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.627+0000 7fb7c0e58640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 0x55f688f13d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33482/0 (socket says 172.21.15.5:33482) 2024-09-15T22:24:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.627+0000 7fb7c0e58640 1 -- 172.21.15.5:0/2656388754 learned_addr learned my addr 172.21.15.5:0/2656388754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.628+0000 7fb7bb7fe640 1 --2- 172.21.15.5:0/2656388754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 0x55f688f11400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.629+0000 7fb7bbfff640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb7a40a34b0 0x7fb7a40a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.632+0000 7fb7bbfff640 1 --2- 172.21.15.5:0/2656388754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f06b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.632+0000 7fb7bb7fe640 1 -- 172.21.15.5:0/2656388754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 msgr2=0x55f688f06b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.632+0000 7fb7bb7fe640 1 --2- 172.21.15.5:0/2656388754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f06b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.632+0000 7fb7bb7fe640 1 -- 172.21.15.5:0/2656388754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 msgr2=0x55f688f13d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.632+0000 7fb7bb7fe640 1 --2- 172.21.15.5:0/2656388754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 0x55f688f13d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.632+0000 7fb7bb7fe640 1 -- 172.21.15.5:0/2656388754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f688bddb90 con 0x55f688f07080 2024-09-15T22:24:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.638+0000 7fb7bb7fe640 1 --2- 172.21.15.5:0/2656388754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 0x55f688f11400 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0005fd0 tx=0x7fb7b000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ea0c0f39d5e0b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.641+0000 7fb7bbfff640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb7a40a34b0 0x7fb7a40a5970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac0096c0 tx=0x7fb7ac000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.646+0000 7fb7a27fc640 1 -- 172.21.15.5:0/2656388754 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0016070 con 0x55f688f07080 2024-09-15T22:24:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.646+0000 7fb7a27fc640 1 -- 172.21.15.5:0/2656388754 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b000de10 con 0x55f688f07080 2024-09-15T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.647+0000 7fb7c4c94740 1 -- 172.21.15.5:0/2656388754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 msgr2=0x55f688f11400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.647+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/2656388754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 0x55f688f11400 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0005fd0 tx=0x7fb7b000be50 comp rx=0 tx=0).stop 2024-09-15T22:24:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.648+0000 7fb7c4c94740 1 -- 172.21.15.5:0/2656388754 shutdown_connections 2024-09-15T22:24:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.648+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/2656388754 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 0x55f688f13d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.648+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/2656388754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 0x55f688f11400 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.648+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/2656388754 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f06b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.648+0000 7fb7c4c94740 1 -- 172.21.15.5:0/2656388754 >> 172.21.15.5:0/2656388754 conn(0x55f688f02cf0 msgr2=0x55f688f030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.649+0000 7fb7c4c94740 1 -- 172.21.15.5:0/2656388754 shutdown_connections 2024-09-15T22:24:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.649+0000 7fb7c4c94740 1 -- 172.21.15.5:0/2656388754 wait complete. 2024-09-15T22:24:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.649+0000 7fb7c4c94740 1 -- start start 2024-09-15T22:24:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.650+0000 7fb7c4c94740 1 Processor -- start 2024-09-15T22:24:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.651+0000 7fb7c4c94740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f0e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.651+0000 7fb7c4c94740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 0x55f688f0ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.651+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f0e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.651+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f0e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:40140/0 (socket says 172.21.15.5:40140) 2024-09-15T22:24:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.651+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 0x55f688f0ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.652+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 0x55f688f0ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33490/0 (socket says 172.21.15.5:33490) 2024-09-15T22:24:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.652+0000 7fb7bbfff640 1 -- 172.21.15.5:0/1686589934 learned_addr learned my addr 172.21.15.5:0/1686589934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.652+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f11940 0x55f688f0f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.652+0000 7fb7c4c94740 1 -- 172.21.15.5:0/1686589934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f688f14560 con 0x55f688f11940 2024-09-15T22:24:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.652+0000 7fb7c4c94740 1 -- 172.21.15.5:0/1686589934 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f688df46e0 con 0x55f688f07080 2024-09-15T22:24:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.652+0000 7fb7c4c94740 1 -- 172.21.15.5:0/1686589934 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f688f146e0 con 0x55f688f06760 2024-09-15T22:24:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.652+0000 7fb7c4c94740 10 client.0 _ll_register_callbacks cb 0x55f688e18a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.657+0000 7fb7bbfff640 1 -- 172.21.15.5:0/1686589934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 msgr2=0x55f688f0ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.657+0000 7fb7bbfff640 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 0x55f688f0ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.657+0000 7fb7bbfff640 1 -- 172.21.15.5:0/1686589934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f11940 msgr2=0x55f688f0f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.657+0000 7fb7bbfff640 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f11940 0x55f688f0f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.657+0000 7fb7bbfff640 1 -- 172.21.15.5:0/1686589934 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f688f1da80 con 0x55f688f06760 2024-09-15T22:24:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.657+0000 7fb7bbfff640 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f0e770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac005ab0 tx=0x7fb7ac02a4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c58f8b4f0fcb0cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.658+0000 7fb79b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.129:3300/0 2024-09-15T22:24:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.661+0000 7fb79b7fe640 1 -- 172.21.15.5:0/1686589934 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7ac04d020 con 0x55f688f06760 2024-09-15T22:24:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.661+0000 7fb79b7fe640 1 -- 172.21.15.5:0/1686589934 <== mon.1 v2:172.21.15.129:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7fb7ac0037f0 con 0x55f688f06760 2024-09-15T22:24:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.661+0000 7fb7c4c94740 1 -- 172.21.15.5:0/1686589934 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b000de10 con 0x55f688f06760 2024-09-15T22:24:36.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.671+0000 7fb79b7fe640 1 -- 172.21.15.5:0/1686589934 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb7ac045040 con 0x55f688f06760 2024-09-15T22:24:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.676+0000 7fb7c4c94740 10 client.27403 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.676+0000 7fb7c4c94740 1 -- 172.21.15.5:0/1686589934 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f688f203f0 con 0x55f688f06760 2024-09-15T22:24:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.676+0000 7fb79b7fe640 1 -- 172.21.15.5:0/1686589934 <== mon.1 v2:172.21.15.129:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b000de10 con 0x55f688f06760 2024-09-15T22:24:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.677+0000 7fb79b7fe640 1 -- 172.21.15.5:0/1686589934 <== mon.1 v2:172.21.15.129:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb7ac05a020 con 0x55f688f06760 2024-09-15T22:24:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.679+0000 7fb7c4c94740 10 client.27403 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.679+0000 7fb7c4c94740 10 client.27403 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:24:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.679+0000 7fb7c4c94740 20 client.27403 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.679+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f688f21900 0x55f688f23dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.679+0000 7fb7c4c94740 4 client.27403 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.679+0000 7fb7c4c94740 1 -- 172.21.15.5:0/1686589934 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f688e799d0 con 0x55f688f21900 2024-09-15T22:24:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.679+0000 7fb7bb7fe640 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f688f21900 0x55f688f23dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.687+0000 7fb79b7fe640 10 client.27403 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.687+0000 7fb7bb7fe640 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f688f21900 0x55f688f23dc0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2de7b209551505a2 server_cookie=682860b3c5298178 in_seq=0 out_seq=0 2024-09-15T22:24:36.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.688+0000 7fb79b7fe640 1 -- 172.21.15.5:0/1686589934 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f688e799d0 con 0x55f688f21900 2024-09-15T22:24:36.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.688+0000 7fb79b7fe640 10 client.27403 handle_command_reply: tid=0 2024-09-15T22:24:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.693+0000 7fb7c4c94740 1 client.27403 shutdown 2024-09-15T22:24:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.700+0000 7fb7c4c94740 1 -- 172.21.15.5:0/1686589934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 msgr2=0x55f688f0e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:36.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.700+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f0e770 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac005ab0 tx=0x7fb7ac02a4b0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.700+0000 7fb7c4c94740 1 -- 172.21.15.5:0/1686589934 shutdown_connections 2024-09-15T22:24:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.700+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f11940 0x55f688f0f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.700+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f688f21900 0x55f688f23dc0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.700+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 0x55f688f0ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.700+0000 7fb79b7fe640 0 client.27403 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.700+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/1686589934 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f0e770 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.700+0000 7fb7c4c94740 1 -- 172.21.15.5:0/1686589934 >> 172.21.15.5:0/1686589934 conn(0x55f688f02cf0 msgr2=0x55f688f08a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.700+0000 7fb7c4c94740 1 -- 172.21.15.5:0/1686589934 shutdown_connections 2024-09-15T22:24:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.700+0000 7fb7c4c94740 1 -- 172.21.15.5:0/1686589934 wait complete. 2024-09-15T22:24:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.701+0000 7fb7c4c94740 20 client.27403 trim_cache size 0 max 16384 2024-09-15T22:24:36.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.736+0000 7fb7c4c94740 1 Processor -- start 2024-09-15T22:24:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.738+0000 7fb7c4c94740 1 -- start start 2024-09-15T22:24:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.738+0000 7fb7c4c94740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f01e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.739+0000 7fb7c4c94740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 0x55f688f29690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.739+0000 7fb7c4c94740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f11940 0x55f688f29bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.739+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f01e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.739+0000 7fb7bbfff640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f01e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.129:3300/0 says I am v2:172.21.15.5:40152/0 (socket says 172.21.15.5:40152) 2024-09-15T22:24:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.739+0000 7fb7bbfff640 1 -- 172.21.15.5:0/561507902 learned_addr learned my addr 172.21.15.5:0/561507902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:36.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.739+0000 7fb7c4c94740 1 -- 172.21.15.5:0/561507902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a80129e0 con 0x55f688f11940 2024-09-15T22:24:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.739+0000 7fb7c4c94740 1 -- 172.21.15.5:0/561507902 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a80102b0 con 0x55f688f07080 2024-09-15T22:24:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.739+0000 7fb7c4c94740 1 -- 172.21.15.5:0/561507902 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a800e120 con 0x55f688f06760 2024-09-15T22:24:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.740+0000 7fb7c0e58640 1 --2- 172.21.15.5:0/561507902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f11940 0x55f688f29bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.740+0000 7fb7c0e58640 1 -- 172.21.15.5:0/561507902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 msgr2=0x55f688f01e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:36.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.740+0000 7fb7c0e58640 1 --2- 172.21.15.5:0/561507902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f01e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.739+0000 7fb7bb7fe640 1 --2- 172.21.15.5:0/561507902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 0x55f688f29690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.741+0000 7fb7c0e58640 1 -- 172.21.15.5:0/561507902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 msgr2=0x55f688f29690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:36.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.741+0000 7fb7c0e58640 1 --2- 172.21.15.5:0/561507902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 0x55f688f29690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.741+0000 7fb7c0e58640 1 -- 172.21.15.5:0/561507902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f688f02b20 con 0x55f688f11940 2024-09-15T22:24:36.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.741+0000 7fb7bb7fe640 1 --2- 172.21.15.5:0/561507902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 0x55f688f29690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:36.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.758+0000 7fb7c0e58640 1 --2- 172.21.15.5:0/561507902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f11940 0x55f688f29bd0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40a5920 tx=0x7fb7b40a1650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c915844422ea6668 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.766+0000 7fb7a1ffb640 1 -- 172.21.15.5:0/561507902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b40b6070 con 0x55f688f11940 2024-09-15T22:24:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.766+0000 7fb7a1ffb640 1 -- 172.21.15.5:0/561507902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b4099850 con 0x55f688f11940 2024-09-15T22:24:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.766+0000 7fb7a1ffb640 1 -- 172.21.15.5:0/561507902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b401f040 con 0x55f688f11940 2024-09-15T22:24:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.768+0000 7fb7c4c94740 1 -- 172.21.15.5:0/561507902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f11940 msgr2=0x55f688f29bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:36.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.768+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/561507902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f11940 0x55f688f29bd0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40a5920 tx=0x7fb7b40a1650 comp rx=0 tx=0).stop 2024-09-15T22:24:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.770+0000 7fb7c4c94740 1 -- 172.21.15.5:0/561507902 shutdown_connections 2024-09-15T22:24:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.770+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/561507902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f11940 0x55f688f29bd0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.770+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/561507902 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f07080 0x55f688f29690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.771+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/561507902 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f01e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.771+0000 7fb7c4c94740 1 -- 172.21.15.5:0/561507902 >> 172.21.15.5:0/561507902 conn(0x55f688f02cf0 msgr2=0x55f688f0d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:36.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.772+0000 7fb7c4c94740 1 -- 172.21.15.5:0/561507902 shutdown_connections 2024-09-15T22:24:36.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.772+0000 7fb7c4c94740 1 -- 172.21.15.5:0/561507902 wait complete. 2024-09-15T22:24:36.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.772+0000 7fb7c4c94740 1 -- start start 2024-09-15T22:24:36.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.773+0000 7fb7c4c94740 1 Processor -- start 2024-09-15T22:24:36.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.773+0000 7fb7c4c94740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f1a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.773+0000 7fb7c4c94740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 0x55f688f15560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.773+0000 7fb7c4c94740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 0x55f688f15b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.773+0000 7fb7c4c94740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a800e770 con 0x55f688f07080 2024-09-15T22:24:36.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.773+0000 7fb7c4c94740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a800fbe0 con 0x55f688f11940 2024-09-15T22:24:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.773+0000 7fb7c4c94740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a800ee40 con 0x55f688f06760 2024-09-15T22:24:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.773+0000 7fb7c4c94740 10 client.0 _ll_register_callbacks cb 0x7fb7ac0315f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:36.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.774+0000 7fb7c0e58640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 0x55f688f15b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.779+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 0x55f688f15560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.779+0000 7fb7bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 0x55f688f15560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34004/0 (socket says 172.21.15.5:34004) 2024-09-15T22:24:36.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.779+0000 7fb7bb7fe640 1 -- 172.21.15.5:0/3017900095 learned_addr learned my addr 172.21.15.5:0/3017900095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:36.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.779+0000 7fb7bbfff640 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f1a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.779+0000 7fb7c0e58640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 0x55f688f15b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.99:3300/0 says I am v2:172.21.15.5:33500/0 (socket says 172.21.15.5:33500) 2024-09-15T22:24:36.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.781+0000 7fb7c0e58640 1 -- 172.21.15.5:0/3017900095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 msgr2=0x55f688f1a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:36.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.781+0000 7fb7c0e58640 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f1a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.781+0000 7fb7c0e58640 1 -- 172.21.15.5:0/3017900095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 msgr2=0x55f688f15560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:36.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.781+0000 7fb7c0e58640 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 0x55f688f15560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:36.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.781+0000 7fb7c0e58640 1 -- 172.21.15.5:0/3017900095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f688f16360 con 0x55f688f11940 2024-09-15T22:24:36.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.788+0000 7fb7c0e58640 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 0x55f688f15b30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40ce6b0 tx=0x7fb7b4005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad32fca82809bf69 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:36.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.788+0000 7fb79b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.99:3300/0 2024-09-15T22:24:36.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.790+0000 7fb79b7fe640 1 -- 172.21.15.5:0/3017900095 <== mon.2 v2:172.21.15.99:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b401f070 con 0x55f688f11940 2024-09-15T22:24:36.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.790+0000 7fb7c4c94740 1 -- 172.21.15.5:0/3017900095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b4099850 con 0x55f688f11940 2024-09-15T22:24:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.790+0000 7fb79b7fe640 1 -- 172.21.15.5:0/3017900095 <== mon.2 v2:172.21.15.99:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7fb7b40a5bc0 con 0x55f688f11940 2024-09-15T22:24:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.792+0000 7fb79b7fe640 1 -- 172.21.15.5:0/3017900095 <== mon.2 v2:172.21.15.99:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb7b40b6040 con 0x55f688f11940 2024-09-15T22:24:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.793+0000 7fb7c4c94740 10 client.27482 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:36.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.793+0000 7fb7c4c94740 1 -- 172.21.15.5:0/3017900095 --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f688f9d6c0 con 0x55f688f11940 2024-09-15T22:24:36.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.797+0000 7fb79b7fe640 1 -- 172.21.15.5:0/3017900095 <== mon.2 v2:172.21.15.99:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b4099850 con 0x55f688f11940 2024-09-15T22:24:36.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.797+0000 7fb79b7fe640 1 -- 172.21.15.5:0/3017900095 <== mon.2 v2:172.21.15.99:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fb7b400f040 con 0x55f688f11940 2024-09-15T22:24:36.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.797+0000 7fb7c4c94740 10 client.27482 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:36.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.797+0000 7fb7c4c94740 10 client.27482 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:24:36.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.797+0000 7fb7c4c94740 20 client.27482 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:36.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.797+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f688f9ddf0 0x55f688fa01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:36.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.797+0000 7fb7c4c94740 4 client.27482 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-15T22:24:36.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.797+0000 7fb7c4c94740 1 -- 172.21.15.5:0/3017900095 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x55f688e73d20 con 0x55f688f9ddf0 2024-09-15T22:24:36.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.798+0000 7fb7bbfff640 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f688f9ddf0 0x55f688fa01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:36.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.808+0000 7fb79b7fe640 10 client.27482 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:36.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:36.808+0000 7fb7bbfff640 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f688f9ddf0 0x55f688fa01e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74045c7ec49978f6 server_cookie=13f71e94ba156c34 in_seq=0 out_seq=0 2024-09-15T22:24:37.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.703+0000 7fb79b7fe640 1 -- 172.21.15.5:0/3017900095 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f688e73d20 con 0x55f688f9ddf0 2024-09-15T22:24:37.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.704+0000 7fb79b7fe640 10 client.27482 handle_command_reply: tid=0 2024-09-15T22:24:37.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.704+0000 7fb7c4c94740 1 client.27482 shutdown 2024-09-15T22:24:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.705+0000 7fb7c4c94740 1 -- 172.21.15.5:0/3017900095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 msgr2=0x55f688f15b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.705+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 0x55f688f15b30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7b40ce6b0 tx=0x7fb7b4005e10 comp rx=0 tx=0).stop 2024-09-15T22:24:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.707+0000 7fb7c4c94740 1 -- 172.21.15.5:0/3017900095 shutdown_connections 2024-09-15T22:24:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.707+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f688f9ddf0 0x55f688fa01e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.707+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f688f11940 0x55f688f15b30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.707+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f688f07080 0x55f688f15560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.707+0000 7fb7c4c94740 1 --2- 172.21.15.5:0/3017900095 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f688f06760 0x55f688f1a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.707+0000 7fb7c4c94740 1 -- 172.21.15.5:0/3017900095 >> 172.21.15.5:0/3017900095 conn(0x55f688f02cf0 msgr2=0x55f688f0ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.708+0000 7fb79b7fe640 0 client.27482 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.708+0000 7fb7c4c94740 1 -- 172.21.15.5:0/3017900095 shutdown_connections 2024-09-15T22:24:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.708+0000 7fb7c4c94740 1 -- 172.21.15.5:0/3017900095 wait complete. 2024-09-15T22:24:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.708+0000 7fb7c4c94740 20 client.27482 trim_cache size 0 max 16384 2024-09-15T22:24:37.711 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:24:37.711 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 58, 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "398ff487": { 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 58, 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 137.7, 2024-09-15T22:24:37.712 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:24:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:24:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:24:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:24:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-15T22:24:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:24:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:37.713 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:37.713 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:24:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.714+0000 7fb7c28bb640 1 -- 172.21.15.5:0/1539872052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb7a40a34b0 msgr2=0x7fb7a40a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.714+0000 7fb7c28bb640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb7a40a34b0 0x7fb7a40a5970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac0096c0 tx=0x7fb7ac000bc0 comp rx=0 tx=0).stop 2024-09-15T22:24:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.714+0000 7fb7c28bb640 1 -- 172.21.15.5:0/1539872052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7bc105f50 msgr2=0x7fb7bc19e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.714+0000 7fb7c28bb640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7bc105f50 0x7fb7bc19e8e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb7b400eaa0 tx=0x7fb7b400ef60 comp rx=0 tx=0).stop 2024-09-15T22:24:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.714+0000 7fb7c28bb640 1 -- 172.21.15.5:0/1539872052 shutdown_connections 2024-09-15T22:24:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.714+0000 7fb7c28bb640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fb7a40a34b0 0x7fb7a40a5970 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.714+0000 7fb7c28bb640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fb7bc105f50 0x7fb7bc19e8e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.714+0000 7fb7c28bb640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7bc101460 0x7fb7bc1a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.714+0000 7fb7c28bb640 1 --2- 172.21.15.5:0/1539872052 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fb7bc100b40 0x7fb7bc1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.715+0000 7fb7c28bb640 1 -- 172.21.15.5:0/1539872052 >> 172.21.15.5:0/1539872052 conn(0x7fb7bc0fc7d0 msgr2=0x7fb7bc1038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.715+0000 7fb7c28bb640 1 -- 172.21.15.5:0/1539872052 shutdown_connections 2024-09-15T22:24:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.715+0000 7fb7c28bb640 1 -- 172.21.15.5:0/1539872052 wait complete. 2024-09-15T22:24:37.776 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:37 smithi005 ceph-mon[30944]: pgmap v492: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.8 MiB/s wr, 277 op/s 2024-09-15T22:24:37.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:37 smithi099 ceph-mon[34892]: pgmap v492: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.8 MiB/s wr, 277 op/s 2024-09-15T22:24:37.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:37 smithi129 ceph-mon[35314]: pgmap v492: 129 pgs: 129 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 2.8 MiB/s wr, 277 op/s 2024-09-15T22:24:38.205 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:24:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.831+0000 7fdff5bc8640 1 -- 172.21.15.5:0/424919126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff0106a90 msgr2=0x7fdff0106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.831+0000 7fdff5bc8640 1 --2- 172.21.15.5:0/424919126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff0106a90 0x7fdff0106e70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0009930 tx=0x7fdfe002a420 comp rx=0 tx=0).stop 2024-09-15T22:24:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.831+0000 7fdff5bc8640 1 -- 172.21.15.5:0/424919126 shutdown_connections 2024-09-15T22:24:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.831+0000 7fdff5bc8640 1 --2- 172.21.15.5:0/424919126 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdff0101430 0x7fdff010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.831+0000 7fdff5bc8640 1 --2- 172.21.15.5:0/424919126 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff0100a90 0x7fdff0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.831+0000 7fdff5bc8640 1 --2- 172.21.15.5:0/424919126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff0106a90 0x7fdff0106e70 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.831+0000 7fdff5bc8640 1 -- 172.21.15.5:0/424919126 >> 172.21.15.5:0/424919126 conn(0x7fdff00fc7d0 msgr2=0x7fdff00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.831+0000 7fdff5bc8640 1 -- 172.21.15.5:0/424919126 shutdown_connections 2024-09-15T22:24:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.831+0000 7fdff5bc8640 1 -- 172.21.15.5:0/424919126 wait complete. 2024-09-15T22:24:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.832+0000 7fdff5bc8640 1 Processor -- start 2024-09-15T22:24:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.832+0000 7fdff5bc8640 1 -- start start 2024-09-15T22:24:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdff5bc8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff0100a90 0x7fdff0196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdff5bc8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff0101430 0x7fdff01967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdff5bc8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdff0106a90 0x7fdff019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdff5bc8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdff010fde0 con 0x7fdff0100a90 2024-09-15T22:24:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdff5bc8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdff010fc60 con 0x7fdff0101430 2024-09-15T22:24:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdfef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff0100a90 0x7fdff0196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdff5bc8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdff010ff60 con 0x7fdff0106a90 2024-09-15T22:24:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdfef7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff0100a90 0x7fdff0196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34026/0 (socket says 172.21.15.5:34026) 2024-09-15T22:24:38.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdfef7fe640 1 -- 172.21.15.5:0/708745432 learned_addr learned my addr 172.21.15.5:0/708745432 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff0101430 0x7fdff01967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdfeffff640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdff0106a90 0x7fdff019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdfef7fe640 1 -- 172.21.15.5:0/708745432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdff0106a90 msgr2=0x7fdff019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdfef7fe640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdff0106a90 0x7fdff019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdfef7fe640 1 -- 172.21.15.5:0/708745432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff0101430 msgr2=0x7fdff01967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.833+0000 7fdfef7fe640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff0101430 0x7fdff01967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.834+0000 7fdfef7fe640 1 -- 172.21.15.5:0/708745432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff019b210 con 0x7fdff0100a90 2024-09-15T22:24:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.834+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff0101430 0x7fdff01967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.834+0000 7fdfef7fe640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff0100a90 0x7fdff0196260 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fdfe002a940 tx=0x7fdfe0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.834+0000 7fdfecff9640 1 -- 172.21.15.5:0/708745432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe003f070 con 0x7fdff0100a90 2024-09-15T22:24:38.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.834+0000 7fdff5bc8640 1 -- 172.21.15.5:0/708745432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff019b4a0 con 0x7fdff0100a90 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.834+0000 7fdfecff9640 1 -- 172.21.15.5:0/708745432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe0031c90 con 0x7fdff0100a90 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.834+0000 7fdff5bc8640 1 -- 172.21.15.5:0/708745432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff019b900 con 0x7fdff0100a90 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.834+0000 7fdfecff9640 1 -- 172.21.15.5:0/708745432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe0035030 con 0x7fdff0100a90 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.836+0000 7fdfecff9640 1 -- 172.21.15.5:0/708745432 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fdfe0042070 con 0x7fdff0100a90 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdff7fa1740 1 Processor -- start 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdff7fa1740 1 -- start start 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03897c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f03897c5d0 0x55f03897ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f038987370 0x55f038989760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0388e22f0 con 0x55f03897db40 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f038889cf0 con 0x55f03897c5d0 2024-09-15T22:24:38.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f0388de2b0 con 0x55f038987370 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdfeffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03897c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdfeffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03897c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34032/0 (socket says 172.21.15.5:34032) 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdfeffff640 1 -- 172.21.15.5:0/1402059565 learned_addr learned my addr 172.21.15.5:0/1402059565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.837+0000 7fdfef7fe640 1 --2- 172.21.15.5:0/1402059565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f03897c5d0 0x55f03897ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/1402059565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f038987370 0x55f038989760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfeffff640 1 -- 172.21.15.5:0/1402059565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f038987370 msgr2=0x55f038989760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1402059565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f038987370 0x55f038989760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfeffff640 1 -- 172.21.15.5:0/1402059565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f03897c5d0 msgr2=0x55f03897ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1402059565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f03897c5d0 0x55f03897ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfeffff640 1 -- 172.21.15.5:0/1402059565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03886e3a0 con 0x55f03897db40 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfef7fe640 1 --2- 172.21.15.5:0/1402059565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f03897c5d0 0x55f03897ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1402059565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03897c090 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8009e60 tx=0x7fdfe8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd7a0775772df7d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfe5ffb640 1 -- 172.21.15.5:0/1402059565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8016070 con 0x55f03897db40 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfe5ffb640 1 -- 172.21.15.5:0/1402059565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe8004d20 con 0x55f03897db40 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfe5ffb640 1 -- 172.21.15.5:0/1402059565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8012870 con 0x55f03897db40 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.838+0000 7fdfecff9640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdfdc0a3430 0x7fdfdc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdfef7fe640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdfdc0a3430 0x7fdfdc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdfecff9640 1 -- 172.21.15.5:0/708745432 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7fdfe00f0880 con 0x7fdff0100a90 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1402059565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 msgr2=0x55f03897c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1402059565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03897c090 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fdfe8009e60 tx=0x7fdfe8010740 comp rx=0 tx=0).stop 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1402059565 shutdown_connections 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1402059565 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f038987370 0x55f038989760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1402059565 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f03897c5d0 0x55f03897ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1402059565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03897c090 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1402059565 >> 172.21.15.5:0/1402059565 conn(0x55f038945310 msgr2=0x55f038978c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdfef7fe640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdfdc0a3430 0x7fdfdc0a58f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55f038979ac0 tx=0x7fdfe0009260 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1402059565 shutdown_connections 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1402059565 wait complete. 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.839+0000 7fdff7fa1740 1 -- start start 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdff7fa1740 1 Processor -- start 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f038986d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038983ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f0389845e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f03898a070 con 0x55f03897db40 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f03889d060 con 0x55f038987370 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f03898a1f0 con 0x55f03897c5d0 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdff7fa1740 10 client.0 _ll_register_callbacks cb 0x55f0385f08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdfeeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038983ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdfeeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038983ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34034/0 (socket says 172.21.15.5:34034) 2024-09-15T22:24:38.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdfeeffd640 1 -- 172.21.15.5:0/1653516507 learned_addr learned my addr 172.21.15.5:0/1653516507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f0389845e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f038986d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdfeeffd640 1 -- 172.21.15.5:0/1653516507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 msgr2=0x55f038986d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f038986d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdfeeffd640 1 -- 172.21.15.5:0/1653516507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 msgr2=0x55f0389845e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f0389845e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdfeeffd640 1 -- 172.21.15.5:0/1653516507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0386533c0 con 0x55f03897db40 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.840+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f038986d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038983ff0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800aa80 tx=0x7fdfd800af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbb6c386c91a3b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdfceffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdfceffd640 1 -- 172.21.15.5:0/1653516507 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd801b030 con 0x55f03897db40 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdfceffd640 1 -- 172.21.15.5:0/1653516507 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7fdfd8004320 con 0x55f03897db40 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1653516507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe8004d20 con 0x55f03897db40 2024-09-15T22:24:38.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdfceffd640 1 -- 172.21.15.5:0/1653516507 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdfd8025050 con 0x55f03897db40 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdff7fa1740 10 client.18027 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1653516507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f038995ab0 con 0x55f03897db40 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdfceffd640 1 -- 172.21.15.5:0/1653516507 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfe8004d20 con 0x55f03897db40 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdfceffd640 1 -- 172.21.15.5:0/1653516507 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdfd8013d20 con 0x55f03897db40 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdff7fa1740 10 client.18027 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdff7fa1740 10 client.18027 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdff7fa1740 20 client.18027 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f038997010 0x55f038999430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdff7fa1740 4 client.18027 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.841+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1653516507 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0388ef860 con 0x55f038997010 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.842+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f038997010 0x55f038999430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.842+0000 7fdfceffd640 10 client.18027 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:24:38.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.842+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f038997010 0x55f038999430 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d012c2c5bf2dff3 server_cookie=b8bc41ffbc33deaf in_seq=0 out_seq=0 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdfceffd640 1 -- 172.21.15.5:0/1653516507 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0388ef860 con 0x55f038997010 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdfceffd640 10 client.18027 handle_command_reply: tid=0 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 1 client.18027 shutdown 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1653516507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 msgr2=0x55f038983ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038983ff0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fdfd800aa80 tx=0x7fdfd800af40 comp rx=0 tx=0).stop 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1653516507 shutdown_connections 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f038997010 0x55f038999430 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f0389845e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038983ff0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1653516507 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f038986d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdfceffd640 0 client.18027 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:24:38.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1653516507 >> 172.21.15.5:0/1653516507 conn(0x55f038945310 msgr2=0x55f03897e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1653516507 shutdown_connections 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1653516507 wait complete. 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.843+0000 7fdff7fa1740 20 client.18027 trim_cache size 0 max 16384 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.856+0000 7fdff7fa1740 1 Processor -- start 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.856+0000 7fdff7fa1740 1 -- start start 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f038993310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03899ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f03899f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdff01b3c50 con 0x55f03897db40 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdff01b1520 con 0x55f038987370 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdff01af390 con 0x55f03897c5d0 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03899ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03899ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34036/0 (socket says 172.21.15.5:34036) 2024-09-15T22:24:38.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeeffd640 1 -- 172.21.15.5:0/3157848285 learned_addr learned my addr 172.21.15.5:0/3157848285 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeffff640 1 --2- 172.21.15.5:0/3157848285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f03899f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeffff640 1 --2- 172.21.15.5:0/3157848285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f038993310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeeffd640 1 -- 172.21.15.5:0/3157848285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 msgr2=0x55f038993310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/3157848285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f038993310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeeffd640 1 -- 172.21.15.5:0/3157848285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 msgr2=0x55f03899f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/3157848285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f03899f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeeffd640 1 -- 172.21.15.5:0/3157848285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f03897f120 con 0x55f03897db40 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeffff640 1 --2- 172.21.15.5:0/3157848285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f03899f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/3157848285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03899ece0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80048f0 tx=0x7fdfd8013600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7826b31208d3bcb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfe57fa640 1 -- 172.21.15.5:0/3157848285 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd801b040 con 0x55f03897db40 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfe57fa640 1 -- 172.21.15.5:0/3157848285 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd802e070 con 0x55f03897db40 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.857+0000 7fdfe57fa640 1 -- 172.21.15.5:0/3157848285 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8003970 con 0x55f03897db40 2024-09-15T22:24:38.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.858+0000 7fdff7fa1740 1 -- 172.21.15.5:0/3157848285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 msgr2=0x55f03899ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.858+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/3157848285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03899ece0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fdfd80048f0 tx=0x7fdfd8013600 comp rx=0 tx=0).stop 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.858+0000 7fdff7fa1740 1 -- 172.21.15.5:0/3157848285 shutdown_connections 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.858+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/3157848285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f03899f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.858+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/3157848285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f03899ece0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.858+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/3157848285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f038993310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.858+0000 7fdff7fa1740 1 -- 172.21.15.5:0/3157848285 >> 172.21.15.5:0/3157848285 conn(0x55f038945310 msgr2=0x55f0389830e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.858+0000 7fdff7fa1740 1 -- 172.21.15.5:0/3157848285 shutdown_connections 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.858+0000 7fdff7fa1740 1 -- 172.21.15.5:0/3157848285 wait complete. 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.858+0000 7fdff7fa1740 1 -- start start 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdff7fa1740 1 Processor -- start 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f03898fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038980e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdff7fa1740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f038981450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdff01af9e0 con 0x55f03897db40 2024-09-15T22:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fdff01b0e50 con 0x55f038987370 2024-09-15T22:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdff7fa1740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fdff01b00b0 con 0x55f03897c5d0 2024-09-15T22:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdff7fa1740 10 client.0 _ll_register_callbacks cb 0x7fdfd800bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038980e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeffff640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f038981450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038980e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34050/0 (socket says 172.21.15.5:34050) 2024-09-15T22:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeeffd640 1 -- 172.21.15.5:0/1230507201 learned_addr learned my addr 172.21.15.5:0/1230507201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f03898fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeeffd640 1 -- 172.21.15.5:0/1230507201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 msgr2=0x55f03898fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f03898fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeeffd640 1 -- 172.21.15.5:0/1230507201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 msgr2=0x55f038981450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f038981450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeeffd640 1 -- 172.21.15.5:0/1230507201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f038981c80 con 0x55f03897db40 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f038981450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfeeffd640 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038980e40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8003650 tx=0x7fdfd800afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd854357b9fef327 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.859+0000 7fdfceffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdfceffd640 1 -- 172.21.15.5:0/1230507201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd8036070 con 0x55f03897db40 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdfceffd640 1 -- 172.21.15.5:0/1230507201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7fdfd8013850 con 0x55f03897db40 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1230507201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfd802e070 con 0x55f03897db40 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdfceffd640 1 -- 172.21.15.5:0/1230507201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdfd8042050 con 0x55f03897db40 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdff7fa1740 10 client.18033 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1230507201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f038a01de0 con 0x55f03897db40 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdfceffd640 1 -- 172.21.15.5:0/1230507201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfd802e070 con 0x55f03897db40 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdfceffd640 1 -- 172.21.15.5:0/1230507201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fdfd8017450 con 0x55f03897db40 2024-09-15T22:24:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdff7fa1740 10 client.18033 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdff7fa1740 10 client.18033 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdff7fa1740 20 client.18033 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f038a02470 0x55f038a04890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdff7fa1740 4 client.18033 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1230507201 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0388e9d70 con 0x55f038a02470 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.860+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f038a02470 0x55f038a04890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdfeffff640 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f038a02470 0x55f038a04890 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8d37dd6868ae476 server_cookie=5060ed81d58f86a0 in_seq=0 out_seq=0 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdfceffd640 10 client.18033 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdfceffd640 1 -- 172.21.15.5:0/1230507201 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0388e9d70 con 0x55f038a02470 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdfceffd640 10 client.18033 handle_command_reply: tid=0 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 1 client.18033 shutdown 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1230507201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 msgr2=0x55f038980e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038980e40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8003650 tx=0x7fdfd800afd0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1230507201 shutdown_connections 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55f038a02470 0x55f038a04890 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f038987370 0x55f038981450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f03897db40 0x55f038980e40 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdfceffd640 0 client.18033 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 1 --2- 172.21.15.5:0/1230507201 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f03897c5d0 0x55f03898fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1230507201 >> 172.21.15.5:0/1230507201 conn(0x55f038945310 msgr2=0x55f03897e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1230507201 shutdown_connections 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 1 -- 172.21.15.5:0/1230507201 wait complete. 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.861+0000 7fdff7fa1740 20 client.18033 trim_cache size 0 max 16384 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 -- 172.21.15.5:0/708745432 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdfdc0a3430 msgr2=0x7fdfdc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdfdc0a3430 0x7fdfdc0a58f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55f038979ac0 tx=0x7fdfe0009260 comp rx=0 tx=0).stop 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 -- 172.21.15.5:0/708745432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff0100a90 msgr2=0x7fdff0196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff0100a90 0x7fdff0196260 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fdfe002a940 tx=0x7fdfe0003b90 comp rx=0 tx=0).stop 2024-09-15T22:24:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 -- 172.21.15.5:0/708745432 shutdown_connections 2024-09-15T22:24:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fdfdc0a3430 0x7fdfdc0a58f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fdff0106a90 0x7fdff019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fdff0101430 0x7fdff01967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 --2- 172.21.15.5:0/708745432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdff0100a90 0x7fdff0196260 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 -- 172.21.15.5:0/708745432 >> 172.21.15.5:0/708745432 conn(0x7fdff00fc7d0 msgr2=0x7fdff00fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 -- 172.21.15.5:0/708745432 shutdown_connections 2024-09-15T22:24:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:37.862+0000 7fdff5bc8640 1 -- 172.21.15.5:0/708745432 wait complete. 2024-09-15T22:24:39.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:39 smithi099 ceph-mon[34892]: pgmap v493: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-09-15T22:24:39.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:39 smithi129 ceph-mon[35314]: pgmap v493: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-09-15T22:24:39.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:39 smithi005 ceph-mon[30944]: pgmap v493: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 4.8 MiB/s wr, 495 op/s 2024-09-15T22:24:40.719 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '398ff487', quiesce took 10.3 seconds. Will release after: 7.399999999999999 2024-09-15T22:24:41.261 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:24:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:41.769 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:41 smithi005 ceph-mon[30944]: pgmap v494: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.2 MiB/s wr, 532 op/s 2024-09-15T22:24:41.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:41 smithi099 ceph-mon[34892]: pgmap v494: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.2 MiB/s wr, 532 op/s 2024-09-15T22:24:41.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:41 smithi129 ceph-mon[35314]: pgmap v494: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 5.2 MiB/s wr, 532 op/s 2024-09-15T22:24:43.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:43 smithi099 ceph-mon[34892]: pgmap v495: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 6.5 MiB/s wr, 657 op/s 2024-09-15T22:24:43.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:43 smithi129 ceph-mon[35314]: pgmap v495: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 6.5 MiB/s wr, 657 op/s 2024-09-15T22:24:43.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:43 smithi005 ceph-mon[30944]: pgmap v495: 129 pgs: 129 active+clean; 2.9 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 6.5 MiB/s wr, 657 op/s 2024-09-15T22:24:45.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.961+0000 7f14b931f640 1 -- 172.21.15.5:0/2570554202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b410e500 msgr2=0x7f14b41108f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:45.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.961+0000 7f14b931f640 1 --2- 172.21.15.5:0/2570554202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b410e500 0x7f14b41108f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009ca0 tx=0x7f14ac02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.961+0000 7f14b931f640 1 -- 172.21.15.5:0/2570554202 shutdown_connections 2024-09-15T22:24:45.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.961+0000 7f14b931f640 1 --2- 172.21.15.5:0/2570554202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b410e500 0x7f14b41108f0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.961+0000 7f14b931f640 1 --2- 172.21.15.5:0/2570554202 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b4101300 0x7f14b410dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.961+0000 7f14b931f640 1 --2- 172.21.15.5:0/2570554202 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4100950 0x7f14b4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.961+0000 7f14b931f640 1 -- 172.21.15.5:0/2570554202 >> 172.21.15.5:0/2570554202 conn(0x7f14b40fc7a0 msgr2=0x7f14b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.961+0000 7f14b931f640 1 -- 172.21.15.5:0/2570554202 shutdown_connections 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.961+0000 7f14b931f640 1 -- 172.21.15.5:0/2570554202 wait complete. 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.961+0000 7f14b931f640 1 Processor -- start 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b931f640 1 -- start start 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b931f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4100950 0x7f14b419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b931f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b4101300 0x7f14b419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b931f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b410e500 0x7f14b419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b931f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4114180 con 0x7f14b4101300 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b931f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4114000 con 0x7f14b4100950 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b931f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f14b4114300 con 0x7f14b410e500 2024-09-15T22:24:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b4101300 0x7f14b419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b4101300 0x7f14b419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46224/0 (socket says 172.21.15.5:46224) 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b27fc640 1 -- 172.21.15.5:0/3624794973 learned_addr learned my addr 172.21.15.5:0/3624794973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4100950 0x7f14b419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.962+0000 7f14b37fe640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b410e500 0x7f14b419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.963+0000 7f14b27fc640 1 -- 172.21.15.5:0/3624794973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b410e500 msgr2=0x7f14b419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.963+0000 7f14b27fc640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b410e500 0x7f14b419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.963+0000 7f14b27fc640 1 -- 172.21.15.5:0/3624794973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4100950 msgr2=0x7f14b419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.963+0000 7f14b27fc640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4100950 0x7f14b419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.963+0000 7f14b27fc640 1 -- 172.21.15.5:0/3624794973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b419f610 con 0x7f14b4101300 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.963+0000 7f14b27fc640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b4101300 0x7f14b419aba0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f149c00e9f0 tx=0x7f149c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.963+0000 7f14abfff640 1 -- 172.21.15.5:0/3624794973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c00cd80 con 0x7f14b4101300 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.963+0000 7f14b931f640 1 -- 172.21.15.5:0/3624794973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b419f900 con 0x7f14b4101300 2024-09-15T22:24:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.963+0000 7f14abfff640 1 -- 172.21.15.5:0/3624794973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f149c005500 con 0x7f14b4101300 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.963+0000 7f14abfff640 1 -- 172.21.15.5:0/3624794973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c010690 con 0x7f14b4101300 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.963+0000 7f14b931f640 1 -- 172.21.15.5:0/3624794973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b41a71e0 con 0x7f14b4101300 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.965+0000 7f14abfff640 1 -- 172.21.15.5:0/3624794973 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f149c0046c0 con 0x7f14b4101300 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.966+0000 7f14bb6f8740 1 Processor -- start 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.966+0000 7f14bb6f8740 1 -- start start 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.966+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac208e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.966+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac208e5d0 0x555ac208ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.966+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac2099370 0x555ac209b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.966+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555ac1ff42f0 con 0x555ac208fb40 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.966+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555ac1f9bcf0 con 0x555ac2099370 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.966+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555ac1ff02b0 con 0x555ac208e5d0 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.966+0000 7f14b37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac208e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.966+0000 7f14b37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac208e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46228/0 (socket says 172.21.15.5:46228) 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.966+0000 7f14b37fe640 1 -- 172.21.15.5:0/3197016509 learned_addr learned my addr 172.21.15.5:0/3197016509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/3197016509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac208e5d0 0x555ac208ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14b27fc640 1 --2- 172.21.15.5:0/3197016509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac2099370 0x555ac209b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14abfff640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f14a00a33b0 0x7f14a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14b37fe640 1 -- 172.21.15.5:0/3197016509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac208e5d0 msgr2=0x555ac208ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14b37fe640 1 --2- 172.21.15.5:0/3197016509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac208e5d0 0x555ac208ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14b37fe640 1 -- 172.21.15.5:0/3197016509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac2099370 msgr2=0x555ac209b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14b37fe640 1 --2- 172.21.15.5:0/3197016509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac2099370 0x555ac209b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14b37fe640 1 -- 172.21.15.5:0/3197016509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac1f803a0 con 0x555ac208fb40 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14abfff640 1 -- 172.21.15.5:0/3624794973 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f149c0d0520 con 0x7f14b4101300 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14b37fe640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f14a00a33b0 0x7f14a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14b37fe640 1 --2- 172.21.15.5:0/3197016509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac208e090 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f14ac005e30 tx=0x7f14ac005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98364a4dc7f58c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14a97fa640 1 -- 172.21.15.5:0/3197016509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac042040 con 0x555ac208fb40 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.967+0000 7f14a97fa640 1 -- 172.21.15.5:0/3197016509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ac036040 con 0x555ac208fb40 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.968+0000 7f14b37fe640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f14a00a33b0 0x7f14a00a5870 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f14ac004af0 tx=0x7f14ac04e000 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.968+0000 7f14a97fa640 1 -- 172.21.15.5:0/3197016509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac02f910 con 0x555ac208fb40 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.968+0000 7f14bb6f8740 1 -- 172.21.15.5:0/3197016509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 msgr2=0x555ac208e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.968+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/3197016509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac208e090 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f14ac005e30 tx=0x7f14ac005fa0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.968+0000 7f14bb6f8740 1 -- 172.21.15.5:0/3197016509 shutdown_connections 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.968+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/3197016509 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac2099370 0x555ac209b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.968+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/3197016509 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac208e5d0 0x555ac208ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.968+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/3197016509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac208e090 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.968+0000 7f14bb6f8740 1 -- 172.21.15.5:0/3197016509 >> 172.21.15.5:0/3197016509 conn(0x555ac2057310 msgr2=0x555ac208ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.968+0000 7f14bb6f8740 1 -- 172.21.15.5:0/3197016509 shutdown_connections 2024-09-15T22:24:45.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.968+0000 7f14bb6f8740 1 -- 172.21.15.5:0/3197016509 wait complete. 2024-09-15T22:24:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14bb6f8740 1 -- start start 2024-09-15T22:24:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14bb6f8740 1 Processor -- start 2024-09-15T22:24:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac2098d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac2096030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac2099370 0x555ac2096640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555ac209c070 con 0x555ac208fb40 2024-09-15T22:24:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x555ac1faf060 con 0x555ac208e5d0 2024-09-15T22:24:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x555ac209c1f0 con 0x555ac2099370 2024-09-15T22:24:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14bb6f8740 10 client.0 _ll_register_callbacks cb 0x555ac1d028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:45.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14b27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac2096030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.104 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:24:45.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14b27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac2096030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46232/0 (socket says 172.21.15.5:46232) 2024-09-15T22:24:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14b27fc640 1 -- 172.21.15.5:0/4000173038 learned_addr learned my addr 172.21.15.5:0/4000173038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.969+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac2098d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14b37fe640 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac2099370 0x555ac2096640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14b27fc640 1 -- 172.21.15.5:0/4000173038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac2099370 msgr2=0x555ac2096640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14b27fc640 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac2099370 0x555ac2096640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14b27fc640 1 -- 172.21.15.5:0/4000173038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 msgr2=0x555ac2098d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14b27fc640 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac2098d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14b27fc640 1 -- 172.21.15.5:0/4000173038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac1d653c0 con 0x555ac208fb40 2024-09-15T22:24:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac2098d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14b37fe640 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac2099370 0x555ac2096640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14b27fc640 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac2096030 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f149c0cd870 tx=0x7f149c09c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aa25c7fe88e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f149a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f149a7fc640 1 -- 172.21.15.5:0/4000173038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c0cdca0 con 0x555ac208fb40 2024-09-15T22:24:45.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f149a7fc640 1 -- 172.21.15.5:0/4000173038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f149c0a4b80 con 0x555ac208fb40 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14bb6f8740 1 -- 172.21.15.5:0/4000173038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ac036040 con 0x555ac208fb40 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f149a7fc640 1 -- 172.21.15.5:0/4000173038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f149c09c3a0 con 0x555ac208fb40 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14bb6f8740 10 client.18045 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.970+0000 7f14bb6f8740 1 -- 172.21.15.5:0/4000173038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ac20a7ba0 con 0x555ac208fb40 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.971+0000 7f149a7fc640 1 -- 172.21.15.5:0/4000173038 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ac036040 con 0x555ac208fb40 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.971+0000 7f149a7fc640 1 -- 172.21.15.5:0/4000173038 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f149c09b7f0 con 0x555ac208fb40 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.971+0000 7f14bb6f8740 10 client.18045 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.971+0000 7f14bb6f8740 10 client.18045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.971+0000 7f14bb6f8740 20 client.18045 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.971+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555ac20a9060 0x555ac20ab450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.971+0000 7f14bb6f8740 4 client.18045 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.971+0000 7f14bb6f8740 1 -- 172.21.15.5:0/4000173038 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ac2001860 con 0x555ac20a9060 2024-09-15T22:24:45.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.971+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555ac20a9060 0x555ac20ab450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.971+0000 7f149a7fc640 10 client.18045 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.971+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555ac20a9060 0x555ac20ab450 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=637b03913463e933 server_cookie=a59aefdea78ae8e0 in_seq=0 out_seq=0 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f149a7fc640 1 -- 172.21.15.5:0/4000173038 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ac2001860 con 0x555ac20a9060 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f149a7fc640 10 client.18045 handle_command_reply: tid=0 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f14bb6f8740 1 client.18045 shutdown 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f14bb6f8740 1 -- 172.21.15.5:0/4000173038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 msgr2=0x555ac2096030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac2096030 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f149c0cd870 tx=0x7f149c09c2a0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f14bb6f8740 1 -- 172.21.15.5:0/4000173038 shutdown_connections 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac2099370 0x555ac2096640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac2096030 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555ac20a9060 0x555ac20ab450 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/4000173038 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac2098d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f14bb6f8740 1 -- 172.21.15.5:0/4000173038 >> 172.21.15.5:0/4000173038 conn(0x555ac2057310 msgr2=0x555ac2090960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.972+0000 7f149a7fc640 0 client.18045 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.973+0000 7f14bb6f8740 1 -- 172.21.15.5:0/4000173038 shutdown_connections 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.973+0000 7f14bb6f8740 1 -- 172.21.15.5:0/4000173038 wait complete. 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.973+0000 7f14bb6f8740 20 client.18045 trim_cache size 0 max 16384 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14bb6f8740 1 Processor -- start 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14bb6f8740 1 -- start start 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac20a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac208fb40 0x555ac2091120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac2099370 0x555ac2091660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41b3b50 con 0x555ac2099370 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41b1420 con 0x555ac208e5d0 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41af290 con 0x555ac208fb40 2024-09-15T22:24:45.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14b37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac2099370 0x555ac2091660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14b37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac2099370 0x555ac2091660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46244/0 (socket says 172.21.15.5:46244) 2024-09-15T22:24:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14b37fe640 1 -- 172.21.15.5:0/2121473535 learned_addr learned my addr 172.21.15.5:0/2121473535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/2121473535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac20a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.986+0000 7f14b27fc640 1 --2- 172.21.15.5:0/2121473535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac208fb40 0x555ac2091120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14b37fe640 1 -- 172.21.15.5:0/2121473535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac208fb40 msgr2=0x555ac2091120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14b37fe640 1 --2- 172.21.15.5:0/2121473535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac208fb40 0x555ac2091120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14b37fe640 1 -- 172.21.15.5:0/2121473535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 msgr2=0x555ac20a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14b37fe640 1 --2- 172.21.15.5:0/2121473535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac20a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14b37fe640 1 -- 172.21.15.5:0/2121473535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac208e2d0 con 0x555ac2099370 2024-09-15T22:24:45.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/2121473535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac20a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14b37fe640 1 --2- 172.21.15.5:0/2121473535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac2099370 0x555ac2091660 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f14ac02ade0 tx=0x7f14ac009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcf1fd4272c734c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14a8ff9640 1 -- 172.21.15.5:0/2121473535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac042040 con 0x555ac2099370 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14a8ff9640 1 -- 172.21.15.5:0/2121473535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ac031820 con 0x555ac2099370 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14a8ff9640 1 -- 172.21.15.5:0/2121473535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0044e0 con 0x555ac2099370 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14bb6f8740 1 -- 172.21.15.5:0/2121473535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac2099370 msgr2=0x555ac2091660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.987+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/2121473535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac2099370 0x555ac2091660 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f14ac02ade0 tx=0x7f14ac009bf0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 -- 172.21.15.5:0/2121473535 shutdown_connections 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/2121473535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac2099370 0x555ac2091660 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/2121473535 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac208fb40 0x555ac2091120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/2121473535 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac20a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 -- 172.21.15.5:0/2121473535 >> 172.21.15.5:0/2121473535 conn(0x555ac2057310 msgr2=0x555ac2095120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 -- 172.21.15.5:0/2121473535 shutdown_connections 2024-09-15T22:24:45.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 -- 172.21.15.5:0/2121473535 wait complete. 2024-09-15T22:24:45.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 -- start start 2024-09-15T22:24:45.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 Processor -- start 2024-09-15T22:24:45.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac209d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac209d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac2099370 0x555ac209deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41af8e0 con 0x555ac208fb40 2024-09-15T22:24:45.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.988+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41b0d50 con 0x555ac208e5d0 2024-09-15T22:24:45.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14bb6f8740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f14b41affb0 con 0x555ac2099370 2024-09-15T22:24:45.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14bb6f8740 10 client.0 _ll_register_callbacks cb 0x7f149c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:45.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac209d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac209d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46254/0 (socket says 172.21.15.5:46254) 2024-09-15T22:24:45.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b27fc640 1 -- 172.21.15.5:0/1904124746 learned_addr learned my addr 172.21.15.5:0/1904124746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:45.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b37fe640 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac2099370 0x555ac209deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac209d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b27fc640 1 -- 172.21.15.5:0/1904124746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac2099370 msgr2=0x555ac209deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:45.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b27fc640 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac2099370 0x555ac209deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b27fc640 1 -- 172.21.15.5:0/1904124746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 msgr2=0x555ac209d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:45.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b27fc640 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac209d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b27fc640 1 -- 172.21.15.5:0/1904124746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac2096fd0 con 0x555ac208fb40 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac209d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14b27fc640 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac209d970 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f149c09c330 tx=0x7f149c09bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d97d18dbe1e9d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f149a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f149a7fc640 1 -- 172.21.15.5:0/1904124746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c098370 con 0x555ac208fb40 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f149a7fc640 1 -- 172.21.15.5:0/1904124746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f149c09b260 con 0x555ac208fb40 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14bb6f8740 1 -- 172.21.15.5:0/1904124746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14ac031820 con 0x555ac208fb40 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f149a7fc640 1 -- 172.21.15.5:0/1904124746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f149c0a4d90 con 0x555ac208fb40 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14bb6f8740 10 client.18051 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.989+0000 7f14bb6f8740 1 -- 172.21.15.5:0/1904124746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555ac2114010 con 0x555ac208fb40 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.990+0000 7f149a7fc640 1 -- 172.21.15.5:0/1904124746 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14ac031820 con 0x555ac208fb40 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.990+0000 7f149a7fc640 1 -- 172.21.15.5:0/1904124746 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f149c0bc0b0 con 0x555ac208fb40 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.990+0000 7f14bb6f8740 10 client.18051 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:45.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.990+0000 7f14bb6f8740 10 client.18051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.990+0000 7f14bb6f8740 20 client.18051 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.990+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555ac21146d0 0x555ac2116af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.990+0000 7f14bb6f8740 4 client.18051 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.990+0000 7f14bb6f8740 1 -- 172.21.15.5:0/1904124746 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ac1ffbd70 con 0x555ac21146d0 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.990+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555ac21146d0 0x555ac2116af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.990+0000 7f14b2ffd640 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555ac21146d0 0x555ac2116af0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2348a4eadf6452c6 server_cookie=218cdc0c4752652e in_seq=0 out_seq=0 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.990+0000 7f149a7fc640 10 client.18051 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f149a7fc640 1 -- 172.21.15.5:0/1904124746 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ac1ffbd70 con 0x555ac21146d0 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f149a7fc640 10 client.18051 handle_command_reply: tid=0 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 1 client.18051 shutdown 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 1 -- 172.21.15.5:0/1904124746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 msgr2=0x555ac209d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac209d970 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f149c09c330 tx=0x7f149c09bb20 comp rx=0 tx=0).stop 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 1 -- 172.21.15.5:0/1904124746 shutdown_connections 2024-09-15T22:24:45.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x555ac2099370 0x555ac209deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555ac208fb40 0x555ac209d970 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x555ac21146d0 0x555ac2116af0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 1 --2- 172.21.15.5:0/1904124746 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x555ac208e5d0 0x555ac209d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 1 -- 172.21.15.5:0/1904124746 >> 172.21.15.5:0/1904124746 conn(0x555ac2057310 msgr2=0x555ac2097890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f149a7fc640 0 client.18051 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 1 -- 172.21.15.5:0/1904124746 shutdown_connections 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 1 -- 172.21.15.5:0/1904124746 wait complete. 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.991+0000 7f14bb6f8740 20 client.18051 trim_cache size 0 max 16384 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 -- 172.21.15.5:0/3624794973 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f14a00a33b0 msgr2=0x7f14a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f14a00a33b0 0x7f14a00a5870 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f14ac004af0 tx=0x7f14ac04e000 comp rx=0 tx=0).stop 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 -- 172.21.15.5:0/3624794973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b4101300 msgr2=0x7f14b419aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b4101300 0x7f14b419aba0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f149c00e9f0 tx=0x7f149c00edf0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 -- 172.21.15.5:0/3624794973 shutdown_connections 2024-09-15T22:24:45.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f14a00a33b0 0x7f14a00a5870 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f14b410e500 0x7f14b419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14b4101300 0x7f14b419aba0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 --2- 172.21.15.5:0/3624794973 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f14b4100950 0x7f14b419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:45.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 -- 172.21.15.5:0/3624794973 >> 172.21.15.5:0/3624794973 conn(0x7f14b40fc7a0 msgr2=0x7f14b40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:45.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 -- 172.21.15.5:0/3624794973 shutdown_connections 2024-09-15T22:24:45.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:44.992+0000 7f14b931f640 1 -- 172.21.15.5:0/3624794973 wait complete. 2024-09-15T22:24:45.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:45 smithi005 ceph-mon[30944]: pgmap v496: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-09-15T22:24:45.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:45 smithi099 ceph-mon[34892]: pgmap v496: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-09-15T22:24:45.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:45 smithi129 ceph-mon[35314]: pgmap v496: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 5.6 MiB/s wr, 587 op/s 2024-09-15T22:24:46.837 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-15T22:24:46.837 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-15T22:24:47.113 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:47.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:47 smithi005 ceph-mon[30944]: pgmap v497: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.7 MiB/s wr, 588 op/s 2024-09-15T22:24:47.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:47 smithi099 ceph-mon[34892]: pgmap v497: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.7 MiB/s wr, 588 op/s 2024-09-15T22:24:47.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:47 smithi129 ceph-mon[35314]: pgmap v497: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 5.7 MiB/s wr, 588 op/s 2024-09-15T22:24:47.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.811+0000 7f43835d0640 1 -- 172.21.15.5:0/249165684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437c102570 msgr2=0x7f437c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.811+0000 7f43835d0640 1 --2- 172.21.15.5:0/249165684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437c102570 0x7f437c1029b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4370009ca0 tx=0x7f437002a950 comp rx=0 tx=0).stop 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.812+0000 7f43835d0640 1 -- 172.21.15.5:0/249165684 shutdown_connections 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.812+0000 7f43835d0640 1 --2- 172.21.15.5:0/249165684 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f437c102ef0 0x7f437c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.812+0000 7f43835d0640 1 --2- 172.21.15.5:0/249165684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437c102570 0x7f437c1029b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.812+0000 7f43835d0640 1 --2- 172.21.15.5:0/249165684 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c100e20 0x7f437c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.812+0000 7f43835d0640 1 -- 172.21.15.5:0/249165684 >> 172.21.15.5:0/249165684 conn(0x7f437c0fc7d0 msgr2=0x7f437c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.812+0000 7f43835d0640 1 -- 172.21.15.5:0/249165684 shutdown_connections 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.812+0000 7f43835d0640 1 -- 172.21.15.5:0/249165684 wait complete. 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.812+0000 7f43835d0640 1 Processor -- start 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.812+0000 7f43835d0640 1 -- start start 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.812+0000 7f43835d0640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c100e20 0x7f437c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f43835d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437c102570 0x7f437c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f43835d0640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f437c102ef0 0x7f437c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f43835d0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1120b0 con 0x7f437c102570 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f43835d0640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f437c111f30 con 0x7f437c100e20 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f43835d0640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f437c112230 con 0x7f437c102ef0 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f4380b6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437c102570 0x7f437c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f4380b6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437c102570 0x7f437c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46274/0 (socket says 172.21.15.5:46274) 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f4380b6b640 1 -- 172.21.15.5:0/821869092 learned_addr learned my addr 172.21.15.5:0/821869092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f438136c640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c100e20 0x7f437c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f4381b6d640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f437c102ef0 0x7f437c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f4380b6b640 1 -- 172.21.15.5:0/821869092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f437c102ef0 msgr2=0x7f437c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f4380b6b640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f437c102ef0 0x7f437c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f4380b6b640 1 -- 172.21.15.5:0/821869092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c100e20 msgr2=0x7f437c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f4380b6b640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c100e20 0x7f437c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f4380b6b640 1 -- 172.21.15.5:0/821869092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c19d4e0 con 0x7f437c102570 2024-09-15T22:24:47.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f4381b6d640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f437c102ef0 0x7f437c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f4380b6b640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437c102570 0x7f437c198a70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4370005cc0 tx=0x7f437002f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f437a7fc640 1 -- 172.21.15.5:0/821869092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370034040 con 0x7f437c102570 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f43835d0640 1 -- 172.21.15.5:0/821869092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c19d770 con 0x7f437c102570 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f437a7fc640 1 -- 172.21.15.5:0/821869092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4370030920 con 0x7f437c102570 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.813+0000 7f437a7fc640 1 -- 172.21.15.5:0/821869092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370004830 con 0x7f437c102570 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.814+0000 7f43835d0640 1 -- 172.21.15.5:0/821869092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c1a50b0 con 0x7f437c102570 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.816+0000 7f437a7fc640 1 -- 172.21.15.5:0/821869092 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4370037020 con 0x7f437c102570 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f43859a9740 1 Processor -- start 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f43859a9740 1 -- start start 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f43859a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789921b60 0x558789921f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f43859a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558789922480 0x55878992c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f43859a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f43859a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558789887280 con 0x55878992cd40 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f43859a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55878982eb10 con 0x558789921b60 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f43859a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x558789883240 con 0x558789922480 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f437a7fc640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43640a33b0 0x7f43640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f4380b6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f4380b6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46278/0 (socket says 172.21.15.5:46278) 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f4380b6b640 1 -- 172.21.15.5:0/2930175061 learned_addr learned my addr 172.21.15.5:0/2930175061 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f4381b6d640 1 --2- 172.21.15.5:0/2930175061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789921b60 0x558789921f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.817+0000 7f438136c640 1 --2- 172.21.15.5:0/2930175061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558789922480 0x55878992c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f4381b6d640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43640a33b0 0x7f43640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f437a7fc640 1 -- 172.21.15.5:0/821869092 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f43700f1000 con 0x7f437c102570 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f4380b6b640 1 -- 172.21.15.5:0/2930175061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558789922480 msgr2=0x55878992c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f4380b6b640 1 --2- 172.21.15.5:0/2930175061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558789922480 0x55878992c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f4380b6b640 1 -- 172.21.15.5:0/2930175061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789921b60 msgr2=0x558789921f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f4380b6b640 1 --2- 172.21.15.5:0/2930175061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789921b60 0x558789921f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f4380b6b640 1 -- 172.21.15.5:0/2930175061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55878983b1f0 con 0x55878992cd40 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f438136c640 1 --2- 172.21.15.5:0/2930175061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558789922480 0x55878992c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f4380b6b640 1 --2- 172.21.15.5:0/2930175061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992f130 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f43700ee510 tx=0x7f437002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aeb854dad4d8af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f4381b6d640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43640a33b0 0x7f43640a5870 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4374006fd0 tx=0x7f4374006c00 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f43637fe640 1 -- 172.21.15.5:0/2930175061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43700bd6f0 con 0x55878992cd40 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f43637fe640 1 -- 172.21.15.5:0/2930175061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43700c5c60 con 0x55878992cd40 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.818+0000 7f43637fe640 1 -- 172.21.15.5:0/2930175061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43700ee5d0 con 0x55878992cd40 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.819+0000 7f43859a9740 1 -- 172.21.15.5:0/2930175061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 msgr2=0x55878992f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.819+0000 7f43859a9740 1 --2- 172.21.15.5:0/2930175061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992f130 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f43700ee510 tx=0x7f437002a950 comp rx=0 tx=0).stop 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.819+0000 7f43859a9740 1 -- 172.21.15.5:0/2930175061 shutdown_connections 2024-09-15T22:24:47.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.819+0000 7f43859a9740 1 --2- 172.21.15.5:0/2930175061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992f130 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.819+0000 7f43859a9740 1 --2- 172.21.15.5:0/2930175061 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558789922480 0x55878992c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.819+0000 7f43859a9740 1 --2- 172.21.15.5:0/2930175061 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789921b60 0x558789921f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.819+0000 7f43859a9740 1 -- 172.21.15.5:0/2930175061 >> 172.21.15.5:0/2930175061 conn(0x55878991e0f0 msgr2=0x55878991e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.819+0000 7f43859a9740 1 -- 172.21.15.5:0/2930175061 shutdown_connections 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.819+0000 7f43859a9740 1 -- 172.21.15.5:0/2930175061 wait complete. 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.819+0000 7f43859a9740 1 -- start start 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.819+0000 7f43859a9740 1 Processor -- start 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f43859a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558789921b60 0x558789929c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f43859a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 0x55878992a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f43859a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f43859a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55878992f6d0 con 0x55878992cd40 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f43859a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x558789841f10 con 0x558789922480 2024-09-15T22:24:47.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f43859a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55878992f850 con 0x558789921b60 2024-09-15T22:24:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f43859a9740 10 client.0 _ll_register_callbacks cb 0x55878980aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4381b6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4381b6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46294/0 (socket says 172.21.15.5:46294) 2024-09-15T22:24:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4381b6d640 1 -- 172.21.15.5:0/1119105728 learned_addr learned my addr 172.21.15.5:0/1119105728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4380b6b640 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 0x55878992a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f438136c640 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558789921b60 0x558789929c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4381b6d640 1 -- 172.21.15.5:0/1119105728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558789921b60 msgr2=0x558789929c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4381b6d640 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558789921b60 0x558789929c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4381b6d640 1 -- 172.21.15.5:0/1119105728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 msgr2=0x55878992a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4381b6d640 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 0x55878992a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4381b6d640 1 -- 172.21.15.5:0/1119105728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587895f86a0 con 0x55878992cd40 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4380b6b640 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 0x55878992a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4381b6d640 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992a6d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4374017d50 tx=0x7f43740047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f716911978c9cd75 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.820+0000 7f4360ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f4360ff9640 1 -- 172.21.15.5:0/1119105728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374027060 con 0x55878992cd40 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f4360ff9640 1 -- 172.21.15.5:0/1119105728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f4374016d00 con 0x55878992cd40 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f43859a9740 1 -- 172.21.15.5:0/1119105728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43700c5c60 con 0x55878992cd40 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f4360ff9640 1 -- 172.21.15.5:0/1119105728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f437402e050 con 0x55878992cd40 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f43859a9740 10 client.18063 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f43859a9740 1 -- 172.21.15.5:0/1119105728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55878993f730 con 0x55878992cd40 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f4360ff9640 1 -- 172.21.15.5:0/1119105728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43700c5c60 con 0x55878992cd40 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f4360ff9640 1 -- 172.21.15.5:0/1119105728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4374030370 con 0x55878992cd40 2024-09-15T22:24:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f43859a9740 10 client.18063 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f43859a9740 10 client.18063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f43859a9740 20 client.18063 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f43859a9740 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558789940bf0 0x558789943020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f43859a9740 4 client.18063 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.821+0000 7f43859a9740 1 -- 172.21.15.5:0/1119105728 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587898947f0 con 0x558789940bf0 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.822+0000 7f438136c640 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558789940bf0 0x558789943020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.822+0000 7f438136c640 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558789940bf0 0x558789943020 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1da630329cdeb7d server_cookie=1ae40cbef871c972 in_seq=0 out_seq=0 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.822+0000 7f4360ff9640 10 client.18063 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f4360ff9640 1 -- 172.21.15.5:0/1119105728 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587898947f0 con 0x558789940bf0 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f4360ff9640 10 client.18063 handle_command_reply: tid=0 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 1 client.18063 shutdown 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 1 -- 172.21.15.5:0/1119105728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 msgr2=0x55878992a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992a6d0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4374017d50 tx=0x7f43740047d0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 1 -- 172.21.15.5:0/1119105728 shutdown_connections 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55878992cd40 0x55878992a6d0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x558789940bf0 0x558789943020 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 0x55878992a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 1 --2- 172.21.15.5:0/1119105728 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x558789921b60 0x558789929c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f4360ff9640 0 client.18063 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 1 -- 172.21.15.5:0/1119105728 >> 172.21.15.5:0/1119105728 conn(0x55878991e0f0 msgr2=0x558789923dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 1 -- 172.21.15.5:0/1119105728 shutdown_connections 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 1 -- 172.21.15.5:0/1119105728 wait complete. 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.823+0000 7f43859a9740 20 client.18063 trim_cache size 0 max 16384 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.840+0000 7f43859a9740 1 Processor -- start 2024-09-15T22:24:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.840+0000 7f43859a9740 1 -- start start 2024-09-15T22:24:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.840+0000 7f43859a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789921b60 0x55878992bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.840+0000 7f43859a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 0x5587899488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f43859a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 0x558789948e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f43859a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1af770 con 0x558789921b60 2024-09-15T22:24:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f43859a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1ad040 con 0x558789922480 2024-09-15T22:24:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f43859a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1aaeb0 con 0x55878992cd40 2024-09-15T22:24:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f438136c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789921b60 0x55878992bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f438136c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789921b60 0x55878992bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46300/0 (socket says 172.21.15.5:46300) 2024-09-15T22:24:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f438136c640 1 -- 172.21.15.5:0/187230322 learned_addr learned my addr 172.21.15.5:0/187230322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f4380b6b640 1 --2- 172.21.15.5:0/187230322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 0x5587899488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f438136c640 1 -- 172.21.15.5:0/187230322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 msgr2=0x558789948e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f4381b6d640 1 --2- 172.21.15.5:0/187230322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 0x558789948e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f438136c640 1 --2- 172.21.15.5:0/187230322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 0x558789948e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f438136c640 1 -- 172.21.15.5:0/187230322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 msgr2=0x5587899488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f438136c640 1 --2- 172.21.15.5:0/187230322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 0x5587899488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f438136c640 1 -- 172.21.15.5:0/187230322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558789928c00 con 0x558789921b60 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f4380b6b640 1 --2- 172.21.15.5:0/187230322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 0x5587899488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f4381b6d640 1 --2- 172.21.15.5:0/187230322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 0x558789948e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f438136c640 1 --2- 172.21.15.5:0/187230322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789921b60 0x55878992bab0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f436c00ba50 tx=0x7f436c0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c297910bcf1c4fd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f4362ffd640 1 -- 172.21.15.5:0/187230322 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c003e20 con 0x558789921b60 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f4362ffd640 1 -- 172.21.15.5:0/187230322 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c011920 con 0x558789921b60 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.841+0000 7f4362ffd640 1 -- 172.21.15.5:0/187230322 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c01ca60 con 0x558789921b60 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.842+0000 7f43859a9740 1 -- 172.21.15.5:0/187230322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789921b60 msgr2=0x55878992bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.842+0000 7f43859a9740 1 --2- 172.21.15.5:0/187230322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789921b60 0x55878992bab0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f436c00ba50 tx=0x7f436c0045b0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.842+0000 7f43859a9740 1 -- 172.21.15.5:0/187230322 shutdown_connections 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.842+0000 7f43859a9740 1 --2- 172.21.15.5:0/187230322 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 0x558789948e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.842+0000 7f43859a9740 1 --2- 172.21.15.5:0/187230322 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789922480 0x5587899488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.842+0000 7f43859a9740 1 --2- 172.21.15.5:0/187230322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789921b60 0x55878992bab0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.842+0000 7f43859a9740 1 -- 172.21.15.5:0/187230322 >> 172.21.15.5:0/187230322 conn(0x55878991e0f0 msgr2=0x55878992b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.842+0000 7f43859a9740 1 -- 172.21.15.5:0/187230322 shutdown_connections 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.842+0000 7f43859a9740 1 -- 172.21.15.5:0/187230322 wait complete. 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.842+0000 7f43859a9740 1 -- start start 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.842+0000 7f43859a9740 1 Processor -- start 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f43859a9740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789921b60 0x558789934f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f43859a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789922480 0x5587899354d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f43859a9740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 0x5587899aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f43859a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1ab500 con 0x558789922480 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f43859a9740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1ac970 con 0x558789921b60 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f43859a9740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1abbd0 con 0x55878992cd40 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f43859a9740 10 client.0 _ll_register_callbacks cb 0x7f437400ccd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4380b6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789922480 0x5587899354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4380b6b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789922480 0x5587899354d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46310/0 (socket says 172.21.15.5:46310) 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4380b6b640 1 -- 172.21.15.5:0/2150625874 learned_addr learned my addr 172.21.15.5:0/2150625874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f438136c640 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789921b60 0x558789934f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4380b6b640 1 -- 172.21.15.5:0/2150625874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 msgr2=0x5587899aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4381b6d640 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 0x5587899aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4380b6b640 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 0x5587899aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4380b6b640 1 -- 172.21.15.5:0/2150625874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789921b60 msgr2=0x558789934f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4380b6b640 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789921b60 0x558789934f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4380b6b640 1 -- 172.21.15.5:0/2150625874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558789919830 con 0x558789922480 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f438136c640 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789921b60 0x558789934f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4381b6d640 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 0x5587899aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4380b6b640 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789922480 0x5587899354d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f43700049d0 tx=0x7f43700c14e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3bef3728bd4fae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4360ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4360ff9640 1 -- 172.21.15.5:0/2150625874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43700c5c60 con 0x558789922480 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f4360ff9640 1 -- 172.21.15.5:0/2150625874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f43700c57d0 con 0x558789922480 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.843+0000 7f43859a9740 1 -- 172.21.15.5:0/2150625874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f436c011920 con 0x558789922480 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f4360ff9640 1 -- 172.21.15.5:0/2150625874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43700bd6f0 con 0x558789922480 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f43859a9740 10 client.18069 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f43859a9740 1 -- 172.21.15.5:0/2150625874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5587899abd20 con 0x558789922480 2024-09-15T22:24:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f4360ff9640 1 -- 172.21.15.5:0/2150625874 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c011920 con 0x558789922480 2024-09-15T22:24:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f4360ff9640 1 -- 172.21.15.5:0/2150625874 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f43700a7030 con 0x558789922480 2024-09-15T22:24:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f43859a9740 10 client.18069 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f43859a9740 10 client.18069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f43859a9740 20 client.18069 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f43859a9740 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587899ac250 0x5587899ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f43859a9740 4 client.18069 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-15T22:24:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f43859a9740 1 -- 172.21.15.5:0/2150625874 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55878988ed00 con 0x5587899ac250 2024-09-15T22:24:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.844+0000 7f438136c640 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587899ac250 0x5587899ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:47.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.845+0000 7f4360ff9640 10 client.18069 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.845+0000 7f438136c640 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587899ac250 0x5587899ae640 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91b9d243b651b308 server_cookie=8515983905257f88 in_seq=0 out_seq=0 2024-09-15T22:24:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f4360ff9640 1 -- 172.21.15.5:0/2150625874 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55878988ed00 con 0x5587899ac250 2024-09-15T22:24:47.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f4360ff9640 10 client.18069 handle_command_reply: tid=0 2024-09-15T22:24:47.890 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:24:47.890 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-15T22:24:47.890 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "9ebef968-4cab-4a1c-a5f1-141bbedc263f", 2024-09-15T22:24:47.890 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-15T22:24:47.891 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:24:47.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 1 client.18069 shutdown 2024-09-15T22:24:47.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 1 -- 172.21.15.5:0/2150625874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789922480 msgr2=0x5587899354d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789922480 0x5587899354d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f43700049d0 tx=0x7f43700c14e0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 1 -- 172.21.15.5:0/2150625874 shutdown_connections 2024-09-15T22:24:47.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55878992cd40 0x5587899aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558789922480 0x5587899354d0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5587899ac250 0x5587899ae640 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 1 --2- 172.21.15.5:0/2150625874 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x558789921b60 0x558789934f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f4360ff9640 0 client.18069 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:47.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 1 -- 172.21.15.5:0/2150625874 >> 172.21.15.5:0/2150625874 conn(0x55878991e0f0 msgr2=0x55878992a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 1 -- 172.21.15.5:0/2150625874 shutdown_connections 2024-09-15T22:24:47.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 1 -- 172.21.15.5:0/2150625874 wait complete. 2024-09-15T22:24:47.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.846+0000 7f43859a9740 20 client.18069 trim_cache size 0 max 16384 2024-09-15T22:24:47.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 -- 172.21.15.5:0/821869092 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43640a33b0 msgr2=0x7f43640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43640a33b0 0x7f43640a5870 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4374006fd0 tx=0x7f4374006c00 comp rx=0 tx=0).stop 2024-09-15T22:24:47.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 -- 172.21.15.5:0/821869092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437c102570 msgr2=0x7f437c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:47.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437c102570 0x7f437c198a70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4370005cc0 tx=0x7f437002f650 comp rx=0 tx=0).stop 2024-09-15T22:24:47.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 -- 172.21.15.5:0/821869092 shutdown_connections 2024-09-15T22:24:47.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f43640a33b0 0x7f43640a5870 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f437c102ef0 0x7f437c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f437c102570 0x7f437c198a70 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 --2- 172.21.15.5:0/821869092 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f437c100e20 0x7f437c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:47.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 -- 172.21.15.5:0/821869092 >> 172.21.15.5:0/821869092 conn(0x7f437c0fc7d0 msgr2=0x7f437c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:47.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 -- 172.21.15.5:0/821869092 shutdown_connections 2024-09-15T22:24:47.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:47.847+0000 7f43835d0640 1 -- 172.21.15.5:0/821869092 wait complete. 2024-09-15T22:24:48.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:24:48.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:48 smithi005 ceph-mon[30944]: scrub queued for path: / 2024-09-15T22:24:48.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:48 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:48.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:48 smithi005 ceph-mon[30944]: scrub summary: active paths [/] 2024-09-15T22:24:48.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:48 smithi005 ceph-mon[30944]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:48.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:24:48.816 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:48.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:48 smithi099 ceph-mon[34892]: scrub queued for path: / 2024-09-15T22:24:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:48 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:48 smithi099 ceph-mon[34892]: scrub summary: active paths [/] 2024-09-15T22:24:48.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:48 smithi099 ceph-mon[34892]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:48 smithi129 ceph-mon[35314]: scrub queued for path: / 2024-09-15T22:24:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:48 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:48 smithi129 ceph-mon[35314]: scrub summary: active paths [/] 2024-09-15T22:24:48.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:48 smithi129 ceph-mon[35314]: scrub summary: idle+waiting paths [/] 2024-09-15T22:24:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.296+0000 7fc55d51f640 1 -- 172.21.15.5:0/906970956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc558101de0 msgr2=0x7fc5581021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.296+0000 7fc55d51f640 1 --2- 172.21.15.5:0/906970956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc558101de0 0x7fc5581021c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc548009900 tx=0x7fc54802a410 comp rx=0 tx=0).stop 2024-09-15T22:24:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.296+0000 7fc55d51f640 1 -- 172.21.15.5:0/906970956 shutdown_connections 2024-09-15T22:24:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.296+0000 7fc55d51f640 1 --2- 172.21.15.5:0/906970956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc558106f10 0x7fc55810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.296+0000 7fc55d51f640 1 --2- 172.21.15.5:0/906970956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc558102790 0x7fc5581067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.296+0000 7fc55d51f640 1 --2- 172.21.15.5:0/906970956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc558101de0 0x7fc5581021c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.296+0000 7fc55d51f640 1 -- 172.21.15.5:0/906970956 >> 172.21.15.5:0/906970956 conn(0x7fc5580fd8c0 msgr2=0x7fc5580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:49.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc55d51f640 1 -- 172.21.15.5:0/906970956 shutdown_connections 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc55d51f640 1 -- 172.21.15.5:0/906970956 wait complete. 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc55d51f640 1 Processor -- start 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc55d51f640 1 -- start start 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc55d51f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc558101de0 0x7fc5581984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc55d51f640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc558102790 0x7fc5581989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc55d51f640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc558106f10 0x7fc55819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc55d51f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc558111f70 con 0x7fc558101de0 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc55d51f640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fc558111df0 con 0x7fc558102790 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc55d51f640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fc5581120f0 con 0x7fc558106f10 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc556ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc558101de0 0x7fc5581984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc556ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc558101de0 0x7fc5581984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46334/0 (socket says 172.21.15.5:46334) 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.297+0000 7fc556ffd640 1 -- 172.21.15.5:0/391814328 learned_addr learned my addr 172.21.15.5:0/391814328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:49.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc5567fc640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc558102790 0x7fc5581989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc5577fe640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc558106f10 0x7fc55819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc556ffd640 1 -- 172.21.15.5:0/391814328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc558106f10 msgr2=0x7fc55819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc556ffd640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc558106f10 0x7fc55819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc556ffd640 1 -- 172.21.15.5:0/391814328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc558102790 msgr2=0x7fc5581989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc556ffd640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc558102790 0x7fc5581989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc556ffd640 1 -- 172.21.15.5:0/391814328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55819d460 con 0x7fc558101de0 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc5577fe640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc558106f10 0x7fc55819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc556ffd640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc558101de0 0x7fc5581984b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc548006010 tx=0x7fc5480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc53ffff640 1 -- 172.21.15.5:0/391814328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54803f030 con 0x7fc558101de0 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc55d51f640 1 -- 172.21.15.5:0/391814328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc55819d6f0 con 0x7fc558101de0 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc53ffff640 1 -- 172.21.15.5:0/391814328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc548003ce0 con 0x7fc558101de0 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc53ffff640 1 -- 172.21.15.5:0/391814328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc548035070 con 0x7fc558101de0 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.298+0000 7fc55d51f640 1 -- 172.21.15.5:0/391814328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5581a5030 con 0x7fc558101de0 2024-09-15T22:24:49.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.299+0000 7fc55d51f640 1 -- 172.21.15.5:0/391814328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5581021c0 con 0x7fc558101de0 2024-09-15T22:24:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.300+0000 7fc53ffff640 1 -- 172.21.15.5:0/391814328 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fc548042030 con 0x7fc558101de0 2024-09-15T22:24:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.301+0000 7fc53ffff640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5400a33b0 0x7fc5400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.301+0000 7fc53ffff640 1 -- 172.21.15.5:0/391814328 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7fc5480f0b30 con 0x7fc558101de0 2024-09-15T22:24:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.301+0000 7fc5567fc640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5400a33b0 0x7fc5400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.302+0000 7fc5567fc640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5400a33b0 0x7fc5400a5870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc54c005fd0 tx=0x7fc54c005850 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:49.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.304+0000 7fc53ffff640 1 -- 172.21.15.5:0/391814328 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5480bd1d0 con 0x7fc558101de0 2024-09-15T22:24:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.525+0000 7fc55d51f640 1 -- 172.21.15.5:0/391814328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc55810a0f0 con 0x7fc558101de0 2024-09-15T22:24:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.527+0000 7fc53ffff640 1 -- 172.21.15.5:0/391814328 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fc5480bcad0 con 0x7fc558101de0 2024-09-15T22:24:49.529 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:49.529 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:24:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:24:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 -- 172.21.15.5:0/391814328 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5400a33b0 msgr2=0x7fc5400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5400a33b0 0x7fc5400a5870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc54c005fd0 tx=0x7fc54c005850 comp rx=0 tx=0).stop 2024-09-15T22:24:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 -- 172.21.15.5:0/391814328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc558101de0 msgr2=0x7fc5581984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc558101de0 0x7fc5581984b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc548006010 tx=0x7fc5480042f0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 -- 172.21.15.5:0/391814328 shutdown_connections 2024-09-15T22:24:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fc5400a33b0 0x7fc5400a5870 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fc558106f10 0x7fc55819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fc558102790 0x7fc5581989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 --2- 172.21.15.5:0/391814328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc558101de0 0x7fc5581984b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 -- 172.21.15.5:0/391814328 >> 172.21.15.5:0/391814328 conn(0x7fc5580fd8c0 msgr2=0x7fc5580fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 -- 172.21.15.5:0/391814328 shutdown_connections 2024-09-15T22:24:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:49.530+0000 7fc55d51f640 1 -- 172.21.15.5:0/391814328 wait complete. 2024-09-15T22:24:49.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:49 smithi005 ceph-mon[30944]: pgmap v498: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-09-15T22:24:49.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:49 smithi099 ceph-mon[34892]: pgmap v498: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-09-15T22:24:49.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:49 smithi129 ceph-mon[35314]: pgmap v498: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 4.6 MiB/s wr, 475 op/s 2024-09-15T22:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.034+0000 7f51a1f2e640 1 -- 172.21.15.5:0/2244916955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f519c10e500 msgr2=0x7f519c110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.034+0000 7f51a1f2e640 1 --2- 172.21.15.5:0/2244916955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f519c10e500 0x7f519c110910 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5194009ca0 tx=0x7f519402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.034+0000 7f51a1f2e640 1 -- 172.21.15.5:0/2244916955 shutdown_connections 2024-09-15T22:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.034+0000 7f51a1f2e640 1 --2- 172.21.15.5:0/2244916955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f519c10e500 0x7f519c110910 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.034+0000 7f51a1f2e640 1 --2- 172.21.15.5:0/2244916955 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f519c101440 0x7f519c10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.034+0000 7f51a1f2e640 1 --2- 172.21.15.5:0/2244916955 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f519c100b20 0x7f519c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.034+0000 7f51a1f2e640 1 -- 172.21.15.5:0/2244916955 >> 172.21.15.5:0/2244916955 conn(0x7f519c0fc7d0 msgr2=0x7f519c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.034+0000 7f51a1f2e640 1 -- 172.21.15.5:0/2244916955 shutdown_connections 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.034+0000 7f51a1f2e640 1 -- 172.21.15.5:0/2244916955 wait complete. 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f51a1f2e640 1 Processor -- start 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f51a1f2e640 1 -- start start 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f51a1f2e640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f519c100b20 0x7f519c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f51a1f2e640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f519c101440 0x7f519c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f51a1f2e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f519c10e500 0x7f519c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f51a1f2e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f519c114250 con 0x7f519c10e500 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f51a1f2e640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1140d0 con 0x7f519c100b20 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f51a1f2e640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1143d0 con 0x7f519c101440 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f519bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f519c10e500 0x7f519c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f519bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f519c10e500 0x7f519c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46342/0 (socket says 172.21.15.5:46342) 2024-09-15T22:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f519bfff640 1 -- 172.21.15.5:0/3757641285 learned_addr learned my addr 172.21.15.5:0/3757641285 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f519b7fe640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f519c100b20 0x7f519c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f5193fff640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f519c101440 0x7f519c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f519bfff640 1 -- 172.21.15.5:0/3757641285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f519c101440 msgr2=0x7f519c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f519bfff640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f519c101440 0x7f519c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f519bfff640 1 -- 172.21.15.5:0/3757641285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f519c100b20 msgr2=0x7f519c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f519bfff640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f519c100b20 0x7f519c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.035+0000 7f519bfff640 1 -- 172.21.15.5:0/3757641285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519c19f6a0 con 0x7f519c10e500 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.036+0000 7f519bfff640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f519c10e500 0x7f519c19efc0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f51940319f0 tx=0x7f5194031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.036+0000 7f51997fa640 1 -- 172.21.15.5:0/3757641285 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194003c30 con 0x7f519c10e500 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.036+0000 7f51997fa640 1 -- 172.21.15.5:0/3757641285 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5194003dd0 con 0x7f519c10e500 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.036+0000 7f51a1f2e640 1 -- 172.21.15.5:0/3757641285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519c19f930 con 0x7f519c10e500 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.036+0000 7f51997fa640 1 -- 172.21.15.5:0/3757641285 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519402fcb0 con 0x7f519c10e500 2024-09-15T22:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.036+0000 7f51a1f2e640 1 -- 172.21.15.5:0/3757641285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f519c1a7270 con 0x7f519c10e500 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.038+0000 7f51997fa640 1 -- 172.21.15.5:0/3757641285 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f5194004050 con 0x7f519c10e500 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.038+0000 7f51a4307740 1 Processor -- start 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.038+0000 7f51a4307740 1 -- start start 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f51a4307740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa95a4a0 0x563eaa95a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f51a4307740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95adc0 0x563eaa965140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f51a4307740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa967a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f51a4307740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563eaa8c02e0 con 0x563eaa965680 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f51a4307740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563eaa867ce0 con 0x563eaa95a4a0 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f51a4307740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563eaa8bc2a0 con 0x563eaa95adc0 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f5193fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa967a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f5193fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa967a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46348/0 (socket says 172.21.15.5:46348) 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f5193fff640 1 -- 172.21.15.5:0/3912939875 learned_addr learned my addr 172.21.15.5:0/3912939875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f51997fa640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51880a3430 0x7f51880a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f5193fff640 1 --2- 172.21.15.5:0/3912939875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa95a4a0 0x563eaa95a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f519b7fe640 1 --2- 172.21.15.5:0/3912939875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95adc0 0x563eaa965140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f51997fa640 1 -- 172.21.15.5:0/3757641285 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f51940f10d0 con 0x7f519c10e500 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f519bfff640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51880a3430 0x7f51880a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f5193fff640 1 -- 172.21.15.5:0/3912939875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95adc0 msgr2=0x563eaa965140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f5193fff640 1 --2- 172.21.15.5:0/3912939875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95adc0 0x563eaa965140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f5193fff640 1 -- 172.21.15.5:0/3912939875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa95a4a0 msgr2=0x563eaa95a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f5193fff640 1 --2- 172.21.15.5:0/3912939875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa95a4a0 0x563eaa95a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f5193fff640 1 -- 172.21.15.5:0/3912939875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eaa84c440 con 0x563eaa965680 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.039+0000 7f5193fff640 1 --2- 172.21.15.5:0/3912939875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa967a70 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5184012700 tx=0x7f518400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c499ec839f690 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f519bfff640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51880a3430 0x7f51880a58f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5194031f80 tx=0x7f5194033040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f5191ffb640 1 -- 172.21.15.5:0/3912939875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184012c80 con 0x563eaa965680 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f5191ffb640 1 -- 172.21.15.5:0/3912939875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5184012e20 con 0x563eaa965680 2024-09-15T22:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f5191ffb640 1 -- 172.21.15.5:0/3912939875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5184010ea0 con 0x563eaa965680 2024-09-15T22:24:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f51a4307740 1 -- 172.21.15.5:0/3912939875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 msgr2=0x563eaa967a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f51a4307740 1 --2- 172.21.15.5:0/3912939875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa967a70 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f5184012700 tx=0x7f518400b6e0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f51a4307740 1 -- 172.21.15.5:0/3912939875 shutdown_connections 2024-09-15T22:24:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f51a4307740 1 --2- 172.21.15.5:0/3912939875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa967a70 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f51a4307740 1 --2- 172.21.15.5:0/3912939875 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95adc0 0x563eaa965140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f51a4307740 1 --2- 172.21.15.5:0/3912939875 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa95a4a0 0x563eaa95a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f51a4307740 1 -- 172.21.15.5:0/3912939875 >> 172.21.15.5:0/3912939875 conn(0x563eaa956a30 msgr2=0x563eaa956e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f51a4307740 1 -- 172.21.15.5:0/3912939875 shutdown_connections 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.040+0000 7f51a4307740 1 -- 172.21.15.5:0/3912939875 wait complete. 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f51a4307740 1 -- start start 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f51a4307740 1 Processor -- start 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f51a4307740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa965040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f51a4307740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa9622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f51a4307740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa965680 0x563eaa9628e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f51a4307740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563eaa968250 con 0x563eaa95adc0 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f51a4307740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x563eaa87b050 con 0x563eaa965680 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f51a4307740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x563eaa9683d0 con 0x563eaa95a4a0 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f51a4307740 10 client.0 _ll_register_callbacks cb 0x563eaa8bb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa9622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa9622f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46360/0 (socket says 172.21.15.5:46360) 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.041+0000 7f519b7fe640 1 -- 172.21.15.5:0/358770076 learned_addr learned my addr 172.21.15.5:0/358770076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f5193fff640 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa965040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f5193fff640 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa965680 0x563eaa9628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f519b7fe640 1 -- 172.21.15.5:0/358770076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 msgr2=0x563eaa965040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f519b7fe640 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa965040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f519b7fe640 1 -- 172.21.15.5:0/358770076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa965680 msgr2=0x563eaa9628e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f519b7fe640 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa965680 0x563eaa9628e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f519b7fe640 1 -- 172.21.15.5:0/358770076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eaa6313c0 con 0x563eaa95adc0 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f5193fff640 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa965680 0x563eaa9628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f519b7fe640 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa9622f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f518c012740 tx=0x7f518c012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900d3d506a38575 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f517affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f517affd640 1 -- 172.21.15.5:0/358770076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c00f920 con 0x563eaa95adc0 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f517affd640 1 -- 172.21.15.5:0/358770076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f518c00fac0 con 0x563eaa95adc0 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f51a4307740 1 -- 172.21.15.5:0/358770076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5184012e20 con 0x563eaa95adc0 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f517affd640 1 -- 172.21.15.5:0/358770076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f518c00fd00 con 0x563eaa95adc0 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f51a4307740 10 client.18087 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f51a4307740 1 -- 172.21.15.5:0/358770076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563eaa973cd0 con 0x563eaa95adc0 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.042+0000 7f517affd640 1 -- 172.21.15.5:0/358770076 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5184012e20 con 0x563eaa95adc0 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.043+0000 7f517affd640 1 -- 172.21.15.5:0/358770076 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f518c01c420 con 0x563eaa95adc0 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.043+0000 7f51a4307740 10 client.18087 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.043+0000 7f51a4307740 10 client.18087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.043+0000 7f51a4307740 20 client.18087 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.043+0000 7f51a4307740 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563eaa975230 0x563eaa977650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.043+0000 7f51a4307740 4 client.18087 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.043+0000 7f51a4307740 1 -- 172.21.15.5:0/358770076 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eaa8cd850 con 0x563eaa975230 2024-09-15T22:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.043+0000 7f5193fff640 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563eaa975230 0x563eaa977650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.043+0000 7f5193fff640 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563eaa975230 0x563eaa977650 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec2598f40026d6e5 server_cookie=39655c0b75673bff in_seq=0 out_seq=0 2024-09-15T22:24:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.043+0000 7f517affd640 10 client.18087 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f517affd640 1 -- 172.21.15.5:0/358770076 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563eaa8cd850 con 0x563eaa975230 2024-09-15T22:24:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f517affd640 10 client.18087 handle_command_reply: tid=0 2024-09-15T22:24:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f51a4307740 1 client.18087 shutdown 2024-09-15T22:24:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f51a4307740 1 -- 172.21.15.5:0/358770076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 msgr2=0x563eaa9622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f51a4307740 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa9622f0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f518c012740 tx=0x7f518c012c00 comp rx=0 tx=0).stop 2024-09-15T22:24:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f51a4307740 1 -- 172.21.15.5:0/358770076 shutdown_connections 2024-09-15T22:24:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f51a4307740 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563eaa975230 0x563eaa977650 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f517affd640 0 client.18087 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:50.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f51a4307740 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa965680 0x563eaa9628e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f51a4307740 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa9622f0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f51a4307740 1 --2- 172.21.15.5:0/358770076 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa965040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f51a4307740 1 -- 172.21.15.5:0/358770076 >> 172.21.15.5:0/358770076 conn(0x563eaa956a30 msgr2=0x563eaa95c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f51a4307740 1 -- 172.21.15.5:0/358770076 shutdown_connections 2024-09-15T22:24:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.044+0000 7f51a4307740 1 -- 172.21.15.5:0/358770076 wait complete. 2024-09-15T22:24:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.045+0000 7f51a4307740 20 client.18087 trim_cache size 0 max 16384 2024-09-15T22:24:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.061+0000 7f51a4307740 1 Processor -- start 2024-09-15T22:24:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.061+0000 7f51a4307740 1 -- start start 2024-09-15T22:24:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f51a4307740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa9714b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f51a4307740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa960bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f51a4307740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa965680 0x563eaa964260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f51a4307740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1b3b50 con 0x563eaa95adc0 2024-09-15T22:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f51a4307740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1b1420 con 0x563eaa965680 2024-09-15T22:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f51a4307740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1af290 con 0x563eaa95a4a0 2024-09-15T22:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa960bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f519b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa960bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46370/0 (socket says 172.21.15.5:46370) 2024-09-15T22:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f519b7fe640 1 -- 172.21.15.5:0/625867045 learned_addr learned my addr 172.21.15.5:0/625867045 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f5193fff640 1 --2- 172.21.15.5:0/625867045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa9714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f519b7fe640 1 -- 172.21.15.5:0/625867045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 msgr2=0x563eaa9714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:50.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f519b7fe640 1 --2- 172.21.15.5:0/625867045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa9714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f519b7fe640 1 -- 172.21.15.5:0/625867045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa965680 msgr2=0x563eaa964260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:50.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f519b7fe640 1 --2- 172.21.15.5:0/625867045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa965680 0x563eaa964260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f519b7fe640 1 -- 172.21.15.5:0/625867045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eaa962c50 con 0x563eaa95adc0 2024-09-15T22:24:50.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f519b7fe640 1 --2- 172.21.15.5:0/625867045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa960bc0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f518c01c910 tx=0x7f518c00ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d002f789a58a501a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:50.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f51917fa640 1 -- 172.21.15.5:0/625867045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c02f070 con 0x563eaa95adc0 2024-09-15T22:24:50.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f51917fa640 1 -- 172.21.15.5:0/625867045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f518c012e50 con 0x563eaa95adc0 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.062+0000 7f51917fa640 1 -- 172.21.15.5:0/625867045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518c02a920 con 0x563eaa95adc0 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.063+0000 7f51a4307740 1 -- 172.21.15.5:0/625867045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 msgr2=0x563eaa960bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.063+0000 7f51a4307740 1 --2- 172.21.15.5:0/625867045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa960bc0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f518c01c910 tx=0x7f518c00ff20 comp rx=0 tx=0).stop 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.063+0000 7f51a4307740 1 -- 172.21.15.5:0/625867045 shutdown_connections 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.063+0000 7f51a4307740 1 --2- 172.21.15.5:0/625867045 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa965680 0x563eaa964260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.063+0000 7f51a4307740 1 --2- 172.21.15.5:0/625867045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa95adc0 0x563eaa960bc0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.063+0000 7f51a4307740 1 --2- 172.21.15.5:0/625867045 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa9714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.063+0000 7f51a4307740 1 -- 172.21.15.5:0/625867045 >> 172.21.15.5:0/625867045 conn(0x563eaa956a30 msgr2=0x563eaa963b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.063+0000 7f51a4307740 1 -- 172.21.15.5:0/625867045 shutdown_connections 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.063+0000 7f51a4307740 1 -- 172.21.15.5:0/625867045 wait complete. 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.063+0000 7f51a4307740 1 -- start start 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.063+0000 7f51a4307740 1 Processor -- start 2024-09-15T22:24:50.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f51a4307740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa969340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f51a4307740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa95adc0 0x563eaa969880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f51a4307740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa969dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f51a4307740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1af8e0 con 0x563eaa965680 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f51a4307740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1b0d50 con 0x563eaa95adc0 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f51a4307740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f519c1affb0 con 0x563eaa95a4a0 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f51a4307740 10 client.0 _ll_register_callbacks cb 0x7f518c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f5193fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa969dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f5193fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa969dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46378/0 (socket says 172.21.15.5:46378) 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f5193fff640 1 -- 172.21.15.5:0/3982290518 learned_addr learned my addr 172.21.15.5:0/3982290518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f519b7fe640 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa95adc0 0x563eaa969880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f5193fff640 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa969340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f5193fff640 1 -- 172.21.15.5:0/3982290518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 msgr2=0x563eaa969340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:50.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f5193fff640 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa969340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f5193fff640 1 -- 172.21.15.5:0/3982290518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa95adc0 msgr2=0x563eaa969880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f5193fff640 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa95adc0 0x563eaa969880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f5193fff640 1 -- 172.21.15.5:0/3982290518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eaa9deef0 con 0x563eaa965680 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f519b7fe640 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa95adc0 0x563eaa969880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f5193fff640 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa969dc0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f51840126a0 tx=0x7f518400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5531d8c4a2569449 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.064+0000 7f517affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f517affd640 1 -- 172.21.15.5:0/3982290518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518400be80 con 0x563eaa965680 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f517affd640 1 -- 172.21.15.5:0/3982290518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f5184010740 con 0x563eaa965680 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f51a4307740 1 -- 172.21.15.5:0/3982290518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f518c012e50 con 0x563eaa965680 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f517affd640 1 -- 172.21.15.5:0/3982290518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5184010940 con 0x563eaa965680 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f51a4307740 10 client.18093 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:50.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f51a4307740 1 -- 172.21.15.5:0/3982290518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563eaa9e0160 con 0x563eaa965680 2024-09-15T22:24:50.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f517affd640 1 -- 172.21.15.5:0/3982290518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f518c012e50 con 0x563eaa965680 2024-09-15T22:24:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f517affd640 1 -- 172.21.15.5:0/3982290518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f5184003ac0 con 0x563eaa965680 2024-09-15T22:24:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f51a4307740 10 client.18093 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f51a4307740 10 client.18093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:24:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f51a4307740 20 client.18093 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f51a4307740 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563eaa975230 0x563eaa9e2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f51a4307740 4 client.18093 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:24:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.065+0000 7f51a4307740 1 -- 172.21.15.5:0/3982290518 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563eaa8c7d60 con 0x563eaa975230 2024-09-15T22:24:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f519b7fe640 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563eaa975230 0x563eaa9e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:50.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f519b7fe640 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563eaa975230 0x563eaa9e2690 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3f9fcc972a0f48c server_cookie=29114e0c39f160e4 in_seq=0 out_seq=0 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f517affd640 10 client.18093 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f517affd640 1 -- 172.21.15.5:0/3982290518 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563eaa8c7d60 con 0x563eaa975230 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f517affd640 10 client.18093 handle_command_reply: tid=0 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f51a4307740 1 client.18093 shutdown 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f51a4307740 1 -- 172.21.15.5:0/3982290518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 msgr2=0x563eaa969dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f51a4307740 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa969dc0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f51840126a0 tx=0x7f518400b6e0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f51a4307740 1 -- 172.21.15.5:0/3982290518 shutdown_connections 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f51a4307740 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563eaa965680 0x563eaa969dc0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f51a4307740 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x563eaa975230 0x563eaa9e2690 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f51a4307740 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x563eaa95adc0 0x563eaa969880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f51a4307740 1 --2- 172.21.15.5:0/3982290518 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x563eaa95a4a0 0x563eaa969340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f517affd640 0 client.18093 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f51a4307740 1 -- 172.21.15.5:0/3982290518 >> 172.21.15.5:0/3982290518 conn(0x563eaa956a30 msgr2=0x563eaa964370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.066+0000 7f51a4307740 1 -- 172.21.15.5:0/3982290518 shutdown_connections 2024-09-15T22:24:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a4307740 1 -- 172.21.15.5:0/3982290518 wait complete. 2024-09-15T22:24:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a4307740 20 client.18093 trim_cache size 0 max 16384 2024-09-15T22:24:50.076 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:24:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-15T22:24:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-15T22:24:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "9ebef968-4cab-4a1c-a5f1-141bbedc263f": { 2024-09-15T22:24:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-15T22:24:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "9ebef968-4cab-4a1c-a5f1-141bbedc263f", 2024-09-15T22:24:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-15T22:24:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:50.077 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:24:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 -- 172.21.15.5:0/3757641285 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51880a3430 msgr2=0x7f51880a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51880a3430 0x7f51880a58f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5194031f80 tx=0x7f5194033040 comp rx=0 tx=0).stop 2024-09-15T22:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 -- 172.21.15.5:0/3757641285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f519c10e500 msgr2=0x7f519c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f519c10e500 0x7f519c19efc0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f51940319f0 tx=0x7f5194031a20 comp rx=0 tx=0).stop 2024-09-15T22:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 -- 172.21.15.5:0/3757641285 shutdown_connections 2024-09-15T22:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f51880a3430 0x7f51880a58f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f519c10e500 0x7f519c19efc0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f519c101440 0x7f519c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 --2- 172.21.15.5:0/3757641285 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f519c100b20 0x7f519c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 -- 172.21.15.5:0/3757641285 >> 172.21.15.5:0/3757641285 conn(0x7f519c0fc7d0 msgr2=0x7f519c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 -- 172.21.15.5:0/3757641285 shutdown_connections 2024-09-15T22:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:50.067+0000 7f51a1f2e640 1 -- 172.21.15.5:0/3757641285 wait complete. 2024-09-15T22:24:50.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:50 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/391814328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:50.825 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --set-id 398ff487 --release --await' 2024-09-15T22:24:50.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --set-id 398ff487 --release --await 2024-09-15T22:24:50.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:50 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/391814328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:50.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:50 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/391814328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:51.452 INFO:tasks.cephfs.filesystem:scrub status for tag:9ebef968-4cab-4a1c-a5f1-141bbedc263f - {'path': '/', 'tag': '9ebef968-4cab-4a1c-a5f1-141bbedc263f', 'options': 'recursive,force'} 2024-09-15T22:24:51.520 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:51.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:51 smithi005 ceph-mon[30944]: pgmap v499: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-09-15T22:24:51.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:51 smithi099 ceph-mon[34892]: pgmap v499: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-09-15T22:24:51.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:51 smithi129 ceph-mon[35314]: pgmap v499: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.5 MiB/s wr, 256 op/s 2024-09-15T22:24:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.013+0000 7f77af48b640 1 -- 172.21.15.5:0/1800199420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77a8074620 msgr2=0x7f77a8074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.013+0000 7f77af48b640 1 --2- 172.21.15.5:0/1800199420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77a8074620 0x7f77a8074a00 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009900 tx=0x7f77a402a480 comp rx=0 tx=0).stop 2024-09-15T22:24:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.013+0000 7f77af48b640 1 -- 172.21.15.5:0/1800199420 shutdown_connections 2024-09-15T22:24:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.013+0000 7f77af48b640 1 --2- 172.21.15.5:0/1800199420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a810e3c0 0x7f77a8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.013+0000 7f77af48b640 1 --2- 172.21.15.5:0/1800199420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a8074f40 0x7f77a810de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.013+0000 7f77af48b640 1 --2- 172.21.15.5:0/1800199420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77a8074620 0x7f77a8074a00 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.013+0000 7f77af48b640 1 -- 172.21.15.5:0/1800199420 >> 172.21.15.5:0/1800199420 conn(0x7f77a80fd8c0 msgr2=0x7f77a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.013+0000 7f77af48b640 1 -- 172.21.15.5:0/1800199420 shutdown_connections 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.013+0000 7f77af48b640 1 -- 172.21.15.5:0/1800199420 wait complete. 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.013+0000 7f77af48b640 1 Processor -- start 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77af48b640 1 -- start start 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77af48b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a8074620 0x7f77a819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77af48b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a8074f40 0x7f77a819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77af48b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77a810e3c0 0x7f77a819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77af48b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8114150 con 0x7f77a810e3c0 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77af48b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77a8113fd0 con 0x7f77a8074620 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77af48b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81142d0 con 0x7f77a8074f40 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77ada28640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77a810e3c0 0x7f77a819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77ada28640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77a810e3c0 0x7f77a819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46398/0 (socket says 172.21.15.5:46398) 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77ada28640 1 -- 172.21.15.5:0/3216062872 learned_addr learned my addr 172.21.15.5:0/3216062872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77aca26640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a8074f40 0x7f77a819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77ad227640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a8074620 0x7f77a819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77ada28640 1 -- 172.21.15.5:0/3216062872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a8074f40 msgr2=0x7f77a819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77ada28640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a8074f40 0x7f77a819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77ada28640 1 -- 172.21.15.5:0/3216062872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a8074620 msgr2=0x7f77a819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77ada28640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a8074620 0x7f77a819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77ada28640 1 -- 172.21.15.5:0/3216062872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a819f5a0 con 0x7f77a810e3c0 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77ad227640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a8074620 0x7f77a819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.014+0000 7f77ada28640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77a810e3c0 0x7f77a819eec0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f77a000eaa0 tx=0x7f77a000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.015+0000 7f779e7fc640 1 -- 172.21.15.5:0/3216062872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a0004d20 con 0x7f77a810e3c0 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.015+0000 7f77af48b640 1 -- 172.21.15.5:0/3216062872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a819f890 con 0x7f77a810e3c0 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.015+0000 7f779e7fc640 1 -- 172.21.15.5:0/3216062872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a0005020 con 0x7f77a810e3c0 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.015+0000 7f77af48b640 1 -- 172.21.15.5:0/3216062872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a81a7170 con 0x7f77a810e3c0 2024-09-15T22:24:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.015+0000 7f779e7fc640 1 -- 172.21.15.5:0/3216062872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a001f060 con 0x7f77a810e3c0 2024-09-15T22:24:52.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.018+0000 7f77b1864740 1 Processor -- start 2024-09-15T22:24:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.018+0000 7f779e7fc640 1 -- 172.21.15.5:0/3216062872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f77a00051d0 con 0x7f77a810e3c0 2024-09-15T22:24:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.018+0000 7f77b1864740 1 -- start start 2024-09-15T22:24:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77b1864740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102587060 0x560102587440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77b1864740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x560102591d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77b1864740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102592240 0x560102594630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77b1864740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5601024ec620 con 0x560102592240 2024-09-15T22:24:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77b1864740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560102493e50 con 0x560102587060 2024-09-15T22:24:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77b1864740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x5601024e85e0 con 0x560102587980 2024-09-15T22:24:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77aca26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102592240 0x560102594630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77aca26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102592240 0x560102594630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46410/0 (socket says 172.21.15.5:46410) 2024-09-15T22:24:52.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77aca26640 1 -- 172.21.15.5:0/4028427276 learned_addr learned my addr 172.21.15.5:0/4028427276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77aca26640 1 --2- 172.21.15.5:0/4028427276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102587060 0x560102587440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77ad227640 1 --2- 172.21.15.5:0/4028427276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x560102591d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77aca26640 1 -- 172.21.15.5:0/4028427276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 msgr2=0x560102591d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77aca26640 1 --2- 172.21.15.5:0/4028427276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x560102591d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77aca26640 1 -- 172.21.15.5:0/4028427276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102587060 msgr2=0x560102587440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77aca26640 1 --2- 172.21.15.5:0/4028427276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102587060 0x560102587440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77aca26640 1 -- 172.21.15.5:0/4028427276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560102484d40 con 0x560102592240 2024-09-15T22:24:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f779e7fc640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f77940a33b0 0x7f77940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77aca26640 1 --2- 172.21.15.5:0/4028427276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102592240 0x560102594630 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f77980117e0 tx=0x7f779800a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1added3f54bfaca server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.019+0000 7f77ad227640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f77940a33b0 0x7f77940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.020+0000 7f779e7fc640 1 -- 172.21.15.5:0/3216062872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f77a00d0240 con 0x7f77a810e3c0 2024-09-15T22:24:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.020+0000 7f77937fe640 1 -- 172.21.15.5:0/4028427276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798011e20 con 0x560102592240 2024-09-15T22:24:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.020+0000 7f77937fe640 1 -- 172.21.15.5:0/4028427276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7798005020 con 0x560102592240 2024-09-15T22:24:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.020+0000 7f77937fe640 1 -- 172.21.15.5:0/4028427276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798004ac0 con 0x560102592240 2024-09-15T22:24:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.020+0000 7f77b1864740 1 -- 172.21.15.5:0/4028427276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102592240 msgr2=0x560102594630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.020+0000 7f77b1864740 1 --2- 172.21.15.5:0/4028427276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102592240 0x560102594630 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f77980117e0 tx=0x7f779800a760 comp rx=0 tx=0).stop 2024-09-15T22:24:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.020+0000 7f77ad227640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f77940a33b0 0x7f77940a5870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x560102584450 tx=0x7f77a4031400 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.020+0000 7f77b1864740 1 -- 172.21.15.5:0/4028427276 shutdown_connections 2024-09-15T22:24:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 --2- 172.21.15.5:0/4028427276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102592240 0x560102594630 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 --2- 172.21.15.5:0/4028427276 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x560102591d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 --2- 172.21.15.5:0/4028427276 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102587060 0x560102587440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 -- 172.21.15.5:0/4028427276 >> 172.21.15.5:0/4028427276 conn(0x5601025835f0 msgr2=0x5601025839d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 -- 172.21.15.5:0/4028427276 shutdown_connections 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 -- 172.21.15.5:0/4028427276 wait complete. 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 -- start start 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 Processor -- start 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x56010258f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102592240 0x56010258fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560102594e60 con 0x560102587060 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x560102474f10 con 0x560102592240 2024-09-15T22:24:52.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.021+0000 7f77b1864740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x560102594fe0 con 0x560102587980 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77b1864740 10 client.0 _ll_register_callbacks cb 0x560102498db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77ad227640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77ad227640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46424/0 (socket says 172.21.15.5:46424) 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77ad227640 1 -- 172.21.15.5:0/1571002171 learned_addr learned my addr 172.21.15.5:0/1571002171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77aca26640 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x56010258f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77ada28640 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102592240 0x56010258fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77ad227640 1 -- 172.21.15.5:0/1571002171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 msgr2=0x56010258f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77ad227640 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x56010258f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77ad227640 1 -- 172.21.15.5:0/1571002171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102592240 msgr2=0x56010258fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77ad227640 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102592240 0x56010258fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77ad227640 1 -- 172.21.15.5:0/1571002171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601021fb580 con 0x560102587060 2024-09-15T22:24:52.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77ada28640 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102592240 0x56010258fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77ad227640 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258f210 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009900 tx=0x7f77a4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552a4d45c3c68631 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f7790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f7790ff9640 1 -- 172.21.15.5:0/1571002171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a404c030 con 0x560102587060 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f7790ff9640 1 -- 172.21.15.5:0/1571002171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f77a4003ac0 con 0x560102587060 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.022+0000 7f77b1864740 1 -- 172.21.15.5:0/1571002171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7798005020 con 0x560102587060 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f7790ff9640 1 -- 172.21.15.5:0/1571002171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f77a4059050 con 0x560102587060 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f77b1864740 10 client.18105 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f77b1864740 1 -- 172.21.15.5:0/1571002171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5601022c5960 con 0x560102587060 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f7790ff9640 1 -- 172.21.15.5:0/1571002171 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7798005020 con 0x560102587060 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f7790ff9640 1 -- 172.21.15.5:0/1571002171 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f77a403c440 con 0x560102587060 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f77b1864740 10 client.18105 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f77b1864740 10 client.18105 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:24:52.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f77b1864740 20 client.18105 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f77b1864740 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5601025a1ed0 0x5601025a4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f77b1864740 4 client.18105 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f77b1864740 1 -- 172.21.15.5:0/1571002171 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601024f9b90 con 0x5601025a1ed0 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.023+0000 7f77aca26640 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5601025a1ed0 0x5601025a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.024+0000 7f77aca26640 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5601025a1ed0 0x5601025a4350 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d372ec20ee32679 server_cookie=fa6ecdb9975178c9 in_seq=0 out_seq=0 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.024+0000 7f7790ff9640 10 client.18105 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f7790ff9640 1 -- 172.21.15.5:0/1571002171 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f7798029e80 con 0x5601025a1ed0 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f7790ff9640 10 client.18105 handle_command_reply: tid=0 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 1 client.18105 shutdown 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 1 -- 172.21.15.5:0/1571002171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 msgr2=0x56010258f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258f210 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f77a4009900 tx=0x7f77a4009260 comp rx=0 tx=0).stop 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 1 -- 172.21.15.5:0/1571002171 shutdown_connections 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x5601025a1ed0 0x5601025a4350 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102592240 0x56010258fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x56010258f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 1 --2- 172.21.15.5:0/1571002171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258f210 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f7790ff9640 0 client.18105 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 1 -- 172.21.15.5:0/1571002171 >> 172.21.15.5:0/1571002171 conn(0x5601025835f0 msgr2=0x5601025892d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 1 -- 172.21.15.5:0/1571002171 shutdown_connections 2024-09-15T22:24:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 1 -- 172.21.15.5:0/1571002171 wait complete. 2024-09-15T22:24:52.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.025+0000 7f77b1864740 20 client.18105 trim_cache size 0 max 16384 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.042+0000 7f77b1864740 1 Processor -- start 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.042+0000 7f77b1864740 1 -- start start 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.042+0000 7f77b1864740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.042+0000 7f77b1864740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x560102591180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.042+0000 7f77b1864740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102592240 0x5601025a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.042+0000 7f77b1864740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81b3c10 con 0x560102587060 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.042+0000 7f77b1864740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81b14e0 con 0x560102592240 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.042+0000 7f77b1864740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81af350 con 0x560102587980 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.042+0000 7f77ad227640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.042+0000 7f77ad227640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46436/0 (socket says 172.21.15.5:46436) 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.042+0000 7f77ad227640 1 -- 172.21.15.5:0/67642441 learned_addr learned my addr 172.21.15.5:0/67642441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.043+0000 7f77aca26640 1 --2- 172.21.15.5:0/67642441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x560102591180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.043+0000 7f77ad227640 1 -- 172.21.15.5:0/67642441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 msgr2=0x560102591180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.043+0000 7f77ad227640 1 --2- 172.21.15.5:0/67642441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x560102591180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.043+0000 7f77ad227640 1 -- 172.21.15.5:0/67642441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102592240 msgr2=0x5601025a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.043+0000 7f77ad227640 1 --2- 172.21.15.5:0/67642441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102592240 0x5601025a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.043+0000 7f77ad227640 1 -- 172.21.15.5:0/67642441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56010258f8b0 con 0x560102587060 2024-09-15T22:24:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.043+0000 7f77aca26640 1 --2- 172.21.15.5:0/67642441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x560102591180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.043+0000 7f77ad227640 1 --2- 172.21.15.5:0/67642441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258a560 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f77a403c860 tx=0x7f77a403cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513180b1cd2ba356 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.043+0000 7f7792ffd640 1 -- 172.21.15.5:0/67642441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4062070 con 0x560102587060 2024-09-15T22:24:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.043+0000 7f7792ffd640 1 -- 172.21.15.5:0/67642441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a403c440 con 0x560102587060 2024-09-15T22:24:52.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.043+0000 7f7792ffd640 1 -- 172.21.15.5:0/67642441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a404c040 con 0x560102587060 2024-09-15T22:24:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.044+0000 7f77b1864740 1 -- 172.21.15.5:0/67642441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 msgr2=0x56010258a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.044+0000 7f77b1864740 1 --2- 172.21.15.5:0/67642441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258a560 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f77a403c860 tx=0x7f77a403cea0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.044+0000 7f77b1864740 1 -- 172.21.15.5:0/67642441 shutdown_connections 2024-09-15T22:24:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.044+0000 7f77b1864740 1 --2- 172.21.15.5:0/67642441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102592240 0x5601025a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.044+0000 7f77b1864740 1 --2- 172.21.15.5:0/67642441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102587980 0x560102591180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.044+0000 7f77b1864740 1 --2- 172.21.15.5:0/67642441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587060 0x56010258a560 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.044+0000 7f77b1864740 1 -- 172.21.15.5:0/67642441 >> 172.21.15.5:0/67642441 conn(0x5601025835f0 msgr2=0x560102590a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.044+0000 7f77b1864740 1 -- 172.21.15.5:0/67642441 shutdown_connections 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.044+0000 7f77b1864740 1 -- 172.21.15.5:0/67642441 wait complete. 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.044+0000 7f77b1864740 1 -- start start 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.044+0000 7f77b1864740 1 Processor -- start 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77b1864740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102587060 0x56010259a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77b1864740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587980 0x560102597200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77b1864740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102592240 0x560102597740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77b1864740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81af9a0 con 0x560102587980 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77b1864740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81b0e10 con 0x560102587060 2024-09-15T22:24:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77b1864740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f77a81b0070 con 0x560102592240 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77b1864740 10 client.0 _ll_register_callbacks cb 0x7f77a40319d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77aca26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587980 0x560102597200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77aca26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587980 0x560102597200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46444/0 (socket says 172.21.15.5:46444) 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77aca26640 1 -- 172.21.15.5:0/4063536417 learned_addr learned my addr 172.21.15.5:0/4063536417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77ad227640 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102587060 0x56010259a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77aca26640 1 -- 172.21.15.5:0/4063536417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102592240 msgr2=0x560102597740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77aca26640 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102592240 0x560102597740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77aca26640 1 -- 172.21.15.5:0/4063536417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102587060 msgr2=0x56010259a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77aca26640 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102587060 0x56010259a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77aca26640 1 -- 172.21.15.5:0/4063536417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560102597c80 con 0x560102587980 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f77aca26640 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587980 0x560102597200 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x560102584970 tx=0x7f779800a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cee9c9f77204a87 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f7790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.045+0000 7f7790ff9640 1 -- 172.21.15.5:0/4063536417 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779800aab0 con 0x560102587980 2024-09-15T22:24:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f7790ff9640 1 -- 172.21.15.5:0/4063536417 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f779800f9d0 con 0x560102587980 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f77b1864740 1 -- 172.21.15.5:0/4063536417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77a403c440 con 0x560102587980 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f7790ff9640 1 -- 172.21.15.5:0/4063536417 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7798038050 con 0x560102587980 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f77b1864740 10 client.18111 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f77b1864740 1 -- 172.21.15.5:0/4063536417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56010260d980 con 0x560102587980 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f7790ff9640 1 -- 172.21.15.5:0/4063536417 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a403c440 con 0x560102587980 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f7790ff9640 1 -- 172.21.15.5:0/4063536417 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f7798010430 con 0x560102587980 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f77b1864740 10 client.18111 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f77b1864740 10 client.18111 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f77b1864740 20 client.18111 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f77b1864740 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56010260dff0 0x560102610410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f77b1864740 4 client.18111 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "398ff487", "release": true, "await": true}] 2024-09-15T22:24:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f77b1864740 1 -- 172.21.15.5:0/4063536417 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "set_id": "398ff487", "release": true, "await": true}) -- 0x560102590020 con 0x56010260dff0 2024-09-15T22:24:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.046+0000 7f77ad227640 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56010260dff0 0x560102610410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.047+0000 7f77ad227640 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56010260dff0 0x560102610410 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1374ab84d37ef90e server_cookie=666c027e6080e3b9 in_seq=0 out_seq=0 2024-09-15T22:24:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.047+0000 7f7790ff9640 10 client.18111 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:52.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f7790ff9640 1 -- 172.21.15.5:0/4063536417 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560102590020 con 0x56010260dff0 2024-09-15T22:24:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f7790ff9640 10 client.18111 handle_command_reply: tid=0 2024-09-15T22:24:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 1 client.18111 shutdown 2024-09-15T22:24:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 1 -- 172.21.15.5:0/4063536417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587980 msgr2=0x560102597200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587980 0x560102597200 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x560102584970 tx=0x7f779800a680 comp rx=0 tx=0).stop 2024-09-15T22:24:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 1 -- 172.21.15.5:0/4063536417 shutdown_connections 2024-09-15T22:24:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x560102592240 0x560102597740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560102587980 0x560102597200 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x56010260dff0 0x560102610410 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 1 --2- 172.21.15.5:0/4063536417 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x560102587060 0x56010259a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f7790ff9640 0 client.18111 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:52.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 1 -- 172.21.15.5:0/4063536417 >> 172.21.15.5:0/4063536417 conn(0x5601025835f0 msgr2=0x560102582750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:52.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 1 -- 172.21.15.5:0/4063536417 shutdown_connections 2024-09-15T22:24:52.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 1 -- 172.21.15.5:0/4063536417 wait complete. 2024-09-15T22:24:52.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.096+0000 7f77b1864740 20 client.18111 trim_cache size 0 max 16384 2024-09-15T22:24:52.100 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:24:52.100 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:24:52.100 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:24:52.100 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 60, 2024-09-15T22:24:52.100 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-15T22:24:52.100 INFO:teuthology.orchestra.run.smithi005.stdout: "398ff487": { 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 60, 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 137.7, 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-15T22:24:52.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-15T22:24:52.102 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-15T22:24:52.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:52.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:52.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:52.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:52.102 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:24:52.102 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 -- 172.21.15.5:0/3216062872 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f77940a33b0 msgr2=0x7f77940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f77940a33b0 0x7f77940a5870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x560102584450 tx=0x7f77a4031400 comp rx=0 tx=0).stop 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 -- 172.21.15.5:0/3216062872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77a810e3c0 msgr2=0x7f77a819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77a810e3c0 0x7f77a819eec0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f77a000eaa0 tx=0x7f77a000eea0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 -- 172.21.15.5:0/3216062872 shutdown_connections 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f77940a33b0 0x7f77940a5870 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77a810e3c0 0x7f77a819eec0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f77a8074f40 0x7f77a819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 --2- 172.21.15.5:0/3216062872 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f77a8074620 0x7f77a819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 -- 172.21.15.5:0/3216062872 >> 172.21.15.5:0/3216062872 conn(0x7f77a80fd8c0 msgr2=0x7f77a80fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 -- 172.21.15.5:0/3216062872 shutdown_connections 2024-09-15T22:24:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:52.097+0000 7f77af48b640 1 -- 172.21.15.5:0/3216062872 wait complete. 2024-09-15T22:24:52.840 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:52 smithi099 ceph-mon[34892]: scrub summary: idle 2024-09-15T22:24:52.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:52 smithi129 ceph-mon[35314]: scrub summary: idle 2024-09-15T22:24:52.854 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:52 smithi005 ceph-mon[30944]: scrub summary: idle 2024-09-15T22:24:52.896 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2024-09-15T22:24:52.958 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/: 2024-09-15T22:24:52.959 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:52.959 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 . 2024-09-15T22:24:52.959 INFO:teuthology.orchestra.run.smithi005.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 .. 2024-09-15T22:24:52.959 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 data 2024-09-15T22:24:52.959 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:52.960 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data: 2024-09-15T22:24:52.960 INFO:teuthology.orchestra.run.smithi005.stdout:total 36 2024-09-15T22:24:52.960 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 . 2024-09-15T22:24:52.960 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 .. 2024-09-15T22:24:52.960 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 15 22:09 base 2024-09-15T22:24:52.960 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Sep 15 22:09 current_logfiles 2024-09-15T22:24:52.960 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 15 22:09 global 2024-09-15T22:24:52.960 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 log 2024-09-15T22:24:52.961 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_commit_ts 2024-09-15T22:24:52.961 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_dynshmem 2024-09-15T22:24:52.961 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4516 Sep 15 22:09 pg_hba.conf 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Sep 15 22:09 pg_ident.conf 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:24 pg_logical 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 15 22:09 pg_multixact 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_notify 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_replslot 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_serial 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_snapshots 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:24 pg_stat 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:24 pg_stat_tmp 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:24 pg_subtrans 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_tblspc 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pg_twophase 2024-09-15T22:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 PG_VERSION 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 60 Sep 15 22:24 pg_wal 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 pg_xact 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Sep 15 22:09 postgresql.auto.conf 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 28074 Sep 15 22:09 postgresql.conf 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Sep 15 22:09 postmaster.opts 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base: 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 15 22:09 . 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 15 22:09 1 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 15 22:09 13493 2024-09-15T22:24:52.963 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 15 22:15 13494 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 pgsql_tmp 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/1: 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:total 7777 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 15 22:09 . 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 15 22:09 .. 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 112 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 113 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 1247 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1247_fsm 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1247_vm 2024-09-15T22:24:52.964 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 15 22:09 1249 2024-09-15T22:24:52.965 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1249_fsm 2024-09-15T22:24:52.965 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1249_vm 2024-09-15T22:24:52.965 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 15 22:09 1255 2024-09-15T22:24:52.965 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1255_fsm 2024-09-15T22:24:52.965 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1255_vm 2024-09-15T22:24:52.965 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 15 22:09 1259 2024-09-15T22:24:52.965 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1259_fsm 2024-09-15T22:24:52.965 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1259_vm 2024-09-15T22:24:52.965 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 15 22:09 13344 2024-09-15T22:24:52.965 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13344_fsm 2024-09-15T22:24:52.965 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13344_vm 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13346 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13348 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13349_fsm 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349_vm 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13351 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13353 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13354_fsm 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354_vm 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13356 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13358 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13359_fsm 2024-09-15T22:24:52.966 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359_vm 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13361 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13363 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1417 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1418 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 174 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 175 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2187 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2224 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2328 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2336 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2337 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2579 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2600 2024-09-15T22:24:52.967 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2600_fsm 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2600_vm 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2601_fsm 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601_vm 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2602 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2602_fsm 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2602_vm 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2603 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2603_fsm 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2603_vm 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2604 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2605 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2605_fsm 2024-09-15T22:24:52.968 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2605_vm 2024-09-15T22:24:52.969 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606 2024-09-15T22:24:52.969 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2606_fsm 2024-09-15T22:24:52.969 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606_vm 2024-09-15T22:24:52.969 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2607 2024-09-15T22:24:52.969 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2607_fsm 2024-09-15T22:24:52.969 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2607_vm 2024-09-15T22:24:52.969 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 15 22:09 2608 2024-09-15T22:24:52.969 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2608_fsm 2024-09-15T22:24:52.969 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2608_vm 2024-09-15T22:24:52.969 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 15 22:09 2609 2024-09-15T22:24:52.969 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2609_fsm 2024-09-15T22:24:52.970 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2609_vm 2024-09-15T22:24:52.970 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2610 2024-09-15T22:24:52.970 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2610_fsm 2024-09-15T22:24:52.970 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2610_vm 2024-09-15T22:24:52.970 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2611 2024-09-15T22:24:52.971 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612 2024-09-15T22:24:52.971 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2612_fsm 2024-09-15T22:24:52.971 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612_vm 2024-09-15T22:24:52.971 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2613 2024-09-15T22:24:52.971 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615 2024-09-15T22:24:52.971 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2615_fsm 2024-09-15T22:24:52.972 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615_vm 2024-09-15T22:24:52.972 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2616 2024-09-15T22:24:52.972 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2616_fsm 2024-09-15T22:24:52.972 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2616_vm 2024-09-15T22:24:52.973 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 15 22:09 2617 2024-09-15T22:24:52.973 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2617_fsm 2024-09-15T22:24:52.973 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2617_vm 2024-09-15T22:24:52.973 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 15 22:09 2618 2024-09-15T22:24:52.973 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2618_fsm 2024-09-15T22:24:52.973 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2618_vm 2024-09-15T22:24:52.974 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 15 22:09 2619 2024-09-15T22:24:52.974 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2619_fsm 2024-09-15T22:24:52.974 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2619_vm 2024-09-15T22:24:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2620 2024-09-15T22:24:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2650 2024-09-15T22:24:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2651 2024-09-15T22:24:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2652 2024-09-15T22:24:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2653 2024-09-15T22:24:52.975 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2654 2024-09-15T22:24:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2655 2024-09-15T22:24:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2656 2024-09-15T22:24:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2657 2024-09-15T22:24:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 15 22:09 2658 2024-09-15T22:24:52.976 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 2659 2024-09-15T22:24:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2660 2024-09-15T22:24:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2661 2024-09-15T22:24:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2662 2024-09-15T22:24:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2663 2024-09-15T22:24:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2664 2024-09-15T22:24:52.977 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2665 2024-09-15T22:24:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2666 2024-09-15T22:24:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2667 2024-09-15T22:24:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2668 2024-09-15T22:24:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2669 2024-09-15T22:24:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2670 2024-09-15T22:24:52.978 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 15 22:09 2673 2024-09-15T22:24:52.979 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 15 22:09 2674 2024-09-15T22:24:52.979 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 15 22:09 2675 2024-09-15T22:24:52.979 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2678 2024-09-15T22:24:52.979 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2679 2024-09-15T22:24:52.979 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2680 2024-09-15T22:24:52.980 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2681 2024-09-15T22:24:52.980 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2682 2024-09-15T22:24:52.980 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2683 2024-09-15T22:24:52.980 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2684 2024-09-15T22:24:52.980 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2685 2024-09-15T22:24:52.980 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2686 2024-09-15T22:24:52.981 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2687 2024-09-15T22:24:52.981 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2688 2024-09-15T22:24:52.981 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2689 2024-09-15T22:24:52.981 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 15 22:09 2690 2024-09-15T22:24:52.981 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 15 22:09 2691 2024-09-15T22:24:52.982 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2692 2024-09-15T22:24:52.982 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2693 2024-09-15T22:24:52.982 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2696 2024-09-15T22:24:52.982 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2699 2024-09-15T22:24:52.982 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2701 2024-09-15T22:24:52.982 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2702 2024-09-15T22:24:52.983 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2703 2024-09-15T22:24:52.983 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2704 2024-09-15T22:24:52.983 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2753 2024-09-15T22:24:52.983 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2753_fsm 2024-09-15T22:24:52.983 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2753_vm 2024-09-15T22:24:52.983 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2754 2024-09-15T22:24:52.984 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2755 2024-09-15T22:24:52.984 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2756 2024-09-15T22:24:52.984 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2757 2024-09-15T22:24:52.984 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2830 2024-09-15T22:24:52.984 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2831 2024-09-15T22:24:52.984 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2832 2024-09-15T22:24:52.984 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2833 2024-09-15T22:24:52.984 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2834 2024-09-15T22:24:52.984 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2835 2024-09-15T22:24:52.984 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2836 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2837 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 15 22:09 2838 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2838_fsm 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2838_vm 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2839 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840_fsm 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2840_vm 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2841 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2995 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2996 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3079_fsm 2024-09-15T22:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079_vm 2024-09-15T22:24:52.986 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3080 2024-09-15T22:24:52.986 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3081 2024-09-15T22:24:52.986 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3085 2024-09-15T22:24:52.986 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3118 2024-09-15T22:24:52.986 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3119 2024-09-15T22:24:52.986 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 15 22:09 3164 2024-09-15T22:24:52.986 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3256 2024-09-15T22:24:52.986 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3257 2024-09-15T22:24:52.986 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3258 2024-09-15T22:24:52.986 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3350 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3351 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3379 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3380 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3381 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3394 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3394_fsm 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3394_vm 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3395 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3429 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3430 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3431 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3433 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3439 2024-09-15T22:24:52.987 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3440 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3455 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 15 22:09 3456 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3456_fsm 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3456_vm 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3466 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3467 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3468 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3501 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3502 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3503 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3534 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541 2024-09-15T22:24:52.988 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3541_fsm 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541_vm 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3542 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3574 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3575 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3576 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3596 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3597 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3598 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3599 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3600_fsm 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600_vm 2024-09-15T22:24:52.989 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3601_fsm 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601_vm 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3602_fsm 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602_vm 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603_fsm 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3603_vm 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3604 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3605 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3606 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3607 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3608 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 3609 2024-09-15T22:24:52.990 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3712 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3764_fsm 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764_vm 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3766 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3767 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3997 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4143 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4144 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4145 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4146 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4147 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4148 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4149 2024-09-15T22:24:52.991 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4150 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4151 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4152 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4153 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4154 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4155 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4156 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4157 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4158 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4159 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4160 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4161 2024-09-15T22:24:52.992 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4162 2024-09-15T22:24:52.993 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4163 2024-09-15T22:24:52.994 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4164 2024-09-15T22:24:52.994 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4165 2024-09-15T22:24:52.994 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4166 2024-09-15T22:24:52.994 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4167 2024-09-15T22:24:52.994 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4168 2024-09-15T22:24:52.994 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4169 2024-09-15T22:24:52.994 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4170 2024-09-15T22:24:52.994 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4171 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4172 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4173 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4174 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 5002 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 548 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 549 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6102 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6104 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6106 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6110 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6111 2024-09-15T22:24:52.995 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6112 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6113 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6117 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 826 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 827 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 828 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 15 22:09 pg_filenode.map 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 PG_VERSION 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/13493: 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:total 7777 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 289 Sep 15 22:09 . 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 15 22:09 .. 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 112 2024-09-15T22:24:52.996 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 113 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 1247 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1247_fsm 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1247_vm 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 15 22:09 1249 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1249_fsm 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1249_vm 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 15 22:09 1255 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1255_fsm 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1255_vm 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 15 22:09 1259 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1259_fsm 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1259_vm 2024-09-15T22:24:52.997 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 15 22:09 13344 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13344_fsm 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13344_vm 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13346 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13348 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13349_fsm 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349_vm 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13351 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13353 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13354_fsm 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354_vm 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13356 2024-09-15T22:24:52.998 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13358 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13359_fsm 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359_vm 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13361 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13363 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1417 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1418 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 174 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 175 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2187 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2224 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2328 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2336 2024-09-15T22:24:52.999 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2337 2024-09-15T22:24:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2579 2024-09-15T22:24:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2600 2024-09-15T22:24:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2600_fsm 2024-09-15T22:24:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2600_vm 2024-09-15T22:24:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601 2024-09-15T22:24:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2601_fsm 2024-09-15T22:24:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601_vm 2024-09-15T22:24:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2602 2024-09-15T22:24:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2602_fsm 2024-09-15T22:24:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2602_vm 2024-09-15T22:24:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2603 2024-09-15T22:24:53.001 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2603_fsm 2024-09-15T22:24:53.001 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2603_vm 2024-09-15T22:24:53.001 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2604 2024-09-15T22:24:53.001 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2605 2024-09-15T22:24:53.001 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2605_fsm 2024-09-15T22:24:53.001 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2605_vm 2024-09-15T22:24:53.001 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606 2024-09-15T22:24:53.001 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2606_fsm 2024-09-15T22:24:53.001 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606_vm 2024-09-15T22:24:53.001 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2607 2024-09-15T22:24:53.001 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2607_fsm 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2607_vm 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 15 22:09 2608 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2608_fsm 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2608_vm 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 15 22:09 2609 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2609_fsm 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2609_vm 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2610 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2610_fsm 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2610_vm 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2611 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612 2024-09-15T22:24:53.002 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2612_fsm 2024-09-15T22:24:53.003 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612_vm 2024-09-15T22:24:53.003 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2613 2024-09-15T22:24:53.003 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615 2024-09-15T22:24:53.003 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2615_fsm 2024-09-15T22:24:53.003 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615_vm 2024-09-15T22:24:53.003 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2616 2024-09-15T22:24:53.004 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2616_fsm 2024-09-15T22:24:53.004 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2616_vm 2024-09-15T22:24:53.004 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 15 22:09 2617 2024-09-15T22:24:53.004 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2617_fsm 2024-09-15T22:24:53.004 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2617_vm 2024-09-15T22:24:53.004 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 15 22:09 2618 2024-09-15T22:24:53.004 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2618_fsm 2024-09-15T22:24:53.004 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2618_vm 2024-09-15T22:24:53.004 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 147456 Sep 15 22:09 2619 2024-09-15T22:24:53.004 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2619_fsm 2024-09-15T22:24:53.004 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2619_vm 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2620 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2650 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2651 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2652 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2653 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2654 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2655 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2656 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2657 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 15 22:09 2658 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 2659 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2660 2024-09-15T22:24:53.005 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2661 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2662 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2663 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2664 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2665 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2666 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2667 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2668 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2669 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2670 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 15 22:09 2673 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 15 22:09 2674 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 15 22:09 2675 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2678 2024-09-15T22:24:53.006 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2679 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2680 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2681 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2682 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2683 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2684 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2685 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2686 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2687 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2688 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2689 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 15 22:09 2690 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 15 22:09 2691 2024-09-15T22:24:53.007 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2692 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2693 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2696 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2699 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2701 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2702 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2703 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2704 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2753 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2753_fsm 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2753_vm 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2754 2024-09-15T22:24:53.008 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2755 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2756 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2757 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2830 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2831 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2832 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2833 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2834 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2835 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2836 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2837 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 15 22:09 2838 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2838_fsm 2024-09-15T22:24:53.009 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2838_vm 2024-09-15T22:24:53.010 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2839 2024-09-15T22:24:53.010 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840 2024-09-15T22:24:53.010 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840_fsm 2024-09-15T22:24:53.010 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2840_vm 2024-09-15T22:24:53.010 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2841 2024-09-15T22:24:53.010 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2995 2024-09-15T22:24:53.010 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2996 2024-09-15T22:24:53.010 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079 2024-09-15T22:24:53.010 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3079_fsm 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079_vm 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3080 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3081 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3085 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3118 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3119 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 15 22:09 3164 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3256 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3257 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3258 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3350 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3351 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3379 2024-09-15T22:24:53.011 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3380 2024-09-15T22:24:53.012 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3381 2024-09-15T22:24:53.012 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3394 2024-09-15T22:24:53.012 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3394_fsm 2024-09-15T22:24:53.012 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3394_vm 2024-09-15T22:24:53.012 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3395 2024-09-15T22:24:53.012 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3429 2024-09-15T22:24:53.012 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3430 2024-09-15T22:24:53.012 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3431 2024-09-15T22:24:53.012 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3433 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3439 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3440 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3455 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 15 22:09 3456 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3456_fsm 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3456_vm 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3466 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3467 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3468 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3501 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3502 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3503 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3534 2024-09-15T22:24:53.013 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3541_fsm 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541_vm 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3542 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3574 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3575 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3576 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3596 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3597 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3598 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3599 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600 2024-09-15T22:24:53.014 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3600_fsm 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600_vm 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3601_fsm 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601_vm 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3602_fsm 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602_vm 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603_fsm 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3603_vm 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3604 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3605 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3606 2024-09-15T22:24:53.015 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3607 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3608 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 3609 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3712 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3764_fsm 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764_vm 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3766 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3767 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3997 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4143 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4144 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4145 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4146 2024-09-15T22:24:53.016 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4147 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4148 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4149 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4150 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4151 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4152 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4153 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4154 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4155 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4156 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4157 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4158 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4159 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4160 2024-09-15T22:24:53.017 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4161 2024-09-15T22:24:53.018 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4162 2024-09-15T22:24:53.018 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4163 2024-09-15T22:24:53.018 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4164 2024-09-15T22:24:53.018 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4165 2024-09-15T22:24:53.018 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4166 2024-09-15T22:24:53.018 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4167 2024-09-15T22:24:53.018 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4168 2024-09-15T22:24:53.018 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4169 2024-09-15T22:24:53.018 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4170 2024-09-15T22:24:53.018 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4171 2024-09-15T22:24:53.018 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4172 2024-09-15T22:24:53.020 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4173 2024-09-15T22:24:53.020 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4174 2024-09-15T22:24:53.020 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 5002 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 548 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 549 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6102 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6104 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6106 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6110 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6111 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6112 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6113 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6117 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 826 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 827 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 828 2024-09-15T22:24:53.021 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 15 22:09 pg_filenode.map 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 PG_VERSION 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/13494: 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:total 549846 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 305 Sep 15 22:15 . 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 15 22:09 .. 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 112 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 113 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 1247 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1247_fsm 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1247_vm 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 434176 Sep 15 22:09 1249 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1249_fsm 2024-09-15T22:24:53.022 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1249_vm 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 663552 Sep 15 22:09 1255 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1255_fsm 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1255_vm 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Sep 15 22:24 1259 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1259_fsm 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1259_vm 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Sep 15 22:09 13344 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13344_fsm 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13344_vm 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13346 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13348 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13349_fsm 2024-09-15T22:24:53.023 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13349_vm 2024-09-15T22:24:53.024 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13351 2024-09-15T22:24:53.024 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13353 2024-09-15T22:24:53.024 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354 2024-09-15T22:24:53.024 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13354_fsm 2024-09-15T22:24:53.024 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13354_vm 2024-09-15T22:24:53.024 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13356 2024-09-15T22:24:53.024 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13358 2024-09-15T22:24:53.024 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359 2024-09-15T22:24:53.024 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 13359_fsm 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13359_vm 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 13361 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 13363 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1417 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 1418 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 439672832 Sep 15 22:24 16396 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Sep 15 22:24 16396_fsm 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:17 16396_vm 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 966656 Sep 15 22:24 16397 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:24 16397_fsm 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:24 16397_vm 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 2981888 Sep 15 22:24 16399 2024-09-15T22:24:53.025 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:24 16399_fsm 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:23 16399_vm 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:24 16400 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:21 16402 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 71901184 Sep 15 22:24 16404 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 39059456 Sep 15 22:24 16406 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:24 16406_fsm 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:22 16406_vm 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 174 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 175 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2187 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2224 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2328 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2336 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2337 2024-09-15T22:24:53.026 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2579 2024-09-15T22:24:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2600 2024-09-15T22:24:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2600_fsm 2024-09-15T22:24:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2600_vm 2024-09-15T22:24:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601 2024-09-15T22:24:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2601_fsm 2024-09-15T22:24:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2601_vm 2024-09-15T22:24:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2602 2024-09-15T22:24:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2602_fsm 2024-09-15T22:24:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2602_vm 2024-09-15T22:24:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2603 2024-09-15T22:24:53.027 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2603_fsm 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2603_vm 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2604 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2605 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2605_fsm 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2605_vm 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2606_fsm 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2606_vm 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2607 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2607_fsm 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2607_vm 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 466944 Sep 15 22:09 2608 2024-09-15T22:24:53.028 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2608_fsm 2024-09-15T22:24:53.029 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2608_vm 2024-09-15T22:24:53.029 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 344064 Sep 15 22:09 2609 2024-09-15T22:24:53.029 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2609_fsm 2024-09-15T22:24:53.029 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2609_vm 2024-09-15T22:24:53.029 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2610 2024-09-15T22:24:53.029 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2610_fsm 2024-09-15T22:24:53.029 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2610_vm 2024-09-15T22:24:53.029 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2611 2024-09-15T22:24:53.029 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2612_fsm 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2612_vm 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2613 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2615_fsm 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2615_vm 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2616 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2616_fsm 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2616_vm 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Sep 15 22:09 2617 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2617_fsm 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2617_vm 2024-09-15T22:24:53.030 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 15 22:09 2618 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2618_fsm 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2618_vm 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 163840 Sep 15 22:24 2619 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:11 2619_fsm 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2619_vm 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2620 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2650 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2651 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2652 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2653 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2654 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2655 2024-09-15T22:24:53.031 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2656 2024-09-15T22:24:53.032 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2657 2024-09-15T22:24:53.032 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Sep 15 22:09 2658 2024-09-15T22:24:53.032 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Sep 15 22:09 2659 2024-09-15T22:24:53.032 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2660 2024-09-15T22:24:53.032 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2661 2024-09-15T22:24:53.032 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2662 2024-09-15T22:24:53.032 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2663 2024-09-15T22:24:53.032 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2664 2024-09-15T22:24:53.032 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2665 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2666 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2667 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2668 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2669 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2670 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 278528 Sep 15 22:09 2673 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 352256 Sep 15 22:09 2674 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Sep 15 22:09 2675 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2678 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2679 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2680 2024-09-15T22:24:53.033 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2681 2024-09-15T22:24:53.034 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2682 2024-09-15T22:24:53.034 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2683 2024-09-15T22:24:53.034 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2684 2024-09-15T22:24:53.034 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2685 2024-09-15T22:24:53.034 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2686 2024-09-15T22:24:53.034 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2687 2024-09-15T22:24:53.034 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2688 2024-09-15T22:24:53.034 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Sep 15 22:09 2689 2024-09-15T22:24:53.034 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Sep 15 22:09 2690 2024-09-15T22:24:53.034 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 253952 Sep 15 22:09 2691 2024-09-15T22:24:53.034 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2692 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2693 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:12 2696 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2699 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2701 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2702 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2703 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 2704 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2753 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2753_fsm 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2753_vm 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2754 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2755 2024-09-15T22:24:53.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2756 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 2757 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2830 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2831 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2832 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2833 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2834 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2835 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2836 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2837 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 491520 Sep 15 22:09 2838 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2838_fsm 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2838_vm 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2839 2024-09-15T22:24:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2840_fsm 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2840_vm 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2841 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2995 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2996 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3079_fsm 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3079_vm 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3080 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3081 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3085 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3118 2024-09-15T22:24:53.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3119 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Sep 15 22:09 3164 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3256 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3257 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3258 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3350 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3351 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3379 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3380 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3381 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3394 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3394_fsm 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3394_vm 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3395 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3429 2024-09-15T22:24:53.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3430 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3431 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3433 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3439 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3440 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Sep 15 22:09 3455 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 221184 Sep 15 22:09 3456 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3456_fsm 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3456_vm 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3466 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3467 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3468 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3501 2024-09-15T22:24:53.039 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3502 2024-09-15T22:24:53.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3503 2024-09-15T22:24:53.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3534 2024-09-15T22:24:53.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541 2024-09-15T22:24:53.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3541_fsm 2024-09-15T22:24:53.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3541_vm 2024-09-15T22:24:53.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3542 2024-09-15T22:24:53.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3574 2024-09-15T22:24:53.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3575 2024-09-15T22:24:53.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3576 2024-09-15T22:24:53.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3596 2024-09-15T22:24:53.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3597 2024-09-15T22:24:53.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3598 2024-09-15T22:24:53.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3599 2024-09-15T22:24:53.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600 2024-09-15T22:24:53.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3600_fsm 2024-09-15T22:24:53.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3600_vm 2024-09-15T22:24:53.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601 2024-09-15T22:24:53.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3601_fsm 2024-09-15T22:24:53.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3601_vm 2024-09-15T22:24:53.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602 2024-09-15T22:24:53.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3602_fsm 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3602_vm 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3603_fsm 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3603_vm 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3604 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3605 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3606 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3607 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3608 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Sep 15 22:09 3609 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3712 2024-09-15T22:24:53.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 3764_fsm 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3764_vm 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3766 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 3767 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3997 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4143 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4144 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4145 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4146 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4147 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4148 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4149 2024-09-15T22:24:53.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4150 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4151 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4152 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4153 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4154 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4155 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4156 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4157 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4158 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4159 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4160 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4161 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4162 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4163 2024-09-15T22:24:53.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4164 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4165 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4166 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4167 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4168 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4169 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4170 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4171 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4172 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4173 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4174 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 5002 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 548 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 549 2024-09-15T22:24:53.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6102 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6104 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6106 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6110 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6111 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6112 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6113 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6117 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 826 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 827 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 828 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 15 22:09 pg_filenode.map 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 151596 Sep 15 22:09 pg_internal.init 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:09 PG_VERSION 2024-09-15T22:24:53.046 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 4 Sep 15 22:09 .. 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/global: 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:total 560 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 58 Sep 15 22:09 . 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1213 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1213_fsm 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1213_vm 2024-09-15T22:24:53.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1214 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1214_fsm 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1214_vm 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 1232 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 1233 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1260 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1260_fsm 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1260_vm 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1261 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1261_fsm 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1261_vm 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1262 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 1262_fsm 2024-09-15T22:24:53.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 1262_vm 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2396 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Sep 15 22:09 2396_fsm 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2396_vm 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2397 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2671 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2672 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2676 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2677 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2694 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2695 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2697 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Sep 15 22:09 2698 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2846 2024-09-15T22:24:53.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2847 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2964 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2965 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 2966 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 2967 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 3592 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 3593 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4060 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4061 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4175 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4176 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4177 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4178 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4181 2024-09-15T22:24:53.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4182 2024-09-15T22:24:53.051 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4183 2024-09-15T22:24:53.051 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4184 2024-09-15T22:24:53.051 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 4185 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 4186 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6000 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6001 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6002 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 6100 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6114 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 6115 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:24 pg_control 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Sep 15 22:09 pg_filenode.map 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 23144 Sep 15 22:09 pg_internal.init 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.053 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/log: 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:total 2 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 . 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1223 Sep 15 22:24 postgresql-Sun.log 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.054 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical: 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:24 . 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 mappings 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Sep 15 22:24 replorigin_checkpoint 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 snapshots 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:24 .. 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.055 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:24 .. 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 15 22:09 . 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 members 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 offsets 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2024-09-15T22:24:53.056 INFO:teuthology.orchestra.run.smithi005.stdout:total 8 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 . 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 15 22:09 .. 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:09 0000 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout:total 8 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 . 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 2 Sep 15 22:09 .. 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Sep 15 22:14 0000 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_notify: 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.057 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_serial: 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.058 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_stat: 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:total 15 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Sep 15 22:24 . 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1952 Sep 15 22:24 db_0.stat 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 10979 Sep 15 22:24 db_13494.stat 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1151 Sep 15 22:24 global.stat 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:24 . 2024-09-15T22:24:53.059 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.060 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.060 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2024-09-15T22:24:53.060 INFO:teuthology.orchestra.run.smithi005.stdout:total 96 2024-09-15T22:24:53.060 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:24 . 2024-09-15T22:24:53.060 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.060 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Sep 15 22:24 000B 2024-09-15T22:24:53.060 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.060 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2024-09-15T22:24:53.060 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.060 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.060 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_wal: 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout:total 966656 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 60 Sep 15 22:24 . 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:24 000000010000000000000060 2024-09-15T22:24:53.061 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:14 000000010000000000000061 2024-09-15T22:24:53.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:14 000000010000000000000062 2024-09-15T22:24:53.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:16 000000010000000000000063 2024-09-15T22:24:53.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 000000010000000000000064 2024-09-15T22:24:53.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:14 000000010000000000000065 2024-09-15T22:24:53.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:17 000000010000000000000066 2024-09-15T22:24:53.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 000000010000000000000067 2024-09-15T22:24:53.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 000000010000000000000068 2024-09-15T22:24:53.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 000000010000000000000069 2024-09-15T22:24:53.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 00000001000000000000006A 2024-09-15T22:24:53.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:17 00000001000000000000006B 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:18 00000001000000000000006C 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 00000001000000000000006D 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 00000001000000000000006E 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:17 00000001000000000000006F 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:16 000000010000000000000070 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:16 000000010000000000000071 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:14 000000010000000000000072 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 000000010000000000000073 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 000000010000000000000074 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 000000010000000000000075 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 000000010000000000000076 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:19 000000010000000000000077 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 000000010000000000000078 2024-09-15T22:24:53.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 000000010000000000000079 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 00000001000000000000007A 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:14 00000001000000000000007B 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:16 00000001000000000000007C 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 00000001000000000000007D 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 00000001000000000000007E 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:15 00000001000000000000007F 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:19 000000010000000000000080 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:21 000000010000000000000081 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:19 000000010000000000000082 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:20 000000010000000000000083 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:21 000000010000000000000084 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:21 000000010000000000000085 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:19 000000010000000000000086 2024-09-15T22:24:53.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:21 000000010000000000000087 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:22 000000010000000000000088 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:21 000000010000000000000089 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:23 00000001000000000000008A 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:20 00000001000000000000008B 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:22 00000001000000000000008C 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:19 00000001000000000000008D 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:21 00000001000000000000008E 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:19 00000001000000000000008F 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:21 000000010000000000000090 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:22 000000010000000000000091 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:19 000000010000000000000092 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:19 000000010000000000000093 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:20 000000010000000000000094 2024-09-15T22:24:53.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:20 000000010000000000000095 2024-09-15T22:24:53.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:21 000000010000000000000096 2024-09-15T22:24:53.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:19 000000010000000000000097 2024-09-15T22:24:53.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:19 000000010000000000000098 2024-09-15T22:24:53.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:19 000000010000000000000099 2024-09-15T22:24:53.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Sep 15 22:23 00000001000000000000009A 2024-09-15T22:24:53.066 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 archive_status 2024-09-15T22:24:53.066 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.066 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2024-09-15T22:24:53.066 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2024-09-15T22:24:53.066 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Sep 15 22:09 . 2024-09-15T22:24:53.066 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 60 Sep 15 22:24 .. 2024-09-15T22:24:53.067 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:53.067 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_xact: 2024-09-15T22:24:53.067 INFO:teuthology.orchestra.run.smithi005.stdout:total 184 2024-09-15T22:24:53.067 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 1 Sep 15 22:09 . 2024-09-15T22:24:53.067 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 25 Sep 15 22:24 .. 2024-09-15T22:24:53.067 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 188416 Sep 15 22:24 0000 2024-09-15T22:24:53.067 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-15T22:24:53.079 INFO:tasks.quiescer:joining Quiescers 2024-09-15T22:24:53.079 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-15T22:24:53.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:24:53.080 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '398ff487', total seconds elapsed: 22.7 2024-09-15T22:24:53.335 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:53.717 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:53 smithi005 ceph-mon[30944]: pgmap v500: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-09-15T22:24:53.839 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:53 smithi099 ceph-mon[34892]: pgmap v500: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-09-15T22:24:53.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:53 smithi129 ceph-mon[35314]: pgmap v500: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-09-15T22:24:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.926+0000 7ff45d1e9640 1 -- 172.21.15.5:0/3085298142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff458106a80 msgr2=0x7ff458106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.926+0000 7ff45d1e9640 1 --2- 172.21.15.5:0/3085298142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff458106a80 0x7ff458106e60 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff448009930 tx=0x7ff44802a420 comp rx=0 tx=0).stop 2024-09-15T22:24:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.926+0000 7ff45d1e9640 1 -- 172.21.15.5:0/3085298142 shutdown_connections 2024-09-15T22:24:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.926+0000 7ff45d1e9640 1 --2- 172.21.15.5:0/3085298142 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff458101420 0x7ff45810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.926+0000 7ff45d1e9640 1 --2- 172.21.15.5:0/3085298142 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff458100a80 0x7ff458100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.926+0000 7ff45d1e9640 1 --2- 172.21.15.5:0/3085298142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff458106a80 0x7ff458106e60 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.926+0000 7ff45d1e9640 1 -- 172.21.15.5:0/3085298142 >> 172.21.15.5:0/3085298142 conn(0x7ff4580fc7a0 msgr2=0x7ff4580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.926+0000 7ff45d1e9640 1 -- 172.21.15.5:0/3085298142 shutdown_connections 2024-09-15T22:24:53.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.926+0000 7ff45d1e9640 1 -- 172.21.15.5:0/3085298142 wait complete. 2024-09-15T22:24:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.927+0000 7ff45d1e9640 1 Processor -- start 2024-09-15T22:24:53.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.927+0000 7ff45d1e9640 1 -- start start 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.927+0000 7ff45d1e9640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff458100a80 0x7ff45819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.927+0000 7ff45d1e9640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff458101420 0x7ff45819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.927+0000 7ff45d1e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff458106a80 0x7ff45819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.927+0000 7ff45d1e9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff458110e80 con 0x7ff458106a80 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.927+0000 7ff45d1e9640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff458110d00 con 0x7ff458100a80 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.927+0000 7ff45d1e9640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff458111000 con 0x7ff458101420 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff45759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff458106a80 0x7ff45819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff45759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff458106a80 0x7ff45819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54690/0 (socket says 172.21.15.5:54690) 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff45659c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff458101420 0x7ff45819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff45759e640 1 -- 172.21.15.5:0/2097807903 learned_addr learned my addr 172.21.15.5:0/2097807903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff456d9d640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff458100a80 0x7ff45819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff45659c640 1 -- 172.21.15.5:0/2097807903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff458100a80 msgr2=0x7ff45819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff45659c640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff458100a80 0x7ff45819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff45659c640 1 -- 172.21.15.5:0/2097807903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff458106a80 msgr2=0x7ff45819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff45659c640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff458106a80 0x7ff45819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff45659c640 1 -- 172.21.15.5:0/2097807903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45819f5f0 con 0x7ff458101420 2024-09-15T22:24:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff45759e640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff458106a80 0x7ff45819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:24:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.928+0000 7ff45659c640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff458101420 0x7ff45819ab80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff44000c970 tx=0x7ff44000ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.929+0000 7ff44ffff640 1 -- 172.21.15.5:0/2097807903 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff440007bb0 con 0x7ff458101420 2024-09-15T22:24:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.929+0000 7ff45d1e9640 1 -- 172.21.15.5:0/2097807903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45819f8e0 con 0x7ff458101420 2024-09-15T22:24:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.929+0000 7ff44ffff640 1 -- 172.21.15.5:0/2097807903 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff440007d50 con 0x7ff458101420 2024-09-15T22:24:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.929+0000 7ff44ffff640 1 -- 172.21.15.5:0/2097807903 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff440004e60 con 0x7ff458101420 2024-09-15T22:24:53.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.929+0000 7ff45d1e9640 1 -- 172.21.15.5:0/2097807903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4581a71c0 con 0x7ff458101420 2024-09-15T22:24:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.930+0000 7ff45d1e9640 1 -- 172.21.15.5:0/2097807903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff458100ee0 con 0x7ff458101420 2024-09-15T22:24:53.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.931+0000 7ff44ffff640 1 -- 172.21.15.5:0/2097807903 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff440005000 con 0x7ff458101420 2024-09-15T22:24:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.933+0000 7ff44ffff640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff4440a33b0 0x7ff4440a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.933+0000 7ff456d9d640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff4440a33b0 0x7ff4440a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:53.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.933+0000 7ff44ffff640 1 -- 172.21.15.5:0/2097807903 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7ff4400cfd20 con 0x7ff458101420 2024-09-15T22:24:53.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.933+0000 7ff456d9d640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff4440a33b0 0x7ff4440a5870 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff448009900 tx=0x7ff4480058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:53.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:53.936+0000 7ff44ffff640 1 -- 172.21.15.5:0/2097807903 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff440014030 con 0x7ff458101420 2024-09-15T22:24:54.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.151+0000 7ff45d1e9640 1 -- 172.21.15.5:0/2097807903 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4581a74c0 con 0x7ff458101420 2024-09-15T22:24:54.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.153+0000 7ff44ffff640 1 -- 172.21.15.5:0/2097807903 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7ff44009bc50 con 0x7ff458101420 2024-09-15T22:24:54.154 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:24:54.154 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:24:54.155 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:24:54.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.154+0000 7ff45d1e9640 1 -- 172.21.15.5:0/2097807903 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff4440a33b0 msgr2=0x7ff4440a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:54.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.155+0000 7ff45d1e9640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff4440a33b0 0x7ff4440a5870 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff448009900 tx=0x7ff4480058d0 comp rx=0 tx=0).stop 2024-09-15T22:24:54.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.155+0000 7ff45d1e9640 1 -- 172.21.15.5:0/2097807903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff458101420 msgr2=0x7ff45819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:54.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.155+0000 7ff45d1e9640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff458101420 0x7ff45819ab80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff44000c970 tx=0x7ff44000ce30 comp rx=0 tx=0).stop 2024-09-15T22:24:54.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.155+0000 7ff45d1e9640 1 -- 172.21.15.5:0/2097807903 shutdown_connections 2024-09-15T22:24:54.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.155+0000 7ff45d1e9640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff4440a33b0 0x7ff4440a5870 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:54.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.155+0000 7ff45d1e9640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff458106a80 0x7ff45819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:54.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.155+0000 7ff45d1e9640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff458101420 0x7ff45819ab80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:54.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.155+0000 7ff45d1e9640 1 --2- 172.21.15.5:0/2097807903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff458100a80 0x7ff45819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:54.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.155+0000 7ff45d1e9640 1 -- 172.21.15.5:0/2097807903 >> 172.21.15.5:0/2097807903 conn(0x7ff4580fc7a0 msgr2=0x7ff45806a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:54.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.155+0000 7ff45d1e9640 1 -- 172.21.15.5:0/2097807903 shutdown_connections 2024-09-15T22:24:54.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:54.155+0000 7ff45d1e9640 1 -- 172.21.15.5:0/2097807903 wait complete. 2024-09-15T22:24:54.785 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24491 quiesce db --cancel --all' 2024-09-15T22:24:54.785 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24491 quiesce db --cancel --all 2024-09-15T22:24:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:24:55.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:54 smithi005 ceph-mon[30944]: pgmap v501: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 970 KiB/s wr, 97 op/s 2024-09-15T22:24:55.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:54 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2097807903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:55.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:54 smithi099 ceph-mon[34892]: pgmap v501: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 970 KiB/s wr, 97 op/s 2024-09-15T22:24:55.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:54 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2097807903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:55.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:54 smithi129 ceph-mon[35314]: pgmap v501: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 970 KiB/s wr, 97 op/s 2024-09-15T22:24:55.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:54 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2097807903' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:24:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.538+0000 7efee5094640 1 -- 172.21.15.5:0/2403735345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0101de0 msgr2=0x7efee01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.538+0000 7efee5094640 1 --2- 172.21.15.5:0/2403735345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0101de0 0x7efee01021c0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7efed0009900 tx=0x7efed002a410 comp rx=0 tx=0).stop 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.538+0000 7efee5094640 1 -- 172.21.15.5:0/2403735345 shutdown_connections 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.538+0000 7efee5094640 1 --2- 172.21.15.5:0/2403735345 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0106f10 0x7efee010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.538+0000 7efee5094640 1 --2- 172.21.15.5:0/2403735345 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee0102790 0x7efee01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.538+0000 7efee5094640 1 --2- 172.21.15.5:0/2403735345 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0101de0 0x7efee01021c0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.538+0000 7efee5094640 1 -- 172.21.15.5:0/2403735345 >> 172.21.15.5:0/2403735345 conn(0x7efee00fd8c0 msgr2=0x7efee00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.538+0000 7efee5094640 1 -- 172.21.15.5:0/2403735345 shutdown_connections 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.538+0000 7efee5094640 1 -- 172.21.15.5:0/2403735345 wait complete. 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.539+0000 7efee5094640 1 Processor -- start 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.539+0000 7efee5094640 1 -- start start 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.539+0000 7efee5094640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0101de0 0x7efee01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.539+0000 7efee5094640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee0102790 0x7efee0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.539+0000 7efee5094640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0106f10 0x7efee019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.539+0000 7efee5094640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efee0111f70 con 0x7efee0106f10 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.539+0000 7efee5094640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efee0111df0 con 0x7efee0101de0 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.539+0000 7efee5094640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efee01120f0 con 0x7efee0102790 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.539+0000 7efedf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0106f10 0x7efee019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efedf59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0106f10 0x7efee019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54706/0 (socket says 172.21.15.5:54706) 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efedf59e640 1 -- 172.21.15.5:0/1734937155 learned_addr learned my addr 172.21.15.5:0/1734937155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efede59c640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee0102790 0x7efee0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efeded9d640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0101de0 0x7efee01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efedf59e640 1 -- 172.21.15.5:0/1734937155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee0102790 msgr2=0x7efee0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efedf59e640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee0102790 0x7efee0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efedf59e640 1 -- 172.21.15.5:0/1734937155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0101de0 msgr2=0x7efee01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efedf59e640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0101de0 0x7efee01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efedf59e640 1 -- 172.21.15.5:0/1734937155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee019d480 con 0x7efee0106f10 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efedf59e640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0106f10 0x7efee019cda0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7efed800eaa0 tx=0x7efed800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efec7fff640 1 -- 172.21.15.5:0/1734937155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed800cdb0 con 0x7efee0106f10 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efee5094640 1 -- 172.21.15.5:0/1734937155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee019d770 con 0x7efee0106f10 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efec7fff640 1 -- 172.21.15.5:0/1734937155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed8005020 con 0x7efee0106f10 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efec7fff640 1 -- 172.21.15.5:0/1734937155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed80116b0 con 0x7efee0106f10 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.540+0000 7efee5094640 1 -- 172.21.15.5:0/1734937155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a5050 con 0x7efee0106f10 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efec7fff640 1 -- 172.21.15.5:0/1734937155 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7efed8005560 con 0x7efee0106f10 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efee746d740 1 Processor -- start 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efee746d740 1 -- start start 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efee746d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5dc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efee746d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dc730 0x55bc4b5e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efee746d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5e93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efee746d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bc4b53c200 con 0x55bc4b5dc730 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efee746d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bc4b4e2dc0 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efee746d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bc4b5381c0 con 0x55bc4b5dbe10 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efeded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dc730 0x55bc4b5e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efeded9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dc730 0x55bc4b5e6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54710/0 (socket says 172.21.15.5:54710) 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efeded9d640 1 -- 172.21.15.5:0/2605600279 learned_addr learned my addr 172.21.15.5:0/2605600279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.543+0000 7efede59c640 1 --2- 172.21.15.5:0/2605600279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5dc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efec7fff640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efec80a33b0 0x7efec80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efede59c640 1 --2- 172.21.15.5:0/2605600279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5e93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efeded9d640 1 -- 172.21.15.5:0/2605600279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 msgr2=0x55bc4b5dc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efeded9d640 1 --2- 172.21.15.5:0/2605600279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5dc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efeded9d640 1 -- 172.21.15.5:0/2605600279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5e6ff0 msgr2=0x55bc4b5e93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efeded9d640 1 --2- 172.21.15.5:0/2605600279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5e93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efeded9d640 1 -- 172.21.15.5:0/2605600279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc4b2ad620 con 0x55bc4b5dc730 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efede59c640 1 --2- 172.21.15.5:0/2605600279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5dc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efec7fff640 1 -- 172.21.15.5:0/1734937155 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7efed80d0260 con 0x7efee0106f10 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efedf59e640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efec80a33b0 0x7efec80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efeded9d640 1 --2- 172.21.15.5:0/2605600279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dc730 0x55bc4b5e6ab0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7efed0009900 tx=0x7efed0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df9cb71dd517b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efec57fa640 1 -- 172.21.15.5:0/2605600279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0040070 con 0x55bc4b5dc730 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efec57fa640 1 -- 172.21.15.5:0/2605600279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed00319f0 con 0x55bc4b5dc730 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.544+0000 7efec57fa640 1 -- 172.21.15.5:0/2605600279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0035040 con 0x55bc4b5dc730 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.545+0000 7efee746d740 1 -- 172.21.15.5:0/2605600279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dc730 msgr2=0x55bc4b5e6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.545+0000 7efedf59e640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efec80a33b0 0x7efec80a5870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efed809c180 tx=0x7efed80bc040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.545+0000 7efee746d740 1 --2- 172.21.15.5:0/2605600279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dc730 0x55bc4b5e6ab0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7efed0009900 tx=0x7efed0005e80 comp rx=0 tx=0).stop 2024-09-15T22:24:55.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.545+0000 7efee746d740 1 -- 172.21.15.5:0/2605600279 shutdown_connections 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.545+0000 7efee746d740 1 --2- 172.21.15.5:0/2605600279 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5e93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.545+0000 7efee746d740 1 --2- 172.21.15.5:0/2605600279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dc730 0x55bc4b5e6ab0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.545+0000 7efee746d740 1 --2- 172.21.15.5:0/2605600279 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5dc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.545+0000 7efee746d740 1 -- 172.21.15.5:0/2605600279 >> 172.21.15.5:0/2605600279 conn(0x55bc4b5d83a0 msgr2=0x55bc4b5d8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.545+0000 7efee746d740 1 -- 172.21.15.5:0/2605600279 shutdown_connections 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.545+0000 7efee746d740 1 -- 172.21.15.5:0/2605600279 wait complete. 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.545+0000 7efee746d740 1 -- start start 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efee746d740 1 Processor -- start 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efee746d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efee746d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5e3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efee746d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5e40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efee746d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bc4b5e9bc0 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efee746d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55bc4b4d6c50 con 0x55bc4b5dc730 2024-09-15T22:24:55.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efee746d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55bc4b5e9d40 con 0x55bc4b5dbe10 2024-09-15T22:24:55.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efee746d740 10 client.0 _ll_register_callbacks cb 0x55bc4b4e8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:55.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efede59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5e40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efede59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5e40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54716/0 (socket says 172.21.15.5:54716) 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efede59c640 1 -- 172.21.15.5:0/1651866441 learned_addr learned my addr 172.21.15.5:0/1651866441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efeded9d640 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5e3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efede59c640 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efede59c640 1 -- 172.21.15.5:0/1651866441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 msgr2=0x55bc4b5e68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efede59c640 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5e68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efede59c640 1 -- 172.21.15.5:0/1651866441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 msgr2=0x55bc4b5e3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efede59c640 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5e3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efede59c640 1 -- 172.21.15.5:0/1651866441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc4b5e48a0 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efeded9d640 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efede59c640 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5e40c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7efed400bf00 tx=0x7efed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b215da96a943392 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.546+0000 7efec2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:55.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efec2ff5640 1 -- 172.21.15.5:0/1651866441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed401b040 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efec2ff5640 1 -- 172.21.15.5:0/1651866441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7efed4005020 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efee746d740 1 -- 172.21.15.5:0/1651866441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed00319f0 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efec2ff5640 1 -- 172.21.15.5:0/1651866441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efed4005220 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efee746d740 10 client.18129 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efee746d740 1 -- 172.21.15.5:0/1651866441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bc4b5f5750 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efec2ff5640 1 -- 172.21.15.5:0/1651866441 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed00319f0 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efec2ff5640 1 -- 172.21.15.5:0/1651866441 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efed40146e0 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efee746d740 10 client.18129 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efee746d740 10 client.18129 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efee746d740 20 client.18129 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efee746d740 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bc4b5f6c10 0x55bc4b5f9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efee746d740 4 client.18129 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.547+0000 7efee746d740 1 -- 172.21.15.5:0/1651866441 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc4b549770 con 0x55bc4b5f6c10 2024-09-15T22:24:55.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.548+0000 7efeded9d640 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bc4b5f6c10 0x55bc4b5f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.548+0000 7efeded9d640 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bc4b5f6c10 0x55bc4b5f9000 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9846a253297e0e61 server_cookie=6a7bef84b365bf45 in_seq=0 out_seq=0 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.548+0000 7efec2ff5640 10 client.18129 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efec2ff5640 1 -- 172.21.15.5:0/1651866441 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc4b549770 con 0x55bc4b5f6c10 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efec2ff5640 10 client.18129 handle_command_reply: tid=0 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 1 client.18129 shutdown 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 1 -- 172.21.15.5:0/1651866441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 msgr2=0x55bc4b5e40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5e40c0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7efed400bf00 tx=0x7efed40046c0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 1 -- 172.21.15.5:0/1651866441 shutdown_connections 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5e40c0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bc4b5f6c10 0x55bc4b5f9000 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5e3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 1 --2- 172.21.15.5:0/1651866441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5e68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efec2ff5640 0 client.18129 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 1 -- 172.21.15.5:0/1651866441 >> 172.21.15.5:0/1651866441 conn(0x55bc4b5d83a0 msgr2=0x55bc4b5d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 1 -- 172.21.15.5:0/1651866441 shutdown_connections 2024-09-15T22:24:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 1 -- 172.21.15.5:0/1651866441 wait complete. 2024-09-15T22:24:55.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.549+0000 7efee746d740 20 client.18129 trim_cache size 0 max 16384 2024-09-15T22:24:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.566+0000 7efee746d740 1 Processor -- start 2024-09-15T22:24:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.566+0000 7efee746d740 1 -- start start 2024-09-15T22:24:55.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.566+0000 7efee746d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5e2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.566+0000 7efee746d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.566+0000 7efee746d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.566+0000 7efee746d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efee01af810 con 0x55bc4b5dbe10 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.566+0000 7efee746d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efee01ad0e0 con 0x55bc4b5dc730 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.566+0000 7efee746d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efee01aaf50 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.566+0000 7efede59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5e2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.566+0000 7efede59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5e2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54732/0 (socket says 172.21.15.5:54732) 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.567+0000 7efede59c640 1 -- 172.21.15.5:0/3857940064 learned_addr learned my addr 172.21.15.5:0/3857940064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.567+0000 7efeded9d640 1 --2- 172.21.15.5:0/3857940064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.567+0000 7efede59c640 1 -- 172.21.15.5:0/3857940064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5e6ff0 msgr2=0x55bc4b5fe880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.567+0000 7efede59c640 1 --2- 172.21.15.5:0/3857940064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.567+0000 7efede59c640 1 -- 172.21.15.5:0/3857940064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 msgr2=0x55bc4b5f2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.567+0000 7efede59c640 1 --2- 172.21.15.5:0/3857940064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5f2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.567+0000 7efede59c640 1 -- 172.21.15.5:0/3857940064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc4b5dc560 con 0x55bc4b5dbe10 2024-09-15T22:24:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.567+0000 7efede59c640 1 --2- 172.21.15.5:0/3857940064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5e2450 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7efed4004690 tx=0x7efed4003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ce66c9f844c303 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.567+0000 7efec4ff9640 1 -- 172.21.15.5:0/3857940064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed40146e0 con 0x55bc4b5dbe10 2024-09-15T22:24:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.567+0000 7efec4ff9640 1 -- 172.21.15.5:0/3857940064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed4014880 con 0x55bc4b5dbe10 2024-09-15T22:24:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.567+0000 7efec4ff9640 1 -- 172.21.15.5:0/3857940064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4012480 con 0x55bc4b5dbe10 2024-09-15T22:24:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.568+0000 7efee746d740 1 -- 172.21.15.5:0/3857940064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dbe10 msgr2=0x55bc4b5e2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.568+0000 7efee746d740 1 --2- 172.21.15.5:0/3857940064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5e2450 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7efed4004690 tx=0x7efed4003e50 comp rx=0 tx=0).stop 2024-09-15T22:24:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.568+0000 7efee746d740 1 -- 172.21.15.5:0/3857940064 shutdown_connections 2024-09-15T22:24:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.568+0000 7efee746d740 1 --2- 172.21.15.5:0/3857940064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5fe880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.568+0000 7efee746d740 1 --2- 172.21.15.5:0/3857940064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5f2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.568+0000 7efee746d740 1 --2- 172.21.15.5:0/3857940064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5e2450 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.568+0000 7efee746d740 1 -- 172.21.15.5:0/3857940064 >> 172.21.15.5:0/3857940064 conn(0x55bc4b5d83a0 msgr2=0x55bc4b5d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.568+0000 7efee746d740 1 -- 172.21.15.5:0/3857940064 shutdown_connections 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.568+0000 7efee746d740 1 -- 172.21.15.5:0/3857940064 wait complete. 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.568+0000 7efee746d740 1 -- start start 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.568+0000 7efee746d740 1 Processor -- start 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efee746d740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5ef3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efee746d740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5ea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efee746d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5eac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efee746d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efee01ab5a0 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efee746d740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7efee01aca10 con 0x55bc4b5dc730 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efee746d740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7efee01abc70 con 0x55bc4b5dbe10 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efee746d740 10 client.0 _ll_register_callbacks cb 0x7efed400bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efede59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5eac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efede59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5eac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54744/0 (socket says 172.21.15.5:54744) 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efede59c640 1 -- 172.21.15.5:0/1672051939 learned_addr learned my addr 172.21.15.5:0/1672051939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efeded9d640 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5ea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efede59c640 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5ef3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efede59c640 1 -- 172.21.15.5:0/1672051939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 msgr2=0x55bc4b5ef3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efede59c640 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5ef3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efede59c640 1 -- 172.21.15.5:0/1672051939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 msgr2=0x55bc4b5ea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efede59c640 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5ea720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efede59c640 1 -- 172.21.15.5:0/1672051939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc4b5eb1a0 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efede59c640 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5eac60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7efed40059b0 tx=0x7efed4004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c88b25832c73601 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efec2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efec2ff5640 1 -- 172.21.15.5:0/1672051939 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4014270 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efec2ff5640 1 -- 172.21.15.5:0/1672051939 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7efed40053c0 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.569+0000 7efee746d740 1 -- 172.21.15.5:0/1672051939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed4014880 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efec2ff5640 1 -- 172.21.15.5:0/1672051939 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efed40055c0 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efee746d740 10 client.18135 fetch_fsmap learned FSMap version 25 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efee746d740 1 -- 172.21.15.5:0/1672051939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bc4b661500 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efec2ff5640 1 -- 172.21.15.5:0/1672051939 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed4014880 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efec2ff5640 1 -- 172.21.15.5:0/1672051939 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7efed402b9f0 con 0x55bc4b5e6ff0 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efee746d740 10 client.18135 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efee746d740 10 client.18135 resolve_mds: validated gid 24491 aka daemon mds.b 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efee746d740 20 client.18135 populate_metadata read hostname 'smithi005' 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efee746d740 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bc4b661b60 0x55bc4b663f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efee746d740 4 client.18135 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efee746d740 1 -- 172.21.15.5:0/1672051939 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55bc4b543c80 con 0x55bc4b661b60 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.570+0000 7efeded9d640 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bc4b661b60 0x55bc4b663f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efec2ff5640 10 client.18135 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efeded9d640 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bc4b661b60 0x55bc4b663f50 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50ce850e89ca3b2d server_cookie=31f37e0f09e4d3df in_seq=0 out_seq=0 2024-09-15T22:24:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efec2ff5640 1 -- 172.21.15.5:0/1672051939 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55bc4b543c80 con 0x55bc4b661b60 2024-09-15T22:24:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efec2ff5640 10 client.18135 handle_command_reply: tid=0 2024-09-15T22:24:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efee746d740 1 client.18135 shutdown 2024-09-15T22:24:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efee746d740 1 -- 172.21.15.5:0/1672051939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 msgr2=0x55bc4b5eac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efee746d740 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5eac60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7efed40059b0 tx=0x7efed4004c70 comp rx=0 tx=0).stop 2024-09-15T22:24:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efee746d740 1 -- 172.21.15.5:0/1672051939 shutdown_connections 2024-09-15T22:24:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efee746d740 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc4b5e6ff0 0x55bc4b5eac60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efee746d740 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55bc4b661b60 0x55bc4b663f50 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efee746d740 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55bc4b5dc730 0x55bc4b5ea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efee746d740 1 --2- 172.21.15.5:0/1672051939 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55bc4b5dbe10 0x55bc4b5ef3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efec2ff5640 0 client.18135 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:24:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efee746d740 1 -- 172.21.15.5:0/1672051939 >> 172.21.15.5:0/1672051939 conn(0x55bc4b5d83a0 msgr2=0x55bc4b5d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.571+0000 7efee746d740 1 -- 172.21.15.5:0/1672051939 shutdown_connections 2024-09-15T22:24:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee746d740 1 -- 172.21.15.5:0/1672051939 wait complete. 2024-09-15T22:24:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee746d740 20 client.18135 trim_cache size 0 max 16384 2024-09-15T22:24:55.583 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:24:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2024-09-15T22:24:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24491, 2024-09-15T22:24:55.583 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 60, 2024-09-15T22:24:55.584 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": {} 2024-09-15T22:24:55.584 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:24:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 -- 172.21.15.5:0/1734937155 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efec80a33b0 msgr2=0x7efec80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efec80a33b0 0x7efec80a5870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efed809c180 tx=0x7efed80bc040 comp rx=0 tx=0).stop 2024-09-15T22:24:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 -- 172.21.15.5:0/1734937155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0106f10 msgr2=0x7efee019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0106f10 0x7efee019cda0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7efed800eaa0 tx=0x7efed800ef60 comp rx=0 tx=0).stop 2024-09-15T22:24:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 -- 172.21.15.5:0/1734937155 shutdown_connections 2024-09-15T22:24:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7efec80a33b0 0x7efec80a5870 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efee0106f10 0x7efee019cda0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7efee0102790 0x7efee0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 --2- 172.21.15.5:0/1734937155 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7efee0101de0 0x7efee01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 -- 172.21.15.5:0/1734937155 >> 172.21.15.5:0/1734937155 conn(0x7efee00fd8c0 msgr2=0x7efee00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 -- 172.21.15.5:0/1734937155 shutdown_connections 2024-09-15T22:24:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:55.572+0000 7efee5094640 1 -- 172.21.15.5:0/1734937155 wait complete. 2024-09-15T22:24:56.404 INFO:tasks.quiescer:done joining Quiescers 2024-09-15T22:24:56.405 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-15T22:24:56.415 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-15T22:24:56.415 INFO:teuthology.task.exec:Executing custom commands... 2024-09-15T22:24:56.415 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-15T22:24:56.416 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-15T22:24:56.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-15T22:24:56.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-15T22:24:56.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.802+0000 7f881e5c4640 1 -- 172.21.15.5:0/160048556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102d00 msgr2=0x7f88181030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:56.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.802+0000 7f881e5c4640 1 --2- 172.21.15.5:0/160048556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102d00 0x7f88181030e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f8808009900 tx=0x7f880802a480 comp rx=0 tx=0).stop 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.803+0000 7f881e5c4640 1 -- 172.21.15.5:0/160048556 shutdown_connections 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.803+0000 7f881e5c4640 1 --2- 172.21.15.5:0/160048556 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f881810c0d0 0x7f881810e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.803+0000 7f881e5c4640 1 --2- 172.21.15.5:0/160048556 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8818103620 0x7f881810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.803+0000 7f881e5c4640 1 --2- 172.21.15.5:0/160048556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102d00 0x7f88181030e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.803+0000 7f881e5c4640 1 -- 172.21.15.5:0/160048556 >> 172.21.15.5:0/160048556 conn(0x7f88180fc690 msgr2=0x7f88180feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.803+0000 7f881e5c4640 1 -- 172.21.15.5:0/160048556 shutdown_connections 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.803+0000 7f881e5c4640 1 -- 172.21.15.5:0/160048556 wait complete. 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.803+0000 7f881e5c4640 1 Processor -- start 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.803+0000 7f881e5c4640 1 -- start start 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f881e5c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102d00 0x7f881819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f881e5c4640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8818103620 0x7f881819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f881e5c4640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f881810c0d0 0x7f881819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:56.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f881e5c4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8818111ed0 con 0x7f8818102d00 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f881e5c4640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8818111d50 con 0x7f8818103620 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f881e5c4640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8818112050 con 0x7f881810c0d0 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f8817fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102d00 0x7f881819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f8817fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102d00 0x7f881819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54762/0 (socket says 172.21.15.5:54762) 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f8817fff640 1 -- 172.21.15.5:0/455307710 learned_addr learned my addr 172.21.15.5:0/455307710 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f88177fe640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8818103620 0x7f881819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f881cb61640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f881810c0d0 0x7f881819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f8817fff640 1 -- 172.21.15.5:0/455307710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f881810c0d0 msgr2=0x7f881819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f8817fff640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f881810c0d0 0x7f881819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f8817fff640 1 -- 172.21.15.5:0/455307710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8818103620 msgr2=0x7f881819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f8817fff640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8818103620 0x7f881819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:56.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f8817fff640 1 -- 172.21.15.5:0/455307710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881819f670 con 0x7f8818102d00 2024-09-15T22:24:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f8817fff640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102d00 0x7f881819a680 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f88080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f88157fa640 1 -- 172.21.15.5:0/455307710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880803f030 con 0x7f8818102d00 2024-09-15T22:24:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f88157fa640 1 -- 172.21.15.5:0/455307710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8808003ce0 con 0x7f8818102d00 2024-09-15T22:24:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f881e5c4640 1 -- 172.21.15.5:0/455307710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881819f900 con 0x7f8818102d00 2024-09-15T22:24:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.804+0000 7f88157fa640 1 -- 172.21.15.5:0/455307710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808035070 con 0x7f8818102d00 2024-09-15T22:24:56.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.805+0000 7f881e5c4640 1 -- 172.21.15.5:0/455307710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88181a71f0 con 0x7f8818102d00 2024-09-15T22:24:56.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.805+0000 7f881e5c4640 1 -- 172.21.15.5:0/455307710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88181030e0 con 0x7f8818102d00 2024-09-15T22:24:56.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.807+0000 7f88157fa640 1 -- 172.21.15.5:0/455307710 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8808042030 con 0x7f8818102d00 2024-09-15T22:24:56.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.807+0000 7f88157fa640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88000a33b0 0x7f88000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:56.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.807+0000 7f88177fe640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88000a33b0 0x7f88000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:56.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.807+0000 7f88157fa640 1 -- 172.21.15.5:0/455307710 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f88080f07e0 con 0x7f8818102d00 2024-09-15T22:24:56.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.808+0000 7f88177fe640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88000a33b0 0x7f88000a5870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8818102970 tx=0x7f880c00b390 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:56.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:56.810+0000 7f88157fa640 1 -- 172.21.15.5:0/455307710 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88080bc830 con 0x7f8818102d00 2024-09-15T22:24:57.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.009+0000 7f881e5c4640 1 -- 172.21.15.5:0/455307710 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f88180fe9b0 con 0x7f88000a33b0 2024-09-15T22:24:57.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.017+0000 7f88157fa640 1 -- 172.21.15.5:0/455307710 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f88180fe9b0 con 0x7f88000a33b0 2024-09-15T22:24:57.019 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-15T00:00:00", "created": "2024-09-15T22:09:03", "first": "2024-09-15T22:10:00", "last": "2024-09-15T22:24:01", "last_pruned": "2024-09-15T22:24:01", "created_count": 15, "pruned_count": 9, "active": true} 2024-09-15T22:24:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 -- 172.21.15.5:0/455307710 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88000a33b0 msgr2=0x7f88000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88000a33b0 0x7f88000a5870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8818102970 tx=0x7f880c00b390 comp rx=0 tx=0).stop 2024-09-15T22:24:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 -- 172.21.15.5:0/455307710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102d00 msgr2=0x7f881819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102d00 0x7f881819a680 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8808006010 tx=0x7f88080042f0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 -- 172.21.15.5:0/455307710 shutdown_connections 2024-09-15T22:24:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f88000a33b0 0x7f88000a5870 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f881810c0d0 0x7f881819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8818103620 0x7f881819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 --2- 172.21.15.5:0/455307710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8818102d00 0x7f881819a680 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 -- 172.21.15.5:0/455307710 >> 172.21.15.5:0/455307710 conn(0x7f88180fc690 msgr2=0x7f88180fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 -- 172.21.15.5:0/455307710 shutdown_connections 2024-09-15T22:24:57.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.019+0000 7f881e5c4640 1 -- 172.21.15.5:0/455307710 wait complete. 2024-09-15T22:24:57.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:56 smithi005 ceph-mon[30944]: pgmap v502: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 719 KiB/s wr, 72 op/s 2024-09-15T22:24:57.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-15T22:24:57.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.142+0000 7f8b67bbd640 1 -- 172.21.15.5:0/2953418967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b60101340 msgr2=0x7f8b6010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.142+0000 7f8b67bbd640 1 --2- 172.21.15.5:0/2953418967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b60101340 0x7f8b6010d870 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009cd0 tx=0x7f8b5802a8e0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.142+0000 7f8b67bbd640 1 -- 172.21.15.5:0/2953418967 shutdown_connections 2024-09-15T22:24:57.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.142+0000 7f8b67bbd640 1 --2- 172.21.15.5:0/2953418967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b60101340 0x7f8b6010d870 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.142+0000 7f8b67bbd640 1 --2- 172.21.15.5:0/2953418967 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b601009a0 0x7f8b60100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.142+0000 7f8b67bbd640 1 --2- 172.21.15.5:0/2953418967 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b601069b0 0x7f8b60106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.142+0000 7f8b67bbd640 1 -- 172.21.15.5:0/2953418967 >> 172.21.15.5:0/2953418967 conn(0x7f8b600779a0 msgr2=0x7f8b600fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.142+0000 7f8b67bbd640 1 -- 172.21.15.5:0/2953418967 shutdown_connections 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.142+0000 7f8b67bbd640 1 -- 172.21.15.5:0/2953418967 wait complete. 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b67bbd640 1 Processor -- start 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b67bbd640 1 -- start start 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b67bbd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b601009a0 0x7f8b601983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b67bbd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b60101340 0x7f8b60198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b67bbd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b601069b0 0x7f8b6019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b67bbd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60110df0 con 0x7f8b60101340 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b67bbd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60110c70 con 0x7f8b601009a0 2024-09-15T22:24:57.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b67bbd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60110f70 con 0x7f8b601069b0 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b65158640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b60101340 0x7f8b60198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b65158640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b60101340 0x7f8b60198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54784/0 (socket says 172.21.15.5:54784) 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b65158640 1 -- 172.21.15.5:0/1199332466 learned_addr learned my addr 172.21.15.5:0/1199332466 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.143+0000 7f8b65959640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b601009a0 0x7f8b601983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b65158640 1 -- 172.21.15.5:0/1199332466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b601069b0 msgr2=0x7f8b6019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b6615a640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b601069b0 0x7f8b6019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b65158640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b601069b0 0x7f8b6019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b65158640 1 -- 172.21.15.5:0/1199332466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b601009a0 msgr2=0x7f8b601983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b65158640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b601009a0 0x7f8b601983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b65158640 1 -- 172.21.15.5:0/1199332466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6019d3d0 con 0x7f8b60101340 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b65959640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b601009a0 0x7f8b601983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:24:57.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b6615a640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b601069b0 0x7f8b6019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:24:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b65158640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b60101340 0x7f8b60198920 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00d950 tx=0x7f8b5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b56ffd640 1 -- 172.21.15.5:0/1199332466 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c017070 con 0x7f8b60101340 2024-09-15T22:24:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b67bbd640 1 -- 172.21.15.5:0/1199332466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6019d6c0 con 0x7f8b60101340 2024-09-15T22:24:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b56ffd640 1 -- 172.21.15.5:0/1199332466 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b5c009c30 con 0x7f8b60101340 2024-09-15T22:24:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b56ffd640 1 -- 172.21.15.5:0/1199332466 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c00f6b0 con 0x7f8b60101340 2024-09-15T22:24:57.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.144+0000 7f8b67bbd640 1 -- 172.21.15.5:0/1199332466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b601a4f50 con 0x7f8b60101340 2024-09-15T22:24:57.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.145+0000 7f8b67bbd640 1 -- 172.21.15.5:0/1199332466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b601020e0 con 0x7f8b60101340 2024-09-15T22:24:57.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.146+0000 7f8b56ffd640 1 -- 172.21.15.5:0/1199332466 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f8b5c0109d0 con 0x7f8b60101340 2024-09-15T22:24:57.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.147+0000 7f8b56ffd640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b4c0a33b0 0x7f8b4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:24:57.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.147+0000 7f8b65959640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b4c0a33b0 0x7f8b4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:24:57.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.147+0000 7f8b56ffd640 1 -- 172.21.15.5:0/1199332466 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9541+0+0 (secure 0 0 0) 0x7f8b5c0cfdf0 con 0x7f8b60101340 2024-09-15T22:24:57.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.148+0000 7f8b65959640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b4c0a33b0 0x7f8b4c0a5870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8b50005fd0 tx=0x7f8b50005850 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:24:57.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.155+0000 7f8b56ffd640 1 -- 172.21.15.5:0/1199332466 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b5c009de0 con 0x7f8b60101340 2024-09-15T22:24:57.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:56 smithi099 ceph-mon[34892]: pgmap v502: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 719 KiB/s wr, 72 op/s 2024-09-15T22:24:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:56 smithi129 ceph-mon[35314]: pgmap v502: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 719 KiB/s wr, 72 op/s 2024-09-15T22:24:57.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.352+0000 7f8b67bbd640 1 -- 172.21.15.5:0/1199332466 --> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f8b6006a200 con 0x7f8b4c0a33b0 2024-09-15T22:24:57.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.360+0000 7f8b56ffd640 1 -- 172.21.15.5:0/1199332466 <== mgr.15228 v2:172.21.15.5:6800/1448662997 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f8b6006a200 con 0x7f8b4c0a33b0 2024-09-15T22:24:57.361 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1m 6m3h 2024-09-15T22:24:57.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 -- 172.21.15.5:0/1199332466 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b4c0a33b0 msgr2=0x7f8b4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b4c0a33b0 0x7f8b4c0a5870 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8b50005fd0 tx=0x7f8b50005850 comp rx=0 tx=0).stop 2024-09-15T22:24:57.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 -- 172.21.15.5:0/1199332466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b60101340 msgr2=0x7f8b60198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:24:57.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b60101340 0x7f8b60198920 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00d950 tx=0x7f8b5c00de10 comp rx=0 tx=0).stop 2024-09-15T22:24:57.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 -- 172.21.15.5:0/1199332466 shutdown_connections 2024-09-15T22:24:57.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f8b4c0a33b0 0x7f8b4c0a5870 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f8b601069b0 0x7f8b6019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b60101340 0x7f8b60198920 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 --2- 172.21.15.5:0/1199332466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f8b601009a0 0x7f8b601983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:24:57.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 -- 172.21.15.5:0/1199332466 >> 172.21.15.5:0/1199332466 conn(0x7f8b600779a0 msgr2=0x7f8b60069af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:24:57.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 -- 172.21.15.5:0/1199332466 shutdown_connections 2024-09-15T22:24:57.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:24:57.362+0000 7f8b67bbd640 1 -- 172.21.15.5:0/1199332466 wait complete. 2024-09-15T22:24:57.375 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-15T22:24:57.384 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-15T22:24:58.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:57 smithi005 ceph-mon[30944]: from='client.18141 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:24:58.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:57 smithi005 ceph-mon[30944]: from='client.18147 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:24:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:57 smithi099 ceph-mon[34892]: from='client.18141 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:24:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:57 smithi099 ceph-mon[34892]: from='client.18147 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:24:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:57 smithi129 ceph-mon[35314]: from='client.18141 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:24:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:57 smithi129 ceph-mon[35314]: from='client.18147 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-15T22:24:59.246 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:24:58 smithi129 ceph-mon[35314]: pgmap v503: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 367 KiB/s wr, 27 op/s 2024-09-15T22:24:59.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:24:58 smithi099 ceph-mon[34892]: pgmap v503: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 367 KiB/s wr, 27 op/s 2024-09-15T22:24:59.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:24:58 smithi005 ceph-mon[30944]: pgmap v503: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 367 KiB/s wr, 27 op/s 2024-09-15T22:25:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:00 smithi099 ceph-mon[34892]: pgmap v504: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 350 KiB/s wr, 27 op/s 2024-09-15T22:25:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:00 smithi129 ceph-mon[35314]: pgmap v504: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 350 KiB/s wr, 27 op/s 2024-09-15T22:25:01.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:00 smithi005 ceph-mon[30944]: pgmap v504: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 350 KiB/s wr, 27 op/s 2024-09-15T22:25:03.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:02 smithi099 ceph-mon[34892]: pgmap v505: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 350 KiB/s wr, 30 op/s 2024-09-15T22:25:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:02 smithi129 ceph-mon[35314]: pgmap v505: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 350 KiB/s wr, 30 op/s 2024-09-15T22:25:03.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:02 smithi005 ceph-mon[30944]: pgmap v505: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 350 KiB/s wr, 30 op/s 2024-09-15T22:25:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:04 smithi099 ceph-mon[34892]: pgmap v506: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 383 KiB/s wr, 32 op/s 2024-09-15T22:25:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:04 smithi129 ceph-mon[35314]: pgmap v506: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 383 KiB/s wr, 32 op/s 2024-09-15T22:25:05.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:04 smithi005 ceph-mon[30944]: pgmap v506: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 383 KiB/s wr, 32 op/s 2024-09-15T22:25:07.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:06 smithi099 ceph-mon[34892]: pgmap v507: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 290 KiB/s wr, 43 op/s 2024-09-15T22:25:07.341 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:06 smithi005 ceph-mon[30944]: pgmap v507: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 290 KiB/s wr, 43 op/s 2024-09-15T22:25:07.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:06 smithi129 ceph-mon[35314]: pgmap v507: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 290 KiB/s wr, 43 op/s 2024-09-15T22:25:09.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:08 smithi005 ceph-mon[30944]: pgmap v508: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 265 KiB/s wr, 39 op/s 2024-09-15T22:25:09.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:08 smithi005 ceph-mon[30944]: osdmap e108: 12 total, 12 up, 12 in 2024-09-15T22:25:09.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:08 smithi099 ceph-mon[34892]: pgmap v508: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 265 KiB/s wr, 39 op/s 2024-09-15T22:25:09.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:08 smithi099 ceph-mon[34892]: osdmap e108: 12 total, 12 up, 12 in 2024-09-15T22:25:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:08 smithi129 ceph-mon[35314]: pgmap v508: 129 pgs: 129 active+clean; 2.9 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 265 KiB/s wr, 39 op/s 2024-09-15T22:25:09.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:08 smithi129 ceph-mon[35314]: osdmap e108: 12 total, 12 up, 12 in 2024-09-15T22:25:11.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:10 smithi099 ceph-mon[34892]: pgmap v510: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 82 KiB/s wr, 29 op/s 2024-09-15T22:25:11.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:10 smithi129 ceph-mon[35314]: pgmap v510: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 82 KiB/s wr, 29 op/s 2024-09-15T22:25:11.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:10 smithi005 ceph-mon[30944]: pgmap v510: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.8 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 82 KiB/s wr, 29 op/s 2024-09-15T22:25:13.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:12 smithi099 ceph-mon[34892]: pgmap v511: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 82 KiB/s wr, 28 op/s 2024-09-15T22:25:13.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:12 smithi129 ceph-mon[35314]: pgmap v511: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 82 KiB/s wr, 28 op/s 2024-09-15T22:25:13.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:12 smithi005 ceph-mon[30944]: pgmap v511: 129 pgs: 2 active+clean+snaptrim, 2 active+clean+snaptrim_wait, 125 active+clean; 2.6 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 82 KiB/s wr, 28 op/s 2024-09-15T22:25:15.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:15 smithi099 ceph-mon[34892]: pgmap v512: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 24 op/s 2024-09-15T22:25:15.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:15 smithi129 ceph-mon[35314]: pgmap v512: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 24 op/s 2024-09-15T22:25:15.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:15 smithi005 ceph-mon[30944]: pgmap v512: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 40 KiB/s wr, 24 op/s 2024-09-15T22:25:17.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:17 smithi099 ceph-mon[34892]: pgmap v513: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-09-15T22:25:17.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:17 smithi129 ceph-mon[35314]: pgmap v513: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-09-15T22:25:17.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:17 smithi005 ceph-mon[30944]: pgmap v513: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-09-15T22:25:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:19 smithi099 ceph-mon[34892]: pgmap v514: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-09-15T22:25:19.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:19 smithi099 ceph-mon[34892]: osdmap e109: 12 total, 12 up, 12 in 2024-09-15T22:25:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:19 smithi129 ceph-mon[35314]: pgmap v514: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-09-15T22:25:19.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:19 smithi129 ceph-mon[35314]: osdmap e109: 12 total, 12 up, 12 in 2024-09-15T22:25:19.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:19 smithi005 ceph-mon[30944]: pgmap v514: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-09-15T22:25:19.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:19 smithi005 ceph-mon[30944]: osdmap e109: 12 total, 12 up, 12 in 2024-09-15T22:25:21.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:21 smithi099 ceph-mon[34892]: pgmap v516: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-09-15T22:25:21.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:21 smithi129 ceph-mon[35314]: pgmap v516: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-09-15T22:25:21.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:21 smithi005 ceph-mon[30944]: pgmap v516: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-09-15T22:25:21.454 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-15T22:25:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:22.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.262+0000 7f3401a6a640 1 -- 172.21.15.5:0/3498136458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc102570 msgr2=0x7f33fc1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:22.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.262+0000 7f3401a6a640 1 --2- 172.21.15.5:0/3498136458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc102570 0x7f33fc1029b0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009900 tx=0x7f33f002a480 comp rx=0 tx=0).stop 2024-09-15T22:25:22.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.262+0000 7f3401a6a640 1 -- 172.21.15.5:0/3498136458 shutdown_connections 2024-09-15T22:25:22.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.262+0000 7f3401a6a640 1 --2- 172.21.15.5:0/3498136458 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33fc102ef0 0x7f33fc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.262+0000 7f3401a6a640 1 --2- 172.21.15.5:0/3498136458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc102570 0x7f33fc1029b0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.262+0000 7f3401a6a640 1 --2- 172.21.15.5:0/3498136458 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33fc100e20 0x7f33fc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.262+0000 7f3401a6a640 1 -- 172.21.15.5:0/3498136458 >> 172.21.15.5:0/3498136458 conn(0x7f33fc0fc7d0 msgr2=0x7f33fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:22.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.262+0000 7f3401a6a640 1 -- 172.21.15.5:0/3498136458 shutdown_connections 2024-09-15T22:25:22.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.262+0000 7f3401a6a640 1 -- 172.21.15.5:0/3498136458 wait complete. 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.262+0000 7f3401a6a640 1 Processor -- start 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f3401a6a640 1 -- start start 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f3401a6a640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33fc100e20 0x7f33fc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f3401a6a640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33fc102570 0x7f33fc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f3401a6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc102ef0 0x7f33fc1a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f3401a6a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc110f80 con 0x7f33fc102ef0 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f3401a6a640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc110e00 con 0x7f33fc102570 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f3401a6a640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc111100 con 0x7f33fc100e20 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc102ef0 0x7f33fc1a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f33fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc102ef0 0x7f33fc1a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34932/0 (socket says 172.21.15.5:34932) 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f33fbfff640 1 -- 172.21.15.5:0/729279364 learned_addr learned my addr 172.21.15.5:0/729279364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f33faffd640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33fc102570 0x7f33fc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33fc100e20 0x7f33fc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f33fbfff640 1 -- 172.21.15.5:0/729279364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33fc100e20 msgr2=0x7f33fc199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f33fbfff640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33fc100e20 0x7f33fc199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f33fbfff640 1 -- 172.21.15.5:0/729279364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33fc102570 msgr2=0x7f33fc1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f33fbfff640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33fc102570 0x7f33fc1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.263+0000 7f33fbfff640 1 -- 172.21.15.5:0/729279364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33fc1a4850 con 0x7f33fc102ef0 2024-09-15T22:25:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.264+0000 7f33fbfff640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc102ef0 0x7f33fc1a4310 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f33f400c970 tx=0x7f33f400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.264+0000 7f33f8ff9640 1 -- 172.21.15.5:0/729279364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4007bb0 con 0x7f33fc102ef0 2024-09-15T22:25:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.264+0000 7f33f8ff9640 1 -- 172.21.15.5:0/729279364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f4007d50 con 0x7f33fc102ef0 2024-09-15T22:25:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.264+0000 7f3401a6a640 1 -- 172.21.15.5:0/729279364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33fc1a4a20 con 0x7f33fc102ef0 2024-09-15T22:25:22.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.264+0000 7f33f8ff9640 1 -- 172.21.15.5:0/729279364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f4018800 con 0x7f33fc102ef0 2024-09-15T22:25:22.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.264+0000 7f3401a6a640 1 -- 172.21.15.5:0/729279364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33fc1a4ee0 con 0x7f33fc102ef0 2024-09-15T22:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.267+0000 7f33f8ff9640 1 -- 172.21.15.5:0/729279364 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f33f400fa50 con 0x7f33fc102ef0 2024-09-15T22:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.267+0000 7f3403e43740 1 Processor -- start 2024-09-15T22:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.267+0000 7f3403e43740 1 -- start start 2024-09-15T22:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f3403e43740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067360 0x55aa19067740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f3403e43740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067c80 0x55aa19076550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f3403e43740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19076a90 0x55aa19078e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f3403e43740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aa18fcd2e0 con 0x55aa19067c80 2024-09-15T22:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f3403e43740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55aa18f74ce0 con 0x55aa19076a90 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f3403e43740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aa18fc92a0 con 0x55aa19067360 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33fb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067c80 0x55aa19076550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33fb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067c80 0x55aa19076550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34936/0 (socket says 172.21.15.5:34936) 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33fb7fe640 1 -- 172.21.15.5:0/546638837 learned_addr learned my addr 172.21.15.5:0/546638837 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33f8ff9640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f33e40a33b0 0x7f33e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33faffd640 1 --2- 172.21.15.5:0/546638837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067360 0x55aa19067740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33faffd640 1 --2- 172.21.15.5:0/546638837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19076a90 0x55aa19078e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33fb7fe640 1 -- 172.21.15.5:0/546638837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067360 msgr2=0x55aa19067740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/546638837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067360 0x55aa19067740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33fb7fe640 1 -- 172.21.15.5:0/546638837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19076a90 msgr2=0x55aa19078e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/546638837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19076a90 0x55aa19078e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33fb7fe640 1 -- 172.21.15.5:0/546638837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa18f59440 con 0x55aa19067c80 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33fbfff640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f33e40a33b0 0x7f33e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33faffd640 1 --2- 172.21.15.5:0/546638837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19076a90 0x55aa19078e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33f8ff9640 1 -- 172.21.15.5:0/729279364 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f33f40d0070 con 0x7f33fc102ef0 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.268+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/546638837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067c80 0x55aa19076550 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0039f0 tx=0x7f33ec003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb37de0d94d393e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f33e1ffb640 1 -- 172.21.15.5:0/546638837 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec004b20 con 0x55aa19067c80 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f33e1ffb640 1 -- 172.21.15.5:0/546638837 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33ec004cc0 con 0x55aa19067c80 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f33e1ffb640 1 -- 172.21.15.5:0/546638837 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ec0127d0 con 0x55aa19067c80 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f3403e43740 1 -- 172.21.15.5:0/546638837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067c80 msgr2=0x55aa19076550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f3403e43740 1 --2- 172.21.15.5:0/546638837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067c80 0x55aa19076550 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f33ec0039f0 tx=0x7f33ec003eb0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f33fbfff640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f33e40a33b0 0x7f33e40a5870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f33f40cd9c0 tx=0x7f33f40913e0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f3403e43740 1 -- 172.21.15.5:0/546638837 shutdown_connections 2024-09-15T22:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f3403e43740 1 --2- 172.21.15.5:0/546638837 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19076a90 0x55aa19078e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f3403e43740 1 --2- 172.21.15.5:0/546638837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067c80 0x55aa19076550 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f3403e43740 1 --2- 172.21.15.5:0/546638837 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067360 0x55aa19067740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f3403e43740 1 -- 172.21.15.5:0/546638837 >> 172.21.15.5:0/546638837 conn(0x55aa19063a30 msgr2=0x55aa19063e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f3403e43740 1 -- 172.21.15.5:0/546638837 shutdown_connections 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.269+0000 7f3403e43740 1 -- 172.21.15.5:0/546638837 wait complete. 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f3403e43740 1 -- start start 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f3403e43740 1 Processor -- start 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f3403e43740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19067360 0x55aa19076370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f3403e43740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067c80 0x55aa19073620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f3403e43740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19076a90 0x55aa19073b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f3403e43740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55aa19079660 con 0x55aa19076a90 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f3403e43740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55aa18f88050 con 0x55aa19067360 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f3403e43740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55aa190797e0 con 0x55aa19067c80 2024-09-15T22:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f3403e43740 10 client.0 _ll_register_callbacks cb 0x55aa18fc8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f33faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19076a90 0x55aa19073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f33faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19076a90 0x55aa19073b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34940/0 (socket says 172.21.15.5:34940) 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.270+0000 7f33faffd640 1 -- 172.21.15.5:0/1834139260 learned_addr learned my addr 172.21.15.5:0/1834139260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33faffd640 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19067360 0x55aa19076370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067c80 0x55aa19073620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33faffd640 1 -- 172.21.15.5:0/1834139260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067c80 msgr2=0x55aa19073620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33faffd640 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067c80 0x55aa19073620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33faffd640 1 -- 172.21.15.5:0/1834139260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19067360 msgr2=0x55aa19076370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33faffd640 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19067360 0x55aa19076370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33faffd640 1 -- 172.21.15.5:0/1834139260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa18d3e3c0 con 0x55aa19076a90 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33faffd640 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19076a90 0x55aa19073b60 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f33f002a9a0 tx=0x7f33f0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2925cef6bdd6a442 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33daffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33daffd640 1 -- 172.21.15.5:0/1834139260 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0035040 con 0x55aa19076a90 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33daffd640 1 -- 172.21.15.5:0/1834139260 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f33f0003b30 con 0x55aa19076a90 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f3403e43740 1 -- 172.21.15.5:0/1834139260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33ec004cc0 con 0x55aa19076a90 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f33daffd640 1 -- 172.21.15.5:0/1834139260 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33f0003d30 con 0x55aa19076a90 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f3403e43740 10 client.18159 fetch_fsmap learned FSMap version 25 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.271+0000 7f3403e43740 1 -- 172.21.15.5:0/1834139260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aa19084f80 con 0x55aa19076a90 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.272+0000 7f33daffd640 1 -- 172.21.15.5:0/1834139260 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33ec004cc0 con 0x55aa19076a90 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.272+0000 7f33daffd640 1 -- 172.21.15.5:0/1834139260 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f33f0049020 con 0x55aa19076a90 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.272+0000 7f3403e43740 10 client.18159 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.272+0000 7f3403e43740 10 client.18159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.272+0000 7f3403e43740 20 client.18159 populate_metadata read hostname 'smithi005' 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.272+0000 7f3403e43740 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55aa19086440 0x55aa19088830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.272+0000 7f3403e43740 4 client.18159 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:22.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.272+0000 7f3403e43740 1 -- 172.21.15.5:0/1834139260 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa18fda850 con 0x55aa19086440 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.272+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55aa19086440 0x55aa19088830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.272+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55aa19086440 0x55aa19088830 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8d3a4d09a8816ba server_cookie=7346e7ab1b0d36eb in_seq=0 out_seq=0 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.272+0000 7f33daffd640 10 client.18159 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f33daffd640 1 -- 172.21.15.5:0/1834139260 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa18fda850 con 0x55aa19086440 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f33daffd640 10 client.18159 handle_command_reply: tid=0 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f3403e43740 1 client.18159 shutdown 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f3403e43740 1 -- 172.21.15.5:0/1834139260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19076a90 msgr2=0x55aa19073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f3403e43740 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19076a90 0x55aa19073b60 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f33f002a9a0 tx=0x7f33f0009260 comp rx=0 tx=0).stop 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f3403e43740 1 -- 172.21.15.5:0/1834139260 shutdown_connections 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f3403e43740 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19076a90 0x55aa19073b60 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f3403e43740 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067c80 0x55aa19073620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f3403e43740 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55aa19086440 0x55aa19088830 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f3403e43740 1 --2- 172.21.15.5:0/1834139260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19067360 0x55aa19076370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f3403e43740 1 -- 172.21.15.5:0/1834139260 >> 172.21.15.5:0/1834139260 conn(0x55aa19063a30 msgr2=0x55aa19069790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.273+0000 7f33daffd640 0 client.18159 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:25:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.274+0000 7f3403e43740 1 -- 172.21.15.5:0/1834139260 shutdown_connections 2024-09-15T22:25:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.274+0000 7f3403e43740 1 -- 172.21.15.5:0/1834139260 wait complete. 2024-09-15T22:25:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.274+0000 7f3403e43740 20 client.18159 trim_cache size 0 max 16384 2024-09-15T22:25:22.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f3403e43740 1 Processor -- start 2024-09-15T22:25:22.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f3403e43740 1 -- start start 2024-09-15T22:25:22.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f3403e43740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa19082900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f3403e43740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19067c80 0x55aa19075590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f3403e43740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19076a90 0x55aa1908dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f3403e43740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1af780 con 0x55aa19067360 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f3403e43740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1ad050 con 0x55aa19067c80 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f3403e43740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1aaec0 con 0x55aa19076a90 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f33faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa19082900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f33faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa19082900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34944/0 (socket says 172.21.15.5:34944) 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f33faffd640 1 -- 172.21.15.5:0/506900319 learned_addr learned my addr 172.21.15.5:0/506900319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f33faffd640 1 -- 172.21.15.5:0/506900319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19076a90 msgr2=0x55aa1908dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f33faffd640 1 --2- 172.21.15.5:0/506900319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19076a90 0x55aa1908dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f33faffd640 1 -- 172.21.15.5:0/506900319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19067c80 msgr2=0x55aa19075590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/506900319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19067c80 0x55aa19075590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f33faffd640 1 --2- 172.21.15.5:0/506900319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19067c80 0x55aa19075590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.291+0000 7f33faffd640 1 -- 172.21.15.5:0/506900319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa190766e0 con 0x55aa19067360 2024-09-15T22:25:22.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f33faffd640 1 --2- 172.21.15.5:0/506900319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa19082900 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005a90 tx=0x7f33f0004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c97edb7e2923af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:22.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f33e17fa640 1 -- 172.21.15.5:0/506900319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0035070 con 0x55aa19067360 2024-09-15T22:25:22.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f33e17fa640 1 -- 172.21.15.5:0/506900319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f0005780 con 0x55aa19067360 2024-09-15T22:25:22.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f33e17fa640 1 -- 172.21.15.5:0/506900319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f003db00 con 0x55aa19067360 2024-09-15T22:25:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f3403e43740 1 -- 172.21.15.5:0/506900319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 msgr2=0x55aa19082900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f3403e43740 1 --2- 172.21.15.5:0/506900319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa19082900 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f33f0005a90 tx=0x7f33f0004560 comp rx=0 tx=0).stop 2024-09-15T22:25:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f3403e43740 1 -- 172.21.15.5:0/506900319 shutdown_connections 2024-09-15T22:25:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f3403e43740 1 --2- 172.21.15.5:0/506900319 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19076a90 0x55aa1908dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f3403e43740 1 --2- 172.21.15.5:0/506900319 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19067c80 0x55aa19075590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f3403e43740 1 --2- 172.21.15.5:0/506900319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa19082900 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f3403e43740 1 -- 172.21.15.5:0/506900319 >> 172.21.15.5:0/506900319 conn(0x55aa19063a30 msgr2=0x55aa19074e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f3403e43740 1 -- 172.21.15.5:0/506900319 shutdown_connections 2024-09-15T22:25:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.292+0000 7f3403e43740 1 -- 172.21.15.5:0/506900319 wait complete. 2024-09-15T22:25:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f3403e43740 1 -- start start 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f3403e43740 1 Processor -- start 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f3403e43740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa1907a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f3403e43740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067c80 0x55aa1907adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f3403e43740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19076a90 0x55aa1907b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f3403e43740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1ab510 con 0x55aa19067360 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f3403e43740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1ac980 con 0x55aa19076a90 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f3403e43740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f33fc1abbe0 con 0x55aa19067c80 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f3403e43740 10 client.0 _ll_register_callbacks cb 0x7f33f0009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f33faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa1907a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f33faffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa1907a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34956/0 (socket says 172.21.15.5:34956) 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.293+0000 7f33faffd640 1 -- 172.21.15.5:0/859796380 learned_addr learned my addr 172.21.15.5:0/859796380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067c80 0x55aa1907adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33faffd640 1 -- 172.21.15.5:0/859796380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067c80 msgr2=0x55aa1907adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33faffd640 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067c80 0x55aa1907adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33faffd640 1 -- 172.21.15.5:0/859796380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19076a90 msgr2=0x55aa1907b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33faffd640 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19076a90 0x55aa1907b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33faffd640 1 -- 172.21.15.5:0/859796380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa190f0070 con 0x55aa19067360 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067c80 0x55aa1907adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33faffd640 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa1907a870 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f33f003d510 tx=0x7f33f0004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5128e5c365febb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33daffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33daffd640 1 -- 172.21.15.5:0/859796380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f0035040 con 0x55aa19067360 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33daffd640 1 -- 172.21.15.5:0/859796380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f33f0046070 con 0x55aa19067360 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f3403e43740 1 -- 172.21.15.5:0/859796380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33f0005780 con 0x55aa19067360 2024-09-15T22:25:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f33daffd640 1 -- 172.21.15.5:0/859796380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33f0059020 con 0x55aa19067360 2024-09-15T22:25:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f3403e43740 10 client.18165 fetch_fsmap learned FSMap version 25 2024-09-15T22:25:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.294+0000 7f3403e43740 1 -- 172.21.15.5:0/859796380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55aa190f1360 con 0x55aa19067360 2024-09-15T22:25:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.295+0000 7f33daffd640 1 -- 172.21.15.5:0/859796380 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33f0005780 con 0x55aa19067360 2024-09-15T22:25:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.295+0000 7f33daffd640 1 -- 172.21.15.5:0/859796380 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f33f0049040 con 0x55aa19067360 2024-09-15T22:25:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.295+0000 7f3403e43740 10 client.18165 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:25:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.295+0000 7f3403e43740 10 client.18165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.295+0000 7f3403e43740 20 client.18165 populate_metadata read hostname 'smithi005' 2024-09-15T22:25:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.295+0000 7f3403e43740 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55aa190f1880 0x55aa190f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.295+0000 7f3403e43740 4 client.18165 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-15T22:25:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.295+0000 7f3403e43740 1 -- 172.21.15.5:0/859796380 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55aa18fd4d60 con 0x55aa190f1880 2024-09-15T22:25:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.295+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55aa190f1880 0x55aa190f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.295+0000 7f33fb7fe640 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55aa190f1880 0x55aa190f3c70 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d30d55fdd4df3e66 server_cookie=dee40cd7208171ea in_seq=0 out_seq=0 2024-09-15T22:25:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.295+0000 7f33daffd640 10 client.18165 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:25:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f33daffd640 1 -- 172.21.15.5:0/859796380 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55aa18fd4d60 con 0x55aa190f1880 2024-09-15T22:25:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f33daffd640 10 client.18165 handle_command_reply: tid=0 2024-09-15T22:25:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 1 client.18165 shutdown 2024-09-15T22:25:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 1 -- 172.21.15.5:0/859796380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 msgr2=0x55aa1907a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa1907a870 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f33f003d510 tx=0x7f33f0004370 comp rx=0 tx=0).stop 2024-09-15T22:25:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 1 -- 172.21.15.5:0/859796380 shutdown_connections 2024-09-15T22:25:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55aa190f1880 0x55aa190f3c70 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55aa19076a90 0x55aa1907b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55aa19067c80 0x55aa1907adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 1 --2- 172.21.15.5:0/859796380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa19067360 0x55aa1907a870 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 1 -- 172.21.15.5:0/859796380 >> 172.21.15.5:0/859796380 conn(0x55aa19063a30 msgr2=0x55aa19062350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f33daffd640 0 client.18165 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:25:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 1 -- 172.21.15.5:0/859796380 shutdown_connections 2024-09-15T22:25:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 1 -- 172.21.15.5:0/859796380 wait complete. 2024-09-15T22:25:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.296+0000 7f3403e43740 20 client.18165 trim_cache size 0 max 16384 2024-09-15T22:25:22.307 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:25:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-15T22:25:22.307 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-15T22:25:22.307 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:25:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 -- 172.21.15.5:0/729279364 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f33e40a33b0 msgr2=0x7f33e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f33e40a33b0 0x7f33e40a5870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f33f40cd9c0 tx=0x7f33f40913e0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 -- 172.21.15.5:0/729279364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc102ef0 msgr2=0x7f33fc1a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc102ef0 0x7f33fc1a4310 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f33f400c970 tx=0x7f33f400ce30 comp rx=0 tx=0).stop 2024-09-15T22:25:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 -- 172.21.15.5:0/729279364 shutdown_connections 2024-09-15T22:25:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f33e40a33b0 0x7f33e40a5870 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33fc102ef0 0x7f33fc1a4310 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f33fc102570 0x7f33fc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 --2- 172.21.15.5:0/729279364 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f33fc100e20 0x7f33fc199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 -- 172.21.15.5:0/729279364 >> 172.21.15.5:0/729279364 conn(0x7f33fc0fc7d0 msgr2=0x7f33fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 -- 172.21.15.5:0/729279364 shutdown_connections 2024-09-15T22:25:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:22.297+0000 7f3401a6a640 1 -- 172.21.15.5:0/729279364 wait complete. 2024-09-15T22:25:23.114 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-15T22:25:23.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:25:23.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:23 smithi099 ceph-mon[34892]: pgmap v517: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-09-15T22:25:23.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:23 smithi129 ceph-mon[35314]: pgmap v517: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-09-15T22:25:23.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:23 smithi005 ceph-mon[30944]: pgmap v517: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 6 op/s 2024-09-15T22:25:23.373 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.880+0000 7ff7f45a8640 1 -- 172.21.15.5:0/3350438897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec102d30 msgr2=0x7ff7ec103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.880+0000 7ff7f45a8640 1 --2- 172.21.15.5:0/3350438897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec102d30 0x7ff7ec103110 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009930 tx=0x7ff7e802a420 comp rx=0 tx=0).stop 2024-09-15T22:25:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.880+0000 7ff7f45a8640 1 -- 172.21.15.5:0/3350438897 shutdown_connections 2024-09-15T22:25:23.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.880+0000 7ff7f45a8640 1 --2- 172.21.15.5:0/3350438897 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7ec1082a0 0x7ff7ec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.880+0000 7ff7f45a8640 1 --2- 172.21.15.5:0/3350438897 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec1036e0 0x7ff7ec107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.880+0000 7ff7f45a8640 1 --2- 172.21.15.5:0/3350438897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec102d30 0x7ff7ec103110 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.880+0000 7ff7f45a8640 1 -- 172.21.15.5:0/3350438897 >> 172.21.15.5:0/3350438897 conn(0x7ff7ec0fc7d0 msgr2=0x7ff7ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.880+0000 7ff7f45a8640 1 -- 172.21.15.5:0/3350438897 shutdown_connections 2024-09-15T22:25:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.881+0000 7ff7f45a8640 1 -- 172.21.15.5:0/3350438897 wait complete. 2024-09-15T22:25:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.881+0000 7ff7f45a8640 1 Processor -- start 2024-09-15T22:25:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.881+0000 7ff7f45a8640 1 -- start start 2024-09-15T22:25:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f45a8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec102d30 0x7ff7ec1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f45a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec1036e0 0x7ff7ec198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f45a8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7ec1082a0 0x7ff7ec19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:23.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f45a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec1120a0 con 0x7ff7ec1036e0 2024-09-15T22:25:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f45a8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec111f20 con 0x7ff7ec102d30 2024-09-15T22:25:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f45a8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec112220 con 0x7ff7ec1082a0 2024-09-15T22:25:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f1b43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec1036e0 0x7ff7ec198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f2344640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec102d30 0x7ff7ec1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f1b43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec1036e0 0x7ff7ec198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53398/0 (socket says 172.21.15.5:53398) 2024-09-15T22:25:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f1b43640 1 -- 172.21.15.5:0/885892348 learned_addr learned my addr 172.21.15.5:0/885892348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f2b45640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7ec1082a0 0x7ff7ec19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:23.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f2b45640 1 -- 172.21.15.5:0/885892348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec102d30 msgr2=0x7ff7ec1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f2b45640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec102d30 0x7ff7ec1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f2b45640 1 -- 172.21.15.5:0/885892348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec1036e0 msgr2=0x7ff7ec198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f2b45640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec1036e0 0x7ff7ec198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f2b45640 1 -- 172.21.15.5:0/885892348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ec19d570 con 0x7ff7ec1082a0 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f2344640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec102d30 0x7ff7ec1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f1b43640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec1036e0 0x7ff7ec198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.882+0000 7ff7f2b45640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7ec1082a0 0x7ff7ec19ce90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009fd0 tx=0x7ff7e400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.883+0000 7ff7e37fe640 1 -- 172.21.15.5:0/885892348 <== mon.1 v2:172.21.15.129:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400ce20 con 0x7ff7ec1082a0 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.883+0000 7ff7e37fe640 1 -- 172.21.15.5:0/885892348 <== mon.1 v2:172.21.15.129:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7e4005020 con 0x7ff7ec1082a0 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.883+0000 7ff7f45a8640 1 -- 172.21.15.5:0/885892348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ec19d860 con 0x7ff7ec1082a0 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.883+0000 7ff7e37fe640 1 -- 172.21.15.5:0/885892348 <== mon.1 v2:172.21.15.129:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4010720 con 0x7ff7ec1082a0 2024-09-15T22:25:23.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.883+0000 7ff7f45a8640 1 -- 172.21.15.5:0/885892348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ec1a5140 con 0x7ff7ec1082a0 2024-09-15T22:25:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.884+0000 7ff7f45a8640 1 -- 172.21.15.5:0/885892348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ec06a850 con 0x7ff7ec1082a0 2024-09-15T22:25:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.886+0000 7ff7e37fe640 1 -- 172.21.15.5:0/885892348 <== mon.1 v2:172.21.15.129:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7ff7e4005540 con 0x7ff7ec1082a0 2024-09-15T22:25:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.887+0000 7ff7e37fe640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff7d80a33b0 0x7ff7d80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.887+0000 7ff7f2344640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff7d80a33b0 0x7ff7d80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.887+0000 7ff7e37fe640 1 -- 172.21.15.5:0/885892348 <== mon.1 v2:172.21.15.129:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7ff7e40d06b0 con 0x7ff7ec1082a0 2024-09-15T22:25:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.887+0000 7ff7f2344640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff7d80a33b0 0x7ff7d80a5870 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009900 tx=0x7ff7e80058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:23.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:23.891+0000 7ff7e37fe640 1 -- 172.21.15.5:0/885892348 <== mon.1 v2:172.21.15.129:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7e409cc90 con 0x7ff7ec1082a0 2024-09-15T22:25:24.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:24 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:25:24.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:24 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:25:24.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:24 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:25:24.108 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:24 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:25:24.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.117+0000 7ff7f45a8640 1 -- 172.21.15.5:0/885892348 --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7ec10a2b0 con 0x7ff7ec1082a0 2024-09-15T22:25:24.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.118+0000 7ff7e37fe640 1 -- 172.21.15.5:0/885892348 <== mon.1 v2:172.21.15.129:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7ff7e409c590 con 0x7ff7ec1082a0 2024-09-15T22:25:24.121 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:25:24.122 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:25:24.122 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.120+0000 7ff7f45a8640 1 -- 172.21.15.5:0/885892348 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff7d80a33b0 msgr2=0x7ff7d80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.120+0000 7ff7f45a8640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff7d80a33b0 0x7ff7d80a5870 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009900 tx=0x7ff7e80058d0 comp rx=0 tx=0).stop 2024-09-15T22:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.120+0000 7ff7f45a8640 1 -- 172.21.15.5:0/885892348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7ec1082a0 msgr2=0x7ff7ec19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.121+0000 7ff7f45a8640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7ec1082a0 0x7ff7ec19ce90 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009fd0 tx=0x7ff7e400ef10 comp rx=0 tx=0).stop 2024-09-15T22:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.121+0000 7ff7f45a8640 1 -- 172.21.15.5:0/885892348 shutdown_connections 2024-09-15T22:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.121+0000 7ff7f45a8640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7ff7d80a33b0 0x7ff7d80a5870 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:24.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.121+0000 7ff7f45a8640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7ff7ec1082a0 0x7ff7ec19ce90 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:24.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.121+0000 7ff7f45a8640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7ec1036e0 0x7ff7ec198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:24.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.121+0000 7ff7f45a8640 1 --2- 172.21.15.5:0/885892348 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7ff7ec102d30 0x7ff7ec1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:24.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.121+0000 7ff7f45a8640 1 -- 172.21.15.5:0/885892348 >> 172.21.15.5:0/885892348 conn(0x7ff7ec0fc7d0 msgr2=0x7ff7ec105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:24.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.121+0000 7ff7f45a8640 1 -- 172.21.15.5:0/885892348 shutdown_connections 2024-09-15T22:25:24.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:24.121+0000 7ff7f45a8640 1 -- 172.21.15.5:0/885892348 wait complete. 2024-09-15T22:25:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:24 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:25:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:24 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:25:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:24 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:25:24.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:24 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:25:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:24 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-15T22:25:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:24 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-15T22:25:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:24 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-15T22:25:24.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:24 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' 2024-09-15T22:25:24.913 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-15T22:25:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:25.298 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:25 smithi005 ceph-mon[30944]: pgmap v518: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 6 op/s 2024-09-15T22:25:25.298 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:25 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/885892348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:25.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:25 smithi099 ceph-mon[34892]: pgmap v518: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 6 op/s 2024-09-15T22:25:25.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:25 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/885892348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:25.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:25 smithi129 ceph-mon[35314]: pgmap v518: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 6 op/s 2024-09-15T22:25:25.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:25 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/885892348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.690+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/451627575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c100a80 msgr2=0x7fa79c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.690+0000 7fa7a0c5c640 1 --2- 172.21.15.5:0/451627575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c100a80 0x7fa79c100ee0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa790009930 tx=0x7fa79002a420 comp rx=0 tx=0).stop 2024-09-15T22:25:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.690+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/451627575 shutdown_connections 2024-09-15T22:25:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.690+0000 7fa7a0c5c640 1 --2- 172.21.15.5:0/451627575 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c101420 0x7fa79c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.690+0000 7fa7a0c5c640 1 --2- 172.21.15.5:0/451627575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c100a80 0x7fa79c100ee0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.690+0000 7fa7a0c5c640 1 --2- 172.21.15.5:0/451627575 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c106a80 0x7fa79c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.690+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/451627575 >> 172.21.15.5:0/451627575 conn(0x7fa79c0fc7a0 msgr2=0x7fa79c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.690+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/451627575 shutdown_connections 2024-09-15T22:25:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.690+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/451627575 wait complete. 2024-09-15T22:25:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa7a0c5c640 1 Processor -- start 2024-09-15T22:25:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa7a0c5c640 1 -- start start 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa7a0c5c640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c100a80 0x7fa79c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa7a0c5c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c101420 0x7fa79c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa7a0c5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c106a80 0x7fa79c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa7a0c5c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c110e80 con 0x7fa79c106a80 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa7a0c5c640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c110d00 con 0x7fa79c101420 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa7a0c5c640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c111000 con 0x7fa79c100a80 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa79ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c106a80 0x7fa79c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa79ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c106a80 0x7fa79c19cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53422/0 (socket says 172.21.15.5:53422) 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa79ad9d640 1 -- 172.21.15.5:0/3788125323 learned_addr learned my addr 172.21.15.5:0/3788125323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.691+0000 7fa799d9b640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c101420 0x7fa79c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa79a59c640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c100a80 0x7fa79c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa79ad9d640 1 -- 172.21.15.5:0/3788125323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c100a80 msgr2=0x7fa79c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa79ad9d640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c100a80 0x7fa79c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa79ad9d640 1 -- 172.21.15.5:0/3788125323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c101420 msgr2=0x7fa79c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa79ad9d640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c101420 0x7fa79c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa79ad9d640 1 -- 172.21.15.5:0/3788125323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa79c19d400 con 0x7fa79c106a80 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa799d9b640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c101420 0x7fa79c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa79a59c640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c100a80 0x7fa79c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa79ad9d640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c106a80 0x7fa79c19cd20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa79400eaa0 tx=0x7fa79400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa78b7fe640 1 -- 172.21.15.5:0/3788125323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa79400cdb0 con 0x7fa79c106a80 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa78b7fe640 1 -- 172.21.15.5:0/3788125323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa794005020 con 0x7fa79c106a80 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa78b7fe640 1 -- 172.21.15.5:0/3788125323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7940116b0 con 0x7fa79c106a80 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/3788125323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa79c19d6f0 con 0x7fa79c106a80 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.692+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/3788125323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa79c1a4fd0 con 0x7fa79c106a80 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.695+0000 7fa78b7fe640 1 -- 172.21.15.5:0/3788125323 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa794005560 con 0x7fa79c106a80 2024-09-15T22:25:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.695+0000 7fa7a3035740 1 Processor -- start 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.695+0000 7fa7a3035740 1 -- start start 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.695+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff808b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.695+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff808b5d0 0x556ff808ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.695+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff8096370 0x556ff8098760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.695+0000 7fa7a3035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ff7ff12f0 con 0x556ff808cb40 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.695+0000 7fa7a3035740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556ff7f98cf0 con 0x556ff808b5d0 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.695+0000 7fa7a3035740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ff7fed2b0 con 0x556ff8096370 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff808b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa78b7fe640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7840a33b0 0x7fa7840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff808b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53424/0 (socket says 172.21.15.5:53424) 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa799d9b640 1 -- 172.21.15.5:0/880806064 learned_addr learned my addr 172.21.15.5:0/880806064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa79ad9d640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7840a33b0 0x7fa7840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa79a59c640 1 --2- 172.21.15.5:0/880806064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff808b5d0 0x556ff808ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa799d9b640 1 --2- 172.21.15.5:0/880806064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff8096370 0x556ff8098760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa78b7fe640 1 -- 172.21.15.5:0/3788125323 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7fa7940d03a0 con 0x7fa79c106a80 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa799d9b640 1 -- 172.21.15.5:0/880806064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff8096370 msgr2=0x556ff8098760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa799d9b640 1 --2- 172.21.15.5:0/880806064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff8096370 0x556ff8098760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa799d9b640 1 -- 172.21.15.5:0/880806064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff808b5d0 msgr2=0x556ff808ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa799d9b640 1 --2- 172.21.15.5:0/880806064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff808b5d0 0x556ff808ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa799d9b640 1 -- 172.21.15.5:0/880806064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff7f7d3a0 con 0x556ff808cb40 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa799d9b640 1 --2- 172.21.15.5:0/880806064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff808b090 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa790000c00 tx=0x7fa790005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b02e7bc0b6df6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa79ad9d640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7840a33b0 0x7fa7840a5870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa794009270 tx=0x7fa7940bc040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa788ff9640 1 -- 172.21.15.5:0/880806064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790035040 con 0x556ff808cb40 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa788ff9640 1 -- 172.21.15.5:0/880806064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa790049030 con 0x556ff808cb40 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.696+0000 7fa788ff9640 1 -- 172.21.15.5:0/880806064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790004bc0 con 0x556ff808cb40 2024-09-15T22:25:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.697+0000 7fa7a3035740 1 -- 172.21.15.5:0/880806064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 msgr2=0x556ff808b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.697+0000 7fa7a3035740 1 --2- 172.21.15.5:0/880806064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff808b090 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fa790000c00 tx=0x7fa790005fd0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.697+0000 7fa7a3035740 1 -- 172.21.15.5:0/880806064 shutdown_connections 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.697+0000 7fa7a3035740 1 --2- 172.21.15.5:0/880806064 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff8096370 0x556ff8098760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.697+0000 7fa7a3035740 1 --2- 172.21.15.5:0/880806064 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff808b5d0 0x556ff808ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.697+0000 7fa7a3035740 1 --2- 172.21.15.5:0/880806064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff808b090 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.697+0000 7fa7a3035740 1 -- 172.21.15.5:0/880806064 >> 172.21.15.5:0/880806064 conn(0x556ff8054310 msgr2=0x556ff8087c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.697+0000 7fa7a3035740 1 -- 172.21.15.5:0/880806064 shutdown_connections 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.697+0000 7fa7a3035740 1 -- 172.21.15.5:0/880806064 wait complete. 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa7a3035740 1 -- start start 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa7a3035740 1 Processor -- start 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808b5d0 0x556ff8095cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff8092f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff8096370 0x556ff80934d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa7a3035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ff8099010 con 0x556ff808cb40 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa7a3035740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x556ff7fac060 con 0x556ff8096370 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa7a3035740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x556ff8099190 con 0x556ff808b5d0 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa7a3035740 10 client.0 _ll_register_callbacks cb 0x556ff7cff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa79a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff8092f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa79a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff8092f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53440/0 (socket says 172.21.15.5:53440) 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa79a59c640 1 -- 172.21.15.5:0/2724831220 learned_addr learned my addr 172.21.15.5:0/2724831220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa799d9b640 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff8096370 0x556ff80934d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa799d9b640 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808b5d0 0x556ff8095cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa79a59c640 1 -- 172.21.15.5:0/2724831220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808b5d0 msgr2=0x556ff8095cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa79a59c640 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808b5d0 0x556ff8095cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa79a59c640 1 -- 172.21.15.5:0/2724831220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff8096370 msgr2=0x556ff80934d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa79a59c640 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff8096370 0x556ff80934d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa79a59c640 1 -- 172.21.15.5:0/2724831220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff7d623c0 con 0x556ff808cb40 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.698+0000 7fa799d9b640 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808b5d0 0x556ff8095cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa79a59c640 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff8092f70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa78c011830 tx=0x7fa78c011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7f17449ba88aff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa781ffb640 1 -- 172.21.15.5:0/2724831220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00fd50 con 0x556ff808cb40 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa781ffb640 1 -- 172.21.15.5:0/2724831220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7fa78c004020 con 0x556ff808cb40 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa7a3035740 1 -- 172.21.15.5:0/2724831220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa790049030 con 0x556ff808cb40 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa781ffb640 1 -- 172.21.15.5:0/2724831220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa78c024050 con 0x556ff808cb40 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa7a3035740 10 client.18183 fetch_fsmap learned FSMap version 25 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa7a3035740 1 -- 172.21.15.5:0/2724831220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556ff80a4ad0 con 0x556ff808cb40 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa781ffb640 1 -- 172.21.15.5:0/2724831220 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa790049030 con 0x556ff808cb40 2024-09-15T22:25:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa781ffb640 1 -- 172.21.15.5:0/2724831220 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa78c018030 con 0x556ff808cb40 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa7a3035740 10 client.18183 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa7a3035740 10 client.18183 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa7a3035740 20 client.18183 populate_metadata read hostname 'smithi005' 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.699+0000 7fa7a3035740 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556ff80a6030 0x556ff80a8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.700+0000 7fa7a3035740 4 client.18183 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.700+0000 7fa7a3035740 1 -- 172.21.15.5:0/2724831220 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ff7ffe860 con 0x556ff80a6030 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.700+0000 7fa799d9b640 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556ff80a6030 0x556ff80a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.700+0000 7fa781ffb640 10 client.18183 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.700+0000 7fa799d9b640 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556ff80a6030 0x556ff80a8450 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33037ee888f46055 server_cookie=c8c97b09a436cbb1 in_seq=0 out_seq=0 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.701+0000 7fa781ffb640 1 -- 172.21.15.5:0/2724831220 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ff7ffe860 con 0x556ff80a6030 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.701+0000 7fa781ffb640 10 client.18183 handle_command_reply: tid=0 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.701+0000 7fa7a3035740 1 client.18183 shutdown 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa7a3035740 1 -- 172.21.15.5:0/2724831220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 msgr2=0x556ff8092f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa7a3035740 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff8092f70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fa78c011830 tx=0x7fa78c011cf0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa7a3035740 1 -- 172.21.15.5:0/2724831220 shutdown_connections 2024-09-15T22:25:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa7a3035740 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556ff80a6030 0x556ff80a8450 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa7a3035740 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff8096370 0x556ff80934d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa7a3035740 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808cb40 0x556ff8092f70 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa781ffb640 0 client.18183 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:25:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa7a3035740 1 --2- 172.21.15.5:0/2724831220 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808b5d0 0x556ff8095cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa7a3035740 1 -- 172.21.15.5:0/2724831220 >> 172.21.15.5:0/2724831220 conn(0x556ff8054310 msgr2=0x556ff808d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa7a3035740 1 -- 172.21.15.5:0/2724831220 shutdown_connections 2024-09-15T22:25:25.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa7a3035740 1 -- 172.21.15.5:0/2724831220 wait complete. 2024-09-15T22:25:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.702+0000 7fa7a3035740 20 client.18183 trim_cache size 0 max 16384 2024-09-15T22:25:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa7a3035740 1 Processor -- start 2024-09-15T22:25:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa7a3035740 1 -- start start 2024-09-15T22:25:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808b5d0 0x556ff8092f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808cb40 0x556ff80a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff8096370 0x556ff80adce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa7a3035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1af6f0 con 0x556ff808b5d0 2024-09-15T22:25:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa7a3035740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1acfc0 con 0x556ff8096370 2024-09-15T22:25:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa7a3035740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1aae30 con 0x556ff808cb40 2024-09-15T22:25:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808b5d0 0x556ff8092f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808b5d0 0x556ff8092f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53442/0 (socket says 172.21.15.5:53442) 2024-09-15T22:25:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa799d9b640 1 -- 172.21.15.5:0/1392362903 learned_addr learned my addr 172.21.15.5:0/1392362903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa799d9b640 1 -- 172.21.15.5:0/1392362903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808cb40 msgr2=0x556ff80a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa799d9b640 1 --2- 172.21.15.5:0/1392362903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808cb40 0x556ff80a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa799d9b640 1 -- 172.21.15.5:0/1392362903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff8096370 msgr2=0x556ff80adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa799d9b640 1 --2- 172.21.15.5:0/1392362903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff8096370 0x556ff80adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa799d9b640 1 -- 172.21.15.5:0/1392362903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ff80a8ab0 con 0x556ff808b5d0 2024-09-15T22:25:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.715+0000 7fa799d9b640 1 --2- 172.21.15.5:0/1392362903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808b5d0 0x556ff8092f40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa790009930 tx=0x7fa790031890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63727c87d4d303eb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa783fff640 1 -- 172.21.15.5:0/1392362903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790035040 con 0x556ff808b5d0 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa783fff640 1 -- 172.21.15.5:0/1392362903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa790043a50 con 0x556ff808b5d0 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa783fff640 1 -- 172.21.15.5:0/1392362903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7900586c0 con 0x556ff808b5d0 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa7a3035740 1 -- 172.21.15.5:0/1392362903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808b5d0 msgr2=0x556ff8092f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa7a3035740 1 --2- 172.21.15.5:0/1392362903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808b5d0 0x556ff8092f40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa790009930 tx=0x7fa790031890 comp rx=0 tx=0).stop 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa7a3035740 1 -- 172.21.15.5:0/1392362903 shutdown_connections 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa7a3035740 1 --2- 172.21.15.5:0/1392362903 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff8096370 0x556ff80adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa7a3035740 1 --2- 172.21.15.5:0/1392362903 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808cb40 0x556ff80a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa7a3035740 1 --2- 172.21.15.5:0/1392362903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff808b5d0 0x556ff8092f40 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa7a3035740 1 -- 172.21.15.5:0/1392362903 >> 172.21.15.5:0/1392362903 conn(0x556ff8054310 msgr2=0x556ff8092830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa7a3035740 1 -- 172.21.15.5:0/1392362903 shutdown_connections 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.716+0000 7fa7a3035740 1 -- 172.21.15.5:0/1392362903 wait complete. 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa7a3035740 1 -- start start 2024-09-15T22:25:25.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa7a3035740 1 Processor -- start 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808b5d0 0x556ff809ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff808cb40 0x556ff809b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa7a3035740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff8096370 0x556ff809e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa7a3035740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1ab480 con 0x556ff8096370 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa7a3035740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1ac8f0 con 0x556ff808cb40 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa7a3035740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa79c1abb50 con 0x556ff808b5d0 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa7a3035740 10 client.0 _ll_register_callbacks cb 0x7fa78c011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff8096370 0x556ff809e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa799d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff8096370 0x556ff809e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53448/0 (socket says 172.21.15.5:53448) 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa79a59c640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff808cb40 0x556ff809b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa799d9b640 1 -- 172.21.15.5:0/334363615 learned_addr learned my addr 172.21.15.5:0/334363615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.717+0000 7fa799d9b640 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808b5d0 0x556ff809ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa799d9b640 1 -- 172.21.15.5:0/334363615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808b5d0 msgr2=0x556ff809ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa799d9b640 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808b5d0 0x556ff809ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa799d9b640 1 -- 172.21.15.5:0/334363615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff808cb40 msgr2=0x556ff809b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa799d9b640 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff808cb40 0x556ff809b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa799d9b640 1 -- 172.21.15.5:0/334363615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ff808e120 con 0x556ff8096370 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa79a59c640 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff808cb40 0x556ff809b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa799d9b640 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff8096370 0x556ff809e270 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa790009850 tx=0x7fa790058360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16669db2fc9740e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa781ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa781ffb640 1 -- 172.21.15.5:0/334363615 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa790035070 con 0x556ff8096370 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa781ffb640 1 -- 172.21.15.5:0/334363615 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7fa790043570 con 0x556ff8096370 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa7a3035740 1 -- 172.21.15.5:0/334363615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa790043a50 con 0x556ff8096370 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa781ffb640 1 -- 172.21.15.5:0/334363615 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa790043770 con 0x556ff8096370 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa7a3035740 10 client.18189 fetch_fsmap learned FSMap version 25 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa7a3035740 1 -- 172.21.15.5:0/334363615 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556ff8088960 con 0x556ff8096370 2024-09-15T22:25:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa781ffb640 1 -- 172.21.15.5:0/334363615 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa790043a50 con 0x556ff8096370 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa781ffb640 1 -- 172.21.15.5:0/334363615 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa7900039b0 con 0x556ff8096370 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa7a3035740 10 client.18189 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa7a3035740 10 client.18189 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa7a3035740 20 client.18189 populate_metadata read hostname 'smithi005' 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.718+0000 7fa7a3035740 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556ff8111700 0x556ff8113af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.719+0000 7fa7a3035740 4 client.18189 mds_command: new command op to 24466 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.719+0000 7fa7a3035740 1 -- 172.21.15.5:0/334363615 --> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ff7ff8d70 con 0x556ff8111700 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.719+0000 7fa79a59c640 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556ff8111700 0x556ff8113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.719+0000 7fa79a59c640 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556ff8111700 0x556ff8113af0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f1d13a61f77ac43 server_cookie=108bc8c6d0263126 in_seq=0 out_seq=0 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.719+0000 7fa781ffb640 10 client.18189 ms_handle_connect on v2:172.21.15.129:6838/2032213198 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.719+0000 7fa781ffb640 1 -- 172.21.15.5:0/334363615 <== mds.2 v2:172.21.15.129:6838/2032213198 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ff7ff8d70 con 0x556ff8111700 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.719+0000 7fa781ffb640 10 client.18189 handle_command_reply: tid=0 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.719+0000 7fa7a3035740 1 client.18189 shutdown 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a3035740 1 -- 172.21.15.5:0/334363615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff8096370 msgr2=0x556ff809e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a3035740 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff8096370 0x556ff809e270 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa790009850 tx=0x7fa790058360 comp rx=0 tx=0).stop 2024-09-15T22:25:25.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a3035740 1 -- 172.21.15.5:0/334363615 shutdown_connections 2024-09-15T22:25:25.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a3035740 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.129:6838/2032213198,v1:172.21.15.129:6839/2032213198] conn(0x556ff8111700 0x556ff8113af0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.759 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:25:25.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a3035740 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ff8096370 0x556ff809e270 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a3035740 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x556ff808cb40 0x556ff809b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa781ffb640 0 client.18189 ms_handle_reset on v2:172.21.15.129:6838/2032213198 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a3035740 1 --2- 172.21.15.5:0/334363615 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x556ff808b5d0 0x556ff809ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a3035740 1 -- 172.21.15.5:0/334363615 >> 172.21.15.5:0/334363615 conn(0x556ff8054310 msgr2=0x556ff808d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a3035740 1 -- 172.21.15.5:0/334363615 shutdown_connections 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a3035740 1 -- 172.21.15.5:0/334363615 wait complete. 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a3035740 20 client.18189 trim_cache size 0 max 16384 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/3788125323 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7840a33b0 msgr2=0x7fa7840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a0c5c640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7840a33b0 0x7fa7840a5870 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa794009270 tx=0x7fa7940bc040 comp rx=0 tx=0).stop 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/3788125323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c106a80 msgr2=0x7fa79c19cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a0c5c640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c106a80 0x7fa79c19cd20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa79400eaa0 tx=0x7fa79400ef60 comp rx=0 tx=0).stop 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/3788125323 shutdown_connections 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a0c5c640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa7840a33b0 0x7fa7840a5870 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a0c5c640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa79c106a80 0x7fa79c19cd20 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a0c5c640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa79c101420 0x7fa79c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a0c5c640 1 --2- 172.21.15.5:0/3788125323 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa79c100a80 0x7fa79c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.720+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/3788125323 >> 172.21.15.5:0/3788125323 conn(0x7fa79c0fc7a0 msgr2=0x7fa79c1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.721+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/3788125323 shutdown_connections 2024-09-15T22:25:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:25.721+0000 7fa7a0c5c640 1 -- 172.21.15.5:0/3788125323 wait complete. 2024-09-15T22:25:26.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-15T22:25:26.805 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:27.088 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:27 smithi005 ceph-mon[30944]: pgmap v519: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 6 op/s 2024-09-15T22:25:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.284+0000 7f4f58fd7640 1 -- 172.21.15.5:0/1076542441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54106f20 msgr2=0x7f4f5410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.284+0000 7f4f58fd7640 1 --2- 172.21.15.5:0/1076542441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54106f20 0x7f4f5410eaa0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009ca0 tx=0x7f4f4c02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.284+0000 7f4f58fd7640 1 -- 172.21.15.5:0/1076542441 shutdown_connections 2024-09-15T22:25:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.284+0000 7f4f58fd7640 1 --2- 172.21.15.5:0/1076542441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54106f20 0x7f4f5410eaa0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.284+0000 7f4f58fd7640 1 --2- 172.21.15.5:0/1076542441 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f541027a0 0x7f4f541067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.284+0000 7f4f58fd7640 1 --2- 172.21.15.5:0/1076542441 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f54101df0 0x7f4f541021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.284+0000 7f4f58fd7640 1 -- 172.21.15.5:0/1076542441 >> 172.21.15.5:0/1076542441 conn(0x7f4f540fd8f0 msgr2=0x7f4f540ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.284+0000 7f4f58fd7640 1 -- 172.21.15.5:0/1076542441 shutdown_connections 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.284+0000 7f4f58fd7640 1 -- 172.21.15.5:0/1076542441 wait complete. 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.285+0000 7f4f58fd7640 1 Processor -- start 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.285+0000 7f4f58fd7640 1 -- start start 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f58fd7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54101df0 0x7f4f541983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f58fd7640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f541027a0 0x7f4f54198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f58fd7640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54106f20 0x7f4f5419cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f58fd7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54111f80 con 0x7f4f54101df0 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f58fd7640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54111e00 con 0x7f4f54106f20 2024-09-15T22:25:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f58fd7640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54112100 con 0x7f4f541027a0 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54101df0 0x7f4f541983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54101df0 0x7f4f541983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53456/0 (socket says 172.21.15.5:53456) 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f5259c640 1 -- 172.21.15.5:0/3027861836 learned_addr learned my addr 172.21.15.5:0/3027861836 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54106f20 0x7f4f5419cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f51d9b640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f541027a0 0x7f4f54198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f5259c640 1 -- 172.21.15.5:0/3027861836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f541027a0 msgr2=0x7f4f54198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f5259c640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f541027a0 0x7f4f54198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f5259c640 1 -- 172.21.15.5:0/3027861836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54106f20 msgr2=0x7f4f5419cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f5259c640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54106f20 0x7f4f5419cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f5259c640 1 -- 172.21.15.5:0/3027861836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5419d370 con 0x7f4f54101df0 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.286+0000 7f4f5259c640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54101df0 0x7f4f541983c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400d770 tx=0x7f4f4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.287+0000 7f4f437fe640 1 -- 172.21.15.5:0/3027861836 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44017070 con 0x7f4f54101df0 2024-09-15T22:25:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.287+0000 7f4f58fd7640 1 -- 172.21.15.5:0/3027861836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5419d660 con 0x7f4f54101df0 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.287+0000 7f4f437fe640 1 -- 172.21.15.5:0/3027861836 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f44005500 con 0x7f4f54101df0 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.287+0000 7f4f437fe640 1 -- 172.21.15.5:0/3027861836 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44004e00 con 0x7f4f54101df0 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.287+0000 7f4f58fd7640 1 -- 172.21.15.5:0/3027861836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f541a4f40 con 0x7f4f54101df0 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f5b3b0740 1 Processor -- start 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f5b3b0740 1 -- start start 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c136090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 0x55e93c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c141370 0x55e93c143760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e93c09c2f0 con 0x55e93c137b40 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e93c043cf0 con 0x55e93c1365d0 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e93c0982b0 con 0x55e93c141370 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f437fe640 1 -- 172.21.15.5:0/3027861836 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f4f44004fa0 con 0x7f4f54101df0 2024-09-15T22:25:27.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f52d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c136090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f52d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c136090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53472/0 (socket says 172.21.15.5:53472) 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f52d9d640 1 -- 172.21.15.5:0/1331249466 learned_addr learned my addr 172.21.15.5:0/1331249466 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.290+0000 7f4f51d9b640 1 --2- 172.21.15.5:0/1331249466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c141370 0x55e93c143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f5259c640 1 --2- 172.21.15.5:0/1331249466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 0x55e93c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f52d9d640 1 -- 172.21.15.5:0/1331249466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c141370 msgr2=0x55e93c143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/1331249466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c141370 0x55e93c143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f52d9d640 1 -- 172.21.15.5:0/1331249466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 msgr2=0x55e93c136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/1331249466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 0x55e93c136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f52d9d640 1 -- 172.21.15.5:0/1331249466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93c0283a0 con 0x55e93c137b40 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f5259c640 1 --2- 172.21.15.5:0/1331249466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 0x55e93c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/1331249466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c136090 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c004af0 tx=0x7f4f4c0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95c181794f6becc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f40ff9640 1 -- 172.21.15.5:0/1331249466 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c003970 con 0x55e93c137b40 2024-09-15T22:25:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f40ff9640 1 -- 172.21.15.5:0/1331249466 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c036040 con 0x55e93c137b40 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f40ff9640 1 -- 172.21.15.5:0/1331249466 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c0043a0 con 0x55e93c137b40 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.291+0000 7f4f437fe640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4f3c0a33b0 0x7f4f3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f5259c640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4f3c0a33b0 0x7f4f3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f437fe640 1 -- 172.21.15.5:0/3027861836 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f4f440cfba0 con 0x7f4f54101df0 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f5259c640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4f3c0a33b0 0x7f4f3c0a5870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4f440ccfa0 tx=0x7f4f44086040 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/1331249466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 msgr2=0x55e93c136090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/1331249466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c136090 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c004af0 tx=0x7f4f4c0098b0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/1331249466 shutdown_connections 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/1331249466 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c141370 0x55e93c143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/1331249466 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 0x55e93c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/1331249466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c136090 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/1331249466 >> 172.21.15.5:0/1331249466 conn(0x55e93c0ff310 msgr2=0x55e93c132c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/1331249466 shutdown_connections 2024-09-15T22:25:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.292+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/1331249466 wait complete. 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f5b3b0740 1 -- start start 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f5b3b0740 1 Processor -- start 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 0x55e93c145550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c145a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c141370 0x55e93c149c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e93c143e90 con 0x55e93c137b40 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55e93c057060 con 0x55e93c1365d0 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55e93c144010 con 0x55e93c141370 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f5b3b0740 10 client.0 _ll_register_callbacks cb 0x55e93bdaa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f51d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f51d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c145a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53478/0 (socket says 172.21.15.5:53478) 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.293+0000 7f4f51d9b640 1 -- 172.21.15.5:0/4082085788 learned_addr learned my addr 172.21.15.5:0/4082085788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 0x55e93c145550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c141370 0x55e93c149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f51d9b640 1 -- 172.21.15.5:0/4082085788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c141370 msgr2=0x55e93c149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f51d9b640 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c141370 0x55e93c149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f51d9b640 1 -- 172.21.15.5:0/4082085788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 msgr2=0x55e93c145550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f51d9b640 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 0x55e93c145550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f51d9b640 1 -- 172.21.15.5:0/4082085788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93be0d3c0 con 0x55e93c137b40 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 0x55e93c145550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f51d9b640 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c145a90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800be00 tx=0x7f4f480117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ca95c2413437f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f39ffb640 1 -- 172.21.15.5:0/4082085788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48012650 con 0x55e93c137b40 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f39ffb640 1 -- 172.21.15.5:0/4082085788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f4f480127f0 con 0x55e93c137b40 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/4082085788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4c036040 con 0x55e93c137b40 2024-09-15T22:25:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f39ffb640 1 -- 172.21.15.5:0/4082085788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f480129f0 con 0x55e93c137b40 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f5b3b0740 10 client.18201 fetch_fsmap learned FSMap version 25 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.294+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/4082085788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e93c153cd0 con 0x55e93c137b40 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.295+0000 7f4f39ffb640 1 -- 172.21.15.5:0/4082085788 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c036040 con 0x55e93c137b40 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.295+0000 7f4f39ffb640 1 -- 172.21.15.5:0/4082085788 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4f4801a3d0 con 0x55e93c137b40 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.295+0000 7f4f5b3b0740 10 client.18201 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.295+0000 7f4f5b3b0740 10 client.18201 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.295+0000 7f4f5b3b0740 20 client.18201 populate_metadata read hostname 'smithi005' 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.295+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e93c155190 0x55e93c157610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.295+0000 7f4f5b3b0740 4 client.18201 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.295+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/4082085788 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e93c0a9860 con 0x55e93c155190 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.295+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e93c155190 0x55e93c157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.295+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e93c155190 0x55e93c157610 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc52db705640ce62 server_cookie=d4e3a7f808c5e08c in_seq=0 out_seq=0 2024-09-15T22:25:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.295+0000 7f4f39ffb640 10 client.18201 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f39ffb640 1 -- 172.21.15.5:0/4082085788 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e93c0a9860 con 0x55e93c155190 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f39ffb640 10 client.18201 handle_command_reply: tid=0 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f5b3b0740 1 client.18201 shutdown 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/4082085788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 msgr2=0x55e93c145a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c145a90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800be00 tx=0x7f4f480117b0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/4082085788 shutdown_connections 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e93c155190 0x55e93c157610 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c141370 0x55e93c149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c137b40 0x55e93c145a90 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f39ffb640 0 client.18201 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/4082085788 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c1365d0 0x55e93c145550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.296+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/4082085788 >> 172.21.15.5:0/4082085788 conn(0x55e93c0ff310 msgr2=0x55e93c138960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.297+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/4082085788 shutdown_connections 2024-09-15T22:25:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.297+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/4082085788 wait complete. 2024-09-15T22:25:27.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.297+0000 7f4f5b3b0740 20 client.18201 trim_cache size 0 max 16384 2024-09-15T22:25:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.314+0000 7f4f5b3b0740 1 Processor -- start 2024-09-15T22:25:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.314+0000 7f4f5b3b0740 1 -- start start 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.314+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c1365d0 0x55e93c149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.314+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c137b40 0x55e93c132e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.314+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c141370 0x55e93c15cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.314+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541af710 con 0x55e93c1365d0 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.314+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541acfe0 con 0x55e93c141370 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.314+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541aae50 con 0x55e93c137b40 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.314+0000 7f4f52d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c1365d0 0x55e93c149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.314+0000 7f4f52d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c1365d0 0x55e93c149bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53484/0 (socket says 172.21.15.5:53484) 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.314+0000 7f4f52d9d640 1 -- 172.21.15.5:0/3823098245 learned_addr learned my addr 172.21.15.5:0/3823098245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.315+0000 7f4f51d9b640 1 --2- 172.21.15.5:0/3823098245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c137b40 0x55e93c132e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.315+0000 7f4f52d9d640 1 -- 172.21.15.5:0/3823098245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c137b40 msgr2=0x55e93c132e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.315+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/3823098245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c137b40 0x55e93c132e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.315+0000 7f4f52d9d640 1 -- 172.21.15.5:0/3823098245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c141370 msgr2=0x55e93c15cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.315+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/3823098245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c141370 0x55e93c15cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.315+0000 7f4f52d9d640 1 -- 172.21.15.5:0/3823098245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e93c145e00 con 0x55e93c1365d0 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.315+0000 7f4f51d9b640 1 --2- 172.21.15.5:0/3823098245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c137b40 0x55e93c132e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.315+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/3823098245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c1365d0 0x55e93c149bf0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c005e30 tx=0x7f4f4c004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fbfc730d09a0b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.315+0000 7f4f3bfff640 1 -- 172.21.15.5:0/3823098245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c031450 con 0x55e93c1365d0 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.315+0000 7f4f3bfff640 1 -- 172.21.15.5:0/3823098245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c036040 con 0x55e93c1365d0 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.315+0000 7f4f3bfff640 1 -- 172.21.15.5:0/3823098245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c004380 con 0x55e93c1365d0 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.316+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/3823098245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c1365d0 msgr2=0x55e93c149bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.316+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/3823098245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c1365d0 0x55e93c149bf0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c005e30 tx=0x7f4f4c004a80 comp rx=0 tx=0).stop 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.316+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/3823098245 shutdown_connections 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.316+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/3823098245 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c141370 0x55e93c15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.316+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/3823098245 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c137b40 0x55e93c132e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.316+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/3823098245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c1365d0 0x55e93c149bf0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.316+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/3823098245 >> 172.21.15.5:0/3823098245 conn(0x55e93c0ff310 msgr2=0x55e93c132700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.316+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/3823098245 shutdown_connections 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.316+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/3823098245 wait complete. 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.316+0000 7f4f5b3b0740 1 -- start start 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f5b3b0740 1 Processor -- start 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c1365d0 0x55e93c151a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c137b40 0x55e93c13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f5b3b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c141370 0x55e93c13fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541ab4a0 con 0x55e93c141370 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541ac910 con 0x55e93c137b40 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f5b3b0740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f4f541abb70 con 0x55e93c1365d0 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f5b3b0740 10 client.0 _ll_register_callbacks cb 0x7f4f4800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f52d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c141370 0x55e93c13fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f52d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c141370 0x55e93c13fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53492/0 (socket says 172.21.15.5:53492) 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f52d9d640 1 -- 172.21.15.5:0/277513327 learned_addr learned my addr 172.21.15.5:0/277513327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f51d9b640 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c137b40 0x55e93c13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c1365d0 0x55e93c151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f52d9d640 1 -- 172.21.15.5:0/277513327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c1365d0 msgr2=0x55e93c151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c1365d0 0x55e93c151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f52d9d640 1 -- 172.21.15.5:0/277513327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c137b40 msgr2=0x55e93c13f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c137b40 0x55e93c13f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.317+0000 7f4f52d9d640 1 -- 172.21.15.5:0/277513327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e93c140360 con 0x55e93c141370 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f52d9d640 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c141370 0x55e93c13fb30 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0095e0 tx=0x7f4f4c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908b269b7cfd1b42 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f39ffb640 1 -- 172.21.15.5:0/277513327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4c003b80 con 0x55e93c141370 2024-09-15T22:25:27.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f39ffb640 1 -- 172.21.15.5:0/277513327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f4f4c04a040 con 0x55e93c141370 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/277513327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4c036040 con 0x55e93c141370 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f39ffb640 1 -- 172.21.15.5:0/277513327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f4c064050 con 0x55e93c141370 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f5b3b0740 10 client.18207 fetch_fsmap learned FSMap version 25 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/277513327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e93c1bfe30 con 0x55e93c141370 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f39ffb640 1 -- 172.21.15.5:0/277513327 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f4c036040 con 0x55e93c141370 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f39ffb640 1 -- 172.21.15.5:0/277513327 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4f4c043070 con 0x55e93c141370 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f5b3b0740 10 client.18207 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f5b3b0740 10 client.18207 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.318+0000 7f4f5b3b0740 20 client.18207 populate_metadata read hostname 'smithi005' 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.319+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e93c1c00c0 0x55e93c1c24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.319+0000 7f4f5b3b0740 4 client.18207 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.319+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/277513327 --> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e93c0a3d70 con 0x55e93c1c00c0 2024-09-15T22:25:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.319+0000 7f4f51d9b640 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e93c1c00c0 0x55e93c1c24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.319+0000 7f4f51d9b640 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e93c1c00c0 0x55e93c1c24b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30d44e6bc180a49b server_cookie=408e9c66a342f1f2 in_seq=0 out_seq=0 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.319+0000 7f4f39ffb640 10 client.18207 ms_handle_connect on v2:172.21.15.99:6836/3952304847 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.319+0000 7f4f39ffb640 1 -- 172.21.15.5:0/277513327 <== mds.1 v2:172.21.15.99:6836/3952304847 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e93c0a3d70 con 0x55e93c1c00c0 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.319+0000 7f4f39ffb640 10 client.18207 handle_command_reply: tid=0 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.319+0000 7f4f5b3b0740 1 client.18207 shutdown 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/277513327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c141370 msgr2=0x55e93c13fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c141370 0x55e93c13fb30 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c0095e0 tx=0x7f4f4c005d80 comp rx=0 tx=0).stop 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/277513327 shutdown_connections 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.99:6836/3952304847,v1:172.21.15.99:6837/3952304847] conn(0x55e93c1c00c0 0x55e93c1c24b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e93c141370 0x55e93c13fb30 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55e93c137b40 0x55e93c13f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f39ffb640 0 client.18207 ms_handle_reset on v2:172.21.15.99:6836/3952304847 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f5b3b0740 1 --2- 172.21.15.5:0/277513327 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55e93c1365d0 0x55e93c151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/277513327 >> 172.21.15.5:0/277513327 conn(0x55e93c0ff310 msgr2=0x55e93c14ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/277513327 shutdown_connections 2024-09-15T22:25:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f5b3b0740 1 -- 172.21.15.5:0/277513327 wait complete. 2024-09-15T22:25:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f5b3b0740 20 client.18207 trim_cache size 0 max 16384 2024-09-15T22:25:27.330 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:25:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f58fd7640 1 -- 172.21.15.5:0/3027861836 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4f3c0a33b0 msgr2=0x7f4f3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f58fd7640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4f3c0a33b0 0x7f4f3c0a5870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4f440ccfa0 tx=0x7f4f44086040 comp rx=0 tx=0).stop 2024-09-15T22:25:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f58fd7640 1 -- 172.21.15.5:0/3027861836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54101df0 msgr2=0x7f4f541983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.320+0000 7f4f58fd7640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54101df0 0x7f4f541983c0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4f4400d770 tx=0x7f4f4400dc30 comp rx=0 tx=0).stop 2024-09-15T22:25:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.321+0000 7f4f58fd7640 1 -- 172.21.15.5:0/3027861836 shutdown_connections 2024-09-15T22:25:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.321+0000 7f4f58fd7640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f4f3c0a33b0 0x7f4f3c0a5870 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.321+0000 7f4f58fd7640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4f54106f20 0x7f4f5419cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.321+0000 7f4f58fd7640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4f541027a0 0x7f4f54198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.321+0000 7f4f58fd7640 1 --2- 172.21.15.5:0/3027861836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f54101df0 0x7f4f541983c0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.321+0000 7f4f58fd7640 1 -- 172.21.15.5:0/3027861836 >> 172.21.15.5:0/3027861836 conn(0x7f4f540fd8f0 msgr2=0x7f4f540fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.321+0000 7f4f58fd7640 1 -- 172.21.15.5:0/3027861836 shutdown_connections 2024-09-15T22:25:27.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:27.321+0000 7f4f58fd7640 1 -- 172.21.15.5:0/3027861836 wait complete. 2024-09-15T22:25:27.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:27 smithi099 ceph-mon[34892]: pgmap v519: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 6 op/s 2024-09-15T22:25:27.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:27 smithi129 ceph-mon[35314]: pgmap v519: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.2 KiB/s wr, 6 op/s 2024-09-15T22:25:28.126 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-15T22:25:28.386 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.909+0000 7f4795a71640 1 -- 172.21.15.5:0/2766822604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4790102790 msgr2=0x7f47901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:28.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.909+0000 7f4795a71640 1 --2- 172.21.15.5:0/2766822604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4790102790 0x7f47901067e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f4784009ca0 tx=0x7f478402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.909+0000 7f4795a71640 1 -- 172.21.15.5:0/2766822604 shutdown_connections 2024-09-15T22:25:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.909+0000 7f4795a71640 1 --2- 172.21.15.5:0/2766822604 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4790106f10 0x7f479010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.909+0000 7f4795a71640 1 --2- 172.21.15.5:0/2766822604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4790102790 0x7f47901067e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.909+0000 7f4795a71640 1 --2- 172.21.15.5:0/2766822604 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4790101de0 0x7f47901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.909+0000 7f4795a71640 1 -- 172.21.15.5:0/2766822604 >> 172.21.15.5:0/2766822604 conn(0x7f47900fd8c0 msgr2=0x7f47900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.909+0000 7f4795a71640 1 -- 172.21.15.5:0/2766822604 shutdown_connections 2024-09-15T22:25:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.909+0000 7f4795a71640 1 -- 172.21.15.5:0/2766822604 wait complete. 2024-09-15T22:25:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.909+0000 7f4795a71640 1 Processor -- start 2024-09-15T22:25:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f4795a71640 1 -- start start 2024-09-15T22:25:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f4795a71640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4790101de0 0x7f4790198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f4795a71640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4790102790 0x7f4790198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f4795a71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4790106f10 0x7f479019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f4795a71640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4790111f30 con 0x7f4790106f10 2024-09-15T22:25:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f4795a71640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f4790111db0 con 0x7f4790101de0 2024-09-15T22:25:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f4795a71640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f47901120b0 con 0x7f4790102790 2024-09-15T22:25:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f478ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4790106f10 0x7f479019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f478ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4790106f10 0x7f479019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53510/0 (socket says 172.21.15.5:53510) 2024-09-15T22:25:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f478ffff640 1 -- 172.21.15.5:0/2753470581 learned_addr learned my addr 172.21.15.5:0/2753470581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f478effd640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4790102790 0x7f4790198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f478f7fe640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4790101de0 0x7f4790198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f478ffff640 1 -- 172.21.15.5:0/2753470581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4790102790 msgr2=0x7f4790198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f478ffff640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4790102790 0x7f4790198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f478ffff640 1 -- 172.21.15.5:0/2753470581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4790101de0 msgr2=0x7f4790198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f478ffff640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4790101de0 0x7f4790198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.910+0000 7f478ffff640 1 -- 172.21.15.5:0/2753470581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f479019d300 con 0x7f4790106f10 2024-09-15T22:25:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.911+0000 7f478ffff640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4790106f10 0x7f479019cc20 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f478800c970 tx=0x7f478800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.911+0000 7f478cff9640 1 -- 172.21.15.5:0/2753470581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788007bb0 con 0x7f4790106f10 2024-09-15T22:25:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.911+0000 7f4795a71640 1 -- 172.21.15.5:0/2753470581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479019d5f0 con 0x7f4790106f10 2024-09-15T22:25:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.911+0000 7f4795a71640 1 -- 172.21.15.5:0/2753470581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47901a4ed0 con 0x7f4790106f10 2024-09-15T22:25:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.911+0000 7f478cff9640 1 -- 172.21.15.5:0/2753470581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4788007d50 con 0x7f4790106f10 2024-09-15T22:25:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.911+0000 7f478cff9640 1 -- 172.21.15.5:0/2753470581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47880186e0 con 0x7f4790106f10 2024-09-15T22:25:28.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.913+0000 7f478cff9640 1 -- 172.21.15.5:0/2753470581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f478800f450 con 0x7f4790106f10 2024-09-15T22:25:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.913+0000 7f4797e4a740 1 Processor -- start 2024-09-15T22:25:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.913+0000 7f4797e4a740 1 -- start start 2024-09-15T22:25:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f408625b40 0x55f408624090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4086245d0 0x55f408624a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40862f370 0x55f408631760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f4797e4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f40858a2f0 con 0x55f408625b40 2024-09-15T22:25:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f4797e4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f408531cf0 con 0x55f40862f370 2024-09-15T22:25:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f4797e4a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f4085862b0 con 0x55f4086245d0 2024-09-15T22:25:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f408625b40 0x55f408624090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f408625b40 0x55f408624090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53520/0 (socket says 172.21.15.5:53520) 2024-09-15T22:25:28.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478effd640 1 -- 172.21.15.5:0/3866402154 learned_addr learned my addr 172.21.15.5:0/3866402154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478cff9640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47780a3430 0x7f47780a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478f7fe640 1 --2- 172.21.15.5:0/3866402154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4086245d0 0x55f408624a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478effd640 1 --2- 172.21.15.5:0/3866402154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40862f370 0x55f408631760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478ffff640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47780a3430 0x7f47780a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478effd640 1 -- 172.21.15.5:0/3866402154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4086245d0 msgr2=0x55f408624a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478effd640 1 --2- 172.21.15.5:0/3866402154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4086245d0 0x55f408624a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478effd640 1 -- 172.21.15.5:0/3866402154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40862f370 msgr2=0x55f408631760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478effd640 1 --2- 172.21.15.5:0/3866402154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40862f370 0x55f408631760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478effd640 1 -- 172.21.15.5:0/3866402154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4085163a0 con 0x55f408625b40 2024-09-15T22:25:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478f7fe640 1 --2- 172.21.15.5:0/3866402154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4086245d0 0x55f408624a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:28.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.914+0000 7f478cff9640 1 -- 172.21.15.5:0/2753470581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f47880d0090 con 0x7f4790106f10 2024-09-15T22:25:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.915+0000 7f478effd640 1 --2- 172.21.15.5:0/3866402154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f408625b40 0x55f408624090 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f4784009bf0 tx=0x7f4784005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0678ca71aeb731f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.915+0000 7f4775ffb640 1 -- 172.21.15.5:0/3866402154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784031650 con 0x55f408625b40 2024-09-15T22:25:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.915+0000 7f4775ffb640 1 -- 172.21.15.5:0/3866402154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4784036040 con 0x55f408625b40 2024-09-15T22:25:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.915+0000 7f4775ffb640 1 -- 172.21.15.5:0/3866402154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784030820 con 0x55f408625b40 2024-09-15T22:25:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.915+0000 7f478ffff640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47780a3430 0x7f47780a58f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f478809bea0 tx=0x7f478806dc90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.916+0000 7f4797e4a740 1 -- 172.21.15.5:0/3866402154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f408625b40 msgr2=0x55f408624090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.916+0000 7f4797e4a740 1 --2- 172.21.15.5:0/3866402154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f408625b40 0x55f408624090 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f4784009bf0 tx=0x7f4784005fa0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.916+0000 7f4797e4a740 1 -- 172.21.15.5:0/3866402154 shutdown_connections 2024-09-15T22:25:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.916+0000 7f4797e4a740 1 --2- 172.21.15.5:0/3866402154 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f40862f370 0x55f408631760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.916+0000 7f4797e4a740 1 --2- 172.21.15.5:0/3866402154 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f4086245d0 0x55f408624a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.916+0000 7f4797e4a740 1 --2- 172.21.15.5:0/3866402154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f408625b40 0x55f408624090 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.916+0000 7f4797e4a740 1 -- 172.21.15.5:0/3866402154 >> 172.21.15.5:0/3866402154 conn(0x55f4085ed310 msgr2=0x55f408620c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.916+0000 7f4797e4a740 1 -- 172.21.15.5:0/3866402154 shutdown_connections 2024-09-15T22:25:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.916+0000 7f4797e4a740 1 -- 172.21.15.5:0/3866402154 wait complete. 2024-09-15T22:25:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.916+0000 7f4797e4a740 1 -- start start 2024-09-15T22:25:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.916+0000 7f4797e4a740 1 Processor -- start 2024-09-15T22:25:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f40862ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f408625b40 0x55f40862bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f40862f370 0x55f40862c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f4797e4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f408631e90 con 0x55f4086245d0 2024-09-15T22:25:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f4797e4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x55f408545060 con 0x55f408625b40 2024-09-15T22:25:28.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f4797e4a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x55f408632010 con 0x55f40862f370 2024-09-15T22:25:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f4797e4a740 10 client.0 _ll_register_callbacks cb 0x55f4082988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f478effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f40862ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f478effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f40862ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53524/0 (socket says 172.21.15.5:53524) 2024-09-15T22:25:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f478effd640 1 -- 172.21.15.5:0/105017452 learned_addr learned my addr 172.21.15.5:0/105017452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f478f7fe640 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f408625b40 0x55f40862bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f478effd640 1 -- 172.21.15.5:0/105017452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f40862f370 msgr2=0x55f40862c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f478effd640 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f40862f370 0x55f40862c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f478effd640 1 -- 172.21.15.5:0/105017452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f408625b40 msgr2=0x55f40862bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f478effd640 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f408625b40 0x55f40862bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f478effd640 1 -- 172.21.15.5:0/105017452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4082fb3c0 con 0x55f4086245d0 2024-09-15T22:25:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f478effd640 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f40862ecb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4784009620 tx=0x7f4784031a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2979703323208a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f476effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:25:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f476effd640 1 -- 172.21.15.5:0/105017452 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784054070 con 0x55f4086245d0 2024-09-15T22:25:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f476effd640 1 -- 172.21.15.5:0/105017452 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f47840305c0 con 0x55f4086245d0 2024-09-15T22:25:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.917+0000 7f4797e4a740 1 -- 172.21.15.5:0/105017452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4784036040 con 0x55f4086245d0 2024-09-15T22:25:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f476effd640 1 -- 172.21.15.5:0/105017452 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4784004b40 con 0x55f4086245d0 2024-09-15T22:25:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f4797e4a740 10 client.18219 fetch_fsmap learned FSMap version 25 2024-09-15T22:25:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f4797e4a740 1 -- 172.21.15.5:0/105017452 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f408631e90 con 0x55f4086245d0 2024-09-15T22:25:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f476effd640 1 -- 172.21.15.5:0/105017452 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4784036040 con 0x55f4086245d0 2024-09-15T22:25:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f476effd640 1 -- 172.21.15.5:0/105017452 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4784005020 con 0x55f4086245d0 2024-09-15T22:25:28.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f4797e4a740 10 client.18219 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:25:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f4797e4a740 10 client.18219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f4797e4a740 20 client.18219 populate_metadata read hostname 'smithi005' 2024-09-15T22:25:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f4797e4a740 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f40863ea90 0x55f408640e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f4797e4a740 4 client.18219 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T22:25:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f4797e4a740 1 -- 172.21.15.5:0/105017452 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f408597860 con 0x55f40863ea90 2024-09-15T22:25:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.918+0000 7f478f7fe640 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f40863ea90 0x55f408640e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.919+0000 7f476effd640 10 client.18219 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:25:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.919+0000 7f478f7fe640 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f40863ea90 0x55f408640e80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ea19dcd8e792da server_cookie=b1fd6d90a2556108 in_seq=0 out_seq=0 2024-09-15T22:25:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.919+0000 7f476effd640 1 -- 172.21.15.5:0/105017452 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f408597860 con 0x55f40863ea90 2024-09-15T22:25:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.919+0000 7f476effd640 10 client.18219 handle_command_reply: tid=0 2024-09-15T22:25:28.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.919+0000 7f4797e4a740 1 client.18219 shutdown 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f4797e4a740 1 -- 172.21.15.5:0/105017452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 msgr2=0x55f40862ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f4797e4a740 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f40862ecb0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4784009620 tx=0x7f4784031a40 comp rx=0 tx=0).stop 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f4797e4a740 1 -- 172.21.15.5:0/105017452 shutdown_connections 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f4797e4a740 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f40862f370 0x55f40862c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f4797e4a740 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f40863ea90 0x55f408640e80 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f4797e4a740 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f408625b40 0x55f40862bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f4797e4a740 1 --2- 172.21.15.5:0/105017452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f40862ecb0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f4797e4a740 1 -- 172.21.15.5:0/105017452 >> 172.21.15.5:0/105017452 conn(0x55f4085ed310 msgr2=0x55f408626930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f476effd640 0 client.18219 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f4797e4a740 1 -- 172.21.15.5:0/105017452 shutdown_connections 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f4797e4a740 1 -- 172.21.15.5:0/105017452 wait complete. 2024-09-15T22:25:28.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.920+0000 7f4797e4a740 20 client.18219 trim_cache size 0 max 16384 2024-09-15T22:25:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.933+0000 7f4797e4a740 1 Processor -- start 2024-09-15T22:25:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.933+0000 7f4797e4a740 1 -- start start 2024-09-15T22:25:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4086245d0 0x55f40863b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f408625b40 0x55f4086466b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f40862f370 0x55f408646bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f4797e4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f477c0128d0 con 0x55f40862f370 2024-09-15T22:25:28.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f4797e4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f477c0101a0 con 0x55f4086245d0 2024-09-15T22:25:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f4797e4a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f477c00e010 con 0x55f408625b40 2024-09-15T22:25:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f478effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f40862f370 0x55f408646bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f478effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f40862f370 0x55f408646bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53538/0 (socket says 172.21.15.5:53538) 2024-09-15T22:25:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f478effd640 1 -- 172.21.15.5:0/307843496 learned_addr learned my addr 172.21.15.5:0/307843496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f478f7fe640 1 --2- 172.21.15.5:0/307843496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f408625b40 0x55f4086466b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f478effd640 1 --2- 172.21.15.5:0/307843496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4086245d0 0x55f40863b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f478effd640 1 -- 172.21.15.5:0/307843496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f408625b40 msgr2=0x55f4086466b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f478effd640 1 --2- 172.21.15.5:0/307843496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f408625b40 0x55f4086466b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f478effd640 1 -- 172.21.15.5:0/307843496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4086245d0 msgr2=0x55f40863b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f478effd640 1 --2- 172.21.15.5:0/307843496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4086245d0 0x55f40863b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f478effd640 1 -- 172.21.15.5:0/307843496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f408624d50 con 0x55f40862f370 2024-09-15T22:25:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f478effd640 1 --2- 172.21.15.5:0/307843496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f40862f370 0x55f408646bf0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f478402c010 tx=0x7f478402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fff832197f656091 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.934+0000 7f47757fa640 1 -- 172.21.15.5:0/307843496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784054040 con 0x55f40862f370 2024-09-15T22:25:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.935+0000 7f47757fa640 1 -- 172.21.15.5:0/307843496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4784036070 con 0x55f40862f370 2024-09-15T22:25:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.935+0000 7f47757fa640 1 -- 172.21.15.5:0/307843496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478402fd20 con 0x55f40862f370 2024-09-15T22:25:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.935+0000 7f4797e4a740 1 -- 172.21.15.5:0/307843496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f40862f370 msgr2=0x55f408646bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.935+0000 7f4797e4a740 1 --2- 172.21.15.5:0/307843496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f40862f370 0x55f408646bf0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f478402c010 tx=0x7f478402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.935+0000 7f4797e4a740 1 -- 172.21.15.5:0/307843496 shutdown_connections 2024-09-15T22:25:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.935+0000 7f4797e4a740 1 --2- 172.21.15.5:0/307843496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f40862f370 0x55f408646bf0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.935+0000 7f4797e4a740 1 --2- 172.21.15.5:0/307843496 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f408625b40 0x55f4086466b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.935+0000 7f4797e4a740 1 --2- 172.21.15.5:0/307843496 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f4086245d0 0x55f40863b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.935+0000 7f4797e4a740 1 -- 172.21.15.5:0/307843496 >> 172.21.15.5:0/307843496 conn(0x55f4085ed310 msgr2=0x55f40862bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.935+0000 7f4797e4a740 1 -- 172.21.15.5:0/307843496 shutdown_connections 2024-09-15T22:25:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.935+0000 7f4797e4a740 1 -- 172.21.15.5:0/307843496 wait complete. 2024-09-15T22:25:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f4797e4a740 1 -- start start 2024-09-15T22:25:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f4797e4a740 1 Processor -- start 2024-09-15T22:25:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f408633490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f408625b40 0x55f4086339d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f4797e4a740 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f40862f370 0x55f4086a8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f4797e4a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f477c00e660 con 0x55f4086245d0 2024-09-15T22:25:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f4797e4a740 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f477c00fad0 con 0x55f408625b40 2024-09-15T22:25:28.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f4797e4a740 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f477c00ed30 con 0x55f40862f370 2024-09-15T22:25:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f4797e4a740 10 client.0 _ll_register_callbacks cb 0x7f4784031e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T22:25:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f478effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f408633490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f478effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f408633490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53546/0 (socket says 172.21.15.5:53546) 2024-09-15T22:25:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f478effd640 1 -- 172.21.15.5:0/863827260 learned_addr learned my addr 172.21.15.5:0/863827260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.936+0000 7f478effd640 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f40862f370 0x55f4086a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f478f7fe640 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f408625b40 0x55f4086339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f478effd640 1 -- 172.21.15.5:0/863827260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f40862f370 msgr2=0x55f4086a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f478effd640 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f40862f370 0x55f4086a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f478effd640 1 -- 172.21.15.5:0/863827260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f408625b40 msgr2=0x55f4086339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T22:25:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f478effd640 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f408625b40 0x55f4086339d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f478effd640 1 -- 172.21.15.5:0/863827260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4086a8bc0 con 0x55f4086245d0 2024-09-15T22:25:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f478effd640 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f408633490 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4784009bf0 tx=0x7f478402f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5b8e53564c3e67 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f476effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-15T22:25:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f476effd640 1 -- 172.21.15.5:0/863827260 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784054040 con 0x55f4086245d0 2024-09-15T22:25:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f476effd640 1 -- 172.21.15.5:0/863827260 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f47840058b0 con 0x55f4086245d0 2024-09-15T22:25:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f4797e4a740 1 -- 172.21.15.5:0/863827260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4784036070 con 0x55f4086245d0 2024-09-15T22:25:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f476effd640 1 -- 172.21.15.5:0/863827260 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f478402fe90 con 0x55f4086245d0 2024-09-15T22:25:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f4797e4a740 10 client.18225 fetch_fsmap learned FSMap version 25 2024-09-15T22:25:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f4797e4a740 1 -- 172.21.15.5:0/863827260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f4086a9d10 con 0x55f4086245d0 2024-09-15T22:25:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.937+0000 7f476effd640 1 -- 172.21.15.5:0/863827260 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4784036070 con 0x55f4086245d0 2024-09-15T22:25:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.938+0000 7f476effd640 1 -- 172.21.15.5:0/863827260 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4784004540 con 0x55f4086245d0 2024-09-15T22:25:28.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.938+0000 7f4797e4a740 10 client.18225 fetch_fsmap finished waiting for FSMap version 25 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.938+0000 7f4797e4a740 10 client.18225 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.938+0000 7f4797e4a740 20 client.18225 populate_metadata read hostname 'smithi005' 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.938+0000 7f4797e4a740 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f4086aa1e0 0x55f4086ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.938+0000 7f4797e4a740 4 client.18225 mds_command: new command op to 24491 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.938+0000 7f4797e4a740 1 -- 172.21.15.5:0/863827260 --> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f408591d70 con 0x55f4086aa1e0 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.938+0000 7f478f7fe640 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f4086aa1e0 0x55f4086ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.938+0000 7f476effd640 10 client.18225 ms_handle_connect on v2:172.21.15.99:6838/3763383986 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.938+0000 7f478f7fe640 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f4086aa1e0 0x55f4086ac5d0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c32c2498817ff645 server_cookie=16471150ce67cceb in_seq=0 out_seq=0 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.938+0000 7f476effd640 1 -- 172.21.15.5:0/863827260 <== mds.0 v2:172.21.15.99:6838/3763383986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f408591d70 con 0x55f4086aa1e0 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f476effd640 10 client.18225 handle_command_reply: tid=0 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 1 client.18225 shutdown 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 1 -- 172.21.15.5:0/863827260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 msgr2=0x55f408633490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f408633490 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f4784009bf0 tx=0x7f478402f3d0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 1 -- 172.21.15.5:0/863827260 shutdown_connections 2024-09-15T22:25:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x55f40862f370 0x55f4086a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.99:6838/3763383986,v1:172.21.15.99:6839/3763383986] conn(0x55f4086aa1e0 0x55f4086ac5d0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x55f408625b40 0x55f4086339d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 1 --2- 172.21.15.5:0/863827260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4086245d0 0x55f408633490 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 1 -- 172.21.15.5:0/863827260 >> 172.21.15.5:0/863827260 conn(0x55f4085ed310 msgr2=0x55f40861f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f476effd640 0 client.18225 ms_handle_reset on v2:172.21.15.99:6838/3763383986 2024-09-15T22:25:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 1 -- 172.21.15.5:0/863827260 shutdown_connections 2024-09-15T22:25:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 1 -- 172.21.15.5:0/863827260 wait complete. 2024-09-15T22:25:28.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4797e4a740 20 client.18225 trim_cache size 0 max 16384 2024-09-15T22:25:28.952 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-15T22:25:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4795a71640 1 -- 172.21.15.5:0/2753470581 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47780a3430 msgr2=0x7f47780a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4795a71640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47780a3430 0x7f47780a58f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f478809bea0 tx=0x7f478806dc90 comp rx=0 tx=0).stop 2024-09-15T22:25:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4795a71640 1 -- 172.21.15.5:0/2753470581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4790106f10 msgr2=0x7f479019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.939+0000 7f4795a71640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4790106f10 0x7f479019cc20 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f478800c970 tx=0x7f478800cd70 comp rx=0 tx=0).stop 2024-09-15T22:25:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.940+0000 7f4795a71640 1 -- 172.21.15.5:0/2753470581 shutdown_connections 2024-09-15T22:25:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.940+0000 7f4795a71640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f47780a3430 0x7f47780a58f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.940+0000 7f4795a71640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4790106f10 0x7f479019cc20 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.940+0000 7f4795a71640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f4790102790 0x7f4790198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.940+0000 7f4795a71640 1 --2- 172.21.15.5:0/2753470581 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f4790101de0 0x7f4790198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.940+0000 7f4795a71640 1 -- 172.21.15.5:0/2753470581 >> 172.21.15.5:0/2753470581 conn(0x7f47900fd8c0 msgr2=0x7f47900feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:28.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.940+0000 7f4795a71640 1 -- 172.21.15.5:0/2753470581 shutdown_connections 2024-09-15T22:25:28.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:28.940+0000 7f4795a71640 1 -- 172.21.15.5:0/2753470581 wait complete. 2024-09-15T22:25:29.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:29 smithi005 ceph-mon[30944]: pgmap v520: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 6 op/s 2024-09-15T22:25:29.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:29 smithi099 ceph-mon[34892]: pgmap v520: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 6 op/s 2024-09-15T22:25:29.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:29 smithi129 ceph-mon[35314]: pgmap v520: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.2 KiB/s wr, 6 op/s 2024-09-15T22:25:29.750 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-15T22:25:29.751 INFO:tasks.fwd_scrub:done joining 2024-09-15T22:25:29.751 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-15T22:25:29.762 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T22:25:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.876+0000 7fa4b8b91640 1 -- 172.21.15.5:0/319945531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b41069d0 msgr2=0x7fa4b4106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.876+0000 7fa4b8b91640 1 --2- 172.21.15.5:0/319945531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b41069d0 0x7fa4b4106db0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4009930 tx=0x7fa4a402a420 comp rx=0 tx=0).stop 2024-09-15T22:25:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.876+0000 7fa4b8b91640 1 -- 172.21.15.5:0/319945531 shutdown_connections 2024-09-15T22:25:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.876+0000 7fa4b8b91640 1 --2- 172.21.15.5:0/319945531 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4b4101360 0x7fa4b410d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.876+0000 7fa4b8b91640 1 --2- 172.21.15.5:0/319945531 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b41009c0 0x7fa4b4100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.876+0000 7fa4b8b91640 1 --2- 172.21.15.5:0/319945531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b41069d0 0x7fa4b4106db0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.876+0000 7fa4b8b91640 1 -- 172.21.15.5:0/319945531 >> 172.21.15.5:0/319945531 conn(0x7fa4b4077a10 msgr2=0x7fa4b40fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.876+0000 7fa4b8b91640 1 -- 172.21.15.5:0/319945531 shutdown_connections 2024-09-15T22:25:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.876+0000 7fa4b8b91640 1 -- 172.21.15.5:0/319945531 wait complete. 2024-09-15T22:25:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b8b91640 1 Processor -- start 2024-09-15T22:25:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b8b91640 1 -- start start 2024-09-15T22:25:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b8b91640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4b41009c0 0x7fa4b41078f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b8b91640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b4101360 0x7fa4b4107e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b8b91640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b41069d0 0x7fa4b4108370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b8b91640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b4111f00 con 0x7fa4b4101360 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b8b91640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b4111d80 con 0x7fa4b41069d0 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b8b91640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b4112080 con 0x7fa4b41009c0 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b4101360 0x7fa4b4107e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b4101360 0x7fa4b4107e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53568/0 (socket says 172.21.15.5:53568) 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b1d9b640 1 -- 172.21.15.5:0/1030926961 learned_addr learned my addr 172.21.15.5:0/1030926961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b259c640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4b41009c0 0x7fa4b41078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b2d9d640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b41069d0 0x7fa4b4108370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b1d9b640 1 -- 172.21.15.5:0/1030926961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4b41009c0 msgr2=0x7fa4b41078f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b1d9b640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4b41009c0 0x7fa4b41078f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b1d9b640 1 -- 172.21.15.5:0/1030926961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b41069d0 msgr2=0x7fa4b4108370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:29.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.877+0000 7fa4b1d9b640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b41069d0 0x7fa4b4108370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.878+0000 7fa4b1d9b640 1 -- 172.21.15.5:0/1030926961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b41aabf0 con 0x7fa4b4101360 2024-09-15T22:25:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.878+0000 7fa4b1d9b640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b4101360 0x7fa4b4107e30 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800c950 tx=0x7fa4a800ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.878+0000 7fa4a37fe640 1 -- 172.21.15.5:0/1030926961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8007c70 con 0x7fa4b4101360 2024-09-15T22:25:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.878+0000 7fa4a37fe640 1 -- 172.21.15.5:0/1030926961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4a8007e10 con 0x7fa4b4101360 2024-09-15T22:25:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.878+0000 7fa4b8b91640 1 -- 172.21.15.5:0/1030926961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b41aaee0 con 0x7fa4b4101360 2024-09-15T22:25:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.878+0000 7fa4a37fe640 1 -- 172.21.15.5:0/1030926961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a8018960 con 0x7fa4b4101360 2024-09-15T22:25:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.878+0000 7fa4b8b91640 1 -- 172.21.15.5:0/1030926961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b41ab350 con 0x7fa4b4101360 2024-09-15T22:25:29.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.879+0000 7fa4b8b91640 1 -- 172.21.15.5:0/1030926961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b4102100 con 0x7fa4b4101360 2024-09-15T22:25:29.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.880+0000 7fa4a37fe640 1 -- 172.21.15.5:0/1030926961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fa4a8018b00 con 0x7fa4b4101360 2024-09-15T22:25:29.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.881+0000 7fa4a37fe640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa49c0a33c0 0x7fa49c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:29.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.881+0000 7fa4a37fe640 1 -- 172.21.15.5:0/1030926961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7fa4a80d0550 con 0x7fa4b4101360 2024-09-15T22:25:29.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.881+0000 7fa4b259c640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa49c0a33c0 0x7fa49c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:29.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.882+0000 7fa4b259c640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa49c0a33c0 0x7fa49c0a5880 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4000c00 tx=0x7fa4a4005a20 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:29.884+0000 7fa4a37fe640 1 -- 172.21.15.5:0/1030926961 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4a8014030 con 0x7fa4b4101360 2024-09-15T22:25:30.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.063+0000 7fa4b8b91640 1 -- 172.21.15.5:0/1030926961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4b4100e20 con 0x7fa4b4101360 2024-09-15T22:25:30.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.066+0000 7fa4a37fe640 1 -- 172.21.15.5:0/1030926961 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+25314 (secure 0 0 0) 0x7fa4b4100e20 con 0x7fa4b4101360 2024-09-15T22:25:30.068 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:25:30.069 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:25:18.970679+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2453332511":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/878276657":"2024-09-16T22:08:53.484141+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1408571851":"2024-09-16T22:08:53.484141+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/801026306":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6800/801026306":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2083481614":"2024-09-16T22:08:53.484141+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:25:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 -- 172.21.15.5:0/1030926961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa49c0a33c0 msgr2=0x7fa49c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa49c0a33c0 0x7fa49c0a5880 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4000c00 tx=0x7fa4a4005a20 comp rx=0 tx=0).stop 2024-09-15T22:25:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 -- 172.21.15.5:0/1030926961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b4101360 msgr2=0x7fa4b4107e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b4101360 0x7fa4b4107e30 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800c950 tx=0x7fa4a800ce10 comp rx=0 tx=0).stop 2024-09-15T22:25:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 -- 172.21.15.5:0/1030926961 shutdown_connections 2024-09-15T22:25:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fa49c0a33c0 0x7fa49c0a5880 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fa4b41069d0 0x7fa4b4108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4b4101360 0x7fa4b4107e30 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 --2- 172.21.15.5:0/1030926961 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fa4b41009c0 0x7fa4b41078f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 -- 172.21.15.5:0/1030926961 >> 172.21.15.5:0/1030926961 conn(0x7fa4b4077a10 msgr2=0x7fa4b4103b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 -- 172.21.15.5:0/1030926961 shutdown_connections 2024-09-15T22:25:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.067+0000 7fa4b8b91640 1 -- 172.21.15.5:0/1030926961 wait complete. 2024-09-15T22:25:30.081 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T22:01:57.184697+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-15T22:06:33.616843+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-15T22:06:36.161538+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-15T22:06:43.068085+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 27, 'snap_epoch': 108, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-15T22:25:30.082 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T22:25:30.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.186+0000 7fcda29fd640 1 -- 172.21.15.5:0/3171630703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd9c10c080 msgr2=0x7fcd9c10e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.186+0000 7fcda29fd640 1 --2- 172.21.15.5:0/3171630703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd9c10c080 0x7fcd9c10e540 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009ca0 tx=0x7fcd9402a950 comp rx=0 tx=0).stop 2024-09-15T22:25:30.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.186+0000 7fcda29fd640 1 -- 172.21.15.5:0/3171630703 shutdown_connections 2024-09-15T22:25:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.186+0000 7fcda29fd640 1 --2- 172.21.15.5:0/3171630703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd9c10c080 0x7fcd9c10e540 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.186+0000 7fcda29fd640 1 --2- 172.21.15.5:0/3171630703 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd9c103580 0x7fcd9c10ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.186+0000 7fcda29fd640 1 --2- 172.21.15.5:0/3171630703 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd9c102c60 0x7fcd9c103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.186+0000 7fcda29fd640 1 -- 172.21.15.5:0/3171630703 >> 172.21.15.5:0/3171630703 conn(0x7fcd9c0fc690 msgr2=0x7fcd9c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.186+0000 7fcda29fd640 1 -- 172.21.15.5:0/3171630703 shutdown_connections 2024-09-15T22:25:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.186+0000 7fcda29fd640 1 -- 172.21.15.5:0/3171630703 wait complete. 2024-09-15T22:25:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.187+0000 7fcda29fd640 1 Processor -- start 2024-09-15T22:25:30.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.187+0000 7fcda29fd640 1 -- start start 2024-09-15T22:25:30.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.187+0000 7fcda29fd640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd9c102c60 0x7fcd9c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.187+0000 7fcda29fd640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd9c103580 0x7fcd9c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.187+0000 7fcda29fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd9c10c080 0x7fcd9c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.187+0000 7fcda29fd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c111e80 con 0x7fcd9c10c080 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.187+0000 7fcda29fd640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c111d00 con 0x7fcd9c103580 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.187+0000 7fcda29fd640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c112000 con 0x7fcd9c102c60 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.187+0000 7fcda0f9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd9c10c080 0x7fcd9c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcda0f9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd9c10c080 0x7fcd9c19ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53584/0 (socket says 172.21.15.5:53584) 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcda0f9a640 1 -- 172.21.15.5:0/3533684839 learned_addr learned my addr 172.21.15.5:0/3533684839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcd9bfff640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd9c102c60 0x7fcd9c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcd9b7fe640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd9c103580 0x7fcd9c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcda0f9a640 1 -- 172.21.15.5:0/3533684839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd9c102c60 msgr2=0x7fcd9c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcda0f9a640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd9c102c60 0x7fcd9c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcda0f9a640 1 -- 172.21.15.5:0/3533684839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd9c103580 msgr2=0x7fcd9c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcda0f9a640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd9c103580 0x7fcd9c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcda0f9a640 1 -- 172.21.15.5:0/3533684839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c19d400 con 0x7fcd9c10c080 2024-09-15T22:25:30.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcd9b7fe640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd9c103580 0x7fcd9c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:25:30.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcda0f9a640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd9c10c080 0x7fcd9c19ccd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcd94006010 tx=0x7fcd9402f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:30.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcd997fa640 1 -- 172.21.15.5:0/3533684839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94034040 con 0x7fcd9c10c080 2024-09-15T22:25:30.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcd997fa640 1 -- 172.21.15.5:0/3533684839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd94003e00 con 0x7fcd9c10c080 2024-09-15T22:25:30.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcd997fa640 1 -- 172.21.15.5:0/3533684839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd940456e0 con 0x7fcd9c10c080 2024-09-15T22:25:30.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcda29fd640 1 -- 172.21.15.5:0/3533684839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c19d690 con 0x7fcd9c10c080 2024-09-15T22:25:30.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.188+0000 7fcda29fd640 1 -- 172.21.15.5:0/3533684839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c1a4f80 con 0x7fcd9c10c080 2024-09-15T22:25:30.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.190+0000 7fcda29fd640 1 -- 172.21.15.5:0/3533684839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9c103040 con 0x7fcd9c10c080 2024-09-15T22:25:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.191+0000 7fcd997fa640 1 -- 172.21.15.5:0/3533684839 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fcd94037020 con 0x7fcd9c10c080 2024-09-15T22:25:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.192+0000 7fcd997fa640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcd840a33c0 0x7fcd840a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.192+0000 7fcd9bfff640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcd840a33c0 0x7fcd840a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.192+0000 7fcd997fa640 1 -- 172.21.15.5:0/3533684839 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7fcd940f11a0 con 0x7fcd9c10c080 2024-09-15T22:25:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.192+0000 7fcd9bfff640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcd840a33c0 0x7fcd840a5880 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0058c0 tx=0x7fcd8c00a2d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.196+0000 7fcd997fa640 1 -- 172.21.15.5:0/3533684839 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd940bd890 con 0x7fcd9c10c080 2024-09-15T22:25:30.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.369+0000 7fcda29fd640 1 -- 172.21.15.5:0/3533684839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd9c1070e0 con 0x7fcd9c10c080 2024-09-15T22:25:30.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.370+0000 7fcd997fa640 1 -- 172.21.15.5:0/3533684839 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) ==== 93+0+10 (secure 0 0 0) 0x7fcd940bd190 con 0x7fcd9c10c080 2024-09-15T22:25:30.373 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-15T22:25:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 -- 172.21.15.5:0/3533684839 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcd840a33c0 msgr2=0x7fcd840a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcd840a33c0 0x7fcd840a5880 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0058c0 tx=0x7fcd8c00a2d0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 -- 172.21.15.5:0/3533684839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd9c10c080 msgr2=0x7fcd9c19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd9c10c080 0x7fcd9c19ccd0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fcd94006010 tx=0x7fcd9402f710 comp rx=0 tx=0).stop 2024-09-15T22:25:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 -- 172.21.15.5:0/3533684839 shutdown_connections 2024-09-15T22:25:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fcd840a33c0 0x7fcd840a5880 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd9c10c080 0x7fcd9c19ccd0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fcd9c103580 0x7fcd9c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 --2- 172.21.15.5:0/3533684839 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fcd9c102c60 0x7fcd9c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 -- 172.21.15.5:0/3533684839 >> 172.21.15.5:0/3533684839 conn(0x7fcd9c0fc690 msgr2=0x7fcd9c0fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:30.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 -- 172.21.15.5:0/3533684839 shutdown_connections 2024-09-15T22:25:30.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.372+0000 7fcda29fd640 1 -- 172.21.15.5:0/3533684839 wait complete. 2024-09-15T22:25:30.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-15T22:25:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.489+0000 7f80b5da6640 1 -- 172.21.15.5:0/3583518678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b010c0e0 msgr2=0x7f80b010e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.489+0000 7f80b5da6640 1 --2- 172.21.15.5:0/3583518678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b010c0e0 0x7f80b010e530 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009ca0 tx=0x7f80a802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.489+0000 7f80b5da6640 1 -- 172.21.15.5:0/3583518678 shutdown_connections 2024-09-15T22:25:30.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.489+0000 7f80b5da6640 1 --2- 172.21.15.5:0/3583518678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b010c0e0 0x7f80b010e530 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.489+0000 7f80b5da6640 1 --2- 172.21.15.5:0/3583518678 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b0103590 0x7f80b010ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.489+0000 7f80b5da6640 1 --2- 172.21.15.5:0/3583518678 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80b0102be0 0x7f80b0102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.489+0000 7f80b5da6640 1 -- 172.21.15.5:0/3583518678 >> 172.21.15.5:0/3583518678 conn(0x7f80b00fc6a0 msgr2=0x7f80b00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.489+0000 7f80b5da6640 1 -- 172.21.15.5:0/3583518678 shutdown_connections 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.489+0000 7f80b5da6640 1 -- 172.21.15.5:0/3583518678 wait complete. 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.490+0000 7f80b5da6640 1 Processor -- start 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.490+0000 7f80b5da6640 1 -- start start 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.490+0000 7f80b5da6640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b0102be0 0x7f80b0198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.490+0000 7f80b5da6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0103590 0x7f80b01988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.490+0000 7f80b5da6640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80b010c0e0 0x7f80b019cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.490+0000 7f80b5da6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f80b0111e90 con 0x7f80b0103590 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.490+0000 7f80b5da6640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f80b0111d10 con 0x7f80b010c0e0 2024-09-15T22:25:30.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.490+0000 7f80b5da6640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f80b0112010 con 0x7f80b0102be0 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80aeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0103590 0x7f80b01988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80aeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0103590 0x7f80b01988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53600/0 (socket says 172.21.15.5:53600) 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80aeffd640 1 -- 172.21.15.5:0/3754827055 learned_addr learned my addr 172.21.15.5:0/3754827055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80af7fe640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b0102be0 0x7f80b0198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80affff640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80b010c0e0 0x7f80b019cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80aeffd640 1 -- 172.21.15.5:0/3754827055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b0102be0 msgr2=0x7f80b0198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80aeffd640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b0102be0 0x7f80b0198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80aeffd640 1 -- 172.21.15.5:0/3754827055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80b010c0e0 msgr2=0x7f80b019cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80aeffd640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80b010c0e0 0x7f80b019cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80aeffd640 1 -- 172.21.15.5:0/3754827055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80b019d370 con 0x7f80b0103590 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80af7fe640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b0102be0 0x7f80b0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80aeffd640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0103590 0x7f80b01988c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f80a400d950 tx=0x7f80a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:30.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80acff9640 1 -- 172.21.15.5:0/3754827055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a4017030 con 0x7f80b0103590 2024-09-15T22:25:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80acff9640 1 -- 172.21.15.5:0/3754827055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80a4004020 con 0x7f80b0103590 2024-09-15T22:25:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80b5da6640 1 -- 172.21.15.5:0/3754827055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80b019d660 con 0x7f80b0103590 2024-09-15T22:25:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80acff9640 1 -- 172.21.15.5:0/3754827055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80a40047a0 con 0x7f80b0103590 2024-09-15T22:25:30.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.491+0000 7f80b5da6640 1 -- 172.21.15.5:0/3754827055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80b01a4ef0 con 0x7f80b0103590 2024-09-15T22:25:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.492+0000 7f80b5da6640 1 -- 172.21.15.5:0/3754827055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80b0102fc0 con 0x7f80b0103590 2024-09-15T22:25:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.493+0000 7f80acff9640 1 -- 172.21.15.5:0/3754827055 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f80a40041d0 con 0x7f80b0103590 2024-09-15T22:25:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.494+0000 7f80acff9640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80980a33c0 0x7f80980a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.494+0000 7f80acff9640 1 -- 172.21.15.5:0/3754827055 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f80a40cfba0 con 0x7f80b0103590 2024-09-15T22:25:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.494+0000 7f80af7fe640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80980a33c0 0x7f80980a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.495+0000 7f80af7fe640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80980a33c0 0x7f80980a5880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009c20 tx=0x7f80a0005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:30.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.498+0000 7f80acff9640 1 -- 172.21.15.5:0/3754827055 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80a409c120 con 0x7f80b0103590 2024-09-15T22:25:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.673+0000 7f80b5da6640 1 -- 172.21.15.5:0/3754827055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f80b002d7c0 con 0x7f80b0103590 2024-09-15T22:25:30.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.674+0000 7f80acff9640 1 -- 172.21.15.5:0/3754827055 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) ==== 104+0+11 (secure 0 0 0) 0x7f80a409ba20 con 0x7f80b0103590 2024-09-15T22:25:30.677 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-15T22:25:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 -- 172.21.15.5:0/3754827055 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80980a33c0 msgr2=0x7f80980a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80980a33c0 0x7f80980a5880 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009c20 tx=0x7f80a0005f90 comp rx=0 tx=0).stop 2024-09-15T22:25:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 -- 172.21.15.5:0/3754827055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0103590 msgr2=0x7f80b01988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0103590 0x7f80b01988c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f80a400d950 tx=0x7f80a400de10 comp rx=0 tx=0).stop 2024-09-15T22:25:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 -- 172.21.15.5:0/3754827055 shutdown_connections 2024-09-15T22:25:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f80980a33c0 0x7f80980a5880 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f80b010c0e0 0x7f80b019cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0103590 0x7f80b01988c0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 --2- 172.21.15.5:0/3754827055 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f80b0102be0 0x7f80b0198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 -- 172.21.15.5:0/3754827055 >> 172.21.15.5:0/3754827055 conn(0x7f80b00fc6a0 msgr2=0x7f80b00fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 -- 172.21.15.5:0/3754827055 shutdown_connections 2024-09-15T22:25:30.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.676+0000 7f80b5da6640 1 -- 172.21.15.5:0/3754827055 wait complete. 2024-09-15T22:25:30.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-15T22:25:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.795+0000 7fd6a6465640 1 -- 172.21.15.5:0/465256420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a010c0e0 msgr2=0x7fd6a010e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.795+0000 7fd6a6465640 1 --2- 172.21.15.5:0/465256420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a010c0e0 0x7fd6a010e530 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fd698009ca0 tx=0x7fd69802a8c0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.795+0000 7fd6a6465640 1 -- 172.21.15.5:0/465256420 shutdown_connections 2024-09-15T22:25:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.795+0000 7fd6a6465640 1 --2- 172.21.15.5:0/465256420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a010c0e0 0x7fd6a010e530 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.795+0000 7fd6a6465640 1 --2- 172.21.15.5:0/465256420 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6a0103590 0x7fd6a010ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.795+0000 7fd6a6465640 1 --2- 172.21.15.5:0/465256420 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0102be0 0x7fd6a0102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.795+0000 7fd6a6465640 1 -- 172.21.15.5:0/465256420 >> 172.21.15.5:0/465256420 conn(0x7fd6a00fc6a0 msgr2=0x7fd6a00feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.795+0000 7fd6a6465640 1 -- 172.21.15.5:0/465256420 shutdown_connections 2024-09-15T22:25:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.795+0000 7fd6a6465640 1 -- 172.21.15.5:0/465256420 wait complete. 2024-09-15T22:25:30.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.795+0000 7fd6a6465640 1 Processor -- start 2024-09-15T22:25:30.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.795+0000 7fd6a6465640 1 -- start start 2024-09-15T22:25:30.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a6465640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6a0102be0 0x7fd6a0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a6465640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0103590 0x7fd6a0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a6465640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a010c0e0 0x7fd6a019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a6465640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0111e90 con 0x7fd6a010c0e0 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a6465640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0111d10 con 0x7fd6a0103590 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a6465640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a0112010 con 0x7fd6a0102be0 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a4a02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a010c0e0 0x7fd6a019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a4a02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a010c0e0 0x7fd6a019cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53616/0 (socket says 172.21.15.5:53616) 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a4a02640 1 -- 172.21.15.5:0/3444082247 learned_addr learned my addr 172.21.15.5:0/3444082247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd69f7fe640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0103590 0x7fd6a0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd69ffff640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6a0102be0 0x7fd6a0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a4a02640 1 -- 172.21.15.5:0/3444082247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6a0102be0 msgr2=0x7fd6a0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a4a02640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6a0102be0 0x7fd6a0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a4a02640 1 -- 172.21.15.5:0/3444082247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0103590 msgr2=0x7fd6a0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a4a02640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0103590 0x7fd6a0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a4a02640 1 -- 172.21.15.5:0/3444082247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a019d330 con 0x7fd6a010c0e0 2024-09-15T22:25:30.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd6a4a02640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a010c0e0 0x7fd6a019cc00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd698031a20 tx=0x7fd698031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:30.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.796+0000 7fd69d7fa640 1 -- 172.21.15.5:0/3444082247 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd698003b70 con 0x7fd6a010c0e0 2024-09-15T22:25:30.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.797+0000 7fd6a6465640 1 -- 172.21.15.5:0/3444082247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a019d5c0 con 0x7fd6a010c0e0 2024-09-15T22:25:30.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.797+0000 7fd69d7fa640 1 -- 172.21.15.5:0/3444082247 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd698003d10 con 0x7fd6a010c0e0 2024-09-15T22:25:30.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.797+0000 7fd69d7fa640 1 -- 172.21.15.5:0/3444082247 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69802fc40 con 0x7fd6a010c0e0 2024-09-15T22:25:30.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.797+0000 7fd6a6465640 1 -- 172.21.15.5:0/3444082247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a01a4eb0 con 0x7fd6a010c0e0 2024-09-15T22:25:30.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.798+0000 7fd6a6465640 1 -- 172.21.15.5:0/3444082247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a0102fc0 con 0x7fd6a010c0e0 2024-09-15T22:25:30.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.799+0000 7fd69d7fa640 1 -- 172.21.15.5:0/3444082247 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd698004b40 con 0x7fd6a010c0e0 2024-09-15T22:25:30.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.799+0000 7fd69d7fa640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd68c0a33c0 0x7fd68c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:30.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.800+0000 7fd69d7fa640 1 -- 172.21.15.5:0/3444082247 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7fd6980f12b0 con 0x7fd6a010c0e0 2024-09-15T22:25:30.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.800+0000 7fd69ffff640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd68c0a33c0 0x7fd68c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:30.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.800+0000 7fd69ffff640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd68c0a33c0 0x7fd68c0a5880 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd6900095f0 tx=0x7fd690005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:30.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.803+0000 7fd69d7fa640 1 -- 172.21.15.5:0/3444082247 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6980bda30 con 0x7fd6a010c0e0 2024-09-15T22:25:30.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.982+0000 7fd6a6465640 1 -- 172.21.15.5:0/3444082247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd6a0107170 con 0x7fd6a010c0e0 2024-09-15T22:25:30.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.983+0000 7fd69d7fa640 1 -- 172.21.15.5:0/3444082247 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) ==== 100+0+11 (secure 0 0 0) 0x7fd6980bd330 con 0x7fd6a010c0e0 2024-09-15T22:25:30.985 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 -- 172.21.15.5:0/3444082247 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd68c0a33c0 msgr2=0x7fd68c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd68c0a33c0 0x7fd68c0a5880 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd6900095f0 tx=0x7fd690005f90 comp rx=0 tx=0).stop 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 -- 172.21.15.5:0/3444082247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a010c0e0 msgr2=0x7fd6a019cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a010c0e0 0x7fd6a019cc00 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fd698031a20 tx=0x7fd698031a50 comp rx=0 tx=0).stop 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 -- 172.21.15.5:0/3444082247 shutdown_connections 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd68c0a33c0 0x7fd68c0a5880 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6a010c0e0 0x7fd6a019cc00 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd6a0103590 0x7fd6a0198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 --2- 172.21.15.5:0/3444082247 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd6a0102be0 0x7fd6a0198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 -- 172.21.15.5:0/3444082247 >> 172.21.15.5:0/3444082247 conn(0x7fd6a00fc6a0 msgr2=0x7fd6a00fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 -- 172.21.15.5:0/3444082247 shutdown_connections 2024-09-15T22:25:30.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:30.985+0000 7fd6a6465640 1 -- 172.21.15.5:0/3444082247 wait complete. 2024-09-15T22:25:30.998 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-15T22:25:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.104+0000 7f31fc677640 1 -- 172.21.15.5:0/3157111749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f4106e70 msgr2=0x7f31f410e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.104+0000 7f31fc677640 1 --2- 172.21.15.5:0/3157111749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f4106e70 0x7f31f410e9f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009ca0 tx=0x7f31ec02a8c0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.104+0000 7f31fc677640 1 -- 172.21.15.5:0/3157111749 shutdown_connections 2024-09-15T22:25:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.104+0000 7f31fc677640 1 --2- 172.21.15.5:0/3157111749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f4106e70 0x7f31f410e9f0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.104+0000 7f31fc677640 1 --2- 172.21.15.5:0/3157111749 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31f406a110 0x7f31f4106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.104+0000 7f31fc677640 1 --2- 172.21.15.5:0/3157111749 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31f4069760 0x7f31f4069b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.104+0000 7f31fc677640 1 -- 172.21.15.5:0/3157111749 >> 172.21.15.5:0/3157111749 conn(0x7f31f40766f0 msgr2=0x7f31f4076b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.104+0000 7f31fc677640 1 -- 172.21.15.5:0/3157111749 shutdown_connections 2024-09-15T22:25:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.104+0000 7f31fc677640 1 -- 172.21.15.5:0/3157111749 wait complete. 2024-09-15T22:25:31.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31fc677640 1 Processor -- start 2024-09-15T22:25:31.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31fc677640 1 -- start start 2024-09-15T22:25:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31fc677640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31f4069760 0x7f31f4198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31fc677640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f4106e70 0x7f31f41988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31fc677640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31f419cc50 0x7f31f419d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31fc677640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4111ed0 con 0x7f31f4106e70 2024-09-15T22:25:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31fc677640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4111d50 con 0x7f31f419cc50 2024-09-15T22:25:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31fc677640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4112050 con 0x7f31f4069760 2024-09-15T22:25:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31f9c12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f4106e70 0x7f31f41988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31f9c12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f4106e70 0x7f31f41988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53640/0 (socket says 172.21.15.5:53640) 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31f9c12640 1 -- 172.21.15.5:0/566994300 learned_addr learned my addr 172.21.15.5:0/566994300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31fac14640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31f419cc50 0x7f31f419d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31fa413640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31f4069760 0x7f31f4198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31f9c12640 1 -- 172.21.15.5:0/566994300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31f4069760 msgr2=0x7f31f4198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31f9c12640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31f4069760 0x7f31f4198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31f9c12640 1 -- 172.21.15.5:0/566994300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31f419cc50 msgr2=0x7f31f419d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31f9c12640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31f419cc50 0x7f31f419d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31f9c12640 1 -- 172.21.15.5:0/566994300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f419d740 con 0x7f31f4106e70 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31fac14640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31f419cc50 0x7f31f419d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.105+0000 7f31f9c12640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f4106e70 0x7f31f41988d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e9c0 tx=0x7f31e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.106+0000 7f31eb7fe640 1 -- 172.21.15.5:0/566994300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e400cd80 con 0x7f31f4106e70 2024-09-15T22:25:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.106+0000 7f31eb7fe640 1 -- 172.21.15.5:0/566994300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e4005500 con 0x7f31f4106e70 2024-09-15T22:25:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.106+0000 7f31fc677640 1 -- 172.21.15.5:0/566994300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f419da30 con 0x7f31f4106e70 2024-09-15T22:25:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.106+0000 7f31eb7fe640 1 -- 172.21.15.5:0/566994300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4010690 con 0x7f31f4106e70 2024-09-15T22:25:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.106+0000 7f31fc677640 1 -- 172.21.15.5:0/566994300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f41a51d0 con 0x7f31f4106e70 2024-09-15T22:25:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.107+0000 7f31fc677640 1 -- 172.21.15.5:0/566994300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f4069b40 con 0x7f31f4106e70 2024-09-15T22:25:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.108+0000 7f31eb7fe640 1 -- 172.21.15.5:0/566994300 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f31e40046c0 con 0x7f31f4106e70 2024-09-15T22:25:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.108+0000 7f31eb7fe640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e00a33c0 0x7f31e00a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.109+0000 7f31fa413640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e00a33c0 0x7f31e00a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.109+0000 7f31eb7fe640 1 -- 172.21.15.5:0/566994300 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f31e40d0600 con 0x7f31f4106e70 2024-09-15T22:25:31.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.109+0000 7f31fa413640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e00a33c0 0x7f31e00a5880 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009aa0 tx=0x7f31f0005f90 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:31.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.113+0000 7f31eb7fe640 1 -- 172.21.15.5:0/566994300 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f31e409cb80 con 0x7f31f4106e70 2024-09-15T22:25:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.289+0000 7f31fc677640 1 -- 172.21.15.5:0/566994300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f31f406a110 con 0x7f31f4106e70 2024-09-15T22:25:31.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.289+0000 7f31eb7fe640 1 -- 172.21.15.5:0/566994300 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v109) ==== 103+0+11 (secure 0 0 0) 0x7f31e409c480 con 0x7f31f4106e70 2024-09-15T22:25:31.291 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-15T22:25:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 -- 172.21.15.5:0/566994300 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e00a33c0 msgr2=0x7f31e00a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e00a33c0 0x7f31e00a5880 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009aa0 tx=0x7f31f0005f90 comp rx=0 tx=0).stop 2024-09-15T22:25:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 -- 172.21.15.5:0/566994300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f4106e70 msgr2=0x7f31f41988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f4106e70 0x7f31f41988d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e9c0 tx=0x7f31e400ee80 comp rx=0 tx=0).stop 2024-09-15T22:25:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 -- 172.21.15.5:0/566994300 shutdown_connections 2024-09-15T22:25:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f31e00a33c0 0x7f31e00a5880 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f31f419cc50 0x7f31f419d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f4106e70 0x7f31f41988d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 --2- 172.21.15.5:0/566994300 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f31f4069760 0x7f31f4198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 -- 172.21.15.5:0/566994300 >> 172.21.15.5:0/566994300 conn(0x7f31f40766f0 msgr2=0x7f31f40ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:31.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 -- 172.21.15.5:0/566994300 shutdown_connections 2024-09-15T22:25:31.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.291+0000 7f31fc677640 1 -- 172.21.15.5:0/566994300 wait complete. 2024-09-15T22:25:31.302 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:31 smithi005 ceph-mon[30944]: pgmap v521: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-09-15T22:25:31.302 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:31 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1030926961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:25:31.302 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:31 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3533684839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:25:31.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:31 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3754827055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:25:31.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:31 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3444082247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:25:31.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-15T22:25:31.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:31 smithi099 ceph-mon[34892]: pgmap v521: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-09-15T22:25:31.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:31 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1030926961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:25:31.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:31 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3533684839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:25:31.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:31 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3754827055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:25:31.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:31 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3444082247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:25:31.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:31 smithi129 ceph-mon[35314]: pgmap v521: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-09-15T22:25:31.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:31 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1030926961' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:25:31.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:31 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3533684839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-15T22:25:31.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:31 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3754827055' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-15T22:25:31.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:31 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3444082247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-15T22:25:31.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.406+0000 7f3da2531640 1 -- 172.21.15.5:0/1118510424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c1069e0 msgr2=0x7f3d9c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:31.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.406+0000 7f3da2531640 1 --2- 172.21.15.5:0/1118510424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c1069e0 0x7f3d9c106dc0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009930 tx=0x7f3d8c02a420 comp rx=0 tx=0).stop 2024-09-15T22:25:31.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.406+0000 7f3da2531640 1 -- 172.21.15.5:0/1118510424 shutdown_connections 2024-09-15T22:25:31.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.406+0000 7f3da2531640 1 --2- 172.21.15.5:0/1118510424 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d9c101370 0x7f3d9c10d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.406+0000 7f3da2531640 1 --2- 172.21.15.5:0/1118510424 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d9c1009d0 0x7f3d9c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.406+0000 7f3da2531640 1 --2- 172.21.15.5:0/1118510424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c1069e0 0x7f3d9c106dc0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.406+0000 7f3da2531640 1 -- 172.21.15.5:0/1118510424 >> 172.21.15.5:0/1118510424 conn(0x7f3d9c0fc6b0 msgr2=0x7f3d9c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.406+0000 7f3da2531640 1 -- 172.21.15.5:0/1118510424 shutdown_connections 2024-09-15T22:25:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.406+0000 7f3da2531640 1 -- 172.21.15.5:0/1118510424 wait complete. 2024-09-15T22:25:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3da2531640 1 Processor -- start 2024-09-15T22:25:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3da2531640 1 -- start start 2024-09-15T22:25:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3da2531640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d9c1009d0 0x7f3d9c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3da2531640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c101370 0x7f3d9c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3da2531640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d9c1069e0 0x7f3d9c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3da2531640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c111f10 con 0x7f3d9c101370 2024-09-15T22:25:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3da2531640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c111d90 con 0x7f3d9c1009d0 2024-09-15T22:25:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3da2531640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9c112090 con 0x7f3d9c1069e0 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3d9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c101370 0x7f3d9c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3d9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c101370 0x7f3d9c198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53648/0 (socket says 172.21.15.5:53648) 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3d9b7fe640 1 -- 172.21.15.5:0/4130283035 learned_addr learned my addr 172.21.15.5:0/4130283035 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3d9bfff640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d9c1009d0 0x7f3d9c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3d9b7fe640 1 -- 172.21.15.5:0/4130283035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d9c1069e0 msgr2=0x7f3d9c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3da0ace640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d9c1069e0 0x7f3d9c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3d9b7fe640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d9c1069e0 0x7f3d9c19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3d9b7fe640 1 -- 172.21.15.5:0/4130283035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d9c1009d0 msgr2=0x7f3d9c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3d9b7fe640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d9c1009d0 0x7f3d9c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.407+0000 7f3d9b7fe640 1 -- 172.21.15.5:0/4130283035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9c19d4d0 con 0x7f3d9c101370 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.408+0000 7f3d9b7fe640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c101370 0x7f3d9c198a20 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000b4a0 tx=0x7f3d9000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:31.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.408+0000 7f3d997fa640 1 -- 172.21.15.5:0/4130283035 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90014070 con 0x7f3d9c101370 2024-09-15T22:25:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.408+0000 7f3d997fa640 1 -- 172.21.15.5:0/4130283035 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d90010af0 con 0x7f3d9c101370 2024-09-15T22:25:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.408+0000 7f3da2531640 1 -- 172.21.15.5:0/4130283035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d9c19d7c0 con 0x7f3d9c101370 2024-09-15T22:25:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.408+0000 7f3d997fa640 1 -- 172.21.15.5:0/4130283035 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90017d10 con 0x7f3d9c101370 2024-09-15T22:25:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.408+0000 7f3da2531640 1 -- 172.21.15.5:0/4130283035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d9c1a5050 con 0x7f3d9c101370 2024-09-15T22:25:31.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.409+0000 7f3da2531640 1 -- 172.21.15.5:0/4130283035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d9c100e30 con 0x7f3d9c101370 2024-09-15T22:25:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.410+0000 7f3d997fa640 1 -- 172.21.15.5:0/4130283035 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f3d90017070 con 0x7f3d9c101370 2024-09-15T22:25:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.411+0000 7f3d997fa640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d840a33c0 0x7f3d840a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.411+0000 7f3d997fa640 1 -- 172.21.15.5:0/4130283035 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f3d900cf440 con 0x7f3d9c101370 2024-09-15T22:25:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.411+0000 7f3d9bfff640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d840a33c0 0x7f3d840a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:31.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.411+0000 7f3d9bfff640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d840a33c0 0x7f3d840a5880 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009900 tx=0x7f3d8c0058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:31.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.415+0000 7f3d997fa640 1 -- 172.21.15.5:0/4130283035 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d9009bad0 con 0x7f3d9c101370 2024-09-15T22:25:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.638+0000 7f3da2531640 1 -- 172.21.15.5:0/4130283035 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f3d9c1089b0 con 0x7f3d9c101370 2024-09-15T22:25:31.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.646+0000 7f3d997fa640 1 -- 172.21.15.5:0/4130283035 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742862 (secure 0 0 0) 0x7f3d9009b3d0 con 0x7f3d9c101370 2024-09-15T22:25:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:25:31.657 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:25:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 26, 2024-09-15T22:25:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "active_gid": 15228, 2024-09-15T22:25:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "active_name": "x", 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addrs": { 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1448662997 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1448662997 2024-09-15T22:25:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addr": "172.21.15.5:6801/1448662997", 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "active_change": "2024-09-15T22:08:53.484377+0000", 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "active_mgr_features": 4540701547738038271, 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "available": true, 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "standbys": [ 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24878, 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "y", 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-09-15T22:25:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-15T22:25:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-15T22:25:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-15T22:25:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-15T22:25:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-15T22:25:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-15T22:25:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-15T22:25:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-15T22:25:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-15T22:25:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-15T22:25:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T22:25:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-15T22:25:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-15T22:25:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-15T22:25:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-15T22:25:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-15T22:25:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T22:25:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-15T22:25:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-15T22:25:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-15T22:25:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-15T22:25:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T22:25:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T22:25:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T22:25:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-15T22:25:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-15T22:25:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T22:25:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-15T22:25:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-15T22:25:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-15T22:25:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T22:25:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-15T22:25:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-15T22:25:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-15T22:25:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T22:25:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-15T22:25:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-15T22:25:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-15T22:25:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T22:25:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-15T22:25:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-15T22:25:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-15T22:25:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-15T22:25:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T22:25:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-15T22:25:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-15T22:25:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T22:25:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-15T22:25:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-15T22:25:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-15T22:25:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-15T22:25:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-15T22:25:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T22:25:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T22:25:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-15T22:25:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-15T22:25:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-15T22:25:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T22:25:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T22:25:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T22:25:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T22:25:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T22:25:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T22:25:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-15T22:25:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-15T22:25:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-15T22:25:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-15T22:25:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T22:25:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-15T22:25:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-15T22:25:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T22:25:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-15T22:25:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-15T22:25:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T22:25:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-15T22:25:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-15T22:25:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-15T22:25:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-15T22:25:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-15T22:25:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-15T22:25:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T22:25:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T22:25:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T22:25:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T22:25:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T22:25:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T22:25:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T22:25:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T22:25:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T22:25:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T22:25:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T22:25:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T22:25:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T22:25:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T22:25:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T22:25:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T22:25:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T22:25:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T22:25:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T22:25:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T22:25:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T22:25:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T22:25:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T22:25:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T22:25:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T22:25:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T22:25:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T22:25:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T22:25:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T22:25:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-15T22:25:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-15T22:25:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-15T22:25:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-15T22:25:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-15T22:25:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-15T22:25:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-15T22:25:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-15T22:25:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-15T22:25:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-15T22:25:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-15T22:25:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-15T22:25:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-15T22:25:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-15T22:25:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-15T22:25:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-15T22:25:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-15T22:25:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-15T22:25:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-15T22:25:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-15T22:25:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-15T22:25:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-15T22:25:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-15T22:25:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-15T22:25:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-15T22:25:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-15T22:25:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-15T22:25:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-15T22:25:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-15T22:25:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-15T22:25:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-15T22:25:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-15T22:25:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T22:25:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-15T22:25:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-15T22:25:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-15T22:25:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-15T22:25:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-15T22:25:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T22:25:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-15T22:25:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-15T22:25:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-15T22:25:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:25:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-15T22:25:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-15T22:25:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T22:25:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-15T22:25:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-15T22:25:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-15T22:25:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-15T22:25:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-15T22:25:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-15T22:25:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-15T22:25:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T22:25:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T22:25:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-15T22:25:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T22:25:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-15T22:25:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-15T22:25:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-15T22:25:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-15T22:25:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-15T22:25:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-15T22:25:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-15T22:25:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-15T22:25:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-15T22:25:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T22:25:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-15T22:25:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-15T22:25:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-15T22:25:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-15T22:25:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-15T22:25:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-15T22:25:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-15T22:25:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-15T22:25:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-15T22:25:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-15T22:25:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-15T22:25:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-15T22:25:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-15T22:25:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-15T22:25:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-15T22:25:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-15T22:25:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-15T22:25:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-15T22:25:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-15T22:25:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-15T22:25:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-15T22:25:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-15T22:25:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-15T22:25:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-15T22:25:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-15T22:25:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-15T22:25:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-15T22:25:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-15T22:25:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-15T22:25:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-15T22:25:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-15T22:25:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-15T22:25:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-15T22:25:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-15T22:25:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-15T22:25:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-15T22:25:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T22:25:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24881, 2024-09-15T22:25:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "z", 2024-09-15T22:25:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-09-15T22:25:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-15T22:25:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-15T22:25:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-15T22:25:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-15T22:25:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-15T22:25:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-15T22:25:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-15T22:25:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T22:25:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-15T22:25:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-15T22:25:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-15T22:25:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-15T22:25:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-15T22:25:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-15T22:25:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T22:25:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-15T22:25:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-15T22:25:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-15T22:25:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T22:25:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:25:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-15T22:25:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-15T22:25:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-15T22:25:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-15T22:25:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T22:25:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:25:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-15T22:25:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-15T22:25:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-15T22:25:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-15T22:25:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T22:25:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-15T22:25:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-15T22:25:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T22:25:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-15T22:25:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-15T22:25:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-15T22:25:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T22:25:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-15T22:25:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-15T22:25:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-15T22:25:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-15T22:25:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-15T22:25:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T22:25:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:25:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-15T22:25:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-15T22:25:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T22:25:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-15T22:25:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-15T22:25:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-15T22:25:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T22:25:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T22:25:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-15T22:25:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-15T22:25:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-15T22:25:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T22:25:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-15T22:25:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-15T22:25:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-15T22:25:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T22:25:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-15T22:25:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-15T22:25:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-15T22:25:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T22:25:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T22:25:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T22:25:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-15T22:25:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-15T22:25:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T22:25:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-15T22:25:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T22:25:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-15T22:25:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T22:25:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-15T22:25:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-15T22:25:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-15T22:25:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-15T22:25:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-15T22:25:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T22:25:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-15T22:25:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-15T22:25:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T22:25:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-15T22:25:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-15T22:25:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-15T22:25:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T22:25:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-15T22:25:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-15T22:25:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T22:25:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T22:25:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-15T22:25:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-15T22:25:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-15T22:25:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T22:25:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-15T22:25:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-15T22:25:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-15T22:25:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T22:25:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-15T22:25:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-15T22:25:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-15T22:25:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-15T22:25:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T22:25:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-15T22:25:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-15T22:25:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T22:25:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-15T22:25:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-15T22:25:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-15T22:25:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T22:25:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T22:25:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-15T22:25:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-15T22:25:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-15T22:25:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T22:25:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-15T22:25:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T22:25:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-15T22:25:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T22:25:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-15T22:25:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-15T22:25:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-15T22:25:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T22:25:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-15T22:25:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-15T22:25:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-15T22:25:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-15T22:25:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T22:25:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-15T22:25:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-15T22:25:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-15T22:25:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T22:25:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T22:25:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-15T22:25:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-15T22:25:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-15T22:25:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-15T22:25:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T22:25:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T22:25:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-15T22:25:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-15T22:25:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T22:25:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T22:25:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T22:25:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T22:25:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T22:25:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-15T22:25:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-15T22:25:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-15T22:25:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-15T22:25:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-15T22:25:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-15T22:25:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-15T22:25:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-15T22:25:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-15T22:25:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-15T22:25:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-15T22:25:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T22:25:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-15T22:25:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-15T22:25:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-15T22:25:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T22:25:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-15T22:25:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-15T22:25:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T22:25:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T22:25:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T22:25:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-15T22:25:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-15T22:25:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T22:25:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-15T22:25:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-15T22:25:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-15T22:25:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T22:25:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-15T22:25:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-15T22:25:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T22:25:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-15T22:25:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-15T22:25:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-15T22:25:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T22:25:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-15T22:25:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-15T22:25:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T22:25:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-15T22:25:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-15T22:25:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-15T22:25:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-15T22:25:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T22:25:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-15T22:25:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-15T22:25:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-15T22:25:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T22:25:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T22:25:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-15T22:25:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T22:25:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T22:25:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T22:25:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T22:25:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T22:25:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T22:25:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T22:25:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T22:25:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T22:25:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T22:25:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T22:25:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T22:25:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T22:25:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-15T22:25:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T22:25:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T22:25:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T22:25:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T22:25:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T22:25:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T22:25:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T22:25:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T22:25:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T22:25:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T22:25:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T22:25:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T22:25:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T22:25:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T22:25:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T22:25:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T22:25:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T22:25:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T22:25:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-15T22:25:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T22:25:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T22:25:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T22:25:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T22:25:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-15T22:25:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T22:25:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T22:25:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T22:25:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T22:25:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T22:25:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-15T22:25:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-15T22:25:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-15T22:25:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-15T22:25:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-15T22:25:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-15T22:25:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-15T22:25:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-15T22:25:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-15T22:25:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-15T22:25:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-15T22:25:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-15T22:25:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-15T22:25:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-15T22:25:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-15T22:25:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-15T22:25:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-15T22:25:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-15T22:25:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-15T22:25:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-15T22:25:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-15T22:25:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-15T22:25:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-15T22:25:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-15T22:25:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-15T22:25:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-15T22:25:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-15T22:25:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-15T22:25:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-15T22:25:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-15T22:25:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-15T22:25:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-15T22:25:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-15T22:25:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T22:25:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T22:25:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-15T22:25:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-15T22:25:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-15T22:25:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T22:25:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-15T22:25:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-15T22:25:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-15T22:25:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-15T22:25:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-15T22:25:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T22:25:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-15T22:25:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-15T22:25:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-15T22:25:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T22:25:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-15T22:25:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-15T22:25:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-15T22:25:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-15T22:25:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T22:25:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-15T22:25:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-15T22:25:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-15T22:25:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-15T22:25:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-15T22:25:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-15T22:25:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-15T22:25:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-15T22:25:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T22:25:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-15T22:25:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-15T22:25:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-15T22:25:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-15T22:25:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-15T22:25:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-15T22:25:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:25:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-15T22:25:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-15T22:25:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-15T22:25:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-15T22:25:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T22:25:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-15T22:25:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-15T22:25:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-15T22:25:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-15T22:25:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-15T22:25:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-15T22:25:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-15T22:25:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-15T22:25:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-15T22:25:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-15T22:25:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T22:25:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-15T22:25:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-15T22:25:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-15T22:25:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-15T22:25:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-15T22:25:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-15T22:25:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-15T22:25:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T22:25:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-15T22:25:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-15T22:25:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-15T22:25:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-15T22:25:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-15T22:25:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T22:25:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-15T22:25:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-15T22:25:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-15T22:25:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-15T22:25:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-15T22:25:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-15T22:25:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-15T22:25:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-15T22:25:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-15T22:25:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-15T22:25:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-15T22:25:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-15T22:25:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-15T22:25:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T22:25:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T22:25:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-15T22:25:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-15T22:25:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T22:25:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T22:25:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-15T22:25:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-15T22:25:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-15T22:25:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-15T22:25:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-15T22:25:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-15T22:25:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-15T22:25:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-15T22:25:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-15T22:25:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-15T22:25:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-15T22:25:32.566 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.566 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.566 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.566 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.569 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.570 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.572 INFO:teuthology.orchestra.run.smithi005.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T22:25:32.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T22:25:32.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:32.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.572 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.572 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T22:25:32.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-15T22:25:32.573 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.574 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.574 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.574 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.574 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.574 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.575 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.575 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.575 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.575 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.575 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.575 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.575 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.575 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.575 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.576 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.576 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.576 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.577 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.577 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.577 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.577 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-15T22:25:32.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.581 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.581 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T22:25:32.581 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.581 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.581 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.581 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.581 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.583 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.584 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.584 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.584 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.585 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.585 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.585 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.588 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-15T22:25:32.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-15T22:25:32.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-15T22:25:32.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.590 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.590 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.590 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.590 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.590 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.590 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-15T22:25:32.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-15T22:25:32.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.591 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.592 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.595 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.595 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.595 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-15T22:25:32.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-15T22:25:32.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.595 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.596 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.596 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.596 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.596 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.596 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-15T22:25:32.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-15T22:25:32.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.597 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.598 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.600 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-15T22:25:32.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-15T22:25:32.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.600 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.602 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.602 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.602 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-15T22:25:32.603 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.603 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.603 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.603 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-15T22:25:32.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-15T22:25:32.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-15T22:25:32.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.605 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.606 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.607 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.607 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.607 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.608 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.609 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.609 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.609 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.609 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.610 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.610 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.610 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.610 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.610 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.613 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.613 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.613 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.613 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.613 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.613 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.613 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.614 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.616 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.616 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.616 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.616 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.616 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.616 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.616 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.617 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.617 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-15T22:25:32.619 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.619 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.619 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.619 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.624 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.624 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-15T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.625 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-15T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-15T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.629 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.629 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.629 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.629 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.629 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.629 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.629 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.632 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.634 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.634 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.634 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.634 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.634 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.634 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-15T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-15T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-15T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-15T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-15T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.636 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-15T22:25:32.637 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.640 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T22:25:32.641 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.641 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-15T22:25:32.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-15T22:25:32.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.641 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-15T22:25:32.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T22:25:32.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.644 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.646 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.647 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-15T22:25:32.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-15T22:25:32.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-15T22:25:32.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-15T22:25:32.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.651 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.653 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.653 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.653 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.653 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.654 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-15T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-15T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.657 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-15T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-15T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-15T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-15T22:25:32.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.660 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-15T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.662 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.662 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.662 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.662 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.662 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.662 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.662 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.663 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.665 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-15T22:25:32.667 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.668 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.669 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.669 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.669 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.669 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.669 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.669 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.669 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.670 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.671 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-15T22:25:32.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-15T22:25:32.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-15T22:25:32.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T22:25:32.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-15T22:25:32.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T22:25:32.677 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.677 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.677 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-15T22:25:32.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T22:25:32.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.677 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.677 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-15T22:25:32.678 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.680 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-15T22:25:32.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-15T22:25:32.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.680 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.680 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.683 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.684 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.684 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.684 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.684 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.684 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.684 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.686 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.686 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.686 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.686 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.687 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.688 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.691 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-15T22:25:32.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-15T22:25:32.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T22:25:32.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.692 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.692 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.692 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.692 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.692 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: "modules": [ 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard", 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: "iostat", 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: "nfs", 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: "restful", 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_schedule" 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-15T22:25:32.693 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.695 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.696 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.696 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.696 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.696 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.696 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.696 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.696 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.697 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-15T22:25:32.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-15T22:25:32.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.704 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-15T22:25:32.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-15T22:25:32.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.709 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.710 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.710 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-15T22:25:32.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.710 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.710 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.710 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.712 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-15T22:25:32.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.712 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-15T22:25:32.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-15T22:25:32.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-15T22:25:32.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.713 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-15T22:25:32.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:25:32.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.713 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-15T22:25:32.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-15T22:25:32.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:32.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:32.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-15T22:25:32.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-15T22:25:32.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-15T22:25:32.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:25:32.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.715 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-15T22:25:32.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-15T22:25:32.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:32.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-15T22:25:32.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:32.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-15T22:25:32.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-15T22:25:32.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.716 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-15T22:25:32.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-15T22:25:32.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-15T22:25:32.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-15T22:25:32.717 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-15T22:25:32.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.717 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-15T22:25:32.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-15T22:25:32.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-15T22:25:32.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.719 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-15T22:25:32.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-15T22:25:32.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:32.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.720 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:32.720 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-15T22:25:32.720 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-15T22:25:32.720 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-15T22:25:32.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-15T22:25:32.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.721 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.722 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.724 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.725 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.725 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.725 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.725 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.725 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.725 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.725 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-15T22:25:32.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.729 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-15T22:25:32.729 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-15T22:25:32.729 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-15T22:25:32.729 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-15T22:25:32.729 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-15T22:25:32.729 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-15T22:25:32.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-15T22:25:32.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.732 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-15T22:25:32.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-15T22:25:32.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-15T22:25:32.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-15T22:25:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.736 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-15T22:25:32.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-15T22:25:32.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.736 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-15T22:25:32.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.738 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.739 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-15T22:25:32.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-15T22:25:32.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.739 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.739 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.739 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.739 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.739 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-15T22:25:32.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-15T22:25:32.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.740 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.740 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-15T22:25:32.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-15T22:25:32.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:32.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-15T22:25:32.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.742 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.742 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-15T22:25:32.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.742 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-15T22:25:32.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-15T22:25:32.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.744 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-15T22:25:32.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-15T22:25:32.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-15T22:25:32.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.745 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-15T22:25:32.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-15T22:25:32.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-15T22:25:32.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.747 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.747 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-15T22:25:32.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.749 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-15T22:25:32.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-15T22:25:32.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.749 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-15T22:25:32.749 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.749 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.749 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.750 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-15T22:25:32.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.750 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.750 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-15T22:25:32.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-15T22:25:32.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-15T22:25:32.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.753 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-15T22:25:32.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.753 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.753 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-15T22:25:32.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.757 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-15T22:25:32.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-15T22:25:32.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-15T22:25:32.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-15T22:25:32.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:32.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.760 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.761 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.762 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-15T22:25:32.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-15T22:25:32.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.762 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.762 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-15T22:25:32.762 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.762 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.762 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-15T22:25:32.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.764 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-15T22:25:32.764 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-15T22:25:32.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.764 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.764 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-15T22:25:32.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.765 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-15T22:25:32.765 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.765 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.765 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-15T22:25:32.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.767 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-15T22:25:32.767 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-15T22:25:32.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.767 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-15T22:25:32.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-15T22:25:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-15T22:25:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-15T22:25:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-15T22:25:32.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.770 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-15T22:25:32.770 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.770 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-15T22:25:32.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-15T22:25:32.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.771 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.771 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-15T22:25:32.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-15T22:25:32.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.775 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.777 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:32.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.777 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.777 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.777 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-15T22:25:32.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.777 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.778 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-15T22:25:32.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.781 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.781 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.781 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.781 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.781 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.781 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.781 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.781 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.781 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.782 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-15T22:25:32.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-15T22:25:32.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.782 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.782 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-15T22:25:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-15T22:25:32.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.784 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.785 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.786 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.786 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.786 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.788 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-15T22:25:32.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.788 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.791 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-15T22:25:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-15T22:25:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-15T22:25:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.792 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-15T22:25:32.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.793 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.793 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-15T22:25:32.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-15T22:25:32.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.796 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-15T22:25:32.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-15T22:25:32.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.796 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-15T22:25:32.796 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.796 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-15T22:25:32.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.799 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-15T22:25:32.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.801 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-15T22:25:32.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-15T22:25:32.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-15T22:25:32.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.802 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-15T22:25:32.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-15T22:25:32.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-15T22:25:32.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.803 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.803 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-15T22:25:32.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-15T22:25:32.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-15T22:25:32.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.805 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-15T22:25:32.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.807 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-15T22:25:32.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-15T22:25:32.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.808 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-15T22:25:32.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.808 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-15T22:25:32.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-15T22:25:32.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-15T22:25:32.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.810 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-15T22:25:32.810 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.811 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.812 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.812 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-15T22:25:32.812 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.812 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.812 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.812 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.813 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.814 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.814 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.814 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.814 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.816 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.817 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.817 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.817 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.818 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.818 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.818 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-15T22:25:32.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.820 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-15T22:25:32.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-15T22:25:32.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-15T22:25:32.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.820 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-15T22:25:32.820 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.821 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.821 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.821 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.821 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-15T22:25:32.821 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.821 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.821 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.821 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-15T22:25:32.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-15T22:25:32.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.822 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.823 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-15T22:25:32.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-15T22:25:32.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.824 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.824 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-15T22:25:32.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-15T22:25:32.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.825 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-15T22:25:32.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-15T22:25:32.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.827 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-15T22:25:32.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-15T22:25:32.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.828 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.828 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.828 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.828 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-15T22:25:32.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-15T22:25:32.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.829 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-15T22:25:32.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-15T22:25:32.829 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.830 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-15T22:25:32.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-15T22:25:32.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.833 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-15T22:25:32.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-15T22:25:32.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-15T22:25:32.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.836 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.839 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.839 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.839 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-15T22:25:32.839 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-15T22:25:32.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.839 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-15T22:25:32.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.841 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-15T22:25:32.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.842 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-15T22:25:32.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-15T22:25:32.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-15T22:25:32.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.843 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.843 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.844 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-15T22:25:32.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-15T22:25:32.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.844 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.844 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.844 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-15T22:25:32.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.848 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.851 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-15T22:25:32.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-15T22:25:32.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.851 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.851 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.851 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-15T22:25:32.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-15T22:25:32.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.855 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.855 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.855 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.855 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.856 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-15T22:25:32.856 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-15T22:25:32.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.856 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-15T22:25:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-15T22:25:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.860 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.860 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.860 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.860 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.860 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-15T22:25:32.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.864 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-15T22:25:32.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-15T22:25:32.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.864 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-15T22:25:32.864 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.864 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.864 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.866 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.867 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-15T22:25:32.867 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-15T22:25:32.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.867 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-15T22:25:32.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.869 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.869 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.871 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-15T22:25:32.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-15T22:25:32.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.871 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.871 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.871 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.871 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.872 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.872 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.872 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-15T22:25:32.872 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-15T22:25:32.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-15T22:25:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.876 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-15T22:25:32.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-15T22:25:32.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.876 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-15T22:25:32.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-15T22:25:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-15T22:25:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-15T22:25:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.878 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.879 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.879 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.880 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.880 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.880 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.880 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.880 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.880 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.880 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.883 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.883 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.883 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.883 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.883 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-15T22:25:32.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.886 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.886 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-15T22:25:32.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.888 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-15T22:25:32.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-15T22:25:32.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-15T22:25:32.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-15T22:25:32.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.891 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-15T22:25:32.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.892 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.892 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.892 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-15T22:25:32.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-15T22:25:32.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-15T22:25:32.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-15T22:25:32.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-15T22:25:32.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.898 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.899 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.899 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.899 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.899 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.899 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.899 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.899 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.900 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.900 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.904 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-15T22:25:32.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-15T22:25:32.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-15T22:25:32.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-15T22:25:32.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.906 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-15T22:25:32.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-15T22:25:32.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-15T22:25:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.908 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-15T22:25:32.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-15T22:25:32.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.910 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:32.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:32.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:32.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-15T22:25:32.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.911 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-15T22:25:32.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-15T22:25:32.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.915 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.916 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.916 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.916 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.917 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.917 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.917 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.917 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.917 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.917 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.917 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-15T22:25:32.919 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:32.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-15T22:25:32.923 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-15T22:25:32.923 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-15T22:25:32.923 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.923 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-15T22:25:32.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-15T22:25:32.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-15T22:25:32.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.924 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-15T22:25:32.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.924 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.924 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-15T22:25:32.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-15T22:25:32.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.925 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-15T22:25:32.925 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.925 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.925 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-15T22:25:32.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-15T22:25:32.926 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-15T22:25:32.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.928 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.929 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.929 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.929 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.932 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.932 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.932 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.932 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.934 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-15T22:25:32.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.942 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.943 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.946 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.948 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.949 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.953 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.954 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.959 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-15T22:25:32.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.964 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-15T22:25:32.967 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-15T22:25:32.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-15T22:25:32.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-15T22:25:32.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-15T22:25:32.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.974 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.975 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.976 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.978 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.979 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.982 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.987 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.991 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:32.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.997 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:32.998 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:32.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.002 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.004 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.005 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.006 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.008 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.009 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.010 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.016 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.018 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.020 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.021 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.025 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.026 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.028 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-15T22:25:33.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.036 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-15T22:25:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.040 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.041 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.044 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-15T22:25:33.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.047 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.049 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.051 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.052 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.054 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-15T22:25:33.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-15T22:25:33.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.059 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "secondary_zone_period_retry_limit": { 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.064 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-15T22:25:33.070 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.074 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.077 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.080 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.084 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.087 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.088 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-15T22:25:33.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.091 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.093 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.094 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.097 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.099 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.100 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.101 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.102 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.103 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.106 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-15T22:25:33.107 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.109 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.112 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.114 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.117 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-15T22:25:33.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.123 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.124 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.125 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-15T22:25:33.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.128 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.129 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-15T22:25:33.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-15T22:25:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.139 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.140 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.141 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.142 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.145 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.147 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.148 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.150 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.151 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.152 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.154 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-15T22:25:33.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-15T22:25:33.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.160 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-15T22:25:33.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "services": { 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard": "https://172.21.15.5:8443/" 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "always_on_modules": { 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "octopus": [ 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-15T22:25:33.165 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "pacific": [ 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-15T22:25:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "quincy": [ 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-15T22:25:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "reef": [ 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "squid": [ 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-15T22:25:33.168 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "last_failure_osd_epoch": 89, 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "active_clients": [ 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3051918257 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "libcephsqlite", 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3540802012 2024-09-15T22:25:33.170 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3283493139 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-15T22:25:33.171 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2947270351 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-15T22:25:33.172 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 4033207008 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 882939927 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-15T22:25:33.173 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:25:33.175 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:32 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/566994300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:25:33.175 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:32 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4130283035' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T22:25:33.175 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:32 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/566994300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:25:33.175 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:32 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4130283035' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T22:25:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.651+0000 7f3da2531640 1 -- 172.21.15.5:0/4130283035 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d840a33c0 msgr2=0x7f3d840a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.651+0000 7f3da2531640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d840a33c0 0x7f3d840a5880 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c009900 tx=0x7f3d8c0058d0 comp rx=0 tx=0).stop 2024-09-15T22:25:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.651+0000 7f3da2531640 1 -- 172.21.15.5:0/4130283035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c101370 msgr2=0x7f3d9c198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.651+0000 7f3da2531640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c101370 0x7f3d9c198a20 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000b4a0 tx=0x7f3d9000b960 comp rx=0 tx=0).stop 2024-09-15T22:25:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.652+0000 7f3da2531640 1 -- 172.21.15.5:0/4130283035 shutdown_connections 2024-09-15T22:25:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.652+0000 7f3da2531640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f3d840a33c0 0x7f3d840a5880 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.652+0000 7f3da2531640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3d9c1069e0 0x7f3d9c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.652+0000 7f3da2531640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d9c101370 0x7f3d9c198a20 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.652+0000 7f3da2531640 1 --2- 172.21.15.5:0/4130283035 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3d9c1009d0 0x7f3d9c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:33.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.652+0000 7f3da2531640 1 -- 172.21.15.5:0/4130283035 >> 172.21.15.5:0/4130283035 conn(0x7f3d9c0fc6b0 msgr2=0x7f3d9c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:33.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.652+0000 7f3da2531640 1 -- 172.21.15.5:0/4130283035 shutdown_connections 2024-09-15T22:25:33.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:31.652+0000 7f3da2531640 1 -- 172.21.15.5:0/4130283035 wait complete. 2024-09-15T22:25:33.182 DEBUG:tasks.check_counter:Getting stats from a 2024-09-15T22:25:33.183 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-15T22:25:33.183 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:32 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/566994300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-15T22:25:33.183 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:32 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4130283035' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-15T22:25:33.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:33 smithi099 ceph-mon[34892]: pgmap v522: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-15T22:25:33.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:33 smithi129 ceph-mon[35314]: pgmap v522: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-15T22:25:33.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:33 smithi005 ceph-mon[30944]: pgmap v522: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-15T22:25:33.457 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:34.021 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:25:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-09-15T22:25:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-09-15T22:25:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-09-15T22:25:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-09-15T22:25:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-15T22:25:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-15T22:25:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.001574829, 2024-09-15T22:25:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T22:25:34.021 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000300560, 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:25:34.022 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 318, 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 307, 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1614336, 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 189360, 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.054146314, 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.023195260, 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.043767474, 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:34.023 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 306, 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.021162159, 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000069157 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1614336, 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 189360 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:34.024 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 7, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 237, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3552, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 278064, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.018770929, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.011592396, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.004376137, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 235, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.027320126, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000116255 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:34.025 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 278064 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-15T22:25:34.026 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:34.027 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:34.028 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148748, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 32, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-15T22:25:34.029 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:34.030 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 321, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1569444, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 321, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-15T22:25:34.031 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 321, 2024-09-15T22:25:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1569444, 2024-09-15T22:25:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-15T22:25:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:34.032 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:34.032 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:25:34.737 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-15T22:25:34.738 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-15T22:25:34.738 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-15T22:25:34.738 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-15T22:25:34.738 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-15T22:25:34.738 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-15T22:25:34.738 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-15T22:25:34.738 DEBUG:tasks.check_counter:Getting stats from d 2024-09-15T22:25:34.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-15T22:25:34.997 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:35.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:35 smithi005 ceph-mon[30944]: pgmap v523: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-15T22:25:35.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:35 smithi099 ceph-mon[34892]: pgmap v523: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-15T22:25:35.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:35 smithi129 ceph-mon[35314]: pgmap v523: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 5.2 KiB/s wr, 6 op/s 2024-09-15T22:25:35.517 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:25:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-09-15T22:25:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-09-15T22:25:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-09-15T22:25:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-09-15T22:25:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-15T22:25:35.517 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.001323152, 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000250123, 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:35.518 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 323, 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 312, 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1615696, 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 192560, 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-15T22:25:35.519 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.061111741, 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.022470364, 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.054711814, 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 311, 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.033798618, 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000108677 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:35.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1615696, 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 192560 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 7, 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 242, 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3552, 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 278944, 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.020065987, 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.012994611, 2024-09-15T22:25:35.521 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.004506678, 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 240, 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.024965472, 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000104022 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:35.522 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 278944 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:35.523 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:35.524 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-15T22:25:35.525 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148714, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:35.526 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:35.527 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 326, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1570094, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 326, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 326, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1570094, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:35.528 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:35.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:35.529 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:25:36.315 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-15T22:25:36.316 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-15T22:25:36.316 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-15T22:25:36.316 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-15T22:25:36.316 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-15T22:25:36.316 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-15T22:25:36.316 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-15T22:25:36.316 DEBUG:tasks.check_counter:Getting stats from g 2024-09-15T22:25:36.316 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-15T22:25:36.575 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:37.067 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:25:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-09-15T22:25:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-09-15T22:25:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-09-15T22:25:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-09-15T22:25:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-15T22:25:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-15T22:25:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.001265034, 2024-09-15T22:25:37.068 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000136131, 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:37.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:25:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:25:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 324, 2024-09-15T22:25:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 311, 2024-09-15T22:25:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1617952, 2024-09-15T22:25:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 191088, 2024-09-15T22:25:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-15T22:25:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-15T22:25:37.070 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.063145517, 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.021724989, 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.055485010, 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 311, 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.032475115, 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000104421 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:37.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1617952, 2024-09-15T22:25:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 191088 2024-09-15T22:25:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 3, 2024-09-15T22:25:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 237, 2024-09-15T22:25:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 480, 2024-09-15T22:25:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 277968, 2024-09-15T22:25:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-15T22:25:37.072 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-15T22:25:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.017994856, 2024-09-15T22:25:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.012888302, 2024-09-15T22:25:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002422297, 2024-09-15T22:25:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 234, 2024-09-15T22:25:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.028289510, 2024-09-15T22:25:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000120895 2024-09-15T22:25:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:37.073 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-15T22:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 277968 2024-09-15T22:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-15T22:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-15T22:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-15T22:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:37.074 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:37.075 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:37.076 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-15T22:25:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:37.077 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148670, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-15T22:25:37.078 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-15T22:25:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-15T22:25:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-15T22:25:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-15T22:25:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-15T22:25:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:37.079 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-15T22:25:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-15T22:25:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-15T22:25:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-15T22:25:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 323, 2024-09-15T22:25:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1569704, 2024-09-15T22:25:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 323, 2024-09-15T22:25:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-15T22:25:37.080 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-15T22:25:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 323, 2024-09-15T22:25:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1569704, 2024-09-15T22:25:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-15T22:25:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:37.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:37.081 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:25:37.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:37 smithi099 ceph-mon[34892]: pgmap v524: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-15T22:25:37.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:37 smithi005 ceph-mon[30944]: pgmap v524: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-15T22:25:37.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:37 smithi129 ceph-mon[35314]: pgmap v524: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-15T22:25:37.831 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-15T22:25:37.831 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-15T22:25:37.831 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-15T22:25:37.831 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-15T22:25:37.831 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-15T22:25:37.831 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-15T22:25:37.831 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-15T22:25:37.831 DEBUG:tasks.check_counter:Getting stats from j 2024-09-15T22:25:37.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-15T22:25:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:38.605 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-15T22:25:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 325, 2024-09-15T22:25:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 314, 2024-09-15T22:25:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1616240, 2024-09-15T22:25:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 193824, 2024-09-15T22:25:38.605 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.058187237, 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.024072532, 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.046137759, 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 313, 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.035632042, 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000113840 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:38.606 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1616240, 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 193824 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 7, 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 243, 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3552, 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 279120, 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.021258229, 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.013564033, 2024-09-15T22:25:38.607 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.004670066, 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 241, 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.042597343, 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000176752 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-09-15T22:25:38.608 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 279120 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.000309658, 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000212745, 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:38.609 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:25:38.610 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:38.611 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:38.612 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-15T22:25:38.613 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148731, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 31, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:38.614 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:38.615 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 328, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1570354, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 328, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 328, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1570354, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-15T22:25:38.616 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-15T22:25:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:38.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-15T22:25:38.617 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-15T22:25:39.240 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-15T22:25:39.240 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-15T22:25:39.241 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-15T22:25:39.241 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-15T22:25:39.241 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-15T22:25:39.241 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-15T22:25:39.241 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-15T22:25:39.241 DEBUG:tasks.check_counter:Getting stats from b 2024-09-15T22:25:39.241 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-15T22:25:39.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:39 smithi099 ceph-mon[34892]: pgmap v525: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6 op/s 2024-09-15T22:25:39.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:39 smithi129 ceph-mon[35314]: pgmap v525: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6 op/s 2024-09-15T22:25:39.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:39 smithi005 ceph-mon[30944]: pgmap v525: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6 op/s 2024-09-15T22:25:39.573 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:25:40.609 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-15T22:25:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 11541, 2024-09-15T22:25:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 10408, 2024-09-15T22:25:40.609 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 3175724, 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 43398812, 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 245, 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 1.038441571, 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.257063519, 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.266202690, 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.448912971, 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 10406, 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.316850606, 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000030448 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:40.610 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2024, 2024-09-15T22:25:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000306776, 2024-09-15T22:25:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000151 2024-09-15T22:25:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 648640, 2024-09-15T22:25:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 196608 2024-09-15T22:25:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 9430, 2024-09-15T22:25:40.611 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 11043, 2024-09-15T22:25:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 4028421, 2024-09-15T22:25:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 21083212, 2024-09-15T22:25:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 10, 2024-09-15T22:25:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 7, 2024-09-15T22:25:40.612 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.613315369, 2024-09-15T22:25:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.226672385, 2024-09-15T22:25:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.248905365, 2024-09-15T22:25:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.137659879, 2024-09-15T22:25:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 11042, 2024-09-15T22:25:40.613 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.659044563, 2024-09-15T22:25:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000059685 2024-09-15T22:25:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5056, 2024-09-15T22:25:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001095169, 2024-09-15T22:25:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000216 2024-09-15T22:25:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.614 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1694128, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 686832 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 24104, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 29688, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 7033862, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 17980432, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 15, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 6, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 1.784958405, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.598579151, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.779265690, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.514645621, 2024-09-15T22:25:40.615 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 29686, 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2.116572570, 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000071298 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 29103, 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.008370421, 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000287 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 43120, 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 138272 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.616 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 22424, 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 7.786142920, 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000347223 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-09-15T22:25:40.617 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-15T22:25:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-15T22:25:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 718, 2024-09-15T22:25:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.033479459, 2024-09-15T22:25:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000046628 2024-09-15T22:25:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-09-15T22:25:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 4768, 2024-09-15T22:25:40.618 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 4768, 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4768, 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 50.214359715, 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.010531535 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 73, 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 7, 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-09-15T22:25:40.619 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 1985, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 149, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 1836, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 1773, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 1047, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 1047, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 2, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 7914, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 6682, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2024-09-15T22:25:40.620 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 300, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 22070, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 1020, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 1569748406, 2024-09-15T22:25:40.621 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 6, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 22070, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 763, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 22, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 741, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 21307, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 43, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 2824, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 4346, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:25:40.622 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 5633, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 2297, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 169, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 1157 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 84, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 126, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-09-15T22:25:40.623 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 385, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 231, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 154, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 15, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 15764, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 22, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-09-15T22:25:40.624 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-09-15T22:25:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 7, 2024-09-15T22:25:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-09-15T22:25:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-09-15T22:25:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-09-15T22:25:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-09-15T22:25:40.625 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 7938, 2024-09-15T22:25:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-09-15T22:25:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-09-15T22:25:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 7938, 2024-09-15T22:25:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-09-15T22:25:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-09-15T22:25:40.626 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 17, 2024-09-15T22:25:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-09-15T22:25:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-09-15T22:25:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 17, 2024-09-15T22:25:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-09-15T22:25:40.627 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-09-15T22:25:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-09-15T22:25:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-09-15T22:25:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 62054754, 2024-09-15T22:25:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2024-09-15T22:25:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-09-15T22:25:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3493, 2024-09-15T22:25:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 45.734428539, 2024-09-15T22:25:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.013093165 2024-09-15T22:25:40.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 1704, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 1957, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 253, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 96, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 105, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 9, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 1985, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 2567, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 582, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 1047, 2024-09-15T22:25:40.629 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 2771, 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 1724, 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 76160, 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 231712 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 5681, 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 4768, 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 363, 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1263, 2024-09-15T22:25:40.630 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 41.000694224, 2024-09-15T22:25:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.032462940 2024-09-15T22:25:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-09-15T22:25:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 196, 2024-09-15T22:25:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.060918842, 2024-09-15T22:25:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000310810 2024-09-15T22:25:40.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-09-15T22:25:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.632 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-09-15T22:25:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 154, 2024-09-15T22:25:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.318791582, 2024-09-15T22:25:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002070075 2024-09-15T22:25:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-09-15T22:25:40.633 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1572, 2024-09-15T22:25:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 3.245716638, 2024-09-15T22:25:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002064705 2024-09-15T22:25:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-09-15T22:25:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.634 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-09-15T22:25:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.635 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-09-15T22:25:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-09-15T22:25:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.637 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-09-15T22:25:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-09-15T22:25:40.639 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 16, 2024-09-15T22:25:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.002349191, 2024-09-15T22:25:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000146824 2024-09-15T22:25:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-09-15T22:25:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 37, 2024-09-15T22:25:40.640 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.126975669, 2024-09-15T22:25:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.003431774 2024-09-15T22:25:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-09-15T22:25:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.641 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-09-15T22:25:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 16, 2024-09-15T22:25:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.229797060, 2024-09-15T22:25:40.642 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.014362316 2024-09-15T22:25:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-09-15T22:25:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 816, 2024-09-15T22:25:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 2.747713190, 2024-09-15T22:25:40.643 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.003367295 2024-09-15T22:25:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-09-15T22:25:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 124, 2024-09-15T22:25:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.175091463, 2024-09-15T22:25:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001412027 2024-09-15T22:25:40.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-09-15T22:25:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 367, 2024-09-15T22:25:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 1.901970526, 2024-09-15T22:25:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.005182480 2024-09-15T22:25:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.645 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-09-15T22:25:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-09-15T22:25:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.646 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-09-15T22:25:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-09-15T22:25:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.015626917, 2024-09-15T22:25:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.007813458 2024-09-15T22:25:40.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-09-15T22:25:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 10, 2024-09-15T22:25:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.071652495, 2024-09-15T22:25:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.007165249 2024-09-15T22:25:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-09-15T22:25:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.648 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001184637, 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000592318 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 9, 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.022454332, 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.002494925 2024-09-15T22:25:40.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 184, 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.293422949, 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001594689 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.651 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 2, 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 7, 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 5, 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 2, 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 738, 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 369, 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 971, 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-09-15T22:25:40.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-15T22:25:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:40.653 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:40.654 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:40.655 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 848741, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 1339, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9720, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 198, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:40.656 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 6271647, 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 92005, 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-09-15T22:25:40.657 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 27271, 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 58945904, 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 37, 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 27271, 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 27271, 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 132.314013526, 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.004851821 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 27271, 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 27678 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.658 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 27271, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 22195, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 5076, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 11, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 103, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 3574, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 135, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 5, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 1001, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-09-15T22:25:40.659 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-09-15T22:25:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-09-15T22:25:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 22092, 2024-09-15T22:25:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 22, 2024-09-15T22:25:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:25:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-09-15T22:25:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:25:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-09-15T22:25:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-09-15T22:25:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-09-15T22:25:40.660 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:25:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-09-15T22:25:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:25:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-09-15T22:25:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-09-15T22:25:40.661 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-09-15T22:25:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-09-15T22:25:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-09-15T22:25:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-09-15T22:25:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-09-15T22:25:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-09-15T22:25:40.662 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-09-15T22:25:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-09-15T22:25:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-09-15T22:25:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-09-15T22:25:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-09-15T22:25:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-09-15T22:25:40.663 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-09-15T22:25:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-09-15T22:25:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-09-15T22:25:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 109, 2024-09-15T22:25:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-09-15T22:25:40.664 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 29, 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 12, 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 12, 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 0, 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 609, 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 0, 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 126 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:25:40.665 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 13676 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 24, 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 8, 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 474, 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 231, 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-09-15T22:25:40.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 45071, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 10083452, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 45071, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 45071, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 10083452, 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:40.667 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:40.668 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 27271, 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 58972099, 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 27271, 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 58972099, 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:40.669 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 27271, 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 27271, 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 27271, 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 27271, 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:25:40.670 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 231, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 26195, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 231, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 81, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 26195, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.671 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x55dd53c30500": { 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 7938, 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 57860450, 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 7938, 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3493, 2024-09-15T22:25:40.672 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 57860450, 2024-09-15T22:25:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:40.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:40.673 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-15T22:25:41.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:41 smithi099 ceph-mon[34892]: pgmap v526: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:25:41.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:41 smithi129 ceph-mon[35314]: pgmap v526: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:25:41.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:41 smithi005 ceph-mon[30944]: pgmap v526: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:25:41.551 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-15T22:25:41.551 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-15T22:25:41.551 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=84 2024-09-15T22:25:41.551 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-15T22:25:41.551 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-15T22:25:41.551 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=16 2024-09-15T22:25:41.551 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=10 2024-09-15T22:25:41.551 DEBUG:tasks.check_counter:Getting stats from e 2024-09-15T22:25:41.552 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-15T22:25:41.797 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:25:42.278 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-15T22:25:42.278 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:42.278 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 359, 2024-09-15T22:25:42.278 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 344, 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1662578, 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 198852, 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.055102562, 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.019913414, 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.049784607, 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000020391, 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 344, 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.023274103, 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000067657 2024-09-15T22:25:42.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1661744, 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 198528 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 1011, 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 1249, 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 248556, 2024-09-15T22:25:42.280 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 896131, 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 8, 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.098614601, 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.032039571, 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.022133549, 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.034478142, 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1246, 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.059750239, 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000047953 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.281 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 575264 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 2199, 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2201, 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 659621, 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 699340, 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.201779310, 2024-09-15T22:25:42.282 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.067733260, 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.052431808, 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.064489507, 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2199, 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.107152935, 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000048728 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:25:42.283 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5359, 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.141868090, 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000026472 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:42.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 0, 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 0, 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.285 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 0, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 0, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 0, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 0, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 0, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-09-15T22:25:42.286 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 1, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-09-15T22:25:42.287 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:25:42.288 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 0, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 0 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.289 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 0, 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 0, 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 0, 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-09-15T22:25:42.290 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-09-15T22:25:42.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 0, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 9, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 0, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-09-15T22:25:42.292 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4196802, 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 4196661, 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 9 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-09-15T22:25:42.293 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 3, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 3, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 0, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 1, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 1, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 0, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 0, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 0, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 0, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 0, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 0, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 0, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 48168, 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 215328 2024-09-15T22:25:42.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 0, 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 0, 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 0, 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.295 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.296 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-09-15T22:25:42.297 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.298 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-09-15T22:25:42.299 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.299 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.299 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.299 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-09-15T22:25:42.299 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.299 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.300 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.303 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-09-15T22:25:42.304 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-09-15T22:25:42.306 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 1, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 4, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 3, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 3, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 0, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 0, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 4823, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:42.307 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:42.308 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:42.309 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 190217, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 54, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-15T22:25:42.310 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9720, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 198, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 8512, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 13, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:42.311 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 3209, 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 21, 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 3, 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 3209, 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3209, 2024-09-15T22:25:42.312 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 14.074683141, 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.004386002 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3209, 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 3210 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 3209, 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 3209, 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 0, 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 2102, 2024-09-15T22:25:42.313 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 0, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 1105, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 0, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 0, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 0, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-09-15T22:25:42.314 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 1, 2024-09-15T22:25:42.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 0, 2024-09-15T22:25:42.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:25:42.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-09-15T22:25:42.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:25:42.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-09-15T22:25:42.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-09-15T22:25:42.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-09-15T22:25:42.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:25:42.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-09-15T22:25:42.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:25:42.315 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-09-15T22:25:42.316 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 109, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 23, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 5, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 7, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 2, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 0, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 2, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 0 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.317 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 0, 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 0 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-09-15T22:25:42.318 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 3565, 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 2228667, 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 3565, 2024-09-15T22:25:42.319 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3565, 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 2228667, 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-15T22:25:42.320 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3209, 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 2503, 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3209, 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 2503, 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 3209, 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 3209, 2024-09-15T22:25:42.322 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 3209, 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 3209, 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-15T22:25:42.323 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:42.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x55efb34426e0": { 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-15T22:25:42.325 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:42.326 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:42.326 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:42.326 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:42.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:42.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:42.326 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-15T22:25:43.064 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-15T22:25:43.065 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-15T22:25:43.065 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-15T22:25:43.065 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-15T22:25:43.065 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-15T22:25:43.065 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:25:43.065 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:25:43.065 DEBUG:tasks.check_counter:Getting stats from h 2024-09-15T22:25:43.065 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-15T22:25:43.308 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:25:43.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:43 smithi099 ceph-mon[34892]: pgmap v527: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:25:43.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:43 smithi129 ceph-mon[35314]: pgmap v527: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:25:43.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:43 smithi005 ceph-mon[30944]: pgmap v527: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:25:43.811 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-15T22:25:43.811 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 62, 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 266, 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 17413, 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 684784, 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 21, 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 8, 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.028591214, 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.008683861, 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.012007027, 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.001565048, 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 263, 2024-09-15T22:25:43.812 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.080211871, 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000304988 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 10, 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000001095, 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000109 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 539200 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 3495, 2024-09-15T22:25:43.813 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 2107, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1184854, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 1105084, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 48, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 4, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.209957752, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.090247598, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.149265495, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.005120104, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2106, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.102457069, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000048650 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.814 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 5277, 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000987614, 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000187 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 4464, 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 130528 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 602, 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 653, 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1730557, 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 349470, 2024-09-15T22:25:43.815 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 15, 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 5, 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.081475960, 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.028204798, 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.071634590, 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000239068, 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 652, 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.049314324, 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000075635 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 442, 2024-09-15T22:25:43.816 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000085523, 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000193 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1672848, 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 202496 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 1, 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-MDSRank": { 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 72, 2024-09-15T22:25:43.817 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.002070455, 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000028756 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "finisher-PurgeQueue": { 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "queue_len": 0, 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "complete_latency": { 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000020194, 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000010097 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "mds": { 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "request": 132, 2024-09-15T22:25:43.818 INFO:teuthology.orchestra.run.smithi099.stdout: "reply": 18, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "reply_latency": { 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 18, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.004645038, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000258057 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_reply": 0, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "forward": 114, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_complete": 0, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_fetch_keys": 0, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_commit": 11, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_split": 0, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_merge": 0, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes": 34, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_top": 9, 2024-09-15T22:25:43.819 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_bottom": 25, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pin_tail": 0, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_pinned": 19, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_expired": 4, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "inodes_with_caps": 4, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "caps": 4, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "subtrees": 3, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse": 235, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_hit": 100, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_forward": 113, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_discover": 22, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:25:43.820 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_remote_ino": 0, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "traverse_lock": 0, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "load_cent": 0, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "q": 0, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "exported": 0, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "exported_inodes": 0, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "imported": 0, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "imported_inodes": 0, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_dir_fetch": 0, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "openino_peer_discover": 0, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rfiles": 1022, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rbytes": 1570010530, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "root_rsnaps": 6, 2024-09-15T22:25:43.821 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_set_tag": 0, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "scrub_file_inodes": 0, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_revoke": 40, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_grant": 38, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:25:43.822 INFO:teuthology.orchestra.run.smithi099.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps": 54, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_cap_release": 8, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "process_request_cap_release": 4 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_cache": { 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update": 0, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_update_receipt": 106, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_try_discover": 64, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_send_discover": 63, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "dir_handle_discover": 0, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays": 0, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_delayed": 0, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:25:43.823 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_created": 0, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_enqueued": 0, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_reintegrated": 0, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "strays_migrated": 0, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_processing": 0, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_started": 0, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "recovery_completed": 0, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_path": 15, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_quiesce_inode": 330, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_exportdir": 0, 2024-09-15T22:25:43.824 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_flush": 0, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_fragstats": 0, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "ireq_inodestats": 0 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_log": { 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "evlrg": 0, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "evadd": 9, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "evex": 0, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "evtrm": 0, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "ev": 9, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "evexg": 0, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "evexd": 0, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "segadd": 2, 2024-09-15T22:25:43.825 INFO:teuthology.orchestra.run.smithi099.stdout: "segex": 0, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "segtrm": 0, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "seg": 2, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "segmjr": 2, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "segexg": 0, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "segexd": 0, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "expos": 4194304, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "wrpos": 4196802, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "rdpos": 0, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "jlat": { 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 8, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.069820794, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.008727599 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.826 INFO:teuthology.orchestra.run.smithi099.stdout: "replayed": 0 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_mem": { 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "ino": 38, 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "ino+": 40, 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "ino-": 2, 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "dir": 86, 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "dir+": 94, 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "dir-": 8, 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "dn": 34, 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "dn+": 38, 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "dn-": 4, 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "cap": 4, 2024-09-15T22:25:43.827 INFO:teuthology.orchestra.run.smithi099.stdout: "cap+": 19, 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "cap-": 15, 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "rss": 53104, 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "heap": 223520 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_server": { 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_client_request": 132, 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "dispatch_server_request": 0, 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_request": 132, 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_client_session": 26, 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "handle_peer_request": 0, 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "req_create_latency": { 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.828 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getattr_latency": { 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 1, 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000152910, 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000152910 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getfilelock_latency": { 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "req_link_latency": { 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.829 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookup_latency": { 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 11, 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.003142441, 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000285676 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookuphash_latency": { 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupino_latency": { 2024-09-15T22:25:43.830 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 2, 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000333421, 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000166710 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupname_latency": { 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupparent_latency": { 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.831 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lookupsnap_latency": { 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "req_lssnap_latency": { 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mkdir_latency": { 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mknod_latency": { 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "req_mksnap_latency": { 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "req_open_latency": { 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 4, 2024-09-15T22:25:43.833 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.001016266, 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000254066 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "req_readdir_latency": { 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rename_latency": { 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.834 INFO:teuthology.orchestra.run.smithi099.stdout: "req_renamesnap_latency": { 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "req_snapdiff_latency": { 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmdir_latency": { 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.835 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmsnap_latency": { 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "req_rmxattr_latency": { 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setattr_latency": { 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.836 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setdirlayout_latency": { 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setfilelock_latency": { 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setlayout_latency": { 2024-09-15T22:25:43.837 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "req_setxattr_latency": { 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "req_symlink_latency": { 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "req_unlink_latency": { 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "req_getvxattr_latency": { 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_sessions": { 2024-09-15T22:25:43.839 INFO:teuthology.orchestra.run.smithi099.stdout: "session_count": 1, 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "session_add": 4, 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "session_remove": 3, 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_open": 1, 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "sessions_stale": 0, 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "total_load": 4, 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "average_load": 4, 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "avg_session_uptime": 1008, 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "mdthresh_evicted": 0 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:43.840 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:43.841 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:43.842 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 272091, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 112, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:43.843 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 9720, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 198, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 232225, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 1106, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "objecter": { 2024-09-15T22:25:43.844 INFO:teuthology.orchestra.run.smithi099.stdout: "op_active": 0, 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "op_laggy": 0, 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send": 63, 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "op_send_bytes": 15403, 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "op_resend": 30, 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "op_reply": 63, 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "op_latency": { 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 63, 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.980977405, 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.015571069 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "op_inflight": 0, 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "oplen_avg": { 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 63, 2024-09-15T22:25:43.845 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 104 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "op": 63, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "op_r": 0, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "op_w": 63, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "op_rmw": 0, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "op_pg": 0, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_stat": 0, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_create": 10, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_read": 0, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_write": 8, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writefull": 7, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_writesame": 0, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_append": 0, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_zero": 0, 2024-09-15T22:25:43.846 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_truncate": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_delete": 6, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_mapext": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_sparse_read": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_clonerange": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_getxattr": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_setxattr": 20, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_rmxattr": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_call": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_watch": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_notify": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:25:43.847 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "osdop_other": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_active": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_send": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_resend": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "linger_ping": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_active": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_send": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "poolop_resend": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_active": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_send": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "poolstat_resend": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_active": 0, 2024-09-15T22:25:43.848 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_send": 0, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "statfs_resend": 0, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "command_active": 0, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "command_send": 0, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "command_resend": 0, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "map_epoch": 109, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "map_full": 0, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "map_inc": 23, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_sessions": 10, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_open": 16, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_session_close": 6, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_laggy": 0, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_wr": 44, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_rd": 0, 2024-09-15T22:25:43.849 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_del": 9 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "oft": { 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_objs": 1, 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_updates": 0, 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "omap_total_removes": 0 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "osd-slow-ops": { 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "slow_ops_count": 299 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "purge_queue": { 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops": 0, 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:25:43.850 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing": 0, 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executing_high_water": 0, 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed_ops": 0, 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_executed": 0, 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "pq_item_in_journal": 0 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 4155, 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 2535372, 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:43.851 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 4155, 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 4155, 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 2535372, 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_bytes": { 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-15T22:25:43.852 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 63, 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 15403, 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 63, 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 15403, 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:43.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-objecter_ops": { 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 1024, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 63, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 63, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 63, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 63, 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:43.854 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 0, 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 0, 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-15T22:25:43.855 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 0, 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 0, 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-write_buf_throttle-0x5621e8122500": { 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 3758096384, 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 9, 2024-09-15T22:25:43.856 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 2498, 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 9, 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 8, 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 2498, 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:43.857 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-15T22:25:44.587 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-15T22:25:44.587 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-15T22:25:44.587 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-15T22:25:44.587 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=106 2024-09-15T22:25:44.587 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-15T22:25:44.587 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:25:44.587 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:25:44.587 DEBUG:tasks.check_counter:Getting stats from k 2024-09-15T22:25:44.588 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-15T22:25:44.831 INFO:teuthology.orchestra.run.smithi099.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.b/config 2024-09-15T22:25:45.246 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:45 smithi099 ceph-mon[34892]: pgmap v528: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:25:45.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:45 smithi129 ceph-mon[35314]: pgmap v528: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:25:45.347 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-15T22:25:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 327, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 316, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 1616784, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 195120, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 4, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 0, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.053192170, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.020335596, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.046298543, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 315, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.028283819, 2024-09-15T22:25:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000089789 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 1616784, 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 195120 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 3, 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 244, 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 480, 2024-09-15T22:25:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 279200, 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 6, 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 3, 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.014221606, 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.009675116, 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.001883943, 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 241, 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.020861373, 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000086561 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 279200 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_messages": 4, 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages": 4, 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_bytes": 3072, 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_bytes": 448, 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_created_connections": 2, 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_active_connections": 1, 2024-09-15T22:25:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_total_time": 0.002324197, 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_send_time": 0.000036383, 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_recv_time": 0.002267746, 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 3, 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.004211890, 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.001403963 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-15T22:25:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: "msgr_send_encrypted_bytes": 448 2024-09-15T22:25:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: "cct": { 2024-09-15T22:25:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: "total_workers": 0, 2024-09-15T22:25:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: "unhealthy_workers": 0 2024-09-15T22:25:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: "mempool": { 2024-09-15T22:25:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_items": 0, 2024-09-15T22:25:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_bytes": 148692, 2024-09-15T22:25:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_anon_items": 30, 2024-09-15T22:25:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_bytes": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_items": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_bytes": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_items": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: "pgmap_items": 0, 2024-09-15T22:25:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_bytes": 928, 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "mds_co_items": 2, 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_1_items": 0, 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "unittest_2_items": 0 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "val": 0, 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "max": 104857600, 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "get_started": 0, 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "get": 330, 2024-09-15T22:25:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "get_sum": 1570614, 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "get_or_fail_success": 330, 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "take": 0, 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "take_sum": 0, 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "put": 330, 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "put_sum": 1570614, 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "wait": { 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "avgcount": 0, 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "sum": 0.000000000, 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "avgtime": 0.000000000 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-15T22:25:45.362 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-15T22:25:45.363 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:45 smithi005 ceph-mon[30944]: pgmap v528: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:25:46.065 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-09-15T22:25:46.065 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-09-15T22:25:46.066 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-09-15T22:25:46.066 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-09-15T22:25:46.066 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-15T22:25:46.066 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-15T22:25:46.066 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-15T22:25:46.066 DEBUG:tasks.check_counter:Getting stats from c 2024-09-15T22:25:46.066 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-15T22:25:46.383 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:25:47.236 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-15T22:25:47.236 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:47.236 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 2063, 2024-09-15T22:25:47.236 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 2050, 2024-09-15T22:25:47.236 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 23164229, 2024-09-15T22:25:47.236 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 741388, 2024-09-15T22:25:47.236 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 6, 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.226225483, 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.065847125, 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.105168088, 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.056616689, 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2049, 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.123766114, 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000060403 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.237 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 1658832, 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 198240 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 1119, 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 1355, 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 17089424, 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 1019394, 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 9, 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 6, 2024-09-15T22:25:47.238 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.109601173, 2024-09-15T22:25:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.034418696, 2024-09-15T22:25:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.035451201, 2024-09-15T22:25:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.030889311, 2024-09-15T22:25:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1353, 2024-09-15T22:25:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.078863305, 2024-09-15T22:25:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000058287 2024-09-15T22:25:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.239 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 3552, 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 665712 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 827, 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 829, 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 20321534, 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 262993, 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 5, 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 4, 2024-09-15T22:25:47.240 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.083191093, 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.022774383, 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.031274588, 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.023083483, 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 827, 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.497703784, 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000601818 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.241 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 1, 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-MDSRank": { 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 6121, 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.153004695, 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000024996 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:47.242 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-PurgeQueue": { 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: "mds": { 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: "request": 0, 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: "reply": 0, 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: "reply_latency": { 2024-09-15T22:25:47.243 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_reply": 0, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "forward": 0, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_complete": 0, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_keys": 0, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_commit": 0, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_split": 0, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_merge": 0, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes": 2283, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_top": 0, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_bottom": 2283, 2024-09-15T22:25:47.244 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pin_tail": 0, 2024-09-15T22:25:47.245 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pinned": 2283, 2024-09-15T22:25:47.245 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_expired": 0, 2024-09-15T22:25:47.245 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_with_caps": 0, 2024-09-15T22:25:47.245 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": 0, 2024-09-15T22:25:47.245 INFO:teuthology.orchestra.run.smithi129.stdout: "subtrees": 2, 2024-09-15T22:25:47.245 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse": 0, 2024-09-15T22:25:47.245 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_hit": 0, 2024-09-15T22:25:47.245 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_forward": 0, 2024-09-15T22:25:47.245 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_discover": 0, 2024-09-15T22:25:47.245 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:25:47.245 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_remote_ino": 0, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_lock": 0, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "load_cent": 0, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "q": 0, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "exported": 0, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "exported_inodes": 0, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "imported": 0, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "imported_inodes": 0, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_dir_fetch": 0, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_peer_discover": 0, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rfiles": 1020, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rbytes": 1569748406, 2024-09-15T22:25:47.246 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rsnaps": 6, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_set_tag": 0, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_file_inodes": 0, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:25:47.247 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps": 0, 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_cap_release": 0, 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "process_request_cap_release": 0 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache": { 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update": 0, 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update_receipt": 0, 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_try_discover": 0, 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_send_discover": 0, 2024-09-15T22:25:47.248 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_handle_discover": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_delayed": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_created": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_enqueued": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_reintegrated": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_migrated": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_processing": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_started": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_completed": 0, 2024-09-15T22:25:47.249 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_path": 0, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_inode": 0, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_exportdir": 0, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_flush": 0, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragstats": 0, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_inodestats": 0 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log": { 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "evlrg": 0, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "evadd": 0, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "evex": 0, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "evtrm": 0, 2024-09-15T22:25:47.250 INFO:teuthology.orchestra.run.smithi129.stdout: "ev": 7938, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "evexg": 0, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "evexd": 0, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "segadd": 0, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "segex": 0, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "segtrm": 0, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "seg": 17, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "segmjr": 2, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "segexg": 0, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "segexd": 0, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "expos": 4194304, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "wrpos": 62054754, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "rdpos": 61992336, 2024-09-15T22:25:47.251 INFO:teuthology.orchestra.run.smithi129.stdout: "jlat": { 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: "replayed": 7938 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_mem": { 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: "ino": 1704, 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: "ino+": 1935, 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: "ino-": 231, 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: "dir": 96, 2024-09-15T22:25:47.252 INFO:teuthology.orchestra.run.smithi129.stdout: "dir+": 105, 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "dir-": 9, 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "dn": 2283, 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "dn+": 2287, 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "dn-": 4, 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "cap": 0, 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "cap+": 0, 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "cap-": 0, 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "rss": 59536, 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "heap": 215328 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_server": { 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_client_request": 0, 2024-09-15T22:25:47.253 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_server_request": 0, 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_request": 0, 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_session": 0, 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_peer_request": 0, 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: "req_create_latency": { 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getattr_latency": { 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.254 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getfilelock_latency": { 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "req_link_latency": { 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookup_latency": { 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.255 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookuphash_latency": { 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupino_latency": { 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupname_latency": { 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.256 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupparent_latency": { 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupsnap_latency": { 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lssnap_latency": { 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.257 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.258 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.258 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mkdir_latency": { 2024-09-15T22:25:47.258 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.258 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.258 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.258 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.258 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mknod_latency": { 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mksnap_latency": { 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: "req_open_latency": { 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.259 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "req_readdir_latency": { 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rename_latency": { 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "req_renamesnap_latency": { 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.260 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "req_snapdiff_latency": { 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmdir_latency": { 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmsnap_latency": { 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.261 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmxattr_latency": { 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setattr_latency": { 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setdirlayout_latency": { 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.262 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setfilelock_latency": { 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setlayout_latency": { 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setxattr_latency": { 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.263 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "req_symlink_latency": { 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "req_unlink_latency": { 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getvxattr_latency": { 2024-09-15T22:25:47.264 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sessions": { 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: "session_count": 2, 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: "session_add": 7, 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: "session_remove": 5, 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_open": 5, 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_stale": 0, 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: "total_load": 0, 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: "average_load": 0, 2024-09-15T22:25:47.265 INFO:teuthology.orchestra.run.smithi129.stdout: "avg_session_uptime": 4159, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "mdthresh_evicted": 0 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:47.266 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:47.267 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 288124, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 2018, 2024-09-15T22:25:47.268 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:47.269 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:47.269 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-09-15T22:25:47.269 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-09-15T22:25:47.269 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:47.269 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:47.269 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:47.269 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:47.269 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 9720, 2024-09-15T22:25:47.269 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 198, 2024-09-15T22:25:47.269 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:47.272 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:47.272 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:47.272 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-09-15T22:25:47.272 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 6074445, 2024-09-15T22:25:47.272 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 86702, 2024-09-15T22:25:47.272 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:47.272 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-09-15T22:25:47.272 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:47.272 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-09-15T22:25:47.272 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.272 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter": { 2024-09-15T22:25:47.273 INFO:teuthology.orchestra.run.smithi129.stdout: "op_active": 0, 2024-09-15T22:25:47.273 INFO:teuthology.orchestra.run.smithi129.stdout: "op_laggy": 0, 2024-09-15T22:25:47.273 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send": 3647, 2024-09-15T22:25:47.273 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send_bytes": 26, 2024-09-15T22:25:47.273 INFO:teuthology.orchestra.run.smithi129.stdout: "op_resend": 4, 2024-09-15T22:25:47.273 INFO:teuthology.orchestra.run.smithi129.stdout: "op_reply": 3647, 2024-09-15T22:25:47.273 INFO:teuthology.orchestra.run.smithi129.stdout: "op_latency": { 2024-09-15T22:25:47.273 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 3647, 2024-09-15T22:25:47.273 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 25.625432355, 2024-09-15T22:25:47.273 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.007026441 2024-09-15T22:25:47.273 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "op_inflight": 0, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "oplen_avg": { 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 3647, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 3648 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "op": 3647, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "op_r": 3647, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "op_w": 0, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "op_rmw": 0, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "op_pg": 0, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_stat": 2176, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_create": 0, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_read": 1468, 2024-09-15T22:25:47.274 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_write": 0, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writefull": 0, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writesame": 0, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_append": 0, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_zero": 0, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_truncate": 0, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_delete": 0, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_mapext": 0, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_sparse_read": 0, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_clonerange": 0, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_getxattr": 2, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_setxattr": 0, 2024-09-15T22:25:47.275 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:25:47.276 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_rmxattr": 0, 2024-09-15T22:25:47.276 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:25:47.276 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_call": 0, 2024-09-15T22:25:47.276 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_watch": 0, 2024-09-15T22:25:47.276 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_notify": 0, 2024-09-15T22:25:47.276 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:25:47.276 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls": 0, 2024-09-15T22:25:47.276 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:25:47.276 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_other": 0, 2024-09-15T22:25:47.276 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_active": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_send": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_resend": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_ping": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_active": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_send": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_resend": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_active": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_send": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_resend": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_active": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_send": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_resend": 0, 2024-09-15T22:25:47.277 INFO:teuthology.orchestra.run.smithi129.stdout: "command_active": 0, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "command_send": 0, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "command_resend": 0, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "map_epoch": 109, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "map_full": 0, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "map_inc": 23, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_sessions": 8, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_open": 8, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_close": 0, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_laggy": 0, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_wr": 0, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_rd": 2, 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_del": 0 2024-09-15T22:25:47.278 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: "oft": { 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_objs": 0, 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_updates": 0, 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_removes": 0 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: "osd-slow-ops": { 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_ops_count": 0 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: "purge_queue": { 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops": 0, 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:25:47.279 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing": 0, 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_high_water": 0, 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed_ops": 0, 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed": 0, 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_item_in_journal": 0 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 4005, 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 60193410, 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:47.280 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 4005, 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 4005, 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 60193410, 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_bytes": { 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-09-15T22:25:47.281 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 3647, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 57897103, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 3647, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 57897103, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.282 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_ops": { 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 1024, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 3647, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 3647, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 3647, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 3647, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.283 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-09-15T22:25:47.284 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle-0x5611eed3a6e0": { 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:47.285 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:47.286 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-15T22:25:47.286 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-15T22:25:47.286 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-09-15T22:25:47.286 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-09-15T22:25:47.286 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:47.286 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:47.286 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:47.286 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:47.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:47.286 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:47.286 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-15T22:25:47.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:47 smithi099 ceph-mon[34892]: pgmap v529: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:25:47.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:47 smithi129 ceph-mon[35314]: pgmap v529: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:25:47.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:47 smithi005 ceph-mon[30944]: pgmap v529: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:25:48.230 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-15T22:25:48.230 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-15T22:25:48.230 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-15T22:25:48.230 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-15T22:25:48.230 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-15T22:25:48.230 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:25:48.231 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:25:48.231 DEBUG:tasks.check_counter:Getting stats from f 2024-09-15T22:25:48.231 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-15T22:25:48.479 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:25:48.973 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-15T22:25:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 3215, 2024-09-15T22:25:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 3217, 2024-09-15T22:25:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 910399, 2024-09-15T22:25:48.973 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 1022377, 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 6, 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.257261945, 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.076998171, 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.067901491, 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.094856471, 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 3215, 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.127619196, 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000039694 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:48.974 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 361, 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 346, 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 1662572, 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 199866, 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 3, 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 2, 2024-09-15T22:25:48.975 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.054708252, 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.020331792, 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.048419720, 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000093331, 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 346, 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.019900574, 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000057516 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.976 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 1662016, 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 199168 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 4, 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 242, 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 748, 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 665582, 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 7, 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 4, 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.015039018, 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.009030102, 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.003659769, 2024-09-15T22:25:48.977 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000025258, 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 239, 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.021766420, 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000091072 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 665264 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.978 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 1, 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-MDSRank": { 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 5307, 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.135399832, 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000025513 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-PurgeQueue": { 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-09-15T22:25:48.979 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "mds": { 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "request": 0, 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "reply": 0, 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "reply_latency": { 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.980 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_reply": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "forward": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_complete": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_keys": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_commit": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_split": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_merge": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_top": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_bottom": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pin_tail": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pinned": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_expired": 0, 2024-09-15T22:25:48.981 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_with_caps": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "subtrees": 1, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_hit": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_forward": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_discover": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_remote_ino": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_lock": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "load_cent": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "q": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "exported": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "exported_inodes": 0, 2024-09-15T22:25:48.982 INFO:teuthology.orchestra.run.smithi129.stdout: "imported": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "imported_inodes": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_dir_fetch": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_peer_discover": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rfiles": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rbytes": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rsnaps": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_set_tag": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:25:48.983 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_file_inodes": 0, 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_revoke": 0, 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_grant": 0, 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps": 0, 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_cap_release": 0, 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "process_request_cap_release": 0 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache": { 2024-09-15T22:25:48.984 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update_receipt": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_try_discover": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_send_discover": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_handle_discover": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_delayed": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_created": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_enqueued": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_reintegrated": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_migrated": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_processing": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:25:48.985 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_started": 0, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_completed": 0, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_path": 0, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_inode": 0, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_exportdir": 0, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_flush": 0, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragstats": 0, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_inodestats": 0 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log": { 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "evlrg": 0, 2024-09-15T22:25:48.986 INFO:teuthology.orchestra.run.smithi129.stdout: "evadd": 0, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "evex": 0, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "evtrm": 0, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "ev": 9, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "evexg": 0, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "evexd": 0, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "segadd": 0, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "segex": 0, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "segtrm": 0, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "seg": 2, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "segmjr": 2, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "segexg": 0, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "segexd": 0, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "expos": 4194304, 2024-09-15T22:25:48.987 INFO:teuthology.orchestra.run.smithi129.stdout: "wrpos": 4196802, 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "rdpos": 4196661, 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "jlat": { 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "replayed": 9 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_mem": { 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "ino": 3, 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "ino+": 3, 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "ino-": 0, 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "dir": 1, 2024-09-15T22:25:48.988 INFO:teuthology.orchestra.run.smithi129.stdout: "dir+": 1, 2024-09-15T22:25:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "dir-": 0, 2024-09-15T22:25:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "dn": 0, 2024-09-15T22:25:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "dn+": 0, 2024-09-15T22:25:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "dn-": 0, 2024-09-15T22:25:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "cap": 0, 2024-09-15T22:25:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "cap+": 0, 2024-09-15T22:25:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "cap-": 0, 2024-09-15T22:25:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "rss": 51008, 2024-09-15T22:25:48.989 INFO:teuthology.orchestra.run.smithi129.stdout: "heap": 215328 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_server": { 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_client_request": 0, 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_server_request": 0, 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_request": 0, 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_session": 0, 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_peer_request": 0, 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "req_create_latency": { 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getattr_latency": { 2024-09-15T22:25:48.990 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getfilelock_latency": { 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "req_link_latency": { 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookup_latency": { 2024-09-15T22:25:48.991 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookuphash_latency": { 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupino_latency": { 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.992 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupname_latency": { 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupparent_latency": { 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupsnap_latency": { 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.993 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lssnap_latency": { 2024-09-15T22:25:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mkdir_latency": { 2024-09-15T22:25:48.994 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mknod_latency": { 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mksnap_latency": { 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.995 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "req_open_latency": { 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "req_readdir_latency": { 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rename_latency": { 2024-09-15T22:25:48.996 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "req_renamesnap_latency": { 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "req_snapdiff_latency": { 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.997 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmdir_latency": { 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmsnap_latency": { 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmxattr_latency": { 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.998 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setattr_latency": { 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setdirlayout_latency": { 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setfilelock_latency": { 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:48.999 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setlayout_latency": { 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setxattr_latency": { 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.000 INFO:teuthology.orchestra.run.smithi129.stdout: "req_symlink_latency": { 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "req_unlink_latency": { 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getvxattr_latency": { 2024-09-15T22:25:49.001 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sessions": { 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "session_count": 1, 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "session_add": 4, 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "session_remove": 3, 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_open": 3, 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_stale": 0, 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "total_load": 0, 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "average_load": 0, 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "avg_session_uptime": 4787, 2024-09-15T22:25:49.002 INFO:teuthology.orchestra.run.smithi129.stdout: "mdthresh_evicted": 0 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:49.003 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:49.004 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 182111, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 48, 2024-09-15T22:25:49.005 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 9720, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 198, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-09-15T22:25:49.006 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 8512, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 13, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter": { 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "op_active": 0, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "op_laggy": 0, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send": 3218, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send_bytes": 21, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "op_resend": 3, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "op_reply": 3218, 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "op_latency": { 2024-09-15T22:25:49.007 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 3218, 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 29.648353459, 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.009213285 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "op_inflight": 0, 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "oplen_avg": { 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 3218, 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 3219 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "op": 3218, 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "op_r": 3218, 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "op_w": 0, 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "op_rmw": 0, 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "op_pg": 0, 2024-09-15T22:25:49.008 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_stat": 2108, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_create": 0, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_read": 1108, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_write": 0, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writefull": 0, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writesame": 0, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_append": 0, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_zero": 0, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_truncate": 0, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_delete": 0, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_mapext": 0, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_sparse_read": 0, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_clonerange": 0, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_getxattr": 1, 2024-09-15T22:25:49.009 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_setxattr": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_rmxattr": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_call": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_watch": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_notify": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_other": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_active": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_send": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_resend": 0, 2024-09-15T22:25:49.010 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_ping": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_active": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_send": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_resend": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_active": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_send": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_resend": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_active": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_send": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_resend": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "command_active": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "command_send": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "command_resend": 0, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "map_epoch": 109, 2024-09-15T22:25:49.011 INFO:teuthology.orchestra.run.smithi129.stdout: "map_full": 0, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "map_inc": 23, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_sessions": 4, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_open": 4, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_close": 0, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_laggy": 0, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_wr": 0, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_rd": 2, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_del": 0 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "oft": { 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_objs": 0, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_updates": 0, 2024-09-15T22:25:49.012 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_removes": 0 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "osd-slow-ops": { 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_ops_count": 0 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "purge_queue": { 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops": 0, 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing": 0, 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_high_water": 0, 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed_ops": 0, 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed": 0, 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_item_in_journal": 0 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.013 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 3576, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 2230546, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 3576, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 3576, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 2230546, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:49.014 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_bytes": { 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 3218, 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 2503, 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 3218, 2024-09-15T22:25:49.015 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 2503, 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_ops": { 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 1024, 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-09-15T22:25:49.016 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 3218, 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 3218, 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 3218, 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 3218, 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-09-15T22:25:49.017 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:49.018 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle-0x5614073986e0": { 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:49.019 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:49.020 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:49.020 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-15T22:25:49.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:49 smithi099 ceph-mon[34892]: pgmap v530: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-15T22:25:49.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:49 smithi129 ceph-mon[35314]: pgmap v530: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-15T22:25:49.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:49 smithi005 ceph-mon[30944]: pgmap v530: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-15T22:25:49.787 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-15T22:25:49.787 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-15T22:25:49.787 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-15T22:25:49.787 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-15T22:25:49.787 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-09-15T22:25:49.787 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:25:49.787 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:25:49.787 DEBUG:tasks.check_counter:Getting stats from i 2024-09-15T22:25:49.788 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-15T22:25:50.032 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:25:50.514 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-15T22:25:50.515 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:50.515 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 269, 2024-09-15T22:25:50.515 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 365, 2024-09-15T22:25:50.515 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 74253, 2024-09-15T22:25:50.516 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 142155, 2024-09-15T22:25:50.516 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 18, 2024-09-15T22:25:50.516 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 4, 2024-09-15T22:25:50.516 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.037903434, 2024-09-15T22:25:50.516 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.009975894, 2024-09-15T22:25:50.516 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.024071850, 2024-09-15T22:25:50.517 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000275164, 2024-09-15T22:25:50.517 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:50.517 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 360, 2024-09-15T22:25:50.517 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.020312148, 2024-09-15T22:25:50.517 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000056422 2024-09-15T22:25:50.517 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.518 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:50.518 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 526, 2024-09-15T22:25:50.518 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000141415, 2024-09-15T22:25:50.518 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000268 2024-09-15T22:25:50.518 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.518 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 3584, 2024-09-15T22:25:50.519 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 4400 2024-09-15T22:25:50.519 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.519 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:50.519 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 3481, 2024-09-15T22:25:50.519 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 2294, 2024-09-15T22:25:50.519 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 1186791, 2024-09-15T22:25:50.520 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 1178387, 2024-09-15T22:25:50.520 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 11, 2024-09-15T22:25:50.520 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 4, 2024-09-15T22:25:50.520 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.195186602, 2024-09-15T22:25:50.520 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.075039202, 2024-09-15T22:25:50.520 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.153998949, 2024-09-15T22:25:50.520 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.007574165, 2024-09-15T22:25:50.520 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:50.521 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2295, 2024-09-15T22:25:50.521 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.126582006, 2024-09-15T22:25:50.521 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000055155 2024-09-15T22:25:50.521 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.521 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:50.521 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 5251, 2024-09-15T22:25:50.521 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000939529, 2024-09-15T22:25:50.522 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000178 2024-09-15T22:25:50.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.522 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-15T22:25:50.522 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 578944 2024-09-15T22:25:50.522 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.522 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:50.522 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 438, 2024-09-15T22:25:50.522 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 424, 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 1677499, 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 744042, 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 52, 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 7, 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.073596421, 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.022870469, 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.063939477, 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000176418, 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 423, 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.025775044, 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000060933 2024-09-15T22:25:50.523 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 52, 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000006168, 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000118 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 1669248, 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 203856 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 1, 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-09-15T22:25:50.524 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-MDSRank": { 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 79, 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.001882345, 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000023827 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: "finisher-PurgeQueue": { 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: "queue_len": 0, 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: "complete_latency": { 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 2, 2024-09-15T22:25:50.525 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000014973, 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000007486 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: "mds": { 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: "request": 145, 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: "reply": 15, 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: "reply_latency": { 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 15, 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.005448838, 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000363255 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_reply": 0, 2024-09-15T22:25:50.526 INFO:teuthology.orchestra.run.smithi129.stdout: "forward": 130, 2024-09-15T22:25:50.527 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_complete": 0, 2024-09-15T22:25:50.527 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_fetch_keys": 0, 2024-09-15T22:25:50.527 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_commit": 11, 2024-09-15T22:25:50.527 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_split": 0, 2024-09-15T22:25:50.527 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_merge": 0, 2024-09-15T22:25:50.527 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes": 34, 2024-09-15T22:25:50.527 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_top": 9, 2024-09-15T22:25:50.527 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_bottom": 25, 2024-09-15T22:25:50.527 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pin_tail": 0, 2024-09-15T22:25:50.527 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_pinned": 21, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_expired": 4, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "inodes_with_caps": 5, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "caps": 5, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "subtrees": 3, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse": 256, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_hit": 102, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_forward": 129, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_discover": 25, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_dir_fetch": 0, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_remote_ino": 0, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "traverse_lock": 0, 2024-09-15T22:25:50.528 INFO:teuthology.orchestra.run.smithi129.stdout: "load_cent": 0, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "q": 0, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "exported": 0, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "exported_inodes": 0, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "imported": 0, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "imported_inodes": 0, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_dir_fetch": 0, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_backtrace_fetch": 0, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "openino_peer_discover": 0, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rfiles": 1022, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rbytes": 1570010530, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "root_rsnaps": 6, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_fetch": 0, 2024-09-15T22:25:50.529 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_set_tag": 0, 2024-09-15T22:25:50.530 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_backtrace_repaired": 0, 2024-09-15T22:25:50.530 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_inotable_repaired": 0, 2024-09-15T22:25:50.530 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_inodes": 0, 2024-09-15T22:25:50.530 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dir_base_inodes": 0, 2024-09-15T22:25:50.530 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_dirfrag_rstats": 0, 2024-09-15T22:25:50.530 INFO:teuthology.orchestra.run.smithi129.stdout: "scrub_file_inodes": 0, 2024-09-15T22:25:50.530 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_inode_file_caps": 0, 2024-09-15T22:25:50.530 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_revoke": 60, 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_grant": 60, 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_trunc": 0, 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps": 73, 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_caps_dirty": 0, 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_cap_release": 8, 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "process_request_cap_release": 5 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_cache": { 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update": 0, 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_update_receipt": 109, 2024-09-15T22:25:50.531 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_try_discover": 67, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_send_discover": 63, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "dir_handle_discover": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_delayed": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "num_strays_enqueuing": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_created": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_enqueued": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_reintegrated": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "strays_migrated": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_processing": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_enqueued": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "num_recovering_prioritized": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_started": 0, 2024-09-15T22:25:50.532 INFO:teuthology.orchestra.run.smithi129.stdout: "recovery_completed": 0, 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_path": 15, 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_quiesce_inode": 330, 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_enqueue_scrub": 0, 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_exportdir": 0, 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_flush": 0, 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragmentdir": 0, 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_fragstats": 0, 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "ireq_inodestats": 0 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_log": { 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "evlrg": 0, 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "evadd": 9, 2024-09-15T22:25:50.533 INFO:teuthology.orchestra.run.smithi129.stdout: "evex": 0, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "evtrm": 0, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "ev": 9, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "evexg": 0, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "evexd": 0, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "segadd": 2, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "segex": 0, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "segtrm": 0, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "seg": 2, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "segmjr": 2, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "segexg": 0, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "segexd": 0, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "expos": 4194304, 2024-09-15T22:25:50.534 INFO:teuthology.orchestra.run.smithi129.stdout: "wrpos": 4196802, 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: "rdpos": 0, 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: "jlat": { 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 8, 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.017150945, 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.002143868 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: "replayed": 0 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_mem": { 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: "ino": 38, 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: "ino+": 40, 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: "ino-": 2, 2024-09-15T22:25:50.535 INFO:teuthology.orchestra.run.smithi129.stdout: "dir": 86, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "dir+": 94, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "dir-": 8, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "dn": 34, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "dn+": 38, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "dn-": 4, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "cap": 5, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "cap+": 17, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "cap-": 12, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "rss": 53380, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "heap": 223520 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_server": { 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_client_request": 145, 2024-09-15T22:25:50.536 INFO:teuthology.orchestra.run.smithi129.stdout: "dispatch_server_request": 0, 2024-09-15T22:25:50.537 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_request": 145, 2024-09-15T22:25:50.537 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_client_session": 26, 2024-09-15T22:25:50.537 INFO:teuthology.orchestra.run.smithi129.stdout: "handle_peer_request": 0, 2024-09-15T22:25:50.537 INFO:teuthology.orchestra.run.smithi129.stdout: "req_create_latency": { 2024-09-15T22:25:50.537 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.537 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.537 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.537 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.537 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getattr_latency": { 2024-09-15T22:25:50.537 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1, 2024-09-15T22:25:50.537 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000256820, 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000256820 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getfilelock_latency": { 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: "req_link_latency": { 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.538 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookup_latency": { 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 13, 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.004815409, 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000370416 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookuphash_latency": { 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupino_latency": { 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.539 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupname_latency": { 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupparent_latency": { 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lookupsnap_latency": { 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.540 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.541 INFO:teuthology.orchestra.run.smithi129.stdout: "req_lssnap_latency": { 2024-09-15T22:25:50.541 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.541 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.541 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.541 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.541 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mkdir_latency": { 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mknod_latency": { 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: "req_mksnap_latency": { 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.542 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: "req_open_latency": { 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 1, 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000376609, 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000376609 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: "req_readdir_latency": { 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rename_latency": { 2024-09-15T22:25:50.543 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: "req_renamesnap_latency": { 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: "req_snapdiff_latency": { 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.544 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmdir_latency": { 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmsnap_latency": { 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "req_rmxattr_latency": { 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.545 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setattr_latency": { 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setdirlayout_latency": { 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setfilelock_latency": { 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.546 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setlayout_latency": { 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: "req_setxattr_latency": { 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: "req_symlink_latency": { 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.547 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: "req_unlink_latency": { 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_revoke_eviction": 0, 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: "cap_acquisition_throttle": 0, 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: "req_getvxattr_latency": { 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:50.548 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_sessions": { 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "session_count": 1, 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "session_add": 4, 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "session_remove": 3, 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_open": 1, 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "sessions_stale": 0, 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "total_load": 4, 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "average_load": 4, 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "avg_session_uptime": 983, 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "mdthresh_evicted": 0 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-09-15T22:25:50.549 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:50.550 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:50.551 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 263867, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 106, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:50.552 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 9720, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 198, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 232553, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 1103, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-09-15T22:25:50.553 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "objecter": { 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "op_active": 0, 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "op_laggy": 0, 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send": 65, 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "op_send_bytes": 15593, 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "op_resend": 29, 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "op_reply": 65, 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "op_latency": { 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 65, 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.546987836, 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.008415197 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "op_inflight": 0, 2024-09-15T22:25:50.554 INFO:teuthology.orchestra.run.smithi129.stdout: "oplen_avg": { 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 65, 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 107 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "op": 65, 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "op_r": 0, 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "op_w": 65, 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "op_rmw": 0, 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "op_pg": 0, 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_stat": 0, 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_create": 10, 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_read": 0, 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_write": 8, 2024-09-15T22:25:50.555 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writefull": 8, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_writesame": 0, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_append": 0, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_zero": 0, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_truncate": 0, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_delete": 6, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_mapext": 0, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_sparse_read": 0, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_clonerange": 0, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_getxattr": 0, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_setxattr": 20, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_cmpxattr": 0, 2024-09-15T22:25:50.556 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_rmxattr": 0, 2024-09-15T22:25:50.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_resetxattrs": 0, 2024-09-15T22:25:50.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_call": 0, 2024-09-15T22:25:50.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_watch": 0, 2024-09-15T22:25:50.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_notify": 0, 2024-09-15T22:25:50.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_src_cmpxattr": 0, 2024-09-15T22:25:50.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls": 0, 2024-09-15T22:25:50.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_pgls_filter": 0, 2024-09-15T22:25:50.557 INFO:teuthology.orchestra.run.smithi129.stdout: "osdop_other": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_active": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_send": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_resend": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "linger_ping": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_active": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_send": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "poolop_resend": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_active": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_send": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "poolstat_resend": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_active": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_send": 0, 2024-09-15T22:25:50.558 INFO:teuthology.orchestra.run.smithi129.stdout: "statfs_resend": 0, 2024-09-15T22:25:50.559 INFO:teuthology.orchestra.run.smithi129.stdout: "command_active": 0, 2024-09-15T22:25:50.559 INFO:teuthology.orchestra.run.smithi129.stdout: "command_send": 0, 2024-09-15T22:25:50.559 INFO:teuthology.orchestra.run.smithi129.stdout: "command_resend": 0, 2024-09-15T22:25:50.559 INFO:teuthology.orchestra.run.smithi129.stdout: "map_epoch": 109, 2024-09-15T22:25:50.559 INFO:teuthology.orchestra.run.smithi129.stdout: "map_full": 0, 2024-09-15T22:25:50.559 INFO:teuthology.orchestra.run.smithi129.stdout: "map_inc": 23, 2024-09-15T22:25:50.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_sessions": 9, 2024-09-15T22:25:50.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_open": 14, 2024-09-15T22:25:50.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_session_close": 5, 2024-09-15T22:25:50.559 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_laggy": 0, 2024-09-15T22:25:50.559 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_wr": 46, 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_rd": 0, 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_del": 9 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: "oft": { 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_objs": 1, 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_kv_pairs": 0, 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_updates": 0, 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: "omap_total_removes": 0 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: "osd-slow-ops": { 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: "slow_ops_count": 299 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: "purge_queue": { 2024-09-15T22:25:50.560 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops": 0, 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_ops_high_water": 0, 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing": 0, 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executing_high_water": 0, 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed_ops": 0, 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_executed": 0, 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "pq_item_in_journal": 0 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:50.561 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 4184, 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 2538653, 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 4184, 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 4184, 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 2538653, 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.562 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_bytes": { 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 65, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 15593, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 65, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 15593, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.563 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-objecter_ops": { 2024-09-15T22:25:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 1024, 2024-09-15T22:25:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-09-15T22:25:50.564 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 65, 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 65, 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 65, 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 65, 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.565 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle": { 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 0, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 0, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 0, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 0, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 0, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.566 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-write_buf_throttle-0x56450630e500": { 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 3758096384, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 9, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 2498, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 9, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 8, 2024-09-15T22:25:50.567 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 2498, 2024-09-15T22:25:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:50.568 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:50.568 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-15T22:25:51.276 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-15T22:25:51.276 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-15T22:25:51.276 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-15T22:25:51.276 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=109 2024-09-15T22:25:51.276 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-15T22:25:51.276 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-15T22:25:51.276 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-15T22:25:51.276 DEBUG:tasks.check_counter:Getting stats from l 2024-09-15T22:25:51.276 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-15T22:25:51.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:51 smithi099 ceph-mon[34892]: pgmap v531: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:25:51.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:51 smithi129 ceph-mon[35314]: pgmap v531: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:25:51.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:51 smithi005 ceph-mon[30944]: pgmap v531: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-15T22:25:51.521 INFO:teuthology.orchestra.run.smithi129.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.c/config 2024-09-15T22:25:52.025 INFO:teuthology.orchestra.run.smithi129.stdout:{ 2024-09-15T22:25:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-0": { 2024-09-15T22:25:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 334, 2024-09-15T22:25:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 321, 2024-09-15T22:25:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 1620672, 2024-09-15T22:25:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 197488, 2024-09-15T22:25:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 4, 2024-09-15T22:25:52.025 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 0, 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.054337263, 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.019810429, 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.049495574, 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 321, 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.027495163, 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000085654 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:52.026 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 1620672, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 197488 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-1": { 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 3, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 245, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 480, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 279376, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 6, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 3, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.014729493, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.009811677, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.001363347, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:52.027 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 242, 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.023574266, 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000097414 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 279376 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:52.028 INFO:teuthology.orchestra.run.smithi129.stdout: "AsyncMessenger::Worker-2": { 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_messages": 0, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages": 2, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_bytes": 0, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_bytes": 0, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_created_connections": 2, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_active_connections": 1, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_total_time": 0.000311026, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_send_time": 0.000000000, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_recv_time": 0.000218943, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_messages_queue_lat": { 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:52.029 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_handle_ack_lat": { 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "msgr_send_encrypted_bytes": 0 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "cct": { 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "total_workers": 0, 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "unhealthy_workers": 0 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "mempool": { 2024-09-15T22:25:52.030 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_bytes": 0, 2024-09-15T22:25:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "bloom_filter_items": 0, 2024-09-15T22:25:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_bytes": 0, 2024-09-15T22:25:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_alloc_items": 0, 2024-09-15T22:25:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_bytes": 0, 2024-09-15T22:25:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_data_items": 0, 2024-09-15T22:25:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-15T22:25:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_onode_items": 0, 2024-09-15T22:25:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-15T22:25:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_meta_items": 0, 2024-09-15T22:25:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_bytes": 0, 2024-09-15T22:25:52.031 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_other_items": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_cache_buffer_items": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_bytes": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_extent_items": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_bytes": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_blob_items": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_shared_blob_items": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_inline_bl_items": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_bytes": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_fsck_items": 0, 2024-09-15T22:25:52.032 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_bytes": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_txc_items": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_deferred_items": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_bytes": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "bluestore_writing_items": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_bytes": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_items": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_bytes": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_reader_items": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_bytes": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "bluefs_file_writer_items": 0, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_bytes": 148721, 2024-09-15T22:25:52.033 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_anon_items": 30, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_bytes": 0, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "buffer_meta_items": 0, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_bytes": 0, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_items": 0, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_bytes": 0, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_mapbl_items": 0, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_bytes": 0, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "osd_pglog_items": 0, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_bytes": 0, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_items": 0, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_bytes": 0, 2024-09-15T22:25:52.034 INFO:teuthology.orchestra.run.smithi129.stdout: "osdmap_mapping_items": 0, 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_bytes": 0, 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "pgmap_items": 0, 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_bytes": 928, 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "mds_co_items": 2, 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_bytes": 0, 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_1_items": 0, 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_bytes": 0, 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "unittest_2_items": 0 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: }, 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "val": 0, 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "max": 104857600, 2024-09-15T22:25:52.035 INFO:teuthology.orchestra.run.smithi129.stdout: "get_started": 0, 2024-09-15T22:25:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "get": 333, 2024-09-15T22:25:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "get_sum": 1571004, 2024-09-15T22:25:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_fail": 0, 2024-09-15T22:25:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "get_or_fail_success": 333, 2024-09-15T22:25:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "take": 0, 2024-09-15T22:25:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "take_sum": 0, 2024-09-15T22:25:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "put": 333, 2024-09-15T22:25:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "put_sum": 1571004, 2024-09-15T22:25:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "wait": { 2024-09-15T22:25:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "avgcount": 0, 2024-09-15T22:25:52.036 INFO:teuthology.orchestra.run.smithi129.stdout: "sum": 0.000000000, 2024-09-15T22:25:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: "avgtime": 0.000000000 2024-09-15T22:25:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:52.037 INFO:teuthology.orchestra.run.smithi129.stdout: } 2024-09-15T22:25:52.037 INFO:teuthology.orchestra.run.smithi129.stdout:} 2024-09-15T22:25:52.772 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-15T22:25:52.773 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-15T22:25:52.773 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-15T22:25:52.773 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-15T22:25:52.773 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-15T22:25:52.773 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-15T22:25:52.773 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-15T22:25:52.773 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/check_counter.py", line 125, in end raise RuntimeError("The following counters failed to be set " RuntimeError: The following counters failed to be set on mds daemons: {'mds.imported', 'mds.exported'} 2024-09-15T22:25:52.774 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-15T22:25:52.783 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-15T22:25:52.783 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T22:25:52.784 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:25:52.819 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T22:25:52.819 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:25:52.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-15T22:25:53.189 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:53.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:53 smithi005 ceph-mon[30944]: pgmap v532: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:25:53.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:53 smithi099 ceph-mon[34892]: pgmap v532: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:25:53.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:53 smithi129 ceph-mon[35314]: pgmap v532: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-15T22:25:53.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.671+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2099612856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c10c170 msgr2=0x7f2c2c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:53.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.671+0000 7f2c32ed1640 1 --2- 172.21.15.5:0/2099612856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c10c170 0x7f2c2c10e610 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2c24009ca0 tx=0x7f2c2402a950 comp rx=0 tx=0).stop 2024-09-15T22:25:53.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.671+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2099612856 shutdown_connections 2024-09-15T22:25:53.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.671+0000 7f2c32ed1640 1 --2- 172.21.15.5:0/2099612856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c10c170 0x7f2c2c10e610 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:53.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.671+0000 7f2c32ed1640 1 --2- 172.21.15.5:0/2099612856 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c2c103660 0x7f2c2c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:53.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.671+0000 7f2c32ed1640 1 --2- 172.21.15.5:0/2099612856 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c2c102d40 0x7f2c2c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:53.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.671+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2099612856 >> 172.21.15.5:0/2099612856 conn(0x7f2c2c0fc7a0 msgr2=0x7f2c2c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:53.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.671+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2099612856 shutdown_connections 2024-09-15T22:25:53.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.671+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2099612856 wait complete. 2024-09-15T22:25:53.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.672+0000 7f2c32ed1640 1 Processor -- start 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.672+0000 7f2c32ed1640 1 -- start start 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.672+0000 7f2c32ed1640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c2c102d40 0x7f2c2c1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.672+0000 7f2c32ed1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c103660 0x7f2c2c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.672+0000 7f2c32ed1640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c2c10c170 0x7f2c2c101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.672+0000 7f2c32ed1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c111f70 con 0x7f2c2c103660 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.672+0000 7f2c32ed1640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c111df0 con 0x7f2c2c102d40 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.672+0000 7f2c32ed1640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2c1120f0 con 0x7f2c2c10c170 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.672+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c103660 0x7f2c2c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.672+0000 7f2c2bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c103660 0x7f2c2c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48236/0 (socket says 172.21.15.5:48236) 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.672+0000 7f2c2bfff640 1 -- 172.21.15.5:0/2739969956 learned_addr learned my addr 172.21.15.5:0/2739969956 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c30c6d640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c2c102d40 0x7f2c2c1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c2bfff640 1 -- 172.21.15.5:0/2739969956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c2c10c170 msgr2=0x7f2c2c101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:53.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c2bfff640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c2c10c170 0x7f2c2c101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c2bfff640 1 -- 172.21.15.5:0/2739969956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c2c102d40 msgr2=0x7f2c2c1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c2bfff640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c2c102d40 0x7f2c2c1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c2bfff640 1 -- 172.21.15.5:0/2739969956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2c1a68d0 con 0x7f2c2c103660 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c2bfff640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c103660 0x7f2c2c100ee0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000d740 tx=0x7f2c2000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c29ffb640 1 -- 172.21.15.5:0/2739969956 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20017070 con 0x7f2c2c103660 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c29ffb640 1 -- 172.21.15.5:0/2739969956 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c2000de50 con 0x7f2c2c103660 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2739969956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2c1a6bc0 con 0x7f2c2c103660 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c29ffb640 1 -- 172.21.15.5:0/2739969956 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2000f620 con 0x7f2c2c103660 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.673+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2739969956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c2c1a7150 con 0x7f2c2c103660 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.674+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2739969956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2c103120 con 0x7f2c2c103660 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.675+0000 7f2c29ffb640 1 -- 172.21.15.5:0/2739969956 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f2c20009c30 con 0x7f2c2c103660 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.676+0000 7f2c29ffb640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2c140a3430 0x7f2c140a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.676+0000 7f2c30c6d640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2c140a3430 0x7f2c140a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:53.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.676+0000 7f2c29ffb640 1 -- 172.21.15.5:0/2739969956 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f2c200cfca0 con 0x7f2c2c103660 2024-09-15T22:25:53.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.676+0000 7f2c30c6d640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2c140a3430 0x7f2c140a58f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c005fd0 tx=0x7f2c1c005850 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:53.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.680+0000 7f2c29ffb640 1 -- 172.21.15.5:0/2739969956 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c2009c280 con 0x7f2c2c103660 2024-09-15T22:25:53.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.873+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2739969956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2c2c101ed0 con 0x7f2c2c103660 2024-09-15T22:25:53.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.874+0000 7f2c29ffb640 1 -- 172.21.15.5:0/2739969956 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v109) ==== 81+0+927 (secure 0 0 0) 0x7f2c2009bb80 con 0x7f2c2c103660 2024-09-15T22:25:53.876 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2453332511 2024-09-16T22:08:53.484141+0000 2024-09-15T22:25:53.876 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1812503826 2024-09-16T21:59:18.773927+0000 2024-09-15T22:25:53.876 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/3785442724 2024-09-16T21:59:42.685255+0000 2024-09-15T22:25:53.876 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4013564474 2024-09-16T21:59:18.773927+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/878276657 2024-09-16T22:08:53.484141+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2658926924 2024-09-16T21:59:18.773927+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1445520321 2024-09-16T21:59:18.773927+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/3785442724 2024-09-16T21:59:42.685255+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1408571851 2024-09-16T22:08:53.484141+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3880791045 2024-09-16T21:59:42.685255+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3146025957 2024-09-16T21:59:42.685255+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3326533830 2024-09-16T21:59:42.685255+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/801026306 2024-09-16T22:08:53.484141+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/801026306 2024-09-16T22:08:53.484141+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1812503826 2024-09-16T21:59:18.773927+0000 2024-09-15T22:25:53.877 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2083481614 2024-09-16T22:08:53.484141+0000 2024-09-15T22:25:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:listed 16 entries 2024-09-15T22:25:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2739969956 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2c140a3430 msgr2=0x7f2c140a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2c140a3430 0x7f2c140a58f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c005fd0 tx=0x7f2c1c005850 comp rx=0 tx=0).stop 2024-09-15T22:25:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2739969956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c103660 msgr2=0x7f2c2c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c103660 0x7f2c2c100ee0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000d740 tx=0x7f2c2000dc00 comp rx=0 tx=0).stop 2024-09-15T22:25:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2739969956 shutdown_connections 2024-09-15T22:25:53.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f2c140a3430 0x7f2c140a58f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f2c2c10c170 0x7f2c2c101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c2c103660 0x7f2c2c100ee0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 --2- 172.21.15.5:0/2739969956 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f2c2c102d40 0x7f2c2c1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2739969956 >> 172.21.15.5:0/2739969956 conn(0x7f2c2c0fc7a0 msgr2=0x7f2c2c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2739969956 shutdown_connections 2024-09-15T22:25:53.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:53.876+0000 7f2c32ed1640 1 -- 172.21.15.5:0/2739969956 wait complete. 2024-09-15T22:25:54.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:54 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2739969956' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:25:54.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:54 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2739969956' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:25:54.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:54 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2739969956' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-15T22:25:54.737 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-15T22:25:54.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:25:54.737 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /tmp/cephfs 2024-09-15T22:25:54.850 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-09-15T22:25:54.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:25:54.850 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2024-09-15T22:25:54.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:25:54.881 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-15T22:25:54.960 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--tmp-cephfs (id: 0) 2024-09-15T22:25:54.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:25:54.960 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--tmp-cephfs 2024-09-15T22:25:55.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:25:55.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-09-15T22:25:55.134 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-15T22:25:55.134 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-15T22:25:55.134 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T22:25:55.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056794,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498696k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18947 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/d9919bd38868efac62a64ced842c98781a4aae485929fecdae89356d25da0873/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J77FNHCZS63DFKU5OY4VXNFBPL:/var/lib/containers/storage/overlay/l/UEYH53GVDYKOUZQH735WIYD3TT,upperdir=/var/lib/containers/storage/overlay/d9919bd38868efac62a64ced842c98781a4aae485929fecdae89356d25da0873/diff,workdir=/var/lib/containers/storage/overlay/d9919bd38868efac62a64ced842c98781a4aae485929fecdae89356d25da0873/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/19d0969efd62277363b779c05f709445a592d6c592ccd59be6dbb88da18969e9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J77FNHCZS63DFKU5OY4VXNFBPL:/var/lib/containers/storage/overlay/l/UEYH53GVDYKOUZQH735WIYD3TT,upperdir=/var/lib/containers/storage/overlay/19d0969efd62277363b779c05f709445a592d6c592ccd59be6dbb88da18969e9/diff,workdir=/var/lib/containers/storage/overlay/19d0969efd62277363b779c05f709445a592d6c592ccd59be6dbb88da18969e9/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249348k,nr_inodes=812337,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/64af27383b4f441589ed7612b3ba45ec6fc96d746a524155f81c2d8e284c0f5f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J77FNHCZS63DFKU5OY4VXNFBPL:/var/lib/containers/storage/overlay/l/UEYH53GVDYKOUZQH735WIYD3TT,upperdir=/var/lib/containers/storage/overlay/64af27383b4f441589ed7612b3ba45ec6fc96d746a524155f81c2d8e284c0f5f/diff,workdir=/var/lib/containers/storage/overlay/64af27383b4f441589ed7612b3ba45ec6fc96d746a524155f81c2d8e284c0f5f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d97a7ceee00496b3797785aa6ce4f4070837939fa21d9674db481c66615acc84/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J77FNHCZS63DFKU5OY4VXNFBPL:/var/lib/containers/storage/overlay/l/UEYH53GVDYKOUZQH735WIYD3TT,upperdir=/var/lib/containers/storage/overlay/d97a7ceee00496b3797785aa6ce4f4070837939fa21d9674db481c66615acc84/diff,workdir=/var/lib/containers/storage/overlay/d97a7ceee00496b3797785aa6ce4f4070837939fa21d9674db481c66615acc84/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/978c9fcf9c52816845c791885727ecc2c74000b21868153ab31ca52c5bccd86a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J77FNHCZS63DFKU5OY4VXNFBPL:/var/lib/containers/storage/overlay/l/UEYH53GVDYKOUZQH735WIYD3TT,upperdir=/var/lib/containers/storage/overlay/978c9fcf9c52816845c791885727ecc2c74000b21868153ab31ca52c5bccd86a/diff,workdir=/var/lib/containers/storage/overlay/978c9fcf9c52816845c791885727ecc2c74000b21868153ab31ca52c5bccd86a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a570665ea4a27305dae5929dff31c77ab727d8d1df05f43c89e531e1766f5a3b/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/J77FNHCZS63DFKU5OY4VXNFBPL:/var/lib/containers/storage/overlay/l/UEYH53GVDYKOUZQH735WIYD3TT,upperdir=/var/lib/containers/storage/overlay/a570665ea4a27305dae5929dff31c77ab727d8d1df05f43c89e531e1766f5a3b/diff,workdir=/var/lib/containers/storage/overlay/a570665ea4a27305dae5929dff31c77ab727d8d1df05f43c89e531e1766f5a3b/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/ba3339e07f8d43b190ef167a5236b880b21b42e7d214855c4396818ad5a5ac2a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J77FNHCZS63DFKU5OY4VXNFBPL:/var/lib/containers/storage/overlay/l/UEYH53GVDYKOUZQH735WIYD3TT,upperdir=/var/lib/containers/storage/overlay/ba3339e07f8d43b190ef167a5236b880b21b42e7d214855c4396818ad5a5ac2a/diff,workdir=/var/lib/containers/storage/overlay/ba3339e07f8d43b190ef167a5236b880b21b42e7d214855c4396818ad5a5ac2a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a4a75b5048794f23e1c6fa0203ff67adcd417d7b37fac8c5162f4b2cfb10847a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J77FNHCZS63DFKU5OY4VXNFBPL:/var/lib/containers/storage/overlay/l/UEYH53GVDYKOUZQH735WIYD3TT,upperdir=/var/lib/containers/storage/overlay/a4a75b5048794f23e1c6fa0203ff67adcd417d7b37fac8c5162f4b2cfb10847a/diff,workdir=/var/lib/containers/storage/overlay/a4a75b5048794f23e1c6fa0203ff67adcd417d7b37fac8c5162f4b2cfb10847a/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/71cfd08d78763dd853953867d14af836bb60b34579aa22d90e990207fe5aa31f/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J77FNHCZS63DFKU5OY4VXNFBPL:/var/lib/containers/storage/overlay/l/UEYH53GVDYKOUZQH735WIYD3TT,upperdir=/var/lib/containers/storage/overlay/71cfd08d78763dd853953867d14af836bb60b34579aa22d90e990207fe5aa31f/diff,workdir=/var/lib/containers/storage/overlay/71cfd08d78763dd853953867d14af836bb60b34579aa22d90e990207fe5aa31f/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/e4e7ff231bdd6c49e7410ea3cada0bf0a8743529212e2c163d31f0fe5c0af3ff/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/J77FNHCZS63DFKU5OY4VXNFBPL:/var/lib/containers/storage/overlay/l/UEYH53GVDYKOUZQH735WIYD3TT,upperdir=/var/lib/containers/storage/overlay/e4e7ff231bdd6c49e7410ea3cada0bf0a8743529212e2c163d31f0fe5c0af3ff/diff,workdir=/var/lib/containers/storage/overlay/e4e7ff231bdd6c49e7410ea3cada0bf0a8743529212e2c163d31f0fe5c0af3ff/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498696k,nr_inodes=819200,mode=755,inode64 0 0 2024-09-15T22:25:55.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:25:55.152 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-15T22:25:55.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T22:25:55.208 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-09-15T22:25:55.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:55 smithi005 ceph-mon[30944]: pgmap v533: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:25:55.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:55 smithi099 ceph-mon[34892]: pgmap v533: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:25:55.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:55 smithi129 ceph-mon[35314]: pgmap v533: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-15T22:25:55.383 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-09-15T22:25:55.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:25:55.385 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-15T22:25:55.397 INFO:tasks.cephadm:Teardown begin 2024-09-15T22:25:55.397 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-15T22:25:55.397 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:25:55.684 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.122+0000 7f1f80b32640 1 -- 172.21.15.5:0/2367812780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c101420 msgr2=0x7f1f7c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.122+0000 7f1f80b32640 1 --2- 172.21.15.5:0/2367812780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c101420 0x7f1f7c10d950 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009930 tx=0x7f1f7402a420 comp rx=0 tx=0).stop 2024-09-15T22:25:56.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.122+0000 7f1f80b32640 1 -- 172.21.15.5:0/2367812780 shutdown_connections 2024-09-15T22:25:56.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.122+0000 7f1f80b32640 1 --2- 172.21.15.5:0/2367812780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c101420 0x7f1f7c10d950 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:56.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.122+0000 7f1f80b32640 1 --2- 172.21.15.5:0/2367812780 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c100a80 0x7f1f7c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:56.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.122+0000 7f1f80b32640 1 --2- 172.21.15.5:0/2367812780 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c106a80 0x7f1f7c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:56.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.122+0000 7f1f80b32640 1 -- 172.21.15.5:0/2367812780 >> 172.21.15.5:0/2367812780 conn(0x7f1f7c0fc7a0 msgr2=0x7f1f7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:56.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.122+0000 7f1f80b32640 1 -- 172.21.15.5:0/2367812780 shutdown_connections 2024-09-15T22:25:56.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.122+0000 7f1f80b32640 1 -- 172.21.15.5:0/2367812780 wait complete. 2024-09-15T22:25:56.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f80b32640 1 Processor -- start 2024-09-15T22:25:56.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f80b32640 1 -- start start 2024-09-15T22:25:56.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f80b32640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c100a80 0x7f1f7c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:56.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f80b32640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c101420 0x7f1f7c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:56.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f80b32640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c106a80 0x7f1f7c1a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:56.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f80b32640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c110ed0 con 0x7f1f7c106a80 2024-09-15T22:25:56.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f80b32640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c110d50 con 0x7f1f7c100a80 2024-09-15T22:25:56.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f80b32640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7c111050 con 0x7f1f7c101420 2024-09-15T22:25:56.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f7ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c106a80 0x7f1f7c1a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:56.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f7ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c106a80 0x7f1f7c1a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48266/0 (socket says 172.21.15.5:48266) 2024-09-15T22:25:56.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f7ad9d640 1 -- 172.21.15.5:0/4285916501 learned_addr learned my addr 172.21.15.5:0/4285916501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:56.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f7a59c640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c100a80 0x7f1f7c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.123+0000 7f1f79d9b640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c101420 0x7f1f7c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.124+0000 7f1f7ad9d640 1 -- 172.21.15.5:0/4285916501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c101420 msgr2=0x7f1f7c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.124+0000 7f1f7ad9d640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c101420 0x7f1f7c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.124+0000 7f1f7ad9d640 1 -- 172.21.15.5:0/4285916501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c100a80 msgr2=0x7f1f7c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.124+0000 7f1f7ad9d640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c100a80 0x7f1f7c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.124+0000 7f1f7ad9d640 1 -- 172.21.15.5:0/4285916501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7c1a4860 con 0x7f1f7c106a80 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.124+0000 7f1f7ad9d640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c106a80 0x7f1f7c1a4320 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1f74004770 tx=0x7f1f74003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.124+0000 7f1f737fe640 1 -- 172.21.15.5:0/4285916501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7403f070 con 0x7f1f7c106a80 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.124+0000 7f1f80b32640 1 -- 172.21.15.5:0/4285916501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7c1a4a30 con 0x7f1f7c106a80 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.124+0000 7f1f737fe640 1 -- 172.21.15.5:0/4285916501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f74031c90 con 0x7f1f7c106a80 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.124+0000 7f1f737fe640 1 -- 172.21.15.5:0/4285916501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f74035030 con 0x7f1f7c106a80 2024-09-15T22:25:56.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.124+0000 7f1f80b32640 1 -- 172.21.15.5:0/4285916501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f7c1a4e90 con 0x7f1f7c106a80 2024-09-15T22:25:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.125+0000 7f1f80b32640 1 -- 172.21.15.5:0/4285916501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f7c100ee0 con 0x7f1f7c106a80 2024-09-15T22:25:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.126+0000 7f1f737fe640 1 -- 172.21.15.5:0/4285916501 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f1f74042070 con 0x7f1f7c106a80 2024-09-15T22:25:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.127+0000 7f1f737fe640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f680a33b0 0x7f1f680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.127+0000 7f1f737fe640 1 -- 172.21.15.5:0/4285916501 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f1f7403be00 con 0x7f1f7c106a80 2024-09-15T22:25:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.127+0000 7f1f7a59c640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f680a33b0 0x7f1f680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.128+0000 7f1f7a59c640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f680a33b0 0x7f1f680a5870 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c005c40 tx=0x7f1f6c00a590 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:56.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.130+0000 7f1f737fe640 1 -- 172.21.15.5:0/4285916501 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1f740bcd40 con 0x7f1f7c106a80 2024-09-15T22:25:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.342+0000 7f1f80b32640 1 -- 172.21.15.5:0/4285916501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f7c1a50c0 con 0x7f1f7c106a80 2024-09-15T22:25:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.343+0000 7f1f737fe640 1 -- 172.21.15.5:0/4285916501 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7f1f740d4e90 con 0x7f1f7c106a80 2024-09-15T22:25:56.346 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:25:56.346 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:25:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:25:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 -- 172.21.15.5:0/4285916501 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f680a33b0 msgr2=0x7f1f680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f680a33b0 0x7f1f680a5870 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c005c40 tx=0x7f1f6c00a590 comp rx=0 tx=0).stop 2024-09-15T22:25:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 -- 172.21.15.5:0/4285916501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c106a80 msgr2=0x7f1f7c1a4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c106a80 0x7f1f7c1a4320 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f1f74004770 tx=0x7f1f74003b90 comp rx=0 tx=0).stop 2024-09-15T22:25:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 -- 172.21.15.5:0/4285916501 shutdown_connections 2024-09-15T22:25:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f1f680a33b0 0x7f1f680a5870 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f7c106a80 0x7f1f7c1a4320 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f1f7c101420 0x7f1f7c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 --2- 172.21.15.5:0/4285916501 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f1f7c100a80 0x7f1f7c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 -- 172.21.15.5:0/4285916501 >> 172.21.15.5:0/4285916501 conn(0x7f1f7c0fc7a0 msgr2=0x7f1f7c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 -- 172.21.15.5:0/4285916501 shutdown_connections 2024-09-15T22:25:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:56.345+0000 7f1f80b32640 1 -- 172.21.15.5:0/4285916501 wait complete. 2024-09-15T22:25:57.041 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-15T22:25:57.303 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:57.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:57 smithi099 ceph-mon[34892]: pgmap v534: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-09-15T22:25:57.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:57 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/4285916501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:57 smithi129 ceph-mon[35314]: pgmap v534: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-09-15T22:25:57.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:57 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/4285916501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:57.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:57 smithi005 ceph-mon[30944]: pgmap v534: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-09-15T22:25:57.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:57 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/4285916501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.823+0000 7fca5a8f8640 1 -- 172.21.15.5:0/351744084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca54102570 msgr2=0x7fca541029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.823+0000 7fca5a8f8640 1 --2- 172.21.15.5:0/351744084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca54102570 0x7fca541029b0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009900 tx=0x7fca3c02a480 comp rx=0 tx=0).stop 2024-09-15T22:25:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.823+0000 7fca5a8f8640 1 -- 172.21.15.5:0/351744084 shutdown_connections 2024-09-15T22:25:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.823+0000 7fca5a8f8640 1 --2- 172.21.15.5:0/351744084 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54102ef0 0x7fca5410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.823+0000 7fca5a8f8640 1 --2- 172.21.15.5:0/351744084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca54102570 0x7fca541029b0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.823+0000 7fca5a8f8640 1 --2- 172.21.15.5:0/351744084 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca54100e20 0x7fca54101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.823+0000 7fca5a8f8640 1 -- 172.21.15.5:0/351744084 >> 172.21.15.5:0/351744084 conn(0x7fca540fc7d0 msgr2=0x7fca540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.823+0000 7fca5a8f8640 1 -- 172.21.15.5:0/351744084 shutdown_connections 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.823+0000 7fca5a8f8640 1 -- 172.21.15.5:0/351744084 wait complete. 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.823+0000 7fca5a8f8640 1 Processor -- start 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.823+0000 7fca5a8f8640 1 -- start start 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca5a8f8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca54100e20 0x7fca54198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca5a8f8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca54102570 0x7fca54198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca5a8f8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54102ef0 0x7fca5419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca5a8f8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca54110f80 con 0x7fca54102570 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca5a8f8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fca54110e00 con 0x7fca54100e20 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca5a8f8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fca54111100 con 0x7fca54102ef0 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca4bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca54102570 0x7fca54198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca4bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca54102570 0x7fca54198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48278/0 (socket says 172.21.15.5:48278) 2024-09-15T22:25:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca4bfff640 1 -- 172.21.15.5:0/2502702175 learned_addr learned my addr 172.21.15.5:0/2502702175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca58e95640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54102ef0 0x7fca5419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca53fff640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca54100e20 0x7fca54198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca4bfff640 1 -- 172.21.15.5:0/2502702175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54102ef0 msgr2=0x7fca5419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca4bfff640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54102ef0 0x7fca5419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca4bfff640 1 -- 172.21.15.5:0/2502702175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca54100e20 msgr2=0x7fca54198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca4bfff640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca54100e20 0x7fca54198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca4bfff640 1 -- 172.21.15.5:0/2502702175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5419d3e0 con 0x7fca54102570 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca58e95640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54102ef0 0x7fca5419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.824+0000 7fca4bfff640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca54102570 0x7fca54198970 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fca3c006010 tx=0x7fca3c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.825+0000 7fca51ffb640 1 -- 172.21.15.5:0/2502702175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c03f070 con 0x7fca54102570 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.825+0000 7fca51ffb640 1 -- 172.21.15.5:0/2502702175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca3c031c30 con 0x7fca54102570 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.825+0000 7fca51ffb640 1 -- 172.21.15.5:0/2502702175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3c035030 con 0x7fca54102570 2024-09-15T22:25:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.825+0000 7fca5a8f8640 1 -- 172.21.15.5:0/2502702175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5419d670 con 0x7fca54102570 2024-09-15T22:25:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.825+0000 7fca5a8f8640 1 -- 172.21.15.5:0/2502702175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca541a4fb0 con 0x7fca54102570 2024-09-15T22:25:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.826+0000 7fca5a8f8640 1 -- 172.21.15.5:0/2502702175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca54101200 con 0x7fca54102570 2024-09-15T22:25:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.827+0000 7fca51ffb640 1 -- 172.21.15.5:0/2502702175 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fca3c042070 con 0x7fca54102570 2024-09-15T22:25:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.828+0000 7fca51ffb640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fca400a33b0 0x7fca400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.828+0000 7fca53fff640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fca400a33b0 0x7fca400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.828+0000 7fca51ffb640 1 -- 172.21.15.5:0/2502702175 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7fca3c0f0e50 con 0x7fca54102570 2024-09-15T22:25:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.828+0000 7fca53fff640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fca400a33b0 0x7fca400a5870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fca44005e10 tx=0x7fca44005da0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:57.831+0000 7fca51ffb640 1 -- 172.21.15.5:0/2502702175 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca3c031de0 con 0x7fca54102570 2024-09-15T22:25:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.043+0000 7fca5a8f8640 1 -- 172.21.15.5:0/2502702175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca5410a0a0 con 0x7fca54102570 2024-09-15T22:25:58.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.043+0000 7fca51ffb640 1 -- 172.21.15.5:0/2502702175 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11897 (secure 0 0 0) 0x7fca3c0bd430 con 0x7fca54102570 2024-09-15T22:25:58.045 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:25:58.045 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-15T22:08:52:078869+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14526,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/93159772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":93159772},{"type":"v1","addr":"172.21.15.5:6835","nonce":93159772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14538,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2795083273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2795083273},{"type":"v1","addr":"172.21.15.5:6837","nonce":2795083273}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14583,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/2796220965","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":2796220965},{"type":"v1","addr":"172.21.15.5:6839","nonce":2796220965}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14595,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/3376413003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":3376413003},{"type":"v1","addr":"172.21.15.5:6841","nonce":3376413003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24418,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.129:6833/3381054167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6832","nonce":3381054167},{"type":"v1","addr":"172.21.15.129:6833","nonce":3381054167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.99:6833/1006057528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":1006057528},{"type":"v1","addr":"172.21.15.99:6833","nonce":1006057528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-15T22:06:37.854436+0000","modified":"2024-09-15T22:08:52.028683+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24491,"mds_1":24467,"mds_2":24466},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.99:6835/157351762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6834","nonce":157351762},{"type":"v1","addr":"172.21.15.99:6835","nonce":157351762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6835/3511675647","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6834","nonce":3511675647},{"type":"v1","addr":"172.21.15.129:6835","nonce":3511675647}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.129:6837/3535375231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6836","nonce":3535375231},{"type":"v1","addr":"172.21.15.129:6837","nonce":3535375231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24466":{"gid":24466,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":16,"addr":"172.21.15.129:6839/2032213198","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6838","nonce":2032213198},{"type":"v1","addr":"172.21.15.129:6839","nonce":2032213198}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":16,"addr":"172.21.15.99:6837/3952304847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6836","nonce":3952304847},{"type":"v1","addr":"172.21.15.99:6837","nonce":3952304847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24491":{"gid":24491,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.99:6839/3763383986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6838","nonce":3763383986},{"type":"v1","addr":"172.21.15.99:6839","nonce":3763383986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24491,"qdb_cluster":[24491,24467,24466]},"id":1}]} 2024-09-15T22:25:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-15T22:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 -- 172.21.15.5:0/2502702175 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fca400a33b0 msgr2=0x7fca400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fca400a33b0 0x7fca400a5870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fca44005e10 tx=0x7fca44005da0 comp rx=0 tx=0).stop 2024-09-15T22:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 -- 172.21.15.5:0/2502702175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca54102570 msgr2=0x7fca54198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca54102570 0x7fca54198970 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fca3c006010 tx=0x7fca3c003c10 comp rx=0 tx=0).stop 2024-09-15T22:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 -- 172.21.15.5:0/2502702175 shutdown_connections 2024-09-15T22:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fca400a33b0 0x7fca400a5870 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fca54102ef0 0x7fca5419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca54102570 0x7fca54198970 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 --2- 172.21.15.5:0/2502702175 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fca54100e20 0x7fca54198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 -- 172.21.15.5:0/2502702175 >> 172.21.15.5:0/2502702175 conn(0x7fca540fc7d0 msgr2=0x7fca5406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 -- 172.21.15.5:0/2502702175 shutdown_connections 2024-09-15T22:25:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:58.046+0000 7fca5a8f8640 1 -- 172.21.15.5:0/2502702175 wait complete. 2024-09-15T22:25:58.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:58 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/2502702175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:58.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:58 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/2502702175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:58.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:58 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/2502702175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-15T22:25:58.867 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-15T22:25:59.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:25:59 smithi005 ceph-mon[30944]: pgmap v535: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 7 op/s 2024-09-15T22:25:59.135 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:25:59.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:25:59 smithi099 ceph-mon[34892]: pgmap v535: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 7 op/s 2024-09-15T22:25:59.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:25:59 smithi129 ceph-mon[35314]: pgmap v535: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 7 op/s 2024-09-15T22:25:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.623+0000 7f350f7a8640 1 -- 172.21.15.5:0/3427937468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f350810c170 msgr2=0x7f350810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.623+0000 7f350f7a8640 1 --2- 172.21.15.5:0/3427937468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f350810c170 0x7f350810e610 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3500009900 tx=0x7f350002a480 comp rx=0 tx=0).stop 2024-09-15T22:25:59.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.623+0000 7f350f7a8640 1 -- 172.21.15.5:0/3427937468 shutdown_connections 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.623+0000 7f350f7a8640 1 --2- 172.21.15.5:0/3427937468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f350810c170 0x7f350810e610 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.623+0000 7f350f7a8640 1 --2- 172.21.15.5:0/3427937468 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3508103660 0x7f350810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.623+0000 7f350f7a8640 1 --2- 172.21.15.5:0/3427937468 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f3508102d40 0x7f3508103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.623+0000 7f350f7a8640 1 -- 172.21.15.5:0/3427937468 >> 172.21.15.5:0/3427937468 conn(0x7f35080fc7a0 msgr2=0x7f35080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.623+0000 7f350f7a8640 1 -- 172.21.15.5:0/3427937468 shutdown_connections 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.623+0000 7f350f7a8640 1 -- 172.21.15.5:0/3427937468 wait complete. 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.623+0000 7f350f7a8640 1 Processor -- start 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.623+0000 7f350f7a8640 1 -- start start 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350f7a8640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3508102d40 0x7f35081009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350f7a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3508103660 0x7f3508100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350f7a8640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f350810c170 0x7f3508101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350f7a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3508111f70 con 0x7f3508103660 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350f7a8640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f3508111df0 con 0x7f3508102d40 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350f7a8640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f35081120f0 con 0x7f350810c170 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350cd43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3508103660 0x7f3508100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350cd43640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3508103660 0x7f3508100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48302/0 (socket says 172.21.15.5:48302) 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350cd43640 1 -- 172.21.15.5:0/3264935050 learned_addr learned my addr 172.21.15.5:0/3264935050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350dd45640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f350810c170 0x7f3508101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350d544640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3508102d40 0x7f35081009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350cd43640 1 -- 172.21.15.5:0/3264935050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f350810c170 msgr2=0x7f3508101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350cd43640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f350810c170 0x7f3508101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350cd43640 1 -- 172.21.15.5:0/3264935050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3508102d40 msgr2=0x7f35081009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350cd43640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3508102d40 0x7f35081009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350cd43640 1 -- 172.21.15.5:0/3264935050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35081a6830 con 0x7f3508103660 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.624+0000 7f350d544640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3508102d40 0x7f35081009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.625+0000 7f350cd43640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3508103660 0x7f3508100ee0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f34f400cce0 tx=0x7f34f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.625+0000 7f35067fc640 1 -- 172.21.15.5:0/3264935050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f400f450 con 0x7f3508103660 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.625+0000 7f350f7a8640 1 -- 172.21.15.5:0/3264935050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35081a6b20 con 0x7f3508103660 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.625+0000 7f35067fc640 1 -- 172.21.15.5:0/3264935050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34f400fad0 con 0x7f3508103660 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.625+0000 7f35067fc640 1 -- 172.21.15.5:0/3264935050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4018c10 con 0x7f3508103660 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.625+0000 7f350f7a8640 1 -- 172.21.15.5:0/3264935050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35081a7060 con 0x7f3508103660 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.626+0000 7f350f7a8640 1 -- 172.21.15.5:0/3264935050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3508103120 con 0x7f3508103660 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.627+0000 7f35067fc640 1 -- 172.21.15.5:0/3264935050 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f34f400f5f0 con 0x7f3508103660 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.628+0000 7f35067fc640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f34f80a33b0 0x7f34f80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.628+0000 7f350d544640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f34f80a33b0 0x7f34f80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.628+0000 7f35067fc640 1 -- 172.21.15.5:0/3264935050 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f34f40d0410 con 0x7f3508103660 2024-09-15T22:25:59.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.628+0000 7f350d544640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f34f80a33b0 0x7f34f80a5870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0058c0 tx=0x7f34fc005ec0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:25:59.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.632+0000 7f35067fc640 1 -- 172.21.15.5:0/3264935050 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34f4014030 con 0x7f3508103660 2024-09-15T22:25:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.813+0000 7f350f7a8640 1 -- 172.21.15.5:0/3264935050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35081a7290 con 0x7f3508103660 2024-09-15T22:25:59.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.814+0000 7f35067fc640 1 -- 172.21.15.5:0/3264935050 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+25314 (secure 0 0 0) 0x7f34f409c9f0 con 0x7f3508103660 2024-09-15T22:25:59.816 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-15T22:25:59.817 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":109,"fsid":"962891f2-73ad-11ef-bceb-c7b262605968","created":"2024-09-15T21:58:48.668955+0000","modified":"2024-09-15T22:25:18.970679+0000","last_up_change":"2024-09-15T22:05:45.350833+0000","last_in_change":"2024-09-15T22:05:21.037693+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T22:01:57.184697+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T22:06:33.616843+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T22:06:36.161538+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-15T22:06:43.068085+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":27,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"32741a24-7f36-4b4d-b9cd-4c4df6a9fa9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6803","nonce":3136788865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6805","nonce":3136788865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6809","nonce":3136788865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":3136788865},{"type":"v1","addr":"172.21.15.5:6807","nonce":3136788865}]},"public_addr":"172.21.15.5:6803/3136788865","cluster_addr":"172.21.15.5:6805/3136788865","heartbeat_back_addr":"172.21.15.5:6809/3136788865","heartbeat_front_addr":"172.21.15.5:6807/3136788865","state":["exists","up"]},{"osd":1,"uuid":"24fdcef5-f6ba-4f4e-8c27-aa2d60d54bb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6801","nonce":2888502880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6803","nonce":2888502880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6807","nonce":2888502880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2888502880},{"type":"v1","addr":"172.21.15.99:6805","nonce":2888502880}]},"public_addr":"172.21.15.99:6801/2888502880","cluster_addr":"172.21.15.99:6803/2888502880","heartbeat_back_addr":"172.21.15.99:6807/2888502880","heartbeat_front_addr":"172.21.15.99:6805/2888502880","state":["exists","up"]},{"osd":2,"uuid":"0cf9aa4e-8865-471c-a841-4be5386a3fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6800","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6801","nonce":2902807483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6802","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6803","nonce":2902807483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6806","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6807","nonce":2902807483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6804","nonce":2902807483},{"type":"v1","addr":"172.21.15.129:6805","nonce":2902807483}]},"public_addr":"172.21.15.129:6801/2902807483","cluster_addr":"172.21.15.129:6803/2902807483","heartbeat_back_addr":"172.21.15.129:6807/2902807483","heartbeat_front_addr":"172.21.15.129:6805/2902807483","state":["exists","up"]},{"osd":3,"uuid":"0fcbf74d-7a3d-471e-9447-688e10d0c350","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6811","nonce":1557519616}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6813","nonce":1557519616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6817","nonce":1557519616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":1557519616},{"type":"v1","addr":"172.21.15.5:6815","nonce":1557519616}]},"public_addr":"172.21.15.5:6811/1557519616","cluster_addr":"172.21.15.5:6813/1557519616","heartbeat_back_addr":"172.21.15.5:6817/1557519616","heartbeat_front_addr":"172.21.15.5:6815/1557519616","state":["exists","up"]},{"osd":4,"uuid":"511bc0a8-e4f3-48e2-824b-7afd2ead90e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6809","nonce":79942025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6811","nonce":79942025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6815","nonce":79942025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":79942025},{"type":"v1","addr":"172.21.15.99:6813","nonce":79942025}]},"public_addr":"172.21.15.99:6809/79942025","cluster_addr":"172.21.15.99:6811/79942025","heartbeat_back_addr":"172.21.15.99:6815/79942025","heartbeat_front_addr":"172.21.15.99:6813/79942025","state":["exists","up"]},{"osd":5,"uuid":"5524a904-d0c6-4276-9e11-2dcf0e6ec037","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6808","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6809","nonce":3964498949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6810","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6811","nonce":3964498949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6814","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6815","nonce":3964498949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6812","nonce":3964498949},{"type":"v1","addr":"172.21.15.129:6813","nonce":3964498949}]},"public_addr":"172.21.15.129:6809/3964498949","cluster_addr":"172.21.15.129:6811/3964498949","heartbeat_back_addr":"172.21.15.129:6815/3964498949","heartbeat_front_addr":"172.21.15.129:6813/3964498949","state":["exists","up"]},{"osd":6,"uuid":"a5b89d06-6596-4cd8-9083-3e023d156603","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6819","nonce":1626090223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6821","nonce":1626090223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6825","nonce":1626090223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1626090223},{"type":"v1","addr":"172.21.15.5:6823","nonce":1626090223}]},"public_addr":"172.21.15.5:6819/1626090223","cluster_addr":"172.21.15.5:6821/1626090223","heartbeat_back_addr":"172.21.15.5:6825/1626090223","heartbeat_front_addr":"172.21.15.5:6823/1626090223","state":["exists","up"]},{"osd":7,"uuid":"4e8598f5-b1c0-4196-8668-780ad8f79e76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6817","nonce":521420883}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6819","nonce":521420883}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6823","nonce":521420883}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":521420883},{"type":"v1","addr":"172.21.15.99:6821","nonce":521420883}]},"public_addr":"172.21.15.99:6817/521420883","cluster_addr":"172.21.15.99:6819/521420883","heartbeat_back_addr":"172.21.15.99:6823/521420883","heartbeat_front_addr":"172.21.15.99:6821/521420883","state":["exists","up"]},{"osd":8,"uuid":"ae5048f4-2e10-4e65-a7e9-130180439ad9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6816","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6817","nonce":2292146431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6818","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6819","nonce":2292146431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6822","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6823","nonce":2292146431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6820","nonce":2292146431},{"type":"v1","addr":"172.21.15.129:6821","nonce":2292146431}]},"public_addr":"172.21.15.129:6817/2292146431","cluster_addr":"172.21.15.129:6819/2292146431","heartbeat_back_addr":"172.21.15.129:6823/2292146431","heartbeat_front_addr":"172.21.15.129:6821/2292146431","state":["exists","up"]},{"osd":9,"uuid":"93835974-5099-475b-86ce-057cf6614dff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6827","nonce":3480349355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6829","nonce":3480349355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6833","nonce":3480349355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3480349355},{"type":"v1","addr":"172.21.15.5:6831","nonce":3480349355}]},"public_addr":"172.21.15.5:6827/3480349355","cluster_addr":"172.21.15.5:6829/3480349355","heartbeat_back_addr":"172.21.15.5:6833/3480349355","heartbeat_front_addr":"172.21.15.5:6831/3480349355","state":["exists","up"]},{"osd":10,"uuid":"d019941f-3a11-449b-8e5d-dac790b1530f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6825","nonce":154036498}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6827","nonce":154036498}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6831","nonce":154036498}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":154036498},{"type":"v1","addr":"172.21.15.99:6829","nonce":154036498}]},"public_addr":"172.21.15.99:6825/154036498","cluster_addr":"172.21.15.99:6827/154036498","heartbeat_back_addr":"172.21.15.99:6831/154036498","heartbeat_front_addr":"172.21.15.99:6829/154036498","state":["exists","up"]},{"osd":11,"uuid":"2cff3d48-81e6-4748-96bb-c20ba054f287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6824","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6825","nonce":4002757789}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6826","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6827","nonce":4002757789}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6830","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6831","nonce":4002757789}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.129:6828","nonce":4002757789},{"type":"v1","addr":"172.21.15.129:6829","nonce":4002757789}]},"public_addr":"172.21.15.129:6825/4002757789","cluster_addr":"172.21.15.129:6827/4002757789","heartbeat_back_addr":"172.21.15.129:6831/4002757789","heartbeat_front_addr":"172.21.15.129:6829/4002757789","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:05.243650+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:29.545283+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:01:54.743307+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:19.379002+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:02:45.484521+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:11.168554+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:03:35.860336+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:01.505669+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:26.652838+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:04:51.752625+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:17.224212+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T22:05:43.069398+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11},{"from":8,"to":6}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":8,"to":4},{"from":7,"to":11}]},{"pgid":"3.6","mappings":[{"from":3,"to":6}]},{"pgid":"3.11","mappings":[{"from":4,"to":7}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.37","mappings":[{"from":1,"to":9}]},{"pgid":"4.6","mappings":[{"from":9,"to":11}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/2453332511":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6801/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6800/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/4013564474":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/878276657":"2024-09-16T22:08:53.484141+0000","172.21.15.5:0/2658926924":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/1445520321":"2024-09-16T21:59:18.773927+0000","172.21.15.5:6801/3785442724":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/1408571851":"2024-09-16T22:08:53.484141+0000","172.21.15.5:0/3880791045":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3146025957":"2024-09-16T21:59:42.685255+0000","172.21.15.5:0/3326533830":"2024-09-16T21:59:42.685255+0000","172.21.15.5:6801/801026306":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6800/801026306":"2024-09-16T22:08:53.484141+0000","172.21.15.5:6800/1812503826":"2024-09-16T21:59:18.773927+0000","172.21.15.5:0/2083481614":"2024-09-16T22:08:53.484141+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]},{"pool":4,"snaps":[{"begin":14,"length":1},{"begin":27,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T22:25:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 -- 172.21.15.5:0/3264935050 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f34f80a33b0 msgr2=0x7f34f80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f34f80a33b0 0x7f34f80a5870 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0058c0 tx=0x7f34fc005ec0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 -- 172.21.15.5:0/3264935050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3508103660 msgr2=0x7f3508100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:25:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3508103660 0x7f3508100ee0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f34f400cce0 tx=0x7f34f400a620 comp rx=0 tx=0).stop 2024-09-15T22:25:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 -- 172.21.15.5:0/3264935050 shutdown_connections 2024-09-15T22:25:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f34f80a33b0 0x7f34f80a5870 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f350810c170 0x7f3508101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3508103660 0x7f3508100ee0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 --2- 172.21.15.5:0/3264935050 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f3508102d40 0x7f35081009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:25:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 -- 172.21.15.5:0/3264935050 >> 172.21.15.5:0/3264935050 conn(0x7f35080fc7a0 msgr2=0x7f35080fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:25:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 -- 172.21.15.5:0/3264935050 shutdown_connections 2024-09-15T22:25:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:25:59.817+0000 7f350f7a8640 1 -- 172.21.15.5:0/3264935050 wait complete. 2024-09-15T22:26:00.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:00 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3264935050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:26:00.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:00 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3264935050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:26:00.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:00 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3264935050' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-15T22:26:00.660 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-15T22:26:00.917 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:01.294 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:01 smithi005 ceph-mon[30944]: pgmap v536: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 6 op/s 2024-09-15T22:26:01.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-mon[34892]: pgmap v536: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 6 op/s 2024-09-15T22:26:01.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-mon[35314]: pgmap v536: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 76 KiB/s wr, 6 op/s 2024-09-15T22:26:01.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.508+0000 7f6018414640 1 -- 172.21.15.5:0/3674949026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010102790 msgr2=0x7f60101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:01.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.508+0000 7f6018414640 1 --2- 172.21.15.5:0/3674949026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010102790 0x7f60101067e0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f6000009900 tx=0x7f600002a410 comp rx=0 tx=0).stop 2024-09-15T22:26:01.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.508+0000 7f6018414640 1 -- 172.21.15.5:0/3674949026 shutdown_connections 2024-09-15T22:26:01.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.508+0000 7f6018414640 1 --2- 172.21.15.5:0/3674949026 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6010106f10 0x7f601010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:01.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.508+0000 7f6018414640 1 --2- 172.21.15.5:0/3674949026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010102790 0x7f60101067e0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:01.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.508+0000 7f6018414640 1 --2- 172.21.15.5:0/3674949026 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010101de0 0x7f60101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:01.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.508+0000 7f6018414640 1 -- 172.21.15.5:0/3674949026 >> 172.21.15.5:0/3674949026 conn(0x7f60100fd8c0 msgr2=0x7f60100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:01.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.509+0000 7f6018414640 1 -- 172.21.15.5:0/3674949026 shutdown_connections 2024-09-15T22:26:01.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.509+0000 7f6018414640 1 -- 172.21.15.5:0/3674949026 wait complete. 2024-09-15T22:26:01.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.509+0000 7f6018414640 1 Processor -- start 2024-09-15T22:26:01.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.509+0000 7f6018414640 1 -- start start 2024-09-15T22:26:01.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f6018414640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101de0 0x7f6010198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:01.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f6018414640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010102790 0x7f6010198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:01.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f6018414640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6010106f10 0x7f601019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:01.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f6018414640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111f70 con 0x7f6010101de0 2024-09-15T22:26:01.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f6018414640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f6010111df0 con 0x7f6010102790 2024-09-15T22:26:01.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f6018414640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f60101120f0 con 0x7f6010106f10 2024-09-15T22:26:01.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60161b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101de0 0x7f6010198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60161b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101de0 0x7f6010198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48316/0 (socket says 172.21.15.5:48316) 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60161b0640 1 -- 172.21.15.5:0/1233165806 learned_addr learned my addr 172.21.15.5:0/1233165806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60159af640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010102790 0x7f6010198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60169b1640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6010106f10 0x7f601019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60161b0640 1 -- 172.21.15.5:0/1233165806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6010106f10 msgr2=0x7f601019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60161b0640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6010106f10 0x7f601019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60161b0640 1 -- 172.21.15.5:0/1233165806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010102790 msgr2=0x7f6010198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60161b0640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010102790 0x7f6010198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60161b0640 1 -- 172.21.15.5:0/1233165806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601019d3e0 con 0x7f6010101de0 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60159af640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010102790 0x7f6010198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60161b0640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101de0 0x7f6010198430 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f600c00d950 tx=0x7f600c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60077fe640 1 -- 172.21.15.5:0/1233165806 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c017030 con 0x7f6010101de0 2024-09-15T22:26:01.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f6018414640 1 -- 172.21.15.5:0/1233165806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601019d6d0 con 0x7f6010101de0 2024-09-15T22:26:01.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.510+0000 7f60077fe640 1 -- 172.21.15.5:0/1233165806 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f600c004020 con 0x7f6010101de0 2024-09-15T22:26:01.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.511+0000 7f6018414640 1 -- 172.21.15.5:0/1233165806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101a4fb0 con 0x7f6010101de0 2024-09-15T22:26:01.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.511+0000 7f60077fe640 1 -- 172.21.15.5:0/1233165806 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c0047a0 con 0x7f6010101de0 2024-09-15T22:26:01.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.512+0000 7f6018414640 1 -- 172.21.15.5:0/1233165806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60101021c0 con 0x7f6010101de0 2024-09-15T22:26:01.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.513+0000 7f60077fe640 1 -- 172.21.15.5:0/1233165806 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f600c0041d0 con 0x7f6010101de0 2024-09-15T22:26:01.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.513+0000 7f60077fe640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5ffc0a33b0 0x7f5ffc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:01.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.514+0000 7f60159af640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5ffc0a33b0 0x7f5ffc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:01.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.514+0000 7f60077fe640 1 -- 172.21.15.5:0/1233165806 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9541+0+0 (secure 0 0 0) 0x7f600c0d01f0 con 0x7f6010101de0 2024-09-15T22:26:01.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.514+0000 7f60159af640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5ffc0a33b0 0x7f5ffc0a5870 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6000000c00 tx=0x7f6000032c70 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:01.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.517+0000 7f60077fe640 1 -- 172.21.15.5:0/1233165806 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f600c09c7d0 con 0x7f6010101de0 2024-09-15T22:26:01.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.735+0000 7f6018414640 1 -- 172.21.15.5:0/1233165806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6010199790 con 0x7f6010101de0 2024-09-15T22:26:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.737+0000 7f60077fe640 1 -- 172.21.15.5:0/1233165806 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f600c09c0d0 con 0x7f6010101de0 2024-09-15T22:26:01.739 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-15T22:26:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 -- 172.21.15.5:0/1233165806 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5ffc0a33b0 msgr2=0x7f5ffc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5ffc0a33b0 0x7f5ffc0a5870 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6000000c00 tx=0x7f6000032c70 comp rx=0 tx=0).stop 2024-09-15T22:26:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 -- 172.21.15.5:0/1233165806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101de0 msgr2=0x7f6010198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:01.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101de0 0x7f6010198430 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f600c00d950 tx=0x7f600c00de10 comp rx=0 tx=0).stop 2024-09-15T22:26:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 -- 172.21.15.5:0/1233165806 shutdown_connections 2024-09-15T22:26:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f5ffc0a33b0 0x7f5ffc0a5870 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f6010106f10 0x7f601019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f6010102790 0x7f6010198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 --2- 172.21.15.5:0/1233165806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010101de0 0x7f6010198430 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 -- 172.21.15.5:0/1233165806 >> 172.21.15.5:0/1233165806 conn(0x7f60100fd8c0 msgr2=0x7f60100fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 -- 172.21.15.5:0/1233165806 shutdown_connections 2024-09-15T22:26:01.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:01.741+0000 7f6018414640 1 -- 172.21.15.5:0/1233165806 wait complete. 2024-09-15T22:26:02.085 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:01 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a[30940]: 2024-09-15T22:26:01.736+0000 7ffbca2ba640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:26:02.086 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-e[64374]: ignoring --setuser ceph since I am not root 2024-09-15T22:26:02.086 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-e[64374]: ignoring --setgroup ceph since I am not root 2024-09-15T22:26:02.086 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-mds[64378]: -- 172.21.15.99:0/3845542020 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f0b03b9860 con 0x55f0b0396800 2024-09-15T22:26:02.086 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-e[64374]: starting mds.e at 2024-09-15T22:26:02.086 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b[64991]: ignoring --setuser ceph since I am not root 2024-09-15T22:26:02.086 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b[64991]: ignoring --setgroup ceph since I am not root 2024-09-15T22:26:02.086 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-mds[64995]: -- 172.21.15.99:0/971648389 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d422853860 con 0x55d422830800 2024-09-15T22:26:02.087 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b[64991]: starting mds.b at 2024-09-15T22:26:02.087 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h[64679]: ignoring --setuser ceph since I am not root 2024-09-15T22:26:02.087 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h[64679]: ignoring --setgroup ceph since I am not root 2024-09-15T22:26:02.087 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-mds[64683]: -- 172.21.15.99:0/3817881158 <== mon.2 v2:172.21.15.99:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577d952b860 con 0x5577d9509400 2024-09-15T22:26:02.087 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:26:01 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h[64679]: starting mds.h at 2024-09-15T22:26:02.088 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f[64638]: ignoring --setuser ceph since I am not root 2024-09-15T22:26:02.088 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f[64638]: ignoring --setgroup ceph since I am not root 2024-09-15T22:26:02.088 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f[64638]: starting mds.f at 2024-09-15T22:26:02.088 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c[64941]: ignoring --setuser ceph since I am not root 2024-09-15T22:26:02.088 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c[64941]: ignoring --setgroup ceph since I am not root 2024-09-15T22:26:02.088 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-mds[64945]: -- 172.21.15.129:0/3720514236 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556469b27860 con 0x556469b04800 2024-09-15T22:26:02.088 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c[64941]: starting mds.c at 2024-09-15T22:26:02.089 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i[65245]: ignoring --setuser ceph since I am not root 2024-09-15T22:26:02.089 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i[65245]: ignoring --setgroup ceph since I am not root 2024-09-15T22:26:02.089 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-mds[65249]: -- 172.21.15.129:0/315915350 <== mon.1 v2:172.21.15.129:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558790ca5860 con 0x558790c83000 2024-09-15T22:26:02.089 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:26:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i[65245]: starting mds.i at 2024-09-15T22:26:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:02 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1233165806' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:26:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:02 smithi099 ceph-mon[34892]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T22:26:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:02 smithi099 ceph-mon[34892]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:26:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:02 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1233165806' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T22:26:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:02 smithi099 ceph-mon[34892]: osdmap e110: 12 total, 12 up, 12 in 2024-09-15T22:26:02.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:02 smithi099 ceph-mon[34892]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-15T22:26:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:02 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1233165806' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:26:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:02 smithi129 ceph-mon[35314]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T22:26:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:02 smithi129 ceph-mon[35314]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:26:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:02 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1233165806' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T22:26:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:02 smithi129 ceph-mon[35314]: osdmap e110: 12 total, 12 up, 12 in 2024-09-15T22:26:02.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:02 smithi129 ceph-mon[35314]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-15T22:26:02.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:02 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1233165806' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-15T22:26:02.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:02 smithi005 ceph-mon[30944]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-15T22:26:02.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:02 smithi005 ceph-mon[30944]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-15T22:26:02.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:02 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1233165806' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-15T22:26:02.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:02 smithi005 ceph-mon[30944]: osdmap e110: 12 total, 12 up, 12 in 2024-09-15T22:26:02.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:02 smithi005 ceph-mon[30944]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-15T22:26:02.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-15T22:26:02.694 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.153+0000 7fd81ddfb640 1 -- 172.21.15.5:0/2096575685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd818102570 msgr2=0x7fd8181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.153+0000 7fd81ddfb640 1 --2- 172.21.15.5:0/2096575685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd818102570 0x7fd8181029b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fd800009900 tx=0x7fd80002a480 comp rx=0 tx=0).stop 2024-09-15T22:26:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.153+0000 7fd81ddfb640 1 -- 172.21.15.5:0/2096575685 shutdown_connections 2024-09-15T22:26:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.153+0000 7fd81ddfb640 1 --2- 172.21.15.5:0/2096575685 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd818102ef0 0x7fd81810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.153+0000 7fd81ddfb640 1 --2- 172.21.15.5:0/2096575685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd818102570 0x7fd8181029b0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.153+0000 7fd81ddfb640 1 --2- 172.21.15.5:0/2096575685 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd818100e20 0x7fd818101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:03.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.153+0000 7fd81ddfb640 1 -- 172.21.15.5:0/2096575685 >> 172.21.15.5:0/2096575685 conn(0x7fd8180fc7d0 msgr2=0x7fd8180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:03.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.153+0000 7fd81ddfb640 1 -- 172.21.15.5:0/2096575685 shutdown_connections 2024-09-15T22:26:03.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.153+0000 7fd81ddfb640 1 -- 172.21.15.5:0/2096575685 wait complete. 2024-09-15T22:26:03.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.153+0000 7fd81ddfb640 1 Processor -- start 2024-09-15T22:26:03.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd81ddfb640 1 -- start start 2024-09-15T22:26:03.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd81ddfb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd818100e20 0x7fd8180fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:03.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd81ddfb640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd818102570 0x7fd8180fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:03.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd81ddfb640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd818102ef0 0x7fd8180fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:03.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd81ddfb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd818110f80 con 0x7fd818100e20 2024-09-15T22:26:03.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd81ddfb640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7fd818110e00 con 0x7fd818102ef0 2024-09-15T22:26:03.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd81ddfb640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7fd818111100 con 0x7fd818102570 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd8177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd818100e20 0x7fd8180fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd8177fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd818100e20 0x7fd8180fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48338/0 (socket says 172.21.15.5:48338) 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd8177fe640 1 -- 172.21.15.5:0/1367126014 learned_addr learned my addr 172.21.15.5:0/1367126014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd817fff640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd818102ef0 0x7fd8180fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd80ffff640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd818102570 0x7fd8180fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd8177fe640 1 -- 172.21.15.5:0/1367126014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd818102570 msgr2=0x7fd8180fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd8177fe640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd818102570 0x7fd8180fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd8177fe640 1 -- 172.21.15.5:0/1367126014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd818102ef0 msgr2=0x7fd8180fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd8177fe640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd818102ef0 0x7fd8180fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd8177fe640 1 -- 172.21.15.5:0/1367126014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8181a6940 con 0x7fd818100e20 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd817fff640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd818102ef0 0x7fd8180fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:03.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.154+0000 7fd8177fe640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd818100e20 0x7fd8180fcc80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd80800d950 tx=0x7fd80800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.155+0000 7fd8157fa640 1 -- 172.21.15.5:0/1367126014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd808017070 con 0x7fd818100e20 2024-09-15T22:26:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.155+0000 7fd81ddfb640 1 -- 172.21.15.5:0/1367126014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8181a6c30 con 0x7fd818100e20 2024-09-15T22:26:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.155+0000 7fd8157fa640 1 -- 172.21.15.5:0/1367126014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd808009c30 con 0x7fd818100e20 2024-09-15T22:26:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.155+0000 7fd8157fa640 1 -- 172.21.15.5:0/1367126014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80800f6b0 con 0x7fd818100e20 2024-09-15T22:26:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.155+0000 7fd81ddfb640 1 -- 172.21.15.5:0/1367126014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8181a7170 con 0x7fd818100e20 2024-09-15T22:26:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.156+0000 7fd81ddfb640 1 -- 172.21.15.5:0/1367126014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd818101200 con 0x7fd818100e20 2024-09-15T22:26:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.157+0000 7fd8157fa640 1 -- 172.21.15.5:0/1367126014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7fd8080109d0 con 0x7fd818100e20 2024-09-15T22:26:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.157+0000 7fd8157fa640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd8040a33b0 0x7fd8040a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.158+0000 7fd80ffff640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd8040a33b0 0x7fd8040a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:03.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.158+0000 7fd8157fa640 1 -- 172.21.15.5:0/1367126014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9711+0+0 (secure 0 0 0) 0x7fd8080cfec0 con 0x7fd818100e20 2024-09-15T22:26:03.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.158+0000 7fd80ffff640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd8040a33b0 0x7fd8040a5870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd8000096c0 tx=0x7fd800000bc0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:03.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.162+0000 7fd8157fa640 1 -- 172.21.15.5:0/1367126014 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd808009de0 con 0x7fd818100e20 2024-09-15T22:26:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: pgmap v538: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 103 KiB/s wr, 9 op/s 2024-09-15T22:26:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.129:6834/149816973,v1:172.21.15.129:6836/149816973] up:boot 2024-09-15T22:26:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.129:6835/621642313,v1:172.21.15.129:6837/621642313] up:boot 2024-09-15T22:26:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.129:6838/3551466044,v1:172.21.15.129:6839/3551466044] up:boot 2024-09-15T22:26:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.99:6834/2536775231,v1:172.21.15.99:6835/2536775231] up:boot 2024-09-15T22:26:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.99:6836/30532124,v1:172.21.15.99:6838/30532124] up:boot 2024-09-15T22:26:03.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: mds.? [v2:172.21.15.99:6837/3751463512,v1:172.21.15.99:6839/3751463512] up:boot 2024-09-15T22:26:03.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-15T22:26:03.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:26:03.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:26:03.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:26:03.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:26:03.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:26:03.341 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:03 smithi099 ceph-mon[34892]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:26:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: pgmap v538: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 103 KiB/s wr, 9 op/s 2024-09-15T22:26:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.129:6834/149816973,v1:172.21.15.129:6836/149816973] up:boot 2024-09-15T22:26:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.129:6835/621642313,v1:172.21.15.129:6837/621642313] up:boot 2024-09-15T22:26:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.129:6838/3551466044,v1:172.21.15.129:6839/3551466044] up:boot 2024-09-15T22:26:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.99:6834/2536775231,v1:172.21.15.99:6835/2536775231] up:boot 2024-09-15T22:26:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.99:6836/30532124,v1:172.21.15.99:6838/30532124] up:boot 2024-09-15T22:26:03.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: mds.? [v2:172.21.15.99:6837/3751463512,v1:172.21.15.99:6839/3751463512] up:boot 2024-09-15T22:26:03.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-15T22:26:03.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:26:03.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:26:03.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:26:03.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:26:03.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:26:03.346 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:03 smithi129 ceph-mon[35314]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:26:03.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: pgmap v538: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 103 KiB/s wr, 9 op/s 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.129:6834/149816973,v1:172.21.15.129:6836/149816973] up:boot 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.129:6835/621642313,v1:172.21.15.129:6837/621642313] up:boot 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.129:6838/3551466044,v1:172.21.15.129:6839/3551466044] up:boot 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.99:6834/2536775231,v1:172.21.15.99:6835/2536775231] up:boot 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.99:6836/30532124,v1:172.21.15.99:6838/30532124] up:boot 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: mds.? [v2:172.21.15.99:6837/3751463512,v1:172.21.15.99:6839/3751463512] up:boot 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-15T22:26:03.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-15T22:26:03.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:03 smithi005 ceph-mon[30944]: from='mgr.15228 172.21.15.5:0/1246612925' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-15T22:26:03.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:03.374+0000 7fd81ddfb640 1 -- 172.21.15.5:0/1367126014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd81802d7c0 con 0x7fd818100e20 2024-09-15T22:26:04.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.091+0000 7fd8157fa640 1 -- 172.21.15.5:0/1367126014 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fd80809c400 con 0x7fd818100e20 2024-09-15T22:26:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.095+0000 7fd81ddfb640 1 -- 172.21.15.5:0/1367126014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd8040a33b0 msgr2=0x7fd8040a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.095+0000 7fd81ddfb640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd8040a33b0 0x7fd8040a5870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fd8000096c0 tx=0x7fd800000bc0 comp rx=0 tx=0).stop 2024-09-15T22:26:04.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.095+0000 7fd81ddfb640 1 -- 172.21.15.5:0/1367126014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd818100e20 msgr2=0x7fd8180fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.095+0000 7fd81ddfb640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd818100e20 0x7fd8180fcc80 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fd80800d950 tx=0x7fd80800de10 comp rx=0 tx=0).stop 2024-09-15T22:26:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.095+0000 7fd81ddfb640 1 -- 172.21.15.5:0/1367126014 shutdown_connections 2024-09-15T22:26:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.095+0000 7fd81ddfb640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7fd8040a33b0 0x7fd8040a5870 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.095+0000 7fd81ddfb640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7fd818102ef0 0x7fd8180fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.095+0000 7fd81ddfb640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7fd818102570 0x7fd8180fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.095+0000 7fd81ddfb640 1 --2- 172.21.15.5:0/1367126014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd818100e20 0x7fd8180fcc80 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:04.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.095+0000 7fd81ddfb640 1 -- 172.21.15.5:0/1367126014 >> 172.21.15.5:0/1367126014 conn(0x7fd8180fc7d0 msgr2=0x7fd81806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.095+0000 7fd81ddfb640 1 -- 172.21.15.5:0/1367126014 shutdown_connections 2024-09-15T22:26:04.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:04.096+0000 7fd81ddfb640 1 -- 172.21.15.5:0/1367126014 wait complete. 2024-09-15T22:26:04.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:04 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1367126014' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:26:04.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:04 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1367126014' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:26:04.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:04 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1367126014' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-15T22:26:04.831 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-15T22:26:05.096 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:05.339 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:05 smithi099 ceph-mon[34892]: pgmap v539: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 133 KiB/s wr, 12 op/s 2024-09-15T22:26:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:05 smithi099 ceph-mon[34892]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T22:26:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:05 smithi099 ceph-mon[34892]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T22:26:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:05 smithi099 ceph-mon[34892]: Cluster is now healthy 2024-09-15T22:26:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:05 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/1367126014' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:26:05.340 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:05 smithi099 ceph-mon[34892]: fsmap 12 up:standby 2024-09-15T22:26:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:05 smithi129 ceph-mon[35314]: pgmap v539: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 133 KiB/s wr, 12 op/s 2024-09-15T22:26:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:05 smithi129 ceph-mon[35314]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T22:26:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:05 smithi129 ceph-mon[35314]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T22:26:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:05 smithi129 ceph-mon[35314]: Cluster is now healthy 2024-09-15T22:26:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:05 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/1367126014' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:26:05.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:05 smithi129 ceph-mon[35314]: fsmap 12 up:standby 2024-09-15T22:26:05.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:05 smithi005 ceph-mon[30944]: pgmap v539: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 133 KiB/s wr, 12 op/s 2024-09-15T22:26:05.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:05 smithi005 ceph-mon[30944]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-15T22:26:05.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:05 smithi005 ceph-mon[30944]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-15T22:26:05.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:05 smithi005 ceph-mon[30944]: Cluster is now healthy 2024-09-15T22:26:05.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:05 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/1367126014' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-15T22:26:05.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:05 smithi005 ceph-mon[30944]: fsmap 12 up:standby 2024-09-15T22:26:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.580+0000 7f0189e3b640 1 -- 172.21.15.5:0/3013513610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0184102d30 msgr2=0x7f0184103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.580+0000 7f0189e3b640 1 --2- 172.21.15.5:0/3013513610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0184102d30 0x7f0184103110 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f0174009930 tx=0x7f017402a420 comp rx=0 tx=0).stop 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.580+0000 7f0189e3b640 1 -- 172.21.15.5:0/3013513610 shutdown_connections 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.580+0000 7f0189e3b640 1 --2- 172.21.15.5:0/3013513610 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f01841082a0 0x7f018410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.580+0000 7f0189e3b640 1 --2- 172.21.15.5:0/3013513610 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01841036e0 0x7f0184107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.580+0000 7f0189e3b640 1 --2- 172.21.15.5:0/3013513610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0184102d30 0x7f0184103110 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.580+0000 7f0189e3b640 1 -- 172.21.15.5:0/3013513610 >> 172.21.15.5:0/3013513610 conn(0x7f01840fc7d0 msgr2=0x7f01840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.580+0000 7f0189e3b640 1 -- 172.21.15.5:0/3013513610 shutdown_connections 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.580+0000 7f0189e3b640 1 -- 172.21.15.5:0/3013513610 wait complete. 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0189e3b640 1 Processor -- start 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0189e3b640 1 -- start start 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0189e3b640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0184102d30 0x7f0184100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0189e3b640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01841036e0 0x7f0184100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0189e3b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01841082a0 0x7f0184101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0189e3b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01841120a0 con 0x7f01841082a0 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0189e3b640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f0184111f20 con 0x7f0184102d30 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0189e3b640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f0184112220 con 0x7f01841036e0 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0183fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01841082a0 0x7f0184101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0183fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01841082a0 0x7f0184101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33430/0 (socket says 172.21.15.5:33430) 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0183fff640 1 -- 172.21.15.5:0/431099651 learned_addr learned my addr 172.21.15.5:0/431099651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f01837fe640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0184102d30 0x7f0184100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f017bfff640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01841036e0 0x7f0184100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0183fff640 1 -- 172.21.15.5:0/431099651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01841036e0 msgr2=0x7f0184100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.581+0000 7f0183fff640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01841036e0 0x7f0184100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.582+0000 7f0183fff640 1 -- 172.21.15.5:0/431099651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0184102d30 msgr2=0x7f0184100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.582+0000 7f0183fff640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0184102d30 0x7f0184100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.582+0000 7f0183fff640 1 -- 172.21.15.5:0/431099651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01841a6a80 con 0x7f01841082a0 2024-09-15T22:26:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.582+0000 7f01837fe640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0184102d30 0x7f0184100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T22:26:05.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.582+0000 7f0183fff640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01841082a0 0x7f0184101410 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f017c00eb10 tx=0x7f017c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:05.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.582+0000 7f01817fa640 1 -- 172.21.15.5:0/431099651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c004b20 con 0x7f01841082a0 2024-09-15T22:26:05.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.582+0000 7f0189e3b640 1 -- 172.21.15.5:0/431099651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01841a6d70 con 0x7f01841082a0 2024-09-15T22:26:05.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.582+0000 7f01817fa640 1 -- 172.21.15.5:0/431099651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f017c004cc0 con 0x7f01841082a0 2024-09-15T22:26:05.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.582+0000 7f01817fa640 1 -- 172.21.15.5:0/431099651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c010760 con 0x7f01841082a0 2024-09-15T22:26:05.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.582+0000 7f0189e3b640 1 -- 172.21.15.5:0/431099651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01841a72b0 con 0x7f01841082a0 2024-09-15T22:26:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.583+0000 7f0189e3b640 1 -- 172.21.15.5:0/431099651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f018406a850 con 0x7f01841082a0 2024-09-15T22:26:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.584+0000 7f01817fa640 1 -- 172.21.15.5:0/431099651 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f017c005020 con 0x7f01841082a0 2024-09-15T22:26:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.585+0000 7f01817fa640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f01700a3430 0x7f01700a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.585+0000 7f01837fe640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f01700a3430 0x7f01700a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.585+0000 7f01817fa640 1 -- 172.21.15.5:0/431099651 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9711+0+0 (secure 0 0 0) 0x7f017c0d0760 con 0x7f01841082a0 2024-09-15T22:26:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.585+0000 7f01837fe640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f01700a3430 0x7f01700a58f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0174009900 tx=0x7f01740058d0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.589+0000 7f01817fa640 1 -- 172.21.15.5:0/431099651 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f017c09cc40 con 0x7f01841082a0 2024-09-15T22:26:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:05.769+0000 7f0189e3b640 1 -- 172.21.15.5:0/431099651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f018410a2b0 con 0x7f01841082a0 2024-09-15T22:26:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.093+0000 7f01817fa640 1 -- 172.21.15.5:0/431099651 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) ==== 181+0+0 (secure 0 0 0) 0x7f017c09c540 con 0x7f01841082a0 2024-09-15T22:26:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-09-15T22:26:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.095+0000 7f0189e3b640 1 -- 172.21.15.5:0/431099651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f01700a3430 msgr2=0x7f01700a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.095+0000 7f0189e3b640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f01700a3430 0x7f01700a58f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0174009900 tx=0x7f01740058d0 comp rx=0 tx=0).stop 2024-09-15T22:26:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.095+0000 7f0189e3b640 1 -- 172.21.15.5:0/431099651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01841082a0 msgr2=0x7f0184101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.095+0000 7f0189e3b640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01841082a0 0x7f0184101410 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f017c00eb10 tx=0x7f017c00c2d0 comp rx=0 tx=0).stop 2024-09-15T22:26:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.096+0000 7f0189e3b640 1 -- 172.21.15.5:0/431099651 shutdown_connections 2024-09-15T22:26:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.096+0000 7f0189e3b640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f01700a3430 0x7f01700a58f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.096+0000 7f0189e3b640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01841082a0 0x7f0184101410 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.096+0000 7f0189e3b640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f01841036e0 0x7f0184100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.096+0000 7f0189e3b640 1 --2- 172.21.15.5:0/431099651 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f0184102d30 0x7f0184100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:06.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.096+0000 7f0189e3b640 1 -- 172.21.15.5:0/431099651 >> 172.21.15.5:0/431099651 conn(0x7f01840fc7d0 msgr2=0x7f0184105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.097+0000 7f0189e3b640 1 -- 172.21.15.5:0/431099651 shutdown_connections 2024-09-15T22:26:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:06.098+0000 7f0189e3b640 1 -- 172.21.15.5:0/431099651 wait complete. 2024-09-15T22:26:06.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:06 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/431099651' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:26:06.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:06 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/431099651' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:26:06.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:06 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/431099651' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:26:06.795 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-15T22:26:07.056 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:07.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:07 smithi005 ceph-mon[30944]: pgmap v540: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 155 KiB/s wr, 29 op/s 2024-09-15T22:26:07.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:07 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/431099651' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:26:07.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:07 smithi005 ceph-mon[30944]: osdmap e111: 12 total, 12 up, 12 in 2024-09-15T22:26:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.559+0000 7f18de099640 1 -- 172.21.15.5:0/4104387141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d81027a0 msgr2=0x7f18d81067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.559+0000 7f18de099640 1 --2- 172.21.15.5:0/4104387141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d81027a0 0x7f18d81067f0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009900 tx=0x7f18c002a410 comp rx=0 tx=0).stop 2024-09-15T22:26:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.559+0000 7f18de099640 1 -- 172.21.15.5:0/4104387141 shutdown_connections 2024-09-15T22:26:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.559+0000 7f18de099640 1 --2- 172.21.15.5:0/4104387141 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18d8106f20 0x7f18d810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.559+0000 7f18de099640 1 --2- 172.21.15.5:0/4104387141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d81027a0 0x7f18d81067f0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.559+0000 7f18de099640 1 --2- 172.21.15.5:0/4104387141 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18d8101df0 0x7f18d81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.559+0000 7f18de099640 1 -- 172.21.15.5:0/4104387141 >> 172.21.15.5:0/4104387141 conn(0x7f18d80fd8f0 msgr2=0x7f18d80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.559+0000 7f18de099640 1 -- 172.21.15.5:0/4104387141 shutdown_connections 2024-09-15T22:26:07.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.559+0000 7f18de099640 1 -- 172.21.15.5:0/4104387141 wait complete. 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18de099640 1 Processor -- start 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18de099640 1 -- start start 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18de099640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8101df0 0x7f18d81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18de099640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18d81027a0 0x7f18d81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18de099640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18d8106f20 0x7f18d819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18de099640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18d8111f80 con 0x7f18d8101df0 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18de099640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f18d8111e00 con 0x7f18d8106f20 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18de099640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f18d8112100 con 0x7f18d81027a0 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8101df0 0x7f18d81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18d77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8101df0 0x7f18d81984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33452/0 (socket says 172.21.15.5:33452) 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18d77fe640 1 -- 172.21.15.5:0/93659704 learned_addr learned my addr 172.21.15.5:0/93659704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18d7fff640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18d8106f20 0x7f18d819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:07.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18ceffd640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18d81027a0 0x7f18d81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18d77fe640 1 -- 172.21.15.5:0/93659704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18d81027a0 msgr2=0x7f18d81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18d77fe640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18d81027a0 0x7f18d81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18d77fe640 1 -- 172.21.15.5:0/93659704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18d8106f20 msgr2=0x7f18d819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18d77fe640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18d8106f20 0x7f18d819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18d77fe640 1 -- 172.21.15.5:0/93659704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d819d450 con 0x7f18d8101df0 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18d7fff640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18d8106f20 0x7f18d819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.560+0000 7f18ceffd640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18d81027a0 0x7f18d81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.561+0000 7f18d77fe640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8101df0 0x7f18d81984a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f18c800d950 tx=0x7f18c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.561+0000 7f18d57fa640 1 -- 172.21.15.5:0/93659704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8017030 con 0x7f18d8101df0 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.561+0000 7f18de099640 1 -- 172.21.15.5:0/93659704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d819d740 con 0x7f18d8101df0 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.561+0000 7f18d57fa640 1 -- 172.21.15.5:0/93659704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c8004020 con 0x7f18d8101df0 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.561+0000 7f18d57fa640 1 -- 172.21.15.5:0/93659704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c80047a0 con 0x7f18d8101df0 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.561+0000 7f18de099640 1 -- 172.21.15.5:0/93659704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d81a5020 con 0x7f18d8101df0 2024-09-15T22:26:07.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.562+0000 7f18de099640 1 -- 172.21.15.5:0/93659704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d81021d0 con 0x7f18d8101df0 2024-09-15T22:26:07.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.563+0000 7f18d57fa640 1 -- 172.21.15.5:0/93659704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f18c80041d0 con 0x7f18d8101df0 2024-09-15T22:26:07.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.564+0000 7f18d57fa640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18c40a33b0 0x7f18c40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:07.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.564+0000 7f18ceffd640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18c40a33b0 0x7f18c40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:07.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.564+0000 7f18d57fa640 1 -- 172.21.15.5:0/93659704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9170+0+0 (secure 0 0 0) 0x7f18c80cfa50 con 0x7f18d8101df0 2024-09-15T22:26:07.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.565+0000 7f18ceffd640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18c40a33b0 0x7f18c40a5870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f18c0000c00 tx=0x7f18c0032c70 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:07.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.569+0000 7f18d57fa640 1 -- 172.21.15.5:0/93659704 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18c809c1a0 con 0x7f18d8101df0 2024-09-15T22:26:07.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:07 smithi099 ceph-mon[34892]: pgmap v540: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 155 KiB/s wr, 29 op/s 2024-09-15T22:26:07.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:07 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/431099651' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:26:07.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:07 smithi099 ceph-mon[34892]: osdmap e111: 12 total, 12 up, 12 in 2024-09-15T22:26:07.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:07 smithi129 ceph-mon[35314]: pgmap v540: 129 pgs: 129 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 155 KiB/s wr, 29 op/s 2024-09-15T22:26:07.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:07 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/431099651' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:26:07.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:07 smithi129 ceph-mon[35314]: osdmap e111: 12 total, 12 up, 12 in 2024-09-15T22:26:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:07.746+0000 7f18de099640 1 -- 172.21.15.5:0/93659704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f18d810a160 con 0x7f18d8101df0 2024-09-15T22:26:08.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.099+0000 7f18d57fa640 1 -- 172.21.15.5:0/93659704 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) ==== 169+0+0 (secure 0 0 0) 0x7f18c809baa0 con 0x7f18d8101df0 2024-09-15T22:26:08.102 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-09-15T22:26:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.102+0000 7f18de099640 1 -- 172.21.15.5:0/93659704 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18c40a33b0 msgr2=0x7f18c40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.102+0000 7f18de099640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18c40a33b0 0x7f18c40a5870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f18c0000c00 tx=0x7f18c0032c70 comp rx=0 tx=0).stop 2024-09-15T22:26:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.102+0000 7f18de099640 1 -- 172.21.15.5:0/93659704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8101df0 msgr2=0x7f18d81984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:08.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.102+0000 7f18de099640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8101df0 0x7f18d81984a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f18c800d950 tx=0x7f18c800de10 comp rx=0 tx=0).stop 2024-09-15T22:26:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.103+0000 7f18de099640 1 -- 172.21.15.5:0/93659704 shutdown_connections 2024-09-15T22:26:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.103+0000 7f18de099640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f18c40a33b0 0x7f18c40a5870 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.103+0000 7f18de099640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f18d8106f20 0x7f18d819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.103+0000 7f18de099640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f18d81027a0 0x7f18d81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.103+0000 7f18de099640 1 --2- 172.21.15.5:0/93659704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d8101df0 0x7f18d81984a0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.103+0000 7f18de099640 1 -- 172.21.15.5:0/93659704 >> 172.21.15.5:0/93659704 conn(0x7f18d80fd8f0 msgr2=0x7f18d80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.104+0000 7f18de099640 1 -- 172.21.15.5:0/93659704 shutdown_connections 2024-09-15T22:26:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:08.104+0000 7f18de099640 1 -- 172.21.15.5:0/93659704 wait complete. 2024-09-15T22:26:08.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:08 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/93659704' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:26:08.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:08 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/93659704' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:26:08.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:08 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/93659704' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:26:08.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 962891f2-73ad-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-15T22:26:09.090 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/mon.a/config 2024-09-15T22:26:09.354 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:09 smithi005 ceph-mon[30944]: pgmap v542: 97 pgs: 97 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:26:09.354 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:09 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/93659704' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:26:09.354 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:09 smithi005 ceph-mon[30944]: osdmap e112: 12 total, 12 up, 12 in 2024-09-15T22:26:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.552+0000 7f48f0d87640 1 -- 172.21.15.5:0/172835847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48ec106f10 msgr2=0x7f48ec10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.552+0000 7f48f0d87640 1 --2- 172.21.15.5:0/172835847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48ec106f10 0x7f48ec10ea90 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009ca0 tx=0x7f48e402a8c0 comp rx=0 tx=0).stop 2024-09-15T22:26:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.552+0000 7f48f0d87640 1 -- 172.21.15.5:0/172835847 shutdown_connections 2024-09-15T22:26:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.552+0000 7f48f0d87640 1 --2- 172.21.15.5:0/172835847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48ec106f10 0x7f48ec10ea90 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.552+0000 7f48f0d87640 1 --2- 172.21.15.5:0/172835847 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec102790 0x7f48ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:09.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.552+0000 7f48f0d87640 1 --2- 172.21.15.5:0/172835847 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48ec101de0 0x7f48ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.552+0000 7f48f0d87640 1 -- 172.21.15.5:0/172835847 >> 172.21.15.5:0/172835847 conn(0x7f48ec0fd8c0 msgr2=0x7f48ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.552+0000 7f48f0d87640 1 -- 172.21.15.5:0/172835847 shutdown_connections 2024-09-15T22:26:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.552+0000 7f48f0d87640 1 -- 172.21.15.5:0/172835847 wait complete. 2024-09-15T22:26:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48f0d87640 1 Processor -- start 2024-09-15T22:26:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48f0d87640 1 -- start start 2024-09-15T22:26:09.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48f0d87640 1 --2- >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48ec101de0 0x7f48ec0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48f0d87640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48ec102790 0x7f48ec074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48f0d87640 1 --2- >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec106f10 0x7f48ec074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:09.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48f0d87640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec111f70 con 0x7f48ec102790 2024-09-15T22:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48f0d87640 1 -- --> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec111df0 con 0x7f48ec106f10 2024-09-15T22:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48f0d87640 1 -- --> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] -- mon_getmap magic: 0 -- 0x7f48ec1120f0 con 0x7f48ec101de0 2024-09-15T22:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48ec102790 0x7f48ec074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48e9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48ec102790 0x7f48ec074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33476/0 (socket says 172.21.15.5:33476) 2024-09-15T22:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48e9d9b640 1 -- 172.21.15.5:0/3578669579 learned_addr learned my addr 172.21.15.5:0/3578669579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-15T22:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.553+0000 7f48ead9d640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec106f10 0x7f48ec074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48ea59c640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48ec101de0 0x7f48ec0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48e9d9b640 1 -- 172.21.15.5:0/3578669579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48ec101de0 msgr2=0x7f48ec0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48e9d9b640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48ec101de0 0x7f48ec0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48e9d9b640 1 -- 172.21.15.5:0/3578669579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec106f10 msgr2=0x7f48ec074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:09.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48e9d9b640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec106f10 0x7f48ec074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48e9d9b640 1 -- 172.21.15.5:0/3578669579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48ec1a67f0 con 0x7f48ec102790 2024-09-15T22:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48ea59c640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48ec101de0 0x7f48ec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T22:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48e9d9b640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48ec102790 0x7f48ec074a30 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f48e000d950 tx=0x7f48e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48db7fe640 1 -- 172.21.15.5:0/3578669579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0017070 con 0x7f48ec102790 2024-09-15T22:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48db7fe640 1 -- 172.21.15.5:0/3578669579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48e0009c30 con 0x7f48ec102790 2024-09-15T22:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48f0d87640 1 -- 172.21.15.5:0/3578669579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48ec1a6ae0 con 0x7f48ec102790 2024-09-15T22:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48db7fe640 1 -- 172.21.15.5:0/3578669579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e000f6b0 con 0x7f48ec102790 2024-09-15T22:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.554+0000 7f48f0d87640 1 -- 172.21.15.5:0/3578669579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48ec1a7020 con 0x7f48ec102790 2024-09-15T22:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.555+0000 7f48f0d87640 1 -- 172.21.15.5:0/3578669579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48ec1021c0 con 0x7f48ec102790 2024-09-15T22:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.557+0000 7f48db7fe640 1 -- 172.21.15.5:0/3578669579 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 26) ==== 138485+0+0 (secure 0 0 0) 0x7f48e00109d0 con 0x7f48ec102790 2024-09-15T22:26:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.557+0000 7f48db7fe640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48d40a33b0 0x7f48d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T22:26:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.557+0000 7f48ea59c640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48d40a33b0 0x7f48d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T22:26:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.557+0000 7f48db7fe640 1 -- 172.21.15.5:0/3578669579 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8582+0+0 (secure 0 0 0) 0x7f48e00cf8f0 con 0x7f48ec102790 2024-09-15T22:26:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.558+0000 7f48ea59c640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48d40a33b0 0x7f48d40a5870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00abe0 tx=0x7f48dc00a5c0 comp rx=0 tx=0).ready entity=mgr.15228 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T22:26:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.561+0000 7f48db7fe640 1 -- 172.21.15.5:0/3578669579 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48e0009de0 con 0x7f48ec102790 2024-09-15T22:26:09.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:09 smithi099 ceph-mon[34892]: pgmap v542: 97 pgs: 97 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:26:09.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:09 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/93659704' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:26:09.590 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:09 smithi099 ceph-mon[34892]: osdmap e112: 12 total, 12 up, 12 in 2024-09-15T22:26:09.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:09 smithi129 ceph-mon[35314]: pgmap v542: 97 pgs: 97 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:26:09.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:09 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/93659704' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:26:09.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:09 smithi129 ceph-mon[35314]: osdmap e112: 12 total, 12 up, 12 in 2024-09-15T22:26:09.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:09.735+0000 7f48f0d87640 1 -- 172.21.15.5:0/3578669579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f48ec075a20 con 0x7f48ec102790 2024-09-15T22:26:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.106+0000 7f48db7fe640 1 -- 172.21.15.5:0/3578669579 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v113) ==== 178+0+0 (secure 0 0 0) 0x7f48e009c290 con 0x7f48ec102790 2024-09-15T22:26:10.108 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' removed 2024-09-15T22:26:10.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.109+0000 7f48f0d87640 1 -- 172.21.15.5:0/3578669579 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48d40a33b0 msgr2=0x7f48d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:10.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.109+0000 7f48f0d87640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48d40a33b0 0x7f48d40a5870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00abe0 tx=0x7f48dc00a5c0 comp rx=0 tx=0).stop 2024-09-15T22:26:10.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.109+0000 7f48f0d87640 1 -- 172.21.15.5:0/3578669579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48ec102790 msgr2=0x7f48ec074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T22:26:10.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.109+0000 7f48f0d87640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48ec102790 0x7f48ec074a30 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f48e000d950 tx=0x7f48e000de10 comp rx=0 tx=0).stop 2024-09-15T22:26:10.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.109+0000 7f48f0d87640 1 -- 172.21.15.5:0/3578669579 shutdown_connections 2024-09-15T22:26:10.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.109+0000 7f48f0d87640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.5:6800/1448662997,v1:172.21.15.5:6801/1448662997] conn(0x7f48d40a33b0 0x7f48d40a5870 secure :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f48dc00abe0 tx=0x7f48dc00a5c0 comp rx=0 tx=0).stop 2024-09-15T22:26:10.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.109+0000 7f48f0d87640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] conn(0x7f48ec106f10 0x7f48ec074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:10.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.109+0000 7f48f0d87640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48ec102790 0x7f48ec074a30 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:10.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.109+0000 7f48f0d87640 1 --2- 172.21.15.5:0/3578669579 >> [v2:172.21.15.129:3300/0,v1:172.21.15.129:6789/0] conn(0x7f48ec101de0 0x7f48ec0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T22:26:10.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.109+0000 7f48f0d87640 1 -- 172.21.15.5:0/3578669579 >> 172.21.15.5:0/3578669579 conn(0x7f48ec0fd8c0 msgr2=0x7f48ec0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T22:26:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.110+0000 7f48f0d87640 1 -- 172.21.15.5:0/3578669579 shutdown_connections 2024-09-15T22:26:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-15T22:26:10.110+0000 7f48f0d87640 1 -- 172.21.15.5:0/3578669579 wait complete. 2024-09-15T22:26:10.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:10 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3578669579' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:26:10.589 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:10 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3578669579' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:26:10.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:10 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3578669579' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-15T22:26:10.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:26:10.986 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:26:11.076 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:26:11.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:11 smithi129 ceph-mon[35314]: pgmap v544: 33 pgs: 33 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:26:11.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:11 smithi129 ceph-mon[35314]: from='client.? 172.21.15.5:0/3578669579' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:26:11.345 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:11 smithi129 ceph-mon[35314]: osdmap e113: 12 total, 12 up, 12 in 2024-09-15T22:26:11.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:11 smithi099 ceph-mon[34892]: pgmap v544: 33 pgs: 33 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:26:11.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:11 smithi099 ceph-mon[34892]: from='client.? 172.21.15.5:0/3578669579' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:26:11.346 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:11 smithi099 ceph-mon[34892]: osdmap e113: 12 total, 12 up, 12 in 2024-09-15T22:26:11.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:11 smithi005 ceph-mon[30944]: pgmap v544: 33 pgs: 33 active+clean; 2.5 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-15T22:26:11.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:11 smithi005 ceph-mon[30944]: from='client.? 172.21.15.5:0/3578669579' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-15T22:26:11.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:11 smithi005 ceph-mon[30944]: osdmap e113: 12 total, 12 up, 12 in 2024-09-15T22:26:11.426 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-15T22:26:11.426 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:26:11.464 DEBUG:teuthology.orchestra.run.smithi099:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:26:11.498 DEBUG:teuthology.orchestra.run.smithi129:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-15T22:26:11.530 INFO:tasks.cephadm:Stopping all daemons... 2024-09-15T22:26:11.530 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-15T22:26:11.530 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.a 2024-09-15T22:26:11.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:11 smithi005 systemd[1]: Stopping Ceph mon.a for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:26:13.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:12 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a[30940]: 2024-09-15T22:26:12.895+0000 7ffbcfac5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:26:13.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:12 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a[30940]: 2024-09-15T22:26:12.895+0000 7ffbcfac5640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-15T22:26:13.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:13 smithi005 podman[120531]: 2024-09-15 22:26:13.15582555 +0000 UTC m=+1.191709665 container died e647ceea0e91a4de76a22cd0c39530d5dfe161dd828180b5016ad05ab14aa160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:26:14.856 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:14 smithi005 podman[120531]: 2024-09-15 22:26:14.729168104 +0000 UTC m=+2.765052215 container cleanup e647ceea0e91a4de76a22cd0c39530d5dfe161dd828180b5016ad05ab14aa160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-15T22:26:14.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:14 smithi005 bash[120531]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a 2024-09-15T22:26:15.316 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 15 22:26:14 smithi005 podman[120543]: 2024-09-15 22:26:14.939206503 +0000 UTC m=+1.778641605 container remove e647ceea0e91a4de76a22cd0c39530d5dfe161dd828180b5016ad05ab14aa160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-15T22:26:15.328 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.a.service' 2024-09-15T22:26:15.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:26:15.404 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-15T22:26:15.404 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-15T22:26:15.404 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.b 2024-09-15T22:26:15.726 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:15 smithi099 systemd[1]: Stopping Ceph mon.b for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:26:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:15 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-b[34888]: 2024-09-15T22:26:15.928+0000 7f2ab7700640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:26:16.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:15 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-b[34888]: 2024-09-15T22:26:15.928+0000 7f2ab7700640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-15T22:26:16.449 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:16 smithi099 podman[71489]: 2024-09-15 22:26:16.123215397 +0000 UTC m=+0.365171864 container died 4f087ea6937a3f4ba6b1bffaff90fe2c4bd09aebd0487f60adb962b479feeda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-15T22:26:16.816 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:16 smithi099 podman[71489]: 2024-09-15 22:26:16.554654768 +0000 UTC m=+0.796611236 container cleanup 4f087ea6937a3f4ba6b1bffaff90fe2c4bd09aebd0487f60adb962b479feeda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-b, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-15T22:26:16.816 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:16 smithi099 bash[71489]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-b 2024-09-15T22:26:17.041 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.b.service' 2024-09-15T22:26:17.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:16 smithi099 podman[71504]: 2024-09-15 22:26:16.815017472 +0000 UTC m=+0.689552606 container remove 4f087ea6937a3f4ba6b1bffaff90fe2c4bd09aebd0487f60adb962b479feeda8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-15T22:26:17.089 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:17 smithi099 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-15T22:26:17.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:17 smithi099 systemd[1]: Stopped Ceph mon.b for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:26:17.090 INFO:journalctl@ceph.mon.b.smithi099.stdout:Sep 15 22:26:17 smithi099 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.b.service: Consumed 16.024s CPU time. 2024-09-15T22:26:17.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:26:17.104 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-15T22:26:17.104 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-15T22:26:17.104 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.c 2024-09-15T22:26:17.595 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:17 smithi129 systemd[1]: Stopping Ceph mon.c for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:26:19.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:19 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-c[35310]: 2024-09-15T22:26:19.360+0000 7f8267c87640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:26:19.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:19 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-c[35310]: 2024-09-15T22:26:19.360+0000 7f8267c87640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-15T22:26:19.845 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:19 smithi129 podman[72209]: 2024-09-15 22:26:19.554877214 +0000 UTC m=+1.722037316 container died 43f1b4bee469bb60e422e7f59c2b956af3e55cef66ebc218d46d80baf77ea757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-15T22:26:23.028 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:22 smithi129 podman[72209]: 2024-09-15 22:26:22.816555434 +0000 UTC m=+4.983715575 container cleanup 43f1b4bee469bb60e422e7f59c2b956af3e55cef66ebc218d46d80baf77ea757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-c, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-15T22:26:23.028 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:22 smithi129 bash[72209]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-c 2024-09-15T22:26:23.257 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.c.service' 2024-09-15T22:26:23.279 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:23 smithi129 podman[72224]: 2024-09-15 22:26:23.026811323 +0000 UTC m=+3.454360706 container remove 43f1b4bee469bb60e422e7f59c2b956af3e55cef66ebc218d46d80baf77ea757 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-15T22:26:23.279 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:23 smithi129 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-15T22:26:23.279 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:23 smithi129 systemd[1]: Stopped Ceph mon.c for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:26:23.279 INFO:journalctl@ceph.mon.c.smithi129.stdout:Sep 15 22:26:23 smithi129 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mon.c.service: Consumed 15.751s CPU time. 2024-09-15T22:26:23.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:26:23.306 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-15T22:26:23.306 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-15T22:26:23.306 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.x 2024-09-15T22:26:23.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:26:23 smithi005 systemd[1]: Stopping Ceph mgr.x for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:26:25.606 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:26:25 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:26:25.343+0000 7f779fff8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:26:25.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:26:25 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-15T22:26:25.343+0000 7f779fff8640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-15T22:26:25.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:26:25 smithi005 podman[120644]: 2024-09-15 22:26:25.38465177 +0000 UTC m=+1.729242269 container died 4b0c5ec44a62dffed6f4b784c0cc9bffd5a517df2f73d8abdaef4dec5b41c4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-15T22:26:27.680 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:26:27 smithi005 podman[120644]: 2024-09-15 22:26:27.468842807 +0000 UTC m=+3.813433305 container cleanup 4b0c5ec44a62dffed6f4b784c0cc9bffd5a517df2f73d8abdaef4dec5b41c4f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-15T22:26:27.680 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 15 22:26:27 smithi005 bash[120644]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-x 2024-09-15T22:26:27.821 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-15T22:26:27.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:26:27.858 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-15T22:26:27.858 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-15T22:26:27.858 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.y 2024-09-15T22:26:28.223 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:26:27 smithi099 systemd[1]: Stopping Ceph mgr.y for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:26:28.590 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:26:28 smithi099 podman[71615]: 2024-09-15 22:26:28.222846746 +0000 UTC m=+0.163711829 container died d366fade87e0218028b575a13be9663c2d9fe7a6c0fa06afe1735326013b9213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-15T22:26:28.995 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:26:28 smithi099 podman[71615]: 2024-09-15 22:26:28.684524955 +0000 UTC m=+0.625390046 container cleanup d366fade87e0218028b575a13be9663c2d9fe7a6c0fa06afe1735326013b9213 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-15T22:26:28.996 INFO:journalctl@ceph.mgr.y.smithi099.stdout:Sep 15 22:26:28 smithi099 bash[71615]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-y 2024-09-15T22:26:29.128 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-15T22:26:29.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:26:29.162 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-15T22:26:29.163 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-15T22:26:29.163 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.z 2024-09-15T22:26:29.449 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:26:29 smithi129 systemd[1]: Stopping Ceph mgr.z for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:26:29.845 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:26:29 smithi129 podman[72334]: 2024-09-15 22:26:29.53455127 +0000 UTC m=+0.175592689 container stop 1ca9519e701b5ef611240d93727f7751deeb240a6758b6a8c245b39e56ac5ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e) 2024-09-15T22:26:29.845 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:26:29 smithi129 podman[72334]: 2024-09-15 22:26:29.560146245 +0000 UTC m=+0.201187663 container died 1ca9519e701b5ef611240d93727f7751deeb240a6758b6a8c245b39e56ac5ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-15T22:26:30.258 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:26:30 smithi129 podman[72334]: 2024-09-15 22:26:30.022576432 +0000 UTC m=+0.663617848 container cleanup 1ca9519e701b5ef611240d93727f7751deeb240a6758b6a8c245b39e56ac5ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-15T22:26:30.259 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:26:30 smithi129 bash[72334]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z 2024-09-15T22:26:30.490 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-15T22:26:30.512 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:26:30 smithi129 podman[72347]: 2024-09-15 22:26:30.257854048 +0000 UTC m=+0.717534186 container remove 1ca9519e701b5ef611240d93727f7751deeb240a6758b6a8c245b39e56ac5ea2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-15T22:26:30.512 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:26:30 smithi129 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-09-15T22:26:30.512 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:26:30 smithi129 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-09-15T22:26:30.513 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:26:30 smithi129 systemd[1]: Stopped Ceph mgr.z for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:26:30.513 INFO:journalctl@ceph.mgr.z.smithi129.stdout:Sep 15 22:26:30 smithi129 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mgr.z.service: Consumed 11.759s CPU time. 2024-09-15T22:26:30.526 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:26:30.526 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-15T22:26:30.526 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-15T22:26:30.526 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.0 2024-09-15T22:26:30.856 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:30 smithi005 systemd[1]: Stopping Ceph osd.0 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:26:31.356 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:30 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0[44440]: 2024-09-15T22:26:30.856+0000 7f8be6746640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:26:31.357 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:30 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0[44440]: 2024-09-15T22:26:30.856+0000 7f8be6746640 -1 osd.0 113 *** Got signal Terminated *** 2024-09-15T22:26:31.357 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:30 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0[44440]: 2024-09-15T22:26:30.856+0000 7f8be6746640 -1 osd.0 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:26:36.818 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:36 smithi005 podman[120765]: 2024-09-15 22:26:36.510648418 +0000 UTC m=+5.773036121 container died dbbba2aea54487d898443c2fc4651e5478e9f85e1bc990acc575f44b03737a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2) 2024-09-15T22:26:37.106 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:36 smithi005 podman[120765]: 2024-09-15 22:26:36.906933711 +0000 UTC m=+6.169321404 container cleanup dbbba2aea54487d898443c2fc4651e5478e9f85e1bc990acc575f44b03737a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True) 2024-09-15T22:26:37.107 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:36 smithi005 bash[120765]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0 2024-09-15T22:26:37.526 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:37 smithi005 podman[120781]: 2024-09-15 22:26:37.200260051 +0000 UTC m=+0.670974250 container remove dbbba2aea54487d898443c2fc4651e5478e9f85e1bc990acc575f44b03737a34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-15T22:26:37.821 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:37 smithi005 podman[120874]: 2024-09-15 22:26:37.428070831 +0000 UTC m=+0.016421819 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:26:37.821 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:37 smithi005 podman[120874]: 2024-09-15 22:26:37.568024671 +0000 UTC m=+0.156375693 container create fbdb546dc07b241c8a5550032f60eaa4cd413a7665a9a12936af7a54806c6986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-15T22:26:37.821 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:37 smithi005 podman[120874]: 2024-09-15 22:26:37.820474774 +0000 UTC m=+0.408825751 container init fbdb546dc07b241c8a5550032f60eaa4cd413a7665a9a12936af7a54806c6986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-15T22:26:38.106 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:37 smithi005 podman[120874]: 2024-09-15 22:26:37.824242132 +0000 UTC m=+0.412593109 container start fbdb546dc07b241c8a5550032f60eaa4cd413a7665a9a12936af7a54806c6986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-15T22:26:38.106 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:37 smithi005 podman[120874]: 2024-09-15 22:26:37.929402409 +0000 UTC m=+0.517753467 container attach fbdb546dc07b241c8a5550032f60eaa4cd413a7665a9a12936af7a54806c6986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:26:38.607 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 15 22:26:38 smithi005 podman[120897]: 2024-09-15 22:26:38.241400821 +0000 UTC m=+0.017583594 container died fbdb546dc07b241c8a5550032f60eaa4cd413a7665a9a12936af7a54806c6986 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-15T22:26:38.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.0.service' 2024-09-15T22:26:38.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:26:38.999 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-15T22:26:38.999 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-15T22:26:39.000 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.1 2024-09-15T22:26:39.339 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:39 smithi099 systemd[1]: Stopping Ceph osd.1 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:26:39.839 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:39 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1[39339]: 2024-09-15T22:26:39.346+0000 7fb4ae359640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:26:39.840 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:39 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1[39339]: 2024-09-15T22:26:39.346+0000 7fb4ae359640 -1 osd.1 113 *** Got signal Terminated *** 2024-09-15T22:26:39.840 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:39 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1[39339]: 2024-09-15T22:26:39.346+0000 7fb4ae359640 -1 osd.1 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:26:45.340 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:45 smithi099 podman[71750]: 2024-09-15 22:26:45.075557255 +0000 UTC m=+5.874889508 container died f8feb1ca0875c62822c1c9287bcc9bea8138306590edc5796e858efc307a83fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:26:45.750 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:45 smithi099 podman[71750]: 2024-09-15 22:26:45.489016661 +0000 UTC m=+6.288348959 container cleanup f8feb1ca0875c62822c1c9287bcc9bea8138306590edc5796e858efc307a83fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:26:45.750 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:45 smithi099 bash[71750]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1 2024-09-15T22:26:46.057 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:45 smithi099 podman[71765]: 2024-09-15 22:26:45.74884489 +0000 UTC m=+0.668899895 container remove f8feb1ca0875c62822c1c9287bcc9bea8138306590edc5796e858efc307a83fe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-15T22:26:46.329 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:46 smithi099 podman[71852]: 2024-09-15 22:26:45.958782877 +0000 UTC m=+0.015834414 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:26:46.329 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:46 smithi099 podman[71852]: 2024-09-15 22:26:46.074934268 +0000 UTC m=+0.131985802 container create e013abfdb79c4be467e703d562838a069872b938ebfef1af831e61bb9efd2b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731) 2024-09-15T22:26:46.329 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:46 smithi099 podman[71852]: 2024-09-15 22:26:46.328076553 +0000 UTC m=+0.385128087 container init e013abfdb79c4be467e703d562838a069872b938ebfef1af831e61bb9efd2b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1-deactivate, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-15T22:26:46.589 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:46 smithi099 podman[71852]: 2024-09-15 22:26:46.331475302 +0000 UTC m=+0.388526835 container start e013abfdb79c4be467e703d562838a069872b938ebfef1af831e61bb9efd2b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-15T22:26:46.590 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:46 smithi099 podman[71852]: 2024-09-15 22:26:46.436619965 +0000 UTC m=+0.493671500 container attach e013abfdb79c4be467e703d562838a069872b938ebfef1af831e61bb9efd2b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-15T22:26:47.025 INFO:journalctl@ceph.osd.1.smithi099.stdout:Sep 15 22:26:46 smithi099 podman[71852]: 2024-09-15 22:26:46.663429162 +0000 UTC m=+0.720480696 container died e013abfdb79c4be467e703d562838a069872b938ebfef1af831e61bb9efd2b9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True) 2024-09-15T22:26:47.327 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.1.service' 2024-09-15T22:26:47.367 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:26:47.367 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-15T22:26:47.367 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-15T22:26:47.367 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.2 2024-09-15T22:26:47.679 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:47 smithi129 systemd[1]: Stopping Ceph osd.2 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:26:48.095 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:47 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2[40242]: 2024-09-15T22:26:47.677+0000 7f62cbcac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:26:48.095 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:47 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2[40242]: 2024-09-15T22:26:47.677+0000 7f62cbcac640 -1 osd.2 113 *** Got signal Terminated *** 2024-09-15T22:26:48.095 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:47 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2[40242]: 2024-09-15T22:26:47.677+0000 7f62cbcac640 -1 osd.2 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:26:53.771 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:53 smithi129 podman[72470]: 2024-09-15 22:26:53.44075345 +0000 UTC m=+5.885554892 container died fc68eca7441354e59d1b9d243db60166e954251778b4c5d1989c5aee385cfe59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-15T22:26:54.095 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:53 smithi129 podman[72470]: 2024-09-15 22:26:53.87762345 +0000 UTC m=+6.322424879 container cleanup fc68eca7441354e59d1b9d243db60166e954251778b4c5d1989c5aee385cfe59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:26:54.095 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:53 smithi129 bash[72470]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2 2024-09-15T22:26:54.489 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:54 smithi129 podman[72485]: 2024-09-15 22:26:54.095748596 +0000 UTC m=+0.657250738 container remove fc68eca7441354e59d1b9d243db60166e954251778b4c5d1989c5aee385cfe59 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, ceph=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-15T22:26:54.784 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:54 smithi129 podman[72574]: 2024-09-15 22:26:54.391524072 +0000 UTC m=+0.015556819 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:26:54.784 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:54 smithi129 podman[72574]: 2024-09-15 22:26:54.513644169 +0000 UTC m=+0.137676954 container create 50cd91939315283a8259b7e800e99fcf604bd36d2a195231fe636fd3c868e6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-15T22:26:54.784 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:54 smithi129 podman[72574]: 2024-09-15 22:26:54.783031106 +0000 UTC m=+0.407063853 container init 50cd91939315283a8259b7e800e99fcf604bd36d2a195231fe636fd3c868e6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-15T22:26:55.095 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:54 smithi129 podman[72574]: 2024-09-15 22:26:54.786580644 +0000 UTC m=+0.410613390 container start 50cd91939315283a8259b7e800e99fcf604bd36d2a195231fe636fd3c868e6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:26:55.095 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:54 smithi129 podman[72574]: 2024-09-15 22:26:54.933552961 +0000 UTC m=+0.557585699 container attach 50cd91939315283a8259b7e800e99fcf604bd36d2a195231fe636fd3c868e6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=squid-8921c8e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-15T22:26:55.447 INFO:journalctl@ceph.osd.2.smithi129.stdout:Sep 15 22:26:55 smithi129 podman[72574]: 2024-09-15 22:26:55.178033041 +0000 UTC m=+0.802065788 container died 50cd91939315283a8259b7e800e99fcf604bd36d2a195231fe636fd3c868e6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-15T22:26:55.859 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.2.service' 2024-09-15T22:26:55.900 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:26:55.900 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-15T22:26:55.900 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-15T22:26:55.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.3 2024-09-15T22:26:56.229 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:26:55 smithi005 systemd[1]: Stopping Ceph osd.3 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:26:56.606 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:26:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3[51213]: 2024-09-15T22:26:56.227+0000 7f8b9dea3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:26:56.606 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:26:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3[51213]: 2024-09-15T22:26:56.227+0000 7f8b9dea3640 -1 osd.3 113 *** Got signal Terminated *** 2024-09-15T22:26:56.607 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:26:56 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3[51213]: 2024-09-15T22:26:56.227+0000 7f8b9dea3640 -1 osd.3 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:26:58.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:26:57 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:26:57.855+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:26:58.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:26:57 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:26:57.923+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:26:59.247 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:26:58 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:26:58.878+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:26:59.339 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:26:58 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:26:58.861+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:26:59.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:26:58 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:26:58.957+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:00.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:26:59 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:26:59.881+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:00.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:26:59 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:26:59.873+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:00.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:26:59 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:26:59.953+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:01.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:00 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:00.874+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:01.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:00 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:00.866+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:01.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:00 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:00.994+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:02.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:01 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:01.863+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:02.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:01.888+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:02.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:02 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:02.026+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:02.357 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:27:01 smithi005 podman[120988]: 2024-09-15 22:27:01.972024757 +0000 UTC m=+5.864239772 container died 67b7b99eb6a038d6e74be7c1c2deee0021fa3602fbf195ea05457ee1cdf570f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-15T22:27:02.856 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:27:02 smithi005 podman[120988]: 2024-09-15 22:27:02.406898552 +0000 UTC m=+6.299113608 container cleanup 67b7b99eb6a038d6e74be7c1c2deee0021fa3602fbf195ea05457ee1cdf570f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-15T22:27:02.857 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:27:02 smithi005 bash[120988]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3 2024-09-15T22:27:02.857 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:27:02 smithi005 podman[121004]: 2024-09-15 22:27:02.612444927 +0000 UTC m=+0.640977385 container remove 67b7b99eb6a038d6e74be7c1c2deee0021fa3602fbf195ea05457ee1cdf570f8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-15T22:27:03.308 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:03 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:03.052+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:03.309 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:27:03 smithi005 podman[121094]: 2024-09-15 22:27:02.937165488 +0000 UTC m=+0.016286550 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:27:03.309 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:27:03 smithi005 podman[121094]: 2024-09-15 22:27:03.055028263 +0000 UTC m=+0.134149367 container create 990bd7eb64cb1b95dc5b449477ffee5fdeb37290c0be5c28df1bd1c27a0938e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-15T22:27:03.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:02 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:02.845+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:03.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:02 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:02.904+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:03.607 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:27:03 smithi005 podman[121094]: 2024-09-15 22:27:03.307794397 +0000 UTC m=+0.386915459 container init 990bd7eb64cb1b95dc5b449477ffee5fdeb37290c0be5c28df1bd1c27a0938e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-15T22:27:03.607 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:27:03 smithi005 podman[121094]: 2024-09-15 22:27:03.311448827 +0000 UTC m=+0.390569889 container start 990bd7eb64cb1b95dc5b449477ffee5fdeb37290c0be5c28df1bd1c27a0938e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-15T22:27:03.607 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:27:03 smithi005 podman[121094]: 2024-09-15 22:27:03.407567975 +0000 UTC m=+0.486689037 container attach 990bd7eb64cb1b95dc5b449477ffee5fdeb37290c0be5c28df1bd1c27a0938e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:27:03.607 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 15 22:27:03 smithi005 podman[121094]: 2024-09-15 22:27:03.501224154 +0000 UTC m=+0.580345215 container died 990bd7eb64cb1b95dc5b449477ffee5fdeb37290c0be5c28df1bd1c27a0938e7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-15T22:27:04.089 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:03 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:03.799+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:04.304 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:04 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:04.089+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:04.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.3.service' 2024-09-15T22:27:04.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:03 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:03.892+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:04.380 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:27:04.381 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-15T22:27:04.381 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-15T22:27:04.381 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.4 2024-09-15T22:27:04.711 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:04 smithi099 systemd[1]: Stopping Ceph osd.4 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:27:05.089 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:04 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:04.816+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:05.090 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:04 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4[45975]: 2024-09-15T22:27:04.709+0000 7f1039f13640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:27:05.090 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:04 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4[45975]: 2024-09-15T22:27:04.709+0000 7f1039f13640 -1 osd.4 113 *** Got signal Terminated *** 2024-09-15T22:27:05.090 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:04 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4[45975]: 2024-09-15T22:27:04.709+0000 7f1039f13640 -1 osd.4 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:27:05.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:04 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:04.940+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:05.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:05 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:05.085+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:06.310 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:05 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4[45975]: 2024-09-15T22:27:05.860+0000 7f1032521640 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:40.499278+0000 front 2024-09-15T22:26:40.499310+0000 (oldest deadline 2024-09-15T22:27:05.799418+0000) 2024-09-15T22:27:06.310 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:05 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:05.857+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:06.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:05 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:05.900+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:06.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:06 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:06.087+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:06.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:06 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:06.087+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:07.092 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:06 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4[45975]: 2024-09-15T22:27:06.823+0000 7f1032521640 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:40.499278+0000 front 2024-09-15T22:26:40.499310+0000 (oldest deadline 2024-09-15T22:27:05.799418+0000) 2024-09-15T22:27:07.093 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:06 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:06.903+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:07.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:06 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:06.899+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:07.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:07 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:07.113+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:07.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:07 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:07.113+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:07.357 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:06 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:06.936+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:08.089 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:07 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4[45975]: 2024-09-15T22:27:07.803+0000 7f1032521640 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:40.499278+0000 front 2024-09-15T22:26:40.499310+0000 (oldest deadline 2024-09-15T22:27:05.799418+0000) 2024-09-15T22:27:08.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:07 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:07.860+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:08.090 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:07 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:07.989+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:08.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:07 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:07.865+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:08.095 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:08 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:08.039+0000 7f7b7e667640 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:41.754515+0000 front 2024-09-15T22:26:41.754350+0000 (oldest deadline 2024-09-15T22:27:07.654428+0000) 2024-09-15T22:27:08.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:07 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:07.818+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:08.356 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:07 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:07.940+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:08.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:08 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:08.126+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:08.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:08 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:08.126+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:09.089 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:08 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4[45975]: 2024-09-15T22:27:08.758+0000 7f1032521640 -1 osd.4 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:40.499278+0000 front 2024-09-15T22:26:40.499310+0000 (oldest deadline 2024-09-15T22:27:05.799418+0000) 2024-09-15T22:27:09.090 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:08 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:08.959+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:09.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:08 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:08.849+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:09.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:08 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:08.849+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:09.095 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:09 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:09.005+0000 7f7b7e667640 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:41.754515+0000 front 2024-09-15T22:26:41.754350+0000 (oldest deadline 2024-09-15T22:27:07.654428+0000) 2024-09-15T22:27:09.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:08 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:08.800+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:09.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:08 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:08.872+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:09.258 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:08 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:08.891+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:09.258 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:09 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:09.110+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:09.258 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:09 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:09.110+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:10.090 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:09 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:09.974+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:10.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:09 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:09.825+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:10.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:09 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:09.825+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:10.095 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:10 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:10.011+0000 7f7b7e667640 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:41.754515+0000 front 2024-09-15T22:26:41.754350+0000 (oldest deadline 2024-09-15T22:27:07.654428+0000) 2024-09-15T22:27:10.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:09 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:09.790+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:10.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:09 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:09.922+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:10.356 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:09 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:09.903+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:10.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:10 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:10.084+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:10.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:10 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:10.084+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:10.815 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:10 smithi099 podman[71968]: 2024-09-15 22:27:10.51002429 +0000 UTC m=+5.931896175 container died 5ba73cbb2aea51dbcf1fd70513f6e47cb57b8c00db8be5f2162479e1e76c4c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-15T22:27:11.089 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:10 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:10.970+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:11.090 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:10 smithi099 podman[71968]: 2024-09-15 22:27:10.959321109 +0000 UTC m=+6.381192994 container cleanup 5ba73cbb2aea51dbcf1fd70513f6e47cb57b8c00db8be5f2162479e1e76c4c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:27:11.090 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:10 smithi099 bash[71968]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4 2024-09-15T22:27:11.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:10 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:10.813+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:11.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:10 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:10.813+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:11.095 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:10 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:10.990+0000 7f7b7e667640 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:41.754515+0000 front 2024-09-15T22:26:41.754350+0000 (oldest deadline 2024-09-15T22:27:07.654428+0000) 2024-09-15T22:27:11.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:10 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:10.807+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:11.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:10 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:10.909+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:11.356 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:10 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:10.895+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:11.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:11 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:11.133+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:11.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:11 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:11.133+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:11.590 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:11 smithi099 podman[71995]: 2024-09-15 22:27:11.185968391 +0000 UTC m=+0.676278715 container remove 5ba73cbb2aea51dbcf1fd70513f6e47cb57b8c00db8be5f2162479e1e76c4c51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e) 2024-09-15T22:27:11.884 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:11 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:11.854+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:11.884 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:11 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:11.854+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:11.884 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:11 smithi099 podman[72084]: 2024-09-15 22:27:11.509013453 +0000 UTC m=+0.015814246 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:27:11.885 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:11 smithi099 podman[72084]: 2024-09-15 22:27:11.622459014 +0000 UTC m=+0.129259849 container create fb2c3b1ff0b854e47fefebc8fdc7a2f296f175ef9efe27b6083af8c0a258feac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-15T22:27:11.885 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:11 smithi099 podman[72084]: 2024-09-15 22:27:11.883463293 +0000 UTC m=+0.390264097 container init fb2c3b1ff0b854e47fefebc8fdc7a2f296f175ef9efe27b6083af8c0a258feac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:27:12.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:12 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:12.001+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:12.340 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:11 smithi099 podman[72084]: 2024-09-15 22:27:11.887042338 +0000 UTC m=+0.393843132 container start fb2c3b1ff0b854e47fefebc8fdc7a2f296f175ef9efe27b6083af8c0a258feac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-15T22:27:12.340 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:11 smithi099 podman[72084]: 2024-09-15 22:27:11.983333668 +0000 UTC m=+0.490134475 container attach fb2c3b1ff0b854e47fefebc8fdc7a2f296f175ef9efe27b6083af8c0a258feac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-15T22:27:12.340 INFO:journalctl@ceph.osd.4.smithi099.stdout:Sep 15 22:27:12 smithi099 podman[72084]: 2024-09-15 22:27:12.082561187 +0000 UTC m=+0.589361982 container died fb2c3b1ff0b854e47fefebc8fdc7a2f296f175ef9efe27b6083af8c0a258feac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-15T22:27:12.345 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:11 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:11.982+0000 7f7b7e667640 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:41.754515+0000 front 2024-09-15T22:26:41.754350+0000 (oldest deadline 2024-09-15T22:27:07.654428+0000) 2024-09-15T22:27:12.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:11 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:11.870+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:12.345 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:11 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:11.856+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:12.356 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:11 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:11.932+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:12.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:12 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:12.179+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:12.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:12 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:12.179+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:12.708 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.4.service' 2024-09-15T22:27:12.749 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:27:12.749 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-15T22:27:12.750 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-15T22:27:12.750 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.5 2024-09-15T22:27:13.013 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:12 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:12.849+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:13.014 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:12 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:12.849+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:13.030 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:12 smithi129 systemd[1]: Stopping Ceph osd.5 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:27:13.030 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:12 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:12.837+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:13.031 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:12 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:12.873+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:13.201 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:12 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:12.899+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:13.339 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:13 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:13.011+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:13.345 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:13 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:13.028+0000 7f7b7e667640 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:41.754515+0000 front 2024-09-15T22:26:41.754350+0000 (oldest deadline 2024-09-15T22:27:07.654428+0000) 2024-09-15T22:27:13.345 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:13 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:13.073+0000 7f7b86059640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:27:13.345 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:13 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:13.073+0000 7f7b86059640 -1 osd.5 113 *** Got signal Terminated *** 2024-09-15T22:27:13.345 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:13 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:13.073+0000 7f7b86059640 -1 osd.5 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:27:13.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:13 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:13.199+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:13.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:13 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:13.199+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:14.095 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:14 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:14.042+0000 7f7b7e667640 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:41.754515+0000 front 2024-09-15T22:26:41.754350+0000 (oldest deadline 2024-09-15T22:27:07.654428+0000) 2024-09-15T22:27:14.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:13 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:13.860+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:14.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:13 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:13.839+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:14.207 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:13 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:13.871+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:14.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:13 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:13.968+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:14.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:13 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:13.968+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:14.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:13 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:13.898+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:14.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:13 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:13.898+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:14.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:14 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:14.205+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:14.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:14 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:14.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:14.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:14 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:14.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:15.095 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:15 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:15.040+0000 7f7b7e667640 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:41.754515+0000 front 2024-09-15T22:26:41.754350+0000 (oldest deadline 2024-09-15T22:27:07.654428+0000) 2024-09-15T22:27:15.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:14 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:14.850+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:15.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:14 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:14.798+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:15.200 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:14 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:14.869+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:15.200 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:14 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:14.869+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:15.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:14 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:14.952+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:15.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:14 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:14.952+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:15.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:14 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:14.936+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:15.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:14 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:14.936+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:15.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:15 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:15.198+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:15.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:15 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:15.198+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:15.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:15 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:15.198+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:16.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:15 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:15.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:16.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:15 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:15.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:16.090 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:15 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:15.961+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:16.090 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:15 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:15.961+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:16.244 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:15 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:15.896+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:16.244 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:15 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:15.896+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:16.345 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:16 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:16.004+0000 7f7b7e667640 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:41.754515+0000 front 2024-09-15T22:26:41.754350+0000 (oldest deadline 2024-09-15T22:27:07.654428+0000) 2024-09-15T22:27:16.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:15 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:15.844+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:16.346 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:15 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:15.870+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:16.346 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:15 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:15.870+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:16.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:16 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:16.243+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:16.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:16 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:16.243+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:16.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:16 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:16.243+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:17.095 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:16 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:16.957+0000 7f7b7e667640 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:41.754515+0000 front 2024-09-15T22:26:41.754350+0000 (oldest deadline 2024-09-15T22:27:07.654428+0000) 2024-09-15T22:27:17.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:16 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:16.899+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:17.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:16 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:16.899+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:17.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:16 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:16.814+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:17.106 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:16 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:16.849+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:17.106 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:16 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:16.849+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:17.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:16 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:16.921+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:17.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:16 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:16.921+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:17.341 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:17 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:17.002+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:17.341 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:17 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:17.002+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:17.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:17.201+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:17.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:17.201+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:17.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:17.201+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:18.239 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:17.859+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:18.239 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:17.859+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:18.339 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:17 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:17.982+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:18.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:17 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:17.982+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:18.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:17 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:17.872+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:18.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:17 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:17.872+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:18.345 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:17 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5[46707]: 2024-09-15T22:27:17.985+0000 7f7b7e667640 -1 osd.5 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:41.754515+0000 front 2024-09-15T22:26:41.754350+0000 (oldest deadline 2024-09-15T22:27:07.654428+0000) 2024-09-15T22:27:18.345 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:17 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:17.913+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:18.345 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:17 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:17.913+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:18.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:17 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:17.862+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:18.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:17 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:17.862+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:18.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:18.237+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:18.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:18.237+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:18.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:18.237+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:19.271 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:18.904+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:19.271 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:18 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:18.904+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:19.292 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:18 smithi129 podman[72692]: 2024-09-15 22:27:18.97449839 +0000 UTC m=+6.030608785 container died 13a4ccd5f72490dc70648f2e160cd625c35de8c740c3ea462c8c097afc65d4b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-15T22:27:19.293 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:18 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:18.924+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:19.293 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:18 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:18.924+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:19.293 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:18 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:18.857+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:19.293 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:18 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:18.857+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:19.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:18 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:18.932+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:19.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:18 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:18.932+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:19.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:18 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:18.863+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:19.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:18 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:18.863+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:19.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:18 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:18.863+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:19.595 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:19 smithi129 podman[72692]: 2024-09-15 22:27:19.407371276 +0000 UTC m=+6.463481687 container cleanup 13a4ccd5f72490dc70648f2e160cd625c35de8c740c3ea462c8c097afc65d4b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-15T22:27:19.595 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:19 smithi129 bash[72692]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5 2024-09-15T22:27:19.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:19 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:19.270+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:19.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:19 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:19.270+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:19.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:19 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:19.270+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:19.910 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:19 smithi129 podman[72707]: 2024-09-15 22:27:19.61716884 +0000 UTC m=+0.634549689 container remove 13a4ccd5f72490dc70648f2e160cd625c35de8c740c3ea462c8c097afc65d4b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:27:19.911 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:19 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:19.835+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:19.911 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:19 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:19.835+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:20.089 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:19 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:19.958+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:20.090 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:19 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:19.958+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:20.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:19 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:19.831+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:20.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:19 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:19.831+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:20.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:19 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:19.831+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:20.162 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:19 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:19.909+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:20.162 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:19 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:19.909+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:20.163 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:20 smithi129 podman[72798]: 2024-09-15 22:27:19.917394012 +0000 UTC m=+0.016063594 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:27:20.163 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:20 smithi129 podman[72798]: 2024-09-15 22:27:20.034944903 +0000 UTC m=+0.133614477 container create a92a8fa64ca7462ded9753759d79c21f4c929a8340aca6b33243b884acd0677d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=squid-8921c8e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-15T22:27:20.285 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:19 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:19.945+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:20.285 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:19 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:19.945+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:20.595 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:20 smithi129 podman[72798]: 2024-09-15 22:27:20.287615531 +0000 UTC m=+0.386285109 container init a92a8fa64ca7462ded9753759d79c21f4c929a8340aca6b33243b884acd0677d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-15T22:27:20.595 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:20 smithi129 podman[72798]: 2024-09-15 22:27:20.290968615 +0000 UTC m=+0.389638192 container start a92a8fa64ca7462ded9753759d79c21f4c929a8340aca6b33243b884acd0677d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-15T22:27:20.595 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:20 smithi129 podman[72798]: 2024-09-15 22:27:20.387488514 +0000 UTC m=+0.486158101 container attach a92a8fa64ca7462ded9753759d79c21f4c929a8340aca6b33243b884acd0677d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2) 2024-09-15T22:27:20.595 INFO:journalctl@ceph.osd.5.smithi129.stdout:Sep 15 22:27:20 smithi129 podman[72798]: 2024-09-15 22:27:20.475887287 +0000 UTC m=+0.574556865 container died a92a8fa64ca7462ded9753759d79c21f4c929a8340aca6b33243b884acd0677d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-15T22:27:20.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:20 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:20.283+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:20.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:20 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:20.283+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:20.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:20 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:20.283+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:21.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:20 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:20.958+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:21.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:20 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:20.958+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:21.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:20 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:20.788+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:21.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:20 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:20.788+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:21.182 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.5.service' 2024-09-15T22:27:21.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:27:21.223 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-15T22:27:21.223 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-15T22:27:21.224 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.6 2024-09-15T22:27:21.255 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:20 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:20.960+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:21.255 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:20 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:20.960+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:21.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:20 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:20.976+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:21.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:20 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:20.976+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:21.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:20 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:20.867+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:21.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:20 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:20.867+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:21.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:20 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:20.867+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:21.542 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:21.273+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:21.542 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:21.273+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:21.542 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:21.273+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:21.543 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:21 smithi005 systemd[1]: Stopping Ceph osd.6 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:27:21.543 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:21.539+0000 7fd245ff0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:27:21.543 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:21.539+0000 7fd245ff0640 -1 osd.6 113 *** Got signal Terminated *** 2024-09-15T22:27:21.543 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:21 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:21.539+0000 7fd245ff0640 -1 osd.6 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:27:22.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:21 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:21.918+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:22.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:21 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:21.918+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:22.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:21 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:21.742+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:22.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:21 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:21.742+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:22.277 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:22 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:21.999+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:22.277 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:22 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:21.999+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:22.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:22 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:22.017+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:22.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:22 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:22.017+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:22.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:21 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:21.874+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:22.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:21 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:21.874+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:22.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:21 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:21.874+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:22.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:22 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:22.275+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:22.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:22 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:22.275+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:22.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:22 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:22.275+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:23.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:22 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:22.936+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:23.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:22 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:22.936+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:23.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:22 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:22.787+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:23.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:22 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:22.787+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:23.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:23.046+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:23.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:23.046+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:23.339 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:22 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:22.987+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:23.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:22 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:22.987+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:23.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:22 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:22.880+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:23.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:22 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:22.880+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:23.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:22 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:22.880+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:23.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:23.308+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:23.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:23.308+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:23.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:23 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:23.308+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:24.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:23 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:23.786+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:24.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:23 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:23.786+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:24.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:23 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:23.924+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:24.096 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:23 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:23.924+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:24.096 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:23 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:23.924+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:24.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:24 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:24.013+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:24.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:24 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:24.013+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:24.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:23 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:23.863+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:24.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:23 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:23.863+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:24.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:23 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:23.863+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:24.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:24 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:24.263+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:24.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:24 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:24.263+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:24.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:24 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:24.263+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:24.357 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:24 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:24.029+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:24.357 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:24 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:24.029+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:24.357 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:24 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:24.029+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:59.522548+0000 front 2024-09-15T22:26:59.522409+0000 (oldest deadline 2024-09-15T22:27:23.622565+0000) 2024-09-15T22:27:25.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:24 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:24.767+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:25.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:24 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:24.767+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:25.096 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:24 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:24.907+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:25.096 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:24 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:24.907+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:25.096 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:24 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:24.907+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:25.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:25 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:25.043+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:25.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:25 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:25.043+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:25.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:25 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:25.043+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.988727+0000 front 2024-09-15T22:26:58.988705+0000 (oldest deadline 2024-09-15T22:27:24.288640+0000) 2024-09-15T22:27:25.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:24 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:24.900+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:25.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:24 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:24.900+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:25.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:24 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:24.900+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:25.357 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:24 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:24.989+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:25.357 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:24 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:24.989+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:25.357 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:24 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:24.989+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:59.522548+0000 front 2024-09-15T22:26:59.522409+0000 (oldest deadline 2024-09-15T22:27:23.622565+0000) 2024-09-15T22:27:25.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:25 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:25.222+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:25.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:25 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:25.222+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:25.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:25 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:25.222+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:25.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:25 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:25.222+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:26.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:25 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:25.893+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:26.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:25 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:25.893+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:26.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:25 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:25.893+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:26.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:25 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:25.732+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:26.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:25 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:25.732+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:26.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:25 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:25.732+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:26.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:26 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:26.020+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:26.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:26 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:26.020+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:26.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:26 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:26.020+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.988727+0000 front 2024-09-15T22:26:58.988705+0000 (oldest deadline 2024-09-15T22:27:24.288640+0000) 2024-09-15T22:27:26.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:25 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:25.914+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:26.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:25 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:25.914+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:26.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:25 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:25.914+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:26.356 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:25 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:25.955+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.418697+0000 front 2024-09-15T22:26:43.418571+0000 (oldest deadline 2024-09-15T22:27:06.318800+0000) 2024-09-15T22:27:26.357 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:25 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:25.955+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.120485+0000 front 2024-09-15T22:26:52.120566+0000 (oldest deadline 2024-09-15T22:27:14.320998+0000) 2024-09-15T22:27:26.357 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:25 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6[57972]: 2024-09-15T22:27:25.955+0000 7fd242606640 -1 osd.6 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:59.522548+0000 front 2024-09-15T22:26:59.522409+0000 (oldest deadline 2024-09-15T22:27:23.622565+0000) 2024-09-15T22:27:26.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:26 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:26.188+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:26.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:26 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:26.188+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:26.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:26 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:26.188+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:26.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:26 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:26.188+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:27.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:26 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:26.939+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:27.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:26 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:26.939+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:27.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:26 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:26.939+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:27.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:26 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:26.773+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:27.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:26 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:26.773+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:27.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:26 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:26.773+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:27.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:27 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:27.070+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:27.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:27 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:27.070+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:27.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:27 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:27.070+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.988727+0000 front 2024-09-15T22:26:58.988705+0000 (oldest deadline 2024-09-15T22:27:24.288640+0000) 2024-09-15T22:27:27.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:26 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:26.903+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:27.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:26 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:26.903+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:27.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:26 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:26.903+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:27.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:26 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:26.903+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:27.508 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:27 smithi005 podman[121208]: 2024-09-15 22:27:27.217784601 +0000 UTC m=+5.780300695 container died 217772f3cd401de7158240e29635af80cb5a8ca80d9e6d6a46549d0a3c28d332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:27:27.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:27 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:27.160+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:27.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:27 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:27.160+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:27.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:27 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:27.160+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:27.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:27 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:27.160+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:27.857 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:27 smithi005 podman[121208]: 2024-09-15 22:27:27.68993816 +0000 UTC m=+6.252454289 container cleanup 217772f3cd401de7158240e29635af80cb5a8ca80d9e6d6a46549d0a3c28d332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-15T22:27:27.857 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:27 smithi005 bash[121208]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6 2024-09-15T22:27:28.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:27 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:27.953+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:28.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:27 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:27.953+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:28.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:27 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:27.953+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:28.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:27 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:27.753+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:28.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:27 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:27.753+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:28.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:27 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:27.753+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:28.316 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:27 smithi005 podman[121226]: 2024-09-15 22:27:27.999122047 +0000 UTC m=+0.784773419 container remove 217772f3cd401de7158240e29635af80cb5a8ca80d9e6d6a46549d0a3c28d332 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True) 2024-09-15T22:27:28.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:28 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:28.143+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:28.316 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:28 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:28.143+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:28.317 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:28 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:28.143+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:28.317 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:28 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:28.143+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:28.339 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:28 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:28.048+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:28.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:28 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:28.048+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:28.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:28 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:28.048+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.988727+0000 front 2024-09-15T22:26:58.988705+0000 (oldest deadline 2024-09-15T22:27:24.288640+0000) 2024-09-15T22:27:28.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:27 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:27.932+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:28.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:27 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:27.932+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:28.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:27 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:27.932+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:28.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:27 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:27.932+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:28.579 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:28 smithi005 podman[121316]: 2024-09-15 22:27:28.218025041 +0000 UTC m=+0.016118049 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:27:28.580 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:28 smithi005 podman[121316]: 2024-09-15 22:27:28.334004105 +0000 UTC m=+0.132097106 container create d3fb5fac0c2aec37d4886a8f43b7ce5daff0b97f226425ab3973fbc59e9c2f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-15T22:27:28.580 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:28 smithi005 podman[121316]: 2024-09-15 22:27:28.578531968 +0000 UTC m=+0.376624979 container init d3fb5fac0c2aec37d4886a8f43b7ce5daff0b97f226425ab3973fbc59e9c2f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-15T22:27:28.857 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:28 smithi005 podman[121316]: 2024-09-15 22:27:28.582102467 +0000 UTC m=+0.380195468 container start d3fb5fac0c2aec37d4886a8f43b7ce5daff0b97f226425ab3973fbc59e9c2f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-15T22:27:28.857 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:28 smithi005 podman[121316]: 2024-09-15 22:27:28.68686652 +0000 UTC m=+0.484959528 container attach d3fb5fac0c2aec37d4886a8f43b7ce5daff0b97f226425ab3973fbc59e9c2f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-15T22:27:28.857 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 15 22:27:28 smithi005 podman[121316]: 2024-09-15 22:27:28.771531624 +0000 UTC m=+0.569624625 container died d3fb5fac0c2aec37d4886a8f43b7ce5daff0b97f226425ab3973fbc59e9c2f44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-15T22:27:29.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:28 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:28.975+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:29.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:28 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:28.975+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:29.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:28 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:28.975+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:29.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:28 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:28.750+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:29.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:28 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:28.750+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:29.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:28 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:28.750+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:29.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:29.018+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:29.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:29.018+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:29.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:29.018+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.988727+0000 front 2024-09-15T22:26:58.988705+0000 (oldest deadline 2024-09-15T22:27:24.288640+0000) 2024-09-15T22:27:29.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:28 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:28.924+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:29.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:28 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:28.924+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:29.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:28 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:28.924+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:29.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:28 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:28.924+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:29.467 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:29 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:29.192+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:29.467 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:29 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:29.192+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:29.467 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:29 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:29.192+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:29.467 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:29 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:29.192+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:29.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.6.service' 2024-09-15T22:27:29.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:27:29.583 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-15T22:27:29.583 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-15T22:27:29.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.7 2024-09-15T22:27:29.906 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:29 smithi099 systemd[1]: Stopping Ceph osd.7 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:27:30.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:30 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:30.007+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:30.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:30 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:30.007+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:30.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:30 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:30.007+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:30.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:29 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:29.787+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:30.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:29 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:29.787+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:30.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:29 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:29.787+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:30.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:29.965+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:30.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:29.965+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:30.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:29.965+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:30.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:29.965+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:30.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:29.904+0000 7f0d8e384640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:27:30.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:29.904+0000 7f0d8e384640 -1 osd.7 113 *** Got signal Terminated *** 2024-09-15T22:27:30.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:29.904+0000 7f0d8e384640 -1 osd.7 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:27:30.341 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:29.989+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:30.341 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:29.989+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:30.341 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:29 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:29.989+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.988727+0000 front 2024-09-15T22:26:58.988705+0000 (oldest deadline 2024-09-15T22:27:24.288640+0000) 2024-09-15T22:27:30.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:30 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:30.223+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:30.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:30 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:30.223+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:30.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:30 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:30.223+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:30.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:30 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:30.223+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:31.009 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:30 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:30.739+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:31.009 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:30 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:30.739+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:31.009 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:30 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:30.739+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:31.265 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:31 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:31.056+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:31.265 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:31 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:31.056+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:31.265 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:31 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:31.056+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:31.339 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:30 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:30.986+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:31.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:30 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:30.986+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:31.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:30 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:30.986+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.988727+0000 front 2024-09-15T22:26:58.988705+0000 (oldest deadline 2024-09-15T22:27:24.288640+0000) 2024-09-15T22:27:31.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:30 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:30.986+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.789669+0000 front 2024-09-15T22:27:07.789395+0000 (oldest deadline 2024-09-15T22:27:30.689562+0000) 2024-09-15T22:27:31.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:31.006+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:31.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:31.006+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:31.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:31.006+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:31.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:31.007+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:31.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:31.007+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:31.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:31 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:31.230+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:31.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:31 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:31.230+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:31.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:31 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:31.230+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:31.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:31 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:31.230+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:32.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:31 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:31.785+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:32.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:31 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:31.785+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:32.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:31 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:31.785+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:32.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:31.961+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:32.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:31.961+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:32.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:31.961+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.988727+0000 front 2024-09-15T22:26:58.988705+0000 (oldest deadline 2024-09-15T22:27:24.288640+0000) 2024-09-15T22:27:32.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:31.961+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.789669+0000 front 2024-09-15T22:27:07.789395+0000 (oldest deadline 2024-09-15T22:27:30.689562+0000) 2024-09-15T22:27:32.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:31.959+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:32.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:31.959+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:32.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:31.959+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:32.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:31.959+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:32.342 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:31.959+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:32.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:32 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:32.100+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:32.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:32 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:32.100+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:32.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:32 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:32.100+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:32.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:32 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:32.100+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:08.685795+0000 front 2024-09-15T22:27:08.685762+0000 (oldest deadline 2024-09-15T22:27:32.086150+0000) 2024-09-15T22:27:32.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:32 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:32.197+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:32.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:32 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:32.197+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:32.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:32 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:32.197+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:32.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:32 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:32.197+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:33.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:32 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:32.813+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:33.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:32 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:32.813+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:33.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:32 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:32.813+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:33.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:32 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:32.813+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:33.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:32 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:32.948+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:33.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:32 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:32.948+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:33.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:32 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:32.948+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.988727+0000 front 2024-09-15T22:26:58.988705+0000 (oldest deadline 2024-09-15T22:27:24.288640+0000) 2024-09-15T22:27:33.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:32 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:32.948+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.789669+0000 front 2024-09-15T22:27:07.789395+0000 (oldest deadline 2024-09-15T22:27:30.689562+0000) 2024-09-15T22:27:33.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:32 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:32.960+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:33.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:32 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:32.960+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:33.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:32 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:32.960+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:33.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:32 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:32.960+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:33.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:32 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:32.960+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:33.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:33 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:33.095+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:33.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:33 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:33.095+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:33.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:33 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:33.095+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:33.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:33 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:33.095+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:08.685795+0000 front 2024-09-15T22:27:08.685762+0000 (oldest deadline 2024-09-15T22:27:32.086150+0000) 2024-09-15T22:27:33.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:33 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:33.183+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:33.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:33 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:33.183+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:33.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:33 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:33.183+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:33.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:33 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:33.183+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:34.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:33 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:33.809+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:34.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:33 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:33.809+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:34.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:33 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:33.809+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:34.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:33 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:33.809+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:34.339 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:33 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:33.956+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:43.286240+0000 front 2024-09-15T22:26:43.286329+0000 (oldest deadline 2024-09-15T22:27:07.386292+0000) 2024-09-15T22:27:34.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:33 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:33.956+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:50.187676+0000 front 2024-09-15T22:26:50.187723+0000 (oldest deadline 2024-09-15T22:27:13.687601+0000) 2024-09-15T22:27:34.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:33 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:33.956+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.988727+0000 front 2024-09-15T22:26:58.988705+0000 (oldest deadline 2024-09-15T22:27:24.288640+0000) 2024-09-15T22:27:34.340 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:33 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7[52468]: 2024-09-15T22:27:33.956+0000 7f0d8a99a640 -1 osd.7 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.789669+0000 front 2024-09-15T22:27:07.789395+0000 (oldest deadline 2024-09-15T22:27:30.689562+0000) 2024-09-15T22:27:34.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:33 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:33.957+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:34.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:33 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:33.957+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:34.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:33 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:33.957+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:34.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:33 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:33.957+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:34.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:33 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:33.957+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:34.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:34 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:34.140+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:34.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:34 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:34.140+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:34.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:34 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:34.140+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:34.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:34 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:34.140+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:08.685795+0000 front 2024-09-15T22:27:08.685762+0000 (oldest deadline 2024-09-15T22:27:32.086150+0000) 2024-09-15T22:27:34.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:34 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:34.176+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:34.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:34 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:34.176+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:34.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:34 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:34.176+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:34.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:34 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:34.176+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:34.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:34 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:34.176+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:35.128 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:34 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:34.857+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:35.128 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:34 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:34.857+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:35.128 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:34 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:34.857+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:35.128 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:34 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:34.857+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:35.241 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:34 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:34.958+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:35.241 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:34 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:34.958+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:35.241 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:34 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:34.958+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:35.241 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:34 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:34.958+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:35.241 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:34 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:34.958+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:35.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:35 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:35.126+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:35.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:35 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:35.126+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:35.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:35 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:35.126+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:35.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:35 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:35.126+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:08.685795+0000 front 2024-09-15T22:27:08.685762+0000 (oldest deadline 2024-09-15T22:27:32.086150+0000) 2024-09-15T22:27:35.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:35 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:35.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:35.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:35 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:35.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:35.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:35 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:35.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:35.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:35 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:35.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:35.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:35 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:35.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:36.089 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:35 smithi099 podman[72187]: 2024-09-15 22:27:35.796254911 +0000 UTC m=+6.013680496 container died 573ca3113b927d202caab4311f12dea8337ae7eddc1998f0bfa057cf5875ece9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-15T22:27:36.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:35 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:35.997+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:36.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:35 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:35.997+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:36.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:35 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:35.997+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:36.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:35 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:35.997+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:36.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:35 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:35.997+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:36.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:35 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:35.824+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:36.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:35 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:35.824+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:36.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:35 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:35.824+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:36.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:35 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:35.824+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:36.431 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:36 smithi099 podman[72187]: 2024-09-15 22:27:36.220678268 +0000 UTC m=+6.438103851 container cleanup 573ca3113b927d202caab4311f12dea8337ae7eddc1998f0bfa057cf5875ece9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-09-15T22:27:36.431 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:36 smithi099 bash[72187]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7 2024-09-15T22:27:36.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:36 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:36.118+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:36.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:36 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:36.118+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:36.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:36 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:36.118+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:36.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:36 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:36.118+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:08.685795+0000 front 2024-09-15T22:27:08.685762+0000 (oldest deadline 2024-09-15T22:27:32.086150+0000) 2024-09-15T22:27:36.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:36 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:36.255+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:36.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:36 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:36.255+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:36.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:36 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:36.255+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:36.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:36 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:36.255+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:36.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:36 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:36.255+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:36.840 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:36 smithi099 podman[72211]: 2024-09-15 22:27:36.430229103 +0000 UTC m=+0.637804503 container remove 573ca3113b927d202caab4311f12dea8337ae7eddc1998f0bfa057cf5875ece9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:27:37.089 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:36 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:36.830+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:37.089 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:36 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:36.830+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:37.089 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:36 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:36.830+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:37.090 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:36 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:36.830+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:37.126 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:36 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:36.987+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:37.126 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:36 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:36.987+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:37.126 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:36 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:36.987+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:37.126 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:36 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:36.987+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:37.126 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:36 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:36.987+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:37.126 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:36 smithi099 podman[72298]: 2024-09-15 22:27:36.744091929 +0000 UTC m=+0.015691473 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:27:37.127 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:36 smithi099 podman[72298]: 2024-09-15 22:27:36.847811996 +0000 UTC m=+0.119411571 container create f7cb9266502059d47d55efdec2102f73e635dcd9494a1091dd7a88fc32832137 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-15T22:27:37.127 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:37 smithi099 podman[72298]: 2024-09-15 22:27:37.026023764 +0000 UTC m=+0.297623297 container init f7cb9266502059d47d55efdec2102f73e635dcd9494a1091dd7a88fc32832137 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-15T22:27:37.127 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:37 smithi099 podman[72298]: 2024-09-15 22:27:37.029268614 +0000 UTC m=+0.300868145 container start f7cb9266502059d47d55efdec2102f73e635dcd9494a1091dd7a88fc32832137 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-15T22:27:37.127 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:37 smithi099 podman[72298]: 2024-09-15 22:27:37.125375609 +0000 UTC m=+0.396975140 container attach f7cb9266502059d47d55efdec2102f73e635dcd9494a1091dd7a88fc32832137 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-15T22:27:37.345 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:37 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:37.087+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:37.345 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:37 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:37.087+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:37.345 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:37 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:37.087+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:37.345 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:37 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:37.087+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:08.685795+0000 front 2024-09-15T22:27:08.685762+0000 (oldest deadline 2024-09-15T22:27:32.086150+0000) 2024-09-15T22:27:37.496 INFO:journalctl@ceph.osd.7.smithi099.stdout:Sep 15 22:27:37 smithi099 podman[72298]: 2024-09-15 22:27:37.217247801 +0000 UTC m=+0.488847344 container died f7cb9266502059d47d55efdec2102f73e635dcd9494a1091dd7a88fc32832137 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:27:37.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:37 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:37.246+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:37.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:37 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:37.246+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:37.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:37 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:37.246+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:37.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:37 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:37.246+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:37.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:37 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:37.246+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:37.844 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.7.service' 2024-09-15T22:27:37.886 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:27:37.886 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-15T22:27:37.886 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-15T22:27:37.886 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.8 2024-09-15T22:27:38.089 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:38.017+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:38.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:38.017+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:38.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:38.017+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:38.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:38.017+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:38.090 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:38.017+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:38.095 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:37 smithi129 systemd[1]: Stopping Ceph osd.8 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:27:38.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:37 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:37.836+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:38.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:37 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:37.836+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:38.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:37 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:37.836+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:38.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:37 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:37.836+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:38.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:38 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:38.123+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:38.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:38 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:38.123+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:38.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:38 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:38.123+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:38.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:38 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:38.123+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:08.685795+0000 front 2024-09-15T22:27:08.685762+0000 (oldest deadline 2024-09-15T22:27:32.086150+0000) 2024-09-15T22:27:38.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:38 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:38.210+0000 7fc5d8029640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:27:38.596 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:38 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:38.210+0000 7fc5d8029640 -1 osd.8 113 *** Got signal Terminated *** 2024-09-15T22:27:38.596 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:38 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:38.210+0000 7fc5d8029640 -1 osd.8 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:27:38.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:38 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:38.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:38.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:38 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:38.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:38.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:38 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:38.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:38.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:38 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:38.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:38.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:38 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:38.206+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:39.172 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:38 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:38.873+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:39.172 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:38 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:38.873+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:39.173 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:38 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:38.873+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:39.173 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:38 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:38.873+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:39.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:38.981+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:39.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:38.981+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:39.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:38.981+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:39.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:38.981+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:39.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:38.981+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:39.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:39 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:39.249+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:39.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:39 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:39.249+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:39.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:39 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:39.249+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:39.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:39 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:39.249+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:39.508 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:39 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:39.249+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:39.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:39 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:39.170+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:39.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:39 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:39.170+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:39.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:39 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:39.170+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:39.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:39 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:39.170+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:08.685795+0000 front 2024-09-15T22:27:08.685762+0000 (oldest deadline 2024-09-15T22:27:32.086150+0000) 2024-09-15T22:27:40.222 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:39 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:39.886+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:40.222 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:39 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:39.886+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:40.222 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:39 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:39.886+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:40.223 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:39 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:39.886+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:40.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:39 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:39.993+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:40.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:39 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:39.993+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:40.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:39 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:39.993+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:40.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:39 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:39.993+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:40.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:39 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:39.993+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:40.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:39 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:39.993+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:40.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:40.220+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:40.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:40.220+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:40.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:40.220+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:40.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:40.220+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:08.685795+0000 front 2024-09-15T22:27:08.685762+0000 (oldest deadline 2024-09-15T22:27:32.086150+0000) 2024-09-15T22:27:40.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:40.220+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:13.786894+0000 front 2024-09-15T22:27:13.786846+0000 (oldest deadline 2024-09-15T22:27:39.587528+0000) 2024-09-15T22:27:40.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:40 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:40.248+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:40.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:40 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:40.248+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:40.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:40 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:40.248+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:40.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:40 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:40.248+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:40.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:40 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:40.248+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:41.207 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:40.913+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:41.207 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:40.913+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:41.207 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:40.913+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:41.207 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:40 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:40.913+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:41.339 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:40 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:40.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:41.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:40 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:40.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:41.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:40 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:40.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:41.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:40 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:40.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:41.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:40 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:40.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:41.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:40 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:40.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:41.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:41.205+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:41.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:41.205+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:41.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:41.205+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:41.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:41.205+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:08.685795+0000 front 2024-09-15T22:27:08.685762+0000 (oldest deadline 2024-09-15T22:27:32.086150+0000) 2024-09-15T22:27:41.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:41.205+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:13.786894+0000 front 2024-09-15T22:27:13.786846+0000 (oldest deadline 2024-09-15T22:27:39.587528+0000) 2024-09-15T22:27:41.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:41.282+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:41.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:41.282+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:41.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:41.282+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:41.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:41.282+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:41.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:41 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:41.282+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:42.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:41.871+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:42.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:41.872+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:42.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:41.872+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:42.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:41.872+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:42.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:41 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:41.872+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:42.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:41 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:41.989+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:42.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:41 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:41.989+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:42.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:41 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:41.989+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:42.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:41 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:41.989+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:42.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:41 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:41.989+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:42.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:41 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:41.989+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:42.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:42.223+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.581569+0000 front 2024-09-15T22:26:42.581534+0000 (oldest deadline 2024-09-15T22:27:07.281520+0000) 2024-09-15T22:27:42.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:42.223+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:49.582356+0000 front 2024-09-15T22:26:49.582333+0000 (oldest deadline 2024-09-15T22:27:14.882435+0000) 2024-09-15T22:27:42.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:42.223+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:26:58.383587+0000 front 2024-09-15T22:26:58.383701+0000 (oldest deadline 2024-09-15T22:27:23.083420+0000) 2024-09-15T22:27:42.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:42.223+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:08.685795+0000 front 2024-09-15T22:27:08.685762+0000 (oldest deadline 2024-09-15T22:27:32.086150+0000) 2024-09-15T22:27:42.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8[53215]: 2024-09-15T22:27:42.223+0000 7fc5d463f640 -1 osd.8 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:13.786894+0000 front 2024-09-15T22:27:13.786846+0000 (oldest deadline 2024-09-15T22:27:39.587528+0000) 2024-09-15T22:27:42.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:42 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:42.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:42.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:42 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:42.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:42.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:42 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:42.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:42.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:42 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:42.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:42.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:42 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:42.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:42.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:42 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:42.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.850298+0000 front 2024-09-15T22:27:16.850435+0000 (oldest deadline 2024-09-15T22:27:42.150187+0000) 2024-09-15T22:27:43.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:42.839+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:43.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:42.839+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:43.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:42.839+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:43.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:42.839+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:43.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:42 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:42.839+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:43.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:42.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:43.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:42.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:43.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:42.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:43.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:42.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:43.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:42.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:43.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:42 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:42.954+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:43.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:43 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:43.254+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:43.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:43 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:43.254+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:43.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:43 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:43.254+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:43.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:43 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:43.254+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:43.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:43 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:43.254+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:43.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:43 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:43.254+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.850298+0000 front 2024-09-15T22:27:16.850435+0000 (oldest deadline 2024-09-15T22:27:42.150187+0000) 2024-09-15T22:27:44.263 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:43 smithi129 podman[72912]: 2024-09-15 22:27:43.959628921 +0000 UTC m=+5.873611350 container died de161a3aed44cb72bcd22f44e192a534ce3cd304dccf41f85db359df6d625d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-15T22:27:44.263 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:43.864+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:44.263 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:43.864+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:44.264 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:43.864+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:44.264 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:43.864+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:44.264 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:43.864+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:44.339 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:43 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:43.990+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:44.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:43 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:43.990+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:44.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:43 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:43.990+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:44.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:43 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:43.990+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:44.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:43 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:43.990+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:44.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:43 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:43.990+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:44.571 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:44 smithi129 podman[72912]: 2024-09-15 22:27:44.352504681 +0000 UTC m=+6.266487111 container cleanup de161a3aed44cb72bcd22f44e192a534ce3cd304dccf41f85db359df6d625d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-15T22:27:44.571 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:44 smithi129 bash[72912]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8 2024-09-15T22:27:44.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:44 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:44.237+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:44.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:44 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:44.237+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:44.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:44 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:44.237+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:44.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:44 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:44.237+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:44.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:44 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:44.237+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:44.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:44 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:44.237+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.850298+0000 front 2024-09-15T22:27:16.850435+0000 (oldest deadline 2024-09-15T22:27:42.150187+0000) 2024-09-15T22:27:44.845 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:44 smithi129 podman[72928]: 2024-09-15 22:27:44.570555407 +0000 UTC m=+0.613283373 container remove de161a3aed44cb72bcd22f44e192a534ce3cd304dccf41f85db359df6d625d92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD) 2024-09-15T22:27:45.124 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:44 smithi129 podman[73017]: 2024-09-15 22:27:44.782341912 +0000 UTC m=+0.015924077 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:27:45.124 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:44 smithi129 podman[73017]: 2024-09-15 22:27:44.888232856 +0000 UTC m=+0.121815021 container create 3901c77d34c7a485ccb22313979dc40dcd84687b04119377ddc8cfd8a1ce6ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:27:45.125 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:45 smithi129 podman[73017]: 2024-09-15 22:27:45.123762265 +0000 UTC m=+0.357344434 container init 3901c77d34c7a485ccb22313979dc40dcd84687b04119377ddc8cfd8a1ce6ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=) 2024-09-15T22:27:45.125 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:44 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:44.860+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:45.125 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:44 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:44.860+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:45.125 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:44 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:44.860+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:45.125 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:44 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:44.860+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:45.125 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:44 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:44.860+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:45.339 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.025+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:45.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.025+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:45.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.025+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:45.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.025+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:45.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.025+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:45.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.025+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:45.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:45 smithi129 podman[73017]: 2024-09-15 22:27:45.127317783 +0000 UTC m=+0.360899949 container start 3901c77d34c7a485ccb22313979dc40dcd84687b04119377ddc8cfd8a1ce6ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8-deactivate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:27:45.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:45 smithi129 podman[73017]: 2024-09-15 22:27:45.232469575 +0000 UTC m=+0.466051749 container attach 3901c77d34c7a485ccb22313979dc40dcd84687b04119377ddc8cfd8a1ce6ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-15T22:27:45.595 INFO:journalctl@ceph.osd.8.smithi129.stdout:Sep 15 22:27:45 smithi129 podman[73040]: 2024-09-15 22:27:45.345008148 +0000 UTC m=+0.016897232 container died 3901c77d34c7a485ccb22313979dc40dcd84687b04119377ddc8cfd8a1ce6ea5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:27:45.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:45.282+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:45.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:45.282+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:45.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:45.282+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:45.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:45.282+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:45.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:45.282+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:45.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:45 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:45.282+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.850298+0000 front 2024-09-15T22:27:16.850435+0000 (oldest deadline 2024-09-15T22:27:42.150187+0000) 2024-09-15T22:27:45.920 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:45 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:45.863+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:45.920 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:45 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:45.863+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:45.921 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:45 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:45.863+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:45.921 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:45 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:45.863+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:45.921 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:45 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:45.863+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:45.947 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.8.service' 2024-09-15T22:27:45.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:27:45.988 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-15T22:27:45.989 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-15T22:27:45.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.9 2024-09-15T22:27:46.269 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:46 smithi005 systemd[1]: Stopping Ceph osd.9 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:27:46.270 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:46 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:46.268+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:46.270 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:46 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:46.268+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:46.270 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:46 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:46.268+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:46.270 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:46 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:46.268+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:46.270 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:46 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:46.268+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:46.270 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:46 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:46.268+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.850298+0000 front 2024-09-15T22:27:16.850435+0000 (oldest deadline 2024-09-15T22:27:42.150187+0000) 2024-09-15T22:27:46.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:46.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:46.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:46.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:46.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:46.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:45 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:45.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:46.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:46 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:46.304+0000 7f2c5572c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:27:46.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:46 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:46.304+0000 7f2c5572c640 -1 osd.9 113 *** Got signal Terminated *** 2024-09-15T22:27:46.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:46 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:46.304+0000 7f2c5572c640 -1 osd.9 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:27:47.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:46 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:46.829+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:47.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:46 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:46.829+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:47.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:46 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:46.829+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:47.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:46 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:46.829+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:47.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:46 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:46.829+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:47.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:46 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:46.981+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:47.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:46 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:46.981+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:47.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:46 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:46.981+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:47.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:46 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:46.981+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:47.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:46 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:46.981+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:47.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:46 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:46.981+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:47.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:47 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:47.288+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:47.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:47 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:47.288+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:47.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:47 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:47.288+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:47.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:47 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:47.288+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:47.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:47 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:47.288+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:47.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:47 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:47.288+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.850298+0000 front 2024-09-15T22:27:16.850435+0000 (oldest deadline 2024-09-15T22:27:42.150187+0000) 2024-09-15T22:27:48.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:47 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:47.824+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:48.220 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:47 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:47.824+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:48.221 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:47 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:47.824+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:48.221 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:47 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:47.824+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:48.221 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:47 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:47.824+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:48.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:47 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:47.969+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:48.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:47 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:47.969+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:48.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:47 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:47.969+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:48.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:47 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:47.969+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:48.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:47 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:47.969+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:48.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:47 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:47.969+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:48.608 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:48.293+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:48.608 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:48.293+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:48.608 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:48.293+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:48.608 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:48.293+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:48.608 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:48.293+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:48.608 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:48 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:48.293+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.850298+0000 front 2024-09-15T22:27:16.850435+0000 (oldest deadline 2024-09-15T22:27:42.150187+0000) 2024-09-15T22:27:49.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:48.841+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:49.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:48.841+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:49.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:48.841+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:49.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:48.841+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:49.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:48 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:48.841+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:49.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:48.944+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:49.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:48.944+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:49.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:48.944+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:49.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:48.944+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:49.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:48.944+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:49.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:48.944+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:49.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:48 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:48.944+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:23.477812+0000 front 2024-09-15T22:27:23.477890+0000 (oldest deadline 2024-09-15T22:27:48.177647+0000) 2024-09-15T22:27:49.606 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:49 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:49.314+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:49.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:49 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:49.314+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:49.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:49 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:49.314+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:49.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:49 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:49.314+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:49.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:49 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:49.314+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:49.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:49 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:49.314+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.850298+0000 front 2024-09-15T22:27:16.850435+0000 (oldest deadline 2024-09-15T22:27:42.150187+0000) 2024-09-15T22:27:50.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:49 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:49.982+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:50.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:49 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:49.982+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:50.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:49 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:49.982+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:50.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:49 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:49.982+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:50.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:49 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:49.982+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:50.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:49 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:49.982+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:50.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:49 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:49.982+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:23.477812+0000 front 2024-09-15T22:27:23.477890+0000 (oldest deadline 2024-09-15T22:27:48.177647+0000) 2024-09-15T22:27:50.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:49 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:49.871+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:50.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:49 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:49.871+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:50.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:49 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:49.871+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:50.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:49 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:49.871+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:50.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:49 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:49.871+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:50.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:50 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:50.292+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:50.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:50 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:50.292+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:50.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:50 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:50.292+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:50.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:50 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:50.292+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:50.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:50 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:50.292+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:50.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:50 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:50.292+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.850298+0000 front 2024-09-15T22:27:16.850435+0000 (oldest deadline 2024-09-15T22:27:42.150187+0000) 2024-09-15T22:27:51.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:50 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:50.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:51.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:50 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:50.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:51.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:50 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:50.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:51.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:50 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:50.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:51.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:50 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:50.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:51.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:50 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:50.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:51.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:50 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:50.995+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:23.477812+0000 front 2024-09-15T22:27:23.477890+0000 (oldest deadline 2024-09-15T22:27:48.177647+0000) 2024-09-15T22:27:51.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:50 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:50.913+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:51.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:50 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:50.913+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:51.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:50 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:50.913+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:51.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:50 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:50.913+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:51.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:50 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:50.913+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:51.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:51.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:32.841890+0000 front 2024-09-15T22:26:32.841867+0000 (oldest deadline 2024-09-15T22:26:58.142085+0000) 2024-09-15T22:27:51.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:51.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.445090+0000 front 2024-09-15T22:26:42.445162+0000 (oldest deadline 2024-09-15T22:27:05.945201+0000) 2024-09-15T22:27:51.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:51.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.246174+0000 front 2024-09-15T22:26:51.246151+0000 (oldest deadline 2024-09-15T22:27:13.546202+0000) 2024-09-15T22:27:51.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:51.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.147928+0000 front 2024-09-15T22:27:01.147906+0000 (oldest deadline 2024-09-15T22:27:24.648029+0000) 2024-09-15T22:27:51.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:51.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:09.348859+0000 front 2024-09-15T22:27:09.348813+0000 (oldest deadline 2024-09-15T22:27:33.949208+0000) 2024-09-15T22:27:51.607 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:51 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9[64675]: 2024-09-15T22:27:51.281+0000 7f2c4dd3a640 -1 osd.9 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.850298+0000 front 2024-09-15T22:27:16.850435+0000 (oldest deadline 2024-09-15T22:27:42.150187+0000) 2024-09-15T22:27:52.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:52.040+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:52.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:52.040+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:52.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:52.040+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:52.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:52.040+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:52.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:52.040+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:52.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:52.040+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:52.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:52 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:52.040+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:23.477812+0000 front 2024-09-15T22:27:23.477890+0000 (oldest deadline 2024-09-15T22:27:48.177647+0000) 2024-09-15T22:27:52.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:51.930+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:52.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:51.930+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:52.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:51.930+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:52.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:51.930+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:52.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:51.930+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:52.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:51.930+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:27:52.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:52 smithi005 podman[121429]: 2024-09-15 22:27:52.083909927 +0000 UTC m=+5.888932368 container died 9ef01b984b6017952963461700246ebb16fdec953e615cea1cbf7c679dbc7790 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-15T22:27:52.721 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:52 smithi005 podman[121429]: 2024-09-15 22:27:52.451320507 +0000 UTC m=+6.256343017 container cleanup 9ef01b984b6017952963461700246ebb16fdec953e615cea1cbf7c679dbc7790 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-15T22:27:52.721 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:52 smithi005 bash[121429]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9 2024-09-15T22:27:53.075 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:52 smithi005 podman[121448]: 2024-09-15 22:27:52.719564281 +0000 UTC m=+0.629647777 container remove 9ef01b984b6017952963461700246ebb16fdec953e615cea1cbf7c679dbc7790 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-15T22:27:53.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:53 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:53.056+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:53.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:53 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:53.056+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:53.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:53 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:53.056+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:53.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:53 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:53.056+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:53.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:53 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:53.056+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:53.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:53 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:53.056+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:53.341 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:53 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:53.056+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:23.477812+0000 front 2024-09-15T22:27:23.477890+0000 (oldest deadline 2024-09-15T22:27:48.177647+0000) 2024-09-15T22:27:53.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:52 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:52.894+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:53.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:52 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:52.894+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:53.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:52 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:52.894+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:53.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:52 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:52.894+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:53.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:52 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:52.894+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:53.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:52 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:52.894+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:27:53.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:53 smithi005 podman[121534]: 2024-09-15 22:27:52.976236012 +0000 UTC m=+0.016255919 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:27:53.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:53 smithi005 podman[121534]: 2024-09-15 22:27:53.078673895 +0000 UTC m=+0.118693803 container create d073d639967b66aec46e7dabda249f27cb860b99b471cfdf63f6f8cdea225400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:27:53.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:53 smithi005 podman[121534]: 2024-09-15 22:27:53.306052193 +0000 UTC m=+0.346072098 container init d073d639967b66aec46e7dabda249f27cb860b99b471cfdf63f6f8cdea225400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:27:53.357 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:53 smithi005 podman[121534]: 2024-09-15 22:27:53.309503015 +0000 UTC m=+0.349522919 container start d073d639967b66aec46e7dabda249f27cb860b99b471cfdf63f6f8cdea225400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-15T22:27:53.784 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:53 smithi005 podman[121534]: 2024-09-15 22:27:53.389276018 +0000 UTC m=+0.429295935 container attach d073d639967b66aec46e7dabda249f27cb860b99b471cfdf63f6f8cdea225400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9-deactivate, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-15T22:27:53.784 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:53 smithi005 podman[121534]: 2024-09-15 22:27:53.503554085 +0000 UTC m=+0.543574002 container died d073d639967b66aec46e7dabda249f27cb860b99b471cfdf63f6f8cdea225400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-15T22:27:54.036 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 15 22:27:54 smithi005 podman[121557]: 2024-09-15 22:27:54.017552267 +0000 UTC m=+0.507885185 container remove d073d639967b66aec46e7dabda249f27cb860b99b471cfdf63f6f8cdea225400 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-15T22:27:54.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.9.service' 2024-09-15T22:27:54.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:27:54.085 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-15T22:27:54.085 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-15T22:27:54.085 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.10 2024-09-15T22:27:54.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:53 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:53.893+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:54.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:53 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:53.893+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:54.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:53 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:53.893+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:54.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:53 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:53.893+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:54.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:53 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:53.893+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:54.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:53 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:53.893+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:27:54.407 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:54 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:54.093+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:54.407 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:54 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:54.093+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:54.407 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:54 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:54.093+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:54.407 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:54 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:54.093+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:54.407 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:54 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:54.093+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:54.407 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:54 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:54.093+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:54.407 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:54 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:54.093+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:23.477812+0000 front 2024-09-15T22:27:23.477890+0000 (oldest deadline 2024-09-15T22:27:48.177647+0000) 2024-09-15T22:27:54.407 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:54 smithi099 systemd[1]: Stopping Ceph osd.10 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:27:54.408 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:54 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:54.405+0000 7f49efcf0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:27:54.408 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:54 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:54.405+0000 7f49efcf0640 -1 osd.10 113 *** Got signal Terminated *** 2024-09-15T22:27:54.408 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:54 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:54.405+0000 7f49efcf0640 -1 osd.10 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:27:55.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:55 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:55.075+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:55.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:55 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:55.075+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:55.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:55 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:55.075+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:55.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:55 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:55.075+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:55.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:55 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:55.075+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:55.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:55 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:55.075+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:55.340 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:55 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:55.075+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:23.477812+0000 front 2024-09-15T22:27:23.477890+0000 (oldest deadline 2024-09-15T22:27:48.177647+0000) 2024-09-15T22:27:55.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:54 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:54.916+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:55.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:54 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:54.916+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:55.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:54 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:54.916+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:55.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:54 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:54.916+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:55.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:54 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:54.916+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:55.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:54 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:54.916+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:27:56.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:55 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:55.957+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:56.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:55 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:55.957+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:56.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:55 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:55.957+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:56.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:55 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:55.957+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:56.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:55 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:55.957+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:56.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:55 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:55.957+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:27:56.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:56 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:56.118+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:56.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:56 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:56.118+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:56.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:56 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:56.118+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:56.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:56 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:56.118+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:56.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:56 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:56.118+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:56.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:56 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:56.118+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:56.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:56 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:56.118+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:23.477812+0000 front 2024-09-15T22:27:23.477890+0000 (oldest deadline 2024-09-15T22:27:48.177647+0000) 2024-09-15T22:27:57.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:56 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:56.910+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:57.346 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:56 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:56.910+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:57.346 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:56 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:56.910+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:57.346 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:56 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:56.910+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:57.346 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:56 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:56.910+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:57.346 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:56 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:56.910+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:27:57.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:57 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:57.090+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:57.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:57 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:57.090+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:57.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:57 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:57.090+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:57.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:57 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:57.090+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:57.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:57 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:57.090+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:57.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:57 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:57.090+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:57.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:57 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:57.090+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:23.477812+0000 front 2024-09-15T22:27:23.477890+0000 (oldest deadline 2024-09-15T22:27:48.177647+0000) 2024-09-15T22:27:58.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:57 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:57.931+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:58.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:57 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:57.931+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:58.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:57 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:57.931+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:58.346 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:57 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:57.931+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:58.346 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:57 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:57.931+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:58.346 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:57 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:57.931+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:27:58.346 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:57 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:57.931+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-15T22:27:31.359545+0000 front 2024-09-15T22:27:31.359517+0000 (oldest deadline 2024-09-15T22:27:57.259467+0000) 2024-09-15T22:27:58.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:58 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:58.135+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:58.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:58 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:58.135+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:58.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:58 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:58.135+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:58.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:58 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:58.135+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:58.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:58 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:58.135+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:58.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:58 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:58.135+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:58.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:58 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:58.135+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:23.477812+0000 front 2024-09-15T22:27:23.477890+0000 (oldest deadline 2024-09-15T22:27:48.177647+0000) 2024-09-15T22:27:59.246 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:58 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:58.909+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:27:59.247 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:58 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:58.909+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:27:59.247 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:58 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:58.909+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:27:59.247 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:58 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:58.909+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:27:59.247 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:58 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:58.909+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:27:59.247 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:58 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:58.909+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:27:59.247 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:58 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:58.909+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-15T22:27:31.359545+0000 front 2024-09-15T22:27:31.359517+0000 (oldest deadline 2024-09-15T22:27:57.259467+0000) 2024-09-15T22:27:59.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:59 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:59.131+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:34.371291+0000 front 2024-09-15T22:26:34.371355+0000 (oldest deadline 2024-09-15T22:26:57.271334+0000) 2024-09-15T22:27:59.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:59 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:59.131+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6804 osd.1 since back 2024-09-15T22:26:42.572194+0000 front 2024-09-15T22:26:42.572214+0000 (oldest deadline 2024-09-15T22:27:08.472385+0000) 2024-09-15T22:27:59.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:59 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:59.131+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:52.474010+0000 front 2024-09-15T22:26:52.474092+0000 (oldest deadline 2024-09-15T22:27:18.373981+0000) 2024-09-15T22:27:59.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:59 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:59.131+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:00.675138+0000 front 2024-09-15T22:27:00.675166+0000 (oldest deadline 2024-09-15T22:27:26.574995+0000) 2024-09-15T22:27:59.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:59 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:59.131+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:07.675696+0000 front 2024-09-15T22:27:07.675731+0000 (oldest deadline 2024-09-15T22:27:30.575857+0000) 2024-09-15T22:27:59.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:59 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:59.131+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:16.477019+0000 front 2024-09-15T22:27:16.476940+0000 (oldest deadline 2024-09-15T22:27:39.376681+0000) 2024-09-15T22:27:59.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:27:59 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10[59475]: 2024-09-15T22:27:59.131+0000 7f49e82fe640 -1 osd.10 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:23.477812+0000 front 2024-09-15T22:27:23.477890+0000 (oldest deadline 2024-09-15T22:27:48.177647+0000) 2024-09-15T22:28:00.295 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:28:00 smithi099 podman[72401]: 2024-09-15 22:28:00.017717793 +0000 UTC m=+5.738385595 container died 9011664cffc8092697b9b26643a26a0a8f70d488214e8109b917465843f05a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS) 2024-09-15T22:28:00.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:59 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:59.859+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:28:00.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:59 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:59.859+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:28:00.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:59 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:59.859+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:28:00.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:59 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:59.859+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:28:00.345 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:59 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:59.859+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:28:00.346 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:59 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:59.859+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:28:00.346 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:27:59 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:27:59.859+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-15T22:27:31.359545+0000 front 2024-09-15T22:27:31.359517+0000 (oldest deadline 2024-09-15T22:27:57.259467+0000) 2024-09-15T22:28:00.589 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:28:00 smithi099 podman[72401]: 2024-09-15 22:28:00.404600776 +0000 UTC m=+6.125268585 container cleanup 9011664cffc8092697b9b26643a26a0a8f70d488214e8109b917465843f05a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-15T22:28:00.589 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:28:00 smithi099 bash[72401]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10 2024-09-15T22:28:01.001 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:28:00 smithi099 podman[72417]: 2024-09-15 22:28:00.689289699 +0000 UTC m=+0.666545615 container remove 9011664cffc8092697b9b26643a26a0a8f70d488214e8109b917465843f05a41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-15T22:28:01.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:00 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:00.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:28:01.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:00 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:00.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:28:01.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:00 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:00.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:28:01.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:00 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:00.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:28:01.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:00 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:00.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:28:01.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:00 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:00.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:28:01.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:00 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:00.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-15T22:27:31.359545+0000 front 2024-09-15T22:27:31.359517+0000 (oldest deadline 2024-09-15T22:27:57.259467+0000) 2024-09-15T22:28:01.301 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:28:01 smithi099 podman[72505]: 2024-09-15 22:28:00.903656948 +0000 UTC m=+0.016149133 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:28:01.302 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:28:01 smithi099 podman[72505]: 2024-09-15 22:28:01.023326843 +0000 UTC m=+0.135819067 container create 7d936babaabb7406d30b3eb336c8f0a0eed86dbdba346aa03729b4ec4c53f299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-15T22:28:01.302 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:28:01 smithi099 podman[72505]: 2024-09-15 22:28:01.22543873 +0000 UTC m=+0.337930908 container init 7d936babaabb7406d30b3eb336c8f0a0eed86dbdba346aa03729b4ec4c53f299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10-deactivate, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2) 2024-09-15T22:28:01.302 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:28:01 smithi099 podman[72505]: 2024-09-15 22:28:01.228794913 +0000 UTC m=+0.341287092 container start 7d936babaabb7406d30b3eb336c8f0a0eed86dbdba346aa03729b4ec4c53f299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-15T22:28:01.302 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:28:01 smithi099 podman[72505]: 2024-09-15 22:28:01.30043292 +0000 UTC m=+0.412925112 container attach 7d936babaabb7406d30b3eb336c8f0a0eed86dbdba346aa03729b4ec4c53f299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-15T22:28:01.590 INFO:journalctl@ceph.osd.10.smithi099.stdout:Sep 15 22:28:01 smithi099 podman[72505]: 2024-09-15 22:28:01.419150381 +0000 UTC m=+0.531642562 container died 7d936babaabb7406d30b3eb336c8f0a0eed86dbdba346aa03729b4ec4c53f299 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-15T22:28:02.051 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.10.service' 2024-09-15T22:28:02.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:02.090 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-15T22:28:02.091 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-15T22:28:02.091 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.11 2024-09-15T22:28:02.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:01.834+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:28:02.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:01.834+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:28:02.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:01.834+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:28:02.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:01.834+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:28:02.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:01.834+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:28:02.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:01.834+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:28:02.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:01 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:01.834+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-15T22:27:31.359545+0000 front 2024-09-15T22:27:31.359517+0000 (oldest deadline 2024-09-15T22:27:57.259467+0000) 2024-09-15T22:28:02.456 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:02 smithi129 systemd[1]: Stopping Ceph osd.11 for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:02.814 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:02 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:02.454+0000 7f3cc76a9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:02.814 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:02 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:02.454+0000 7f3cc76a9640 -1 osd.11 113 *** Got signal Terminated *** 2024-09-15T22:28:02.814 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:02 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:02.454+0000 7f3cc76a9640 -1 osd.11 113 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-15T22:28:02.814 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:02 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:02.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:28:02.814 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:02 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:02.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:28:02.814 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:02 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:02.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:28:02.814 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:02 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:02.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:28:02.815 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:02 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:02.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:28:02.815 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:02 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:02.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:28:02.815 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:02 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:02.812+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-15T22:27:31.359545+0000 front 2024-09-15T22:27:31.359517+0000 (oldest deadline 2024-09-15T22:27:57.259467+0000) 2024-09-15T22:28:04.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:03 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:03.791+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:28:04.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:03 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:03.791+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:28:04.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:03 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:03.791+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:28:04.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:03 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:03.791+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:28:04.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:03 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:03.791+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:28:04.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:03 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:03.791+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:28:04.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:03 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:03.791+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-15T22:27:31.359545+0000 front 2024-09-15T22:27:31.359517+0000 (oldest deadline 2024-09-15T22:27:57.259467+0000) 2024-09-15T22:28:05.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:04 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:04.769+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:28:05.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:04 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:04.769+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:28:05.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:04 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:04.769+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:28:05.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:04 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:04.769+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:28:05.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:04 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:04.769+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:28:05.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:04 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:04.769+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:28:05.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:04 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:04.769+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-15T22:27:31.359545+0000 front 2024-09-15T22:27:31.359517+0000 (oldest deadline 2024-09-15T22:27:57.259467+0000) 2024-09-15T22:28:06.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:05 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:05.788+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:28:06.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:05 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:05.788+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:28:06.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:05 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:05.788+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:28:06.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:05 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:05.788+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:28:06.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:05 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:05.788+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:28:06.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:05 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:05.788+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:28:06.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:05 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:05.788+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-15T22:27:31.359545+0000 front 2024-09-15T22:27:31.359517+0000 (oldest deadline 2024-09-15T22:27:57.259467+0000) 2024-09-15T22:28:06.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:05 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:05.788+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6820 osd.8 since back 2024-09-15T22:27:40.660734+0000 front 2024-09-15T22:27:40.660708+0000 (oldest deadline 2024-09-15T22:28:05.360786+0000) 2024-09-15T22:28:07.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:06 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:06.755+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-15T22:26:35.351632+0000 front 2024-09-15T22:26:35.351563+0000 (oldest deadline 2024-09-15T22:26:57.651568+0000) 2024-09-15T22:28:07.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:06 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:06.755+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6804 osd.2 since back 2024-09-15T22:26:51.654088+0000 front 2024-09-15T22:26:51.654063+0000 (oldest deadline 2024-09-15T22:27:16.954057+0000) 2024-09-15T22:28:07.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:06 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:06.755+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-15T22:27:01.055352+0000 front 2024-09-15T22:27:01.055205+0000 (oldest deadline 2024-09-15T22:27:25.155215+0000) 2024-09-15T22:28:07.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:06 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:06.755+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6812 osd.4 since back 2024-09-15T22:27:06.256251+0000 front 2024-09-15T22:27:06.256228+0000 (oldest deadline 2024-09-15T22:27:32.056732+0000) 2024-09-15T22:28:07.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:06 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:06.755+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6812 osd.5 since back 2024-09-15T22:27:17.858067+0000 front 2024-09-15T22:27:17.858037+0000 (oldest deadline 2024-09-15T22:27:41.358245+0000) 2024-09-15T22:28:07.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:06 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:06.755+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-15T22:27:26.059246+0000 front 2024-09-15T22:27:26.059198+0000 (oldest deadline 2024-09-15T22:27:51.359061+0000) 2024-09-15T22:28:07.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:06 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:06.755+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.99:6820 osd.7 since back 2024-09-15T22:27:31.359545+0000 front 2024-09-15T22:27:31.359517+0000 (oldest deadline 2024-09-15T22:27:57.259467+0000) 2024-09-15T22:28:07.096 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:06 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11[59797]: 2024-09-15T22:28:06.755+0000 7f3cbfcb7640 -1 osd.11 113 heartbeat_check: no reply from 172.21.15.129:6820 osd.8 since back 2024-09-15T22:27:40.660734+0000 front 2024-09-15T22:27:40.660708+0000 (oldest deadline 2024-09-15T22:28:05.360786+0000) 2024-09-15T22:28:08.498 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:08 smithi129 podman[73131]: 2024-09-15 22:28:08.209029619 +0000 UTC m=+5.909375172 container died 1281fae7a845ac3d989054d67a80e6cc9352bfb677ccd93cca0481ae4b3eba7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-09-15T22:28:08.834 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:08 smithi129 podman[73131]: 2024-09-15 22:28:08.598375759 +0000 UTC m=+6.298721268 container cleanup 1281fae7a845ac3d989054d67a80e6cc9352bfb677ccd93cca0481ae4b3eba7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-15T22:28:08.834 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:08 smithi129 bash[73131]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11 2024-09-15T22:28:09.095 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:08 smithi129 podman[73147]: 2024-09-15 22:28:08.83308254 +0000 UTC m=+0.616896198 container remove 1281fae7a845ac3d989054d67a80e6cc9352bfb677ccd93cca0481ae4b3eba7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-15T22:28:09.421 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:09 smithi129 podman[73234]: 2024-09-15 22:28:09.142176078 +0000 UTC m=+0.104811970 container create 96206ddbdc737fe4ad756ab81d6af589f766dd1d435caf0de65bf6eb7368d0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-15T22:28:09.421 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:09 smithi129 podman[73234]: 2024-09-15 22:28:09.053301048 +0000 UTC m=+0.015936962 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-15T22:28:09.421 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:09 smithi129 podman[73234]: 2024-09-15 22:28:09.344595754 +0000 UTC m=+0.307231646 container init 96206ddbdc737fe4ad756ab81d6af589f766dd1d435caf0de65bf6eb7368d0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:28:09.421 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:09 smithi129 podman[73234]: 2024-09-15 22:28:09.347818817 +0000 UTC m=+0.310454711 container start 96206ddbdc737fe4ad756ab81d6af589f766dd1d435caf0de65bf6eb7368d0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-15T22:28:09.421 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:09 smithi129 podman[73234]: 2024-09-15 22:28:09.419497416 +0000 UTC m=+0.382133318 container attach 96206ddbdc737fe4ad756ab81d6af589f766dd1d435caf0de65bf6eb7368d0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-15T22:28:09.831 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:09 smithi129 podman[73234]: 2024-09-15 22:28:09.537945533 +0000 UTC m=+0.500581426 container died 96206ddbdc737fe4ad756ab81d6af589f766dd1d435caf0de65bf6eb7368d0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-15T22:28:10.089 INFO:journalctl@ceph.osd.11.smithi129.stdout:Sep 15 22:28:10 smithi129 podman[73257]: 2024-09-15 22:28:10.07314814 +0000 UTC m=+0.529343623 container remove 96206ddbdc737fe4ad756ab81d6af589f766dd1d435caf0de65bf6eb7368d0ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-15T22:28:10.095 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@osd.11.service' 2024-09-15T22:28:10.173 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:10.173 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-15T22:28:10.173 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-15T22:28:10.174 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.a 2024-09-15T22:28:10.496 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:28:10 smithi005 systemd[1]: Stopping Ceph mds.a for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:10.768 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:28:10 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-a[71425]: 2024-09-15T22:28:10.494+0000 7fd72bb0b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:10.769 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:28:10 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-a[71425]: 2024-09-15T22:28:10.494+0000 7fd72bb0b640 -1 mds.a *** got signal Terminated *** 2024-09-15T22:28:12.746 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:28:12 smithi005 podman[121636]: 2024-09-15 22:28:12.461767298 +0000 UTC m=+2.093805957 container died 18bcf8c0d601b76db0a774a8e98b2a783715988251938914fd3adbcb336198b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-a, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-15T22:28:13.106 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:28:12 smithi005 podman[121636]: 2024-09-15 22:28:12.927793139 +0000 UTC m=+2.559831856 container cleanup 18bcf8c0d601b76db0a774a8e98b2a783715988251938914fd3adbcb336198b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:28:13.106 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 15 22:28:12 smithi005 bash[121636]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-a 2024-09-15T22:28:13.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.a.service' 2024-09-15T22:28:13.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:13.405 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-15T22:28:13.406 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-15T22:28:13.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.d 2024-09-15T22:28:13.725 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:28:13 smithi005 systemd[1]: Stopping Ceph mds.d for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:14.106 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:28:13 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-d[70235]: 2024-09-15T22:28:13.723+0000 7f4e6e614640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:14.107 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:28:13 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-d[70235]: 2024-09-15T22:28:13.723+0000 7f4e6e614640 -1 mds.d *** got signal Terminated *** 2024-09-15T22:28:16.552 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:28:16 smithi005 podman[121771]: 2024-09-15 22:28:16.262066914 +0000 UTC m=+2.632583886 container died ab9a334f36f0c3fc13d7c8515d28800d1327a3f6327bd05fc5387b9836e58535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-15T22:28:16.856 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:28:16 smithi005 podman[121771]: 2024-09-15 22:28:16.731913215 +0000 UTC m=+3.102430236 container cleanup ab9a334f36f0c3fc13d7c8515d28800d1327a3f6327bd05fc5387b9836e58535 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:28:16.856 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 15 22:28:16 smithi005 bash[121771]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-d 2024-09-15T22:28:17.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.d.service' 2024-09-15T22:28:17.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:17.125 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-15T22:28:17.125 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-15T22:28:17.125 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.g 2024-09-15T22:28:17.356 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:28:17 smithi005 systemd[1]: Stopping Ceph mds.g for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:17.856 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:28:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-g[71006]: 2024-09-15T22:28:17.453+0000 7f39f6173640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:17.857 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:28:17 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-g[71006]: 2024-09-15T22:28:17.453+0000 7f39f6173640 -1 mds.g *** got signal Terminated *** 2024-09-15T22:28:21.046 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:28:20 smithi005 podman[121891]: 2024-09-15 22:28:20.778058797 +0000 UTC m=+3.432364342 container died 99ebb54bd236fa17df78451f76ef44a0067f6505b3cc5cb3893e165a1231edef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-g, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2) 2024-09-15T22:28:21.356 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:28:21 smithi005 podman[121891]: 2024-09-15 22:28:21.249524118 +0000 UTC m=+3.903829659 container cleanup 99ebb54bd236fa17df78451f76ef44a0067f6505b3cc5cb3893e165a1231edef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-15T22:28:21.356 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 15 22:28:21 smithi005 bash[121891]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-g 2024-09-15T22:28:21.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.g.service' 2024-09-15T22:28:21.671 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:21.671 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-15T22:28:21.671 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-15T22:28:21.671 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.j 2024-09-15T22:28:22.042 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:28:21 smithi005 systemd[1]: Stopping Ceph mds.j for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:22.357 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:28:22 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-j[69843]: 2024-09-15T22:28:22.040+0000 7f4741893640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:22.357 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:28:22 smithi005 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-j[69843]: 2024-09-15T22:28:22.040+0000 7f4741893640 -1 mds.j *** got signal Terminated *** 2024-09-15T22:28:25.225 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:28:24 smithi005 podman[122012]: 2024-09-15 22:28:24.935419803 +0000 UTC m=+3.027521206 container died f40634a25b83278242096d6568d96cc0f125bfa28e7c3cde39d6a72d5cea7f27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-15T22:28:25.584 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:28:25 smithi005 podman[122012]: 2024-09-15 22:28:25.340834612 +0000 UTC m=+3.432936011 container cleanup f40634a25b83278242096d6568d96cc0f125bfa28e7c3cde39d6a72d5cea7f27 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:28:25.584 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 15 22:28:25 smithi005 bash[122012]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-j 2024-09-15T22:28:25.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.j.service' 2024-09-15T22:28:25.742 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:25.742 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-15T22:28:25.742 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-15T22:28:25.742 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.b 2024-09-15T22:28:26.089 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:28:25 smithi099 systemd[1]: Stopping Ceph mds.b for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:26.590 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:28:26 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b[64991]: 2024-09-15T22:28:26.126+0000 7f428c462640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:26.590 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:28:26 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b[64991]: 2024-09-15T22:28:26.126+0000 7f428c462640 -1 mds.b *** got signal Terminated *** 2024-09-15T22:28:30.145 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:28:29 smithi099 podman[72609]: 2024-09-15 22:28:29.864808024 +0000 UTC m=+3.933250684 container died 51a3dc6feead20303ee11e71e434d74e9b55d0a409e378bbe8a8f08489541854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-15T22:28:30.478 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:28:30 smithi099 podman[72609]: 2024-09-15 22:28:30.301809573 +0000 UTC m=+4.370252221 container cleanup 51a3dc6feead20303ee11e71e434d74e9b55d0a409e378bbe8a8f08489541854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-15T22:28:30.478 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:28:30 smithi099 bash[72609]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b 2024-09-15T22:28:30.722 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.b.service' 2024-09-15T22:28:30.745 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:28:30 smithi099 podman[72633]: 2024-09-15 22:28:30.477734308 +0000 UTC m=+0.612162638 container remove 51a3dc6feead20303ee11e71e434d74e9b55d0a409e378bbe8a8f08489541854 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-15T22:28:30.745 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:28:30 smithi099 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-15T22:28:30.745 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:28:30 smithi099 systemd[1]: Stopped Ceph mds.b for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:28:30.745 INFO:journalctl@ceph.mds.b.smithi099.stdout:Sep 15 22:28:30 smithi099 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.b.service: Consumed 34.605s CPU time. 2024-09-15T22:28:30.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:30.758 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-15T22:28:30.758 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-15T22:28:30.759 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.e 2024-09-15T22:28:31.082 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:28:30 smithi099 systemd[1]: Stopping Ceph mds.e for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:31.340 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:28:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-e[64374]: 2024-09-15T22:28:31.080+0000 7f9662d63640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:31.340 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:28:31 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-e[64374]: 2024-09-15T22:28:31.080+0000 7f9662d63640 -1 mds.e *** got signal Terminated *** 2024-09-15T22:28:34.166 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:28:33 smithi099 podman[72742]: 2024-09-15 22:28:33.857816844 +0000 UTC m=+2.876187140 container died 3c51429a29f34d2fb032cbfc67a4c0580aa5073dacfdc19bdcd78c6ff1f886b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:28:34.523 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:28:34 smithi099 podman[72742]: 2024-09-15 22:28:34.255229839 +0000 UTC m=+3.273600135 container cleanup 3c51429a29f34d2fb032cbfc67a4c0580aa5073dacfdc19bdcd78c6ff1f886b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-15T22:28:34.524 INFO:journalctl@ceph.mds.e.smithi099.stdout:Sep 15 22:28:34 smithi099 bash[72742]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-e 2024-09-15T22:28:34.675 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.e.service' 2024-09-15T22:28:34.708 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:34.708 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-15T22:28:34.708 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-15T22:28:34.708 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.h 2024-09-15T22:28:35.089 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:28:34 smithi099 systemd[1]: Stopping Ceph mds.h for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:35.089 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:28:35 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h[64679]: 2024-09-15T22:28:35.025+0000 7f56c5a7e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:35.090 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:28:35 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h[64679]: 2024-09-15T22:28:35.025+0000 7f56c5a7e640 -1 mds.h *** got signal Terminated *** 2024-09-15T22:28:38.090 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:28:37 smithi099 podman[72863]: 2024-09-15 22:28:37.883231211 +0000 UTC m=+2.957155156 container died 6f5ba94cdc815f3e2f349cbe1b59350d1c40302e47f53106809aa46f37c7fa3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-15T22:28:38.502 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:28:38 smithi099 podman[72863]: 2024-09-15 22:28:38.275080067 +0000 UTC m=+3.349004011 container cleanup 6f5ba94cdc815f3e2f349cbe1b59350d1c40302e47f53106809aa46f37c7fa3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-15T22:28:38.502 INFO:journalctl@ceph.mds.h.smithi099.stdout:Sep 15 22:28:38 smithi099 bash[72863]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-h 2024-09-15T22:28:38.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.h.service' 2024-09-15T22:28:38.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:38.668 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-15T22:28:38.669 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-15T22:28:38.669 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.k 2024-09-15T22:28:38.839 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:28:38 smithi099 systemd[1]: Stopping Ceph mds.k for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:39.339 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:28:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-k[64073]: 2024-09-15T22:28:38.994+0000 7f764301c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:39.340 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:28:38 smithi099 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-k[64073]: 2024-09-15T22:28:38.994+0000 7f764301c640 -1 mds.k *** got signal Terminated *** 2024-09-15T22:28:42.088 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:28:41 smithi099 podman[72984]: 2024-09-15 22:28:41.811877994 +0000 UTC m=+2.925474482 container died 7f7f5a520440cd1c6c78bc890cd4266fe4c4358b094f80a6322623cab9982e7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-15T22:28:42.339 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:28:42 smithi099 podman[72984]: 2024-09-15 22:28:42.169648915 +0000 UTC m=+3.283245396 container cleanup 7f7f5a520440cd1c6c78bc890cd4266fe4c4358b094f80a6322623cab9982e7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:28:42.340 INFO:journalctl@ceph.mds.k.smithi099.stdout:Sep 15 22:28:42 smithi099 bash[72984]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-k 2024-09-15T22:28:42.593 DEBUG:teuthology.orchestra.run.smithi099:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.k.service' 2024-09-15T22:28:42.626 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:42.626 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-15T22:28:42.626 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-15T22:28:42.626 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.c 2024-09-15T22:28:43.008 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:28:42 smithi129 systemd[1]: Stopping Ceph mds.c for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:43.267 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:28:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c[64941]: 2024-09-15T22:28:43.007+0000 7f63ff30c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:43.267 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:28:43 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c[64941]: 2024-09-15T22:28:43.007+0000 7f63ff30c640 -1 mds.c *** got signal Terminated *** 2024-09-15T22:28:46.093 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:28:45 smithi129 podman[73337]: 2024-09-15 22:28:45.822484719 +0000 UTC m=+3.003609293 container died d13e9bfd1eefb51f920fc826c063fea7885232658bfb857bd7a2e3200ec15855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-15T22:28:46.345 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:28:46 smithi129 podman[73337]: 2024-09-15 22:28:46.189973516 +0000 UTC m=+3.371098089 container cleanup d13e9bfd1eefb51f920fc826c063fea7885232658bfb857bd7a2e3200ec15855 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-15T22:28:46.345 INFO:journalctl@ceph.mds.c.smithi129.stdout:Sep 15 22:28:46 smithi129 bash[73337]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-c 2024-09-15T22:28:46.615 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.c.service' 2024-09-15T22:28:46.651 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:46.651 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-15T22:28:46.651 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-15T22:28:46.652 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.f 2024-09-15T22:28:47.095 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:28:46 smithi129 systemd[1]: Stopping Ceph mds.f for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:47.095 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:28:46 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f[64638]: 2024-09-15T22:28:46.977+0000 7fe514dac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:47.095 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:28:46 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f[64638]: 2024-09-15T22:28:46.977+0000 7fe514dac640 -1 mds.f *** got signal Terminated *** 2024-09-15T22:28:50.095 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:28:49 smithi129 podman[73471]: 2024-09-15 22:28:49.824728081 +0000 UTC m=+2.949756149 container died 44bb71243fa874eef6486747a9814378487b2d933f48cb4f2ad82d0e2a7b36a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:28:50.497 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:28:50 smithi129 podman[73471]: 2024-09-15 22:28:50.244834591 +0000 UTC m=+3.369862646 container cleanup 44bb71243fa874eef6486747a9814378487b2d933f48cb4f2ad82d0e2a7b36a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-15T22:28:50.497 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:28:50 smithi129 bash[73471]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f 2024-09-15T22:28:50.729 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.f.service' 2024-09-15T22:28:50.752 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:28:50 smithi129 podman[73483]: 2024-09-15 22:28:50.496224288 +0000 UTC m=+0.666144026 container remove 44bb71243fa874eef6486747a9814378487b2d933f48cb4f2ad82d0e2a7b36a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-f, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-15T22:28:50.752 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:28:50 smithi129 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.f.service: Deactivated successfully. 2024-09-15T22:28:50.752 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:28:50 smithi129 systemd[1]: Stopped Ceph mds.f for 962891f2-73ad-11ef-bceb-c7b262605968. 2024-09-15T22:28:50.752 INFO:journalctl@ceph.mds.f.smithi129.stdout:Sep 15 22:28:50 smithi129 systemd[1]: ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.f.service: Consumed 2.524s CPU time. 2024-09-15T22:28:50.763 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:50.763 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-15T22:28:50.763 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-15T22:28:50.763 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.i 2024-09-15T22:28:51.074 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:28:50 smithi129 systemd[1]: Stopping Ceph mds.i for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:51.345 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:28:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i[65245]: 2024-09-15T22:28:51.073+0000 7ff098599640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:51.345 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:28:51 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i[65245]: 2024-09-15T22:28:51.073+0000 7ff098599640 -1 mds.i *** got signal Terminated *** 2024-09-15T22:28:54.095 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:28:53 smithi129 podman[73591]: 2024-09-15 22:28:53.818875946 +0000 UTC m=+2.838591493 container died 54daa7354c84a78d28f77acb2ac7826bec264c8aabe37039e0bba273c0f7efec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-15T22:28:54.484 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:28:54 smithi129 podman[73591]: 2024-09-15 22:28:54.215252178 +0000 UTC m=+3.234967724 container cleanup 54daa7354c84a78d28f77acb2ac7826bec264c8aabe37039e0bba273c0f7efec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-8921c8e) 2024-09-15T22:28:54.497 INFO:journalctl@ceph.mds.i.smithi129.stdout:Sep 15 22:28:54 smithi129 bash[73591]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-i 2024-09-15T22:28:54.611 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.i.service' 2024-09-15T22:28:54.645 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:54.645 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-15T22:28:54.646 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-15T22:28:54.646 DEBUG:teuthology.orchestra.run.smithi129:> sudo systemctl stop ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.l 2024-09-15T22:28:54.845 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:28:54 smithi129 systemd[1]: Stopping Ceph mds.l for 962891f2-73ad-11ef-bceb-c7b262605968... 2024-09-15T22:28:55.345 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:28:54 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-l[64337]: 2024-09-15T22:28:54.952+0000 7f0c56726640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-15T22:28:55.345 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:28:54 smithi129 ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-l[64337]: 2024-09-15T22:28:54.952+0000 7f0c56726640 -1 mds.l *** got signal Terminated *** 2024-09-15T22:28:59.229 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:28:58 smithi129 podman[73712]: 2024-09-15 22:28:58.966941647 +0000 UTC m=+4.107553401 container died e98fd1bf7eed02365a983f1f9df51606521b26a48733437c55daf7fdee34d6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-15T22:28:59.595 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:28:59 smithi129 podman[73712]: 2024-09-15 22:28:59.311445367 +0000 UTC m=+4.452057108 container cleanup e98fd1bf7eed02365a983f1f9df51606521b26a48733437c55daf7fdee34d6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True) 2024-09-15T22:28:59.595 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:28:59 smithi129 bash[73712]: ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-l 2024-09-15T22:28:59.595 INFO:journalctl@ceph.mds.l.smithi129.stdout:Sep 15 22:28:59 smithi129 podman[73726]: 2024-09-15 22:28:59.462380832 +0000 UTC m=+0.486934187 container remove e98fd1bf7eed02365a983f1f9df51606521b26a48733437c55daf7fdee34d6c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-962891f2-73ad-11ef-bceb-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-15T22:28:59.664 DEBUG:teuthology.orchestra.run.smithi129:> sudo pkill -f 'journalctl -f -n 0 -u ceph-962891f2-73ad-11ef-bceb-c7b262605968@mds.l.service' 2024-09-15T22:28:59.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-15T22:28:59.702 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-15T22:28:59.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 962891f2-73ad-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T22:28:59.916 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:29:02.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 962891f2-73ad-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T22:29:02.871 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:29:05.369 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 962891f2-73ad-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-15T22:29:05.569 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:29:08.045 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:29:08.079 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:29:08.112 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-15T22:29:08.141 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-15T22:29:08.143 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224/remote/smithi005/crash 2024-09-15T22:29:08.144 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/crash -- . 2024-09-15T22:29:08.194 INFO:teuthology.orchestra.run.smithi005.stderr:tar: /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T22:29:08.195 INFO:teuthology.orchestra.run.smithi005.stderr:tar: Error is not recoverable: exiting now 2024-09-15T22:29:08.197 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224/remote/smithi099/crash 2024-09-15T22:29:08.199 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/crash -- . 2024-09-15T22:29:08.239 INFO:teuthology.orchestra.run.smithi099.stderr:tar: /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T22:29:08.239 INFO:teuthology.orchestra.run.smithi099.stderr:tar: Error is not recoverable: exiting now 2024-09-15T22:29:08.241 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224/remote/smithi129/crash 2024-09-15T22:29:08.242 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/crash -- . 2024-09-15T22:29:08.279 INFO:teuthology.orchestra.run.smithi129.stderr:tar: /var/lib/ceph/962891f2-73ad-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-15T22:29:08.280 INFO:teuthology.orchestra.run.smithi129.stderr:tar: Error is not recoverable: exiting now 2024-09-15T22:29:08.281 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-15T22:29:08.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-15T22:29:08.370 INFO:tasks.cephadm:Compressing logs... 2024-09-15T22:29:08.370 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T22:29:08.373 DEBUG:teuthology.orchestra.run.smithi099:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T22:29:08.377 DEBUG:teuthology.orchestra.run.smithi129:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T22:29:08.407 INFO:teuthology.orchestra.run.smithi129.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-15T22:29:08.408 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T22:29:08.408 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T22:29:08.411 INFO:teuthology.orchestra.run.smithi129.stderr:gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-15T22:29:08.412 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T22:29:08.414 INFO:teuthology.orchestra.run.smithi005.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-15T22:29:08.414 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-15T22:29:08.414 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-15T22:29:08.414 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.log: /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-15T22:29:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T22:29:08.415 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T22:29:08.415 INFO:teuthology.orchestra.run.smithi129.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-15T22:29:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.9.log: /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-15T22:29:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.audit.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-15T22:29:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T22:29:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-15T22:29:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-15T22:29:08.420 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-15T22:29:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mon.a.log: /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.log 2024-09-15T22:29:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-15T22:29:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-15T22:29:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-15T22:29:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-15T22:29:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T22:29:08.421 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.4% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-15T22:29:08.421 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-15T22:29:08.421 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-15T22:29:08.422 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-15T22:29:08.425 INFO:teuthology.orchestra.run.smithi099.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-15T22:29:08.425 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-15T22:29:08.426 INFO:teuthology.orchestra.run.smithi099.stderr:gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-15T22:29:08.426 INFO:teuthology.orchestra.run.smithi099.stderr:gzip/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.10.log: -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-15T22:29:08.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-15T22:29:08.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-15T22:29:08.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-15T22:29:08.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-15T22:29:08.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-15T22:29:08.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-15T22:29:08.430 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-15T22:29:08.431 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.log 2024-09-15T22:29:08.431 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-15T22:29:08.431 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-15T22:29:08.431 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-15T22:29:08.431 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-15T22:29:08.431 INFO:teuthology.orchestra.run.smithi099.stderr:/var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 94.4% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-15T22:29:08.441 INFO:teuthology.orchestra.run.smithi005.stderr: 86.9% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T22:29:08.472 INFO:teuthology.orchestra.run.smithi099.stderr: 82.3% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T22:29:08.485 INFO:teuthology.orchestra.run.smithi005.stderr: 84.8% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T22:29:08.521 INFO:teuthology.orchestra.run.smithi129.stderr: 82.3% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-15T22:29:08.529 INFO:teuthology.orchestra.run.smithi129.stderr: 92.3% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-15T22:29:08.530 INFO:teuthology.orchestra.run.smithi129.stderr: 91.0% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T22:29:08.559 INFO:teuthology.orchestra.run.smithi099.stderr: 92.2% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-15T22:29:08.590 INFO:teuthology.orchestra.run.smithi129.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-15T22:29:08.592 INFO:teuthology.orchestra.run.smithi099.stderr: 86.9% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T22:29:08.598 INFO:teuthology.orchestra.run.smithi129.stderr: 86.9% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-15T22:29:08.604 INFO:teuthology.orchestra.run.smithi099.stderr: 90.3% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-15T22:29:08.606 INFO:teuthology.orchestra.run.smithi129.stderr: 90.3% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-15T22:29:08.631 INFO:teuthology.orchestra.run.smithi005.stderr: 94.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-15T22:29:08.714 INFO:teuthology.orchestra.run.smithi005.stderr: 90.2% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-15T22:29:08.723 INFO:teuthology.orchestra.run.smithi099.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-15T22:29:08.917 INFO:teuthology.orchestra.run.smithi005.stderr: 91.0% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T22:29:08.989 INFO:teuthology.orchestra.run.smithi099.stderr: 91.0% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-15T22:29:09.770 INFO:teuthology.orchestra.run.smithi099.stderr: 94.5% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-15T22:29:10.731 INFO:teuthology.orchestra.run.smithi129.stderr: 94.7% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-15T22:29:11.450 INFO:teuthology.orchestra.run.smithi005.stderr: 90.4% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-15T22:29:11.952 INFO:teuthology.orchestra.run.smithi099.stderr: 93.4% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-15T22:29:12.238 INFO:teuthology.orchestra.run.smithi129.stderr: 93.4% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-15T22:29:14.346 INFO:teuthology.orchestra.run.smithi129.stderr: 91.8% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-15T22:29:17.003 INFO:teuthology.orchestra.run.smithi099.stderr: 91.7% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-15T22:29:21.004 INFO:teuthology.orchestra.run.smithi129.stderr: 94.4% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-15T22:29:24.747 INFO:teuthology.orchestra.run.smithi005.stderr: 90.3% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-15T22:29:49.109 INFO:teuthology.orchestra.run.smithi099.stderr: 95.1% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-15T22:34:37.840 INFO:teuthology.orchestra.run.smithi099.stderr: 90.8% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-15T22:34:53.258 INFO:teuthology.orchestra.run.smithi129.stderr: 90.6% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-15T22:35:55.512 INFO:teuthology.orchestra.run.smithi099.stderr: 90.7% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-15T22:37:58.598 INFO:teuthology.orchestra.run.smithi099.stderr: 90.7% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-15T22:38:26.050 INFO:teuthology.orchestra.run.smithi099.stderr: 90.8% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi099.stderr: 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:real 9m17.660s 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:user 5m57.103s 2024-09-15T22:38:26.052 INFO:teuthology.orchestra.run.smithi099.stderr:sys 0m13.684s 2024-09-15T22:38:54.406 INFO:teuthology.orchestra.run.smithi129.stderr: 90.7% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-15T22:39:54.948 INFO:teuthology.orchestra.run.smithi129.stderr: 90.9% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-15T22:40:33.760 INFO:teuthology.orchestra.run.smithi129.stderr: 90.7% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-15T22:40:33.762 INFO:teuthology.orchestra.run.smithi129.stderr: 2024-09-15T22:40:33.762 INFO:teuthology.orchestra.run.smithi129.stderr:real 11m25.371s 2024-09-15T22:40:33.763 INFO:teuthology.orchestra.run.smithi129.stderr:user 6m28.920s 2024-09-15T22:40:33.763 INFO:teuthology.orchestra.run.smithi129.stderr:sys 0m14.856s 2024-09-15T22:43:44.039 INFO:teuthology.orchestra.run.smithi005.stderr: 90.8% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-15T22:44:39.086 INFO:teuthology.orchestra.run.smithi005.stderr: 90.8% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-15T22:45:56.830 INFO:teuthology.orchestra.run.smithi005.stderr: 90.6% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-15T22:46:32.771 INFO:teuthology.orchestra.run.smithi005.stderr: 90.7% -- replaced with /var/log/ceph/962891f2-73ad-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-15T22:46:32.804 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-15T22:46:32.804 INFO:teuthology.orchestra.run.smithi005.stderr:real 17m24.417s 2024-09-15T22:46:32.804 INFO:teuthology.orchestra.run.smithi005.stderr:user 7m57.498s 2024-09-15T22:46:32.804 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m18.969s 2024-09-15T22:46:32.805 INFO:tasks.cephadm:Archiving logs... 2024-09-15T22:46:32.806 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224/remote/smithi005/log 2024-09-15T22:46:32.807 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T22:47:51.219 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224/remote/smithi099/log 2024-09-15T22:47:51.244 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T22:48:53.301 DEBUG:teuthology.misc:Transferring archived files from smithi129:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224/remote/smithi129/log 2024-09-15T22:48:53.302 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T22:50:01.755 INFO:tasks.cephadm:Removing cluster... 2024-09-15T22:50:01.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 962891f2-73ad-11ef-bceb-c7b262605968 --force 2024-09-15T22:50:02.417 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:50:03.255 DEBUG:teuthology.orchestra.run.smithi099:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 962891f2-73ad-11ef-bceb-c7b262605968 --force 2024-09-15T22:50:03.544 INFO:teuthology.orchestra.run.smithi099.stdout:Deleting cluster with fsid: 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:50:04.211 DEBUG:teuthology.orchestra.run.smithi129:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 962891f2-73ad-11ef-bceb-c7b262605968 --force 2024-09-15T22:50:04.787 INFO:teuthology.orchestra.run.smithi129.stdout:Deleting cluster with fsid: 962891f2-73ad-11ef-bceb-c7b262605968 2024-09-15T22:50:05.541 INFO:tasks.cephadm:Removing cephadm ... 2024-09-15T22:50:05.541 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T22:50:05.562 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T22:50:05.582 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-15T22:50:05.631 INFO:tasks.cephadm:Teardown complete 2024-09-15T22:50:05.631 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T22:50:05.642 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T22:50:05.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T22:50:05.645 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T22:50:05.647 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T22:50:05.765 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T22:50:05.776 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T22:50:05.776 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:50:05.779 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:50:05.782 DEBUG:teuthology.orchestra.run.smithi129:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T22:50:05.795 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:50:05.795 INFO:teuthology.orchestra.run.smithi129.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:50:05.796 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-15T22:50:05.806 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:50:05.806 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-15T22:50:05.806 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +1114us[+1102us] +/- 55ms 2024-09-15T22:50:05.806 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 -995us[-1006us] +/- 53ms 2024-09-15T22:50:05.806 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 -147us[ -147us] +/- 77ms 2024-09-15T22:50:05.806 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:50:05.825 INFO:teuthology.orchestra.run.smithi129.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:50:05.826 INFO:teuthology.orchestra.run.smithi129.stdout:=============================================================================== 2024-09-15T22:50:05.826 INFO:teuthology.orchestra.run.smithi129.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +1125us[+1142us] +/- 55ms 2024-09-15T22:50:05.826 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -973us[ -973us] +/- 53ms 2024-09-15T22:50:05.826 INFO:teuthology.orchestra.run.smithi129.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 -115us[ -115us] +/- 77ms 2024-09-15T22:50:05.826 INFO:teuthology.orchestra.run.smithi129.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:50:05.885 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T22:50:05.885 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-09-15T22:50:05.885 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +1087us[+1076us] +/- 55ms 2024-09-15T22:50:05.885 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -1014us[-1026us] +/- 53ms 2024-09-15T22:50:05.885 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 -188us[ -200us] +/- 77ms 2024-09-15T22:50:05.885 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T22:50:05.897 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T22:50:05.908 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T22:50:05.909 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T22:50:05.920 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T22:50:06.123 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T22:50:06.237 DEBUG:teuthology.orchestra.run.smithi129:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T22:50:06.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:50:06.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:50:06.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:50:06.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:50:06.432 DEBUG:teuthology.orchestra.run.smithi129:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T22:50:06.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:50:06.460 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T22:50:06.471 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T22:50:06.482 INFO:teuthology.task.internal:Duration was 3481.250347 seconds 2024-09-15T22:50:06.483 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T22:50:06.494 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T22:50:06.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T22:50:06.497 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T22:50:06.499 DEBUG:teuthology.orchestra.run.smithi129:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T22:50:06.551 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:50:06.565 INFO:teuthology.orchestra.run.smithi129.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:50:06.570 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T22:50:06.948 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T22:50:06.948 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-09-15T22:50:06.948 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T22:50:06.981 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-09-15T22:50:06.981 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T22:50:07.022 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi129.front.sepia.ceph.com 2024-09-15T22:50:07.022 DEBUG:teuthology.orchestra.run.smithi129:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T22:50:07.047 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T22:50:07.048 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T22:50:07.050 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T22:50:07.065 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T22:50:07.114 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T22:50:07.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T22:50:07.121 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T22:50:07.136 DEBUG:teuthology.orchestra.run.smithi129:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T22:50:08.629 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T22:50:08.640 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T22:50:08.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T22:50:08.667 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T22:50:08.699 DEBUG:teuthology.orchestra.run.smithi129:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T22:50:08.735 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T22:50:08.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:50:08.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:50:08.761 DEBUG:teuthology.orchestra.run.smithi129:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:50:08.789 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-09-15T22:50:08.797 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-09-15T22:50:08.817 INFO:teuthology.orchestra.run.smithi129.stdout:kernel.core_pattern = core 2024-09-15T22:50:08.840 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:50:08.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:50:08.873 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:50:08.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:50:08.889 DEBUG:teuthology.orchestra.run.smithi129:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T22:50:08.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T22:50:08.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T22:50:08.918 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T22:50:08.919 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224/remote/smithi005 2024-09-15T22:50:08.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T22:50:08.993 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224/remote/smithi099 2024-09-15T22:50:08.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T22:50:09.034 DEBUG:teuthology.misc:Transferring archived files from smithi129:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906224/remote/smithi129 2024-09-15T22:50:09.034 DEBUG:teuthology.orchestra.run.smithi129:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T22:50:09.074 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T22:50:09.075 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T22:50:09.077 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T22:50:09.080 DEBUG:teuthology.orchestra.run.smithi129:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T22:50:09.124 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T22:50:09.137 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T22:50:09.137 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T22:50:09.148 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T22:50:09.149 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T22:50:09.151 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T22:50:09.153 DEBUG:teuthology.orchestra.run.smithi129:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T22:50:09.164 INFO:teuthology.orchestra.run.smithi005.stdout: 265722 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 22:50 /home/ubuntu/cephtest 2024-09-15T22:50:09.167 INFO:teuthology.orchestra.run.smithi099.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 22:50 /home/ubuntu/cephtest 2024-09-15T22:50:09.181 INFO:teuthology.orchestra.run.smithi129.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 22:50 /home/ubuntu/cephtest 2024-09-15T22:50:09.182 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T22:50:09.194 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T22:50:09.227 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/with-quiesce 6-workunit/postgres}} duration: 3481.250347137451 failure_reason: 'The following counters failed to be set on mds daemons: {''mds.imported'', ''mds.exported''}' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-09-15T22:50:09.228 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T22:50:09.312 INFO:teuthology.run:FAIL